Research conducted for the UX Upskill Program @ Hyper Island
This research was conducted and presented as part of the Field Work module at the UX Upskill Program at the school Hyper Island.
We were introduced to the art of doing field work in order to understand what people truly want, need and desire. Learned the fundamentals about how to gather data, what research methods there are and how we conduct interviews in the real world. Also, we practiced Nudging.
We also reviewed work processes and quality standards within UX Design and parts of UX research, such as user testing, data gathering and analytics as well as performing quantitative and qualitative field studies.
We were presented with the challenge: How might we help product teams when they are trying to understand their users?
We were to choose from three different target groups, Product Manager, Researched and Product/UX Designer.
I choose UX Designers as the target group within this research.
I used the Double Diamon Framework from Design Council (https://www.designcouncil.org.uk/news-opinion/what-framework-innovation-design-councils-evolved-double-diamond).
1 – Mapping out the UX Designers needs, I simulated a workshop where the needs (coming from my earlier experiences) was mapped out and where I selected 3/5, top important- and unknown, needs for a UX Designer to be used for the hypothesis.
2 – An experiment guide was used to create hypothesis out of the top 3 needs. The guide helped me to develop both the quantitative questions as well as the qualitative questions
3 – The survey was set up in Google Forms. I decided from the beginning that I wanted something I find important in research, a large enough sample of the dataset. My preconceptions about UX Designers are that they are creative and that the like memes. So I created a survey that would give the user a “fun” and “relaxed” feeling when filled in. Since we also couldn’t reward the ones participating with gift cards or such, I brainstormed and came up with that a book tip was something I think most appreciate, but mostly, it triggers peoples curiousness!
I joined different Slack, Facebook and LinkedIn groups for UX designers and posted in them. I connected over LinkedIn with random UX Designers and asked them to fill it in. All of these acts worked to get me 110 answers on the survey and 9 people that was up for interviews!
Along with the answers coming in and me having the interviews I’ve learned that the survey, if redoing it, would benefit from being more clear on the exact type of UX Designers I was looking for as well as rewriting some of the questions for getting even more insights from it. Though a great learning opportunity and I’ve also recieved good and constructive feedback from people answering the survey and wanting to help out.
4 – For the last part the data from the quantitative survey data and qualitative data from the interviews was synthetized where the insights, principles and the next steps are defined.
A Generative Research Project
THE CHALLENGE
How might we help product teams when they are trying to understand their users?
Methodology
Defining the Target Group
UX Designers were selected as the target group within a product team. A kickoff workshop narrowed down the thoughts on a UX Designers needs to three unknown and important needs. These were transferred to an experiment canvas. It helped guiding me into defining a hypothesis for each of the three needs.
UX DESIGNER
Kickoff workshop defined a UX Designer to be: Digitally gifted - Passion driven - Work is Life - Creative souls
Collecting the Quantitative Data
On top of defining three hypothesis with the experiment canvas, a set of qualitative & quantitative questions emerged from it. The quantitative questions was added to a survey that would be sent out to gather participants for proposed interviews to collect qualitative data. The survey was constructed to grasp the attention of many, it used memes, created curiosity by giving away a book tip (nudging users into taking the survey since we take advantage of that people doesn't want the be left out in not knowing what book it is) and was posted in social media and sent directly to random UX Designers on LinkedIn that I added as connections. It was a success with 110 responses and 9 people signed up for interviews.
Survey
110 responses
Screener- & Quantitative questions.
Conducting the Interviews [Qualitative Data]
Within the survey, participants for an interview was collected. There was 9 responses and 5 was selected to take part of the interview. Calendly.com was used to schedule times and within a week the interviews was completed. Each interview was transcribed manually for myself to better understand how I perform as an interviewer.
Interviews
5 participants
30 minute interviews online with UX designers working with digital products. Both junior and senior designers was part of the interviews.
Synthesis the Data
The quantitative- and qualitative data (survey & interviews) were used to find those insights we are after. Themes and patterns was created for the interviews and matched together with relevant data from the survey. It was synthesised into insights, principles and recommendations for the next steps.
Please see below for the results of this research.
Please see below for the results of this research.
Synthesis
Insights, principles & recommendations
Guided by the help of the journey map
See the results below
Hypothesis 1
We believe UX Designers prefer working with products for which they belong to the target group
Definition of Success
We know this is true when UX Designers express posivitely about being part of the target group or negatively about not being part of the target group
Results [Quantitative Survey]
In the survey we asked the question on how closely related UX Designers would define themselves to the products target group. This question is meant to show if there is a clear indication if UX Designers today are part of their products target group or not. What we did find out was that the results was very scattered with a tilt towards that UX Designers aren't part of their products target group.
Results [Qualitative Interviews]
The interviews started by asking if the UX Designers product had a specific type of user and as a follow up question asking them how closely related they would be to the user and if they were, or weren't, if that helped or hindered them as a UX designer. The interviews results showed that:
- UX Designers being part of the target group found it to be positive
- UX Designers who defines them being in the middle found that knowing more about the target group was positive
- UX Designers that weren't part of the target group found it was a struggle not to be it
- UX Designers being part of the target group found it to be positive
- UX Designers who defines them being in the middle found that knowing more about the target group was positive
- UX Designers that weren't part of the target group found it was a struggle not to be it
Interview Quotes
I worked with the product myself before joining the company, so you figured out some work-arounds on problems which I directly knew I needed to flag and do something about. Though I believe the positive outweights the negative since you always have the "user with you".
UX Designer - Start-up SaaS [Stockholm]
It helps me having an in-depth knowledge about the type of users we have. I work alongside colleagues that are more or less identical to our end users. In conversations with them I learn and can filtrate what they are saying through my expertise and fine a golden middle path.
UX Designer & Marketer - Infrastructure-as-a-Service [Stockholm]
It's always a struggle to design for a different culture than my own. Either it's the language or that certain images does and doesn't work in different markets.
UX Designer - Ed-tech [Stockholm]
Insights
UX Designers feel that the more knowledge they have about the products target group the better they can perform in designing the product
Being part of the target group can make you biased in designing more according to the pains you have experienced instead of the pains in the bigger group. Though, by always being able to "ask yourself" on target group issues can provide true gains in your design process. The opportunities in being part of the target group will exceed the negative aspects of it. What have been brought up in the interviews are that Google Analytics as well as Google Ads has been some of the major digital tools for UX Designers to gather data on their target group. Within your team, colleagues, as the form of developers can be a part of understanding your target group and get closer to them since your colleagues might be part of the target group themselves.
Principle
Allow UX Designers to explore and learn more about their target groups
Next step
Concept session: Explore how we can support our UX Designers in learning more and be closer to the products target group.
Hypothesis 2
We believe UX Designers would benefit to have a structured forum where users and the company can communicate with each other
Definition of Success
We know this is true when UX Designers can't refer to a unified place where they can access and share communication with their users
Results [Quantitative Survey]
In the survey we asked if their company offered an open forum for users to ask questions on the product. We received that most (60%) didn't offer this type of forum, 29% offered a forum like this and the last 10% was shared between other types of answers. Contrasting this with the question on how important to be transparent and share potential issues and errors with their users where the majority (60%) rated it an 8 or above on it being important it seems there might be a disconnect between what is offered and what is sought after.
Results [Qualitative Interviews]
We asked the UX Designers how their process normally looked like to gather information about their users. We received most answers on them initially using Google Analytics, but mostly to get demographics, but for getting more in-depth user insights they would be setting up interviews, but often they felt they were lacking the resources for setting up more in-depth knowledge of what issues their users might have. It was also brought up that the issues that are being brought up are those who reports it, but all those who doesn't report them, what issues might be out there? One interpretation of this could be that if it's too cumbersome for a user to share issues, like if you have to yourself just email the company on the standard "info@companyabc.com" many might not do it. With an easier forum platform to share issues, as well as noticing that the company openly asks and receives questions might trigger more people to share issues.
Interview Quotes
We sometimes set up user tests, asks users on the website, sends out NPS surveys, though nothing that happens on a regular basis which I would like to have!
UX Designer - Ed-tech [Stockholm]
We have a community driven library in which teams share things they created and use instead of or on top of what is available for them via our service and we use this to distribute it to all users.
Head of UX - Product company [Stockholm]
I've added surveys on the website, feedback tabs, minor stuff. Though there I get ahold of those who are the most engaged. Almost have to trust that those with a problem is those who will answer. So those with a problem I can start with. Then I have to do more... well it is a bit of a teaser, difficult to get ahold of those who doesn't answer!
UX Designer - Product Company [Stockholm]
Insights
UX Designers are lacking a unified platform where users and the product company can communicate.
According to the interviews, there seems to be a pain point in retrieving information on the users and we can see that there is only roughly 30% which actually are collecting user insights/problems via a forum, even if most users (60%) answered that its important to be transparent with their users. The gains in having a forum where users can communicate seems to be high since it allows the task of retrieving user insights into an "automatic" task when the users can share it themselves. There seems to be a great opportunity in saving resources but still getting the users insights. There is tools that can be used for this, one that have been brought up in the interviews is Github. This is also a tool that can be used by your team since they can themselves share and review information that is pushed onto Github.
Principle
Allow UX Designers to receive communication from users in a structured way.
Next step
Set up and test a basic community driven web forum to see if it would benefit UX Designers when they understand their users.
Hypothesis 3
We don't believe UX Designers think that demographic data is important when understanding their target group
Definition of Success
We know this is true when UX Designers don't bring up demographics data during conversations about understanding their target group
Results [Quantitative Survey]
Is demographics of importance when understanding your users? This was the question many might wonder since it's extremely common to see questions on age, gender, income, etc. on various surveys and also those regarding UX Design. I asked this questions to over 100 UX Designers, and the results show that the answers vary but it's also very clear that it's neither very important nor not important. It's a bit over every fourth person we asked (28.2%) which said it's an 8 or above in importance (1-10 scale) and almost every fourth (23.7%) who is a 3 and below (1-10 scale). We didn't collect user demographics on this survey so we could stay true to our hypothesis, but it could've been interesting to see if there is a difference in the younger generation who has been brought up in a society with less focus on people being defined by their demographics...
Results [Qualitative Interviews]
Within the interviews I chose to not bluntly ask, "how important is demographics on understanding your users?" since I believed this might mean I would be getting answers which didn't was true to nature. Also since I already had that data from the survey so I chose to try to get the interview people to bring it up themselves. I asked questions such as "Does your product have a specific type of users?" with follow up questions like "How well do you know your users, what can you tell me about them". Out of the five people we interviewed, I could only find one time that they brought up something related to their users demographics being important. Otherwise they talked more about type of interest they had, the role they had at the company and about finding out about their users behaviour.
Though, it's important to include here that Google Analytics often was brought up in the interviews so a possible reason could be that Google Analytics have made it so easy to access demographics that UX Designers aren't even reflecting that it's not an option to know all about the demographics about your users.
Though, it's important to include here that Google Analytics often was brought up in the interviews so a possible reason could be that Google Analytics have made it so easy to access demographics that UX Designers aren't even reflecting that it's not an option to know all about the demographics about your users.
Interview Quotes
The data we get is demographics via Google Analytics, though difficult to see problems and behaviours.
UX Designer - Product Company [Stockholm]
No more quotes
Which might state that the hypothesis is true
No more quotes!
Which might state that the hypothesis is true
Insights
UX Designers doesn't share a common viewpoint on the importance of demographics in understanding their target group.
A true pain point that was talked about in the interviews was how to understand their users behaviour and that is something they aren't finding in the demographic data. Though according to the survey, there is a slight tilt towards demographics being an opportunity in understanding their users, for those using tools like Google Analytics would be considered a real gain. Though looking into the spread of how people responded in the survey, you have to take into consideration that there is multiple different views if it's a gain or pain in knowing the demographics for different colleagues in your team.
Principle
Allow UX Designers to be open to both use demographics and not for understanding their target group.
Next step
Research to find out if demographics is of importance for different types of products or if it's that different UX Designers think it's important or not
What I've learned
Jump.
Well yes, this exercise taught me all about the methodology that can be used for user research. But it did it, in what I would believe is the Hyper Island way, it did it by forcing me to "trust the process" and just do it. I was not in the clear when we started, I quickly started writing what a UX Designers is, their needs, jumping into how it should be prioritised and created that survey.. But then, when I normally would stop, when I normally would sit there, change a little, doubt a little, wait a little.. I was forced to send out the survey. It had, what I can see know, many parts that could have been better. Though also many parts that worked really well. Even as cliché at it might sound, all the wrongs I did, those were the ones I learned the most from! I learned that sending out a survey to UX Designers, then you should define way more in the screener survey so you can really trust the data you're getting. I should have straight away read up more on how the double diamond framework worked. I should have done the interviews in English so I could have used the quotes without translating and risking loosing that "personal touch" from the person I interviewed. Though, all of those things I brought up here and many, many more, are things I've learned, without a teacher, book, or class-mate pointing it out. So, I would like to thank Hyper Island and Martina (the one leading this journey for us) for allowing us to learn this way.
Thanks!
Thanks!
Final words
Trust the process and jump!