UX Research, Striking a Balance
Striking a balance between Quantitative & Qualitative Data
UX Research is a significant piece of the process in creating or improving new products. When and how you do the research should be planned. But what types of UX research should you decide to incorporate into your plan? Generally, there are two types of data that is collected in user studies: Qualitative and Quantitative.
Quantitative — metrics and actual statistics.· i.e. The time it takes to complete a specific task.
Qualitative — observational findings, behaviors and emotions. i.e. Identifying design features as being difficult or easy to use.
“Qualitative data is critical for measuring impact and ROI. Quantitative data can tell you the magnitude of a specific problem or behavior.”
Here are some real-life examples:·
Quantitative — The average time spent on the eCommerce checkout page is 13.5 seconds.
Qualitative — While conducting user testing, it was noticed that some of the participants had to re-read the first paragraph on the homepage before they understood the message.
Defective Metrics
The key to using both qual and quant is to not confuse the data and produce a defective metric.
Let’s say you are running three co-design sessions. During each of the sessions a participant states that they do not use the find my location functionality. Although you heard this response in every session, you can not say “100% of people do not use the find my location functionality.” You did not hear it from all the participants, and it’s likely that most of the participants were not directly asked if they use the functionality.
You should also be careful not to say: “At 100% of the sessions people said they don’t use the find my location functionality.” For example, if you had 10 people at each session and you received that feedback from one person at each session and the other people actually all use the functionality. That would mean that “10% of participants do not use the find my location functionality”.
If you compare the two side-by-side:
“At 100% of the session participants said they don’t use the find my location functionality”
“10% of participants don’t use the find my location functionality”
As you can see, the first claim can be recognized to have a prominent value, compared to the second.
The Method
It is always best to use both qual and quant. By using both you can identify the hypothesis and then find metrics that can then prove as the evidence to support the hypothesis. When only doing one type of research, you are often left with how the outcome has been translated.
Let’s look at a four-step process. You have captured quant research to examine the time spent on each of the four steps:
You might find that Step 2 takes a long time. You then conduct interviews with some of your end users. While reviewing the process , all of the participants state that “Step 1 is confusing”, “it isn’t explained very clearly”, “All I do is check a box and then click apply.” The issue is then not how long it takes to complete Step 2, but the abruptness of Step 1. You now have the qual of the observation and the quant of the time spent to present evidence and prove where the issue in the process truly is.
Research Techniques
Below is a list of some of the most day-to-day techniques that you can use for quant and qual research:
Analytics (Quant)
Analytics is used for the discovery and explanation of meaningful patterns in data.
Adding analytics to your website or app is a common way to capture quant data.
This data can generally be separated into two groups:
1) User
2) Technical
Feel free to contact: nikki@wolfeandsmith.com for more information on how to use google analytics in user research.
Funnel Analysis (Quant / Qual)
Funnel analysis is the process of monitoring the steps or events that occur during a process to lead the user to a select outcome. Relying on whether the process is tracked through analytics or is tested using prototypes. The data collected will be either quant or qual.
Heat maps (Quant)
A heat map records the locations of interactions on an interface. These can be configured to capture the clicks/touches or the mouse position. Software is available to capture eye movement and eye-tracking. This is often done during usability testing. The eye movement observations can then be aggregated to build an alternate heat map.
Surveys (Quant / Qual)
Surveys are a great way to capture information on a large-scale.
A/B testing (Qual / Quant)
A/B testing is a method where you compare two versions of a webpage or app against each other to govern the one that performs the best. For example, A/B tests are often conducted through an interactive prototype (with the use of analytics.)
Cohort Analysis (Quant)
Cohort Analysis implies breaking down data into similar groups for analysis. Comparing these groups can provide extra clarity around what influences specific behaviors.
Usability testing (Quant / Qual)
The results are often more qual than quant but some data trends can be identified. Usability testing is the approach to test interfaces with users before releasing them. The results give companies the confidence to move forward with product launches and reduces the risk of failure.
There are many ways in which usability testing can be conducted:
Storyboarding — using storyboards to understand the flow of user’s interactions with a product over time.
Tree testing — enable users to test the information architecture by asking them to accomplish a task. Conducting these tests allow us to look into how users would navigate to the appropriate place to complete an action or task. Tree testing is a method that is a quick, iterative evaluation of menu labels and categories.
Interactive Wireframes — Interactive wireframes or a proof of concept allow users to interact with a technology touchpoint. This can help identify gaps or missing requirements, design errors & usability issues.
Card sorting — enable users to organize and group content into logical areas to guide the information architecture. This is also a great tool in helping align or define hierarchies to make smarter and more validated design decisions. i.e Understanding how users group information can provide user-centered strategies in improving a navigation.
Contextual Inquiries (Qual)
Contextual Inquiry sessions are where users are observed and questioned while they work in their own environments.
Interviews (Qual)
Interviews allow you to ask questions to users to a deeper view into their processes and actions. Conducting interviews allows you to dig deeper into their motivations, relationships, frustrations/friction points and context.
Focus groups (Qual)
Much like interviews, however as opposed to one participant, the session is with a group of contributors. While working in group sessions it is continually vital to make sure all voices are heard.
Diary studies (Qual)
Diary studies are user logs (diaries) of daily activities as they occur. These logs provide contextual insights about real-time user behaviors and needs. Most importantly, these logs assist in helping define feature requirements.
I hope that this gives you a quick overview and you have found this informative. Please do not hesitate to contact me: nikki@wolfeandsmith.com if you want more in depth knowledge on any of these techniques listed or alternative techniques or methods that were not listed in this article.