When do I use Generative vs. Evaluative research methods?

When do I use Qualitative vs. Quantitative research methods? #

Generative vs. Evaluative -UX Research
 

Generative


Evaluative


Purpose


Defining the problem & identifying a solution


Understanding the effectiveness of an existing product


Orientation


Future


Present & near future


Timeframe


In the beginning of a research project


Throughout the product design process


Goal


  1. Generate insights
  2. Identify needs
  3. Spark innovation

  1. Assessing concepts
  2. Verification

Qualitative methods


  1. Stakeholder interviews
  2. Ethnography
  3. User interviews
  4. Focus groups
  5. Diary studies
  6. Card sorting

  1. Lab testing
  2. Moderated usability testing
  3. Focus groups
  4. Moderated card sorting

Quantitative methods


  1. Surveys
  2. Interviews

  1. Survey
  2. Tree testing
  3. A/B testing
  4. First click testing
  5. Preference testing
  6. Task analysis
  7. KPI monitoring

What is Generative research? #

Generative research. also called exploratory research, is a method you use at the very beginning of the product cycle when you don’t even know what problem to solve for your customers. It’s all about generating a deep understanding of your costumers motivations, pain points, behaviours, and needs. 

The purpose is for you to explore and formulate a problem statement to solve. You do this by going places, watching people, interviewing customers, experts and sometimes also random people, researching the Internet very broadly amongst other activities.

In interviews, you ask very broad and generic open question without even giving the interviewees tasks to solve. Instead, you want to understand their life, environment, behaviours to engage in their problems to be able to identify them.

It is not about the customers telling you their problems, instead you need to synthesize, detect and understand patterns and map out your own pattern and thus detect a problem to be solved. In this stage it’s all about keeping an open mind as you don’t yet have a clue of the final solution. The goal and purpose of generative research is about discovering and formulating, which problem you solve (for whom?) and why this makes sense for your customers. 

Why is Generative research valuable? #

Generative research is important and valuable because if you fail to address the right problem your team ends up designing and building a product that  your customers don’t truly understand, need or use. The key with generative research is focusing on developing the best solution by properly identifying the problem, not developing the best solution by doing lots of evaluative research and refining your design. 

Generative research can help you identify new opportunities for an existing product or improve the solution you’ve already created. It is great to use when you are trying to innovate as it’s an effective way to discover new insights. Put your efforts on designing solutions that solve relevant problems and add real value to your costumers.

Examples of Generative research: #

Diary studies: ask your participants to write down their everyday activities, behaviours, and thoughts for a given period of time when they perform a task such as moving flats, buying a car or grocery shopping.  This way you will get a deep understanding of their habits and challenges along their journey.  

Focus groups: invite ten of your customers to openly discuss their actions, thoughts, and feelings around your product. 

What is Evaluative research? #

Evaluative research, also called as evaluation research, is a method you use after you have defined and confirmed your problem statement. In this stage you evaluate and test your solution ideas and product concept to ensure usability and ground in form of wants, needs, and desires. You conduct this by approaching people with your ideas and try to find out which work best and how to twist and turn them. This method helps you find a match between your product and your brand messages and potential vision.

For best results make sure to conduct these kinds of UX tests frequently, by integrating them smoothly into your development flow. Introduce the tests early in the phases of your design process to test existing or new solutions and continue to employ it in an iterative way until the product becomes ‘final’.

The goal of evaluative research  is to understand which solutions will work the best and if your solutions are getting closer to solving the problem you have chosen. Did you do enough explanatory research to come up with the right solutions? And does the solution meet your costumer’s needs in terms of is it easy to access and use, and is it hopefully valuable? 

Why is Evaluative research valuable? #

Generative research is important and valuable because it helps you make sure that your team’s efforts and resources aren’t wasted. By continuously testing your solutions and collecting valuable costumer feedback can help you refine and improve the user experience along the journey. Findings from evaluative research are key to assessing what works and what doesn’t and identifying areas of improvement. By evaluating your product early and often, you’ll know whether you’re building the right solution for your costumers. The insights also helps you show the effectiveness and impact of your projects, data you can present to your stakeholders to get buy-in for future projects and achieve your product goals efficiently. Build it right the first time.

Examples of Evaluative research: #

Usability testing: test your concept with real users by having them speak aloud while completing a list of tasks. 

Survey: create a questionary with closed-ended questions. 

UX Dimensions - Research Toolbox Method-ttitudinal vs Behavioural methods?