How to select the right research method with maximum gain

Let's look at the cycle "Idea — Testing — Development — Evaluation" again. Whether you're working on a product or a marketing campaign, you:

1. Look for unmet needs and formulate ideas to solve those needs — IDEA
   
2. Check ideas, leaving only viable ones, prioritise — TESTING
   
3. Design your product, focusing on implementation, choosing the best options — DEVELOPMENT
   
4. Evaluate the obtained results — EVALUATION

Take a look at the steps listed above. Which stage are you at? This will define the type of methods you should use to get the best outcome.

(*) in case a product is already available

ADDITIONAL CONSIDERATIONS
When selecting a research method

Necessity and sufficiency. Choose the method requiring the least investment of time and money, provided that the information obtained is sufficient to make the required business decision. Time spent = organisation + data collection + analysis. Money = direct costs (including the cost of internal resources) + own time. Different methods have a different balance of these costly parts.

To what extent are consumers aware of their own behaviour?
You need to understand your consumer's behaviour and the level of their awareness during the purchasing cycle. Is your consumer able to reflect on the process of selection, purchase and use and clearly communicate it in words?
For example if you have a situation with high awareness you would choose polls and interviews. This would apply to a situation where a consumer has to select a tariff plan for a mobile phone for example. If you have a situation with low awareness you would use data analysis, observation or experiment (including AB testing). This would apply in situations where you have a product like orange juice: imagining buying juice in a supermarket, where all juices are about the same and the decision is heavily influenced by advertising, placement on shelves, packaging design, and even temperature and music in the sales area.

Key objectives of the idea stage:
1) to understand the needs of the target audience,
2) to find, formulate and shape up product and marketing ideas,
3) understand the reasons and find solutions to the observed problems.

Input:

Ideas (hypotheses) are not direct products of research. They are born in your head. Research merely gives you food for thought, clues, additional stimuli for the increasing neural networks in your head.

Let’s see what clues you can gain from each method.

Data. If the product is already launched, you have sales data (volumes, seasonal variation data, channels, frequency). You may also have data on how people use your product. Analysis of this data will help to find out:

If there are any problems?
If so, which ones?
At what stages do they arise?
How is the product actually used?

Then you might want to describe the current state of affairs, correlate with expectations, make a diagnosis or formulate hypotheses. You can also search for similar data on competing or related products that meet similar needs of your target audience (industry reports, similarweb …)

Observation. Here we would study in detail what exactly happens at one or another stage of using the product. This can be an observation in a "natural environment" or in an artificially created simulation. You can see more detail with your own eyes than in the data. For example, simply by looking at the way a user moves their cursor, you can see that the person is in doubt or hesitating to decide where they want to go next on the page. Or observing a person in a supermarket can tell you what attracts their attention and make assumptions about the factors that influence their behavior.

The interview will help to understand in detail how people behave, why they behave the way they do, why they use or do not use the product, what they lack, what is their need and how they are now solving it. An interview is a conversation "about life", about how a person understands it or perceives it. The interview will also help you: understand in detail the reaction of people to your ideas (what is catching on, what isn’t and why) and to formulate ideas in a language that is clear to people in order to test them at subsequent stages.

Key objectives of the testing stage: make a go\no-go decision, prioritize.

Input: product and marketing ideas formulated at the previous stage.

So, now you have some product or creative ideas. Should you develop them right away? You could, but it would be better to check them first. This will help: 1) not to waste efforts on something that won’t work with consumers; 2) focus on the most important or promising ideas first; 3) refine the idea and thereby increase the likelihood that you will end up with something really good.

At the previous stage, you have generated and formulated ideas. And you used qualitative methods to do that. Now you need to quickly test the ideas you have developed and assess their potential. For this you can use quantitative tools. How many people would potentially buy your new product A, how many would prefer product B? How many users of your product need a particular feature? How many people find your USP relevant?

Survey. Let’s find out how many people like your idea. Let’s evaluate the potential and compare ideas against each other. You should formulate the idea in a format that is clear to people, demonstrate it to the target audience and ask them to evaluate it by filling out a questionnaire. Key questions: who to interview, how to invite people to take part in the survey, how many people do you need to evaluate the idea.

Experiment, behavioral data analysis, A / B testing. These methods are suitable when we no longer have just an idea, but a working product or a prototype of a product. Moreover, said prototype should be able to give people a good idea of the real experience with the product allowing them to adequately perceive it. These methods allow you to test ideas without asking any questions.

The development stage objectives:
1) if there is more than one option for the implementation of product or advertising elements, choose the best one;
2) evaluate the "suitability" of the implementation option and make a decision to launch as is or refine.

Input: product and advertising ideas that were selected for implementation at the previous stage.

Elements of an idea

Okay, now that you have the marketing and product ideas that have already been tested at the previous stage you can start developing them. To do it well, you need to make sure that all the constituent elements are done well, because the overall success of the product or advertisement depends on the quality of all of these elements put together.

Depending on what exactly you are developing (product, feature, advertising, etc…), the result will be composed of different elements. Here are just some examples of it: name, packaging, design, functionality and how the product works, custom features, USP, price, advertising messages, visual images, logo, mechanics and promo terms, prizes, characters, celebrities.


What format should you test in (designing and presenting the idea elements for testing)

To find out how many people like a particular implementation, people need to be shown something to evaluate. This something can vary greatly depending on what exactly you are developing. For a bank, for example, this can be a landing page of a product or a page with a loan calculator. For a physical product, this could be a look and feel or a "how it works" diagram. For a promo, this can be a description of the mechanics of participation and prizes. For advertising — a choice of several celebrities. The main thing is that the respondent should interact with a self-sufficient element of the offering otherwise the testing will be of no use (it makes no sense to test the color of a button in isolation from the rest of the interface).


To find out if it’s good, you need to know the potential scope

To solve the problems in this stage, you will need quantitative tools again, because you need to answer the question "how many":
How many people like this implementation option?
Which of the implementation options do you like more?

Survey. Let’s find out how many people like the implementation of the element, and how many don’t. If there are several options for an element, then we compare them with each other. We demonstrate the options to the target audience and ask them to evaluate them by filling out a questionnaire.

Key questions: who to interview, how to invite them, how many people should evaluate the implementation option.

Experiment, behavioral data analysis, A / B test. Stress testing. These methods are suitable when the implementation of the element can be demonstrated to respondents in a way that they can "feel" it in some form, close to reality (prototype). This allows you to test ideas without asking questions.

The tasks of the evaluation stage:
1) evaluate the actual achieved results of advertising or the product;
2) correlate the results with the set goals and make a decision whether to continue, modify or not to continue;
3) draw conclusions for the future.

Input: a product that has been out there for some time (so there are real users) or an ongoing or recently completed advertising campaign.

So, you've launched a product or ran an ad campaign. A certain number of people interacted with it and formed their own view of it.

Result
What is the result? The result is (achieved or not) what we want to evaluate at this stage.

Some results can be measured directly through the available metrics (sales, conversions, online ad reach …), but this isn’t always the case. For example, recommendations often happen offline or through channels that cannot be measured (word of mouth), which is why there is such an indirect assessment method as NPS ("intent to recommend" instead of "how many actually recommended"). In digital advertising and in the case of a digital product, it is easy to trace the entire funnel from impression to purchase. However, for offline advertising (TV, outdoor, etc…) or assessing the impact of a campaign on sales is much more difficult as there are no reliable universal "points of measurement". Therefore, metrics are not always enough and you have to supplement with surveys.

Analysis of behavioral data and metrics. For any product and for any advertisement, there is always going to be some "data" available that tells about their effectiveness.

Survey. Suitable for the following situations: 1) the business goal is defined in terms of satisfaction or perception; 2) there is no way to evaluate through "hard" metrics based on data. Hard metrics are site visits, page views, clicks, downloads, conversions and other metrics that are directly connected with the actual purchase)