Why Fastuna Has One Set Of Norms For All Categories
Our platform uses a set of built-in norms to evaluate creative materials or product ideas in simple terms: excellent, good, average or bad. These norms help measure your idea against all other ideas tested on Fastuna's platform. One set of norms is used across all industries.
On occasion, Fastuna users have wanted us to add an industry-specific norm. They suggested it might work better for people in their field, since the product or service categories can be so different. For instance, some people in the pharmaceutical industry think that norms should be lower than average as their field is very specific and developing a creative for medicines is challenging. Other people in FMCG believe that norms should be higher than average as they develop a creative for a wide range of product categories where competition is high.
So, we set out to see if the norms really do change depending on the type of product or service, and here’s what we found.
Comparing norms in different categories: no significant differences
We used actual data to test this hypothesis. Specifically, we used all of the historic test results from the Video Test solution — one of Fastuna's most popular solutions. There are more than 1500 ad tests in the platform database, allowing us to compare results across industries.
Advertising is a high-risk category because it can be very expensive to develop and distribute creative materials. If you make a mistake, you’ll take a financial hit. Video ads are where the main battle on TV for consumer attention takes place — for example, a digital service ad can be followed by an ad for children’s products and an ad for new vehicles. In the digital space, however, there’s even more competition — it’s not easy to break through the social media noise.
Preliminary evaluation of creative ideas is essential for choosing the best one for development, improving final details and predicting audience reaction.
Solution used to test the hypothesis: Video Test
Base: 1506 concepts
Categories: FMCG, electronics and appliances, pharmaceuticals, automotive, apparel, telecom, finance, online services, retail, hospitality, travel and tourism, education, media and entertainment, real estate and more.
How we calculate norms: We calculate norms based on the data collected from projects using a particular solution. For each metric (KPI), we calculate four ranges: bad (lower 25%), average, good or excellent (higher 25%). We then analyse all of the historic ads within those ranges. The top 2 scores are used in our analysis — the percentage of people rating 4 and 5 out of 5. This value is compared to the existing benchmark ranges to determine whether an ad is considered excellent, good, average or poor.
Base: 1506 concepts
Categories: FMCG, electronics and appliances, pharmaceuticals, automotive, apparel, telecom, finance, online services, retail, hospitality, travel and tourism, education, media and entertainment, real estate and more.
How we calculate norms: We calculate norms based on the data collected from projects using a particular solution. For each metric (KPI), we calculate four ranges: bad (lower 25%), average, good or excellent (higher 25%). We then analyse all of the historic ads within those ranges. The top 2 scores are used in our analysis — the percentage of people rating 4 and 5 out of 5. This value is compared to the existing benchmark ranges to determine whether an ad is considered excellent, good, average or poor.
We allocated all ads to their respective categories. For each category with a sound base we calculated benchmarks, and we eventually produced summary tables to showcase findings.
Table showing scores for the "Like" metric
As you can see, there are no significant differences between industries when it comes to the results of this analysis. Therefore, general market norms are relevant when evaluating a creative for any category.
Let’s get specific
It is true that there were differences in some metrics between categories, but they do not affect the combined score of the creative and therefore cannot be critical. As a matter of course, we have looked into those differences to ensure that no stone has been left unturned.
The financial products and services ads typically score lower on "Clarity", "Relevance", "Believability" and "Will try / use" parameters. This is not surprising, because an ad for a financial product such as a stock requires a sophisticated decision-making process in order to arrive at the stage where consumers are ready to invest their money. An ad for an FMCG product such as chewing gum, on the other hand, can easily invite the consumer to try the product because its low cost does not require a long decision making process.
At the beginning of this study, one of the assumptions was that the pharmaceutical market is very specific, so creativity in this industry is very limited, and therefore benchmarks should also be specific to it. However, the study did not reveal significant differences with the market average in this category.
Why do market averages matter?
The market is a united information field. Within different communication channels you may find competition from companies that offer products similar to yours but also from companies that are not apparent competitors. Moreover, you may have consumers outside of your main target audience.
Because of the competition for consumers' attention, your creative work must stand out in the information-saturated environment. It should generate interest and desire to learn more, try the product or service, and also make consumers want to share information about it. In contrast with industry-specific norms, market-wide norms give you a chance to evaluate your creative potential in terms of the whole market.
While, it is worth bearing in mind the small differences in the way ideas are evaluated between different categories, it is important to remember that scores that are too high or too low in different categories can give a skewed view and lead to incorrect interpretation of the test results. This is why it is more reliable and effective to base your analysis on the whole market data.
To keep up-to-date, we update the norms on our platform every year. Who knows what the market will look like in the future! Stay FASTuned!
Innovate with confidence
Validate product and marketing decisions with real people in 24h using our hassle-free testing tool