Concept Testing Blog

How Amazing Is My Concept?

Ask 10 different market insights professionals how they manage concept testing, and you’re liable to get 10 different answers.

How, then, do you know which approach to use to answer the question that all marketers, product managers, and brand managers commonly ask? Namely, “Just how amazing is this new product/service concept going to be?”

There likely isn’t a singular correct answer, I’m sorry to say. Some options are going to be better than others, but any testing is better than just guessing or, worse, hoping things will turn out for the best. Often, the answer is going to depend on how much time and budget is available to test the concept(s).

Let’s be clear. Concept testing isn’t product testing. These are new(ish) products or service ideas, not prototypes. Concept testing precedes product testing and is often used to decide whether an idea will move down the new product development pipeline to product testing. I’ve had the pleasure of testing a variety of new concepts, such as:

A common question is, “Do you test concepts monadically (one per participant) or not?” My response is typically that we prefer to test them monadically, but we recognize it’s not always feasible to do so. A monadic approach requires larger sample sizes since each concept should have a representative group of consumers evaluating it. An alternative to this is sequential monadic testing, where each participant evaluates multiple concepts one at a time in random order. A sequential monadic approach requires less sample but has a longer survey length to account for repeating questions for multiple concepts.

The next question that arises, then, is, “How many concepts can you test at the same time?” The answer is it depends. You can test as many concepts as you want if you have enough sample to go around. The better question is how many concepts do you have each participant review? I like to cap it at 4 or 5, if possible, but I’ve had participants review up to 8 concepts in some circumstances. That makes for a long survey, and we all know that long surveys call into question respondent fatigue. If you have many concepts, you should likely think of the project more as concept screening and less full-fledged concept testing. Reserve the more in-depth concept testing for a final set of viable concepts. An experimental design will help you allocate the concepts equally across your total participants, rather than forcing them all into the same 3, 4, or 5 buckets without any variety in their mix.

Another question I often hear is, “How detailed does my concept need to be?” Concepts don’t need to be detailed to be evaluated, but they should at least include a basic description and reasons to buy. Beyond that, you can include images of the product or service that help explain what it does, show a brand name (or leave it unbranded), and provide a brief list of available options. We normally don’t recommend including pricing initially but rather bringing it in after asking a number of key metrics to understand if there is a disconnect between concept appeal and value measures. Here is an example of a very thorough branded product concept:

It doesn’t have to be that elaborate if you are at an earlier stage in concept development. You could get away with testing a simple description such as “This new facial cream is made from all-organic and responsibly farmed ingredients, including, starfish extract, aloe vera, tea tree oil, almond oil, and organic beeswax.” Whatever level of finished concept you are evaluating, the key is to ensure that all concepts evaluated against each other have the same level of finish. That should go without saying.

Key metrics in concept testing often include:

You then compare the results of the key metrics across the concepts or to your normative database to find the concepts with the highest scores. These you take to the next level. Many clients will apply weights to the key metrics to arrive at a composite concept score for each concept—individual key metrics may be confusing to evaluate if they offer conflicting viewpoints (high appeal but low priced purchase interest, for example).

You can see why there are so many different paths to successful concept testing, even though the basic premise is the same across the board. The results I discussed are focused on quantitative testing, but there are qualitative methods that also can be employed. If you’re in a quandary about what solution might work well for your situation, feel free to reach out. I’m always happy to chat about research!

Tom Allen

Senior Vice President

Email Tom

Tom has over 20 years of research experience spanning several industries, including retail, restaurants, consumer packaged goods, and financial services. He earned a Bachelor of Business Administration from The University of Texas at Austin and a Master of Business Administration in Marketing from The University of Texas at Arlington.