Jerry W. Thomas
Based upon 30 years of marketing research experience, spanning thousands of research projects, I am convinced that
product testing is the single most valuable marketing research that most companies ever do. The
great value of product testing is, perhaps, best illustrated by some of its many uses. It can be used to:
- Achieve product superiority over competitive products.
- Continuously improve product performance and customer satisfaction (i.e., to maintain product superiority, especially as consumer tastes evolve over time).
- Monitor the potential threat levels posed by competitive products to understand competitive strengths and weaknesses.
- Cost-reduce product formulations and/or processing methods, while maintaining product superiority.
- Measure the effects of aging upon product quality (shelf-life studies).
- Implicitly measure the effects of price, brand name, or packaging upon perceived product performance/quality.
- Provide guidance to research and development in creating new products or upgrading existing products.
- Monitor product quality from different factories, through different channels of distribution, and from year to year.
- Predict consumer acceptance of new products.
Companies committed to rigorous product testing and continuous product improvement can, in most instances, achieve product superiority over their
competitors. Product superiority, in turn, helps strengthen brand share, magnifies the positive effects of all marketing activities (advertising,
promotion, selling, etc.), and often allows the superior product to command a premium price relative to competitors.
Most companies, unfortunately, do very little product testing. Few companies really understand the power of continuous product improvement and product
testing. Even fewer companies know how to do product testing the right way. Fewer yet budget enough money to support a serious product-testing program.
These shortcomings in the majority of companies create opportunities for the minority of companies who are dedicated to continuous product improvement. How
can companies realize optimal value from product testing?
Product Testing Secrets
The secrets to truly accurate and actionable product testing are several:
- A systems approach. The methods and procedures of product testing should constitute a standardized
system, so that every like product is tested exactly the same way, including identical product preparation, age, packaging, and coding; identical questionnaires
(of course, parts of the questionnaire must be adapted to different product categories); identical sampling plans, typically employing blocking-screening
grids to ensure matched samples; identical data preparation and tabulation methods; and similar analytical methods.
- Normative data. As products are tested over time, the goal is to build normative databases, so
that successive product tests become more meaningful and valuable. The normative data, or norms, continually improve a companys ability to correctly
interpret product-testing scores, and the norms help reveal exactly how good, or how bad, the test product is.
- Same research company. Use one research company for all of your product testing. This is the only
way you can make sure all tests are conducted in exactly the same way.
- Real environment test. If the product is used in offices, it should be tested in offices by people
who work in offices. If the product is typically used at home, it should be tested at home. If the product is consumed in restaurants, it should be tested
in restaurants, and so on. In general, this kind of real environment test will produce the most accurate results. For example, for food products,
an in-home usage test is almost always more accurate and predictive than a central-location taste test.
- Relevant universe. Sampling is a critical variable in product testing. For new products or low-share
products, the sample should reflect, or represent, the brand share makeup of the market. For well-established, high-share or highly differentiated products,
the sample should contain a readable subsample of that products users, and a readable cell of nonusers. If the product category is underdeveloped
(e.g., a relatively new category), then the sample should include nonusers of the category, as well as users. Also, its always important to represent
medium to heavy users of the product category in the final sample.
In summary, if a companys brand share is very low, its important to assign more weight (or importance) to the opinions from nonusers of the brand.
If brand share is very high, then what brand users think is more important.
- Critical variables.
Product performance and quality must be defined from the consumers perspective, not the manufacturers. What aspects of the product are truly important
to consumers? What critical variables determine the consumers satisfaction with the product? These critical variables must be identified for each product
category (typically, with focus groups or depth interviews) to design an accurate product-testing system.
- Conservative actions. The formulation of an established product should never be changed without
careful testing and evaluation of the new formulation. Once you are sure you have a better product, introduce it into a limited geographic area for a reasonable
time period (several product repeat purchase cycles). Then, and only then, roll the new product out to all markets. The smaller the market share, the
greater the risks that can be taken with a new formulation. The larger the market share, the more conservative one should be in introducing a new formulation.
The Major Techniques
The monadic, sequential monadic, paired-comparison, and protomonadic are the most widely used research designs for product testing.
- Monadic testing typically is the best method. Testing a product by its own offers many advantages. Interaction between products (which occurs in
paired-comparison tests) is eliminated. The monadic test simulates real life (thats the way we usually use products, one at a time). By focusing
the respondents attention upon one product, the monadic test provides the most accurate and actionable diagnostic information.
Additionally, the monadic design permits the use of normative data and the development of norms and action standards. Virtually all products can be tested
monadically, whereas many cannot be accurately tested in paired-comparison designs. For example, a product with a very strong flavor (hot peppers,
alcohol, etc.) may deaden or inhibit the taste buds so that the respondent cannot really taste the second product.
- Sequential monadic designs are often used to reduce costs. In this design, each respondent evaluates
two products (he or she uses one product and evaluates it, then uses the second product and evaluates it). The sequential monadic design works reasonably
well in most instances, and offers some of the same advantages as pure monadic testing.
One must be aware of what we call the suppression effect in sequential monadic testing, however. All the test scores will be lower in a
sequential monadic design, compared to a pure monadic test. Therefore, the results from sequential monadic tests cannot be compared to results from
monadic tests. Also, as in paired-comparison testing, an interaction effect is at work in sequential monadic designs. If one of the two
products is exceptionally good, then the other products test scores are disproportionately lower, and vice versa.
- Paired-comparison designs (in which the consumer is asked to use two products and determine which product
is better) appeal to our common sense. Its a wonderful design if presenting evidence to a jury, because of its face value or face validity.
The paired-comparison can be a very sensitive testing technique (i.e., it can measure very small differences) between two products. Also, the paired-comparison
test is often less expensive than other methods, because sample sizes can be smaller in some instances.
Paired-comparison testing, however, is limited in value for a serious, ongoing product-testing program. The paired-comparison test does not tell us when
both products are bad. The paired-comparison test does not lend itself to the use of normative data.
The paired-comparison test is heavily influenced by the interaction effect (i.e., any variations in the control product will create corresponding
variance in the test products scores).
- The protomonadic design (and the definition of this term varies greatly from researcher to researcher) begins as a monadic
test, followed by a paired-comparison. Often, sequential monadic tests are also followed by a paired-comparison test. The protomonadic design
yields good diagnostic data, and the paired-comparison at the end can be thought of as a safety netas added insurance that the results
are correct. The protomonadic design is typically used in central-location taste testing, not in-home testing (because of the
complexity of execution in the home).
Nonpackaged Goods Categories
While most product testing is conducted in the food and beverage industries, the concepts and methods of product testing are applicable to virtually all
product categories, although the structure and mechanics of execution will vary greatly from product category to product category. For example, computer
software can be tested, furniture can be tested, store environments can be tested, dog food can be tested, airline service can be tested, equipment
prototypes can be tested, etc.
The ultimate benefit of product testing is competitive advantage. Product superiority is the surest way to dominate a product category or an industry.
Companies dedicated to ongoing product improvement and product testing can achieve product superiority, and achieve a competitive advantage of great strategic
Companies that ignore product improvement and product testing, on the other hand, may wake up one morning to find themselves on the brink of extinction
from a competitor who has built a better mousetrap.
Copyright © 1993 by Decision Analyst, Inc.
This article may not be copied, published, or used in any way without written
permission of Decision Analyst.
|Share This Article
| Download This Article