You are here:
Home |
Research Services | Product Testing
Product Testing
Product testing is, perhaps, the single-most-important type of consumer research any company ever conducts.
A company with consistently superior products tends to consistently outperform its competitors in the marketplace.
Achieving clear-cut product superiority in a category is the surest way to build brand share, engender customer loyalty, and
boost profitability. Better products tend to command higher prices and be more responsive to advertising investments. |
 |
Our Product-Testing Philosophy
Consumer tastes and preferences evolve over time. A consumer's palate is a moving target. That's why product testing must be viewed as a strategic activity.
"Real environment" testing (i.e., testing a product the way it is typically used by consumers) is almost always the most accurate method of product
evaluation. For example, it is generally best to let consumers evaluate food products in their homes rather than in a laboratory or test kitchen.
Products should be tested using a standardized analytical research system, so that each product is tested in exactly the same way. Our analytical research system is Optima®.
Optima® Product Testing
Optima® is Decision Analyst's proprietary, copyrighted product-testing system. The Optima® core questionnaire consists of the following modules:
- Screening/Qualification questions
- Overall rating of product
- Likes
- Suggested product improvements
- Diagnostic ratings of product attributes
- Ratings of product components
- Purchase intent, priced
- Expected purchase frequency
- Value rating
- Demographics
If the product test is a part of a new-product volumetric forecast, then additional modules are added to the core questionnaire.
How Does Optima® Work?
Typically, a representative sample of category users (150 to 200 households)
are given a test product to use in home for a few days. Then these consumers
are asked a series of standard questions about the product.
Based on internal diagnostics, normative data, and analytical models, these
questions tell us whether the product is optimal or not, and indicate what needs
to be changed to improve the product. Our primary analytic model is Pii®.
Product Improvement Index® (Pii®)
Decision Analyst developed the Pii® mathematical model to help guide
product-development efforts for new products and the reformulation of existing
products. Pii® was developed primarily because of
problems encountered in using various types of regression models in product-testing
analyses. Regression models assume that all of the input variables are independent
(i.e., not intercorrelated in any way).
The reality is, however, that virtually all of the input variables that might explain a product's performance are typically intercorrelated. The result is a
regression equation that masks or omits important input variables. For example, if the color and the sweetness of a product happened to be highly correlated with each
other, the regression equation would omit one of the variables. We might think we had a color problem, when in fact we had a color and a sweetness problem.
The Pii® model was designed to circumvent the "missing variables" problem associated with regression. This model is based upon a type of correlation, using "dummy"
variables, to examine the relationship between the diagnostic ratings (e.g., too sweet, about right, not sweet enough; or too much salt, about right, not
enough salt) and the consumer's overall rating of the product. The overall rating is typically measured with an 11-point scale.
The output of the Pii® model is a table of important explanatory variables along with the Pii® rating and the indicated action, as illustrated here.
| Diagnostic Variable |
Pii® Score |
Indicated Action |
| Too sweet |
18.65 |
Reduce sweetness |
| Too dark in color |
14.72 |
Make product lighter |
| Too soft |
12.95 |
Make product firmer |
| Not enough salt |
9.48 |
Add some salt |
| Not enough crunch |
5.23 |
Make product crunchier |
Generally, any Pii® score greater than 4.0 indicates that some modification of the product might be necessary. The greater the Pii® score, the more
important that variable is and the more that variable should be modified.
Optimization Methods
In addition to Pii® analyses, response-surface and choice-modeling analyses
are the primary optimization techniques. Experimental designs and simulation
models are employed to optimize products. By testing chosen subsets of product
possibilities, response surface and choice modeling can simulate and predict
consumer preferences for hundreds of product possibilities, as defined by variations
in ingredients, features, elements, or packages. The resulting equations are
used to build an optimization simulator so that “what if” products
can be fully explored and understood. The goal of optimization can vary. It
might be maximizing consumer preference, or maximizing the profit margin without
losing market share, or maximizing sales potential. The optimization simulator
also helps reveal “cause and effect” as inputs are changed and outcomes
vary.
Optima® Uses
The value of Optima® product testing is illustrated by its many uses:
- To evaluate and improve existing products.
- To measure the threat posed by competitive products.
- To continuously improve product performance over time (i.e., to optimize products).
- To evaluate cost-reduction formulations while maintaining product superiority.
- To measure the effects of aging upon product quality (shelf-life studies).
- To provide guidance to R&D in developing or upgrading products.
- To monitor product quality from different suppliers and/or different factories.
- To implicitly evaluate marketing variables (i.e., packaging, pricing, sizing, etc.).
- To predict the success of new products.
Optima® Features
The essential features of Optima® product testing are:
- Monadic Design. Each product is tested alone. This provides the most accurate evaluation of the product and the best
diagnostic feedback about how to improve the product.
- Standardized Systems. The sampling, data collection, data preparation, and data tabulation methods and procedures (i.e., the system)
are standardized for each product. It is essential that every product be tested in precisely the same way.
- Standardized Questions. The standardized questionnaire is modular in structure and flexible in design. All of the questionnaire modules
are copyrighted by Decision Analyst.
- Analytical Research System. Optima® is an integrated research system with mathematical models to yield greater analytical insights.
Sampling Options
The recommended sample source for most product tests is American Consumer Opinion® Online. This Decision Analyst panel of over
8 million online consumers is an economical way to screen for low-incidence product categories, and it provides high-quality data and fast turnaround at
the conclusion of each product test.
Other sampling options include mall-intercept, recruit to central location, and telephone recruitment. Central location, in-store, and on-site product testing is also an option.
Nonfood Product Testing
The concepts, methods, and techniques of product testing can be adapted and applied to almost any product category. Our staff has evaluated:
- Airliner seats
- Calculators
- Cellular phones
- Comforters
- Computers
- Educational toys
- Electronic games
- Film & film processing
- Food service products
- Frying pans
- Game prototypes
- Gaming devices
|
- Glass windows
- Hotel rooms
- Microwave ovens
- Pagers
- PDA's
- Personal computers
- Restaurant entrees & side dishes
- Restaurant exteriors &iInteriors
- Retail store layouts
- Software
- Washing machines
- Websites
|
Product Testing Services
Decision Analyst is a recognized leader in consumer product testing and optimization.
Its staff has evaluated more than 1,000 foods, beverages, and other products
during the past 32 years. The firms has over 50 staff members with extensive
experience in the conduct and analysis of producting test and optimization studies.
The company is a leader in the development of analytical techniques to enhance
product testing and optimization.
If you would like more information on product testing, please contact Jerry W. Thomas by emailing jthomas@decisionanalyst.com
or calling 1-800-ANALYSIS (262-5974) or 1-817-640-6166.
Additional Resources From Decision Analyst
Related Services
Brochures
Case Studies
Related White Papers
