Realism in Research:
Innovative Use of 3D Animation for
Qualitative & Quantitative Research Methodologies
Michael Richarme, Ph.D. and John Colias, Ph.D.
To a very large degree, people experience and interact with their surroundings from a visual perspective. An often-cited advantage of mail- and Internet-based
surveys over telephone surveys is the ability to present information (such as concepts, scales, and question wording) in a visual manner, easing the cognitive
burden on respondents and allowing for richer, more meaningful responses.
Yet all surveys inherently contain a potentially deadly flaw—the act of completing a survey does not mimic the true decision-making process utilized
The use of 3D animation helps to mitigate this limitation of surveys, giving respondents a more realistic decision-making context. This article will demonstrate
one 3D animation model and will discuss the incorporation of 3D animation into most choice-modeling research tasks.
An empirical study was conducted to compare the use of a traditional grid-based choice-modeling approach to a 3D animation choice-modeling approach. The sample
consisted of over 600 nationally representative adults in the United States. The respondents were randomly assigned to one of the two modeling approaches.
Analysis of the data shows encouraging support that the use of 3D animation in choice-modeling projects can result in a more accurate estimate of future
purchases as compared to traditional text-based choice modeling methodologies. The respondents tend to be more engaged in the research and tend to report higher
levels of satisfaction with the 3D animation approach. Respondent cognitive stress is reduced, and the level of frustration with the repetitive choice-modeling
task is reduced. Finally, the use of 3D animation reduces the price elasticities of the stimulus product.
Though the 3D animation methodology was utilized in this experiment with a single consumer packaged good category, it is expected that the methodology
could be similarly utilized in a wide variety of choice-modeling situations, where store-shelf comparisons are used by consumers to make purchase decisions.
The 3D animation lends an increasing amount of realism to the consumer decision-making process that is lacking in the traditional text-based, choice-modeling approaches.
The use of realism in advertising has been increasing (Warlaumont, 1998). This is partly due to an increased level of both short-term and long-term brand
recall when realistic ads are placed in a consumer’s stream of consciousness (Nelson, 2002). As an empirical finding, this has great practical value to practitioners.
Of considerable importance, however, is the need to understand the underlying theory of why the use of reality causes this increased brand recall, and whether
or not the use of reality stimulates purchase intent. To do this, it helps to understand how people learn about products. The more information and understanding
that people have about products, the more likely the firm is to exhibit long-term profit performance (Wernerfelt, 1996).
In terms of presentation format, Jiang and Benbasat (2007) examined four different presentation formats for online consumers, attempting to determine which of
the formats had the most efficiency and salience for building better product understanding. The four formats were presentation of a static picture, presentation
of a video without narration, presentation of a video with narration, and a virtual product experience. In this empirical study, they found that the virtual
product experience provided consumers with a higher level of product understanding than the other three formats.
Berneburg (2007) conducted a laboratory experiment in Germany with a single consumer product to examine whether or not virtual reality could be utilized
in a choice-based conjoint (CBC) study. Her conclusions were that the use of virtual reality (3D simulation) did not cause respondents to focus on the cool
technique at the cost of enhanced accuracy, and in fact it increased the accuracy of the part-worth estimation and resulting purchase intentions.
This empirical research utilized both qualitative and quantitative methodologies to examine the efficacy of 3D animation for choice modeling tasks versus traditional
grid-based, choice-modeling tasks. Online depth interviews were conducted for a small number of respondents, in which the respondents were connected to an
audio- and text-conferencing service, which allowed the researchers to converse with the respondent via the respondent’s computer monitor being mirrored
on the researcher’s desktop. When the survey was launched by the respondent, the researcher was able to talk with the respondent about the survey as it was
The second methodology was to conduct an Internet survey with random assignment of respondents to two different treatments. The first treatment utilized a traditional
grid-based, choice-modeling task, and the second treatment utilized a 3D animation-based, choice-modeling task. The survey and incorporated choice-modeling tasks were
exactly identical except for the use of traditional grid-based, or 3D animation-based, choice-modeling formats.
A nationally representative sample of 600 adults in the United States, aged 18-65, was selected from Decision Analyst’s 8 million member global consumer
panel, American Consumer Opinion® Panel. Using Icion®, a proprietary multivariate sampling tool developed by Decision Analyst, population characteristics
of age, gender, ethnicity, household income, education, and marital status were simultaneously considered for the sample. Respondents were subsequently screened
to be food and beverage shoppers for the household, and had to have purchased the stimulus item within the past three months to qualify for the study. In
addition, the respondents had to have purchased the stimulus item most often at a grocery, drug, mass market, or club store.
For the purposes of the choice-modeling task, the respondents were randomly divided into three blocks of eight scenarios, creating an experimental design
of 24 total scenarios. Each respondent viewed eight scenarios (eight shopping trips) for the choice-modeling task.
In addition, because respondents were each provided a cash incentive of $5 to complete the survey, a security question involving a brief video was utilized
to eliminate computer-generated responses and to ensure that the respondent’s computer equipment was capable of handling the survey. At the end of the survey,
respondents were asked another quality-control question to ensure that the answers were not computer-generated.
The choice-modeling stimulus utilized in the survey was microwave popcorn. This stimulus was selected for a variety of reasons. The researchers wanted
a stimulus that was relatively common in usage, that was purchased by all segments of the population, and that had relatively few numbers of brands from which
to choose. A moderate level of stimulus importance was desired to allow for brand switching and a manageable consideration set. The stimulus also had to
be available across the entire United States.
Using Decision Analyst’s Health and Nutrition Strategist™ (a large scale, longitudinal, searchable syndicated database) microwave popcorn was determined
to meet the stimulus criteria. From that database microwave popcorn was found to have been utilized by roughly half of the adult population in the past 30
days, with little difference in gender or income. Families with children had a slightly higher microwave popcorn usage than families without children (56%
vs. 47%, respectively), and popcorn with health claims (lower calories, low fat, etc.) tends to have a slightly higher consumption skew toward higher-income households.
Microwave popcorn had a moderate number of nationally available stock keeping units (SKUs), which allowed for a manageable choice task. A total of 47 SKUs
were collected for the choice modeling task.
Analysis and Results
In addition to the depth interviews conducted with several respondents, all survey respondents were asked a series of open-ended questions at the completion
of the survey regarding their survey experience. When assessing the 3D animation methodology, respondents indicated that the methodology was not unfamiliar,
given the prevalence of animated movies and computer games with this type of technology. In addition, the similarity to their real-world purchase environment
provided a better ability to simulate actual decision-making processes. Some comments from the qualitative research component relative to 3D animation are
“The graphics actually worked. It was interesting to see things placed side by side so I could visually choose.”
“It was like being in a real store with the same choices.”
“It actually showed me how I tend to choose my popcorn item. I tend to like the butter flavor but price has a lot of meaning.”
“You get to see all the products in front of you as though you were looking at a store shelf. I think having pictures with questions is helpful.”
When assessing the traditional methodology, respondents were not favorably impressed with the typical choice-model card. Some comments from the qualitative
research component of the project are as follows:
“Pics too small and my eyes straining; didn’t want to take so much extra time to click on each one to make them bigger.”
“Too many screens of the same information.”
With online or in-person qualitative depth interviews, it appears that the use of 3D animation triggers better recall of actual purchasing memories, thus
leading to more cogent and useful discussion with the moderators about purchase intentions and behaviors.
Question 1: Purchase Volume
There was a significant difference between the traditional choice-model technique and the 3D animation choice model technique, with the latter providing a reduced
stated purchase intention that was much more aligned with resulting purchase behavior. Thus, there is support for the hypothesis that the 3D animation methodology
reduces stated purchase volume in survey data. While the absolute accuracy of the reduced purchase volume relative to actual market data is the subject of
a future study, the finding of reduced stated purchase volume suggests that the 3D animation choice modeling approach more accurately predicts actual purchase
behavior than does the traditional grid-based choice methodology.
When asked to decide how many items would be purchased within one month, 3D respondents indicated a significantly lower purchase volume.
Question 2: Satisfaction with Methodology
Another element of investigation was whether or not the respondents viewed the 3D methodology as more satisfying than the traditional choice model approach.
A between-groups comparison revealed evidence that the 3D choice-modeling methodology was viewed as more interesting, engaging, and stimulating than the traditional
Other than selecting the same brand and selecting the same price, all the other items in this scale were significantly different between the two groups
The 3D animation methodology was perceived by respondents to be a more realistic, stimulating, unique, and effective way of presenting information. Only when
respondents were asked about complexity and level of boredom did the traditional methodology have a statistically significant higher difference. Thus, there
is support for this hypothesis that the use of 3D animation in a choice-modeling environment will lead to a more engaged respondent.
More Favorable Attitudes to 3D Animation Approach
There were two natural groups of attitudes about the survey experience (based on factor analysis):
Attitude about the Survey Approach
- Survey approach was boring
- Survey approach was stimulating
- I’d like to do this type of survey more often
- Creative way of presenting information
- Choices were too complex
- Survey approach was unique
Attitude about the Choice Task
- The way I made the choices is similar to the way I make choices in the store
- Adequate amount of information for each choice
- Similar to my current decision-process
- Helped me explore different choices
- The survey approach was realistic
- Approach helped me form decisions
Question 3: Mental Labor to Respondents
Additionally, a cognitive strain task was provided to the respondents immediately upon completing the choice-modeling tasks. The respondents were asked to go
through eight iterations of the choice-modeling task. The variables were taken from a validated cognitive stress scale (the NASA Task Load Index).
This monadic approach to the evaluation of the methodology allowed for a direct comparison of the two groups of respondents.
The traditional choice-modeling task is perceived by the respondents to be more mentally demanding, and the choice task is perceived by the respondents
to result in less success in accomplishing the task. In addition, the amount of work and level of frustration are both higher for the traditional approach
than the 3D animation approach.
The four items discussed in this section are significantly different between groups. Thus, there is support for the hypothesis that the cognitive stress
level is higher for the traditional choice-modeling methodology relative to the 3D animation choice-modeling methodology.
Question 4: Price Elasticity
We tested the hypothesis that price elasticity is reduced when estimated from 3D animation choice data vs. traditional grid-based choice data. We applied
a between-subjects statistical test to measure the opposite of our fourth hypothesis—that the averages of price coefficients were equal. We rejected the hypothesis of
equality, suggesting that less negative price coefficient (i.e., lower price sensitivity) was significant.
Simulated demand curve for Act II Movie Theater Butter Popcorn
· Lower purchase volume
· Lesser price sensitivity
Conclusions and Managerial Recommendations
It is clear from the data that the utilization of 3D animation is preferred over traditional choice-modeling techniques in terms of respondent satisfaction
and engagement with the methodology, and in terms of presenting a more realistic survey environment. The data suggests a greater accuracy in measurement of purchase
intention; however, comparison to in-market data must be done to be more conclusive. Though still not as realistic as a physical store shelf set, this methodology
presents the opportunity to conduct choice-modeling tasks with nationally representative samples and using the full power of part-worth estimation (something that is
more difficult, time-consuming, and expensive in physical store shelf-set projects).
The research focused on one limited category of a consumer packaged good, popcorn. Replication across a variety of consumer packaged goods would examine the degree
of applicability across segments. In addition, this research was conducted with a sample of adults who are representative of the U.S. population. Conducting
similar research in other countries with high Internet usage would help to examine whether or not this technique is useful on a global scale.
- Berneburg, Alma (2007), “Interactive 3-D Simulations in Measuring Consumer Preferences: Friend or Foe to Test Results?”
Journal of Interactive Advertising, 8:1, 1-37.
- Jiang, Zhenhui (Jack), and Izak Benbasat (2007), “The Effects of Presentation Formats and Task Complexity on Online Consumers’ Product Understanding,”
MIS Quarterly, 31:3, 475-500.
- Nelson, Michelle R. (2002), “Recall of Brand Placements in Computer/Video Games,” Journal of Advertising Research, 42:2, 80-92.
- Warlaumont, Hazel G. (1998), “Realism in Magazine Advertising: Meaning, Prevalence, and Intention,” Journal of Current Issues in Advertising,
- Wernerfelt, B. (1996), “Efficient Marketing Communication: Helping the Customer Learn,” Journal of Marketing Research, 33, 239-246.
Copyright © 2009 by Decision Analyst, Inc.
This article may not be copied, published, or used in any way without written permission of Decision Analyst.