Home |
Analytical Consulting |
Database Analytic Services
| Text Mining or Text Analytics
Text Mining or Text Analytics
The terms “text mining” and “text analytics” are often
used interchangeably and refer to the extraction of data or information from
text. The text (words, sentences, paragraphs) could come from open-ended questions
in a survey or CRM system, from customer complaints or comments, the entries
of salespeople, comments on a website, etc.
Qualitative Technique
An array of techniques may be employed to derive meaning from text. The most
accurate method is an intelligent, trained human being reading the text and
interpreting its meaning. This is the slowest method and the most costly, but
the most accurate and powerful. Ideally, the reader is trained in qualitative
research techniques and understands the industry and contextual framework of
the text. A well-trained qualitative researcher can extract extraordinary understanding
and insight from text. In a typical project, the qualitative researcher might
read hundreds of paragraphs to analyze the text, develop hypotheses, draw conclusions,
and write a report. This type of analysis is subject to the risks of bias and
misinterpretation on the part of the qualitative researcher, but these limitations
are with us always—regardless of method. The power of the human mind cannot
be equaled by any software or any computer system. Decision Analyst’s
team of highly trained qualitative researchers are experts at understanding
text.
The history of text analytics traces back to World War II and the development
of “content analysis” by governmental intelligence services. That
is, intelligence analysts would read documents, magazines, records, dispatches,
etc., and assign numeric codes to different topics, concepts, or ideas. By summing
up these numeric codes, the analyst could quantify the different concepts or
ideas, and track them over time. This approach was further developed by the
survey research industry after the war. Today as then, open-end questions in
surveys are analyzed by someone reading the textual answers and assigning numeric
codes. These codes are then summarized in tables, so that the analyst has a
quantitative sense of what people are saying. This remains a powerful method
of text mining or text analytics. It leverages the power of the human mind to
discern subtleties and context.
The first step is careful selection of a representative sample of respondents
or responses. In surveys the sample is usually representative and comparatively
small (less than 2,000), so all open-ended questions are coded. However, in
the case of a CRM system, or customer complaint system, the text might be made
up of millions of customer comments. So the first step is the random selection
of a few thousand records, and these records are checked for duplicates, geographic
distribution, etc. Then, a human being reads each and every paragraph of text
and assigns numeric codes to different meanings and ideas. These codes are tabulated
and statistical summaries are prepared for the analyst. This is text mining
or text analytics at its apogee. Open-end coding offers the strength of numbers
(statistical significance) and the intelligence of the human mind. Decision
Analyst operates a large multilanguage coding facility with highly trained staff
specifically for content analysis and text analytics.
Machine Text Mining or Text Analytics
With the explosion of keyboard-generated text related to the spread of PCs
and the Internet over the past two decades, many companies are searching for
automated ways to analyze large volumes of textual data. Decision Analyst offers
several text-analytic services, based on different software systems, to analyze
and report on textual data. These software systems are very powerful, but they
cannot take the place of the thinking human brain. The results from these software
systems should be thought of as approximations, as crude indicators of truth
and trends, but the results must always be verified by other methods and other
data.
Text Mining Services
For more information on Text Mining or Text Analytics, please contact Rod
Carver, Vice President, Director of Database Analytic Services (rcarver@decisionanalyst.com),
or call 1-800-ANALYSIS (262-5974) or 1-817-640-6166.
Additional Resources from Decision Analyst
Database Analytic Services
Advanced Analytic Brochures
Database Analytic Case Histories
