Text analytics 101: Uncovering insights in qualitative data

Hidden in messy text data are nuggets of insight that hold the power to transform business, marketing, and management strategies.

But what is text analysis? And how can this potential be unlocked?

When you think of analytics, you probably think of numbers. But a lot of the world’s valuable data is contained in words, such as survey responses or social media posts.

In fact, numerical data and text data are often two sides of the same coin.

Where numerical data tells you what is happening (for example, how many people said they would purchase your new product?), text data can reveal the all-important why (why do they or don’t they want to purchase the product?), and sometimes even the coveted what’s next (which product attributes should you avoid or innovate in the future?). 

If you want to take your business intelligence to the next level but you’re not sure where to start, this guide is for you.

We break down the fundamentals of text analysis and how you can transform your understanding of customers, target audiences and employees, covering the following topics:

What is text analytics?
Why do you need text analytics in your BI toolkit?
What are examples of insight-rich text data?
Text analysis tools: how do they work and what can they do?
The science behind text analysis, natural language processing and machine learning

What is text analytics (and is it the same as text mining)?

To put it simply, text analysis is the process of distilling objective insights from text data (aka words).

While traditionally analysts have relied on manual methods of text analysis, AI text analysis tools make the process more efficient.

They leverage intelligent natural language processing and machine learning (we’ll talk about these in more detail below), to allow you to analyze large volumes of data and uncover insights, at scale.

Text analytics can help you drive your business strategy by providing objective answers to questions such as:

What are customers enjoying about your product?
Why is employee satisfaction low?
How do people perceive us as a brand?
How are audiences responding to competitor propositions?
Why is footfall increasing or decreasing?
How are audiences adapting to external factors like pandemics or recessions?

In answering these questions precisely and objectively, you can drive your business strategy with confidence and purpose.

So, where does that leave us on text analytics vs text mining?

For all intents and purposes, the goal of text analytics and text mining is the same: to find insights in text data. Most people simply use the two terms interchangeably.

Why do you need text analytics in your BI toolkit?

Text analytics is your secret weapon for investigating everyday business mysteries.

Traditional business intelligence (BI) tools were not designed with qualitative text data in mind and often fall short when it comes to identifying the root cause behind trends and behaviors.

That’s where text analytics comes in with additional insights.

Perhaps there is a steady decline in repeat purchases from customers but you don’t know why?

Using survey text analysis to compare survey responses from repeat customers against one-time customers can help you unearth deeper insights that you will then turn into actionable next steps.

Perhaps employee turnover has been remarkably high in one department and you can’t pinpoint the problem?

Analyzing employee engagement surveys can shed some light on why the team members are leaving and inform HR strategy going forward.

But there’s more to text analytics than detective work.

When done well, it can also help you anticipate — and capture! — business opportunities before anyone else. From putting out fires to planting the seed for the next big thing, text analysis can be a game-changer for business intelligence.

Five insight-rich text data examples

The very first thing you need to kickstart your text analysis is the text data you’ll be working on.

Text data is just another way of saying “lots of words”.

From survey open-ends and customer reviews to social media and online forums, there is a plethora of unstructured text data out there waiting to be put into use.

And the good news is: you probably have some text data already.

1. Open-end survey responses

Open-end questions can be the most revealing part of a survey.

Let’s say you want to determine your NPS (Net Promoter Score) to benchmark your business against competitors.

A multiple-choice question will let you quickly determine your score.

But without conducting survey text analysis of the open-end responses, you might not know what it is that your customers are loving (or not) about your brand. In other words, you’ll be missing the key element that takes your insight from informative to actionable.

2. Online reviews

Review platforms like TripAdvisor, TrustPilot, and Amazon are a treasure trove of text data.

Text analytics allows you to use the star rating associated with reviews to segment and compare the positive and negative responses on these platforms.

From revealing the drivers of customer satisfaction to inspiring new features in product design, online reviews can be an incredibly powerful source of insights for business intelligence.

There is also a big opportunity here to learn how your competitors are perceived — and refine your brand positioning to stand out from the crowd.

3. Social media text analysis

People talk. And the great thing about social media is that you can listen in and take notes.

Many global brands have set up alerts to know when people are posting about them and keep abreast of online perception. But you can, and should, think bigger.

To truly use social media data to its full potential, you need to keep tabs on broader online conversations and take advantage of new opportunities before anyone else.

You can use trend analysis to monitor how public discourse around relevant topics changes over time, or geographic analysis to learn how people talk about the same subject across different markets.

4. Customer service transcripts

In theory, getting to talk to customers one-on-one to hear their immediate feedback is every marketer’s dream.

In practice, this data often sits stored away somewhere on the company server, unanalyzed and unused.

Customer service calls and interviews can be transcribed (using transcription software such as rev.com) to become valuable text data. Just a half-hour interview can yield over 3,000 words of valuable insight.

Taken en masse, this speech to text analytics can build a broader picture of brand perception and key drivers behind customer satisfaction.

Who knows, maybe the answer to that one bugging question you haven’t been able to solve is hidden away somewhere in this data?

5. Marketing content

We talked about customers’ perceptions of your brand, but what about competitors’ brand positioning?

How do other players in the market talk about their products?

And what are they missing?

(Cue: that’s where you come in.)

Website copy, blogs, landing pages, and other types of marketing content are fantastic text data examples for market competitor analysis.

Text analysis tools: How do they work and what can they do?

So, you’ve gathered your text data and you’re ready to start. What’s next?

Technically, you can analyze your data manually. But in practice, this can be an efficiency nightmare.

Manual analysis is labor and time intensive. And equally, manually tagging your data points is prone to human bias.

A dedicated text analysis tool is a much more efficient and objective way to draw insights from your text data, delivering results at scale and quantifying qualitative data.

When choosing the right text analytics solution to add to your BI toolbox, consider the key features of text analysis tools.

Word frequency analysis

Word frequency analysis is the most basic form of text analytics. No surprises here: it analyzes text data to identify the most commonly used words (or in some cases, phrases).

It’s a way of understanding the dominant words in a data set and often features in social media listening tools.

Topical text classification

Topical text classification is where you start getting the juicy insights, in this case, key themes within your text data.

The challenge of automated linguistic analysis is that words can take on different meanings in different contexts.

There is a crucial difference between “right” in “I turned right into the driveway” and “right” in “It turned out I was right”. Or you could say “I was his right-hand man” and that would mean something else entirely.

With topical text classification, the quality of the output very much depends on the tool you use.

Basic text classification tools would struggle to distinguish between the examples above. But AI text analysis tools can make a more nuanced and reliable judgment on how a particular word is being used in a given context.

Emotional or sentiment analysis

Sentiment analysis tools attempt to classify text as positive, negative, or neutral. This can be helpful on a basic level, but often it’s not sophisticated enough to tap into the real emotions at play.

To get truly game-changing insights into how people feel about your brand and products, you need to go deeper than that — which is where emotional text analysis comes in.

AI text analysis tools like Relative Insight offer much more sophisticated insights.

While sentiment analysis tools might tell you your text is mostly positive, emotional text analysis can, for example, help you distinguish between joy, contentment, or hope.

Let’s say we use social listening to take a temperature check on fandom during the Super Bowl.

We might see lots of comments like “that 30 minutes of Snoop Dog was the best action we saw all night”, or “awesome, great to see the refs giving away penalties like there’s no tomorrow 🙄”. Only a very sophisticated tool would be able to detect the sarcasm, understanding the emojis too.

Visual text analysis

With so much data on hand, visualization is key.

Clear visualizations allow you to understand what’s happening in your data at a glance and spot trends that might have gone unnoticed.

These visualizations are a powerful tool for data storytelling and can help boil down a complex analysis into a format that is easily understood by stakeholders.

In some cases, text analysis tools can even feed visualizations directly into BI tools and dashboards like Tableau or Power BI, integrating text analytics with your existing processes and reports for a holistic overview.

Some qualitative data visualization examples are Relative Insight’s Heartbeat charts, which allow you to track changes in the prevalence of key topics in customer feedback and social media conversations over time.

Heatmaps is another handy visualization tool that helps you quickly identify which data sets have the most and least in common.

Data management

Inconsistent methods of data collection and storage add unnecessary frustration to qualitative analysis, particularly when working with large volumes of data.

Hassle-free data management often makes the difference between a good text mining tool and a great one.

Text analysis tools tend to offer a range of options when it comes to uploading data for analysis, from a simple upload from .csv or Excel files to joining up data sources directly using a text analytics API connection. API connections are superior in terms of ease of management: they automatically push your data into text analytics platforms for analysis, without any need for manual intervention.

Most text analytics platforms will also provide a suite of tools to help you filter, clean and segment your data.

Finding the right text analysis software

There is a range of text analysis tools out there, with some offering more sophisticated features than others.

The best text analysis software will allow you to go beyond basic data points and mine powerful insights through emotional text analytics or accurate topical text classification.

Ultimately, to do text analysis well, you need a platform that combines the best of these features, complete with data management and visualization that align with your business goals. The type of text data you have, the insights you’re looking for, and the impact you’re hoping to make, all factor into deciding which tool is right for you.

Real-world text analytics examples

One of the most common questions from people entering the world of text analytics is “What is text analytics used for?” A more apt question would be to ask what it’s not used for.

From analyzing customer feedback to target market research to competitor benchmarking, businesses in the know use text analytics across the board to up their data game and stay ahead of the curve.

Text analytics examples for customer insights

Great customer experience is in the detail. Especially when it comes to taking customer experience from good to excellent, it can be tricky to pinpoint the little touches that make all the difference.

One of the most common text analytics use cases for customer insights is comparing positive feedback from survey open-ends or reviews against negative feedback.

Understanding what you’re doing well and zeroing in on areas of improvement allows you to create a laser-focused business roadmap that addresses real customer sentiments.

Once you have your roadmap, a good text mining tool will allow you to define the topics you want to track over time.

Let’s say you noticed that customers complain about delays in home deliveries. You can visualize how the prevalence of delays is changing over time in your text data to measure success and keep tabs on this topic in the future.

Text mining applications in marketing

Companies that use text analysis are consistently amazed at how it transforms their understanding of customers and target audiences and informs strategic decision-making.

It is always the mission of the marketer to identify that opportunistic gap in the market that precisely pinpoints customers’ wants and needs – and text mining can be just the tool to identify this gap.

Perhaps you’re a recipe kit start-up looking to carve a space in the market away from big players like Gousto and Hello Fresh.

In this case, you could use text analytics to discover general trends in topical online conversations; you could track responses to competitor campaigns or macro-level trends like the cost of living; you could assess regional differences between consumers in different areas; you could even compare positive and negative reviews of yours and competitors’ products.

This data-driven strategic approach could help you to find a niche brand positioning for your recipe kit brand, that capitalizes on a real and unanswered need of consumers. All facilitated by text analytics.

Text analytics use cases for market research

More and more brands are realizing that text analytics is a market research game-changer.

Clever businesses are using these insights to inform everything from product development to marketing comms, and you should too.

Let’s say you’re a skincare brand.

To stay relevant, you start using a social listening tool to monitor social media conversations around beauty products in your target markets. Your analysis reveals a lot of online chatter around sustainability and reusable packaging, and you dig deeper to discover that none of the big beauty players are offering a skincare refill scheme.

Bingo. Not only did you just uncover a new business opportunity, but you also found a new avenue to strengthen your brand and engage in authentic conversations with your target audience.

The science behind text analysis, natural language processing and machine learning

It’s time to get geeky.

It’s always helpful to know exactly where your insights have come from, so let’s talk about how text analytics works from a more technical perspective.

What are the steps in text analytics?

First, the data is prepared for analysis using Natural Language Processing (NLP). This is an essential step because, unlike humans, computers can’t make sense of our everyday language.

You can think of NLP as a sort of interpreter, translating text data from human language to computer language so it can be analyzed further.

NLP is not perfect, though. At least not yet.

The best text analysis tools use machine learning to improve text mining algorithms so they can become more precise and pick up on nuance.

In particular, machine learning helps to make judgments about the context in which words are being used, like distinguishing between ‘lead’ being used to describe the metal or the act of leading a group. Not only is language a tricky thing in itself, it’s also always evolving, throwing constant curveballs at text analysis techniques.

Once the data set has been prepared using NLP, it’s time for the text analytics.

This can take many forms, depending on what insights you’re looking for. From sentiment analysis of text to topic modeling, text analysis will process the data and distill the information you need to fuel your business strategy.

By the way, Relative Insight also allows you to do text analytics for the healthcare sector. Our platform features a specialized medical lexicon that’ll aid you in your medical sector research efforts.

Now you’re ready to go

And that’s it: your 101 on text analytics.

From understanding how text analysis tools work, to how you can use them, to what they can deliver for you – we hope all your questions are answered.

Put simply, text analytics opens a world of insight and opportunity, just waiting to be uncovered.