The fundamentals of survey analysis and design

Just about every business conducts surveys. However, the sad reality is that survey data is often wasted because surveys are poorly designed or poorly analyzed — or a mix of the two. 

In this complete guide to survey analysis, we map out everything you need to know about survey design and analyzing quantitative and qualitative survey data, covering the following points: 

• What is survey analysis? 
• What are the advantages of survey analysis?
• How to gather survey data
• What are the different types of surveys?
• The basic steps in conducting a survey
• How do you analyze survey responses? 
• Automating survey coding with the right survey data analysis software 
• Survey analysis examples and use cases
• How do you create a compelling survey results report?

What is survey analysis? 

Survey analysis refers to the process of analyzing survey responses from any type of survey to extract actionable insights that are relevant to your research and business objectives. That being said, survey analysis contains both quantitative and a qualitative elements. 

Quantitative survey questions prompt respondents to select from predetermined options such as yes/no, multiple choice and rating scales and are easily analyzed using the tools available in most survey platforms or excel. Qualitative survey questions, on the other hand, generate unstructured open-ends that allow respondents to express their opinions in their own words and require specialist tools to analyze at scale. 

What are the advantages of survey analysis? 

Surveys are widely regarded as one of the most critical sources of information and insights for just about any organization. Businesses most commonly conduct surveys for one of three reasons: market research, customer satisfaction or employee experience. 

The insights gained from survey analysis are regularly used to improve processes and strategies across all areas of the business, such as product offerings, customer service, marketing or communications. In a world where many companies are struggling in their transition to becoming data-driven, we’ve outlined a foolproof approach for making the most of your survey data.

Gathering survey data 

In today’s digital world, we’re used to being able to do almost anything online – the same holds true for surveys. Although surveys can be conducted face-to-face, in paper form or over the telephone, the most common way of gathering survey data is through online survey tools. Digital surveys allow you to gain faster access to data and insights, reach new, more expansive audiences and are much cheaper to run than traditional offline surveys.  

And you know what’s the best part? There are numerous free online survey tools, including Survey Monkey, Qualtrics and Google Forms that you can use to get started.

What are the different types of surveys? 

Although there exists a myriad of different types of survey, most surveys fall into one of the following three categories: 

1. Customer experience survey

CX surveys are questionnaires aimed at getting inside the heads of customers to understand any aspect of their experience with your products, services or brand as a whole. The overall goal of conducting a customer experience survey is to improve the customer journey by boosting satisfaction, increasing customer lifetime value and decreasing customer churn. 

Customer surveys can take on many names including membership surveys, passenger surveys or fan surveys depending on your sector. 

Three common examples of customer experience surveys include:

• Net Promoter Score (NPS): used to understand how likely a customer is to recommend your brand
• Customer Satisfaction Score (CSAT): assesses customer satisfaction levels
• Customer Effort Score (CES) : measures the amount of effort required by customers when having an interaction with your business

2. Market research survey 

One of the most cost-efficient and reliable ways to conduct market research is through a market research survey. Market research surveys allow you to collect data surrounding target audiences and competitors, as well as pricing and other market trends. They are an essential source of intel for businesses looking to expand into new markets, launch new products, acquire new customers, keep up with current trends or improve business strategies.  

Market research or marketing surveys are commonly used by marketing teams to measure brand perception, shape go-to-market plans and map the competitor landscape. Depending on their focus, market research surveys can also produce fruitful insights to steer product development, sales and customer service strategies.

A particularly useful type of market research surveys are competitor surveys. Competitor surveys enable you to assess your brand’s strengths and weaknesses, as well as those of your competitors or the industry as a whole. This provides you with the opportunity to uncover market gaps and white spaces while leveraging these insights to have an edge over your competition.  

3. Employee surveys

Many say that happy customers are a company’s most important asset. We dare to disagree. 

Without happy, engaged, capable and productive employees your business will struggle to attract and retain satisfied customers. This is why employees need to be regarded as a business’s biggest asset. 

But how can you find out how your employees are truly feeling? What factors drive employee engagement and retention? Which of your policies and perks are most effective? 

The best way to generate employee feedback is through employee satisfaction surveys, employee engagement surveys and exit surveys. Feedback from these types of surveys helps you understand, in which areas your business is already succeeding and where you need to improve. 

The basic steps in conducting a survey

A survey is only as good as its design. Always keep this important truth in mind when you plan any type of survey. 

To ensure you’re on the right track when it comes to your survey design, we broke the process down into five steps: 

1. Know your goals 

Understanding the intended outcome of your research project is the absolute first step in survey design and analysis. 

At this point it’s important to mention that you should refrain from creating one survey that attempts to answer all of your business questions and instead aim to send out multiple surveys as needed. You don’t want to overwhelm respondents with an excessively long survey that may cause high abandonment rates!

To determine your goals, it’s paramount to ask yourself the following questions:

• What do I want to uncover?
• Who is my target audience? 
• How do I plan to utilize the insights?

These three points will shape the questions you choose to ask, but they will also guide the course of analysis once you have your responses in hand. 

2. Ask the right questions

Once you’ve defined your research goals, you should have a clear idea of what types of survey questions you’d like to ask. As a general rule, you should try to be specific, avoid jargon and ambiguity, be as engaging as possible and ask the most important questions early on. 

3. Pair closed questions with open ends

There’s been a historical tendency to focus surveys on closed questions that generate structured data that’s easy to analyze. The problem with this approach is that yes/no, multiple choice and rating scale questions only tell you what is happening but fail to reveal why. Yet understanding the why is the only way to fix the root cause of problems.

For example, a closed survey question might tell you that customers are unhappy with your customer service. You might assume that this is caused by bad customer service agent performance. However, asking an accompanying open-end question might reveal that the actual driver of customer dissatisfaction is long wait times rather than the service experience itself. This example underpins the importance of pairing open-ended and closed questions to generate the most powerful insights. 

Here’s what such a question pairing could look like: 

How likely are you to recommend our brand to a friend?

• Unlikely
• Somewhat likely
• Likely
• Very likely

Explain the reason behind your selection. 

4. Prioritize user-friendly design

Have you ever started answering a survey and quit because you became frustrated about the navigation, loading time or any other design or technical issue? This is exactly the scenario you want to avoid when sending out a survey. 

A user-friendly design entails clear navigation and a transparent progress bar. What’s more, your survey should be overall visually appealing to respondents to maintain engagement. Be clear about how long the survey will take.

Be aware of making your survey too long. The beauty of using text analysis tools like Relative Insight is that you can extract detailed intelligence through open-ended questions, rather than trying to find out this information through multiple closed questions.

Alban de Courville, Senior Manager, Business Insights at Medtronic Diabetes EMEA, explored this topic in one of our Spotlight Series webinars. 

5. Solicit responses at the right moments

Now that you’ve carefully crafted your survey questions, it’s time to send your surveys out to target audiences. 

Try to ask for responses in moments when you believe your audience will be most likely to have opinions to share. For a B2B software company like us conducting a customer survey, this might be after completing the onboarding process, hitting a key platform usage milestone, upon successful contract renewal (or after a failed one).

B2C companies, on the other hand, may want to send surveys a day or two after delivery of an order, if a customer is demonstrating loyalty with a certain number of purchases in a defined time frame or if they have returned a product. 

Caution – pinpointing the right time to send out a survey isn’t enough, you also need to determine the best way to reach respondents. For example, you might want to consider using social media channels, emails or pop-ups on your website, depending on who you’re targeting with your survey.   

How do you analyze survey responses?

Having learned why surveys are important, the different types of surveys and how to best go about designing a survey, it’s likely that you are left with the following question: What analysis method is used for surveys (particularly the open-ends)?

When it comes to survey analysis methods there is a surefire step-by-step approach we recommend: 

1. Understand who your respondents are and assess survey performance 

This might go without saying, but in order to pinpoint who and what the insights from your survey data analysis are applicable to, you need to first understand who your respondents are. 

As a first step, you will want to assess your survey performance for different respondent groups and distribution channels. This will allow you to verify whether the respondent profile of your survey mirrors your target audience profile. For example, if your goal is to gather feedback on your entire product line and you realize that the majority of your responses are from people who only purchase one particular product, you need to reassess your research goals to avoid misguided conclusions. 

2. Analyze closed questions 

Closed survey questions can help businesses report on customer health metrics, overall brand perception, employee satisfaction and more. However, they are a means of collecting data points that’ll allow you to segment qualitative responses for analysis. Such data points include demographic and behavioral attributes or positive and negative ratings. 

The good part about analyzing closed questions that produce structured data is that the analysis process is fairly easy and in most cases done automatically by the surveying tool you are using.

3. Segment, analyze and compare open-ends 

Here is where the data points you extracted from your closed questions come into play. There are various different ways you can segment, analyze and compare your open-ended responses. Examples include demographics (e.g. age, occupation, location, gender), satisfaction score, time period and behavioral attributes like frequency of purchase, average order value and more. 

Automating survey coding with the right survey data analysis software 

Qualitative survey responses (a.k.a. open-ends) are typically where people run into challenges with their survey analysis. There are two different approaches to coding qualitative questionnaire responses. 

The traditional approach is manual survey coding, where open-ends are read and categorized into a set of predefined themes by hand. As you can probably imagine, this approach is very time, labor and cost-intensive. But that’s not all – manual survey analysis is also prone to bias, meaning that insights are often shaped by the predetermined hypotheses of those doing the coding.  

The second approach to coding qualitative survey data is automated survey coding using survey data analysis software. Not only does this strategy speed up the analysis process tremendously, saving time, money and resources, it also minimizes the potential for human error or bias. 

Automated survey coding, however, does not replace your human insights teams, it complements it, enabling insights professionals to focus on data-driven storytelling and strategizing rather than spending days (or and weeks) reading through questionnaire responses. 

But beware! Many survey data analysis software platforms struggle to analyze context-specific meanings of words. Just think of a word like ‘spring’ which can be used in reference to a season, body of water, mechanical component or as a verb. 

Relative Insight’s approach stands out from the masses because our survey data analysis software applies a comparative approach to uncover the most important differences and similarities across data sets, helping you to zone in on what truly matters. 

Using Relative Insight for your survey analysis is easy – all you have to do is upload your data files and segment your survey responses based on your research goals. The platform will then analyze the survey data and uncover the differences and similarities in the words, phrases, grammar, topic, and emotions each group used.

Survey analysis examples and use cases 

Now that you know how to leverage survey data analysis software to automate the most cumbersome aspects of your survey analysis process, you are probably wondering what all of this could look like in practice. 

Using survey analysis to craft localized advertising campaigns

Co-op Funeralcare and Global, one of the world’s leading media and entertainment groups, wanted to understand how end-of-life planning differs across the UK in order to develop localized advertising campaigns. To do this, they conducted a survey to assess Brits’ opinions regarding end-of-life wishes, funerals and end-of-life celebrations. 

Using Relative Insight for the analysis of the qualitative survey responses, they were able to pinpoint what makes each regional UK audience unique. These insights informed Co-op’s location-specific advertising copy across Global’s network of outdoor advertising sites. 

Monitoring brand perception with survey open-ends 

Nowadays, brands’ social impact, diversity and inclusion policy and overall reputation play an enormous role when it comes to consumers’ buying decisions. Diversity and inclusion are terms that can mean very different things to different people. This is why we conducted a survey to gather UK and US consumers’ thoughts on diversity and inclusion and how brands can stack up. 

Using our survey analysis tool we grouped qualitative questionnaire responses based on various attributes including gender, race and sexuality. We then examined the extent to which people view diversity and inclusion through the prism of their lives, differences in language use and the impact brands, media and other cultural touch points have on people’s feelings about diversity and inclusion. 

Keeping up with consumer behavior in times of economic crises 

Inflation and other economic crises can have a tremendous impact on consumer behavior. One area that springs to mind as being under the threat of cost-cutting is giving to charity. To find out whether this is the case, how inflation is impacting consumer spending generally and which industries are most likely to get cut, we conducted a cost-of-living survey. 

Leveraging our comparative survey analysis software, we were able to identify some areas organizations can hone in on to dissuade existing givers from reducing donations and encourage more people to make a contribution.  

Market research survey: Uncovering how vegan fast food competitors stack up  

There’s no greater indicator of the rise of the plant-based movement than fast food chains jumping on the bandwagon. To pinpoint what the most popular vegan fast food trends are and how people feel about fast food vendors’ vegan offerings, we conducted a snap survey of Americans who have eaten vegan fast food at either McDonald’s, Burger King or KFC. 

While the quantitative survey analysis revealed that Burger King was the respondent favorite, the qualitative survey analysis uncovered the reason why: the chain’s Impossible Whopper. Additionally, we learned that vegan fast food brands encouraging meat eaters to ‘try’ their food is a winning strategy while highlighting a lack of cruelty to animals will resonate with Gen Z consumers.  

Driving action: How do you create a compelling survey results report? 

So far so good – you’ve crafted your survey in line with our survey design best practices, collected representative data and got the most out of your survey analysis with survey data analysis software. The truth is that none of this matters unless you create a compelling survey results report to present to stakeholders. Stick to the following steps for maximum impact: 

1) Identify key insights to include in your survey results report

First things first – the foundation for any good survey results report are high-quality insights. To determine which key insights to include in your survey analysis report, ask yourself if the findings in question fulfill the four characteristics of a good insight: focused, actionable, authoritative and surprising. 

2) Make use of visualization tools 

Want to keep your stakeholders engaged throughout your survey reporting? Make sure to include compelling visuals, such as graphics, graphs and other types of data overviews in your survey summary report – powerful visualizations are essential to storytelling and are a useful way to communicate complex ideas in a way that can be easily understood. 

Relative Insight’s survey data analysis software offers several ways of visualizing your survey data. For one, you can bookmark and group similar findings together into a single visual that also includes verbatim quotes to communicate context. Another powerful way to visually support effective data storytelling is through Relative Insight Heatmaps, which offer a top-level view of your market and customers, by visualizing the most and the least pronounced differences between respondent groups. 

3) Frame your survey analysis report as a compelling narrative 

Storytelling has the power to turn even the most boring stats into a compelling call to action for stakeholders. The only way to make your insights matter and inspire stakeholders to take action is by making your survey analysis report interesting and relevant. How do you do that? Convey the who, what, why and what’s next of your insights. 

4) Present your report to stakeholders and drive action 

Shoutout to you for making it this far! Now you are ready to present your survey summary report. If you’ve followed our step-by-step recommendations, there’s nothing standing in the way of you being able to compel your stakeholders to take action! 

Ready to put your newly acquired expert knowledge around survey analysis and survey design into practice? Book a no-commitment discovery call to learn more and leverage the goldmine of insights hidden in your qualitative survey data!