Analysis of open-ended survey responses

Published on November 4, 2022 by Akanksha Singh

In research, the inclusion of appropriate questions that are in sync with the organisation’s objectives and help draw the right insights is paramount. Both close- and open-ended questions have their own set of benefits and limitations. While close-ended questions are easy to analyse, they may fail to unearth the details, as well as fresh ideas, opinions or insights open-ended questions reveal.

Having said that, dissecting the data generated from open-ended surveys and using them effectively are critical.

What are open-ended responses?

Open-ended questions, designed to extract more subjective and explanatory responses from respondents, seek to understand a respondent’s viewpoint and, thus, do not restrict them to limited (pre-set) possibilities.

The principal difference between close- and open-ended responses is this. Close-ended responses offer limited options and quantitative feedback, usually expressed in numeric values and are used to generate statistics. In contrast, open-ended responses generate qualitative feedback in the unique language of the respondent, sometimes surprising researchers with the information or insights they present.

Benefits of open-ended survey questions

Open-ended survey questions may require additional tools to interpret the results, but the data generated from them offer a deeper and more holistic avenue to study the underlying concepts. They also offer a plethora of other benefits.

Let’s discuss a few of them.

1. Insights from verbatim responses

Respondents to open-ended questions have an opportunity to convey their opinion. Consequently, the output may broaden researchers’ understanding. In contrast, close-ended questions, offering a pre-defined set of options, rob researchers of this benefit.

2. Prospects of new opportunities from explanatory feedback

Open-ended surveys permit researchers to be as broad or specific, based on their objective(s). The respondents may expand their thoughts and dwell on additional ideas in their response, giving a new direction to researchers and/or creating opportunities for organisations.

3. Gains beyond sentiment analysis

Data gathered from open-ended responses can provide supplementary benefits. For example, when respondents share their perspectives on products/services, preferences and buying process, the data can be used to fine-tune sales and marketing efforts and make it more relatable.

Common methods to analyse open-ended responses

1. Coding the data

Open-ended responses may provide anecdotal information. Data coding can prove to be highly effective to ascertain the number of respondents who provided the same response. For example, data coding may help establish the number of respondents who do not buy a car because they want to see a greener earth.

In market research, coding involves collating the various responses of a survey into sets of various key ‘topics’, each of which is provided a code. Once coded, patterns emerge, framing one or more story outlines, which enrich the result. Thus, data become easier to analyse, as the analysis can be performed in the same manner as single- and multiple-response questions.

2. Data visualisation using Gestalt principles

Once open-ended responses are thematically coded, they can be visually represented using the following Gestalt principles:

• Colour and shape. They can be used to highlight variances or similarities among the respondents or topics. Examples include visualisation techniques, such as spectrum display or heat map.

• Proximity and connection. They can be used to display the interconnection of ideas, for instance, by using Venn diagrams or process charts, to establish cause-effect relationship in interviews or focus group. In open-ended survey data, proximity can be used to highlight the interrelation of different themes with topics, with closely related topics placed together and loosely related or unrelated ones placed apart visually.

• Weight and size. They underscore the significance of information, as well as the hierarchy of insights collected from data. Word cloud may be considered one of the most popular methods. However, a packed bubble diagram can also highlight notable themes and display a second dimension.

3. Augmentation using natural language processing

Natural language processing (NLP) – a subset of artificial intelligence – supports computers process, translate and understand human language. It helps computers decipher human communications. Few NLP subsets used to derive extra insights from open-ended survey responses are:

• It is the process of decomposing text into meaningful words, phrases or other elements called tokens. These tokens help understand the meaning of text or develop a model for NLP. Tokenisation requires an underlying model of the language based on application and cannot be performed by applying simple rules, as each language has its own grammatical construct.

• It is the act of analysing the context and converting a word to a common base form, known as a lemma. For example, a lemmatisation algorithm would figure out that the word better is derived from the word good and, hence, the lemma is ‘good’. Lemmatisation is used to assess the significance of each word, as the word cloud may contain various forms of the same word. To assess the correct lemma of a word, the technique performs the morphological analysis of each word using dictionaries.

• Part-of-speech tagging. It categorises each word according to its syntactic function, i.e., based on its context in the text. The relevancy and importance of each part of speech may vary, for example, nouns and adjectives may contain more important information than prepositions. This technique is a crucial pre-processing step in content or free-form text analysis, as it weeds out the less vital parts of speech.

Conclusion

Open-ended responses are key to quantitative research. Proper understanding of when to use, formulate and analyse these responses can provide a richer context to organisations. These insights can help businesses achieve their objectives and open new vistas.

How Acuity Knowledge Partners can help

Acuity Knowledge Partners is a global one-stop shop for market research solutions, offering the latest technologies, optimum execution techniques, robust quality, consultancy and extensive experience. We provide market research services across an array of industry verticals and have a scalable pool of domain experts.

Sources:

• Dcipher Analytics | How to analyze open-ended survey responses

• How to analyze open ended survey questions - Ascribe (goascribe.com)

• How to Analyze Open-ended Responses for Powerful Results (monkeylearn.com)

• What to Do With All Those Open-Ended Responses? Data Visualization Techniques for Survey

• Getting the most out of open ended responses — Lab42 Research

---

---