Survey Design

A Beginners Guide to Open Text Questions

By January 23, 2018 No Comments

Open text questions (AKA text response questions) are a common feature in all types of surveys. They are a useful tool and when used correctly can provide many advantages to the researcher. They also have disadvantages as well. In this post, I outline what an open text question is, when you should use them and cover off their advantages and disadvantages.

According to a study by SurveyGizmo, 64% reported using text box questions while 52% using essay-style questions. Open-ended questions go hand and hand with closed questions and can provide a wealth of information which closed-ended questions are not capable of providing.

Types of Open Text Questions

There are several different types of open text questions. The most common ones used in consultation or engagement surveys are;

  1. Extension Questions,
  2. Expansion Questions,
  3. or General Open Text Questions.
1. Extension Questions

Extension questions are often found at the end of a list and often take the form of “Other: Please specify” followed by a text box. When designing surveys, it is best practice to have a relevant option available for all respondents, and an extension question serves this function. The extension question also allows for options which you have not thought of in the closed question section and ensures all options are covered.

Example of an extension question

2. Expansion Questions

Expansion questions follow on from a closed-ended question and allow the respondent to elaborate further on their response. The purpose of the expansion question is to clarify, expand on or explain their answer to a closed question. Expansion questions often ask why or how.

Example of an expansion question

3. General Questions

General questions are often found at the end of a section of a survey or on the last page of the survey and often take the form of “Do you have any additional comments?“. The general open text question allows the respondent to elaborate or expand on their closed answers and share their views about the themes of the survey.

Example of a general open text question

Advantages of Open Text Questions

In an article comparing open and closed-ended questions, it was noted that open text questions allow the respondent an opinion which is not influenced by the researcher.  With closed-ended questions, the researcher gets to choose what answers are available to the respondent, and the respondent has to fit their response into one of the pre-selected answers.

O’ Cathain and Thomas studied the use of general questions concluding that they can add value to a series of closed questions by allowing the respondent to elaborate on their response and acts as a safety net identifying issues or themes not considered by the researcher. Open text questions allow the respondent to elaborate on or explain their closed-ended answers which can provide additional insights when the researcher interprets closed text answers.

Disadvantages of Open Text Questions

Three significant disadvantages of open text questions are:

  1. Increased Survey Fatigue
  2. Irrelevant or messy data collected
  3. Analysis of collected data can be resource intensive
1. Increased Survey Fatigue

One of the disadvantages of open text questions is missing data due to the question being skipped or invalid answers.   Answering open-ended questions requires significantly more resources in comparison to closed-ended questions. The respondent has to think about their answer, type it into the text box which requires additional time meaning the survey takes longer to complete which increases survey fatigue. Often open-end questions are skipped which can result in incomplete data.

2. Irrelevant or messy data collected

Invalid answers are responses which don’t add anything to your data, a common example is in response to the question “Do you have any further comments?” and reply with “no”, “none” or mash the keyboard providing a jibberish response. Some respondents use the open-end questions to comment on totally unrelated issues. Invalid answers add time and cost to a project regarding data cleaning but also every response has to be read. This irrelevant data is removed during the cleaning process and stored in a junk file commonly known as file 13.

During a recent survey we conducted on proposed changes to Dog Control by-laws one respondent shared what they thought about cyclists and proposed cycleways. Another recent example from a survey on redeveloping a towns CBD is one respondent who wrote “Ban 1080” in every open text box.

3. Analysis of collected data can be resource intensive

Another downside to open text responses is how to translate the responses into actionable data and incorporate them in the report. This is often done through text analysis or coding which takes considerable skill and time. A recent survey included six extension questions plus a general question resulting in between 150 and 200 responses for each extension question and just under 300 general responses. Researchers often face the dilemma of whether or not to analyse the open text responses due to the resources required and these may not have been allocated for during project designed. Open text questions have become standard in surveys to the point that O’Cathain & Thomas believe that some researchers use the general open text question without much forethought as it has become standard practice. A 2016 study, two out of five researchers, confessed to launching a survey with open text questions without any intent to analyse the responses. Best practice is to only ask open text questions only if you are prepared to analyse the responses.

Analysing open text responses

Open text questions can be very helpful allowing the respondent the ability to expand on their responses and cover relevant topics which the researcher may have overlooked. There are some disadvantages as well including the skill and time that is required to clean and analyze the responses along with the question of how the responses will be included in the report. Open text responses can be a very useful survey tool if some thought is provided during the project design stage.