Open-ended against closed-ended questions

Closed questions inherit the designer's assumptions

Every option list is a hypothesis about what people think, and anything missing from it is invisible in the results. If a feedback form lists five reasons somebody might not have bought, and the real reason is a sixth nobody considered, the data will confidently report that the five listed reasons account for everything. This is the strongest argument for keeping at least one open question, and it is also the argument for running a small open ended round before writing the closed version. Ten open responses from the right people will usually tell you what the option list should contain, and the closed survey that follows will be both shorter and more accurate than one written from a meeting room.

Open questions cost more than people budget for

Two hundred free text answers takes a couple of hours to read properly, build a coding frame for, and code consistently. Two thousand takes days, or a tool and a method. That cost is real and it is the reason most open questions on most surveys are never analysed at all, which is worse than not asking, because respondents spent effort on something nobody read. Before adding an open box, decide who will read the answers and when. If there is no answer to that, cut the question. The exception is a form rather than a survey, where each response is read individually anyway, and an open box costs the reader nothing extra.

The number of open questions decides the response rate

One open question at the end of a short survey is close to free. A second one measurably reduces the number of people who answer the first, because the sight of two empty boxes reads as work. Three turns a survey into a writing task. If you genuinely need several open answers, spread them across different surveys or different audiences rather than stacking them. Making the box optional helps, and so does a specific prompt: what would have made this more useful gets better answers than any other comments, which reliably produces either silence or a complaint about something unrelated.

Coding open answers without losing the point

The standard method is to read a sample, write down the themes that recur, turn those into a small set of codes, then code every answer against them, allowing more than one code per answer. Keep the code list under about fifteen entries, because a longer frame becomes inconsistent between coders and even within one coder over a long session. Keep an other bucket and read it at the end, since it usually contains either the most interesting answer or the evidence that a code is missing. Quote a few verbatim answers in the report alongside the counts, because a single sentence in a respondent's own words does more to change a decision than a bar chart of themes.

Building both in a form

Closed questions use radio for one answer, checkbox for several, or a dropdown where the list is long, and a matrix where several statements share one scale. Open questions use a long text field. A classic field's description is not shown to the respondent, so any instruction such as one or two sentences is enough has to go in the label, the placeholder, or a paragraph block above the field. On a conversational form the description does render, under the question label. Answers export one column per field, so an open question arrives as a full text column ready to code.

Using open answers to improve the closed ones

The two formats work best as a cycle rather than as rivals. Run the first version of a survey with a generous open question and a thin set of closed ones, read what comes back, and use it to build a better option list for the next round. Repeat once and the closed questions will cover the ground people actually talk about, in the words they use, which improves both the accuracy and the completion rate. Keeping one open question in every subsequent round then acts as an early warning: when answers start appearing that no option covers, the list has aged and the survey needs revisiting. This is also the cheapest defence against a survey that slowly stops measuring anything, which happens to most recurring surveys within about two years as the business changes around a question set nobody has reopened.

Questions people ask