Survey and form glossary: question types and bias

Question types, and what each one is good at

The type pages cover the shapes a question can take: dichotomous, multiple choice, dropdown against radio, ranking, rating scales, sliders, semantic differential and the two standard loyalty measures, net promoter score and customer effort score. Each page says what the type measures well, what it measures badly, and what it costs the respondent in effort. Effort matters more than it looks, because a question that takes ten seconds longer than it should is the question people abandon. Where a shape has no direct equivalent in a form builder, the page says what stands in for it instead of pretending otherwise.

Bias pages, and when to read them

The bias pages cover the ways a question changes its own answer: leading questions, double barrelled questions, acquiescence, social desirability, primacy and recency. Read these before writing, not after. The neutral option and not applicable pages sit between the two groups, because whether to offer a midpoint is a design decision with a measurable effect on the distribution you get back. Screening and demographic questions have their own pages, since both are about who answers rather than what they answer, and both are the usual cause of a sample that does not represent anyone.

Writing order that avoids most of the trouble

Decide what decision the results will inform, then write the smallest set of questions that could inform it. Choose the type after the wording, not before, because picking a scale first tends to bend the question to fit it. Put screening questions at the top, demographics at the bottom, and the questions people find easiest in between. Then read the survey fatigue page and cut. Anything that would repeat a question already asked, or that nobody will act on, is length you are spending on the respondent's patience for no return.

Every page in this collection

• Anonymous vs confidential survey: the difference

  The two words get used as if they meant the same thing, and respondents notice when a survey promised one and delivered the other. The promise shapes how candid the answers are, so it has to match what the form actually records.

• Confidence level vs confidence interval

  The two terms travel together and get swapped constantly, which produces sentences that sound precise and mean very little. Keeping them apart makes it clear what a reader can and cannot conclude from a single survey result.

• Cross-sectional vs longitudinal survey

  The choice decides which questions the results can ever answer, and it is usually made by accident when a one off survey gets repeated. Knowing which design you are running stops a snapshot being read as a trend.

• Do progress bars help form completion?

  Progress bars are so common that most people assume they help, and the research says the answer is more awkward than that. A bar is a promise about the effort still ahead, and when the promise is discouraging or turns out to be wrong it does more harm than having no bar at all.

• Dropdown against radio buttons

  This looks like a styling preference and it measurably changes how people answer. Options that are visible get considered, and options hidden behind a control mostly do not.

• Fillable PDF vs flat PDF

  The choice matters most at the two ends of a document's life: when a form goes out, and when a completed one comes back to be filed. Send the wrong kind and either people cannot type into it, or anyone can quietly change what was typed.

• Form field vs component vs block

  The words sound interchangeable, but the difference decides what becomes a column in your data, what can be marked required and what counts as a question when you estimate how long a form takes. Forms built without the distinction look complete on screen and export badly.

• Inline vs on submit validation

  When an error appears matters almost as much as what it says. Shown too early it scolds people for answers they have not finished typing, and shown too late it sends them hunting back through a form they thought was done.

• Open-ended against closed-ended questions

  The choice is usually framed as depth against speed, which is only half of it. The real trade is that closed questions can only find what you already thought of, and open questions can only be used if somebody is going to read them.

• Primacy and recency effects in survey answers

  Order effects are one of the few survey biases with a clean, replicated explanation, and one of the least often controlled for. The order you happened to type the options in is quietly part of your results.

• Registration form vs application form vs enrolment form

  Calling an application a registration sets the wrong expectation, and people who assumed they had a place write in upset when they learn otherwise. The name at the top of the form is a promise about what happens after somebody presses submit.

• Required vs optional fields: how many is too many

  Required marks are the cheapest setting on a form and one of the most expensive in effect, because each one is a point where a person with no good answer must either lie or leave. Getting the balance right is mostly a matter of asking what you would actually do with a response that lacked the answer.

• Response rate vs completion rate

  The two numbers fail for different reasons, so mixing them up sends the fix to the wrong place. A low response rate is a problem with the invitation and the audience, while a low completion rate is a problem with the form.

• Should a survey include a not applicable option?

  These three options are usually treated as one escape hatch and dropped into a question list without much thought. Getting them right is the difference between a clean denominator and an analysis built on guesses.

• Should a survey scale have a neutral option?

  The argument for removing the midpoint is that it collects people who could not be bothered. The argument for keeping it is that it also collects people who genuinely have no view, and removing it does not give you their real opinion, only a guess.

• Single select against multi select questions

  Choosing between them looks like a formatting decision and is really a decision about what the numbers will mean. One produces a share of people; the other produces a share of mentions, and the two get confused in almost every report.

• Skip logic vs branching logic

  The terms are used loosely, and each survey tool draws the lines in a slightly different place. Knowing which one you need saves building a tangle of rules when a single jump would have done.

• The NPS question: exact wording and scale

  The wording matters more than most single questions, because the value of the metric comes entirely from everybody asking it the same way. Change the scale or the phrasing and the benchmark you were chasing stops applying.

• What does embedding a form mean?

  Choosing between an embed and a plain link decides where people meet the form, how it behaves on a phone and whether other sites can copy it. It looks like a small technical choice and it often decides how many visitors finish.

• What is 360 degree feedback?

  The value lies in the gaps between what a manager sees, what peers see and what the person believes about themselves. Those gaps only show up honestly when every rater trusts that their own answers will stay out of view.

• What is a consent checkbox?

  Consent is only worth having if you can later show the person gave it knowingly, and a pre ticked or bundled box is the quickest way to lose that proof. Getting the box right costs one line of copy, while getting it wrong can leave a whole mailing list without a defensible basis.

• What is a demographic question?

  These are the questions most likely to make somebody close a survey, and the ones most often included out of habit. Every one of them should have a named analysis waiting for it before it goes on the form.

• What is a dichotomous question?

  Two options make a question fast to answer and trivial to count, which is why forms are full of them. The cost appears when the thing being asked about was never binary in the first place.

• What is a domain restricted form?

  The same phrase covers two unrelated checks, and confusing them leaves a staff survey open to the public or an embedded form refusing to load on its own website. Knowing which domain is being tested tells you what the restriction actually proves about the person answering.

• What is a double-barrelled question?

  It is the most common flaw in survey writing and the easiest to fix, which is why it is worth learning to spot. Once you can see the joining word, you cannot unsee it.

• What is a form submission?

  The word sounds too simple to need defining, but the line between a submission and everything around it decides what you can count. Response rates, completion rates and every export depend on knowing exactly what became a record and what did not.

• What is a form theme?

  Keeping the look apart from the questions is what lets a form be rebranded in a minute instead of rebuilt in an afternoon. It also concentrates accessibility in one place, because a theme that fails contrast fails on every field at once.

• What is a good survey response rate?

  Chasing a benchmark number is how teams end up celebrating a response rate that hides a skewed sample. The useful question is whether the rate is normal for how and whom you asked, and whether the missing people would have answered differently.

• What is a hidden field in a form?

  Hidden fields answer the questions people are bad at answering about themselves, such as which email or poster brought them to the form. They are also easy to trust too much, because anything that arrives in a link can be changed by anyone who can edit a link.

• What is a JSON payload?

  Anyone connecting a form to another system ends up reading one of these, usually while working out why a value landed in the wrong place. Knowing how the structure fits together turns that from guesswork into a two minute check.

• What is a lead capture form?

  The difference shows in what happens after submit: a contact form lands in a shared inbox, while a lead form is sorted, routed and followed up against a clock. Designing it as one or the other decides how many people fill it in and how useful the ones who do turn out to be.

• What is a leading question in a survey?

  Nobody sets out to write one. They appear because the person writing the survey already knows what they hope to find, and that expectation leaks into the wording in ways that are invisible from the inside.

• What is a multi step form?

  Splitting a form is one of the cheapest changes you can make to how long it feels, because the first screen decides whether anyone starts. It is also easy to do badly, and a wizard with arbitrary breaks can be more tiring than the long page it replaced.

• What is a partial response in a survey?

  Every survey loses some people part of the way through, and what happens to their answers changes both the dataset and the response rate you report. Whether partials are kept is a design decision worth making before launch, not a setting discovered after the results arrive.

• What is a pulse survey?

  The value of a pulse is the line it draws over time, which only exists while the questions stay the same and people keep answering. Programmes that change the items every month, or never act on the results, usually lose both within a few rounds.

• What is a ranking question?

  Ranking is the right question when you need to know what wins, and the wrong one when you need to know how much anything is liked. Most surveys reach for it in the second case and end up with data that cannot answer the question asked.

• What is a rating scale question?

  Rating scales are the most used question type in feedback of any kind, and the two decisions that shape the data are made before a single response arrives: how many points, and what the points are called.

• What is a regex pattern in a form field?

  Patterns catch typing slips that no other rule can, such as a letter in a postcode or a missing digit in a business number. They also cause some of the worst form errors on the web, because a pattern that is slightly too strict silently turns away real people with real answers.

• What is a representative sample?

  Most survey mistakes that survive into decisions are sampling mistakes rather than wording mistakes. A carefully written questionnaire sent to the wrong slice of people still produces a confident, wrong answer.

• What is a screening question?

  Screeners protect the quality of everything that follows, and they are also the easiest place to teach people what answer to give. A badly written screener recruits exactly the respondents it was meant to exclude.

• What is a secret link form?

  Most forms that feel private are really unlisted, and for most purposes that is the right amount of protection. The trouble starts when a form that needed a lock is treated as though an obscure address were one.

• What is a semantic differential scale?

  It is the scale you reach for when the thing you are measuring has no natural agree or disagree framing. Asking whether a website felt cluttered or spacious gets a cleaner answer than asking whether somebody agrees the website was cluttered.

• What is a slider question?

  Sliders look precise and modern, which is most of their appeal. What they actually deliver depends heavily on the device somebody is holding and on whether the thing being measured has a hundred distinguishable levels in anybody's mind.

• What is a thank you page and what should it say?

  The moment after pressing submit is when people are least sure whether anything happened, and an unclear confirmation produces duplicate submissions and anxious follow up emails. It is also the one point where you have their full attention and nothing left to ask of them.

• What is a waiver form?

  People sign waivers without reading them, and organisers rely on them without knowing how far the law lets them reach. Both assumptions tend to fail at the worst possible moment, which is after somebody has been injured.

• What is a webhook? Explained with a form submission

  Webhooks are how a form stops being a list somebody checks and becomes part of a system that acts on each entry. They are simple to set up and easy to get subtly wrong, because the sending side decides what happens when your end is slow or offline.

• What is a welcome screen on a form?

  People decide whether to start a form within seconds, usually before they have read a single question. The welcome screen is the only place to answer what they are silently asking first: how long, why me, and what happens next.

• What is acquiescence bias?

  Agreeing is easier than disagreeing. It takes less thought, it feels more cooperative, and on a long grid of statements it is the path of least resistance, which is why agreement scales flatter almost everything they measure.

• What is an intake form?

  Intake is where a practice learns enough to use the first appointment well, and where most of its privacy obligations begin. A focused one shortens that first session, while a bloated one gets filled in carelessly, or not at all.

• What is an order form?

  Confusing the two causes real trouble: an order form that looks like a bill can be paid by an accounts team as though it were one, and an invoice raised before an order is confirmed invites disputes. An order form earns its place by capturing a request precisely enough that nobody has to phone to clarify it.

• What is customer effort score?

  Effort is the one service metric that points straight at a fixable cause. A low satisfaction score tells you somebody is unhappy; a low effort score tells you they had to repeat themselves, chase a reply or explain the problem twice.

• What is form validation?

  Validation is what separates a form that collects usable answers from one that collects whatever was typed. Done badly it is also the most common source of frustration on a form, because a rule that rejects a real answer blocks the very person the form was built for.

• What is form versioning?

  Forms change after they go live, often within days, and a changed question quietly breaks every comparison with the answers collected before it. Versioning is what lets you improve a form and still explain your data afterwards.

• What is margin of error?

  It is the most quoted number in a survey report and the most misread, usually as a guarantee that the result is accurate. It covers one source of error only, and it grows quickly when a result is split into smaller groups.

• What is non-response bias?

  It is the quiet failure behind many confident survey results, because nothing in the collected data announces who is missing. The only defence is to think about who stayed silent before reading what everyone else said.

• What is piping in a survey?

  A question that names the thing somebody just told you about is easier to answer and harder to misread than one that says the option you selected above. It is also one of the easiest survey features to break, because a blank or unexpected answer lands straight in the middle of a sentence.

• What is social desirability bias?

  People are not lying when this happens. They are managing an impression, often without noticing, and the effect is large enough to make self reported behaviour a poor substitute for measured behaviour on sensitive subjects.

• What is survey fatigue?

  Fatigue rarely announces itself. The survey still gets responses, the dashboard still updates, and the quality of the answers drops quietly enough that nobody notices until a finding turns out to be wrong.

• What is survey weighting?

  Most published polls are weighted and most internal business surveys are not, which is one reason the two so often disagree. The arithmetic is simple, but it quietly trades one kind of error for extra uncertainty of another kind.

• What sample size do I need for a survey?

  Most surveys either collect far more answers than the decision needs or far fewer than the conclusions drawn from them. Working out the number before sending anything tells you how many people to invite and which comparisons the results can honestly support.