What is a screening question?

Never let people see what qualifies them

The fastest way to ruin a screener is to make the target obvious. A question asking whether somebody has bought a product in the last three months, sitting on an invitation that promised a voucher for product owners, is answered yes by people who have never bought it. The standard defences are to bury the qualifying option in a longer list, to ask about a category rather than the specific item, and to use two mild questions rather than one obvious one. Avoid questions where the honest answer is obviously less convenient than the alternative. It is also worth avoiding an invitation that describes the audience, since anyone who reads it now knows the answers before they arrive at the first question.

How many screeners a survey can carry

Every screening question is a chance to lose somebody who would have qualified, through misreading, impatience or an answer that is technically outside the criteria but practically fine. Three is a lot. If your eligibility rule needs six conditions, the criteria are probably narrower than they need to be, and it is worth asking which of them would genuinely change the analysis. Where several conditions are unavoidable, order them so the most likely disqualifier comes first, which means people who will not qualify leave after one question instead of four. That ordering also reduces the time cost for the respondent, which matters when nothing is being offered in return.

Ending the survey with some grace

A disqualified respondent has given you their attention and should be told clearly and pleasantly that the survey is not for them. Avoid the word disqualified, which reads as a judgement, and avoid vague endings that leave people wondering whether their answers were recorded. A short screen saying this survey is for people who did a particular thing, thanking them, and where relevant offering another way to give feedback, is enough. If an incentive was offered, be explicit about whether it applies. The other thing worth doing is counting how many people are screened out and at which question, since a high rate means the recruitment source is wrong rather than that the screener is working.

Screening for quality, not just eligibility

Some screeners exist to remove inattentive or fraudulent responses rather than the wrong audience, particularly on panels. The usual devices are a question about a product or organisation that does not exist, an instruction embedded mid scale asking the respondent to select a particular point, and a check for impossibly fast completion. Each has costs. Fake brand questions catch some genuine respondents who assume they have simply forgotten, and attention checks irritate careful people. Use one, not three, place it well into the survey rather than at the start, and decide the exclusion rule before you look at the results rather than after.

Building screeners in a form

A screening question is usually a radio field with two or more options. Conditional logic here is jump rules only, so an ineligible answer jumps the respondent to an ending rather than hiding later questions in place. That is a clean fit for screening, since the whole point is to end the form early. Because a classic field's description is not shown to the respondent, any context belongs in the label or in a paragraph block above the field, or in a conversational form where the description does render. Keep the ending screens short, and give each route its own.

Keeping the screener honest over time

Screeners age quietly. A criterion written for last year's audience keeps admitting people who no longer match the group you care about, and nobody notices because the survey still fills. Review the eligibility rule whenever the audience, the product or the purpose of the research changes, and check the screen out counts each wave rather than only the completed responses. A sudden fall in the screen out rate usually means either the recruitment source changed or somebody worked out which answer qualifies. It is also worth reading a sample of qualified responses end to end once per wave, because a respondent who passed the screener and then answered every open question with a single word is telling you something the screening question could not. Where a panel or an incentive is involved, expect the rate of gaming to rise as the incentive does.

Questions people ask