Forms · Compared
Confidence level vs confidence interval
The confidence level is how often a method of building ranges would capture the true population value if a survey were repeated many times, usually 95 percent. The confidence interval is the range itself for one survey, such as 38 to 46 percent. The level is chosen before collecting; the interval is calculated from the answers.
The two terms travel together and get swapped constantly, which produces sentences that sound precise and mean very little. Keeping them apart makes it clear what a reader can and cannot conclude from a single survey result.
Nuwan Madhusanka · Co-founder
5 min read · Published
| Confidence level | Margin of error | Confidence interval | What the choice trades |
|---|---|---|---|
| 90% | Plus or minus 3.6 points | 38.4% to 45.6% | Narrower, but the method misses the true value more often |
| 95% | Plus or minus 4.3 points | 37.7% to 46.3% | The usual reporting standard |
| 99% | Plus or minus 5.7 points | 36.3% to 47.7% | Wider, suited to decisions where a wrong call is costly |
What the level is a promise about
The NIST Engineering Statistics Handbook puts it in operational terms: if the same population is sampled on numerous occasions and an interval is built each time, those intervals would bracket the true value in approximately 95 percent of cases. The promise is about the procedure, not about any one survey. Imagine Coastline Credit Union running its app question a hundred times, each time with a fresh random sample of 500 members. Each run would produce a slightly different percentage and a slightly different interval. About 95 of those intervals would contain the true share of all members who would use the app, and about 5 would miss it entirely. The trouble is that any single survey cannot tell you whether it is one of the 95 or one of the 5. The level is the long run hit rate you signed up for.
The sentence that gets it slightly wrong
Reports routinely say there is a 95 percent chance the true value lies between 37.7 and 46.3 percent. Under the standard approach to statistics that is not quite right, because the true value is fixed and the interval either contains it or does not. The accurate version is longer: the interval was produced by a method that captures the true value 95 percent of the time. For most business decisions the practical difference is small, and the loose reading does little harm. It does real harm in two situations. One is when a reader treats the edges of the interval as hard limits, as if 46.4 percent were impossible. The other is when a reader forgets that the 95 percent covers sampling luck only, and assumes it also covers a badly worded question or a sample that left out the members who never use digital services.
Trading level against width
With the sample fixed, a higher confidence level produces a wider interval, as the table shows: moving from 90 to 99 percent widens the credit union's range from about 7 points to more than 11. That is the trade. A narrow interval at a low level is easy to state and wrong more often; a wide interval at a high level is rarely wrong and sometimes too vague to act on. The only way to get both a high level and a narrow interval is more responses. Keeping a margin of 5 points while moving from 95 to 99 percent confidence raises the completed responses needed for a large population from 385 to 664. Choose the level with the decision in mind, then work out the sample size, rather than choosing the level afterwards to make a result look tighter.
Choosing a level for a business survey
Ninety five percent is the default for a reason: it is widely understood, it matches the five percent threshold used in most significance testing, and a reader will not ask why you chose it. Ninety percent suits early, exploratory questions where the cost of being wrong is a follow up survey. Ninety nine percent suits results that will be quoted publicly, used in a regulatory setting, or used to justify a large spend. Whatever the choice, fix it before the survey runs and state it in the report next to the interval and the number of responses. Switching from 95 to 90 percent after seeing that a result fell just short of a threshold is the survey equivalent of moving the goalposts, and an experienced reader will notice.
Working it out from form responses
A yes or no question built as a radio field exports as its own column in the CSV, with the option labels written out, so counting is one filter. Divide the yes count by the number of rows that answered the question to get the proportion. The standard error is the square root of the proportion times one minus the proportion, divided by that count. Multiply it by 1.645, 1.96 or 2.576 for 90, 95 or 99 percent, then subtract and add the result to get the interval. If the question was optional, use the answered count rather than the total submissions.
Questions people ask
Why is 95 percent the convention?
Largely habit inherited from early twentieth century statistics, where a one in twenty chance of being misled was judged an acceptable risk for most scientific work. It stuck because it balances the width of the interval against the frequency of misses. There is nothing special about it mathematically, and some fields routinely use 90 or 99 percent.
If two intervals overlap, is there no real difference?
Not necessarily. Two intervals can overlap slightly while the difference between the groups is still larger than its own uncertainty. The right check is an interval or test on the difference itself. The reverse holds, though: if the intervals do not overlap at all, the difference is very unlikely to be sampling noise.
Does a confidence interval account for bias?
No. It reflects only the uncertainty from sampling. A survey that reached only enthusiastic members, or asked a leading question, can produce a narrow interval around the wrong value. Bias moves the whole interval, and no confidence level widens it enough to compensate, so check how the sample was drawn before trusting the range.
What is a credible interval?
It is the Bayesian counterpart to a confidence interval. It combines the data with a stated prior belief and does support the sentence people want to say, that there is a 95 percent probability the true value lies in the range, given the model. It is less common in business surveys but increasingly offered by analysis software.
Is confidence level the same as statistical significance?
They are two sides of one choice. A 95 percent confidence level corresponds to a significance threshold of 5 percent. If a 95 percent interval for a difference between two groups excludes zero, that difference is significant at the 5 percent level. Reporting the interval is usually more useful than the bare significance verdict, because it shows the size.
Can an interval run below zero or above 100 percent?
With the simple formula it can, when a result is close to either end and the sample is small, such as 2 percent from 60 responses. That is a sign the approximation is failing. Methods such as the Wilson interval stay within the possible range and behave better for small samples and extreme results.
Make one with forms
The button opens the generator with this use case already described. Change the wording to match your own.
Create a form with OneCraftRelated questions
- What is margin of error?What is margin of error? The range a survey result is likely to sit within, how it is calculated at 95 percent confidence, and a table by sample size.
- What sample size do I need for a survey?What sample size do I need for a survey? Work it out from population, confidence level and margin of error, with a table for 90, 95 and 99 percent.
- What is a representative sample?What is a representative sample? One that mirrors the population on the traits that shape the answers. Random, stratified, convenience and quota compared.
Step by step in the builder: Create a form from scratch with AI, then View, export and manage your responses.
Written and checked by the OneCraft team. Last checked .