Analysis
Confidence interval
Also called CI.
Confidence interval is a range around an estimate that would contain the true value in a stated share of repeated samples. A 95 percent interval is a method's long-run record, not a probability about one result.
How it is measured
It is computed from the estimate, the sample size, and the variation in the data. For a rate, the interval narrows as the number of observations grows roughly with its square root.
Report the lower and upper bounds together with the confidence level and the n. A point estimate without a range hides how much the number could move on a new draw.
Worked example
A pet-food shop sees 38 orders from 1,100 visitors on a new landing page, a 3.5 percent rate. A rough 95 percent interval runs from 2.4 to 4.6 percent.
The old page ran 3.1 percent. That value sits inside the new range, so the shop cannot yet say the new page is better. After three more weeks, with 3,900 visitors, the interval tightens to 3.0 to 4.0 percent.
How it differs
A confidence interval gives a range of plausible values. Statistical significance gives a yes or no against a null. The interval is more informative because it shows size as well as direction.
Common errors
Reading 95 percent as 'a 95 percent chance the truth is here'. Using too few observations. Ignoring that peeking widens the real error. Comparing two overlapping intervals as if overlap proved no difference. Dropping the interval from the report.
In practice
Show the range beside every rate you share. If the range is wide enough to cover a decision either way, collect more data first. Keep the confidence level the same across reports.