GlossaryStatistics and interpretation
Statistical significance
Statistical significance means that a result meets a stated threshold in a statistical test under that test's assumptions.
Statistical significance means that a result meets a stated threshold in a statistical test under that test's assumptions. It helps assess whether an observed difference or association is compatible with a specified explanation involving chance. It does not, by itself, show that a difference is large, important, or caused by a particular policy.
What a significance statement needs
A report should identify the estimate or comparison, test, threshold, and relevant uncertainty. The Census Bureau's testing tool, for example, compares survey estimates using their margins of error and a chosen confidence level. Simply identifying the larger printed estimate is a different task.
Significance is a property of an analysis, not a label automatically attached to every change in an administrative count. A complete count can still have reporting limitations that a sampling-based test does not address.
Example
Hypothetical example: A large survey finds a difference of one percentage point between two groups and reports that the difference is statistically significant under its method. A planning team must decide whether that size of difference changes service needs.
The test result addresses statistical evidence. The planning decision also needs the magnitude, affected population, cost, and practical context. A small difference can meet a statistical threshold without answering those policy questions.
Common confusion
“Not statistically significant” does not prove that two population values are exactly equal. An imprecise study may be unable to distinguish a real difference from sampling variation.
A statistically significant association also does not establish causation. Bias or another variable may help explain the association.
Source differences
Read the stated confidence level and method rather than imposing one universal threshold. For repeated comparisons or complex surveys, follow the source's recommended approach. A dashboard that displays no uncertainty information may not support an independent significance claim.