Contact

GlossaryPublic records, privacy, and data quality

Data validation

Data validation checks records against stated rules. Learn what passing a check establishes, what it cannot prove, and why flagged values need review.

Data validation checks whether information meets defined rules. In public-safety records, those rules might concern required fields, allowed category codes, sensible date sequences, or relationships between values. Passing validation means a record passed those checks; it does not prove that every detail accurately describes the event.

What the checks are designed to find

A validation rule needs a clear purpose. A date-format check identifies a value the system cannot interpret as a date. A category check identifies a code outside the permitted list. A timing check can flag a sequence that needs explanation.

The U.S. Geological Survey's quality-control guidance recommends evaluating data against defined criteria, marking known issues, checking transformed values, and tracking repairs. These are general data-management practices rather than public-safety reporting rules. USGS quality-control guidance.

Example

Hypothetical: A response record shows dispatch at 10:08 and arrival at 10:03 on the same date and time basis. A rule expecting arrival after dispatch flags the record.

The flag identifies a question; it does not supply the answer. The source may contain a mistyped time, a misunderstood field, or a workflow that the rule did not anticipate. Reviewing the original record and field definitions is necessary before changing the value or excluding it from a calculation.

Common confusion

Validation and correction are separate steps. A system can detect a problem without knowing the correct replacement. Automatically replacing an unusual value with an average would create an estimate, not recover the original fact.

Validity also differs from accuracy. A response time of eight minutes may be in an allowed range while still being entered incorrectly. A plausible value can pass a format or range check and remain wrong.

An outlier is not automatically an error. An unusually long response can be real. Treating every unusual observation as invalid would remove precisely the events that may deserve explanation.

How validation affects published figures

Readers benefit from knowing which rules affect inclusion. If a response-time chart excludes invalid timestamps, it should show how many records were excluded and what counted as invalid. Different exclusion rules can produce different averages from the same source file.

The version of the rule matters too. When allowed codes or field meanings change, checks need to match the relevant data version. A useful validation record preserves the issue, the review decision, and any correction so later readers can understand why a value changed.

Sources