FQM — Field Quality Monitoring

Field Quality Monitoring tracks whether your data is well-parsed: how completely and correctly fields are extracted, and how compliant a sourcetype or data model is with the Splunk Common Information Model (CIM). Where the feeds components ask “is the data arriving?”, FQM asks “is the data usable?”

Unlike the other components, FQM works in two phases: a collect job samples and scores your events, and a monitor job then turns those results into TrackMe entities and alerts.

Fields quality: collect, then monitor One collect job with two phases. Phase 1 (collect) runs a scheduled search in sampling or head mode over a data context and its dictionary, calls the trackmefieldsquality command to score each field against models (missing, empty, unknown, regex), and indexes the JSON results to a summary index under the sourcetype trackme:fields_quality. Phase 2 (monitor), scheduled 10 minutes later, parses those events, discovers the @global entity plus one entity per field from the break-by definition, and applies compliance thresholds that drive TrackMe stateful alerts and rich email notifications. Fields quality — collect, then monitor One collect job, two phases: sample and score your events, then turn the results into TrackMe entities and alerts. PHASE 1 · COLLECT scheduled search — sampling or head mode Data context + dictionary CIM data model or raw events Score every field | trackmefieldsquality · missing/empty/regex Index the JSON samples → sourcetype trackme:fields_quality JSON samples +10 min PHASE 2 · MONITOR runs 10 minutes after the collect job Parse the collected events one source per collect job Discover the entities @global + one per field (break-by) Thresholds and alerting compliance % · stateful alerts · email The data dictionary is generated from a CIM data model or reused from a previous job — dictionaries can be shared between collect jobs. The same approach applies to CIM and to any raw events: only the data context and its dictionary change. Thresholds are set automatically on the compliance percentage, so alerting reuses TrackMe's stateful alerts and rich email notifications.
FQM entities view, one entity per evaluated field, with coverage % and success % An FQM field-quality entity overview with its thresholds

What FQM tracks

Each entity is the extraction quality of one field — typically one entity per (data model, dataset, field), or per a custom SPL pattern. FQM measures:

Metric

Meaning

Coverage %

The share of events in which the field is present / extracted.

Success %

The share in which the field’s value is valid / well-formed.

CIM compliance

Drift against CIM-specific expectations.

Inactivity

The dataset has stopped producing data.

You set coverage and success thresholds per metric — the same dynamic, optionally time-sliced threshold model as FLX.

Creating an FQM tracker

  1. Pick the source — a CIM data model and dataset, or a custom SPL pattern.

  2. Choose the fields to evaluate.

  3. Set thresholds — coverage % and success %.

  4. Review and create.

FQM leans on the Common Information Model: standard CIM lookups provide ready-made field patterns, while custom SPL covers non-CIM fields.

Tip

Aim for a realistic bar. 95% coverage is a common, healthy target — insisting on 100% will generate noise. Outlier detection is supported but not recommended for FQM, as field-quality metrics are slow-moving.

See also