AI FQM Advisor¶
Overview¶
The AI FQM Advisor handles FQM (Field Quality Monitoring) — TrackMe’s component for tracking field parsing quality and CIM compliance. It is the most feature-rich advisor of the family: it manages dynamic thresholds and a per-tracker data dictionary that defines per-field validation logic (regex, datatype, required/optional, expected values, CIM datamodel mapping). It also has a unique wizard-time dictionary generation capability that proposes a starter dictionary from sampled-field statistics — before the entity even exists.
See The AI Advisors for the shared inspect/act model and the structured output contract, and Use Case Demo: Fields Quality (CIM and non-CIM) for the FQM component itself.
What FQM tracks¶
FQM answers “are the fields in this sourcetype being parsed correctly and conforming to CIM
expectations?” For each tracked sourcetype, you define a data dictionary mapping field names to
validation rules. A scheduled collect-job samples real events and computes percent_success (events
that pass field validation) and percent_coverage (events where the field was present at all);
dynamic thresholds turn those numbers into alerts.
This gives the advisor three flavours of work:
Threshold tuning — “your
percent_successthreshold of 99% is unrealistic for this noisy sourcetype; the observed median is 95% — recommend 90%.”Dictionary maintenance — “your
src_ipfield has a 15% failure rate because the regex does not match IPv6 — update the regex”; or “you have no validation foruser_idbut it is consistently a 10-digit number — add a rule.”Wizard-time dictionary generation — propose a starter dictionary from observed field types, value distributions, and CIM datamodel hints.
When to use it¶
An FQM entity is RED because of a field-quality threshold breach and you want to validate whether the threshold is realistic.
A specific field has a high failure rate and you want the advisor to inspect failing samples and propose a regex fix.
The data dictionary was set up months ago and the sourcetype’s payload has drifted (new field, renamed field, format change) — a “review my dictionary” pass.
You are creating a new FQM tracker via the wizard and want a starter dictionary proposal from sampled-field statistics.
What it reads and writes¶
In inspect mode the advisor reads the full FQM entity context, the data dictionary (and which trackers share it), individual field entries, the quality history, sampled failing events, the collect-job health, peer-entity thresholds, and the CIM datamodel diff. It can also test a candidate regex against real sampled values before recommending it.
In act mode it can add / update / delete dynamic thresholds, update individual field-dictionary entries, bulk-update the dictionary, and update the entity’s monitoring state or priority. A separate dictionary_generate mode runs at wizard time with no tools — it reasons purely from the sampled field statistics to propose a dictionary.
Note
The data dictionary is per-tracker, not per-entity — a dictionary change affects every entity sharing the tracker. Bulk dictionary updates are atomic (all-or-nothing), and operator review of a generated dictionary is always required regardless of the confidence score.
Example 1 — a CIM tracker with a generated dictionary¶
The FQM Advisor’s dictionary generation is integrated directly into the Fields Quality Monitoring
tracker-creation wizard. When creating a collect job for a CIM datamodel, the wizard steps you through
selecting the tracker type, defining the search, sampling fields, and previewing a generated data
dictionary. The embedded Wizard Assistant offers to “generate the data dictionary based on the
sampled fields” and explains the validation options (e.g. allow_unknown vs
allow_empty_or_missing).
Example 2 — a non-CIM tracker with a brand-new dictionary¶
The same flow applies to a non-CIM tracker, where there is no datamodel to align against. The advisor proposes a brand-new dictionary purely from the observed field types and value distributions — proposing presence-only validation where a field has no clear pattern rather than inventing a regex. The proposed dictionary is rendered as an editable table for operator review before it is applied.
Threshold tuning and dictionary review¶
Beyond the wizard, the AI FQM Advisor tab on FQM entity overviews runs the standard inspect/act flow for existing trackers — validating thresholds against observed quality history, inspecting failing samples, testing candidate regexes, and proposing dictionary updates (which can also be proposed by the AI Assistant as a consent card). It participates in the nightly automated batch for threshold tuning and dictionary maintenance; dictionary generation is wizard-only by design. See AI Agents automation.