Metrics reference¶

TrackMe writes numeric KPIs to its metrics index (default trackme_metrics), all under the trackme.* namespace. There are two kinds:

Metrics TrackMe controls — a fixed catalogue it emits for every entity and tracker: the SLA state, the feed KPIs (delay, latency, volume…), workload counters, and operational health. These are documented below.
Metrics you define — for Flex and FQM, the metric names come from your tracker’s SPL, so there is no fixed list (see below).

Tip

The authoritative list for your deployment is always live in Splunk. List every metric name with:

| mcatalog values(metric_name) WHERE index=trackme_metrics

The tables below name the metrics as of current TrackMe; treat mcatalog as the source of truth if a name differs in your version.

Common to all components¶

One metric is emitted for every entity, regardless of component — it is the backbone of SLA reporting:

Metric	Meaning	Unit
`trackme.sla.object_state`	The entity’s health state expressed as a number (the green / orange / red / blue state, including ack), one sample per entity per cycle. This is what SLA dashboards integrate over time.	state code

Operational / system-level¶

These describe how TrackMe itself is running — the health and runtime of its trackers, and the scoring pipeline. They are per tracker / per entity rather than per data feed.

Metric	Meaning	Unit
`trackme.components_register.runtime`	Execution run time of a tracker / report — how long a hybrid tracker (or any component tracker) took on its last run.	seconds
`trackme.components_register.status`	Last execution status of a tracker — `1` success, `2` failure, `3` other.	status code
`trackme.scoring.score`	The computed score sample for an entity.	score
`trackme.impact_score.*`	Durable record of impact-score adjustments (false-positive and manual increase / decrease). Leaves are the numeric keys of the adjustment record.	score

The components_register metrics are the ones to watch for tracker performance — for example, average hybrid-tracker runtime per tenant — and feed the Performance Deepdive dashboards.

Feed KPIs — DSM, DHM, MHM¶

The data-flow components share a single family, trackme.splk.feeds.*, emitted by their hybrid trackers. These are the delay, latency, and volume KPIs behind the entities table and the entity overview charts. The object_category dimension (splk-dsm / splk-dhm / splk-mhm) tells you which component a sample belongs to.

Metric	Meaning	Unit
`trackme.splk.feeds.lag_event_sec`	Delay — seconds since the most recent event (now − last event time).	seconds
`trackme.splk.feeds.lag_ingestion_sec`	Latency — seconds between an event’s time and its index time.	seconds
`trackme.splk.feeds.eventcount_4h`	Event count over the tracker window.	count
`trackme.splk.feeds.hostcount_4h`	Distinct host count over the tracker window.	count
`trackme.splk.feeds.avg_eventcount_5m`	Average events per 5-minute bucket.	count
`trackme.splk.feeds.latest_eventcount_5m`	Latest 5-minute event count.	count
`trackme.splk.feeds.perc95_eventcount_5m`	95th-percentile 5-minute event count.	count
`trackme.splk.feeds.stdev_eventcount_5m`	Standard deviation of the 5-minute event count.	count
`trackme.splk.feeds.avg_latency_5m`	Average ingestion latency per 5 minutes.	seconds
`trackme.splk.feeds.latest_latency_5m`	Latest 5-minute latency.	seconds
`trackme.splk.feeds.perc95_latency_5m`	95th-percentile 5-minute latency.	seconds
`trackme.splk.feeds.stdev_latency_5m`	Standard deviation of the 5-minute latency.	seconds
`trackme.splk.feeds.avg_dcount_host_5m`	Average distinct host count per 5 minutes.	count
`trackme.splk.feeds.latest_dcount_host_5m`	Latest 5-minute distinct host count.	count
`trackme.splk.feeds.perc95_dcount_host_5m`	95th-percentile 5-minute distinct host count.	count
`trackme.splk.feeds.stdev_dcount_host_5m`	Standard deviation of the 5-minute distinct host count.	count
`trackme.splk.feeds.global_dcount_host`	Global distinct host count for the entity.	count

Note

Not every component emits every leaf. DSM emits the full set above. DHM emits a subset — the event-count, latency, eventcount_4h, and the two lag metrics — but not the distinct-host-count metrics. MHM rides the same family for the lag / availability metrics (notably lag_event_sec).

Component-specific metrics¶

On top of the shared feed KPIs, each data component emits a small set under its own root (note the underscore — trackme.splk_dsm etc., distinct from the dotted trackme.splk.feeds):

Metric	Meaning	Unit
`trackme.splk_dsm.sampling.model_pct_match`	DSM data sampling — % of sampled events matching a format model (per `model_name`).	percent
`trackme.splk_dsm.sampling.model_count_matched`	DSM data sampling — events matched by the model.	count
`trackme.splk_dsm.sampling.model_count_parsed`	DSM data sampling — events parsed.	count
`trackme.splk_dsm.sampling.run_time`	DSM data-sampling run time.	seconds
`trackme.splk_dhm.last_eventcount`	DHM — last event count for the host, per index / sourcetype.	count
`trackme.splk_dhm.last_ingest_lag`	DHM — last ingestion lag.	seconds
`trackme.splk_dhm.last_event_lag`	DHM — last event lag (delay).	seconds
`trackme.splk_mhm.last_metric_lag`	MHM — last lag of the monitored metric series, per `metric_category`.	seconds

Workload (WLK)¶

Workload monitoring emits a fixed set of scheduled-search execution KPIs under trackme.splk.wlk.* (object_category=splk-wlk):

Metric	Meaning	Unit
`trackme.splk.wlk.count_execution`	Scheduled-search executions.	count
`trackme.splk.wlk.count_completed`	Executions that completed.	count
`trackme.splk.wlk.count_skipped`	Skipped executions.	count
`trackme.splk.wlk.count_errors`	Errored executions.	count
`trackme.splk.wlk.elapsed`	Search run time.	seconds
`trackme.splk.wlk.scan_count`	Events scanned.	count
`trackme.splk.wlk.pct_cpu`	CPU usage.	percent
`trackme.splk.wlk.pct_memory`	Memory usage.	percent
`trackme.splk.wlk.svc_usage`	Splunk Cloud SVC consumption.	SVC
`trackme.splk.wlk.count_ess_notable`	ES notable events produced (where applicable).	count
`trackme.splk.wlk.status`	Last status of the scheduled search.	status code

Flex and FQM — your own metrics¶

Flex and FQM are different: you decide the metrics. Whatever numeric keys your tracker’s SPL puts in its metrics JSON become metrics, one per key:

Flex → trackme.splk.flx.<your_metric_key>
FQM → trackme.splk.fqm.<your_field_metric> (field-quality results land under the trackme.splk.fqm.fields_quality.* sub-family, e.g. percent_success).

So there is no fixed catalogue for these — list what a given tracker produces with mcatalog filtered by object_id. A few leaves are reserved (TrackMe’s own, not your data): trackme.splk.flx.status, and for converging trackers trackme.splk.flx.count_entities_up / count_entities_down; likewise trackme.splk.fqm.status.

Note

There is no trackme.splk.cim.* namespace — CIM compliance tracking is built on Flex, so its KPIs are emitted as trackme.splk.flx.* from the CIM tracker’s SPL.

Dimensions and how to query¶

Every metric carries identifying dimensions — most commonly tenant_id, object, object_id, and object_category (the component), plus alias and priority on the SLA metric, component / tracker on the operational metrics, model_name on DSM sampling, idx / st on DHM, and metric_category on MHM. The value itself is in _value.

# List every metric name in the index
| mcatalog values(metric_name) WHERE index=trackme_metrics

# See the dimensions carried by a metric
| mcatalog values(_dims) WHERE index=trackme_metrics metric_name="trackme.splk.feeds.lag_event_sec"

# Preview raw data points
| mpreview index=trackme_metrics filter="metric_name=trackme.sla.object_state" earliest=-1h

# Aggregate — average DSM latency per entity over 5-minute spans
| mstats avg(trackme.splk.feeds.avg_latency_5m) AS avg_latency
    WHERE index=trackme_metrics object_category="splk-dsm" BY object span=5m