Feeds thresholds (delay & latency)

For the splk-feeds family — DSM and DHM — the core question is “is my data on time?”. Two KPIs answer it, and each has a threshold an entity must stay under:

  • Delay — how old the most recent event is (now − last_event_time); breached above data_max_delay_allowed.

  • Latency — the gap between an event’s timestamp and when Splunk indexed it (_indextime − _time); breached above data_max_lag_allowed.

Tip

New to these two, or not sure which is which? Start with Delay vs latency: what they actually mean — a plain-English, diagram-led explanation of the difference (and how they appear in the UI).

When discovered, every entity starts with a 1-hour (3600-second) maximum for both. From there, TrackMe gives you several ways to set the right threshold for each feed — from a fixed value to fully automated, machine-learning-driven thresholds.

Ways to set a delay threshold

Approach

When to use it

Static threshold

A single fixed value — the default. Set it per entity, simulate it first, and read the two worked anatomy examples.

Lagging classes

Rule-based thresholds at scale — match entities by index, sourcetype or priority and apply a shared delay (and optional latency) threshold.

Adaptive delay (ML)

Let TrackMe learn each feed’s normal lag from history and maintain the threshold for you.

Variable delay

Time-aware thresholds that change by day-of-week and hour-of-day — tight during business hours, relaxed at night and on weekends.

Who manages a threshold: the Threshold Lock and the AI Advisor

Two cross-cutting ideas decide who owns an entity’s delay threshold.

The Threshold Lock. Setting thresholds used to mean answering two easily-confused per-entity questions — “allow adaptive delay to auto-tune this?” and “should this entity override lagging classes?” — that interacted in non-obvious ways and could each silently rewrite a value you had set by hand. A single per-entity Threshold Lock (a red 🔒, pinning delay and latency together) replaces both with one yes/no:

  • Locked — this value is mine. TrackMe pins your manual thresholds and every background writer skips the entity: adaptive delay, variable-delay auto-review, and lagging-class management. A periodic reconcile even restores your pinned values (and, for a variable entity, your slot schedule) if anything drifts.

  • Unlocked — TrackMe, manage it. Lagging classes and adaptive / variable adaptive delay are free to set and maintain the thresholds.

(The lock is the single control behind the older allow_adaptive_delay and data_override_lagging_class fields, which are now derived from it and no longer edited directly. Locking is governed per tenant by delay_threshold_lock_enabled.)

Important

The AI Feed Lifecycle Advisor takes precedence. When the AI Feed Lifecycle Advisor is enabled to cover DSM/DHM, it becomes the authority for delay thresholds: the legacy adaptive-delay and variable-delay auto-review mechanisms are automatically disabled (their per-tenant toggles are forced off and their trackers short-circuit at runtime). Turning the advisor back off does not re-enable them automatically — you re-toggle them by hand if you want them back. See the Artificial Intelligence section.