Cribl Stream reference¶
This page is the reference companion of Cribl Stream: the catalogue of built-in policies, the jobs and their schedules, the Leader request budget, the traffic ML models, the rules for silent and removed objects, the severity scale, every tenant setting, the indexed finding events, the permissions, the Configuration Guardian checks, the REST endpoints, the capacity and the troubleshooting table. The concepts shared with the other findings-based tenants (Finding groups, triage, assignment, findings actions, AI Agent) are in Findings, Finding groups and findings actions.
Policy catalogue¶
A Cribl Stream tenant evaluates 16 built-in policies, shown in four groups on the Policies tab. Every policy has the same common settings:
Enabled — on or off.
Severity — the severity of the findings it opens (see Severity scale).
Open after / Recover after — how many consecutive failing checks open a finding and how many healthy checks recover it (1 to 12). Metric policies count five-minute intervals; Worker and Leader policies count checks at least five minutes apart, so repeated manual collections never shorten a confirmation.
Worker Groups — the Worker Groups it applies to (empty = all).
Selected objects — the objects it applies to (up to 128; empty = all).
Bank holidays — Hold new findings or Keep opening findings while a bank holiday of the tenant’s calendars is active and the tenant suppresses new alerts on holidays. Default: Traffic outliers holds, every other policy keeps opening findings; the four Leader policies have no such setting. A finding already open keeps being evaluated.
Threshold policies are bad at or above the threshold, good at or below the recovery threshold, and hold their current condition in between. Missing data is never turned into a zero, except by the Traffic outliers policy for silent objects (see Silent and removed objects). A saved policy applies from the first collection interval after the save.
Cribl Leader
Policy |
What it detects |
Default |
Severity |
|---|---|---|---|
Leader availability |
The configured Leader API cannot be reached or used. |
On |
Critical |
Leader CPU |
Operating-system CPU of each Leader node. |
On |
Medium |
Leader memory |
Memory in use on each Leader node. |
Off |
Medium |
Leader disk |
Used space of each filesystem a Leader node reports. |
On |
Medium |
Data flow
Policy |
What it detects |
Default |
Severity |
|---|---|---|---|
Reported health |
A Source or Destination Cribl reports degraded or unhealthy. |
On |
Medium → Critical |
Traffic outliers |
Traffic outside the range its traffic ML model expects. |
On |
Medium |
Expected traffic |
A selected or automatically expected object carries no traffic. |
On |
Medium |
Pipeline errors |
Pipeline errors at or above the threshold. |
On |
Medium |
Destination backpressure |
A Destination reports backpressure or is blocked. |
On |
High → Critical |
Worker processes
Policy |
What it detects |
Default |
Severity |
|---|---|---|---|
Process CPU |
CPU of each worker process. |
On |
Medium |
Worker machines
Policy |
What it detects |
Default |
Severity |
|---|---|---|---|
Worker machine CPU |
Operating-system CPU of each Worker. |
On |
Medium |
Worker machine memory |
Memory in use on each Worker. |
Off |
Medium |
Worker machine disk |
Used space of the filesystem Cribl reports for each Worker. |
On |
Medium |
Worker availability |
A Worker is disconnected or its heartbeat is too old. |
On |
High |
Worker version drift |
A connected Worker does not run the running Leader’s version. |
On |
Low |
Worker configuration drift |
A Worker does not run its Worker Group’s configuration. |
On |
Low |
Default settings
What each policy applies to, its checks to open and to recover, and its own settings with their defaults.
Policy |
Applies to |
Open / recover |
Settings and notes (defaults) |
|---|---|---|---|
Leader availability |
The Leader API |
2 / 2 |
Fails on the network, the TLS certificate or host name, authentication, an unavailable service or a malformed answer. A rate limited answer is unknown, never a failure. |
Leader CPU |
Leader node |
3 / 2 |
Threshold 90 %, recovery 80 %. |
Leader memory |
Leader node |
3 / 2 |
Threshold 90 %, recovery 80 %. File cache included (see the note below). |
Leader disk |
Leader node filesystem |
3 / 2 |
Threshold 90 %, recovery 80 %, per named filesystem. |
Reported health |
Source, Destination |
3 / 2 |
Critical when Cribl reports it unhealthy, the policy severity when degraded. Only objects that carried traffic in the last 24 hours, are selected, or are already failing (Include resources without traffic: off). |
Traffic outliers |
Source, Destination, Pipeline, Route (per measurement) |
2 / 2 |
Detecting models only. Direction: drops and spikes (or drops only, spikes only; also per model). The silence of an object counts as zero traffic, under guards. |
Expected traffic |
Source, Destination, Pipeline, Route |
3 / 1 |
Opens nothing until objects are selected or expected. A new tenant uses Selected + automatic: objects qualify after 7 days; Sources and Destinations. |
Pipeline errors |
Pipeline |
3 / 2 |
Threshold 1 error per interval. |
Destination backpressure |
Destination |
2 / 2 |
Critical when fully blocked. At most one finding per Destination and condition. |
Process CPU |
Worker process |
3 / 2 |
Threshold 90 %, recovery 80 %; statistic Average (or Peak). |
Worker machine CPU |
Worker |
3 / 2 |
Threshold 90 %, recovery 80 %. |
Worker machine memory |
Worker |
3 / 2 |
Threshold 90 %, recovery 80 %. Cache included (see the note below). |
Worker machine disk |
Worker filesystem |
3 / 2 |
Threshold 90 %, recovery 80 %. |
Worker availability |
Worker |
3 / 2 |
Heartbeat limit 600 s (60 to 3,600). |
Worker version drift |
Worker |
3 / 2 |
Compared with the version of the running Leader. |
Worker configuration drift |
Worker |
3 / 2 |
Compared with the Worker Group’s configuration version. |
Note
Why memory policies are off. The memory the operating system reports as used includes its file cache, which fills a healthy Linux host. Leader memory and Worker machine memory are available but not enabled by default; enable them knowingly, with a threshold that suits your hosts.
Each policy’s editor also has an Actions tab (drilldowns, links and webhooks offered on its findings) and an Automations tab (rules that run an action automatically): see Findings actions.
Severity scale¶
Cribl Stream findings use the TrackMe scale Informational, Low, Medium, High, Critical.
Rule |
Effect |
|---|---|
Policy severity |
Each policy has its default severity (catalogue above), which a TrackMe administrator of the tenant can change. |
Raised by the condition |
Reported health: a Source or Destination Cribl reports unhealthy is Critical. Destination backpressure: a fully blocked Destination is Critical. Such a raise is an escalated transition. |
Operator override |
Set severity on a finding or a Finding group overrides the policy’s; Use policy severity restores it. |
Finding group |
The highest severity of its failing members, unless overridden on the group. |
Settled findings |
A recovered, ended or resolved finding is shown Informational; the severity its
episode reached stays recorded ( |
Earlier builds |
A severity stored as warning reads Medium. |
Notifications filter on the minimum severity (Medium and above by default).
Jobs and schedules¶
Each tenant is served by scheduled reports, each with its own lock and run receipt. Schedules are offset per tenant (from the tenant identifier), so several tenants reading the same Leader never start in the same minute.
Job |
Schedule |
What it does |
|---|---|---|
Live metrics |
every 5 minutes |
Checks the Leader API, reads the newest settled five-minute interval of every metric family, scores traffic against the models, publishes the snapshot every tab reads, refreshes the Workers and the Leader resources, evaluates every policy, then delivers the queued notifications. |
Inventory |
every 10 minutes; a new cycle about hourly |
Reads the configuration and the Workers from the Leader. A new cycle starts once the previous one is 50 minutes old; a Worker Group whose configuration version did not change is confirmed without a read (and re-read at least once a day); at most 10 Worker Groups are read per run, the changed ones first. |
Traffic ML models |
twice an hour |
Learns the models when a pass is due (every 6 hours, or while bootstrapping); returns at once otherwise. After repeated failures it waits 1, 2, 4 then 6 hours. |
Monitoring policies |
on demand only |
Evaluates the policies again on the newest snapshot. The scheduled evaluation is the one that ends every live run. |
AI Agent |
hourly, when enabled |
Scheduled triage by the Cribl Stream AI Agent (off by default; the report stays unscheduled until it is enabled). |
Manual collection. Settings → Collection & models → Scheduled collection is on by default; off, the reports are kept but unscheduled and Collection health reads Manual collection.
Run now. Collect now (status strip) runs the live job; Run now on each Collection health card runs that job; Read every Worker Group again (Inventory card) starts a cycle that reads every Worker Group regardless of its configuration version, for edits saved on the Leader but not yet deployed. Cooldowns per tenant, whoever asks: inventory 120 s, live 60 s.
Changing the source. Changing the connection, the metric index, the metric prefix or the Worker Groups starts a new source: collection and model learning restart, and the findings of the previous source close. Changing the schedule or the learning history keeps everything.
Leader request budget¶
TrackMe is gentle with the Cribl Leader at any size. Every caller that reads one Leader — every tenant, every job, the connection test, the wizard discovery — draws from one shared budget for that Leader:
Requests per hour — 1 200 by default (120 to 20 000), a setting of the Cribl connection.
Burst — 150 by default (10 to 2 000), also on the connection.
When several connections reach the same Leader, the lowest values apply.
At most two runs read one Leader at a time, at most 8 requests per second together.
A run that would exceed the budget is deferred and resumes exactly where it stopped (inventory page, Worker directory page, machine-sample position): protection, not failure. When the Leader answers slow down (HTTP 429 with a wait), every later caller waits for it; after three failing runs in a row the Leader is left alone for 60 seconds, doubling up to 30 minutes. The Collection health Cribl Leader card shows the requests of the last hour against the budget, the last run’s requests and data received, and any requested wait.
Only the live and inventory jobs, the connection test, the wizard discovery and the live status of a Source or Destination read the Leader. Every tab, Topology Studio, the policies and the model training read what TrackMe already stored.
Traffic ML models¶
Aspect |
Behaviour |
|---|---|
What gets a model |
One model per object and measurement, all worker processes combined: Sources (events in, bytes in), Destinations (events out, bytes out), Pipelines and Routes (events in, events out). Up to 20 000 models per tenant; the busiest series get one first. |
How it learns |
A normal distribution per hour of the day, then per weekday and hour once 28 days of history are learned. Recent days count more (half-life 14 days by default). |
Learning history |
At creation, the models learn from the history already in the metric index, up to the Model learning history (30 days by default, 0 to 90), bounded by the index’s retention. |
Retraining |
No periodic full retrain: a pass every 6 hours reads only the intervals since the last one and merges them in. Each run has a bounded time budget and continues at the next run where it stopped. |
Relearn |
Relearn one model (or a selection) or Relearn models (all) re-reads the history. TrackMe administrators of the tenant. |
States |
Learning (less than a day, or hours not covered), Low confidence (range shown, opens no finding, until 7 days), Detecting (opens findings), No data (the object stopped reporting). |
Expected range |
Expected value ± z deviations: Normal 3.0, Low 4.0, High 2.5 (custom 2.0 to 6.0); deviations smaller than 10 % of the expected value or 1 event or byte per five minutes are ignored. |
Per-model settings |
Up to 500 models with their own settings (Tune model): training days (7 to 90), direction, pause (closes its Traffic outliers findings), up to 20 excluded periods. |
Not an anomaly |
Excludes the episode (from an hour before it opened to its recovery, 7 days at most) from what the model learns, relearns it at the next training run and dismisses the finding. Power and admin roles of the tenant; for non-administrators the excluded periods of a model total at most 7 days and never leave less history than the model needs to open findings. |
Bank holidays |
With the tenant’s Exclude from ML training effect on (the default), the hours of its bank holidays are not learned, like an excluded period, by training, the history bootstrap, a relearn and the tuning simulation. From the next training run on; no automatic relearn. With Suppress new alerts on, the policies set to hold (Traffic outliers by default, see Policy catalogue) open no new finding while a holiday is active (reason Bank holiday); open findings keep being evaluated. See Bank holidays. |
Expected traffic: automatic inclusion¶
Rule |
Value |
|---|---|
Modes |
Selected (only the objects you select) or Selected + automatic (default of a new tenant; tenants created on earlier builds keep Selected). |
Qualify after |
An object is expected automatically once it carried traffic with no gap longer than 10 minutes for the whole window: 7 days (default) or 2 days. |
Resource types |
Sources and Destinations by default; Pipelines and Routes optional. |
Stops expecting |
After 7 days of silence, when the object is removed from Cribl, or when it leaves the scope. |
Per object |
Exclude, Include again, Keep as selected (adds it to the selected objects). |
Silent and removed objects¶
Rule |
Value |
|---|---|
Shown under No data |
From the second missing five-minute interval. |
Silence counted as zero traffic |
By Traffic outliers only, and only when the Inventory is current and lists the object, monitoring is on, its family is collected, its Worker Group reports, it carried traffic within 24 hours, its model is Detecting (and watches drops), and no mass silence is in progress. A silence can open or keep a finding, never recover one. |
Many objects stopping together |
Most objects of a type in one Worker Group stopping in the same interval is a mass silence: not counted as zero, reported in Collection health (objects stopped together). |
Retention of a silent object |
Removed 30 days after its last measurement (7 days for a worker process), never while a policy selects it or an unresolved finding names it. |
Removed from Cribl |
Missing from two consecutive complete Inventory readings: its findings close at once (removed from Cribl), its silence is never counted, it leaves No data and is purged after 7 days. |
Mass removal |
More than half of a Worker Group’s objects of one type (at least 5) missing at once are held, not removed, and noted in Collection health. |
Forget this object |
Purges a silent or removed object at once. TrackMe administrators of the tenant, up to 100 objects per request, audited. |
Findings lifecycle¶
Aspect |
Value |
|---|---|
Condition |
Failing, Recovered, Awaiting data (unknown), Superseded. |
Status |
Open, Investigating, Resolved, Dismissed. A measured recovery resolves an open or investigating finding; a new failure reopens it. |
Dismiss |
Until it recovers (default), for 1, 7, 30 or 90 days (then back to open), or permanently. |
Snooze |
Up to 30 days; not a status. Stops notifications and colours the object blue in Topology Studio. |
Automatic closures |
Source or connection changed; object not observed or not evaluated for 24 hours; traffic no longer reported; Worker no longer listed; monitoring turned off; model paused; removed from Cribl. |
Finding groups |
The findings of one object form one group automatically (up to 50 members); merge, add, remove and ungroup by hand. See Finding groups. |
Retention |
Closed findings and Finding groups are kept 90 days (20 000 of each at most). At most 5 000 unresolved subjects per tenant. |
Settings reference¶
Settings are saved by a TrackMe administrator of the tenant, except the policies (the tenant’s admin roles) and the monitoring state (the tenant’s power and admin roles).
Section |
Setting |
Default |
Meaning |
|---|---|---|---|
Source & Worker Groups |
Cribl connection |
— |
Fixed once saved: monitoring another deployment is another tenant. |
Cribl metric index |
— |
The metric index that receives Cribl’s internal metrics. |
|
Metric prefix |
|
Letters, digits and dots, ending with a dot. |
|
Worker Groups |
all discovered |
1 to 100. |
|
Repair workspace |
— |
Recreates the tenant’s stores and reports; keeps findings and settings. |
|
Collection & models |
Scheduled collection |
On |
Off = manual collection. |
Model learning history |
30 days |
0 to 90 days; applies the next time the models are relearned. |
|
Traffic ML models |
Sensitivity |
Normal (3.0) |
Low 4.0, High 2.5, or custom 2.0 to 6.0 deviations. |
Seasonality |
Automatic |
Hour of day, weekday and hour, or none. |
|
Weekday pattern after |
28 days |
History needed before the weekday × hour pattern is used. |
|
Recent days count more |
14 days (half-life) |
How fast older days fade. |
|
Open findings once |
7 days of history |
When a model moves from Low confidence to Detecting. |
|
Ignore deviations smaller than |
10 % and 1 event or byte per 5 minutes |
Smallest deviation that counts. |
|
Notifications |
Enable notifications |
Off |
Email delivery of finding transitions. |
Events |
opened, reopened, added to the group, escalated, recovered, dismissal ended, closed |
Other transitions can be added. |
|
Minimum severity |
Medium and above |
Low and above, Medium and above, High and above, Critical only. |
|
Policies |
all |
Limit the emails to some policies. |
|
Delivery account |
the first configured account |
A TrackMe email delivery account, or Default (localhost MTA). |
|
Recipients |
— |
Up to 50, filtered by the account’s allowed domains. |
|
Mode |
Digest |
One digest per live run, or one threaded email per Finding group. |
|
Include charts |
On |
The finding’s chart and key measurements of 24 hours, inline. |
|
AI Agent |
Scheduled triage |
Off |
Hourly review of open findings. |
Mode |
Inspect (recommend only) |
Or Act (apply decisions). |
|
AI provider |
First available provider |
The provider of the scheduled runs. |
|
Review interval (days) |
1 |
0 to 365; a reviewed finding is skipped this long unless a new episode starts. |
|
Findings per run |
10 |
1 to 50. |
|
Time budget per run (seconds) |
3 000 |
60 to 86 400. |
|
Custom instructions |
empty |
Plain English the agent follows on every run, asked or scheduled (at most 4000 characters); Evaluate… tries an unsaved draft in inspect mode and writes nothing. |
|
Allowed actions |
Mark investigating, Dismiss for a time, Not an anomaly |
Resolve and Stop monitoring are off by default. |
|
Longest dismissal (days) |
7 |
1 to 90. |
|
Proven inactivity (days) |
7 |
1 to 365; how long an object must be inactive before the agent may stop monitoring it. |
|
Assignment |
Assignment groups, rules |
none |
|
Finding actions |
Actions, automation rules |
none |
See Findings actions. |
Policies tab |
Every policy, Expected traffic mode |
see Policy catalogue |
Tenant admin roles. |
Inventory tab |
Exclusion rules, objects turned off |
none |
Up to 200 rules and 20 000 objects turned off; tenant power and admin roles. |
Cribl connection |
Verify SSL certificate, CA certificate chain, requests per hour, burst |
On, —, 1 200, 150 |
Configuration → Cribl connections (Cribl connections). |
The Source & Worker Groups section also has two settings available through the REST API only: the bucket span (300 s) and the settle delay (120 s).
Finding events¶
Every transition is written as one JSON event in the tenant’s notable index, whatever the notification settings:
sourcetype=trackme:cribl:finding— one event per finding transition (event_kind=finding_transition);sourcetype=trackme:cribl:finding_group— one event per Finding group transition (event_kind=finding_group_transition).
Main fields: tenant_id and object_category (cribl_stream), finding_id,
group_id, transition and transition_label (Finding opened, Severity raised,
Measurement recovered, Closed automatically, …), transition_id (deduplicate on it),
policy_id and policy_name, severity, display_severity, peak_severity,
condition, status, reason, family, worker_group, entity_name (the
object), kpi, value, expected_lower, expected_upper, assignee, actor.
index=trackme_notable sourcetype=trackme:cribl:finding tenant_id="<tenant>"
transition IN ("opened", "reopened", "escalated") severity="critical"
index=trackme_notable sourcetype=trackme:cribl:finding_group tenant_id="<tenant>"
| stats latest(_time) as last_at latest(status) as status latest(severity) as severity
latest(object) as object latest(worker_group) as worker_group by group_id
| where status IN ("open", "investigating")
Use them for Splunk alerts, Enterprise Security correlation searches or SOAR playbooks.
Permissions¶
Permissions combine a TrackMe capability with the tenant’s roles. Members of admin or
sc_admin can do everything on every tenant. A TrackMe administrator of one tenant has no
right on another.
Tier |
Who |
Actions |
|---|---|---|
Read |
|
Every tab, every chart, the investigation drawer, the AI Assistant, drilldown and link actions. |
Triage |
|
Status, dismiss, snooze, severity, assignment, notes, Finding groups, Not an anomaly, monitoring on / off and exclusion rules, Stop monitoring this object, Ask the AI Agent and apply its recommendations, send a webhook action. |
Policies |
|
Edit policies, the Expected traffic mode and its automatic list. |
Manage (a TrackMe administrator of this tenant) |
|
Source settings, Collect now and Run now, model tuning and relearn, Repair workspace, Forget this object, notifications, AI Agent settings, assignment and finding actions. |
Tenant owner. The tenant’s reports run as its owner: nobody, a Splunk administrator, or
an account holding trackmeuseroperations, trackmepoweroperations,
trackmeadminoperations, schedule_search, list_settings, list_storage_passwords
and admin_all_objects. The scheduled AI Agent needs an owner that is a Splunk
administrator account (not nobody).
Cribl connections are managed in Configuration → Cribl connections (Cribl connections).
Configuration Guardian checks¶
cribl_collection_health (tenant scope, warning; evaluated at every tenant health cycle) is
raised with one or more of these problems. A disabled tenant, a tenant without a configured
source, or a tenant on manual collection reports nothing.
Problem code |
Meaning |
|---|---|
|
No live metrics run since the source was configured, or none finished in the last 20 minutes. |
|
Live runs happen but publish nothing, or their traffic scoring fails, for 30 minutes. |
|
No Cribl metric reached the metric index for 30 minutes. |
|
Selected Worker Groups send no metric. |
|
No inventory run since the source was configured, or none finished for 3 hours. |
|
Inventory runs do not complete for 90 minutes. |
|
The inventory was last published more than a day ago (runs keep waiting for the Leader or do not finish). |
|
A full inventory cycle takes more than a day under the Leader request budget. |
|
Worker machine samples do not keep up under the Leader request budget. |
|
The Worker directory is not refreshed (30 minutes): Worker availability and versions are out of date. |
|
The directory of objects that stopped reporting is not kept up to date. |
|
Many objects stopped reporting in the same interval (for 30 minutes). |
|
Policy evaluation does not complete for 30 minutes. |
|
Model training does not complete for 90 minutes. |
|
A capacity limit stopped collection, scoring or training, or some traffic series have no model (more series than the tenant keeps models for). |
|
Finding emails fail for 30 minutes. |
Notes shown in Collection health that never raise the check: candidates capped, objects removed together, Expected traffic capped or out of time.
System checks on Cribl connections:
cribl_connection_health— the daily connection probe fails: warning, critical after 24 hours; visible to administrators.cribl_connection_tls_unverified— a self-managed connection has certificate verification turned off: warning.
Also relevant: insufficient_tenant_owner_capabilities (the tenant owner lacks a required
capability), ai_automation_owner_is_system_user (scheduled AI Agent with a nobody
owner), findings_action_delivery_failing (webhooks or other findings actions failing) and
topology_view_stale_references. See Configuration Guardian.
REST endpoints¶
All endpoints are under /services/trackme/v2/ and described in the REST API Reference
(describe=true). Reads run with the caller’s session; changes are authorized on the
caller’s session first, then applied and audited.
Group |
Operations ( |
|---|---|
Workspace and reads |
|
Triage |
|
Policies |
|
Manage |
|
Other |
|
Capacity¶
Limit |
Value |
|---|---|
Monitored objects per tenant (Sources, Destinations, Pipelines, Routes, worker processes in the snapshot) |
About 20 000; split a larger deployment across tenants by Worker Group. |
Worker Groups per tenant |
100 |
Traffic ML models |
20 000 |
Models with their own settings |
500 |
Unresolved findings (subjects) |
5 000; beyond, new failures are refused while every recorded one keeps being evaluated. |
Closed findings and Finding groups kept |
20 000 each, 90 days |
Selected objects per policy |
128 |
Exclusion rules / objects turned off one by one |
200 / 20 000 |
Notification recipients |
50 |
Forget this object |
100 objects per request |
Models need about a week of history before they open findings, unless the metric index already holds that history at creation.
Logs¶
Every line starts with tenant_id="<tenant>".
Log |
Sourcetype |
Contains |
|---|---|---|
|
|
One summary line per run; WARNING when a run did not complete; Cribl collector failure for an internal error. |
|
|
Failures of the tuning simulation. |
|
|
One line per test email. |
|
|
Degraded paths of the REST API. |
index=_internal sourcetype=trackme:custom_commands:trackmecriblcollect tenant_id="<tenant>"
index=_internal sourcetype=trackme:custom_commands:trackmecriblcollect tenant_id="<tenant>" job="live"
| stats count, latest(_time) as last, values(reason) as reasons by complete
Tokens, passwords, email addresses, request bodies and search strings are never logged.
Troubleshooting¶
Symptom |
Likely cause |
Fix |
|---|---|---|
The wizard proposes no metric index |
Cribl’s internal metrics do not reach a Splunk metric index, or they use another prefix. |
Route Cribl’s internal metrics to a Splunk metric index; set the metric prefix your deployment uses. |
Metrics not arriving in Collection health |
The metrics stopped reaching the index, or the index or prefix is wrong. |
Check the Cribl Splunk Destination and the metric index; compare the prefix in Settings. |
Worker Groups without metrics |
A selected Worker Group does not send its internal metrics. |
Enable the internal metrics on that Worker Group, or deselect it. |
Leader · unavailable, a Critical Leader availability finding |
Network, certificate, credentials or service problem on the Leader API (the tooltip names it). |
Certificate not trusted or host name mismatch: add the CA chain to the connection. Credentials refused: update them. Then test the connection. |
Leader · rate limited, deferred runs |
The Leader asked TrackMe to slow down, or the request budget is spent. |
Wait; lower the connection’s requests per hour if the Leader is shared; check the Cribl Leader card in Collection health. |
Inventory cycle slow / Worker sampling slow |
The deployment needs more requests than the Leader budget allows. |
Raise the connection’s requests per hour if the Leader can take it, or split the deployment across tenants. |
Traffic outliers open nothing |
Models are still Learning or Low confidence. |
Wait until they are Detecting (7 days of history), or recreate with a longer Model learning history. |
Too many Traffic outliers findings |
The models are too sensitive for this traffic. |
Use Not an anomaly on expected episodes; lower the sensitivity or raise Ignore deviations smaller than; limit the direction to drops. |
Worker memory findings on healthy hosts |
Memory used includes the file cache. |
Keep Worker machine memory off, or raise its threshold. |
A deleted object still shows under No data |
The Inventory has not seen it missing twice yet, or it was turned off. |
Wait for the next inventory cycle, or Forget this object. |
The scheduled AI Agent does not run |
AI Assistant disabled, no provider, or a |
Enable the AI Assistant, configure a provider, and give the tenant a Splunk administrator owner. |
No email |
Notifications off, events or severity filtered out, or delivery failing. |
Check Settings → Notifications, then Send test email; see
|
See also
Cribl Stream — the Cribl Stream user guide.
Cribl connections — Cribl connections, TLS and the Leader budget.
Findings, Finding groups and findings actions — findings-based monitoring.
Cribl Edge reference — the Cribl Edge reference.