Cribl monitoring¶

TrackMe can monitor one or more Cribl Stream (Logstream) environments at scale, using the Flex Objects component and a ready-made library of Cribl use cases built on Cribl’s internal metrics in Splunk.

Note

Cribl monitoring uses the Flex component (splk-flx), which is a restricted, Enterprise / Unlimited capability (not available with the Foundation Edition trial).

What it tracks¶

The built-in library provides the following Cribl Stream use cases. A single tracker covers all worker groups — the searches break by Cribl’s group dimension, which also names the resulting entities.

Use case (`uc_ref`)	Monitors	Metrics
`cribl_logstream_health_inputs`	Cribl Logstream health inputs status	`cribl_logstream.health.health_inputs`
`cribl_logstream_health_outputs`	Cribl Logstream health outputs status	`cribl_logstream.health.health_outputs`
`cribl_logstream_hosts_cpu_usage`	Hosts CPU usage; triggers on high-usage thresholds	`cribl_logstream.avg_cpu_usage`
`cribl_logstream_output_destination_pressure`	Destination outputs blocked / under backpressure	`cribl_logstream.output.blocked_outputs`, `cribl_logstream.output.backpressure_outputs`
`cribl_logstream_pipeline`	Pipelines	`cribl_logstream.pipeline.in_events`, `.out_events`, `.dropped_events`, `.pct_sent_events`, `.pct_dropped_events`
`cribl_logstream_route_traffic`	Route traffic	`cribl_logstream.route.route_in_bytes` / `.route_out_bytes` / `.route_in_mbytes` / `.route_out_mbytes` / `.route_in_events` / `.route_out_events`
`cribl_logstream_total_traffic_inputs`	Total input traffic	`cribl_logstream.total.total_in_bytes`, `.total_in_events`, `.total_in_mbytes`
`cribl_logstream_total_traffic_outputs`	Total output traffic	`cribl_logstream.total.total_out_bytes`, `.total_out_events`, `.total_out_mbytes`

These use cases ship in the Flex Object use-case library; you can also customise a built-in use case or add your own. Review the use-case details before creating them with:

| trackmesplkflxgetuc | search uc_vendor=Cribl uc_category=cribl_logstream

Requirements¶

Cribl internal metrics¶

TrackMe’s Cribl monitoring relies on Cribl internal metrics indexed in Splunk. The pre-built searches search all metric indexes by default (where index=*); for performance you can pin your real Cribl metrics index when creating the Flex trackers, for example:

| mstats sum(cribl.logstream.route.in_bytes) as route_in_bytes, sum(cribl.logstream.route.in_events) as route_in_events, sum(cribl.logstream.route.out_bytes) as route_out_bytes, sum(cribl.logstream.route.out_events) as route_out_events where index=* host=* by group, name

For more on Cribl Logstream metrics in Splunk, see https://docs.cribl.io/stream/internal-metrics.

Hint

Multiple worker groups

If your Cribl deployment has multiple worker groups, there is nothing extra to do — a single tracker manages all worker groups individually. All searches break against the Cribl group dimension, which is also used to create and maintain the entities.

A Flex-enabled tenant¶

You need a TrackMe tenant with the Flex Object component enabled — either a dedicated tenant for Cribl monitoring or any existing tenant. Once the Flex trackers are created, TrackMe automatically groups the resulting Cribl entities:

Use case (`uc_ref`)	Entity group
`cribl_logstream_health_inputs`	`Cribl_Logstream:health`
`cribl_logstream_health_outputs`	`Cribl_Logstream:health`
`cribl_logstream_hosts_cpu_usage`	`Cribl_Logstream:infrastructure`
`cribl_logstream_output_destination_pressure`	`Cribl_Logstream:Destination`
`cribl_logstream_pipeline`	`Cribl_Logstream:pipeline_traffic`
`cribl_logstream_route_traffic`	`Cribl_Logstream:route_traffic`
`cribl_logstream_total_traffic_inputs`	`Cribl_Logstream:traffic_in_total`
`cribl_logstream_total_traffic_outputs`	`Cribl_Logstream:traffic_out_total`

Setting it up¶

In your Flex-enabled tenant, open Flex tracker management and load one or more Cribl use cases:

Enter a meaningful name for the tracker.
Select Cribl as the vendor.
Select the use-case reference identifier (uc_ref).
Review the high-level info TrackMe shows for the use case (generated metrics, requirements, recommendations).
Review the generated SPL — in particular set your real Cribl metrics index explicitly for performance.
Click Simulate, review the results, then create the tracker (earliest/latest times are set automatically).

Once created, TrackMe maintains the entities and detects issues automatically from Cribl’s internal metrics — for example, a Splunk S2S destination outage surfaces as a state change on the corresponding Cribl_Logstream:Destination entities.