TrackMe Sourcetypes & Metrics
About TrackMe sourcetypes and metrics
TrackMe generates various events and metrics for the purposes of its activity.
The destination indexes for both events and metrics are entirely configurable, and potentially on a per Virtual Tenant basis.
Events generated by TrackMe are JSON events, and parsed using KV_MODE=json.
Metrics are generated and stored in the Splunk metrics store using metric indexes as per the Virtual Tenant configuration.
TrackMe events & sourcetypes
The following event sourcetypes are generated by TrackMe:
To list all events from a Splunk search:
| tstats count where index=trackme_* by sourcetype
Sourcetype |
Purpose |
|---|---|
|
TrackMe audit events, for instance when modifications are made against TrackMe entities |
|
TrackMe flipping state events, flipping events are generated when entities switch from one state to another |
|
Handler events are generated when TrackMe logics, such as trackers, are executed against a given entity. These events allow identifying which logics are maintaining and performing operations on which entities. |
|
Health events are generated to monitor the health status of TrackMe components and logics, for instance these events are used to identify when a given Virtual Tenant is degraded due to an issue in a tracker. |
|
TrackMe has a concept of SLA tracking, when entities are breaching their SLA and the associated SLA class definition, an SLA breach event will be generated. (one event every 24 hours by default) |
|
State events are generated when entities are maintained by TrackMe, and contain key information about the entities statuses. |
|
Stateful events are generated via the concept of state aware alerting, when incidents are opened, updated or closed. |
|
TrackMe notable events produced by the TrackMe notable alert action |
|
TrackMe SmartStatus events produced by the TrackMe SmartStatus alert action, when an alert triggers and has the SmartStatus alert action enabled, automated investigations are performed and their results are indexed as SmartStatus events. |
|
These events are specific to the Workload component (splk-wlk), and are generated when an updated version (or first time discovered) of a monitored Splunk scheduled search occurs. |
TrackMe main metrics
TrackMe generates various metrics per component, using the following strict convention:
To list all metrics from a Splunk search:
| mcatalog values(metric_name) as metrics, values(_dims) as dimensions where index=trackme_metrics
To view the content of metrics in a practical way such as you would with events in Splunk:
| mpreview index=trackme_metrics filter="metric_name=trackme.*"
metric_name |
Purpose |
|---|---|
|
Metrics generated by the Feeds tracking components (splk-dsm/splk-dhm/splk-mhm), which contain notably metrics are latency and availability for TrackMe feeds entities. |
|
Metrics generated by the Flex Object splk-flx component. In Flex, a tracker can generate any kind of metrics depending on the use cases, from system metrics to functional or business metrics. |
|
Metrics generated by the CIM compliance component. (splk-cim) |
|
Metrics generated by the Workload component. (splk-wlk) |
|
Metrics generated when trackers are executed, which trace the health status and runtime of the various TrackMe components and logics such as trackers. |
|
These metrics are generated for SLA tracking purposes, and contain the state of the entities in a numerical format. |
TrackMe logging events
About TrackMe logging
TrackMe has essentially two types of components generating log messages, REST API endpoints and custom commands.
Both carefully perform message logging, according to the logging level defined in the TrackMe global configurations.
REST API endpoints log messages
To access REST API log messages, you would use the following search:
index=_internal sourcetype=trackme:rest_api
Custom commands log messages
To access custom commands log messages, you would use the following search:
index=_internal sourcetype=trackme:custom_commands:*