Use Case Demo: 360 Services Monitoring with TrackMe¶
Use Case Demo: 360 Services Monitoring with TrackMe
This use case demo demonstrates how TrackMe can be used to perform a 360 degree monitoring of the different services that are commonly composing Splunk environments, with addition of third parties notably Cribl Logstream.
- The purpose of this demo is to show step by step how to design and implement TrackMe concepts and features, notably:
Data Tiers:Monitoring high priority Splunk Feeds availability and performance using TrackMe componentsplk-dsm.Data Tiers:Monitoring high priority endpoints availability (Think about Active Directory domain controllers, Checkpoint firewalls, etc.) using TrackMe componentsplk-dsm.Data Tiers:Monitoring abnormal volume variations in Splunk indexes and Splunk license usage using TrackMe componentsplk-flx. (Since TrackMe 2.4.18 thesplk-volcomponent covers per-index licensed volume out of the box — see VOL — Volume Outliers.)Splunk Tiers:Monitoring key aspects and metrics of Splunk Indexers Cluster using TrackMe componentsplk-flx.Splunk Tiers:Monitoring key aspects and metrics of Splunk Search Head Cluster or Search Heads using TrackMe componentsplk-flx.Splunk Tiers:Monitoring key aspects of Splunk Heavy Forwarder tiers using TrackMe componentsplk-flx. (Since TrackMe 2.4.22 the Splunk S2S tenant covers the receivers and intermediate forwarders, and since 2.4.24 the Splunk Forwarder Collectors tenant covers the heavy forwarders used as collection points and their inputs — see Splunk S2S and Splunk Forwarder Collectors.)Splunk Tiers:Monitoring Splunk deployment servers and clients using TrackMe componentsplk-flx. (Since TrackMe 2.4.22 the Splunk UF Fleet tenant covers the forwarding agents and their Deployment Servers — see Splunk UF Fleet.)Use Cases & Controls:Monitoring Splunk core & Splunk Enterprise Security use cases using TrackMe Workload componentsplk-wlk.Use Cases & Controls:Monitoring various environments control points using TrackMe componentsplk-flx.Cribl Logstream:Monitoring Cribl Logstream availability and performance using TrackMe componentsplk-flx. (Since TrackMe 2.4.19 the Cribl Stream and Cribl Edge tenants monitor Cribl directly — see Cribl Stream and Cribl Edge.)Splunk SOAR tier:Monitoring Splunk SOAR platforms using TrackMe componentsplk-flx.
Some of the components leveraged in this demo are restricted features available in TrackMe Enterprise Edition & Unlimited Edition.
This demo documentation is currently a work in progress and will be updated in the future to reflect the latest features and capabilities of TrackMe.
Pictures Gallery¶
The following image shows a template Splunk dashboard which calls TrackMe Flex converging entities, which transparently correlate the status of TrackMe entities to form the representation of the different tiers in the environment: (find this template in the API & Tooling menu, from TrackMe 2.1.18)
The following images show TrackMe Virtual Tenants Home page:
An incident is affecting the Splunk Indexers Cluster tier:
The Services Monitoring Virtual Tenant view:
The Services Monitoring Dashboard view:
The StateFul Opening incident Email notification:
The StateFul Opening incident notification from the Splunk Indexer tier tenant:
Several views of the faulty Splunk indexer:
A few pictures from the global Splunk cluster entity view:
After some time, the issue is resolved, the faulty indexer is back in service, the incident is closed and the dashboard shows the updated situation for our Splunk tiers:
Closure incident Email notifications were sent to the team:
Notes: for the documentation purposes, we show notification for the service as well as the entities, but you may choose to only send notifications for the tiers services.