splk-feeds - Creating and managing Hybrid Trackers
##################################################

Introduction to Hybrid Trackers
===============================

**Hybrid Trackers are created and managed through TrackMe, these are basically scheduled backend jobs orchestrating the entities discovery and management for TrackMe splk-feeds components:**

- Hybrid Trackers are scheduled reports which involve various TrackMe backend tools depending on the TrackMe component
- A single Hybrid Tracker can discovery and manage a few or many entities according to the needs
- Hybrid Trackers are integrating into a main application workflow, which involves concepts such as registering their execution statuses, registering run time performance etc
- Hybrid Trackers can be created at any time, through guided user interfaces or TrackMe REST endpoints
- In the context of Splunk feeds tracking, Hybrid Trackers can as well be created during the initial creation of the Virtual Tenant
- When creating a trackers, the related knowledge objects will be owned by the owner defined at the Virtual Tenant level
- TrackMe keeps records of the knowledge objects related to the Hybrid Trackers, therefore you need to manage its life cycle through TrackMe

.. image:: img_v2/hybrid_trackers/overview.png
   :alt: overview.png
   :align: center
   :width: 1200px
   :class: with-border

Creating an Hybrid Tracker for splk-feeds
=========================================

*These instructions are related to the splk-dsm component, options for splk-dhm/splk-mhm may differ but the logic underneath is similar.*

**To create a new Hybrid Tracker, access to the tenant then click on "Manage: Hybrid Trackers":**

.. image:: img_v2/hybrid_trackers/splk_dsm/screen1.png
   :alt: screen1.png
   :align: center
   :width: 1200px
   :class: with-border

.. image:: img_v2/hybrid_trackers/splk_dsm/screen2.png
   :alt: screen2.png
   :align: center
   :width: 1200px
   :class: with-border

**splk-dsm Hybrid Tracker creation wizard:**

.. image:: img_v2/hybrid_trackers/splk_dsm/screen3.png
   :alt: screen3.png
   :align: center
   :width: 1200px
   :class: with-border

**Once in the creation wizard, follow the guided steps:**

*Hybrid Tracker identifier:*

- Provide with a name for the Hybrid Tracker, this will be included in the name of the Splunk Knowledge Objects related to this tracker
- In the example below, we will name our tracker "endpoints_os_data" as it deals with events originating from Operating Systems

.. image:: img_v2/hybrid_trackers/splk_dsm/screen4.png
   :alt: screen4.png
   :align: center
   :width: 1200px
   :class: with-border

*Target Splunk deployment:*

- Specify if the data is searchable locally on the Splunk deployment, or if the trackers deals with a remote Splunk deployment
- If a remote Splunk deployment is selected, TrackMe performs first a connectivity check to that environment

.. image:: img_v2/hybrid_trackers/splk_dsm/screen5.png
   :alt: screen5.png
   :align: center
   :width: 1200px
   :class: with-border

*Search mode and search root constraint:*

*tstats versus raw*

- Then, define the search mode, you can choose between tstats and raw
- tstats is generally recommended as it provides much faster and efficient searches relying on Splunk tsidx files
- However, tstats requires all fields to indexed fields, while a raw search can deal with search time extracted fields
- Therefore, raw search provides much more flexibility, but the cost is as well much higher
- Depending on your context, raw searches may be fully valid but if a tstats search can be used equally, use tstats

*root search constraint:*

- Define the Splunk root search constraint, the constraint comes pre-filled with normally expected constraints which requires valid data, excludes TrackMe related, etc
- Add to your own search filters according to your needs, in our example we add an index filter "(index=linux* OR index=win*)"

.. image:: img_v2/hybrid_trackers/splk_dsm/screen6.png
   :alt: screen6.png
   :align: center
   :width: 1200px
   :class: with-border

*break by logic:*

- You can optionally add an additional break by logic field
- This defaults to "none" which means entities are going to match the combo ``index + ":" + sourcetype``
- For instance, we have an indexed field ``region``, we can leverage it here to distinguish entities per region, our entities creation logic became ``index + ":" + sourcetype + ":" + region``

.. image:: img_v2/hybrid_trackers/splk_dsm/screen7.png
   :alt: screen7.png
   :align: center
   :width: 1200px
   :class: with-border

.. image:: img_v2/hybrid_trackers/splk_dsm/screen8.png
   :alt: screen8.png
   :align: center
   :width: 1200px
   :class: with-border

*Time quantifiers:*

- Review and update if necessary indexed time earliest and latest, as well as time range earliest and latest
- These time quantifiers drive the period of data that the tracker is going to cover
- Generally, you will want to have a large event time range period to cover data with high latency, while the period for indexed time range can be more restricted for performance optimization purposes
- What will work best and be the most efficient depends a lot on your context and environment, start with these values, review and adapt if necessary

.. image:: img_v2/hybrid_trackers/splk_dsm/screen9.png
   :alt: screen9.png
   :align: center
   :width: 1200px
   :class: with-border

*Cron schedule:*

- Define the cron schedule for the Hybrid Tracker
- It defaults to every 5 minutes, note that TrackMe will automatically dispatch cron schedule for optimization purposes

.. image:: img_v2/hybrid_trackers/splk_dsm/screen10.png
   :alt: screen10.png
   :align: center
   :width: 1200px
   :class: with-border

*Test and review:*

- Click on the button to execute in preview the Hybrid Tracker

.. image:: img_v2/hybrid_trackers/splk_dsm/screen11.png
   :alt: screen11.png
   :align: center
   :width: 1200px
   :class: with-border

*Finally, validate the Hybrid Tracker creation:*

.. image:: img_v2/hybrid_trackers/splk_dsm/screen12.png
   :alt: screen12.png
   :align: center
   :width: 1200px
   :class: with-border

*Once created, you can choose to run the Tracker immediately to discover and create entities in the Virtual Tenant:*

.. image:: img_v2/hybrid_trackers/splk_dsm/screen13.png
   :alt: screen13.png
   :align: center
   :width: 1200px
   :class: with-border

Managing Hybrid Trackers for splk-feeds
=======================================

Deleting an Hybrid Tracker through the UI
-----------------------------------------

If you want to delete an existing Hybrid Tracker, this operation must be done via TrackMe.

The reason is that the application keeps track of all knowledge objects that were created for a given tenant, to honour various features such as managing the life cycle of the tenant (enabling / disabling, etc) or the life cycle of the tracker itself.

**To manage Hybrid Trackers, click on:**

.. image:: img_v2/hybrid_trackers/manage/screen1.png
   :alt: screen1.png
   :align: center
   :width: 1200px
   :class: with-border

**The user interface shows available trackers and their related objects:**

.. image:: img_v2/hybrid_trackers/manage/screen2.png
   :alt: screen2.png
   :align: center
   :width: 1200px
   :class: with-border

**Select one of more trackers to be deleted:**

.. image:: img_v2/hybrid_trackers/manage/screen3.png
   :alt: screen3.png
   :align: center
   :width: 1200px
   :class: with-border

The related knowledge objects will be deleted and the Virtual Tenant record cleaned up automatically.

For splk-feeds, the entities that were created through these Hybrid Trackers will **not** be deleted. (however, unless another Tracker is created, these will not be maintained anymore)

Deleting an Hybrid Tracker through REST
---------------------------------------

**You can delete a Tracker through the following REST endpoint, example in SPL:**

::

   | trackme mode=post url="/services/trackme/v2/splk_hybrid_trackers/admin/hybrid_tracker_delete" body="{'tenant_id': 'mytenant', 'component': 'dsm', 'hybrid_trackers_list': 'test:001,test:002'}"