Auto deletion or management of TrackMe entities
Introduction to automated management of TrackMe entities
About this paper:
In some use cases, you may want to automatically manage entities in TrackMe based on different criterias.
For instance, if you track hosts in TrackMe, you could for example automatically delete machines that have been tagged as decommissioned in your CMDB.
This way, you would automatically keep your TrackMe entities in sync with your CMDB and purge entities as needed.
Automatically deleting entities in TrackMe with a custom search
In this example, we are managing hosts in TrackMe through the splk-dhm component, we can access to the real time view very easily using the following command:
Note: Click on the “Search table” button in TrackMe’s UI to access this search.
Hint
Replace tenant and component name:
Replace “mytenant” with the tenant ID you want to delete entities from.
| trackmegetcoll tenant_id=hosts-dsm-tracking component=dsm
Then, let’s add a table to focus on specific fields:
| trackmegetcoll tenant_id=host-dhm-tracking component=dhm
| table keyid, object, alias, object_state
Now, let’s assume that we want to filter on entities which have stopped forwarding data to Splunk, therefore that are currently in a red state.
| trackmegetcoll tenant_id=host-dhm-tracking component=dhm
| table keyid, object, alias, object_state
``` select entities in alert ```
| where object_state="red"
Then, we will cross with our CMDB which is available in Splunk, we will use the alias field which contains the true host as seen in Splunk, you obviously can add any additional logic and/or customise the logic to match your context and requirements:
| trackmegetcoll tenant_id=host-dhm-tracking component=dhm
| table keyid, object, alias, object_state
``` select entities in alert ```
| where object_state="red"
``` cross with your CMDB ```
| lookup my_hosts_cmdb ci_id as alias OUTPUT ci_status
| where isnotnull(ci_status) AND ci_status!=""
The next step is to turn this into a comma separated list of keyids, so that we can use it in a delete command using TrackMe’s API endpoints and built in pure SPL, we will also use the map command from Splunk:
| trackmegetcoll tenant_id=host-dhm-tracking component=dhm
| table keyid, object, alias, object_state
``` select entities in alert ```
| where object_state="red"
``` cross with your CMDB ```
| lookup my_hosts_cmdb ci_id as alias OUTPUT ci_status
| where isnotnull(ci_status) AND ci_status!=""
``` build the keyid comma separated list ```
| stats values(keyid) as keyid
| eval keyid=mvjoin(keyid, ",")
Last, and final step, call the endpoint:
| trackmegetcoll tenant_id=host-dhm-tracking component=dhm
| table keyid, object, alias, object_state
``` select entities in alert ```
| where object_state="red"
``` cross with your CMDB ```
| lookup my_hosts_cmdb ci_id as alias OUTPUT ci_status
| where isnotnull(ci_status) AND ci_status!=""
``` build the keyid comma separated list ```
| stats values(keyid) as keyid
| eval keyid=mvjoin(keyid, ",")
``` call map and the TrackMe entities deletion endpoint ```
| map search="| trackme url=\"/services/trackme/v2/splk_dhm/write/dh_delete\" mode=\"post\" body=\"{'tenant_id':'host-dhm-tracking', 'deletion_type': 'temporary', 'keys_list':'$keyid$', 'comment_update': 'This host was detected as decommissioned in the CMDB and automatically purged.'}\""
We will schedule this search to run automatically, it doesn’t need to be running on a frequent basis, and most likely once per day or every few hours would be more than enough.
Any entities that are detected as decommissioned in the CMDB will be automatically purged from TrackMe, we also have TrackMe audit events should we need to review the deletion history.
Alternative using the Flex object component (splk-flex)
Licensed TrackMe users can also leverage instead the Flex object component to track hosts, the same logic can be used very easily too:
| trackmegetcoll tenant_id=hosts-flx-tracking component=flx
| table keyid, object, alias, object_state, object_description
``` select entities in alert ```
| where object_state="red"
``` cross with your CMDB ```
| lookup my_hosts_cmdb ci_id as alias OUTPUT ci_status
| where isnotnull(ci_status) AND ci_status!=""
``` build the keyid comma separated list ```
| stats values(keyid) as keyid
| eval keyid=mvjoin(keyid, ",")
``` call map and the TrackMe entities deletion endpoint ```
| map search="| trackme url=\"/services/trackme/v2/splk_flx/write/flx_delete\" mode=\"post\" body=\"{'tenant_id':'hosts-flx-tracking', 'deletion_type': 'temporary', 'keys_list':'$keyid$', 'comment_update': 'This host was detected as decommissioned in the CMDB and automatically purged.'}\""
Alternative logic disabling rather then deleting entities
Alternatively, we may want to disable entities, rather than deleting them, the same logic would apply and we would simply call a different endpoint.
With our splk-dhm example:
| trackmegetcoll tenant_id=host-dhm-tracking component=dhm
| table keyid, object, alias, object_state
``` select entities in alert ```
| where object_state="red"
``` cross with your CMDB ```
| lookup my_hosts_cmdb ci_id as alias OUTPUT ci_status
| where isnotnull(ci_status) AND ci_status!=""
``` build the keyid comma separated list ```
| stats values(keyid) as keyid
| eval keyid=mvjoin(keyid, ",")
``` call map and the TrackMe entities deletion endpoint ```
| map search="| trackme url=\"/services/trackme/v2/splk_dhm/write/dh_monitoring\" mode=\"post\" body=\"{'tenant_id':'host-dhm-tracking', 'action': 'disable', 'keys_list':'$keyid$', 'comment_update': 'This host was detected as decommissioned in the CMDB and automatically disabled.'}\""
With our splk-flx example:
| trackmegetcoll tenant_id=host-flx-tracking component=dhm
| table keyid, object, alias, object_state
``` select entities in alert ```
| where object_state="red"
``` cross with your CMDB ```
| lookup my_hosts_cmdb ci_id as alias OUTPUT ci_status
| where isnotnull(ci_status) AND ci_status!=""
``` build the keyid comma separated list ```
| stats values(keyid) as keyid
| eval keyid=mvjoin(keyid, ",")
``` call map and the TrackMe entities deletion endpoint ```
| map search="| trackme url=\"/services/trackme/v2/splk_dhm/write/flx_monitoring\" mode=\"post\" body=\"{'tenant_id':'host-flx-tracking', 'action': 'disable', 'keys_list':'$keyid$', 'comment_update': 'This host was detected as decommissioned in the CMDB and automatically disabled.'}\""
Hint
The same type of logic applies to anything in TrackMe:
In fact, you totally can use the same type of logic in many variations, whatever the component you are dealing with.
You can for example decide to have entities with a monitored state defined as disabled when entities are discovered, then wait for at least 7 days of historical knowledge to be collected before enabling the monitoring, or any other concept that makes sense for your context.
Consult the REST API reference dashboard in TrackMe, menu API & Tooling to find which endpoints are available for each component, as well as their usage and usage examples.
