Feeds - Tags enrichments management
Tags enrichments for splk-dsm (Data Sources)
TrackMe version 2.0.84
These instructions require TrackMe version 2.0.84 and later
For Data Sources, TrackMe allows defining per entity tags manually, and/or in a more effiscient manner using tag policies.
Accessing, defining and managing tags policies
You can access tags policies from the main TrackMe’s tenant screen via the button Manage: tags policies:
About Tags policies:
Regex based: Tags polciies are regular expressions which define a number of tags for matching entities
Tags submission: These polcies are monitored are monitored by a scheduled job named
trackme_dsm_tags_tracker_tenant_<tenant_id>Multiple matching policies: Multiple policies can be apply, this means a given entity can receive tags from one or more tags policies
Lower case: Tags are automatically stored in lower case format
Removing tags policies: When tags policies are removed, the tags were associated with entities and from the removed policies are removed from the entities automatically
Tags in the KVstore records: Tags are stored in the entities KVstore records, and can be used for alerting filtering purposes
Tags in the user interface: Tags are displayed in the user interface for the entities
Tags policies usage example
In the following example, we have a number of entities which have in comming some patterns we can leverage for tags policies purposes:
geo-locations: the entities naming convention includes the source location of these feeds
technology information: we can also leverage some of these information to associate these entities with technology related context tags
Example of tags policies definition:
Once we have created our policies, the tags trackers applies the policies automatically, we can also manually execute the tags tracker and the hybrid tracker(s) to reflect these changes immediately:
After this, tags become visible in the user interface:
Tags can also be searched in the user interface filters:
Tags can be used when creating a TrackMe alert:
Tags are stored as comma separated list of values (CSV) in the KVstore collections:
| inputlookup trackme_dsm_tenant_secops-<replace with tenant_id>
| table object, *tags*
Tags are also included in the trackme:state events:
index=trackme_summary sourcetype="trackme:state" tenant_id=<replace with tenant_id>
Tags can also be accessed through the properties fields of TrackMe notable events:
index=trackme_notable tenant_id=<replace with tenant_id>
Tags are stored in a dedicated KVstore collection per tenant:
| inputlookup trackme_common_tag_policies_tenant_secops-merged-all
Troubleshoot tags policies
Tags policies tracker
The first component is the scheduled job which applies the tags policies to the entities, this job is called as follows:
trackme_dsm_tags_tracker_tenant_<tenant_id>
This jobs should be scheduled, and properly executed:
You can find its logs as follows:
index=_internal sourcetype=trackme:custom_commands:trackmesplktags
When you have multiple tenants, you may want to filter on the tenant identifier:
index=_internal sourcetype=trackme:custom_commands:trackmesplktags tenant_id=<replace with tenant_id>
In case of any issues with the tags policies, ensure that the job exists, is scheduled and executes properly and continue with the following steps.
Tags policies REST API endpoint
The scheduled job calls the REST API endpoint to apply the tags policies, you can review the logs for this endpoint as follows:
index=_internal sourcetype=trackme:rest_api post_tag_policies_apply
With multiple tenants, filter on the tenant identifier:
index=_internal sourcetype=trackme:rest_api post_tag_policies_apply tenant_id=<replace with tenant_id>
In case of issues, carefully review the logs and verify if there are exceptions or issues with the tags policies:
index=_internal sourcetype=trackme:rest_api post_tag_policies_apply log_level=error
The REST API endpoint can be applied manually as well, you can easily get the list of endpoints and their usage, review the REST API reference dashboard:
In fact, you can apply the tags policies manually:
| trackme mode=post url="/services/trackme/v2/splk_tag_policies/write/tag_policies_apply" body="{'tenant_id': '<replace with the tenant id>'}"
Tags policies KVstore collection
When tags policies are applied, TrackMe entities get a record with the list of associated tags, if any:
| inputlookup trackme_dsm_tags_tenant_<replace with tenant id>
When TrackMe trackers are executed, tags records are automatically crossed enriched with the entities records to form the final entities records tags fields:
| inputlookup trackme_dsm_tenant_<replace with tenant id> | table object, *tags*
Tags enrichments for splk-dhm/splk-mhm
TrackMe version 2.0.53
These instructions require TrackMe version 2.0.53 as some enhancements were made for this purpose
For the Data Hosts tracking and Metric Hosts tracking components, tag enrichments rely on a lookup call approach which can be system wide customised.
You can use this feature to map TrackMe entities against any knowledge lookup in Splunk, such as your CMDB or Splunk Enterprise Security Assets and Identities lookups.
This relies on the two following macros:
# splk_dhm_tags
[trackme_get_splk_dhm_tags]
definition = `trackme_tags_default_message`
iseval = 0
# splk_mhm_tags
[trackme_get_splk_mhm_tags]
definition = `trackme_tags_default_message`
iseval = 0
By default, these macros refer to the macro trackme_tags_default_message:
[trackme_tags_default_message]
definition = eval tags = "Tags enrichment is not configured yet, consult the configuration UI TrackMe manage and configure."
iseval = 0
In the user interface, the tags enrichments can be accessed as follows:
Tags tracker logs:
You can review the tags tracker logs if you encounter any issues with tags policies:
index=_internal sourcetype=trackme:custom_commands:trackmesplktags
Configuration for splk-dsm/splk-mhm
The configuration is very simple, let’s assume with the example above that we have a CMDB lookup containing the following fields:
asset: defines the value for the asset (could be hosts, etc)
domain: a domain context
location: location information
tags: The actual tags for that asset, these values can be searched and filtered from the user interface
In a simplistic manner, you would update the macro trackme_get_splk_dhm_tags for splk-dhm and trackme_get_splk_mhm_tags for splk-mhm to:
lookup cmdb_demo asset as alias
Notes: the alias represents the raw name of the entity without any addition from TrackMe.
At the next call, TrackMe performs the call to the lookup accordingly:
You can of course include much more content to the macro, such as restricting the output or manipulating the results if there is no match in your CMDB and so forth.
Filtering on tags from the user interface:
