TrackMe Tags enrichment

Introduction to tags enrichments

Hint

About Tags in TrackMe

  • Tags are used to provide additional context to the entities

  • Tags can be used for filtering purposes in the user interface

  • Tags can be used for alerting purposes

  • Tags can be used for additional context in the user interface

tags_overview.png

Implementing tags enrichments in TrackMe (all components)

Extension of Tags features for all components in TrackMe 2.0.98

  • Since TrackMe version 2.0.98, tags enrichments are available for all components in TrackMe with the exact same level of features, and user experience.

Tags enrichments in TrackMe can be defined by:

  • Tags policies: Regular expressions policies which define a number of tags for matching entities, once defined, these policies are applied automatically to the entities.

  • Manual Tags: In Addition, you can manually add tags to the entities from the user interface.

Accessing, defining and managing tags policies

You can access tags policies from the main TrackMe’s tenant screen via the button Manage: tags policies:

policies_editor1.png

About Tags policies:

  • Regex based: Tags polciies are regular expressions which define a number of tags for matching entities

  • Tags submission: These polcies are monitored are monitored by a scheduled job named trackme_<component_suffix>_tags_tracker_tenant_<tenant_id>

  • Multiple matching policies: Multiple policies can be apply, this means a given entity can receive tags from one or more tags policies

  • Lower case: Tags are automatically stored in lower case format

  • Removing tags policies: When tags policies are removed, the tags were associated with entities and from the removed policies are removed from the entities automatically

  • Tags in the KVstore records: Tags are stored in the entities KVstore records, and can be used for alerting filtering purposes

  • Tags in the user interface: Tags are displayed in the user interface for the entities

Tags policies usage example

In the following example, we have a number of entities which have in comming some patterns we can leverage for tags policies purposes:

  • geo-locations: the entities naming convention includes the source location of these feeds

  • technology information: we can also leverage some of these information to associate these entities with technology related context tags

Example of tags policies definition:

In this example, we are going to rely on the index naming convention to define tags, so we can automatically classify TrackMe entities.

In short, we are going to apply the simple following logic:

geographical location:

regular expression

tags

.*_na_.*

north_america

.*_eu_.*

europe

.*_uk_.*

united_kingdom

technology information:

regular expression

tags

.*_syslog:.*

network

.*_linux:.*

os

Once configured in TrackMe’s UI:

tags_policies_overview.png

Tags can be used for filtering purposes in the user interface:

tags_filter_ui.png

Tags can also be accessed per entity, and modified from here too:

tags_review1.png tags_review2.png

Tags are parts of TrackMe notable events, so you can use these for filtering, enrichment, decision making and so forth:

index=trackme_notable tenant_id=<replace with tenant_id>
dsm_tag_policies18.png

Tags are included in the trackme:state events:

index=trackme_summary sourcetype="trackme:state" tenant_id=<replace with tenant_id>
tags_trackmestate.png

Tags can be accessed through the TrackMe get collection command, which renders entities from Decision Maker (same as the UI view):

| trackmegetcoll tenant_id=<tenant_id> component=<component_suffix>
command_get_records.png command_get_records2.png

Troubleshoot tags policies

Tags policies tracker

The first component is the scheduled job which applies the tags policies to the entities, this job is called as follows:

  • trackme_<component_suffix>_tags_tracker_tenant_<tenant_id>

troubleshoot1.png

This jobs should be scheduled, and properly executed:

troubleshoot2.png

You can find its logs as follows:

index=_internal sourcetype=trackme:custom_commands:trackmesplktags

When you have multiple tenants, you may want to filter on the tenant identifier:

index=_internal sourcetype=trackme:custom_commands:trackmesplktags tenant_id=<replace with tenant_id>

In case of any issues with the tags policies, ensure that the job exists, is scheduled and executes properly and continue with the following steps.

Tags policies REST API endpoint

The scheduled job calls the REST API endpoint to apply the tags policies, you can review the logs for this endpoint as follows:

index=_internal sourcetype=trackme:rest_api post_tag_policies_apply

With multiple tenants, filter on the tenant identifier:

index=_internal sourcetype=trackme:rest_api post_tag_policies_apply tenant_id=<replace with tenant_id>
troubleshoot3.png

In case of issues, carefully review the logs and verify if there are exceptions or issues with the tags policies:

index=_internal sourcetype=trackme:rest_api post_tag_policies_apply log_level=error

The REST API endpoint can be applied manually as well, you can easily get the list of endpoints and their usage, review the REST API reference dashboard:

troubleshoot4.png

In fact, you can apply the tags policies manually:

| trackme mode=post url="/services/trackme/v2/splk_tag_policies/write/tag_policies_apply" body="{'tenant_id': '<replace with the tenant id>'}"
troubleshoot5.png

Tags policies KVstore collection

When tags policies are applied, TrackMe entities get a record with the list of associated tags, if any:

| inputlookup trackme_dsm_tags_tenant_<replace with tenant id>
troubleshoot6.png

When TrackMe trackers are executed, tags records are automatically crossed enriched with the entities records to form the final entities records tags fields:

| inputlookup trackme_dsm_tenant_<replace with tenant id> | table object, *tags*
troubleshoot7.png