Injecting Expected Sources and Hosts into TrackMe¶

Inject expected data sources or hosts from a Splunk lookup — with a guided wizard, simulation preview, and optional recurring schedule

This guide walks you through the Inject Expected Sources / Hosts wizard, available directly from the Tenant Home actions menu.
Use this feature to manually insert entities into TrackMe that have not yet been discovered, based on a CMDB, asset inventory, or any Splunk lookup table.
When injected, these entities are added as if they were discovered by TrackMe under normal circumstances.
If these entities are not yet sending data to Splunk, they will appear in a red state. As soon as real data arrives and the trackers run, the entities will be updated and their status will reflect the actual health of the data.
The wizard supports both one-shot injection and recurring scheduled imports, so new entries added to the lookup over time are automatically picked up.
Supported for Data Source Monitoring (DSM) and Data Host Monitoring (DHM).
This feature requires TrackMe 2.3.19 or higher.

Injecting expected data hosts (DHM)¶

In this example, we inject 400 expected hosts from a lookup into a DHM tenant. The lookup contains a host field with the hostnames we expect to see in Splunk.

Step 1 — Open the wizard¶

From the Tenant Home for a DHM tenant, click the three-dot menu (top-right) and select Inject expected hosts under the One shot actions section:

Tenant Home three-dot menu showing Inject expected hosts under One shot actions

Step 2 — Select the Splunk deployment¶

The wizard opens. The first step lets you choose the Splunk deployment where the lookup resides. Select local for the local instance, or pick a configured remote account if the lookup lives on a different Splunk deployment:

Inject expected hosts wizard — Step 1 Splunk deployment, local selected

Step 3 — Select the lookup and preview its content¶

Select the lookup transform from the dropdown. The wizard loads the lookup fields and displays a preview table showing sample rows, so you can verify the content before proceeding:

Inject expected hosts wizard — Step 2 Select lookup, preview of example_expected_hosts with host and cluster_name columns

Step 4 — Map lookup fields to entity fields¶

Map the lookup fields to the required entity fields. For DHM, the host field is required. The wizard auto-detects matching field names — if your lookup has a column called host, it will be pre-selected automatically:

Inject expected hosts wizard — Step 3 Field mapping, host mapped to host

Step 5 — Simulate the injection¶

Before executing, run a simulation to preview exactly what will happen. The simulation checks each row against the existing KV store and reports:

Lookup rows — total rows in the lookup
New entities — entities that will be created (not yet in TrackMe)
Already existing — entities that already exist and will be skipped

A collapsible table shows the full list of new entities to be created, with their object name and host value:

Inject expected hosts wizard — Step 4 Simulate, 400 new entities, 0 existing

Step 6 — Configure options and execute¶

The final step lets you optionally enable a recurring import before executing. When toggled on, a saved search is created to re-execute the injection on the configured schedule. You can customize both the report name and the cron schedule:

Inject expected hosts wizard — Step 5 Options & Execute, recurring import enabled with report name and cron schedule

Click Inject expected hosts to execute. The wizard reports the results — entities created, entities skipped, and whether the recurring schedule was created:

Inject expected hosts wizard — Injection completed successfully, 400 entities created, recurring schedule created

Result — Entities visible in Tenant Home¶

After closing the wizard and refreshing, the 400 injected hosts are now visible in the Tenant Home. Entities that are not yet actively sending data to Splunk appear in a red state, which is expected:

Tenant Home after injection — 400 enabled data hosts, 168 in alert state

Hint

For DHM entities, the index and sourcetype fields will initially appear empty for hosts that have not yet sent data to Splunk. This is normal — once real data arrives and the DHM tracker runs, these fields are automatically populated and the empty placeholders are cleaned up.

Clicking on an entity that has not yet sent data shows the indexes/sourcetypes summary with empty placeholder values. Once the host starts sending data, these values are automatically replaced with the real indexes and sourcetypes:

Entity indexes/sourcetypes summary showing empty placeholder for a host awaiting data

As soon as trackers are executed and the hosts start sending data to Splunk, the entities will transition to a green state and their metrics will be fully populated.

Injecting expected data sources (DSM)¶

The process for data sources is very similar, with the key difference that DSM requires both an index and a sourcetype field from the lookup.

Open the wizard (DSM)¶

From the Tenant Home for a DSM tenant, click the three-dot menu and select Inject expected sources under the One shot actions section:

Tenant Home three-dot menu showing Inject expected sources under One shot actions

Select the Splunk deployment (DSM)¶

Choose the Splunk deployment where the lookup resides:

Inject expected sources wizard — Step 1 Splunk deployment, local selected

Select the lookup and preview (DSM)¶

Select the lookup transform. The preview table shows the index and sourcetype columns from the lookup:

Inject expected sources wizard — Step 2 Select lookup, preview showing index and sourcetype columns

Map lookup fields (DSM)¶

For DSM, both index and sourcetype mappings are required. The wizard auto-detects matching field names:

Inject expected sources wizard — Step 3 Field mapping, index and sourcetype mapped

Simulate the injection (DSM)¶

Run the simulation to preview the results. In this example, 12 new data source entities will be created:

Inject expected sources wizard — Step 4 Simulate, 12 new entities to create

Configure options and execute (DSM)¶

Enable the recurring import if desired, customize the report name and cron schedule, then click Inject expected sources:

Inject expected sources wizard — Step 5 Options & Execute with recurring import enabled

The wizard confirms the injection was successful and the recurring schedule was created:

Inject expected sources wizard — Injection completed successfully, 12 entities created, schedule created

Entities visible in Tenant Home (DSM)¶

The 12 injected data sources are now visible in the Tenant Home. Entities not yet covered by the tracker scope appear in red:

Tenant Home after injection — 12 data sources, all in alert state

As soon as the trackers are executed and the data sources become active within their scope, the entities transition to a green state:

Tenant Home after tracker execution — 12 data sources, all green

Recurring imports¶

When you enable Set as recurring import in the wizard, a Splunk saved search is created with the specified name and cron schedule. This saved search:

Runs on behalf of the tenant owner (automatically re-assigned)
Reads the lookup on each execution and injects any new entities not already in TrackMe
Skips entities that already exist — no duplicates, no overwrites
For recurring runs where nothing new is found, no audit event is recorded and no searches are executed (the pre-filter detects that all entities already exist and short-circuits)

You can also create a recurring schedule after the initial injection by toggling the recurring import switch and clicking Create schedule in the footer.

The recurring schedule can be managed from Splunk’s Settings > Searches, Reports and Alerts page. Look for reports with the naming convention TrackMe - Inject Expected {Sources|Hosts} - {tenant_id} - {lookup_name}:

Splunk Edit Report dialog showing the recurring injection saved search with title, description, search command, and schedule

How it works under the hood¶

The inject wizard leverages the trackmepushdatasource streaming command, which is the same mechanism used by the TrackMe push API. This ensures injected entities are created with all required fields properly initialized — delay/latency statistics, timestamps, and other metadata that the decision maker expects.

For DSM, each lookup row with an index and sourcetype produces a data source entity with the format {index}:{sourcetype}.

For DHM, each lookup row with a host value produces a host entity with the format key:host|{hostname}. The raw hostname is stored as the entity’s alias for display purposes.

Key behaviors

Existing entities are never overwritten. The wizard checks against the KV store before injection and skips any entity that already exists.
Empty placeholder cleanup for DHM. Injected hosts start with empty index/sourcetype values. Once real data is detected by the DHM tracker, these empty placeholders are automatically cleaned up — no manual intervention required.
Batching for large lookups. Lookups with more than 500 entities are automatically processed in batches to stay within Splunk’s SPL query length limits.
Remote deployment support. The lookup can reside on a different Splunk deployment. Select the configured remote account in the first step of the wizard.

Troubleshooting¶

If you encounter issues with the injection, check the following logs:

Inject expected REST handler logs:

index=_internal sourcetype=trackme:rest_api source="*trackme_rest_api_splk_inject_expected*"

trackmepushdatasource command logs:

index=_internal sourcetype=trackme:custom_commands:trackmepushdatasource

Advanced usage with the trackmepushdatasource command¶

The inject wizard uses the trackmepushdatasource streaming command under the hood. For advanced use cases, you can call this command directly in SPL.

Command arguments¶

Argument	Required	Default	Description
tenant_id	Yes	None	The tenant identifier
component	Yes	None	The component to use (`dsm` or `dhm`)
search_type	Yes	None	The type of search to perform (`tstats` or `raw`)
show_search_query	No	False	If true, includes the search query in the summary output
show_search_results	No	False	If true, includes the search results in the summary output
pretend_latest	No	-24h	Relative time value in Splunk format for data_last_time_seen

Example — Pushing expected data sources via SPL¶

| inputlookup example_expected_data_sources.csv
| trackmepushdatasource search_type=tstats tenant_id=demo show_search_query=True show_search_results=True pretend_latest="-24h" component="dsm"

Example — Pushing expected hosts via SPL¶

| inputlookup example_expected_hosts.csv
| trackmepushdatasource search_type=tstats tenant_id=demo show_search_query=True show_search_results=True pretend_latest="-24h" component="dhm"

Controlling the break-by logic for DSM¶

For expected data sources, if the TrackMe tracker logic includes a specific break-by logic, you can submit the value for the object accordingly:

| inputlookup ds_expected.csv
| fields index, sourcetype, cribl_env
| eval object = index . ":" . sourcetype . ":" . "|key:cribl_env|" . cribl_env
| trackmepushdatasource search_type=tstats tenant_id=secops show_search_query=True show_search_results=True pretend_latest="-24h" component="dsm"

Controlling the host metadata key for DHM¶

For expected hosts, you can control the metadata key for the host:

| inputlookup dh_expected.csv
| fields host
| eval host = "key:forwarder|" . host
| trackmepushdatasource search_type=tstats tenant_id=endpoints show_search_query=True show_search_results=True pretend_latest="-24h" component="dhm"