Remote Splunk deployments

A tenant can monitor a remote Splunk deployment — Enterprise or Cloud — instead of (or as well as) the local search head. One TrackMe instance then becomes a single pane of glass across many Splunk estates. Remote access is configured once as a remote account and referenced by tenants and trackers.

Remote accounts

A remote account stores everything TrackMe needs to reach a remote search head:

  • a name — the identifier tenants and trackers reference;

  • one or more Splunk REST URLs;

  • a bearer token (HTTPS only, stored encrypted in Splunk’s credential store);

  • the remote app namespace (defaults to search);

  • the allowed Splunk roles (account-level RBAC).

The implicit account local represents on-box data.

High availability and resilience

  • Failover — the URL field accepts a comma-separated list of REST endpoints. TrackMe tests them and picks a reachable one at random, so a search-head cluster with no load balancer still gets transparent failover.

  • Retry with backoff — transient connection and 5xx errors are retried with exponential backoff (configurable attempts, initial delay, and multiplier per account).

  • Connectivity test — building or editing an account runs a connectivity and authentication check, and refuses to save on failure.

All remote searches flow through the splunkremotesearch command, which TrackMe uses transparently: it splits an SPL search into a remote part (run on the remote SH) and a local part, so trackers work against remote data without any special handling.

Automatic token rotation

TrackMe rotates each account’s bearer token automatically — by default every 7 days. Once a day, for each account, it creates a new token on the remote, verifies it, stores it locally, and revokes the old one. The remote service account therefore needs the token capabilities (list_tokens_own and edit_tokens_own) in addition to read access to its indexes (including _internal).

Note

The initial token you create by hand is never revoked by TrackMe — disable it yourself once the first rotation has succeeded, per your token policy. A failed rotation is non-destructive: it is logged and the existing token stays in use.

You can also force a rotation on demand — for all accounts or a single one — from the Remote Accounts Overview. And for Splunk Cloud accounts, TrackMe can auto-heal the target URL list as the search-head-cluster members are recreated. Both are covered in Remote deployments — in depth.

Access control

Each account lists the local Splunk roles permitted to use it (rbac_roles) — by both membership and inheritance. A user not in a listed role is refused by splunkremotesearch. This prevents a tenant from being pointed at an unrelated Splunk to exfiltrate data. See Roles & access control.

A tenant selects its remote account at creation time (and trackers and Flex objects can each target a remote account too); the same connectivity check runs automatically.

See also