Upgrading TrackMe¶
Upgrading TrackMe is a standard Splunk application upgrade followed by an automated, per-tenant schema migration that TrackMe runs on its own. The package replaces the shipped code; TrackMe then reshapes each Virtual Tenant’s knowledge objects and KVstore collections to the new release, tenant by tenant, and reports its progress in the user interface.
This section is the upgrade guide. It is written to be executed, not only read: the procedure page is a step-by-step runbook with the screen to look at, the search to run and the expected result at every step, and the other pages hold the reference material behind it.
The step-by-step runbook: before the upgrade, upgrade day, after the upgrade, with checkpoints.
How the automated per-tenant migration works, how to follow it, what runs between releases.
The built-in validation screens, the extended checklist, the behaviour changes to brief operators on, the evidence searches.
Why a rollback is a restore and never a re-install of the older package, and how to do it.
The upgrade procedure at a glance. Step 0 is done once per estate; steps 1 to 3 run on every TrackMe deployment.
What a version change actually touches¶
Every question about what survives an upgrade reduces to which of four layers the state lives in.
Application code and defaults (
etc/apps/trackme:default/,bin/,lib/) are replaced by the package. This is the only layer an upgrade replaces.Local configuration (
local/) holds every Virtual Tenant definition, every per-tenant tracker and saved search, every per-tenant KVstore transform, the settings overrides and the account credentials. The package replacement leaves it entirely untouched. The schema migration that follows may then refresh the TrackMe-managed objects it contains (per-tenant transforms, lookup definitions, saved searches) to the new release’s shape; user-owned configuration (tenants, settings overrides, credentials, your own knowledge objects) is retained.KVstore data (
kv_trackme_*collections: entities, rules, fitted ML models, policies, state) is untouched by the package. Only the forward-only schema migration reshapes it: it creates collections and fields, rewrites values (for example the Outliers model engine) and resets state. This is what the pre-upgrade backups exist to undo.Indexes (
trackme_summary,trackme_metrics,trackme_audit,trackme_notable) are never touched.
Danger
Remove and reinstall is not an upgrade path.
Deleting the application directory destroys local/: every tenant, every tracker, every per-tenant KVstore
transform and every credential. The KVstore data survives, but becomes unreachable from SPL until the
transforms are rebuilt. Always upgrade in place. If a reinstall is ever unavoidable, restore a TrackMe
backup immediately afterwards (see Backup & restore).
Principles¶
Upgrade in place, from a backup. Take a TrackMe backup immediately before the change window (and, on Splunk Enterprise, a copy of the application directory plus a KVstore backup). TrackMe also takes an automatic safety backup before the first migration runs, but the explicit one is the one you will want to restore from.
Follow the migration from the screen behind the version number. On the Virtual Tenants page the version number in the header is a link: it opens the Tenants Update statuses screen, which shows every tenant’s schema version against the required one, its updated / pending status, and a one-click drilldown to the migration logs. It is the authoritative progress view; do not judge anything before it reads fully updated.
One release for an estate. When several TrackMe deployments are upgraded in a programme (managed service providers, multi-instance customers), validate one release on a representative test deployment and deploy that same release everywhere. Move the whole estate to a newer release later, in one deliberate wave.
Land on a current release. We recommend running the latest release at all times (see Compatibility). Never land on the 2.3.22 to 2.4.1 builds: they carry a regression where ML outliers model training silently stops for every component (fixed in 2.4.2).
A rollback is a restore, not an un-install. Schema migrations are forward-only. Reinstalling an older package reverts the code and none of the migrated data; the reliable rollback is to restore the pre-upgrade backups. See Rollback.
Validate the data, not the version marker. After a downgrade, the schema marker is silently realigned to the running version, so the update-status screen reports updated even though the data kept its migrated shape. Post-downgrade validation must inspect actual behaviour and actual records.
Licensing after an upgrade
Deployments that ran the discontinued Free Community Edition start a 90-day Foundation trial automatically after upgrading to 2.4.x; nothing stops, but the trial must be converted by registering a licence key. Licensed deployments keep their licence. Registering a key is step 2.4 of the Upgrade procedure; see also License registration.
Where to go next¶
Read the Upgrade procedure end to end once, then execute it with its checkpoints.
Keep Schema migrations open on upgrade day: it explains every line of the migration log.
Use Validating after an upgrade for formal validation records and to brief operators on the behaviour changes between releases.
Read Rollback before the change window, not after.