Maintenance mode

Maintenance mode suppresses alerting during planned (or unplanned) work, and excludes that time from SLA accounting. TrackMe offers it at three scopes, plus a persistent record that drives the SLA exclusion.

The three mechanisms

Mechanism

What it does

Global / tenant maintenance mode

A runtime on/off switch that silences alerting for the whole instance or one tenant. It suppresses stateful-alert transitions and auto-ack actions — but the decision maker keeps running, so you still see red state in the UI. It is alert silencing, not sleep.

Per-entity maintenance

Forces one entity (or a bulk selection) into the blue, non-alerting state for an explicit window. It has top precedence over computed state and every other protection layer, and auto-returns to normal when the window expires.

Maintenance Knowledge Database (KDB)

A persistent log of windows — planned or unplanned, past or future — by scope. SLA accounting reads the KDB (it needs historical intervals), not the live switch.

Enabling then disabling maintenance mode automatically writes a planned KDB record, so your SLA stays honest with no extra steps.

How it interacts with SLA

SLA exclusion reads the KDB. Only active and expired windows are excluded — not future scheduled ones — and, by default, only planned windows (a global setting changes this). A background tracker flips windows scheduled → active → expired every few minutes and writes the audit trail.

Note

  • Always set a timeout. Global maintenance left on without an end time stays on indefinitely.

  • Re-enabling does not retroactively fire alerts missed during the window; an open stateful record may need manual cleanup.

  • “SLA still counting downtime” almost always means the window’s scope did not match the entity.

  • Managing maintenance requires admin capability; reading the KDB is available to all TrackMe users.

Under the hood, maintenance status is driven by the trackme_maintenance_mode_tracker tracker, and the global setting maintenance_kdb_exclusion_behaviour controls whether only planned maintenance is excluded from SLA. Global maintenance can be toggled from the REST API — /services/trackme/v2/maintenance/global_maintenance_enable and …/global_maintenance_disable (both mode=post), accepting maintenance_duration, update_comment, and an optional tenants_scope to limit the window to specific tenants:

curl -u user https://mysplunk:8089/services/trackme/v2/maintenance/global_maintenance_enable \
    -X POST -d "{\"maintenance_duration\": \"3600\", \"tenants_scope\": \"secops\",
                 \"update_comment\": \"1-hour maintenance for the secops tenant\"}"

See also