Backtesting API

A backtest replays a rule version over your history and compares it with the rule’s live version. These are the endpoints the Backtest panel uses on a rule’s page and in the rule builder.

Before you start

Who can call them. Signed-in people only, like the Rules API; an API key gets 401 sign in required. Sign in once with curl and keep the session cookie, then add -b ~/.sp-cookies to every request below.

Runs are background jobs. Starting a run only queues it and returns its ID. A process running the worker role picks it up. Follow it by reading the run until its status is succeeded, failed or cancelled. If no worker runs backtests, a run stays queued.

Endpoints at a glance

Endpoint	Does	Permission	Success
POST /api/v1/backtests	Queue a backtest	rules.backtest	202 {"id": …}
GET /api/v1/backtests	Recent runs, optionally for one rule or version	rules.view	200
GET /api/v1/backtests/{id}	One run, with its progress and results	rules.view	200
POST /api/v1/backtests/{id}/cancel	Stop a queued or running run	rules.backtest	200 {"status": …}
GET /api/v1/backtests/{id}/hits	The sample of hits	rules.backtest or alerts.view	200

By default, Rule authors, Rule approvers and the MLRO hold rules.backtest.

Start a backtest

POST /api/v1/backtests

Field	Required	Meaning
version_id	One of version_id or definition	A saved version in any state: draft, pending approval, approved or active. Required for a scheduled rule, which can’t be tested from a definition
definition	One of version_id or definition	An unsaved definition, in the same shape as a version (expression, features, params, severity, score, action, explanation). This is how the rule builder tests a draft
rule_id	No	The rule. Implied by version_id; with a definition, the rule being edited, whose live version is the comparison
mode, entity_scope	For a definition without rule_id	post_event or realtime; customer, account or counterparty. For a rule that has been saved, the rule’s own values are used
params	No	Parameters to try instead of the version’s. Only those you send change, each must already be a parameter of the version, and the result is checked against the version’s parameter limits, as when you tune it. Any type works, including maps such as A4’s limits
schedule	No	Scheduled rules only: a schedule to simulate instead of the rule’s own, written as for PUT /api/v1/rules/{id}/schedule: hourly, hourly :MM, daily HH:MM or weekly DAY HH:MM, in the workspace’s time zone. Left out, the rule’s own schedule is simulated
from	Yes	Start of the period (included), RFC 3339
to	Yes	End of the period (excluded). At most 180 days after from, and not in the future

The definition is checked like a new version (it must compile, and its action must suit the rule’s mode) before anything is queued. The exact definition replayed, and its SHA-256 definition_hash, are stored with the run.

# Version 4 of a rule, with a higher threshold, over the last 90 days
curl -s -b ~/.sp-cookies -H 'Content-Type: application/json' \
  -d '{
    "version_id": "0192f3c1-bbbb-7ccc-8ddd-0123456789ab",
    "params": {"threshold": 75000},
    "from": "2026-07-11T00:00:00Z",
    "to": "2026-10-09T00:00:00Z"
  }' \
  "$SP/api/v1/backtests"

{"id": "0192f6a0-1111-7222-8333-444455556666"}

Scheduled rules

Send the version_id of a scheduled rule and the run simulates its schedule: every slot of the schedule in the period is run once, in order, reading nothing that occurred after it. The result lists each run (result.runs) and compares the total with what the rule’s live version actually raised in the period (result.baseline), not with a replay of the live version. params works as for any version, and is also checked against the scenario’s hard limits. See Backtesting a scheduled rule for what the simulation assumes.

# Version 3 of a scheduled rule over the last 90 days, as if it ran every hour at 20 past
curl -s -b ~/.sp-cookies -H 'Content-Type: application/json' \
  -d '{
    "version_id": "0192f3c1-cccc-7ddd-8eee-0123456789ab",
    "schedule": "hourly :20",
    "from": "2026-07-13T00:00:00Z",
    "to": "2026-10-11T00:00:00Z"
  }' \
  "$SP/api/v1/backtests"

Follow a run

GET /api/v1/backtests/{id}

Field	Meaning
status	queued, running, succeeded, failed or cancelled
phase	While running: warm_up, replay or saving
schedule	Scheduled rules: the schedule whose runs are simulated
progress	0 to 1
events_replayed	Transactions and account events replayed so far, warm-up included. For a scheduled rule: simulated runs done
error	Why it failed, in plain words. See When a run fails
rule_id, rule_key, rule_name, mode, entity_scope	The rule; mode is post_event, realtime or scheduled. An unsaved new rule has rule_key draft and rule_name Unsaved rule
version_id, version_number, version_state	The version tested, and its state when the run started. Absent for an unsaved definition
params_override	true when params changed the version’s values
definition, definition_hash	Exactly what was replayed, and its SHA-256
baseline_version_id, baseline_definition	The live version it was compared with. Absent when the rule had none, and always for a scheduled rule, which is compared with what it raised
from, to	The period
warmup_from	Where the warm-up started: history replayed before from without counting hits. For a scheduled rule: the start of the first run’s window
data_watermark	The latest time a transaction it read was received. Data received later wasn’t seen
history_source	Where the run read its transactions: postgres (the main database) or clickhouse (the optional analytics store). When a long run read postgres on an installation that has the analytics store, result.notes says why. See Where a backtest reads its history
cancel_requested	true once someone asked to cancel it
created_by, created_at, started_at, heartbeat_at, finished_at	Who ran it (display name) and when
candidate, baseline	The tested and the live version’s figures (below), once finished
result	The full result (below), once succeeded

result holds:

Field	Meaning
candidate, baseline	For a scheduled rule, baseline is what the live rule actually raised in the period. Each: hits; suppressed (hits an approved suppression would have swallowed); alerts (after merging); entities and alerted_entities; amount_base_minor (total value hit, base currency minor units); eval_errors; and for real-time rules outcomes (allow, step_up, review, block counts)
overlap	new (only the tested version), both, removed (only the live version). For a scheduled rule the comparison is per subject and per run, with what was raised
runs	Scheduled rules only: one entry per simulated run, in order (below)
daily	One entry per day of the period in the workspace’s time zone: date, hits, baseline_hits, alerts, baseline_alerts, and for real-time rules changed (decisions whose outcome changes)
transitions	Real-time rules: changed decisions counted by "<live outcome>→<tested outcome>", for example "allow→review": 37
notes	How this run was computed, in plain sentences. See How a backtest differs from live
stats	events, warmup_events, transactions (in the period), evaluated, duration_ms, events_per_second, and the estimated feature memory used. For a scheduled rule: runs, transactions (in the span the runs read) and duration_ms

Each entry of result.runs:

Field	Meaning
at	The schedule slot simulated. The run read nothing that occurred later
data_from	The earliest transaction time it read
candidates	Subjects the scenario matched
cooldown	Matched subjects left out because an earlier simulated run flagged them within the rule’s window
hits, suppressed, alerts	Hits the run would have written, how many an approved suppression would have swallowed, and alerts it would have opened
actual_hits, actual_alerts	Hits the live rule wrote after the previous slot and up to this one, and the alerts those hits opened or joined, each alert counted once
source	postgres or clickhouse: where this run read its transactions
duration_ms	How long the simulated run took

Read the run every few seconds until it finishes:

curl -s -b ~/.sp-cookies "$SP/api/v1/backtests/0192f6a0-1111-7222-8333-444455556666" | jq '{status, phase, progress, error}'

List runs

GET /api/v1/backtests

Query parameter	Meaning
rule_id	Only runs of this rule
version_id	Only runs of this version
limit	1 to 100 (default 20)

Newest first. Each run has the summary fields above (no definition or result), with candidate and baseline once finished.

Cancel a run

POST /api/v1/backtests/{id}/cancel

No body. A queued run is cancelled at once and the answer is {"status": "cancelled"}. A running one answers {"status": "running"} and stops within a few seconds, when it next reports progress; its status then becomes cancelled.

Read the sample of hits

GET /api/v1/backtests/{id}/hits

Query parameter	Meaning
category	new, removed or both; all three when left out
limit	1 to 500 (default 500)

The hits are transactions, so this needs rules.backtest or alerts.view; rules.view alone, as the Administrator has, isn’t enough.

A run keeps up to 200 new hits, 100 removed and 100 both, chosen at random across the period. They come in time order:

Field	Meaning
category	new, removed or both
transaction_id, transaction_at, external_id	The transaction. For a scheduled rule, the newest contributing transaction (the nil UUID when none is recorded)
run_at, transaction_ids	Scheduled rules only: the simulated run the hit belongs to (for a removed hit, the live run that raised it), and up to 50 contributing transactions, newest first
subject_type, subject_id, subject_name	Who the hit is about. subject_name is a pseudonym such as Customer 3f9a1c for people without entities.view
customer_id	The customer, for linking. Left out for people without entities.view
amount_base_minor	The amount, in base currency minor units
severity, score, suppressed	As the hit would have been recorded
features	The feature values the rule saw
explanation	The explanation analysts would have read
outcome, baseline_outcome	Real-time rules: the decision’s outcome with the tested version and with today’s live rules

Errors

Status	Meaning	Examples
401	Not signed in, or called with an API key	sign in required
403	Missing permission	missing permission rules.backtest; sampled hits are transactions: they need rules.backtest or alerts.view
404	No such run, rule or version in your workspace	backtest, rule or version not found
409	Too many runs, or cancelling a finished one	conflict: 5 backtests are already queued or running in this workspace; wait for one to finish or cancel one; conflict: the run has already finished
422	The request doesn’t fit the rules	invalid: send either version_id or definition; invalid: the period must end after it starts; invalid: the period can be at most 180 days; invalid: the period can't end in the future; invalid: the version belongs to another rule; invalid: a definition of a new rule needs mode and entity_scope; invalid: a scheduled rule is backtested from a saved version: send version_id; invalid: the rule has no schedule yet: set one, or send schedule; invalid: no run of the schedule (daily 02:00) falls in the period: choose a longer period; invalid: schedule applies to scheduled rules only; invalid: params.x isn't a parameter of this version; invalid: params.round_to must be one of 100, 1000 (or another parameter limit); rule is not valid (with each problem listed, as for a new version)

Related

• Backtesting
• Scheduled rules
• Rules API
• Worker: backtests