Using model scores in rules

In short

A risk model’s score does nothing until a rule reads it. Rules read it through the model variable:

model.available && model.percentile >= 99 && txn.amount >= params.min_amount

That reads: there is a live model, this transaction is among the most unusual 1% of what the model learned from, and it is not small.

• Only the live model feeds rules. A shadow model’s scores never reach a rule.
• A rule that reads the score is a rule like any other. It has versions, needs a second person’s approval, and can run in shadow first.
• Real-time and post-event rules can read model. Scheduled rules can’t.

The fields

Field	Type	Meaning	Without a live model
model.available	bool	A live model scored this transaction	false
model.score	number	The anomaly score, 0 to 1. About 0.5 is ordinary; higher is more unusual	0.0
model.percentile	number	Where the score falls among the scores of the model’s training data, 0 to 100. 99 and above is the most unusual 1%	0.0
model.reasons	list of text	The keys of the features that pushed the score up most, strongest first; up to three	Empty
model.top_reason	text	The strongest of those, in plain words, for explanations	Empty

Two more fields are there for rules written by hand: model.version (the live version’s number, 0 without one) and model.reason_labels (the reasons in plain words, in the same order as model.reasons).

In the rule builder, the five fields in the table appear in the field list as Risk model score, Risk model score percentile, Has a live risk model, Risk model reasons and Risk model top reason.

Always guard with model.available

Without a live model every field has a neutral value, so a rule that reads model.score never fails. It just doesn’t fire on the score.

That is safe for conditions that ask for a high score: model.score >= 0.7 is simply false when the score is 0. It is not safe the other way round:

// Wrong: with no live model the score is 0, so this fires on every transaction.
model.score < 0.4 && txn.amount >= params.min_amount

// Right: say what you mean when there is no model.
model.available && model.score < 0.4 && txn.amount >= params.min_amount

Make it a habit to start with model.available &&. The same guard covers the moments a model is missing: before the first version goes live, and after the live one is retired.

Prefer the percentile

model.score’s scale shifts a little between trained versions. model.percentile means the same thing for every version: how this transaction ranks against the model’s own training data.

You want	Write
The most unusual 1%	model.percentile >= 99
The most unusual 5%	model.percentile >= 95.0
The most unusual 1 in 1,000	model.percentile >= 99.9

A rule written on the percentile keeps its meaning when a new version is promoted. A rule written on a raw score may fire more or less often overnight.

Whole numbers and decimals both work here

model.score and model.percentile are decimals. You can compare them with a whole number or a decimal: model.percentile >= 99, as the Models page writes it, and model.percentile >= 99.0 mean the same. To compare with a parameter, convert it with double(params.min_percentile), as the shipped packs do for any decimal field. See Type rules that trip people up.

Combine it; don’t use it alone

An anomaly score on its own flags every bonus and every house purchase. Use it to narrow a condition you already believe in, or to catch what your other rules don’t describe.

Narrow a broad rule. Large transfers out are common. Large transfers out that are also unusual for this customer are worth a look:

model.available
  && model.percentile >= 97.0
  && txn.direction == "out"
  && txn.amount >= params.min_amount

Ask why it is unusual. model.reasons holds feature keys, so a rule can react only when a particular feature drove the score:

model.available
  && model.percentile >= 99
  && "amount_vs_avg_30d" in model.reasons

The standard feature keys are:

Key	Feature
txn_amount	Amount of this transaction
txn_hour	Hour of the day
customer_age_days	Days since the customer was onboarded
customer_count_24h, customer_sum_24h	The customer’s transactions and their total value, last 24 hours
customer_count_7d	The customer’s transactions, last 7 days
customer_sum_out_7d, customer_sum_in_7d	The customer’s money out and money in, last 7 days
customer_avg_30d, customer_max_30d	The customer’s average and largest transaction, last 30 days
customer_counterparties_30d	Different counterparties, last 30 days
account_count_24h, account_sum_30d	The account’s transactions in 24 hours, and total value in 30 days
counterparty_count_30d	The counterparty’s transactions, last 30 days
amount_vs_avg_30d	The amount compared with the customer’s 30-day average
day_vs_week	The customer’s last 24 hours compared with their last 7 days

A model trained with your rules’ aggregates has more keys. The version’s model card lists every one.

Keep new customers out, or in, on purpose. A customer with little history has small aggregates, which the model can read as unusual. Decide which you want:

model.available && model.percentile >= 99 && customer.age_days >= 30

Explanations

The explanation an analyst reads can quote the score and its strongest reason:

Unusual for this customer (more unusual than {{model.percentile}}% of recent transactions): {{model.top_reason}}. Amount {{txn.base_currency}} {{txn.amount}}.

{{model.score}}, {{model.percentile}} and {{model.top_reason}} are filled in like any other placeholder. Whatever the explanation says, the alert and the decision also show the score with all its reasons: see Reviewing an alert.

Write explanations that don’t overclaim

Unusual for this customer is true. Suspected fraud is not something the model can know. The explanation is read by analysts and ends up quoted in cases; keep it to what the score means.

Which action to give the rule

Start gently, and tighten only with evidence.

Stage	Mode and action	Why
First	Post-event, score_only or tag	See how often it would fire, with nobody’s queue affected
Then	Post-event, alert	Analysts review what it finds; their dismissals tell you whether it earns its place
Only with care	Real-time, review or step_up	A held payment costs a customer something. Combine the score with a strong second condition

A real-time block on an anomaly score alone is almost never right: the model can’t know that the unusual payment is the customer’s house deposit.

Testing a rule that reads the score

• The builder’s test has no model: model.available is false and the score is 0. It checks that the rule is valid, not how it behaves with a score.
• A backtest scores each past transaction with today’s live model, and says so in its notes. The model that was live at the time may have been another version, or none. With no live model, the notes say that model.score reads 0.
• Shadow mode is the truest test: the shadow rule version reads the live model’s real scores on live traffic, and creates no alerts.

So the order is: get the model live, backtest the rule, run the rule in shadow, then make it live.

When the score isn’t there

Situation	What rules read
No live model (none yet, or the live one was retired)	model.available is false; score and percentile 0
A shadow model only	The same: shadow scores never reach rules
A real-time decision that couldn’t finish in time (degraded)	Rules weren’t evaluated, so nothing is read; the failure policy answers
A scheduled rule	Scheduled rules don’t have the model variable
A new version was just promoted	Rules read the new version within about 15 seconds

Related

• Risk models, the concept
• Risk models: train, shadow, go live
• CEL reference
• Writing a custom rule
• Explanations