Idempotency and retries

In short

Networks fail. A request can time out after SpeculaPro has stored your transactions but before you got the answer. The only safe reaction is to send the same records again, and SpeculaPro is built so that this never counts anything twice.

The key is external_id: your unique reference for each transaction. SpeculaPro remembers every one it has stored. Sending it again returns duplicate instead of creating a second transaction.

For project leads

“Idempotent” means “doing it twice has the same effect as doing it once”. It matters because double-counted transactions would trigger false alerts, such as a customer appearing to deposit twice as much, and missed ones would hide real activity.

The rules

You send a transaction whose external_id…	Result
SpeculaPro has never seen	accepted
SpeculaPro already stored, with identical content	duplicate, with the original id. Nothing changes
SpeculaPro already stored, with any difference	rejected: external_id was already used for a different transaction
Appears twice in the same batch, identical	The first is processed, the second is duplicate
Appears twice in the same batch, different	The second is rejected: external_id repeated in this batch with different content

“Identical” means every field, including optional ones, status and attributes. A transaction sent once with "status": "completed" and once with no status counts as identical, because completed is the default.

Times are compared as moments, not as text: occurred_at is converted to UTC before the comparison, so 2026-10-09T08:14:03Z and 2026-10-09T09:14:03+01:00 are identical.

Choosing external_id

A good external_id:

• is generated once by your system when the transaction happens, and stored with it;
• never changes, however many times the transaction is exported or sent;
• is unique across your whole institution for that workspace, not just per day, branch or channel. Prefixing by source helps: MOMO-…, CARD-…, CBS-…;
• has no leading or trailing spaces.

Never build external_id from something that can change

A file row number, an export timestamp, or a sequence that restarts each day produces a new external_id for a transaction you’ve already sent. SpeculaPro then has no way of knowing it’s the same transaction, and counts it twice.

How SpeculaPro makes resending safe

When a batch arrives, each transaction’s external_id is first claimed in SpeculaPro’s database, and only then put on the queue. The claim is marked complete once the queue confirms it holds the transaction.

If the connection drops in between, the claim stays incomplete. When you resend, SpeculaPro sees the incomplete claim and puts the transaction on the queue again with the same internal id as the first attempt, so the database can never end up with two copies. You get accepted.

You don’t need to know any of this to integrate. It is why the instruction is simply: when in doubt, send it again.

Transactions that went through a real-time decision

A transaction your system sent to the real-time decision API is recorded from there, under the same external_id. You don’t need to send it through the ingest API as well. If you do, an identical copy comes back duplicate, and one with different content is rejected with external_id was already used for a transaction sent to the decision API (or, once the decision API’s short-term registry has forgotten it, external_id was already used for a different transaction), so it is never counted twice.

While the decision API is still deciding that external_id, which usually takes a fraction of a second, the ingest API refuses it with the same external_id was already used for a transaction sent to the decision API, even an identical copy. Resend it later and it comes back duplicate. A transaction that was blocked is stored with status blocked, whatever status you send.

Account events

Account events are protected differently: an event sent again is stored once and comes back accepted, but rule counts only ignore the repeat for 30 minutes. Resend events promptly, and only the ones that need it.

The 90-day window

SpeculaPro remembers external_ids for 90 days by default (the operator’s SP_WORKER_INGEST_KEY_RETENTION). After that, a regular clean-up (every 6 hours by default) forgets them, to keep the registry fast.

Resending a transaction older than 90 days creates a second copy

After the window, an old external_id is treated as new. If you re-send last quarter’s file “just to be sure”, every transaction in it that SpeculaPro first received more than 90 days ago is accepted again and counted twice.

• Never replay old data without checking with your SpeculaPro operator first.
• Keep your own record of what was accepted, so you only resend what failed.
• If you must back-fill history, agree it with the operator; they can extend the window for the duration.

The window is counted from when SpeculaPro received the transaction, not from occurred_at.

Customers and accounts

Customers and accounts don’t use the same registry. They are updated in place, and the version field decides what wins:

• resending the same record with the same version returns stale and changes nothing;
• a higher version is applied;
• a lower one is ignored (stale);
• a record without a version is always applied, so resending it simply writes the same data again.

So resending customers and accounts is always safe too. A version adds protection against an older record arriving after a newer one, which matters whenever records can be retried or sent from more than one place. See Versions.

FX rates are simply replaced for the same currency and date, so resending them is safe.

A sender that never loses or duplicates

The logic below works for any language. It sends batches, resends only what needs resending, and backs off when SpeculaPro is unavailable.

for each batch of up to 500 records:
    attempt = 0
    pending = batch
    while pending is not empty:
        response = POST pending

        if response is a network error, a timeout, 500 or 503:
            wait backoff(attempt); attempt += 1
            continue                         # resend the same records, unchanged

        if response.status is 401 or 403:
            stop and alert                   # key problem; retrying won't help

        if response.status is 413:
            split pending in two and send each half
            break

        if response.status is 422:
            log the listed errors with their item positions
            move the bad records to a "needs fixing" queue
            pending = pending without the bad records
            continue

        # 200: look at each record
        next = []
        for each result in response.results:
            record = pending[result.index]
            if result.status in (accepted, duplicate, stale):
                mark record as delivered
            else if result.status is rejected:
                send record and result.errors to the "needs fixing" queue
            else if result.status is retry:
                next.append(record)
        if next is not empty:
            wait backoff(attempt); attempt += 1
        pending = next

backoff(n) = min(60 s, 1 s × 2^n) plus a random 0–1 s

Points that matter:

• Resend records unchanged. A changed amount, account, counterparty or any other field makes it a different transaction under the same external_id, and it gets rejected. How the time is written doesn’t matter: 2026-10-09T08:14:03Z, 2026-10-09T08:14:03+00:00 and 2026-10-09T09:14:03+01:00 are the same moment, and SpeculaPro converts times to UTC before comparing, so all three are a duplicate of each other.
• Keep a “needs fixing” queue and give it an owner. Rejected records are real activity that isn’t being monitored until someone fixes the data.
• Cap the backoff, don’t give up. If SpeculaPro is down for an hour, keep the records and resend when it’s back. They are not lost while your sender holds them.

Related

• Responses and statuses
• Transactions
• Troubleshooting