Offboarding a workspace

In short

Regulatory reports are retention records: nothing in the app or the API deletes them, and the database refuses to delete a workspace that holds any. When an institution leaves, the operator runs speculapro tenant offboard. It:

1. exports every report of the workspace and every goAML file it filed, with a manifest of SHA-256 checksums, to a folder you hand to the institution;
2. records the offboarding in the audit log, with the operator’s name and the manifest’s checksum;
3. deletes the workspace and everything in it, in one transaction.

Only this command can delete a workspace’s reports, and only with the schema owner’s database connection.

Deleting a workspace can’t be undone

Everything in the workspace goes: customers, transactions, alerts, cases, rules, people and settings. Take a backup first, and agree the date and what the institution receives with its MLRO in writing.

Before you start

• The schema owner’s connection. Run the command with the same SP_DB_URL you use for speculapro migrate: the role that owns the schema. The application roles can’t delete reports, even inside the command.
• The encryption keys. The filed goAML files are stored encrypted with the workspace’s key, so the command needs the same key settings as the running installation. See Encryption keys.
• A safe, empty folder. The export holds customers’ identity in full. Use an encrypted volume that only you can read. The folder must not exist yet, or be empty.

How to offboard a workspace

1. Export first, without deleting, and check what you get:

   speculapro tenant offboard --slug kweku-pay --out /secure/kweku-pay-reports \
     --operator "Jane Doe" --export-only

   It prints how many reports and filed exports it wrote, and the manifest’s SHA-256. An --export-only run is recorded in the audit log too, with deleted: false.

2. Hand the folder to the institution, by a secure channel, and have its MLRO confirm they received it and the manifest’s checksum matches.

3. Delete, confirming the slug a second time:

   speculapro tenant offboard --slug kweku-pay --out /secure/kweku-pay-final \
     --operator "Jane Doe" --confirm kweku-pay

   This exports again (to a new, empty folder), writes the audit entry, then deletes the workspace. It ends with the workspace and everything in it were deleted; the audit log keeps the workspace.offboard entry, followed by one line about the analytics store.

4. Remove the export from your own systems once the institution has it, according to your data protection policy.

In a cell of a multi-region deployment, deleting the workspace also releases its slug in the directory, so the name can be used again in any region. If the directory couldn’t be reached, the command says the directory still lists … : release it with speculapro cell release-slug <slug>; run that once the directory is back. To move a workspace to another region without ending the relationship, see Moving a workspace between regions.

The analytics store

Only if your installation has the optional analytics store (ClickHouse). The store holds a copy of the workspace’s transactions, decisions, rule hits, alerts and cases. Offboarding deletes that copy too. There is nothing to do before step 3, and no order to get right.

What happens when step 3 runs:

1. The deletion of the copy is recorded in PostgreSQL, in the same database transaction that deletes the workspace. Either both happen or neither does.

2. The copy is deleted, by whichever of these applies:

   The tenant offboard command	What it prints	When the copy is deleted
   Has SP_CLICKHOUSE_URL set and can reach ClickHouse	the workspace’s copy in the analytics store was deleted too	Immediately
   Has no SP_CLICKHOUSE_URL, or can’t reach ClickHouse just then	the analytics store’s copy is deleted by the analytics role (the deletion is recorded; `speculapro analytics status` shows it until it is applied)	By the analytics role, on its next pass (every 30 seconds by default). If ClickHouse is down, the role retries until it has

3. It stays deleted. The analytics writer checks the recorded deletions before every insert, so a replay of Kafka (a rebuild from Kafka, or a consumer that was behind) never writes the workspace back.

To check, run:

speculapro analytics status

The line deletions: none waiting means the store has caught up with every recorded deletion, and the offboarded workspace no longer appears. While a deletion is still waiting, that line counts it, and the workspace is listed under not in Postgres any more (offboarded), still in the store as deletion recorded, being applied. See How deletions reach the store.

A copy left behind by an earlier release

A workspace offboarded before SpeculaPro did this has no recorded deletion. speculapro analytics status lists its copy by ID as orphaned: analytics purge --tenant-id. Delete it with:

speculapro analytics purge --tenant-id <id> --confirm <id>

See How to delete one workspace’s copy.

ClickHouse backups you keep hold the workspace’s data until they expire.

What the export contains

File	Contents
register.json	Every report of the workspace, whatever its status: number, type, status, narrative, approvals, the FIC’s reference and answer, voids, and the transactions each report covered
exports/<report number>-<file name>	Each goAML file as it was filed: the exact bytes, decrypted
manifest.json	The workspace’s slug, name and ID, when it was exported, the operator, and every file’s path, SHA-256 and size

Before writing a filed goAML file, the command checks its SHA-256 against the one recorded when it was exported, and stops if they differ: what the institution receives is exactly what it filed.

What is recorded

The audit entry workspace.offboard (actor: the system, user agent speculapro-cli) records the workspace’s slug and name, the operator, how many reports and filed exports were written, the manifest’s SHA-256, and whether the workspace was deleted. Its entity is the workspace (type tenant, with the workspace’s ID).

• After an --export-only run, the entry belongs to the workspace, like its other entries.
• When the workspace is deleted, the entry is written as a platform entry, with no workspace (tenant_id empty), so it names the workspace as its subject after the workspace is gone. Find it in the database with WHERE action = 'workspace.offboard' AND entity_id = '<the workspace's ID>'.

What is deleted, and what stays

The delete removes the workspace’s row and every row in every table that names the workspace: its people, customers, accounts, transactions, decisions, rule hits, alerts, cases, reports, screening data, settings and the rest. The list of tables is not written down anywhere to be forgotten: the database works it out when the delete runs, from every table that has a workspace column, so a table added in a later version is covered by default. The provider console’s scheduled deletion uses the same delete.

A short, explicit list of tables is kept, because each is a record about the workspace rather than the workspace’s own data:

Kept	Why
The audit log (audit_log)	The record of what was done in the workspace, and of the delete itself
The provider console’s audit log, approvals and tombstones (provider_audit_log, provider_change_requests, provider_workspace_tombstones)	Who at the provider did what to the workspace, and who approved its deletion
Usage tallies (license_usage_daily)	Volumes already reported under the licence
The analytics deletion queue (analytics_deletions)	It is what tells the analytics store to delete its copy
The record of a move between regions (workspace_moves)	It outlives the copy that was moved away

Rows left by earlier offboardings

Before database migration 00058, offboarding deleted the workspace’s row and everything linked to it, but the largest tables are not linked: transactions, decisions, rule hits, customer events and ingest keys. Their rows stayed in the database, out of reach of every workspace but still there. If you offboarded a workspace with an earlier version, remove what it left:

speculapro tenant purge-orphans --dry-run
speculapro tenant purge-orphans --operator "Jane Doe" --confirm purge-orphans

Run it with the schema owner’s SP_DB_URL, as for migrate. The first command only lists, per deleted workspace and table, how many rows are left. The second lists them again and deletes them, in one transaction, and writes a platform audit entry workspace.orphans_purged for each workspace, with the operator and the rows deleted per table. Without --confirm purge-orphans nothing is deleted.

It only touches rows whose workspace no longer exists. A workspace that is being moved to or from another region is left alone, and so are the tables in the list above. Deleting can’t be undone: take a backup first.

If it stops

Message	What to do
--slug, --out and --operator are required	Give all three
deleting a workspace can't be undone: pass --confirm with its slug (or --export-only)	Add --confirm <slug>, or run with --export-only
offboard: <dir> is not empty	Choose a new or empty folder
offboard: no workspace "<slug>"	Check the slug
offboard: export of <number> can't be decrypted	The key settings don’t match the installation’s. Use the same keys as the running servers
offboard: the stored export of <number> doesn't match its SHA-256	The stored file has been altered or damaged. Stop, restore it from a backup and investigate before going further
offboard: delete the workspace (run this with the schema owner's connection, as for migrate)	Nothing was deleted, and the audit entry was rolled back with the delete. The files already written stay in the folder. Run again, into a new folder, with the owner’s SP_DB_URL

Related

• Regulatory reporting: reports are never deleted
• Backups
• Data protection