Design decisions
In short
Section titled “In short”Every system is shaped by a handful of big decisions. This page records SpeculaPro’s, with the alternatives that were considered and why they lost. It is useful for security and architecture reviewers, procurement questionnaires, and anyone wondering “why didn’t they just…?”.
| Decision | Chosen | Instead of |
|---|---|---|
| Where the data lives | SpeculaPro’s own copy | Querying your database live |
| Rule language | CEL | Code, a scripting language, or a home-made language |
| HTTP framework | Huma on chi | Fiber |
| Web app | Vite + React SPA built into the Go binary | Next.js |
| Feature store | Valkey or Redis | Postgres, or an in-process store only |
| Messaging | Kafka or Redpanda, keyed by customer | A job queue, or direct calls between services |
| Deployment | One binary with roles | Many separate microservices |
1. Keep our own copy of the data
Section titled “1. Keep our own copy of the data”Decision. SpeculaPro stores the transactions and entities it evaluates. It never queries an institution’s systems at evaluation time.
Why. Long-window AML rules can’t run against a remote database at high volume. Regulators need proof of exactly what a rule saw. Every institution’s schema differs. And monitoring must keep working when the source system is down.
Trade-off. SpeculaPro holds personal data, so it must protect it (encryption, retention, access logs). Self-hosting is available for institutions that cannot send data out. See Data ownership and ingestion.
2. CEL for rules
Section titled “2. CEL for rules”Decision. Rules are expressions in Google’s Common Expression Language, run by cel-go.
Why. CEL is sandboxed (a rule cannot touch files, network or data), always terminates (no loops), is type-checked when saved, and evaluates in microseconds once compiled. Rules can be edited by tenants without a deploy, and a visual builder can generate them.
Alternatives. Rules as Go code would need a release for every change. A general scripting language (Lua, JavaScript) would need heavy sandboxing and can loop forever. A home-made language would need its own parser, type checker and documentation. See Rule engine.
3. Huma on chi, not Fiber
Section titled “3. Huma on chi, not Fiber”Decision. The HTTP API uses Huma (which generates the OpenAPI description from the code) on the chi router, both built on Go’s standard net/http.
Why. Fiber is built on fasthttp, which is not net/http. That means no HTTP/2, and awkward integration with OpenTelemetry and standard middleware. Routing speed is not SpeculaPro’s bottleneck, so the standard library’s compatibility wins. Huma also keeps the published API description in step with the code automatically.
4. A single-page app built into the binary, not Next.js
Section titled “4. A single-page app built into the binary, not Next.js”Decision. The web app is a Vite + React + TypeScript + Tailwind single-page application, compiled into the Go binary and served by it.
Why.
- No Node.js runtime to ship to self-hosted customers: one fewer thing to install, patch and secure.
- Same origin as the API, so sign-in uses a simple, secure cookie with no cross-site complications.
- Works air-gapped, with no external CDN or server-side rendering service.
The API client used by the web app is generated from the same OpenAPI description, so the two can’t drift apart.
5. Valkey or Redis for features
Section titled “5. Valkey or Redis for features”Decision. Running totals and profiles live in Redis or Valkey, in time buckets. Standalone, Sentinel and Cluster layouts are all supported.
Why. Real-time rules need answers in microseconds; Postgres can’t provide that under thousands of updates a second. The data is derived, never the record, so an in-memory store is appropriate (it is not trivially rebuilt, though: see Feature store). Valkey is offered alongside Redis because its licence is simpler for self-hosted deployments. See Feature store.
6. Kafka or Redpanda, keyed by customer
Section titled “6. Kafka or Redpanda, keyed by customer”Decision. Components communicate through Kafka-compatible topics, keyed by tenant_id:entity_id, through the franz-go client.
Why. Keying by customer means each customer’s events are processed in order by one worker, so running totals stay correct without locks. Kafka also provides buffering, replay and back-pressure. franz-go works with both Apache Kafka and Redpanda, so institutions can use what they already run. See Components and Kafka topics.
Consequence. Partition counts are set high (64) from day one, because changing them later breaks ordering.
7. One binary with roles
Section titled “7. One binary with roles”Decision. Everything ships as one program; --role chooses what each copy does.
Why. Small institutions run one process. Large ones scale each role independently. Both run identical, tested software, and there’s one artefact to sign, scan and upgrade. See Architecture overview.
Smaller decisions worth knowing
Section titled “Smaller decisions worth knowing”| Topic | Decision |
|---|---|
| Database | Postgres 18, Pigsty for production high availability; sqlc for type-checked SQL; monthly partitions for large tables |
| Tenancy | tenant_id on every row, enforced by Postgres row-level security. See Tenant isolation |
| Money | Integer minor units + currency + base-currency amount + the FX rate used. See Data model |
| Analytics | ClickHouse, optional, added only when backtesting and dashboards outgrow Postgres. A derived copy that can be rebuilt; Postgres stays the system of record. See Analytics store |
| Machine learning | Rules first. A workspace can add an anomaly model trained on its own transactions; rules read its score, and the model never blocks or alerts by itself. Models trained on labelled outcomes are not built |
Still open
Section titled “Still open”These questions are not yet settled. Pages that depend on them will change.
- Confirming the performance targets against a pilot’s real volumes.
- Each FIU’s reporting specifics. The goAML export and the CENTIF declaration archive are built, but the codes, thresholds and submission method they default to must still be confirmed with each FIU. See Regulatory reporting.
- What each central bank and data protection authority requires about where hosted data is kept. Region pinning is built (see Multi-region); the requirements themselves are yours to confirm with counsel.
Three questions once listed here are settled. Self-hosted installations run under a signed licence that works offline. Scoring by a model is in: the anomaly model above. And hosted data can be pinned to a region.