Docker Compose in production

In short

The Compose deployment files (deploy/compose/production) run a complete SpeculaPro installation on one server: SpeculaPro itself, PostgreSQL 18, a single Redpanda node (for Kafka) and Valkey (for Redis). A script does the first run in one command: it generates every password and the encryption keys, prepares the database, creates the Kafka topics and your first workspace, and starts everything.

It suits smaller institutions, pilots and test installations. It has no high availability: if the server fails, SpeculaPro stops until it is back. For more capacity or resilience, use Kubernetes with Helm.

Validated configuration, not yet a tested install

The Compose files and the first-run script have been checked for correctness, but not yet run end to end against Docker on a fresh server. Try them on a test server before you rely on them, and tell [email protected] about anything that doesn’t work as described here.

	Docker Compose	Helm
Size	One server	Any size; each role scales on its own
PostgreSQL	A container on the same server	Pigsty, or any managed PostgreSQL 18
Kafka	One Redpanda node	A Redpanda or Kafka cluster
Redis	One Valkey, with persistence	Valkey Sentinel or Cluster
High availability	None (containers restart automatically)	Replicas, disruption budgets, spread across zones

Before you start

• Docker Engine with the Compose plugin v2.24 or newer.
• An HTTPS reverse proxy in front: nginx, Caddy, HAProxy or your load balancer. Sign-in cookies are marked Secure, so browsers will only sign in over HTTPS.
• The image digest of the release you verified. See Verifying releases.
• The deployment files on the server: the compose/production/ folder and the sql/ folder it uses, side by side under one deploy/ folder. They come from SpeculaPro, not from a public download: ask [email protected] for the set that matches your release.

How to install

1. Pin the release. Copy .env.example to .env, make it readable by you only (chmod 600 .env), and set SPECULAPRO_IMAGE=ghcr.io/spalce/speculapro@sha256:<digest>. (The script creates .env for you if you skip this, but then uses the default image tag.)

2. Run the first-run script with your first workspace’s details:

   cd deploy/compose/production
   ./first-run.sh --slug acme --name "Acme Bank" --kind bank --admin-email [email protected]

   It:

   1. fills in every empty password in .env and generates the encryption keys;
   2. starts PostgreSQL, Redpanda and Valkey and waits until they are healthy;
   3. applies the database migrations as the owner role, then grants.sql;
   4. creates the Kafka topics;
   5. creates the workspace and prints its administrator’s temporary password, once;
   6. starts SpeculaPro.

   Other options: --admin-name, --country (default GH), --currency (GHS), --timezone (Africa/Accra), and --skip-workspace to run it again after the workspace exists. Steps 2 to 4 and 6 are safe to repeat.

3. Back up .env now, somewhere other than this server. It holds SP_SECURITY_MASTER_KEYS and SP_SECURITY_BLIND_INDEX_KEY. Without them, the encrypted customer data in your database can’t be read by anyone, including SpeculaPro.

4. Set the public address. Set SP_NOTIFY_PUBLIC_URL in .env to the https:// address people use, so links in emails work, and run docker compose up -d.

5. Point your reverse proxy at 127.0.0.1:8080 (and at 127.0.0.1:8081 for decisions in the split layout). SP_BIND and SP_HTTP_PORT change the address and port.

6. Sign in with the temporary password and change it, then install your licence under Settings → License. See Licensing.

Proxy settings

.env sets SP_HTTP_TRUST_PROXY=true and SP_HTTP_PROXY_HOPS=1, which is right for one reverse proxy on the same server. If a load balancer sits in front of that proxy, set the hops to 2. And keep the published ports on 127.0.0.1 (or a private address only your proxy can reach): with SP_HTTP_TRUST_PROXY=true, anyone who can reach SpeculaPro directly could claim any client address. See Configuration: HTTP server.

Layouts

Choose with COMPOSE_PROFILES in .env, then run docker compose up -d.

Profile	Containers	When
all (the default)	speculapro, running every role	Most single-server installations
split	api (the api and ingest roles) on port 8080; decision on its own port, 8081 (SP_DECISION_PORT); background (stream, persist, worker, connector)	When real-time decisions mustn’t compete with people using the web app, or with large file imports

Each container’s health check covers only what its roles need:

Container	Health check	Healthy when
speculapro (all)	/readyz?role=all on 8080	PostgreSQL answers. It serves decisions, which keep answering with the workspace’s failure policy while Kafka or Valkey is down
api (split)	/readyz?role=api,ingest on 8080	PostgreSQL and Kafka answer
decision (split)	/readyz?role=decision on 8080	PostgreSQL answers, for the same fail-open reason
background (split)	/readyz on the metrics port 9090	PostgreSQL, Kafka and Valkey answer (stream needs all three)

See Health checks.

Analytics store (optional)

The Compose file can also run a single-node ClickHouse as the analytics store, which lets the Analytics page cover 24 months instead of 3 and takes long backtests off PostgreSQL. It is off unless you add the analytics profile.

1. In .env:

   COMPOSE_PROFILES=all,analytics
   SP_CLICKHOUSE_URL=clickhouse://clickhouse:9000
   SP_CLICKHOUSE_PASSWORD=<a long random password>

   With the split layout, use COMPOSE_PROFILES=split,analytics and also set SP_BACKGROUND_ROLES=stream,persist,worker,connector,analytics.

2. Start ClickHouse, create its schema and restart SpeculaPro:

   docker compose up -d clickhouse
   docker compose run --rm cli migrate up
   docker compose up -d

3. Copy the history you already have, once:

   docker compose run --rm cli analytics backfill

Container	clickhouse (clickhouse/clickhouse-server:25.8), data in the clickhouse volume, not published outside the Compose network
Memory	Limited to CLICKHOUSE_MEMORY (4G by default) and 2 CPUs. On a single host it shares the machine with PostgreSQL, Redpanda and Valkey: check the host has the room before enabling it
Who fills it	With all, the speculapro container, automatically. With split, the background container, once analytics is in SP_BACKGROUND_ROLES
Health	With split and the analytics role, the background container is healthy only when ClickHouse answers too
ClickHouse elsewhere	Leave the analytics profile out and point SP_CLICKHOUSE_URL, SP_CLICKHOUSE_USER and SP_CLICKHOUSE_PASSWORD at it. Other SP_CLICKHOUSE_* settings go in speculapro.env

The data protection purge reaches the store

What the retention purge or an erasure removes in PostgreSQL is removed from the copy too, by the worker role. ClickHouse’s own backups are the exception. See how the purge reaches the copy.

What the containers look like

SpeculaPro	Runs as a non-root user (65532) with a read-only filesystem, no Linux capabilities and no-new-privileges. /tmp is a 512 MB in-memory area for uploaded files. Health is checked with speculapro healthcheck, since the image has no curl
Shutdown	stop_grace_period: 60s: SpeculaPro needs up to about 35 seconds after the stop signal to finish requests and hand over decided transactions
PostgreSQL 18	The three SpeculaPro roles are created from .env when its data volume is first created. Settings such as shared_buffers come from .env
Redpanda	One node, automatic topic creation off
Valkey	Password-protected; append-only persistence every second; maxmemory-policy noeviction, so running totals are never dropped silently
File drops	Files fetched from SFTP or S3 are staged in the spool volume (SP_CONNECTOR_SPOOL_DIR) while they load, so it needs room for the largest file
Limits	Each container has a CPU and memory limit in compose.yaml. Adjust them to the server
Logs	JSON files, rotated at 50 MB, five kept per container

Settings not in .env (any variable in the configuration reference) go in a speculapro.env file next to compose.yaml, which every SpeculaPro container loads.

Day-to-day commands

From deploy/compose/production:

Task	Command
See the health of every container	docker compose ps
Follow the logs	docker compose logs -f speculapro (or api, decision, background)
Run a one-off command, as the owner role	docker compose run --rm cli <command>
Check the version	docker compose run --rm cli version
Check the licence	docker compose run --rm cli license status
Get the installation fingerprint	docker compose run --rm cli license fingerprint
Create another workspace	docker compose run --rm cli tenant create --slug … --name … --admin-email …
Give someone a new temporary password	docker compose run --rm cli tenant reset-password --slug acme --email …
See what the analytics store holds (with the analytics profile)	docker compose run --rm cli analytics status

Backups

PostgreSQL holds everything that matters. The optional analytics store is a copy that can be rebuilt from PostgreSQL, so it needs no backup of its own. For a daily dump:

docker compose exec -T postgres pg_dump -U postgres -Fc speculapro > backups/speculapro-$(date +%F).dump

To restore into a fresh database volume: stop SpeculaPro, recreate the postgres service with an empty volume (which re-creates the roles from .env), then:

docker compose exec -T postgres pg_restore -U postgres -d speculapro --no-owner --role=speculapro_owner < backups/<file>.dump
docker compose exec -T postgres psql -v ON_ERROR_STOP=1 -U postgres -d speculapro -f /speculapro-sql/grants.sql

A daily dump can lose up to a day of data. For continuous backups and point-in-time recovery on one server, run pgBackRest or WAL-G against the PostgreSQL data volume, or move to Pigsty. Keep the encryption keys backed up separately: a database backup is unreadable without them. See Backups and restore.

Upgrades

See Upgrading: with Docker Compose.

Apply grants.sql again after the migrations

first-run.sh applies grants.sql once. An upgrade doesn’t, so run it yourself after migrate up and before replacing the containers. It is safe to repeat:

docker compose exec -T postgres psql -v ON_ERROR_STOP=1 -U postgres -d speculapro -f /speculapro-sql/grants.sql

Use the grants.sql of the release you are upgrading to. Without it, functions a new release adds can’t be called. After migration 00056, for example, the worker can no longer create monthly partitions (permission denied for function sp_ensure_monthly_partitions in its log) until the grants are applied.

Related

• Kubernetes with Helm
• Production checklist
• Licensing
• Encryption keys