Operate Thoryn

This section is the operations how-to for running the Thoryn Identity Platform. It serves two audiences on one artefact set: a self-managed customer running the platform on their own Kubernetes, and Thoryn SRE running the managed service. Both operate the identical Helm chart, container images, Flyway migrations, and configuration schema — what differs is who runs the infrastructure, not what runs. The architectural basis for that is Delivery-model architecture; this section is the operational instantiation of it.

Shared-responsibility split

Concern	Managed (sovereign SaaS)	Self-managed
Infrastructure (cluster, nodes, storage, TLS)	Thoryn	You, via the Helm chart
Deploy / upgrade / rollback	Thoryn	You (this section)
Secrets backend (OpenBao) provisioning + unseal	Thoryn	You (the chart's init Job)
Backups + disaster-recovery drills	Thoryn	You (this section)
Tenant / client / federation / policy configuration	The customer (console + API)	The customer (same console + API)

The customer plane — tenant configuration, OAuth-client registration, federation, RBAC/FGA, audit — is self-service in both models through the console and the product API. This section covers the layer underneath that: getting the servers running and keeping them healthy.

New to the platform? Walk the self-managed quickstart tutorial first to stand up a running stack, then use these runbooks for the production-grade depth — required configuration, secrets, recovery — the quickstart defers here.

The one hard rule: rebuildable from git, zero manual steps

Every runbook here is written to a single discipline that both delivery models inherit:

The entire deployed state of any environment must be rebuildable from git plus a clean deploy, with zero manual steps.

Infrastructure ships as Helm chart/values, schema as Flyway migrations, configuration as application*.yml, behaviour as source. kubectl edit / patch / apply against live resources, console clicks, and hand-rolled secrets are not how changes are made — they leave no reproducible trail and drift on the next deploy. Emergency incident response is the only exception, and even then the manual change is a mitigation that must be back-filled into the chart/migration/config in the same or an immediately-following change. A fix is "done" only when a clean, from-scratch deploy reproduces the working state.

Source: CLAUDE.md (reproducible deployments); deploy/helm/thoryn/.

The runbooks

Runbook	What it covers
Production go-live cutover runbook	The ordered, gated sequence to take the platform to production: the current blocking gates at a glance, numbered phases with entry/exit gates (pre-flight, provisioning, migrations, the test-mode flip, DNS/TLS cutover, verification), and a per-phase rollback plan. Start here for a cutover.
Deploy	First deploy of the self-managed distribution: prerequisites, the required Helm values (and why each fails loud), OpenBao init/unseal, DNS/TLS, first-boot Postgres + Flyway ordering, and a from-scratch verify.
Upgrade	Rolling helm upgrade: the immutable image-SHA contract, forward-only Flyway on boot, readiness gating, and rollback.
Backup and recovery	What is backed up (Postgres dumps, OpenBao file-backend), and — for recovery — the forward-patch-only Flyway procedure, the never-DROP DATABASE rule, OpenBao unseal recovery, and Helm SSA field-ownership reclaim.
Tenant BYO-SMTP activation	Turning on per-tenant bring-your-own SMTP: the chart-owned credential-encryption key, the activation flag, the three outcomes to verify, and why rotating that key is destructive.

The operator plane (Thoryn-employee cross-tenant operations, VPN-gated, admin:* scope namespace) is a separate surface documented under operate/operator-plane/ and is not part of the self-managed deploy path.

Two things to internalise before you start

• The secrets backend is OpenBao (the Linux Foundation fork of Vault), run on persistent storage with auto-unseal, holding all signing keys via the Transit engine. A backend pod roll is a non-event because keys persist and the store re-unseals itself — see Signing and crypto and ADR adrs/2026-06-07-vault-to-openbao.md. Historically the platform ran Vault -dev mode, which wiped keys on restart; several runbooks still carry that vocabulary for the documented rollback path.
• Schema recovery is forward-patch only. DROP DATABASE / TRUNCATE / drop-and- re-bootstrap is forbidden in every environment — a CI guard rejects it — because the cost of destroying user-created data is unacceptable even in staging. Recovery converges the schema with a new migration or escalates; it never nukes.

Configuration reference

The knobs these runbooks set are documented in the generated Configuration reference — start with Infrastructure for Postgres / Redis / OpenBao, and the per-service pages for the hub, identity-service, product-api, and api-gateway.