DEPLOYMENT — production deployment guide (P20/R5a, ADR-037)

Canonical runtime: PHP 8.5.12 EVERYWHERE (stdlib-only; owner directive

2026-10-25 — prod /opt/php85, CI, Docker and the agent sandbox via

packages.sury.org; the floor is 8.5 and bin/iya doctor checks it). Database: SQLite (WAL). TLS:

Caddy 2 automatic HTTPS. Two supported topologies: **(A) docker

compose and (B) bare-metal systemd. A minimal VPS — 1 vCPU /

1 GB RAM / 10 GB disk** — is sufficient for beta-scale traffic; SQLite +

built-in PHP workers keep the moving parts at exactly three processes

(web, worker loop, reverse proxy).

Contents: Requirements · Prod .env · A: docker compose · B: bare-metal · TLS · Backups · Restore runbook · Upgrades · SMTP/DNS mail · Log rotation · Monitoring · Security notes

1. Requirements

Debian/Ubuntu packages for option B:

sudo apt-get update
sudo apt-get install -y php-cli php-sqlite3 php-mbstring php-xml \
    php-curl php-intl php-zip caddy
php -m | grep -Ei 'pdo_sqlite|sqlite3|mbstring|intl|zip|curl|dom|simplexml|Zend OPcache'

Get the code (either topology):

sudo mkdir -p /opt/iya-platform && sudo useradd -r -s /usr/sbin/nologin iya || true
# from a release tarball:
sudo tar xzf iya-platform_<stamp>_<commit>.tar.gz -C /opt/iya-platform --strip-components=1
# …or from git:
# sudo -u iya git clone <your-repo-url> /opt/iya-platform
sudo chown -R iya:iya /opt/iya-platform/var
cd /opt/iya-platform && sudo -u iya php bin/iya doctor

var/ is the ONLY directory that must be writable at runtime (SQLite DB, uploads, contrib quarantine, metrics counters, logs). Everything else can be root-owned and read-only — both systemd units ship ProtectSystem=full + ReadWritePaths=/opt/iya-platform/var.

2. Production .env

Copy the template and edit — the real .env is never committed:

sudo -u iya cp .env.example .env && sudo -u iya chmod 600 .env
sudo -u iya php bin/iya key:generate     # -> 64 hex chars for APP_KEY

Non-negotiable production values (drift from dev defaults is deliberate):

Contribution pipeline kill switches stay OFF until you mean it (CONTRIB_ENABLED, CONTRIB_RUNNER_ENABLED, CONTRIB_AUTO_PROMOTE=false always — a human confirms promotions).

First boot:

sudo -u iya php bin/iya migrate      # idempotent; docker `app` does this on boot
# php bin/iya seed                   # NEVER in production (demo data)
sudo -u iya php bin/iya doctor

3. Option A — docker compose

Stack (repo files deploy/docker-compose.yml + deploy/Caddyfile, image from the repo-root Dockerfile, base php:8.5-cli-alpine):

• app — migrate-on-boot, serves on :8080 (4 CLI workers), healthcheck

on /api/v1/health;

• worker — same image/volume, `while true; do php bin/iya worker --once;

sleep ${IYA_WORKER_SLEEP:-60}; done; starts only after app` is healthy (boot migration finished);

• caddy — ports 80/443(+udp), automatic HTTPS, reverse-proxies to

app:8080, access log in the caddy-data volume;

• volumes — iya-var → /app/var (ALL persistent state: SQLite, uploads,

contrib, metrics, logs), caddy-data (certs!), caddy-config.

cd /opt/iya-platform
cp .env.example .env && chmod 600 .env       # fill in §2 values; add:
echo 'TRUSTED_PROXIES=172.30.0.0/24' >> .env # fixed compose subnet
docker compose -f deploy/docker-compose.yml config --quiet   # sanity
docker compose -f deploy/docker-compose.yml up -d --build
curl -sf -H 'Host: abuseipdb.ru' http://127.0.0.1/api/v1/health && echo OK

Operational notes:

• certs live in caddy-data — back it up or lose the rate-limit budget

of Let's Encrypt (`docker run --rm -v iya-platform_caddy-data:/d -v $(pwd)/backups:/b alpine tar czf /b/caddy-data.tgz -C /d .`).

• The compose network subnet is FIXED (172.30.0.0/24) so TRUSTED_PROXIES

is deterministic; app pins to 172.30.0.10.

• Rebuild after upgrades: `docker compose -f deploy/docker-compose.yml up -d

--build` (app re-runs migrations on boot; worker waits for health).

• SQLite + docker volume: keep the DB on a LOCAL volume (never NFS — WAL

needs real POSIX locks).

• Backups: run deploy/backup.sh from a cron on the HOST with

IYA_BACKUP_DIR pointing at a host dir mounted into the app container, or docker compose exec app php bin/iya …-style snapshot (§6 adapts: the tarball contents are identical — var/ lives in the iya-var volume).

4. Option B — bare-metal systemd

Units ship in deploy/ (conventions: app root /opt/iya-platform, run user iya, secrets in /opt/iya-platform/.env — adjust the unit headers if your layout differs; they carry no developer-home paths):

sudo install -m 644 deploy/iya-web.service    /etc/systemd/system/
sudo install -m 644 deploy/iya-worker.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now iya-web iya-worker
systemctl status iya-web --no-pager | head -5
curl -sf -H 'Host: abuseipdb.ru' http://127.0.0.1:8080/api/v1/health && echo OK

Caddy reverse proxy (bare metal) — install deploy/Caddyfile to /etc/caddy/Caddyfile, replacing the upstream app:8080 with 127.0.0.1:8080:

sudo sed 's/app:8080/127.0.0.1:8080/' deploy/Caddyfile | sudo tee /etc/caddy/Caddyfile
sudo systemctl reload caddy

Set TRUSTED_PROXIES=127.0.0.1 in .env so the app reads Caddy's X-Forwarded-For (and only Caddy's).

Cron alternative (no systemd — e.g. container-less minimal hosting):

# worker pass every minute
* * * * * cd /opt/iya-platform && /usr/bin/php bin/iya worker --once >> var/logs/cron-worker.log 2>&1
# nightly backup at 03:17 UTC (§6)
17 3 * * * IYA_BACKUP_DIR=/var/backups/iya IYA_OFFSITE_DIR=/mnt/offsite/iya /opt/iya-platform/deploy/backup.sh >> /var/log/iya-backup.log 2>&1

The web tier under cron-less setups still needs a supervised long-runner: php bin/iya serve … under runit/s6, or @reboot + a keepalive cron line (`pgrep -f 'bin/iya serve' >/dev/null || (cd /opt/iya-platform && nohup php bin/iya serve --host=127.0.0.1 --port=8080 >> var/logs/serve.log 2>&1 &)`). systemd is the supported path; cron-only is best-effort.

5. TLS — Caddy

Caddy obtains and renews Let's Encrypt certificates automatically for every site address in the Caddyfile ({$IYA_APEX:abuseipdb.ru}, www., agent-swarm.) — HTTP-01 needs ports 80/443 open and DNS pointing at the box. Set ACME_EMAIL (compose passes it from .env) for expiry notices.

Tenant wildcard subdomains (*.abuseipdb.ru → per-user sites) require a wildcard certificate, and Let's Encrypt issues those only via DNS-01:

1. build Caddy with your DNS provider plugin:

xcaddy build --with github.com/caddy-dns/cloudflare (or route53/gandi/…);

1. swap the image in deploy/docker-compose.yml (caddy: → your build);
2. uncomment the *.{$IYA_APEX:abuseipdb.ru} block in deploy/Caddyfile and

export the provider token (CF_API_TOKEN etc.) to the caddy service.

Alternative: buy a commercial *.abuseipdb.ru certificate, terminate it on an upstream LB, and run Caddy (or the app) plain-HTTP behind it — then set TRUSTED_PROXIES to that LB and keep COOKIE_SECURE=true.

Renewals need no cron (Caddy's background ACME manager); verify with caddy fmt --overwrite /etc/caddy/Caddyfile && systemctl reload caddy after edits and watch /data/iya-access.log + the caddy journal.

6. Backups

deploy/backup.sh (crontab-ready, set -euo pipefail) produces a versioned tarball iya-platform_<UTCstamp>_<commit>.tar.gz + .sha256 sidecar with:

• the SQLite snapshot taken via VACUUM INTO (online, WAL-safe,

integrity-checked with PRAGMA integrity_check before packaging — no sqlite3 CLI needed, just PHP);

• the whole var/ tree except cache/+test/ scratch (uploads, contrib

quarantine, metrics counters, logs);

• .env as env.snapshot (tarball is chmod 600 — it carries secrets);
• a VERSION stamp (git commit or var/RELEASE, platform semver, PHP).

Environment knobs: IYA_BACKUP_DIR (default /var/backups/iya), IYA_OFFSITE_DIR (offsite mirror — a mounted NAS/S3-fuse/another disk), IYA_BACKUP_KEEP (local generations to keep, default 14).

Crontab (also printed in the script header):

17 3 * * * IYA_BACKUP_DIR=/var/backups/iya IYA_OFFSITE_DIR=/mnt/offsite/iya /opt/iya-platform/deploy/backup.sh >> /var/log/iya-backup.log 2>&1

Rotation rule (project trap #33, enforced by the script): an old local tarball is deleted ONLY after its sha256 was verified against the offsite copy (or, without offsite, against its own sidecar). Unverified files are kept and reported on stderr. Offsite copies are verified right after upload (sha256sum -c). Do not "clean up" backup dirs by hand.

Docker topology: run the same script on the host against a bind-mounted var/ (or mount iya-var read-only into a one-shot container); the archive layout is identical, so restore.sh works for both.

7. Restore runbook

deploy/restore.sh <tarball> [target-app-dir] is deliberately conservative — it verifies the sidecar sha256, then stages:

• env.snapshot → <target>/.env.restore (live .env is NEVER overwritten;

diff and merge by hand, and rotate secrets if the archive came from a compromised box);

• iya.sqlite3 → <target>/var/data/iya.sqlite3.restore (live DB is NEVER

overwritten; promotion is an explicit mv printed by the script);

• the rest of var/ merged file-by-file (no deletions).

Full bare-metal restore on a fresh VPS:

1. provision §1 (packages, iya user, /opt/iya-platform from the release

tarball matching VERSION/commit in the backup — or any newer one: migrations move the schema forward, never back);

1. `deploy/restore.sh /path/iya-platform_<stamp>_<commit>.tar.gz

/opt/iya-platform`;

1. merge .env.restore → .env (chmod 600);
2. promote the DB: `mv var/data/iya.sqlite3{,.fresh} && mv

var/data/iya.sqlite3{.restore,}`;

1. php bin/iya migrate && php bin/iya doctor;
2. systemctl start iya-web iya-worker (docker: up -d);
3. verify curl -sf -H 'Host: abuseipdb.ru' http://127.0.0.1:8080/api/v1/health,

log in, spot-check a tenant redirect and /status;

1. Caddy re-issues certs automatically; if you saved caddy-data, restore

it to skip issuance entirely.

Drill it: a restore you never rehearsed is a hypothesis.

8. Upgrade procedure

Zero-downtime-ish single-box upgrade (order matters — migrate BEFORE swap):

cd /opt/iya-platform
# 0) backup first (§6) — every upgrade starts with a fresh tarball
deploy/backup.sh
# 1) fetch the new release NEXT TO the live root
sudo -u iya tar xzf iya-platform_<new>.tar.gz -C /opt/iya-platform.new --strip-components=1
# 2) carry over state + secrets (var/ and .env live outside the code tree)
sudo -u iya cp -a var /opt/iya-platform.new/var.live   # or keep var/ as a bind-mount/symlink shared by both roots
sudo -u iya cp .env /opt/iya-platform.new/.env && sudo chmod 600 /opt/iya-platform.new/.env
# 3) gates on the NEW tree against the LIVE database copy:
cd /opt/iya-platform.new && sudo -u iya php bin/iya lint | tail -1
sudo -u iya php tests/run.php | tail -1        # optional but cheap insurance
sudo -u iya php bin/iya migrate                # applies pending migrations
# 4) swap + restart
sudo systemctl stop iya-web iya-worker
sudo mv /opt/iya-platform /opt/iya-platform.old && sudo mv /opt/iya-platform.new /opt/iya-platform
sudo systemctl start iya-web iya-worker
curl -sf -H 'Host: abuseipdb.ru' http://127.0.0.1:8080/api/v1/health
# 5) watch /status + metrics for CONTRIB_BETA_WINDOW; rollback = swap back
#    (.old still there) and restore the pre-upgrade DB from step 0 tarball.

Docker: docker compose -f deploy/docker-compose.yml up -d --build — the app container migrates on boot, the worker waits for health. Rollback = docker tag the previous image back to iya-platform:latest + up -d, then restore the DB from the step-0 tarball if migrations were destructive (they are designed not to be).

Never skip: backup → lint/tests → migrate → swap. Migrations are forward-only; the rollback path is always "previous tarball".

9. SMTP + SPF/DKIM/DMARC

Mail leaves the platform via MAIL_DRIVER=smtp (drivers: log (dev), smtp, sendmail). Prod config: MAIL_HOST, MAIL_PORT=587, MAIL_ENCRYPTION=tls, MAIL_USER, MAIL_PASS, MAIL_FROM=no-reply@abuseipdb.ru, MAIL_FROM_NAME=abuseipdb.ru. Test with a password-reset on a real mailbox and watch var/logs/app-<day>.log.

Full reference: [EMAIL.md](EMAIL.md) — every MAIL_* key, DKIM key generation and rotation (selector iya1, rsa2048), the DMARC rollout ladder, the deliverability checklist, the php bin/iya mail:test / doctor verification runbook and the live-SMTP drill.

DNS checklist for deliverability (all on the sending domain, e.g. abuseipdb.ru):

Post-setup validation: send to a mailbox you control, check Authentication-Results headers (spf=pass dkim=pass dmarc=pass), and register the domain at Google Postmaster Tools / Microsoft SNDS. Bounce handling: start with the rua= report address, automate later.

10. Log rotation

The app writes DATED files under var/logs/ (app-<day>.log, access-<day>.log, js-errors-<day>.log (RUM, when enabled), cron-worker.log from §4) plus daily var/metrics/counter-<day>.json counters. Two layers:

1. app-side: dated naming means no giant single file; a weekly cron

prunes anything older than the retention you choose:

   30 3 * * 0 find /opt/iya-platform/var/logs -name '*.log' -mtime +30 -delete; \
              find /opt/iya-platform/var/metrics -name 'counter-*.json' -mtime +400 -delete

(keep metrics ~13 months: the /status uptime ring reads 90 days, beta verdicts compare windows.)

1. system-side (journals + Caddy): stock logrotate/journald limits —

SystemMaxUse=200M in /etc/systemd/journald.conf; Caddy's own file log rolls at 10 MB × 5 (set in deploy/Caddyfile).

Disk-pressure smoke check: du -sh /opt/iya-platform/var/* monthly; the contrib quarantine (var/contrib/) has its own retention (CONTRIB_QUARANTINE_DAYS=7).

11. Monitoring

• GET /api/v1/health — liveness (used by compose healthcheck + Caddy).
• GET /api/v1/metrics — Prometheus text exposition (R5b): public minimum

(iya_up, iya_info, queue depths) + PRIVATE rows (per-zone request counters, rate-limit hits, contrib pipeline counters, uptime, RUM js_errors) gated by METRICS_TOKEN — send Authorization: Bearer <METRICS_TOKEN>. Scrape every 15–60 s.

• GET /status — public status page: 90-day uptime ring + incident history

(admin CRUD at /admin/incidents, R5b).

• Worker liveness: systemctl is-active iya-worker / container health; the

visit queue depth (iya_queue_depth in metrics) growing steadily means the worker loop is stuck.

• Alerts: the platform has an internal alert pipeline (alert_events,

webhook link.target_down etc.) — wire WEBHOOK_ENABLED=true receivers to your ops channel.

12. Security notes

• Run as the unprivileged iya user; units ship hardened

(NoNewPrivileges, ProtectSystem=full, ProtectHome, PrivateTmp).

• .env is mode 600 and inside the app root — never serve it (the front

controller only maps public/); backups carry it as env.snapshot, so treat backup storage as SECRET storage (offsite dir: encrypt at rest or restrict mounts).

• Dedicated WEBHOOK_SECRET_ENC_KEY + CREDENTIAL_SIGNING_KEY (§2) —

dev-derived keys in prod are forbidden (ADR-035.7/ADR-036.9).

• Keep ADMIN_REQUIRE_2FA=true; scope ADMIN_IP_ALLOWLIST if admins have

static IPs.

• Firewall: only 80/443 (+SSH from your management range) are public; the

app listens on loopback (bare-metal) or the compose network (docker).

• Updates: OS security patches via unattended-upgrades; PHP minor bumps are

safe (floor 8.5, canonical 8.5.12 everywhere); re-run §8 for every app release.

13. Continuity endpoint (qwen-gateway) + prod mail state — s30 notes

qwen-gateway. deploy/qwen-access/qwen-gateway.php is a self-contained stdlib endpoint for agent continuity (workspace mirror over qwen_access/, multi-transport: HTTP JSON/GET/raw-stream/multipart, MCP JSON-RPC, CLI stdin/argv, raw socket --serve). Install it into the site root (/var/www/iya/data/www/abuseipdb.ru/qwen-gateway.php) — NOT inside qwen_access/ (that zone is download-only by design and stays the gateway's data store). Full owner checklist (token rotation via SetEnv QWEN_TOKEN, optional QWEN_ALLOW_CIDR, nginx advice, smoke test) and the agent runbook: docs/QWEN-ACCESS.md. The app itself does not depend on the gateway; not installing it breaks nothing. The qwen_access/ store zone is served world-readable with directory listings ON (semi-public by design, docs/QWEN-ACCESS.md §8.2): the live exposure probe and the owner checklist for narrowing or closing it (options A/B/C + apply-and-verify + rollback) are docs/QWEN-ACCESS.md §8.3 — an owner decision; the gateway reads the store from the filesystem, so deny all breaks nothing.

Ops dashboard + continuation worker (s45, gateway v1.1.0). The gateway file doubles as: (1) the owner ops dashboard — https://abuseipdb.ru/qwen-gateway.php?op=dash (md5 password gate per owner spec; panels: prod sha/age, services, cert days, disk, alerts, worker heartbeat, #101 residue, deploy history, agent notes); (2) the hourly continuation worker — root cron php …/qwen-gateway.php --worker (install: --worker-install; control: store flag deploy/worker-disabled; output: _audit/status-latest.json + status-history.ndjson + alerts.ndjson; bundle: deploy/qwen-access/WORKER.md); (3) agent context-economy ops status/probe/batch. Details: QWEN-ACCESS §4.1+§9.

Prod mail (STATUS s44: CUTOVER DONE). The live .env on abuseipdb.ru carries MAIL_DRIVER=smtp — the owner executed the switch on 2026-10-24 (s41): local Exim relay MAIL_HOST=127.0.0.1 port 25, dry-run + live send OK, systemctl reload php-fpm85 performed (Config caches .env per worker process — the reload is MANDATORY), and the setting survived self-deploys #12/#13/#14. Waitlist/notification mail (P21/T1 contour) is now really delivered; the remaining R1 step is the owner acceptance on a real external mailbox. Verification marker: php bin/iya mail:test --to=<address> (expects the MAIL-TEST OK marker). SPF/DKIM/DMARC prerequisites: §9 and docs/EMAIL.md. The exact switch procedure on the live box (P22/R1) — including the worker liveness check, the honest failure semantics of the synchronous guest paths and the rollback — is docs/EMAIL.md §11.

Migration 022 reconciliation (one-time). If php bin/iya migrate on the already-migrated prod DB aborts with a 022 checksum mismatch after deploying the repo tree (the repo's schema-parity reconstruction vs the bytes the hotfix applied), follow docs/QWEN-ACCESS.md §7 — compare PRAGMA table_info(agent_owner_binds), then either reconcile the schema_migrations checksum or adopt the prod bytes into the repo.

14. SSE streams in production (P21/T3)

GET /api/v1/agents/me/stream (and the dashboard realtime streams) hold one PHP worker for the whole connection window. Production checklist:

• php-fpm: request_terminate_timeout must exceed the stream window

(SSE_WINDOW_SEC, hard-capped by SSE_MAX_SECONDS=300), or the worker is killed mid-stream and the client sees a truncated response instead of the polite event: end. Size the pool accordingly: every concurrent stream parks one worker for up to the window (pm.max_children >= expected concurrent streams + normal traffic).

• nginx / reverse proxy: the response already sends

X-Accel-Buffering: no and Cache-Control: no-store, no-transform; additionally set proxy_read_timeout >= the window (default 60 s is fine for the default 30 s window) and keep proxy_buffering off for the /api/v1/agents/me/stream location if your build ignores X-Accel-Buffering.

• Kill switch: AGENT_SSE_ENABLED=false => honest 403 sse_disabled

(agents fall back to polling). SSE_MAX_CONN caps concurrent streams per token (slot lock files in var/run/sse/; flock dies with the worker — no stale locks after crashes, the directory needs write access for the web user).

• Dev router: php -S is single-threaded by default — a stream blocks

every other request until the window closes. For local multi-client testing start it with PHP_CLI_SERVER_WORKERS=4.

15. Billing (Stripe S1–S5) in production

The billing contour ships DARK (BILLING_ENABLED=false): zero Stripe calls, /pricing stays variant A, upgrade/success/cancel/webhook/portal routes are 404s. Turning it on is an owner-side procedure with a rehearsal requirement — the full runbook (Product/Price creation, webhook endpoint registration with the exact 4-event v1 set, customer-portal configuration, secret rotation, go-live checklist, stripe trigger verification, test cards) lives in docs/BILLING.md. Deploy-side summary:

• Env keys (server .env only — the deploy hook's rsync preserves

.env/var/): BILLING_ENABLED, STRIPE_SECRET_KEY (secret), STRIPE_PUBLISHABLE_KEY, STRIPE_WEBHOOK_SECRET (secret), STRIPE_API_BASE (keep default), STRIPE_PRICE_PRO, BILLING_GRACE_DAYS=3, MAX_LINKS_FREE=500. Secrets are masked in php bin/iya doctor output — verify after every .env edit.

• Migrations: 023_billing (tables stripe_subscriptions,

billing_events) — applied automatically by the deploy hook's bin/iya migrate; nothing to do manually.

• Webhook URL: https://abuseipdb.ru/billing/webhook — registered in the

Stripe dashboard with the signing secret copied into STRIPE_WEBHOOK_SECRET. The route's CSRF exemption is IN CODE (Csrf::SIGNATURE_EXEMPT_PATHS), never via env.

• Egress: php-fpm needs outbound HTTPS to api.stripe.com (443) for

checkout/portal session creation; inbound: Stripe must reach /billing/webhook (no allowlisting of Stripe IPs is configured — the HMAC signature is the authenticator).

• Restart: after any billing .env change, restart php-fpm85 (the

deploy hook does this; manual flips must too).

• Verification after go-live flip: docs/BILLING.md §7 steps 6–8

(surface checks + first test payment + 48 h monitoring of billing_events/audit_log).

• Honesty note: the live Stripe API was never exercised from the

sandbox (no internet) — drills prove the contract against a fake sink; the §7 rehearsal in TEST mode is mandatory before live keys.

_Last updated: 2026-10-23 (s38 Stripe S5 billing runbook). Companion files: deploy/ (compose, Caddyfile, systemd units, backup/restore, qwen-access/), .github/workflows/ci.yml (deploy-config job), scripts/_deploy_check.py (sandbox drill), docs/QWEN-ACCESS.md (continuity endpoint), docs/BILLING.md (billing runbook)._