Skip to content

11 — Backend & Infrastructure Deployment (smart-work-permit-api)

Read 10-DEPLOYMENT-OVERVIEW.md first. This file is the runbook for the single VM that hosts the API, Postgres, Redis, MinIO, nginx, and the Cloudflare Tunnel.

Substitute <domain> and <owner> (GitHub org/user) throughout.


1. Prerequisites

  • A VM: 2 vCPU / 4 GB RAM minimum, Ubuntu 22.04 LTS, 40 GB+ disk. Postgres, Redis and MinIO share this box, so disk is the constraint that bites first — permit photo evidence accumulates.
  • A domain on Cloudflare (Registrar or nameservers delegated). Zone SSL/TLS mode set to Full.
  • GitHub repo with Actions enabled and GHCR available.

2. VM baseline (once)

bash
sudo apt-get update && sudo apt-get upgrade -y
curl -fsSL https://get.docker.com | sh

sudo useradd -m -s /bin/bash deploy
sudo usermod -aG docker deploy
sudo mkdir -p /opt/swp/backups && sudo chown -R deploy:deploy /opt/swp

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw enable

No inbound web ports. The tunnel is outbound-only.

CI deploy key — generate on your workstation, not the VM:

bash
ssh-keygen -t ed25519 -f swp_deploy_key -N ""
ssh-copy-id -i swp_deploy_key.pub deploy@<vm-host>

Public half → /home/deploy/.ssh/authorized_keys. Private half → GitHub secret VM_SSH_KEY.

3. Dockerfile (repo root)

dockerfile
FROM oven/bun:1.3.13-slim AS deps
WORKDIR /app
COPY package.json bun.lock ./
RUN bun install --frozen-lockfile

FROM oven/bun:1.3.13-slim AS build
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN bunx prisma generate

FROM oven/bun:1.3.13-slim AS runtime
WORKDIR /app
ENV NODE_ENV=production TZ=UTC
RUN apt-get update && apt-get install -y --no-install-recommends openssl curl \
 && rm -rf /var/lib/apt/lists/*
COPY --from=build /app ./
EXPOSE 3000
CMD ["sh", "-c", "bunx prisma migrate deploy && bun run src/index.ts"]

openssl is required by Prisma's query engine on slim images. curl backs the healthcheck.

4. Compose file — /opt/swp/docker-compose.prod.yml

Keep a copy in the repo at deploy/docker-compose.prod.yml and scp it to the VM on change.

yaml
services:
  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: ${PG_USER}
      POSTGRES_PASSWORD: ${PG_PASSWORD}
      POSTGRES_DB: ${PG_DB}
      TZ: UTC
    volumes:
      - pgdata:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${PG_USER}"]
      interval: 10s
      retries: 5

  redis:
    image: redis:7-alpine
    restart: unless-stopped
    command: redis-server --appendonly yes --requirepass ${REDIS_PASSWORD}
    volumes:
      - redisdata:/data
    healthcheck:
      test: ["CMD", "redis-cli", "-a", "${REDIS_PASSWORD}", "ping"]
      interval: 15s
      retries: 5

  minio:
    image: minio/minio:RELEASE.2025-04-22T22-12-26Z   # PIN. never :latest
    restart: unless-stopped
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: ${MINIO_ROOT_USER}
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
      MINIO_BROWSER_REDIRECT_URL: https://storage.<domain>
      TZ: UTC
    volumes:
      - miniodata:/data
    healthcheck:
      test: ["CMD", "mc", "ready", "local"]
      interval: 15s
      retries: 5

  minio-init:
    image: minio/mc:latest
    restart: "no"
    depends_on:
      minio: { condition: service_healthy }
    environment:
      MINIO_ROOT_USER: ${MINIO_ROOT_USER}
      MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
    entrypoint: >
      /bin/sh -c "
      mc alias set local http://minio:9000 $$MINIO_ROOT_USER $$MINIO_ROOT_PASSWORD &&
      mc mb --ignore-existing local/swp-permits &&
      mc anonymous set none local/swp-permits &&
      echo bucket-ready"

  api:
    image: ghcr.io/<owner>/smart-work-permit-api:${IMAGE_TAG:-latest}
    restart: unless-stopped
    env_file: .env
    depends_on:
      postgres: { condition: service_healthy }
      redis:    { condition: service_healthy }
      minio:    { condition: service_healthy }
    healthcheck:
      test: ["CMD", "curl", "-fsS", "http://localhost:3000/"]
      interval: 15s
      timeout: 5s
      retries: 5
      start_period: 40s

  nginx:
    image: nginx:1.27-alpine
    restart: unless-stopped
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      api: { condition: service_healthy }

  cloudflared:
    image: cloudflare/cloudflared:latest
    restart: unless-stopped
    command: tunnel --no-autoupdate run --token ${TUNNEL_TOKEN}
    depends_on: [nginx, minio]

volumes:
  pgdata:
  redisdata:
  miniodata:

The healthcheck hits GET / — the bare health route outside /api (04-api-contract.md §1). Do not point it at /api/v1/...; every route there is guarded and returns 401.

5. nginx — /opt/swp/nginx.conf

nginx
upstream api_upstream { server api:3000; keepalive 32; }

server {
  listen 80;
  client_max_body_size 25M;      # photo evidence via POST /api/v1/upload

  proxy_set_header X-Forwarded-Proto https;   # Cloudflare terminated TLS
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header Host $host;

  location / {
    proxy_pass http://api_upstream;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_read_timeout 60s;
  }
}

No CORS headers here. Elysia owns them.

6. /opt/swp/.env — create by hand, chmod 600, never commit

bash
NODE_ENV=production
TZ=UTC
PORT=3000

PG_USER=swp
PG_PASSWORD=<openssl rand -base64 24>
PG_DB=swp
DATABASE_URL=postgresql://swp:<PG_PASSWORD>@postgres:5432/swp?schema=public

REDIS_PASSWORD=<openssl rand -base64 24>
REDIS_URL=redis://:<REDIS_PASSWORD>@redis:6379

BETTER_AUTH_URL=https://api.<domain>
BETTER_AUTH_SECRET=<openssl rand -base64 32>
COOKIE_DOMAIN=.<domain>
CORS_ORIGIN=https://app.<domain>,https://safety.<domain>
USER_DISABLE_SIGNUP=true

MINIO_ROOT_USER=swpadmin
MINIO_ROOT_PASSWORD=<openssl rand -base64 32>
S3_ENDPOINT=http://minio:9000
S3_PUBLIC_ENDPOINT=https://storage.<domain>
S3_REGION=us-east-1
S3_BUCKET=swp-permits
S3_ACCESS_KEY=swpadmin
S3_SECRET_KEY=<MINIO_ROOT_PASSWORD>
S3_FORCE_PATH_STYLE=true

TUNNEL_TOKEN=<from step 7>
IMAGE_TAG=latest

COOKIE_DOMAIN=.<domain> is the single line that lets both frontend subdomains share the session.

Feature flags — every one of them defaults to OFF

Two optional variables tighten behaviour. Neither switches itself on, and that is deliberate: PERMIT_POSITION_REQUIRED once locked every contractor out of submitting, so a rule that can refuse work must be an explicit deployment decision. Both are read per request and compared case-insensitively against TRUE.

VariableUnset (default)Set to TRUE
CERT_TYPE_REQUIREDany unexpired certificate satisfies any permit typethe permit's type requires its matching certType at both the submit check and the entrant scan
PPE_REQUIREDa permit may be submitted with no PPE declared400 PPE_REQUIRED when a permit declares no PPE at submit

Removed on 2026-09-11 (round 4, ticket 106), with Area: PERMIT_AREA_REQUIRED and AREA_VISIBILITY_SCOPED. Nothing reads them any more — if either is set in /opt/esw/.env, delete it rather than leaving a switch that looks live and is not.

The pin requirement at submit is not a flag. PERMIT_POSITION_REQUIRED switches on by data: once safety has an active pin on an active plan, every submit needs a pin. Nothing in the environment turns it off, so do not activate a plan and place pins on production until contractors are expected to pick one.

Three server-owned safety numbers are compiled in, not environment variables — deliberately, so a threshold cannot be changed without a code review: the 30-minute Fire Watch and the gas re-test interval (120 minutes) with its 30-minute grace, in src/libs/config/permit.config.ts; and, since round 4, the 12-hour cap on starting an inspector visit from history (HISTORY_START_WINDOW_CAP_HOURS, src/modules/permit/lib/work-window.util.ts). That window is the whole safety argument for the history start, which is why it is not configurable.

The API runs its own scheduler

PermitExpiryCronPlugin ticks once a minute inside the API process and drives two sweeps: permit expiry (with the Fire Watch grace alert) and the gas-reading overdue/escalation sweep. There is no droplet crontab entry for either.

This is a second reason to keep api at one replica. The migration race is the first; a second replica would also double every notification these sweeps send.

MinIO client details that will bite

forcePathStyle: true is mandatory. MinIO serves endpoint/bucket/key; the AWS SDK v3 defaults to virtual-host style (bucket.endpoint/key) and will generate URLs that resolve to nothing.

Presign against the public endpoint. The host is part of the SigV4 canonical request. If you sign with http://minio:9000 and then string-replace the host afterwards, the signature is invalid. Configure the presigning client with S3_PUBLIC_ENDPOINT; use S3_ENDPOINT only for server-side put/delete inside the Compose network.

7. Cloudflare Tunnel

bash
cloudflared tunnel login
cloudflared tunnel create swp

Dashboard → Zero Trust → Networks → Tunnels → swp → Public Hostname, add two:

HostnameService
api.<domain>HTTPnginx:80
storage.<domain>HTTPminio:9000

Copy the connector token into TUNNEL_TOKEN. Port 9001 (MinIO console) is not published — reach it via ssh -L 9001:localhost:9001 deploy@<vm-host>, or put it behind Cloudflare Access with an email policy if it must be remote.

8. Bucket CORS

Only needed if the browser talks to storage.<domain> directly (presigned PUT, or <img> loading a presigned GET). If every upload goes through POST /api/v1/upload and every read through GET /api/v1/file?filePath=, the request is same-origin to the API and you can skip this — check which your implementation actually does before assuming.

json
{"CORSRules":[{
  "AllowedOrigins":["https://app.<domain>","https://safety.<domain>"],
  "AllowedMethods":["GET","PUT","HEAD"],
  "AllowedHeaders":["*"],
  "ExposeHeaders":["ETag"],
  "MaxAgeSeconds":3000
}]}
bash
docker compose exec minio mc alias set local http://minio:9000 "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD"
docker compose exec minio mc cors set local/swp-permits /tmp/cors.json

9. First bring-up

bash
cd /opt/swp
docker login ghcr.io -u <owner>      # PAT with read:packages
docker compose -f docker-compose.prod.yml up -d --wait
docker compose -f docker-compose.prod.yml logs -f api

Migrations run automatically on api start. Then seed the first safety officer — self-registration is disabled (USER_DISABLE_SIGNUP), and contractor accounts are provisioned via POST /api/v1/users by a safety officer, so somebody has to exist first:

bash
docker compose exec api bun run src/scripts/seed-admin.ts   # or your repo's equivalent

10. CI/CD — .github/workflows/deploy.yml

yaml
name: Deploy API
on:
  push: { branches: [main] }
  workflow_dispatch:

concurrency: { group: api-deploy, cancel-in-progress: false }

env:
  IMAGE: ghcr.io/${{ github.repository_owner }}/smart-work-permit-api

jobs:
  test:
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16-alpine
        env: { POSTGRES_PASSWORD: test, POSTGRES_DB: test }
        options: --health-cmd pg_isready --health-interval 10s --health-retries 5
        ports: ['5432:5432']
      redis:
        image: redis:7-alpine
        ports: ['6379:6379']
    env:
      DATABASE_URL: postgresql://postgres:test@localhost:5432/test
      REDIS_URL: redis://localhost:6379
      TZ: UTC
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v2
        with: { bun-version: 1.3.13 }
      - run: bun install --frozen-lockfile
      - run: bunx prisma migrate deploy
      - run: bun test

  build:
    needs: test
    runs-on: ubuntu-latest
    permissions: { contents: read, packages: write }
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - uses: docker/build-push-action@v6
        with:
          push: true
          tags: ${{ env.IMAGE }}:${{ github.sha }},${{ env.IMAGE }}:latest
          cache-from: type=gha
          cache-to: type=gha,mode=max

  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: appleboy/ssh-action@v1
        with:
          host: ${{ secrets.VM_HOST }}
          username: deploy
          key: ${{ secrets.VM_SSH_KEY }}
          script: |
            set -euo pipefail
            cd /opt/swp
            echo "${{ secrets.GHCR_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
            export IMAGE_TAG=${{ github.sha }}
            docker compose -f docker-compose.prod.yml pull api
            docker compose -f docker-compose.prod.yml up -d --wait api nginx
            docker image prune -f

--wait blocks on the healthcheck, so a container that dies on a bad migration fails the job instead of reporting a green deploy over a broken API.

Rollback is export IMAGE_TAG=<previous-sha> and re-run up -d --wait. Note this rolls back code only — a migration that has already applied is not undone. Forward-only migrations, always.

11. Publish the contract — .github/workflows/contract.yml

yaml
name: Publish contract
on:
  push:
    branches: [main]
    paths: ['prisma/**', 'src/modules/**', 'src/routes/**']
jobs:
  dump:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: oven-sh/setup-bun@v2
      - run: bun install --frozen-lockfile
      - run: ./scripts/dump-openapi.sh
      - uses: actions/upload-artifact@v4
        with: { name: openapi, path: docs/openapi.json }

Both frontends consume this. See §4 of the overview for the ordering rule.

12. Verification

bash
curl -i https://api.<domain>/                        # bare object, 200
curl -i https://api.<domain>/api/v1/permits          # 401 UNAUTHENTICATED
curl -i -X POST https://api.<domain>/api/v1/auth/user/public/login \
  -H 'content-type: application/json' -d '{"email":"...","password":"..."}'

The login response must carry:

Set-Cookie: __Secure-better-auth.session_token=…; Domain=.<domain>; Path=/; SameSite=Lax; Secure

If the __Secure- prefix is missing, NODE_ENV isn't production. If Domain is missing, both frontends will look logged-out. Then run the repo's scripts/smoke-api.mjs against the live host — it asserts envelope, pagination, error-body, and permit-detail shapes.

Finally, confirm timezone end to end: create a permit, mark a hot-work permit complete, and check that fireWatch.remainingSeconds counts down from ~1800 and not from a 7-hour offset.

13. Backups

/opt/swp/backup.sh, chmod +x:

bash
#!/bin/bash
set -euo pipefail
D=$(date +%F)
cd /opt/swp
docker compose -f docker-compose.prod.yml exec -T postgres \
  pg_dump -U "$PG_USER" "$PG_DB" | gzip > "backups/db-$D.sql.gz"
docker compose -f docker-compose.prod.yml exec -T minio \
  mc mirror --overwrite local/swp-permits /data-backup/"$D"
find backups/ -mtime +14 -delete

0 3 * * * /opt/swp/backup.sh >> /opt/swp/backups/cron.log 2>&1

Mount a host path to /data-backup on the minio service, and push a weekly copy off-box (rsync to another machine, or mc mirror to an external S3 target). The audit log is hash-chained and append-only, which detects tampering but does nothing about a lost disk. A facility with a compliance audit trail should not have its only copy of permit photo evidence on one volume.

Test a restore before go-live. An untested backup is a guess.

14. Two standing constraints

Do not scale api to two replicas as-is. prisma migrate deploy runs on container start; two containers would race the same migration. If you need a second replica, move migrations into a one-shot Compose service or a pre-deploy SSH step first.

MinIO is AGPL v3. For an internal single-facility deployment this is normally fine, but if the platform is sold or hosted for other companies' facilities, have someone review the terms before go-live. MinIO's community distribution and console have changed meaningfully in recent releases and this document's information is not current — pin the image tag (as the Compose file does) and verify current terms on their site. This is not legal advice.