Murat ÖZDEMİR 5ddba7eba4 docs: update production roadmap for HA Vault and shared storage

- Refactor production setup documentation to reflect a 3-node Vault Raft cluster starting from launch.
- Update all paths to use StorageBox mounts for shared state (SWAG config, TLS certs, Monitoring data).
- Switch Nginx configuration convention from proxy-confs to site-confs to align with SWAG's auto-include behavior.
- Standardize TLS private key extensions to .pem.
- Update node failover and recovery facts to include monitoring services.
- Align deployment pipeline instructions with the latest environment variable-driven approach.

2026-05-16 16:18:21 +03:00

1.9 KiB

Raw Blame History

06 — cert-reloader Sidecar Service (Prod)

Context

Service definition is identical to test (see test-env-setup/06-cert-reloader.md). In prod, Vault runs as a 3-node Raft cluster; cert distribution is handled via the StorageBox shared mount — no SSH required.

Prod flow (3-node Vault Raft)

SWAG renews cert → writes to SWAG_CONFIG_DIR (/mnt/storagebox/swag/config)
cert-reloader detects MD5 change
  → copies to /mnt/storagebox/ssl/  (shared across all app nodes)
  → docker service update --force iklimco_vault
Vault (3 replicas) restarts
  → each instance has /mnt/storagebox/ssl/ mounted → reads the new cert
  → healthcheck checks sealed status every 30 seconds
  → if sealed: reads vault_unseal_key Docker secret and auto-unseals

No SSH distribution, additional secrets, or cert-reloader script changes are needed.

Auto-unseal mechanism

The Vault healthcheck is already implemented in docker-stack-infra.yml:

healthcheck:
  test:
    - "CMD"
    - "sh"
    - "-c"
    - >-
      vault status -format=json 2>/dev/null | grep -q '"sealed":false' ||
      vault operator unseal $$(cat /run/secrets/vault_unseal_key 2>/dev/null)
  interval: 30s
  timeout: 10s
  start_period: 15s
  retries: 5

Each Vault container runs its own healthcheck independently — all 3 replicas unseal separately. The cert renewal → restart → auto-unseal chain requires no manual intervention.

Verification

docker service ps iklimco_cert-reloader
docker service logs iklimco_cert-reloader --tail 20

Expected: [cert-reloader] started, no error lines.

Confirm Vault cert is current after SWAG renewal:

# Check cert expiry on Vault's TLS endpoint from inside the overlay
docker exec $(docker ps -q -f name=iklimco_vault) \
  sh -c 'echo | openssl s_client -connect vault.iklim.co:8200 2>/dev/null \
         | openssl x509 -noout -dates'

notAfter should match the cert in /mnt/storagebox/ssl/STAR.iklim.co.full.crt.

1.9 KiB Raw Blame History