Murat ÖZDEMİR 76f87aa2f9 Integrate DB nodes into Swarm and refine prod service deployment

- Database nodes now join the Docker Swarm as workers with `role=db` labels, allowing Swarm to manage their dedicated services.
- The `docker-stack-infra.yml` has been updated for production to focus solely on application-level infrastructure components.
- Dedicated database services (PostgreSQL, MongoDB, Patroni-etcd) are now explicitly deployed in separate Swarm stacks on `iklim-db-XX` nodes.
- Standardizes node naming conventions (`iklim-app-XX`, `iklim-db-XX`) across the production roadmap documentation.
- Clarifies that the `etcd` service within `docker-stack-infra.yml` is exclusively for APISIX configuration, distinct from Patroni's etcd cluster.

2026-05-11 14:53:21 +03:00

1.9 KiB

Raw Permalink Blame History

06 — cert-reloader Sidecar Service (Prod)

Context

Same service definition as test (see test-env-setup/06-cert-reloader.md). Prod-specific consideration: Vault is single-instance on the manager node (same as SWAG), so the cert copy to /opt/iklimco/ssl/ works without cross-node distribution.

When Vault is expanded to a 3-node Raft cluster (see 07-vault-raft-plan.md), the cert-reloader must be updated to distribute the cert to the other Vault nodes.

Current behavior (single-Vault prod)

SWAG (manager) renews cert → swag-vl
cert-reloader (manager) detects change → copies to /opt/iklimco/ssl/ → reloads Vault
Vault (manager) reads /opt/iklimco/ssl/ → serves new cert

No cross-node distribution needed.

Future behavior (3-node Vault Raft — see step 07)

When Vault runs on iklim-app-01, iklim-app-02, iklim-app-03:

cert-reloader detects cert change
→ copies cert to /opt/iklimco/ssl/ on iklim-app-01 (local)
→ SSH copy to iklim-app-02:/opt/iklimco/ssl/
→ SSH copy to iklim-app-03:/opt/iklimco/ssl/
→ docker service update --force iklimco_vault  (restarts all 3 replicas)

This requires:

An SSH key that cert-reloader can use to reach iklim-app-02 and iklim-app-03
That key mounted as a Docker secret into cert-reloader
Known_hosts for iklim-app-02 and iklim-app-03 pre-configured

Script update for this phase is tracked in 07-vault-raft-plan.md.

Verification

docker service ps iklimco_cert-reloader
docker service logs iklimco_cert-reloader --tail 20

Expected: [cert-reloader] started, no error lines.

Confirm Vault cert is current after SWAG renewal:

# Check cert expiry on Vault's TLS endpoint from inside the overlay
docker exec $(docker ps -q -f name=iklimco_vault) \
  sh -c 'echo | openssl s_client -connect vault.iklim.co:8200 2>/dev/null \
         | openssl x509 -noout -dates'

notAfter should match the cert in /opt/iklimco/ssl/STAR.iklim.co.full.crt.

1.9 KiB Raw Permalink Blame History