- Refactor production setup documentation to reflect a 3-node Vault Raft cluster starting from launch. - Update all paths to use StorageBox mounts for shared state (SWAG config, TLS certs, Monitoring data). - Switch Nginx configuration convention from proxy-confs to site-confs to align with SWAG's auto-include behavior. - Standardize TLS private key extensions to .pem. - Update node failover and recovery facts to include monitoring services. - Align deployment pipeline instructions with the latest environment variable-driven approach.
62 lines
1.9 KiB
Markdown
62 lines
1.9 KiB
Markdown
# 06 — cert-reloader Sidecar Service (Prod)
|
|
|
|
## Context
|
|
Service definition is identical to test (see `test-env-setup/06-cert-reloader.md`).
|
|
In prod, Vault runs as a 3-node Raft cluster; cert distribution is handled via the StorageBox shared mount — no SSH required.
|
|
|
|
## Prod flow (3-node Vault Raft)
|
|
|
|
```
|
|
SWAG renews cert → writes to SWAG_CONFIG_DIR (/mnt/storagebox/swag/config)
|
|
cert-reloader detects MD5 change
|
|
→ copies to /mnt/storagebox/ssl/ (shared across all app nodes)
|
|
→ docker service update --force iklimco_vault
|
|
Vault (3 replicas) restarts
|
|
→ each instance has /mnt/storagebox/ssl/ mounted → reads the new cert
|
|
→ healthcheck checks sealed status every 30 seconds
|
|
→ if sealed: reads vault_unseal_key Docker secret and auto-unseals
|
|
```
|
|
|
|
No SSH distribution, additional secrets, or cert-reloader script changes are needed.
|
|
|
|
## Auto-unseal mechanism
|
|
|
|
The Vault healthcheck is already implemented in `docker-stack-infra.yml`:
|
|
|
|
```yaml
|
|
healthcheck:
|
|
test:
|
|
- "CMD"
|
|
- "sh"
|
|
- "-c"
|
|
- >-
|
|
vault status -format=json 2>/dev/null | grep -q '"sealed":false' ||
|
|
vault operator unseal $$(cat /run/secrets/vault_unseal_key 2>/dev/null)
|
|
interval: 30s
|
|
timeout: 10s
|
|
start_period: 15s
|
|
retries: 5
|
|
```
|
|
|
|
Each Vault container runs its own healthcheck independently — all 3 replicas unseal separately.
|
|
The cert renewal → restart → auto-unseal chain requires no manual intervention.
|
|
|
|
## Verification
|
|
|
|
```bash
|
|
docker service ps iklimco_cert-reloader
|
|
docker service logs iklimco_cert-reloader --tail 20
|
|
```
|
|
|
|
Expected: `[cert-reloader] started`, no error lines.
|
|
|
|
Confirm Vault cert is current after SWAG renewal:
|
|
```bash
|
|
# Check cert expiry on Vault's TLS endpoint from inside the overlay
|
|
docker exec $(docker ps -q -f name=iklimco_vault) \
|
|
sh -c 'echo | openssl s_client -connect vault.iklim.co:8200 2>/dev/null \
|
|
| openssl x509 -noout -dates'
|
|
```
|
|
|
|
`notAfter` should match the cert in `/mnt/storagebox/ssl/STAR.iklim.co.full.crt`.
|