Murat ÖZDEMİR
67dc2986dd
- Anglicized setup and facts markdown file names for better consistency. - Updated 01-swarm-init-multinode.md to highlight Ansible automation of Swarm initialization and labeling. - Overhauled 03-infra-stack-changes.md to describe the single monolithic file strategy and reflect current Redis, RabbitMQ, and etcd cluster configurations. - Fixed minor overrides and typos in Patroni templates and Ansible bootstrap documents. - Restructured README and roadmap mapping to align with the renamed setup documents.
84 lines
2.8 KiB
Markdown
84 lines
2.8 KiB
Markdown
# 07 — Vault Raft Stack and Bootstrap Automation (Prod)
|
|
|
|
## Context
|
|
|
|
Production Vault is a 3-node Raft cluster, but it is no longer initialized through a manual post-deploy runbook.
|
|
|
|
Current references:
|
|
|
|
- Setup source: `../../setup/09-prod-runner-ha-and-swarm.md`
|
|
- Stack file: root `docker-stack-vault.yml`
|
|
- Bootstrap script: root `init/vault/vault-bootstrap.sh`
|
|
- Template: root `init/vault/vault-template-v2.json`
|
|
|
|
## Current Model
|
|
|
|
Vault is deployed separately from `docker-stack-infra_db-prod.yml`.
|
|
|
|
The Vault stack uses:
|
|
|
|
- 3 replicas, one per service node when placement allows it.
|
|
- Docker volumes such as `vault-data-vl` and `vault-logs-vl`.
|
|
- `/opt/iklimco/ssl:/vault/certs:ro` for TLS certificates.
|
|
- `iklimco-net` as an external overlay network.
|
|
- `vault_unseal_key` as a Docker secret.
|
|
|
|
The production workflow calls `init-infra-prod.sh`, which calls `init/vault/vault-bootstrap.sh`. The bootstrap script handles stack deploy, initialization, unseal key secret rotation, peer join, and peer unseal.
|
|
|
|
## Certificate Flow
|
|
|
|
Vault does not read TLS certificates directly from `/mnt/storagebox/ssl`.
|
|
|
|
The current flow is:
|
|
|
|
```text
|
|
SWAG renews certificate
|
|
cert-reloader copies renewed files to /mnt/storagebox/ssl
|
|
cert-distributor syncs certificate files to /opt/iklimco/ssl on service nodes
|
|
Vault reads /opt/iklimco/ssl through the /vault/certs mount
|
|
```
|
|
|
|
## Bootstrap Flow
|
|
|
|
Normal production bootstrap is automated:
|
|
|
|
1. Create or refresh the placeholder `vault_unseal_key` secret when needed.
|
|
2. Deploy `docker-stack-vault.yml`.
|
|
3. Initialize Vault with one key share and one threshold if it is not initialized.
|
|
4. Replace the placeholder `vault_unseal_key` secret with the real unseal key.
|
|
5. Unseal the leader.
|
|
6. Join peers to the Raft cluster.
|
|
7. Unseal peers.
|
|
8. Verify Raft peers and service health.
|
|
|
|
These operations belong to `vault-bootstrap.sh`, not to a manual operator checklist.
|
|
|
|
## Verification
|
|
|
|
Use the current setup verification flow:
|
|
|
|
```bash
|
|
docker service ps iklimco_vault
|
|
docker exec $(docker ps -q -f name=iklimco_vault | head -1) vault status
|
|
docker exec $(docker ps -q -f name=iklimco_vault | head -1) vault operator raft list-peers
|
|
```
|
|
|
|
Expected state:
|
|
|
|
- Vault service has 3 running tasks.
|
|
- `vault status` reports `Sealed false`.
|
|
- Raft list shows one leader and two followers.
|
|
|
|
## Historical / Superseded by Setup
|
|
|
|
The previous manual procedure is superseded:
|
|
|
|
- Deploying Vault through `docker-stack-infra.yml` + `docker-stack-infra.prod.yml`.
|
|
- Creating `/opt/iklimco/vault/data` host-path directories on each app node.
|
|
- Running `vault operator init` manually.
|
|
- Manually copying/storing unseal keys.
|
|
- Manually running `vault operator raft join` on peers.
|
|
- Manually unsealing each peer after join.
|
|
|
|
Keep those notes only as historical context. For current prod, use `docker-stack-vault.yml` and `init/vault/vault-bootstrap.sh`.
|