# 07 — Vault Raft Stack and Bootstrap Automation (Prod)

## Context

Production Vault is a 3-node Raft cluster, but it is no longer initialized through a manual post-deploy runbook.

Current references:

- Setup source: `../../setup/09-prod-runner-ha-and-swarm.md`
- Stack file: root `docker-stack-vault.yml`
- Bootstrap script: root `init/vault/vault-bootstrap.sh`
- Template: root `init/vault/vault-template-v2.json`

## Current Model

Vault is deployed separately from `docker-stack-infra_db-prod.yml`.

The Vault stack uses:

- 3 replicas, one per service node when placement allows it.
- Docker volumes such as `vault-data-vl` and `vault-logs-vl`.
- `/opt/iklimco/ssl:/vault/certs:ro` for TLS certificates.
- `iklimco-net` as an external overlay network.
- `vault_unseal_key` as a Docker secret.

The production workflow calls `init-infra-prod.sh`, which calls `init/vault/vault-bootstrap.sh`. The bootstrap script handles stack deploy, initialization, unseal key secret rotation, peer join, and peer unseal.

## Certificate Flow

Vault does not read TLS certificates directly from `/mnt/storagebox/ssl`.

The current flow is:

```text
SWAG renews certificate
cert-reloader copies renewed files to /mnt/storagebox/ssl
cert-distributor syncs certificate files to /opt/iklimco/ssl on service nodes
Vault reads /opt/iklimco/ssl through the /vault/certs mount
```

## Bootstrap Flow

Normal production bootstrap is automated:

1. Create or refresh the placeholder `vault_unseal_key` secret when needed.
2. Deploy `docker-stack-vault.yml`.
3. Initialize Vault with one key share and one threshold if it is not initialized.
4. Replace the placeholder `vault_unseal_key` secret with the real unseal key.
5. Unseal the leader.
6. Join peers to the Raft cluster.
7. Unseal peers.
8. Verify Raft peers and service health.

These operations belong to `vault-bootstrap.sh`, not to a manual operator checklist.

## Verification

Use the current setup verification flow:

```bash
docker service ps iklimco_vault
docker exec $(docker ps -q -f name=iklimco_vault | head -1) vault status
docker exec $(docker ps -q -f name=iklimco_vault | head -1) vault operator raft list-peers
```

Expected state:

- Vault service has 3 running tasks.
- `vault status` reports `Sealed false`.
- Raft list shows one leader and two followers.

## Historical / Superseded by Setup

The previous manual procedure is superseded:

- Deploying Vault through `docker-stack-infra.yml` + `docker-stack-infra.prod.yml`.
- Creating `/opt/iklimco/vault/data` host-path directories on each app node.
- Running `vault operator init` manually.
- Manually copying/storing unseal keys.
- Manually running `vault operator raft join` on peers.
- Manually unsealing each peer after join.

Keep those notes only as historical context. For current prod, use `docker-stack-vault.yml` and `init/vault/vault-bootstrap.sh`.