From fd6a0b4f46980418114be41aa5cf93074582f3d2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Murat=20=C3=96ZDEM=C4=B0R?= Date: Sat, 16 May 2026 16:52:48 +0300 Subject: [PATCH] docs: fix roadmap inconsistencies between test-env and prod-env Corrects six documentation files to match the actual deployed pipeline behavior and align test/prod approaches where they share the same code. prod-env/02-godaddy-credentials.md - Step 1: correct secret file from .env.secrets.shared to .env.secrets.swag; add clarifying note that .env.secrets.shared holds AppRole/DB secrets and must not be used for GoDaddy credentials. - Step 4: document that GoDaddy A records are now managed automatically by the pipeline's 'Update DNS Records' step via the GoDaddy API; reference the Gitea variable PROD_FLOATING_IP that must be set once. prod-env/08-deploy-pipeline-update.md - Add Step 2 documenting the new 'Update DNS Records' pipeline step (GoDaddy API, idempotent check-before-update, requires jq and vars.PROD_FLOATING_IP). - Renumber subsequent steps 3-8 to accommodate the new step. - Fix DB hostnames in Step 7 (Run Database Init Scripts) from iklimco_postgresql/iklimco_mongodb to postgresql/mongodb, matching how Swarm overlay DNS resolves service names inside iklimco-net. - Update context block: correct DB hostname description, replace outdated storagebox path note with env-var approach, list new steps. - Update final step order to 24 steps including the DNS step and Release Deploy Lock; mark Wait for etcd as NEW. prod-env/09-verify.md - Insert check #2 for the precipitation image directory (/mnt/storagebox/precipitation/images) and iklimco_image-data volume bind mount, mirroring the equivalent check in test-env/08-verify.md. - Renumber all subsequent checks (3-12) to maintain sequential ordering. test-env/03-infra-stack-changes.md - Update SWAG service volume snippet: replace hardcoded paths (swag-vl:/config, /opt/iklimco/swag/dns-conf, /opt/iklimco/swag/site-confs) with env-var forms (${SWAG_CONFIG_DIR:-swag-vl}, ${SWAG_DNS_CONF_DIR:-...}, ${SWAG_SITE_CONFS_DIR:-...}) to match docker-stack-infra.yml. - Update cert-reloader volume snippet: replace swag-vl and /opt/iklimco/ssl with ${SWAG_CONFIG_DIR:-swag-vl} and ${SWAG_CERT_DIR:-/opt/iklimco/ssl}, enabling StorageBox override in prod without changing the base file. test-env/04-swag-nginx-configs.md - Replace RESTRICTED_IP_1/RESTRICTED_IP_2 individual env vars with RESTRICTED_IPS (comma-separated CIDR list) in the required-vars section, matching env-test/.env and the actual pipeline. - Update all three IP-restricted template examples (apigw, rabbitmq, grafana) from allow ${RESTRICTED_IP_1}; allow ${RESTRICTED_IP_2}; to ${RESTRICTED_IPS_BLOCK}, matching the actual .conf.tpl files in the repo. - Rewrite the deploy step section to match the real pipeline: docker run alpine for file writing, RESTRICTED_IPS_BLOCK generation via sed, and envsubst with explicit SWAG_VARS filter to protect nginx $upstream_* vars. test-env/07-deploy-pipeline-update.md - Step 2 (Prepare SWAG Directories): replace sudo-tee approach with the actual docker-run-alpine method used in deploy-test.yml; add nginx reload block; update notes to reflect RESTRICTED_IPS_BLOCK generation. - Step 4 (Re-order): correct step numbering to match actual pipeline (21 steps); mark 'Wait for etcd' as already present in pipeline rather than a new addition; add Bootstrap Vault TLS Placeholder which was missing from the documented order. --- roadmap/prod-env/02-godaddy-credentials.md | 25 ++-- roadmap/prod-env/08-deploy-pipeline-update.md | 108 ++++++++++++------ roadmap/prod-env/09-verify.md | 34 ++++-- roadmap/test-env/03-infra-stack-changes.md | 10 +- roadmap/test-env/04-swag-nginx-configs.md | 45 +++++--- roadmap/test-env/07-deploy-pipeline-update.md | 70 +++++++----- 6 files changed, 190 insertions(+), 102 deletions(-) diff --git a/roadmap/prod-env/02-godaddy-credentials.md b/roadmap/prod-env/02-godaddy-credentials.md index ea56a9f..b77adca 100644 --- a/roadmap/prod-env/02-godaddy-credentials.md +++ b/roadmap/prod-env/02-godaddy-credentials.md @@ -12,11 +12,11 @@ If credentials were shared in any chat log, Slack message, or email, **revoke th **Never commit credentials to the repository.** -## Step 1 — Add credentials to storagebox `.env.secrets.shared` (prod path) +## Step 1 — Add credentials to storagebox `.env.secrets.swag` (prod path) Open the file at storagebox path: ``` -prod/secrets/iklim.co/.env.secrets.shared +prod/secrets/iklim.co/.env.secrets.swag ``` Add: @@ -25,6 +25,9 @@ GODADDY_KEY= GODADDY_SECRET= ``` +> `.env.secrets.swag` contains SWAG/GoDaddy credentials only. +> `.env.secrets.shared` contains AppRole IDs, DB passwords, and other runtime secrets — do not mix. + ## Step 2 — Repo template file Same file as test: `swag/dns-conf/godaddy.ini.tpl` (already created in test step 02). @@ -41,17 +44,21 @@ envsubst < swag/dns-conf/godaddy.ini.tpl > "$SWAG_DNS_CONF_DIR/godaddy.ini" chmod 600 "$SWAG_DNS_CONF_DIR/godaddy.ini" ``` -## Step 4 — GoDaddy A records for prod subdomains +## Step 4 — GoDaddy A records for prod subdomains (handled by pipeline) -In GoDaddy DNS panel for `iklim.co`, add/update A records pointing to the **Floating IP** (`iklim-prod-app-fip`). -To get the Floating IP value: `terraform output prod_floating_ip` +The deploy pipeline's **Update DNS Records** step automatically manages A records via GoDaddy API. +It reads the Floating IP from the Gitea variable `vars.PROD_FLOATING_IP` — set this once in Gitea project settings. + +To get the Floating IP: `terraform output prod_floating_ip` | Record | Value | |--------|-------| -| `api` | `` | -| `apigw` | `` | -| `rabbitmq` | `` | -| `grafana` | `` | +| `api` | `vars.PROD_FLOATING_IP` | +| `apigw` | `vars.PROD_FLOATING_IP` | +| `rabbitmq` | `vars.PROD_FLOATING_IP` | +| `grafana` | `vars.PROD_FLOATING_IP` | + +Logic: for each record, pipeline queries the current value via GoDaddy API. If already correct, it skips. Otherwise it creates/updates the record. > The Floating IP is assigned to `iklim-app-01` (`06-prod-terraform-iaac.md` — `floating_ip.tf`). > If failover is needed, the Floating IP can be reassigned to another app node; DNS does not change. diff --git a/roadmap/prod-env/08-deploy-pipeline-update.md b/roadmap/prod-env/08-deploy-pipeline-update.md index dbcc652..026a230 100644 --- a/roadmap/prod-env/08-deploy-pipeline-update.md +++ b/roadmap/prod-env/08-deploy-pipeline-update.md @@ -6,8 +6,9 @@ adapted for prod paths and prod runner. - **Prod-specific differences from test:** - `SPRING_PROFILES_ACTIVE=prod` (not `test`) in Run APISIX Init - - DB hostnames use Swarm VIP prefixes: `iklimco_postgresql`, `iklimco_mongodb` - - Storagebox paths use `prod/` instead of `test/` + - DB hostnames: `postgresql`, `mongodb` (Swarm overlay DNS — same as test) + - Storagebox paths via env vars (`SWAG_CERT_DIR`, `SWAG_CONFIG_DIR`, vb.) instead of local host paths + - Extra steps: Update DNS Records (GoDaddy API), Wait for etcd ## Step 1 — Remove manual cert scp lines from `Initialize Servers` @@ -23,7 +24,43 @@ Also remove from `Prepare Init Files`: sudo cp STAR.iklim.co.full.crt STAR.iklim.co_key.pem /opt/iklimco/ssl/ ``` -## Step 2 — Add `Prepare SWAG Directories` step +## Step 2 — Add `Update DNS Records` step + +Insert **after** `Docker Login to Harbor` and **before** `Prepare SWAG Directories`. + +```yaml + - name: Update DNS Records + run: | + set -a; . ./.env; . ./.env.secrets.swag; set +a + FLOATING_IP="${{ vars.PROD_FLOATING_IP }}" + DOMAIN="iklim.co" + + for record in api apigw rabbitmq grafana; do + CURRENT=$(curl -s \ + -H "Authorization: sso-key ${GODADDY_KEY}:${GODADDY_SECRET}" \ + "https://api.godaddy.com/v1/domains/${DOMAIN}/records/A/${record}" \ + 2>/dev/null | jq -r '.[0].data // empty' 2>/dev/null || true) + + if [ "$CURRENT" = "$FLOATING_IP" ]; then + echo "✅ ${record}.${DOMAIN} → ${FLOATING_IP} (mevcut, atlanıyor)" + else + curl -sf -X PUT \ + -H "Authorization: sso-key ${GODADDY_KEY}:${GODADDY_SECRET}" \ + -H "Content-Type: application/json" \ + "https://api.godaddy.com/v1/domains/${DOMAIN}/records/A/${record}" \ + -d "[{\"data\":\"${FLOATING_IP}\",\"ttl\":600}]" + echo "✅ ${record}.${DOMAIN} → ${FLOATING_IP} (eklendi/güncellendi)" + fi + done + working-directory: /workspace/iklim.co +``` + +> `GODADDY_KEY` ve `GODADDY_SECRET` `.env.secrets.swag`'dan okunur. +> `PROD_FLOATING_IP` Gitea project variable olarak tanımlanmalı (`terraform output prod_floating_ip`). +> `jq` gereklidir — `Update Apt Repository` adımına eklenmiş olmalı: `apt-get install -y gettext tree jq`. +> Her deploy'da çalışır; mevcut ve doğru kayıtlar atlanır (idempotent). + +## Step 3 — Add `Prepare SWAG Directories` step Insert **before** `Bootstrap Vault TLS Placeholder`: @@ -67,7 +104,7 @@ Insert **before** `Bootstrap Vault TLS Placeholder`: > `.env` is sourced first so `API_SUBDOMAIN=api.iklim.co` (prod values) are used. > Ensure these vars are in `prod/secrets/iklim.co/.env.prod` on storagebox. -## Step 3 — Add `Wait for etcd` step +## Step 4 — Add `Wait for etcd` step Insert **after** `Deploy Swarm Stack` and **before** `Run APISIX Init`. APISIX reads its entire configuration from etcd; init script will fail silently if etcd is not ready. @@ -90,7 +127,7 @@ APISIX reads its entire configuration from etcd; init script will fail silently > **Note:** In prod, the standalone `etcd` service from `docker-stack-infra.yml` still runs (Docker Compose overlay files cannot remove services). APISIX currently uses this etcd; the Patroni etcd migration happens via `docker-stack-infra.prod.yml`. The `http://etcd:2379/health` check targets this standalone service and is correct for the current setup. -## Step 4 — Add `Run APISIX Init` step +## Step 5 — Add `Run APISIX Init` step Insert **after** `Wait for etcd` and **before** `Bootstrap SWAG Certificate`. @@ -115,7 +152,7 @@ Insert **after** `Wait for etcd` and **before** `Bootstrap SWAG Certificate`. > The init script is idempotent (PUT semantics); safe to re-run on subsequent deploys. > With `replicas: 3` in prod, all APISIX instances read the same etcd state — no per-replica init needed. -## Step 5 — Add `Bootstrap SWAG Certificate` step +## Step 6 — Add `Bootstrap SWAG Certificate` step Insert **after** `Run APISIX Init`: @@ -163,7 +200,7 @@ Insert **after** `Run APISIX Init`: working-directory: /workspace/iklim.co ``` -## Step 6 — Add `Run Database Init Scripts` step +## Step 7 — Add `Run Database Init Scripts` step Insert **after** `Bootstrap SWAG Certificate` and **before** `Review Environment`. @@ -176,7 +213,7 @@ Insert **after** `Bootstrap SWAG Certificate` and **before** `Review Environment until docker run --rm --network iklimco-net \ -e PGPASSWORD="${DATABASE_POSTGRES_ROOT_PASSWD}" \ postgis/postgis:17-3.5 \ - pg_isready -h iklimco_postgresql -U "${DATABASE_POSTGRES_ROOT_USER}" -q 2>/dev/null; do + pg_isready -h postgresql -U "${DATABASE_POSTGRES_ROOT_USER}" -q 2>/dev/null; do sleep 5 done for sql_file in $(ls ./init/postgresql/*.sql 2>/dev/null | sort); do @@ -184,31 +221,31 @@ Insert **after** `Bootstrap SWAG Certificate` and **before** `Review Environment docker run --rm -i --network iklimco-net \ -e PGPASSWORD="${DATABASE_POSTGRES_ROOT_PASSWD}" \ postgis/postgis:17-3.5 \ - psql -h iklimco_postgresql -U "${DATABASE_POSTGRES_ROOT_USER}" < "$sql_file" + psql -h postgresql -U "${DATABASE_POSTGRES_ROOT_USER}" < "$sql_file" done echo "⏳ Waiting for MongoDB..." until docker run --rm --network iklimco-net mongo:8 \ - mongosh "mongodb://${DATABASE_MONGODB_ROOT_USER}:${DATABASE_MONGODB_ROOT_PASSWD}@iklimco_mongodb/admin" \ + mongosh "mongodb://${DATABASE_MONGODB_ROOT_USER}:${DATABASE_MONGODB_ROOT_PASSWD}@mongodb/admin" \ --eval "db.runCommand({ping:1})" --quiet 2>/dev/null; do sleep 5 done for js_file in $(ls ./init/mongodb/*.js 2>/dev/null | sort); do echo "▶ $(basename "$js_file")" docker run --rm -i --network iklimco-net mongo:8 \ - mongosh "mongodb://${DATABASE_MONGODB_ROOT_USER}:${DATABASE_MONGODB_ROOT_PASSWD}@iklimco_mongodb/admin" \ + mongosh "mongodb://${DATABASE_MONGODB_ROOT_USER}:${DATABASE_MONGODB_ROOT_PASSWD}@mongodb/admin" \ --quiet < "$js_file" done echo "✅ Database init scripts completed" working-directory: /workspace/iklim.co ``` -> **Prod-specific:** DB hostnames are `iklimco_postgresql` and `iklimco_mongodb` (Swarm VIP service names). +> **Prod-specific:** DB hostnames are `postgresql` and `mongodb` (Swarm VIP service names). > Test pipeline uses `postgresql` / `mongodb` (unqualified aliases within the same stack). > SQL and JS files are generated by `Prepare Init Files` step via `init_postgresql` / `init_mongodb` functions in `common-functions.sh`. > Step is idempotent — scripts use `CREATE IF NOT EXISTS` / `createCollection` semantics. -## Step 7 — Ensure subdomain env vars are in prod `.env` +## Step 8 — Ensure subdomain env vars are in prod `.env` Add to `prod/secrets/iklim.co/.env.prod` on storagebox: @@ -221,24 +258,27 @@ GRAFANA_SUBDOMAIN=grafana.iklim.co ## Step 8 — Final step order for prod pipeline -1. Checkout Branch -2. Prepare Folders -3. Set up SSH Key and Add to known_hosts -4. Update Apt Repository and Install Required Tools -5. Fetch Service Secret Files -6. Initialize Servers ← cert scp lines removed -7. Upload Updated Secrets to Storagebox -8. Provision Vault AppRole IDs and Docker Secrets -9. Upload Updated Env to Storagebox -10. Prepare Init Files ← cert copy lines removed -11. Initialize Docker Swarm -12. Stop Docker Compose Services -13. Docker Login to Harbor -14. **Prepare SWAG Directories** ← NEW -15. Bootstrap Vault TLS Placeholder -16. Deploy Swarm Stack -17. **Wait for etcd** ← NEW -18. **Run APISIX Init** ← NEW (`SPRING_PROFILES_ACTIVE=prod`) -19. **Bootstrap SWAG Certificate** ← NEW -20. **Run Database Init Scripts** ← NEW (`iklimco_postgresql`, `iklimco_mongodb`) -21. Review Environment +1. Acquire Deploy Lock +2. Checkout Branch +3. Prepare Folders +4. Set up SSH Key and Add to known_hosts +5. Update Apt Repository and Install Required Tools (`gettext tree jq`) +6. Fetch Service Secret Files +7. Initialize Servers ← cert scp lines removed +8. Upload Updated Secrets to Storagebox +9. Provision Vault AppRole IDs and Docker Secrets +10. Upload Updated Env to Storagebox +11. Prepare Init Files ← cert copy lines removed +12. Initialize Docker Swarm +13. Stop Docker Compose Services +14. Docker Login to Harbor +15. **Update DNS Records** ← NEW (GoDaddy API, idempotent) +16. **Prepare SWAG Directories** ← NEW +17. Bootstrap Vault TLS Placeholder +18. Deploy Swarm Stack +19. **Wait for etcd** ← NEW +20. **Run APISIX Init** ← NEW (`SPRING_PROFILES_ACTIVE=prod`) +21. **Bootstrap SWAG Certificate** ← NEW +22. **Run Database Init Scripts** ← NEW (`postgresql`, `mongodb`) +23. Review Environment +24. Release Deploy Lock diff --git a/roadmap/prod-env/09-verify.md b/roadmap/prod-env/09-verify.md index aae4566..1df77bb 100644 --- a/roadmap/prod-env/09-verify.md +++ b/roadmap/prod-env/09-verify.md @@ -15,7 +15,21 @@ docker service ls --filter label=project=co.iklim ``` All services show `REPLICAS X/X` (target met). -## 2 — SWAG cert is valid +## 2 — Precipitation image directory exists + +```bash +ls -ld /mnt/storagebox/precipitation/images +``` + +Expected: directory exists. This must be created before `iklimco_precipitation-service` is deployed. + +```bash +docker volume inspect iklimco_image-data +``` + +Expected: `Options.device` is `/mnt/storagebox/precipitation/images`. + +## 3 — SWAG cert is valid ```bash docker exec $(docker ps -q -f name=iklimco_swag) certbot certificates @@ -29,14 +43,14 @@ echo | openssl s_client -connect api.iklim.co:443 -servername api.iklim.co 2>/de ``` Expected: `CN=*.iklim.co`, `notAfter` > 2026-07-15 (cert is Let's Encrypt, not expiring old one). -## 3 — Public API +## 4 — Public API ```bash curl -si https://api.iklim.co/health ``` HTTP 2xx, no TLS errors. -## 4 — IP restriction working +## 5 — IP restriction working From a non-whitelisted IP: ```bash @@ -53,7 +67,7 @@ curl -si https://apigw.iklim.co # HTTP 200 APISIX Dashboard curl -si https://rabbitmq.iklim.co # HTTP 200 RabbitMQ Management ``` -## 5 — Vault not reachable externally +## 6 — Vault not reachable externally ```bash # From outside — must fail @@ -68,14 +82,14 @@ docker exec $(docker ps -q -f name=iklimco_apisix | head -1) \ # Expected: {"sealed":false,...} ``` -## 6 — cert-reloader watching +## 7 — cert-reloader watching ```bash docker service logs iklimco_cert-reloader --tail 5 ``` Expected: `[cert-reloader] started`, no errors. -## 7 — No unexpected published ports +## 8 — No unexpected published ports ```bash docker service ls --format "{{.Name}}\t{{.Ports}}" \ @@ -83,7 +97,7 @@ docker service ls --format "{{.Name}}\t{{.Ports}}" \ ``` Only `iklimco_swag` should show `*:80->80/tcp, *:443->443/tcp`. -## 8 — DB nodes running correct services +## 9 — DB nodes running correct services ```bash # Patroni (PostgreSQL HA) stack @@ -104,21 +118,21 @@ docker service ps iklim-db_mongodb-03 All tasks should show node names matching `iklim-db-01`, `iklim-db-02`, or `iklim-db-03` with placement constraint `role=db`. -## 9 — APISIX replicas +## 10 — APISIX replicas ```bash docker service ps iklimco_apisix ``` Expected: 3 tasks, all `Running`, on different nodes. -## 10 — fail2ban active +## 11 — fail2ban active ```bash docker exec $(docker ps -q -f name=iklimco_swag) fail2ban-client status ``` Expected: multiple jails listed. -## 11 — Microservice health (post-deploy) +## 12 — Microservice health (post-deploy) After microservices are deployed (separate pipeline), verify via the public API: ```bash diff --git a/roadmap/test-env/03-infra-stack-changes.md b/roadmap/test-env/03-infra-stack-changes.md index e44e46b..f71a47e 100644 --- a/roadmap/test-env/03-infra-stack-changes.md +++ b/roadmap/test-env/03-infra-stack-changes.md @@ -45,9 +45,9 @@ Add after the `apisix-dashboard` service block: - EMAIL=muratozdemir@tarla.io - DNSPROPAGATION=90 volumes: - - swag-vl:/config - - /opt/iklimco/swag/dns-conf:/config/dns-conf:ro - - /opt/iklimco/swag/site-confs:/config/nginx/site-confs:ro + - ${SWAG_CONFIG_DIR:-swag-vl}:/config + - ${SWAG_DNS_CONF_DIR:-/opt/iklimco/swag/dns-conf}:/config/dns-conf + - ${SWAG_SITE_CONFS_DIR:-/opt/iklimco/swag/site-confs}:/config/nginx/site-confs ports: - target: 80 published: 80 @@ -78,8 +78,8 @@ Add after the `swag` service block: cert-reloader: image: docker:27-cli volumes: - - swag-vl:/swag-config:ro - - /opt/iklimco/ssl:/host-ssl + - ${SWAG_CONFIG_DIR:-swag-vl}:/swag-config:ro + - ${SWAG_CERT_DIR:-/opt/iklimco/ssl}:/host-ssl - /var/run/docker.sock:/var/run/docker.sock entrypoint: ["/bin/sh", "-c"] command: diff --git a/roadmap/test-env/04-swag-nginx-configs.md b/roadmap/test-env/04-swag-nginx-configs.md index b9b5304..3f4c9bd 100644 --- a/roadmap/test-env/04-swag-nginx-configs.md +++ b/roadmap/test-env/04-swag-nginx-configs.md @@ -12,8 +12,8 @@ API_SUBDOMAIN=api-test.iklim.co APIGW_SUBDOMAIN=apigw-test.iklim.co RABBITMQ_SUBDOMAIN=rabbitmq-test.iklim.co GRAFANA_SUBDOMAIN=grafana-test.iklim.co -RESTRICTED_IP_1=78.187.87.109 -RESTRICTED_IP_2=95.70.151.248 +# Comma-separated list of allowed CIDRs for IP-restricted subdomains +RESTRICTED_IPS="78.187.87.109/32,95.70.151.248/32" ``` ## Files to create @@ -78,8 +78,7 @@ server { client_max_body_size 0; location / { - allow ${RESTRICTED_IP_1}; - allow ${RESTRICTED_IP_2}; +${RESTRICTED_IPS_BLOCK} deny all; include /config/nginx/proxy.conf; @@ -92,6 +91,9 @@ server { } ``` +> `${RESTRICTED_IPS_BLOCK}` is generated at deploy time from `RESTRICTED_IPS` (comma-separated CIDRs) +> as multi-line `allow` directives with `/32` suffix. See `07-deploy-pipeline-update.md` for the pipeline step. + ### `swag/site-confs/rabbitmq.conf.tpl` RabbitMQ Management UI — IP restricted. @@ -107,8 +109,7 @@ server { client_max_body_size 0; location / { - allow ${RESTRICTED_IP_1}; - allow ${RESTRICTED_IP_2}; +${RESTRICTED_IPS_BLOCK} deny all; include /config/nginx/proxy.conf; @@ -136,8 +137,7 @@ server { client_max_body_size 0; location / { - allow ${RESTRICTED_IP_1}; - allow ${RESTRICTED_IP_2}; +${RESTRICTED_IPS_BLOCK} deny all; include /config/nginx/proxy.conf; @@ -153,20 +153,31 @@ server { ## Deploy step (handled by pipeline — see `07-deploy-pipeline-update.md`) ```bash -# Process templates and write to host -mkdir -p /opt/iklimco/swag/site-confs +set -a; . ./.env; . ./.env.secrets.swag; set +a -set -a; . ./.env; set +a -export RESTRICTED_IP_1="78.187.87.109" -export RESTRICTED_IP_2="95.70.151.248" +docker run --rm -v /opt/iklimco/swag:/output alpine \ + mkdir -p /output/dns-conf /output/site-confs +envsubst < swag/dns-conf/godaddy.ini.tpl | docker run --rm -i \ + -v /opt/iklimco/swag/dns-conf:/output \ + alpine sh -c "cat > /output/godaddy.ini && chmod 600 /output/godaddy.ini" + +# RESTRICTED_IPS → multi-line allow block (indented 8 spaces per nginx style) +export RESTRICTED_IPS_BLOCK="$(echo "$RESTRICTED_IPS" | tr ',' '\n' | sed 's|.*| allow &;|')" + +# Explicit var list prevents nginx $upstream_* from being substituted by envsubst +SWAG_VARS='${API_SUBDOMAIN}${APIGW_SUBDOMAIN}${GRAFANA_SUBDOMAIN}${RABBITMQ_SUBDOMAIN}${RESTRICTED_IPS_BLOCK}' for tpl in swag/site-confs/*.conf.tpl; do - out="/opt/iklimco/swag/site-confs/$(basename "${tpl%.tpl}")" - envsubst < "$tpl" > "$out" - echo "✅ $out" + fname=$(basename "${tpl%.tpl}") + envsubst "$SWAG_VARS" < "$tpl" | docker run --rm -i \ + -v /opt/iklimco/swag/site-confs:/output \ + alpine sh -c "cat > /output/${fname}" + echo "✅ ${fname}" done -cp swag/site-confs/default.conf /opt/iklimco/swag/site-confs/default.conf +cat swag/site-confs/default.conf | docker run --rm -i \ + -v /opt/iklimco/swag/site-confs:/output \ + alpine sh -c "cat > /output/default.conf" ``` ## Verification diff --git a/roadmap/test-env/07-deploy-pipeline-update.md b/roadmap/test-env/07-deploy-pipeline-update.md index 470baff..c8f9b50 100644 --- a/roadmap/test-env/07-deploy-pipeline-update.md +++ b/roadmap/test-env/07-deploy-pipeline-update.md @@ -36,31 +36,45 @@ Insert this step **before** `Deploy Swarm Stack`: run: | set -a; . ./.env; . ./.env.secrets.swag; set +a - # GoDaddy credentials file - sudo mkdir -p /opt/iklimco/swag/dns-conf - envsubst < swag/dns-conf/godaddy.ini.tpl | sudo tee /opt/iklimco/swag/dns-conf/godaddy.ini > /dev/null - sudo chmod 600 /opt/iklimco/swag/dns-conf/godaddy.ini + docker run --rm -v /opt/iklimco/swag:/output alpine \ + mkdir -p /output/dns-conf /output/site-confs + + envsubst < swag/dns-conf/godaddy.ini.tpl | docker run --rm -i \ + -v /opt/iklimco/swag/dns-conf:/output \ + alpine sh -c "cat > /output/godaddy.ini && chmod 600 /output/godaddy.ini" echo "✅ godaddy.ini written" - # Nginx site conf files - sudo mkdir -p /opt/iklimco/swag/site-confs - - export RESTRICTED_IP_1="78.187.87.109" - export RESTRICTED_IP_2="95.70.151.248" + # RESTRICTED_IPS → multi-line allow block (indented 8 spaces per nginx style) + export RESTRICTED_IPS_BLOCK="$(echo "$RESTRICTED_IPS" | tr ',' '\n' | sed 's|.*| allow &;|')" + # Explicit var list prevents nginx $upstream_* from being substituted by envsubst + SWAG_VARS='${API_SUBDOMAIN}${APIGW_SUBDOMAIN}${GRAFANA_SUBDOMAIN}${RABBITMQ_SUBDOMAIN}${RESTRICTED_IPS_BLOCK}' for tpl in swag/site-confs/*.conf.tpl; do - out="/opt/iklimco/swag/site-confs/$(basename "${tpl%.tpl}")" - envsubst < "$tpl" | sudo tee "$out" > /dev/null - echo "✅ $out" + fname=$(basename "${tpl%.tpl}") + envsubst "$SWAG_VARS" < "$tpl" | docker run --rm -i \ + -v /opt/iklimco/swag/site-confs:/output \ + alpine sh -c "cat > /output/${fname}" + echo "✅ ${fname}" done - sudo cp swag/site-confs/default.conf /opt/iklimco/swag/site-confs/default.conf + cat swag/site-confs/default.conf | docker run --rm -i \ + -v /opt/iklimco/swag/site-confs:/output \ + alpine sh -c "cat > /output/default.conf" + echo "✅ SWAG directories ready" + + SWAG_CTR=$(docker ps -q -f name=iklimco_swag 2>/dev/null | head -1) + if [ -n "$SWAG_CTR" ]; then + docker exec "$SWAG_CTR" nginx -t && docker exec "$SWAG_CTR" nginx -s reload + echo "✅ SWAG nginx reloaded" + fi working-directory: /workspace/iklim.co ``` -> `GODADDY_KEY` and `GODADDY_SECRET` must be present in `.env.secrets.swag` (see step 02). -> `API_SUBDOMAIN`, `APIGW_SUBDOMAIN`, etc. must be in `.env` (see step 04). +> `GODADDY_KEY` ve `GODADDY_SECRET` `.env.secrets.swag` içinde olmalı (bkz. step 02). +> `API_SUBDOMAIN`, `APIGW_SUBDOMAIN` vb. `.env` içinde olmalı (bkz. step 04). +> Dosyalar `docker run alpine` ile yazılır — host'a `sudo` erişimi gerekmez. +> `RESTRICTED_IPS` comma-separated CIDR listesinden (`env-test/.env`) her satıra `allow` direktifi üretilir. ## Step 3 — Add `Bootstrap SWAG Certificate` step @@ -123,20 +137,22 @@ Final step order in the pipeline: 3. Set up SSH Key 4. Update Apt / Install Tools 5. Fetch Service Secret Files -6. Initialize Servers -7. Upload Updated Secrets to Storagebox -8. Provision Vault AppRole IDs and Docker Secrets -9. Upload Updated Env to Storagebox -10. Prepare Init Files ← `sudo cp STAR.iklim.co.*.crt` lines removed -11. Initialize Docker Swarm +6. Initialize Docker Swarm +7. Initialize Servers +8. Upload Updated Secrets to Storagebox +9. Provision Vault AppRole IDs and Docker Secrets +10. Upload Updated Env to Storagebox +11. Prepare Init Files ← `sudo cp STAR.iklim.co.*.crt` lines removed 12. Stop Docker Compose Services 13. Docker Login to Harbor -14. **Prepare SWAG Directories** ← NEW -15. Deploy Swarm Stack -16. **Run APISIX Init** ← NEW (Swarm etcd volume'süz başlar; idempotent PUT) -17. **Bootstrap SWAG Certificate** ← NEW -18. **Run Database Init Scripts** ← NEW (önceki oturumda eklendi) -19. Review Environment +14. **Prepare SWAG Directories** ← NEW +15. Bootstrap Vault TLS Placeholder +16. Deploy Swarm Stack +17. **Wait for etcd** ← zaten pipeline'da mevcut; etcd health endpoint'i kontrol eder +18. **Run APISIX Init** ← NEW (`SPRING_PROFILES_ACTIVE=test`) +19. **Bootstrap SWAG Certificate** ← NEW +20. **Run Database Init Scripts** ← NEW +21. Review Environment > Steps 8 (Provision Vault) runs before SWAG because it creates Docker secrets and > AppRole IDs — Vault must be reachable for this. On re-deploys, Vault is already