Environment_Infrastructure/roadmap/test-env/07-deploy-pipeline-update.md

07 — Deploy Pipeline Update (Test)

Context

• File: .gitea/workflows/deploy-test.yml
• Changes:
  1. Remove manual scp STAR.iklim.co.full.crt steps (SWAG now owns cert lifecycle).
  2. Add SWAG host directories preparation (dns-conf, nginx site-confs).
  3. Add cert bootstrap step: on first deploy, wait for SWAG to obtain cert, then copy to /opt/iklimco/ssl/ so Vault can start.
  4. Ensure GODADDY_KEY and GODADDY_SECRET are available from .env.secrets.swag.

Step 1 — Update Initialize Servers step

Remove the two scp lines that copy the TLS cert files:

# DELETE these two lines from the "Initialize Servers" step:
          scp -P 23 ${{ vars.STORAGEBOX_USER }}@${{ vars.STORAGEBOX_USER }}.your-storagebox.de:test/app/iklim.co/ssl/STAR.iklim.co.full.crt ./STAR.iklim.co.full.crt
          scp -P 23 ${{ vars.STORAGEBOX_USER }}@${{ vars.STORAGEBOX_USER }}.your-storagebox.de:test/app/iklim.co/ssl/STAR.iklim.co_key.pem ./STAR.iklim.co_key.pem

Also remove any references to STAR.iklim.co.full.crt and STAR.iklim.co_key.pem in the Prepare Init Files step's sudo cp commands:

# DELETE or make conditional:
          sudo cp STAR.iklim.co.full.crt STAR.iklim.co_key.pem /opt/iklimco/ssl/ 2>/dev/null || true

Step 2 — Add Prepare SWAG Directories step

Insert this step before Deploy Swarm Stack:

      - name: Prepare SWAG Directories
        run: |
          set -a; . ./.env; . ./.env.secrets.swag; set +a

          docker run --rm -v /opt/iklimco/swag:/output alpine \
            mkdir -p /output/dns-conf /output/site-confs

          envsubst < template/swag/dns-conf/godaddy.ini.tpl | docker run --rm -i \
            -v /opt/iklimco/swag/dns-conf:/output \
            alpine sh -c "cat > /output/godaddy.ini && chmod 600 /output/godaddy.ini"
          echo "✅ godaddy.ini written"

          # RESTRICTED_IPS → multi-line allow block (indented 8 spaces per nginx style)
          export RESTRICTED_IPS_BLOCK="$(echo "$RESTRICTED_IPS" | tr ',' '\n' | sed 's|.*|        allow &;|')"

          # Explicit var list prevents nginx $upstream_* from being substituted by envsubst
          SWAG_VARS='${API_SUBDOMAIN}${APIGW_SUBDOMAIN}${GRAFANA_SUBDOMAIN}${RABBITMQ_SUBDOMAIN}${RESTRICTED_IPS_BLOCK}'
          for tpl in template/swag/site-confs/*.conf.tpl; do
            fname=$(basename "${tpl%.tpl}")
            envsubst "$SWAG_VARS" < "$tpl" | docker run --rm -i \
              -v /opt/iklimco/swag/site-confs:/output \
              alpine sh -c "cat > /output/${fname}"
            echo "✅ ${fname}"
          done

          cat template/swag/site-confs/default.conf | docker run --rm -i \
            -v /opt/iklimco/swag/site-confs:/output \
            alpine sh -c "cat > /output/default.conf"

          echo "✅ SWAG directories ready"

          SWAG_CTR=$(docker ps -q -f name=iklimco_swag 2>/dev/null | head -1)
          if [ -n "$SWAG_CTR" ]; then
            docker exec "$SWAG_CTR" nginx -t && docker exec "$SWAG_CTR" nginx -s reload
            echo "✅ SWAG nginx reloaded"
          fi
        working-directory: /workspace/iklim.co

GODADDY_KEY ve GODADDY_SECRET .env.secrets.swag içinde olmalı (bkz. step 02). API_SUBDOMAIN, APIGW_SUBDOMAIN vb. .env içinde olmalı (bkz. step 04). Dosyalar docker run alpine ile yazılır — host'a sudo erişimi gerekmez. RESTRICTED_IPS comma-separated CIDR listesinden (env-test/.env) her satıra allow direktifi üretilir.

Step 3 — Add Bootstrap SWAG Certificate step

Insert this step after Deploy Swarm Stack and before any step that depends on Vault being accessible (e.g., Provision Vault AppRole IDs):

      - name: Bootstrap SWAG Certificate
        run: |
          echo "Waiting for SWAG container to start..."
          SWAG_CTR=""
          for i in $(seq 1 24); do
            SWAG_CTR=$(docker ps -q -f name=iklimco_swag 2>/dev/null | head -1)
            [ -n "$SWAG_CTR" ] && break
            sleep 10
          done

          if [ -z "$SWAG_CTR" ]; then
            echo "❌ SWAG container did not start in time"
            exit 1
          fi

          CERT_PATH="/config/etc/letsencrypt/live/iklim.co/fullchain.pem"
          echo "Waiting for SWAG to obtain Let's Encrypt cert (up to 10 min)..."
          for i in $(seq 1 20); do
            if docker exec "$SWAG_CTR" test -f "$CERT_PATH" 2>/dev/null; then
              echo "✅ Cert obtained by SWAG"
              break
            fi
            echo "  attempt $i/20 — waiting 30s..."
            sleep 30
          done

          if ! docker exec "$SWAG_CTR" test -f "$CERT_PATH" 2>/dev/null; then
            echo "❌ SWAG did not obtain cert in time. Check logs:"
            docker service logs iklimco_swag --tail 50
            exit 1
          fi

          # Copy cert to host for Vault bootstrap
          sudo mkdir -p /opt/iklimco/ssl
          docker exec "$SWAG_CTR" cat "$CERT_PATH" | \
            sudo tee /opt/iklimco/ssl/STAR.iklim.co.full.crt > /dev/null
          docker exec "$SWAG_CTR" cat "/config/etc/letsencrypt/live/iklim.co/privkey.pem" | \
            sudo tee /opt/iklimco/ssl/STAR.iklim.co_key.pem > /dev/null
          echo "✅ Cert bootstrapped to /opt/iklimco/ssl/"
        working-directory: /workspace/iklim.co

First deploy only: SWAG contacts Let's Encrypt via GoDaddy DNS challenge. This step waits up to 10 minutes. On subsequent deploys the cert is already in swag-vl (persisted volume) and SWAG starts immediately — wait loop exits fast.

Step 4 — Re-order steps

Final step order in the pipeline:

1. Checkout Branch
2. Prepare Folders
3. Set up SSH Key
4. Update Apt / Install Tools
5. Fetch Service Secret Files
6. Initialize Docker Swarm
7. Initialize Servers
8. Upload Updated Secrets to Storagebox
9. Provision Vault AppRole IDs and Docker Secrets
10. Upload Updated Env to Storagebox
11. Prepare Init Files ← sudo cp STAR.iklim.co.*.crt lines removed
12. Stop Docker Compose Services
13. Docker Login to Harbor
14. Prepare SWAG Directories ← NEW
15. Bootstrap Vault TLS Placeholder
16. Deploy Swarm Stack
17. Wait for etcd ← zaten pipeline'da mevcut; etcd health endpoint'i kontrol eder
18. Run APISIX Init ← NEW (SPRING_PROFILES_ACTIVE=test)
19. Bootstrap SWAG Certificate ← NEW
20. Run Database Init Scripts ← NEW
21. Review Environment

Steps 8 (Provision Vault) runs before SWAG because it creates Docker secrets and AppRole IDs — Vault must be reachable for this. On re-deploys, Vault is already running with the previous cert. On first deploy, step 16 handles the cert wait before any further Vault interaction is needed post-deploy.

If Vault provisioning (step 8) fails on first deploy because Vault has no cert yet, move step 16 before step 8. Adjust based on observed behavior.

Notes

• .env must contain the subdomain env vars added in step 04. Add them to storagebox test/secrets/iklim.co/.env before the first deploy.
• RESTRICTED_IP_1 and RESTRICTED_IP_2 are hardcoded in the pipeline step above. Move to .env if they change often.
• Precipitation service expects its image-data bind mount at /mnt/storagebox/precipitation/images. This directory is provisioned by the test Ansible bootstrap through storagebox_managed_directories; do not rely on the deploy pipeline to create it.