# 08 - Prod DB Cluster Kurulumu (Swarm) Bu aşamanın amacı üç DB node'unu Docker Swarm'a worker olarak eklemek, MongoDB replica set ve Patroni + etcd ile yönetilen PostgreSQL yüksek erişilebilirlik konfigürasyonunu yapmaktır. `07-prod-ansible-bootstrap.md` tüm DB node'larında tamamlanmış olmalıdır. ## Mimari ``` iklim-app-01/02/03 (Swarm manager'lar, 10.20.10.11/12/13) | |-- iklimco-net (overlay) | iklim-db-01 (Swarm worker, 10.20.20.11) mongodb-01 [rs0 member 0 — preferred primary] etcd-01 [etcd cluster member] patroni-01 [Patroni + PostgreSQL — ilk primary adayı] iklim-db-02 (Swarm worker, 10.20.20.12) mongodb-02 [rs0 member 1] etcd-02 [etcd cluster member] patroni-02 [Patroni + PostgreSQL — standby] iklim-db-03 (Swarm worker, 10.20.20.13) mongodb-03 [rs0 member 2] etcd-03 [etcd cluster member] patroni-03 [Patroni + PostgreSQL — standby] ``` DB container'ları birbirlerini overlay DNS adıyla değil, **Hetzner private IP üzerinden** tanıyor. Bu nedenle her servis portunu `host` modda yayımlar; replikasyon ve etcd trafiği doğrudan private network üzerinden gecer. Hetzner Cloud firewall ve prod `db` firewall zaten bu portlara izin vermektedir. ## 1. Firewall Güncellemesi `terraform/hetzner/prod/firewall.tf` dosyasında aşağıdaki kuralların mevcut olduğunu doğrula; eksik varsa ekle ve `terraform apply` çalıştır. `hcloud_firewall.swarm` içinde (DB subnet'ten Swarm portlarına): ```hcl rule { direction = "in" protocol = "tcp" port = "2377" source_ips = [local.db_subnet_cidr] description = "Docker Swarm control plane from DB subnet" } rule { direction = "in" protocol = "tcp" port = "7946" source_ips = [local.db_subnet_cidr] description = "Docker Swarm node discovery (TCP) from DB subnet" } rule { direction = "in" protocol = "udp" port = "7946" source_ips = [local.db_subnet_cidr] description = "Docker Swarm node discovery (UDP) from DB subnet" } rule { direction = "in" protocol = "udp" port = "4789" source_ips = [local.db_subnet_cidr] description = "Docker Swarm VXLAN overlay from DB subnet" } ``` `hcloud_firewall.db` içinde (app subnet'ten Swarm portlarına + overlay; DB subnet içi etcd/Patroni trafiği): ```hcl rule { direction = "in" protocol = "tcp" port = "2377" source_ips = [local.app_subnet_cidr] description = "Docker Swarm control plane from app subnet" } rule { direction = "in" protocol = "tcp" port = "7946" source_ips = [local.app_subnet_cidr] description = "Docker Swarm node discovery (TCP) from app subnet" } rule { direction = "in" protocol = "udp" port = "7946" source_ips = [local.app_subnet_cidr] description = "Docker Swarm node discovery (UDP) from app subnet" } rule { direction = "in" protocol = "udp" port = "4789" source_ips = [local.app_subnet_cidr] description = "Docker Swarm VXLAN overlay from app subnet" } rule { direction = "in" protocol = "tcp" port = "2379" source_ips = [local.db_subnet_cidr] description = "etcd client port within DB subnet" } rule { direction = "in" protocol = "tcp" port = "2380" source_ips = [local.db_subnet_cidr] description = "etcd peer port within DB subnet" } rule { direction = "in" protocol = "tcp" port = "8008" source_ips = [local.db_subnet_cidr] description = "Patroni REST API within DB subnet" } ``` ```bash cd terraform/hetzner/prod terraform plan terraform apply ``` ## 2. DB Node'larını Swarm'a Ekleme **Swarm manager'lardan birinde** (iklim-app-01) join token al: ```bash docker swarm join-token worker ``` **Her DB node'unda** (iklim-db-01, iklim-db-02, iklim-db-03): ```bash docker swarm join --token 10.20.10.11:2377 ``` **iklim-app-01 üzerinde** node'ları etiketle: ```bash docker node update --label-add role=db --label-add db-index=01 iklim-db-01 docker node update --label-add role=db --label-add db-index=02 iklim-db-02 docker node update --label-add role=db --label-add db-index=03 iklim-db-03 docker node ls ``` ## 3. StorageBox Dizin Yapısı Her DB node'unda (`/mnt/storagebox` zaten mount edilmiş olmalı): ```bash # iklim-db-01 üzerinde: mkdir -p /mnt/storagebox/prod/db/mongodb-01/{data,log,config} mkdir -p /mnt/storagebox/prod/db/postgresql-01/{data,config} mkdir -p /mnt/storagebox/prod/db/etcd-01/data # iklim-db-02 üzerinde: mkdir -p /mnt/storagebox/prod/db/mongodb-02/{data,log,config} mkdir -p /mnt/storagebox/prod/db/postgresql-02/{data,config} mkdir -p /mnt/storagebox/prod/db/etcd-02/data # iklim-db-03 üzerinde: mkdir -p /mnt/storagebox/prod/db/mongodb-03/{data,log,config} mkdir -p /mnt/storagebox/prod/db/postgresql-03/{data,config} mkdir -p /mnt/storagebox/prod/db/etcd-03/data ``` ## 4. MongoDB Replica Set ### mongod.conf Her DB node'unda `/mnt/storagebox/prod/db/mongodb-0X/config/mongod.conf`: ```yaml net: port: 27017 storage: engine: "wiredTiger" dbPath: "/data/db" directoryPerDB: true systemLog: verbosity: 0 timeStampFormat: "iso8601-local" destination: file path: "/data/log/mongo.log" logAppend: true logRotate: rename replication: replSetName: "rs0" security: authorization: enabled keyFile: "/data/configdb/rs-auth.key" ``` ### Replica Set Auth Key Tüm DB node'larında **aynı** key dosyası olmalıdır: ```bash # iklim-db-01 üzerinde oluştur: openssl rand -base64 756 > /mnt/storagebox/prod/db/mongodb-01/config/rs-auth.key chmod 400 /mnt/storagebox/prod/db/mongodb-01/config/rs-auth.key # Aynı içeriği diğer node'lara kopyala: cat /mnt/storagebox/prod/db/mongodb-01/config/rs-auth.key \ > /mnt/storagebox/prod/db/mongodb-02/config/rs-auth.key cat /mnt/storagebox/prod/db/mongodb-01/config/rs-auth.key \ > /mnt/storagebox/prod/db/mongodb-03/config/rs-auth.key chmod 400 /mnt/storagebox/prod/db/mongodb-0{2,3}/config/rs-auth.key ``` ### Stack Dosyası — MongoDB `/opt/iklimco/stacks/prod-db-mongo.yml`: ```yaml version: "3.8" networks: iklimco-net: external: true services: mongodb-01: image: mongo:8 environment: MONGO_INITDB_ROOT_USERNAME: mongo-root MONGO_INITDB_ROOT_PASSWORD: "${MONGO_ROOT_PASSWORD}" volumes: - /mnt/storagebox/prod/db/mongodb-01/data:/data/db - /mnt/storagebox/prod/db/mongodb-01/log:/data/log - /mnt/storagebox/prod/db/mongodb-01/config:/data/configdb networks: - iklimco-net ports: - target: 27017 published: 27017 protocol: tcp mode: host command: ["--config", "/data/configdb/mongod.conf"] deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-01 restart_policy: condition: on-failure mongodb-02: image: mongo:8 environment: MONGO_INITDB_ROOT_USERNAME: mongo-root MONGO_INITDB_ROOT_PASSWORD: "${MONGO_ROOT_PASSWORD}" volumes: - /mnt/storagebox/prod/db/mongodb-02/data:/data/db - /mnt/storagebox/prod/db/mongodb-02/log:/data/log - /mnt/storagebox/prod/db/mongodb-02/config:/data/configdb networks: - iklimco-net ports: - target: 27017 published: 27017 protocol: tcp mode: host command: ["--config", "/data/configdb/mongod.conf"] deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-02 restart_policy: condition: on-failure mongodb-03: image: mongo:8 environment: MONGO_INITDB_ROOT_USERNAME: mongo-root MONGO_INITDB_ROOT_PASSWORD: "${MONGO_ROOT_PASSWORD}" volumes: - /mnt/storagebox/prod/db/mongodb-03/data:/data/db - /mnt/storagebox/prod/db/mongodb-03/log:/data/log - /mnt/storagebox/prod/db/mongodb-03/config:/data/configdb networks: - iklimco-net ports: - target: 27017 published: 27017 protocol: tcp mode: host command: ["--config", "/data/configdb/mongod.conf"] deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-03 restart_policy: condition: on-failure ``` ### Replica Set Başlangıç Stack deploy edildikten sonra **bir kez** çalıştırılır: ```bash # iklim-db-01 üzerinde: docker exec -it $(docker ps -q -f name=iklim-db_mongodb-01) mongosh \ -u mongo-root -p "${MONGO_ROOT_PASSWORD}" --authenticationDatabase admin # mongosh içinde: rs.initiate({ _id: "rs0", members: [ { _id: 0, host: "10.20.20.11:27017", priority: 2 }, { _id: 1, host: "10.20.20.12:27017", priority: 1 }, { _id: 2, host: "10.20.20.13:27017", priority: 1 } ] }) # Durum kontrol: rs.status() ``` `"stateStr": "PRIMARY"` ve iki `"SECONDARY"` görülünce replica set hazırdır. ## 5. PostgreSQL — Patroni + etcd Patroni, PostgreSQL primary/standby rollerini etcd üzerinden koordine eder. Primary düşerse diğer node'lardan biri otomatik olarak seçim kazanır ve primary olur. Swarm servisi container'ı yeniden başlatır; Patroni kaldığı yerden devam eder. ### 5.1 Özel Image (Patroni + PostGIS) `postgis/postgis:17-3.5` imajı üzerine Patroni kurulur. Bu imaj Harbor'a push edilip stack'te kullanılır. `Environment_Infrastructure/docker/patroni-postgis/Dockerfile`: ```dockerfile FROM postgis/postgis:17-3.5 USER root RUN apt-get update && apt-get install -y --no-install-recommends \ python3-pip \ python3-dev \ gcc \ libpq-dev \ && pip3 install --no-cache-dir 'patroni[etcd3]' \ && apt-get purge -y gcc python3-dev \ && apt-get autoremove -y \ && rm -rf /var/lib/apt/lists/* USER postgres ENTRYPOINT ["patroni", "/etc/patroni/patroni.yml"] ``` Build ve push: ```bash cd Environment_Infrastructure/docker/patroni-postgis docker build -t registry.iklim.co/infra/patroni-postgis:17-3.5 . docker push registry.iklim.co/infra/patroni-postgis:17-3.5 ``` ### 5.2 etcd Kümesi #### Stack Dosyası — etcd `/opt/iklimco/stacks/prod-db-etcd.yml`: ```yaml version: "3.8" networks: iklimco-net: external: true services: etcd-01: image: bitnami/etcd:3 environment: ALLOW_NONE_AUTHENTICATION: "yes" ETCD_NAME: etcd-01 ETCD_INITIAL_ADVERTISE_PEER_URLS: http://10.20.20.11:2380 ETCD_LISTEN_PEER_URLS: http://0.0.0.0:2380 ETCD_ADVERTISE_CLIENT_URLS: http://10.20.20.11:2379 ETCD_LISTEN_CLIENT_URLS: http://0.0.0.0:2379 ETCD_INITIAL_CLUSTER: "etcd-01=http://10.20.20.11:2380,etcd-02=http://10.20.20.12:2380,etcd-03=http://10.20.20.13:2380" ETCD_INITIAL_CLUSTER_STATE: new ETCD_INITIAL_CLUSTER_TOKEN: iklimco-etcd-prod volumes: - /mnt/storagebox/prod/db/etcd-01/data:/bitnami/etcd/data networks: - iklimco-net ports: - target: 2379 published: 2379 protocol: tcp mode: host - target: 2380 published: 2380 protocol: tcp mode: host deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-01 restart_policy: condition: on-failure etcd-02: image: bitnami/etcd:3 environment: ALLOW_NONE_AUTHENTICATION: "yes" ETCD_NAME: etcd-02 ETCD_INITIAL_ADVERTISE_PEER_URLS: http://10.20.20.12:2380 ETCD_LISTEN_PEER_URLS: http://0.0.0.0:2380 ETCD_ADVERTISE_CLIENT_URLS: http://10.20.20.12:2379 ETCD_LISTEN_CLIENT_URLS: http://0.0.0.0:2379 ETCD_INITIAL_CLUSTER: "etcd-01=http://10.20.20.11:2380,etcd-02=http://10.20.20.12:2380,etcd-03=http://10.20.20.13:2380" ETCD_INITIAL_CLUSTER_STATE: new ETCD_INITIAL_CLUSTER_TOKEN: iklimco-etcd-prod volumes: - /mnt/storagebox/prod/db/etcd-02/data:/bitnami/etcd/data networks: - iklimco-net ports: - target: 2379 published: 2379 protocol: tcp mode: host - target: 2380 published: 2380 protocol: tcp mode: host deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-02 restart_policy: condition: on-failure etcd-03: image: bitnami/etcd:3 environment: ALLOW_NONE_AUTHENTICATION: "yes" ETCD_NAME: etcd-03 ETCD_INITIAL_ADVERTISE_PEER_URLS: http://10.20.20.13:2380 ETCD_LISTEN_PEER_URLS: http://0.0.0.0:2380 ETCD_ADVERTISE_CLIENT_URLS: http://10.20.20.13:2379 ETCD_LISTEN_CLIENT_URLS: http://0.0.0.0:2379 ETCD_INITIAL_CLUSTER: "etcd-01=http://10.20.20.11:2380,etcd-02=http://10.20.20.12:2380,etcd-03=http://10.20.20.13:2380" ETCD_INITIAL_CLUSTER_STATE: new ETCD_INITIAL_CLUSTER_TOKEN: iklimco-etcd-prod volumes: - /mnt/storagebox/prod/db/etcd-03/data:/bitnami/etcd/data networks: - iklimco-net ports: - target: 2379 published: 2379 protocol: tcp mode: host - target: 2380 published: 2380 protocol: tcp mode: host deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-03 restart_policy: condition: on-failure ``` **Önemli:** etcd cluster ilk kez başlatılırken üç node aynı anda `ETCD_INITIAL_CLUSTER_STATE: new` ile deploy edilmelidir. Cluster kurulduktan sonra bu değer `existing` olarak güncellenmeli ve stack yeniden deploy edilmelidir (node yeniden başlasın diye `new` kalmamalı; aksi hâlde data dizini sıfırlanır). ### 5.3 Patroni Konfigürasyonu Her node için ayrı bir `patroni.yml` dosyası oluşturulur. Farklılıklar yalnızca `name` ve `connect_address` alanlarındadır. **Node 01** — `/mnt/storagebox/prod/db/postgresql-01/config/patroni.yml`: ```yaml scope: iklim-postgres namespace: /db/ name: postgresql-01 restapi: listen: 0.0.0.0:8008 connect_address: 10.20.20.11:8008 etcd3: hosts: - 10.20.20.11:2379 - 10.20.20.12:2379 - 10.20.20.13:2379 bootstrap: dcs: ttl: 30 loop_wait: 10 retry_timeout: 10 maximum_lag_on_failover: 1048576 postgresql: use_pg_rewind: true parameters: wal_level: replica hot_standby: "on" wal_keep_size: 512 max_wal_senders: 5 max_replication_slots: 5 initdb: - encoding: UTF8 - data-checksums pg_hba: - host replication replicator 10.20.20.0/24 scram-sha-256 - host all all 10.20.10.0/24 scram-sha-256 - host all all 10.20.20.0/24 scram-sha-256 users: postgres: password: "${POSTGRES_PASSWORD}" options: - superuser postgresql: listen: 0.0.0.0:5432 connect_address: 10.20.20.11:5432 data_dir: /var/lib/postgresql/data/pgdata pgpass: /tmp/pgpass0 authentication: replication: username: replicator password: "${REPLICATOR_PASSWORD}" superuser: username: postgres password: "${POSTGRES_PASSWORD}" parameters: unix_socket_directories: "/var/run/postgresql" tags: nofailover: false noloadbalance: false clonefrom: false nosync: false ``` **Node 02** — `/mnt/storagebox/prod/db/postgresql-02/config/patroni.yml`: Node 01 ile aynı içerik, yalnızca şu alanlar farklı: ```yaml name: postgresql-02 restapi: connect_address: 10.20.20.12:8008 postgresql: connect_address: 10.20.20.12:5432 data_dir: /var/lib/postgresql/data/pgdata ``` **Node 03** — `/mnt/storagebox/prod/db/postgresql-03/config/patroni.yml`: ```yaml name: postgresql-03 restapi: connect_address: 10.20.20.13:8008 postgresql: connect_address: 10.20.20.13:5432 data_dir: /var/lib/postgresql/data/pgdata ``` ### 5.4 Stack Dosyası — Patroni `/opt/iklimco/stacks/prod-db-patroni.yml`: ```yaml version: "3.8" networks: iklimco-net: external: true services: patroni-01: image: registry.iklim.co/infra/patroni-postgis:17-3.5 environment: DATABASE_POSTGRES_ROOT_USER: "${DATABASE_POSTGRES_ROOT_USER}" POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}" REPLICATOR_PASSWORD: "${REPLICATOR_PASSWORD}" TZ: "Europe/Istanbul" volumes: - /mnt/storagebox/prod/db/postgresql-01/data:/var/lib/postgresql/data - /mnt/storagebox/prod/db/postgresql-01/config/patroni.yml:/etc/patroni/patroni.yml:ro networks: - iklimco-net ports: - target: 5432 published: 5432 protocol: tcp mode: host - target: 8008 published: 8008 protocol: tcp mode: host deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-01 restart_policy: condition: on-failure patroni-02: image: registry.iklim.co/infra/patroni-postgis:17-3.5 environment: DATABASE_POSTGRES_ROOT_USER: "${DATABASE_POSTGRES_ROOT_USER}" POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}" REPLICATOR_PASSWORD: "${REPLICATOR_PASSWORD}" TZ: "Europe/Istanbul" volumes: - /mnt/storagebox/prod/db/postgresql-02/data:/var/lib/postgresql/data - /mnt/storagebox/prod/db/postgresql-02/config/patroni.yml:/etc/patroni/patroni.yml:ro networks: - iklimco-net ports: - target: 5432 published: 5432 protocol: tcp mode: host - target: 8008 published: 8008 protocol: tcp mode: host deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-02 restart_policy: condition: on-failure patroni-03: image: registry.iklim.co/infra/patroni-postgis:17-3.5 environment: DATABASE_POSTGRES_ROOT_USER: "${DATABASE_POSTGRES_ROOT_USER}" POSTGRES_PASSWORD: "${POSTGRES_PASSWORD}" REPLICATOR_PASSWORD: "${REPLICATOR_PASSWORD}" TZ: "Europe/Istanbul" volumes: - /mnt/storagebox/prod/db/postgresql-03/data:/var/lib/postgresql/data - /mnt/storagebox/prod/db/postgresql-03/config/patroni.yml:/etc/patroni/patroni.yml:ro networks: - iklimco-net ports: - target: 5432 published: 5432 protocol: tcp mode: host - target: 8008 published: 8008 protocol: tcp mode: host deploy: replicas: 1 placement: constraints: - node.hostname == iklim-db-03 restart_policy: condition: on-failure ``` ### 5.5 Durum Kontrolü ```bash # Herhangi bir DB node'unda: docker exec -it $(docker ps -q -f name=iklim-patroni_patroni-01) \ patronictl -c /etc/patroni/patroni.yml list ``` Beklenen çıktı: bir `Leader` ve iki `Replica` satırı, hepsinin `State` sütunu `running`. ```bash # etcd cluster sağlığı: docker exec -it $(docker ps -q -f name=iklim-etcd_etcd-01) \ etcdctl endpoint health \ --endpoints=http://10.20.20.11:2379,http://10.20.20.12:2379,http://10.20.20.13:2379 ``` ```bash # Mevcut primary'i öğren: docker exec -it $(docker ps -q -f name=iklim-patroni_patroni-01) \ patronictl -c /etc/patroni/patroni.yml topology ``` ## 6. Deploy Sıra önemlidir: önce etcd, ardından MongoDB ve Patroni stack'leri. ```bash # iklim-app-01 üzerinde (Swarm manager): export $(cat /opt/iklimco/stacks/.env | xargs) # 1. etcd cluster: docker stack deploy \ --compose-file /opt/iklimco/stacks/prod-db-etcd.yml \ --with-registry-auth \ iklim-etcd # etcd cluster'ın kurulmasını bekle (yaklaşık 30 saniye): sleep 30 # etcd sağlığını doğrula: docker exec -it $(docker ps -q -f name=iklim-etcd_etcd-01) \ etcdctl endpoint health \ --endpoints=http://10.20.20.11:2379,http://10.20.20.12:2379,http://10.20.20.13:2379 # 2. MongoDB: docker stack deploy \ --compose-file /opt/iklimco/stacks/prod-db-mongo.yml \ --with-registry-auth \ iklim-db # 3. Patroni (PostgreSQL): docker stack deploy \ --compose-file /opt/iklimco/stacks/prod-db-patroni.yml \ --with-registry-auth \ iklim-patroni docker stack services iklim-etcd docker stack services iklim-db docker stack services iklim-patroni ``` **etcd'yi ikinci kez deploy etmeden önce** `ETCD_INITIAL_CLUSTER_STATE` değerini `new` yerine `existing` olarak güncelle; aksi hâlde veri dizini sıfırlanır. İlk deploy başarılı olduktan sonra `prod-db-etcd.yml` içindeki her serviste bu değeri `existing` yap ve `docker stack deploy` ile güncelle. ### .env Dosyası `/opt/iklimco/stacks/.env` (repo'ya commit edilmez): ```env DATABASE_POSTGRES_ROOT_USER=postgres POSTGRES_PASSWORD= REPLICATOR_PASSWORD= MONGO_ROOT_PASSWORD= ``` ### MongoDB Replica Set Başlatma MongoDB stack deploy edildikten sonra bir kez çalıştırılır: ```bash docker exec -it $(docker ps -q -f name=iklim-db_mongodb-01) mongosh \ -u mongo-root -p "${MONGO_ROOT_PASSWORD}" --authenticationDatabase admin # mongosh içinde: rs.initiate({ _id: "rs0", members: [ { _id: 0, host: "10.20.20.11:27017", priority: 2 }, { _id: 1, host: "10.20.20.12:27017", priority: 1 }, { _id: 2, host: "10.20.20.13:27017", priority: 1 } ] }) ``` ## 7. App Servislerinden Erişim ### MongoDB Replica Set Connection String ``` mongodb://mongo-root:@10.20.20.11:27017,10.20.20.12:27017,10.20.20.13:27017/?replicaSet=rs0&authSource=admin ``` ### PostgreSQL — Patroni Patroni her an primary olan node'u yönetir. Uygulama katmanı tüm üç IP'yi vererek primary'e yazabilir, secondary'den okuyabilir: ``` # Yazma — sadece primary kabul eder: jdbc:postgresql://10.20.20.11:5432,10.20.20.12:5432,10.20.20.13:5432/iklimdb?targetServerType=primary # Okuma (yük dengeleme): jdbc:postgresql://10.20.20.11:5432,10.20.20.12:5432,10.20.20.13:5432/iklimdb?targetServerType=preferSecondary ``` PostgreSQL JDBC sürücüsü `targetServerType=primary` ile bağlanmaya çalışacağı tüm node'lara bağlanır ve primary olanı otomatik bulur. ### Patroni REST API Patroni, 8008 portundan HTTP endpoint sunar. Bu endpoint HAProxy veya benzeri bir load balancer ile kullanılarak primary'i otomatik yönlendirme sağlanabilir: ```bash # Primary kontrolü (HTTP 200 = primary, HTTP 503 = replica): curl -s http://10.20.20.11:8008/primary ``` ## Kabul Kriterleri - `docker stack services iklim-etcd` — üç servis `1/1` - `docker stack services iklim-db` — üç MongoDB servisi `1/1` - `docker stack services iklim-patroni` — üç Patroni servisi `1/1` - `patronictl list` — 1 `Leader`, 2 `Replica`, hepsi `running` - `etcdctl endpoint health` — üç endpoint `healthy` - `rs.status()` — 1 PRIMARY, 2 SECONDARY - App node'larından MongoDB ve PostgreSQL'e erişim sağlanır - `5432`, `27017`, `2379`, `2380`, `8008` portları public internet'ten kapalıdır - Bir DB node yeniden başlatıldığında Patroni otomatik seçim yapar, yeni primary belirlenir - Patroni primary geçişi sırasında eski primary standby olarak re-join olur (split-brain yoktur)