docs(roadmap): add sticky session plan for rabbitmq websocket upstream

- Implemented Consistent Hashing (chash) logic in APISIX upstream configuration for prod.
- Added instructions for real IP detection in APISIX configuration template.
- Documented the bypass of Swarm VIP for better session persistence on RabbitMQ nodes.
This commit is contained in:
Murat ÖZDEMİR 2026-05-18 09:46:51 +03:00
parent 4c3b7faad6
commit 52bd6a59ac

View File

@ -371,7 +371,59 @@ docker exec "$CTR" rabbitmqctl cluster_status
> '{"queue-type":"quorum"}' --apply-to queues > '{"queue-type":"quorum"}' --apply-to queues
> ``` > ```
## Step 7 — Create `docker-stack-infra.prod.yml` ## Step 7 — RabbitMQ WebSocket Sticky Sessions (Consistent Hash)
RabbitMQ Web STOMP (over WebSocket) requires a persistent connection. In a 3-node RabbitMQ cluster, if an APISIX instance uses the default Swarm VIP for the `rabbitmq` upstream, it may cause unnecessary inter-node traffic or connection drops if the session doesn't persist on the same node.
To optimize this, we implement **Consistent Hashing (chash)** at the APISIX layer based on the client's IP address (`remote_addr`).
### 1. Update APISIX Upstream Configuration (init.sh)
Update the `rabbitmq` upstream definition in `init/apisix-core/init.sh` to target specific cluster nodes instead of the generic service name, enabling the `chash` algorithm for prod.
```bash
# Update upstream rabbitmq block in init.sh
if [[ "$PROFILE" == "prod" ]]; then
# Direct node DNS names to bypass Swarm VIP and allow chash to work effectively
RABBITMQ_NODES='{"rabbitmq-iklim-app-01:15674":1, "rabbitmq-iklim-app-02:15674":1, "rabbitmq-iklim-app-03:15674":1}'
LB_TYPE="chash"
HASH_KEY="remote_addr"
else
RABBITMQ_NODES='{"rabbitmq:15674":1}'
LB_TYPE="roundrobin"
HASH_KEY=""
fi
call_api "upstream rabbitmq" -X PUT "$APISIX_ADMIN_URL/upstreams/rabbitmq-upstream" \
-H "X-API-KEY: $API_KEY" -H "Content-Type: application/json" \
-d '{
"name": "rabbitmq-upstream",
"type": "'"$LB_TYPE"'",
"key": "'"$HASH_KEY"'",
"nodes": '"$RABBITMQ_NODES"',
"timeout": {"connect": 10, "send": 3600, "read": 3600},
"scheme": "http",
'"$HC"'
}'
```
### 2. Enable Real IP Detection in APISIX
Consistent hashing by `remote_addr` requires APISIX to see the actual client IP, not the internal IP of the SWAG (Nginx) proxy.
Update `template/apisix-core/config.yaml.template`:
```yaml
nginx_config:
http:
real_ip_header: "X-Forwarded-For"
real_ip_from:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
```
## Step 8 — Create `docker-stack-infra.prod.yml`
Create this file in the repo root alongside `docker-stack-infra.yml`. It combines all prod-specific overrides from Steps 26: Create this file in the repo root alongside `docker-stack-infra.yml`. It combines all prod-specific overrides from Steps 26: