docs(roadmap): add sticky session plan for rabbitmq websocket upstream
- Implemented Consistent Hashing (chash) logic in APISIX upstream configuration for prod. - Added instructions for real IP detection in APISIX configuration template. - Documented the bypass of Swarm VIP for better session persistence on RabbitMQ nodes.
This commit is contained in:
parent
4c3b7faad6
commit
52bd6a59ac
@ -371,7 +371,59 @@ docker exec "$CTR" rabbitmqctl cluster_status
|
|||||||
> '{"queue-type":"quorum"}' --apply-to queues
|
> '{"queue-type":"quorum"}' --apply-to queues
|
||||||
> ```
|
> ```
|
||||||
|
|
||||||
## Step 7 — Create `docker-stack-infra.prod.yml`
|
## Step 7 — RabbitMQ WebSocket Sticky Sessions (Consistent Hash)
|
||||||
|
|
||||||
|
RabbitMQ Web STOMP (over WebSocket) requires a persistent connection. In a 3-node RabbitMQ cluster, if an APISIX instance uses the default Swarm VIP for the `rabbitmq` upstream, it may cause unnecessary inter-node traffic or connection drops if the session doesn't persist on the same node.
|
||||||
|
|
||||||
|
To optimize this, we implement **Consistent Hashing (chash)** at the APISIX layer based on the client's IP address (`remote_addr`).
|
||||||
|
|
||||||
|
### 1. Update APISIX Upstream Configuration (init.sh)
|
||||||
|
|
||||||
|
Update the `rabbitmq` upstream definition in `init/apisix-core/init.sh` to target specific cluster nodes instead of the generic service name, enabling the `chash` algorithm for prod.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Update upstream rabbitmq block in init.sh
|
||||||
|
if [[ "$PROFILE" == "prod" ]]; then
|
||||||
|
# Direct node DNS names to bypass Swarm VIP and allow chash to work effectively
|
||||||
|
RABBITMQ_NODES='{"rabbitmq-iklim-app-01:15674":1, "rabbitmq-iklim-app-02:15674":1, "rabbitmq-iklim-app-03:15674":1}'
|
||||||
|
LB_TYPE="chash"
|
||||||
|
HASH_KEY="remote_addr"
|
||||||
|
else
|
||||||
|
RABBITMQ_NODES='{"rabbitmq:15674":1}'
|
||||||
|
LB_TYPE="roundrobin"
|
||||||
|
HASH_KEY=""
|
||||||
|
fi
|
||||||
|
|
||||||
|
call_api "upstream rabbitmq" -X PUT "$APISIX_ADMIN_URL/upstreams/rabbitmq-upstream" \
|
||||||
|
-H "X-API-KEY: $API_KEY" -H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"name": "rabbitmq-upstream",
|
||||||
|
"type": "'"$LB_TYPE"'",
|
||||||
|
"key": "'"$HASH_KEY"'",
|
||||||
|
"nodes": '"$RABBITMQ_NODES"',
|
||||||
|
"timeout": {"connect": 10, "send": 3600, "read": 3600},
|
||||||
|
"scheme": "http",
|
||||||
|
'"$HC"'
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Enable Real IP Detection in APISIX
|
||||||
|
|
||||||
|
Consistent hashing by `remote_addr` requires APISIX to see the actual client IP, not the internal IP of the SWAG (Nginx) proxy.
|
||||||
|
|
||||||
|
Update `template/apisix-core/config.yaml.template`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
nginx_config:
|
||||||
|
http:
|
||||||
|
real_ip_header: "X-Forwarded-For"
|
||||||
|
real_ip_from:
|
||||||
|
- 10.0.0.0/8
|
||||||
|
- 172.16.0.0/12
|
||||||
|
- 192.168.0.0/16
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 8 — Create `docker-stack-infra.prod.yml`
|
||||||
|
|
||||||
Create this file in the repo root alongside `docker-stack-infra.yml`. It combines all prod-specific overrides from Steps 2–6:
|
Create this file in the repo root alongside `docker-stack-infra.yml`. It combines all prod-specific overrides from Steps 2–6:
|
||||||
|
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user