Uncontrolled Memory Allocation in SGLang with Mooncake Backend

Executive Summary

This vulnerability in SGLang through 0.5.22 with the Mooncake disaggregation backend allows unauthenticated network-adjacent attackers to exhaust Prefill memory. Attackers exploit the /route endpoint to obtain rank addresses and send STAGING_RSP messages with large chunk_idx values under fresh room IDs. This causes the scheduler to be terminated due to out-of-memory conditions.

Useful 0 Not Useful 0

Detection Guidance

Monitor for abnormal memory usage spikes in the Prefill scheduler process, particularly increases exceeding typical workloads. Check for OOM killer events targeting the Prefill scheduler. Inspect network traffic for unexpected STAGING_RSP messages with large chunk_idx values sent to the ZeroMQ rank socket. Review logs for sudden slowdowns in overlapping inference requests.

Useful 0 Not Useful 0

Impact Analysis

The vulnerability can cause system instability by exhausting memory resources, leading to crashes or degraded performance. It may disrupt disaggregated inference operations, causing incorrect behavior or service interruptions in environments using SGLang with Mooncake backend.

Useful 0 Not Useful 0

Compliance Impact

This vulnerability could indirectly impact compliance with GDPR and HIPAA by enabling denial-of-service conditions through memory exhaustion. GDPR requires systems to ensure availability of data processing services, and HIPAA mandates safeguards against disruptions that could affect protected health information access. Memory exhaustion attacks may violate these requirements by degrading or terminating critical services.

Useful 0 Not Useful 0

Mitigation Strategies

Restrict network access to the Prefill worker's ZeroMQ rank socket using firewall rules. Implement input validation to enforce maximum chunk_idx limits in STAGING_RSP messages. Add authentication and authorization checks for all incoming messages. Update to the latest patched version if available. Monitor Prefill scheduler memory usage and set up alerts for abnormal increases.

Useful 0 Not Useful 0