Memory Exhaustion in vLLM via Unbounded Mel Config Caching

Executive Summary

This vulnerability affects vLLM versions between 0.24.0 and 0.31.0, specifically in the MOSS-Audio model serving component. It involves an unbounded dictionary that caches a processor for every unique request-controlled mm_processor_kwargs value. Attackers can send distinct or oversized mel_config values to the /tokenize endpoint without authentication, bypassing API key requirements. This exhausts host memory and crashes the API process.

Useful 0 Not Useful 0

Detection Guidance

Monitor for unusually high memory usage on systems running vLLM versions 0.25.0 to 0.30.0, particularly those serving the MOSS-Audio model. Check logs for repeated requests to the /tokenize endpoint with distinct or large mel_config values. Use system monitoring tools like top, htop, or ps to track memory consumption spikes.

Useful 0 Not Useful 0

Impact Analysis

Unauthenticated attackers can exploit this to exhaust server memory, causing system crashes and denial of service. The impact includes service unavailability, potential data loss, and resource exhaustion on the host system. It specifically affects deployments serving the MOSS-Audio model.

Useful 0 Not Useful 0

Compliance Impact

This vulnerability could lead to denial of service by exhausting system memory, potentially causing service disruptions. For GDPR, this may impact availability of personal data processing systems, requiring incident response under Article 33. For HIPAA, it could disrupt healthcare services handling protected health information, violating the Security Rule's availability requirements.

Useful 0 Not Useful 0

Mitigation Strategies

• Upgrade vLLM to version 0.31.0 or later to address the memory exhaustion issue.
• Disable or restrict access to the /tokenize endpoint if not required, as it bypasses API key protection.
• Implement rate limiting or throttling on the /tokenize endpoint to prevent abuse.
• Use a reverse proxy with authentication to add an additional layer of protection.
• Monitor and clear the _hf_processor_cache periodically if the cache cannot be disabled.

Useful 0 Not Useful 0