LightLLM Inference Worker Denial of Service via Malformed Parameters

Executive Summary

This vulnerability is a denial of service issue in LightLLM through version 1.2.0. Unauthenticated attackers can crash the inference worker by sending a request with a malicious pd_kv_trans_params field to the /generate endpoint. This causes pickle.loads to fail in InferSamplingParams, killing the inference thread while the /health endpoint remains up. All subsequent requests then hang until the service is restarted.

Useful 0 Not Useful 0

Detection Guidance

Monitor for hanging requests to the /generate endpoint or failed health checks despite /health returning 200. Check logs for UnpicklingError exceptions in inference threads. Use network traffic analysis to detect malformed pd_kv_trans_params fields in requests to /generate.

Useful 0 Not Useful 0

Impact Analysis

This vulnerability allows attackers to disrupt the availability of the LightLLM service by crashing the inference worker. The service becomes unresponsive to all subsequent requests, causing downtime until a manual restart. The /health endpoint may still report the service as healthy, masking the outage from monitoring tools.

Useful 0 Not Useful 0

Compliance Impact

This vulnerability could impact compliance with GDPR and HIPAA by enabling denial-of-service attacks that disrupt service availability. GDPR requires maintaining service availability for data processing, while HIPAA mandates continuous access to protected health information systems. The crash of inference workers and hanging requests may violate these requirements.

Useful 0 Not Useful 0

Mitigation Strategies

Block or sanitize the pd_kv_trans_params field in requests to /generate. Update to a patched version if available. Restart affected nodes to clear hanging threads. Implement rate limiting or WAF rules to prevent malformed requests.

Useful 0 Not Useful 0