Executive Summary

This vulnerability exists in the SimpleDirectoryReader component of the llama_index project (version 0.12.23). It causes uncontrolled memory consumption because the limit on the number of files to read (num_files_limit) is applied only after all files in a directory are loaded into memory. This means that if a directory contains many files, all of them are loaded first, potentially exhausting available memory and degrading performance. The issue was fixed by changing the file loading logic to enforce the file limit during directory traversal, preventing excessive memory use. [1]

Useful 0 Not Useful 0

Impact Analysis

This vulnerability can lead to memory exhaustion on systems using the affected SimpleDirectoryReader component, especially in environments with limited resources. Excessive memory consumption can cause degraded performance, application crashes, or denial of service, impacting the reliability and availability of applications relying on this component. [1]

Useful 0 Not Useful 0

Detection Guidance

This vulnerability manifests as uncontrolled memory consumption when the SimpleDirectoryReader component loads files from a directory. To detect it, monitor the memory usage of processes using llama_index version 0.12.23, especially when reading directories with many files. Commands like 'top' or 'htop' on Linux can show real-time memory usage. Additionally, you can use 'ps aux --sort=-%mem | head' to identify processes consuming high memory. For more targeted detection, monitor logs or application behavior for performance degradation or crashes during directory reading operations. [1]

Useful 0 Not Useful 0

Mitigation Strategies

The immediate mitigation is to upgrade the llama_index package to version 0.12.41 or later, where the vulnerability is fixed by enforcing the file limit during directory traversal to prevent excessive memory use. If upgrading is not immediately possible, limit the number of files processed by the SimpleDirectoryReader manually or avoid processing directories with a large number of files to reduce memory consumption. [1]

Useful 0 Not Useful 0