Memory Leak in FFmpeg via libx265 Dolby Vision Encoding

Executive Summary

This vulnerability is a memory leak in FFmpeg through version 9.0.2. It occurs in the libavcodec/libx265.c component where the function libx265_encode_frame() fails to free memory allocated for rpu.payload during Dolby Vision HEVC or AV1 file processing. Attackers can exploit this by providing specially crafted files that cause repeated memory allocations without release, leading to excessive memory consumption and system crashes.

Useful 0 Not Useful 0

Detection Guidance

Detecting this vulnerability requires monitoring for memory leaks during FFmpeg operations, particularly when processing Dolby Vision HEVC or AV1 files. Use tools like 'valgrind' to track memory allocations and leaks during FFmpeg transcoding. Check for processes consuming excessive memory when handling such files.

Useful 0 Not Useful 0

Impact Analysis

This vulnerability can impact you by causing your system to run out of memory during video transcoding tasks. If you process Dolby Vision HEVC or AV1 files using FFmpeg, an attacker could supply malicious files that gradually consume all available memory, leading to system slowdowns, crashes, or denial of service. This is particularly risky for servers or systems handling continuous media processing.

Useful 0 Not Useful 0

Mitigation Strategies

Immediately update FFmpeg to the latest version that patches this vulnerability. Avoid processing untrusted Dolby Vision HEVC or AV1 files until patched. Monitor system memory usage for abnormal growth during transcoding operations.

Useful 0 Not Useful 0