https://bugs.freedesktop.org/show_bug.cgi?id=105368

Bug ID: 105368 Summary: Crash in ruvd_end_frame when calling vaBeginPicture/vaEndPicture without rendering anything Product: Mesa Version: git Hardware: All OS: All Status: NEW Severity: normal Priority: medium Component: Drivers/Gallium/radeonsi Assignee: dri-devel@lists.freedesktop.org Reporter: k.philipp@gmail.com QA Contact: dri-devel@lists.freedesktop.org

VAAPI testing has revealed that ruvd_end_frame does not handle a particular edge case (see below), i.e. it crashes.

Source of the crash is here: https://cgit.freedesktop.org/mesa/mesa/tree/src/gallium/drivers/radeon/radeo...

The memset fails when you call vaBeginPicture/vaEndPicture without any relevant vaRenderPicture calls in-between and have previously decoded some frames using the context. Then ruvd_begin_frame (triggered by data buffers) is not called to set up a new bs_ptr, and the old pointer that was unmapped already is still around, so memset will segfault. Inserting dec->bs_ptr = NULL after the buffer_unmap works for me, but I don't know if this is the solution or just a workaround.

ffmpeg seems to do this under certain circumstances, which is how this bug surfaced. The vaapi documentation does not seem to forbid this, even if it does not make a lot of sense.