[Bug 793889] New: vaapisink: Crashes on 24-bit RGB video with AMD card

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

Bug ID: 793889
Summary: vaapisink: Crashes on 24-bit RGB video with AMD card
Classification: Platform
Product: GStreamer
Version: git master
OS: Linux
Status: NEW
Severity: normal
Priority: Normal
Component: gstreamer-vaapi
Assignee: gstreamer-***@lists.freedesktop.org
Reporter: ***@igalia.com
QA Contact: gstreamer-***@lists.freedesktop.org
CC: ***@gmail.com, ***@igalia.com
GNOME version: ---

$ gst-launch-1.0 multifilesrc location="/tmp/white.png"
caps="image/png,framerate=\(fraction\)3/1" ! pngdec ! videoconvert ! identity
silent=false ! vaapisink -v
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'vaapisink0': gst.vaapi.Display=context,
gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx1";
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = event *******
(identity0:sink) E (type: stream-start (10254), GstEventStreamStart,
stream-id=(string)116c669c7f52880c9f1b179be1334a19,
flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)0;) 0x55bdcbc9a8f0
/GstPipeline:pipeline0/GstMultiFileSrc:multifilesrc0.GstPad:src: caps =
image/png, width=(int)1, height=(int)1, framerate=(fraction)3/1
/GstPipeline:pipeline0/GstPngDec:pngdec0.GstPad:sink: caps = image/png,
width=(int)1, height=(int)1, framerate=(fraction)3/1
/GstPipeline:pipeline0/GstPngDec:pngdec0.GstPad:src: caps = video/x-raw,
format=(string)RGB, width=(int)320, height=(int)240,
interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1,
colorimetry=(string)sRGB, framerate=(fraction)3/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:src: caps =
video/x-raw, format=(string)BGRx, width=(int)320, height=(int)240,
framerate=(fraction)3/1, interlace-mode=(string)progressive,
pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = event *******
(identity0:sink) E (type: caps (12814), GstEventCaps,
caps=(GstCaps)"video/x-raw\,\ format\=\(string\)BGRx\,\ width\=\(int\)320\,\
height\=\(int\)240\,\ framerate\=\(fraction\)3/1\,\
interlace-mode\=\(string\)progressive\,\ pixel-aspect-ratio\=\(fraction\)1/1";)
0x7f5cd8005030
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:src: caps = video/x-raw,
format=(string)BGRx, width=(int)320, height=(int)240, framerate=(fraction)3/1,
interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstVaapiSink:vaapisink0.GstPad:sink: caps = video/x-raw,
format=(string)BGRx, width=(int)320, height=(int)240, framerate=(fraction)3/1,
interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:sink: caps = video/x-raw,
format=(string)BGRx, width=(int)320, height=(int)240, framerate=(fraction)3/1,
interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps =
video/x-raw, format=(string)RGB, width=(int)320, height=(int)240,
interlace-mode=(string)progressive, pixel-aspect-ratio=(fraction)1/1,
colorimetry=(string)sRGB, framerate=(fraction)3/1
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = event *******
(identity0:sink) E (type: segment (17934), GstEventSegment,
segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE,
rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME,
base=(guint64)0, offset=(guint64)0, start=(guint64)0,
stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0,
duration=(guint64)18446744073709551615;";) 0x55bdcbc9aa40
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain *******
(identity0:sink) (307200 bytes, dts: none, pts: 0:00:00.000000000, duration:
0:00:00.333333333, offset: -1, offset_end: -1, flags: 00000040 discont , meta:
GstVideoMeta, GstVaapiVideoMeta) 0x7f5cd8009410
Caught SIGSEGV
#0 0x00007f5d1112d3db in poll () from /lib64/libc.so.6
#1 0x00007f5d11a7ae99 in g_main_context_iterate.isra ()
#2 0x00007f5d11a7b232 in g_main_loop_run () from /lib64/libglib-2.0.so.0
#3 0x00007f5d121df543 in gst_bus_poll () from /lib64/libgstreamer-1.0.so.0
#4 0x000055bdc9cc7aab in event_loop ()
#5 0x000055bdc9cc6b8e in main ()
Spinning. Please run 'gdb gst-launch-1.0 46210' to continue debugging, Ctrl-C
to quit, or Ctrl-\ to dump core.
^Chandling interrupt.
Interrupt: Stopping pipeline ...
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #1 from Alicia Boya García <***@igalia.com> ---
Created attachment 369045
--> https://bugzilla.gnome.org/attachment.cgi?id=369045&action=edit
Test vector (indexed color PNG without alpha channel)

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #2 from Alicia Boya García <***@igalia.com> ---
Notice format=(string)BGRx in the output above. Adding a caps filter to use
BGRA, or RGBA will avoid the crash. (The color will still be very wrong because
of https://bugzilla.gnome.org/show_bug.cgi?id=793886)

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #3 from Alicia Boya García <***@igalia.com> ---
Created attachment 369047
--> https://bugzilla.gnome.org/attachment.cgi?id=369047&action=edit
Backtrace

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #4 from Alicia Boya García <***@igalia.com> ---
No output is written to dmesg during the crash.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #5 from Alicia Boya García <***@igalia.com> ---
*** Bug 793890 has been marked as a duplicate of this bug. ***

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #6 from Alicia Boya García <***@igalia.com> ---
Created attachment 369092
--> https://bugzilla.gnome.org/attachment.cgi?id=369092&action=edit
Test vector (MP4 raw RGB video)

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #7 from Alicia Boya García <***@igalia.com> ---
The attached MP4 file seems to trigger the same crash with a slightly different
pixel format (RGB instead of BGRx).

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #8 from Alicia Boya García <***@igalia.com> ---
(In reply to Alicia Boya García from comment #7)

Post by "GStreamer" (GNOME Bugzilla)
The attached MP4 file seems to trigger the same crash with a slightly
different pixel format (RGB instead of BGRx).

The playbin pipeline includes a videoconvert that converts RGB to RGBx.

RGB is not accepted by vaapisink caps, so the problem only occurs with RGBx and
BGRx.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #9 from Alicia Boya García <***@igalia.com> ---
The crash can be reproduced with videotestsrc too. All the following segfault:

$ gst-launch-1.0 -v videotestsrc ! video/x-raw, format=RGBx ! vaapisink
$ gst-launch-1.0 -v videotestsrc ! video/x-raw, format=BGRx ! vaapisink

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

Alicia Boya García <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Summary|vaapisink: Crashes on |vaapisink: Crashes on
|24-bit RGB video with AMD |padded (non-alpha) RGB
|card |video with AMD card

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

Julien Isorce <***@gmail.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@gmail.com

--- Comment #10 from Julien Isorce <***@gmail.com> ---
Thx for the report. Is it also with mesa 17.2.4 ? Does it succeed with other
format (NV12) ? Same result with a tee element in between ? (maybe something
woring when the sink proposes a pool)

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #11 from Julien Isorce <***@gmail.com> ---
Can you try also with GST_VAAPI_ENABLE_DIRECT_RENDERING=0 (not sure if it
affects vaapisink though) ?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #12 from Alicia Boya García <***@igalia.com> ---
(In reply to Julien Isorce from comment #10)

Post by "GStreamer" (GNOME Bugzilla)
Thx for the report. Is it also with mesa 17.2.4 ? Does it succeed with other
format (NV12) ? Same result with a tee element in between ? (maybe something
woring when the sink proposes a pool)

Yes to all.

Name : mesa-dri-drivers
Version : 17.3.5
OpenGL renderer string: AMD Radeon (TM) RX 460 Graphics (AMD POLARIS11 / DRM
3.19.0 / 4.14.16-300.fc27.x86_64, LLVM 5.0.0)
Fedora 27
X11

(In reply to Julien Isorce from comment #11)

Post by "GStreamer" (GNOME Bugzilla)
Can you try also with GST_VAAPI_ENABLE_DIRECT_RENDERING=0 (not sure if it
affects vaapisink though) ?

It makes no difference.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

--- Comment #13 from Alicia Boya García <***@igalia.com> ---
(In reply to Alicia Boya García from comment #12)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to Julien Isorce from comment #10)

Post by "GStreamer" (GNOME Bugzilla)
Thx for the report. Is it also with mesa 17.2.4 ? Does it succeed with other
format (NV12) ? Same result with a tee element in between ? (maybe something
woring when the sink proposes a pool)

Yes to all.

I'll be more explicit just in case:

* It is also mesa 17.2.
* It succeeds with NV12.
* It crashes with a tee element in between too.

Post by "GStreamer" (GNOME Bugzilla)
Name : mesa-dri-drivers
Version : 17.3.5

Pasted wrong console output, it's actually:

Installed Packages
Name : mesa-dri-drivers
Version : 17.2.4

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=793889

GStreamer system administrator <***@gstreamer.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |OBSOLETE

--- Comment #14 from GStreamer system administrator <***@gstreamer.net> ---
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been
closed from further activity.

You can subscribe and participate further through the new bug through this link
to our GitLab instance:
https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/86.