[Bug 770535] New: Playbin fails to decode stream when gst-vaapi is installed

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

Bug ID: 770535
Summary: Playbin fails to decode stream when gst-vaapi is
installed
Classification: Platform
Product: GStreamer
Version: 1.8.3
OS: Linux
Status: NEW
Severity: normal
Priority: Normal
Component: gstreamer-vaapi
Assignee: gstreamer-***@lists.freedesktop.org
Reporter: ***@gmail.com
QA Contact: gstreamer-***@lists.freedesktop.org
CC: ***@gmail.com, ***@igalia.com
GNOME version: ---

URL:
https://vodcdn.mobcrush.com/u/video/57a5c1a051f31fbb6d62cd53/recording.mp4.m3u8

Attempting to play this stream fails with a not-negotiated error.

0:00:00.745837289 18551 0x1972ad0 WARN basetransform
gstbasetransform.c:1414:gst_base_transform_setcaps:<capsfilter0> transform
could not transform video/x-h264, stream-format=(string)byte-stream,
alignment=(string)nal, width=(int)1196, height=(int)720,
framerate=(fraction)0/1, parsed=(boolean)true, profile=(string)baseline,
level=(string)3.1 in anything we support

GStreamer 1.8.3
VA-API version: 0.39 (libva 1.7.1)
Intel i965 driver for Intel(R) Haswell Desktop - 1.7.1

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #1 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
The error is actually

:00:01.280772134 1240 0x23e76d0 DEBUG vaapidecode
gstvaapidecode.c:1205:gst_vaapidecode_sink_getcaps:<vaapidecode0> Returning
sink caps EMPTY

This means that the codec profile is not supported by the hardware.

Checking the logs of the media the codec profile is

video/x-h264, parsed=(boolean)true, stream-format=(string)byte-stream,
alignment=(string)nal, width=(int)1196, height=(int)720,
framerate=(fraction)0/1, profile=(string)baseline, level=(string)3.1

And h264 baseline profile is not supported by Haswell.

The interesting part is that playbin, in the case of local media, can change to
other available decoder (libav, for example) if this happens in
gstreamer-vaapi; but in this case of remote media, it doesn't, it just drops
the negotiation.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

Holger Kaelberer <***@getslash.de> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@getslash.de

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

Julien Isorce <***@gmail.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@gmail.com

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

Tim-Philipp Müller <***@zen.co.uk> changed:

What |Removed |Added
----------------------------------------------------------------------------

Post by "GStreamer" (GNOME Bugzilla)
The interesting part is that playbin, in the case of local media,
can change to other available decoder (libav, for example) if
this happens in gstreamer-vaapi; but in this case of remote media
it doesn't, it just drops the negotiation.

Really?

$ wget
https://vodcdn.mobcrush.com/u/video/57a5c1a051f31fbb6d62cd53/recording.mp4Frag1Num0.ts
$ gst-play-1.0 recording.mp4Frag1Num0.ts

=> not-negotiated as well here.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #3 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
(In reply to Tim-Philipp Müller from comment #2)

Post by "GStreamer" (GNOME Bugzilla)

Post by "GStreamer" (GNOME Bugzilla)
The interesting part is that playbin, in the case of local media,
can change to other available decoder (libav, for example) if
this happens in gstreamer-vaapi; but in this case of remote media
it doesn't, it just drops the negotiation.

Really?
$ wget
https://vodcdn.mobcrush.com/u/video/57a5c1a051f31fbb6d62cd53/recording.
mp4Frag1Num0.ts
$ gst-play-1.0 recording.mp4Frag1Num0.ts
=> not-negotiated as well here.

D'oh!

Interesting. Why in this specific media playbin doesn't try another decoder
when vaapih264dec returns EMPTY at getcaps()???

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

Hyunjun Ko <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@igalia.com

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #4 from Hyunjun Ko <***@igalia.com> ---
I think this depends on the container spec.
Isomp4 might have SPS/PPS information in the header, which means qtdemux can
include profile/level in src caps even before processing buffer.
But in case of mpeg ts, IIRC, it doesn't. This means tsdemux can't recognize
profile/level earlier.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #5 from Hyunjun Ko <***@igalia.com> ---
(In reply to Hyunjun Ko from comment #4)

Post by "GStreamer" (GNOME Bugzilla)
I think this depends on the container spec.
Isomp4 might have SPS/PPS information in the header, which means qtdemux can
include profile/level in src caps even before processing buffer.

so vaapi decoder can reject during connecting pad in this case.

Post by "GStreamer" (GNOME Bugzilla)
But in case of mpeg ts, IIRC, it doesn't. This means tsdemux can't recognize
profile/level earlier.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #6 from sreerenj <***@gmail.com> ---
(In reply to Hyunjun Ko from comment #4)

Post by "GStreamer" (GNOME Bugzilla)
I think this depends on the container spec.
Isomp4 might have SPS/PPS information in the header, which means qtdemux can
include profile/level in src caps even before processing buffer.
But in case of mpeg ts, IIRC, it doesn't. This means tsdemux can't recognize
profile/level earlier.

IIUC,whatever be the container, h264parse should get autoplugged and parse the
relevant headers...

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #7 from Hyunjun Ko <***@igalia.com> ---
(In reply to sreerenj from comment #6)

Post by "GStreamer" (GNOME Bugzilla)
IIUC,whatever be the container, h264parse should get autoplugged and parse
the relevant headers...

Yes.

In case of ts, h264 parser does update profile/level to src caps during
reconfigure. This is too late to change decoder in decodebin.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #8 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
The only solution I see to this is to expose the supported profiles when
registering the decoders.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #9 from Hyunjun Ko <***@igalia.com> ---
(In reply to Víctor Manuel Jáquez Leal from comment #8)

Post by "GStreamer" (GNOME Bugzilla)
The only solution I see to this is to expose the supported profiles when
registering the decoders.

I don't think it's possible to work. Even if we expose supported profiles to
caps, h264parse would not care during negotiation at first.

I think this is about only byte-stream/nal.
If we support only au alignment, this will work fine.
I know that's not an option, just for example :)
In this case, h264parse updates src caps at preroll time, then decodebin can
reject vaapidecoder and replace with another decoder.

IMHO, this issue looks h264parse's issue.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #10 from Holger Kaelberer <***@getslash.de> ---
Same problem here: MPEG-TS live stream with a h.264 profile 3.x, not supported
by our gst-vaapi/libva stack and the playbin hangs up with an autoplugged
vaapidecode.

IIUC there is now mechanism in place to have this handled automatically on a
decodebin-level, right?

What would be the recommended workaround by now? Listen to the src-caps of
h264parse and react on blacklisted h.264 profile-levels by pausing the pipeline
and replacing the h264-decoder manually?

Thanks,
Holger

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #11 from Hyunjun Ko <***@igalia.com> ---
(In reply to Holger Kaelberer from comment #10)

Post by "GStreamer" (GNOME Bugzilla)
Same problem here: MPEG-TS live stream with a h.264 profile 3.x, not
supported by our gst-vaapi/libva stack and the playbin hangs up with an
autoplugged vaapidecode.
IIUC there is now mechanism in place to have this handled automatically on a
decodebin-level, right?
What would be the recommended workaround by now? Listen to the src-caps of
h264parse and react on blacklisted h.264 profile-levels by pausing the
pipeline and replacing the h264-decoder manually?
Thanks,
Holger

Workaround is forcing to expose only au alignment in sink caps for h264
vaapidecoder.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #12 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Just played a bit on this.

(In reply to Hyunjun Ko from comment #11)

Post by "GStreamer" (GNOME Bugzilla)
Workaround is forcing to expose only au alignment in sink caps for h264
vaapidecoder.

AFAIK, the problem using intel's backend, with this particular stream (in
comment #2) is that this VA driver doesn't provide baseline profile (see bug
#732213).

Otherwise you could play that stream with this command (by forcing the caps):

gst-launch-1.0 filesrc location=bug770535.ts ! tsdemux ! h264parse !
video/x-h264, alignment=au ! vaapih264dec

Still it bails out because of the unsupported profile. Are there other sample
with different profile?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Summary|Playbin fails to decode |autoconnect fails with
|stream when gst-vaapi is |tsdemux and h264parse, it
|installed |didn't try other decoders

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

--- Comment #13 from Hyunjun Ko <***@igalia.com> ---
(In reply to Víctor Manuel Jáquez Leal from comment #12)

Post by "GStreamer" (GNOME Bugzilla)
Just played a bit on this.
(In reply to Hyunjun Ko from comment #11)

Post by "GStreamer" (GNOME Bugzilla)
Workaround is forcing to expose only au alignment in sink caps for h264
vaapidecoder.

AFAIK, the problem using intel's backend, with this particular stream (in
comment #2) is that this VA driver doesn't provide baseline profile (see bug
#732213).

I know va doesn't support baseline profile and this issue is originated from it
:)

The workaround(or maybe hack?) that I mentioned above is to make decodebin
choosing another decoder when vaapi decoder fails.

See the following. This is what I meant for "workaround" and you can see
decodebin succed in replacing with another decoder. This is why I suspect
h264parse.

diff --git a/gst/vaapi/gstvaapidecodebin.c b/gst/vaapi/gstvaapidecodebin.c
index 295c12b..1708e89 100644
--- a/gst/vaapi/gstvaapidecodebin.c
+++ b/gst/vaapi/gstvaapidecodebin.c
@@ -84,7 +84,7 @@ static const char gst_vaapi_decode_bin_sink_caps_str[] =
GST_CAPS_CODEC("video/x-divx")
GST_CAPS_CODEC("video/x-xvid")
GST_CAPS_CODEC("video/x-h263")
- GST_CAPS_CODEC("video/x-h264")
+ GST_CAPS_CODEC("video/x-h264, alignment=au")
#if USE_H265_DECODER
GST_CAPS_CODEC("video/x-h265")
#endif

diff --git a/gst/vaapi/gstvaapidecode.c b/gst/vaapi/gstvaapidecode.c
index c2bb42a..ea961d8 100644
--- a/gst/vaapi/gstvaapidecode.c
+++ b/gst/vaapi/gstvaapidecode.c
@@ -68,7 +68,7 @@ static const char gst_vaapidecode_sink_caps_str[] =
GST_CAPS_CODEC("video/x-divx")
GST_CAPS_CODEC("video/x-xvid")
GST_CAPS_CODEC("video/x-h263")
- GST_CAPS_CODEC("video/x-h264")
+ GST_CAPS_CODEC("video/x-h264, alignment=au")
#if USE_H265_DECODER
GST_CAPS_CODEC("video/x-h265")
#endif
@@ -114,7 +114,7 @@ static const GstVaapiDecoderMap vaapi_decode_map[] = {
{GST_VAAPI_CODEC_MPEG4, GST_RANK_PRIMARY, "mpeg4",
"video/mpeg, mpegversion=4"},
{GST_VAAPI_CODEC_H263, GST_RANK_PRIMARY, "h263", "video/x-h263"},
- {GST_VAAPI_CODEC_H264, GST_RANK_PRIMARY, "h264", "video/x-h264"},
+ {GST_VAAPI_CODEC_H264, GST_RANK_PRIMARY, "h264", "video/x-h264,
alignment=au"},
{GST_VAAPI_CODEC_VC1, GST_RANK_PRIMARY, "vc1",
"video/x-wmv, wmvversion=3, format={WMV3,WVC1}"},
#if USE_VP8_DECODER

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Whiteboard| |P2

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770535

GStreamer system administrator <***@gstreamer.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |OBSOLETE

--- Comment #14 from GStreamer system administrator <***@gstreamer.net> ---
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been
closed from further activity.

You can subscribe and participate further through the new bug through this link
to our GitLab instance:
https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/43.