Discussion:
[Bug 778163] New: parsebin: Don't expose endpad if corresponding decoder is not supported
(too old to reply)
"GStreamer" (GNOME Bugzilla)
2017-02-04 04:29:32 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

Bug ID: 778163
Summary: parsebin: Don't expose endpad if corresponding decoder
is not supported
Classification: Platform
Product: GStreamer
Version: 1.11.1
OS: Linux
Status: NEW
Severity: normal
Priority: Normal
Component: gst-plugins-base
Assignee: gstreamer-***@lists.freedesktop.org
Reporter: ***@gmail.com
QA Contact: gstreamer-***@lists.freedesktop.org
GNOME version: ---

Created attachment 344907
--> https://bugzilla.gnome.org/attachment.cgi?id=344907&action=edit
Dot graph for playbin2 with missing audio decoder

Hello All.
In Playbin3, negotiation failure is observed if we can not find available audio
decoder.
Actually, ACCEPT CAPS query returns 'not accepted' in gstdecodebin3-parse.c
However, In Playbin2, endpad of a chain is not exposed from decodebin when
corresponding decoder is missing(e.g. unknown type).
And then, playback works well without unknown audio type.

So, I think it is more better and good to play likes playbin2.

Thanks.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-04 04:32:00 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #1 from HoonHee Lee <***@gmail.com> ---
Created attachment 344908
--> https://bugzilla.gnome.org/attachment.cgi?id=344908&action=edit
parsebin: Don't expose endpad if corresponding decoder is not supported

Remove the endpad if decoder is missing before parsebin is exposing all
endpads.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-04 04:33:22 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #2 from HoonHee Lee <***@gmail.com> ---
Created attachment 344909
--> https://bugzilla.gnome.org/attachment.cgi?id=344909&action=edit
Dot graph for playbin3 with missing audio decoder in fixed version

Dot graph for playbin3 with missing audio decoder in fixed version
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-04 04:41:15 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #3 from HoonHee Lee <***@gmail.com> ---
Created attachment 344910
--> https://bugzilla.gnome.org/attachment.cgi?id=344910&action=edit
gst-play: make unsupported decoder by adjusting rank

If you use make_unsupported_codec () function, you can adjust rank of decoder
what you don't want to support as NONE.

Thanks.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-04 12:20:27 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

Edward Hervey <***@bilboed.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@bilboed.com

--- Comment #4 from Edward Hervey <***@bilboed.com> ---
This will be quite unfortunate if the sink supports that format (like
passthrough HDMI) but there is no decoder present.

Also this will totally break the behaviour of parsebin outside of decodebin3.
One might want to use parsebin for re-muxing purposes for example.

I would say the solution here is to (finally) implement the signals that were
removed from db3/pb3 : https://bugzilla.gnome.org/show_bug.cgi?id=769079
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-06 04:40:19 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #5 from HoonHee Lee <***@gmail.com> ---
(In reply to Edward Hervey from comment #4)
Post by "GStreamer" (GNOME Bugzilla)
This will be quite unfortunate if the sink supports that format (like
passthrough HDMI) but there is no decoder present.
Also this will totally break the behaviour of parsebin outside of
decodebin3. One might want to use parsebin for re-muxing purposes for
example.
I would say the solution here is to (finally) implement the signals that
were removed from db3/pb3 : https://bugzilla.gnome.org/show_bug.cgi?id=769079
As general purpose for parsebin, I agree with your opinion.

Let's pretend that there is a stream 1 video and 1 audio but, we can not find
available decoder for the audio.
When playbin2, we can play video without audio.
But, negotiation failure is happened and can not play in playbin3.
So, I think that most developers and end users may recognize this is a kind of
side effect or regression compared with playbin2.
Also, they want to play without audio in playbin3.


1) Can you acceptable my approach if I add new property likes 'needs-decoder'
in parsebin and then checking expose endpad or not?

2) I am not sure I can cope unsupported decoder after I implement the removed
signals from https://bugzilla.gnome.org/show_bug.cgi?id=769079.
Can I get a hint how to handle it with these signals?

Below is my observations after non available audio stream is exposed in
parsebin.
First one, we may have to handle then negotiation failure problem by Accept
Caps query from gstdecodebin3-parse.c.
Second, we may have to synchronize between available and non-available audio
stream by clock(e.g. running time) and drop buffers likes MQ or input-selector
if non-available audio stream is exposed in parsebin.
Third, we have have to handle preroll state by sending gap event when
transiting paused state if non-available audio stream is exposed in parsebin.

It would be very helpful if either my patch would be acceptable or you give a
hint to use these removed signals as well.

Thanks.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-06 18:28:21 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

Olivier Crête <***@ocrete.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@ocrete.ca

--- Comment #6 from Olivier Crête <***@ocrete.ca> ---
Shouldn't playbin give parsebin also the caps of the sink, then the check for
"decoder" should also check for a sink too?

Something like: ((IS_FACTORY (x) && (gst_element_factory_list_is_type(x,
.._DECODER) || gst_element_factory_list_is_type(x, .._SINK))) || (IS_ELEMENT(x)
&& GST_ELEMENT_IS_SINK(x)))
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-10 04:32:20 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #7 from HoonHee Lee <***@gmail.com> ---
(In reply to Olivier Crête from comment #6)
Post by "GStreamer" (GNOME Bugzilla)
Shouldn't playbin give parsebin also the caps of the sink, then the check
for "decoder" should also check for a sink too?
Something like: ((IS_FACTORY (x) && (gst_element_factory_list_is_type(x,
.._DECODER) || gst_element_factory_list_is_type(x, .._SINK))) ||
(IS_ELEMENT(x) && GST_ELEMENT_IS_SINK(x)))
Hello Olivier Crête
Sorry, I don't get it what you mean exactly.
You want me to use autoplug-{factories/select/continue} signals in playbin3?
It would be helpful if you give me detailed explanation.

Thanks.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-10 16:59:18 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #8 from Olivier Crête <***@ocrete.ca> ---
Yes, I think that when we want to add support for decoding sinks in playbin3,
we'll to have parsebin delegate the decision on whether this is an acceptable
caps to expose or not to the parent playbin3 which knows about the sinks.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-13 05:57:40 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #9 from HoonHee Lee <***@gmail.com> ---
(In reply to Olivier Crête from comment #8)
Post by "GStreamer" (GNOME Bugzilla)
Yes, I think that when we want to add support for decoding sinks in
playbin3, we'll to have parsebin delegate the decision on whether this is an
acceptable caps to expose or not to the parent playbin3 which knows about
the sinks.
Hello Olivier Crête
Thanks for your reply.

I have one more question.
In latest version, parsebin emits "autoplug-factories" to decodebin3 in
analyze_new_pad () but, decodebin3 does not add callback function likes
"g_signal_connect (parsebin, "autoplug-factories"...)".
Thus, decodebin3 can not response the "autoplug-factories" signal and then
can not emit "autoplug-factories" signal to playbin3.

Is there any intention not to implement "autoplug-factories" signal between
decodebin3 and playbin3? or just missed to implement?

Thanks.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-13 07:29:00 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #10 from Edward Hervey <***@bilboed.com> ---
(In reply to Olivier Crête from comment #6)
Post by "GStreamer" (GNOME Bugzilla)
Something like: ((IS_FACTORY (x) && (gst_element_factory_list_is_type(x,
.._DECODER) || gst_element_factory_list_is_type(x, .._SINK))) ||
(IS_ELEMENT(x) && GST_ELEMENT_IS_SINK(x)))
You would need to instantiate the sink (potentially with the correct device
id) to know exactly what the sink supports in terms of caps.

(In reply to HoonHee Lee from comment #9)
Post by "GStreamer" (GNOME Bugzilla)
I have one more question.
In latest version, parsebin emits "autoplug-factories" to decodebin3 in
analyze_new_pad () but, decodebin3 does not add callback function likes
"g_signal_connect (parsebin, "autoplug-factories"...)".
Thus, decodebin3 can not response the "autoplug-factories" signal and then
can not emit "autoplug-factories" signal to playbin3.
Is there any intention not to implement "autoplug-factories" signal between
decodebin3 and playbin3? or just missed to implement?
The reason this was removed or deactivated is because we want to use a more
correct/streamlined signal for this (as mentionned in the review signals bug).

One idea could be the following:
1) parsebin always exposes everything (regardless of decoder being present or
not)
2) streams go through multiqueue
3) When deciding whether to use a decoder or not (downstream of multiqueue),
fire the new signal.

This signal would provide:
* The GstStream object
* The list of decoders (if any) that decodebin3 will try to link

The callback can return:
* The re-sorted list of decoders (potentially with new elements in it)
* Whether the stream should:
1) Be exposed as-is (without any decoder)
2) Be discarded (never exposed, i.e. blacklisted, until the stream
changed)
3) Try to plug in the decoders in the provided order.

This is the same thing as what was done previously. But just with one signal
instead of 4.

This should solve your problem as far as I understand.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-13 07:59:42 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

Gabby Park <***@gmail.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@gmail.com
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-14 10:54:52 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #11 from HoonHee Lee <***@gmail.com> ---

(In reply to Edward Hervey from comment #10)
Post by "GStreamer" (GNOME Bugzilla)
(In reply to Olivier Crête from comment #6)
Post by "GStreamer" (GNOME Bugzilla)
Something like: ((IS_FACTORY (x) && (gst_element_factory_list_is_type(x,
.._DECODER) || gst_element_factory_list_is_type(x, .._SINK))) ||
(IS_ELEMENT(x) && GST_ELEMENT_IS_SINK(x)))
You would need to instantiate the sink (potentially with the correct
device id) to know exactly what the sink supports in terms of caps.
(In reply to HoonHee Lee from comment #9)
Post by "GStreamer" (GNOME Bugzilla)
I have one more question.
In latest version, parsebin emits "autoplug-factories" to decodebin3 in
analyze_new_pad () but, decodebin3 does not add callback function likes
"g_signal_connect (parsebin, "autoplug-factories"...)".
Thus, decodebin3 can not response the "autoplug-factories" signal and then
can not emit "autoplug-factories" signal to playbin3.
Is there any intention not to implement "autoplug-factories" signal between
decodebin3 and playbin3? or just missed to implement?
The reason this was removed or deactivated is because we want to use a
more correct/streamlined signal for this (as mentionned in the review
signals bug).
1) parsebin always exposes everything (regardless of decoder being present
or not)
2) streams go through multiqueue
3) When deciding whether to use a decoder or not (downstream of
multiqueue), fire the new signal.
* The GstStream object
* The list of decoders (if any) that decodebin3 will try to link
* The re-sorted list of decoders (potentially with new elements in it)
1) Be exposed as-is (without any decoder)
2) Be discarded (never exposed, i.e. blacklisted, until the stream
changed)
3) Try to plug in the decoders in the provided order.
This is the same thing as what was done previously. But just with one
signal instead of 4.
This should solve your problem as far as I understand.
Hello Edward Hervey
Thanks for your detailed guide.
I understood most of things that you suggested.

For 2), You want to maintain the GstStream which is not exposed in current
collection? Or Removing the GstStream in current collection and posting it?
I think it is important to show either all tracks or supported tracks to player
side.

Please feedback about my question.
Thanks.

Thanks.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-14 13:55:52 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #12 from Edward Hervey <***@bilboed.com> ---
(In reply to HoonHee Lee from comment #11)
Post by "GStreamer" (GNOME Bugzilla)
Post by "GStreamer" (GNOME Bugzilla)
* The re-sorted list of decoders (potentially with new elements in it)
1) Be exposed as-is (without any decoder)
2) Be discarded (never exposed, i.e. blacklisted, until the stream
changed)
3) Try to plug in the decoders in the provided order.
For 2), You want to maintain the GstStream which is not exposed in current
collection? Or Removing the GstStream in current collection and posting it?
I think it is important to show either all tracks or supported tracks to
player side.
Good point.

One could argue that since something *outside* of playbin3 decided to
blacklist a stream, it shouldn't appear in the resulting collection (which btw,
would have to be re-posted by playbin3).

Let me think about it a bit more and collect some other opinions.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-15 09:03:28 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #13 from Edward Hervey <***@bilboed.com> ---
Another argument that popped up is the following: Even if the stream is not
exploitable by a player/app .. one might still want to be able to list it in
the application (and maybe have it greyed out).
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-15 23:39:00 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #14 from HoonHee Lee <***@gmail.com> ---
(In reply to Edward Hervey from comment #13)
Post by "GStreamer" (GNOME Bugzilla)
Another argument that popped up is the following: Even if the stream is not
exploitable by a player/app .. one might still want to be able to list it in
the application (and maybe have it greyed out).
Thanks for collecting another opinion.
I agree with this behaviour. Because, I think it's more general concept.

Eventually, decodebin3(playbin3) would switch track between available
streams(tracks) and we don't need to worry about handling preroll
state(async-done) when non-available stream(track) is selected and pipeline
tries to change paused state.

Do you agree with following behaviour if you also want to maintain all streams?

1) If we have a stream that contains 1-video and 2-audios and 1st audio stream
is not available(not supported decoder or in blacklist).
How about changing automatically from 1st audio to 2nd audio when initial
pre-rolling? I think that changing automatically is more better instead of mute
sound.

Thanks.
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
"GStreamer" (GNOME Bugzilla)
2017-02-16 07:53:40 UTC
Permalink
Raw Message
https://bugzilla.gnome.org/show_bug.cgi?id=778163

--- Comment #15 from Edward Hervey <***@bilboed.com> ---
(In reply to HoonHee Lee from comment #14)
Post by "GStreamer" (GNOME Bugzilla)
1) If we have a stream that contains 1-video and 2-audios and 1st audio
stream is not available(not supported decoder or in blacklist).
How about changing automatically from 1st audio to 2nd audio when initial
pre-rolling? I think that changing automatically is more better instead of
mute sound.
For this use-case, the way I see it happening should be:
* The 1st audio stream is initially selected automatically
* When that 1st audio stream arrives downstream of multiqueue, decodebin3
fires the 'stream-handling' signal (or whatever we'll call it) with no
available decoder factories
* playbin3 can't handle it, returns that it does not want to have it exposed
* decodebin3 marks it blacklisted for now, and re-triggers the initial
selection (update_requested_selection) which would avoid using that stream (in
the same way that we sort by SELECT/UNSELECT streams) => The 2nd audio stream
ends up being selected instead.
* decodebin3 triggers the stream switch update (it sees the current stream it
wanted to expose isn't selected, and switches)

That would solve the issue of exposing a stream that can't be handled

It leaves the issue of:
* What to do if the app decide to select that stream. How do we notify the
user/app ? Do we still expose it if it was requested ?
--
You are receiving this mail because:
You are the QA Contact for the bug.
You are the assignee for the bug.
Loading...