[Bug 770528] New: tsdemux: add support ATSC EAC3

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Bug ID: 770528
Summary: tsdemux: add support ATSC EAC3
Classification: Platform
Product: GStreamer
Version: git master
OS: Linux
Status: NEW
Severity: normal
Priority: Normal
Component: gst-plugins-bad
Assignee: gstreamer-***@lists.freedesktop.org
Reporter: ***@gmail.com
QA Contact: gstreamer-***@lists.freedesktop.org
GNOME version: ---

EAC3 bit streams shall be identified with a stream_type value of 0x87 when
transmitted as PES streams conforming to ATSC-published standards. It is
specified in ATSC Standard A/52, so I added it to support this type.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

--- Comment #1 from Wonchul Lee <***@gmail.com> ---
Created attachment 334325
--> https://bugzilla.gnome.org/attachment.cgi?id=334325&action=edit
tsdemux: support ATSC EAC3

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Olivier Crête <***@ocrete.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
CC| |***@ocrete.ca
Resolution|--- |FIXED

--- Comment #2 from Olivier Crête <***@ocrete.ca> ---
Merged:

commit 35d147930d5ab1c3549e4413202954f21a7db0fa
Author: Wonchul Lee <***@collabora.com>
Date: Mon Aug 29 11:40:54 2016 +0900

tsdemux: Support ATSC EAC3

EAC3 bit streams shall be identified with a stream_type value of 0x87 when
transmitted as PES streams conforming to ATSC-published standards. It is
specified
in ATSC Standard A/52.

https://bugzilla.gnome.org/show_bug.cgi?id=770528

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Olivier Crête <***@ocrete.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|git master |1.9.3

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Olivier Crête <***@ocrete.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #334325|none |committed
status| |

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Edward Hervey <***@bilboed.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
CC| |***@bilboed.com
Resolution|FIXED |---

--- Comment #3 from Edward Hervey <***@bilboed.com> ---
0x87 is only valid when used on ATSC streams. Streams based on other standards
(base standard, DVB, ISDB, ...) could use this stream_id for other reasons.

We should have a way to identify such streams (presence of ATSC-specific
identifiers or descriptors ?)

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

--- Comment #4 from Wonchul Lee <***@gmail.com> ---
Oh, I didn't know, I referred to stream type table here.
https://en.wikipedia.org/wiki/Program-specific_information
Does this type stream (0x87) could be the other standards for some reason?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

--- Comment #5 from Wonchul Lee <***@gmail.com> ---
Created attachment 335438
--> https://bugzilla.gnome.org/attachment.cgi?id=335438&action=edit
tsdemux: Add conditions to identify ATSC EAC3

Added to identify '0x87' stream type as ATSC E-AC3 using both of descriptor and
registration ID, but I'd like to leave it to identify as ATSC E-AC3 stream even
if it doesn't contain relevant values with printing warning message. Then maybe
we could add each use case (base standards, ISDB, DVB, etc) later because it
has not been used in Tsdemux except blue-ray case.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

--- Comment #6 from Edward Hervey <***@bilboed.com> ---
(In reply to Wonchul Lee from comment #4)

Post by "GStreamer" (GNOME Bugzilla)
Oh, I didn't know, I referred to stream type table here.
https://en.wikipedia.org/wiki/Program-specific_information
Does this type stream (0x87) could be the other standards for some reason?

Unfortunately that page isn't wrong nor right. It's just missing to specify
which specs the elemtary stream type comes from.

The base standard for mpeg-ts is ISO/IEC 13818-1 (or H.222.0) [0]. You can
*not* go against that one. All other standards will abide by that one, and
there are various ways you can create derived standards without interfering
with those.

Now, You'll find the base elementary stream types in there. Note in there that
0x24-0x7E are *reserved* for ITU/ISO usage (so only a spec from ITU or ISO can
use them in derived standards). Furthermore 0x80-0xff are called "User
Private". That means that derived standards *can* (and will) use those to
define other kind of streams.

There are quite a few "derived" standards. DVB, ATSC, SCTE, BDMV (Bluray
standard), ISDB, HDV, ... Where it gets tricky is that, while they all use the
"User Private" range (if needed) to define new stream types (as to not
interfere with the base standard), they might have different meaning between
standards ! Oh, and for total enjoyment, some specs aren't available.

In order to know what the stream types in the User Private range correspond to,
you can't just blindly look at the stream_type. You also need to look at extra
information to figure out to which spec it corresponds to.

The various tricks are:
1) A "Registration ID Descriptor" in the top-level descriptor list of a PMT.
That essentially defines the "type" of specification that is being followed by
the stream. That's the safest/best bet. See how we deal with bluray
stream_types for examples in tsdemux. Sadly not all specifications use that,
let alone bad muxers/content-creators.

2) A "Registration ID Descriptor" in stream-level descriptor list of a PMT.
Generally used in the private PES packet stream types. It's a supplementary
information to tell you what the exact type is. See the examples in tsdemux
again.

3) Magic. No, not really, but you get to the point where it's a combination of
specs that were badly thought about to not conflict with other specifications
(early version of DigiCipher and SCTE) or come from crap content creator (yes,
there are a lot of those). So you need to figure out another way to "identify"
from what specification it corresponds to, in order to avoid mis-recognizing
such streams. Some tips could be to check for the presence of some streams that
only/don't exist in other standards (the ATSC 0x1ffb PSI stream for example).

[0] Latest free copy available here :
https://www.itu.int/rec/T-REC-H.222.0-201206-S/en

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Olivier Crête <***@ocrete.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #335438|none |needs-work
status| |

--- Comment #7 from Olivier Crête <***@ocrete.ca> ---
Review of attachment 335438:
--> (https://bugzilla.gnome.org/review?bug=770528&attachment=335438)

::: gst/mpegtsdemux/tsdemux.c
@@ +1517,3 @@
+ }
+
+ GST_WARNING ("E-AC3 stream type found but no guaranteed "

GST_WARNING_OBJECT (...)
maybe even GST_ELEMENT_WARNING() ?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Wonchul Lee <***@gmail.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #335438|needs-work |none
status| |
Attachment #335438|0 |1
is obsolete| |

--- Comment #8 from Wonchul Lee <***@gmail.com> ---
Created attachment 335609
--> https://bugzilla.gnome.org/attachment.cgi?id=335609&action=edit
tsdemux: Add conditions to identify ATSC EAC3

Thanks for the comprehensive details Edward. I fixed the previous patch to use
GST_ELEMENT_WARNING first. Some conditions had been added to identify ATSC eac3
audio stream based on the top-level/steam-level descriptors in the PMT, but not
yet checked specific (the ATSC 0x1ffb PSI stream) that exist most of the others
standards specifications.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Sebastian Dröge (slomo) <***@coaxion.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@coaxion.net
Target Milestone|1.9.3 |1.9.90

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Sebastian Dröge (slomo) <***@coaxion.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|1.9.90 |1.10.0
Severity|normal |blocker

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Olivier Crête <***@ocrete.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #335609|none |committed
status| |

--- Comment #9 from Olivier Crête <***@ocrete.ca> ---
Comment on attachment 335609
--> https://bugzilla.gnome.org/attachment.cgi?id=335609
tsdemux: Add conditions to identify ATSC EAC3

Merged

commit 69b866326ea69afe348c274b3b84e135468a567f
Author: Wonchul Lee <***@collabora.com>
Date: Tue Sep 13 22:07:05 2016 +0900

tsdemux: Add conditions to identify ATSC EAC3

https://bugzilla.gnome.org/show_bug.cgi?id=770528

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Olivier Crête <***@ocrete.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution|--- |FIXED
Target Milestone|1.10.0 |1.9.91

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=770528

Sebastian Dröge (slomo) <***@coaxion.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
Target Milestone|1.9.91 |1.10.0