[Bug 792034] New: dmabuf import with icamerasrc plugin does not working with gst-vaapi-1.12.2

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Bug ID: 792034
Summary: dmabuf import with icamerasrc plugin does not working
with gst-vaapi-1.12.2
Classification: Platform
Product: GStreamer
Version: 1.12.2
OS: Linux
Status: NEW
Severity: normal
Priority: Normal
Component: gstreamer-vaapi
Assignee: gstreamer-***@lists.freedesktop.org
Reporter: ***@intel.com
QA Contact: gstreamer-***@lists.freedesktop.org
CC: ***@gmail.com, ***@igalia.com
GNOME version: ---

The dmabuf import previous working with gst-vaapi 1.8.3 version.
Aware that in gst-vaapi 1.12.2 version got a lot of design have been changing
and no longer working with icamerasrc plugin in 1.12.2 version.

Previously we added the workaround to get dmabuf import with icamerasrc plugin
working with gst-vaapi.

Now to get it properly working, icamerasrc plugin developer will working with
gst-vaapi developer what needs to done and how to handle it.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #1 from sreerenj <***@gmail.com> ---
We should remove the super hack in gstremaer-vaapi which is to support the
dmabuf export from vaapi element to upstream (eg: gst-launch-1.0 v4l2src
io-mode=dmabuf-import ! vaapipostproc format=nv12 ! fakesink):
https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst/vaapi/gstvaapipluginbase.c#n425

We had some discussion regarding the same:
https://bugzilla.gnome.org/show_bug.cgi?id=765435#c7

Not sure whether v4l2src already implemented it, I guess not.

BTW, IIUC regression is not because of gstreamer-vaapi code, but because of
icamerasrc changes, right?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Nicolas Dufresne (stormer) <***@ndufresne.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@ndufresne.ca

--- Comment #2 from Nicolas Dufresne (stormer) <***@ndufresne.ca> ---
The hack is no-op now since dmabuf export is default in V4L2 now. If icamsrc do
the same, then it will work for both, I hope dmabuf caps feature won't be
needed.

Can you put a link to some icamsrc code, it's not clear where this is hosted,
if it's open source, what hardware is available with that can/ISP, and which
drivers (kernel and user space) are needed.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #3 from sreerenj <***@gmail.com> ---
(In reply to Nicolas Dufresne (stormer) from comment #2)

Post by "GStreamer" (GNOME Bugzilla)
The hack is no-op now since dmabuf export is default in V4L2 now. If icamsrc
do the same, then it will work for both, I hope dmabuf caps feature won't be
needed.

The dmabuf export from source element can work without any hack (both v4l2 and
icamerasrc) in gstreamer-vaapi.

But IIUC, they want dmabuf-import and I don't think it can work without having
the hack in gstreamer-vaapi (I guess it is the same situation for v4l2src too).

Post by "GStreamer" (GNOME Bugzilla)
Can you put a link to some icamsrc code, it's not clear where this is
hosted, if it's open source, what hardware is available with that can/ISP,
and which drivers (kernel and user space) are needed.

Here is the repo: https://github.com/intel/icamerasrc
I told them to comment here, hopefully, we will get more comments soon :)

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #4 from Nicolas Dufresne (stormer) <***@ndufresne.ca> ---
Ok, indeed automatic importation from vaapi dmabuf to the camera is not
automatically supported. It is also broken in v4l2src, so if there is a hack to
enable it, please remove right away, I'll remove it if it is still there when
I'm back in Jan (unless I have a patch that adds the missing validation).

What is complicated is that you need to tell the driver the strides and offsets
of the allocated images. And as there is currently no kernel API to help with
negotiating the allocation between two drivers, these need to match on first
try. Drivers like UVC, that could support arbitrary alignement simply don't
implement it, and enforce a certain alignment. V4L2 API is not very friendly as
it merge the format and alignment ll together.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #5 from sreerenj <***@gmail.com> ---
(In reply to Nicolas Dufresne (stormer) from comment #4)

Post by "GStreamer" (GNOME Bugzilla)
Ok, indeed automatic importation from vaapi dmabuf to the camera is not
automatically supported. It is also broken in v4l2src, so if there is a hack
to enable it, please remove right away, I'll remove it if it is still there
when I'm back in Jan (unless I have a patch that adds the missing
validation).

Does it mean to remove the "dmabuf-import" option completely from v4l2src (+
the corresponding code from gstreamer-vaapi)?

Post by "GStreamer" (GNOME Bugzilla)
What is complicated is that you need to tell the driver the strides and
offsets of the allocated images. And as there is currently no kernel API to
help with negotiating the allocation between two drivers, these need to
match on first try. Drivers like UVC, that could support arbitrary
alignement simply don't implement it, and enforce a certain alignment. V4L2
API is not very friendly as it merge the format and alignment ll together.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #6 from Nicolas Dufresne (stormer) <***@ndufresne.ca> ---
(In reply to sreerenj from comment #5)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to Nicolas Dufresne (stormer) from comment #4)
Does it mean to remove the "dmabuf-import" option completely from v4l2src (+

That I can't, it's in the API now.

Post by "GStreamer" (GNOME Bugzilla)
the corresponding code from gstreamer-vaapi)?

That's the code I'll remove as it's unsafe (or at least make sure it's not used
without user knowledge of the risk). The fix in the source will cause pipeline
error in many cases, instead of crash or corrupted image, it's not much better.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #7 from sreerenj <***@gmail.com> ---
(In reply to Nicolas Dufresne (stormer) from comment #6)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to sreerenj from comment #5)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to Nicolas Dufresne (stormer) from comment #4)
Does it mean to remove the "dmabuf-import" option completely from v4l2src (+

That I can't, it's in the API now.

Post by "GStreamer" (GNOME Bugzilla)
the corresponding code from gstreamer-vaapi)?

That's the code I'll remove as it's unsafe (or at least make sure it's not
used without user knowledge of the risk). The fix in the source will cause
pipeline error in many cases, instead of crash or corrupted image, it's not
much better.

Tha hack in gstreamer-vaapi is for checking whether user explicitly enabled
upstream element's "io-mode" to "dmabuf-import". Yes, it a super hack, which
was added a long time ago when there were no dmabuf negotiation support in
GStreamer and inherited all the way till 1.12.x.


BTW, good to see "dmabuf-export" as default in v4l2src. Great!
So I am fine with removing the hack for v4l2 from gstreamer-vaapi, but please
keep it for "icamerasrc", hopefully we can add some other fix than the nasty
hack.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #9 from ***@intel.com ---
(In reply to sreerenj from comment #1)

Post by "GStreamer" (GNOME Bugzilla)
We should remove the super hack in gstremaer-vaapi which is to support the
dmabuf export from vaapi element to upstream (eg: gst-launch-1.0 v4l2src
https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst/vaapi/
gstvaapipluginbase.c#n425
https://bugzilla.gnome.org/show_bug.cgi?id=765435#c7
Not sure whether v4l2src already implemented it, I guess not.
BTW, IIUC regression is not because of gstreamer-vaapi code, but because of
icamerasrc changes, right?

(In reply to sreerenj from comment #1)

Post by "GStreamer" (GNOME Bugzilla)
We should remove the super hack in gstremaer-vaapi which is to support the
dmabuf export from vaapi element to upstream (eg: gst-launch-1.0 v4l2src
https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst/vaapi/
gstvaapipluginbase.c#n425
https://bugzilla.gnome.org/show_bug.cgi?id=765435#c7
Not sure whether v4l2src already implemented it, I guess not.
BTW, IIUC regression is not because of gstreamer-vaapi code, but because of
icamerasrc changes, right?

(In reply to sreerenj from comment #1)

Post by "GStreamer" (GNOME Bugzilla)
We should remove the super hack in gstremaer-vaapi which is to support the
dmabuf export from vaapi element to upstream (eg: gst-launch-1.0 v4l2src
https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/gst/vaapi/
gstvaapipluginbase.c#n425
https://bugzilla.gnome.org/show_bug.cgi?id=765435#c7
Not sure whether v4l2src already implemented it, I guess not.
BTW, IIUC regression is not because of gstreamer-vaapi code, but because of
icamerasrc changes, right?

Can't simply say it caused by icamerasrc change. The icamerasrc is only
validated on certain version of gstreamer like 1.8.3. For about one year, the
gstreamer version stays on this version. And we do the upgrade gstreamer
version to 1.12.2, the issue is captured.
Go back to check the history, we found
1 code refactoring in icamerasrc to support multi streams and icamerasrc is not
inherited from PUSHSRC.
2 vaapi remove the option GST_BUFFER_POOL_OPTION_DMABUF_MEMORY.

Now it becomes the allocator and pool is empty when using the dmabuf import.

Since the memory:dmabuf is still in development. I suggest to still add the
hack in vappi for icamerasrc to enable the dmabuf import.

When the memory:dmabuf filter is enabled, the hack can be removed throughly.

A hack of hack patch is attached.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

***@intel.com changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@intel.com

--- Comment #8 from ***@intel.com ---
Created attachment 366233
--> https://bugzilla.gnome.org/attachment.cgi?id=366233&action=edit
hack for icamerasrc to enable the dmabuf import with vaapi

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #10 from Lim Siew Hoon <***@intel.com> ---
Hi,
Any update on this?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #11 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Sorry for the delay.

I have a different proposal. I haven't tried it, but it is an idea:

Instead of using that hack, the gstreamer-vaapi elements will add, in the
propose_allocation() method's query, two allocators: the normal one for VAAPI
surfaces and another dmabuf-based.

Thus, when the icamerasrc processes the offered buffer pool, and dmabuf-import
is enabled, will look for the dmabuf-base allocator in the query, and will set
that allocator in the buffer pool configuration.

what do you think?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #12 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
said that, that would go to master. For 1.12, dunno, perhaps merge that code
out-of-tree, but not in upstream.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #13 from sreerenj <***@gmail.com> ---
(In reply to Víctor Manuel Jáquez Leal from comment #11)

Post by "GStreamer" (GNOME Bugzilla)
Sorry for the delay.
Instead of using that hack, the gstreamer-vaapi elements will add, in the
propose_allocation() method's query, two allocators: the normal one for
VAAPI surfaces and another dmabuf-based.
Thus, when the icamerasrc processes the offered buffer pool, and
dmabuf-import is enabled, will look for the dmabuf-base allocator in the
query, and will set that allocator in the buffer pool configuration.
what do you think?

That could be the only way to do it :),
Two allocators or two separate pools?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #14 from sreerenj <***@gmail.com> ---
(In reply to Víctor Manuel Jáquez Leal from comment #12)

Post by "GStreamer" (GNOME Bugzilla)
said that, that would go to master. For 1.12, dunno, perhaps merge that code
out-of-tree, but not in upstream.

Yup, not in 1.12 for sure.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #15 from Lim Siew Hoon <***@intel.com> ---
(In reply to sreerenj from comment #14)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to Víctor Manuel Jáquez Leal from comment #12)

Post by "GStreamer" (GNOME Bugzilla)
said that, that would go to master. For 1.12, dunno, perhaps merge that code
out-of-tree, but not in upstream.

Yup, not in 1.12 for sure.

Currently, we are using it based on Gstreamer framework 1.12.2 version and
Gstreamer-vaapi 1.12.2 version. And also we are using libva v1.8-branch.
There is no plan for us to move to libva 2.0. But, I think we will be ok to
have the code patch can apply to 1.12 version and put on recipes side.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #16 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
(In reply to Lim Siew Hoon from comment #15)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to sreerenj from comment #14)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to Víctor Manuel Jáquez Leal from comment #12)

Post by "GStreamer" (GNOME Bugzilla)
said that, that would go to master. For 1.12, dunno, perhaps merge that code
out-of-tree, but not in upstream.

Yup, not in 1.12 for sure.

Currently, we are using it based on Gstreamer framework 1.12.2 version and
Gstreamer-vaapi 1.12.2 version. And also we are using libva v1.8-branch.
There is no plan for us to move to libva 2.0. But, I think we will be ok to
have the code patch can apply to 1.12 version and put on recipes side.

Does this mean that you need something else beside the posted patch?
(attachment 366233)

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #17 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Created attachment 366741
--> https://bugzilla.gnome.org/attachment.cgi?id=366741&action=edit
plugins: remove dmabuf-import hack

Remove the hack to check if an upstream element has enabled the
property io-mode enabled as dmabuf-import.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #18 from Nicolas Dufresne (stormer) <***@ndufresne.ca> ---
Review of attachment 366741:
--> (https://bugzilla.gnome.org/review?bug=792034&attachment=366741)

Thanks. This removes a bad friend.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #366741|none |committed
status| |

--- Comment #19 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Comment on attachment 366741
--> https://bugzilla.gnome.org/attachment.cgi?id=366741
plugins: remove dmabuf-import hack

Attachment 366741 pushed as 8688e81 - plugins: remove dmabuf-import hack

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #20 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
(In reply to Víctor Manuel Jáquez Leal from comment #19)

Comment on attachment 366741 [details] [review]
plugins: remove dmabuf-import hack
Attachment 366741 [details] pushed as 8688e81 - plugins: remove
dmabuf-import hack

This was merged in master, not in branch 1.12

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Summary|dmabuf import with |implement dmabuf-import
|icamerasrc plugin does not |negotiation with upstream
|working with |
|gst-vaapi-1.12.2 |

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #22 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Created attachment 368086
--> https://bugzilla.gnome.org/attachment.cgi?id=368086&action=edit
vaapivideobufferpool: add dmabuf pool option

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #21 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Created attachment 368085
--> https://bugzilla.gnome.org/attachment.cgi?id=368085&action=edit
vaapivideobufferpool: dynamically build options

Instead of return a static list of options, the array should be
constructed dinamycally with the options processed by the
configuration.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #23 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Created attachment 368087
--> https://bugzilla.gnome.org/attachment.cgi?id=368087&action=edit
plugins: propose a second pool with dmabuf capabilites

If the caps are raw of have the Dmabuf feature, a second pool is
proposed upstream with a dmabuf allocator and dmabuf
capabilities. Also the allocator is appended in the query.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Depends on| |793271, 789476

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #24 from Lim Siew Hoon <***@intel.com> ---
Ok, noted. We will start porting into gst-vaapi 1.12.2 version and verify with
icamerasrc plugin side once icamerasrc plugin rework is done.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #368085|0 |1
is obsolete| |

--- Comment #25 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Created attachment 368133
--> https://bugzilla.gnome.org/attachment.cgi?id=368133&action=edit
vaapivideobufferpool: dynamically build options

Instead of return a static list of options, the array should be
constructed dinamycally with the options processed by the
configuration.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #26 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Created attachment 368134
--> https://bugzilla.gnome.org/attachment.cgi?id=368134&action=edit
vaapivideobufferpool: handle dmabuf pool option

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #368087|0 |1
is obsolete| |

--- Comment #27 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
Created attachment 368135
--> https://bugzilla.gnome.org/attachment.cgi?id=368135&action=edit
plugins: propose a second pool with dmabuf capabilites

If the caps are raw or have the DMAbuf feature, a second pool is
proposed upstream with a DMAbuf allocator and capabilities.

It is the upstream obligation to look for the appropiate pool in
decide_allocation().

The DMABuf allocator is not appended in the query because it is not
certain if it has a custom alloc function or not.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #368086|0 |1
is obsolete| |

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Tim-Philipp Müller <***@zen.co.uk> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@zen.co.uk
Severity|normal |enhancement

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034
Bug 792034 depends on bug 789476, which changed state.

Bug 789476 Summary: vaapi plugins: append allocator in allocation query
https://bugzilla.gnome.org/show_bug.cgi?id=789476

What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution|--- |FIXED

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #28 from Lim Siew Hoon <***@intel.com> ---
(In reply to Víctor Manuel Jáquez Leal from comment #22)

Created attachment 368086 [details] [review]
vaapivideobufferpool: add dmabuf pool option

I can't find this macro in gst-plugins-base master branch for
GST_BUFFER_POOL_OPTION_VIDEO_DMABUF.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Depends on| |793270

--- Comment #29 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
(In reply to Lim Siew Hoon from comment #28)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to Víctor Manuel Jáquez Leal from comment #22)

Created attachment 368086 [details] [review] [review]
vaapivideobufferpool: add dmabuf pool option

I can't find this macro in gst-plugins-base master branch for
GST_BUFFER_POOL_OPTION_VIDEO_DMABUF.

You need to apply the patch in
https://bugzilla.gnome.org/show_bug.cgi?id=793270

It is under discussion and it may change in the future.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034
Bug 792034 depends on bug 789476, which changed state.

Bug 789476 Summary: vaapi plugins: append allocator in allocation query
https://bugzilla.gnome.org/show_bug.cgi?id=789476

What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |REOPENED
Resolution|FIXED |---

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034
Bug 792034 depends on bug 789476, which changed state.

Bug 789476 Summary: vaapi plugins: append allocator in allocation query
https://bugzilla.gnome.org/show_bug.cgi?id=789476

What |Removed |Added
----------------------------------------------------------------------------
Status|REOPENED |RESOLVED
Resolution|--- |FIXED

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Nicolas Dufresne (stormer) <***@ndufresne.ca> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@intel.com

--- Comment #30 from Nicolas Dufresne (stormer) <***@ndufresne.ca> ---
*** Bug 794247 has been marked as a duplicate of this bug. ***

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Julien Isorce <***@gmail.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@gmail.com

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #31 from deshui <***@intel.com> ---
i notice with all above patches, the sinkpad_allocator for plugin always be
video memory, not dma-buf memory.
to use the dma-buf import feature, icamsrc have to check the mem type, since
the memory type is video memory, not dma-buf memory, icamsrc plugin can't work
will.
so, would you like to tell me how to make the sinkpad_allocator be dma-buf
memory?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #32 from ***@intel.com ---
Any feekback to deshui's question?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #33 from Nicolas Dufresne (stormer) <***@ndufresne.ca> ---
(In reply to renwei.wu from comment #32)

Post by "GStreamer" (GNOME Bugzilla)
Any feekback to deshui's question?

It was broadcasted on multiple channels, also bugzilla is not a forum. The
answer is that icamsrc must negotiate memory:DMABuf caps feature if it cannot
always produce DMAbuf (like v4l2src does).

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #34 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
(In reply to deshui from comment #31)

Post by "GStreamer" (GNOME Bugzilla)
i notice with all above patches, the sinkpad_allocator for plugin always be
video memory, not dma-buf memory.
to use the dma-buf import feature, icamsrc have to check the mem type, since
the memory type is video memory, not dma-buf memory, icamsrc plugin can't
work will.
so, would you like to tell me how to make the sinkpad_allocator be dma-buf
memory?

Since there must two pools, it doesn't matter if plugin->sinkpad_allocator is
vaapi memory based because:

Two pools are offered upstream in propose_allocation(): one produces buffers
with vaapi memory and another with dmabuf memory.

The first one, will set the sinkpad_allocator and sinkpad_buffer_pool in
gstvaapipluginbase. But the second one is not hold by gstvaapipluginbase, it
is only offered and forgotten. Upstream element could hold it and use it,
otherwise it is unrrefed.

If the dmabuf-memory-based pool is held by upstream element (camerasrc should
be do this in this case), the source element will use that pool to produce the
output buffers and later they will be imported by vaapi elements as they were
imported before.

Read the patches proposed for v4l2src in bug #793271 for how I proposed to
handle this. The logic, basically is, if the io-mode property in v4l2src in
dmabuf-import, then check for all the offered pool in the query and if it is
there one with DMABUF property, it is hold and use.

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #35 from deshui <***@intel.com> ---
after apply the patches from bug #793271 and bug #793274 on master branch
other format works well except format NV12 WITH command
gst-launch-1.0 v4l2src io-mode=5 num-buffers=100 !
video/x-raw,format=NV12,width=1280,height=720 ! vaapipostproc ! filesink
location=1.raw
so， gstreamer-vappi seems not support NV12 now?
by the way, vappisink works well but filesink failed

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #36 from deshui <***@intel.com> ---
(In reply to deshui from comment #35)

Post by "GStreamer" (GNOME Bugzilla)
after apply the patches from bug #793271 and bug #793274 on master branch
other format works well except format NV12 WITH command
gst-launch-1.0 v4l2src io-mode=5 num-buffers=100 !
video/x-raw,format=NV12,width=1280,height=720 ! vaapipostproc ! filesink
location=1.raw
so， gstreamer-vappi seems not support NV12 now?
by the way, vappisink works well but filesink failed

sorry bug #793274 should be bug #792034

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #37 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
(In reply to deshui from comment #35)

Post by "GStreamer" (GNOME Bugzilla)
after apply the patches from bug #793271 and bug #793274 on master branch
other format works well except format NV12 WITH command
gst-launch-1.0 v4l2src io-mode=5 num-buffers=100 !
video/x-raw,format=NV12,width=1280,height=720 ! vaapipostproc ! filesink
location=1.raw
so， gstreamer-vappi seems not support NV12 now?

That's because my (and I assume yours) v4l2src device doesn't handle NV12

0:00:00.207682288 5580 0x1e652d0 DEBUG v4l2src
gstv4l2src.c:504:gst_v4l2src_negotiate:<v4l2src0> caps of src: video/x-raw,
format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)10/1; video/x-raw,
format=(string)YUY2, width=(int)960, height=(int)540,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1; video/x-raw,
format=(string)YUY2, width=(int)848, height=(int)480,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1; video/x-raw,
format=(string)YUY2, width=(int)640, height=(int)480,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)424, height=(int)240,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)352, height=(int)288,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)320, height=(int)180,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)1280, height=(int)720, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)960,
height=(int)540, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1,
15/1 }; image/jpeg, width=(int)848, height=(int)480,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)640,
height=(int)360, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1,
15/1 }; image/jpeg, width=(int)424, height=(int)240,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)352, height=(int)288, pixel-aspect-ratio=(fraction)1/1,
framerate=(fraction){ 30/1, 15/1 }; image/jpeg, width=(int)320,
height=(int)240, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1,
15/1 }; image/jpeg, width=(int)320, height=(int)180,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.211450330 5580 0x1e652d0 DEBUG v4l2src
gstv4l2src.c:512:gst_v4l2src_negotiate:<v4l2src0> caps of peer: video/x-raw,
format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)[
0/1, 2147483647/1 ], interlace-mode=(string){ progressive, interleaved, mixed }
0:00:00.211518706 5580 0x1e652d0 DEBUG v4l2src
gstv4l2src.c:518:gst_v4l2src_negotiate:<v4l2src0> intersect: EMPTY
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data
stream error.

Post by "GStreamer" (GNOME Bugzilla)
by the way, vappisink works well but filesink failed

fails how?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #38 from deshui <***@intel.com> ---
(In reply to Víctor Manuel Jáquez Leal from comment #37)

Post by "GStreamer" (GNOME Bugzilla)
(In reply to deshui from comment #35)

Post by "GStreamer" (GNOME Bugzilla)
after apply the patches from bug #793271 and bug #793274 on master branch
other format works well except format NV12 WITH command
gst-launch-1.0 v4l2src io-mode=5 num-buffers=100 !
video/x-raw,format=NV12,width=1280,height=720 ! vaapipostproc ! filesink
location=1.raw
so， gstreamer-vappi seems not support NV12 now?

That's because my (and I assume yours) v4l2src device doesn't handle NV12
0:00:00.207682288 5580 0x1e652d0 DEBUG v4l2src
gstv4l2src.c:504:gst_v4l2src_negotiate:<v4l2src0> caps of src: video/x-raw,
format=(string)YUY2, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)10/1; video/x-raw,
format=(string)YUY2, width=(int)960, height=(int)540,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)15/1; video/x-raw,
format=(string)YUY2, width=(int)848, height=(int)480,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1; video/x-raw,
format=(string)YUY2, width=(int)640, height=(int)480,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)424, height=(int)240,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)352, height=(int)288,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)320, height=(int)240,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
video/x-raw, format=(string)YUY2, width=(int)320, height=(int)180,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)960, height=(int)540,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)848, height=(int)480,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)640, height=(int)480,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)640, height=(int)360,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)424, height=(int)240,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)352, height=(int)288,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)320, height=(int)240,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 };
image/jpeg, width=(int)320, height=(int)180,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction){ 30/1, 15/1 }
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.211450330 5580 0x1e652d0 DEBUG v4l2src
gstv4l2src.c:512:gst_v4l2src_negotiate:<v4l2src0> caps of peer: video/x-raw,
format=(string)NV12, width=(int)1280, height=(int)720, framerate=(fraction)[
0/1, 2147483647/1 ], interlace-mode=(string){ progressive, interleaved,
mixed }
0:00:00.211518706 5580 0x1e652d0 DEBUG v4l2src
gstv4l2src.c:518:gst_v4l2src_negotiate:<v4l2src0> intersect: EMPTY
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal
data stream error.

Post by "GStreamer" (GNOME Bugzilla)
by the way, vappisink works well but filesink failed

fails how?

I use vivid, which should support NV12
I can find "video/x-raw, format=(string)NV12, width=(int)1280, height=(int)720"
in v4l2src gstv4l2src.c:504:gst_v4l2src_negotiate:

and the error log is:

(gst-launch-1.0:745): GStreamer-CRITICAL **: gst_memory_unmap: assertion 'mem
!= NULL' failed
0:00:00.445999797 745 0x564d5250b540 DEBUG v4l2src
gstv4l2src.c:913:gst_v4l2src_create:<v4l2src0> ts: 0:01:08.072851000 now
0:01:08.073164703 delay 0:00:00.000313703
0:00:00.446087104 745 0x564d5250b540 INFO v4l2src
gstv4l2src.c:949:gst_v4l2src_create:<v4l2src0> sync to 0:00:00.133333332 out ts
0:00:00.090602346
0:00:00.446215700 745 0x564d5250b540 ERROR vaapivideomemory
gstvaapivideomemory.c:278:map_vaapi_memory: failed to make image current

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

Víctor Manuel Jáquez Leal <***@igalia.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO

--- Comment #39 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
please upload a complete log vaapi*:5,v4l2*:5

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #40 from deshui <***@intel.com> ---
Created attachment 370288
--> https://bugzilla.gnome.org/attachment.cgi?id=370288&action=edit
v4l2src and vaapi logs

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #41 from deshui <***@intel.com> ---
(In reply to Víctor Manuel Jáquez Leal from comment #39)

Post by "GStreamer" (GNOME Bugzilla)
please upload a complete log vaapi*:5,v4l2*:5

please refer to the attachment

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #42 from Víctor Manuel Jáquez Leal <***@igalia.com> ---
That failure is when using filesink?

I'm lost now. If I understand correctly, there are two different issues

1\ NV12 doesn't work (it is not clear to me why and how in your case)
2\ when using filesink, you got the error in attachment 370288,
2.1\ but using vaapisink, the pipeline works

Am I right?

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

--- Comment #43 from deshui <***@intel.com> ---
NV12 + filesink doesn't work
NV12 + vaapisink works
other format + filesink works

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034
Bug 792034 depends on bug 793270, which changed state.

Bug 793270 Summary: videopool: define GST_BUFFER_POOL_OPTION_VIDEO_DMABUF
https://bugzilla.gnome.org/show_bug.cgi?id=793270

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |OBSOLETE

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034
Bug 792034 depends on bug 793271, which changed state.

Bug 793271 Summary: v4l2object: Check for all allocators and pools in query
https://bugzilla.gnome.org/show_bug.cgi?id=793271

What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |OBSOLETE

["GStreamer" (GNOME Bugzilla)]

https://bugzilla.gnome.org/show_bug.cgi?id=792034

GStreamer system administrator <***@gstreamer.net> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |RESOLVED
Resolution|--- |OBSOLETE

--- Comment #44 from GStreamer system administrator <***@gstreamer.net> ---
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been
closed from further activity.

You can subscribe and participate further through the new bug through this link
to our GitLab instance:
https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/81.