Discussion:
[Bug 683470] New: Unix socket fd leaks and memory leaks when use pipeline
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:07:26 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

Summary: Unix socket fd leaks and memory leaks when use
pipeline
Classification: Platform
Product: GStreamer
Version: 0.11.x
OS/Version: Linux
Status: UNCONFIRMED
Severity: critical
Priority: Normal
Component: gstreamer (core)
AssignedTo: gstreamer-***@lists.freedesktop.org
ReportedBy: ***@yahoo.com.cn
QAContact: gstreamer-***@lists.freedesktop.org
GNOME version: ---


We usually use the code as below.
//=====================================
GMainLoop *loop;
GstBus *bus;
GstElement *pipeline,*source,*typefind,*sink;

gst_init(NULL, NULL);
loop = g_main_loop_new(NULL, FALSE);

//create element
pipeline = gst_pipeline_new("my_pipeline");
typefind = gst_element_factory_make("typefind","typefind1");
sink = gst_element_factory_make("fakesink","sink1");
source = gst_element_factory_make("souphttpsrc","source1");

if (!pipeline || !source || !typefind || !sink)
{
return;
}

g_object_set (G_OBJECT(source), "location", _cUrl, NULL);

bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline)); //here will ref bus
gst_bus_add_watch(bus, bus_watch, loop); //here will ref
bus
gst_object_unref(bus); //but
only unref bus once

gst_bin_add_many(GST_BIN(pipeline), source, typefind, sink, NULL);
gst_element_link(source,typefind);
gst_element_link(typefind,sink);

g_main_loop_run(loop);

gst_element_set_state(pipeline,GST_STATE_NULL);
gst_object_unref(GST_OBJECT(pipeline));

//=====================================

Gstreamer example code is the same usage.
Note this:

bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline)); //here will ref bus
gst_bus_add_watch(bus, bus_watch, loop); //here will ref
bus
gst_object_unref(bus); //but
only unref bus once

(gst_pipeline_get_bus --> gst_element_get_bus) will ref bus.
(gst_bus_add_watch ---> gst_bus_add_watch_full_unlocked--->
gst_bus_create_watch) will ref bus.
Then will ref bus twice, and only unref once. So when we call
"gst_object_unref(GST_OBJECT(pipeline));" at end of the application, the
pipeline bus will not freed.

When the pipeline created, it will create bus, and bus will create gstpoll.
look at gstpoll codes,

GstPoll *
gst_poll_new (gboolean controllable)
{
GstPoll *nset;

GST_DEBUG ("controllable : %d", controllable);

nset = g_slice_new0 (GstPoll);
g_mutex_init (&nset->lock);
#ifndef G_OS_WIN32
nset->mode = GST_POLL_MODE_AUTO;
nset->fds = g_array_new (FALSE, FALSE, sizeof (struct pollfd));
nset->active_fds = g_array_new (FALSE, FALSE, sizeof (struct pollfd));
nset->control_read_fd.fd = -1;
nset->control_write_fd.fd = -1;
{
gint control_sock[2];

if (socketpair (PF_UNIX, SOCK_STREAM, 0, control_sock) < 0)
goto no_socket_pair;

//=================================

It will use socketpair to create two sockets, but not closed when pipeline
destroyed because the bus ref count is 2.

You can use lsof to check fd number
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:09:38 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #1 from zhangyanping <***@yahoo.com.cn> 2012-09-06 09:09:33 UTC ---
That's to say the bus of the pipeline is not destroyed, because the ref count
of the bus is not 1
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:16:10 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

Tim-Philipp Müller <t.i.m> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|UNCONFIRMED |NEEDINFO
CC| |***@zen.co.uk

--- Comment #2 from Tim-Philipp Müller <***@zen.co.uk> 2012-09-06 09:16:08 UTC ---
The bus watch probably keeps a ref to the bus as well. Try:

id = gst_bus_add_watch (..);
...
g_source_remove (id);
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:35:28 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #3 from zhangyanping <***@yahoo.com.cn> 2012-09-06 09:35:24 UTC ---
If I use
gst_element_set_state(pipeline,GST_STATE_NULL);
gst_object_unref(GST_OBJECT(pipeline));
g_source_remove(id);

It will be ok. Thank you Tim. I think you should add this to the example code.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:39:48 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

Wim Taymans <wim.taymans> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |***@gmail.com

--- Comment #4 from Wim Taymans <***@gmail.com> 2012-09-06 09:39:43 UTC ---
The problem really is because the mainloop is not unreffed.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:44:00 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #5 from zhangyanping <***@yahoo.com.cn> 2012-09-06 09:43:55 UTC ---
Then how we can avoid it. Is it a gstreamer inner bug or a programmer bug ?
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:45:24 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

Wim Taymans <wim.taymans> changed:

What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |RESOLVED
Resolution| |FIXED

--- Comment #6 from Wim Taymans <***@gmail.com> 2012-09-06 09:45:18 UTC ---
programmer bug. Do g_main_loop_unref (loop) after unreffing the pipeline.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:45:31 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

Wim Taymans <wim.taymans> changed:

What |Removed |Added
----------------------------------------------------------------------------
Resolution|FIXED |NOTABUG
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:47:32 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x
Post by "GStreamer" (bugzilla.gnome.org)
The problem really is because the mainloop is not unreffed.
But the watch is hooked up on the default main context (which I believe is
never destroyed once created), so you can do multiple
g_main_loop_run()/quit()/run(), so I think the fd leak is really due to the
source not having been removed.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:51:33 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #8 from zhangyanping <***@yahoo.com.cn> 2012-09-06 09:51:32 UTC ---
I use
gst_element_set_state(pipeline,GST_STATE_NULL);
gst_object_unref(GST_OBJECT(pipeline));
g_main_loop_unref(loop);

still have fd leak. Every time leak two fds.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 09:59:17 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #9 from zhangyanping <***@yahoo.com.cn> 2012-09-06 09:59:15 UTC ---
Acossing to my debuging, I think the source is removed.


#0 0x2b0c4e20 in socketpair () from
/opt/host_libc_arm1176jzf_s/arm-avl-linux-gnueabi/arm-avl-linux-gnueabi/sysroot/lib/libc.so.6
#1 0x2ac74df0 in gst_poll_new (controllable=1) at gstpoll.c:604
#2 0x2ac74f48 in gst_poll_new_timer () at gstpoll.c:667
#3 0x2ac1d534 in gst_bus_constructed (object=0x94bf0) at gstbus.c:150
#4 0x2ad1ee2c in g_object_newv () from
/home/zhangyp/rootfs_for_hunter/usr/lib/libgobject-2.0.so.0
#5 0x2ac1d96c in gst_bus_new () at gstbus.c:270
#6 0x2ac67f28 in gst_pipeline_init (pipeline=0x111030) at gstpipeline.c:220
#7 0x2ad37158 in g_type_create_instance () from
/home/zhangyp/rootfs_for_hunter/usr/lib/libgobject-2.0.so.0
#8 0x2ad1f398 in g_object_constructor () from
/home/zhangyp/rootfs_for_hunter/usr/lib/libgobject-2.0.so.0
#9 0x2ad1e98c in g_object_newv () from
/home/zhangyp/rootfs_for_hunter/usr/lib/libgobject-2.0.so.0
#10 0x2ad1f22c in g_object_new_valist () from
/home/zhangyp/rootfs_for_hunter/usr/lib/libgobject-2.0.so.0
#11 0x2ad1f35c in g_object_new () from
/home/zhangyp/rootfs_for_hunter/usr/lib/libgobject-2.0.so.0
#12 0x2ac38fb0 in gst_element_factory_create (factory=0x32090, name=0x2aae6534
"my_pipeline") at gstelementfactory.c:382
#13 0x2ac393ec in gst_element_factory_make (factoryname=0x2ace57a4 "pipeline",
name=0x2aae6534 "my_pipeline") at gstelementfactory.c:453
#14 0x2ac68480 in gst_pipeline_new (name=0x2aae6534 "my_pipeline") at
gstpipeline.c:314


The leaked fd is created by socketpair. It belongs to the pipeline element, not
belongs to src
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 10:00:20 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #10 from Wim Taymans <***@gmail.com> 2012-09-06 10:00:19 UTC ---
Ah, indeed. Then you need to explicitly remove the source or make a custom
context. It's not so nice..

Does the source really need a ref to the bus? If removing the last ref on the
bus would also remove the source, it would perhaps be nicer..
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 10:03:08 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #11 from Wim Taymans <***@gmail.com> 2012-09-06 10:03:02 UTC ---
(In reply to comment #9)
Post by "GStreamer" (bugzilla.gnome.org)
Acossing to my debuging, I think the source is removed.
The leaked fd is created by socketpair. It belongs to the pipeline element, not
belongs to src
The socketpair is from the bus object, which is not freed because the source
has a ref to it and is not freed.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 10:04:56 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #12 from zhangyanping <***@yahoo.com.cn> 2012-09-06 10:04:54 UTC ---
Hi Wim,
Then How to write my code. Do you mean

explicitly call gst_object_unref(GST_OBJECT(src));

or call gst_object_unref(bus) twice.

Calling gst_object_unref(bus) twice will cause segment failed.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 10:09:02 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #13 from zhangyanping <***@yahoo.com.cn> 2012-09-06 10:08:59 UTC ---
(In reply to comment #11)
Post by "GStreamer" (bugzilla.gnome.org)
(In reply to comment #9)
Post by "GStreamer" (bugzilla.gnome.org)
Acossing to my debuging, I think the source is removed.
The leaked fd is created by socketpair. It belongs to the pipeline element, not
belongs to src
The socketpair is from the bus object, which is not freed because the source
has a ref to it and is not freed.
The only way to close the socket is when bus is destroyed. When the bus
destroyed, it call gst_poll_free. In gst_poll_free, it will close the socket.

static void
gst_bus_dispose (GObject * object)
{
GstBus *bus = GST_BUS (object);

if (bus->priv->queue) {
GstMessage *message;

g_mutex_lock (&bus->priv->queue_lock);
do {
message = gst_atomic_queue_pop (bus->priv->queue);
if (message)
gst_message_unref (message);
} while (message != NULL);
gst_atomic_queue_unref (bus->priv->queue);
bus->priv->queue = NULL;
g_mutex_unlock (&bus->priv->queue_lock);
g_mutex_clear (&bus->priv->queue_lock);

if (bus->priv->poll)
gst_poll_free (bus->priv->poll);
bus->priv->poll = NULL;
}

G_OBJECT_CLASS (parent_class)->dispose (object);
}
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 10:11:42 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #14 from zhangyanping <***@yahoo.com.cn> 2012-09-06 10:11:36 UTC ---
I think the cause is that the bus is not freed. How to free the bus?
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 10:13:03 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #15 from Wim Taymans <***@gmail.com> 2012-09-06 10:13:00 UTC ---
(In reply to comment #14)
Post by "GStreamer" (bugzilla.gnome.org)
I think the cause is that the bus is not freed. How to free the bus?
What Tim said:

id = gst_bus_add_watch (..);
...
g_source_remove (id);
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 10:15:59 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #16 from zhangyanping <***@yahoo.com.cn> 2012-09-06 10:15:57 UTC ---
(In reply to comment #15)
Post by "GStreamer" (bugzilla.gnome.org)
(In reply to comment #14)
Post by "GStreamer" (bugzilla.gnome.org)
I think the cause is that the bus is not freed. How to free the bus?
id = gst_bus_add_watch (..);
...
g_source_remove (id);
Yes , this way is ok as I had replied in comment 3.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 11:55:49 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #17 from zhangyanping <***@yahoo.com.cn> 2012-09-06 11:55:43 UTC ---
Hello All,
Thank you for your replay with patience. I think gst example codes
should also add this code. I will mark it NOTABUG.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
"GStreamer" (bugzilla.gnome.org)
2012-09-06 12:00:23 UTC
Permalink
https://bugzilla.gnome.org/show_bug.cgi?id=683470
GStreamer | gstreamer (core) | 0.11.x

--- Comment #18 from Tim-Philipp Müller <***@zen.co.uk> 2012-09-06 12:00:19 UTC ---
We'll fix the example, thanks.
--
Configure bugmail: https://bugzilla.gnome.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
You are the assignee for the bug.
Loading...