[pygtk] Garbage collection prematurely clears cyclical objects referenced from GTK callbacks

[Hrvoje Nikšić]

On Wed, 2008-08-13 at 12:41 +0100, Mark Seaborn wrote:
> On Wed, 2008-08-13 at 11:29 +0200, Hrvoje Nikšić wrote:
> 
> > I don't quite understand how this bug occurs, and why it fails to appear
> > when the cyclical reference is removed.  After all, if the object is not
> > correctly increffed from GTK, it should die as soon as Buggy() finishes,
> > but it doesn't happen.
> 
> Something I don't understand is why the bug doesn't cause a crash.  I
> would expect breaking the cycle to cause the refcount of the closure to
> drop to zero, causing its memory to be freed and potentially reused
> before the closure is called by Gtk.

The way I see it, pygobject's traverse misreports the refcount by
unconditionally descending into the GObject.  That makes the GC miscount
the references to containers by +1, causing the "internal refcount" to
appear equal to ob_refcnt.  Normally this would imply that all
references come from within the cycle, so the GC proceeds to clear the
containers.  Since some of the containers are in fact reachable from the
outside, this clearing doesn't cause those particular containers (such
as the closure) to be deallocated.

tp_clear normally leads to deallocation due to it being called only on
objects unreachable from the outside.  Once that invariant is broken,
tp_clear just cripples the reachable objects.

> * Something is incrementing the refcount of the closure after Python
> attempts to break the cycle.
> * Luck: the closure's memory is not reused and Python does not zero out
> any other fields when freeing the block or check that the refcount is
> sane when using the closure.  This should be testable with valgrind.

I don't think it's luck because different kinds of objects reliably keep
working, and Python keeps happily chugging along.  I'm pretty sure it
would crash much sooner if it were really (mis)using deallocated memory.

> However, toggle refs are not *always* used.  They are only used when the
> PyGObject's __dict__ is accessed (either directly or by assigning to
> attributes).

This leads to the truly weird-looking part of the workaround I've now
installed in production code.  After creating the gtk.Dialog, I access
its __dict__, and presto: problem solved.

Hopefully PyGTK 2.14 will include Gustavo's fix.