GitLab

Working on Gio.ListStore.insert_sorted's compare_func

See merge request !610

Closes: #326

April maintenance

See merge request !616

Until now, mostly we didn't care about the nursery because only plain JS
objects are created there; objects with finalizers, which comprise all of
our custom objects in GJS, are automatically promoted to the tenured heap
in SpiderMonkey's implementation of generational garbage collection.

However, with the taking-out-the-garbage patches we will create a lot more
Proxies (GObject wrappers) and plain objects (GObject expando objects)
which can be allocated in the nursery. This can result in them being
missing from heap dumps when we use the IgnoreNurseryObjects flag to
js::DumpHeap().

The alternative is the CollectNurseryBeforeDump flag, which promotes any
live objects in the nursery to the tenured heap. This way, we are
certain to capture all of our wrappers and expando objects in the heap
dump. The downside is that dumping the heap can now reduce performance.
But it is something that should only be done for debugging, anyway.

So here's this commit in advance of...

Proxies will play a large role in GJS, after the taking-out-the-garbage
patches are merged, so they should be formatted specially in heapgraph
output.

See #217.

We should set GJS_USE_UNINSTALLED_FILES (as the tests do) when hacking on
an uninstalled copy. Additionally, if this environment variable is set,
then we should not add the installed typelib location to the search path.

Otherwise this can lead to the confusing situation, if you are modifying
GjsPrivate, that an out-of-date installed typelib file can take precedence
over the one in the build directory, even when running the tests.

GObject: Ensure to call setter methods for construct-only properties

See merge request !614

Construct-only properties on JS-defined GObject subclasses takes some
shortcuts to define these JS properties as readonly ones. However, this
hardcodes the given G/JSValue without giving an opportunity to any
defined setter methods to be called. These setter methods could do
more things than setting the value (e.g. initialize related data,
ensure the given object is in an specific state, ...) and are thus
not trivial to bypass.

To fix this, ensure to set these properties, before locking them as
readonly.

Related: GNOME/gnome-shell#3926

gjs-test-tools: Throw error if we can't create threads

See merge request !618

Do not fail if we can't create a thread in idle, we may be out of
resources and this is not a failure for us.
So we can wait for allocated threads to finish before allocating other
instances (and go out of memory).

When running in some machines we may not be able to created many
threads, in those cases throw the error so that we can catch it at JS
side.

This was a more recent change than the branch that was just merged.

Unreviewed, pushing to fix build.

arg-cache: Never throw when building the argument cache

Closes #386

See merge request GNOME/gjs!590

Threads safer objects

Closes #297, #404, and #406

See merge request !613

Add a new CI job running the tests with ThreadSanitizer, that greatly
helps in checking what we could do wrong, especially now that we have
code that we explicitly force to run in multiple threads.

Also required when compiling with -Db_sanitize=thread.

Avoid repeating the same string checks while debug logging.

We can just save the enabled topics in a vector (if logging is enabled
at all) and quickly check whether a topic is enabled.

ThreadSanitizer is complaining that we may set and use static values in
a racy way, and that's true.

So, handle this in an init/cleanup functions that we set and unset at
context creation and cleanup.

At each toggle notification coming from another thread we check if
we are destroying, however this value is set from the main thread (when
the final gc starts), so we have to access to this atomically.

Use std::atomic, and make ThreadSanitizer happy.

When handling the toggle events from the main thread, we may have
received a notification for a toggle event from multiple threads
concurrently causing us to consider this an error.

For example:
  Object 0x555555643b30 Toggle up, from main thread 1, queued down 1, up 1
  Object 0x555555643b30 Toggle down, from main thread 1, queued down 1, up 1
  Object 0x555555643b30 Toggle up, from main thread 0, queued down 0, up 0
  Object 0x555555643b30 Toggle up, from main thread 1, queued down 0, up 0
  Object 0x555555643b30 Toggle down, from main thread 0, queued down 0, up 0
  Object 0x555555643b30 Toggle down, from main thread 1, queued down 1, up 0

Till the introduction of locked operations during toggle notifications
we may not be sure how to handle this, but now we can be quite confident
that we're working with the actual toggle queue state and so when an
already-toggled event happens we can just be sure that it will be
handled at next idle, and so we can stop considering it an error.

To avoid having a queue of events that have no result, change the
enqueue code to cancel a previously queued event that is the opposite of
the one we're adding. We need to be able to add multiple events of the
same type though (as could come from different threads), so adapt the
cancellation code to this.

As it's just something we can't control our level, but depends on other
threads fired by applications. What we can do is only react properly,
and now we do.

Fixes: #297

While the ToggleQueue is expected to be thread-safe, unfortunately the
way we use isn't always true.
In fact, while we always perform operations locking the shared data, we
may act on it depending on results that may have been changed by another
thread. For example, we check if an object is queued or we cancel a
queued object and we assume that this is true for the rest of the
execution, but this may have been already changed meanwhile, thus we
error later.

This happens repeatedly with applications doing many fast threaded
operations (an example can be triggered via GFileMonitor when the fs is
particularly active as bug #297 shows).

To avoid this, we need to ensure that all the access to the queue are
locked till we're not done with it, but at the same time we must not to
stop the owner thread to access multiple time to the queue while it owns
the lock (as it may happen on recursion of we want group various
operations).

To do this, control public access to the queue only via a structure that
uses a spin-lock to block non-holder threads and that uses a refcounted
system to handle the unlocking.

So, any time we get the queue, we have a locked control of it, once the
ToggleQueue::Locked structure is released, we unlock it only if no other
instances in the current thread are using it.

This serves us particularly well in the toggle notification callback
where we had cases in which we were adding duplicate objects to the queue
even if they were already there or vice versa we were not adding them
because we the situation changed since the call to is_queued().
Now instead, we can be sure that each thread will work on the current
queue state and behave accordingly.

Related-to: #297

While the ToggleQueue serves us to track GObject's using their pointers
is too fragile for us, because they can be also finalized from other
threads and we may not be able to track when this happens, leaving them
in the queue to be handled by the main thread, causing us to access to
invalid memory.

At the contrary, we totally control the life time of ObjectInstance's
and we're sure that they always are created and destroyed in the main
thread, so let's use them even inside the ToggleQueue.

In this way:
 - We can keep using the same strategy to enqueue them
 - We don't have to rely on monitoring GObject's finalization to clear
   the queue (that again may not happen in main thread).
 - When an object wrapper is destroyed, we are sure that we've removed
   the gobject toggle reference, and so that we'll never get another
   toggle event queued again till another wrapper is created.

Related to: #404

While GObject provides native tools to check if an object has been
finalized using GWeakRef's, they have proved to be too fragile for us
because they don't work well when used with toggle references
notifications (not thread safe as glib#2384 demonstrates) and because
may not be cleared on finalization (glib#2390).

So, instead of adding further qdata to monitor a queued object
finalization, let's just ensure that a wrapper cancels the toggle queue
when its wrapped object get finalized.

Since we don't touch the references of the object while adding it to the
queue, we can also ignore remembering the current handled object, as no
recursion can happen now.

It may still happen (mostly due to the way we lock) that a finalized
or unhandled object will be handled by the toggle handler, but this
can't be really fixed for all the racy cases even at enqueue phase
(like in the case a thread enqueues a toggle event just after we've
disassociated the wrapper for such object).

Fixes: #404

While it's true that a void function has no return value, this is not
true in the case it's a void pointer.

Not doing this, will cause a crash at ffi level because we're passing to
it a null return value pointer, while it's expecting to fill it with
the returned data.

Fixes: #406

In this way we can join later in the tests or have more control on them.

Since we use this for testing actions from multiple threads, it's better
to ensure that we have safe access to the variables.

So use an atomic pointer to keep track of the saved object that allows
us better control and make ThreadSanitizer happier.

Some simple Visual Studio fixes for master

See merge request !612

object: Block access to object only if it is finalized

See merge request !611

...on Visual Studio-style builds, since we need to explicitly mark the
prototypes in the header for export, otherwise the build will fail as
running g-ir-scanner will fail to find the export library for the DLL
that was built.

We are now using C++17 attributes in place of the GLib macros as needed, so we
can ignore warning C5030 that comes from attributes that are not recognized by
the compiler, in particular the GCC-specific ones, since they simply not apply
for Visual Studio builds.

For some time now we're warning whether a disposed object is used and
this is sane, however the data it holds may be still valid and in
general all the C code we access to is meant to be used after
disposition. While such code has not to be called when the object is
finalized.

Since now we properly track objects finalization we can go back to allow
most of the object actions (all but the signals connections, these
involve having toggle notifications that are not a tool we can use
at this point) with the actual object pointer.

In case the object is finalized instead we behave exactly as before.

Adapted tests and added ones for finalization case

examples: add examples of GtkBuilder templates

See merge request !607

Add some basic examples of using GtkBuilder templates in GJS, with a
few comments covering the few options available.

doc: document shebang for ESModules

See merge request !608

object: Discard disposed GObject's and do not create wrappers for them

Closes #395, #399, and #294

See merge request !593

When an object is disposed we discard its wrapper, if a new wrapper for
such objects is created it will stay connected to the object till the
wrapper is alive or the object is finalized.

However, we need to make sure that also the "disposed" wrapper object is
always unbound from its wrapped or we'll crash as in issue #395.

To test this, we can just repeatedly get from a C function a disposed
object so that we're going it to rewrap it over and over.