As event handling goes:
1) Events get generated and queued by the seat (from another thread in
native, in the same thread in X11)
2) The MetaBackend gets those events and forwards them to Clutter
via clutter_do_event()
3) The events get queued in the ClutterStage
4) At the time of processing a frame, the input events are processed,
5) Motion events are throttled, only the last is effectively handled
6) Events are filtered, wayland and WM handling happens here
7) Events maybe reach to clutter
This commit moves 6 to happen between 2 and 3. The end result is that:
- Throttling only applies to Clutter event handling, The wayland event
forwarding bits will handle the event stream as soon as it comes, as
timely as possible.
- WM event handling is also unthrottled, but that's more of a side
effect.
- This all still happens on the main thread, so there's the possibility
that other busy areas (e.g. relayout) temporarily block this event
forwarding.
- Sending events unthrottled inherently means more CPU, probably
dependent on input devices' frequency. The impact is not measured.
This should bring the best of both worlds with e.g. 1000Hz mice, wayland
clients get unthrottled events, while GNOME Shell UI still behaves like
it used to do.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1915>
If we wait till finalize, dispose will destroy the actor hierarchy
and cause untimely repicks. Ensure to free the pointer/touch info
first, so the hooked signal callbacks are gone when destroying the
actors.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1915>
In the case a11y is required, the screen reader is very much
interested in getting an uninterrupted flow of key events. It attempts
so by setting a ::captured-event callback on the ClutterStage, but
that falls short with our MetaDisplay event handler, as clutter events
can be stopped before a11y gets a chance to see them.
This kind of selective amnesia wrt key events is not new, in X11 those
go unheard of by the WM as long as a client is focused and no grabs hold,
so it is clients' responsibility to talk with AT bridge.
This commit doesn't yet change that for X11, but we can do this right
away from the compositor on Wayland, and without any chance to be
tampered by clients.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1328>
If one would end up with an actor attached to mapped actor, where the
attached actor doesn't itself have an up to date stage view list while
listening on the stage for updating, when clearing the stage views of
the list, anything that would query the stage views list at this time
would end up accessing freed memory.
This could happen if
1) An actor was added to a newly created container actor attached to
the stage
2) The actor got a timeline attached to it
3) The actor was moved to a container that already was mapped
4) A hotplug happened
After (1) both the container and actor would not have any stage views.
After (2) the timeline would listen on the stage for stage views
updates. After (3) the actor would still listen on the stage for stage
views updates. When (4) happened, the actor would be signalled when the
stage got its stage view cleared, at which point it would traverse up
its actor's tree finding an appropriate stage view to base its animation
on. The problem here would be that it'd query the already mapped
container and its yet-to-be-cleared stage view list, resulting in
use-after free, resulting in for example the following backtrace:
0) g_type_check_instance_cast ()
1) CLUTTER_STAGE_VIEW ()
2) clutter_actor_pick_frame_clock ()
3) clutter_actor_pick_frame_clock ()
4) update_frame_clock ()
5) on_frame_clock_actor_stage_views_changed ()
6) g_closure_invoke ()
7) signal_emit_unlocked_R ()
8) g_signal_emit_valist ()
9) g_signal_emit ()
10) clear_stage_views_cb ()
11) _clutter_actor_traverse_depth ()
12) _clutter_actor_traverse ()
13) clutter_actor_clear_stage_views_recursive ()
14) clutter_stage_clear_stage_views ()
...
Avoid this issue by making sure that we don't emit 'stage-views-changed'
signals while the actor tree is in an invalid state. While we now end up
traversing tree twice, it doesn't change the Big-O notation. It has not
been measured whether this has any noticible performance impact.
Closes: https://gitlab.gnome.org/GNOME/mutter/-/issues/1950
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2025>
This does two things to frown upon:
- Modifies ClutterEvent structs, while the effort is to have those
completely opaque, and readonly after creation from the input
thread side.
- Stores state in the ClutterInputDevice struct, event though those
are also considered static after creation, managed by the input
thread, etc.
Stop doing that. This makes all events just forwarded as-is in
the ClutterStage/clutter-main.c code.
Handling of click count sounds like material for a ClutterGestureAction
(or perhaps ClutterClickAction), all of both callers now do it in place
at the moment, while gestures lack a better state tracking and management.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2024>
This will not try the captured-event shenanigans to emulate grab
behavior, instead relying on event delivery being influenced by
other grab mechanisms.
While at it, improve handling of additional touchpoints by
cancelling the click action right away, as the differences in
event handling make this unwanted behavior surface.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2024>
By default, the pan action performs matrix translations on the
child widget. Nobody wants that (or, nobody wants *just* that).
It's cleaner not to mix mechanism and effect in ClutterGestureAction
subclasses, so drop this base implementation, and change the signal
accumulator so it's more similar to event signals (not that it's
used any longer, anyway).
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2024>
This warning is actually dead code, since should_be_mapped and
must_be_realized are always set to the same value, so it does not
make sense to check for "a && !b".
Turn this into an assert so we avoid the dead branch, but do not
remove the variable duplication so the more aptly named variable
is used where it belongs, for clarity.
CID: #1506254
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2061>
It was a feature relevant for when Clutter was an application toolkit
that wanted the application window to communicate a minimum size to the
windowing system.
Now, clutter is part of the windowing system component, so this feature
doesn't make any sense, so remove it.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2002>
This feature was configured depending on whether the Cogl backend
reported COGL_WINSYS_FEATURE_MULTIPLE_ONSCREEN or not. All cogl backends
do report this, so any code handled the 'static' case were never used.
While we only ever use one stage, it's arguable more correct to
consilidate on the single stage case, but multiple stages is something
that might be desirable for e.g. a remote lock screen, so lets keep this
logic intact.
This has the side effect of completely removing backend features, as
this was the only left-over feature detection that they handled.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2002>
This changes the setup phase of clutter to not be result of calling an
init function that sets up a few global singletons, via global singleton
setup vfuncs.
The way it worked was that mutter first did some initial setup
(connecting to the X11 server), then set a "custom backend" setup vfunc
global, before calling clutter_init().
During the clutter_init() call, the context and backend was setup by
calling the global singleton getters, which implicitly created the
backend and context on-demand.
This has now changed to mutter explicitly creating a `ClutterContext`
(which is actually a `ClutterMainContext`, but with the name shortened to
be consistent with `CoglContext` and `MetaContext`), calling it with a
backend constructor vfunc and user data pointer.
This function now explicitly creates the backend, without having to go
via the previously set global vfunc.
This changes the behavior of some "get_default()" like functions, which
will now fail if called after mutter has shut down, as when it does so,
it now destroys the backends and contexts, not only its own, but the
clutter ones too.
The "ownership" of the clutter backend is also moved to
`ClutterContext`, and MetaBackend is changed to fetch it via the clutter
context.
This also removed the unused option parsing that existed in clutter.
In some places, NULL checks for fetching the clutter context, or
backend, and fetching the cogl context from the clutter backend, had to
be added.
The reason for this is that some code that handles EGL contexts attempts
to restore the cogl EGL context tracking so that the right EGL context
is used by cogl the next time. This makes no sense to do before Cogl and
Clutter are even initialized, which was the case. It wasn't noticed
because the relevant singletons were initialized on demand via their
"getters".
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2002>
This one is a trivial wrapper around clutter_actor_get_children(), so just
use that in the two places where clutter_container_get_children() is used,
and remove clutter_container_get_children().
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2057>
Right now we damage the stage even if an actor is not mapped, for
example in the overview.
Stop doing so, reducing over-paint significantly in some situations.
Clones will still do stage damage on their own.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2035>
ClutterText implements its own get_paint_volume() with its own cache,
but was not invalidating the actor paint volume when when it has
changed. This sometimes could result in labels, especially quickly
changing ones, using the old paint volume which either would cut off the
label or leave parts of the old label on screen.
Fixes: https://gitlab.gnome.org/GNOME/mutter/-/issues/1943
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2006>
This mode is passed along by the ClutterInputMethod, the
ClutterInputFocus will preserve it and ensure it is honored
whenever the IM is being reset.
This mode is immediate. The ClutterInputFocus commits the
text directly without queueing a CLUTTER_IM_COMMIT event.
This is important so events are serialized in the right order
in the wayland implementations (i.e. commit before wl_pointer.press).
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1940>
In line with GTK, the input method context should be reset when clicks
are handled by the ClutterInputFocus user. The reset action can then
either clear or commit the preedit text, as configured by the IM module.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1940>
Make sure that when we've recreated views that we'll actually paint a
new frame for it. This was very rarely a problem, as views tend to
result in getting damage etc being queued as side effects of various
things, like layout, but e.g. when running certain tests, this might not
happen. There is no situation where we want to create a new view that
should remain unpainted, so just make sure we initialize it to become up
to date.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1947>
This code sneaked unconditionally, even though we can disable
tracing code with -Dprofiler=false. Add some COGL_HAS_TRACING
checks so that this code is also optionally built.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1951>
Will be used to trace a lot more, and with more details, and thus may
have a larger impact on what is actually measured. This potential impact
is the reason for enabling only when needed.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1700>
The failure to allocate was not properly handled, causing crashes later
on due to the offscreen being NULL.
#0 cogl_gl_framebuffer_bind (target=36160, gl_framebuffer=0x0)
#1 _cogl_driver_gl_flush_framebuffer_state (...)
#2 cogl_context_flush_framebuffer_state (read_buffer=0x55f48f386780, draw_buffer=0x55f48f386780, ...)
#3 cogl_framebuffer_clear4f (framebuffer=0x55f48f386780, ...)
#4 clutter_layer_node_pre_draw (...)
#5 clutter_paint_node_paint (...)
...
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1942>
We only listen to it for 2 settings (drag threshold, double click
time), and we already have the stock ClutterSettings object tracking
the source of these. This code is redundant.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1862>
Not sure how to update the damage or redraw clip or something; at least
this works properly when under a constantly-redrawing window, which is
ok for debugging purposes.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1762>
Max render time shows how early the frame clock needs to be dispatched
to make it to the predicted next presentation time. Before this commit
it was set to refresh interval minus 2 ms. This means Mutter would
always start compositing 14.7 ms before a display refresh on a 60 Hz
screen or 4.9 ms before a display refresh on a 144 Hz screen. However,
Mutter frequently does not need as much time to finish compositing and
submit buffer to KMS:
max render time
/------------\
---|---------------|---------------|---> presentations
D----S D--S
D - frame clock dispatch
S - buffer submission
This commit aims to automatically compute a shorter max render time to
make Mutter start compositing as late as possible (but still making it
in time for the presentation):
max render time
/-----\
---|---------------|---------------|---> presentations
D----S D--S
Why is this better? First of all, Mutter gets application contents to
draw at the time when compositing starts. If new application buffer
arrives after the compositing has started, but before the next
presentation, it won't make it on screen:
---|---------------|---------------|---> presentations
D----S D--S
A-------------X----------->
^ doesn't make it for this presentation
A - application buffer commit
X - application buffer sampled by Mutter
Here the application committed just a few ms too late and didn't make on
screen until the next presentation. If compositing starts later in the
frame cycle, applications can commit buffers closer to the presentation.
These buffers will be more up-to-date thereby reducing input latency.
---|---------------|---------------|---> presentations
D----S D--S
A----X---->
^ made it!
Moreover, applications are recommended to render their frames on frame
callbacks, which Mutter sends right after compositing is done. Since
this commit delays the compositing, it also reduces the latency for
applications drawing on frame callbacks. Compare:
---|---------------|---------------|---> presentations
D----S D--S
F--A-------X----------->
\____________________/
latency
---|---------------|---------------|---> presentations
D----S D--S
F--A-------X---->
\_____________/
less latency
F - frame callback received, application starts rendering
So how do we actually estimate max render time? We want it to be as low
as possible, but still large enough so as not to miss any frames by
accident:
max render time
/-----\
---|---------------|---------------|---> presentations
D------S------------->
oops, took a little too long
For a successful presentation, the frame needs to be submitted to KMS
and the GPU work must be completed before the vblank. This deadline can
be computed by subtracting the vblank duration (calculated from display
mode) from the predicted next presentation time.
We don't know how long compositing will take, and we also don't know how
long the GPU work will take, since clients can submit buffers with
unfinished GPU work. So we measure and estimate these values.
The frame clock dispatch can be split into two phases:
1. From start of the dispatch to all GPU commands being submitted (but
not finished)—until the call to eglSwapBuffers().
2. From eglSwapBuffers() to submitting the buffer to KMS and to GPU
work completing. These happen in parallel, and we want the latest of
the two to be done before the vblank.
We measure these three durations and store them for the last 16 frames.
The estimate for each duration is a maximum of these last 16 durations.
Usually even taking just the last frame's durations as the estimates
works well enough, but I found that screen-capturing with OBS Studio
increases duration variability enough to cause frequent missed frames
when using that method. Taking a maximum of the last 16 frames smoothes
out this variability.
The durations are naturally quite variable and the estimates aren't
perfect. To take this into account, an additional constant 2 ms is added
to the max render time.
How does it perform in practice? On my desktop with 144 Hz monitors I
get a max render time of 4–5 ms instead of the default 4.9 ms (I had
1 ms manually configured in sway) and on my laptop with a 60 Hz screen I
get a max render time of 4.8–5.5 ms instead of the default 14.7 ms (I
had 5–6 ms manually configured in sway). Weston [1] went with a 7 ms
default.
The main downside is that if there's a sudden heavy batch of work in the
compositing, which would've made it in default 14.7 ms, but doesn't make
it in reduced 6 ms, there is a delayed frame which would otherwise not
be there. Arguably, this happens rarely enough to be a good trade-off
for reduced latency. One possible solution is a "next frame is expected
to be heavy" function which manually increases max render time for the
next frame. This would avoid this single dropped frame at the start of
complex animations.
[1]: https://www.collabora.com/about-us/blog/2015/02/12/weston-repaint-scheduling/
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1762>
This fixes a warning/error:
In function 'parse_settings',
inlined from 'read_settings' at ../clutter/clutter/x11/xsettings/xsettings-client.c:398:25:
../clutter/clutter/x11/xsettings/xsettings-client.c:202:13: error: 'buffer.byte_order' may be used uninitialized [-Werror=maybe-uninitialized]
202 | if (buffer.byte_order != MSBFirst &&
| ~~~~~~^~~~~~~~~~~
This is needed to bump the CI image from F33 to F34, which includes a
upgraded compiler.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1865>
A simply wrapper around `CoglTexture`, making it easy to reuse
content without roundtrip from GPU to CPU memory and back.
It optionally takes a clip rectangle which is implemented by
creating a `CoglSubTexture`. A limitation here is that floating
point clips are not supported.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1888>
When using `CLUTTER_PAINT=damage-region` highlighting was conspicuously
absent during fullscreen animations like entering or leaving the
overview. That was because `queued_redraw_clip` was empty, because it
had been initialized from `redraw_clip == NULL` (full stage redraw).
Now we paint the damage region as the full view (which it is) instead
of nothing at all.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1890>
This commit adds scaling support to clutter_stage_capture_into, which
is currently used when screencasting monitors. This is supposed to
fix graphical issues that arise when using fractional scaling.
Fixes#1131
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1855>
All pointer a11y is a fabrication of Clutter backend-independent
code, with the help of a ClutterVirtualInputDevice and with some
UI on top.
On the other hand, MetaInputSettings is a backend implementation
detail, this has 2 gotchas:
- In the native backend, the MetaInputSettings (and pointer a11y
with it) are initialized early, before the ClutterSeat core
pointer is set up.
- Doing this from the MetaInputSettings also means another dubious
access from the input thread into main thread territory.
Move the pointer a11y into ClutterSettings, making this effectively
backend-independent business, invariably done from the main thread
and ensured to happen after seat initialization.
Fixes: https://gitlab.gnome.org/GNOME/mutter/-/issues/1765
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1849>
Since commit d2f8a30625 we use Graphene to union paint volumes, it
turns out a quite severe issue snuck in during review of that MR though:
Unioned paint volumes (so paint volumes of any actors with children) now
have negative heights. Once projected to 2d coordinates they luckily are
correct again, which is why everything is still working.
The problem is that obvious once looking closer: For the y coordinates
of the unioned paint volume we confused the maximum and the minimum
points and simply used the wrong coordinates to create the unioned paint
volume.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1827>
The graphene functions used by clutter for picking assume that boxes are
inclusive in both there start and end coordinates, so picking at y
coordinate 32 for an actor with the height 32 placed at y coordinate 0
would still be considered a hit. This however is wrong as 32 is the
first position that is not in the actor anymore.
Usually this would not be much of a problem, because motion events are
rarely ever at exactly these borders and even if they are there will be
another motion event soon after. But since actors in gnome-shell usually
are aligned with the pixel grid and on X11 enter/leave events are
generated by the X server at integer coordinates, this case is much
more likely for those.
This can cause issues with Firefox which when using client side
decorations, still requests MWM_DECOR_BORDER via _MOTIF_WM_HINTS to have
mutter draw a border + shadow. This means that the Firefox window even
when using CSD is still reparented. For such windows we receive among
others XI_RawMotion and XI_Enter events, but no XI_Motion events. And
the raw motion events are discarded after an enter event, because that
sets has_pointer_focus to TRUE in MetaSeatX11. So when moving the cursor
from the panel to a maximized Firefox window the last event clutter
receives is the enter event at exactly integer coordinates. Since the
panel is 32px tall and the generated enter event is at y position 32,
the picking code will pick a panel actor and the focus will remain on it
as long as the cursor does not leave the Firefox window.
Fix this by excluding the bottom and right border of a box when picking.
Fixes https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/4041
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1842>
Turns out ClutterClones need a bit of extra handling as always, there's
currently nothing that invalidates a clones paint volume when the source
actors paint volume changes.
Since ClutterClones get_paint_volume() implementation simply takes the
source actors paint volume and returns that, we should make sure they
are kept in sync and invalidate the clones paint volume as soon as the
source actor gets its PV invalidated.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1829>
Updating the last_paint_volume while painting has proven itself to be
quite prone to issues: First we had to make sure actors painted by
offscreen effects get their last_paint_volumes updated correctly (see
0320649a1c), and now a new issue turned up
where we don't update the paint volumes while a fullscreen unredirect is
happening.
To stop those issues from happening and to lay the foundation for using
the last_paint_volume for other things, update the last_paint_volume in
a separate step before painting instead of doing it in
clutter_actor_paint().
To save some resources, avoid introducing another traversal of the
scenegraph and add that step into the existing step of updating the
stage_views lists of actors. To properly update the paint volumes, we
need to do that after finishing the queued redraws, which is why we move
clutter_stage_maybe_finish_queue_redraws() to happen before the new
clutter_stage_finish_layout().
Fixes https://gitlab.gnome.org/GNOME/mutter/-/issues/1699
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1773>
