'screen-cast/monitor-src: Use clutter_stage_paint_to_buffer'
(6c818cd8d5) made the non-dma-buf path use
clutter_stage_paint_to_buffer() to avoid running into direct scanout
issues. At a glance, the dma-buf paths didn't have the same issue since
it explicitly handled dma-bufs by blitting them.
What it also did was move the recording to an idle callback, to avoid
paint reentry issues. A side effect of this, however, is that it also
broke the dma-buf paths, as they rely on the back buffer existing, and
the stage view direct scanout already being setup, which it isn't in an
idle callback.
Fix this by using the dma-buf variant of
clutter_stage_paint_to_buffer(): clutter_stage_paint_to_framebuffer().
This has some negative performance impact, but we can't use
cogl_blit_framebuffer() when using an idle callback for recording.
Potential performance improvements to make things work more as they did
before is to enhance 'cogl_blit_framebuffer()' a bit, making it a vfunc
that could be implemented by MetaOnscreenNative. A flag to say whether
to look at the back or front buffer would let MetaOnscreenNative know
whether to use the already committed-to-KMS buffer, or the current back
buffer.
Fixes: 6c818cd8d5
Closes: https://gitlab.gnome.org/GNOME/mutter/-/issues/2282
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2462>
With the unthrottled input emission, we ended up often getting the
cursor updates long before any damage had been posted, meaning that if
you moved around the mouse pointer where the mouse had a high enough
refresh rate, we'd effectively stall the screen cast stream by only
sending cursor updates and nothing else.
Fix this by scheduling an update when we get a cursor update, then
sending a cursor-only frame after any damage and relayout has been
processed, but only if there is no queued damage that will cause an
actual repaint.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2393>
When the before-paint function is executed, it's only purpose
is to check if there's any scanout queue, and immediately
record it if any.
However, since [1], we regressed in this specific case with the
introduction of an idle callback in the before-paint function.
The regression only happens when the PipeWire stream is using
DMA-BUF buffers, and it would operate as follows:
1. In before-paint, when there's a scanout available, we queue
an idle callback to capture the monitor. The idle callback
(almost always) executes after the scanout is pulled from
the stage view
2. meta_screen_cast_stream_src_maybe_record_frame() is called
by the idle callback. In the DMA-BUF case, it then runs
meta_screen_cast_monitor_stream_src_record_to_framebuffer()
3. In meta_screen_cast_monitor_stream_src_record_to_framebuffer(),
because the stage view doesn't have a scanout anymore, it
ends up calling cogl_blit_framebuffer() with the stage view
framebuffer. This is the regression bug.
This regression presents itself in the form of the screencast
stream showing the desktop when there's an unredirected fullscreen
application window running.
Revert before-paint - and only that - back to immediately capturing
any available scanout. Only record these frames when the target
buffer is a DMA-BUF handle. Nothing is captured on before-paint if
the stream is not using DMA-BUF, since the regular paint routine
will handle these frames regularly post-paint.
[1] https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1914
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2186>
It works correctly with scanouts, in contrast to
clutter_stage_capture_into. Inspired by
meta_screen_cast_area_stream_src_record_to_buffer.
maybe_paint_cursor_sprite is now unused and thus removed.
v2:
* clutter_stage_paint_to_buffer requires switching to recording from an
idle callback as well. (Jonas Ådahl)
v3:
* Set human readable name for idle source. (Ivan Molodetskikh)
Closes: https://gitlab.gnome.org/GNOME/mutter/-/issues/1940
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1914>
This commit adds scaling support to clutter_stage_capture_into, which
is currently used when screencasting monitors. This is supposed to
fix graphical issues that arise when using fractional scaling.
Fixes#1131
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1855>
The area source, window source, and monitor source, currently set up the
stream size up front, given the area, maximum allowed window size or
monitor resolution, but for to be introduced sources, the size will be
negotiated using PipeWire, instead of specified via the D-Bus API. This
commit changes the internal source API to allow for this. There are
currently no users of this new behaviour.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1698>
PipeWire recently introduced busy buffers, which actually fixes the last remaining
issue that blocked us from downgrading these cogl_framebuffer_finish() calls into
cogl_framebuffer_flush() ones.
Switch to cogl_framebuffer_flush() in all three stream sources.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1701>
Instead of getters, pass the width, height and stride around when
relevant. This also removes the redudant "stream_size" and
"stream_height" variables from the src struct, as they are already part
of the video format.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1697>
If the monitor configuration changed, even though the streamed monitor
didn't change, we'd still fail to continue streaming, as we failed to
update the stage watchers, meaning we wouldn't be notified about when
the stage views were painted.
Fix this by reattaching the stage watches, i.e. update the painted
signalling listeners to listen to the right views, when monitor changes
happens.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1691>
We are aiming for a split of HW and SW cursor rendering management.
Given the HW plane is a limited resource and the amount of cursor
renderers may be >1 (due to tablets, even though we currently use an
always-software cursor renderer there), it would ideally be able to
switch between renderers.
Being MetaCursorRenderer not really a singleton, having cursor
inhibitor accounting here doesn't pan out. Make it MetaBackend API
so all cursor renderers get the same picture.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1403>
These use now more of a "pull" model, where they receive update
notifications and the relevant input position is queried, instead
of the coordinates being passed along.
This allows to treat cursor renderers all the same independently
of the device they track. This notifying of position changes should
ideally be more backend-y than core-y, a better location will be
figured out in future commits.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1403>
The cursor tracker may give us a valid position, and a
valid cursor sprite, and yet the cursor can be hidden,
meaning we must hide the cursor on the stream as well.
Remove cursor from stream buffer if it's hidden.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1421
Scanouts are taken away after painting. However, when we're
streaming, what we actually want is to capture whatever is
going to end up on screen - and that includes the scanout
if there's any.
Add a before-paint watch that only records new frames if a
scanout is set.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1421
When there's a direct scanout set in the stage view, we
have to use it instead of the view's regular onscreen
framebuffer.
Use the new CoglScanout API to implement blitting to the
stream framebuffer.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1421
PipeWire reuses buffers, and buffer metadatas, when streaming. When
the cursor is moved to outside the stream, the cursor meta also needs
to be updated, otherwise it'll use the cursor position of whatever is
in the buffer.
Don't bail out when cursor is outside the stream, and ensure to record
a metadata-only frame. This only applies to metadata streams.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1417
Always force-track the cursor position (so that the X11 backend can keep
it up to date), and if the cursor wasn't part of the sampled
framebuffer when reading pixels into CPU memory, draw it in an extra
pass using cairo after the fact. The cairo based cursor painting only
happens on the X11 backend, as we otherwise inhibit the hw cursor.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1391
During animation or other things that cause multiple frames in a row
being painted, we might skip recording frames if the max framerate is
reached.
Doing so means we might end up skipping the last frame in a series,
ending with the last frame we sent was not the last one, making things
appear to get stuck sometimes.
Handle this by creating a timeout if we ever throttle, and at the time
the timeout callback is triggered, make sure we eventually send an up to
date frame.
This is handle differently depending on the source type. A monitor
source type reports 1x1 pixel damage on each view its monitor overlaps,
while a window source type simply records a frame from the surface
directly, except without recording a timestamp, so that timestamps
always refer to when damage actually happened.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1361
Now that we don't use the record function to early out depending on
implicit state (don't record pixels if only cursor moved for example),
let it simply report an error when it fails, as we should no longer ever
return without pixels if nothing failed.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1361
Both do more or less the same but with different methods - one puts
pixels into a buffer using the CPU, the other puts pixels into a buffer
using the GPU.
However, they are behaving slightly different, which they shouldn't.
Lets first address the misleading disconnect in naming, and later we'll
make them behave more similarly.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1361
We'd check if there was any queued redraw on the stage, but this is
inappropriate for two reasons:
1) A monitor and area screen cast source only cares about damage on a
subset of the stage.
2) The global pending-redraw is going away when paint scheduling will be
more view centric.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1285
If there is a paint context available (i.e. for the phases that are
during the actual stage paint), pass it along the callbacks, so that
the callback implementations can change their operation depending on the
paint context state.
This also means we can get the current view from the paint context,
instead of the temporarily used field in the instance struct.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1207
Even though cogl_framebuffer_flush() was supposed to be enough,
it ends up creating streams with odd visual glitches that look
very much like unfinished frames.
Switch back to cogl_framebuffer_finish(), which is admittedly
an overkill, but it's what works for now. There is anedoctal
evidence showing it doesn't incur in worse performance.
https://gitlab.gnome.org/GNOME/mutter/merge_requests/1086
Prior to this commit the stage was drawn separately for each logical
monitor. This allowed to draw different parts of the stage with
different transformations, e.g. with a different viewport to implement
HiDPI support.
Go even further and have one view per CRTC. This causes the stage to
e.g. draw two mirrored monitors twice, instead of using the same
framebuffer on both. This enables us to do two things: one is to support
tiled monitors and monitor mirroring using the EGLStreams backend; the
other is that it'll enable us to tie rendering directly to the CRTC it
will render for. It is also a requirement for rendering being affected
by CRTC state, such as gamma.
It'll be possible to still inhibit re-drawing of the same content
twice, but it should be implemented differently, so that it will still
be possible to implement features requiring the CRTC split.
https://gitlab.gnome.org/GNOME/mutter/merge_requests/1042
This is inspired by 98892391d7 where the usage of
`g_signal_handler_disconnect()` without resetting the corresponding
handler id later resulted in a bug. Using `g_clear_signal_handler()`
makes sure we avoid similar bugs and is almost always the better
alternative. We use it for new code, let's clean up the old code to
also use it.
A further benefit is that it can get called even if the passed id is
0, allowing us to remove a lot of now unnessecary checks, and the fact
that `g_clear_signal_handler()` checks for the right type size, forcing us
to clean up all places where we used `guint` instead of `gulong`.
No functional changes intended here and all changes should be trivial,
thus bundled in one big commit.
https://gitlab.gnome.org/GNOME/mutter/merge_requests/940
Make the monitor implementation do things strictly related to its own
source type, leaving the Spa related logic and cursor read back in the
generic layer, later to be reused by the window source type
implementation.
https://gitlab.gnome.org/GNOME/mutter/merge_requests/413
It scaled the logical monitor rect with scale to get the stream
dimensions, but that is only valid when having
'scale-monitor-framebuffers' enabled. Even when it was, it didn't work
properly, as clutter_stage_capture_into() doesn't work properly with
scaled monitor framebuffers yet.
https://gitlab.gnome.org/GNOME/mutter/merge_requests/415
The 'cursor-mode', which currently is limited to RecordMonitor(), allows
the user to either do screen casts where the cursor is hidden, embedded
in the framebuffer, or sent as PipeWire stream metadata.
The latter allows the user to get cursor updates sent, including the
cursor sprite, without requiring a stage paint each frame. Currently
this is done by using the cursor sprite texture, and either reading
directly from, or drawing to an offscreen framebuffer which is read from
instead, in case the texture is scaled.
https://gitlab.gnome.org/GNOME/mutter/merge_requests/357
To get a consistent behaviour no matter whether HW cursors are in use or
not, make sure to copy the framebuffer content before the stage overlays
(cursor sprite textures) are painted.
https://gitlab.gnome.org/GNOME/mutter/merge_requests/357
The order and way include macros were structured was chaotic, with no
real common thread between files. Try to tidy up the mess with some
common scheme, to make things look less messy.
As of commit 5f5ef3de2cdc816dab82cb7eb5d7171bee0ad2c5 in pipewire the
stream creator can find out the node ID of the stream it created.
So instead of using a special purpose entry to the info property box to
let the application discover stream by monitoring added nodes searching
for the given special purpose entry, just pass the node directly.
https://bugzilla.gnome.org/show_bug.cgi?id=784199
This commit adds basic screen casting and remote desktoping
functionalty. This works by exposing two D-Bus API services:
org.gnome.Mutter.ScreenCast and org.gnome.Mutter.RemoteDesktop.
The remote desktop API is used to create remote desktop sessions. For
each session, a D-Bus object is created, and an application can manage
the session by sending messages to the session object. A remote desktop
session the user to emit input events using the D-Bus methods on the
session object. To get framebuffer content, the application should
create an associated screen cast session.
The screen cast API is used to create screen cast sessions. One can so
far either create stand-alone screen cast sessions, or a screen cast
session associated with a remote desktop session. A remote desktop
associated screen cast session is managed by the remote desktop session.
So far only remote desktop managed screen cast sessions are implemented.
Each screen cast session may have one or more streams. A screen cast
stream is a stream of buffers of some part of the compositor content.
So far API exists for creating streams of monitors and windows, but
only monitor streams are implemented.
When a screen cast session is started, the one PipeWire stream is
created for each screen cast stream created for the session. When this
has happened, a PipeWireStreamAdded signal is emitted on the stream
object, passing a unique identifier. The application may use this
identifier to find the associated stream being advertised by the
PipeWire daemon.
The remote desktop and screen cast functionality must be explicitly be
enabled at ./configure time by passing --enable-remote-desktop to
./configure. Doing this will build both screen cast and remote desktop
support.
To actually enable the screen casting and remote desktop, the user must
enable the experimental feature. See
org.gnome.mutter.experimental-features.
https://bugzilla.gnome.org/show_bug.cgi?id=784199
