When a stream source subclass asks for a DMA-BUF only frame record,
it is legitimate to return FALSE in do_record_frame() - meaning that
a frame was not recorded - but not return an error - meaning nothing
actually failed.
This avoids spamming the journal with warnings on a legitimate case.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2804>
Add meta_screen_cast_stream_src_maybe_record_frame_with_timestamp()
which operates on arbitrary timestamps; and make the current function
meta_screen_cast_stream_src_maybe_record_frame() just call into the
new variant, passing g_get_monotonic_time() as the timestamp.
This will be useful later we start using the target timestamp of the
frame for screencasting.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2804>
This change will export the damaged regions (when available) out
to the pipewire client. This change is currently specific to
virtual streams only (where I was able to test the change) and
maintains the current behavior for other screencast stream types.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2775>
Streams are generally recoverable by the client and errors may happen
e.g. on negotiation failures. Right now we close the stream and
corresponding session, which is neither necessary nor expected by
clients.
Just disable the stream instead and let clients handle things as they
seem fit. This allows clients to e.g. try several Gstreamer pipelines
with limited caps on a single stream.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2850>
The fields of 'priv->video_format.max_framerate' are all of type
uint32_t. Multiplying by G_USEC_PER_SEC can overflow, and equally,
dividing a large numerical type by uint32_t can err too.
Since the variable holding the result is int64_t, cast all uint32_t
fields to int64_t before doing any maths on it.
Spotted while trying to investigating an issue with framerates on
HDMI screencasts.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2762>
meta_screen_cast_stream_src_set_cursor_sprite_metadata() receives
the cursor sprite, position, and scale, and with that it downloads
the cursor sprite by drawing it into a separate framebuffer, then
calls cogl_framebuffer_read_pixels() in it - this is the offscren
path that is very common when using screen capturing applications
such as OBS Studio.
There's a sneaky issue in this code path though: the 'scale' value
is a float. The cursor size is then determined by multiplying the
sprite width and height - two integer variables - by scale, and
this relies on standard float-to-int conversions. This is problematic
as sometimes the rounded values disagree with what is expected by
cogl_framebuffer_read_pixels(). If the packing of either the cursor
width or height is off by one, glReadPixels() will try to write into
off bounds, which crashes.
This can be reproduced by enabling fractional scaling, setting a 150%
zoom level, on a 4K screen, and opening any commit with an image diff
in gitlab.gnome.org, all while screencasting. When hovering the new
image, the cursor sprite will be such that it triggers this code path,
and reproduces this issue.
Fix this by always ceiling the cursor sprite sizes.
Closes https://gitlab.gnome.org/GNOME/mutter/-/issues/2542
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2736>
With this header we can set a flag to signal that the whole buffer is
corrupt and should be ignored. With this we can cater to all cases:
* Window buffer fine, but cursor broken:
Use the spa_meta_cursor properties like id or offset accordingly
* Window buffer broken, but cursor fine:
Use the chunk flags
* Both are broken / the dequeued buffer is not usable
Use the spa_meta_header flag
Additionally clients can now check if a buffer contains spa_meta_header
data and can thus only check for the new or the old behaviour.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2516>
PipeWire supports flags to signal a corrupted buffer. We should use the
flag SPA_CHUNK_FLAG_CORRUPTED for `chunk->flags` instead of setting
`chunk->size = 0` since the size isn't well defined for arbitrary dmabufs
and should be set to 0.
Sadly clients like obs are using a chunk size of 0 to decide if a buffer
should be imported. Thus we should offer both until clients are using
the flag.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2323>
If the remote desktop service emits absolute input events (e.g. absolute
pointer events) before the stream has started streaming, we don't have a
virtual monitor, as the size has not been negotiated yet. When this
happens, just drop the event. Remote desktop services should probably
make sure not to send events before the streaming has started, but them
doing so anyway shouldn't trigger a crash, which would be the case
otherwise.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2270>
Keep the virtual monitor around if it's being resized. This reduces the
number of unnecessary object rebuilding that happen during monitor
rebuilding.
This changes finalize() vfunc into a dispose() vfunc in the abstract
stream source object implementation, as it needs the abstract stream
source object to close the stream early, so that various signal
listeners get disconnected early.
Closes: https://gitlab.gnome.org/GNOME/mutter/-/issues/1904
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2270>
Add some debug logging that allows checking whether we're using DMA
buffers for screencasting or system memory buffers. This can be useful
for debugging screencasting performance and CPU usage.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2256>
With the ability to query the renderer for DMA-BUF support we can
announce support for implicit modifiers. This allows PipeWire to check
for matching modifiers while negotiation.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1939>
Next commits will reintroduce a certain behavior of stage
capturing that can only happen with DMA-BUF buffers. To
control this, add a new flag tp MetaScreenCastRecordFlags
for this behavior.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/2186>
The area source, window source, and monitor source, currently set up the
stream size up front, given the area, maximum allowed window size or
monitor resolution, but for to be introduced sources, the size will be
negotiated using PipeWire, instead of specified via the D-Bus API. This
commit changes the internal source API to allow for this. There are
currently no users of this new behaviour.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1698>
When recording the screen and real time encoding it using a gstreamer
pipeline, that pipeline can stall when the encoder is too slow. Log a
debug message using the new SCREEN_CAST debug topic in that case so we
know when framedrops are happening.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1709>
The other end of the PipeWire stream can set the buffer data type to a
bitmask of supported buffer types. We should respect this, and not
attempt to allocate a DMA buffer if it isn't asked for.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1697>
Instead of getters, pass the width, height and stride around when
relevant. This also removes the redudant "stream_size" and
"stream_height" variables from the src struct, as they are already part
of the video format.
Part-of: <https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1697>
A first step towards abandoning the CoglObject type system: convert
CoglFramebuffer, CoglOffscreen and CoglOnscreen into GObjects.
CoglFramebuffer is turned into an abstract GObject, while the two others
are currently final. The "winsys" and "platform" are still sprinkled
'void *' in the the non-abstract type instances however.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1496
Currently, the maximum size for a mouse pointer bitmap for screen
casting is 64x64 pixels.
However, this limit is hit way too often as it is way too low and
results in crashes in either gnome-remote-desktop or mutter.
For example: The a11y settings in g-c-c allow setting a larger pointer
bitmap in order to increase the visibility of the mouse pointer.
With the current limit of 64x64 pixels it is not possible to use the
larger variants of the default mouse pointer bitmap, without
experiencing any crash.
Another way to hit the limit is when display scaling is used or some
game uses a custom (large) mouse pointer bitmap.
The VNC backend in gnome-remote-desktop does not seem to have a maximum
pointer bitmap size.
The RDP backend on the other hand has a maximum pointer bitmap size at
384x384.
Use this size (384x384) as maximum size instead of the current 64x64
size for mouse pointer bitmaps to avoid crashes in mutter and
gnome-remote-desktop and to ensure that bigger mouse pointer bitmaps
can be used.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1414
Always force-track the cursor position (so that the X11 backend can keep
it up to date), and if the cursor wasn't part of the sampled
framebuffer when reading pixels into CPU memory, draw it in an extra
pass using cairo after the fact. The cairo based cursor painting only
happens on the X11 backend, as we otherwise inhibit the hw cursor.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1391
During animation or other things that cause multiple frames in a row
being painted, we might skip recording frames if the max framerate is
reached.
Doing so means we might end up skipping the last frame in a series,
ending with the last frame we sent was not the last one, making things
appear to get stuck sometimes.
Handle this by creating a timeout if we ever throttle, and at the time
the timeout callback is triggered, make sure we eventually send an up to
date frame.
This is handle differently depending on the source type. A monitor
source type reports 1x1 pixel damage on each view its monitor overlaps,
while a window source type simply records a frame from the surface
directly, except without recording a timestamp, so that timestamps
always refer to when damage actually happened.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1361
Now that we don't use the record function to early out depending on
implicit state (don't record pixels if only cursor moved for example),
let it simply report an error when it fails, as we should no longer ever
return without pixels if nothing failed.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1361
Both do more or less the same but with different methods - one puts
pixels into a buffer using the CPU, the other puts pixels into a buffer
using the GPU.
However, they are behaving slightly different, which they shouldn't.
Lets first address the misleading disconnect in naming, and later we'll
make them behave more similarly.
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1361
The stream will clean up the buffers, so let it do that before we
destroy them under its feet. Note that it'll only do this after the
following PipeWire commit:
commit fbaa4ddedd84afdffca16f090dcc4b0db8ccfc29
Author: Wim Taymans <wtaymans@redhat.com>
Date: Mon Jun 1 15:36:09 2020 +0200
stream: allow NULL param and 0 buffers in disconnect
https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1283
We're iterating inside the PipeWire loop when detecting PipeWire errors,
and shouldn't destroy the PipeWire objects mid-iteration. Avoid this by
first disabling the stream src (effectively stopping the recording),
then notifying about it being closed in an idle callback. The
notification eventually makes the rest of the screen cast code clean up
the objects, including the src and the associated PipeWire objects, but
will do so outside the PipeWire loop iteration.
Closes: https://gitlab.gnome.org/GNOME/mutter/-/issues/1251https://gitlab.gnome.org/GNOME/mutter/-/merge_requests/1251
