Add pipelinea for h264 software encoding using openh264. This is the least
problematic (from a patent perspective) software encoder for h264, Fedora
ships it in a pre-installed repo and it can be enabled very easily. Most
people should have it enabled already to allow for decoding of h264 content.
Unlike vp8enc, this encoder is optimized for realtime and is really fast, it
outperforms the vp8 encoder by an order of magnitude and should allow for
smooth recordings even on older hardware.
The reason why mp4 was chosen as a container format over mkv is that mp4
can be played inside firefox and chromium, whereas mkv can't be played.
It's ensured that the mp4 file is still playable in case the recording
got cancelled by using the "first-moov-then-finalise" fragment-mode on
mp4mux.
Even though h264 is problematic from a patent perspective and often can't
be shipped by default, it still the best supported and most popular codec
out there. The software encoders and decoders are really fast, it's used
everywhere on the web, and it can be hardware decoded on almost any device
out there.
See also: https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2080https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/7335
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/3211>
Depending on the encoder we want to use a different container format and
therefore a different file extension. Right now this file extension is
forced to be webm, so shuffle things around a bit to make that more
dynamic.
Note that this also introduces removing for the old file created by the
recorder, otherwise it would create an empty "mp4" file every time it
falls back from "mp4" to "webm".
To be nice to external (ie. not gnome-shell) consumers of the
screencastService, let's not break the API completely, and detect the
".webm" suffix a little longer to not end up with weird file.webm.webm
filenames.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/3211>
When gstreamer crashes during recording, it pulls the whole screencastService
down with it.
These crashes are typically caused by the gstreamer pipeline that's in use,
so to avoid running into them again and again, we can blocklist the last
used pipeline in case the recorder didn't shut down (aka crashed) last time.
To store this state, create a file (gnome-shell-screencast-pipeline-blocklist)
in the XDG runtime dir and store the ID of the current pipeline in that file
before we try to start.
Now when we crash while running the pipeline, the entry in that file will stay
around and we'll pick it up on the next start of the screencastService as a
blocklist.
When the recording was successful on the other hand, we'll call
`this._updateServiceCrashBlocklist([...this._blocklistFromPreviousCrashes])`
and remove the new entry from the file again before shutting down the recorder.
In addition to that, we can now encourage the user to try recording again
after a crash happened. Adjust the failure notification a bit to say
"please try again".
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/6747
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2976>
Since we now propagate error types back to gnome-shell now, let's start
with showing a special error message in case the disk ran out of space,
which is probably the most typical error.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2976>
We'll be using hardware encoding for screencasts soon, so we'll likely see
more things go wrong in the future, including crashes of the whole
screencastService. To deal with this, we'll introduce logic to blocklist
certain recording pipelines in case of failure and also add some logic
to retry the recording automatically.
To allow for better messaging to the user in those failure cases, we want
to be aware in gnome-shell, what exactly the error in the recorder was.
So propagate the most common types of errors that can happen in the
ScreencastService to gnome-shell using the new DBusError module.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2976>
The module is shared between the various D-Bus services and the
main gnome-shell process, so it was originally left out to allow
porting different bits at their own speed.
Now that everything has been ported to ESM, there is no reason
to not move that particular module as well.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2868>
We want to replace gjs' custom (and now legacy) imports system
with standard EcmaScript modules: JS developers are already
familiar with them, they have better tooling support and using
standard features over non-standard ones is generally the right
thing to do.
Our D-Bus services are separate from the main process, and thus
can be ported separately (except for the few imports that are
shared with the main process' code base).
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2786>
When introducing the pipeline fallback mechanism, support for defining a
custom pipeline was unintentionally dropped, breaking extensions such as
EasyScreenCast.
Fixes 9cb40c4814b6be908edde4fb77a8d900a4cac51a
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2782>
Calling _teardownPipeline() before _tryNextPipeline() was actually not a
good idea, it sets the pipelineState to STOPPED, which means we can't try
any of the following pipelines anymore.
Instead what we want to do is set the pipeline state of the old pipeline to
NULL when trying a new one, without calling _teardownPipeline() for that.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2754>
This pipeline imports dmabufs and does format conversion using GL.
The `video/x-raw(memory:DMABuf)` filter ensures format negotiation
between `pipewiresrc` and Mutter will only succeed if Mutter advertises
dmabuf support as well, falling back to the next pipeline otherwise.
Using this pipeline frees Mutter from downloading buffer content on the
main thread which can have a big impact on compositor performance.
Doing format conversion on the GPU should further improve the overall
performance on most hardware.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2597>
Make sure gnome-shell gets notified of errors that happen during screen
recording using the screencastService, so that it can properly notify the
user about the error and tear down its state, too.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2752>
Pipewire versions < 0.3.67 may not fail immediatly on negotiation
errors, thus use the last/fallback pipeline directly.
Technically, a similar recent version of Wireplumber is required
as well, but we can't check that easily and the combination of old
Wireplumber and new Pipewire is quite unlikely.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2609>
Introduce a fallback mechanism for gstreamer pipelines that allows to
define multiple pipelines and prefer them over each other.
The way this works is that we introduce a new STARTING PipelineState.
While the Recorder is in that state, it is allowed to tear down the
current pipeline and start another one whenever an error happens, this is
used to try multiple pipelines in a fixed order until a working one is
found.
Right now there's just a single pipeline using the existing vp8 encoder, the
actual new encoders and pipelines will be added in a separate commit.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2609>
The check for the Pipewire version was originally introduced in
d32c0348 which states:
> Since it is not clear yet when a proper solution will arrive,
> this makes use of `always-copy` as a workaround for now and
> should be reverted once it is no longer needed.
The check for a stable Gstreamer version with the mention proper fix was
introduced in d7b44319 and carried for the 43 cycle.
By the time Gnome 44 will be released all distros should have had enough
time to update their Gstreamer version - or backport the patches, in
case of not ustream-supported versions.
Thus lets drop it now.
Note: `always-copy` is not suitable for dmabuf buffers as it copies via
mmap.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2609>
Right now when we tell gstreamer to move the pipeline to the state
PLAYING, we pretend that happens immediately and set our PipelineState
to PLAYING right afterwards.
In reality though it's more complicated than that: Gstreamer changes
states asynchronously and set_state() returns a Gst.StateChangeReturn.ASYNC.
In that case we should wait until the according STATE_CHANGED event happens
on the bus, and only then set our PipelineState to PLAYING.
Since the STATE_CHANGED event will also happen when set_state() returns
SUCCESS, we can use the same async logic for that.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2197>
Gstreamer can produce various errors, we shouldn't pretend those don't
exist and go on as usual when one happens. Instead, when an error
happens, tear down the pipeline, set our PipelineState to the new ERROR
state and bail out with a proper error message.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2197>
We're tracking three different state-machines in the Recorder object:
The state of the gstreamer pipeline, the state of the screencast
session, and the sender of our dbus invocation that might vanish.
Properly handling errors that might appear in any of those three "black
boxes" is not easy, especially tearing down the other two when one of
them breaks.
So refactor the error handling here: Add a single error path for each of
those three states we're tracking, and make them all subsequently call
the _bailOutOnError() method. From there we tear down the other states and
call the error callbacks.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2197>
The videos directory doesn't necessarily have to exist, users are free to
delete it. Right now we don't handle this case and screencasting fails.
Let's handle it and fall back to the users home directory instead when
xdg-videos doesn't exist.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2594>
Some gstreamer plugins require a connection to the display server,
so they block until the server is up and running. That's why we
moved the check into the D-Bus service, so that the blocking would
not lock up the compositor itself.
However the block can still delay the service initialization so
much that auto-shutdown hits immediately when returning from the
constructor. If that happens, the proxy on the shell side is no
longer backed by a remote object when the init callback runs, and
all properties therefore resolve as `null`.
As a result, gnome-shell thinks that screencasts aren't supported
and hides the screencast button.
Fix this by holding the service during the gstreamer checks, so
that the auto-shutdown timeout only starts after the service is
ready.
https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/6051
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2533>
Pipewire 0.3.52 via commit a1f33a99 introduced a change that affects how
long pipewiresrc holds onto the pw_buffers it dequeued. Before that
change the pw_buffer was held until the end of the videoconvert element
at the beginning of the pipeline. After that change the pw_buffer was
held onto until the filesink at the end of the pipeline. This was
starving MetaScreenCastStreamSrc of pw_buffers to record new frames
into, resulting in the majority of frames being missed, especially in
situations in which the encoder was taking longer.
Pipewire 0.3.57 via commit 1ea1d525 will allow queuing the pw_buffer
early again via the `always-copy` option. This however is only a
workaround until a proper solution is found in either pipewire or
gstreamer that does not depend on copying the buffer contents and
instead queues the pw_buffer again after videoconvert as prior to
a1f33a99.
Since it is not clear yet when a proper solution will arrive, this makes
use of `always-copy` as a workaround for now and should be reverted once
it is no longer needed.
Related: https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/5585
Related: https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/2461
Related: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/283
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2436>
Screenshots use `%Y-%m-%d %H-%M-%S` for the timestamp, which has the
advantage of allowing proper lexicographical sorting.
The screencast file name pattern, on the other hand, uses
locale-dependent expansions, which break sorting based on file name, and
introduces the chance of potentially invalid characters on different
file systems.
Fixes: #5115
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2300>
If one of these libraries breaks its GIR API in future, then upgrading
packages unrelated to gnome-shell might pull in the newer version,
causing gnome-shell to crash when it gets a newer GIR API that is
incompatible with its expectations. For example, this seems to be
happening in Debian testing at the moment, when GNOME Shell 41.4
imports GWeather and can get version 4.0 instead of the version 3.0 that
it expected.
Adding explicit API versions at the time the newer version is released
is too late, because that will still let the newer version of the GIR API
break pre-existing GNOME Shell packages. Prevent similar crashes in
future by making the desired versions explicit.
This is done for all third-party libraries except GLib, similar to the
common practice in Python code; if GLib breaks API, then that will be
a disruptive change to the whole GLib/GObject ecosystem, regardless.
Gvc, Meta, Shell, Shew, St are not included because they're private
(only exist in a non-default search path entry).
Clutter and Cogl *are* included, because we need to import the fork of
them that comes with Meta, as opposed to their deprecated standalone
versions.
Signed-off-by: Simon McVittie <smcv@debian.org>
Bug-Debian: https://bugs.debian.org/1008926
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2261>
Instead of testing if the videos directory exists and using the home
directory otherwise, just try to create the target directory. This
aligns with how the screenshot UI handles the screenshots folder, and
it's convenient for putting screencasts into a subdirectory.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/2102>
If parsing the pipeline fails for some reason, we currently end up
with a zombie session that leads to a stuck recording indicator in
gnome-shell.
Instead, properly tear down the session to allow mutter and gnome-shell
to correctly update their state.
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/1878>
The current gstreamer pipeline performs quite bad on slower machines and
is dropping lots of frames, improve the pipeline by changing a few
things:
- Use threads for videoconvert and improve speed of videoconvert by
disabling some unneeded things
- Add a queue before the encoding step, this allows the encoder to work
at its own pace and will lead to a lot more stability
- Remove the fixed quantizer and only set a max quantizer, this helps
quite a bit with performance
- Change the deadline parameter of vp8enc to 1: This makes the encoder
go into real time mode, which will make it a lot faster
- Set cpu-used to 16, the maximum possible value.
- Set static-threshold to 1000, static-threshold is the motion detection
threshold, and while a value of 100 is recommended for screencasting in
the gstreamer documentation (see [1]), using 1000 appears to perform a
lot better and still outputs fairly good quality
- Set a larger buffer size than the default size, this seems to get a
bit more stability during high load scenarios
All in all, those changes make the pipeline drop no more frames when
recording at 30 FPS and 2K screen resolution. That was tested on a
fairly recent mobile core-i5 processor.
Also, because we now have two %T replacement strings for the number of
threads, we need to switch to replaceAll(). For that to work, we have to
put the %T matching expression into quotes.
[1] https://gstreamer.freedesktop.org/documentation/vpx/GstVPXEnc.html?gi-language=c#GstVPXEnc:static-threshold
Part-of: <https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/1633>
Move the screencasting into a separate D-Bus service process, using
PipeWire instead of Clutter API. The service is implemented in
Javascript using the dbusService.js helper, and implements the same API
as was done by screencast.js and the corresponding C code.
https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/1372
