citadel/docs/realms.md

Citadel Realms

Citadel contains only the base operating system and the Gnome desktop, it does not include any applications. To be able to install and run applications Citadel can create spaces which are called Realms.

A Realm is a container similar to a Docker or LXC container in which any Linux distribution could be installed. We use a Debian based image but it would not be difficult to create an image for another Linux distribution.

The realm containers are launched with systemd-nspawn but this is a detail of how they are implemented and not something it is necessary to learn about in order to use them.

Citadel provides a command-line tool realms for creating, managing, and launching Realm instances.

The default realm

One realm is always selected to be the default realm. This realm starts automatically when the system boots. The realms utility can be used to change which realm is the default realm. Switching the default realm changes the symlink /realm/default.realm to point to a different realm instance directory.

citadel:~# realms default
Default Realm: main

citadel:~# realms default project
[+] default realm changed from 'main' to 'project'

citadel:~# realms default
Default Realm: project

The current realm

Multiple realms may be launched at once but the Gnome Desktop is only associated with one of the running realms. This realm is called the current realm.

When displaying applications available to launch from the desktop, Gnome will only be aware of applications that are installed in the realm which is set as current and any application launched from the desktop will run inside this current realm.

Setting another realm as current does not affect any applications that are already running. Changing the current realm only means that any further applications which are launched will now run in the newly chosen realm.

Changing or querying the current realm is done with the realms current command, and if you choose a realm which is not currently running it will be automatically started.

citadel:~# realms current
Current Realm: main

citadel:~ # realms current project
[+]: Started realm 'project'
[+]: Realm 'project' set as current realm

citadel:~ # realms current
Current Realm: project

Underneath the hood, this command just changes the symlink /run/realms/current.realm to point to a new realm. This directory is monitored for changes with inotify and when the symlink changes a new set of .desktop files is swapped into a temporary directory where Gnome will look for metadata about which applications are installed.

Creating a new realm

New realms are created with the command realms new <realm name>

When a new realm is created a btrfs snapshot of some application image is created at /realms/realm-$name/rootfs. By default it is the base image (base.appimg) which is cloned as a snapshot. Application images are described in detail in a later section.

citadel:~ # realms new project
[+]: Populating realm home directory with files from /realms/skel
Create a snapshot of '/storage/appimg/base.appimg' in '/realms/realm-project/rootfs'

A new empty home directory is also created for the realm instance. Any file which are placed into the /realm/skel directory will be copied into any newly created realm home directory.

Realms configuration file

All of the curretly supported configuration options are listed below with their default values assigned.

use-shared-dir = true
use-sound = true
use-x11 = true
use-wayland = true
use-gpu = false
use-kvm = false
use-network = true
network-zone = "clear"

If you wish to change any of these options to something other than what is listed above add the corresponding line to the file /realms/realm-$name/config

citadel:~ # echo "use-gpu = true" > /realms/realm-main/config

Option use-shared-dir

Set to false to disable mounting the shared directory /realms/Shared into this realm at /home/user/Shared.

Option use-sound

Set to false to prevent mounting pulse audio socket and sound device into this realm.

Option use-x11

Set to false to prevent mounting /tmp/.X11-unix into the realm. This is the socket for communicating with the XWayland X11 compatibility daemon.

Option use-wayland

Set to false to prevent mounting the wayland display server socket /run/user/1000/wayland-0 into the realm.

Option use-gpu

Set to true to mount the device /dev/dri/renderD128 into the realm. Adding this device will make hardware graphics acceleration available to applications running in the realm.

Option use-kvm

Set to true to mount the device /dev/kvm into the realm. This will make it possible to run Qemu and other KVM based tools with hardware virtualization inside the realm.

Option use-network

Set to false to disable configuring the realm with access to the internet. The realm instance will only have a localhost network interface.

Option network-zone

Setting a name here will create bridge device in citadel with the name vz-$name if it doesn't already exist and attach this realm instance to that bridge.

Realms base directory layout

The realms base directory is stored on the storage partition at /storage/realms and is bind mounted to /realms on the root filesystem for convenience.

/realms
    config
    /Shared
    /skel
    /default.realm -> realm-main
    /realm-main
    /realm-project
    /realm-testing

File /realms/config

This file is a template of the configuration file for individual realms. When a new realm is created this file in copied into the new realm instance directory. By modifying this file, the default configuration for new realm instances can be changed.

Directory /realms/Shared

This directory is bind mounted to /home/user/Shared of each running realm that has the option use-shared-dir enabled. It's a convenient way to move files between different realms and between citadel and realms.

Directory /realms/skel

Files which are added to this directory will be copied into the home directory of any newly created realm. The directory is copied as a tree of files and may contain subdirectories.

Symlink /realms/default.realm

A symlink which points to a realm instance directory of the default realm. The default realm is the realm which starts when the system is booted.

Directory /realms/realm-$name

This is a realm instance directory, for a realm with $name as the realm name.

/realm-main
    config
    /home
    /rootfs

config

Configuration file for the realm instance copied from /realms/config or created by the user.

/home

Home directory for this realm. It will be mounted to /home/user in the realm instance.

/rootfs

The root filesystem of this realm. It is cloned from (a btrfs subvolume snapshot of) some application image.

Application Images

(Not to be confused with the AppImage packaging system)

The root filesystem for realms are called Application Images but we often use the shorter name appimg.

We have created a framework for building a Debian based images and we use this to build the default appimg that we ship.

We also encourage users to experiment with building their own custom images.

Tree Application Images are the only type of application image which are currently implemented for realms.

The rootfs is a tree of files on the filesystem, and it is also a btrfs subvolume which is cloned at zero cost (internally with btrfs subvolume snapshot) to use as the root filesystem of newly created realms.

Block Application Images (and also Sealed Application Images)

In the future we will add another type of application image called a Block Application Image. This type of image will be stored as a disk volume image file and will be mounted with a loop device rather than existing as a tree of files on the filesystem.

This will make it possible to enforce dm-verity verification over the image and ensure that no malicous or unintended modifications can be made to any of the the files on the root filesystem. Signature verification over the dm-verity root hash is done from the citadel rootfs image which is also secured with dm-verity. When enforcement of boot integrity is also implemented this will create a chain of cryptographic assurances that no component of the system has been tampered with.

Block images with signatures and dm-verify verification enabled are called Sealed Application Images

Updating an Application Image

To modify or update an application image run the realms update-appimg command. A container will be created for updating the image and a root shell session will open. From this session regular package management commands can be run. Any changes made will only affect future realms created from this appimg.

citadel:~ # realms update-appimg
[+]: Entering root shell on base appimg
root@base-appimg-update:/# apt update

[...]