1
0
forked from brl/citadel

appimg-builder renamed to realmfs-builder

This commit is contained in:
Bruce Leidl 2019-04-02 15:38:19 -04:00
parent a8947e599a
commit e1bda9ee5e
27 changed files with 242 additions and 51 deletions

View File

@ -1,10 +1,10 @@
Application Image Builder
Base RealmFS Image Builder
=========================
Application Images (or appimgs for short) are created with this builder
framework. The build is controlled by a configuration file which can customize
the build process in various ways such as adding extra packages and specifying
shell functions to be run at certain stages.
RealmFS Images are created with this builder framework. The build is controlled
by a configuration file which can customize the build process in various ways
such as adding extra packages and specifying shell functions to be run at certain
stages.
The configuration file is really just a shell script but it should follow the
conventions described in the Configuration File section of this document.
@ -20,17 +20,17 @@ directory the rootfs is built on.If you are tweaking a config and making repeate
builds this is not only a lot faster, but will also avoid hammering SSD drives with
excessive writes (and write amplification).
By default the application image builder is self-hosting and can always be
By default the RealmFS image builder is self-hosting and can always be
run from inside images that it creates. Building an image is as easy as:
mkdir work && cd work : Make a directory to work in
appimg-builder --new : writes a template file build.conf in current directory
realmfs-builder --new : writes a template file build.conf in current directory
vim build.conf : (optionally) make some changes to the template
sudo appimg-builder : Build an application image
sudo realmfs-builder : Build a RealmFS image
If you want you can even skip the steps of creating and editing a config file and
just run appimg-build in a work directory and it will build the default appimg we use
with Citadel.
just run realmfs-build in a work directory and it will build the default base realmfs
image we use with Citadel.
Stage One
---------
@ -44,7 +44,7 @@ Stage Two
The stage-two.sh script mostly just orchestrates the execution of small
fragments of shell script code that are called 'modules'. The base framework
modules can be found in the directory /usr/share/appimg-builder/appimg-modules.
modules can be found in the directory /usr/share/realmfs-builder/realmfs-modules.
It imports the configuration file with the 'source' command after all the key
variables and functions have been defined. It's possible to override any of
@ -81,7 +81,7 @@ corresponding 'base' variable that could be overidden if necessary.
- Modules
Modules can be functions that you define or they can be loaded from files on
disk. To use files rather than functions a directory named 'appimg-modules'
disk. To use files rather than functions a directory named 'realmfs-modules'
must exist as a subdirectory of the directory containing the configuration file.
Any files you place in this directory will be found by name during the module
execution stages.
@ -89,7 +89,7 @@ execution stages.
- Installing Files
If you would like to have external files such as configuration files copied into
the image, create 'appimg-files' as a subdirectory of the directory containing
the image, create 'realmfs-files' as a subdirectory of the directory containing
the configuration file. You can then use the install_file command inside of a
module to copy the files from this directory. You can either store the files to
install in a flat directory or organize them into subdirectories mirroring the
@ -99,14 +99,14 @@ refers to the directory in which your configuration file is located.
(1): install_file [mode] [file] [target directory]
Example: Install BASE/appimg-files/my_config.conf
Example: Install BASE/realmfs-files/my_config.conf
to /etc/mydaemon/my_config.conf
install_file 0644 my_config.conf /etc/mydaemon
(2): install_file [mode] [full path]
Example: Install BASE/appimg-files/etc/mydaemon/my_config.conf
Example: Install BASE/realmfs-files/etc/mydaemon/my_config.conf
to /etc/mydaemon/my_config.conf
install_file 0644 /etc/mydaemon/my_config.conf

View File

@ -1,14 +1,14 @@
#
# Application Image Builder configuration file template
# RealmFS Image Builder configuration file template
#
# Synopsis:
#
# mkdir work && cd work
# appimg-builder --new
# realmfs-builder --new
# vim build.conf
# sudo appimg-builder build.conf
# sudo realmfs-builder build.conf
#
# Full documentation in /usr/share/appimg-builder/README
# Full documentation in /usr/share/realmfs-builder/README
#
#
@ -62,11 +62,11 @@ example-module() {
}
#
# If you add a subdirectory called 'appimg-files' to the
# If you add a subdirectory called 'realmfs-files' to the
# directory containing this build config file and then
# add a file 'example.service':
#
# THISDIR/appimg-files/example.service
# THISDIR/realmfs-files/example.service
#
# You can then install it by running install_file in a module
# like this:

View File

@ -2,11 +2,11 @@
: ${DEBIAN_MIRROR:="https://deb.debian.org/debian"}
: ${DEBIAN_RELEASE:="buster"}
BASE_PACKAGES="debootstrap iproute2 less xz-utils sudo dbus libpam-systemd openssh-client packagekit-gtk3-module libcanberra-gtk3-module libpulse0 fonts-roboto-hinted bash-completion"
BASE_PACKAGES="iproute2 less xz-utils sudo dbus libpam-systemd openssh-client packagekit-gtk3-module libcanberra-gtk3-module libpulse0 fonts-roboto-hinted bash-completion"
BASE_PRE_INSTALL_MODULES="utility-library configure-locale create-user set-hostname no-install-recommends write-apt-sources"
BASE_POST_INSTALL_MODULES="loginctl-enable-linger configure-systemd install-configure-host0 install-launch-script setup-theme-symlinks"
BASE_POST_INSTALL_MODULES="loginctl-enable-linger configure-systemd install-configure-host0 install-launch-script setup-theme-symlinks apt-cacher-ng add-xsettings-schema"
trap "exit 1" TERM
export TOP_PID=$$

View File

@ -0,0 +1,12 @@
# This configuration snipped is intended to be stored in /etc/apt/apt.conf.d/
# on the client system in order to change a regular setup to use apt-cacher-ng.
#
Acquire::http::Proxy "http://172.17.0.213:3142/";
# Little optimization. A value of 10 has been used in earlier version of
# apt-get but was disabled in the beginning of the second decade because of
# incompatibilities with certain HTTP proxies. However, it still beneficial
# with proxy servers that support it good enough (like apt-cacher-ng).
#
Acquire::http::Pipeline-Depth "23";

View File

@ -1,5 +1,6 @@
#!/bin/bash
export REALM_NAME
export GDK_BACKEND
export DISPLAY=:0
export DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
export XDG_RUNTIME_DIR=/run/user/1000

View File

@ -0,0 +1,83 @@
<!-- This file is generated by glib-mkenums, do not modify it. This code is licensed under the same license as the containing project. Note that it links to GLib, so must comply with the LGPL linking clauses. -->
<schemalist>
<enum id="org.gnome.settings-daemon.GsdFontAntialiasingMode">
<value nick="none" value="0"/>
<value nick="grayscale" value="1"/>
<value nick="rgba" value="2"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdFontHinting">
<value nick="none" value="0"/>
<value nick="slight" value="1"/>
<value nick="medium" value="2"/>
<value nick="full" value="3"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdFontRgbaOrder">
<value nick="rgba" value="0"/>
<value nick="rgb" value="1"/>
<value nick="bgr" value="2"/>
<value nick="vrgb" value="3"/>
<value nick="vbgr" value="4"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdSmartcardRemovalAction">
<value nick="none" value="0"/>
<value nick="lock-screen" value="1"/>
<value nick="force-logout" value="2"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdTouchpadScrollMethod">
<value nick="disabled" value="0"/>
<value nick="edge-scrolling" value="1"/>
<value nick="two-finger-scrolling" value="2"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdBellMode">
<value nick="on" value="0"/>
<value nick="off" value="1"/>
<value nick="custom" value="2"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdTouchpadHandedness">
<value nick="right" value="0"/>
<value nick="left" value="1"/>
<value nick="mouse" value="2"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdWacomRotation">
<value nick="none" value="0"/>
<value nick="cw" value="1"/>
<value nick="ccw" value="2"/>
<value nick="half" value="3"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdWacomActionType">
<value nick="none" value="0"/>
<value nick="custom" value="1"/>
<value nick="switch-monitor" value="2"/>
<value nick="help" value="3"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdPowerActionType">
<value nick="blank" value="0"/>
<value nick="suspend" value="1"/>
<value nick="shutdown" value="2"/>
<value nick="hibernate" value="3"/>
<value nick="interactive" value="4"/>
<value nick="nothing" value="5"/>
<value nick="logout" value="6"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdPowerButtonActionType">
<value nick="nothing" value="0"/>
<value nick="suspend" value="1"/>
<value nick="hibernate" value="2"/>
<value nick="interactive" value="3"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdUpdateType">
<value nick="all" value="0"/>
<value nick="security" value="1"/>
<value nick="none" value="2"/>
</enum>
<enum id="org.gnome.settings-daemon.GsdNumLockState">
<value nick="unknown" value="0"/>
<value nick="on" value="1"/>
<value nick="off" value="2"/>
</enum>
</schemalist>
<!-- Generated data ends here -->

View File

@ -0,0 +1,35 @@
<?xml version="1.0" encoding="UTF-8"?>
<schemalist>
<schema gettext-domain="gnome-settings-daemon" id="org.gnome.settings-daemon.plugins.xsettings" path="/org/gnome/settings-daemon/plugins/xsettings/">
<key name="antialiasing" enum="org.gnome.settings-daemon.GsdFontAntialiasingMode">
<default>'grayscale'</default>
<summary>Antialiasing</summary>
<description>The type of antialiasing to use when rendering fonts. Possible values are: “none” for no antialiasing, “grayscale” for standard grayscale antialiasing, and “rgba” for subpixel antialiasing (LCD screens only).</description>
</key>
<key name="hinting" enum="org.gnome.settings-daemon.GsdFontHinting">
<default>'slight'</default>
<summary>Hinting</summary>
<description>The type of hinting to use when rendering fonts. Possible values are: “none” for no hinting and “slight” for fitting only to the Y-axis like Microsofts ClearType, DirectWrite and Adobes proprietary font rendering engine. Ignores native hinting within the font, generates hints algorithmically. Used on Ubuntu by default. Recommended. The meaning of “medium” and “full” depends on the font format (.ttf, .otf, .pfa/.pfb) and the installed version of FreeType. They usually try to fit glyphs to both the X and the Y axis (except for .otf: Y-only). This can lead to distortion and/or inconsistent rendering depending on the quality of the font, the font format and the state of FreeTypes font engines.</description>
</key>
<key name="rgba-order" enum="org.gnome.settings-daemon.GsdFontRgbaOrder">
<default>'rgb'</default>
<summary>RGBA order</summary>
<description>The order of subpixel elements on an LCD screen; only used when antialiasing is set to “rgba”. Possible values are: “rgb” for red on left (most common), “bgr” for blue on left, “vrgb” for red on top, “vbgr” for red on bottom.</description>
</key>
<key name="disabled-gtk-modules" type="as">
<default>[]</default>
<summary>List of explicitly disabled GTK+ modules</summary>
<description>A list of strings representing the GTK+ modules that will not be loaded, even if enabled by default in their configuration.</description>
</key>
<key name="enabled-gtk-modules" type="as">
<default>[]</default>
<summary>List of explicitly enabled GTK+ modules</summary>
<description>A list of strings representing the GTK+ modules that will be loaded, usually in addition to conditional and forcibly disabled ones.</description>
</key>
<key type="a{sv}" name="overrides">
<default>{}</default>
<summary>A dictionary of XSETTINGS to override</summary>
<description>This dictionary maps XSETTINGS names to overrides values. The values must be either strings, signed int32s or (in the case of colors), 4-tuples of uint16 (red, green, blue, alpha; 65535 is fully opaque).</description>
</key>
</schema>
</schemalist>

View File

@ -0,0 +1,15 @@
[Unit]
Description=Apt-Cacher NG software download proxy
After=network.target
ConditionPathExists=/usr/share/apt-cacher-ng/conf
[Service]
ExecStart=/usr/sbin/apt-cacher-ng -c /usr/share/apt-cacher-ng/conf ForeGround=1
User=1000
Group=1000
# this can be changed to notify if the support was enabled at build time
Type=notify
Restart=on-failure
[Install]
WantedBy=multi-user.target

View File

@ -0,0 +1,5 @@
SCHEMA_DIR=/usr/share/glib-2.0/schemas
install_file 0644 org.gnome.settings-daemon.enums.xml $SCHEMA_DIR
install_file 0644 org.gnome.settings-daemon.plugins.xsettings.gschema.xml $SCHEMA_DIR
glib-compile-schemas $SCHEMA_DIR

View File

@ -0,0 +1,17 @@
info "Setting up apt-cacher-ng"
apt-get --assume-no install apt-cacher-ng
systemctl -q disable apt-cacher-ng.service
#
# Configure RealmFS to use download updates through apt-cacher realm
#
install_file 0644 000apt-cacher-ng-proxy /etc/apt/apt.conf.d/
sed -i 's%https:%http://HTTPS/%' /etc/apt/sources.list
#
# Configure RealmFS to potentially run as apt-cacher system realm
#
mkdir -p /usr/share/apt-cacher-ng
install_file 0644 realm-apt-cacher-ng.service /usr/lib/systemd/system
systemctl -q enable realm-apt-cacher-ng.service

View File

@ -14,3 +14,7 @@ cat > ${loginconfdir}/50-no-kill-user-processes.conf << EOF
[Login]
KillUserProcesses=no
EOF
info "Creating /var/log/journal"
mkdir -p /var/log/journal
systemd-tmpfiles --create --prefix /var/log/journal

View File

@ -1,4 +1,4 @@
local hostname=${APPIMG_HOSTNAME:-"subgraph"}
local hostname=${REALMFS_HOSTNAME:-"subgraph"}
info "Setting hostname to '$hostname' in /etc/hosts and /etc/hostname"
printf "127.0.0.1\t${hostname} localhost\n" > /etc/hosts
printf "${hostname}\n" > /etc/hostname

View File

@ -1,6 +1,6 @@
info "utility library loaded"
SEARCH_PATH="/tmp/appimg-build;${APPIMG_BUILDER_BASE}"
SEARCH_PATH="/tmp/realmfs-build;${REALMFS_BUILDER_BASE}"
# Searches for modules and files in SEARCH_PATH directories
#
@ -9,7 +9,7 @@ SEARCH_PATH="/tmp/appimg-build;${APPIMG_BUILDER_BASE}"
#
search_item() {
local IFS=";" subtype=${1} item=${2}
local subdir="appimg-${subtype}s"
local subdir="realmfs-${subtype}s"
for dir in ${SEARCH_PATH}; do
local fullpath=${dir}/${subdir}/${item}
@ -25,7 +25,7 @@ search_item() {
#
# Execute a module by name. If a function exists with the
# requested module name it will be executed, otherwise scripts
# with this name are searched for in SEARCH_PATH /appimg-modules
# with this name are searched for in SEARCH_PATH /realmfs-modules
# subdirectories.
#

View File

@ -1,7 +1,7 @@
#!/bin/bash
: ${APPIMG_BUILDER_BASE:="/usr/share/appimg-builder"}
source ${APPIMG_BUILDER_BASE}/common.inc
: ${REALMFS_BUILDER_BASE:="/usr/share/realmfs-builder"}
source ${REALMFS_BUILDER_BASE}/common.inc
umount_if_tmpfs() {
if findmnt -t tmpfs -M ${1} > /dev/null; then
@ -46,14 +46,14 @@ setup_chroot() {
mount chsys ${ROOTFS}/sys -t sysfs
mount chtmp ${ROOTFS}/tmp -t tmpfs
# Install a copy of appimg-builder inside new image
# Install a copy of realmfs-builder inside new image
mkdir -p ${ROOTFS}/usr/share
cp -a ${APPIMG_BUILDER_BASE} ${ROOTFS}/usr/share
ln -s /usr/share/appimg-builder/stage-one.sh ${ROOTFS}/usr/bin/appimg-builder
cp -a ${REALMFS_BUILDER_BASE} ${ROOTFS}/usr/share
ln -s /usr/share/realmfs-builder/stage-one.sh ${ROOTFS}/usr/bin/realmfs-builder
# $BUILDFILE and any extra files go in /tmp/appimg-build of rootfs
mkdir -p ${ROOTFS}/tmp/appimg-build
cp ${BUILDFILE} ${ROOTFS}/tmp/appimg-build/build.conf
# $BUILDFILE and any extra files go in /tmp/realmfs-build of rootfs
mkdir -p ${ROOTFS}/tmp/realmfs-build
cp ${BUILDFILE} ${ROOTFS}/tmp/realmfs-build/build.conf
}
cleanup_chroot() {
@ -79,14 +79,14 @@ run_chroot_stage() {
DEBCONF_NONINTERACTIVE_SEEN=true \
LC_ALL=C LANGUAGE=C LANG=C \
DEBIAN_RELEASE=${DEBIAN_RELEASE} DEBIAN_MIRROR=${DEBIAN_MIRROR} \
chroot ${ROOTFS} /usr/share/appimg-builder/stage-two.sh /tmp/appimg-build/build.conf
chroot ${ROOTFS} /usr/share/realmfs-builder/stage-two.sh /tmp/realmfs-build/build.conf
info "chroot installation stage finished, cleaning chroot setup"
cleanup_chroot
}
generate_tarball() {
local tarball=${WORKDIR}/appimg-rootfs.tar
local tarball=${WORKDIR}/realmfs-rootfs.tar
info "----- Generating rootfs tarball -----"
tar -C ${ROOTFS} --numeric-owner -c --xattrs --xattrs-include=* -f $tarball .
@ -100,20 +100,31 @@ generate_tarball() {
echo
}
generate_image() {
# BLOCKS=$(du -ks ${ROOTFS} | cut -f1)
# BLOCKS=$(expr ${BLOCKS} \* 12 / 10)
# SIZE=$(expr ${BLOCKS} \* 1024)
# echo "Size is ${SIZE}"
BLOCKS=$(expr 340 \* 1024)
# allow online resize up to 32G
dd if=/dev/zero of=${WORKDIR}/citadel-realmfs.ext4 seek=${BLOCKS} count=0 bs=4096
mkfs.ext4 -d ${ROOTFS} -i 4096 -b 4096 -F ${WORKDIR}/citadel-realmfs.ext4 ${BLOCKS}
}
usage() {
cat <<-EOF
USAGE: appimg-builder [options] [config-file]
USAGE: realmfs-builder [options] [config-file]
OPTIONS
--new Create a configuration file template called build.conf in the current directory
-d <directory> Choose a non-default directory for build output (currently: $(pwd)/appimg)
-d <directory> Choose a non-default directory for build output (currently: $(pwd)/realmfs)
-t Create a tarball but don't compress it
-z Create a tarball compressed with xz
--no-tmpfs Do not use tmpfs as rootfs build directory
--no-confirm Do not ask for confirmation before beginning
For more documentation see /usr/share/appimg-builder/README
For more documentation see /usr/share/realmfs-builder/README
EOF
exit 0
@ -123,7 +134,7 @@ ask_confirm() {
local use_tmpfs="No"
[[ ${USE_TMPFS} -eq 1 ]] && use_tmpfs="Yes"
printf "About to build application image with the following parameters:\n\n"
printf "About to build RealmFS image with the following parameters:\n\n"
printf "\tBuild Configuration File : ${BUILDFILE}\n"
printf "\tOutput rootfs directory : ${ROOTFS}\n"
printf "\tBuild rootfs on tmpfs : ${use_tmpfs}\n"
@ -141,12 +152,13 @@ try_config() {
printf "${rp}"
}
WORKDIR="$(pwd)/appimg"
WORKDIR="$(pwd)/realmfs"
DO_TAR=0
DO_XZ=0
USE_TMPFS=1
NO_CONFIRM=0
DO_IMG=0
while [[ $# -gt 0 ]]; do
key=${1}
@ -167,6 +179,10 @@ while [[ $# -gt 0 ]]; do
DO_TAR=1 DO_XZ=1
shift
;;
-i)
DO_IMG=1
shift
;;
--no-tmpfs)
USE_TMPFS=0
@ -179,7 +195,7 @@ while [[ $# -gt 0 ]]; do
;;
--new)
cp --verbose ${APPIMG_BUILDER_BASE}/build-template.conf build.conf
cp --verbose ${REALMFS_BUILDER_BASE}/build-template.conf build.conf
exit 0
;;
@ -199,12 +215,12 @@ while [[ $# -gt 0 ]]; do
done
if [ "$EUID" -ne 0 ]; then
echo "The appimg-builder must be run with root privileges."
echo "The realmfs-builder must be run with root privileges."
exit 1
fi
if [[ -z ${BUILDFILE} ]]; then
BUILDFILE=$(try_config "${PWD}/build.conf" || try_config "${APPIMG_BUILDER_BASE}/basic-image.conf") || fatal "Could not find a configuration file to use"
BUILDFILE=$(try_config "${PWD}/build.conf" || try_config "${REALMFS_BUILDER_BASE}/basic-image.conf") || fatal "Could not find a configuration file to use"
fi
ROOTFS=${WORKDIR}/rootfs
@ -213,9 +229,9 @@ CACHE_DIR=${WORKDIR}/var-cache-apt-archives
[[ ${NO_CONFIRM} -ne 1 ]] && ask_confirm
# black magick from stack overflow
exec > >(tee -a $WORKDIR/appimg-build.log) 2>&1
exec > >(tee -a $WORKDIR/realmfs-build.log) 2>&1
info "Starting build of application image from configuration file ${BUILDFILE}"
info "Starting build of RealmFS image from configuration file ${BUILDFILE}"
source ${BUILDFILE}
@ -225,10 +241,13 @@ run_debootstrap
run_chroot_stage
info "rootfs build is completed:"
info " $(du -sh ${ROOTFS})"
if [[ ${DO_TAR} -eq 1 ]]; then
if [[ ${DO_IMG} -eq 1 ]]; then
generate_image
elif [[ ${DO_TAR} -eq 1 ]]; then
generate_tarball
fi

View File

@ -1,12 +1,12 @@
#!/bin/bash
APPIMG_BUILDER_BASE="/usr/share/appimg-builder"
REALMFS_BUILDER_BASE="/usr/share/realmfs-builder"
#
# If you're looking for the base lists of packages that are always installed
# and modules that are always run, they're defined here in common.inc
#
source ${APPIMG_BUILDER_BASE}/common.inc
source ${REALMFS_BUILDER_BASE}/common.inc
if [ "$EUID" -ne 0 ]; then
fatal "The stage-two.sh script is not running as root."
@ -22,7 +22,7 @@ fi
# running module 'utility-library' replaces this function with a more powerful version
module() {
local modpath=${APPIMG_BUILDER_BASE}/appimg-modules/${1}
local modpath=${REALMFS_BUILDER_BASE}/realmfs-modules/${1}
[[ -f ${modpath} ]] || fatal "Could not find module '${1}'"
source ${modpath}
}