Support variant of bootc install to-filesystem omitting bootupctl, fstab

[scottcwang]

I'm playing with booting fedora-bootc as a live image. The end goal is a diskless setup where PXE downloads GRUB, GRUB downloads the initramfs and kernel, and dracut (livenet module) downloads the root squashfs (remotely built), mounts it as a live image, and lets ostree boot it. I don't need day-two updates. If my understanding is correct, this resembles how Fedora CoreOS runs from RAM.

I put together a working proof of concept (below) where I make a disk image with a GRUB boot partition, ESP, boot partition, and root partition, and run bootc install to-filesystem on the root partition. bootc install to-filesystem installs fedora-bootc to the root partition, writes fstab, and runs bootupctl to install GRUB to the boot partition. However, I end up deleting the generated fstab and disregarding the boot partition, since I really only need the squashfs of the root partition without fstab, and I'll roll my own GRUB config (writing my own /boot/loader/entries/ostree-1.conf).

In other words, this workflow would be much simpler if bootc install to-filesystem had an option to only install the container image to a given root, and omit writing fstab and running bootupctl. Are there any considerations I'm not thinking of that would make booting fedora-bootc as a live image a bad idea?

Proof of concept

Constructing the squashfs:

#!/usr/bin/env bash

mkdir output

TMPDIR=$(mktemp --directory --tmpdir=output)
mkdir ${TMPDIR}/{efi,boot,root}

truncate ${TMPDIR}/disk.raw --size 16GiB
# boot partition needs to be on the same disk image, since otherwise bootupctl will complain
sfdisk ${TMPDIR}/disk.raw << EOF
label: gpt
label-id: $(uuidgen)
start=1M, size=1M, type="BIOS boot", uuid="$(uuidgen)"
size=100M, type=uefi, uuid="$(uuidgen)"
size=8G, type=linux, uuid="$(uuidgen)"
type=linux, uuid="$(uuidgen)"
EOF
DEVICE=$(sudo losetup --nooverlap --show --partscan --find ${TMPDIR}/disk.raw)

sudo mkfs.fat ${DEVICE}p2
sudo mount ${DEVICE}p2 ${TMPDIR}/efi

bootuuid=$(uuidgen)
sudo mkfs.ext4 -O "^orphan_file" -U ${bootuuid} ${DEVICE}p3
sudo mount ${DEVICE}p3 ${TMPDIR}/boot

rootuuid=$(uuidgen)
sudo mkfs.ext4 -O "^orphan_file" -U ${rootuuid} ${DEVICE}p4
sudo mount ${DEVICE}p4 ${TMPDIR}/root

sudo podman build --tag fedora-bootc-live - <<EOF
FROM quay.io/fedora/fedora-bootc:latest
RUN dnf install --assumeyes dracut-live
RUN echo 'add_dracutmodules+=" livenet "' > /usr/lib/dracut/dracut.conf.d/01-livenet.conf
RUN mkdir /tmp/dracut ; dracut --rebuild \$(find /usr/lib/modules -name initramfs.img)
RUN echo "root" | passwd -s root
EOF

# without --skip-fetch-check: ERROR Installing to filesystem: Verifying fetch: Creating importer: failed to invoke method OpenImage: failed to invoke method OpenImage: pinging container registry localhost: Get "https://localhost/v2/": dial tcp [::1]:443: connect: connection refused
sudo podman run \
    --rm \
    --privileged \
    --pull=newer \
    --security-opt label=type:unconfined_t \
    --volume ./${TMPDIR}/root:/var/root \
    --volume ./${TMPDIR}/boot:/var/root/boot \
    --volume ./${TMPDIR}/efi:/var/root/boot/efi \
    --volume /var/lib/containers:/var/lib/containers \
    --pid host \
    localhost/fedora-bootc-live:latest \
        bootc install to-filesystem --generic-image --skip-fetch-check /var/root

sudo mount -o remount,rw ${DEVICE}p4 ${TMPDIR}/root
OSTREE_DEPLOY=$(cd ${TMPDIR}/root ; find ostree/boot.1/default -name 0 -type l | head -1)
sudo rm ${TMPDIR}/root/${OSTREE_DEPLOY}/etc/fstab
KERNEL_DIR=$(find ${TMPDIR}/root/${OSTREE_DEPLOY}/usr/lib/modules -mindepth 1 -maxdepth 2 -type d | head -1)
INITRAMFS_IMG=output/initramfs.img
VMLINUZ=output/vmlinuz
sudo cp ${KERNEL_DIR}/initramfs.img ${INITRAMFS_IMG}
sudo cp ${KERNEL_DIR}/vmlinuz ${VMLINUZ}

sync
sudo umount ${TMPDIR}/boot
sudo umount ${TMPDIR}/efi
sudo umount ${TMPDIR}/root

sudo mkdir ${TMPDIR}/root/LiveOS
sudo dd if=${DEVICE}p4 of=${TMPDIR}/root/LiveOS/rootfs.img status=progress
SQUASHFS_IMG=output/squashfs.img
sudo mksquashfs ${TMPDIR}/root ${SQUASHFS_IMG}

sudo losetup --detach ${DEVICE}

sudo rm -rf ${TMPDIR}

Taking it for a spin. This makes a disk image with just a boot partition, installs GRUB manually, adds the squashfs / initramfs / kernel, and writes a config that references them:

#!/usr/bin/env bash

SQUASHFS_IMG=$1
INITRAMFS_IMG=$2
VMLINUZ=$3
OSTREE_DEPLOY=$4

mkdir output

TMPDIR=$(mktemp --directory --tmpdir=output)
mkdir ${TMPDIR}/{efi,boot}

truncate ${TMPDIR}/disk.raw --size 16GiB
sfdisk ${TMPDIR}/disk.raw << EOF
label: gpt
label-id: $(uuidgen)
start=1M, size=1M, type="BIOS boot", uuid="$(uuidgen)"
size=100M, type=uefi, uuid="$(uuidgen)"
type=linux, uuid="$(uuidgen)"
EOF
DEVICE=$(sudo losetup --nooverlap --show --partscan --find ${TMPDIR}/disk.raw)

sudo mkfs.fat ${DEVICE}p2
sudo mount ${DEVICE}p2 ${TMPDIR}/efi

bootuuid=$(uuidgen)
sudo mkfs.ext4 -O "^orphan_file" -U ${bootuuid} ${DEVICE}p3
sudo mount ${DEVICE}p3 ${TMPDIR}/boot

## --write-uuid is required. --with-static-configs doesn't work
sudo podman run \
    --rm \
    --privileged \
    --pull=newer \
    --security-opt label=type:unconfined_t \
    --device ${DEVICE}:${DEVICE}:rwm \
    --volume ./${TMPDIR}/boot:/var/root/boot \
    --volume ./${TMPDIR}/efi:/var/root/boot/efi \
    --pid host \
    localhost/fedora-bootc-live:latest \
        bootupctl backend install --device ${DEVICE} /var/root --write-uuid

sudo mkdir --parents ${TMPDIR}/boot/loader/entries
sudo tee ${TMPDIR}/boot/loader/entries/ostree-1.conf <<EOF
title Fedora Linux
version 1
options root=live:UUID=$(sudo blkid ${DEVICE}p3 -s UUID -o value) rw rd.live.ram boot=UUID=$(sudo blkid ${DEVICE}p3 -s UUID -o value) rw console=tty0 console=ttyS0 ostree=/${OSTREE_DEPLOY}
linux /vmlinuz
initrd /initramfs.img
EOF
sudo mkdir ${TMPDIR}/boot/LiveOS
sudo cp ${SQUASHFS_IMG} ${TMPDIR}/boot/LiveOS/squashfs.img
sudo cp ${INITRAMFS_IMG} ${TMPDIR}/boot/initramfs.img
sudo cp ${VMLINUZ} ${TMPDIR}/boot/vmlinuz
sync

sudo umount ${TMPDIR}/{boot,efi}
sudo losetup --detach ${DEVICE}

cp ${TMPDIR}/disk.raw .
sudo rm -rf ${TMPDIR}

#### TEST

sudo virt-install --name fedora-bootc --cpu host --vcpus 4 --memory 4096 --disk disk.raw --os-variant fedora-eln --import --boot uefi,menu=on,useserial=on --noautoconsole
sudo virsh console fedora-bootc
sudo virsh destroy fedora-bootc
sudo virsh undefine --nvram fedora-bootc

[scottcwang]

Related, not quite the same:

• Add support for iso osbuild/bootc-image-builder#433
• Support diskless directly with bootc #898

[cgwalters]

Thanks so much for filing this! I definitely want to handle this use case! I think #898 is definitely related in that the target root wouldn't be squashfs, it'd be a container image fetched from a registry. I think that'd make it more firmly in the bootc domain.

The only thing here is that it would move some of the container image fetching code into the initramfs and so size would be a concern, but I think we could make it small enough to be quite viable.

I have no opposition to also supporting the architecture you're talking about but I think #898 would have a lot of compelling advantages, enough that I'd say we should target it first.

[scottcwang]

That sounds very good, thank you!

[scottcwang]

This use case was also discussed here:

#527