Hello,
I'm not sure if what you call 'diskless' or 'stateless' you also mean a 'non-destructible' filesystem, which was a concept used on early Linux routers.
Anyway, this is what I do.
I'm using CentOS 7 on Raspberry Pi 3 to create Linux routers/gateways for multiple devices (communicating with radio-frequency) on an IoT environment. The Linux system must be capable to survive to power outages at any moment without suffering filesystem corruption. Hence the concept 'non-destructible' filesystem.
I accomplish this by creating a squashfs of the root filesystem with very minimal set of packages. That is, right after the installation of CentOS 7 image, I do a lot of 'yum remove' and 'rpm -e' over multiple packages that I don't need. Once done, I squashfs the root filesystem into a file that occupies no more than 270MB.
With that image I can now create a new microSD with /boot, /data and /root partitions, and copy the squahsfs image directly into /root partition. I use the /data partition as the non-volatile space to save configurations, metric data, notifications, etc. coming from the IoT devices. Then I use Busybox to create a small filesystem that will be used as the initramfs. You can use this tree [1] as a good starting point, or make your own. Don't forget to include the squashfs and overlayfs kernel modules.
In that initramfs I create the /init program which can be a simple shell script. Such init file will be the responsible to setup the system booted on the initramfs to switch to the root filesystem on the squashfs image:
- mount /proc and /sys filesystems. - make /dev with 'mdev -s'. - install the squashfs and overlayfs kernel modules (which won't be installed by default in the current boot with initramfs). - mount the squashfs (i.e: in /ro). - create the tmpfs (no less than 300MB) and mount it (i.e: in /rw). - mount the writable overlayfs combining /ro and /rw. - prepare the switch with 'mount --move' /rw and /ro into the overlay. - and finally 'switch_root /root /sbin/init'.
In the /boot (FAT) partition I place the same Raspberry Pi files that CentOS 7 have in its own /boot. The only change is the line 'initramfs initramfs.gz' appended into the /boot/config.txt file.
Caveats. I'm still using the kernel version 4.9.80-v7.1 because the 4.14.43-v7.1 version panics randomly during boot. It looks like the kernel is unable to detect /dev/mmcblk0 sometimes. I've not tested the latest 4.14.111 yet.
1 - https://github.com/raspberrypi/target_fs
I hope that helps you. Regards.
On 6/4/19 8:16 PM, Seth Meyer wrote:
Hello,
A colleague and I are trying to get an ARM server to boot stateless (i.e. diskless). Has anyone had any success with this? Our primary source of inspiration is the documentation for the Pegasus cluster:
http://web.mst.edu/~vojtat/pegasus/administration.htm
Basically, the idea is to have the kernel load a custom initramfs that decompresses a file system image onto a tmpfs at boot time (along with some other setup), then execute a `switch_root` to said tmpfs.
We have succeeded in getting an x86_64 machine to boot (mostly) diskless, using the disk only for the boot process by creating a GRUB entry. However, when we try this with my ARM (ThunderX) server, the kernel panics during the boot process because it cannot find init. We have also tried passing `init=/init` as a kernel parameter with no luck. The error message is
Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/init.txt for guidance.
We looked at the referenced documentation and it's not a ton of help for my specific case. The initramfs environment is based on Busybox, which we compiled on a ThunderX machine (the same one we are trying to boot from, in fact).
We have considered compiling my own kernel with the initramfs built in, but we are trying to avoid doing that. Further, we have tried with the 3.10 and 4.4 kernels on x86, and with 4.5 and 4.14 kernels on aarch64.
Does anyone have any guidance? Also, please let us know if you need more information.
Best regards,
Marc Patton and Seth Meyer ARC-TS - https://arc-ts.umich.edu/ University of Michigan
Arm-dev mailing list Arm-dev@centos.org https://lists.centos.org/mailman/listinfo/arm-dev