[Arm-dev] Stateless CentOS on ARM question

[Jordi Sanfeliu]

Hello,

I'm not sure if what you call 'diskless' or 'stateless' you also mean a 
'non-destructible' filesystem, which was a concept used on early Linux 
routers.

Anyway, this is what I do.

I'm using CentOS 7 on Raspberry Pi 3 to create Linux routers/gateways 
for multiple devices (communicating with radio-frequency) on an IoT 
environment. The Linux system must be capable to survive to power 
outages at any moment without suffering filesystem corruption. Hence the 
concept 'non-destructible' filesystem.

I accomplish this by creating a squashfs of the root filesystem with 
very minimal set of packages. That is, right after the installation of 
CentOS 7 image, I do a lot of 'yum remove' and 'rpm -e' over multiple 
packages that I don't need. Once done, I squashfs the root filesystem 
into a file that occupies no more than 270MB.

With that image I can now create a new microSD with /boot, /data and 
/root partitions, and copy the squahsfs image directly into /root 
partition. I use the /data partition as the non-volatile space to save 
configurations, metric data, notifications, etc. coming from the IoT 
devices. Then I use Busybox to create a small filesystem that will be 
used as the initramfs. You can use this tree [1] as a good starting 
point, or make your own. Don't forget to include the squashfs and 
overlayfs kernel modules.

In that initramfs I create the /init program which can be a simple shell 
script. Such init file will be the responsible to setup the system 
booted on the initramfs to switch to the root filesystem on the squashfs 
image:

- mount /proc and /sys filesystems.
- make /dev with 'mdev -s'.
- install the squashfs and overlayfs kernel modules (which won't be 
installed by default in the current boot with initramfs).
- mount the squashfs (i.e: in /ro).
- create the tmpfs (no less than 300MB) and mount it (i.e: in /rw).
- mount the writable overlayfs combining /ro and /rw.
- prepare the switch with 'mount --move' /rw and /ro into the overlay.
- and finally 'switch_root /root /sbin/init'.

In the /boot (FAT) partition I place the same Raspberry Pi files that 
CentOS 7 have in its own /boot. The only change is the line 'initramfs 
initramfs.gz' appended into the /boot/config.txt file.

Caveats. I'm still using the kernel version 4.9.80-v7.1 because the 
4.14.43-v7.1 version panics randomly during boot. It looks like the 
kernel is unable to detect /dev/mmcblk0 sometimes. I've not tested the 
latest 4.14.111 yet.

1 - <https://github.com/raspberrypi/target_fs>

I hope that helps you.
Regards.



On 6/4/19 8:16 PM, Seth Meyer wrote:
> Hello,
> 
> A colleague and I are trying to get an ARM server to boot stateless 
> (i.e. diskless).  Has anyone had any success with this?  Our primary 
> source of inspiration is the documentation for the Pegasus cluster:
> 
> http://web.mst.edu/~vojtat/pegasus/administration.htm
> 
> Basically, the idea is to have the kernel load a custom initramfs that 
> decompresses a file system image onto a tmpfs at boot time (along with 
> some other setup), then execute a `switch_root` to said tmpfs.
> 
> We have succeeded in getting an x86_64 machine to boot (mostly) 
> diskless, using the disk only for the boot process by creating a GRUB 
> entry.  However, when we try this with my ARM (ThunderX) server, the 
> kernel panics during the boot process because it cannot find init.  We 
> have also tried passing `init=/init` as a kernel parameter with no luck. 
>    The error message is
> 
> Kernel panic - not syncing: No working init found. Try passing init= 
> option to kernel. See Linux Documentation/init.txt for guidance.
> 
> We looked at the referenced documentation and it's not a ton of help for 
> my specific case.  The initramfs environment is based on Busybox, which 
> we compiled on a ThunderX machine (the same one we are trying to boot 
> from, in fact).
> 
> We have considered compiling my own kernel with the initramfs built in, 
> but we are trying to avoid doing that.  Further, we have tried with the 
> 3.10 and 4.4 kernels on x86, and with 4.5 and 4.14 kernels on aarch64.
> 
> Does anyone have any guidance?  Also, please let us know if you need 
> more information.
> 
> Best regards,
> 
> Marc Patton and Seth Meyer
> ARC-TS - https://arc-ts.umich.edu/
> University of Michigan
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Arm-dev mailing list
> Arm-dev at centos.org
> https://lists.centos.org/mailman/listinfo/arm-dev
> 

-- 
Jordi Sanfeliu
FIBRANET Network Services Provider
https://www.fibranet.cat