[Arm-dev] Stateless CentOS on ARM question

Wed Jun 5 00:16:27 UTC 2019
R C <cjvijf at gmail.com>

Hi there,

I built/d a bunch of RHEL, Centos and other 'clones' clusters. First of 
all, if you need a disk to start a cluster node, well, it's not 
diskless. Typically, you'd boot a node using PXE. (also, local disks in 
a diskless environment always causes weird trouble).

After that there are several routes you can take. You can do something 
straight forward, like use DHCP/tftp or for example use perceus and vnfs 
for creating your boot images (or something similar). Perceus and vnfs 
are older, but very manageable, especially on larger scale clusters 
(dozens/hundreds/thousands) and are still used.

Since you're trying to netboot a RPI, I'd probably keep the initram disk 
very small and actually use something like perceus and vnfs.  (with vnfs 
you can actually build your whole image, deciding what needs to be on 
the ramdisk, and what are links to an nfs mount). Using Perceus you'd 
boot the RPI using a kernel for handling the vnfs image/capsule, and 
after all of that is taken care of, it switches (kexec) to the kernel 
you actually want to run.

The stateless part; maybe, but depending on what sort of cluster you 
want to setup up, you'd start changing "the state" of an image right 
after boot, before you re-connect, or use different vnfs capsules all 
together.

It might be easier to start with dhcp/tftp, to get things going. If what 
you want to accomplish is more elaborate then just bring up a few 
networked diskless nodes, you probably want to look into something like 
perceus or so.

If you want to "run" a complete distro, you'd probably have to since the 
RPI only has 2Gb, and you can't stick a complete filesystem into that.


Ron


On 6/4/19 12:16 PM, Seth Meyer wrote:
> Hello,
>
> A colleague and I are trying to get an ARM server to boot stateless 
> (i.e. diskless).  Has anyone had any success with this?  Our primary 
> source of inspiration is the documentation for the Pegasus cluster:
>
> http://web.mst.edu/~vojtat/pegasus/administration.htm
>
> Basically, the idea is to have the kernel load a custom initramfs that 
> decompresses a file system image onto a tmpfs at boot time (along with 
> some other setup), then execute a `switch_root` to said tmpfs.
>
> We have succeeded in getting an x86_64 machine to boot (mostly) 
> diskless, using the disk only for the boot process by creating a GRUB 
> entry.  However, when we try this with my ARM (ThunderX) server, the 
> kernel panics during the boot process because it cannot find init.  We 
> have also tried passing `init=/init` as a kernel parameter with no 
> luck.   The error message is
>
> Kernel panic - not syncing: No working init found. Try passing init= 
> option to kernel. See Linux Documentation/init.txt for guidance.
>
> We looked at the referenced documentation and it's not a ton of help 
> for my specific case.  The initramfs environment is based on Busybox, 
> which we compiled on a ThunderX machine (the same one we are trying to 
> boot from, in fact).
>
> We have considered compiling my own kernel with the initramfs built 
> in, but we are trying to avoid doing that.  Further, we have tried 
> with the 3.10 and 4.4 kernels on x86, and with 4.5 and 4.14 kernels on 
> aarch64.
>
> Does anyone have any guidance?  Also, please let us know if you need 
> more information.
>
> Best regards,
>
> Marc Patton and Seth Meyer
> ARC-TS - https://arc-ts.umich.edu/
> University of Michigan
>
>
>
>
>
>
> _______________________________________________
> Arm-dev mailing list
> Arm-dev at centos.org
> https://lists.centos.org/mailman/listinfo/arm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/arm-dev/attachments/20190604/f5f6fb26/attachment-0006.html>