[Arm-dev] Chrooting/Docker to armv7hl and armv5tel on aarch64

[Gordan Bobic]

On Mon, 28 Jan 2019, 11:15 Fabian Arrotin <arrfab at centos.org wrote:

> On 27/01/2019 13:18, Gordan Bobic wrote:
> > Figured it out. Two kernel configuration options were in play:
> >
> > CONFIG_COMPAT=y
> > CONFIG_LSM_MMAP_MIN_ADDR=32768
> >
> > Above is what is required to run armv5tel and armv7hl chroots on aarch64.
> > Default for the latter option is 65535, which prevents 32-bit binaries
> > from running. Up and running with 4.9.153 now.
>
> <snip>
>
> For the centos armhfp/armv7hl builds, instead of using chroot'ed env
> directly on the aarch64 nodes, we just converted those into hypervisors
> and run armhfp VMs on top.
>

Full fat virtualization tax at ~25%+ at full saturation load is too rich
for me.

And besides, isn't containerization/docker/kubernettes all the
drvops-as-hell buzzword rage these days? ;-)

But, as Stephen also pointed out, it unfortunately only works on some
> aarch64 cpus and not all of them : I realized that when I wanted to use
> a thunderX node as replacement for a mustang board and couldn't get any
> armhfp VM to start : then, after some discussion with ARM specialists,
> they confirm that, per hardware design, the thunderx (and also
> thunderx2) nodes can't run 32 bits at all, so no way to get armhfp VMs
> running on that anymore.
>

It's always important to pick the right set of tools for the job. :-)


> That means that so far, the only boards on which we can still use the
> "armhfp VMs on top of aarch64 nodes" build process is on mustang or
> merlin (so basically X-Gene and X-Gene 2)
>

Yup, that's what I have.


> I've heard also that the new Ampere eMAG should be able to also do that,
> but unfortunately, haven't verified (no access to such hardware)


My experience is that despite all the song and dance that makes the news
when a manufacturer announces a big ARM server, they are either unavailable
or ludicrously expensive compared to x86. Gigabyte MP30 AR0/AR1 is to date
the only reasonable option I found for anything much bigger than the
various Pi and clones (and I like my 128GB of RAM for large scale compile
jobs).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/arm-dev/attachments/20190128/ad46e291/attachment-0001.html>