Things required to "support" Pi3 aarch64 that aren't already in place in core CentOS (or at least I haven't managed to find them):

1) Pi3 firmware blobs Trivially downloadable from https://github.com/raspberrypi/firmware/tree/master/boot

2) UEFI bootloader There are two options, u-boot and Tianocore. My current Pi3 aarch64 image works with u-boot that I grabbed from the Fedora 26 image. I'm currently trying to get it working with Tianocore from here: https://github.com/andreiw/RaspberryPiPkg I _almost_ have it working (gets as far as booting grub, but grub then doesn't manage to boot up the kernel, almost certainly a dtb issue somewhere).

3) Kernel I keep my own mainline kernel build for aarch64, loosely based on, IIRC, 4.5.x that shipped with CentOS aarch64, but with some modifications. I have a build that works on both my X-Gene and the Pi3. You can find it here: http://ftp.redsleeve.org/pub/misc/kernel/aarch64/RPMS/ (Note: I only included Pi 3 SoC configuration as of 4.9.73).

So it's not exactly an insurmountable problem, it's just a case off dropping a tarball of 5-6 files onto the /boot/efi FAT partition, having the appropriate kernel installed in the image, and it should "just work". I can have a working image with u-boot EFI as soon as I find half an hour to spare. The one with Tianocore EFI will take a little longer.

On Thu, Jan 4, 2018 at 9:55 PM, Fabian Arrotin arrfab@centos.org wrote:

On 04/01/18 19:26, Christopher Ursich wrote:

...

Hi, all. First-timer here.

I am setting up a new Raspberry Pi 3. When I review the AltArch pages, I see that most of the RPi3 coverage is categorized under Arm32,

including

...

https://wiki.centos.org/SpecialInterestGroup/AltArch/

Arm32/RaspberryPi3

Because we targeted armhfp even for the Pi3 initially, as even the Pi Foundation had no plan to provide/build at the beginning aarch64 kernel/code for the pi3 TBH (my own opinion) it doesn't even really make sense to use aarch64 code on the pi3 itself with such low specs .. only benefit is probably that epel exists for aarch64 vs armhfp and also same tree if you want to deploy to "real" aarch64 nodes in Datacenter ...

Now, I'll let Jim (the aarch64 maintainer) explain his plans for aarch64 tree for pi3, but at this stage of meltdown and spectre, I guess we all have other urgent things to do too :-)

-- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 56BEC54E | twitter: @arrfab

---

Arm-dev mailing list Arm-dev@centos.org https://lists.centos.org/mailman/listinfo/arm-dev

On 01/04/2018 04:30 PM, Gordan Bobic wrote:

Things required to "support" Pi3 aarch64 that aren't already in place in core CentOS (or at least I haven't managed to find them):

1. Pi3 firmware blobs

Trivially downloadable from https://github.com/raspberrypi/firmware/tree/master/boot

I assume we include those in our armhfp kernel so this is not a problem for an official CentOS image.

2. UEFI bootloader

There are two options, u-boot and Tianocore. My current Pi3 aarch64 image works with u-boot that I grabbed from the Fedora 26 image.  I'm currently trying to get it working with Tianocore from here: https://github.com/andreiw/RaspberryPiPkg I _almost_ have it working (gets as far as booting grub, but grub then doesn't manage to boot up the kernel, almost certainly a dtb issue somewhere).

We kind of need to be able to BUILD this, not just take it from something else an use it. At least that is certainly preferable. I suppose we can just include it as a SOURCE for the SRPM, but that is not ideal.

3. Kernel

I keep my own mainline kernel build for aarch64, loosely based on, IIRC, 4.5.x that shipped with CentOS aarch64, but with some modifications. I have a build that works on both my X-Gene and the Pi3. You can find it here: http://ftp.redsleeve.org/pub/misc/kernel/aarch64/RPMS/ (Note: I only included Pi 3 SoC configuration as of 4.9.73).

So it's not exactly an insurmountable problem, it's just a case off dropping a tarball of 5-6 files onto the /boot/efi FAT partition, having the appropriate kernel installed in the image, and it should "just work". I can have a working image with u-boot EFI as soon as I find half an hour to spare. The one with Tianocore EFI will take a little longer.

Is there something about OUR generic CentOS 4.9.x kernel SRPM that does not work if compiled on aarch64:

http://vault.centos.org/altarch/7.4.1708/experimental/Source/i386/Source/SPa...

That SRPM is what we use for i386/x86_64 and the generic armhfp kernel. If we need a new config file, we can likely figure that out.

We would obviously need to add in whatever is necessary for the firmware blobs and the pi uboot config, etc.

I am not opposed to doing this if we can do it right .. which is that it is secure and reproducible to build and install via the RPMs (the kernel), which are produced completely produce automatically from the SRPM .. AND for the image, it is completely buildable via a kickstart file or one of the normal tools and everything does not have to be hand added, etc. If we can make that happen, then I am happy for us to have an aarch64 build.

But, it is odd that the RPI foundation doesn't care enough to make that happen.

We might get some ideas from the SRPMS these guys do:

https://www.linux.com/news/learn/intro-to-linux/first-64-bit-and-enterprise-...

(if we can see them)

Thanks, Johnny Hughes

On Thu, Jan 4, 2018 at 9:55 PM, Fabian Arrotin <arrfab@centos.org mailto:arrfab@centos.org> wrote:

On 04/01/18 19:26, Christopher Ursich wrote:
> Hi, all.  First-timer here.
>
> I am setting up a new Raspberry Pi 3.  When I review the AltArch
pages,
> I see that most of the RPi3 coverage is categorized under Arm32,
including
>
>
  https://wiki.centos.org/SpecialInterestGroup/AltArch/Arm32/RaspberryPi3
<https://wiki.centos.org/SpecialInterestGroup/AltArch/Arm32/RaspberryPi3>

Because we targeted armhfp even for the Pi3 initially, as even the Pi
Foundation had no plan to provide/build at the beginning aarch64
kernel/code for the pi3
TBH (my own opinion) it doesn't even really make sense to use aarch64
code on the pi3 itself with such low specs .. only benefit is probably
that epel exists for aarch64 vs armhfp and also same tree if you want to
deploy to "real" aarch64 nodes in Datacenter ...

Now, I'll let Jim (the aarch64 maintainer) explain his plans for aarch64
tree for pi3, but at this stage of meltdown and spectre, I guess we all
have other urgent things to do too :-)

There is a LICENCE.broadcom file in there with the redistribution permission details, but it looked reasonable to my non-lawyer eyes.

On Mon, Jan 8, 2018 at 6:54 PM, Jim Perrin jperrin@centos.org wrote:

On 01/04/2018 02:30 PM, Gordan Bobic wrote:

...

Things required to "support" Pi3 aarch64 that aren't already in place in core CentOS (or at least I haven't managed to find them):

1. Pi3 firmware blobs

Trivially downloadable from https://github.com/raspberrypi/firmware/tree/master/boot

Keep in mind, we also need to be able to redistribute this if we're building images. It's not just a matter of getting it. Not saying yes or no, just that it's another thing we have to look at as we do this.

-- Jim Perrin The CentOS Project | http://www.centos.org twitter: @BitIntegrity | GPG Key: FA09AD77

On Mon, Jan 8, 2018 at 3:05 PM, Jim Perrin jperrin@centos.org wrote:

On 01/04/2018 10:26 AM, Christopher Ursich wrote:

...

Hi, all. First-timer here.

I am setting up a new Raspberry Pi 3. When I review the AltArch pages, I see that most of the RPi3 coverage is categorized under Arm32,

including

...

https://wiki.centos.org/SpecialInterestGroup/AltArch/

Arm32/RaspberryPi3

...

I'm under the impression that the RPi3 is actually a AArch64 system. I'm not sure whether I should use

http://mirror.centos.org/altarch/7/isos/armhfp/CentOS-

Userland-7-armv7hl-Minimal-1708-RaspberryPi3.img.xz

...

versus

http://mirror.centos.org/altarch/7/isos/aarch64/CentOS-

7-aarch64-rootfs-7.4.1708.tar.xz

...

Can a more-experienced person clarify this for me?

The aarch64 build we produce was aimed at server systems like Cavium's ThunderX initially. There are some limitations with the aarch64 distribution kernel (currently based on RH's kernel-alt 4.11) which make it terrible to run on the rpi.

1. It's set up for 64k pagesize. This, combined with some CMA math in

the kernel means that you'll take a pretty severe memory hit when running the CentOS aarch64 build on the rpi. In some cases this memory hit consumes nearly all of the available ram.

[...]

In short, the rpi currently works better (by default) as a 32bit setup and we want users to have a better experience.

Is there any evidence to indicate that 64KB page size is making the experience better for anyone?

I'm happy to maintain an unofficial Pi3 aarch64 image and a generic mainline LT kernel more suitable for the Pi3 (while also suitable for the usual bigger aarch64 systems) for the foreseeable future, if that helps. I'll be doing it anyway for my own requirements anyway.