[Arm-dev] Gigabyte MP30-AR0

Mon Mar 14 23:39:09 UTC 2016
Gordan Bobic <gordan at redsleeve.org>

On 2016-03-14 22:58, Gordan Bobic wrote:
> On 2016-03-14 22:25, Gordan Bobic wrote:
>> On 2016-03-14 22:14, Michael Howard wrote:
>>> On 14/03/2016 21:47, Gordan Bobic wrote:
>>>> On 2016-03-14 21:41, Michael Howard wrote:
>>>>> On 14/03/2016 17:16, Gordan Bobic wrote:
>>>>>> On 2016-03-14 17:13, Michael Howard wrote:
>>>>>>> On 14/03/2016 16:56, Gordan Bobic wrote:
>>>>>>>> On 2016-03-14 15:00, Gordan Bobic wrote:
>>>>>>>>> On 2016-03-01 22:32, Michael Howard wrote:
>>>>>>>>>> On 01/03/2016 22:26, Richard W.M. Jones wrote:
>>>>>>>>>>> On Mon, Feb 29, 2016 at 03:20:03PM +0000, Michael Howard 
>>>>>>>>>>> wrote:
>>>>>>>>>>>> Just to let you know, I can't get this to work. aarch64 is 
>>>>>>>>>>>> supposed
>>>>>>>>>>>> to be binary compatible, with the correct libraries 
>>>>>>>>>>>> installed, but
>>>>>>>>>>>> I'm thinking the cpu isn't.
>>>>>>>>>>>> 
>>>>>>>>>>>> All I get is 'cannot execute binary file: Exec format 
>>>>>>>>>>>> error',
>>>>>>>>>>>> regardless  of what I try.
>>>>>>>>>>> As I understand it the problem is page size - 64K was chosen 
>>>>>>>>>>> by
>>>>>>>>>>> Red Hat for aarch64, where as 4K is the norm on armv7.
>>>>>>>>>>> 
>>>>>>>>>>> Anyway, you can run a 32 bit VM and it works well -- in fact 
>>>>>>>>>>> a lot
>>>>>>>>>>> faster than regular 32 bit armv7 hardware.
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> Yes, with CONFIG_ARM64_4K_PAGES=y and CONFIG_COMPAT=y, 32 bit 
>>>>>>>>>> binaries run fine.
>>>>>>>>> 
>>>>>>>>> I built a kernel with these options enabled, but chrooting into 
>>>>>>>>> an
>>>>>>>>> armv5tel subtree segfaults immediately. :-(
>>>>>>>>> 
>>>>>>>>> # grep -E "CONFIG_COMPAT=|CONFIG_ARM64_4K_PAGES=" 
>>>>>>>>> /boot/config-4.4.5
>>>>>>>>> CONFIG_ARM64_4K_PAGES=y
>>>>>>>>> CONFIG_COMPAT=y
>>>>>>>>> 
>>>>>>>>> # chroot /orcone/docker/rsel6/
>>>>>>>>> Segmentation fault
>>>>>>>>> 
>>>>>>>>> The chroot is armv5tel soft-float, which I think should work.
>>>>>>>>> Oddly, I see no mention of a segfault in dmesg or in 
>>>>>>>>> /var/log/messages
>>>>>>>>> on the host...
>>>>>>>>> 
>>>>>>>>> # strace chroot /orcone/docker/media/
>>>>>>>>> execve("/sbin/chroot", ["chroot", "/orcone/docker/media/"], [/* 
>>>>>>>>> 18 vars */]) = 0
>>>>>>>>> brk(0)                                  = 0x153eb000
>>>>>>>>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
>>>>>>>>> MAP_PRIVATE|MAP_ANONYMOUS, -1,
>>>>>>>>> 0) = 0x7f98108000
>>>>>>>>> faccessat(AT_FDCWD, "/etc/ld.so.preload", R_OK) = -1 ENOENT (No 
>>>>>>>>> such
>>>>>>>>> file or directory)
>>>>>>>>> openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
>>>>>>>>> fstat(3, {st_mode=S_IFREG|0644, st_size=23876, ...}) = 0
>>>>>>>>> mmap(NULL, 23876, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f98102000
>>>>>>>>> close(3)                                = 0
>>>>>>>>> openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
>>>>>>>>> read(3,
>>>>>>>>> "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0\267\0\1\0\0\0\270\r\2\0\0\0\0\0"..., 
>>>>>>>>> 832) = 832
>>>>>>>>> fstat(3, {st_mode=S_IFREG|0755, st_size=1801536, ...}) = 0
>>>>>>>>> mmap(NULL, 1528796, PROT_READ|PROT_EXEC, 
>>>>>>>>> MAP_PRIVATE|MAP_DENYWRITE, 3,
>>>>>>>>> 0) = 0x7f97f66000
>>>>>>>>> mprotect(0x7f980c2000, 65536, PROT_NONE) = 0
>>>>>>>>> mmap(0x7f980d2000, 24576, PROT_READ|PROT_WRITE,
>>>>>>>>> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15c000) = 
>>>>>>>>> 0x7f980d2000
>>>>>>>>> mmap(0x7f980d8000, 13276, PROT_READ|PROT_WRITE,
>>>>>>>>> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f980d8000
>>>>>>>>> close(3)                                = 0
>>>>>>>>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
>>>>>>>>> MAP_PRIVATE|MAP_ANONYMOUS, -1,
>>>>>>>>> 0) = 0x7f98101000
>>>>>>>>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
>>>>>>>>> MAP_PRIVATE|MAP_ANONYMOUS, -1,
>>>>>>>>> 0) = 0x7f98100000
>>>>>>>>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
>>>>>>>>> MAP_PRIVATE|MAP_ANONYMOUS, -1,
>>>>>>>>> 0) = 0x7f980ff000
>>>>>>>>> mprotect(0x7f980d2000, 16384, PROT_READ) = 0
>>>>>>>>> mprotect(0x41f000, 4096, PROT_READ)     = 0
>>>>>>>>> mprotect(0x7f9810b000, 4096, PROT_READ) = 0
>>>>>>>>> munmap(0x7f98102000, 23876)             = 0
>>>>>>>>> brk(0)                                  = 0x153eb000
>>>>>>>>> brk(0x1540c000)                         = 0x1540c000
>>>>>>>>> brk(0)                                  = 0x1540c000
>>>>>>>>> openat(AT_FDCWD, "/usr/lib/locale/locale-archive", 
>>>>>>>>> O_RDONLY|O_CLOEXEC) = 3
>>>>>>>>> fstat(3, {st_mode=S_IFREG|0644, st_size=109669264, ...}) = 0
>>>>>>>>> mmap(NULL, 109669264, PROT_READ, MAP_PRIVATE, 3, 0) = 
>>>>>>>>> 0x7f916cf000
>>>>>>>>> close(3)                                = 0
>>>>>>>>> chroot("/orcone/docker/media/")         = 0
>>>>>>>>> chdir("/")                              = 0
>>>>>>>>> execve("/bin/bash", ["/bin/bash", "-i"], [/* 18 vars */]) = 
>>>>>>>>> -1053305918634065933
>>>>>>>>> --- SIGSEGV {si_signo=SIGSEGV, si_code=SI_KERNEL, si_addr=0} 
>>>>>>>>> ---
>>>>>>>>> +++ killed by SIGSEGV +++
>>>>>>>>> Segmentation fault
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> What am I doing differently?
>>>>>>>> 
>>>>>>>> Just to make sure we are as much on the same page as possible, 
>>>>>>>> here
>>>>>>>> is the minimal chroot I am trying:
>>>>>>>> http://ftp.redsleeve.org/pub/el6-staging/rootfs/rsel6-minimal.tar.xz
>>>>>>>> 
>>>>>>>> Built from the latest RedSleeve 6 binaries using:
>>>>>>>> yum --installroot=/some/path install @core
>>>>>>>> 
>>>>>>>> Can you extract that into an empty folder and chroot into it
>>>>>>>> from your aarch64 CentOS 7 install? Does it work for you or
>>>>>>>> does it segfault?
>>>>>>>> 
>>>>>>>> If it works for you, any chance you could post your kernel
>>>>>>>> config somewhere? It's the only thing I can think of that
>>>>>>>> could plausibly be causeing the discrepancy (I am on 4.4.5
>>>>>>>> and IIRC you were on 4.5rc).
>>>>>>> 
>>>>>>> Downloading now but it will be a couple of hours before I can 
>>>>>>> check it out.
>>>>>> 
>>>>>> Thanks, most appreciated. You may want to re-download, as I 
>>>>>> literally
>>>>>> just replaced with tar ball with a more compressed version. If 
>>>>>> that happened
>>>>>> during your download, what you get may end up being corrupted 
>>>>>> (check whether
>>>>>> it matches the md5 checksum).
>>>>> 
>>>>> I did re-download, just in case. It all works here as expected.
>>>>> 
>>>>> [root at mp30 ~]# chroot ~/CHROOT2
>>>>> [root at mp30 ~]# echo "nameserver 192.168.1.2" > /etc/resolv.conf
>>>>> [root at mp30 /]# yum search linux
>>>>> base                                                     | 3.8 kB   
>>>>>   00:00
>>>>> updates                                                  | 3.0 kB   
>>>>>   00:00
>>>>> ============================== N/S Matched: linux 
>>>>> ==============================
>>>>> libselinux.armv5tel : SELinux library and simple utilities
>>>>> libselinux-utils.armv5tel : SELinux libselinux utilies
>>>>> python-linux-procfs.noarch : Linux /proc abstraction classes
>>>>> selinux-policy.noarch : SELinux policy configuration
>>>>> selinux-policy-doc.noarch : SELinux policy documentation
>>>>> selinux-policy-minimum.noarch : SELinux minimum base policy
>>>>> selinux-policy-mls.noarch : SELinux mls base policy
>>>>> selinux- SELinux policy compiler
>>>>> epel-release.noarch : Extra Packages for Enterprise Linux 
>>>>> repository
>>>>>                     : configuration
>>>>> filesystem.armv5tel : The basic directory layout for a Linux system
>>>>> iptables.armv5tel : Tools for managing Linux kernel packet 
>>>>> filtering
>>>>>                   : capabilities
>>>>> libsemanage.armv5tel : SELinux binary policy manipulation library
>>>>> libsepol.armv5tel : SELinux binary policy manipulation library
>>>>> man-pages.noarch : Man (manual) pages from the Linux Documentation 
>>>>> Project
>>>>> man-pages-cs.noarch : Czech man pages from the Linux Documentation 
>>>>> Project
>>>>> man-pages-es.noarch : Spanish man pages from the Linux 
>>>>> Documentation Project
>>>>> man-pages-fr.noarch : French version of the Linux man-pages
>>>>> man-pages-it.noarch : Italian man (manual) pages from the Linux 
>>>>> Documentation
>>>>>                     : Project
>>>>> man-pages-pl.noarch : Polish man pages from the Linux Documentation 
>>>>> Project
>>>>> man-pages-ru.noarch : Russian man pages from the Linux 
>>>>> Documentation Project
>>>>> man-pages-uk.noarch : Ukrainian man pages from the Linux 
>>>>> Documentation Project
>>>>> policycoreutils.armv5tel : SELinux policy core utilities
>>>>> redhat-bookmarks.noarch : Red Hat Enterprise Linux bookmarks
>>>>> redhat-indexhtml.noarch : Browser default start page for Red Hat 
>>>>> Enterprise
>>>>>                         : Linux
>>>>> redsleeve-release.armv5tel : Red Sleeve Enterprise Linux release 
>>>>> file
>>>>> rhel-guest-image-6.noarch : Red Hat Enterprise Linux Guest Images
>>>>> util-linux-ng.armv5tel : A collection of basic system utilities
>>>>> 
>>>>>   Name and summary matches only, use "search all" for everything.
>>>>> [root at mp30 /]# exit
>>>>> [root at mp30 ~]#
>>>> 
>>>> That is most puzzling. Any chance you could please post your kernel 
>>>> config file
>>>> (pastebin it or similar)? I'd like to try to build one exactly the 
>>>> same and see
>>>> if I can get it working.
>>> 
>>> No problem;
>>> 
>>> http://pastebin.com/4mrgmNGv
>>> 
>>> If all else fails (don't see why it should) I can put up the binary
>>> and modules so you can triple check. I didn't build an RPM  as I came
>>> across a packaging problem, an aarch64/UTS_MACHINE issue I believe 
>>> and
>>> I had no idea how to solve it nor could I spend any real time
>>> investigating it.
>> 
>> Thanks for this. I'm building a new kernel with your config now. 
>> Hoping to
>> report success in the morning. :)
> 
> 
> I forgot how fast this thing builds the kernel. :)
> I'm happy to say that 4.5.0 release kernel works fine, and I can run 
> chroots!
> 
> Thanks for your help Michael, I really appreciate it.
> 
> Now to see if I can make the same thing work with the LT 4.4.x 
> kernel...

Using that same config with 4.4.5 (make oldconfig, take all defaults) 
also works.
There must have been an extra config option that I missed the first time 
around.

Gordan