Hello Guys,
I have been doing centos package compiling on arm hardware for a while now , and it has been going OK . One drawback is that some bigger packages would run for 10hrs to compile . I would like to speed up that process , and compile on a much more powerful machine . I tried cross-compiling but ended up with issues when the arm package got contaminated with i386 libraries no matter what I tired .
I also tried qemu VM running on my core2duo 4 core desktop machine . This worked OK as packages were perfect , but it was slower to compile than using arm hardware . Same package that took 10 hrs on arm hw , compiled for 3 and a half days on the desktop .
My question is if I used a chroot approach to run the emulation will this be quicker than a qemu VM . If so can anyone please share their instructions on how to configure a chroot to run a centos 7 arm image on a x86 desktop.
Best Regards
Milorad
On 23 December 2016 at 02:35, mo.ucina mo.ucina@gmail.com wrote:
Hello Guys,
I have been doing centos package compiling on arm hardware for a while now , and it has been going OK . One drawback is that some bigger packages would run for 10hrs to compile . I would like to speed up that process , and compile on a much more powerful machine . I tried cross-compiling but ended up with issues when the arm package got contaminated with i386 libraries no matter what I tired .
I also tried qemu VM running on my core2duo 4 core desktop machine . This worked OK as packages were perfect , but it was slower to compile than using arm hardware . Same package that took 10 hrs on arm hw , compiled for 3 and a half days on the desktop .
My question is if I used a chroot approach to run the emulation will this be quicker than a qemu VM . If so can anyone please share their instructions on how to configure a chroot to run a centos 7 arm image on a x86 desktop.
I am probably misunderstanding what you are wanting.. but my initial thought is that you can't.
The ARM uses a completely different computer architecture to x86_64. This means that you can't just chroot into an arm environment because the underlying processor (x86_64) isn't changing. It is still going to try and interpret machine code 32 to be one thing even if on an ARM that meant something completely different. [Also the fact that even if machine code 32 was the same the ARM has different number of registers and other internals than an x86]
You are running into the same problems we have had with ARM development over the last N years. The only solutions are
a) Patience with the slow hardware you have. b) Get more hardware but still have patience c) Get large scale server hardware and have patience.
The emulation problem you are running into is pretty much standard because of the translation of a RISC architecture on a CISC architecture. The most off the shelf ARM systems are equivalent to a P400 MHZ from the early 2000's in some ways but also faster in others. [Trying to use a P400 to compile say texlive is a multi day event while on an arm system it might be only 10-20 hours.] You have a slightly better emulation result if you have an 64 bit arm 'emulate' a 32 bit arm but that is hardware dependent and still requires patience.
Best Regards
Milorad
Arm-dev mailing list Arm-dev@centos.org https://lists.centos.org/mailman/listinfo/arm-dev
Userspace emulation won't be meaningfully faster. There is no real alternative to getting a faster ARM machine.
On the cheaper end, an decently specced 8 core (4+4) can be had for under $200. Samsung Chromebook 2 is more expensive but I find it more convenient.
On the higher end, a Gigabyte MP30-AR0/AR1 is a monster and it takes standard DDR3 DIMMs, but it is quite expensive. I got one specifically for compiling packages. If you decide to go down that route, if you rebuild the kernel with the more reasonable 4KB page size, you can run a 32-bit chroot or docker container on an aarch64 host (that is the setup I use). I filled mine up with 128GB of RAM, so even bloadware like LibreOffice and web browsers can compile entirely on tmpfs, which helps significantly.
On Fri, Dec 23, 2016 at 6:18 PM, Stephen John Smoogen smooge@gmail.com wrote:
On 23 December 2016 at 02:35, mo.ucina mo.ucina@gmail.com wrote:
Hello Guys,
I have been doing centos package compiling on arm hardware for a while
now ,
and it has been going OK . One drawback is that some bigger packages
would
run for 10hrs to compile . I would like to speed up that process , and compile on a much more powerful machine . I tried cross-compiling but
ended
up with issues when the arm package got contaminated with i386 libraries
no
matter what I tired .
I also tried qemu VM running on my core2duo 4 core desktop machine . This worked OK as packages were perfect , but it was slower to compile than
using
arm hardware . Same package that took 10 hrs on arm hw , compiled for 3
and
a half days on the desktop .
My question is if I used a chroot approach to run the emulation will
this be
quicker than a qemu VM . If so can anyone please share their
instructions on
how to configure a chroot to run a centos 7 arm image on a x86 desktop.
I am probably misunderstanding what you are wanting.. but my initial thought is that you can't.
The ARM uses a completely different computer architecture to x86_64. This means that you can't just chroot into an arm environment because the underlying processor (x86_64) isn't changing. It is still going to try and interpret machine code 32 to be one thing even if on an ARM that meant something completely different. [Also the fact that even if machine code 32 was the same the ARM has different number of registers and other internals than an x86]
You are running into the same problems we have had with ARM development over the last N years. The only solutions are
a) Patience with the slow hardware you have. b) Get more hardware but still have patience c) Get large scale server hardware and have patience.
The emulation problem you are running into is pretty much standard because of the translation of a RISC architecture on a CISC architecture. The most off the shelf ARM systems are equivalent to a P400 MHZ from the early 2000's in some ways but also faster in others. [Trying to use a P400 to compile say texlive is a multi day event while on an arm system it might be only 10-20 hours.] You have a slightly better emulation result if you have an 64 bit arm 'emulate' a 32 bit arm but that is hardware dependent and still requires patience.
Best Regards
Milorad
Arm-dev mailing list Arm-dev@centos.org https://lists.centos.org/mailman/listinfo/arm-dev
-- Stephen J Smoogen. _______________________________________________ Arm-dev mailing list Arm-dev@centos.org https://lists.centos.org/mailman/listinfo/arm-dev
Am 23.12.2016 um 19:38 schrieb Gordan Bobic:
On the higher end, a Gigabyte MP30-AR0/AR1 is a monster and it takes standard DDR3 DIMMs, but it is quite expensive. I got one specifically for compiling packages. If you decide to go down that route, if you rebuild the kernel with the more reasonable 4KB page size, you can run a 32-bit chroot or docker container on an aarch64 host (that is the setup I use). I filled mine up with 128GB of RAM, so even bloadware like LibreOffice and web browsers can compile entirely on tmpfs, which helps significantly.
Instead of buying you can rent such a machine at packet.net for $0.5 per hour. https://www.packet.net/bare-metal/servers/type-2a/
Cheers Uli
W dniu 23.12.2016 o 19:38, Gordan Bobic pisze:
On the higher end, a Gigabyte MP30-AR0/AR1 is a monster and it takes standard DDR3 DIMMs, but it is quite expensive. I got one specifically for compiling packages.
If you decide to go down that route, if you rebuild the kernel with the more reasonable 4KB page size, you can run a 32-bit chroot or docker container on an aarch64 host (that is the setup I use).
Or create 32-bit arm VM and use it for anything.
A VM comes with a very substantial performance hit, especially on heavily concurrent loads, such as compiling (make -j). A container comes with almost no performance overhead.
On 24 Dec 2016 11:49, "Marcin Juszkiewicz" marcin.juszkiewicz@linaro.org wrote:
W dniu 23.12.2016 o 19:38, Gordan Bobic pisze:
On the higher end, a Gigabyte MP30-AR0/AR1 is a monster and it takes standard DDR3 DIMMs, but it is quite expensive. I got one specifically for compiling packages.
If you decide to go down that route, if you rebuild the kernel with the more reasonable 4KB page size, you can run a 32-bit chroot or docker container on an aarch64 host (that is the setup I use).
Or create 32-bit arm VM and use it for anything. _______________________________________________ Arm-dev mailing list Arm-dev@centos.org https://lists.centos.org/mailman/listinfo/arm-dev
Or: "Speed is just a question of money. How fast do you want to go?"
On 24 Dec 2016 22:54, "mo.ucina" mo.ucina@gmail.com wrote:
Thanks Guys,
Appreciate your input . I guess the moral of the story is "Patience is a virtue" .
Regards
/Milorad
On 24/12/16 22:54, Gordan Bobic wrote:
A VM comes with a very substantial performance hit, especially on heavily concurrent loads, such as compiling (make -j). A container comes with almost no performance overhead.
On 24 Dec 2016 11:49, "Marcin Juszkiewicz" marcin.juszkiewicz@linaro.org wrote:
W dniu 23.12.2016 o 19:38, Gordan Bobic pisze:
On the higher end, a Gigabyte MP30-AR0/AR1 is a monster and it takes standard DDR3 DIMMs, but it is quite expensive. I got one specifically for compiling packages.
If you decide to go down that route, if you rebuild the kernel with the more reasonable 4KB page size, you can run a 32-bit chroot or docker container on an aarch64 host (that is the setup I use).
Or create 32-bit arm VM and use it for anything. _______________________________________________ Arm-dev mailing list Arm-dev@centos.org https://lists.centos.org/mailman/listinfo/arm-dev
Arm-dev mailing listArm-dev@centos.orghttps://lists.centos.org/mailman/listinfo/arm-dev
Arm-dev mailing list Arm-dev@centos.org https://lists.centos.org/mailman/listinfo/arm-dev
On 12/23/2016 02:35 AM, mo.ucina wrote:
Hello Guys,
I have been doing centos package compiling on arm hardware for a while now , and it has been going OK . One drawback is that some bigger packages would run for 10hrs to compile . I would like to speed up that process , and compile on a much more powerful machine . I tried cross-compiling but ended up with issues when the arm package got contaminated with i386 libraries no matter what I tired .
The most elegant solution I've seen to this problem is multiarch:
https://wiki.debian.org/Multiarch/LibraryPathOverview
I've also seen cross-compilation toolchain wrappers that seem to make things work.
But .rpm distributions don't seem to be adopting multiarch and even it doesn't make the stubborn packages like Perl any easier to cross-compile.
I also tried qemu VM running on my core2duo 4 core desktop machine . This worked OK as packages were perfect , but it was slower to compile than using arm hardware . Same package that took 10 hrs on arm hw , compiled for 3 and a half days on the desktop .
There may be performance tuning and enhancements such as Multi-Threaded TCG that would improve the speed of this approach.
My question is if I used a chroot approach to run the emulation will this be quicker than a qemu VM . If so can anyone please share their instructions on how to configure a chroot to run a centos 7 arm image on a x86 desktop.
I haven't done exactly what you're asking before, but I've read of others doing similar things:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
https://rwmj.wordpress.com/2013/12/22/how-to-run-aarch64-binaries-on-an-x86-...
http://blog.hansenpartnership.com/constructing-architecture-emulation-contai... http://blog.hansenpartnership.com/unprivileged-build-containers/ http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=94... http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=4a...
https://wiki.debian.org/QemuUserEmulation
Cheers, Cov