Hi there,
I've started to work on packaging the VirtualBox Guest Additions. Unfortunately, the upstream build process doesn't work on CentOS Linux 6.8 and 7.2.
On 7.2 gcc fails on <linux/compiler.h> from the kernel-devel package (log attached as vbox-ga-c7.txt). I think this might be GCC bug #49718, fixed in January 2014 in the 4.9 branch. [1] With RHEL 7.3 in beta, would a backport of the fix to Red Hat's gcc-4.8 still be possible in a reasonable timeframe? Should I file a bug report against gcc on the CentOS tracker?
On 6.8 the compilation stops with an error later, while trying to build vboxvideo (log attached as vbox-ga-c6.txt). On #vbox-dev I was told that this directory requires at least kernel 3.11 to build, and it shouldn't even try. There seems to be no ./configure option for excluding vboxvideo from the build process.
Their advice is to install the Guest Additions in a VM using their installer on the .iso image they provide. This should leave all the necessary sources and Makefiles for building the kernel modules in /usr/src, which I could use for the RPMs instead of the original source (the source that their installer creates is able to detect what the guest supports, and will only build what's possible). As far as SRPMs go, this might be somewhat unusual... We'd need to do this for every new version, and we have to look into how package upgrades and removals could work.
Regards, Laurențiu
On 13/09/16 16:03, Laurentiu Pancescu wrote:
The advice from #vbox-dev is to install the Guest Additions in a VM using their installer on the .iso image they provide. This should leave all the necessary sources and Makefiles for building the kernel modules in /usr/src, which I could use for the RPMs instead of the original source.
Over the last few days, I installed the Guest Additions in a Vagrant box and took a closer look at the installer. The source for the kernel modules that it copies in /usr/src compiles without any issues on both CentOS 6 and 7 and doesn't even require kBuild like the regular build process.
Besides the kernel modules' source, the Guest Additions .iso delivers a few binaries, precompiled for i386 and x86_64, as well as a quite extensive runtime, implementing memory allocation, pseudo-random numbers, math routines, a crypto library, etc. [1] We are unable to compile the userland on CentOS due to the unfixed bug in GCC 4.8 (which seems to be triggered by our own kernel headers, not the VirtualBox source), as well as having kernels older than 3.11. Vagrant's check for the guest additions still passes if I remove the userland files, but the shared folders stop working - having just the kernel modules is not enough.
Regards, Laurențiu
[1] https://www.virtualbox.org/browser/vbox/trunk/src/VBox/Runtime/common
On Sat, Sep 17, 2016 at 3:01 AM, Laurentiu Pancescu lpancescu@gmail.com wrote:
On 13/09/16 16:03, Laurentiu Pancescu wrote:
The advice from #vbox-dev is to install the Guest Additions in a VM using their installer on the .iso image they provide. This should leave all the necessary sources and Makefiles for building the kernel modules in /usr/src, which I could use for the RPMs instead of the original source.
Over the last few days, I installed the Guest Additions in a Vagrant box and took a closer look at the installer. The source for the kernel modules that it copies in /usr/src compiles without any issues on both CentOS 6 and 7 and doesn't even require kBuild like the regular build process.
Besides the kernel modules' source, the Guest Additions .iso delivers a few binaries, precompiled for i386 and x86_64, as well as a quite extensive runtime, implementing memory allocation, pseudo-random numbers, math routines, a crypto library, etc. [1] We are unable to compile the userland on CentOS due to the unfixed bug in GCC 4.8 (which seems to be triggered by our own kernel headers, not the VirtualBox source), as well as having kernels older than 3.11. Vagrant's check for the guest additions still
Can you use the much more recent gcc in the dev-toolset-4 repositories, which is in turn enabled by the the centos-release-scl and centos-release-scl-rh packages? It would mean using a customized koji or mock setup and activating a BuildRequires
passes if I remove the userland files, but the shared folders stop working - having just the kernel modules is not enough.
Regards, Laurențiu
[1] https://www.virtualbox.org/browser/vbox/trunk/src/VBox/Runtime/common _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On 17/09/16 21:52, Nico Kadel-Garcia wrote:
Can you use the much more recent gcc in the dev-toolset-4 repositories, which is in turn enabled by the the centos-release-scl and centos-release-scl-rh packages? It would mean using a customized koji or mock setup and activating a BuildRequires
No, we have to use the same compiler used for building the kernel. [1] The only way is to get the patch from the gcc 4.9 branch backported (it seems small in the diff, but I don't know how much different were the code bases of 4.8 and 4.9 by that time). Even then, having kernels older than 3.11 is likely to remain a problem, if we insist on choosing this route to the Guest Additions.
I spent almost the entire last week investigating this, reading VirtualBox code and trying different things - probably a few days more in total, since I started. I started reading Packer's intro Thursday evening, and the missing bits about its "virtualbox-iso" builder and the "vagrant" postprocessor the next morning. [2] By the end of the day, I already had an automated, repeatable way of building Vagrant images for CentOS 6 and 7, based on our official kickstarts and our Netinstall ISOs, with the VirtualBox Guest Additions preinstalled and fully working. I'm much more inclined to go this way. I'm not sure if it would be possible to use CBS, but I could use Jenkins to generate the images, by allocating a node to run VirtualBox and Packer natively. Would this be acceptable from others' perspective?
Would the SCL SIG be willing to also provide Packer, besides Vagrant? Right now, I'm downloading the Packer binary directly from upstream; for production purposes, I'd feel more comfortable with getting it from SCL.
Best regards, Laurențiu
[1] https://www.virtualbox.org/manual/ch12.html#ts_linux-kernelmodule-fails-to-l... [2] https://en.wikipedia.org/wiki/Law_of_holes
On Tuesday, 20 September 2016, Laurentiu Pancescu lpancescu@gmail.com wrote:
On 17/09/16 21:52, Nico Kadel-Garcia wrote:
Can you use the much more recent gcc in the dev-toolset-4
repositories, which is in turn enabled by the the centos-release-scl and centos-release-scl-rh packages? It would mean using a customized koji or mock setup and activating a BuildRequires
No, we have to use the same compiler used for building the kernel. [1] The only way is to get the patch from the gcc 4.9 branch backported (it seems small in the diff, but I don't know how much different were the code bases of 4.8 and 4.9 by that time). Even then, having kernels older than 3.11 is likely to remain a problem, if we insist on choosing this route to the Guest Additions.
I spent almost the entire last week investigating this, reading VirtualBox code and trying different things - probably a few days more in total, since I started. I started reading Packer's intro Thursday evening, and the missing bits about its "virtualbox-iso" builder and the "vagrant" postprocessor the next morning. [2] By the end of the day, I already had an automated, repeatable way of building Vagrant images for CentOS 6 and 7, based on our official kickstarts and our Netinstall ISOs, with the VirtualBox Guest Additions preinstalled and fully working. I'm much more inclined to go this way. I'm not sure if it would be possible to use CBS, but I could use Jenkins to generate the images, by allocating a node to run VirtualBox and Packer natively. Would this be acceptable from others' perspective?
Would the SCL SIG be willing to also provide Packer, besides Vagrant? Right now, I'm downloading the Packer binary directly from upstream; for production purposes, I'd feel more comfortable with getting it from SCL.
Best regards, Laurențiu
[1] https://www.virtualbox.org/manual/ch12.html#ts_linux-kernelm odule-fails-to-load [2] https://en.wikipedia.org/wiki/Law_of_holes _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
For what it's worth, we had been using Packer to build CentOS Vagrant boxes from the templates provided at https://github.com/chef/bento until recently, as there wasn't an 'official' CentOS box for the VMware provider. Since I am currently working on using Packer's vmware-vmx builder to repackage your new VMware box to include the VMware Tools (the VMware equivalent to the Virtualbox guest additions), I would be interested to know if you would pursue a similar approach (using Packer) for VMware?
I realise this is a bit off-topic for your current issue with Virtualbox, but it would be great to have the official CentOS Vagrant boxes well supported under both Virtualbox and VMware.
Let me know what I can do to assist in this.
On Mon, Sep 19, 2016 at 7:50 PM, Michael Vermaes mvermaes@gmail.com wrote:
On Tuesday, 20 September 2016, Laurentiu Pancescu lpancescu@gmail.com wrote:
On 17/09/16 21:52, Nico Kadel-Garcia wrote:
Can you use the much more recent gcc in the dev-toolset-4 repositories, which is in turn enabled by the the centos-release-scl and centos-release-scl-rh packages? It would mean using a customized koji or mock setup and activating a BuildRequires
No, we have to use the same compiler used for building the kernel. [1] The only way is to get the patch from the gcc 4.9 branch backported (it seems small in the diff, but I don't know how much different were the code bases of 4.8 and 4.9 by that time). Even then, having kernels older than 3.11 is likely to remain a problem, if we insist on choosing this route to the Guest Additions.
I spent almost the entire last week investigating this, reading VirtualBox code and trying different things - probably a few days more in total, since I started. I started reading Packer's intro Thursday evening, and the missing bits about its "virtualbox-iso" builder and the "vagrant" postprocessor the next morning. [2] By the end of the day, I already had an automated, repeatable way of building Vagrant images for CentOS 6 and 7, based on our official kickstarts and our Netinstall ISOs, with the VirtualBox Guest Additions preinstalled and fully working. I'm much more inclined to go this way. I'm not sure if it would be possible to use CBS, but I could use Jenkins to generate the images, by allocating a node to run VirtualBox and Packer natively. Would this be acceptable from others' perspective?
Would the SCL SIG be willing to also provide Packer, besides Vagrant? Right now, I'm downloading the Packer binary directly from upstream; for production purposes, I'd feel more comfortable with getting it from SCL.
Best regards, Laurențiu
[1] https://www.virtualbox.org/manual/ch12.html#ts_linux-kernelmodule-fails-to-l... [2] https://en.wikipedia.org/wiki/Law_of_holes _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
For what it's worth, we had been using Packer to build CentOS Vagrant boxes from the templates provided at https://github.com/chef/bento until recently, as there wasn't an 'official' CentOS box for the VMware provider. Since I am currently working on using Packer's vmware-vmx builder to repackage your new VMware box to include the VMware Tools (the VMware equivalent to the Virtualbox guest additions), I would be interested to know if you would pursue a similar approach (using Packer) for VMware?
CentOS 7 ships open-vm-tools, so enabling the vmtoolsd unit should be enough. Or am I missing anything?
I realise this is a bit off-topic for your current issue with Virtualbox, but it would be great to have the official CentOS Vagrant boxes well supported under both Virtualbox and VMware.
Let me know what I can do to assist in this.
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On Tuesday, 20 September 2016, François Cami <fcami@fedoraproject.org javascript:_e(%7B%7D,'cvml','fcami@fedoraproject.org');> wrote:
On Mon, Sep 19, 2016 at 7:50 PM, Michael Vermaes mvermaes@gmail.com wrote:
On Tuesday, 20 September 2016, Laurentiu Pancescu lpancescu@gmail.com wrote:
On 17/09/16 21:52, Nico Kadel-Garcia wrote:
Can you use the much more recent gcc in the dev-toolset-4 repositories, which is in turn enabled by the the centos-release-scl and centos-release-scl-rh packages? It would mean using a customized koji or mock setup and activating a BuildRequires
No, we have to use the same compiler used for building the kernel. [1]
The
only way is to get the patch from the gcc 4.9 branch backported (it
seems
small in the diff, but I don't know how much different were the code
bases
of 4.8 and 4.9 by that time). Even then, having kernels older than
3.11 is
likely to remain a problem, if we insist on choosing this route to the
Guest
Additions.
I spent almost the entire last week investigating this, reading
VirtualBox
code and trying different things - probably a few days more in total,
since
I started. I started reading Packer's intro Thursday evening, and the missing bits about its "virtualbox-iso" builder and the "vagrant" postprocessor the next morning. [2] By the end of the day, I already
had an
automated, repeatable way of building Vagrant images for CentOS 6 and 7, based on our official kickstarts and our Netinstall ISOs, with the VirtualBox Guest Additions preinstalled and fully working. I'm much
more
inclined to go this way. I'm not sure if it would be possible to use
CBS,
but I could use Jenkins to generate the images, by allocating a node to
run
VirtualBox and Packer natively. Would this be acceptable from others' perspective?
Would the SCL SIG be willing to also provide Packer, besides Vagrant? Right now, I'm downloading the Packer binary directly from upstream; for production purposes, I'd feel more comfortable with getting it from SCL.
Best regards, Laurențiu
[1] https://www.virtualbox.org/manual/ch12.html#ts_linux-kernelm
odule-fails-to-load
[2] https://en.wikipedia.org/wiki/Law_of_holes _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
For what it's worth, we had been using Packer to build CentOS Vagrant
boxes
from the templates provided at https://github.com/chef/bento until
recently,
as there wasn't an 'official' CentOS box for the VMware provider. Since
I am
currently working on using Packer's vmware-vmx builder to repackage your
new
VMware box to include the VMware Tools (the VMware equivalent to the Virtualbox guest additions), I would be interested to know if you would pursue a similar approach (using Packer) for VMware?
CentOS 7 ships open-vm-tools, so enabling the vmtoolsd unit should be enough. Or am I missing anything?
I realise this is a bit off-topic for your current issue with Virtualbox, but it would be great to have the official CentOS Vagrant boxes well supported under both Virtualbox and VMware.
Let me know what I can do to assist in this.
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
Including open-vm-tools in the box would be a necessary first step, but my experience is that it's still necessary to load the vmhgfs driver as well, in order to get shared folder support working. As discussed at https://kb.vmware.com/kb/2073804:
*- Why does an operating system release not include the vmhgfs driver?*
*The vmhgfs driver has not been contributed upstream. To work around this situation, install VMware Tools bundled with the Workstation or Fusion products, which will install the missing vmhgfsdrivers. The VMware Tools installer will not disturb inbox VMware drivers included in the OS.*
On Mon, Sep 19, 2016 at 7:50 PM, Michael Vermaes mvermaes@gmail.com wrote:
For what it's worth, we had been using Packer to build CentOS Vagrant boxes from the templates provided at https://github.com/chef/bento until recently, as there wasn't an 'official' CentOS box for the VMware provider. Since I am currently working on using Packer's vmware-vmx builder to repackage your new VMware box to include the VMware Tools (the VMware equivalent to the Virtualbox guest additions), I would be interested to know if you would pursue a similar approach (using Packer) for VMware?
That's unlikely. If I understand correctly, Packer always creates a VM using the the targeted virtualization technology, i.e. VMware must be installed on the build machine. I think someone from CentOS already contacted VMware when we thought about also automatically testing the Vagrant images for VMware on cico, but I've not heard anything about a reply yet. Our build system relies on KVM to perform the CentOS installation, and Image Factory to convert the resulting image to different other formats. VirtualBox is a special case: we have difficulties packaging it, but it's free software, licensed under the GPL (Packer uses VNC to connect to the VM and send the necessary keypresses to change the boot parameters, so Oracle's proprietary Extension Pack isn't needed).
As a side note, Vagrant images for proprietary virtualization technologies are always going to be second-class citizens, since we cannot test them automatically, and we cannot use them natively for installation. For VMware it's just a licensing problem, since it is able to run on CentOS, but e.g. Hyper-V or Parallels only run on proprietary operating systems. The best we can do is use Image Factory to convert the KVM images, and hope to find some volunteers to test them before each monthly release.
Best regards, Laurențiu
On Tue, Sep 20, 2016 at 3:37 AM, Laurențiu Păncescu lpancescu@gmail.com wrote:
On Mon, Sep 19, 2016 at 7:50 PM, Michael Vermaes mvermaes@gmail.com wrote:
For what it's worth, we had been using Packer to build CentOS Vagrant boxes from the templates provided at https://github.com/chef/bento until recently, as there wasn't an 'official' CentOS box for the VMware provider. Since I am currently working on using Packer's vmware-vmx builder to repackage your new VMware box to include the VMware Tools (the VMware equivalent to the Virtualbox guest additions), I would be interested to know if you would pursue a similar approach (using Packer) for VMware?
That's unlikely. If I understand correctly, Packer always creates a VM using the the targeted virtualization technology, i.e. VMware must be installed on the build machine. I think someone from CentOS already contacted VMware when we thought about also automatically testing the Vagrant images for VMware on cico, but I've not heard anything about a reply yet. Our build system relies on KVM to perform the CentOS installation, and Image Factory to convert the resulting image to different other formats. VirtualBox is a special case: we have difficulties packaging it, but it's free software, licensed under the GPL (Packer uses VNC to connect to the VM and send the necessary keypresses to change the boot parameters, so Oracle's proprietary Extension Pack isn't needed).
As a side note, Vagrant images for proprietary virtualization technologies are always going to be second-class citizens, since we cannot test them automatically, and we cannot use them natively for installation. For VMware it's just a licensing problem, since it is able to run on CentOS, but e.g. Hyper-V or Parallels only run on proprietary operating systems. The best we can do is use Image Factory to convert the KVM images, and hope to find some volunteers to test them before each monthly release.
Best regards, Laurențiu
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
OK. That's disappointing, but I understand the limitations a bit better now. Thanks for explaining the background. And please let me know if/how I can help with the Fusion monthly testing.
Michael
On 20/09/16 06:59, Michael Vermaes wrote:
OK. That's disappointing, but I understand the limitations a bit better now. Thanks for explaining the background. And please let me know if/how I can help with the Fusion monthly testing.
Sure, that would be very helpful (we had already discussed testing on #centos, but you were gone before I could ask how I could contact you). Would it be okay to email you off-list when we have an image to test?
Laurențiu
On Tue, Sep 20, 2016 at 3:45 PM, Laurentiu Pancescu lpancescu@gmail.com wrote:
On 20/09/16 06:59, Michael Vermaes wrote:
OK. That's disappointing, but I understand the limitations a bit better now. Thanks for explaining the background. And please let me know if/how I can help with the Fusion monthly testing.
Sure, that would be very helpful (we had already discussed testing on #centos, but you were gone before I could ask how I could contact you). Would it be okay to email you off-list when we have an image to test?
Laurențiu
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
Ah, sorry I missed that. Yes, email is fine.
On 09/17/2016 02:52 PM, Nico Kadel-Garcia wrote:
On Sat, Sep 17, 2016 at 3:01 AM, Laurentiu Pancescu lpancescu@gmail.com wrote:
On 13/09/16 16:03, Laurentiu Pancescu wrote:
The advice from #vbox-dev is to install the Guest Additions in a VM using their installer on the .iso image they provide. This should leave all the necessary sources and Makefiles for building the kernel modules in /usr/src, which I could use for the RPMs instead of the original source.
Over the last few days, I installed the Guest Additions in a Vagrant box and took a closer look at the installer. The source for the kernel modules that it copies in /usr/src compiles without any issues on both CentOS 6 and 7 and doesn't even require kBuild like the regular build process.
Besides the kernel modules' source, the Guest Additions .iso delivers a few binaries, precompiled for i386 and x86_64, as well as a quite extensive runtime, implementing memory allocation, pseudo-random numbers, math routines, a crypto library, etc. [1] We are unable to compile the userland on CentOS due to the unfixed bug in GCC 4.8 (which seems to be triggered by our own kernel headers, not the VirtualBox source), as well as having kernels older than 3.11. Vagrant's check for the guest additions still
Can you use the much more recent gcc in the dev-toolset-4 repositories, which is in turn enabled by the the centos-release-scl and centos-release-scl-rh packages? It would mean using a customized koji or mock setup and activating a BuildRequires
That would mean that running it may (probably likely) require newer components that in the standard C7 release.
If it does, then we would need to build with the standard packages as well, since I suspect more people need el7 vagrant boxes that run on/with standard c7 than ones that run on/with scl enabled gcc/glibc, etc.
passes if I remove the userland files, but the shared folders stop working - having just the kernel modules is not enough.
-
François Cami -
Johnny Hughes -
Laurentiu Pancescu -
Michael Vermaes -
Nico Kadel-Garcia