Hi everyone,
I'd like to revisit the thread about how the CentOS 7 AMIs are created ( https://lists.centos.org/pipermail/centos-devel/2015-July/013652.html) and see if the process can be published in the https://github.com/CentOS/sig-cloud-instance-build repository or another relevant location.
With CentOS 7 AMIs only being available in the Marketplace, all resulting EC2 instances have the Marketplace codes attached to the EBS volumes. A significant restriction of this is that a resulting image cannot be the non-primary volume of an instance unless it is powered down. This presents itself to be a problem in at least the following scenarios:
- Unable to attach a CentOS 7 boot volume to another instance for repair without either creating a temporary instance or shutting down an existing one. For example, if you messed up the /etc/sudoers file and logged out, and wanted to repair, you would not be able to repair by mounting to another instance and editing the file without incurring additional (albeit small) cost, or having an existing instance be temporarily unavailable. - The "amazon-chroot" Packer Builder ( https://www.packer.io/docs/builders/amazon-chroot.html) does not work because it starts by mounting a copy of the snapshot tied to the AMI as part of a scripted operation and therefore cannot power off to do so
Custom AMIs, snapshots, copied EBS volumes, etc, all have the marketplace codes copied to them and inherit the restrictions. If an org was to use these features for automating environments and was disconnected from the original Marketplace agreement, they may be unaware of this limitation.
I would also appreciate being able to have the additional transparency of seeing how an AWS AMI is created as the docker/openstack/etc images from the repository referenced above. This would be useful in environments with regulatory compliance concerns, such as AWS GovCloud, HIPAA, FedRAMP, etc.
I understand the benefit that Marketplace registrations allow for the ability to notify users of any changes, and I am not necessarily advocating for switching away from the Marketplace as the primary AMI location. I would like to be provided the opportunity to build a private AMI in the exact same procedure as the official image so as to avert the restrictions provided by the Marketplace.
*[Note: I previously posted this question to centos-virt https://lists.centos.org/pipermail/centos-virt/2016-February/004907.html, but I did not receive any feedback and I have since noticed that **https://wiki.centos.org/SpecialInterestGroup/Cloud https://wiki.centos.org/SpecialInterestGroup/Cloud refers to the CentOS-Devel list for discussion. My apologies to subscribers of both lists.**]*
Thank you, Alan
As a long-time user, I second Alan's concerns. Reproducibility of cloud images doesn't seem to be a priority for the major distros, and at best, the marketplace throws up a roadblock requiring acceptance through the AWS dashboard when setting up a new account. On Wed, Feb 10, 2016 at 10:53 AM Alan Ivey alanivey@gmail.com wrote:
Hi everyone,
I'd like to revisit the thread about how the CentOS 7 AMIs are created ( https://lists.centos.org/pipermail/centos-devel/2015-July/013652.html) and see if the process can be published in the https://github.com/CentOS/sig-cloud-instance-build repository or another relevant location.
With CentOS 7 AMIs only being available in the Marketplace, all resulting EC2 instances have the Marketplace codes attached to the EBS volumes. A significant restriction of this is that a resulting image cannot be the non-primary volume of an instance unless it is powered down. This presents itself to be a problem in at least the following scenarios:
- Unable to attach a CentOS 7 boot volume to another instance for
repair without either creating a temporary instance or shutting down an existing one. For example, if you messed up the /etc/sudoers file and logged out, and wanted to repair, you would not be able to repair by mounting to another instance and editing the file without incurring additional (albeit small) cost, or having an existing instance be temporarily unavailable.
- The "amazon-chroot" Packer Builder (
https://www.packer.io/docs/builders/amazon-chroot.html) does not work because it starts by mounting a copy of the snapshot tied to the AMI as part of a scripted operation and therefore cannot power off to do so
Custom AMIs, snapshots, copied EBS volumes, etc, all have the marketplace codes copied to them and inherit the restrictions. If an org was to use these features for automating environments and was disconnected from the original Marketplace agreement, they may be unaware of this limitation.
I would also appreciate being able to have the additional transparency of seeing how an AWS AMI is created as the docker/openstack/etc images from the repository referenced above. This would be useful in environments with regulatory compliance concerns, such as AWS GovCloud, HIPAA, FedRAMP, etc.
I understand the benefit that Marketplace registrations allow for the ability to notify users of any changes, and I am not necessarily advocating for switching away from the Marketplace as the primary AMI location. I would like to be provided the opportunity to build a private AMI in the exact same procedure as the official image so as to avert the restrictions provided by the Marketplace.
*[Note: I previously posted this question to centos-virt https://lists.centos.org/pipermail/centos-virt/2016-February/004907.html, but I did not receive any feedback and I have since noticed that **https://wiki.centos.org/SpecialInterestGroup/Cloud https://wiki.centos.org/SpecialInterestGroup/Cloud refers to the CentOS-Devel list for discussion. My apologies to subscribers of both lists.**]*
Thank you, Alan _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On 11/02/16 01:59, Jeremy Voorhis wrote:
As a long-time user, I second Alan's concerns. Reproducibility of cloud images doesn't seem to be a priority for the major distros, and at best, the marketplace throws up a roadblock requiring acceptance through the AWS dashboard when setting up a new account.
Couple of things here..
the CentOS AMI's are just the GenericCloud image imported into a block device and registered as an AMI. there is no magic beyond that.
Secondly, the AMIs are hosted in a vendor environment, we only started publishing there once the Amazon folks were willing to reach out and help endorse our existence - and the mechanics they recommend ( and well, highly encourage ) from our side is participation in the Market Place. I've been repeatedly told by them that the best user experience in their infra is via the market place.
In the mean time I've been experimenting with generic AMIs, cross copied over to othr regions and available in the public catalogue for the CentOS Atomic Host, and have noticed far more people asking for it in the Marketplace than folks asking for the CentOS Cloud image outside the marketplace. Admittedly, this is no metric and most conversations are face2face or over private email(ugh!), but its a thing.
Let me circle back and in the next monthly build we do, I can also publish some AMIs outside of the marketplace and see what we get from there.
In the mean time, if anyone can help open channels with Digital Ocean and Linode, so we can help them run updated images, in line with the rest of the vendors - would be very appreciated.
regards,
Thank you for your reply. I was able to import a GenericCloud raw image into EC2 and run successfully. This satisfies my concern of having Marketplace codes attached to EBS volumes.
For the compliance portion of my question, on how the GenericCloud images were created; I see that this point is being discussed in https://github.com/CentOS/sig-cloud-instance-build/issues/11. I can continue this portion of the conversation in GitHub.
Alan
On Thu, Feb 11, 2016 at 5:35 AM, Karanbir Singh mail-lists@karan.org wrote:
On 11/02/16 01:59, Jeremy Voorhis wrote:
As a long-time user, I second Alan's concerns. Reproducibility of cloud images doesn't seem to be a priority for the major distros, and at best, the marketplace throws up a roadblock requiring acceptance through the AWS dashboard when setting up a new account.
Couple of things here..
the CentOS AMI's are just the GenericCloud image imported into a block device and registered as an AMI. there is no magic beyond that.
Secondly, the AMIs are hosted in a vendor environment, we only started publishing there once the Amazon folks were willing to reach out and help endorse our existence - and the mechanics they recommend ( and well, highly encourage ) from our side is participation in the Market Place. I've been repeatedly told by them that the best user experience in their infra is via the market place.
In the mean time I've been experimenting with generic AMIs, cross copied over to othr regions and available in the public catalogue for the CentOS Atomic Host, and have noticed far more people asking for it in the Marketplace than folks asking for the CentOS Cloud image outside the marketplace. Admittedly, this is no metric and most conversations are face2face or over private email(ugh!), but its a thing.
Let me circle back and in the next monthly build we do, I can also publish some AMIs outside of the marketplace and see what we get from there.
In the mean time, if anyone can help open channels with Digital Ocean and Linode, so we can help them run updated images, in line with the rest of the vendors - would be very appreciated.
regards,
-- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh GnuPG Key : http://www.karan.org/publickey.asc _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
Just to be clear, is this done by the CentOS team using the ec2-import-volume and ec2-import-image commands?
On Thu, Feb 11, 2016 at 12:46 PM, Alan Ivey alanivey@gmail.com wrote:
Thank you for your reply. I was able to import a GenericCloud raw image into EC2 and run successfully. This satisfies my concern of having Marketplace codes attached to EBS volumes.
For the compliance portion of my question, on how the GenericCloud images were created; I see that this point is being discussed in https://github.com/CentOS/sig-cloud-instance-build/issues/11. I can continue this portion of the conversation in GitHub.
Alan
On Thu, Feb 11, 2016 at 5:35 AM, Karanbir Singh mail-lists@karan.org wrote:
On 11/02/16 01:59, Jeremy Voorhis wrote:
As a long-time user, I second Alan's concerns. Reproducibility of cloud images doesn't seem to be a priority for the major distros, and at best, the marketplace throws up a roadblock requiring acceptance through the AWS dashboard when setting up a new account.
Couple of things here..
the CentOS AMI's are just the GenericCloud image imported into a block device and registered as an AMI. there is no magic beyond that.
Secondly, the AMIs are hosted in a vendor environment, we only started publishing there once the Amazon folks were willing to reach out and help endorse our existence - and the mechanics they recommend ( and well, highly encourage ) from our side is participation in the Market Place. I've been repeatedly told by them that the best user experience in their infra is via the market place.
In the mean time I've been experimenting with generic AMIs, cross copied over to othr regions and available in the public catalogue for the CentOS Atomic Host, and have noticed far more people asking for it in the Marketplace than folks asking for the CentOS Cloud image outside the marketplace. Admittedly, this is no metric and most conversations are face2face or over private email(ugh!), but its a thing.
Let me circle back and in the next monthly build we do, I can also publish some AMIs outside of the marketplace and see what we get from there.
In the mean time, if anyone can help open channels with Digital Ocean and Linode, so we can help them run updated images, in line with the rest of the vendors - would be very appreciated.
regards,
-- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh GnuPG Key : http://www.karan.org/publickey.asc _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
There has been no further input from CentOS on https://github.com/CentOS/sig-cloud-instance-build/issues/11. It would be greatly appreciated if the current GenericCloud kickstart and associated files and the AWS uploading-related files could be dumped into a Gist for quick view.
On Thu, Feb 11, 2016 at 3:46 PM, Alan Ivey alanivey@gmail.com wrote:
Thank you for your reply. I was able to import a GenericCloud raw image into EC2 and run successfully. This satisfies my concern of having Marketplace codes attached to EBS volumes.
For the compliance portion of my question, on how the GenericCloud images were created; I see that this point is being discussed in https://github.com/CentOS/sig-cloud-instance-build/issues/11. I can continue this portion of the conversation in GitHub.
Alan
On Thu, Feb 11, 2016 at 5:35 AM, Karanbir Singh mail-lists@karan.org wrote:
On 11/02/16 01:59, Jeremy Voorhis wrote:
As a long-time user, I second Alan's concerns. Reproducibility of cloud images doesn't seem to be a priority for the major distros, and at best, the marketplace throws up a roadblock requiring acceptance through the AWS dashboard when setting up a new account.
Couple of things here..
the CentOS AMI's are just the GenericCloud image imported into a block device and registered as an AMI. there is no magic beyond that.
Secondly, the AMIs are hosted in a vendor environment, we only started publishing there once the Amazon folks were willing to reach out and help endorse our existence - and the mechanics they recommend ( and well, highly encourage ) from our side is participation in the Market Place. I've been repeatedly told by them that the best user experience in their infra is via the market place.
In the mean time I've been experimenting with generic AMIs, cross copied over to othr regions and available in the public catalogue for the CentOS Atomic Host, and have noticed far more people asking for it in the Marketplace than folks asking for the CentOS Cloud image outside the marketplace. Admittedly, this is no metric and most conversations are face2face or over private email(ugh!), but its a thing.
Let me circle back and in the next monthly build we do, I can also publish some AMIs outside of the marketplace and see what we get from there.
In the mean time, if anyone can help open channels with Digital Ocean and Linode, so we can help them run updated images, in line with the rest of the vendors - would be very appreciated.
regards,
-- Karanbir Singh +44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh GnuPG Key : http://www.karan.org/publickey.asc _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
-
Alan Ivey -
Jeremy Voorhis -
Karanbir Singh