Hi there,
I've been looking at making it easier for Vagrant users to verify our images when adding them. The Vagrant documentation mentions that the checksum can be added to the box metadata[1], and that this is done automatically if you build the box on Atlas - indeed, the source code shows they are verifying a checksum from the downloaded metadata[2].
Unfortunately, Atlas does not seem to provide such a checksum for any of the boxes I checked - neither those hosted by them, like debian/jessie64, nor external ones like ours' or Fedora's. The Bento boxes seemed to offer a checksum, but that's just their complete JSON metadata somehow ending up in the description field on Atlas (I assume that's an automatic step not doing what the Bento developers intended - I saw no message about verifying the checksum when adding bento/debian-8.7).
We already host the images on cloud.centos.org. We could also generate the needed JSON metadata (we only need one file for all centos/7 images, and one for centos/6), including the SHA256 checksums. We could also create two Apache aliases (e.g. cloud.centos.org/vagrant/7) to make life easier for our users - after an initial "vagrant box add https://cloud.centos.org/vagrant/7", which would prove the checksum automatically, they would also be notified when new images appear and be able to use "vagrant box update centos/7", just like they do now.
This would allow us to even move away from Atlas, if desired. We would finally be able to completely automate our Vagrant releases, instead of manually adding the releases to Atlas every month, and not even having embedded checksums. There was an Atlas CLI that proved not to work as expected: our 1701 and 1702 releases didn't end up on Atlas, we had to intervene. The big question is how we could communicate this to our users, not the technical side: I already tested this with a local webserver, serving the centos/7 JSON downloaded from Atlas, which I edited to add an SHA256 checksum - Vagrant automatically verified the checksum after the download was finished.
Any thoughts?
Laurențiu
[1] https://www.vagrantup.com/docs/boxes/format.html#box-metadata [2] https://github.com/mitchellh/vagrant/blob/master/lib/vagrant/action/builtin/...
On Wed, Mar 22, 2017 at 11:23 PM, Laurentiu Pancescu lpancescu@gmail.com wrote:
Hi there,
I've been looking at making it easier for Vagrant users to verify our images when adding them. The Vagrant documentation mentions that the checksum can be added to the box metadata[1], and that this is done automatically if you build the box on Atlas - indeed, the source code shows they are verifying a checksum from the downloaded metadata[2].
Unfortunately, Atlas does not seem to provide such a checksum for any of the boxes I checked - neither those hosted by them, like debian/jessie64, nor external ones like ours' or Fedora's. The Bento boxes seemed to offer a checksum, but that's just their complete JSON metadata somehow ending up in the description field on Atlas (I assume that's an automatic step not doing what the Bento developers intended - I saw no message about verifying the checksum when adding bento/debian-8.7).
We already host the images on cloud.centos.org. We could also generate the needed JSON metadata (we only need one file for all centos/7 images, and one for centos/6), including the SHA256 checksums. We could also create two Apache aliases (e.g. cloud.centos.org/vagrant/7) to make life easier for our users - after an initial "vagrant box add https://cloud.centos.org/vagrant/7", which would prove the checksum automatically, they would also be notified when new images appear and be able to use "vagrant box update centos/7", just like they do now.
This would allow us to even move away from Atlas, if desired. We would finally be able to completely automate our Vagrant releases, instead of manually adding the releases to Atlas every month, and not even having embedded checksums. There was an Atlas CLI that proved not to work as expected: our 1701 and 1702 releases didn't end up on Atlas, we had to intervene. The big question is how we could communicate this to our users, not the technical side: I already tested this with a local webserver, serving the centos/7 JSON downloaded from Atlas, which I edited to add an SHA256 checksum - Vagrant automatically verified the checksum after the download was finished.
Any thoughts?
Laurențiu
[1] https://www.vagrantup.com/docs/boxes/format.html#box-metadata [2] https://github.com/mitchellh/vagrant/blob/master/lib/vagrant/action/builtin/... _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
Hi Laurențiu
I think there is still some benefit to maintaining a presence on Atlas if possible, as it is where Vagrant users are likely to search for CentOS boxes first. But hosting the metadata on the CentOS infrastructure makes a lot of sense for the reasons you mentioned. I guess you could maybe leave a final release in Atlas pointing people to the new location when it's available.
By the way, from what I can understand from the commit history [1], it looks like the Bento metadata was added for a similar reason, to allow them to consider standing up their own metadata server in place of Atlas.
On 23/03/17 09:32, Michael Vermaes wrote:
I think there is still some benefit to maintaining a presence on Atlas if possible, as it is where Vagrant users are likely to search for CentOS boxes first. But hosting the metadata on the CentOS infrastructure makes a lot of sense for the reasons you mentioned. I guess you could maybe leave a final release in Atlas pointing people to the new location when it's available.
It would be great if Atlas would also allow redirects for metadata, not just the boxes - not sure if that's possible. I'm also not sure how many users search directly on Atlas for official images released by CentOS, instead of Google or our own website. If we decide to move away from Atlas, we should definitely release a last image there, with the description pointing people to the new location (ideally, a web page dedicated to our Vagrant images).
By the way, from what I can understand from the commit history [1], it looks like the Bento metadata was added for a similar reason, to allow them to consider standing up their own metadata server in place of Atlas.
Thanks, I didn't know that. But I realized another benefit to hosting our own metadata, besides checksums: we could also get statistics on how many active users we have, on which host OSes, since Vagrant will download the metadata to check if a new image has been released (at least when a box is booted).
Laurențiu
On Thu, Mar 23, 2017 at 9:53 AM, Laurentiu Pancescu lpancescu@gmail.com wrote:
On 23/03/17 09:32, Michael Vermaes wrote:
I think there is still some benefit to maintaining a presence on Atlas if possible, as it is where Vagrant users are likely to search for CentOS boxes first.
I'm not using cloud.centos.org, but the benefit from atlas is a simplified handing of URLs and versioning of the boxes, so in the Vagrantfile one can have:
machine.vm.box = "centos/7"
Marcin
But hosting the metadata on the CentOS
infrastructure makes a lot of sense for the reasons you mentioned. I guess you could maybe leave a final release in Atlas pointing people to the new location when it's available.
It would be great if Atlas would also allow redirects for metadata, not just the boxes - not sure if that's possible. I'm also not sure how many users search directly on Atlas for official images released by CentOS, instead of Google or our own website. If we decide to move away from Atlas, we should definitely release a last image there, with the description pointing people to the new location (ideally, a web page dedicated to our Vagrant images).
By the way, from what I can understand from the commit history [1], it
looks like the Bento metadata was added for a similar reason, to allow them to consider standing up their own metadata server in place of Atlas.
Thanks, I didn't know that. But I realized another benefit to hosting our own metadata, besides checksums: we could also get statistics on how many active users we have, on which host OSes, since Vagrant will download the metadata to check if a new image has been released (at least when a box is booted).
Laurențiu
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On 23/03/17 10:12, Marcin Dulak wrote:
I'm not using cloud.centos.org, but the benefit from atlas is a simplified handing of URLs and versioning of the boxes, so in the Vagrantfile one can have:
machine.vm.box = "centos/7"
That's right: machine.vm.box = "https://cloud.centos.org/vagrant/7" would be somewhat longer to type.
On Thu, Mar 23, 2017 at 10:44 AM, Laurentiu Pancescu lpancescu@gmail.com wrote:
On 23/03/17 10:12, Marcin Dulak wrote:
I'm not using cloud.centos.org, but the benefit from atlas is a simplified handing of URLs and versioning of the boxes, so in the Vagrantfile one can have:
machine.vm.box = "centos/7"
That's right: machine.vm.box = "https://cloud.centos.org/vagrant/7" would be somewhat longer to type.
can see now that a link to the latest box has been on cloud.centos.org for a long time https://lists.centos.org/pipermail/centos-announce/2015-June/021162.html
Marcin
CentOS-devel mailing list CentOS-devel@centos.org https://lists.centos.org/mailman/listinfo/centos-devel
On 23/03/17 11:19, Marcin Dulak wrote:
can see now that a link to the latest box has been on cloud.centos.org for a long time https://lists.centos.org/pipermail/centos-announce/2015-June/021162.html
This was always the case, our images are actually stored on cloud.centos.org, not on Atlas. Atlas just issues HTTP redirects to the actual files that "vagrant add" requests:
$ curl -LI https://atlas.hashicorp.com/centos/boxes/7/versions/1605.01/providers/libvir... HTTP/1.1 302 Found Cache-Control: no-cache Content-Type: text/html; charset=utf-8 Date: Thu, 23 Mar 2017 10:24:38 GMT Location: http://cloud.centos.org/centos/7/vagrant/x86_64/images/CentOS-7-x86_64-Vagra... Server: nginx + Phusion Passenger 5.0.29 Status: 302 Found Strict-Transport-Security: max-age=31536000; includeSubDomains; preload X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN X-Powered-By: Phusion Passenger 5.0.29 X-Request-Id: 98388880-8b0d-4737-9546-1d01cd47acc5 X-Runtime: 0.020304 X-XSS-Protection: 1; mode=block Connection: keep-alive
HTTP/1.1 200 OK Date: Thu, 23 Mar 2017 10:20:55 GMT Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips Last-Modified: Wed, 08 Jun 2016 15:26:02 GMT ETag: "1d45ad5e-534c5ec58d680" Accept-Ranges: bytes Content-Length: 491105630 Content-Type: application/vnd.previewsystems.box
-
Laurentiu Pancescu -
Marcin Dulak -
Michael Vermaes