-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 11/10/16 08:46, Fabian Arrotin wrote:
As announced some time ago ( https://lists.centos.org/pipermail/centos-announce/2016-September/0220
65.html
) , there was a big hardware migration/DC relocation happening yesterday
that impacted several of our public services. Here is a small report of that migration, that finally took less time that previously announced.
- 2016-10-10 - 1pm UTC : we started to preconfigure all changes,
and powered down all the services in correct order * 2016-10-10 - 2pm => 6pm UTC : hardware was moved/migrated * 2016-10-10 - 6pm => 8pm UTC : hardware was (in serial order) plugged back in and ipmi/oob interfaces reconfigured, and network started to be validated * 2016-10-10 - 8:30pm UTC : https://accounts.centos.org was back online and operational * 2016-10-10 - 9pm UTC : https://cbs.centos.org was back online but builders were still disabled for validation * 2016-10-10 - 9:45pm UTC : https://cbs.centos.org : builders were enabled and working again * 2016-10-11 - 3:30am UTC : https://ci.centos.org now fully operational (after some network reconfiguration needed post-migration, and validations
Thanks a lot to Brian Stinson (on-site) and DC people who helped for that migration (Matt, Jon, Nathan, Nina, Phil)
Should you encounter an issue , feel free to discuss with us in #centos-devel on irc.freenode.net
PS : sorry for the previous and incomplete mail : sent by accident before finished and probably due to lack of coffee and short night ...
Well done Fabian, Brian and everyone involved in this process, its been a long way getting here, but its fantastic to see the move executed so well.
Moving hundreds of machines across DC's is never fun, and to have done this, brought all the equipment back up on an entirely new network, in under 15 hrs, is commendable.
regards,
- -- Karanbir Singh, Project Lead, The CentOS Project +44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS GnuPG Key : http://www.karan.org/publickey.asc