[Ci-users] [infra] : Hardware migration status (full/correct version)

On Tue, Oct 11, 2016 at 5:38 AM, Karanbir Singh <kbsingh at centos.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 11/10/16 08:46, Fabian Arrotin wrote:
>> As announced some time ago (
>> https://lists.centos.org/pipermail/centos-announce/2016-September/0220
> 65.html
>>
>>
> ) , there was a big hardware migration/DC relocation happening yesterday
>> that impacted several of our public services. Here is a small
>> report of that migration, that finally took less time that
>> previously announced.
>>
>> * 2016-10-10 - 1pm UTC : we started to preconfigure all changes,
>> and powered down all the services in correct order * 2016-10-10 -
>> 2pm => 6pm UTC : hardware was moved/migrated * 2016-10-10 - 6pm =>
>> 8pm UTC : hardware was (in serial order) plugged back in and
>> ipmi/oob interfaces reconfigured, and network started to be
>> validated * 2016-10-10 - 8:30pm UTC : https://accounts.centos.org
>> was back online and operational * 2016-10-10 - 9pm UTC :
>> https://cbs.centos.org was back online but builders were still
>> disabled for validation * 2016-10-10 - 9:45pm UTC :
>> https://cbs.centos.org : builders were enabled and working again *
>> 2016-10-11 - 3:30am UTC : https://ci.centos.org now fully
>> operational (after some network reconfiguration needed
>> post-migration, and validations
>>
>>
>> Thanks a lot to Brian Stinson (on-site) and DC people who helped
>> for that migration (Matt, Jon, Nathan, Nina, Phil)
>>
>> Should you encounter an issue , feel free to discuss with us in
>> #centos-devel on irc.freenode.net
>>
>> PS : sorry for the previous and incomplete mail : sent by accident
>> before finished and probably due to lack of coffee and short night
>> ...
>>
>
> Well done Fabian, Brian and everyone involved in this process, its
> been a long way getting here, but its fantastic to see the move
> executed so well.
>
> Moving hundreds of machines across DC's is never fun, and to have done
> this, brought all the equipment back up on an entirely new network, in
> under 15 hrs, is commendable.
>
+1 -- I was super pumped to see everything back online first thing
this morning. Good job, all!

> regards,
>
> - --
> Karanbir Singh, Project Lead, The CentOS Project
> +44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS
> GnuPG Key : http://www.karan.org/publickey.asc
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (GNU/Linux)
>
> iQEcBAEBAgAGBQJX/LMJAAoJEI3Oi2Mx7xbtjKYH/38E1cKChoNxa5O/uXgvNSNC
> Zykjw17y/09vzq5JVj+ucO5iVtXeFgsyp2hoOjmcOauubOAW3/wQdzmnEc3EyfBu
> eIwHHQStl00s4cqRa/Krx6ITLPlYDR/Zf4/dKHc7qbhXzyfvOLeuQyGUhRyT5J0u
> JtCm9WVPKaHvgBipGVLu7BJjnQV9FPol9D0qPUEJac2PsTepNvoPDJLBXT2GErKB
> YtqaE8fnQ1PO0XFmcgCaFVoTWV+oom7XRZThoeK7qQ2b2dkPpdr3bKYVKvN5C4rO
> pbmYp/60ITjXd/kkDFT6gJtJHF6aTYz192Ukq16e/Rxg0VKFx1P2PRoPLLTwOjs=
> =9Xof
> -----END PGP SIGNATURE-----
> _______________________________________________
> Ci-users mailing list
> Ci-users at centos.org
> https://lists.centos.org/mailman/listinfo/ci-users