[Ci-users] [Infra] - Pre-announced major outage for several services (please read)

[David Moreau Simard]

I'm not sure I follow.

We definitely don't have to delete and re-deploy the virtual machines.

If a compute node is shut down properly (i.e, shutdown -h now), the virtual
machines will be soft shutdown - the kvm process isn't killed.
When the node boots back up, the VMs will be started again.
We /can/ shutdown the VMs prior to the maintenance for additional safety
but that also means they won't be powered back on automatically when the
compute node is back on.

As far as shutting down the whole cloud, there's nothing very specific
since we don't have anything HA. I would shut down the compute nodes first
and then the controller at the end.

David Moreau Simard
Senior Software Engineer | Openstack RDO

dmsimard = [irc, github, twitter]


On Feb 27, 2017 6:18 AM, "Fabian Arrotin" <arrfab at centos.org> wrote:

On 15/02/17 17:55, Brian Stinson wrote:
>> On 27/01/17 15:03, Fabian Arrotin wrote:
>>> Due to some reorganization at the DC/Cage level, we'll have to
>>> shutdown/move/reconfigure a big part of our hosted infra for the
>>> following services :
>>> - cbs.centos.org (Koji)
>>> - accounts.centos.org (auth backend)
>>> - ci.centos.org (jenkins-driven CI environment)
>>>
>>> We're working on a plan to minimize the downtime/reconfiguration part,
>>> but at first sight, due to the hardware move of the racks/recabling
>>> parts/etc, the announced downtime will be probably ~48h.
>>>
>>> What does that mean ? That during this window, nobody will be able to
>>> build/tests packages, nor be able to triggers automatically CI jobs
>>> (important).
>>>
>>> We don't have the final dates confirmed (yet) but it seems that the
>>> migration will be scheduled for March 14th.
>>>
>>> We'll keep you informed when we'll have confirmation about those dates.
>>>
>>> Feel free to answer in this thread, or find us in #centos-devel on
freenode.
>>>
>
>> Just as a (periodic) reminder, and also to confirm now the dates :
>> starting on March 14th
>
>
>> --
>> Fabian Arrotin
>> The CentOS Project | http://www.centos.org
>> gpg key: 56BEC54E | twitter: @arrfab
>
>
> Cross-posting this message from centos-devel. Notice that this outage
> will affect the entire CI Infrastructure on 14-March.
>
> Cheers!
>
> --Brian

@Brian, @David : are you ready for the openstack shutdown/power on
operation ?
What about Virtual Machines running on that openstack ? KB just told me
that openstack can't do basic things like "power off / power on"
previously deployed VMs, and that those have to be deleted and
redeployed. If so, are you ready for those slaves agents running on such
VMs ?
What's the procedure to stop/power off all the openstack nodes
(controller/nodes)

Cheers,

--
Fabian Arrotin
The CentOS Project | http://www.centos.org
gpg key: 56BEC54E | twitter: @arrfab


_______________________________________________
Ci-users mailing list
Ci-users at centos.org
https://lists.centos.org/mailman/listinfo/ci-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/ci-users/attachments/20170227/fd77f063/attachment.html>