[CentOS] CentOS-announce Digest, Vol 134, Issue 9

Sun Apr 17 12:00:02 UTC 2016
centos-announce-request at centos.org <centos-announce-request at centos.org>

Send CentOS-announce mailing list submissions to
	centos-announce at centos.org

To subscribe or unsubscribe via the World Wide Web, visit
or, via email, send a message with subject or body 'help' to
	centos-announce-request at centos.org

You can reach the person managing the list at
	centos-announce-owner at centos.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of CentOS-announce digest..."

Today's Topics:

   1. *correction* Notification of outage : mirrorlist.centos.org
      (ipv4) 2016-04-16 08:58 UTC (Karanbir Singh)


Message: 1
Date: Sat, 16 Apr 2016 23:16:12 +0100
From: Karanbir Singh <kbsingh at centos.org>
To: centos-announce at centos.org
Subject: [CentOS-announce] *correction* Notification of outage :
	mirrorlist.centos.org (ipv4) 2016-04-16 08:58 UTC
Message-ID: <5712B9AC.10600 at centos.org>
Content-Type: text/plain; charset=windows-1252

Hash: SHA1


Apologies for this, the correct date should be : 2016-04-16 Sat. The
timestamps remain accurate.

On 16/04/16 10:09, Karanbir Singh wrote:
> Hi,
> At 05:00 UTC 2016-05-16 an automated component update caused the 
> mirrorlist service for CentOS to go down for all IPv4 based
> services. The IPv6 service for mirrorlist.centos.org was
> unaffected.
> By 08:11 UTC 2016-05-16 I rolled back the impacted components, 
> disabled the update mechanics, and restarted services.
> By 08:15 UTC 2016-05-16 Services were returning to normal.
> By 08:23 UTC 2016-05-16 we had multiple confirmations from around
> the world that services were restored.
> --------------- system wide followup:
> - We will work to extend and add where needed, better testing
> around each component involved in such roles.
> - I will work with Fabian and make sure that all our automated 
> component and system changes that impact a production service are
> only run during regular working hours for the team.
> --------------- root cause:
> all the components involved in the downtime are backed up and we
> will start looking at the root cause of why services went down, for
> now the immediate focus was to restore services which was done by a
> rollback.
> --------------- reporting issues:
> Note that for real time, time sensitive issues always drop into 
> #centos-devel on irc.freenode.net and let us know, along with
> filing a bug report at bugs.centos.org - For any non time critical
> issues, please report them at bugs.centos.org against the
> 'Infrastructure' project, and we will aim to address them as soon
> as possible.

- -- 
Karanbir Singh, Project Lead, The CentOS Project
+44-207-0999389 | http://www.centos.org/ | twitter.com/CentOS
GnuPG Key : http://www.karan.org/publickey.asc
Version: GnuPG v2.0.22 (GNU/Linux)



CentOS-announce mailing list
CentOS-announce at centos.org

End of CentOS-announce Digest, Vol 134, Issue 9