Hi folks,
We now should be back up and running now from a chassis failure. At around 0030 UTC the storage cards in hufty.ci.centos.org got in an inconsistent state, this caused all of the servers to shut down including the node hosting http://ci.centos.org/artifacts
We are working on gathering log data so we can report this issue to the vendor, and will monitor the situation through the weekend.
As always if you have questions or comments, please let us know here.
Cheers! Brian
Hey Brian, at the end of my Atomic Host - CBS job, the images produced are rsynced to artifacts.ci.centos.org -- that part of the script isn't working for me right now -- might be related?
Jason
----- Original Message -----
From: "Brian Stinson" bstinson@redhat.com To: ci-users@centos.org Sent: Friday, September 11, 2015 7:16:40 PM Subject: [Ci-users] Unscheduled outage **hufty.ci.centos.org** 00:30 - 02:00 UTC
Hi folks,
We now should be back up and running now from a chassis failure. At around 0030 UTC the storage cards in hufty.ci.centos.org got in an inconsistent state, this caused all of the servers to shut down including the node hosting http://ci.centos.org/artifacts
We are working on gathering log data so we can report this issue to the vendor, and will monitor the situation through the weekend.
As always if you have questions or comments, please let us know here.
Cheers! Brian _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
On Sep 12 18:11, Jason Brooks wrote:
Hey Brian, at the end of my Atomic Host - CBS job, the images produced are rsynced to artifacts.ci.centos.org -- that part of the script isn't working for me right now -- might be related?
Jason
----- Original Message -----
From: "Brian Stinson" bstinson@redhat.com To: ci-users@centos.org Sent: Friday, September 11, 2015 7:16:40 PM Subject: [Ci-users] Unscheduled outage **hufty.ci.centos.org** 00:30 - 02:00 UTC
Hi folks,
We now should be back up and running now from a chassis failure. At around 0030 UTC the storage cards in hufty.ci.centos.org got in an inconsistent state, this caused all of the servers to shut down including the node hosting http://ci.centos.org/artifacts
We are working on gathering log data so we can report this issue to the vendor, and will monitor the situation through the weekend.
As always if you have questions or comments, please let us know here.
Cheers! Brian _______________________________________________ Ci-users mailing list Ci-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
Quite right. I just now kicked rsyncd on that machine, and service seemed to be restored for me. Can you confirm?
Thanks for letting us know!
--Brian
-
Brian Stinson -
Jason Brooks