On 02/12/17 19:48, Fabian Arrotin wrote:
> On 02/12/17 17:28, Fabian Arrotin wrote:
>> Hi,
>>
>> Our monitoring platform informed us that the ppc64{le} builders used in
>> our Koji environment are actually unreachable (same for the underlying
>> IBM P8 hypervisor node)
>> After investigation it seems that it's hardware issue
>>
>> I'll work with the DC people next monday to see how we can try to fix
>> the issue and I'll also investigate some options to have at least one
>> builder per arch asap (so ppc64 and ppc64le)
>>
>> That means that people building for multiple arches will be impacted as
>> long as those builders will not be back online (or some equivalent node
>> that can take those $arch jobs)
>>
>> I'll keep the list informed about a status update as soon as possible.
>>
>
> Update :
> As at first sight, it seems some time will be needed to fix the hardware
> issue on the current node, I decided to deploy two VMs on a different
> setup/node and we now have two builders back in action at the CBS level
> (one for ppc64le and other one for ppc64)
>
> Keep in mind that it can be slower than before though, but at least
> there is no service disruption for those arches.
>
As I promised to keep the list informed about that, here are the latest
news :
Hardware has been fixed completely last friday, and "stress tested"
during the week-end. As it was working fine, I decide to put back the
Koji/CBS workload back on the fixed node.
That also means that we are now back at full capacity (2 builders per
arch, instead of just one, running in/from a different
environment/setup) so that should speed up your builds if you build for
ppc64/ppc64le at the same time as for other arches.
--
Fabian Arrotin
The CentOS Project | https://www.centos.org
gpg key: 56BEC54E | twitter: @arrfab
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://lists.centos.org/pipermail/centos-devel/attachments/20180129/7b8ac4d4/attachment-0005.sig>