On 02/12/17 19:48, Fabian Arrotin wrote:
On 02/12/17 17:28, Fabian Arrotin wrote:
Hi,
Our monitoring platform informed us that the ppc64{le} builders used in our Koji environment are actually unreachable (same for the underlying IBM P8 hypervisor node) After investigation it seems that it's hardware issue
I'll work with the DC people next monday to see how we can try to fix the issue and I'll also investigate some options to have at least one builder per arch asap (so ppc64 and ppc64le)
That means that people building for multiple arches will be impacted as long as those builders will not be back online (or some equivalent node that can take those $arch jobs)
I'll keep the list informed about a status update as soon as possible.
Update : As at first sight, it seems some time will be needed to fix the hardware issue on the current node, I decided to deploy two VMs on a different setup/node and we now have two builders back in action at the CBS level (one for ppc64le and other one for ppc64)
Keep in mind that it can be slower than before though, but at least there is no service disruption for those arches.
As I promised to keep the list informed about that, here are the latest news :
Hardware has been fixed completely last friday, and "stress tested" during the week-end. As it was working fine, I decide to put back the Koji/CBS workload back on the fixed node. That also means that we are now back at full capacity (2 builders per arch, instead of just one, running in/from a different environment/setup) so that should speed up your builds if you build for ppc64/ppc64le at the same time as for other arches.