Re: [Ci-users] [infra outage] : nfs storage server hosting PVs for openshift

On 02/10/2020 13:11, Fabian Arrotin wrote:

we had a kernel panic on the storage box used as nfs server for openshift (both okd and ocp) and machine doesn't come back online due to md device refusing to start.

machine is now in single-mode to analyze the situation and try to fix it.

We'll send more details and progress when possible

Machine is now back online, after I had to check the md device and reassemble it "by hand". From that point, I mounted one-by-one the LVs from the VG on top of the md device, and ansible reconfigured the exports. NFS node is now available, but don't know how that impacts the pods in openshift and if they have to be rescheduled or if they can reconnect the PersistenVolumes automatically