<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Il giorno gio 7 gen 2021 alle ore 15:58 Jeremey Wise <<a href="mailto:jeremey.wise@gmail.com">jeremey.wise@gmail.com</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>I have a test environment. Three node HCI cluster. CentOS8 build. Gluster as file system with standard cockpit deploy of HCI. </div></div></blockquote><div><br></div><div>Hi, I would recommend to reach <a href="mailto:users@ovirt.org">users@ovirt.org</a> mailing list for oVirt related issues.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div><br></div><div>Converted to CentOS Streams which seemed to go fine. Did a yum update and no issues.</div><div><br></div><div>Did a reboot.. and now engine will no longer start. So I can no longer start my Virtual machines. I posted as bug <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1911910" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1911910</a> I posted to CentOS forum <a href="https://forums.centos.org/viewtopic.php?f=54&t=76716" target="_blank">https://forums.centos.org/viewtopic.php?f=54&t=76716</a> but no responses.</div><div><br></div><div>Can anyone provide means or next step to root cause and or fix?</div></div></blockquote><div><br></div><div>We pushed a fix which is in current nightly but please keep using CentOS Linux for oVirt till it will be officially announced its full compatibility with CentOS Stream.</div><div>Currently it's a tech preview up to oVirt 4.4.4.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>It would take me days to rebuild the entire solution and I really hate to "reload" as a fix.. but after two weeks... nothing changing... just have to find some means to get cluster back working.</div><div><br></div><div>Thanks.</div><div><br></div><div>##### all three servers are looping below events in /var/log/messages ### </div><div>Jan 7 09:48:03 thor journal[2375050]: ovirt-ha-broker ovirt_hosted_engine_ha.broker.broker.Broker ERROR Failed initializing the broker: [Errno 107] Transport endpoint is not connected: '/rhev/data-center/mnt/glusterSD/thorst.penguinpages.local:_engine/3afc47ba-afb9-413f-8de5-8d9a2f45ecde/ha_agent/hosted-engine.metadata'<br>Jan 7 09:48:03 thor journal[2375050]: ovirt-ha-broker ovirt_hosted_engine_ha.broker.broker.Broker ERROR Traceback (most recent call last):#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/broker.py", line 64, in run#012 self._storage_broker_instance = self._get_storage_broker()#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/broker.py", line 143, in _get_storage_broker#012 return storage_broker.StorageBroker()#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", line 97, in __init__#012 self._backend.connect()#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 408, in connect#012 self._check_symlinks(self._storage_path, volume.path, service_link)#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", line 105, in _check_symlinks#012 os.unlink(service_link)#012OSError: [Errno 107] Transport endpoint is not connected: '/rhev/data-center/mnt/glusterSD/thorst.penguinpages.local:_engine/3afc47ba-afb9-413f-8de5-8d9a2f45ecde/ha_agent/hosted-engine.metadata'<br>Jan 7 09:48:03 thor journal[2375050]: ovirt-ha-broker ovirt_hosted_engine_ha.broker.broker.Broker ERROR Trying to restart the broker<br>Jan 7 09:48:03 thor platform-python[2375050]: detected unhandled Python exception in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker'<br>Jan 7 09:48:03 thor abrt-server[2375084]: Not saving repeating crash in '/usr/share/ovirt-hosted-engine-ha/ovirt-ha-broker'<br>Jan 7 09:48:03 thor systemd[1]: ovirt-ha-broker.service: Main process exited, code=exited, status=1/FAILURE<br>Jan 7 09:48:03 thor systemd[1]: ovirt-ha-broker.service: Failed with result 'exit-code'.<br>Jan 7 09:48:04 thor systemd[1]: ovirt-ha-broker.service: Service RestartSec=100ms expired, scheduling restart.<br>Jan 7 09:48:04 thor systemd[1]: ovirt-ha-broker.service: Scheduled restart job, restart counter is at 44569.<br>Jan 7 09:48:04 thor systemd[1]: Stopped oVirt Hosted Engine High Availability Communications Broker.<br>Jan 7 09:48:04 thor systemd[1]: Started oVirt Hosted Engine High Availability Communications Broker.<br>Jan 7 09:48:06 thor systemd[1]: ovirt-ha-agent.service: Service RestartSec=10s expired, scheduling restart.<br>Jan 7 09:48:06 thor systemd[1]: ovirt-ha-agent.service: Scheduled restart job, restart counter is at 22270.<br>Jan 7 09:48:06 thor systemd[1]: Stopped oVirt Hosted Engine High Availability Monitoring Agent.<br>Jan 7 09:48:06 thor systemd[1]: Started oVirt Hosted Engine High Availability Monitoring Agent.<br>Jan 7 09:48:06 thor systemd[1]: Started Session c44598 of user root.<br>Jan 7 09:48:06 thor systemd[1]: session-c44598.scope: Succeeded.<br>Jan 7 09:48:06 thor journal[2375091]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Failed to start necessary monitors<br>Jan 7 09:48:06 thor journal[2375091]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Traceback (most recent call last):#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 85, in start_monitor#012 response = self._proxy.start_monitor(type, options)#012 File "/usr/lib64/python3.6/xmlrpc/client.py", line 1112, in __call__#012 return self.__send(self.__name, args)#012 File "/usr/lib64/python3.6/xmlrpc/client.py", line 1452, in __request#012 verbose=self.__verbose#012 File "/usr/lib64/python3.6/xmlrpc/client.py", line 1154, in request#012 return self.single_request(host, handler, request_body, verbose)#012 File "/usr/lib64/python3.6/xmlrpc/client.py", line 1166, in single_request#012 http_conn = self.send_request(host, handler, request_body, verbose)#012 File "/usr/lib64/python3.6/xmlrpc/client.py", line 1279, in send_request#012 self.send_content(connection, request_body)#012 File "/usr/lib64/python3.6/xmlrpc/client.py", line 1309, in send_content#012 connection.endheaders(request_body)#012 File "/usr/lib64/python3.6/http/client.py", line 1264, in endheaders#012 self._send_output(message_body, encode_chunked=encode_chunked)#012 File "/usr/lib64/python3.6/http/client.py", line 1040, in _send_output#012 self.send(msg)#012 File "/usr/lib64/python3.6/http/client.py", line 978, in send#012 self.connect()#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py", line 74, in connect#012 self.sock.connect(base64.b16decode(self.host))#012FileNotFoundError: [Errno 2] No such file or directory#012#012During handling of the above exception, another exception occurred:#012#012Traceback (most recent call last):#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 131, in _run_agent#012 return action(he)#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 55, in action_proper#012 return he.start_monitoring()#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 437, in start_monitoring#012 self._initialize_broker()#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 561, in _initialize_broker#012 m.get('options', {}))#012 File "/usr/lib/python3.6/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 91, in start_monitor#012 ).format(t=type, o=options, e=e)#012ovirt_hosted_engine_ha.lib.exceptions.RequestError: brokerlink - failed to start monitor via ovirt-ha-broker: [Errno 2] No such file or directory, [monitor: 'network', options: {'addr': '172.16.100.1', 'network_test': 'dns', 'tcp_t_address': '', 'tcp_t_port': ''}]<br>Jan 7 09:48:06 thor journal[2375091]: ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.Agent ERROR Trying to restart agent<br>Jan 7 09:48:06 thor systemd[1]: ovirt-ha-agent.service: Main process exited, code=exited, status=157/n/a<br>Jan 7 09:48:06 thor systemd[1]: ovirt-ha-agent.service: Failed with result 'exit-code'.<br></div><div>####</div><div><br></div><div>Is their a means to start a VM... when oVirt / engine is offline?</div><br clear="all"><div><br></div>-- <br><div>penguinpages</div></div>
_______________________________________________<br>
CentOS-virt mailing list<br>
<a href="mailto:CentOS-virt@centos.org" target="_blank">CentOS-virt@centos.org</a><br>
<a href="https://lists.centos.org/mailman/listinfo/centos-virt" rel="noreferrer" target="_blank">https://lists.centos.org/mailman/listinfo/centos-virt</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px"><span>Sandro</span> <span>Bonazzola</span><span style="text-transform:uppercase;color:rgb(170,170,170);margin:0px"></span></p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:12px;margin:0px"><span>MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV</span></p><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;margin:0px 0px 4px;font-size:12px"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat <span>EMEA</span></a></p><div style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:medium;margin-bottom:4px"></div><p style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;margin:0px;font-size:12px"><span style="margin:0px;padding:0px"><a href="mailto:sbonazzo@redhat.com" style="color:rgb(0,0,0);margin:0px" target="_blank">sbonazzo@redhat.com</a> </span></p><div style="color:rgb(0,0,0);font-family:RedHatText,sans-serif;font-size:medium;margin-top:12px"><div style="margin-top:12px"><table border="0"><tbody><tr><td width="100px"><a href="https://www.redhat.com/" target="_blank"><img src="https://static.redhat.com/libs/redhat/brand-assets/2/corp/logo--200.png" width="96" height="23"></a></td><td style="font-size:12px"><div></div></td></tr></tbody></table></div></div><table border="0"><tbody><tr></tr></tbody></table><div style="margin-top:12px"><font color="#000000" face="arial, sans-serif" size="1"><b></b></font></div><div style="margin-top:12px"><font color="#000000" face="arial, sans-serif" size="1"><b><a href="https://mojo.redhat.com/docs/DOC-1199578" target="_blank">Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.<br></a></b></font></div><div style="margin-top:12px"><font color="#000000" face="arial, sans-serif" size="1"><b><br><br></b></font></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div>