Hello!
After recent struggles with compilation under Vagrant over NFS, I'd like to ask if it would be possible to automatically sync all machines provided by Duffy using NTP.
We encountered several nasty race conditions caused by system time discrepancies between a Duffy host and a VM, which I managed to mitigate by installing & enabling chrony. Unfortunately, it's not perfect, as the time synchronization apparently takes some time, so the race condition is present, but it's much less probable.
Going through Jenkins logs it looks that all machines have their time significantly off:
● chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2020-01-12 16:06:53 GMT; 5h 51min left Docs: man:chronyd(8) man:chrony.conf(5) Main PID: 1992 (chronyd) CGroup: /system.slice/chronyd.service └─1992 /usr/sbin/chronyd
Jan 12 16:06:53 n15.pufty systemd[1]: Starting NTP client/server... Jan 12 16:06:53 n15.pufty chronyd[1992]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG) Jan 12 16:06:53 n15.pufty systemd[1]: Started NTP client/server. Jan 12 16:07:01 n15.pufty chronyd[1992]: Selected source xx.xx.xx.xx Jan 12 16:07:01 n15.pufty chronyd[1992]: System clock wrong by -21251.336765 seconds, adjustment started Jan 12 10:12:50 n15.pufty chronyd[1992]: System clock was stepped by -21251.336765 seconds
I suspect waiting for the synchronization to fully complete using chrony's CLI utilities would help, but having this to be handled automatically by each machine would be much better.
Thanks!
Frantisek
On Sun, Jan 12, 2020, at 06:15, František Šumšal wrote:
Hello!
After recent struggles with compilation under Vagrant over NFS, I'd like to ask if it would be possible to automatically sync all machines provided by Duffy using NTP.
We encountered several nasty race conditions caused by system time discrepancies between a Duffy host and a VM, which I managed to mitigate by installing & enabling chrony. Unfortunately, it's not perfect, as the time synchronization apparently takes some time, so the race condition is present, but it's much less probable.
Going through Jenkins logs it looks that all machines have their time significantly off:
● chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2020-01-12 16:06:53 GMT; 5h 51min left Docs: man:chronyd(8) man:chrony.conf(5) Main PID: 1992 (chronyd) CGroup: /system.slice/chronyd.service └─1992 /usr/sbin/chronyd
Jan 12 16:06:53 n15.pufty systemd[1]: Starting NTP client/server... Jan 12 16:06:53 n15.pufty chronyd[1992]: chronyd version 3.4 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +SECHASH +IPV6 +DEBUG) Jan 12 16:06:53 n15.pufty systemd[1]: Started NTP client/server. Jan 12 16:07:01 n15.pufty chronyd[1992]: Selected source xx.xx.xx.xx Jan 12 16:07:01 n15.pufty chronyd[1992]: System clock wrong by -21251.336765 seconds, adjustment started Jan 12 10:12:50 n15.pufty chronyd[1992]: System clock was stepped by -21251.336765 seconds
I suspect waiting for the synchronization to fully complete using chrony's CLI utilities would help, but having this to be handled automatically by each machine would be much better.
Thanks!
Frantisek
-- PGP Key ID: 0xFB738CE27B634E4B
CI-users mailing list CI-users@centos.org https://lists.centos.org/mailman/listinfo/ci-users
*Attachments:*
- signature.asc
Chronyd *should* be enabled by default. Unfortunately the system clocks on this hardware are not very reliable.
If your builds are sensitive to time shifts, it may make sense for you to put some guards/checks in place, but we'll look into a job that resets the system clocks on a regular basis.
--Brian
-
Brian Stinson -
František Šumšal