Hi,
We have a little over 100 servers, almost all running CentOS 5.7. Virtually all are Dell servers, generally a mix of 1950s, R610s, and R410s.
We use NTP and/or PTP to sync their clocks. One phenomenon we've noticed is that (1) on reboot, the clocks are all greatly out of sync, and (2) if the PTP or NTP process is stopped, the clocks start drifting very quickly.
If it was isolated to one or two servers, I'd dismiss the issue. I also had this problem under CentOS 4.
I suspect something is mis-configured, because I can't imagine the hardware clock on ALL these servers is *that* bad.
Anyone else dealt with anything similar?
Thanks! Matt
On Tue, 2013-01-22 at 21:16 -0600, Matt Garman wrote:
Hi,
We have a little over 100 servers, almost all running CentOS 5.7. Virtually all are Dell servers, generally a mix of 1950s, R610s, and R410s.
We use NTP and/or PTP to sync their clocks. One phenomenon we've noticed is that (1) on reboot, the clocks are all greatly out of sync, and (2) if the PTP or NTP process is stopped, the clocks start drifting very quickly.
If it was isolated to one or two servers, I'd dismiss the issue. I also had this problem under CentOS 4.
I suspect something is mis-configured, because I can't imagine the hardware clock on ALL these servers is *that* bad.
Well -- in my experience ( 15+ years with RH variants of Linux, and ~25 with various Unix flavors ) they CAN be that bad -- especially with some of the "economy" chipsets used with the Intel architecture. It gets worse when you have a CMOS battery that's getting old and weak. The clock may default back to its initial value, or it might just run slow.
Some folks might consider this a "brute force" approach, but I keep it simple and just reset the hardware clock once a week via cron. I prefer to do it in the wee hours, shortly before the weekly cron jobs run on Sunday morning. Put something like this in root's crontab.
3 3 * * 0 /sbin/hwclock --systohc
For the gory details, refer to the man page for "hwclock" and it will tell all.
Anyone else dealt with anything similar?
Thanks! Matt _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Am 23.01.2013 um 06:21 schrieb Ron Loftin reloftin@twcny.rr.com:
On Tue, 2013-01-22 at 21:16 -0600, Matt Garman wrote:
Hi,
We have a little over 100 servers, almost all running CentOS 5.7. Virtually all are Dell servers, generally a mix of 1950s, R610s, and R410s.
We use NTP and/or PTP to sync their clocks. One phenomenon we've noticed is that (1) on reboot, the clocks are all greatly out of sync, and (2) if the PTP or NTP process is stopped, the clocks start drifting very quickly.
If it was isolated to one or two servers, I'd dismiss the issue. I also had this problem under CentOS 4.
I suspect something is mis-configured, because I can't imagine the hardware clock on ALL these servers is *that* bad.
Well -- in my experience ( 15+ years with RH variants of Linux, and ~25 with various Unix flavors ) they CAN be that bad -- especially with some of the "economy" chipsets used with the Intel architecture. It gets worse when you have a CMOS battery that's getting old and weak. The clock may default back to its initial value, or it might just run slow.
Some folks might consider this a "brute force" approach, but I keep it simple and just reset the hardware clock once a week via cron. I prefer to do it in the wee hours, shortly before the weekly cron jobs run on Sunday morning. Put something like this in root's crontab.
3 3 * * 0 /sbin/hwclock --systohc
this is already done in /etc/rc.d/init.d/halt on halt/reboot.
-- LF
From: Matt Garman matthew.garman@gmail.com
I suspect something is mis-configured, because I can't imagine the hardware clock on ALL these servers is *that* bad.
Tried SYNC_HWCLOCK=yes in /etc/sysconfig/ntpd?
JD
On Jan 22, 2013, at 8:16 PM, Matt Garman wrote:
Hi,
We have a little over 100 servers, almost all running CentOS 5.7. Virtually all are Dell servers, generally a mix of 1950s, R610s, and R410s.
We use NTP and/or PTP to sync their clocks. One phenomenon we've noticed is that (1) on reboot, the clocks are all greatly out of sync, and (2) if the PTP or NTP process is stopped, the clocks start drifting very quickly.
If it was isolated to one or two servers, I'd dismiss the issue. I also had this problem under CentOS 4.
I suspect something is mis-configured, because I can't imagine the hardware clock on ALL these servers is *that* bad.
Anyone else dealt with anything similar?
---- pretty much everyone deals with this issue in one way or another.
You don't actually show us your ntp.conf but generally, I would recommend that you make this the very first line of ntp.conf: tinker panic 0
Craig
Craig White wrote:
On Jan 22, 2013, at 8:16 PM, Matt Garman wrote:
We have a little over 100 servers, almost all running CentOS 5.7. Virtually all are Dell servers, generally a mix of 1950s, R610s, and R410s.
Hmmm... May I *STRONGLY* urge you to replace the 1950's ASAP. We've surplussed all of ours. We started to lose them to hardware issues: I think it was the RAID controller dying. We were really amazed at the quality control... - in a short time (literally weeks) we had several (three? four? five?) die with this error. I'd say you're on borrowed time. <snip>
You don't actually show us your ntp.conf but generally, I would recommend that you make this the very first line of ntp.conf: tinker panic 0
Absolutely. This is something I've read, and done; it lets it update (tinker with the time) when they go that much out.
mark
-
Craig White -
John Doe -
Leon Fauster -
m.roth＠5-cent.us -
Matt Garman -
Ron Loftin