Timesync Problem detected with SLES SP3

SUSE Linux Enterprise Server SLES Configure-Administer
1

I have a very unusual problem with all our our SLES 11 SP3 (also one SP2 but symptoms identical) servers. We host some 40 XEN servers on SLES 11 SP3 Bare metal blade servers. 12 of the servers are OES11 SP1

All servers are using ntpd to synchronize time. A typical config file is as follows

[CODE]driftfile /var/lib/ntp/drift/ntp.drift

path for drift file

logfile /var/log/ntp
server 192.168.100.1
server au.pool.ntp.org

Authentication stuff

keys /etc/ntp.keys # path for keys file
trustedkey 1 # define trusted keys
requestkey 1 # key (7) for accessing server variables[/CODE]

The bare metal servers, and all the OES11 virtual servers have perfect time sync. The OES time is tightly monitored and no issues have been detected in long term logs.

A couple of days ago,we noticed that all SLES 11 SP3 servers (XEN virtual only) were about 5 minutes behind. You could reset the time using ‘rcntpd ntptimeset’ and the time would be corrected, but within the minute, the time would revert to 5 minutes behind.

After hours of searching, I finally decided to try a patch update. (The servers are only ever a couple of months out in the patch cycle). With all SLES 11 SP3 servers (excluding the bare metal) updated to the latest patches and restarted, the exact same symptom exists, except that the servers are all now 1 minute 28 secs fast. I can run ‘rcntpd ntptimeset’ and the time will be correct. Within a minute, the time will suddenly jump ahead nearly two minutes.

This is new on a production system that otherwise is running very well. I don’t believe it is a configuration issue, or a time server issue (As all configs are similar or identical and the timeservers are the same for all servers, those in sync and those not. I suspect the patch cycle before the current one has introduced a bug!

Version information
Linux svl-bne-xen-16 3.0.101-0.7.17-xen #1 SMP Tue Feb 4 13:24:49 UTC 2014 (90aac76) x86_64 x86_64 x86_64 GNU/Linux

Is anyone else seeing this problem?
Thanks
Eric.

2

If you stop and disable NTP temporarily on the Xen VMs does that make a
difference? I’m guessing it will not, so we can then rule out NTP entirely.

What is the time set to on the Xen hosts(s)? It is, perhaps, two minutes
fast?

–
Good luck.

If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…

3

As part of our debugging, we have run ntptimeset and then switch off ntpd. The time is still reset as in jumps nearly two minutes ahead within the minute,
The VM hosts are spot on time. They are using NTP and the same NTP servers to sync off.

4

Hi ucba,

have you enabled “independet wallclock” within the VMs? You should do this whenever you use NTP inside a VM.

yourhost:~ # sysctl -e xen.independent_wallclock xen.independent_wallclock = 1

With regards,
Jens