Dear guys,
I think lot of answers flied through on the original questions and I didn't see proper answers to them, so I would like to add my contribution to this topic:
I collected a lot of information some week ago because of industrial servers where the time sync was an unresolved problem.
As the questions raised by Fred has overlaps with my research results, I give some answer with links. Hopefully lot of people can use it in the future.
"The time server is quite close to the computer clock. What causes the discrepancy? The offset in the time server response is about 10 min. The offset is measured from what to what and how is it measured?"
Once you synced the time and experience later huge offset, it means that one side of your synchronization doesn't keep the time properly or the synchronization doesn't work itself (you can make sure by running ntpq -p and verify the output. note: ntpd has to run!). Assume, that you choosed the proper time source for sync, so it means that the mechanism which steps your clock locally is broken. It can be battery reasons on the mainboard, temperature can also influence (but not too much nowadays), if you use virtual machine, that matters a lot, as it gets the CPU cycles from the host machine, and it's inner time stepping highly depends on the CPU time given by the host, finally I have never read anything about the effect of overclocking the CPU, it might have also effect on this, but test should be run to be sure.
And a quotation about the offset calculation:
"Synchronizing a client to a network server consists of several packet exchanges where each exchange is a pair of request and reply. When sending out a request, the client stores its own time (originate timestamp) into the packet being sent. When a server receives such a packet, it will in turn store its own time (receive timestamp) into the packet, and the packet will be returned after putting a transmit timestamp into the packet. When receiving the reply, the receiver will once more log its own receipt time to estimate the travelling time of the packet. The travelling time (delay) is estimated to be half of "the total delay minus remote processing time", assuming symmetrical delays.
Those time differences can be used to estimate the time offset between both machines, as well as the dispersion (maximum offset error). The shorter and more symmetric the round-trip time, the more accurate the estimate of the current time.
Time is not believed until several packet exchanges have taken place, each passing a set of sanity checks. Only if the replies from a server satisfy the conditions defined in the protocol specification, the server is considered valid. Time cannot be synchronized from a server that is considered invalid by the protocol. Some essential values are put into multi-stage filters for statistical purposes to improve and estimate the quality of the samples from each server. All used servers are evaluated for a consistent time. In case of disagreements, the largest set of agreeing servers (truechimers) is used to produce a combined reference time, thereby declaring other servers as invalid (falsetickers).
Usually it takes about five minutes (five good samples) until a NTP server is accepted as synchronization source. Interestingly, this is also true for local reference clocks that have no delay at all by definition."
And about ntpdate.
Ntpdate, believe me, is a phantastic tool which helps you the fastest way in the most critical situations where there is no time to wait until everything gets better.
We know, that some encryption algorithm depends very much on accurate time, so for example in a huge infrastructure where Apache webservers using TLS encryption, and the server also have Kerberos authentication enabled, and very important things the server is doing, there the communication can not be broken just because NTP was not able to step the clock enough fast.
Ntpd has higher threshold to step the clock.
"After some time, small offsets (significantly less than a second) will be slewed (adjusted slowly), while larger offsets will cause the clock to be stepped (set anew)."
In contrast, ntpdate:
"The default is to step the time using settimeofday() if the offset is greater than +-128 ms. "
So in a situtation like that I never wait to ntpd to sync the time slowly. I use ntpdate, but it can be used only if ntpd or other sync programs doesn't use UDP123 port. So I stop ntpd and make an ntpdate <IP to ntpserver> and the clock is stepped right after the syncronization.
About chrony:
chrony has more builtin mechanism to keep the time more accurately on the server even if it is disconnected from the network.
I think the future is for chrony.
Details (and sorry for the redhat link, but just because it is RedHat, it is very correct information):
So guys, I hope it was useful.
Best regards,
Tamas Fekete