Problem with timezone in Perl (and possibly elsewhere)
A colleague reported the following problem, which I am attempting to relay
as accurately as possible: When one sets the 'TZ' environment variable in
one's Perl script, then calls localtime(), the time returned is correct
for the chosen time zone. However, if one sets 'TZ' -again-, then calls
localtime(), the zone does not change.
This problem seems to be specific to Debian. The user has tested on
non-Debian systems, and they do not seem to suffer from this problem. It
may be a problem with Perl, or with the underlying system function(s)
which localtime() calls.
The writeup below is from the user who discovered the problem, 'otto'.
This problem can be demonstrated at the following URLs, which include a
printout of their own source code:
http://otto.twu.net/timetest.cgi
http://otto.twu.net/timetest.cgi?callfirst=y
=======================================================================
START QUOTE...
=======================================================================
I have found the problem with 'hist'... and the problem is that Perl's
localtime() is broken on TWU (and other systems- read on...) in a somewhat
subtle way. It is, fortunately, a way that's easy to code around- this time.
But there are other things I've done here and there that might not be so
easily modified.
In my code, the array @loc_time holds the results of the calls to localtime()
that are used by the presentation layer to show the current time, adjusted for
timezone as specified by the user. This is present in both the the
command-line and CGI versions, as the former also needs to display the current
time when drawing its graph. Coincidentally, the CLI version was also
intended to be capable of handling timezone offsets. When declared, this
variable was also initialized, like so:
my @loc_time = localtime();
The idea was that the data would be replaced, if needed, by subsequent code
when a timezone offset was requested by the user; otherwise it would be used
as-is, already holding the system's local time as a default. This line of
code remained in the CGI version, even after a new system to select a default
timezone was put in place; the CGI version always assigns a value to TZ and
explicitly replaces the contents of @loc_time, acting as if the user chose my
home timezone (programmer's perogative...) if no value is actually selected.
It was originally done simply because TWU was not in my home timezone at the
time, and using the mechanism set up for interpreting CGI parameters was the
easiest way to have 'hist' always act as if it were unless told otherwise.
Doing it this way caused no error when 'hist' was originally placed on TWU in
2003, and is not the source of the problem the program has had since the most
recent move. It was, in fact, the initialization of the variable shown above
that was bringing the problem to light.
Simply put, Perl was reading the TZ variable during the first invocation of
localtime()... and ONLY the first invocation. Any subsequent changes to TZ
were ignored. Since TZ had not been explicitly set at the time of variable
declaration, any existing value for TZ (or the system default if none was
specified) would therefore be used throughout the rest of the program,
regardless of what the user specified. Changing the variable declaration to
remove the initialization avoided the problem:
my @loc_time;
With this one line rewritten, the program does not call localtime() until
after TZ is changed, and so timezone shifting works as expected.
Your own example CGI code has been modified to be closer to the way 'hist'
does things (CGI objects invoked for a few things, data structures changed
around, taint checking turned on) to reduce the number of variables that might
have affected the handling of the TZ and the storage/display of results from
localtime(). The results are posted here:
http://otto.twu.net/timetest.cgi
Full source is presented. Load that as-is and you'll see the results you
expect, but give it the parameter 'callfirst=y' and you'll see that the bug is
reproduced- it outputs Central (machine local) time even though it will still
attempt to spit out out Eastern.
This is not normal behavior for localtime() and/or the handling of the TZ
variable.
A brief test was run on several systems. The test is a success if 'date' and
the Perl code do NOT output the same time. All systems were in the Central
timezone except as noted; Pacific time was used for TZ in this series of tests
rather than Eastern as in our previous discussions. The results were as
follows:
discovery (Mac OS X 10.4.11 [intel]; Perl 5.8.6):
[meneleig@discovery:~]$ date; perl -e '@time = localtime(); $ENV{TZ} =
"PST8PDT"; @time = localtime(); printf("%02i:%02i:%02i\n", $time[2],
$time[1],
$time[0]);'
Mon Jun 9 23:33:26 CDT 2008
21:33:26
saturn (Sun OS 5.8; Perl 5.005_03):
[meneleig@saturn:~]$ date; perl -e '@time = localtime(); $ENV{TZ} =
"PST8PDT";
@time = localtime(); printf("%02i:%02i:%02i\n", $time[2], $time[1],
$time[0]);'
Mon Jun 9 23:33:36 CDT 2008
21:33:36
A friend's system (Fedoara Core 8 i386; Perl 5.8.8; this system is on Eastern
time):
$ date; perl -e '@time = localtime(); $ENV{TZ} = "PST8PDT"; @time =
localtime(); printf("%02i:%02i:%02i\n", $time[2], $time[1], $time[0]);'
Tue Jun 10 00:23:29 EDT 2008
21:23:29
goshawk (Debian GNU/Linux parisc; Perl 5.8.8):
[meneleig@goshawk:~]$ date; perl -e '@time = localtime(); $ENV{TZ} =
"PST8PDT"; @time = localtime(); printf("%02i:%02i:%02i\n", $time[2],
$time[1],
$time[0]);'
Mon Jun 9 23:30:19 CDT 2008
23:30:19
pippin (TWU) (Debian GNU/Linux i686; Perl 5.8.8):
otto@pippin:~$ date; perl -e '@time = localtime(); $ENV{TZ} = "PST8PDT"; @time
= localtime(); printf("%02i:%02i:%02i\n", $time[2], $time[1], $time[0]);'
Mon Jun 9 23:26:54 CDT 2008
23:26:54
With this evidence it's clear that this problem is expressed on Debian
GNU/Linux for at least two processor architectures when running Perl 5.8.8.
The same version of Perl on Fedora does not exhibit this behavior, nor do a
limited number of other versions of Perl on other Unix systems not derived
from Linux. It is unknown at this time whether other versions of Perl
distributed for Debian have this problem. I do not recall what version of
Perl was installed on the previous incarnation of TWU, but I recall it being
one of the Red Hat based distributions. As 'hist' worked just fine there (as
it had on every incarnation of TWU since 2003) I must conclude that it was
also not affected.
It can also be shown conclusively that the broken localtime() respects the
first reading of a modified TZ variable, but ignores subsequent
modifications,
as seen here:
otto@pippin:~$ date; perl -e '$ENV{TZ} = "UTC"; @time = localtime(); $ENV{TZ}
= "PST8PDT"; @time = localtime(); printf("%02i:%02i:%02i\n", $time[2],
$time[1], $time[0]);'
Tue Jun 10 00:00:14 CDT 2008
05:00:14
Subsequent calls to localtime() within the same process do show that the time
is being updated- it is only the TZ variable that is ignored (even if reset
before every call):
otto@pippin:~$ date; perl -e '@time = localtime(); while(1){ $ENV{TZ} =
"PST8PDT"; @time = localtime(time()); printf("%02i:%02i:%02i\n", $time[2],
$time[1], $time[0]); sleep(20); }'
Tue Jun 10 00:06:23 CDT 2008
00:06:23
00:06:43
00:07:03
00:07:23
00:07:43
If I may be allowed a bit of conjecture here, I suspect that the error results
from someone on the Debian team being 'clever' and optimizing the guts behind
localtime() so that it would only check the TZ variable once in a given
process' lifetime, thus perhaps saving some system resources if localtime() is
called in a tight loop. Then again, as we've discovered, it has the side
effect of breaking any application that depends on modifying its TZ variable
between invocations of localtime(). I'd normally think a distro maintainer
wouldn't do something as potentially damaging as that, but... Well, this
wouldn't be the first time the Debian team modified package code and failed to
imagine the consequences. Just look at what they did with the code that
generates SSL keys- just to avoid a compiler warning. I hold open the
possibility that the 'guts' of which I speak lie below the level of Perl
itself, in a library that (possibly many) other programs are linked to, but
digging into C sources and/or writing C to test this is beyond the scope of
this discussion, as is determining whether TZ is the only variable Perl treats
as "read once and only once" on Debian.
I am comfortable, for the moment, with the primary conclusion of this
investigation- that the problem did not originate with the code of 'hist', but
rather with a faulty port of Perl (or an underlying library or libraries) on
Debian.
=======================================================================
...END QUOTE
=======================================================================
Reply to: