www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Article Review: Migrating from std.date to std.datetime

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
It recently came to my attention that an article on converting code from using 
std.date to using std.datetime would be of value, so I wrote one up. Since 
it's an article, and it's within the time period set by Walter for the article 
contest, I guess that it's in the article contest, but I wrote it because it 
needed writing rather than for the contest. It might as well be part of the 
contest though.

In any case, I put it in my d-programming-language.org repository on github: 
https://github.com/jmdavis/d-programming-language.org/tree/article_datetime

Or if you just want to download it without building d-programming-language.org 
yourself, go here: http://is.gd/jFAmDy

So, feel free to read it and give feeback on it. Hopefully it does its job 
fairly well. Writing it did give me some insight into a couple of tweaks that 
need to be made to std.datetime though. In particular, I really should come up 
with a better way to save time zones for those that care about restoring the 
_exact_ time zone that they had in the SysTime that they saved in a database 
or to disk or whatever. ISO is great for storing the exact time - including 
what the offset from UTC of the original time zone was - but it's not 
generally enough for restoring the exact time zone that you were dealing with 
in the first place. Most code won't care, but I really should find a way to do 
it. I knew about the problem, but it's definitely a pain to solve, so I left 
it alone. I should work on fixing that.

In any case, the article is now available for you to read and review.

- Jonathan M Davis
May 14 2011
next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
Thanks a lot for writing the article. I was just about to get rid of 
std.date and migrate to std.datetime, so it's perfect timing. ;-)

A few nitpicks:

- a short motivation for using hecto-nano-seconds would be nice, it's 
not really the most obvious choice.

- Reading this code:


 d_time dTime = "myfile.txt".lastModified();
 SysTime sysTime = "myfile.txt".timeLastModified();


Using UFCS distracted me quite a bit from the actual point, better use 
standard call syntax.

When migrating, I hit a few problems:

- there is an alias for std.string.indexOf in std.datetime, that causes 
ambiguities when imported with other modules. (AFAICT it is used to 
disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad 
by themselves, but it'd be better to use explicite std.string.indexOf in 
the few places in std.datetime).

- the documentation for std.datetime.WindowsTimeZone is missing on the 
website

- the documentation for std.file.getTimesWin/Posix is not found on the 
website (why the different function names?)

- my current uses of datetime are comparing file times and displaying 
the file time. Much better than std.date, the times displayed are now 
the same as shown by Explorer/dir most of the time, but some are off by 
one hour. It seems this happens for times with a different daylight 
saving. Is this a bug? Or do I need to call something else than

writeln(timeLastModified(file).toSimpleString());

The ISO versions or conversion to DateTime produce the same output. My 
local time is UTC+1+DST.

- running the unittests for std.datetime takes more than 10 minutes 
until it assert here:

Failed time zone: America/Los_Angeles
core.exception.AssertError std\datetime.d(27696): _assertPred!"==" 
failed: [Pazifik Normalzeit] is not == [Pacific Standard Time].

It seems the test strings need to be localized aswell.

Do the tests also run this long under linux? My impression from stopping 
execution inside a debugger is that constructing SysTime is rather slow 
(most time seems to be spent in SystemTimeToTzSpecificLocalTime).

Rainer


Jonathan M Davis wrote:

 It recently came to my attention that an article on converting code from using 
 std.date to using std.datetime would be of value, so I wrote one up. Since 
 it's an article, and it's within the time period set by Walter for the article 
 contest, I guess that it's in the article contest, but I wrote it because it 
 needed writing rather than for the contest. It might as well be part of the 
 contest though.
 
 In any case, I put it in my d-programming-language.org repository on github: 
 https://github.com/jmdavis/d-programming-language.org/tree/article_datetime
 
 Or if you just want to download it without building d-programming-language.org 
 yourself, go here: http://is.gd/jFAmDy
 
 So, feel free to read it and give feeback on it. Hopefully it does its job 
 fairly well. Writing it did give me some insight into a couple of tweaks that 
 need to be made to std.datetime though. In particular, I really should come up 
 with a better way to save time zones for those that care about restoring the 
 _exact_ time zone that they had in the SysTime that they saved in a database 
 or to disk or whatever. ISO is great for storing the exact time - including 
 what the offset from UTC of the original time zone was - but it's not 
 generally enough for restoring the exact time zone that you were dealing with 
 in the first place. Most code won't care, but I really should find a way to do 
 it. I knew about the problem, but it's definitely a pain to solve, so I left 
 it alone. I should work on fixing that.
 
 In any case, the article is now available for you to read and review.
 
 - Jonathan M Davis
May 15 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-15 03:34, Rainer Schuetze wrote:

 Thanks a lot for writing the article. I was just about to get rid of
 std.date and migrate to std.datetime, so it's perfect timing. ;-)
 
 A few nitpicks:
 
 - a short motivation for using hecto-nano-seconds would be nice, it's
 not really the most obvious choice.


It's the highest precision that you can have and still have a decent range of 
values in a 64-bit integer. Going by 10 ns units would result in the range 
being too small, but 100 ns gives you from around 29,000 B.C. to 29,000 A.D. 



 - Reading this code:
  > d_time dTime = "myfile.txt".lastModified();
  > SysTime sysTime = "myfile.txt".timeLastModified();
 
 Using UFCS distracted me quite a bit from the actual point, better use
 standard call syntax.
 
 When migrating, I hit a few problems:
 
 - there is an alias for std.string.indexOf in std.datetime, that causes
 ambiguities when imported with other modules. (AFAICT it is used to
 disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad
 by themselves, but it'd be better to use explicite std.string.indexOf in
 the few places in std.datetime).


That's a bug. I shouldn't have done that. I didn't think about the alias being 
public when I did it. It has now been fixed on github.


 - the documentation for std.datetime.WindowsTimeZone is missing on the
 website


I don't know why it would be. It's probably a bug relating to the new StdDdoc 
versioning that we switched to (instead of using D_Ddoc which screwed up 
people who build the documentation at the same time as their code - a bad 
practice IMHO, but we decided to not set it up so that it didn't work when 
using Phobos). As WindowsTimeZone is only on Windows, it definitely had to be 
versioned with StdDdoc.


 - the documentation for std.file.getTimesWin/Posix is not found on the
 website (why the different function names?)


Probably a StdDdoc problem as well. As for the different names, the that's 
because of the fact that they give you different values. Windows has file 
creation times. Posix does not. Instead, it gives you the last time that the 
file status was changed (which historically, std.file has mistakenly been 
returning for the creation time of files on Posix).


 - my current uses of datetime are comparing file times and displaying
 the file time. Much better than std.date, the times displayed are now
 the same as shown by Explorer/dir most of the time, but some are off by
 one hour. It seems this happens for times with a different daylight
 saving. Is this a bug? Or do I need to call something else than
 
 writeln(timeLastModified(file).toSimpleString());
 
 The ISO versions or conversion to DateTime produce the same output. My
 local time is UTC+1+DST.


I'd need more data to have any idea what's going on, but it sounds like a bug.


 - running the unittests for std.datetime takes more than 10 minutes
 until it assert here:
 
 Failed time zone: America/Los_Angeles
 core.exception.AssertError std\datetime.d(27696): _assertPred!"=="
 failed: [Pazifik Normalzeit] is not == [Pacific Standard Time].
 
 It seems the test strings need to be localized aswell.


I didn't even know that time zone strings were ever localized, and there's no 
way that the unit tests are going to be able to deal with that other than not 
testing stuff that would ideally be being tested. I'll have to figure out what 
to about that.


 Do the tests also run this long under linux? My impression from stopping
 execution inside a debugger is that constructing SysTime is rather slow
 (most time seems to be spent in SystemTimeToTzSpecificLocalTime).


The unit tests have never taken anywhere near that long for me. I'm in the 
midst of rewriting them though (some of that rewrite was in the last release), 
so their performance characteristics could change. It tests a lot though, so 
I'd expect it to take longer than the other Phobos modules.

- Jonathan M Davis
May 15 2011
next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
Jonathan M Davis wrote:

 On 2011-05-15 03:34, Rainer Schuetze wrote:

 Thanks a lot for writing the article. I was just about to get rid of
 std.date and migrate to std.datetime, so it's perfect timing. ;-)

 A few nitpicks:

 - a short motivation for using hecto-nano-seconds would be nice, it's
 not really the most obvious choice.


 
 It's the highest precision that you can have and still have a decent range of 
 values in a 64-bit integer. Going by 10 ns units would result in the range 
 being too small, but 100 ns gives you from around 29,000 B.C. to 29,000 A.D. 



Thanks for the clarification (I know it's also the resolution from the 
Windows API). I suggest to put it somewhere in the article or the 
documentation.



 - there is an alias for std.string.indexOf in std.datetime, that causes
 ambiguities when imported with other modules. (AFAICT it is used to
 disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad
 by themselves, but it'd be better to use explicite std.string.indexOf in
 the few places in std.datetime).


 
 That's a bug. I shouldn't have done that. I didn't think about the alias being 
 public when I did it. It has now been fixed on github.
 


Unfortunately, for some obscure reason, private module declarations are 
still placed into the overload set, so your fix does not help.
You should either alias it to something unlikely to clash with something 
else or use fully qualified names (though this does not allow UFCS).


 - the documentation for std.datetime.WindowsTimeZone is missing on the
 website


 
 I don't know why it would be. It's probably a bug relating to the new StdDdoc 
 versioning that we switched to (instead of using D_Ddoc which screwed up 
 people who build the documentation at the same time as their code - a bad 
 practice IMHO, but we decided to not set it up so that it didn't work when 
 using Phobos). As WindowsTimeZone is only on Windows, it definitely had to be 
 versioned with StdDdoc.


I just noticed that it shows up on 
http://www.d-programming-language.org/phobos/std_datetime.html,
but not on http://www.digitalmars.com/d/2.0/phobos/std_datetime.html


 

 - the documentation for std.file.getTimesWin/Posix is not found on the
 website (why the different function names?)


 
 Probably a StdDdoc problem as well. 


also on d-programming-lanuage.org.
May 15 2011
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-15 05:26, Rainer Schuetze wrote:

 Jonathan M Davis wrote:

 On 2011-05-15 03:34, Rainer Schuetze wrote:

 Thanks a lot for writing the article. I was just about to get rid of
 std.date and migrate to std.datetime, so it's perfect timing. ;-)
 
 A few nitpicks:
 
 - a short motivation for using hecto-nano-seconds would be nice, it's
 not really the most obvious choice.


 
 It's the highest precision that you can have and still have a decent
 range of values in a 64-bit integer. Going by 10 ns units would result
 in the range being too small, but 100 ns gives you from around 29,000



 
 Thanks for the clarification (I know it's also the resolution from the
 Windows API). I suggest to put it somewhere in the article or the
 documentation.
 

 - there is an alias for std.string.indexOf in std.datetime, that causes
 ambiguities when imported with other modules. (AFAICT it is used to
 disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad
 by themselves, but it'd be better to use explicite std.string.indexOf in
 the few places in std.datetime).


 
 That's a bug. I shouldn't have done that. I didn't think about the alias
 being public when I did it. It has now been fixed on github.


 
 Unfortunately, for some obscure reason, private module declarations are
 still placed into the overload set, so your fix does not help.
 You should either alias it to something unlikely to clash with something
 else or use fully qualified names (though this does not allow UFCS).
 

 - the documentation for std.datetime.WindowsTimeZone is missing on the
 website


 
 I don't know why it would be. It's probably a bug relating to the new
 StdDdoc versioning that we switched to (instead of using D_Ddoc which
 screwed up people who build the documentation at the same time as their
 code - a bad practice IMHO, but we decided to not set it up so that it
 didn't work when using Phobos). As WindowsTimeZone is only on Windows,
 it definitely had to be versioned with StdDdoc.


 
 I just noticed that it shows up on
 http://www.d-programming-language.org/phobos/std_datetime.html,
 but not on http://www.digitalmars.com/d/2.0/phobos/std_datetime.html
 

 - the documentation for std.file.getTimesWin/Posix is not found on the
 website (why the different function names?)


 
 Probably a StdDdoc problem as well.


 
 also on d-programming-lanuage.org.


Well, Walter obviously isn't using the d-programming-language.org makefile to 
generate the documentation for www.digitalmars.com, since the style is 
completely different. I would guess that he didn't adjust what he was doing to 
use -version=StdDdoc.

- Jonathan M Davis
May 15 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-15 05:26, Rainer Schuetze wrote:

 Jonathan M Davis wrote:

 On 2011-05-15 03:34, Rainer Schuetze wrote:

 - there is an alias for std.string.indexOf in std.datetime, that causes
 ambiguities when imported with other modules. (AFAICT it is used to
 disambiguate std.algorithm.indexOf and std.string.indexOf, which are bad
 by themselves, but it'd be better to use explicite std.string.indexOf in
 the few places in std.datetime).


 
 That's a bug. I shouldn't have done that. I didn't think about the alias
 being public when I did it. It has now been fixed on github.


 
 Unfortunately, for some obscure reason, private module declarations are
 still placed into the overload set, so your fix does not help.
 You should either alias it to something unlikely to clash with something
 else or use fully qualified names (though this does not allow UFCS).


Bug reported: http://d.puremagic.com/issues/show_bug.cgi?id=6013

I also altered the private alias for indexOf so that it's using a name which 
won't conflict, so hopefully it won't cause any further problems, though I'd 
definitely prefer it if the bug were fixed and private alias worked properly.

- Jonathan M Davis
May 15 2011
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Fantastic work! I was lost in that std.datetime jungle for a while
now. It's a great read (your English is pretty fluent!).

Btw, I think there's a missing word in this sentence: "If what you're
doesn't need that extra boost of efficiency. (doing?).
May 16 2011
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
Jonathan M Davis wrote:

 On 2011-05-15 03:34, Rainer Schuetze wrote:

 - my current uses of datetime are comparing file times and displaying
 the file time. Much better than std.date, the times displayed are now
 the same as shown by Explorer/dir most of the time, but some are off by
 one hour. It seems this happens for times with a different daylight
 saving. Is this a bug? Or do I need to call something else than

 writeln(timeLastModified(file).toSimpleString());

 The ISO versions or conversion to DateTime produce the same output. My
 local time is UTC+1+DST.


 
 I'd need more data to have any idea what's going on, but it sounds like a bug.


I have investigated it a bit, and I'm currently thinking that Explorer, 
dir and Total Commander are wrong, and std.datetime is right.

What I did to test:
- create a new file, verify the dates printed are correct. fine in all 
programs
- set the system time back to February
- create the file again, verify the dates printed are correct. fine in 
all programs
- set the system time back to May
- now the displayed file time is the same when using std.datetime, but 
an hour later with all other programs.

Maybe it's an XP or NTFS issue, but I would expect the file time not to 
change with the date it is displayed.
May 16 2011
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
 Jonathan M Davis wrote:

 On 2011-05-15 03:34, Rainer Schuetze wrote:

 - my current uses of datetime are comparing file times and displaying
 the file time. Much better than std.date, the times displayed are now
 the same as shown by Explorer/dir most of the time, but some are off by
 one hour. It seems this happens for times with a different daylight
 saving. Is this a bug? Or do I need to call something else than
 
 writeln(timeLastModified(file).toSimpleString());
 
 The ISO versions or conversion to DateTime produce the same output. My
 local time is UTC+1+DST.


 
 I'd need more data to have any idea what's going on, but it sounds like a
 bug.


 
 I have investigated it a bit, and I'm currently thinking that Explorer,
 dir and Total Commander are wrong, and std.datetime is right.
 
 What I did to test:
 - create a new file, verify the dates printed are correct. fine in all
 programs
 - set the system time back to February
 - create the file again, verify the dates printed are correct. fine in
 all programs
 - set the system time back to May
 - now the displayed file time is the same when using std.datetime, but
 an hour later with all other programs.
 
 Maybe it's an XP or NTFS issue, but I would expect the file time not to
 change with the date it is displayed.


Well, Microsoft made the mistake of making it so that Windows boxes keep their 
time in local time (which they probably can't fix and maintain backwards 
compatability, so we're likely stuck), and that can have some interesting 
consequences, since that means that the system clock actually gets changed 
when DST begins or ends. I would have thought that the OS and file system 
would handle that well enough for Explorer to display the time correctly, but 
I don't know. Given how hard it can be to catch DST-related bugs and the fact 
that people don't generally examine the times on their files closely enough to 
verify that their times aren't 1 hour off 6 months later, it wouldn't entirely 
surprise me if there were bugs with regards to that on XP. But I still would 
have thought that they would have caught it.

- Jonathan M Davis
May 16 2011
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
 Fantastic work! I was lost in that std.datetime jungle for a while
 now. It's a great read (your English is pretty fluent!).


I would hope so. I'm a native of California. English is my native language. 
I'd be more worried about my French fluency (which is good but likely 
deteriorating since I haven't been to Europe recently) than my English 
fluency. And, of course, I'm not fluent in any other language, so I'd be 
complete rubbish at anything else.


 Btw, I think there's a missing word in this sentence: "If what you're
 doesn't need that extra boost of efficiency. (doing?).


Yes. It sounds like it's the word doing that's missing. I'll fix it this 
evening.

- Jonathan M Daviss
May 16 2011