digitalmars.D - Death by design?
- Bob W (13/13) Apr 21 2005 Although D catches lots of runtime errors,
- Ben Hinkle (8/21) Apr 21 2005 It could be the giant gc collection going on at program exit. Try
- Walter (2/2) Apr 21 2005 For huge files, using std.stream or std.mmfile to read it is a better
- Bob W (15/39) Apr 21 2005 Thanks for your reply, but I cannot help to believe
- Ben Hinkle (6/47) Apr 22 2005 You are probably right - one would think that if the GC were to blame it...
- Bob W (5/14) Apr 22 2005 I could but for now I will follow Walter's advice:
- Manfred Nowak (10/13) Apr 21 2005 [...]
- Thomas Kuehne (10/21) Apr 21 2005 -----BEGIN PGP SIGNED MESSAGE-----
- Manfred Nowak (4/5) Apr 22 2005 See bugs group.
Although D catches lots of runtime errors, programs tend to die silently if the following code is used with huge text files: chararr=cast(char[])read(bigfile); If I remember correctly, the problem starts with files of approx. 6MB+ in size. In some cases the file will load, the program will finish all operations, but will never return to the command line (DmD 0.121, Win version). I always had the perception that range and stack overflow checking is enabled by default and the according exceptions would be flagged. Right or wrong?
Apr 21 2005
"Bob W" <nospam aol.com> wrote in message news:d49lvn$19h7$1 digitaldaemon.com...Although D catches lots of runtime errors, programs tend to die silently if the following code is used with huge text files: chararr=cast(char[])read(bigfile); If I remember correctly, the problem starts with files of approx. 6MB+ in size. In some cases the file will load, the program will finish all operations, but will never return to the command line (DmD 0.121, Win version). I always had the perception that range and stack overflow checking is enabled by default and the according exceptions would be flagged. Right or wrong?It could be the giant gc collection going on at program exit. Try recompiling phobos with the function gc_term in phobos/internal/gc/gc.d set to a no-op. Currently it collects and given a large amount of memory I remember posts from long ago where the program could take a minute (maybe they or my memory were exagerating..) to exit while it runs though checking memory.
Apr 21 2005
"Walter" <newshound digitalmars.com> For huge files, using std.stream or std.mmfile to read it is a better choice.
Apr 21 2005
"Ben Hinkle" <ben.hinkle gmail.com> wrote in message news:d49m8b$19lr$1 digitaldaemon.com..."Bob W" <nospam aol.com> wrote in message news:d49lvn$19h7$1 digitaldaemon.com...Thanks for your reply, but I cannot help to believe that there are some other things going wrong: Given the most simplistic program: read file to buffer, display buffer length, done. Examples: - File size 154,223,881: returns within the fraction of a sec. - File size 158,923,657: waits forever (i.e. until my patience is exhausted and/or ^C is pressed). (Before anyone asks: 1GB installed) If you have a good explanation why a 3% file size increase could possibly cause a maybe almost infinite delay increase, I'll do the Phobos recompilation. Otherwise I just patiently wait for further info.Although D catches lots of runtime errors, programs tend to die silently if the following code is used with huge text files: chararr=cast(char[])read(bigfile); If I remember correctly, the problem starts with files of approx. 6MB+ in size. In some cases the file will load, the program will finish all operations, but will never return to the command line (DmD 0.121, Win version). I always had the perception that range and stack overflow checking is enabled by default and the according exceptions would be flagged. Right or wrong?It could be the giant gc collection going on at program exit. Try recompiling phobos with the function gc_term in phobos/internal/gc/gc.d set to a no-op. Currently it collects and given a large amount of memory I remember posts from long ago where the program could take a minute (maybe they or my memory were exagerating..) to exit while it runs though checking memory.
Apr 21 2005
"Bob W" <nospam aol.com> wrote in message news:d49qpp$1dcp$1 digitaldaemon.com..."Ben Hinkle" <ben.hinkle gmail.com> wrote in message news:d49m8b$19lr$1 digitaldaemon.com...You are probably right - one would think that if the GC were to blame it would be linear. I don't know what the cause is. Maybe you can try putting some printfs in internal/dmain2.d after your main returns and see where it gets stuck."Bob W" <nospam aol.com> wrote in message news:d49lvn$19h7$1 digitaldaemon.com...Thanks for your reply, but I cannot help to believe that there are some other things going wrong: Given the most simplistic program: read file to buffer, display buffer length, done. Examples: - File size 154,223,881: returns within the fraction of a sec. - File size 158,923,657: waits forever (i.e. until my patience is exhausted and/or ^C is pressed). (Before anyone asks: 1GB installed) If you have a good explanation why a 3% file size increase could possibly cause a maybe almost infinite delay increase, I'll do the Phobos recompilation. Otherwise I just patiently wait for further info.Although D catches lots of runtime errors, programs tend to die silently if the following code is used with huge text files: chararr=cast(char[])read(bigfile); If I remember correctly, the problem starts with files of approx. 6MB+ in size. In some cases the file will load, the program will finish all operations, but will never return to the command line (DmD 0.121, Win version). I always had the perception that range and stack overflow checking is enabled by default and the according exceptions would be flagged. Right or wrong?It could be the giant gc collection going on at program exit. Try recompiling phobos with the function gc_term in phobos/internal/gc/gc.d set to a no-op. Currently it collects and given a large amount of memory I remember posts from long ago where the program could take a minute (maybe they or my memory were exagerating..) to exit while it runs though checking memory.
Apr 22 2005
I could but for now I will follow Walter's advice: Stay away from std.file.read() for huge files. He probably has had some info on these issues before. And, who knows, maybe the read() is already shining in DMD 0.122 or DMD 0.123? ; )If you have a good explanation why a 3% file size increase could possibly cause a maybe almost infinite delay increase, I'll do the Phobos recompilation. Otherwise I just patiently wait for further info.You are probably right - one would think that if the GC were to blame it would be linear. I don't know what the cause is. Maybe you can try putting some printfs in internal/dmain2.d after your main returns and see where it gets stuck.
Apr 22 2005
"Ben Hinkle" <ben.hinkle gmail.com> wrote: [...]It could be the giant gc collection going on at program exit. Try recompiling phobos with the function gc_term in phobos/internal/gc/gc.d set to a no-op.[...] I remember that posts, but what is the purpose of a collect at exit anyway? There is another point: I once tried to allocate all memory but did not succeed because no outOfMemoryException was thrown. Instead the program did not return. I reported on that, but nobody seemed interested. -manfred
Apr 21 2005
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Manfred Nowak schrieb am Fri, 22 Apr 2005 03:39:05 +0000 (UTC):"Ben Hinkle" <ben.hinkle gmail.com> wrote: [...]How did your try to "allocate all memory"? Thomas -----BEGIN PGP SIGNATURE----- iD8DBQFCaHS53w+/yD4P9tIRAq7OAJ0VKdl78oinqzs5TDHlhTShgpuoegCfdv6/ OTfr+vjnpOF0UvzzxeXE1OM= =n59e -----END PGP SIGNATURE-----It could be the giant gc collection going on at program exit. Try recompiling phobos with the function gc_term in phobos/internal/gc/gc.d set to a no-op.[...] I remember that posts, but what is the purpose of a collect at exit anyway? There is another point: I once tried to allocate all memory but did not succeed because no outOfMemoryException was thrown. Instead the program did not return.
Apr 21 2005
Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> wrote: [...]How did your try to "allocate all memory"?See bugs group. -manfred
Apr 22 2005