www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - strange memory dmd bug

reply Timon Gehr <timon.gehr gmx.ch> writes:
void main(){
     mixin({
             string r;
             foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
             return r;
         }());
}

When I compile this program, DMD sometimes uses about 1GB of memory, 
sometimes about 2GB of memory and sometimes it makes my OS crash.

Can anyone reproduce this?
Sep 09 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/09/2011 10:30 PM, Timon Gehr wrote:

 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
 return r;
 }());
 }

 When I compile this program, DMD sometimes uses about 1GB of memory,
 sometimes about 2GB of memory and sometimes it makes my OS crash.

 Can anyone reproduce this?


Similarly this:

template TT(string s){enum TT=s;}

void main(){
     mixin({
             string r;
             foreach(i;0..12000) r~=q{{enum x=TT!"";};};
             return r;
         }());
}

That does use a somewhat smaller amount of memory, but it is still huge 
and non-deterministic in size.
Sep 09 2011
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 10.09.2011 0:55, Timon Gehr wrote:

 On 09/09/2011 10:30 PM, Timon Gehr wrote:

 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
 return r;
 }());
 }

 When I compile this program, DMD sometimes uses about 1GB of memory,
 sometimes about 2GB of memory and sometimes it makes my OS crash.

 Can anyone reproduce this?


 Similarly this:

 template TT(string s){enum TT=s;}

 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{{enum x=TT!"";};};
 return r;
 }());
 }

 That does use a somewhat smaller amount of memory, but it is still huge
 and non-deterministic in size.


I guess issue 6498, i.e. it's CTFE doing memory allocation on each 
append in foreach loop.
http://d.puremagic.com/issues/show_bug.cgi?id=6498

As for as non-determinism, maybe heap fragmentation?

-- 
Dmitry Olshansky
Sep 09 2011
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/09/2011 11:44 PM, Dmitry Olshansky wrote:

 On 10.09.2011 0:55, Timon Gehr wrote:

 On 09/09/2011 10:30 PM, Timon Gehr wrote:

 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
 return r;
 }());
 }

 When I compile this program, DMD sometimes uses about 1GB of memory,
 sometimes about 2GB of memory and sometimes it makes my OS crash.

 Can anyone reproduce this?


 Similarly this:

 template TT(string s){enum TT=s;}

 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{{enum x=TT!"";};};
 return r;
 }());
 }

 That does use a somewhat smaller amount of memory, but it is still huge
 and non-deterministic in size.


 I guess issue 6498, i.e. it's CTFE doing memory allocation on each
 append in foreach loop.
 http://d.puremagic.com/issues/show_bug.cgi?id=6498


Well, I don't get the issue if I don't mixin mixins or template 
instantiations.

void main(){
     mixin({
             string r;
             foreach(i;0..12000) r~="{}{}{}{}{}{}{}{}"; 

             return r;
         }());
}


 As for as non-determinism, maybe heap fragmentation?


That might well be it. Thanks!
Sep 09 2011
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, September 09, 2011 23:58:56 Timon Gehr wrote:

 On 09/09/2011 11:44 PM, Dmitry Olshansky wrote:

 On 10.09.2011 0:55, Timon Gehr wrote:

 On 09/09/2011 10:30 PM, Timon Gehr wrote:

 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{mixin(q{{enum x="";}});};
 return r;
 }());
 }
 
 When I compile this program, DMD sometimes uses about 1GB of memory,
 sometimes about 2GB of memory and sometimes it makes my OS crash.
 
 Can anyone reproduce this?


 
 Similarly this:
 
 template TT(string s){enum TT=s;}
 
 void main(){
 mixin({
 string r;
 foreach(i;0..12000) r~=q{{enum x=TT!"";};};
 return r;
 }());
 }
 
 That does use a somewhat smaller amount of memory, but it is still
 huge
 and non-deterministic in size.


 
 I guess issue 6498, i.e. it's CTFE doing memory allocation on each
 append in foreach loop.
 http://d.puremagic.com/issues/show_bug.cgi?id=6498


 
 Well, I don't get the issue if I don't mixin mixins or template
 instantiations.
 
 void main(){
      mixin({
              string r;
              foreach(i;0..12000) r~="{}{}{}{}{}{}{}{}";
 
              return r;
          }());
 }
 

 As for as non-determinism, maybe heap fragmentation?


 
 That might well be it. Thanks!


Excessive memory consumption during compilation does not surprise me. While 
some bugs in that area have been fixed, there are still several issues with dmd 
using way too much memory (one of the main issues being that it never actually 
releases any of it). The non-determinism, however, is a bit odd. Heap 
fragmentation may be a good explanation though. I don't know.

- Jonathan M Davis
Sep 09 2011