www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.internals - DMD with GC

reply RazvanN <razvan.nitu1305 gmail.com> writes:
Hi all,

Currently the compiler is not garbage collected due to the fact 
that the GC slows it down. This means that at some point you 
could enable the GC, while at the moment if you do that you get 
ugly segfaults (see [1]). I am interested in this because the 
compiler library should have the possibility of being garbage 
collected and I am curios if somebody knows what was the change 
which made the compiler incompatible with the GC.

Thanks,
RazvanN

[1] 
http://forum.dlang.org/thread/dywncjvlezvrbjkfkbsz forum.dlang.org
Nov 07 2017
next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 07.11.2017 10:04, RazvanN wrote:
 Hi all,
 
 Currently the compiler is not garbage collected due to the fact that the 
 GC slows it down. This means that at some point you could enable the GC, 
 while at the moment if you do that you get ugly segfaults (see [1]). I 
 am interested in this because the compiler library should have the 
 possibility of being garbage collected and I am curios if somebody knows 
 what was the change which made the compiler incompatible with the GC.
 
 Thanks,
 RazvanN
 
 [1] http://forum.dlang.org/thread/dywncjvlezvrbjkfkbsz forum.dlang.org
I recently built a semantic server for Visual D from the dmd front end with the GC enabled, and it didn't crash AFAICT. I don't think I had to make any patch but the one to make it compile (https://github.com/dlang/dmd/pull/7156), but you have to stop the GC before calling the backend and actually generate code. It has some other issues though prohibiting it to be a viable solution for an IDE ATM: - the allocation through the rmem module has the disadvantage that type information is lost, so that every block has to be scanned conservatively, even if it never contains pointers. This makes it more likely to have false pointers. - there are a number of (private or function static) global variables that are populated during compilation, and it's not simple (though possible) to reset them to restart the analysis - it eats memory like crazy. One problem I identified is that there are a number of pools that allocate memory in large chunks that continue to contain old references. - the semantic analysis does quite a bit of transformations that make it difficult to match the AST to the source. Worst of all: the error propagation prunes complete sub trees of the AST, so there is no information to lookup while the code doesn't compile. This prohibits any help while editing, e.g. code completion or tool tips with inferred type information. So it worked for displaying semantic errors during editing (Visual D currently only displays parsing errors), but it's not really fast. It would be nice to have some reasonable dependency tracking so that a complete analysis doesn't have to be redone with every edit.
Nov 07 2017
parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Tuesday, 7 November 2017 at 21:14:07 UTC, Rainer Schuetze 
wrote:
 I recently built a semantic server for Visual D from the dmd 
 front end with the GC enabled, and it didn't crash AFAICT. I 
 don't think I had to make any patch but the one to make it 
 compile (https://github.com/dlang/dmd/pull/7156), but you have 
 to stop the GC before calling the backend and actually generate 
 code.
I tried to compile the parser as a library with the GC disabled, but running the example in dmd/src/examples/test_parser.d results in a segmentation fault. Running with the GC disabled completes successfully, so I guess there still are some corner cases, though I am noob when it comes to GCed dmd. The patch you mentioned does not fix the case presented in [1]. [1] http://forum.dlang.org/thread/dywncjvlezvrbjkfkbsz forum.dlang.org
Nov 08 2017
parent reply Jacob Carlborg <doob me.com> writes:
On 2017-11-09 07:54, RazvanN wrote:

 I tried to compile the parser as a library with the GC disabled,
 but running the example in dmd/src/examples/test_parser.d results
 in a segmentation fault. Running with the GC disabled completes
 successfully, so I guess there still are some corner cases, though
 I am noob when it comes to GCed dmd. The patch you mentioned does not
 fix the case presented in [1].
 
 [1] http://forum.dlang.org/thread/dywncjvlezvrbjkfkbsz forum.dlang.org
Looks like the problem occurs when importing the ddmd.root.rmem [1] module, which is basically always done one way or other. This module will override some of the D runtime functions [2], which somehow causes a conflict with GC. When the module defines these functions [2] they will end up in the object file for the module. When that happens the linker won't search for these symbols in any external libraries, i.e. druntime (statically linked into Phobos), which it normally does otherwise. The solution is available in that module, compile with the "GC" version enabled, i.e. "dmd -version=GC". [1] https://github.com/dlang/dmd/blob/master/src/ddmd/root/rmem.d [2] https://github.com/dlang/dmd/blob/master/src/ddmd/root/rmem.d#L195 -- /Jacob Carlborg
Nov 09 2017
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 09.11.2017 09:20, Jacob Carlborg wrote:
 On 2017-11-09 07:54, RazvanN wrote:
 
 I tried to compile the parser as a library with the GC disabled,
 but running the example in dmd/src/examples/test_parser.d results
 in a segmentation fault. Running with the GC disabled completes
 successfully, so I guess there still are some corner cases, though
 I am noob when it comes to GCed dmd. The patch you mentioned does not
 fix the case presented in [1].

 [1] http://forum.dlang.org/thread/dywncjvlezvrbjkfkbsz forum.dlang.org
Looks like the problem occurs when importing the ddmd.root.rmem [1] module, which is basically always done one way or other. This module will override some of the D runtime functions [2], which somehow causes a conflict with GC. When the module defines these functions [2] they will end up in the object file for the module. When that happens the linker won't search for these symbols in any external libraries, i.e. druntime (statically linked into Phobos), which it normally does otherwise. The solution is available in that module, compile with the "GC" version enabled, i.e. "dmd -version=GC". [1] https://github.com/dlang/dmd/blob/master/src/ddmd/root/rmem.d [2] https://github.com/dlang/dmd/blob/master/src/ddmd/root/rmem.d#L195
Yes, that's what I did, too: compile the dmd frontend with -version=GC. IIRC I could also compile phobos unittests with the GC version (it was just 10-20% slower), but building them all with a single dmd invocation crashed during code generation (which has to run with the GC disabled). AFAICT it got further than the regular version but probably I didn't have enough swap space when the process needed more than 16 GB).
Nov 09 2017
prev sibling parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 7 November 2017 at 09:04:36 UTC, RazvanN wrote:
 Hi all,

 Currently the compiler is not garbage collected due to the fact 
 that the GC slows it down. This means that at some point you 
 could enable the GC, while at the moment if you do that you get 
 ugly segfaults (see [1]). I am interested in this because the 
 compiler library should have the possibility of being garbage 
 collected and I am curios if somebody knows what was the change 
 which made the compiler incompatible with the GC.
Seeing this late, but the dmd compiler itself was never a GC'ed program. The frontend was written in C++ then mostly automatically translated to D, and the old manual allocation scheme was kept. To make it actually work with GC for the first time would probably require some work, same for your library. The good news is your library is nogc for free! ;)
Dec 19 2017