• RazvanN (66/66) May 24 2017 After I compiled the parser as a library I wanted to test it to
• Mathias Lang (2/6) May 24 2017
• RazvanN (3/10) May 25 2017 Thank you. It does, indeed, solve the problem. But I am curious,
• Mathias Lang (7/18) May 26 2017 DMD uses a mix of allocations strategy.

RazvanN <razvan.nitu1305 gmail.com> writes:

After I compiled the parser as a library I wanted to test it to 
see if it successfully parses the phobos code. The code I am 
using is the following:


// test_parser.d
import std.stdio;
import std.file;

import ddmd.parse;
import ddmd.astbase;

import ddmd.id;
import ddmd.globals;

void main()
{
     string path = "../../phobos/std/";
     string regex = "*.d";

     // parser initializations
     Id.initialize();
     global._init();
     global.params.isLinux = true;
     global.params.is64bit = (size_t.sizeof == 8);
     ASTBase.Type._init();

     // get recursively all the .d files
     auto dFiles = dirEntries(path, regex, SpanMode.depth);
     foreach (d; dFiles)
     {
         writeln("Processing:", d.name);

         // comment the following line and it works
         string content = readText(d.name);

         scope p = new Parser!ASTBase(null, content, false);
         p.nextToken();
         p.parseModule();

         writeln("Finished!");
     }
}

This code fails with segmentation fault. The dFiles iterator gets 
broken somehow. If the line which reads a file is commented, than 
the code works. Also, if the parser imports are commented 
(ddmd.astbase and ddmd.parse) and all the lines of code which use 
code from those modules are commented, it also works.

I compiled the lexer and the parser with the following targets:


$G/lexer.a: $(LEXER_SRCS) $(LEXER_ROOT)
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -lib -of$  $(MODEL_FLAG) 
-I/tmp/.host_dmd-2.072.2/dmd2/src/druntime/import/ -J$G 
-L-lstdc++ $(DFLAGS) $(LEXER_SRCS) $(LEXER_ROOT)


$G/parser.a: $(PARSER_SRCS) $G/lexer.a $(ROOT_SRCS)
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -lib -of$  $(MODEL_FLAG) 
-I/tmp/.host_dmd-2.072.2/dmd2/src/druntime/import/ -J$G 
-L-lstdc++ $(DFLAGS) $(PARSER_SRCS) $G/lexer.a $(ROOT_SRCS) 
-Iddmd -J../res/

Then, the test file is compiled as follows:

parser_test: $G/parser.a test_parser.d
»   CC=$(HOST_CXX) $(HOST_DMD_RUN) -of$  $(MODEL_FLAG) -vtls -J$G 
-L-lstdc++ $(DFLAGS) -I$G $G/lexer.a $G/parser.a test_parser.d 
-Iddmd -J../res/

I thought it may be a mismatch between the versions of druntime 
used for the compilation of the lexer and parser libraries and 
the one used for phobos. So I recompiled druntime (master->head) 
with the latest stable dmd (2.072.2) and then compiled phobos 
(master->head) with the aforementioned dmd and druntime, still I 
get the segfault.

I also tried to compile the lexer and the parser using -conf= 
-defaultlib= -debuglib= and then I tried to pass the path to 
druntime with -I when compiling test_parser.d, but running the 
program also ends with segmentation fault.

Does anyone have any idea what the problem might be?

Thank you,
RazvanN

Mathias Lang <mathias.lang sociomantic.com> writes:

On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
 Does anyone have any idea what the problem might be?

 Thank you,
 RazvanN

Since my emails don't seem to go through in a timely fashion:
 You should also disable the GC.

RazvanN <razvan.nitu1305 gmail.com> writes:

On Wednesday, 24 May 2017 at 10:57:40 UTC, Mathias Lang wrote:
 On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
 Does anyone have any idea what the problem might be?

 Thank you,
 RazvanN

 Since my emails don't seem to go through in a timely fashion:
 You should also disable the GC.


Thank you. It does, indeed, solve the problem. But I am curious, 
why was it segfaulting?

Mathias Lang <mathias.lang sociomantic.com> writes:

On Thursday, 25 May 2017 at 16:48:00 UTC, RazvanN wrote:
 On Wednesday, 24 May 2017 at 10:57:40 UTC, Mathias Lang wrote:
 On Wednesday, 24 May 2017 at 10:40:42 UTC, RazvanN wrote:
 Does anyone have any idea what the problem might be?

 Thank you,
 RazvanN

 Since my emails don't seem to go through in a timely fashion:
 You should also disable the GC.


 Thank you. It does, indeed, solve the problem. But I am 
 curious, why was it segfaulting?

DMD uses a mix of allocations strategy.

My guess is that some references to GC-allocated memory are 
stored into non GC-allocated memory, and those chunks of memory 
are not added as GC roots. So when the GC decides to collect 
memory, it thinks those objects are dead and recycle them. From 
there, you end up with a dangling pointer.