www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - segfault

reply spir <denis.spir gmail.com> writes:
Hello D programmers,


I'm blocked by a mysterious segfault I seem to be unable to diagnose. There=
 is probably some point of D I have not yet understood. Below relevant piec=
e of code; "***" marks debug instructions.

The constructor works fine, assertions pass, and writeln writes:
    ([0-9]+ ("+" [0-9]+)*)
But when I call the method 'check', writeln writes
    (null null)
Which means that this.pattern's sub-patterns have disappeared. See line con=
structing this.pattern in constructor to have an idea (I even tried to repl=
ace sep & element by this.sep & this.element, but this does not help).

When matching with this.pattern later in check(), I get a segfault because =
the code tries to recursively match its sub-patterns (the null items).


Denis


class List : Pattern {
    // ...
    Pattern element;
    Pattern sep;
    Tuple pattern;
    uint min;
   =20
    this (Pattern element, Pattern sep, uint min=3D2) {
        this.min =3D min;
        // for output
        this.element =3D element;
        this.sep =3D sep;
        // construct pattern
        this.pattern =3D new Tuple(
            element,=20
            new ZeroOrMore(new Tuple(sep,element))
            );
        assert ((this.element !is null) && (this.sep !is null));    // ***
        assert (this.pattern !is null);                             // ***
        writeln(this.pattern);                                      // ***
    }
   =20
    override Result check (Source source) {
        assert (this.pattern !is null);                             // ***
        writeln(this.pattern);                                      // ***
        // ...

-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 10 2010
next sibling parent "Daniel Murphy" <yebblies nospamgmail.com> writes:
What happens if you make pattern const?
Nov 10 2010
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 10 Nov 2010 07:13:21 -0500, spir <denis.spir gmail.com> wrote:


 Hello D programmers,


 I'm blocked by a mysterious segfault I seem to be unable to diagnose.  
 There is probably some point of D I have not yet understood. Below  
 relevant piece of code; "***" marks debug instructions.


This is not enough code to understand the problem.  For instance,  
writefln(pattern) prints a class, but we don't see Pattern's toString  
function.  We don't see the code that creates the object and then calls  
check.  Maybe something happens between calls?

It's impossible to diagnose something like this without a working example,  
so you need to trim it down to something that still compiles and fails,  
and then share that entire code.

-Steve
Nov 11 2010
next sibling parent spir <denis.spir gmail.com> writes:
On Thu, 11 Nov 2010 07:47:01 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:


 On Wed, 10 Nov 2010 07:13:21 -0500, spir <denis.spir gmail.com> wrote:
=20

 Hello D programmers,


 I'm blocked by a mysterious segfault I seem to be unable to diagnose. =




=20

 There is probably some point of D I have not yet understood. Below =20
 relevant piece of code; "***" marks debug instructions.


=20
 This is not enough code to understand the problem.  For instance, =20
 writefln(pattern) prints a class, but we don't see Pattern's toString =20
 function.  We don't see the code that creates the object and then calls =


=20

 check.  Maybe something happens between calls?
=20
 It's impossible to diagnose something like this without a working example=


, =20

 so you need to trim it down to something that still compiles and fails, =


=20

 and then share that entire code.
=20
 -Steve


All right, I'll try to do that. It'll be difficult, but I need to progress =
anyway.

Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 11 2010
prev sibling parent reply spir <denis.spir gmail.com> writes:
On Thu, 11 Nov 2010 07:47:01 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:


 This is not enough code to understand the problem.  For instance,  
 writefln(pattern) prints a class, but we don't see Pattern's toString  
 function.  We don't see the code that creates the object and then calls  
 check.  Maybe something happens between calls?
 
 It's impossible to diagnose something like this without a working example,  
 so you need to trim it down to something that still compiles and fails,  
 and then share that entire code.
 
 -Steve


Hello,

I tried to build a example mini-app to show the issue more simply, but
couldn't. The point is the bug only shows when using the most complicated
element of my code (class List), built on top of ~ 1000 lines. I guess the best
is to try to explain the case and provide example test. See code of the class
and test case below.

This is a PEG matching lib. Patterns and whole grammars are written in plain D,
eg:
    DOT = new Character('.');
    digits = new String(new Klass("0-9"));
    integer = new Tuple(digits, DOT, digits);
There are about 15 little test suites running fine for all bases (Node types,
Exceptions, Pattern), utilities, and every Pattern type, including 3 other
helpers like List.

List is a helper for matching lists of separated items -- a very common &
annoying case. It does not only build the actual pattern, but also extracts the
elements out of the mess for you (see embedded doc).

Its actual pattern is constructed in this(), as a Tuple formed of a sub-pattern
matching the first element, and a ZeroOrMore repetition matching more (sep
pattern) Tuples (see this()). The test case is as follows:
    auto number = new String(new Klass("0-9"));
    auto PLUS = new Literal("+");
    auto addition = new List(number, PLUS);
Writing out the constructed pattern attribute at the end of this() correctly
gives:
    pattern: ([0-9]+ ("+" [0-9]+)*)

(Since a List's actual pattern is an instance of Tuple, what is shown here is
the result of Tuple's toString(), which itself uses a tool func listText() -->
code below after the class and test case.)

Using a List instance (here 'addition') is calling an operational method, of
which only 'match' is implemented yet (in Pattern). Match does little stuff
more that delegating to check(source) (specialised in List). If I then write
out the pattern from check(), I read:
    pattern: (null null)
Thus, no surprise that I get a segfault when matching, since the top pattern
will delegate to null subpatterns.

But strangely enough, I also get a segfault by simply typing
"writeln(addition.pattern)" in the test func.

I can also post the whole module somewhere online if needed.

PS: Last minute! I just retried to write out the pattern from inside check().
This time, instead of "(null null)", I get "(DeeMatch.List DeeMatch.Source)".
???
(DeeMatch is the temp. module name, so it's 2 class names.)


Thank you,
Denis


==================== test func ======================
void testList2 () {
    writeln("=== List ========================");
    auto number = new String(new Klass("0-9"));
    auto PLUS = new Literal("+");
    auto addition = new List(number, PLUS);
    
    // This check works fine:
    writeln(addition);          // --> List([0-9]+, "+", 2)
    
    // But if I uncomment the following: segfault
    // writeln(addition.pattern);
    
    // If I run a test as is, I get a segfault when
    // accessing one of addition's patterns subpattern.
    // Trick to make code execute w/o segfault:
    // redefine addition.pattern by hand
    addition.pattern = new Tuple(
        number, 
        new ZeroOrMore(new Tuple(PLUS,number))
    );
    
    // Then, I can use addition: 
    auto node = addition.match("1+23+456");
    assert (node.toString() == `["1" "23" "456"]`);
    // instead of: `("1" [("+" "23") ("+" "456")])`
}

==================== List class =====================
class List : Pattern {
    /**  pattern type matching a list of separated elements
        Example:
            moreElements = ZeroOrMore(Tuple(sep, element));
            list = Tuple(element, moreElements);
            --> (e1 ((s e2) (s e3)))
        Using List instead works as follows:
            list = List(element, sep);
            --> (e1 e2 e3)
        Parameter min defines minimal number of elements
        (typically 0 1 or 2). */
    
    static string typename = "List";
    static string errorMessageForm = 
        "Cannot find at least %s elements for List pattern %s.";
    
    Pattern element;
    Pattern sep;
    uint min;
    Tuple pattern;
    
    this (Pattern element, Pattern sep, uint min=2) {
        this.min = min;
        // for output
        this.element = element;
        this.sep = sep;
        // Construct pattern.
        this.pattern = new Tuple(
            element, 
            new ZeroOrMore(new Tuple(sep,element))
        );
        writeln(this.pattern);
    }
    
    // operational
    override Result check (Source source) {
        /**  Match text and form a list of elements. */
        // Try & match pattern.
        // Note: match mail fail if min==0
        Result result;
        TupleNode node;
        try {
            result = this.pattern.check(source);
            node = cast(TupleNode)(result.node);
        } catch (MatchError e) {
            if (this.min == 0) {
                auto zeroNode = new ListNode(this, []);
                return new Result(zeroNode, source);
            }
            else
                throw e;
        }
        
        // Extract element nodes.
        // First, get & record first element.
        writeln("*** ", node);
        Node firstElement = node.nodes[0];      // first element node
        Node[] listNodes = [firstElement];      // actual result!
        
        // Then, get to list of additional elements.
        auto nodeOfPairs = cast(ListNode)(node.nodes[1]);
        TupleNode[] pairNodes = cast(TupleNode[])(nodeOfPairs.nodes);
        
        // Extract additional elements
        if (pairNodes.length > 0) {
            foreach (TupleNode pairNode ; pairNodes)    // (sep,element) pairs
                listNodes ~= pairNode.nodes[1];
        }
        
        // Check number of elements.
        if (listNodes.length < this.min) {
            string message = format(this.errorMessageForm, this.min, this);
            throw new MatchFailure(this, source, message);
        }
        
        // Construct result node.
        auto listNode = new ListNode(this, listNodes);
        return new Result(listNode, source);
    }
    
    // feedback
    override string toString () {
        /** "List(element, sep, min)" */
        return format("List(%s, %s, %s)", this.element,this.sep,this.min);
    }
}

==================== Tuple toString() =======================
    override string toString () {
        /** "(p1 p2 ...)" */
        return listText!Pattern(this.patterns, " " ,"(",")");
    }
=== using tool func
string listText(Element) (Element[] elements,
    string sep, string lDelim,string rDelim) {
    /** textual representation of elements held in a plain array */
    string[] elementTexts = new string[elements.length];
    foreach (uint i, Element element ; elements)
        elementTexts[i] = to!string(element);
    string content = join(elementTexts, sep);
    return format("%s%s%s", lDelim, content, rDelim);
}
Nov 11 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 11 Nov 2010 10:34:41 -0500, spir <denis.spir gmail.com> wrote:


 On Thu, 11 Nov 2010 07:47:01 -0500
 "Steven Schveighoffer" <schveiguy yahoo.com> wrote:


 This is not enough code to understand the problem.  For instance,
 writefln(pattern) prints a class, but we don't see Pattern's toString
 function.  We don't see the code that creates the object and then calls
 check.  Maybe something happens between calls?

 It's impossible to diagnose something like this without a working  
 example,
 so you need to trim it down to something that still compiles and fails,
 and then share that entire code.

 -Steve


 Hello,

 I tried to build a example mini-app to show the issue more simply, but  
 couldn't. The point is the bug only shows when using the most  
 complicated element of my code (class List), built on top of ~ 1000  
 lines. I guess the best is to try to explain the case and provide  
 example test. See code of the class and test case below.


This gives a little more information.  From your code, it appears that the  
constructor has this line:

writeln(this.pattern);

Which succeeds.  Then your test code looks like this:


      auto addition = new List(number, PLUS);
     // This check works fine:
      writeln(addition);          // --> List([0-9]+, "+", 2)
     // But if I uncomment the following: segfault
      // writeln(addition.pattern);

Which means between constructor (first line in test code), you printed the  
item itself, then print the item's pattern field.

What it looks like to me is that between printing it in the constructor,  
and printing it outside the constructor, something bad happened.  There  
are several pieces of code running that I don't have the source for, so I  
can assume either you have a compiler bug, or it's in one of those pieces.

Try this: inside the constructor do:

writeln(this.pattern);
writeln(this);
writeln(this.pattern);

And see if that fails.  If that fails, then we know it's one of those  
three lines thats messing things up.

Comment out writeln(this), and see if it still fails.  If it does, then  
it's something being printed in your Tuple class that is causing it to  
corrupt something so it can't be printed again.  If not, then it's  
something being printed in your List class.  I don't have the code that  
converts your Pattern class to a string, etc. so I can't say whether they  
are an issue.

If those three lines work in the constructor, I'd say there is a possible  
compiler error, because there is no difference between calling that code  
 from in the constructor or in the test code.

If that's the case, create a function that prints out all the addresses of  
things (you can print out the address of a class by simply casting the  
class reference to a void *).  Print that out between each line and see  
what changes, then focus there.  Again, without more code, I can't say for  
certain what's going on.

If you have a debugger, it might help even more than these little tricks.

-Steve
Nov 11 2010
parent reply spir <denis.spir gmail.com> writes:
On Thu, 11 Nov 2010 10:52:50 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:


 On Thu, 11 Nov 2010 10:34:41 -0500, spir <denis.spir gmail.com> wrote:
=20

 On Thu, 11 Nov 2010 07:47:01 -0500
 "Steven Schveighoffer" <schveiguy yahoo.com> wrote:


 This is not enough code to understand the problem.  For instance,
 writefln(pattern) prints a class, but we don't see Pattern's toString
 function.  We don't see the code that creates the object and then calls
 check.  Maybe something happens between calls?

 It's impossible to diagnose something like this without a working =20
 example,
 so you need to trim it down to something that still compiles and fails,
 and then share that entire code.

 -Steve


 Hello,

 I tried to build a example mini-app to show the issue more simply, but =




=20

 couldn't. The point is the bug only shows when using the most =20
 complicated element of my code (class List), built on top of ~ 1000 =20
 lines. I guess the best is to try to explain the case and provide =20
 example test. See code of the class and test case below.




Thank you very much for your help, Steve. I think now the bug is not in my =
code. An evidence is I never play with memory: every element is a class ins=
tance (many should later become structs or even simpler thingies, but I fir=
st chose this design for simplicity); there is no malloc or such, not even =
a single pointer; and indeed, I do not deallocate anything manually, and as=
sign nothing to null. So, how come a segfault? See also below.


 This gives a little more information.  From your code, it appears that th=


e =20

 constructor has this line:
=20
 writeln(this.pattern);
=20
 Which succeeds.  Then your test code looks like this:
=20
=20
       auto addition =3D new List(number, PLUS);
      // This check works fine:
       writeln(addition);          // --> List([0-9]+, "+", 2)
      // But if I uncomment the following: segfault
       // writeln(addition.pattern);
=20
 Which means between constructor (first line in test code), you printed th=


e =20

 item itself, then print the item's pattern field.
=20
 What it looks like to me is that between printing it in the constructor, =


=20

 and printing it outside the constructor, something bad happened.


Exactly. But as you can see above, there is no code running in between. The=
 construtor does only what you've read in the piece of code posted in last =
message (& the superclass has no this(), it's used for "genericity" and som=
e interface methods to the user).
The pattern (([0-9]+ ("+" [0-9]+)*)) becomes (null null), or other meaningl=
ess thing, when the constructor returns. This expression means actually a T=
uple of 2 sub-patterns. these sub-patterns exist inside this, and are corre=
ct, but then disppear.


  There =20
 are several pieces of code running that I don't have the source for, so I=


 =20

 can assume either you have a compiler bug, or it's in one of those pieces.


Nothing else happens.The constructor records data, builds the pattern and r=
eturns.


 Try this: inside the constructor do:
=20
 writeln(this.pattern);
 writeln(this);
 writeln(this.pattern);


Runs fine, see also below. I'm sure now that:
* the pattern is ok as long as we remain inside this()
* the pattern is corrupted as soon as we quit this()
(Actually, its contents are corrupted, precisely its field 'pattern', an ar=
ray that holds its subpatterns.)


 [...]
 If not, then it's =20
 something being printed in your List class.  I don't have the code that =


=20

 converts your Pattern class to a string, etc. so I can't say whether they=


 =20

 are an issue.


I printed it in last post (but was at the end). It's just a call to a tool =
func listText that wrap join()Here it is again:

class Tuple : Pattern {
    // ........
    override string toString () {
        /** "(p1 p2 ...)" */
        return listText!Pattern(this.patterns, " " ,"(",")");
    }
}

string listText(Element) (Element[] elements,
        string sep, string lDelim,string rDelim) {
    /** textual representation of elements held in a plain array */
    string[] elementTexts =3D new string[elements.length];
    foreach (uint i, Element element ; elements)
        elementTexts[i] =3D to!string(element);
    string content =3D join(elementTexts, sep);
    return format("%s%s%s", lDelim, content, rDelim);
}


 If those three lines work in the constructor, I'd say there is a possible=


 =20

 compiler error, because there is no difference between calling that code =


=20

  from in the constructor or in the test code.


I agree with your reasoning. (But would not dare concluding there is a bug =
in dmd :-)


 If that's the case, create a function that prints out all the addresses o=


f =20

 things (you can print out the address of a class by simply casting the =20
 class reference to a void *).  Print that out between each line and see =


=20

 what changes, then focus there.  Again, without more code, I can't say fo=


r =20

 certain what's going on.


Here it is: I added some debug code to the constructor and the test func. B=
elow code and output. The segfault happens here on call to match. I added n=
ot only the address of the pattern, but also of its .patterns field, and of=
 its 2 sub-patterns.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D List.this =3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
    this (Pattern element, Pattern sep, uint min=3D2) {
        this.min =3D min;
        // for output
        this.element =3D element;
        this.sep =3D sep;
        // Construct pattern.
        this.pattern =3D new Tuple(
            element,=20
            new ZeroOrMore(new Tuple(sep,element))
        );
        // checking
        writeln("--- in List.this ---");
        writeln(this.pattern);
        writeln(this);
        writeln(this.pattern);
        writeln(cast(void*)(this));
        writeln(cast(void*)(this.pattern));
        writeln(cast(void*)(this.pattern.patterns));    // array
        writeln(cast(void*)(this.pattern.patterns[0]));
        writeln(cast(void*)(this.pattern.patterns[1]));
    }
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D testList2 =3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
void testList2 () {
    writeln("=3D=3D=3D List =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D");
    auto number =3D new String(new Klass("0-9"));
    auto PLUS =3D new Literal("+");
    auto addition =3D new List(number, PLUS);
   =20
    writeln("--- in testList2 ---");
    writeln(addition);
    writeln(cast(void*)(addition));
    writeln(cast(void*)(addition.pattern));
    writeln(cast(void*)(addition.pattern.patterns));    // array
    writeln(cast(void*)(addition.pattern.patterns[0]));
    writeln(cast(void*)(addition.pattern.patterns[1]));
   =20
    // use
    auto node =3D addition.match("1+23+456");
    assert (node.toString() =3D=3D `["1" "23" "456"]`);
}
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D output =3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- in List.this ---
([0-9]+ ("+" [0-9]+)*)
List([0-9]+, "+", 2)
([0-9]+ ("+" [0-9]+)*)
46DF40
46ECA0
BFA107E0
46ECE0
46EC80
--- in testList2 ---
List([0-9]+, "+", 2)
46DF40
46ECA0
BFA107E0
BFA1070A
38C4E0
Segmentation fault


As you see, the addresses of the 2 sub-patterns have changed (but the array=
 that holds them hasn't moved -- and I just checked that cast(void*)array a=
ctually returns the ptr). Don't know what to conclude.


Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 11 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 11 Nov 2010 14:50:10 -0500, spir <denis.spir gmail.com> wrote:


 Thank you very much for your help, Steve. I think now the bug is not in  
 my code. An evidence is I never play with memory: every element is a  
 class instance (many should later become structs or even simpler  
 thingies, but I first chose this design for simplicity); there is no  
 malloc or such, not even a single pointer; and indeed, I do not  
 deallocate anything manually, and assign nothing to null. So, how come a  
 segfault? See also below.


You're welcome, and there are other ways to have issues with pointers.   
The most common is not deleting something from the heap, but rather  
returning stack memory from a function.


 Try this: inside the constructor do:

 writeln(this.pattern);
 writeln(this);
 writeln(this.pattern);


 Runs fine, see also below. I'm sure now that:
 * the pattern is ok as long as we remain inside this()
 * the pattern is corrupted as soon as we quit this()
 (Actually, its contents are corrupted, precisely its field 'pattern', an  
 array that holds its subpatterns.)


This really sounds like an issue where stack data is being returned.


 [...]
 If not, then it's
 something being printed in your List class.  I don't have the code that
 converts your Pattern class to a string, etc. so I can't say whether  
 they
 are an issue.


 I printed it in last post (but was at the end). It's just a call to a  
 tool func listText that wrap join()Here it is again:

 class Tuple : Pattern {
     // ........
     override string toString () {
         /** "(p1 p2 ...)" */
         return listText!Pattern(this.patterns, " " ,"(",")");
     }
 }

 string listText(Element) (Element[] elements,
         string sep, string lDelim,string rDelim) {
     /** textual representation of elements held in a plain array */
     string[] elementTexts = new string[elements.length];
     foreach (uint i, Element element ; elements)
         elementTexts[i] = to!string(element);
     string content = join(elementTexts, sep);
     return format("%s%s%s", lDelim, content, rDelim);
 }


It's the to!string(element) that I don't have the code to -- each object's  
toString function.  But I'm no longer worried about that.  If it works  
properly inside the ctor, it should work properly outside.


 If those three lines work in the constructor, I'd say there is a  
 possible
 compiler error, because there is no difference between calling that code
  from in the constructor or in the test code.


 I agree with your reasoning. (But would not dare concluding there is a  
 bug in dmd :-)


It's not unheard of :)  But I still would rule out all other possibilities  
before concluding that.  I've had very strange dmd bugs where things only  
fail when certain ordering of structures or code affects them.  It does  
happen.


 If that's the case, create a function that prints out all the addresses  
 of
 things (you can print out the address of a class by simply casting the
 class reference to a void *).  Print that out between each line and see
 what changes, then focus there.  Again, without more code, I can't say  
 for
 certain what's going on.


 Here it is: I added some debug code to the constructor and the test  
 func. Below code and output. The segfault happens here on call to match.  
 I added not only the address of the pattern, but also of its .patterns  
 field, and of its 2 sub-patterns.

 ============== List.this ======================
     this (Pattern element, Pattern sep, uint min=2) {
         this.min = min;
         // for output
         this.element = element;
         this.sep = sep;
         // Construct pattern.
         this.pattern = new Tuple(
             element,
             new ZeroOrMore(new Tuple(sep,element))
         );
         // checking
         writeln("--- in List.this ---");
         writeln(this.pattern);
         writeln(this);
         writeln(this.pattern);
         writeln(cast(void*)(this));
         writeln(cast(void*)(this.pattern));
         writeln(cast(void*)(this.pattern.patterns));    // array
         writeln(cast(void*)(this.pattern.patterns[0]));
         writeln(cast(void*)(this.pattern.patterns[1]));
     }
 ============== testList2 ======================
 void testList2 () {
     writeln("=== List ========================");
     auto number = new String(new Klass("0-9"));
     auto PLUS = new Literal("+");
     auto addition = new List(number, PLUS);
    writeln("--- in testList2 ---");
     writeln(addition);
     writeln(cast(void*)(addition));
     writeln(cast(void*)(addition.pattern));
     writeln(cast(void*)(addition.pattern.patterns));    // array
     writeln(cast(void*)(addition.pattern.patterns[0]));
     writeln(cast(void*)(addition.pattern.patterns[1]));
    // use
     auto node = addition.match("1+23+456");
     assert (node.toString() == `["1" "23" "456"]`);
 }
 ============== output ======================
 --- in List.this ---
 ([0-9]+ ("+" [0-9]+)*)
 List([0-9]+, "+", 2)
 ([0-9]+ ("+" [0-9]+)*)
 46DF40
 46ECA0
 BFA107E0
 46ECE0
 46EC80
 --- in testList2 ---
 List([0-9]+, "+", 2)
 46DF40
 46ECA0
 BFA107E0
 BFA1070A
 38C4E0
 Segmentation fault


 As you see, the addresses of the 2 sub-patterns have changed (but the  
 array that holds them hasn't moved -- and I just checked that  
 cast(void*)array actually returns the ptr). Don't know what to conclude.


OK, can we see the code for Tuple?  That may be where something fishy is  
occurring.

-Steve
Nov 11 2010
parent reply spir <denis.spir gmail.com> writes:
On Thu, 11 Nov 2010 15:11:34 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:


 As you see, the addresses of the 2 sub-patterns have changed (but the =




=20

 array that holds them hasn't moved -- and I just checked that =20
 cast(void*)array actually returns the ptr). Don't know what to conclude.
 =20


=20
 OK, can we see the code for Tuple?  That may be where something fishy is =


=20

 occurring.


Well, since the pattern is OK _after_ call to Tuple's constructor (which do=
es nothing more than recording its sub-patterns, see below) and only gets w=
rong when qutting this(), I fail to see how Tuple could be cause of anythin=
g. Also, the corruption is visible _before_ calling match() (which here del=
egates to Tuple's check()).

I understand your note about returning an object actually allocated on the =
stack -- but here there are only implicitely referenced objects (class inst=
ances). This would mean that D creates the 2 sub-patterns on the stack? But=
 why those objects, precisely? (Also note that they are of different classe=
s: one is here a "String", the other a "ZeroOrMore"). They are stored in an=
 array.

What's troubling is that the array elements, meaning the supposed subpatter=
n addresses, have changed. Maybe the patterns themselves are still ok, but =
the array data only are corrupted?
Oh, I may try to cast to String the memory area pointed inside this()......=
. Does not seem to work: I recorded the pointer read in this() (as String*)=
 into a global; then in the test func:
    writeln("p: ",p);       	// ok, same address as in this()
    writeln(cast(String)(*p));	// segfault!

Anyway, just in case my reasoning is wrong, here is the whole Tuple class:

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
class Tuple : Pattern {
    /**  pattern type for tuple of given patterns */
    static string typename =3D "Tuple";
    Pattern[] patterns;
    this (Pattern[] patterns...) {
        this.patterns =3D patterns;
    }
    // operational
    override Result check (Source source) {
        /**  Check whether text holds successive matches of all patterns. */
        // try & match
        // case success: rest text slice is given by each match result
        // case failure: MatchFailure error is thrown by sub-pattern
        Node[] nodes;
        Result result;
        foreach (Pattern pattern ; patterns) {
            result =3D pattern.check(source);
            nodes ~=3D result.node;
            source =3D result.source;
        }
        auto node =3D new TupleNode(this, nodes);
        return new Result(node, source);
    }
    // feedback
    override string toString () {
        /** "(p1 p2 ...)" */
        return listText!Pattern(this.patterns, " " ,"(",")");
    }
}


Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 11 2010
parent reply =?UTF-8?B?UGVsbGUgTcOlbnNzb24=?= <pelle.mansson gmail.com> writes:
On 11/11/2010 10:15 PM, spir wrote:


 Well, since the pattern is OK _after_ call to Tuple's constructor (which does
nothing more than recording its sub-patterns, see below) and only gets wrong
when qutting this(), I fail to see how Tuple could be cause of anything. Also,
the corruption is visible _before_ calling match() (which here delegates to
Tuple's check()).

 I understand your note about returning an object actually allocated on the
stack -- but here there are only implicitely referenced objects (class
instances). This would mean that D creates the 2 sub-patterns on the stack? But
why those objects, precisely? (Also note that they are of different classes:
one is here a "String", the other a "ZeroOrMore"). They are stored in an array.

 What's troubling is that the array elements, meaning the supposed subpattern
addresses, have changed. Maybe the patterns themselves are still ok, but the
array data only are corrupted?
 Oh, I may try to cast to String the memory area pointed inside this().......
Does not seem to work: I recorded the pointer read in this() (as String*) into
a global; then in the test func:
      writeln("p: ",p);       	// ok, same address as in this()
      writeln(cast(String)(*p));	// segfault!

 Anyway, just in case my reasoning is wrong, here is the whole Tuple class:

 ====================================================
 class Tuple : Pattern {
      /**  pattern type for tuple of given patterns */
      static string typename = "Tuple";
      Pattern[] patterns;
      this (Pattern[] patterns...) {
          this.patterns = patterns;


You need to dup that. Arguments are passed on the stack.
Nov 11 2010
parent reply spir <denis.spir gmail.com> writes:
On Thu, 11 Nov 2010 23:45:39 +0100
Pelle M=C3=A5nsson <pelle.mansson gmail.com> wrote:


 On 11/11/2010 10:15 PM, spir wrote:
=20

 Well, since the pattern is OK _after_ call to Tuple's constructor (whic=




h does nothing more than recording its sub-patterns, see below) and only ge=
ts wrong when qutting this(), I fail to see how Tuple could be cause of any=
thing. Also, the corruption is visible _before_ calling match() (which here=
 delegates to Tuple's check()).

 I understand your note about returning an object actually allocated on =




the stack -- but here there are only implicitely referenced objects (class =
instances). This would mean that D creates the 2 sub-patterns on the stack?=
 But why those objects, precisely? (Also note that they are of different cl=
asses: one is here a "String", the other a "ZeroOrMore"). They are stored i=
n an array.

 What's troubling is that the array elements, meaning the supposed subpa=




ttern addresses, have changed. Maybe the patterns themselves are still ok, =
but the array data only are corrupted?

 Oh, I may try to cast to String the memory area pointed inside this()..=




..... Does not seem to work: I recorded the pointer read in this() (as Stri=
ng*) into a global; then in the test func:

      writeln("p: ",p);       	// ok, same address as in this()
      writeln(cast(String)(*p));	// segfault!

 Anyway, just in case my reasoning is wrong, here is the whole Tuple cla=




ss:

 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=




=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D

 class Tuple : Pattern {
      /**  pattern type for tuple of given patterns */
      static string typename =3D "Tuple";
      Pattern[] patterns;
      this (Pattern[] patterns...) {
          this.patterns =3D patterns;


=20
 You need to dup that. Arguments are passed on the stack.


Sorry, but I don't understand your hint. Where is the dup-ed array supposed=
 to be allocated? Isn't the assignment supposed to copy it to the field, an=
yway? How else could one store an array to a field? And how are contained r=
eferenced objects supposed to stop vanishing thank to dup?
(I tried, anyway, but the did not stop segfault.)

I have one more example of segfault, in a brand new demo func matching 4 op=
erations. Very strange. With additions, all works fine. When I replace '+' =
by '-', I get a segfault _sometimes_. When I use '*' or '/', seems I get a =
segfault _everytime_. The patterns for the 4 operations are all the same, i=
ndeed.
Also, since the demo matches calculations of several operations, there can =
be whitespace. If I add some space after the operation, I get a segfault ev=
en for '+'.

What I intend to do is uninstall dmd2 to replace it by dmd1 and see what ha=
ppens. What do you think?


Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 12 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 12 Nov 2010 06:34:02 -0500, spir <denis.spir gmail.com> wrote:


 On Thu, 11 Nov 2010 23:45:39 +0100
 Pelle Månsson <pelle.mansson gmail.com> wrote:


 On 11/11/2010 10:15 PM, spir wrote:


 Well, since the pattern is OK _after_ call to Tuple's constructor  


 (which does nothing more than recording its sub-patterns, see below)  
 and only gets wrong when qutting this(), I fail to see how Tuple could  
 be cause of anything. Also, the corruption is visible _before_ calling  
 match() (which here delegates to Tuple's check()).

 I understand your note about returning an object actually allocated  


 on the stack -- but here there are only implicitely referenced objects  
 (class instances). This would mean that D creates the 2 sub-patterns on  
 the stack? But why those objects, precisely? (Also note that they are  
 of different classes: one is here a "String", the other a  
 "ZeroOrMore"). They are stored in an array.

 What's troubling is that the array elements, meaning the supposed  


 subpattern addresses, have changed. Maybe the patterns themselves are  
 still ok, but the array data only are corrupted?

 Oh, I may try to cast to String the memory area pointed inside  


 this()....... Does not seem to work: I recorded the pointer read in  
 this() (as String*) into a global; then in the test func:

      writeln("p: ",p);       	// ok, same address as in this()
      writeln(cast(String)(*p));	// segfault!

 Anyway, just in case my reasoning is wrong, here is the whole Tuple  


 class:

 ====================================================
 class Tuple : Pattern {
      /**  pattern type for tuple of given patterns */
      static string typename = "Tuple";
      Pattern[] patterns;
      this (Pattern[] patterns...) {
          this.patterns = patterns;


 You need to dup that. Arguments are passed on the stack.


 Sorry, but I don't understand your hint. Where is the dup-ed array  
 supposed to be allocated? Isn't the assignment supposed to copy it to  
 the field, anyway? How else could one store an array to a field? And how  
 are contained referenced objects supposed to stop vanishing thank to dup?
 (I tried, anyway, but the did not stop segfault.)


Pelle, I spent all this time helping him, and you swoop in with the answer  
:)

Yes, he is right, you need to dup the patterns argument.  It is something  
I very recently discovered.

Here is what happens:

void foo(int[] arg...) {}

foo(1,2,3);

What the compiler does is push 1, 2, and 3 onto the stack, then passes in  
a dynamic array reference to that stack data.  It does this because heap  
allocations are much more expensive than stack allocations.  When you just  
"save" the data, it is no longer valid.  The reason the code all works  
within the List constructor is because that is the stack frame where the  
array is pushed onto the stack.

Note, you can do foo([1,2,3]), and you will be wasting time duping, but I  
have found a solution to that, overloading:

void foo(int[] arg) {}

If you pass in an array, then the second overload is used, if you pass in  
individual arguments the first overload is used.

Essentially, if you change your line above to:

this.patterns = patterns.dup;

All is well, you should be good.

-Steve
Nov 12 2010
next sibling parent reply spir <denis.spir gmail.com> writes:
On Fri, 12 Nov 2010 08:03:26 -0500
"Steven Schveighoffer" <schveiguy yahoo.com> wrote:



 Pelle, I spent all this time helping him, and you swoop in with the answe=


r =20

 :)
=20
 Yes, he is right, you need to dup the patterns argument.  It is something=


 =20

 I very recently discovered.


Wow, first thought dup did not solve the issue, but it fact I had introduce=
d another bug in the meanyrime ;-) All works fine, now. I'll be able to res=
tart working on this. D soon has a prototype for a PEG lib.


 Here is what happens:
=20
 void foo(int[] arg...) {}
=20
 foo(1,2,3);
=20
 What the compiler does is push 1, 2, and 3 onto the stack, then passes in=


 =20

 a dynamic array reference to that stack data.  It does this because heap =


=20

 allocations are much more expensive than stack allocations.  When you jus=


t =20

 "save" the data, it is no longer valid.  The reason the code all works =20
 within the List constructor is because that is the stack frame where the =


=20

 array is pushed onto the stack.


Right.


 Note, you can do foo([1,2,3]), and you will be wasting time duping, but I=


 =20

 have found a solution to that, overloading:
=20
 void foo(int[] arg) {}
=20
 If you pass in an array, then the second overload is used, if you pass in=


 =20

 individual arguments the first overload is used.


All right. I find a bit strange that the compiler accepts f([1,2,3]) when t=
he declaration reads eg void f(int[] ints...). Anyway, it cannot harm, I gu=
ess.


 Essentially, if you change your line above to:
=20
 this.patterns =3D patterns.dup;


Works, but I don't understand why: isn't the duplicate also allocated on th=
e stack? I mean, dup is supposed to just duplicate, isn't it? what does it =
give to the new structure that the original one hasn't? I thought I would h=
ave to do for instance:
	ints[] x;
	void f(int[] ints...) {
	  x.length =3D ints.length;   // allocate new area on the heap
	  x[] =3D ints[];             // copy elements in there
	}


 All is well, you should be good.


yo!


 -Steve


Thank you and Pelle for your help,
Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 13 2010
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 13 Nov 2010 15:57:50 -0500, spir <denis.spir gmail.com> wrote:


 On Fri, 12 Nov 2010 08:03:26 -0500
 "Steven Schveighoffer" <schveiguy yahoo.com> wrote:



 Essentially, if you change your line above to:

 this.patterns = patterns.dup;


 Works, but I don't understand why: isn't the duplicate also allocated on  
 the stack? I mean, dup is supposed to just duplicate, isn't it? what  
 does it give to the new structure that the original one hasn't? I  
 thought I would have to do for instance:
 	ints[] x;
 	void f(int[] ints...) {
 	  x.length = ints.length;   // allocate new area on the heap
 	  x[] = ints[];             // copy elements in there
 	}


When doing this:

arr1 = arr2;

It copies the reference to the data, not the data itself.  Since the data  
is on the stack, you are still pointing at the stack

arr1 = arr2.dup;

copies the data itself, then returns a reference to the new data.  The  
reference is stored wherever the reference is stored.  In your case, the  
reference this.patterns is stored in the class, so it's also stored on the  
heap.

I hope this is clearer.

-Steve
Nov 15 2010
prev sibling parent =?UTF-8?B?UGVsbGUgTcOlbnNzb24=?= <pelle.mansson gmail.com> writes:
On 11/12/2010 02:03 PM, Steven Schveighoffer wrote:

 Pelle, I spent all this time helping him, and you swoop in with the
 answer :)


I was in a rush when answering, causing the swoopiness of my post. :-)

I only knew the answer because I had almost exactly the same bug a week 
ago, or so. Coincidentally, for a parser generator.
Nov 14 2010
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
spir:


 I'm blocked by a mysterious segfault I seem to be unable to diagnose.


I have added an enhancement request to avoid the D pitfall you have found, so
the time you have used to find this bug is not wasted and may help future D
programmers:

http://d.puremagic.com/issues/show_bug.cgi?id=5212

Bye,
bearophile
Nov 13 2010