digitalmars.D.learn - Trouble with anon delegates.
- Pragma (26/26) Jan 16 2007 All,
- Frits van Bommel (21/55) Jan 16 2007 That class has a member function returning a delegate. That returned
- BCS (9/45) Jan 16 2007 Really, some BIG RED TEXT needs to be added to the doc in few places. T...
- Pragma (27/79) Jan 16 2007 Thanks for the clarification. I wanted to be sure that I wasn't out of ...
- Frits van Bommel (22/32) Jan 16 2007 Maybe we could get Walter to implement a slight change to such
- BCS (22/45) Jan 16 2007 Cool!
- Pragma (11/58) Jan 17 2007 I would have done the same, but I'm making heavy use of polymorphism to ...
- BCS (99/99) Jan 16 2007 You might try somthing like this (warning it has been called "evil"), th...
All,
I ran into a small problem regarding the use of anon delegates, and what
their context (.ptr) is set to. If anyone
can share any insight as to if this is a bug or not, I would appreciate it.
In short: I'm attempting to create delegates on the fly and use them elsewhere.
After many strange errors, I narrowed
it down to the following case. Note the assertions at the end:
abstract class Foobar{
Fn getFn();
}
alias void delegate() Fn;
auto foo = new class Foobar{
int x;
Fn getFn(){
return { x = x };
}
};
auto fn = foo.getFn();
assert((&foo.getFn).ptr is foo); // passes
assert(fn.ptr is foo); // fails (ptr does *not* point to the object)
So by this, anon delegates are not the same as actual object delegates. Is
this a bug, or is this intentional?
"When comparing with nested functions, the function form is analogous to static
or non-nested functions, and the
delegate form is analogous to non-static nested functions. In other words, a
delegate literal can access stack variables
in its enclosing function, a function literal cannot."
As you can see, the docs are as clear as mud. I suppose a "non-static nested
function" wouldn't know about the 'this'
pointer, nor care - but then why encumber it with the 'delegate' keyword?
--
- EricAnderton at yahoo
Jan 16 2007
Frits van Bommel <fvbommel REMwOVExCAPSs.nl> Pragma wrote:All, I ran into a small problem regarding the use of anon delegates, and what their context (.ptr) is set to. If anyone can share any insight as to if this is a bug or not, I would appreciate it. In short: I'm attempting to create delegates on the fly and use them elsewhere. After many strange errors, I narrowed it down to the following case. Note the assertions at the end: abstract class Foobar{ Fn getFn(); } alias void delegate() Fn; auto foo = new class Foobar{ int x; Fn getFn(){ return { x = x }; } };That class has a member function returning a delegate. That returned delegate has the stack frame of said member function as its context pointer and should thus not be called after it returns. Basically, the return value is useless as it can't safely be called.auto fn = foo.getFn();'fn' now contains said delegate, with the context pointer set to an invalidated stackframe.assert((&foo.getFn).ptr is foo); // passes(&foo.getFn) is a delegate with context foo and function pointer Foo.getFn.assert(fn.ptr is foo); // fails (ptr does *not* point to the object)fn.ptr is the invalid stack frame of the call to foo.getFn(), not foo itself.So by this, anon delegates are not the same as actual object delegates. Is this a bug, or is this intentional?Intentional."When comparing with nested functions, the function form is analogous to static or non-nested functions, and the delegate form is analogous to non-static nested functions. In other words, a delegate literal can access stack variables in its enclosing function, a function literal cannot." As you can see, the docs are as clear as mud. I suppose a "non-static nested function" wouldn't know about the 'this' pointer, nor care - but then why encumber it with the 'delegate' keyword?It *does* know about 'this'. However, its context pointer is a stack frame (that happens to store 'this' in your case), not 'this' itself. There are simply several 'types' of delegates. One is a delegate that has a class or struct (or perhaps even a union) instance as its context pointer. These are created by (&instance.memberFunction). Another 'type' has a stack frame as its context pointer. These are created by (&nestedFunction) or { foo(); }. They should not be called after the function call (of the function directly containing their definition) in which they were created returns. Spec reference: http://www.digitalmars.com/d/function.html#closures
Jan 16 2007
Reply to Pragma,
All,
I ran into a small problem regarding the use of anon delegates, and
what their context (.ptr) is set to. If anyone
can share any insight as to if this is a bug or not, I would
appreciate it.
In short: I'm attempting to create delegates on the fly and use them
elsewhere. After many strange errors, I narrowed it down to the
following case. Note the assertions at the end:
abstract class Foobar{
Fn getFn();
}
alias void delegate() Fn;
auto foo = new class Foobar{
int x;
Fn getFn(){
return { x = x };
}
};
auto fn = foo.getFn();
assert((&foo.getFn).ptr is foo); // passes
assert(fn.ptr is foo); // fails (ptr does *not* point to the object)
So by this, anon delegates are not the same as actual object
delegates. Is this a bug, or is this intentional?
"When comparing with nested functions, the function form is analogous
to static or non-nested functions, and the delegate form is analogous
to non-static nested functions. In other words, a delegate literal can
access stack variables in its enclosing function, a function literal
cannot."
As you can see, the docs are as clear as mud. I suppose a "non-static
nested function" wouldn't know about the 'this' pointer, nor care -
but then why encumber it with the 'delegate' keyword?
Really, some BIG RED TEXT needs to be added to the doc in few places. This
is one of them. Anon delegates are totally invalid after the function they
are declared within returns. Any statement of the form:
return {...};
is ALWAYS invalid. This has gotten about a dozen people in the last week
or so and, as you said. "the docs are as clear as mud" on this. Anon delegate
in any scope use the current function's frame pointer as the context. Once
the function returns, this more than likely is junk.
Jan 16 2007
BCS wrote:Reply to Pragma,Thanks for the clarification. I wanted to be sure that I wasn't out of my mind. This is really disappointing. I was working on a runtime Spirit engine (as a counterpart to your own library) that would use inline delegates as valid parse expressions: auto rule = ws >> '(' >> +Argument >> ')' >> { writefln("args parsed"); }; It works fine for this kind of stuff, but once you graduate beyond there and want to bind terminals to class members, you need the ability to perform filtering and aritmetic on them. But once you do that, you find that your stack frame is all out of whack. :( // This construct is used to help create a context for binding and manipulating parse terms. // create() is called within LambdaExpr's constructor to create and cache the Expr to be used // at this stage of the parser. Of course, the inline delegates are invalid once this returns. /** 'factor' rule from the infamous 'infix calculator' BNF grammar. */ auto factor = new class LambdaExpr{ real result,term; Expr create(){ auto posFactor = T('+') >> ws >> term[&term] >> { result += term; }; auto negFactor = T('-') >> ws >> term[&term] >> { result +- term; }; return ws >> term[&result] >> -+(ws >> (posFactor | negFactor)) >> { return result; }; } }; It would be nice if things so succinct would work as I expected. Instead I have to break all the delegates out into proper class methods, which bloats the code and runs contrary to the whole point of having a parser framework like this. The ability to declare *first class* delegates as anonymous and inline like the above is just too tempting to leave alone. I suppose the only way to have your cake and eat it too is to preserve the stack somehow. Maybe this is a job for stack threads? -- - EricAnderton at yahooAll, I ran into a small problem regarding the use of anon delegates, and what their context (.ptr) is set to. If anyone can share any insight as to if this is a bug or not, I would appreciate it. In short: I'm attempting to create delegates on the fly and use them elsewhere. After many strange errors, I narrowed it down to the following case. Note the assertions at the end: abstract class Foobar{ Fn getFn(); } alias void delegate() Fn; auto foo = new class Foobar{ int x; Fn getFn(){ return { x = x }; } }; auto fn = foo.getFn(); assert((&foo.getFn).ptr is foo); // passes assert(fn.ptr is foo); // fails (ptr does *not* point to the object) So by this, anon delegates are not the same as actual object delegates. Is this a bug, or is this intentional? "When comparing with nested functions, the function form is analogous to static or non-nested functions, and the delegate form is analogous to non-static nested functions. In other words, a delegate literal can access stack variables in its enclosing function, a function literal cannot." As you can see, the docs are as clear as mud. I suppose a "non-static nested function" wouldn't know about the 'this' pointer, nor care - but then why encumber it with the 'delegate' keyword?Really, some BIG RED TEXT needs to be added to the doc in few places. This is one of them. Anon delegates are totally invalid after the function they are declared within returns. Any statement of the form: return {...}; is ALWAYS invalid. This has gotten about a dozen people in the last week or so and, as you said. "the docs are as clear as mud" on this. Anon delegate in any scope use the current function's frame pointer as the context. Once the function returns, this more than likely is junk.
Jan 16 2007
Pragma wrote: [snip]It would be nice if things so succinct would work as I expected. Instead I have to break all the delegates out into proper class methods, which bloats the code and runs contrary to the whole point of having a parser framework like this. The ability to declare *first class* delegates as anonymous and inline like the above is just too tempting to leave alone. I suppose the only way to have your cake and eat it too is to preserve the stack somehow. Maybe this is a job for stack threads?Maybe we could get Walter to implement a slight change to such delegates: instead of getting the innermost context pointer, they should get the outermost possible context pointer that still contains all the variables/members used? So then in your case they would get the 'this' pointer, in nested functions they could get the stackframe of the outer function if they didn't reference variables local to the inner function, and in member functions of (non-static) nested classes they could even contain a copy of the 'outer' pointer of the nested class, so only the outer class is referenced if that's all that's needed. I have no idea how difficult this would be to implement, but I think this will go a long way towards making delegates more usable. The only disadvantage would be that the validity of a delegate after a function call returns would depend on the code it contains, which may be a source of tricky bugs... An alternative would be the explicit context syntax I somewhere today (can't find it anymore :( ). Your example delegate would then look something like 'this.{ x = x; }'. Not as pretty, but still much better than having to define a new member function for this operator, don't you think?
Jan 16 2007
Reply to Pragma, [...]Thanks for the clarification. I wanted to be sure that I wasn't out of my mind. This is really disappointing. I was working on a runtime Spirit engine (as a counterpart to your own library) that would use inline delegates as valid parse expressions:Cool!auto rule = ws >> '(' >> +Argument >> ')' >> { writefln("args parsed"); }; It works fine for this kind of stuff, but once you graduate beyond there and want to bind terminals to class members, you need the ability to perform filtering and aritmetic on them. But once you do that, you find that your stack frame is all out of whack. :(I tend to use structs for that kind of thing. They entail less overhead than classes. [...]It would be nice if things so succinct would work as I expected. Instead I have to break all the delegates out into proper class methods, which bloats the code and runs contrary to the whole point of having a parser framework like this.I've said it befor, What is needed is to allow delegate leterals to expicetly state there context. auto c = new C; return c.(char[] foo){retrun this.toString() ~ foo;} Another thing would be non static init for anon structs. This and a few others would make for some neat code. int delegate(int,int,int) Bar(int i, int j, int k) { return new struct { int a = i; int b = j; int c = k; }.(int m, int n, int o){ return a*m + b*n + c*o;}; }The ability to declare *first class* delegates as anonymous and inline like the above is just too tempting to leave alone. I suppose the only way to have your cake and eat it too is to preserve the stack somehow.I guess that is more or less what the above does. (Where's the 2.0 wish list? <g> )
Jan 16 2007
BCS wrote:Reply to Pragma,I would have done the same, but I'm making heavy use of polymorphism to get the operators to work as expected. However, that could be refactored into a struct-oriented design with some clever templates and mixins to emulate inheritance.auto rule = ws >> '(' >> +Argument >> ')' >> { writefln("args parsed"); }; It works fine for this kind of stuff, but once you graduate beyond there and want to bind terminals to class members, you need the ability to perform filtering and aritmetic on them. But once you do that, you find that your stack frame is all out of whack. :(I tend to use structs for that kind of thing. They entail less overhead than classes.[...]I read the other thread here, in D.learn, on this concept. I like the ability to anonymously declare any construct, but the ability to add literal methods/members to those is where it really shines. I've been using similar techniques in Javascript programming, and it completely transforms the way you approach solving particular problems. As Walter has written a compliant JS engine, you'd think he's be wise to the advantages here? Maybe there's something else blocking this kind of syntax from inclusion into D.It would be nice if things so succinct would work as I expected. Instead I have to break all the delegates out into proper class methods, which bloats the code and runs contrary to the whole point of having a parser framework like this.I've said it befor, What is needed is to allow delegate leterals to expicetly state there context. auto c = new C; return c.(char[] foo){retrun this.toString() ~ foo;} Another thing would be non static init for anon structs. This and a few others would make for some neat code. int delegate(int,int,int) Bar(int i, int j, int k) { return new struct { int a = i; int b = j; int c = k; }.(int m, int n, int o){ return a*m + b*n + c*o;}; }You mean, we dont' have one yet?! -- - EricAnderton at yahooThe ability to declare *first class* delegates as anonymous and inline like the above is just too tempting to leave alone. I suppose the only way to have your cake and eat it too is to preserve the stack somehow.I guess that is more or less what the above does. (Where's the 2.0 wish list? <g> )
Jan 17 2007
Pragma wrote:BCS wrote:Ah, you must be doing something more complicated than I was thinking of. I often find myself using the pattern of a struct that is used in exactly one place: struct S { int i; int j(){return i} } auto s = new S; s.i=2; return &s.j this can be done with a class, but it has some more overhead. class C { int i; this(int k){i=k;} int j(){return i} } return &(new C(1)).jReply to Pragma,I would have done the same, but I'm making heavy use of polymorphism to get the operators to work as expected. However, that could be refactored into a struct-oriented design with some clever templates and mixins to emulate inheritance.auto rule = ws >> '(' >> +Argument >> ')' >> { writefln("args parsed"); }; It works fine for this kind of stuff, but once you graduate beyond there and want to bind terminals to class members, you need the ability to perform filtering and aritmetic on them. But once you do that, you find that your stack frame is all out of whack. :(I tend to use structs for that kind of thing. They entail less overhead than classes.Not that I know of :-|(Where's the 2.0 wish list? <g> )You mean, we dont' have one yet?!
Jan 17 2007
BCS wrote:Pragma wrote:Yea, sorry to talk so much about something that is basically vaporware. I've learned to be cautious of releasing stuff without having completely worked the kinks out of it myself - if it's at all useful, poor quality offerings breed a contemptible support situation for everyone involved (especially for me). But since the cat's out of the bag, I'll explain what I'm talking about. Basically, it's something like this (paraphrased for clarity): abstract class Expr{ Expr opShr(Expr expr){ return new AndGroup(this,expr); } Expr opShr(String value){ return opShr(new Terminal(value)); } Expr opOr(Expr expr){ return new OrGroup(this,expr); } Expr opOr(String value){ return opOr(new Terminal(value)); } Expr opPlus(Expr expr){ return new Multiple(this); } Expr opNeg(Expr expr){ return new Optional(this); } bool Parse(IScanner scanner); } class Terminal: Expr{} class Optional: Expr{} class Multiple: Expr{} class AndGroup: Expr{} class OrGroup: Expr{} ... and so on, plus a few shims to get some expressions to work. So that: Expr ws = /* defined elsewhere */ Expr rule = ws >> 'hello' >> ws >> 'world'; is equivalent to: Expr rule = new AndGroup( ws, new AndGroup( new Terminal('hello'), new AndGroup( ws, new Terminal('world') ) ) ); The various operators are overridden in the sub-classes so optimizations can be made. For instance, AndGroup and OrGroup will maintain a list of terms such that the redundant nesting in the above never happens. Now, to bring it back to where I started, I had some methods on Expr that did this: Expr opShr(void delegate() predicate){ return opShr(new PredicateExpr(predicate)); } Expr opOr(void delegate() predicate){ return opOr(new PredicateExpr(predicate)); } ... but that's dangerous for the reasons we discussed earlier. FWIW, another limitation I ran into is that D offers no static operator overloads in the global scope, so we can't get operators to generate templates in the same way that Spirit can in C++. Granted, that's a small limitation, but a 100% static parser /might/ help somewhat.BCS wrote:Ah, you must be doing something more complicated than I was thinking of.Reply to Pragma,I would have done the same, but I'm making heavy use of polymorphism to get the operators to work as expected. However, that could be refactored into a struct-oriented design with some clever templates and mixins to emulate inheritance.auto rule = ws >> '(' >> +Argument >> ')' >> { writefln("args parsed"); }; It works fine for this kind of stuff, but once you graduate beyond there and want to bind terminals to class members, you need the ability to perform filtering and aritmetic on them. But once you do that, you find that your stack frame is all out of whack. :(I tend to use structs for that kind of thing. They entail less overhead than classes.I often find myself using the pattern of a struct that is used in exactly one place: struct S { int i; int j(){return i} } auto s = new S; s.i=2; return &s.j this can be done with a class, but it has some more overhead. class C { int i; this(int k){i=k;} int j(){return i} } return &(new C(1)).j-- - EricAnderton at yahooNot that I know of :-|(Where's the 2.0 wish list? <g> )You mean, we dont' have one yet?!
Jan 17 2007
You might try somthing like this (warning it has been called "evil"), this
vertion dosn't work but it can actualy be made to by using structs and a
little refactoring. The basic idea is use structs with delegate as members
to build operatio trees. In this case I used it for the reduction, you'd
could using it for parsing as well.
p.s. It realy is evil, you have been warned.
// a base type for derivation
class Type{}
// a templated derived type to store things in
class TypeT(T) : Type
{
public T val;
static TypeT!(T) opCall(T v)
{
auto ret = new TypeT!(T);
ret.val = v;
return ret;
}
}
/*
Doc ::=
A:a B:b C:c = new Fig(a,b,c)
| D:d E:e = new Fig(d,e)
*/
TypeT!(Fig) delegate() Parse_Doc(inout char[] from)
{
char[] tmp;
// copy from
tmp = from;
if(auto a = Parse_A(tmp)) // try to parse an "a"
if(auto b = Parse_B(tmp)) // try to parse an "b"
if(auto c = Parse_C(tmp)) // try to parse an "c"
{ // if all's good
// update what's been parsed
from = tmp;
return &(
// make an array from the args
[cast(Type delegate())a,b,c]
// copy it (force off stack)
.dup
)
// form delegate from array
// and function literal
.function TypeT!(Fig)(Type delegate()[] args)
{
// literal calls given code
auto ret =
new Fig(
// call args for values
cast(a.typeof)args[0]().val,
cast(b.typeof)args[1]().val,
cast(b.typeof)args[2]().val);
// place in object and return
return TypeT!(ret.typeof)(ret);
};
}
// restore tmp
tmp = from;
// blah, blah, blah
if(auto d = Parse_D(tmp))
if(auto e = Parse_E(tmp))
{
from = tmp;
return &([cast(Type delegate())d,e].dup)
.function TypeT!(Fig)(Type delegate()[] args)
{
auto ret =
new Fig(
cast(d.typeof)args[0]().val,
cast(e.typeof)args[1]().val);
return TypeT!(ret.typeof)(ret);
};
}
return null;
}
/*
A ::=
"hello":v = v
*/
TypeT!(char[]) delegate() Parse_A(inout char[] from)
{
char[] tmp;
tmp=from;
// check for a "hello"
if(from[0.."hello".length] == "hello")
{
// update from
from = from["hello".length..$];
// return delegate returning "hello" in obj
return &("hello".dup).function TypeT!(char[])()
{
auto ret =
from[0.."hello".length];
return TypeT!(ret.typeof)(ret);
};
}
return null;
}
/// ........
Jan 16 2007