www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Delegates

reply David Medlock <amedlock nospam.org> writes:
The following code did not produce results I expected:

int delegate() dg;

alias int delegate() dg_t ;

dg_t fn(int a )
{
   int inner() { int i = a; return i + 5; }
   dg_t result = &inner;
   return result;
}


void main()
{
   dg_t    delfunct, delfunct2;

   delfunct = fn( 100 );
   delfunct2 = fn( 55 );

   printf( "%i\n", delfunct() );
   printf( "%i\n", delfunct2() );
   printf( "%i\n", delfunct() );
   printf( "%i\n", delfunct2() );
}

After reading the spec I see this is normal behavior:

"The stack variables, however, are not valid once the function declaring 
them has exited, in the same manner that pointers to stack variables are 
not valid upon exit from a function.."

My question is why this limitation exists?
The only difference for a delegate and a normal function pointer is one 
uses the stack for its local variables and a delegate would use a local 
block of memory(which can change).  Maybe I am not seeing some other 
underlying technical issue.

Sorry if this topic has come up before( I am a little new to D ).
Thanks,
David
Jul 21 2004
parent reply Sean Kelly <sean f4.ca> writes:
David Medlock wrote:

 "The stack variables, however, are not valid once the function declaring 
 them has exited, in the same manner that pointers to stack variables are 
 not valid upon exit from a function.."
 
 My question is why this limitation exists?


This has been discussed a bit, but it's been a few months.  I think the 
primary reason was performance.  If 95% of the delgates used are not 
returned from functions in this manner then it's unnecessarily costly to 
always copy the stack frame.  Some more advanced lexing might be 
possible to determine when this is necessary but I think the response 
was "maybe but not right now."  One slightly lame fix that might work in 
a single-threaded environment is:

dg_t fn(int a )
{
   static int cpy = a;
   int inner() { int i = cpy; return i + 5; }
   dg_t result = &inner;
   return result;
}

I suppose if this works you could use thread local storage to do the 
same thing in a multithreaded program.


Sean
Jul 21 2004
next sibling parent reply David Medlock <amedlock nospam.org> writes:
Sean Kelly wrote:

 David Medlock wrote:
 

 "The stack variables, however, are not valid once the function 
 declaring them has exited, in the same manner that pointers to stack 
 variables are not valid upon exit from a function.."

 My question is why this limitation exists?


 
 
 This has been discussed a bit, but it's been a few months.  I think the 
 primary reason was performance.  If 95% of the delgates used are not 
 returned from functions in this manner then it's unnecessarily costly to 
 always copy the stack frame.  Some more advanced lexing might be 
 possible to determine when this is necessary but I think the response 
 was "maybe but not right now."  One slightly lame fix that might work in 
 a single-threaded environment is:
 
 dg_t fn(int a )
 {
   static int cpy = a;
   int inner() { int i = cpy; return i + 5; }
   dg_t result = &inner;
   return result;
 }
 
 I suppose if this works you could use thread local storage to do the 
 same thing in a multithreaded program.
 
 
 Sean


Hmmm.  The only performance hit would be at creation and not during use, 
so I would consider that a minor issue at best.  And since the delegate 
'carries' its local variables, its pretty thread safe isn't it?

Adding a 'const int i = cpy' would perhaps cause the compiler to inline 
the whole thing, but as you say more advanced checking could optimize 
even in the absence of const.

It just borders so close to true closures, just seems a shame they 
aren't there, especially since delegate is D-only and doesn't rely on C 
linkability(or does it?).  Adding this in makes D much more functional 
and may attract people from functional language communities, imo.
Jul 21 2004
parent Sean Kelly <sean f4.ca> writes:
In article <cdm2ml$2qfb$1 digitaldaemon.com>, David Medlock says...

Sean Kelly wrote:

 David Medlock wrote:
 

 "The stack variables, however, are not valid once the function 
 declaring them has exited, in the same manner that pointers to stack 
 variables are not valid upon exit from a function.."

 My question is why this limitation exists?


 
 
 This has been discussed a bit, but it's been a few months.  I think the 
 primary reason was performance.  If 95% of the delgates used are not 
 returned from functions in this manner then it's unnecessarily costly to 
 always copy the stack frame.  Some more advanced lexing might be 
 possible to determine when this is necessary but I think the response 
 was "maybe but not right now."  One slightly lame fix that might work in 
 a single-threaded environment is:
 
 dg_t fn(int a )
 {
   static int cpy = a;
   int inner() { int i = cpy; return i + 5; }
   dg_t result = &inner;
   return result;
 }
 
 I suppose if this works you could use thread local storage to do the 
 same thing in a multithreaded program.
 
 
 Sean


Hmmm.  The only performance hit would be at creation and not during use, 
so I would consider that a minor issue at best.  And since the delegate 
'carries' its local variables, its pretty thread safe isn't it?


The thread-safety issue was only with my workaround, not with a language-based
stack copy.  Assume this bit of code:

void func() {
char buf1[65536*64];
void deleg() { ... }
for( ... ) { deleg(); }
}

The delegate is never used outside the function but the stack would have to be
copied every time func() is called (assuming no advanced lexing) to make sure
that deleg() always has correct data in its copy of buf.  And because buf is
pretty large there's a risk of out of memory conditions plus the cost of
allocating the memory and copying the data.  Things get a bit more complicated
when you consider class method functions, as a delegate should have access to
not only its own stack but to class member data as well, though I think the
already existing hidden this pointer may suffice in that case. 


Adding a 'const int i = cpy' would perhaps cause the compiler to inline 
the whole thing, but as you say more advanced checking could optimize 
even in the absence of const.


True enough.  Though I'm not sure a const would work--do const values have
static linkage?


It just borders so close to true closures, just seems a shame they 
aren't there, especially since delegate is D-only and doesn't rely on C 
linkability(or does it?).  Adding this in makes D much more functional 
and may attract people from functional language communities, imo.


I agree.  I think if this were to be done the best thing would be to add a new
keyword that signals to the compiler that the stack must be copied.  Perhaps use
"auto" in the function declaration to signal that it is a truly local functions,
while the new default behavior is to copy the stack?


Sean
Jul 21 2004
prev sibling parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
I am a big proponent of having some functionality that enables us to 
copy stack delegates.  We don't want copying to be the default, however. 
  Consider this code:


int foo() {
   int counter = 0;

   bar(void delegate(int val) { counter += arg; });

   return counter;
}
void bar(void delegate() dg) {
   foreach(whatever)
     dg(something);
}


You see, in this code, you want to give the delegate the capability to 
modify the real values on the stack.  You don't want it to be using a 
duplicate.

So what I've advocated is that we have the option to copy or not.

Sean Kelly wrote:

 David Medlock wrote:
 

 "The stack variables, however, are not valid once the function 
 declaring them has exited, in the same manner that pointers to stack 
 variables are not valid upon exit from a function.."

 My question is why this limitation exists?


 
 
 This has been discussed a bit, but it's been a few months.  I think the 
 primary reason was performance.  If 95% of the delgates used are not 
 returned from functions in this manner then it's unnecessarily costly to 
 always copy the stack frame.  Some more advanced lexing might be 
 possible to determine when this is necessary but I think the response 
 was "maybe but not right now."  One slightly lame fix that might work in 
 a single-threaded environment is:
 
 dg_t fn(int a )
 {
   static int cpy = a;
   int inner() { int i = cpy; return i + 5; }
   dg_t result = &inner;
   return result;
 }
 
 I suppose if this works you could use thread local storage to do the 
 same thing in a multithreaded program.
 
 
 Sean
Jul 21 2004
next sibling parent reply David Medlock <amedlock nospam.org> writes:
Russ Lewis wrote:


 I am a big proponent of having some functionality that enables us to 
 copy stack delegates.  We don't want copying to be the default, however. 
  Consider this code:
 
 
 int foo() {
   int counter = 0;
 
   bar(void delegate(int val) { counter += arg; });
 
   return counter;
 }
 void bar(void delegate() dg) {
   foreach(whatever)
     dg(something);
 }
 
 
 You see, in this code, you want to give the delegate the capability to 
 modify the real values on the stack.  You don't want it to be using a 
 duplicate.
 
 So what I've advocated is that we have the option to copy or not.
 
 Sean Kelly wrote:


I assume you meant to write 'counter += val'.

Actually that seems an error in my definition of a closure. In the
above situation the behavior I would expect from the compiler is all
accessible variables not passed as params are *in*, so counter would
have to be redeclared within the inner function.

Pseudo example of what I am thinking of(internals):
struct delegate { int size; void * data; void *funct; }

Assigning a function to a delegate would mean ensuring the size/data 
parameters are valid.

The only problem that remains would be giving a delegate function access 
to the *data* within the struct.

The compiler could:
Have code inserted before and after a delegate is called to handle 
copying the data to and from the stack(as mentioned).  Performance is 
extremely minimal because in most cases its just a few Push/Pops.

-or-

For each non-static function, have another version internally(AST) which 
is used when an assignment to a delegate happens.  The delegate version 
accesses local variables in its storage space instead of the stack(using 
the data*). Having a different version probably wouldnt be that bad, 
templates do this already.

I realize this would be a PITA, but this would allow many functional 
idioms to be used.

Sorry for the rambling, I hope this makes some sense.
Jul 22 2004
next sibling parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
David Medlock wrote:

 Russ Lewis wrote:
 

 I am a big proponent of having some functionality that enables us to 
 copy stack delegates.  We don't want copying to be the default, 
 however.  Consider this code:


 int foo() {
   int counter = 0;

   bar(void delegate(int val) { counter += arg; });

   return counter;
 }
 void bar(void delegate() dg) {
   foreach(whatever)
     dg(something);
 }


 I assume you meant to write 'counter += val'.


Yes.  and I also noticed that the argument to bar() has the wrong 
argument type.  My bad. :(


 Actually that seems an error in my definition of a closure. In the
 above situation the behavior I would expect from the compiler is all
 accessible variables not passed as params are *in*, so counter would
 have to be redeclared within the inner function.


I see what you're thinking of, and it makes sense.  However, what I'm 
pointing out is that what you're suggesting is only appropriate in some 
designs.  In your design, the creator of the delegate cannot get any 
results out of the delegate (unless, of course, you pass pointers into 
the delegate).

What I was pointing out is that there are two different paradigms here. 
  One paradigm is supported by D currently; you are saying that it 
should be something else.

I would like the compiler to support both paradigms.


 Pseudo example of what I am thinking of(internals):
 struct delegate { int size; void * data; void *funct; }


Essentially, a stack delegate is syntax sugar for declaring a new struct 
type.  Here's old C++ code:

struct func_workingData { int counter; }
int func() {
   func_workingData data;
   data.counter = 0;
   bar(&func_inner_delegate, &data);
   return data.counter;
}
void func_inner_delegate(void *arg, int val) {
   func_workingData *this = (func_workingData*)arg;
   this.counter += val;
}
void bar(void (*callback)(void*,int), void *arg) {
   for(<whatever>)
     callback(arg, <something>);
}

So you see, when you create a stack delegate you are essentially passing 
a pointer to an anonymous struct on the stack.  When you duplicate, you 
are doing something else:

void func2() {
   func_workingData data;
   data.counter = 0;
   func_workingData *copy = malloc(sizeof(data));
   memcpy(copy, &data, sizeof(data));
   bar(&func_inner_delegate, copy);
}

Note that there's no need to push/pop the duplicated stack variables. 
You can simply access them directly from the heap.  The delegate code 
doesn't need to know whether the pointer points to the heap or the stack.
Jul 22 2004
prev sibling parent reply Sha Chancellor <schancel pacific.net> writes:
If delegates are allowed to point at the stack they were instatiated within then
the compiler has to be 
smart enough to leave the stack around for the garbage collector to clean up
when the delegate is 
finally done away with.   This seems strange...

Otherwise you'll end up with stackless delegates..

In article <cdoc6l$mkg$1 digitaldaemon.com>, David Medlock says...

Russ Lewis wrote:


 I am a big proponent of having some functionality that enables us to 
 copy stack delegates.  We don't want copying to be the default, however. 
  Consider this code:
 
 
 int foo() {
   int counter = 0;
 
   bar(void delegate(int val) { counter += arg; });
 
   return counter;
 }
 void bar(void delegate() dg) {
   foreach(whatever)
     dg(something);
 }
 
 
 You see, in this code, you want to give the delegate the capability to 
 modify the real values on the stack.  You don't want it to be using a 
 duplicate.
 
 So what I've advocated is that we have the option to copy or not.
 
 Sean Kelly wrote:


I assume you meant to write 'counter += val'.

Actually that seems an error in my definition of a closure. In the
above situation the behavior I would expect from the compiler is all
accessible variables not passed as params are *in*, so counter would
have to be redeclared within the inner function.

Pseudo example of what I am thinking of(internals):
struct delegate { int size; void * data; void *funct; }

Assigning a function to a delegate would mean ensuring the size/data 
parameters are valid.

The only problem that remains would be giving a delegate function access 
to the *data* within the struct.

The compiler could:
Have code inserted before and after a delegate is called to handle 
copying the data to and from the stack(as mentioned).  Performance is 
extremely minimal because in most cases its just a few Push/Pops.

-or-

For each non-static function, have another version internally(AST) which 
is used when an assignment to a delegate happens.  The delegate version 
accesses local variables in its storage space instead of the stack(using 
the data*). Having a different version probably wouldnt be that bad, 
templates do this already.

I realize this would be a PITA, but this would allow many functional 
idioms to be used.

Sorry for the rambling, I hope this makes some sense.
Jul 22 2004
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Stack delegates work just like any code that passes pointers to stack 
variables: they are valid only as long as the function doesn't return, 
or leave the block of code where the variable resides.  So this code is 
perfectly ok in C/C++ and D:
   void foo() {
     int var = 0;
     bar(&var);
   }
so long as bar() doesn't save the pointer.  bar() should never use the 
pointer after it returns to foo().

Likewise, a stack delegate is valid only as long as the stack frame is 
valid.  So you're right, you can end up with stackless delegates...but 
this is just the way they work.  The programmer has to be aware of it.

Of course, this whole discussion is about the fact that this would not 
be an issue if it were possible to have stack delegates which use copies 
of the stack.

Sha Chancellor wrote:

 If delegates are allowed to point at the stack they were instatiated within
then
 the compiler has to be 
 smart enough to leave the stack around for the garbage collector to clean up
 when the delegate is 
 finally done away with.   This seems strange...
 
 Otherwise you'll end up with stackless delegates..
Jul 22 2004
parent reply pragma <EricAnderton at yahoo dot com> <pragma_member pathlink.com> writes:
In article <cdpbel$14k4$1 digitaldaemon.com>, Russ Lewis says...

Stack delegates work just like any code that passes pointers to stack 
variables: they are valid only as long as the function doesn't return, 
or leave the block of code where the variable resides.  So this code is 
perfectly ok in C/C++ and D:
   void foo() {
     int var = 0;
     bar(&var);
   }
so long as bar() doesn't save the pointer.  bar() should never use the 
pointer after it returns to foo().

Likewise, a stack delegate is valid only as long as the stack frame is 
valid.  So you're right, you can end up with stackless delegates...but 
this is just the way they work.  The programmer has to be aware of it.

Of course, this whole discussion is about the fact that this would not 
be an issue if it were possible to have stack delegates which use copies 
of the stack.


Here's a few thoughts: what happens when a delegate, executing on it's own
stack-frame (copied into GC'd memory for argument's sake) calls another
function?  Would the called function's stack frame be allocated back on the
thread's stack (provided it's not another stack delegate)?  How would recursion
within a stack-delegate be handled? ...on the thread's stack in all cases,
perhaps?

The reason why I bring this up is that you may be able to create a full-on
reference implementation in D itself (with some liberal application of assembler
of course).  All you'd need to to is push the stack-segment and stack-pointer
onto the current stack for safe-keeping before setting it to what stack you're
going to use (this means that whatever stack space the delegate needs would have
to be at least 16 bytes to accomodate this).  This would apply to entering the
stack-delegate as well as calling functions that lead out of it.

- Pragma
Jul 22 2004
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
pragma  wrote:

 Here's a few thoughts: what happens when a delegate, executing on it's own
 stack-frame (copied into GC'd memory for argument's sake) calls another
 function?  Would the called function's stack frame be allocated back on the
 thread's stack (provided it's not another stack delegate)?  How would recursion
 within a stack-delegate be handled? ...on the thread's stack in all cases,
 perhaps?
 
 The reason why I bring this up is that you may be able to create a full-on
 reference implementation in D itself (with some liberal application of
assembler
 of course).  All you'd need to to is push the stack-segment and stack-pointer
 onto the current stack for safe-keeping before setting it to what stack you're
 going to use (this means that whatever stack space the delegate needs would
have
 to be at least 16 bytes to accomodate this).  This would apply to entering the
 stack-delegate as well as calling functions that lead out of it.


I think that perhaps I miscommunicated here.  I don't think that the 
stack frame of a delegate should ever reside on the heap.  Instead, I 
think that some of the local variables of a certain delegate should 
reside inside a struct which resides on the heap.  The difference may be 
small for the programmer, but the difference is massive for 
implementation.  I'm going to try to illustrate.  Please forgive the 
amount of hex is will require...

   int delegate() foo() {
     int counter;
     return stack_frame.dup.delegate int(int val)
       { counter += val; return val; }
   }

   void bar() {
     int delegate() del = foo();
     foreach(int i; <whatever>)
       printf("Value = %d, retval = %d\n", i,del(i));
   }

Ok, so the code here has a stack delegate which gets duplicated.  What 
happens when bar() calls that delegate?  This is how I imagine the stack 
would look:

   ADDRESS 0xB080 (STACK)
     Local variable 'del' (8 bytes)
       obj = 0x8000
       fptr = 0x4000
     Local variable 'i' (4 bytes)
   ADDRESS 0xB070 (STACK)
     Argument 'this' (4 bytes)
       ptr = 0x8000  /* note: this is the 'obj' from above */

   ADDRESS 0x8000 (HEAP)
     Variable 'counter'

The point I'm trying to communicate here is that the stack frame for the 
delegate resides on the same stack as everybody else.  That stack frame 
includes a pointer variable which points to a location in the heap where 
copied variables (things that USED TO BE on the stack) now reside.

A delegate can then call any other function it wants to.  It continues 
to use the stack just like any other function does.  No magic necessary.

You can see how you can implement this in one of my previous posts in 
this thread.  All you have to do is to create a struct type, and enclose 
all of the copied stack variables in a struct of that type.
Jul 22 2004
parent reply "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Forgive me Russ for not quite following the reasoning behind this. Would you
mind clarifying why this delegate example is so much more valuable that,
say, foo() returning an Interface or even a base-class? I mean, wouldn't
something like this provide the equivalent functionality?

IDelegate foo()
{
    class Foo : IDelegate
    {
       int counter;
       dg(int val) {counter += val; return counter;}
     }

    return new Foo;
}

etc. etc.

Certainly, there's a memory allocation there; but that cost can be avoided
at runtime through various means. What's the important bit I'm missing?
(yeah, yeah; apart from a brain ... :~)

- Kris


"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:cdq7ii$1ho2$1 digitaldaemon.com...

 pragma  wrote:

 Here's a few thoughts: what happens when a delegate, executing on it's




own

 stack-frame (copied into GC'd memory for argument's sake) calls another
 function?  Would the called function's stack frame be allocated back on




the

 thread's stack (provided it's not another stack delegate)?  How would




recursion

 within a stack-delegate be handled? ...on the thread's stack in all




cases,

 perhaps?

 The reason why I bring this up is that you may be able to create a




full-on

 reference implementation in D itself (with some liberal application of




assembler

 of course).  All you'd need to to is push the stack-segment and




stack-pointer

 onto the current stack for safe-keeping before setting it to what stack




you're

 going to use (this means that whatever stack space the delegate needs




would have

 to be at least 16 bytes to accomodate this).  This would apply to




entering the

 stack-delegate as well as calling functions that lead out of it.


 I think that perhaps I miscommunicated here.  I don't think that the
 stack frame of a delegate should ever reside on the heap.  Instead, I
 think that some of the local variables of a certain delegate should
 reside inside a struct which resides on the heap.  The difference may be
 small for the programmer, but the difference is massive for
 implementation.  I'm going to try to illustrate.  Please forgive the
 amount of hex is will require...

    int delegate() foo() {
      int counter;
      return stack_frame.dup.delegate int(int val)
        { counter += val; return val; }
    }

    void bar() {
      int delegate() del = foo();
      foreach(int i; <whatever>)
        printf("Value = %d, retval = %d\n", i,del(i));
    }

 Ok, so the code here has a stack delegate which gets duplicated.  What
 happens when bar() calls that delegate?  This is how I imagine the stack
 would look:

    ADDRESS 0xB080 (STACK)
      Local variable 'del' (8 bytes)
        obj = 0x8000
        fptr = 0x4000
      Local variable 'i' (4 bytes)
    ADDRESS 0xB070 (STACK)
      Argument 'this' (4 bytes)
        ptr = 0x8000  /* note: this is the 'obj' from above */

    ADDRESS 0x8000 (HEAP)
      Variable 'counter'

 The point I'm trying to communicate here is that the stack frame for the
 delegate resides on the same stack as everybody else.  That stack frame
 includes a pointer variable which points to a location in the heap where
 copied variables (things that USED TO BE on the stack) now reside.

 A delegate can then call any other function it wants to.  It continues
 to use the stack just like any other function does.  No magic necessary.

 You can see how you can implement this in one of my previous posts in
 this thread.  All you have to do is to create a struct type, and enclose
 all of the copied stack variables in a struct of that type.
Jul 22 2004
next sibling parent David Medlock <amedlock nospam.org> writes:
Kris wrote:

 Forgive me Russ for not quite following the reasoning behind this. Would you
 mind clarifying why this delegate example is so much more valuable that,
 say, foo() returning an Interface or even a base-class? I mean, wouldn't
 something like this provide the equivalent functionality?
 
 IDelegate foo()
 {
     class Foo : IDelegate
     {
        int counter;
        dg(int val) {counter += val; return counter;}
      }
 
     return new Foo;
 }
 
 etc. etc.
 
 Certainly, there's a memory allocation there; but that cost can be avoided
 at runtime through various means. What's the important bit I'm missing?
 (yeah, yeah; apart from a brain ... :~)
 
 - Kris
 
 
 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
 news:cdq7ii$1ho2$1 digitaldaemon.com...
 

pragma  wrote:


Here's a few thoughts: what happens when a delegate, executing on it's




 
 own
 

stack-frame (copied into GC'd memory for argument's sake) calls another
function?  Would the called function's stack frame be allocated back on




 
 the
 

thread's stack (provided it's not another stack delegate)?  How would




 
 recursion
 

within a stack-delegate be handled? ...on the thread's stack in all




 
 cases,
 

perhaps?

The reason why I bring this up is that you may be able to create a




 
 full-on
 

reference implementation in D itself (with some liberal application of




 
 assembler
 

of course).  All you'd need to to is push the stack-segment and




 
 stack-pointer
 

onto the current stack for safe-keeping before setting it to what stack




 
 you're
 

going to use (this means that whatever stack space the delegate needs




 
 would have
 

to be at least 16 bytes to accomodate this).  This would apply to




 
 entering the
 

stack-delegate as well as calling functions that lead out of it.


I think that perhaps I miscommunicated here.  I don't think that the
stack frame of a delegate should ever reside on the heap.  Instead, I
think that some of the local variables of a certain delegate should
reside inside a struct which resides on the heap.  The difference may be
small for the programmer, but the difference is massive for
implementation.  I'm going to try to illustrate.  Please forgive the
amount of hex is will require...

   int delegate() foo() {
     int counter;
     return stack_frame.dup.delegate int(int val)
       { counter += val; return val; }
   }

   void bar() {
     int delegate() del = foo();
     foreach(int i; <whatever>)
       printf("Value = %d, retval = %d\n", i,del(i));
   }

Ok, so the code here has a stack delegate which gets duplicated.  What
happens when bar() calls that delegate?  This is how I imagine the stack
would look:

   ADDRESS 0xB080 (STACK)
     Local variable 'del' (8 bytes)
       obj = 0x8000
       fptr = 0x4000
     Local variable 'i' (4 bytes)
   ADDRESS 0xB070 (STACK)
     Argument 'this' (4 bytes)
       ptr = 0x8000  /* note: this is the 'obj' from above */

   ADDRESS 0x8000 (HEAP)
     Variable 'counter'

The point I'm trying to communicate here is that the stack frame for the
delegate resides on the same stack as everybody else.  That stack frame
includes a pointer variable which points to a location in the heap where
copied variables (things that USED TO BE on the stack) now reside.

A delegate can then call any other function it wants to.  It continues
to use the stack just like any other function does.  No magic necessary.

You can see how you can implement this in one of my previous posts in
this thread.  All you have to do is to create a struct type, and enclose
all of the copied stack variables in a struct of that type.


 


This is what I meant above:

"For each non-static function, have another version internally(AST) 
which is used when an assignment to a delegate happens.  The delegate 
version accesses local variables in its storage space instead of the 
stack(using the data*). Having a different version probably wouldnt be 
that bad, templates do this already."

The delegate version would not change local variables on the stack, but 
as dereferenced through a pointer on the stack.

Doing this requires that an assignment of a function to a delegate 
ensures memory is set aside, whereas assigning a delegate to another 
just copies the data(struct mentioned above) from the delegate.
Jul 23 2004
prev sibling parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Kris wrote:

 Forgive me Russ for not quite following the reasoning behind this. Would you
 mind clarifying why this delegate example is so much more valuable that,
 say, foo() returning an Interface or even a base-class? I mean, wouldn't
 something like this provide the equivalent functionality?
 
 IDelegate foo()
 {
     class Foo : IDelegate
     {
        int counter;
        dg(int val) {counter += val; return counter;}
      }
 
     return new Foo;
 }


A class provides equivalent functionality, but, as you noted, it 
requires much more overhead.  Plus, I would like to declare the whole 
duplicated stack delegate on a single line, rather than having the 
complexity and readability issues of having to declare a whole class or 
struct just for this purpose.

Incidentally, when you are only using a single interface function (and 
you don't need to do anything to the object itself), delegates are 
always more flexible than interfaces.  They can interact with other 
classes, which don't share any common base class or interface, and don't 
require the declaration of some additional Interface.  So I'd prefer 
this code, if I was going to use a class:

int delegate() foo() {
   class Foo {
     int counter;
     this() { counter = 0; }
     int dg(int val) { counter += val; return counter; }
   }

   Foo ret = new Foo;
   return &ret.dg;
}
Jul 23 2004
parent reply "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Thanks!

Sure, delegates are great for certain tasks. I guess my fear is trying to
make them do things that might perhaps be better off approached from the
other direction (a syntax and methodology for non-verbose & low-cost class
instantiation). The latter would expose the same benefits you propose, yet
support more than one callback and provide arguably better encapsulation.

Just an observation though. What you propose is certainly interesting.

- Kris

"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:cdr9gr$23j0$1 digitaldaemon.com...

 Kris wrote:

 Forgive me Russ for not quite following the reasoning behind this. Would




you

 mind clarifying why this delegate example is so much more valuable that,
 say, foo() returning an Interface or even a base-class? I mean, wouldn't
 something like this provide the equivalent functionality?

 IDelegate foo()
 {
     class Foo : IDelegate
     {
        int counter;
        dg(int val) {counter += val; return counter;}
      }

     return new Foo;
 }


 A class provides equivalent functionality, but, as you noted, it
 requires much more overhead.  Plus, I would like to declare the whole
 duplicated stack delegate on a single line, rather than having the
 complexity and readability issues of having to declare a whole class or
 struct just for this purpose.

 Incidentally, when you are only using a single interface function (and
 you don't need to do anything to the object itself), delegates are
 always more flexible than interfaces.  They can interact with other
 classes, which don't share any common base class or interface, and don't
 require the declaration of some additional Interface.  So I'd prefer
 this code, if I was going to use a class:

 int delegate() foo() {
    class Foo {
      int counter;
      this() { counter = 0; }
      int dg(int val) { counter += val; return counter; }
    }

    Foo ret = new Foo;
    return &ret.dg;
 }
Jul 23 2004
parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Kris wrote:

 Thanks!
 
 Sure, delegates are great for certain tasks. I guess my fear is trying to
 make them do things that might perhaps be better off approached from the
 other direction (a syntax and methodology for non-verbose & low-cost class
 instantiation). The latter would expose the same benefits you propose, yet
 support more than one callback and provide arguably better encapsulation.
 
 Just an observation though. What you propose is certainly interesting.


I agree that it would be nice to have some mechanism for very low-cost 
classes.  Whether that is a "better" approach or not...well, we'll 
probably not know until they are available! :)

It has occurred to me before that, if a single-function interface is 
just a delegate, then isn't a multifunction interface just a struct of 
delegates, sort of?

This interface:
   interface FooI {
     int foo();
     void bar(int);
     char baz(char[]);
   }

is a lot like this struct:
   struct FooS {
     int delegate() foo;
     void delegate(int) bar;
     char delegate(char[]) baz;

     void delegate() do_delete;
       /* deletes the object */
     Object delegate() getObj;
       /* gets the obj pointer, so you can cast it */
   }

so you could do this:
   class MyClass {
     int asdf() {...}
     void jkl(int) {...}
     char qwerty(char[]) {...}

     ~this() {...}

     // this function would be unnecessary if
     // stack duplicate delegates were implemented
     Object getMe() { return this; }
   }

   FooS *GetFooS(MyClass obj) {
     FooS *ret = new FooS[1];
     ret.foo = &obj.asdf;
     ret.bar = &obj.jkl;
     ret.baz = &obj.qwerty;
     ret.do_delete = &.obj~this;
     ret.getObj = &obj.getMe;
     return ret;
   }

So a struct of delegates is more adaptible than an interface...the names 
in the class don't have to match the names in the interface.  Of course, 
this sort of thing has a couple of big disadvantages:
   * Size.  Since each ptr is a delegate, it takes 8 bytes.  Plus, each 
object has to have its own struct...no shared vtbl
   * Hard to declare

However, if we all agreed that this sort of thing was useful, then it 
might be possible to devise (for 2.0) some new, better solution for this.
Jul 23 2004
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <cdn6a7$76i$1 digitaldaemon.com>, Russ Lewis says...

I am a big proponent of having some functionality that enables us to 
copy stack delegates.  We don't want copying to be the default, however. 


I agree.  The only reason I suggested that was because it would allow us to use
the existing "auto" keyword in a consistent manner.


Sean
Jul 22 2004
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Sean Kelly wrote:

 In article <cdn6a7$76i$1 digitaldaemon.com>, Russ Lewis says...
 

I am a big proponent of having some functionality that enables us to 
copy stack delegates.  We don't want copying to be the default, however. 


 
 
 I agree.  The only reason I suggested that was because it would allow us to use
 the existing "auto" keyword in a consistent manner.


I apologize, but I'm clueless about how 'auto' comes in here.  Can you 
give an example?  Sounds interesting.
Jul 22 2004
parent reply Sean Kelly <sean f4.ca> writes:
In article <cdp115$vo6$1 digitaldaemon.com>, Russ Lewis says...

Sean Kelly wrote:

 In article <cdn6a7$76i$1 digitaldaemon.com>, Russ Lewis says...
 

I am a big proponent of having some functionality that enables us to 
copy stack delegates.  We don't want copying to be the default, however. 


 
 
 I agree.  The only reason I suggested that was because it would allow us to use
 the existing "auto" keyword in a consistent manner.


I apologize, but I'm clueless about how 'auto' comes in here.  Can you 
give an example?  Sounds interesting.


It was in an earlier post in this thread.  Here's an example:







In the above code, d1 will get a copy of the stack frame that it can carry
outside of func, while d2 will not since it is declared as "auto."  This seems
consistent with the existing auto semantics for variables.  Only catch is that,
as you said, I think the default behavior should remain how it is.


Sean
Jul 22 2004
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Sean Kelly wrote:






 
 In the above code, d1 will get a copy of the stack frame that it can carry
 outside of func, while d2 will not since it is declared as "auto."  This seems
 consistent with the existing auto semantics for variables.  Only catch is that,
 as you said, I think the default behavior should remain how it is.


Ok, I see what you're saying.  The syntax makes a lot of sense, except 
that I would prefer not to overload the "auto" keyword.  Maybe I'm the 
only one, but it didn't make immediate sense to me.  Remember how the 
keyword "static" got overloaded in C...ick.

How would you declare a delegate literal using the above syntax, 
however?  Would you declare
   void delegate() foo = auto delegate void() {...}
?

Ok, weird ponder here.  I've been playing with the idea that it would be 
nice to declare a delegate that has any arbitrary 'this' pointer, using 
the following syntax:
   MyClass foo = <whatever>;
   void delegate() del = foo.delegate void() { foo.x(); foo.y(); }

The cool thing is, I just realized that that syntax makes stack 
duplicates easier to create:
   void foo() {
     void delegate() del = stack_frame.dup.delegate void() { ... }
   }

Then, you realize that the current stack delegate syntax can be viewed 
as syntax sugar:
   void bar() {
     void delegate() d1 = delegate void() {...}
     void delegate() d2 = stack_frame.delegate void() {...}
   }
In the code above, both delegate literals are non-copying delegates.



Thoughts?
Jul 22 2004
next sibling parent David Medlock <amedlock nospam.org> writes:
Russ Lewis wrote:

 Sean Kelly wrote:
 







 In the above code, d1 will get a copy of the stack frame that it can 
 carry
 outside of func, while d2 will not since it is declared as "auto."  
 This seems
 consistent with the existing auto semantics for variables.  Only catch 
 is that,
 as you said, I think the default behavior should remain how it is.


 
 
 Ok, I see what you're saying.  The syntax makes a lot of sense, except 
 that I would prefer not to overload the "auto" keyword.  Maybe I'm the 
 only one, but it didn't make immediate sense to me.  Remember how the 
 keyword "static" got overloaded in C...ick.
 
 How would you declare a delegate literal using the above syntax, 
 however?  Would you declare
   void delegate() foo = auto delegate void() {...}
 ?
 
 Ok, weird ponder here.  I've been playing with the idea that it would be 
 nice to declare a delegate that has any arbitrary 'this' pointer, using 
 the following syntax:
   MyClass foo = <whatever>;
   void delegate() del = foo.delegate void() { foo.x(); foo.y(); }
 
 The cool thing is, I just realized that that syntax makes stack 
 duplicates easier to create:
   void foo() {
     void delegate() del = stack_frame.dup.delegate void() { ... }
   }
 
 Then, you realize that the current stack delegate syntax can be viewed 
 as syntax sugar:
   void bar() {
     void delegate() d1 = delegate void() {...}
     void delegate() d2 = stack_frame.delegate void() {...}
   }
 In the code above, both delegate literals are non-copying delegates.
 
 
 
 Thoughts?
 


Interesting.

I think you could almost make this work with templates alone(using a 
class and opCmd) if function themselves had a *.size* property to reveal 
the amount of stack space needed for the functions variables.  On first 
call of the function the space is allocated/initialized, with subsequent 
calls just referring to the stack frame.

I know the behavior I would like to use delegates for: like that in 
lua/scheme/Javascript (true closures).  It is just a hard feature to add 
with C because functions are not created, just declared.  When you 
specify 'static int a' within a inner function, static really means 
'singular' because no matter where it is accessed, there is only one.

If delegates were forced to be created like other objects, implementing 
the underlying semantics would probably be very simple.
Jul 22 2004
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
In article <cdp2td$10ib$1 digitaldaemon.com>, Russ Lewis says...

How would you declare a delegate literal using the above syntax, 
however?  Would you declare
   void delegate() foo = auto delegate void() {...}


Yeah I suppose.  And this does look a tad weird.


Ok, weird ponder here.  I've been playing with the idea that it would be 
nice to declare a delegate that has any arbitrary 'this' pointer, using 
the following syntax:
   MyClass foo = <whatever>;
   void delegate() del = foo.delegate void() { foo.x(); foo.y(); }


Interesting idea.  The only weird thing here is that this feature would enable a
user to break encapsulation rules by accessing hidden members via these
delegates.


Then, you realize that the current stack delegate syntax can be viewed 
as syntax sugar:
   void bar() {
     void delegate() d1 = delegate void() {...}
     void delegate() d2 = stack_frame.delegate void() {...}
   }
In the code above, both delegate literals are non-copying delegates.


Interesting idea.  Never thought of nesting keyowrds in this manner.  I'm not
sure if I like it though, I'll have to think about it for a bit.  Am I right in
assuming that stack_frame is a keyword as well and not a type sitting off
somewhere?


Sean
Jul 22 2004
next sibling parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Sean Kelly wrote:

Ok, weird ponder here.  I've been playing with the idea that it would be 
nice to declare a delegate that has any arbitrary 'this' pointer, using 
the following syntax:
  MyClass foo = <whatever>;
  void delegate() del = foo.delegate void() { foo.x(); foo.y(); }


 
 Interesting idea.  The only weird thing here is that this feature would enable
a
 user to break encapsulation rules by accessing hidden members via these
 delegates.


You've confused me again here.  Assuming that the code in the delegate 
can't access anything that the function can access, how does this break 
encapsulation?

You can already sort of do this in D, although it requires you to create 
a wrapper class that has no purpose other than to give you a delegate. 
You can do this in D using the following code:

   MyClass foo = <whatever>;
   void hand_coded_delegate(void *arg) {
     MyClass foo = cast(MyClass)arg;
     foo.x(); foo.y();
   }
   union {
     void delegate() del;
     struct {
       void *arg;
       void function(void*) fptr;
     }
   } convert;
   convert.arg = cast(void*)foo;
   convert.fptr = &hand_coded_delegate;
   void delegate() del = convert.del;

So you see, the delegate can't do anything that the enclosing block 
can't do.


Then, you realize that the current stack delegate syntax can be viewed 
as syntax sugar:
  void bar() {
    void delegate() d1 = delegate void() {...}
    void delegate() d2 = stack_frame.delegate void() {...}
  }
In the code above, both delegate literals are non-copying delegates.


 
 Interesting idea.  Never thought of nesting keyowrds in this manner.  I'm not
 sure if I like it though, I'll have to think about it for a bit.  Am I right in
 assuming that stack_frame is a keyword as well and not a type sitting off
 somewhere?


Yes.  My assumption is that stack_frame is a new keyword, which is a 
pointer to an anonymous structure type which models the local function 
variables.  It's sort of like 'this' for a stack frame.
Jul 22 2004
parent Sean Kelly <sean f4.ca> writes:
In article <cdp6ms$12g3$1 digitaldaemon.com>, Russ Lewis says...

Sean Kelly wrote:

Ok, weird ponder here.  I've been playing with the idea that it would be 
nice to declare a delegate that has any arbitrary 'this' pointer, using 
the following syntax:
  MyClass foo = <whatever>;
  void delegate() del = foo.delegate void() { foo.x(); foo.y(); }


 
 Interesting idea.  The only weird thing here is that this feature would enable
a
 user to break encapsulation rules by accessing hidden members via these
 delegates.


You've confused me again here.  Assuming that the code in the delegate 
can't access anything that the function can access, how does this break 
encapsulation?


Oh I misunderstood.  When you said "hidden this pointer" I assumed you meant
that the delegate would behave as if it were a first class member of foo.
Interesting idea.


Sean
Jul 22 2004
prev sibling parent reply pragma <EricAnderton at yahoo dot com> <pragma_member pathlink.com> writes:
I have a few friendly questions.  :)

I've followed the conversation so far, but one thing still escapes me: what
would such a closure accomplish that a class with opCall cannot?  Is this trying
to achieve some kind of efficency gain or just a better syntax (or both)?

In article <cdp5cu$11ui$1 digitaldaemon.com>, Sean Kelly says...

Then, you realize that the current stack delegate syntax can be viewed 
as syntax sugar:
   void bar() {
     void delegate() d1 = delegate void() {...}
     void delegate() d2 = stack_frame.delegate void() {...}
   }
In the code above, both delegate literals are non-copying delegates.


Interesting idea.  Never thought of nesting keyowrds in this manner.  I'm not
sure if I like it though, I'll have to think about it for a bit.  Am I right in
assuming that stack_frame is a keyword as well and not a type sitting off
somewhere?


Personally, I like how succinct this concept is.  I have the same question: is
'stack_frame' a new keyword or some type?  If it is keyword, wouldn't
overloading 'new' be a little more appropriate, since it's getting stack frame
all to itself?


   void bar() {
     void delegate() d1 = delegate void() {...}
     void delegate() d2 = new delegate void() {...}
   }
Jul 22 2004
next sibling parent Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
pragma  wrote:

 I have a few friendly questions.  :)
 
 I've followed the conversation so far, but one thing still escapes me: what
 would such a closure accomplish that a class with opCall cannot?  Is this
trying
 to achieve some kind of efficency gain or just a better syntax (or both)?


It's both.  If you implement this sort of functionality with a class 
with opCall, you have an unnecessary level of indirection; the 'this' 
argument to the delegate will be a pointer to your wrapper class, and 
you have to then read a member variable of the class to get the pointer 
to 'foo'.  Worse, you have to allocate, construct, and then later 
garbage collect a new class object on the heap.

In my reply to Sean, I wrote some D code that avoids all of these 
inefficiencies.  However, it's many lines of code.  So I'm suggesting 
that it would be good to use my syntax as it is far cleaner, 
easier-to-read code.


 In article <cdp5cu$11ui$1 digitaldaemon.com>, Sean Kelly says...
 

Then, you realize that the current stack delegate syntax can be viewed 
as syntax sugar:
  void bar() {
    void delegate() d1 = delegate void() {...}
    void delegate() d2 = stack_frame.delegate void() {...}
  }
In the code above, both delegate literals are non-copying delegates.


Interesting idea.  Never thought of nesting keyowrds in this manner.  I'm not
sure if I like it though, I'll have to think about it for a bit.  Am I right in
assuming that stack_frame is a keyword as well and not a type sitting off
somewhere?


 
 Personally, I like how succinct this concept is.  I have the same question: is
 'stack_frame' a new keyword or some type?  If it is keyword, wouldn't
 overloading 'new' be a little more appropriate, since it's getting stack frame
 all to itself?
 

  void bar() {
    void delegate() d1 = delegate void() {...}
    void delegate() d2 = new delegate void() {...}
  }




This might make a lot of sense.  To its favor, it makes more explicit 
that new memory is being allocated (although using .dup is pretty 
explicit, as well).  One strike against it, however, is that it doesn't 
make immediately clear that something is being copied.  New typically 
allocates uninitialized data.
Jul 22 2004
prev sibling next sibling parent Sean Kelly <sean f4.ca> writes:
In article <cdp6ub$12kh$1 digitaldaemon.com>, pragma <EricAnderton at yahoo dot
com> says...

I have a few friendly questions.  :)

I've followed the conversation so far, but one thing still escapes me: what
would such a closure accomplish that a class with opCall cannot?  Is this trying
to achieve some kind of efficency gain or just a better syntax (or both)?


Nothing really, but for some additional simplicity.  A returned class needs its
own external declaration while delegates can be declared as a part of the
function prototype.  Also, returning a class would require the programmer to
manually copy the data he cares about while this dynamic closure idea would make
the process automatic.


Seam
Jul 22 2004
prev sibling parent reply "Blandger" <zeroman prominvest.com.ua> writes:
"pragma" <EricAnderton at yahoo dot compragma_member pathlink.com> wrote in
message news:cdp6ub$12kh$1 digitaldaemon.com...

 I have a few friendly questions.  :)

 I've followed the conversation so far, but one thing still escapes me:


what

 would such a closure accomplish that a class with opCall cannot?  Is this


trying

 to achieve some kind of efficency gain or just a better syntax (or both)?


I tryed to folow you conversation. And now I have only one wish.

Pls, could someone of you add some of  this (or other) useful/essential
examples to the : dsource.org/tutorial / delegates section (or Wiki)? I
think many people will be grateful to you (especially me). Thanks.

-----
Yuriy.
Jul 23 2004
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
I posted an example, but need to edit it.  How do I do that?

Blandger wrote:

 "pragma" <EricAnderton at yahoo dot compragma_member pathlink.com> wrote in
 message news:cdp6ub$12kh$1 digitaldaemon.com...
 

I have a few friendly questions.  :)

I've followed the conversation so far, but one thing still escapes me:


 
 what
 

would such a closure accomplish that a class with opCall cannot?  Is this


 
 trying
 

to achieve some kind of efficency gain or just a better syntax (or both)?


 
 
 I tryed to folow you conversation. And now I have only one wish.
 
 Pls, could someone of you add some of  this (or other) useful/essential
 examples to the : dsource.org/tutorial / delegates section (or Wiki)? I
 think many people will be grateful to you (especially me). Thanks.
 
 -----
 Yuriy.
Jul 26 2004
parent reply "Blandger" <zeroman aport.ru> writes:
"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:ce3u1v$1h7j$1 digitaldaemon.com...


 I posted an example, but need to edit it.  How do I do that?


Sorry I didn't understand a question.

I'd like to see good delegete examples in the dsource.org -> tutorial
section. To post it you can apply to Brad. He is admin there 'brad at
dsource dot org'

You can post them in the appropriate D Wiki page. Though I don't have a
experience using it :(

----
Yuriy.
Jul 26 2004
parent reply Russ Lewis <spamhole-2001-07-16 deming-os.org> writes:
Blandger wrote:

 "Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
 news:ce3u1v$1h7j$1 digitaldaemon.com...
 

I posted an example, but need to edit it.  How do I do that?


 
 Sorry I didn't understand a question.
 
 I'd like to see good delegete examples in the dsource.org -> tutorial
 section. To post it you can apply to Brad. He is admin there 'brad at
 dsource dot org'


I have posted 4 examples.  See 
http://dsource.org/tutorials/index.php?show_topic=Delegates

I'll email Brad to ask him how to fix that one broken title...
Jul 27 2004
parent "Blandger" <zeroman aport.ru> writes:
"Russ Lewis" <spamhole-2001-07-16 deming-os.org> wrote in message
news:ce6230$2e0l$1 digitaldaemon.com...

 I have posted 4 examples.  See
 http://dsource.org/tutorials/index.php?show_topic=Delegates

 I'll email Brad to ask him how to fix that one broken title...


Thanks !!

-----
Yuriy.
Jul 27 2004