www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - One minute to twelve: last chance to fix D2 ABI?

reply Lionello Lunesu <lio lunesu.remove.com> writes:
The following has been discussed for years, and the biggest problem
(years ago) was that it was too late to change the D (1) ABI.

Is the D2 ABI compatible with D1? Should it be? If not, please consider
the following change:

alias void delegate() action_d;
void Callback() { }             // global
action_d = &Callback;           // should work

A function should be assignable to a delegate (but not vice versa*.)
This can work, since the code invoking the delegate will end up calling
the function with a context pointer ("this") in a register that's simply
ignored by the code in the function.

This is fairly easy too: the calling convention for functions and
delegates should be identical, except for the context pointer, which
should be stored in a register that's unused in the function calling
convention.

The reason for this change is that many times classes expose delegates
for callbacks, but it's impossible to bind a function to those
delegates. The code above can easily be made to compile:

action_d = { Callback(); }; //wrapped, works

but causes a dummy function to be called, only because the parameter
ABI's don't overlap:

_D1t4mainFZi12__dgliteral1MFZi  comdat            ;delegate literal
        assume  CS:_D1t4mainFZi12__dgliteral1MFZi
L0:             push    EAX
                call    near ptr _D1t8FunctionFZi ;global function
                pop     ECX
                ret
_D1t4mainFZi12__dgliteral1MFZi  ends

Of course, all of this applies to extern "D" code only.

* Binding a delegate to a function pointer will always need a (dynamic)
thunk, because the function will have been invoked without any context
pointer and it's the thunk's responsibility to assign the correct
context pointer.
Mar 02 2010
next sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 02 Mar 2010 21:02:51 -0500, Lionello Lunesu  
<lio lunesu.remove.com> wrote:

 The following has been discussed for years, and the biggest problem
 (years ago) was that it was too late to change the D (1) ABI.

 Is the D2 ABI compatible with D1? Should it be? If not, please consider
 the following change:

 alias void delegate() action_d;
 void Callback() { }             // global
 action_d = &Callback;           // should work

 A function should be assignable to a delegate (but not vice versa*.)
 This can work, since the code invoking the delegate will end up calling
 the function with a context pointer ("this") in a register that's simply
 ignored by the code in the function.


Except this also effects member functions, nested functions, etc.


 This is fairly easy too: the calling convention for functions and
 delegates should be identical, except for the context pointer, which
 should be stored in a register that's unused in the function calling
 convention.


This breaks the synergy of returns in EAX and params in EAX.


 The reason for this change is that many times classes expose delegates
 for callbacks, but it's impossible to bind a function to those
 delegates. The code above can easily be made to compile:

 action_d = { Callback(); }; //wrapped, works

 but causes a dummy function to be called, only because the parameter
 ABI's don't overlap:

 _D1t4mainFZi12__dgliteral1MFZi  comdat            ;delegate literal
         assume  CS:_D1t4mainFZi12__dgliteral1MFZi
 L0:             push    EAX
                 call    near ptr _D1t8FunctionFZi ;global function
                 pop     ECX
                 ret
 _D1t4mainFZi12__dgliteral1MFZi  ends

 Of course, all of this applies to extern "D" code only.

 * Binding a delegate to a function pointer will always need a (dynamic)
 thunk, because the function will have been invoked without any context
 pointer and it's the thunk's responsibility to assign the correct
 context pointer.


It would be fairly easy for D (or anyone else) to implicitly define a  
simple adapter that stores the function pointer in the delegate's this  
pointer. The adapter function pointer points to one of two common stub  
functions which either calls the this pointer or pops a value off the  
stack into EAX and then calls the this pointer.
Mar 02 2010
parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
On 3-3-2010 10:49, Robert Jacques wrote:

 On Tue, 02 Mar 2010 21:02:51 -0500, Lionello Lunesu
 <lio lunesu.remove.com> wrote:

 The following has been discussed for years, and the biggest problem
 (years ago) was that it was too late to change the D (1) ABI.

 Is the D2 ABI compatible with D1? Should it be? If not, please consider
 the following change:

 alias void delegate() action_d;
 void Callback() { }             // global
 action_d = &Callback;           // should work

 A function should be assignable to a delegate (but not vice versa*.)
 This can work, since the code invoking the delegate will end up calling
 the function with a context pointer ("this") in a register that's simply
 ignored by the code in the function.


 
 Except this also effects member functions, nested functions, etc.


Well, yes. That doesn't make it any harder or less possible.


 This is fairly easy too: the calling convention for functions and
 delegates should be identical, except for the context pointer, which
 should be stored in a register that's unused in the function calling
 convention.


 
 This breaks the synergy of returns in EAX and params in EAX.


How exactly? I don't understand. What does the return value have to do
with any of this?


 The reason for this change is that many times classes expose delegates
 for callbacks, but it's impossible to bind a function to those
 delegates. The code above can easily be made to compile:

 action_d = { Callback(); }; //wrapped, works

 but causes a dummy function to be called, only because the parameter
 ABI's don't overlap:

 _D1t4mainFZi12__dgliteral1MFZi  comdat            ;delegate literal
         assume  CS:_D1t4mainFZi12__dgliteral1MFZi
 L0:             push    EAX
                 call    near ptr _D1t8FunctionFZi ;global function
                 pop     ECX
                 ret
 _D1t4mainFZi12__dgliteral1MFZi  ends

 Of course, all of this applies to extern "D" code only.

 * Binding a delegate to a function pointer will always need a (dynamic)
 thunk, because the function will have been invoked without any context
 pointer and it's the thunk's responsibility to assign the correct
 context pointer.


 
 It would be fairly easy for D (or anyone else) to implicitly define a
 simple adapter that stores the function pointer in the delegate's this
 pointer. The adapter function pointer points to one of two common stub
 functions which either calls the this pointer or pops a value off the
 stack into EAX and then calls the this pointer.


It is easy. Like I should, you could just enter a pair of {} and it
would work. And to prevent context capturing (and heap allocation) you
could use such an adapter function. But without changing the ABI, even
in the best case you would have a wrapper function with an extra call
operation, which simply should not be necessary.

L.
Mar 03 2010
parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 03 Mar 2010 05:28:46 -0500, Lionello Lunesu  
<lio lunesu.remove.com> wrote:

 On 3-3-2010 10:49, Robert Jacques wrote:

 On Tue, 02 Mar 2010 21:02:51 -0500, Lionello Lunesu




[snip]

 Except this also effects member functions, nested functions, etc.


 Well, yes. That doesn't make it any harder or less possible.


It greatly increases the scope of the change, which does increase the  
implementation difficulty.

[snip]

 This breaks the synergy of returns in EAX and params in EAX.


 How exactly? I don't understand. What does the return value have to do
 with any of this?


The ABI has been designed so that there is a synergy between the return  
value of a function (EAX) and where the last parameter to the next  
function is (also EAX). This allows function chaining to occur without  
mucking with register moves, etc.


 It would be fairly easy for D (or anyone else) to implicitly define a
 simple adapter that stores the function pointer in the delegate's this
 pointer. The adapter function pointer points to one of two common stub
 functions which either calls the this pointer or pops a value off the
 stack into EAX and then calls the this pointer.


 It is easy. Like I should, you could just enter a pair of {} and it
 would work. And to prevent context capturing (and heap allocation) you
 could use such an adapter function. But without changing the ABI, even
 in the best case you would have a wrapper function with an extra call
 operation, which simply should not be necessary.

 L.


At the cost of forcing every single member function call to do an  
additional stack push? DMD doesn't pass args in ECX and EDX for a reason.  
Adapting free functions to delegates is a rare enough operation that  
making everything else slower to save one function call isn't worth it.
Mar 03 2010
parent reply Lionello Lunesu <lio lunesu.remove.com> writes:
On 3-3-2010 23:51, Robert Jacques wrote:

 At the cost of forcing every single member function call to do an
 additional stack push? DMD doesn't pass args in ECX and EDX for a
 reason. Adapting free functions to delegates is a rare enough operation
 that making everything else slower to save one function call isn't worth
 it.


That's a great point. I hadn't thought about it. Thanks.

L.
Mar 04 2010
parent "Robert Jacques" <sandford jhu.edu> writes:
On Thu, 04 Mar 2010 05:00:46 -0500, Lionello Lunesu  
<lio lunesu.remove.com> wrote:


 On 3-3-2010 23:51, Robert Jacques wrote:

 At the cost of forcing every single member function call to do an
 additional stack push? DMD doesn't pass args in ECX and EDX for a
 reason. Adapting free functions to delegates is a rare enough operation
 that making everything else slower to save one function call isn't worth
 it.


 That's a great point. I hadn't thought about it. Thanks.

 L.


No problem. I'd never have thought about it and think it through without  
you raising your point. The really odd thing is that 2-word structs are  
returned in register, but not sent in registers. Maybe this will change in  
x86.
Mar 04 2010
prev sibling next sibling parent Daniel Keep <daniel.keep.lists gmail.com> writes:
That or just use something like this: http://gist.github.com/263824
Mar 02 2010
prev sibling next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
I consider the function-delegate thing a reasonably solved problem, starting in
the next release of Phobos.  See:

http://dsource.org/projects/phobos/changeset/1428
Mar 02 2010
next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
dsimcha wrote:

 I consider the function-delegate thing a reasonably solved problem, starting in
 the next release of Phobos.  See:
 
 http://dsource.org/projects/phobos/changeset/1428


Nice.

Btw, I'm guessing that pragma(msg) in line 562 should be commented out 
before release? :)
Mar 02 2010
prev sibling next sibling parent Lionello Lunesu <lio lunesu.remove.com> writes:
On 3-3-2010 11:51, dsimcha wrote:

 I consider the function-delegate thing a reasonably solved problem, starting in
 the next release of Phobos.  See:
 
 http://dsource.org/projects/phobos/changeset/1428


But there's still the unnecessary call that has to be made. My point was
that it fixable by just adjusting the ABI a little.

Perhaps the status quo isn't that bad if the compiler could use
toDelegate transparently whenever a function pointer is assigned to a
delegate.

L.
Mar 03 2010
prev sibling parent Max Samukha <spambox d-coding.com> writes:
On 03.03.2010 5:51, dsimcha wrote:

 I consider the function-delegate thing a reasonably solved problem, starting in
 the next release of Phobos.  See:

 http://dsource.org/projects/phobos/changeset/1428


Your code works only if return/parameter types are accessible from 
std.functional. An implementation that handles any types is way more 
complex.

You'll have to extract storage classes from the function pointer type 
and then reconstruct the delegate type using those extracted storage 
classes combined with the parameter types as elements of the parameter 
type tuple. An example demonstrating what is meant:

auto toDelegate(F)(F fp) {
     ...
     alias ParameterTypeTuple!(F) Params;

     // CTFE generating the delegate type string
     static string genDgStr() {
         auto paramSCs = extractParamSCs!(F);
         auto dgStr = extractReturnSCs!(F) ~ " ReturnType!(F) delegate(";
         foreach (i, paramSC; paramSCs) {
             if (i) dgStr ~= ", ";
             dgStr ~= paramSCs[i] ~ " Params[" ~ to!string(i) ~ "]";
         }
         return dgStr ~ ")";
     }

     mixin ("alias " ~ genDgStr() ~ " Dg;");
     ...
}



So, for the function type 'ref SomeAlienStruct function(ref 
SomeAlienStruct s)', the mixed-in delegate type alias will be:

alias ref ReturnType!(F) delegate(ref Params[0]) Dg;

Note that there are no stringof's involved in the alias declaration.

Terrible.
Mar 03 2010
prev sibling parent reply Steve Teale <steve.teale britseyeview.com> writes:
 

 * Binding a delegate to a function pointer will always need a (dynamic)
 thunk, because the function will have been invoked without any context
 pointer and it's the thunk's responsibility to assign the correct
 context pointer.


At present, D's way of handling delegates requires patching GCC for the 
GDC implementation. If an ABI change was made it would be nice if it 
fitted into the GCC trampoline model.

Steve
Mar 02 2010
parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 03 Mar 2010 01:00:46 -0500, Steve Teale  
<steve.teale britseyeview.com> wrote:

 * Binding a delegate to a function pointer will always need a (dynamic)
 thunk, because the function will have been invoked without any context
 pointer and it's the thunk's responsibility to assign the correct
 context pointer.


 At present, D's way of handling delegates requires patching GCC for the
 GDC implementation. If an ABI change was made it would be nice if it
 fitted into the GCC trampoline model.

 Steve


Um, how exactly? I just read the wikipedia page on GCC trampoline and it  
a) targeted only at nested functions b) utilizes self-modifying code which  
is disabled on any sensible platform c) would require dynamic allocation  
and d) actually uses more memory than D's method. Some time ago there was  
a link to an excellent article (by Walter? Don?) on different ways to do  
this. I remember d's way coming out the best, but I've seemed to have lost  
the link.
Mar 02 2010
parent reply Steve Teale <steve.teale britseyeview.com> writes:
On Wed, 03 Mar 2010 01:38:58 -0500, Robert Jacques wrote:


 On Wed, 03 Mar 2010 01:00:46 -0500, Steve Teale
 <steve.teale britseyeview.com> wrote:

 * Binding a delegate to a function pointer will always need a
 (dynamic) thunk, because the function will have been invoked without
 any context pointer and it's the thunk's responsibility to assign the
 correct context pointer.


 At present, D's way of handling delegates requires patching GCC for the
 GDC implementation. If an ABI change was made it would be nice if it
 fitted into the GCC trampoline model.

 Steve


 
 Um, how exactly? I just read the wikipedia page on GCC trampoline and it
 a) targeted only at nested functions b) utilizes self-modifying code
 which is disabled on any sensible platform c) would require dynamic
 allocation and d) actually uses more memory than D's method. Some time
 ago there was a link to an excellent article (by Walter? Don?) on
 different ways to do this. I remember d's way coming out the best, but
 I've seemed to have lost the link.


I'm just quoting from something I noticed a couple of days ago. In David 
Friedman's make file for GDC there is:


D_EXTRA_DEFINES += -DD_NO_TRAMPOLINES=1

Steve
Mar 03 2010
parent "Robert Jacques" <sandford jhu.edu> writes:
On Wed, 03 Mar 2010 10:34:48 -0500, Steve Teale  
<steve.teale britseyeview.com> wrote:


 On Wed, 03 Mar 2010 01:38:58 -0500, Robert Jacques wrote:


 On Wed, 03 Mar 2010 01:00:46 -0500, Steve Teale
 <steve.teale britseyeview.com> wrote:

 * Binding a delegate to a function pointer will always need a
 (dynamic) thunk, because the function will have been invoked without
 any context pointer and it's the thunk's responsibility to assign the
 correct context pointer.


 At present, D's way of handling delegates requires patching GCC for the
 GDC implementation. If an ABI change was made it would be nice if it
 fitted into the GCC trampoline model.

 Steve


 Um, how exactly? I just read the wikipedia page on GCC trampoline and it
 a) targeted only at nested functions b) utilizes self-modifying code
 which is disabled on any sensible platform c) would require dynamic
 allocation and d) actually uses more memory than D's method. Some time
 ago there was a link to an excellent article (by Walter? Don?) on
 different ways to do this. I remember d's way coming out the best, but
 I've seemed to have lost the link.


 I'm just quoting from something I noticed a couple of days ago. In David
 Friedman's make file for GDC there is:


 GCC.
 D_EXTRA_DEFINES += -DD_NO_TRAMPOLINES=1

 Steve


I think that's because GCC's internal nested function AST construct would  
normally use trampolines, but D defines them as delegates (IIRC). As this  
hack is around GCC's AST handling and not D's ABI, it would be required  
regardless. BTW nested function support is a language extension specific  
to GCC.
Mar 03 2010