www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Using a C++ class in a D associative array

reply Jacob Carlborg <doob me.com> writes:
I'm playing around with using the DMD frontend as a library. I wanted to 
store some nodes from the AST in a set. Since there doesn't seem to be a 
set container that is shipped with Phobos I wrapped an associative array 
and added some functions to use the AA as a set.

To my surprise when I did that my code started to behave strangely. It 
started to print out things like this:

foo 0x10bc70800
bar 0x10bc70c00

I've been banging my head against the wall for several days trying to 
figure out why the above output was printed. I've been modifying my own 
code and removed as much as possible, but it didn't contained any calls 
to any "write" or "printf" functions. The problem seemed to appear when 
I put an AST node in the set (associative array). So I started to look 
through the AST classes and looking for overrides of "toHash". None of 
the AST nodes override "toHash". Then I noticed a suspicious "printf" 
call in a "print" method in the RootObject class in the DMD source code 
[1] (this class is the root of all classes in DMD). I removed the call 
to "printf" and now the output was gone. Trying to figure out what was 
calling this "print" method, that had called "printf", I added a failing 
assert. Thanks to the druntime printing out the stacktrace of an 
uncaught exception, I got this output (reduce):


??:? _d_assertp [0xd4d3311]
../../../.dub/packages/dmd-master/dmd/src/dmd/func.d:494 
_ZN15FuncDeclaration5printEv [0xd41d97c]
??:? const nothrow  trusted ulong object.TypeInfo_Class.getHash(scope 
const(void*)) [0xd4c825f]
??:? _aaGetY [0xd4e5adc]
~/.dvm/compilers/dmd-2.081.0/osx/bin/../../src/druntime/import/object.d:326 
pure nothrow  safe void 
dlp.core.set.Set!(dmd.func.FuncDeclaration).Set.put(dmd.func.FuncDeclaration) 
[0xd30cab9]

At the third line there's a call from object.TypeInfo_Class.getHash. I 
looked up to see what the "getHash" method is doing in druntime [2], the 
method looks like this:

override size_t getHash(scope const void* p)  trusted const
{
     auto o = *cast(Object*)p;
     return o ? o.toHash() : 0;
}

The method is basically calling "toHash" on the passed in object. But 
none of the AST nodes were overriding "toHas", hmm.

Then I started to think, all the AST nodes in DMD are C++ classes (both 
LDC and GDC are written in C++ and need to use the AST). Without really 
thinking of it I had put instance of C++ classes as keys in a D 
associative array. This is, for some reason, calling 
object.TypeInfo_Class.getHash which only expects to operate on D 
classes. I'm guessing that somehow what is usually the "toHash" method 
in a D class matched up with the "print" method in the C++ class.

All this just compiled without any error or warnings. No runtime 
exceptions or asserts were triggered. I just got a really weird behavior.

[1] 
https://github.com/dlang/dmd/blob/c3f6320d59ec14d6fa81a18f92b59393671b346a/src/dmd/root/rootobject.d#L55-L58

[2] 
https://github.com/dlang/druntime/blob/430585650930797369eeb5b3b142faba10572f10/src/object.d#L1712-L1716

-- 
/Jacob Carlborg
Aug 20 2018
next sibling parent reply kinke <noone nowhere.com> writes:
On Monday, 20 August 2018 at 22:16:09 UTC, Jacob Carlborg wrote:

 At the third line there's a call from 
 object.TypeInfo_Class.getHash. I looked up to see what the 
 "getHash" method is doing in druntime [2], the method looks 
 like this:

 override size_t getHash(scope const void* p)  trusted const
 {
     auto o = *cast(Object*)p;
     return o ? o.toHash() : 0;
 }


I guess the compiler uses the AA key type's TypeInfo, which is 
available for extern(C++) classes too. The 
TypeInfo_Class.getHash() then uses the dynamic type via virtual 
call, (wrongly) assuming it's always a D class. For an 
extern(C++) class, it will either call another virtual function 
(no inherited virtual functions from Object), what you were 
seeing, or attempt to call... something. ;)


 All this just compiled without any error or warnings. No 
 runtime exceptions or asserts were triggered. I just got a 
 really weird behavior.


This is somewhat special due to the common TypeInfo.getHash() 
signature for all kinds of types, and so just taking a hairy 
void* pointer to the byte/real/static array/AA/object/… to be 
hashed.

Polishing C++ interop with extern(C++) classes (matching 
ctors/dtors, mixed class hiearchies, ability to easily 
allocate/construct/destruct/free on the other language side etc.) 
has started with v2.081 and is still on-going; there are probably 
more places in druntime silently assuming a D class.
Aug 20 2018
parent reply Jacob Carlborg <doob me.com> writes:
On 2018-08-21 02:07, kinke wrote:

 On Monday, 20 August 2018 at 22:16:09 UTC, Jacob Carlborg wrote:

 At the third line there's a call from object.TypeInfo_Class.getHash. I 
 looked up to see what the "getHash" method is doing in druntime [2], 
 the method looks like this:

 override size_t getHash(scope const void* p)  trusted const
 {
     auto o = *cast(Object*)p;
     return o ? o.toHash() : 0;
 }


 
 I guess the compiler uses the AA key type's TypeInfo, which is available 
 for extern(C++) classes too. The TypeInfo_Class.getHash() then uses the 
 dynamic type via virtual call, (wrongly) assuming it's always a D class. 
 For an extern(C++) class, it will either call another virtual function 
 (no inherited virtual functions from Object), what you were seeing, or 
 attempt to call... something. ;)
 

 All this just compiled without any error or warnings. No runtime 
 exceptions or asserts were triggered. I just got a really weird behavior.


 
 This is somewhat special due to the common TypeInfo.getHash() signature 
 for all kinds of types, and so just taking a hairy void* pointer to the 
 byte/real/static array/AA/object/… to be hashed.
 
 Polishing C++ interop with extern(C++) classes (matching ctors/dtors, 
 mixed class hiearchies, ability to easily 
 allocate/construct/destruct/free on the other language side etc.) has 
 started with v2.081 and is still on-going; there are probably more 
 places in druntime silently assuming a D class.


This could be solved, I think, with having "TypeInfo.getHash" a template 
taking the actual type and not void*. That template can then inspect if 
the passed type is a D class or any other type of class and act accordingly.

-- 
/Jacob Carlborg
Aug 22 2018
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 22 August 2018 at 19:25:40 UTC, Jacob Carlborg 
wrote:

 This could be solved, I think, with having "TypeInfo.getHash" a 
 template taking the actual type and not void*. That template 
 can then inspect if the passed type is a D class or any other 
 type of class and act accordingly.


It could be simpler (and slower ;)) by using `m_flags & 
ClassFlags.isCPPclass`.
Aug 22 2018
parent Jacob Carlborg <doob me.com> writes:
On 2018-08-22 23:00, kinke wrote:


 It could be simpler (and slower ;)) by using `m_flags & 
 ClassFlags.isCPPclass`.


There's Objective-C classes as well, which does not seem to have an 
entry in TypeInfo_Class.ClassFlags.

-- 
/Jacob Carlborg
Aug 25 2018
prev sibling parent Paul O'Neil <redballoon36 gmail.com> writes:
Potentially related:
https://issues.dlang.org/show_bug.cgi?id=13875
Aug 20 2018