D.gnu - Can GDC make DLLs?
- Ben Davis (22/22) Feb 23 2013 Hi,
- Iain Buclaw (13/26) Feb 23 2013 couldn't find any evidence of other people hitting it. Have I got it rig...
- Ben Davis (24/57) Feb 23 2013 I see. :)
- Iain Buclaw (17/67) Feb 24 2013 GCC-style inline asm for the offending block. Now I've run into another
- Ben Davis (26/56) Feb 24 2013 That may be true, but what GDC is generating is wrong nevertheless.
- Ben Davis (82/84) Feb 24 2013 That's weird. Whatever further problem I thought I was hitting, I'm not
- Iain Buclaw (24/82) Feb 24 2013 else who wants to build a DLL with GDC. Also in case you want to fix the
- Ben Davis (24/29) Feb 24 2013 Hey, I just wanted to get it working as quickly as possible - Googling
- Iain Buclaw (18/33) Feb 24 2013 how to use Intel syntax with GCC's inline asm was faster than trying to
Hi,
Has anyone had any success using GDC to make DLLs to be called from C/C++?
The reason I ask is, for me, the following snippet inside dll.d /
dll_fixTLS() seems to have compiled to a call to abort():
void** peb;
asm
{
mov EAX,FS:[0x30];
mov peb, EAX;
}
and thus dll_process_attach() crashes the process.
It seems like a bug that would affect more people than just me, yet I
couldn't find any evidence of other people hitting it. Have I got it
right what's happening, or is something else at work?
If I'm right, then I'm just wondering if anyone has any ideas on whether
it could be fixed, and how?
Also, I found some discussion about D-style inline asm being problematic
and worthy of removal, but didn't find any explanation as to what those
problems were. I'm curious :)
Thanks in advance if anyone can help. This is my first post in this
group, so I hope it meets the necessary standards :)
Ben :)
Feb 23 2013
On Feb 23, 2013 6:20 PM, "Ben Davis" <entheh cantab.net> wrote:Hi, Has anyone had any success using GDC to make DLLs to be called from C/C++? The reason I ask is, for me, the following snippet inside dll.d /dll_fixTLS() seems to have compiled to a call to abort():void** peb; asm { mov EAX,FS:[0x30]; mov peb, EAX; } and thus dll_process_attach() crashes the process. It seems like a bug that would affect more people than just me, yet Icouldn't find any evidence of other people hitting it. Have I got it right what's happening, or is something else at work?If I'm right, then I'm just wondering if anyone has any ideas on whetherit could be fixed, and how?Also, I found some discussion about D-style inline asm being problematicand worthy of removal, but didn't find any explanation as to what those problems were. I'm curious :)Only because shared libraries requires PIC, and quite a few of the IASM routines clobber the pic register. Regards -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Feb 23 2013
On 23/02/2013 18:38, Iain Buclaw wrote:
On Feb 23, 2013 6:20 PM, "Ben Davis" <entheh cantab.net
<mailto:entheh cantab.net>> wrote:
>
> Hi,
>
> Has anyone had any success using GDC to make DLLs to be called from
C/C++?
>
> The reason I ask is, for me, the following snippet inside dll.d /
dll_fixTLS() seems to have compiled to a call to abort():
>
> void** peb;
> asm
> {
> mov EAX,FS:[0x30];
> mov peb, EAX;
> }
>
> and thus dll_process_attach() crashes the process.
>
> It seems like a bug that would affect more people than just me, yet I
couldn't find any evidence of other people hitting it. Have I got it
right what's happening, or is something else at work?
>
> If I'm right, then I'm just wondering if anyone has any ideas on
whether it could be fixed, and how?
>
> Also, I found some discussion about D-style inline asm being
problematic and worthy of removal, but didn't find any explanation as to
what those problems were. I'm curious :)
>
Only because shared libraries requires PIC, and quite a few of the IASM
routines clobber the pic register.
I see. :)
I've managed to build my DLL with a replacement for dll.d, using
GCC-style inline asm for the offending block. Now I've run into another
problem. This code crashes:
static bool tlsCtorRun;
static this() { tlsCtorRun = true; }
64841F0F 8B 15 D4 E2 95 64 mov edx,dword ptr ds:[6495E2D4h]
64841F15 64 A1 2C 00 00 00 mov eax,dword ptr fs:[0000002Ch]
64841F1B 8B 04 90 mov eax,dword ptr [eax+edx*4]
64841F1E C6 80 1C 50 96 64 01 mov byte ptr _tls_start+4
(6496501Ch)[eax],1 <-- this line crashes
64841F25 5D pop ebp
64841F26 C3 ret
It works when built with DMD. The code is functionally identical, and
gets identical values (give or take some randomless with exact memory
layout), with one exception: the offending line effectively says "byte
ptr [eax+10h]" instead. So the constant displacement in the instruction
is 10h instead of 6496501Ch.
I'm no expert on how DLLs are loaded, but I think I've ruled out any
load-time code offset adjustment - because I found the exact sequence of
code bytes in the DLL file on disk, including that number 6496501Ch and
all the surrounding code mnemonics.
Is this a compiler TLS codegen bug?
Feb 23 2013
On 23 Feb 2013 22:00, "Ben Davis" <entheh cantab.net> wrote:On 23/02/2013 18:38, Iain Buclaw wrote:GCC-style inline asm for the offending block. Now I've run into another problem. This code crashes:On Feb 23, 2013 6:20 PM, "Ben Davis" <entheh cantab.net <mailto:entheh cantab.net>> wrote: > > Hi, > > Has anyone had any success using GDC to make DLLs to be called from C/C++? > > The reason I ask is, for me, the following snippet inside dll.d / dll_fixTLS() seems to have compiled to a call to abort(): > > void** peb; > asm > { > mov EAX,FS:[0x30]; > mov peb, EAX; > } > > and thus dll_process_attach() crashes the process. > > It seems like a bug that would affect more people than just me, yet I couldn't find any evidence of other people hitting it. Have I got it right what's happening, or is something else at work? > > If I'm right, then I'm just wondering if anyone has any ideas on whether it could be fixed, and how? > > Also, I found some discussion about D-style inline asm being problematic and worthy of removal, but didn't find any explanation as to what those problems were. I'm curious :) > Only because shared libraries requires PIC, and quite a few of the IASM routines clobber the pic register.I see. :) I've managed to build my DLL with a replacement for dll.d, usingstatic bool tlsCtorRun; static this() { tlsCtorRun = true; } 64841F0F 8B 15 D4 E2 95 64 mov edx,dword ptr ds:[6495E2D4h] 64841F15 64 A1 2C 00 00 00 mov eax,dword ptr fs:[0000002Ch] 64841F1B 8B 04 90 mov eax,dword ptr [eax+edx*4] 64841F1E C6 80 1C 50 96 64 01 mov byte ptr _tls_start+4(6496501Ch)[eax],1 <-- this line crashes64841F25 5D pop ebp 64841F26 C3 ret It works when built with DMD. The code is functionally identical, andgets identical values (give or take some randomless with exact memory layout), with one exception: the offending line effectively says "byte ptr [eax+10h]" instead. So the constant displacement in the instruction is 10h instead of 6496501Ch.I'm no expert on how DLLs are loaded, but I think I've ruled out anyload-time code offset adjustment - because I found the exact sequence of code bytes in the DLL file on disk, including that number 6496501Ch and all the surrounding code mnemonics.Is this a compiler TLS codegen bug?GDC TLS is differrent to whatever DMD uses, so assembly code that works for DMD may not necessarily be correct for GDC. Regards ---- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Feb 24 2013
On 24/02/2013 09:38, Iain Buclaw wrote:
On 23 Feb 2013 22:00, "Ben Davis" <entheh cantab.net
<mailto:entheh cantab.net>> wrote:
> I've managed to build my DLL with a replacement for dll.d, using
GCC-style inline asm for the offending block. Now I've run into another
problem. This code crashes:
>
> static bool tlsCtorRun;
> static this() { tlsCtorRun = true; }
> 64841F0F 8B 15 D4 E2 95 64 mov edx,dword ptr ds:[6495E2D4h]
> 64841F15 64 A1 2C 00 00 00 mov eax,dword ptr fs:[0000002Ch]
> 64841F1B 8B 04 90 mov eax,dword ptr [eax+edx*4]
> 64841F1E C6 80 1C 50 96 64 01 mov byte ptr _tls_start+4
(6496501Ch)[eax],1 <-- this line crashes
> 64841F25 5D pop ebp
> 64841F26 C3 ret
>
> It works when built with DMD. The code is functionally identical, and
gets identical values (give or take some randomless with exact memory
layout), with one exception: the offending line effectively says "byte
ptr [eax+10h]" instead. So the constant displacement in the instruction
is 10h instead of 6496501Ch.
>
> I'm no expert on how DLLs are loaded, but I think I've ruled out any
load-time code offset adjustment - because I found the exact sequence of
code bytes in the DLL file on disk, including that number 6496501Ch and
all the surrounding code mnemonics.
>
> Is this a compiler TLS codegen bug?
GDC TLS is differrent to whatever DMD uses, so assembly code that works
for DMD may not necessarily be correct for GDC.
That may be true, but what GDC is generating is wrong nevertheless.
Otherwise it wouldn't crash! There should be enough evidence here for
someone to investigate the bug if they want to.
I can also report that the bug only exists in the latest version:
gcc-4.6.1-tdm64-1-gdc-20130108-D2.060.7z - buggy
gcc-4.6.1-tdm64-1-gdc-20121117-D2.060.7z - works
And I suspect it only affects 32-bit code.
With the older build, the codegen looks like this:
static bool tlsCtorRun;
static this() { tlsCtorRun = true; }
64841F0F 8B 15 D4 C2 94 64 mov edx,dword ptr ds:[6494C2D4h]
64841F15 64 A1 2C 00 00 00 mov eax,dword ptr fs:[0000002Ch]
64841F1B 8B 04 90 mov eax,dword ptr [eax+edx*4]
64841F1E C6 80 04 00 00 00 01 mov byte ptr [eax+4],1
64841F25 5D pop ebp
64841F26 C3 ret
And it doesn't crash.
My DLL needs to be 32-bit - but while I had the broken version
installed, I also tried a simple hello world exe (Visual D's default) in
both 32-bit and 64-bit. The 32-bit one crashed somewhere inside the GC
thread, and the 64-bit one worked. When I switched to the non-broken
version, they both worked.
I'm hitting more problems getting my DLL to start up, but I'll post
about those separately later if necessary.
Hope that helps :)
Feb 24 2013
On 24/02/2013 15:13, Ben Davis wrote:I'm hitting more problems getting my DLL to start up, but I'll post about those separately later if necessary.That's weird. Whatever further problem I thought I was hitting, I'm not hitting it any more. I'll put it down to user error on my part. :) Yay! GDC built my DLL and it works! I'll post here a summary of what I had to do, in case it helps anyone else who wants to build a DLL with GDC. Also in case you want to fix the bugs I'm posting a workaround for. I get the feeling you guys are already too busy, but it's here if you want it. :) So, to summarise - in order for 32-bit DLLs to work, you need to: - Not use the 20130108 release because of the 32-bit TLS bug; - Replace dll.d and threadaux.d (both in core/sys/windows, containing D-style asm) with versions that contain GCC-style asm instead (my versions below). Because the user's DllMain is responsible for calling down into these, it's easy to declare one's own copies with the fixes applied, and call those instead. I put mine in a subdirectory (does D call those packages?) called 'gdcdll', and then imported them instead of the Windows ones. (They're Boost-licensed so it looks as if this is OK no matter what end-user licence your own code has. I'll put my changes under Boost too; why not? So you guys can use them.) Here are all the specific changes I made for MinGW, along with the old (DMD) versions. You'll find the 'else' parts already in the source. In dll.d somewhere near the top - I think this change already exists somewhere but the only place I found it was in a .di file somewhere: version (MinGW) { extern __gshared void* _tls_start; extern __gshared void* _tls_end; extern __gshared void* __xl_a; alias _tls_start _tlsstart; alias _tls_end _tlsend; alias __xl_a _tls_callbacks_a; } else { extern __gshared void* _tlsstart; extern __gshared void* _tlsend; extern __gshared void* _tls_callbacks_a; } extern __gshared int _tls_index; //This is unchanged Further down in dll.d: void** peb; version (MinGW) { asm { ".intel_syntax noprefix\n" "mov EAX,FS:[0x30]\n" ".att_syntax noprefix\n" :"=a"(peb); } } else { asm { mov EAX,FS:[0x30]; mov peb, EAX; } } Somewhere in the middle of threadaux.d: version(MinGW) { void** teb; asm { ".intel_syntax noprefix\n" "mov EAX,FS:[0x18]\n" "ret\n" ".att_syntax noprefix\n" :"=a"(teb); } return teb; } else version(Win32) { asm { naked; mov EAX,FS:[0x18]; ret; } } else version(Win64) { snip }
Feb 24 2013
On Feb 24, 2013 3:50 PM, "Ben Davis" <entheh cantab.net> wrote:On 24/02/2013 15:13, Ben Davis wrote:hitting it any more. I'll put it down to user error on my part. :)I'm hitting more problems getting my DLL to start up, but I'll post about those separately later if necessary.That's weird. Whatever further problem I thought I was hitting, I'm notYay! GDC built my DLL and it works! I'll post here a summary of what I had to do, in case it helps anyoneelse who wants to build a DLL with GDC. Also in case you want to fix the bugs I'm posting a workaround for. I get the feeling you guys are already too busy, but it's here if you want it. :)I leave monger to Daniel, and don't really affiliate any modifications to gcc-proper any part if gdc. However that's not to say that I don't think his work is invaluable. :)So, to summarise - in order for 32-bit DLLs to work, you need to: - Not use the 20130108 release because of the 32-bit TLS bug; - Replace dll.d and threadaux.d (both in core/sys/windows, containingD-style asm) with versions that contain GCC-style asm instead (my versions below). Because the user's DllMain is responsible for calling down into these, it's easy to declare one's own copies with the fixes applied, and call those instead. I put mine in a subdirectory (does D call those packages?) called 'gdcdll', and then imported them instead of the Windows ones. (They're Boost-licensed so it looks as if this is OK no matter what end-user licence your own code has. I'll put my changes under Boost too; why not? So you guys can use them.)Here are all the specific changes I made for MinGW, along with the old(DMD) versions. You'll find the 'else' parts already in the source.In dll.d somewhere near the top - I think this change already existssomewhere but the only place I found it was in a .di file somewhere:version (MinGW) { extern __gshared void* _tls_start; extern __gshared void* _tls_end; extern __gshared void* __xl_a; alias _tls_start _tlsstart; alias _tls_end _tlsend; alias __xl_a _tls_callbacks_a; } else { extern __gshared void* _tlsstart; extern __gshared void* _tlsend; extern __gshared void* _tls_callbacks_a; } extern __gshared int _tls_index; //This is unchanged Further down in dll.d: void** peb; version (MinGW) { asm { ".intel_syntax noprefix\n" "mov EAX,FS:[0x30]\n" ".att_syntax noprefix\n" :"=a"(peb); } } else { asm { mov EAX,FS:[0x30]; mov peb, EAX; } } Somewhere in the middle of threadaux.d: version(MinGW) { void** teb; asm { ".intel_syntax noprefix\n" "mov EAX,FS:[0x18]\n" "ret\n" ".att_syntax noprefix\n" :"=a"(teb); }I think it's horrible that you use .intel_syntax. \n shouldn't be required either if you just have it as one string. ;) Regards -- Iain Buclaw *(p < e ? p++ : p) = (c & 0x0f) + '0';
Feb 24 2013
On 25/02/2013 00:53, Iain Buclaw wrote:I leave monger to Daniel, and don't really affiliate any modifications to gcc-proper any part if gdc. However that's not to say that I don't think his work is invaluable. :)Can't say I know what monger is, but OK :)I think it's horrible that you use .intel_syntax. \n shouldn't be required either if you just have it as one string. ;)Hey, I just wanted to get it working as quickly as possible - Googling how to use Intel syntax with GCC's inline asm was faster than trying to learn the absolute %hideous &satan-$born that(%is,$abomination,%syntax),%at&t;. ;) About the string though, do you mean a "multiline string like this"? I guess I've been using Java too long, where those aren't allowed. Good to be reminded that's possible. :) As it happens, I ran into some of the more general problems with GDC which I'm sure people are actively working on - e.g. missing stack traces, failure to generate catchable throwables in response to null pointer access violations and the like - and also, my DLL stopped working when I turned debug info off (I didn't investigate why beyond checking that the export table contained the required entries, which it did) - so after a whole weekend of it, I decided to cut my losses and shell out for the DMC source so I could fix the bug that led me to GDC in the first place. (I also noticed that the GDC build was over 6 MB and the DMD build was less than 1 MB.) I do hope to take advantage of GCC's alleged faster code one day, but my current project is hardly going to tax any CPUs, so I'll go with more stuff working for now. But definitely keep up the good work :)
Feb 24 2013
Iain Buclaw <ibuclaw ubuntu.com> On Feb 25, 2013 3:00 AM, "Ben Davis" <entheh cantab.net> wrote:On 25/02/2013 00:53, Iain Buclaw wrote:Autocomplete fail on tablet. MingGW port.I leave monger to Daniel, and don't really affiliate any modifications to gcc-proper any part if gdc. However that's not to say that I don't think his work is invaluable. :)Can't say I know what monger is, but OK :)how to use Intel syntax with GCC's inline asm was faster than trying to learn the absolute %hideous &satan-$born that(%is,$abomination,%syntax),%at&t;. ;)I think it's horrible that you use .intel_syntax. \n shouldn't be required either if you just have it as one string. ;)Hey, I just wanted to get it working as quickly as possible - GooglingAbout the string though, do you mean a "multiline string like this"? I guess I've been using Java too long, where those aren't allowed.Good to be reminded that's possible. :)Yes.As it happens, I ran into some of the more general problems with GDCwhich I'm sure people are actively working on - e.g. missing stack traces, failure to generate catchable throwables in response to null pointer access violations and the like - and also, my DLL stopped working when I turned debug info off (I didn't investigate why beyond checking that the export table contained the required entries, which it did) - so after a whole weekend of it, I decided to cut my losses and shell out for the DMC source so I could fix the bug that led me to GDC in the first place. (I also noticed that the GDC build was over 6 MB and the DMD build was less than 1 MB.)I do hope to take advantage of GCC's alleged faster code one day, but mycurrent project is hardly going to tax any CPUs, so I'll go with more stuff working for now.But definitely keep up the good work :)
Feb 24 2013