www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - DLLs: Cleaning up

reply "Chris" <wendlec tcd.ie> writes:
I have a DLL written in D I load into a Python application via 
ctypes like so:

lib = CDLL("mydll")

The DLL loads and can be used no problem. However, once the DLL 
is discarded of by the program, the program either doesn't react 
or crashes. I still haven't worked out how to clean up the DLL 
correctly before it is unloaded / detached (from Python). I guess 
it's the GC and/or some C stuff I've overlooked. I have tried 
both approaches described on this page: http://dlang.org/dll.html.

Maybe someone of yous once had a similar problem and found a 
solution. Any hints or suggestions would be appreciated. Thanks.
Jul 11 2013
next sibling parent reply "dnewbie" <run3 myopera.com> writes:
On Thursday, 11 July 2013 at 12:58:42 UTC, Chris wrote:

 I have a DLL written in D I load into a Python application via 
 ctypes like so:

 lib = CDLL("mydll")

 The DLL loads and can be used no problem. However, once the DLL 
 is discarded of by the program, the program either doesn't 
 react or crashes. I still haven't worked out how to clean up 
 the DLL correctly before it is unloaded / detached (from 
 Python). I guess it's the GC and/or some C stuff I've 
 overlooked. I have tried both approaches described on this 
 page: http://dlang.org/dll.html.

 Maybe someone of yous once had a similar problem and found a 
 solution. Any hints or suggestions would be appreciated. Thanks.


Ye. Please try compiling your DLL with GDC.
Jul 11 2013
parent reply "Chris" <wendlec tcd.ie> writes:
On Thursday, 11 July 2013 at 19:37:31 UTC, dnewbie wrote:

 On Thursday, 11 July 2013 at 12:58:42 UTC, Chris wrote:

 I have a DLL written in D I load into a Python application via 
 ctypes like so:

 lib = CDLL("mydll")

 The DLL loads and can be used no problem. However, once the 
 DLL is discarded of by the program, the program either doesn't 
 react or crashes. I still haven't worked out how to clean up 
 the DLL correctly before it is unloaded / detached (from 
 Python). I guess it's the GC and/or some C stuff I've 
 overlooked. I have tried both approaches described on this 
 page: http://dlang.org/dll.html.

 Maybe someone of yous once had a similar problem and found a 
 solution. Any hints or suggestions would be appreciated. 
 Thanks.


 Ye. Please try compiling your DLL with GDC.


Thanks for the advice, I might try that later. However, I have 
noticed that my DLL does not cause any errors nor does it crash 
the app when switching between my DLL and other third party 
plugins. There is only one other plugin that crashes the app 
after my dll was active (otherwise there's no crash). I've tried 
several solutions now but to no avail. I don't think it's the 
other plugin's fault, as it works usually fine, but maybe there 
is a conflict somewhere in the dark realms of memory.
Jul 12 2013
parent "Chris" <wendlec tcd.ie> writes:
On Friday, 12 July 2013 at 13:15:34 UTC, Chris wrote:

 On Thursday, 11 July 2013 at 19:37:31 UTC, dnewbie wrote:

 On Thursday, 11 July 2013 at 12:58:42 UTC, Chris wrote:

 I have a DLL written in D I load into a Python application 
 via ctypes like so:

 lib = CDLL("mydll")

 The DLL loads and can be used no problem. However, once the 
 DLL is discarded of by the program, the program either 
 doesn't react or crashes. I still haven't worked out how to 
 clean up the DLL correctly before it is unloaded / detached 
 (from Python). I guess it's the GC and/or some C stuff I've 
 overlooked. I have tried both approaches described on this 
 page: http://dlang.org/dll.html.

 Maybe someone of yous once had a similar problem and found a 
 solution. Any hints or suggestions would be appreciated. 
 Thanks.


 Ye. Please try compiling your DLL with GDC.


 Thanks for the advice, I might try that later. However, I have 
 noticed that my DLL does not cause any errors nor does it crash 
 the app when switching between my DLL and other third party 
 plugins. There is only one other plugin that crashes the app 
 after my dll was active (otherwise there's no crash). I've 
 tried several solutions now but to no avail. I don't think it's 
 the other plugin's fault, as it works usually fine, but maybe 
 there is a conflict somewhere in the dark realms of memory.


Sorry, forgot to mention that I wrote a small test app in Python 
and everything works fine there. I only wonder, if dlls in D 
intrude on the memory / main thread of the main app somehow. 
Forgive me my ignorance, I am not a Windows programmer and 
usually work on Linux/Unix systems.
Jul 12 2013
prev sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 07/11/2013 05:58 AM, Chris wrote:

 I have a DLL written in D I load into a Python application via ctypes
 like so:

 lib = CDLL("mydll")

 The DLL loads and can be used no problem. However, once the DLL is
 discarded of by the program, the program either doesn't react or
 crashes. I still haven't worked out how to clean up the DLL correctly
 before it is unloaded / detached (from Python). I guess it's the GC
 and/or some C stuff I've overlooked. I have tried both approaches
 described on this page: http://dlang.org/dll.html.

 Maybe someone of yous once had a similar problem and found a solution.
 Any hints or suggestions would be appreciated. Thanks.


hmm. pyd uses the example under 'DLLs with a C Interface' for its 
windows dll code and it seems pretty stable, but then it doesn't use 
ctypes. It doesn't look like you need to be mucking with rt_init and 
rt_term, so maybe the garbage collector is trying to collect something 
that python still has a reference to?

Also, if you can finagle a dll out of gdc I would love to hear about it. 
I have not used it on windows, though.
Jul 14 2013
parent reply "Chris" <wendlec tcd.ie> writes:
On Sunday, 14 July 2013 at 21:10:53 UTC, Ellery Newcomer wrote:

 On 07/11/2013 05:58 AM, Chris wrote:

 I have a DLL written in D I load into a Python application via 
 ctypes
 like so:

 lib = CDLL("mydll")

 The DLL loads and can be used no problem. However, once the 
 DLL is
 discarded of by the program, the program either doesn't react 
 or
 crashes. I still haven't worked out how to clean up the DLL 
 correctly
 before it is unloaded / detached (from Python). I guess it's 
 the GC
 and/or some C stuff I've overlooked. I have tried both 
 approaches
 described on this page: http://dlang.org/dll.html.

 Maybe someone of yous once had a similar problem and found a 
 solution.
 Any hints or suggestions would be appreciated. Thanks.


 hmm. pyd uses the example under 'DLLs with a C Interface' for 
 its windows dll code and it seems pretty stable, but then it 
 doesn't use ctypes. It doesn't look like you need to be mucking 
 with rt_init and rt_term, so maybe the garbage collector is 
 trying to collect something that python still has a reference 
 to?

 Also, if you can finagle a dll out of gdc I would love to hear 
 about it. I have not used it on windows, though.


Yes, I think so too, that it has something to do with GC. What 
happens in the program is that Python passes a string to the DLL 
but nothing is returned. The DLL somehow interferes with the main 
thread. Other DLLs/threads are fine. I'll check again.

Thanks for the info about Pyd, unfortunately it is out of date 
and doesn't work with newer versions of dmd, but I'll have a look 
at the source code anyway.

"Finagle" is the right word. Phew. But first I could try and 
write a C wrapper (with rt_init etc), I did that once and it 
worked.
Jul 15 2013
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 07/15/2013 07:18 AM, Chris wrote:

 doesn't work with newer versions of dmd


does too. (I'm the maintainer)

https://bitbucket.org/ariovistus/pyd
Jul 15 2013
parent reply "Chris" <wendlec tcd.ie> writes:
On Monday, 15 July 2013 at 15:26:49 UTC, Ellery Newcomer wrote:

 On 07/15/2013 07:18 AM, Chris wrote:

 doesn't work with newer versions of dmd


 does too. (I'm the maintainer)

 https://bitbucket.org/ariovistus/pyd


Thank you very much (I used an old version of pyd I had found at 
http://pyd.dsource.org/), which gave me loads of warnings from 
the latest dmd compiler. But I have installed the latest version 
now and the obligatory "hello world" program works! I'll try to 
build my dll with pyd and will let you know what happened. I hope 
it can cope with the third party libraries my dll uses ... We'll 
see. Thanks again.
Jul 15 2013
parent reply "Chris" <wendlec tcd.ie> writes:
On Monday, 15 July 2013 at 15:59:42 UTC, Chris wrote:

 On Monday, 15 July 2013 at 15:26:49 UTC, Ellery Newcomer wrote:

 On 07/15/2013 07:18 AM, Chris wrote:

 doesn't work with newer versions of dmd


 does too. (I'm the maintainer)

 https://bitbucket.org/ariovistus/pyd


 Thank you very much (I used an old version of pyd I had found 
 at http://pyd.dsource.org/), which gave me loads of warnings 
 from the latest dmd compiler. But I have installed the latest 
 version now and the obligatory "hello world" program works! 
 I'll try to build my dll with pyd and will let you know what 
 happened. I hope it can cope with the third party libraries my 
 dll uses ... We'll see. Thanks again.


Ok, for the record: I've sorted it out now. First of all, the D 
code was INNOCENT! The whole thing was down to how paths and 
search paths are handled in Python, with some nasty surprises as 
regards obtaining (valid) paths on Windows as opposed to Linux / 
Mac.

The D DLL works fine and doesn't interfere with the program's 
memory. No need to use GDC. A normal DMD compilation is enough (I 
used 2.063)

However, I will try to set up a Pyd implementation as well, which 
would avoid using ctypes and make the code more concise. (Any 
time and life saving advice about linking to other libraries / 
DLLs?)

Thanks everyone!
Jul 17 2013
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 07/17/2013 08:13 AM, Chris wrote:

 with some nasty surprises as regards obtaining
 (valid) paths on Windows as opposed to Linux / Mac.


Do tell.


 (Any time and life saving advice about linking to other libraries / DLLs?)

 Thanks everyone!


celerid should be up to the task.
Jul 17 2013