• Chris (13/13) Nov 12 2014 The following causes the DLL to crash on Windows:
• ketmar via Digitalmars-d-learn (8/24) Nov 12 2014 seems that you are using two different allocators here. one is that
• Chris (7/35) Nov 12 2014 That makes a lot of sense. Hm. How can I work around this problem
• ketmar via Digitalmars-d-learn (6/44) Nov 12 2014 if you can extend C DLL, just add wrapper for `free()` there. so you
• Chris (6/56) Nov 12 2014 I initially had an implementation that did exactly that (I
• Chris (5/17) Nov 12 2014 I've changed the code so that the memory is freed in C. Although
• ketmar via Digitalmars-d-learn (5/26) Nov 12 2014 this also can happen due to allocators conflict somehow. or due to
• Chris (12/42) Nov 13 2014 Thanks a million! Just checked it this morning. It was the
• ketmar via Digitalmars-d-learn (11/13) Nov 13 2014 seems that libc allocator is not marking free pages as "unreadable",
• Chris (19/40) Nov 14 2014 To whom it may concern: For Windows I had to write safe code. My
• ketmar via Digitalmars-d-learn (3/19) Nov 12 2014 p.s. i mean "different C runtimes".

"Chris" <wendlec tcd.ie> writes:

The following causes the DLL to crash on Windows:

Input: immutable(short)* data (immutable because in separate 
thread).
// Later
core.stdc.stdlib.free(cast(short *)data);

(short* data is provided by the C library, where the memory is 
allocated)

On Linux it works fine and never crashes, in the Windows DLL it 
randomly causes an access violation in memory (both read and 
write). Note that it doesn't crash immediately, it goes on for a 
while, but sooner or later it crashes. If I comment out this 
line, everything works fine. However, if I don't free the memory, 
I'll have a memory leak. Any hints/advice/guesses?

ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:

On Wed, 12 Nov 2014 12:40:30 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:

 The following causes the DLL to crash on Windows:
=20
 Input: immutable(short)* data (immutable because in separate=20
 thread).
 // Later
 core.stdc.stdlib.free(cast(short *)data);
=20
 (short* data is provided by the C library, where the memory is=20
 allocated)
=20
 On Linux it works fine and never crashes, in the Windows DLL it=20
 randomly causes an access violation in memory (both read and=20
 write). Note that it doesn't crash immediately, it goes on for a=20
 while, but sooner or later it crashes. If I comment out this=20
 line, everything works fine. However, if I don't free the memory,=20
 I'll have a memory leak. Any hints/advice/guesses?

seems that you are using two different allocators here. one is that
comes with DLL and other that comes with D. i bet they either using
different runtimes, or C runtime is doesn't know about another C
runtime in DLL.

on GNU/Linux there is only one runtime (most of the time), so there
is no problem with different allocators.

"Chris" <wendlec tcd.ie> writes:

On Wednesday, 12 November 2014 at 12:58:19 UTC, ketmar via 
Digitalmars-d-learn wrote:
 On Wed, 12 Nov 2014 12:40:30 +0000
 Chris via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:

 The following causes the DLL to crash on Windows:
 
 Input: immutable(short)* data (immutable because in separate 
 thread).
 // Later
 core.stdc.stdlib.free(cast(short *)data);
 
 (short* data is provided by the C library, where the memory is 
 allocated)
 
 On Linux it works fine and never crashes, in the Windows DLL 
 it randomly causes an access violation in memory (both read 
 and write). Note that it doesn't crash immediately, it goes on 
 for a while, but sooner or later it crashes. If I comment out 
 this line, everything works fine. However, if I don't free the 
 memory, I'll have a memory leak. Any hints/advice/guesses?

 seems that you are using two different allocators here. one is 
 that
 comes with DLL and other that comes with D. i bet they either 
 using
 different runtimes, or C runtime is doesn't know about another C
 runtime in DLL.

 on GNU/Linux there is only one runtime (most of the time), so 
 there
 is no problem with different allocators.

That makes a lot of sense. Hm. How can I work around this problem 
then? What's involved are a C-dll and a D-dll that uses the C-dll 
and is loaded into a Python program. To complicate things further 
short* data is passed to C by D. short* data is then allocated 
and populated in C.

ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:

On Wed, 12 Nov 2014 14:11:35 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:

 On Wednesday, 12 November 2014 at 12:58:19 UTC, ketmar via=20
 Digitalmars-d-learn wrote:
 On Wed, 12 Nov 2014 12:40:30 +0000
 Chris via Digitalmars-d-learn=20
 <digitalmars-d-learn puremagic.com> wrote:

 The following causes the DLL to crash on Windows:
=20
 Input: immutable(short)* data (immutable because in separate=20
 thread).
 // Later
 core.stdc.stdlib.free(cast(short *)data);
=20
 (short* data is provided by the C library, where the memory is=20
 allocated)
=20
 On Linux it works fine and never crashes, in the Windows DLL=20
 it randomly causes an access violation in memory (both read=20
 and write). Note that it doesn't crash immediately, it goes on=20
 for a while, but sooner or later it crashes. If I comment out=20
 this line, everything works fine. However, if I don't free the=20
 memory, I'll have a memory leak. Any hints/advice/guesses?

 seems that you are using two different allocators here. one is=20
 that
 comes with DLL and other that comes with D. i bet they either=20
 using
 different runtimes, or C runtime is doesn't know about another C
 runtime in DLL.

 on GNU/Linux there is only one runtime (most of the time), so=20
 there
 is no problem with different allocators.

=20
 That makes a lot of sense. Hm. How can I work around this problem=20
 then? What's involved are a C-dll and a D-dll that uses the C-dll=20
 and is loaded into a Python program. To complicate things further=20
 short* data is passed to C by D. short* data is then allocated=20
 and populated in C.

if you can extend C DLL, just add wrapper for `free()` there. so you
will not call `free()` from D, but call C DLL function which will free
the memory. it's a good practice anyway, 'cause it's recommended to
free memory in the same library where you allocated it.

"Chris" <wendlec tcd.ie> writes:

On Wednesday, 12 November 2014 at 14:26:15 UTC, ketmar via 
Digitalmars-d-learn wrote:
 On Wed, 12 Nov 2014 14:11:35 +0000
 Chris via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:

 On Wednesday, 12 November 2014 at 12:58:19 UTC, ketmar via 
 Digitalmars-d-learn wrote:
 On Wed, 12 Nov 2014 12:40:30 +0000
 Chris via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:

 The following causes the DLL to crash on Windows:
 
 Input: immutable(short)* data (immutable because in 
 separate thread).
 // Later
 core.stdc.stdlib.free(cast(short *)data);
 
 (short* data is provided by the C library, where the memory 
 is allocated)
 
 On Linux it works fine and never crashes, in the Windows 
 DLL it randomly causes an access violation in memory (both 
 read and write). Note that it doesn't crash immediately, it 
 goes on for a while, but sooner or later it crashes. If I 
 comment out this line, everything works fine. However, if I 
 don't free the memory, I'll have a memory leak. Any 
 hints/advice/guesses?

 seems that you are using two different allocators here. one 
 is that
 comes with DLL and other that comes with D. i bet they 
 either using
 different runtimes, or C runtime is doesn't know about 
 another C
 runtime in DLL.

 on GNU/Linux there is only one runtime (most of the time), 
 so there
 is no problem with different allocators.

 
 That makes a lot of sense. Hm. How can I work around this 
 problem then? What's involved are a C-dll and a D-dll that 
 uses the C-dll and is loaded into a Python program. To 
 complicate things further short* data is passed to C by D. 
 short* data is then allocated and populated in C.

 if you can extend C DLL, just add wrapper for `free()` there. 
 so you
 will not call `free()` from D, but call C DLL function which 
 will free
 the memory. it's a good practice anyway, 'cause it's 
 recommended to
 free memory in the same library where you allocated it.

I initially had an implementation that did exactly that (I 
usually do that), but for some reason it didn't work properly in 
this particular case and caused all sorts of undefined behavior. 
But I'll have a look at it again.

"Chris" <wendlec tcd.ie> writes:

On Wednesday, 12 November 2014 at 14:42:34 UTC, Chris wrote:
 On Wednesday, 12 November 2014 at 14:26:15 UTC, ketmar via
 if you can extend C DLL, just add wrapper for `free()` there. 
 so you
 will not call `free()` from D, but call C DLL function which 
 will free
 the memory. it's a good practice anyway, 'cause it's 
 recommended to
 free memory in the same library where you allocated it.

 I initially had an implementation that did exactly that (I 
 usually do that), but for some reason it didn't work properly 
 in this particular case and caused all sorts of undefined 
 behavior. But I'll have a look at it again.

I've changed the code so that the memory is freed in C. Although 
it works "better" it crashes too every now and then

(WindowsError : exception : access violation writing 0x0310A1B4)

Will look into it.

ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:

On Wed, 12 Nov 2014 16:03:08 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:

 On Wednesday, 12 November 2014 at 14:42:34 UTC, Chris wrote:
 On Wednesday, 12 November 2014 at 14:26:15 UTC, ketmar via
 if you can extend C DLL, just add wrapper for `free()` there.=20
 so you
 will not call `free()` from D, but call C DLL function which=20
 will free
 the memory. it's a good practice anyway, 'cause it's=20
 recommended to
 free memory in the same library where you allocated it.

 I initially had an implementation that did exactly that (I=20
 usually do that), but for some reason it didn't work properly=20
 in this particular case and caused all sorts of undefined=20
 behavior. But I'll have a look at it again.

=20
 I've changed the code so that the memory is freed in C. Although=20
 it works "better" it crashes too every now and then
=20
 (WindowsError : exception : access violation writing 0x0310A1B4)
=20
 Will look into it.

this also can happen due to allocators conflict somehow. or due to
other code which stores the pointer somewhere and then accesses the
memory. i think that it will be hard to trace without debugger.

"Chris" <wendlec tcd.ie> writes:

On Wednesday, 12 November 2014 at 16:10:34 UTC, ketmar via 
Digitalmars-d-learn wrote:
 On Wed, 12 Nov 2014 16:03:08 +0000
 Chris via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:

 On Wednesday, 12 November 2014 at 14:42:34 UTC, Chris wrote:
 On Wednesday, 12 November 2014 at 14:26:15 UTC, ketmar via
 if you can extend C DLL, just add wrapper for `free()` 
 there. so you
 will not call `free()` from D, but call C DLL function 
 which will free
 the memory. it's a good practice anyway, 'cause it's 
 recommended to
 free memory in the same library where you allocated it.

 I initially had an implementation that did exactly that (I 
 usually do that), but for some reason it didn't work 
 properly in this particular case and caused all sorts of 
 undefined behavior. But I'll have a look at it again.

 
 I've changed the code so that the memory is freed in C. 
 Although it works "better" it crashes too every now and then
 
 (WindowsError : exception : access violation writing 
 0x0310A1B4)
 
 Will look into it.

 this also can happen due to allocators conflict somehow. or due 
 to
 other code which stores the pointer somewhere and then accesses 
 the
 memory. i think that it will be hard to trace without debugger.

Thanks a million! Just checked it this morning. It was the 
latter. I kept a reference to short* data in a class variable in 
D and didn't clear that reference when freeing the memory in the 
C library.

Interesting though that it never crashes on Linux, only on 
Windows did this cause problems.

It is also interesting that core.stdc.stdlib.free() and free() in 
the C library produced a slightly different crash behavior. But 
that might be down to the fact that the two happen in different 
places in the program.

ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:

On Thu, 13 Nov 2014 10:08:47 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:

 Interesting though that it never crashes on Linux, only on=20
 Windows did this cause problems.

seems that libc allocator is not marking free pages as "unreadable",
and windows libc allocator does something like this. i got alot of such
"use after free" things in my C code, but valgrind is brilliant to
track that down. i miss valgrind on windows. ;-)

now i prefer to write code in GNU/Linux if possible and use valgrind to
track down memory issues, and only after it works on linux and valgrind
is happy, i'm starting to port it to windows. this, of course, hard to
do with winapi-tied code, but wine/winelib can help there too (to some
extent).

"Chris" <wendlec tcd.ie> writes:

On Thursday, 13 November 2014 at 10:17:35 UTC, ketmar via 
Digitalmars-d-learn wrote:
 On Thu, 13 Nov 2014 10:08:47 +0000
 Chris via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:

 Interesting though that it never crashes on Linux, only on 
 Windows did this cause problems.

 seems that libc allocator is not marking free pages as 
 "unreadable",
 and windows libc allocator does something like this. i got alot 
 of such
 "use after free" things in my C code, but valgrind is brilliant 
 to
 track that down. i miss valgrind on windows. ;-)

 now i prefer to write code in GNU/Linux if possible and use 
 valgrind to
 track down memory issues, and only after it works on linux and 
 valgrind
 is happy, i'm starting to port it to windows. this, of course, 
 hard to
 do with winapi-tied code, but wine/winelib can help there too 
 (to some
 extent).

To whom it may concern: For Windows I had to write safe code. My 
"skating on razor blades" code[1] didn't work there (dodgy 
casts). I have to make an immutable copy of the short[] array I 
get from C (an extra step I wanted to avoid). In a way it's much 
cleaner. I copy the array and free the memory in C immediately 
afterwards. It's not the Windows crashes with the unsafe code, 
it's just that freeing the memory and reading the array are out 
of sync (across threads), giving rise to undefined behavior (it 
reads random values from memory).

Having said this (repentantly), it took me a while to check the C 
code and the memory allocations/deallocations that go on there, 
to make sure everything is fine in C (and D). I have to say that 
it's much nicer to work with GC than with manual memory 
management, because it is a tedious task that keeps you from 
writing useful code.

[1] 
http://forum.dlang.org/thread/yhticlpnygtjkdcfjihk forum.dlang.org

ketmar via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> writes:

On Wed, 12 Nov 2014 12:40:30 +0000
Chris via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:

 The following causes the DLL to crash on Windows:
=20
 Input: immutable(short)* data (immutable because in separate=20
 thread).
 // Later
 core.stdc.stdlib.free(cast(short *)data);
=20
 (short* data is provided by the C library, where the memory is=20
 allocated)
=20
 On Linux it works fine and never crashes, in the Windows DLL it=20
 randomly causes an access violation in memory (both read and=20
 write). Note that it doesn't crash immediately, it goes on for a=20
 while, but sooner or later it crashes. If I comment out this=20
 line, everything works fine. However, if I don't free the memory,=20
 I'll have a memory leak. Any hints/advice/guesses?

p.s. i mean "different C runtimes".