www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DMD 2.055 Crashing on Windows 7 x64

reply Mehrdad <wfunction hotmail.com> writes:
DMD 2.055 seems to be crashing on my computer (Windows 7 x64).

When I press Debug, I see it's doing so in CloseHandle, probably because 
I've turned on "Enable close exception" with Visual Studio's "Global 
Flags" tool. (It takes a reboot to take effect.)

So this means there's an incorrect handle being used somewhere... but 
interestingly, by the time it crashes, the output file is already 
produced. I don't have the time to make a debug build right now, but I 
thought I'd mention this so that it could be looked into at some point.
Sep 26 2011
next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 7:55 AM, Mehrdad wrote:

 DMD 2.055 seems to be crashing on my computer (Windows 7 x64).

 When I press Debug, I see it's doing so in CloseHandle, probably 
 because I've turned on "Enable close exception" with Visual Studio's 
 "Global Flags" tool. (It takes a reboot to take effect.)

 So this means there's an incorrect handle being used somewhere... but 
 interestingly, by the time it crashes, the output file is already 
 produced. I don't have the time to make a debug build right now, but I 
 thought I'd mention this so that it could be looked into at some point.


Actually, I just found out that the program itself crashes the same way, 
too.

Is there a bug in snn.lib or something?
Sep 26 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/26/2011 8:06 AM, Mehrdad wrote:

 Actually, I just found out that the program itself crashes the same way, too.


What program?
Sep 26 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 15:05:44 -0400, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 9/26/2011 8:06 AM, Mehrdad wrote:

 Actually, I just found out that the program itself crashes the same  
 way, too.


 What program?


I think he means both dmd and the program it generates crash in the same  
way.

-Steve
Sep 26 2011
parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 12:19 PM, Steven Schveighoffer wrote:

 On Mon, 26 Sep 2011 15:05:44 -0400, Walter Bright 
 <newshound2 digitalmars.com> wrote:


 On 9/26/2011 8:06 AM, Mehrdad wrote:

 Actually, I just found out that the program itself crashes the same 
 way, too.


 What program?


 I think he means both dmd and the program it generates crash in the 
 same way.

 -Steve


Right; what Steve said. The program is empty.

It seems to be failing in the _close() method, but ONLY when I run it 
from my text editor, SciTE. It runs fine from the command prompt.

I don't believe it's a problem with the text editor (since it's just a 
pipe, and since it works fine with other languages) but I'll keep 
looking. Does anyone happen to know if there's a stray _close() method 
in snn.lib that closes a handle that might be already closed?
Sep 26 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 16:38:10 -0400, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 12:19 PM, Steven Schveighoffer wrote:

 On Mon, 26 Sep 2011 15:05:44 -0400, Walter Bright  
 <newshound2 digitalmars.com> wrote:


 On 9/26/2011 8:06 AM, Mehrdad wrote:

 Actually, I just found out that the program itself crashes the same  
 way, too.


 What program?


 I think he means both dmd and the program it generates crash in the  
 same way.

 -Steve


 Right; what Steve said. The program is empty.

 It seems to be failing in the _close() method, but ONLY when I run it  
 from my text editor, SciTE. It runs fine from the command prompt.

 I don't believe it's a problem with the text editor (since it's just a  
 pipe, and since it works fine with other languages) but I'll keep  
 looking. Does anyone happen to know if there's a stray _close() method  
 in snn.lib that closes a handle that might be already closed?


AHA!

Yes, there is a bug in snn.lib regarding pipes.  And I fixed it, waiting  
for Walter to incorporate it :)  I needed it for the new std.process.

What it comes down to is, DMC's FILE * implementation does not expect some  
of the quirks of pipe HANDLEs.  For instance, if you open a FILE * around  
a pipe handle, it still tries to do a seek on that handle, and crashes.   
Also, when the write end of a pipe is closed, reading from the read end  
results in EPIPE from ReadFile, but this is translated to EBADF by the  
runtime.  Therefore, FILE * sets an error instead of EOF.

Is the email address you have for this message correct?  If so, I can send  
you a new version of snn.lib to try linking your code against (if you are  
willing to go through these steps), to see if it fixes your problem.

Using the command line dmd to build should be sufficient (I think).

-Steve
Sep 26 2011
next sibling parent Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:

 AHA!


Holy cow what happened XD


 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it, 
 waiting for Walter to incorporate it :)  I needed it for the new 
 std.process.


yay!


 What it comes down to is, DMC's FILE * implementation does not expect 
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE 
 * around a pipe handle, it still tries to do a seek on that handle, 
 and crashes.  Also, when the write end of a pipe is closed, reading 
 from the read end results in EPIPE from ReadFile, but this is 
 translated to EBADF by the runtime.  Therefore, FILE * sets an error 
 instead of EOF.

 Is the email address you have for this message correct?  If so, I can 
 send you a new version of snn.lib to try linking your code against (if 
 you are willing to go through these steps), to see if it fixes your 
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve


Yup, feel free to send it to this one. It's not my main one but I do 
check it so I'll get it. :)
Thanks!
Sep 26 2011
prev sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:

 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it, 
 waiting for Walter to incorporate it :)  I needed it for the new 
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect 
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE 
 * around a pipe handle, it still tries to do a seek on that handle, 
 and crashes.  Also, when the write end of a pipe is closed, reading 
 from the read end results in EPIPE from ReadFile, but this is 
 translated to EBADF by the runtime.  Therefore, FILE * sets an error 
 instead of EOF.

 Is the email address you have for this message correct?  If so, I can 
 send you a new version of snn.lib to try linking your code against (if 
 you are willing to go through these steps), to see if it fixes your 
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve


Thanks for the email.

It seems like it still crashes from SciTE, though. :(
When I try debugging, I see it's doing so inside _close() which is 
/immediately/ above kernel32.dll! BaseThreadInitThunk 12()  -- in other 
words, it seems to be called directly from the entrypoint of the 
program, assuming it hasn't undergone optimizations. The error is: 
0xC0000008: An invalid handle was specified.

Is there any other place in which this can happen?
Sep 26 2011
next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:30 PM, Mehrdad wrote:

 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:

 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it, 
 waiting for Walter to incorporate it :)  I needed it for the new 
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect 
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE 
 * around a pipe handle, it still tries to do a seek on that handle, 
 and crashes.  Also, when the write end of a pipe is closed, reading 
 from the read end results in EPIPE from ReadFile, but this is 
 translated to EBADF by the runtime.  Therefore, FILE * sets an error 
 instead of EOF.

 Is the email address you have for this message correct?  If so, I can 
 send you a new version of snn.lib to try linking your code against 
 (if you are willing to go through these steps), to see if it fixes 
 your problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve


 Thanks for the email.

 It seems like it still crashes from SciTE, though. :(
 When I try debugging, I see it's doing so inside _close() which is 
 /immediately/ above kernel32.dll! BaseThreadInitThunk 12()  -- in 
 other words, it seems to be called directly from the entrypoint of the 
 program, assuming it hasn't undergone optimizations. The error is: 
 0xC0000008: An invalid handle was specified.

 Is there any other place in which this can happen?


Also, now that we're on the topic... this might or might not sound 
silly, but why not just use msvcrt.dll (if not the whole library, at 
least the I/O functions)? It's not like it introduces a new dependency 
(it's already in every version of Windows) and it's also proven to work. 
Not to mention that it reduces code size as well...
Sep 26 2011
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 17:35:20 -0400, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 2:30 PM, Mehrdad wrote:

 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:

 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it,  
 waiting for Walter to incorporate it :)  I needed it for the new  
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect  
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE  
 * around a pipe handle, it still tries to do a seek on that handle,  
 and crashes.  Also, when the write end of a pipe is closed, reading  
 from the read end results in EPIPE from ReadFile, but this is  
 translated to EBADF by the runtime.  Therefore, FILE * sets an error  
 instead of EOF.

 Is the email address you have for this message correct?  If so, I can  
 send you a new version of snn.lib to try linking your code against (if  
 you are willing to go through these steps), to see if it fixes your  
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve


 Thanks for the email.

 It seems like it still crashes from SciTE, though. :(
 When I try debugging, I see it's doing so inside _close() which is  
 /immediately/ above kernel32.dll! BaseThreadInitThunk 12()  -- in other  
 words, it seems to be called directly from the entrypoint of the  
 program, assuming it hasn't undergone optimizations. The error is:  
 0xC0000008: An invalid handle was specified.

 Is there any other place in which this can happen?


 Also, now that we're on the topic... this might or might not sound  
 silly, but why not just use msvcrt.dll (if not the whole library, at  
 least the I/O functions)? It's not like it introduces a new dependency  
 (it's already in every version of Windows) and it's also proven to work.  
 Not to mention that it reduces code size as well...


I think the reason Walter always gives is that dmd would then have to  
output COFF objects, whereas it currently outputs OMF.  OPTLINK also does  
not accept COFF objects.

I think a lot of people would very much welcome this change, but it's  
somewhat of an underwhelming change.  Instead of using DMC's runtime for C  
functions, you are using VC++'s runtime.  But who cares?  I want D's  
runtime to be better :)

I think the best path is weaning ourselves off of C dependencies.   
Starting with FILE *, which has a lot of limitations.  It's part of the  
reason I'm working on reworking std.stdio.

For example, with my version of std.stdio, you would never have this  
problem, because it deals with the HANDLE directly instead of through  
DMC's file descriptor layer.

-Steve
Sep 26 2011
next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:53 PM, Steven Schveighoffer wrote:

 I think the reason Walter always gives is that dmd would then have to 
 output COFF objects, whereas it currently outputs OMF.  OPTLINK also 
 does not accept COFF objects.


Wait, why does it have to output COFF? Can't you just use an OMF version 
of msvcrt.lib, like we're doing for all the other Windows libraries?
I'm confused as to how this is an issue.


 I think a lot of people would very much welcome this change, but it's 
 somewhat of an underwhelming change.  Instead of using DMC's runtime 
 for C functions, you are using VC++'s runtime.  But who cares?  I want 
 D's runtime to be better :)


Well, sort of. It's not so much Visual C++'s runtime, but it's 
Microsoft's C runtime that's packaged with Windows (which Microsoft 
ironically doesn't want you to use, but which compilers like GCC use 
anyway... and it turns out better than anything bundled with Visual 
Studio). You don't actually need Visual Studio to use it (in fact, it's 
not even included) -- the Windows Driver Kit has the bundled stuff required.

In all honesty I don't see a single thing about it that I would call 
"better". Do you have any examples of things that snn.lib does 'better' 
than msvcrt.lib?

The only incompatibility (which isn't "better" or "worse"... it's just 
an incompatibility) I can see is exception handling, but that's 
obviously an exception that can be left in snn.lib. Almost all the rest 
of the runtime, though, can be easily removed and redirected to 
msvcrt.dll... and I don't think there'd be any problems. (Are there?)


 I think the best path is weaning ourselves off of C dependencies.  
 Starting with FILE *, which has a lot of limitations.  It's part of 
 the reason I'm working on reworking std.stdio.
 For example, with my version of std.stdio, you would never have this 
 problem, because it deals with the HANDLE directly instead of through 
 DMC's file descriptor layer.


100% agree, but we weren't talking about D in the first place. :) This 
is all regarding what's /already/ taking place in snn.lib /before/ D 
runs, which is obviously unrelated to what's going on in std.stdio.

I'm really curious to know what specific problems are keeping us from 
doing the switch (i.e. what snn.lib does better than msvcrt.dll). :)
Sep 26 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 18:10:05 -0400, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 2:53 PM, Steven Schveighoffer wrote:

 I think the reason Walter always gives is that dmd would then have to  
 output COFF objects, whereas it currently outputs OMF.  OPTLINK also  
 does not accept COFF objects.


 Wait, why does it have to output COFF? Can't you just use an OMF version  
 of msvcrt.lib, like we're doing for all the other Windows libraries?
 I'm confused as to how this is an issue.


I don't know, maybe there's another reason.


 I think a lot of people would very much welcome this change, but it's  
 somewhat of an underwhelming change.  Instead of using DMC's runtime  
 for C functions, you are using VC++'s runtime.  But who cares?  I want  
 D's runtime to be better :)


 Well, sort of. It's not so much Visual C++'s runtime, but it's  
 Microsoft's C runtime that's packaged with Windows (which Microsoft  
 ironically doesn't want you to use, but which compilers like GCC use  
 anyway... and it turns out better than anything bundled with Visual  
 Studio). You don't actually need Visual Studio to use it (in fact, it's  
 not even included) -- the Windows Driver Kit has the bundled stuff  
 required.


It's the same thing.  MSVCRT.dll is included with every windows.  Just the  
version may be different from the runtime bundled with Visual Studio.   
That's why you can install the visual studio redistributable package.

The lib file (which contains the linking information) is not included  
(with Windows).  Neither is the statically linked version.

But make no mistake, it's the Visual C++ runtime (MicroSoftVisualCRunTime).


 In all honesty I don't see a single thing about it that I would call  
 "better". Do you have any examples of things that snn.lib does 'better'  
 than msvcrt.lib?


I think it may conform to the standard better.  But I don't think any C  
runtime is better than what we could make in D :)


 The only incompatibility (which isn't "better" or "worse"... it's just  
 an incompatibility) I can see is exception handling, but that's  
 obviously an exception that can be left in snn.lib. Almost all the rest  
 of the runtime, though, can be easily removed and redirected to  
 msvcrt.dll... and I don't think there'd be any problems. (Are there?)


I don't know.  For certain, there are some pieces of phobos written  
specifically knowing the internals of Digital Mars' runtime (like  
std.stdio).


 I think the best path is weaning ourselves off of C dependencies.   
 Starting with FILE *, which has a lot of limitations.  It's part of the  
 reason I'm working on reworking std.stdio.
 For example, with my version of std.stdio, you would never have this  
 problem, because it deals with the HANDLE directly instead of through  
 DMC's file descriptor layer.


 100% agree, but we weren't talking about D in the first place. :) This  
 is all regarding what's /already/ taking place in snn.lib /before/ D  
 runs, which is obviously unrelated to what's going on in std.stdio.

 I'm really curious to know what specific problems are keeping us from  
 doing the switch (i.e. what snn.lib does better than msvcrt.dll). :)


I'm not really qualified to answer that.  It seems like the right move to  
me, so there must be good reasons compiler-wise.

-Steve
Sep 26 2011
parent Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 3:38 PM, Steven Schveighoffer wrote:

 On Mon, 26 Sep 2011 18:10:05 -0400, Mehrdad <wfunction hotmail.com> 
 wrote:


 On 9/26/2011 2:53 PM, Steven Schveighoffer wrote:

 I think the reason Walter always gives is that dmd would then have 
 to output COFF objects, whereas it currently outputs OMF.  OPTLINK 
 also does not accept COFF objects.


 Wait, why does it have to output COFF? Can't you just use an OMF 
 version of msvcrt.lib, like we're doing for all the other Windows 
 libraries?
 I'm confused as to how this is an issue.


 I don't know, maybe there's another reason.


 I think a lot of people would very much welcome this change, but 
 it's somewhat of an underwhelming change.  Instead of using DMC's 
 runtime for C functions, you are using VC++'s runtime.  But who 
 cares?  I want D's runtime to be better :)


 Well, sort of. It's not so much Visual C++'s runtime, but it's 
 Microsoft's C runtime that's packaged with Windows (which Microsoft 
 ironically doesn't want you to use, but which compilers like GCC use 
 anyway... and it turns out better than anything bundled with Visual 
 Studio). You don't actually need Visual Studio to use it (in fact, 
 it's not even included) -- the Windows Driver Kit has the bundled 
 stuff required.


 It's the same thing.  MSVCRT.dll is included with every windows.  Just 
 the version may be different from the runtime bundled with Visual 
 Studio.  That's why you can install the visual studio redistributable 
 package.

 The lib file (which contains the linking information) is not included 
 (with Windows).  Neither is the statically linked version.

 But make no mistake, it's the Visual C++ runtime 
 (MicroSoftVisualCRunTime).


 In all honesty I don't see a single thing about it that I would call 
 "better". Do you have any examples of things that snn.lib does 
 'better' than msvcrt.lib?


 I think it may conform to the standard better.  But I don't think any 
 C runtime is better than what we could make in D :)


 The only incompatibility (which isn't "better" or "worse"... it's 
 just an incompatibility) I can see is exception handling, but that's 
 obviously an exception that can be left in snn.lib. Almost all the 
 rest of the runtime, though, can be easily removed and redirected to 
 msvcrt.dll... and I don't think there'd be any problems. (Are there?)


 I don't know.  For certain, there are some pieces of phobos written 
 specifically knowing the internals of Digital Mars' runtime (like 
 std.stdio).


 I think the best path is weaning ourselves off of C dependencies.  
 Starting with FILE *, which has a lot of limitations.  It's part of 
 the reason I'm working on reworking std.stdio.
 For example, with my version of std.stdio, you would never have this 
 problem, because it deals with the HANDLE directly instead of 
 through DMC's file descriptor layer.


 100% agree, but we weren't talking about D in the first place. :) 
 This is all regarding what's /already/ taking place in snn.lib 
 /before/ D runs, which is obviously unrelated to what's going on in 
 std.stdio.

 I'm really curious to know what specific problems are keeping us from 
 doing the switch (i.e. what snn.lib does better than msvcrt.dll). :)


 I'm not really qualified to answer that.  It seems like the right move 
 to me, so there must be good reasons compiler-wise.

 -Steve


I hope this isn't too huge of a request (Walter?), but would it be 
possible to PLEASE release an object or library file which would be 
helpful for implementing the use of msvcrt.lib?

Right now, most crucial thing I need is an object file which defines 
this symbol correctly (since it seems to be providing an anchor for the 
rest runtime and the GC):

phobos.lib(memory) Error 42: Symbol Undefined __xi_a

Having something that also implements the following would also be 
awesome, but I can't tell if it's as crucial as the above until I have 
the above and try it out first:

Test.obj(Test) Error 42: Symbol Undefined __except_list
phobos.lib(deh) Error 42: Symbol Undefined __tls_array
phobos.lib(deh) Error 42: Symbol Undefined __tls_index
phobos.lib(thread) Error 42: Symbol Undefined __tlsend
phobos.lib(thread) Error 42: Symbol Undefined __tlsstart
phobos.lib(memory) Error 42: Symbol Undefined __end

If I could _only_ have these, then I could very well try to see if I can 
integrate the C runtime with msvcrt.dll, which I think many people would 
appreciate.

I'm pretty sure that the definition of __xi_a itself must be incredibly 
small (maybe 2 lines of code total?) so I hope it's not too much trouble 
to release something about it... I have no idea how to define it myself. 
If a working definition was released then it'd be pretty awesome.

Thank you!
Sep 26 2011
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, September 26, 2011 14:53 Steven Schveighoffer wrote:

 On Mon, 26 Sep 2011 17:35:20 -0400, Mehrdad <wfunction hotmail.com> wrote:

 On 9/26/2011 2:30 PM, Mehrdad wrote:

 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:

 AHA!
 
 Yes, there is a bug in snn.lib regarding pipes. And I fixed it,
 waiting for Walter to incorporate it :) I needed it for the new
 std.process.
 
 What it comes down to is, DMC's FILE * implementation does not expect
 some of the quirks of pipe HANDLEs. For instance, if you open a FILE
 * around a pipe handle, it still tries to do a seek on that handle,
 and crashes. Also, when the write end of a pipe is closed, reading
 from the read end results in EPIPE from ReadFile, but this is
 translated to EBADF by the runtime. Therefore, FILE * sets an error
 instead of EOF.
 
 Is the email address you have for this message correct? If so, I can
 send you a new version of snn.lib to try linking your code against (if
 you are willing to go through these steps), to see if it fixes your
 problem.
 
 Using the command line dmd to build should be sufficient (I think).
 
 -Steve


 
 Thanks for the email.
 
 It seems like it still crashes from SciTE, though. :(
 When I try debugging, I see it's doing so inside _close() which is
 /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other
 words, it seems to be called directly from the entrypoint of the
 program, assuming it hasn't undergone optimizations. The error is:
 0xC0000008: An invalid handle was specified.
 
 Is there any other place in which this can happen?


 
 Also, now that we're on the topic... this might or might not sound
 silly, but why not just use msvcrt.dll (if not the whole library, at
 least the I/O functions)? It's not like it introduces a new dependency
 (it's already in every version of Windows) and it's also proven to work.
 Not to mention that it reduces code size as well...


 
 I think the reason Walter always gives is that dmd would then have to
 output COFF objects, whereas it currently outputs OMF. OPTLINK also does
 not accept COFF objects.
 
 I think a lot of people would very much welcome this change, but it's
 somewhat of an underwhelming change. Instead of using DMC's runtime for C
 functions, you are using VC++'s runtime. But who cares? I want D's
 runtime to be better :)


The gain in switching to COFF would be _huge_, because then D code would be 
properly interoperable with most of the C code on Windows - and in particular 
C code compiled with Microsoft's compiler. The only thing saving the current 
situation from being a complete disaster is that you can still use dlls.

- Jonathan M Davis
Sep 26 2011
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 27.09.2011 2:33, Jonathan M Davis wrote:

 On Monday, September 26, 2011 14:53 Steven Schveighoffer wrote:

 On Mon, 26 Sep 2011 17:35:20 -0400, Mehrdad<wfunction hotmail.com>  wrote:

 On 9/26/2011 2:30 PM, Mehrdad wrote:

 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:

 AHA!

 Yes, there is a bug in snn.lib regarding pipes. And I fixed it,
 waiting for Walter to incorporate it :) I needed it for the new
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect
 some of the quirks of pipe HANDLEs. For instance, if you open a FILE
 * around a pipe handle, it still tries to do a seek on that handle,
 and crashes. Also, when the write end of a pipe is closed, reading
 from the read end results in EPIPE from ReadFile, but this is
 translated to EBADF by the runtime. Therefore, FILE * sets an error
 instead of EOF.

 Is the email address you have for this message correct? If so, I can
 send you a new version of snn.lib to try linking your code against (if
 you are willing to go through these steps), to see if it fixes your
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve


 Thanks for the email.

 It seems like it still crashes from SciTE, though. :(
 When I try debugging, I see it's doing so inside _close() which is
 /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other
 words, it seems to be called directly from the entrypoint of the
 program, assuming it hasn't undergone optimizations. The error is:
 0xC0000008: An invalid handle was specified.

 Is there any other place in which this can happen?


 Also, now that we're on the topic... this might or might not sound
 silly, but why not just use msvcrt.dll (if not the whole library, at
 least the I/O functions)? It's not like it introduces a new dependency
 (it's already in every version of Windows) and it's also proven to work.
 Not to mention that it reduces code size as well...


 I think the reason Walter always gives is that dmd would then have to
 output COFF objects, whereas it currently outputs OMF. OPTLINK also does
 not accept COFF objects.

 I think a lot of people would very much welcome this change, but it's
 somewhat of an underwhelming change. Instead of using DMC's runtime for C
 functions, you are using VC++'s runtime. But who cares? I want D's
 runtime to be better :)


 The gain in switching to COFF would be _huge_, because then D code would be
 properly interoperable with most of the C code on Windows - and in particular
 C code compiled with Microsoft's compiler. The only thing saving the current
 situation from being a complete disaster is that you can still use dlls.


It's not that easy. I mean to gain this level of compatibility we'd have 
to do both: switch to MS runtime and use COFF object format. Linking 
together two C runtimes is *ehm* tricky.

But most importantly newer versions of VC++ use newer runtimes, I'm not 
sure how compatible they are, but judging by my previous experience - 
you still has to keep things separated like in DLLs and never mix 
allocation/FILE* stuff between them.



 - Jonathan M Davis



-- 
Dmitry Olshansky
Sep 26 2011
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, September 26, 2011 16:07 Dmitry Olshansky wrote:

 On 27.09.2011 2:33, Jonathan M Davis wrote:

 On Monday, September 26, 2011 14:53 Steven Schveighoffer wrote:

 On Mon, 26 Sep 2011 17:35:20 -0400, Mehrdad<wfunction hotmail.com> 






wrote:

 On 9/26/2011 2:30 PM, Mehrdad wrote:

 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:

 AHA!
 
 Yes, there is a bug in snn.lib regarding pipes. And I fixed it,
 waiting for Walter to incorporate it :) I needed it for the new
 std.process.
 
 What it comes down to is, DMC's FILE * implementation does not expect
 some of the quirks of pipe HANDLEs. For instance, if you open a FILE
 * around a pipe handle, it still tries to do a seek on that handle,
 and crashes. Also, when the write end of a pipe is closed, reading
 from the read end results in EPIPE from ReadFile, but this is
 translated to EBADF by the runtime. Therefore, FILE * sets an error
 instead of EOF.
 
 Is the email address you have for this message correct? If so, I can
 send you a new version of snn.lib to try linking your code against
 (if you are willing to go through these steps), to see if it fixes
 your problem.
 
 Using the command line dmd to build should be sufficient (I think).
 
 -Steve


 
 Thanks for the email.
 
 It seems like it still crashes from SciTE, though. :(
 When I try debugging, I see it's doing so inside _close() which is
 /immediately/ above kernel32.dll! BaseThreadInitThunk 12() -- in other
 words, it seems to be called directly from the entrypoint of the
 program, assuming it hasn't undergone optimizations. The error is:
 0xC0000008: An invalid handle was specified.
 
 Is there any other place in which this can happen?


 
 Also, now that we're on the topic... this might or might not sound
 silly, but why not just use msvcrt.dll (if not the whole library, at
 least the I/O functions)? It's not like it introduces a new dependency
 (it's already in every version of Windows) and it's also proven to
 work. Not to mention that it reduces code size as well...


 
 I think the reason Walter always gives is that dmd would then have to
 output COFF objects, whereas it currently outputs OMF. OPTLINK also does
 not accept COFF objects.
 
 I think a lot of people would very much welcome this change, but it's
 somewhat of an underwhelming change. Instead of using DMC's runtime for
 C functions, you are using VC++'s runtime. But who cares? I want D's
 runtime to be better :)


 
 The gain in switching to COFF would be _huge_, because then D code would
 be properly interoperable with most of the C code on Windows - and in
 particular C code compiled with Microsoft's compiler. The only thing
 saving the current situation from being a complete disaster is that you
 can still use dlls.


 
 It's not that easy. I mean to gain this level of compatibility we'd have
 to do both: switch to MS runtime and use COFF object format. Linking
 together two C runtimes is *ehm* tricky.
 
 But most importantly newer versions of VC++ use newer runtimes, I'm not
 sure how compatible they are, but judging by my previous experience -
 you still has to keep things separated like in DLLs and never mix
 allocation/FILE* stuff between them.


That may be, but the fact that I can't just link C code built with Visual 
Studio with D code is a major problem in enviroments where Visual Studio is 
what is used on Windows (such as where I work). In comparison, in Linux it 
works, because dmd actually uses the compiler which is the de facto standard 
on Linux - gcc. But since dmd uses optlink on Windows, you just don't have 
that level of compatibility. dmc and optlink may very well be solid tools, but 
as a D programmer trying to interoperate with C code, compatability matters a 
lot, and dmd on Windows just doesn't have it.

- Jonathan M Davis
Sep 26 2011
prev sibling parent Kagamin <spam here.lot> writes:
Mehrdad Wrote:


 Also, now that we're on the topic... this might or might not sound 
 silly, but why not just use msvcrt.dll (if not the whole library, at 
 least the I/O functions)? It's not like it introduces a new dependency 
 (it's already in every version of Windows) and it's also proven to work. 
 Not to mention that it reduces code size as well...


gdc already does it.
Sep 27 2011
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 17:30:42 -0400, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 1:51 PM, Steven Schveighoffer wrote:

 AHA!

 Yes, there is a bug in snn.lib regarding pipes.  And I fixed it,  
 waiting for Walter to incorporate it :)  I needed it for the new  
 std.process.

 What it comes down to is, DMC's FILE * implementation does not expect  
 some of the quirks of pipe HANDLEs.  For instance, if you open a FILE *  
 around a pipe handle, it still tries to do a seek on that handle, and  
 crashes.  Also, when the write end of a pipe is closed, reading from  
 the read end results in EPIPE from ReadFile, but this is translated to  
 EBADF by the runtime.  Therefore, FILE * sets an error instead of EOF.

 Is the email address you have for this message correct?  If so, I can  
 send you a new version of snn.lib to try linking your code against (if  
 you are willing to go through these steps), to see if it fixes your  
 problem.

 Using the command line dmd to build should be sufficient (I think).

 -Steve


 Thanks for the email.

 It seems like it still crashes from SciTE, though. :(
 When I try debugging, I see it's doing so inside _close() which is  
 /immediately/ above kernel32.dll! BaseThreadInitThunk 12()  -- in other  
 words, it seems to be called directly from the entrypoint of the  
 program, assuming it hasn't undergone optimizations. The error is:  
 0xC0000008: An invalid handle was specified.

 Is there any other place in which this can happen?


Hm... its hard for me to say.  Why would it be calling _close before  
main() is called?  Very strange.  Is there any more stack information?

Can you get it to crash by using a pipe from the command line?

In other words:

myprogram | more

or echo hi | miprogram

?

If this isn't killing the program in the same way, then it might be a  
different issue than the one I fixed.

-Steve
Sep 26 2011
parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:36 PM, Steven Schveighoffer wrote:

 Hm... its hard for me to say.  Why would it be calling _close before 
 main() is called?  Very strange.  Is there any more stack information?

 Can you get it to crash by using a pipe from the command line?

 In other words:

 myprogram | more

 or echo hi | miprogram

 ?

 If this isn't killing the program in the same way, then it might be a 
 different issue than the one I fixed.

 -Steve


Nope, pipes/redirections from the command prompt are all fine -- it just 
crashes when run from SciTE (and also Notepad++, which I just verified).

After somewhat painful debugging, it seems like the stack trace is a 
little off -- it actually happens when _exit() is called, which in turn 
calls _fcloseall() or something.

Not sure what else I can do about it, though -- it's pretty clear the 
handle was destroyed sometime before, but I don't know when.
Sep 26 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 17:52:02 -0400, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 2:36 PM, Steven Schveighoffer wrote:

 Hm... its hard for me to say.  Why would it be calling _close before  
 main() is called?  Very strange.  Is there any more stack information?

 Can you get it to crash by using a pipe from the command line?

 In other words:

 myprogram | more

 or echo hi | miprogram

 ?

 If this isn't killing the program in the same way, then it might be a  
 different issue than the one I fixed.

 -Steve


 Nope, pipes/redirections from the command prompt are all fine -- it just  
 crashes when run from SciTE (and also Notepad++, which I just verified).

 After somewhat painful debugging, it seems like the stack trace is a  
 little off -- it actually happens when _exit() is called, which in turn  
 calls _fcloseall() or something.

 Not sure what else I can do about it, though -- it's pretty clear the  
 handle was destroyed sometime before, but I don't know when.


This is likely a DMC issue, and is probably best reported as a DMC bug.  I  
don't know how much more you want to pursue this, but the next steps I'd  
recommend are:

1. obtain the dmc compiler (it's free) if you haven't already.
2. Compile an empty C program and run it using SciTE and Notepad++.   
Verify the same error occurs
3. report the failure using DMC's bugzilla.

I'm guessing there's something in the way SciTE or Notepad++ sets up the  
pipes before executing the process which causes the problem.  I have a  
sinking feeling we'll see more of this issue when the new std.process is  
released :(

-Steve
Sep 26 2011
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
SciTE (and probably N++ since it's based on scintilla) has a subsystem
setting which can affect how it invokes a process. See the "Command
subsystem" section here: http://www.scintilla.org/SciTEDoc.html

Usually I run with subsystem 0.
Sep 26 2011
parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 3:07 PM, Andrej Mitrovic wrote:

 SciTE (and probably N++ since it's based on scintilla) has a subsystem
 setting which can affect how it invokes a process. See the "Command
 subsystem" section here: http://www.scintilla.org/SciTEDoc.html

 Usually I run with subsystem 0.


(1) Scintilla is just the text editor. It doesn't have anything to do 
with the integrated command line.

(2) I tried both 0 and 1, and neither made a difference. In fact, while 
using 1 DID pop up an extra command-line window, in either case the I/O 
took place inside SciTE itself. And in both cases, both DMD and my 
program crashed.

You should be able to easily reproduce this if you set the "Enable close 
exception" and  "Enable bad handles detection" Global Flags, using the 
Global Flags utility that comes with Debugging Tools for Windows. (Just 
make sure you reboot before you expect to see the change.)
Sep 26 2011
parent "Regan Heath" <regan netmail.co.nz> writes:
On Mon, 26 Sep 2011 23:14:43 +0100, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 3:07 PM, Andrej Mitrovic wrote:

 SciTE (and probably N++ since it's based on scintilla) has a subsystem
 setting which can affect how it invokes a process. See the "Command
 subsystem" section here: http://www.scintilla.org/SciTEDoc.html

 Usually I run with subsystem 0.


 (1) Scintilla is just the text editor. It doesn't have anything to do  
 with the integrated command line.

 (2) I tried both 0 and 1, and neither made a difference. In fact, while  
 using 1 DID pop up an extra command-line window, in either case the I/O  
 took place inside SciTE itself. And in both cases, both DMD and my  
 program crashed.

 You should be able to easily reproduce this if you set the "Enable close  
 exception" and  "Enable bad handles detection" Global Flags, using the  
 Global Flags utility that comes with Debugging Tools for Windows. (Just  
 make sure you reboot before you expect to see the change.)


On newer versions of windows you can use Application Verifier.. provided  
you run the exe you want to verify in a debugger (WinDbg or MSVC).

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Sep 27 2011
prev sibling next sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:

 This is likely a DMC issue, and is probably best reported as a DMC 
 bug.  I don't know how much more you want to pursue this, but the next 
 steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.  
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up 
 the pipes before executing the process which causes the problem.  I 
 have a sinking feeling we'll see more of this issue when the new 
 std.process is released :(

 -Steve


I'm suspecting it's /not/ a DMC issue (it's probably just an improperly 
used handle in snn.lib), because I don't see how it could affect my D 
program this way, if it was a DMC issue.
And yeah, I just tested it with a simple hello world C example using DMC 
as well, and got the same error. Pretty darn sure now that the handle is 
getting closed twice or something...
Sep 26 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 18:17:49 -0400, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:

 This is likely a DMC issue, and is probably best reported as a DMC  
 bug.  I don't know how much more you want to pursue this, but the next  
 steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.   
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up  
 the pipes before executing the process which causes the problem.  I  
 have a sinking feeling we'll see more of this issue when the new  
 std.process is released :(

 -Steve


 I'm suspecting it's /not/ a DMC issue (it's probably just an improperly  
 used handle in snn.lib), because I don't see how it could affect my D  
 program this way, if it was a DMC issue.
 And yeah, I just tested it with a simple hello world C example using DMC  
 as well, and got the same error. Pretty darn sure now that the handle is  
 getting closed twice or something...


My apologies, I meant DMC runtime issue (whose library is called snn.lib).

At least with this test, we can rule out D being the culprit.  However, we  
still have to deal with this problem...

-Steve
Sep 26 2011
parent Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 3:37 PM, Steven Schveighoffer wrote:

 On Mon, 26 Sep 2011 18:17:49 -0400, Mehrdad <wfunction hotmail.com> 
 wrote:


 On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:

 This is likely a DMC issue, and is probably best reported as a DMC 
 bug.  I don't know how much more you want to pursue this, but the 
 next steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.  
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up 
 the pipes before executing the process which causes the problem.  I 
 have a sinking feeling we'll see more of this issue when the new 
 std.process is released :(

 -Steve


 I'm suspecting it's /not/ a DMC issue (it's probably just an 
 improperly used handle in snn.lib), because I don't see how it could 
 affect my D program this way, if it was a DMC issue.
 And yeah, I just tested it with a simple hello world C example using 
 DMC as well, and got the same error. Pretty darn sure now that the 
 handle is getting closed twice or something...


 My apologies, I meant DMC runtime issue (whose library is called 
 snn.lib).

 At least with this test, we can rule out D being the culprit.  
 However, we still have to deal with this problem...

 -Steve


Ahhh ok I see. If I'm in the mood sometime I'll post it somewhere 
DMC-related as well. :)
Sep 26 2011
prev sibling parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:

 This is likely a DMC issue, and is probably best reported as a DMC 
 bug.  I don't know how much more you want to pursue this, but the next 
 steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.  
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up 
 the pipes before executing the process which causes the problem.  I 
 have a sinking feeling we'll see more of this issue when the new 
 std.process is released :(

 -Steve


OK, I think I found the bug, and I'm pretty sure it's what I was saying 
originally:

A handle is being closed that isn't actually valid.

Which handle? stderr.  stderr is being closed when in fact it is the 
same handle as stdout. So when _fcloseallp() is called, it goes through 
the structures in __iob and closes each one in turn -- which causes a 
duplicate close operation on stderr, because it already gets closed when 
stdout gets closed.

So it's an snn.lib issue. Is it easy to fix? (I hope it is...)

Thanks!
Sep 26 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 18:36:16 -0400, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 2:58 PM, Steven Schveighoffer wrote:

 This is likely a DMC issue, and is probably best reported as a DMC  
 bug.  I don't know how much more you want to pursue this, but the next  
 steps I'd recommend are:

 1. obtain the dmc compiler (it's free) if you haven't already.
 2. Compile an empty C program and run it using SciTE and Notepad++.   
 Verify the same error occurs
 3. report the failure using DMC's bugzilla.

 I'm guessing there's something in the way SciTE or Notepad++ sets up  
 the pipes before executing the process which causes the problem.  I  
 have a sinking feeling we'll see more of this issue when the new  
 std.process is released :(

 -Steve


 OK, I think I found the bug, and I'm pretty sure it's what I was saying  
 originally:

 A handle is being closed that isn't actually valid.

 Which handle? stderr.  stderr is being closed when in fact it is the  
 same handle as stdout. So when _fcloseallp() is called, it goes through  
 the structures in __iob and closes each one in turn -- which causes a  
 duplicate close operation on stderr, because it already gets closed when  
 stdout gets closed.

 So it's an snn.lib issue. Is it easy to fix? (I hope it is...)


Probably -- let the OS close the handles instead of manually closing them  
:)

-Steve
Sep 26 2011
parent reply Mehrdad <wfunction hotmail.com> writes:
On 9/26/2011 3:41 PM, Steven Schveighoffer wrote:

 Probably -- let the OS close the handles instead of manually closing 
 them :)

 -Steve


I don't think that's a good idea because snn.lib likely does its own 
internal buffering of the data, which would be lost if the stream isn't 
closed.
Sep 26 2011
next sibling parent Rainer Schuetze <r.sagitario gmx.de> writes:
I think, http://d.puremagic.com/issues/show_bug.cgi?id=1550 is very much 
related.

IIRC, the handle for stdin, stdout and stderr are retreived by 
GetStdHandle(), which should not be closed at all. Definitely should be 
fixed in snn.lib.


On 27.09.2011 01:15, Mehrdad wrote:

 On 9/26/2011 3:41 PM, Steven Schveighoffer wrote:

 Probably -- let the OS close the handles instead of manually closing
 them :)

 -Steve


 I don't think that's a good idea because snn.lib likely does its own
 internal buffering of the data, which would be lost if the stream isn't
 closed.
Sep 27 2011
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 26 Sep 2011 19:15:08 -0400, Mehrdad <wfunction hotmail.com> wrote:


 On 9/26/2011 3:41 PM, Steven Schveighoffer wrote:

 Probably -- let the OS close the handles instead of manually closing  
 them :)

 -Steve


 I don't think that's a good idea because snn.lib likely does its own  
 internal buffering of the data, which would be lost if the stream isn't  
 closed.


That's what fflush is for.  The handles should simply be closed by the OS.

-Steve
Sep 27 2011
prev sibling parent Don <nospam nospam.com> writes:
On 26.09.2011 16:55, Mehrdad wrote:

 DMD 2.055 seems to be crashing on my computer (Windows 7 x64).

 When I press Debug, I see it's doing so in CloseHandle, probably because
 I've turned on "Enable close exception" with Visual Studio's "Global
 Flags" tool. (It takes a reboot to take effect.)

 So this means there's an incorrect handle being used somewhere... but
 interestingly, by the time it crashes, the output file is already
 produced. I don't have the time to make a debug build right now, but I
 thought I'd mention this so that it could be looked into at some point.


That's interesting. I recently found another bug, which is also Windows7 
specific.
http://d.puremagic.com/issues/show_bug.cgi?id=6660
Although I assumed that it's stack corruption, it could also be the same 
handle issue.
Sep 26 2011