www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Seemingly patternless optlink premature termination

reply pineapple <meapineapple gmail.com> writes:
I'm working on my dumb library and I have run into a case where 
when I add this line to a module, I get an optlink error 
attempting to compile it:

     unittest{}

The module compiles fine without it. So do the modules which 
depend on it.

But with that line? optlink errors everywhere.

A screencap of the error box that pops up:

     http://puu.sh/tfmBK/c5068eb2fb.png

And the contents of stdout:

     checkpoint(256)
     --- errorlevel 1

I encountered a similar error right before this and eventually 
worked out that I had introduced a cyclic dependency into my 
project some time ago - strangely not rearing its head as an 
error until I made a completely unrelated addition to the tests 
in a module incidentally importing one of the modules involved in 
that dependency cycle - and addressing that issue seemed to 
resolve the optlink error. (At least until I tried to add some 
extra unit tests to one of the modules involved in that cycle.)

I am utterly failing to find a way to make a simple repro case 
from this, or to understand what could conceivably be causing 
this error to occur in a module whose dependency graph is small 
and simple.
Jan 08 2017
next sibling parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 8 January 2017 at 14:15:26 UTC, pineapple wrote:

 I'm working on my dumb library and I have run into a case where 
 when I add this line to a module, I get an optlink error 
 attempting to compile it:

     unittest{}


After deciding to let my inability to add some unit tests to that 
module I started working on some other code in the same project. 
At this point it seems completely arbitrary which lines produce 
optlink errors. Since the `unittest{}` issue I have, in other 
modules, had optlink errors occur depending on the presence of 
arbitrary method and function calls, too.
Jan 08 2017
parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 8 January 2017 at 14:42:57 UTC, pineapple wrote:

 After deciding to let my inability to add some unit tests to 
 that module I started working on some other code in the same 
 project. At this point it seems completely arbitrary which 
 lines produce optlink errors. Since the `unittest{}` issue I 
 have, in other modules, had optlink errors occur depending on 
 the presence of arbitrary method and function calls, too.


Update from IRC: Stefan was able to determine that this is a 
stack corruption bug with optlink. (How and why I should be the 
one to stumble upon it is yet to be determined.)
Jan 08 2017
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 8 January 2017 at 17:14:13 UTC, pineapple wrote:

 On Sunday, 8 January 2017 at 14:42:57 UTC, pineapple wrote:

 After deciding to let my inability to add some unit tests to 
 that module I started working on some other code in the same 
 project. At this point it seems completely arbitrary which 
 lines produce optlink errors. Since the `unittest{}` issue I 
 have, in other modules, had optlink errors occur depending on 
 the presence of arbitrary method and function calls, too.


 Update from IRC: Stefan was able to determine that this is a 
 stack corruption bug with optlink. (How and why I should be the 
 one to stumble upon it is yet to be determined.)


No I was not.
I think! it could be one.
I'll post it when there is something concrete to post.
Jan 08 2017
prev sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 8 January 2017 at 17:14:13 UTC, pineapple wrote:

 On Sunday, 8 January 2017 at 14:42:57 UTC, pineapple wrote:

 After deciding to let my inability to add some unit tests to 
 that module I started working on some other code in the same 
 project. At this point it seems completely arbitrary which 
 lines produce optlink errors. Since the `unittest{}` issue I 
 have, in other modules, had optlink errors occur depending on 
 the presence of arbitrary method and function calls, too.


 Update from IRC: Stefan was able to determine that this is a 
 stack corruption bug with optlink. (How and why I should be the 
 one to stumble upon it is yet to be determined.)


No I was able to determine it not.
I think, it could be one.
I'll post it when there is something concrete to post.
Jan 08 2017
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 6:15 AM, pineapple wrote:

 I'm working on my dumb library and I have run into a case where when I add this
 line to a module, I get an optlink error attempting to compile it:

     unittest{}

 The module compiles fine without it. So do the modules which depend on it.

 But with that line? optlink errors everywhere.

 A screencap of the error box that pops up:

     http://puu.sh/tfmBK/c5068eb2fb.png

 And the contents of stdout:

     checkpoint(256)
     --- errorlevel 1

 I encountered a similar error right before this and eventually worked out that
I
 had introduced a cyclic dependency into my project some time ago - strangely
not
 rearing its head as an error until I made a completely unrelated addition to
the
 tests in a module incidentally importing one of the modules involved in that
 dependency cycle - and addressing that issue seemed to resolve the optlink
 error. (At least until I tried to add some extra unit tests to one of the
 modules involved in that cycle.)

 I am utterly failing to find a way to make a simple repro case from this, or to
 understand what could conceivably be causing this error to occur in a module
 whose dependency graph is small and simple.


Please post bug reports to bugzilla. I transcribed the error here:

EAX=00000010
EBX=0000001C
ECX=00000000
EDX=004337CF
ESI=0044AAE8
EDI=00000100
EBP=0018FF54
ESP=0018FDEC
EIP=0040347E
First=00402000

Try to include the code that causes the problem.

The way to debug this (for anyone interested) is to get it to fail running
under 
windbg.exe. Then examine the instructions around where it failed. Look in the 
source code to find where those instructions are, and hence the location in the 
code where it faulted.

P.S. Isn't it maddening that Windows doesn't allow copying the data from a 
message window?
Jan 08 2017
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 On 1/8/2017 6:15 AM, pineapple wrote:

 I'm working on my dumb library and I have run into a case 
 where when I add this
 line to a module, I get an optlink error attempting to compile 
 it:

     unittest{}

 The module compiles fine without it. So do the modules which 
 depend on it.

 But with that line? optlink errors everywhere.

 A screencap of the error box that pops up:

     http://puu.sh/tfmBK/c5068eb2fb.png

 And the contents of stdout:

     checkpoint(256)
     --- errorlevel 1

 I encountered a similar error right before this and eventually 
 worked out that I
 had introduced a cyclic dependency into my project some time 
 ago - strangely not
 rearing its head as an error until I made a completely 
 unrelated addition to the
 tests in a module incidentally importing one of the modules 
 involved in that
 dependency cycle - and addressing that issue seemed to resolve 
 the optlink
 error. (At least until I tried to add some extra unit tests to 
 one of the
 modules involved in that cycle.)

 I am utterly failing to find a way to make a simple repro case 
 from this, or to
 understand what could conceivably be causing this error to 
 occur in a module
 whose dependency graph is small and simple.


 Please post bug reports to bugzilla. I transcribed the error 
 here:

 EAX=00000010
 EBX=0000001C
 ECX=00000000
 EDX=004337CF
 ESI=0044AAE8
 EDI=00000100
 EBP=0018FF54
 ESP=0018FDEC
 EIP=0040347E
 First=00402000

 Try to include the code that causes the problem.

 The way to debug this (for anyone interested) is to get it to 
 fail running under windbg.exe. Then examine the instructions 
 around where it failed. Look in the source code to find where 
 those instructions are, and hence the location in the code 
 where it faulted.

 P.S. Isn't it maddening that Windows doesn't allow copying the 
 data from a message window?


Any chance to get a debug build of optlink ?
Jan 08 2017
prev sibling next sibling parent reply jkpl <jkpl nowhere.de> writes:
On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 P.S. Isn't it maddening that Windows doesn't allow copying the 
 data from a message window?


Not really, because you can, Ctrl+C...
Jan 08 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 10:56 AM, jkpl wrote:

 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 P.S. Isn't it maddening that Windows doesn't allow copying the data from a
 message window?


 Not really, because you can, Ctrl+C...


That frequently does not work. For example, take the "Help" window of 
Thunderbird, which is my email program. You cannot copy data from it.
Jan 08 2017
prev sibling next sibling parent reply Jerry <hurricane hereiam.com> writes:
On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 P.S. Isn't it maddening that Windows doesn't allow copying the 
 data from a message window?


A bit more maddening that D is still trying to support optlink, a 
linker that hasn't seen a commit in 2 years when it is hardly bug 
free.
Jan 08 2017
parent reply jkpl <jkpl somewhere.de> writes:
On Sunday, 8 January 2017 at 19:15:49 UTC, Jerry wrote:

 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 P.S. Isn't it maddening that Windows doesn't allow copying the 
 data from a message window?


 A bit more maddening that D is still trying to support optlink, 
 a linker that hasn't seen a commit in 2 years when it is hardly 
 bug free.


Your comment is not pertinent. The object format doesn't change 
so the linker doesn't need to be updated. The problem here seems 
to be a bug. Until someone finds it, it can't be fixed.
Jan 08 2017
parent reply Jerry <hurricane hereiam.com> writes:
On Sunday, 8 January 2017 at 20:36:38 UTC, jkpl wrote:

 On Sunday, 8 January 2017 at 19:15:49 UTC, Jerry wrote:

 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 P.S. Isn't it maddening that Windows doesn't allow copying 
 the data from a message window?


 A bit more maddening that D is still trying to support 
 optlink, a linker that hasn't seen a commit in 2 years when it 
 is hardly bug free.


 Your comment is not pertinent. The object format doesn't change 
 so the linker doesn't need to be updated. The problem here 
 seems to be a bug. Until someone finds it, it can't be fixed.


We are discussing Optlink and a bug related to it. The fact it's 
20 years old, written in assembly, and hasn't had a bug fix in 
years even though they have been reported is very very very much 
relevant. It's legacy software that shouldn't be relied on unless 
you also consider D to be legacy (I think most probably won't).
Jan 08 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On Sunday, 8 January 2017 at 19:15:49 UTC, Jerry wrote:

 A bit more maddening that D is still trying to support optlink, a linker that
 hasn't seen a commit in 2 years when it is hardly bug free.



Optlink has had many bugs fixed in it. There are currently 7 open issues with
it 
on bugzilla:

https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=dmd&list_id=212971&query_format=advanced&short_desc=optlink&short_desc_type=allwordssubstr

This includes the issue in this thread. Only 1 of the other issues looks like a 
bug in Optlink.

If you know of unreported issues with Optlink, please report them to bugzilla.
Jan 08 2017
parent reply Jerry <hurricane hereiam.com> writes:
On Monday, 9 January 2017 at 01:26:48 UTC, Walter Bright wrote:

 On Sunday, 8 January 2017 at 19:15:49 UTC, Jerry wrote:

 A bit more maddening that D is still trying to support 
 optlink, a linker that
 hasn't seen a commit in 2 years when it is hardly bug free.



 Optlink has had many bugs fixed in it. There are currently 7 
 open issues with it on bugzilla:

 https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&component=dmd&list_id=212971&query_format=advanced&short_desc=optlink&short_desc_type=allwordssubstr

 This includes the issue in this thread. Only 1 of the other 
 issues looks like a bug in Optlink.


There's more, just the bugs are so obscure the people filing the 
bugs don't know that the problem lies with Optlink. There's 
problems with shared libraries not being loaded and unloaded 
properly. Random crashes during runtime, more often crashes when 
a debugger is attached. Some more that I ran into as well before 
I switched away from optlink. What point is there to report these 
bugs though, Microsoft's linker works with fewer bugs and 
actually supports the format of the platform. So no need to 
convert .lib files with it. This is my point, it's just one more 
thing that has to be maintained, but it's not even being 
maintained. Supporting MinGW would be a better option as then you 
don't have to maintain the linker and not have it degrade over 
time.


 If you know of unreported issues with Optlink, please report 
 them to bugzilla.


To what end? There isn't even an Optlink specific bug list. Look 
at the bug with Optlink related to data that has more than 16 MB. 
What happened with that? Was that known bug fixed? No, a 
limitation was introduced into DMD to accommodate Optlink's bug. 
Rather than doing the sane decision to fix the bug in Optlink. 
Why? Probably because no one wants to deal with assembly.
Jan 08 2017
next sibling parent reply pineapple <meapineapple gmail.com> writes:
On Monday, 9 January 2017 at 02:34:31 UTC, Jerry wrote:

 before I switched away from optlink. What point is there to 
 report these bugs though, Microsoft's linker works with fewer 
 bugs and actually supports the format of the platform. So no 
 need to convert .lib files with it. This is my point, it's just 
 one more thing that has to be maintained, but it's not even 
 being maintained. Supporting MinGW would be a better option as 
 then you don't have to maintain the linker and not have it 
 degrade over time.


On this subject, I spent a good chunk of my day trying to compile 
my code using the -m32mscoff switch and was never able to get it 
to work. (My most recent obstacle is this error, occurring even 
for a very simple test program importing no dependencies, if 
anyone cares to take a look: https://hastebin.com/vezojehaxo.txt)

On Monday, 9 January 2017 at 01:27:27 UTC, Walter Bright wrote:

 On 1/8/2017 12:58 PM, pineapple wrote:

 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 Please post bug reports to bugzilla.


 Here you go: https://issues.dlang.org/show_bug.cgi?id=17077


 Thank you.


Regarding the above: Because I have not managed to get an 
alternative to optlink working, I currently am unable to work on 
mach on my Windows desktop. In order to keep up progress I've 
been having to write and compile code on OSX on a laptop; it is 
nowhere near an ideal workflow. I understand you likely have a 
lot on your plate, but please prioritize solving this problem. 
It's causing me a great deal of irritation and inconvenience, and 
it's embarrassing that my library now simply won't compile on 
Windows.
Jan 08 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 8:24 PM, pineapple wrote:

 On this subject, I spent a good chunk of my day trying to compile my code using
 the -m32mscoff switch and was never able to get it to work. (My most recent
 obstacle is this error, occurring even for a very simple test program importing
 no dependencies, if anyone cares to take a look:
 https://hastebin.com/vezojehaxo.txt)


Please post bug reports to bugzilla, not pastebins.



 Regarding the above: Because I have not managed to get an alternative to
optlink
 working, I currently am unable to work on mach on my Windows desktop. In order
 to keep up progress I've been having to write and compile code on OSX on a
 laptop; it is nowhere near an ideal workflow. I understand you likely have a
lot
 on your plate, but please prioritize solving this problem. It's causing me a
 great deal of irritation and inconvenience, and it's embarrassing that my
 library now simply won't compile on Windows.


You could try compiling it with -m64.
Jan 08 2017
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 6:34 PM, Jerry wrote:

 There's more, just the bugs are so obscure the people filing the bugs don't
know
 that the problem lies with Optlink. There's problems with shared libraries not
 being loaded and unloaded properly. Random crashes during runtime, more often
 crashes when a debugger is attached. Some more that I ran into as well before I
 switched away from optlink. What point is there to report these bugs though,


If you don't report bugs, they are guaranteed to not get fixed. There's nothing 
anyone can do with the statement you made.



 If you know of unreported issues with Optlink, please report them to bugzilla.


 To what end?


Complaining that somebody didn't fix problems neither you nor anyone else 
reported is not very sporting.



 There isn't even an Optlink specific bug list.


I found the list I posted by searching bugzilla for "optlink". If there are any 
I missed, please let me know.
Jan 08 2017
next sibling parent ag0aep6g <anonymous example.com> writes:
On 01/09/2017 08:05 AM, Walter Bright wrote:

 I found the list I posted by searching bugzilla for "optlink". If there
 are any I missed, please let me know.


snn.lib is closely related to optlink, because COFF tools have no 
business with snn.lib.

https://issues.dlang.org/buglist.cgi?quicksearch=snn
Jan 09 2017
prev sibling next sibling parent reply Mike Wey <mike-wey example.com> writes:
On 01/09/2017 08:05 AM, Walter Bright wrote:

 There isn't even an Optlink specific bug list.


 I found the list I posted by searching bugzilla for "optlink". If there
 are any I missed, please let me know.


https://issues.dlang.org/show_bug.cgi?id=15418

Is there anything i should add to mark it as an optlink bug?

-- 
Mike Wey
Jan 09 2017
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/9/2017 10:57 AM, Mike Wey wrote:

 https://issues.dlang.org/show_bug.cgi?id=15418

 Is there anything i should add to mark it as an optlink bug?



I put Optlink in the title for you. Also created an optlink keyword.
Jan 09 2017
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/9/2017 10:57 AM, Mike Wey wrote:

 Is there anything i should add to mark it as an optlink bug?



I added "optlink" as a keyword, so you can search that way:

https://issues.dlang.org/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&keywords=optlink&keywords_type=allwords&list_id=213005&query_format=advanced

Of course, this relies on bugzilla issues about optlink to be tagged with the 
keyword.
Jan 09 2017
prev sibling next sibling parent Jerry <hurricane hereiam.com> writes:
On Monday, 9 January 2017 at 07:05:27 UTC, Walter Bright wrote:

 If you don't report bugs, they are guaranteed to not get fixed. 
 There's nothing anyone can do with the statement you made.


If it's an Optlink bug it is pretty much a guarantee regardless 
even if the bug is reported.


 To what end?


 Complaining that somebody didn't fix problems neither you nor 
 anyone else reported is not very sporting.


Guess you missed literally the rest of that paragraph that 
outlines how bugs are actually handled in regards to optlink, let 
me repost it:

Look at the bug with Optlink related to data that has more than 
16 MB. What happened with that? Was that known bug fixed? No, a 
limitation was introduced into DMD to accommodate Optlink's bug. 
Rather than doing the sane decision to fix the bug in Optlink. 
Why? Probably because no one wants to deal with assembly.


Bugs get reported, reported bugs don't get fixed, limitations are 
introduced to accommodate bugs. That's the way Optlink is handled.

I'd more "sporting" if "please report bugs for optlink" wasn't 
synonymous for, "please think we care about (and might actually 
fix it) the bug you found with our project even though no one has 
touched optlink in 2 years". At some point you have to say, yes 
this isn't working, we should do something else. It becomes that 
much more difficult when the other party isn't even willing to 
accept there's a problem when optlink hasn't had a commit in 2 
years. Stern words need to be used to get it through to the 
stubborn party.


 There isn't even an Optlink specific bug list.


 I found the list I posted by searching bugzilla for "optlink". 
 If there are any I missed, please let me know.


I meant a subsection of like dmd/druntime/phobos, but I guess 
that wouldn't help if people don't know the bugs they are 
reporting are for optlink.
Jan 09 2017
prev sibling next sibling parent Jerry <hurricane hereiam.com> writes:
On Monday, 9 January 2017 at 07:05:27 UTC, Walter Bright wrote:

 Complaining that somebody didn't fix problems neither you nor 
 anyone else reported is not very sporting.


Just to clarify, I'm not complaining about a problem not being 
fixed. I'm advocating for the removal of Optlink, I don't care if 
it gets fixed or not cause I'm not using it anymore. I don't want 
new people to go through the problems I had to go through using 
it, before I realized I could do the more sane thing and just use 
Microsoft's linker.
Jan 09 2017
prev sibling parent Anonymouse <asdf asdf.net> writes:
On Monday, 9 January 2017 at 07:05:27 UTC, Walter Bright wrote:

 If you don't report bugs, they are guaranteed to not get fixed. 
 There's nothing anyone can do with the statement you made.


A good quote is "unreported bugs can only be fixed by accident."
Jan 10 2017
prev sibling parent reply pineapple <meapineapple gmail.com> writes:
On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 Please post bug reports to bugzilla.


Here you go: https://issues.dlang.org/show_bug.cgi?id=17077
Jan 08 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/8/2017 12:58 PM, pineapple wrote:

 On Sunday, 8 January 2017 at 18:49:50 UTC, Walter Bright wrote:

 Please post bug reports to bugzilla.


 Here you go: https://issues.dlang.org/show_bug.cgi?id=17077


Thank you.
Jan 08 2017