www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Yet more OPTLINK woes

reply Daniel Keep <daniel.keep.lists gmail.com> writes:
That's right, it's time for everyone's favourite [1] game: guess why
OPTLINK's not working! [2]

*sigh*  I'm writing a math eval library.  There are two test
applications.  LexerTest only touches part of the code.  AstTest touches
everything.

Now, the following works and creates an executable:

dmd -ofLexerTest (appropriate .d files)

So far, so good.  I get LexerTest.map, LexerTest.obj and LexerTest.exe.
 Let's try the other one...

dmd -ofAstTest (more .d files)

This creates a legitimate-looking AstTest.obj and a completely empty
AstTest.map file.

That's it.

No executable, no error message, no register dump, nothing.

Adding or removing -g, -debug, -unittest, -release, -inline, -O does
nothing.  Changing the target filename does nothing useful.

There are no spurious link.exe or dmd.exe processes running.  Invoking
OPTLINK directly changes nothing.

Does anyone have any idea, any idea at all, on what could be causing
this?  I've tried everything myself and several others on #d could think of.

*miserable sob*

After over five years of this sort of shit, I am so, so completely and
utterly sick to death of OPTLINK.



[1] I am, of course, being sarcastic.

[2] It is, of course, entirely possible that it's not OPTLINK's fault.
Frankly though, I find that hard to believe.
May 12 2010
next sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Daniel Keep <daniel.keep.lists gmail.com> wrote:


 That's right, it's time for everyone's favourite [1] game: guess why
 OPTLINK's not working! [2]


[...]

 Does anyone have any idea, any idea at all, on what could be causing
 this?  I've tried everything myself and several others on #d could think  
 of.


<channeling src="Walter">Please reduce this to a 1-line test case.
</channeling>

You know for sure OPTLINK is being run?
If so, that wisdom of Walter's is my best idea. Try and remove stuff
until it starts working, then remove other stuff until there's nothing
left to remove. I know, it's boring and should be unnecessary. Still,
sometimes it's the only thing that works.

-- 
Simen
May 12 2010
parent reply div0 <div0 users.sourceforge.net> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Simen kjaeraas wrote:

 Daniel Keep <daniel.keep.lists gmail.com> wrote:
 

 That's right, it's time for everyone's favourite [1] game: guess why
 OPTLINK's not working! [2]


 [...]

 Does anyone have any idea, any idea at all, on what could be causing
 this?  I've tried everything myself and several others on #d could
 think of.


 
 <channeling src="Walter">Please reduce this to a 1-line test case.
 </channeling>
 
 You know for sure OPTLINK is being run?
 If so, that wisdom of Walter's is my best idea. Try and remove stuff
 until it starts working, then remove other stuff until there's nothing
 left to remove. I know, it's boring and should be unnecessary. Still,
 sometimes it's the only thing that works.
 


Yah, it might not actually be OPTLINK for a change.

I've had problems before with DMD silently exiting without an error
message and with error success and leaving behind a corrupt .obj.

Can't remember how I worked that out though.

You could try the -v switch and/or trying to dump the contents of the .obj

- --
My enormous talent is exceeded only by my outrageous laziness.
http://www.ssTk.co.uk
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFL6ux+T9LetA9XoXwRArL6AKC0Lv88VmvKRle4ZD+Tv/P/0qdRlQCfaGg5
iDfslwtQJjROqjUug23VqlE=
=dtSa
-----END PGP SIGNATURE-----
May 12 2010
parent Don <nospam nospam.com> writes:
div0 wrote:

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 Simen kjaeraas wrote:

 Daniel Keep <daniel.keep.lists gmail.com> wrote:


 That's right, it's time for everyone's favourite [1] game: guess why
 OPTLINK's not working! [2]


 [...]

 Does anyone have any idea, any idea at all, on what could be causing
 this?  I've tried everything myself and several others on #d could
 think of.


 <channeling src="Walter">Please reduce this to a 1-line test case.
 </channeling>

 You know for sure OPTLINK is being run?
 If so, that wisdom of Walter's is my best idea. Try and remove stuff
 until it starts working, then remove other stuff until there's nothing
 left to remove. I know, it's boring and should be unnecessary. Still,
 sometimes it's the only thing that works.


 
 Yah, it might not actually be OPTLINK for a change.
 
 I've had problems before with DMD silently exiting without an error
 message and with error success and leaving behind a corrupt .obj.


Yes, it sounds to me like a DMD stack overflow. I've seen that behaviour 
frequently, and optlink was never to blame. CTFE recursion is the only 
reported unfixed bug that does that -- but I've fixed half a dozen of 
them in the last year, so there may be more.
May 12 2010
prev sibling next sibling parent torhu <no spam.invalid> writes:
On 12.05.2010 13:22, Daniel Keep wrote:

 That's right, it's time for everyone's favourite [1] game: guess why
 OPTLINK's not working! [2]

 *sigh*  I'm writing a math eval library.  There are two test
 applications.  LexerTest only touches part of the code.  AstTest touches
 everything.

 Now, the following works and creates an executable:

 dmd -ofLexerTest (appropriate .d files)

 So far, so good.  I get LexerTest.map, LexerTest.obj and LexerTest.exe.
   Let's try the other one...

 dmd -ofAstTest (more .d files)

 This creates a legitimate-looking AstTest.obj and a completely empty
 AstTest.map file.

 That's it.

 No executable, no error message, no register dump, nothing.


Post the source and I'll try to help. I like debugging weird problems. :)
May 12 2010
prev sibling next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Some general replies:

There have been three responses along the lines of "are you sure OPTLINK
is running?".  Quoting myself:

"Invoking OPTLINK directly changes nothing."

Or to be more specific:


 link AstTest,AstTest,,user32+kernel32/noi+tango.lib;


(zero-byte .map file pops into existence)

Yes, I'm sure.  I'm also sure it's the DigitalMars link.exe and in the
correct directory.


 I've had problems before with DMD silently exiting without an error
 message and with error success and leaving behind a corrupt .obj.


I had a look at the .obj file (I actually used lib to stuff it into a
.lib file and then dump the symbols), and it looked fine.

Well, except for this:

_D11TokenStream<80><8d><8d>5_ctorMFC6Source<f6>P<80><90><91>
  DFS8Location<80><89><89>AaYvJS6<80><85><cf>s5<f4>Zb<80>
  <99><aa>ZC<81><9a><87>

(broken across 3 lines; <xx> are bytes in hex since several of them were
non-printable.)

The same had happened to a number of symbols from tango.io.stream.Format.

In the case of the first one, the appropriate declarations (with
fully-qualified module name commented) are:


 final class /*TokenStream.*/TokenStream
 {
     alias bool function(Source, LocErr, out Token) NextToken;
     this(Source src, NextToken next, LocErr err) { ... }
 }

 final class /*Source.*/Source { ... }

 alias void delegate(Location, char[], ...) /*Location.*/LocErr;

 struct /*Location.*/Location { ... }


As far as I understand D's name mangling, that symbol name is corrupt.


 CTFE recursion is the only reported unfixed bug that does that ...


There is exactly one CTFE function that calls itself.  However, it
cannot recurse more than once (it does so to change types) *and* this
function is used in the LexerTest program which does compile.

AstTest uses an additional CTFE function, but this just concatenates
some strings together.


 Try and remove stuff until it starts working


I've gutted the changes between LexerTest and AstTest to the point that
I get an .exe.  This involved stripping out more or less every global
and member function that was added and every derived class.

What tipped it over into compiling was removing the reference to
StructuredOutput from AstDumpVisitor.  Which is odd, because
StructuredOutput is still being used in AstTest.

StructuredOutput, once gutted, is this:


 module StructuredOutput;

 import tango.io.model.IConduit : OutputStream;
 import tango.io.stream.Format : FormatOutput;

 final class StructuredOutput
 {
     alias StructuredOutput This;
     this(OutputStream os)
     {
     }
 }


I honestly can't see what it could be upset about.


 Post the source and I'll try to help. I like debugging weird problems.
 :)


Attached both regular and decaffeinated^Hgutted versions.
May 13 2010
next sibling parent reply div0 <div0 users.sourceforge.net> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Daniel Keep wrote:

  
 Well, except for this:
 
 _D11TokenStream<80><8d><8d>5_ctorMFC6Source<f6>P<80><90><91>
   DFS8Location<80><89><89>AaYvJS6<80><85><cf>s5<f4>Zb<80>
   <99><aa>ZC<81><9a><87>
 
 (broken across 3 lines; <xx> are bytes in hex since several of them were
 non-printable.)



 As far as I understand D's name mangling, that symbol name is corrupt.


I thought that as well, but they aren't corrupt,
according to WB they are compressed:

http://d.puremagic.com/issues/show_bug.cgi?id=2238

Strange what OPTLINK is doing; normally it crashes at the drop of a hat,
so it seems unreasonable that it processes the .obj a bit then
(un)successfully exits. Maybe the .obj is corrupt in some way that makes
it look empty, so it does nothing.

Walter did post about converting it to C, did he finish that in the end?

- --
My enormous talent is exceeded only by my outrageous laziness.
http://www.ssTk.co.uk
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFL7EEKT9LetA9XoXwRAh05AKCbEe/EqK8vhwvml/yha6J6OpqmtQCePg0a
2509Ij9EAmos2Fa8/dC+Ui4=
=VtO5
-----END PGP SIGNATURE-----
May 13 2010
parent "Nick Sabalausky" <a a.a> writes:
"div0" <div0 users.sourceforge.net> wrote in message 
news:hshfe9$o06$1 digitalmars.com...

 Strange what OPTLINK is doing; normally it crashes at the drop of a hat,
 so it seems unreasonable that it processes the .obj a bit then
 (un)successfully exits. Maybe the .obj is corrupt in some way that makes
 it look empty, so it does nothing.

 Walter did post about converting it to C, did he finish that in the end?


When he talked about it, it sounded to me like a slow long-term project. So 
I'm sure he's still in the middle of it.
May 17 2010
prev sibling next sibling parent Don <nospam nospam.com> writes:
Daniel Keep wrote:

 Some general replies:
 
 There have been three responses along the lines of "are you sure OPTLINK
 is running?".  Quoting myself:
 
 "Invoking OPTLINK directly changes nothing."
 
 Or to be more specific:
 

 link AstTest,AstTest,,user32+kernel32/noi+tango.lib;


 (zero-byte .map file pops into existence)
 
 Yes, I'm sure.  I'm also sure it's the DigitalMars link.exe and in the
 correct directory.
 

 I've had problems before with DMD silently exiting without an error
 message and with error success and leaving behind a corrupt .obj.


 
 I had a look at the .obj file (I actually used lib to stuff it into a
 .lib file and then dump the symbols), and it looked fine.
 
 Well, except for this:
 
 _D11TokenStream<80><8d><8d>5_ctorMFC6Source<f6>P<80><90><91>
   DFS8Location<80><89><89>AaYvJS6<80><85><cf>s5<f4>Zb<80>
   <99><aa>ZC<81><9a><87>
 
 (broken across 3 lines; <xx> are bytes in hex since several of them were
 non-printable.)
 
 The same had happened to a number of symbols from tango.io.stream.Format.
 
 In the case of the first one, the appropriate declarations (with
 fully-qualified module name commented) are:
 

 final class /*TokenStream.*/TokenStream
 {
     alias bool function(Source, LocErr, out Token) NextToken;
     this(Source src, NextToken next, LocErr err) { ... }
 }

 final class /*Source.*/Source { ... }

 alias void delegate(Location, char[], ...) /*Location.*/LocErr;

 struct /*Location.*/Location { ... }


 
 As far as I understand D's name mangling, that symbol name is corrupt.
 

 CTFE recursion is the only reported unfixed bug that does that ...


 
 There is exactly one CTFE function that calls itself.  However, it
 cannot recurse more than once (it does so to change types) *and* this
 function is used in the LexerTest program which does compile.
 
 AstTest uses an additional CTFE function, but this just concatenates
 some strings together.
 

 Try and remove stuff until it starts working


 
 I've gutted the changes between LexerTest and AstTest to the point that
 I get an .exe.  This involved stripping out more or less every global
 and member function that was added and every derived class.
 
 What tipped it over into compiling was removing the reference to
 StructuredOutput from AstDumpVisitor.  Which is odd, because
 StructuredOutput is still being used in AstTest.
 
 StructuredOutput, once gutted, is this:
 

 module StructuredOutput;

 import tango.io.model.IConduit : OutputStream;
 import tango.io.stream.Format : FormatOutput;

 final class StructuredOutput
 {
     alias StructuredOutput This;
     this(OutputStream os)
     {
     }
 }


 
 I honestly can't see what it could be upset about.
 

 Post the source and I'll try to help. I like debugging weird problems.
 :)


 
 Attached both regular and decaffeinated^Hgutted versions.


Thanks! I hope torhu is able to create a bug report from this.
May 13 2010
prev sibling parent reply torhu <no spam.invalid> writes:
On 13.05.2010 10:39, Daniel Keep wrote:

 Attached both regular and decaffeinated^Hgutted versions.


Most likely DMD turns VisitorCtfe.d into an invalid object file. But 
since you don't need to link with objects that contain only ctfe 
functions...

Here's what I get:
------
d:\prog\test\D\matheval>dmd -c AstTest.d Ast.d AstDumpVisitor.d Lexer.d 
Location.d Parser.d Source.d StructuredOutput.d Tokens.d TokenStream.d 
VisitorCtfe.d

d:\prog\test\D\matheval>dmd -ofAstTest.exe *.obj -v
binary    d:\prog\dmd\bin\dmd.exe
version   v1.060
config    d:\prog\dmd\bin\sc.ini
d:\prog\dmd\bin\..\..\dm\bin\link.exe 
"*","AstTest.exe",,user32+kernel32/noi;
OPTLINK (R) for Win32  Release 8.00.1
Copyright (C) Digital Mars 1989-2004  All rights reserved.
VisitorCtfe.obj(VisitorCtfe)  Offset 004D8H Record Type 009D
  Error 16: Index Range
--- errorlevel 1

d:\prog\test\D\matheval>del VisitorCtfe.obj

d:\prog\test\D\matheval>dmd -ofAstTest.exe *.obj -v
binary    d:\prog\dmd\bin\dmd.exe
version   v1.060
config    d:\prog\dmd\bin\sc.ini
d:\prog\dmd\bin\..\..\dm\bin\link.exe 
"*","AstTest.exe",,user32+kernel32/noi;

d:\prog\test\D\matheval>AstTest.exe
Can't initialize the TangoTrace LGPL stuff
Usage: AstTest.exe CODE

d:\prog\test\D\matheval>YAY
--------
May 13 2010
next sibling parent torhu <no spam.invalid> writes:
Seems I've got an older version of optlink, 8.00.1.  With 8.00.2 I get 
the same behavior you did, but it still works if you don't link with 
VisitorCtfe.obj.

When running 8.00.2 in the msvc debugger, it just looks like optlink 
successfully exits.  No crash or anything.  But no useful output either.

So I guess this is two bug reports, not just one...
May 13 2010
prev sibling parent reply torhu <no spam.invalid> writes:
On 13.05.2010 21:07, torhu wrote:

 On 13.05.2010 10:39, Daniel Keep wrote:

 Attached both regular and decaffeinated^Hgutted versions.


 Most likely DMD turns VisitorCtfe.d into an invalid object file. But
 since you don't need to link with objects that contain only ctfe
 functions...


http://d.puremagic.com/issues/show_bug.cgi?id=4315
Jun 14 2010
parent torhu <no spam.invalid> writes:
On 15.06.2010 01:34, torhu wrote:

 On 13.05.2010 21:07, torhu wrote:

 On 13.05.2010 10:39, Daniel Keep wrote:

 Attached both regular and decaffeinated^Hgutted versions.


 Most likely DMD turns VisitorCtfe.d into an invalid object file.
 But since you don't need to link with objects that contain only
 ctfe functions...


 http://d.puremagic.com/issues/show_bug.cgi?id=4315


Linker bug is http://d.puremagic.com/issues/show_bug.cgi?id=4324
Jun 15 2010
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"Daniel Keep" <daniel.keep.lists gmail.com> wrote in message 
news:hse30q$1tnt$1 digitalmars.com...

 That's right, it's time for everyone's favourite [1] game: guess why
 OPTLINK's not working! [2]

 *sigh*  I'm writing a math eval library.  There are two test
 applications.  LexerTest only touches part of the code.  AstTest touches
 everything.

 Now, the following works and creates an executable:

 dmd -ofLexerTest (appropriate .d files)

 So far, so good.  I get LexerTest.map, LexerTest.obj and LexerTest.exe.
 Let's try the other one...

 dmd -ofAstTest (more .d files)

 This creates a legitimate-looking AstTest.obj and a completely empty
 AstTest.map file.

 That's it.

 No executable, no error message, no register dump, nothing.

 Adding or removing -g, -debug, -unittest, -release, -inline, -O does
 nothing.  Changing the target filename does nothing useful.

 There are no spurious link.exe or dmd.exe processes running.  Invoking
 OPTLINK directly changes nothing.

 Does anyone have any idea, any idea at all, on what could be causing
 this?  I've tried everything myself and several others on #d could think 
 of.

 *miserable sob*

 After over five years of this sort of shit, I am so, so completely and
 utterly sick to death of OPTLINK.



 [1] I am, of course, being sarcastic.

 [2] It is, of course, entirely possible that it's not OPTLINK's fault.
 Frankly though, I find that hard to believe.



Disclaimer: There's probably a good chance that I'm completely off-base 
here....

Did you try deleting all the object files before trying to compile AstTest? 
I've often had linker problems (although in my case, they were always linker 
errors, not silence) when compiling two different targets that share code 
(or two configurations of the same target) without clearing the object files 
in-between. (Because of that, I've been in the habit of always setting up my 
builds so that each target/configuration combination has it's own separate 
directory for object files.)
May 17 2010