www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - Required to link windows header modules (?)

reply Sean Kelly <sean f4.ca> writes:
The standard C headers I've defined don't need to be linked against my 
app in any circumstances I've encountered so far--they serve as pure 
import modules.  However, I've recently discovered that this does not 
hold true for the Windows headers I use, which seem to require me to 
link against them if I reference any struct they declare.  Is it 
possible that extern (C) declarations tell the compiler that the struct 
will be available elsewhere and extern (Windows) doesn't?  It's worth 
noting that the link errors I get display a D-style mangled name for the 
struct definitions rather than a C-style name as I'd expect.  Should 
these structs all be declared as extern (C)?


Sean
Jun 08 2006
next sibling parent reply Walter Bright <newshound digitalmars.com> writes:
Sean Kelly wrote:

 The standard C headers I've defined don't need to be linked against my 
 app in any circumstances I've encountered so far--they serve as pure 
 import modules.  However, I've recently discovered that this does not 
 hold true for the Windows headers I use, which seem to require me to 
 link against them if I reference any struct they declare.  Is it 
 possible that extern (C) declarations tell the compiler that the struct 
 will be available elsewhere and extern (Windows) doesn't?  It's worth 
 noting that the link errors I get display a D-style mangled name for the 
 struct definitions rather than a C-style name as I'd expect.  Should 
 these structs all be declared as extern (C)?


What link errors are you getting?
Jun 08 2006
parent reply Sean Kelly <sean f4.ca> writes:
Walter Bright wrote:

 Sean Kelly wrote:

 The standard C headers I've defined don't need to be linked against my 
 app in any circumstances I've encountered so far--they serve as pure 
 import modules.  However, I've recently discovered that this does not 
 hold true for the Windows headers I use, which seem to require me to 
 link against them if I reference any struct they declare.  Is it 
 possible that extern (C) declarations tell the compiler that the 
 struct will be available elsewhere and extern (Windows) doesn't?  It's 
 worth noting that the link errors I get display a D-style mangled name 
 for the struct definitions rather than a C-style name as I'd expect.  
 Should these structs all be declared as extern (C)?


 
 What link errors are you getting?


For example, if I modify the Phobos makefile by removing any reference 
to std.c.windows.windows, then rebuilding Phobos gives me this at the 
end.  It's essentially the same as the issues I've been having:


Digital Mars Librarian Version 8.00n
Copyright (C) Digital Mars 2000-2002 All Rights Reserved www.digitalmars.com
Digital Mars Librarian complete.

\bin\dmd\bin\dmd unittest -g
C:\bin\dmd\bin\..\..\dm\bin\link.exe unittest,,,user32+kernel32/co/noi;
OPTLINK (R) for Win32  Release 7.50B1
Copyright (C) Digital Mars 1989 - 2001  All Rights Reserved

phobos.lib(date)
  Error 42: Symbol Undefined 
__init_3std1c7windows7windows21TIME_ZONE_INFORMATION

phobos.lib(file)
  Error 42: Symbol Undefined __init_3std1c7windows7windows16WIN32_FIND_DATAW
phobos.lib(file)
  Error 42: Symbol Undefined __init_3std1c7windows7windows15WIN32_FIND_DATA
phobos.lib(gcx)
  Error 42: Symbol Undefined __init_3std1c7windows7windows7CONTEXT
phobos.lib(syserror)
  Error 42: Symbol Undefined _MAKELANGID 8
--- errorlevel 5

--- errorlevel 5

C:\bin\dmd\src\phobos>
Jun 08 2006
parent reply Walter Bright <newshound digitalmars.com> writes:
Sean Kelly wrote:

 phobos.lib(date)
  Error 42: Symbol Undefined 
 __init_3std1c7windows7windows21TIME_ZONE_INFORMATION
 
 phobos.lib(file)
  Error 42: Symbol Undefined __init_3std1c7windows7windows16WIN32_FIND_DATAW
 phobos.lib(file)
  Error 42: Symbol Undefined __init_3std1c7windows7windows15WIN32_FIND_DATA
 phobos.lib(gcx)
  Error 42: Symbol Undefined __init_3std1c7windows7windows7CONTEXT
 phobos.lib(syserror)
  Error 42: Symbol Undefined _MAKELANGID 8


Ok, these are the static initializers for the struct, and a function for 
MAKELANGID. My next question is what's the issue with linking with 
windows.obj?
Jun 08 2006
parent reply Sean Kelly <sean f4.ca> writes:
Walter Bright wrote:

 Sean Kelly wrote:

 phobos.lib(date)
  Error 42: Symbol Undefined 
 __init_3std1c7windows7windows21TIME_ZONE_INFORMATION

 phobos.lib(file)
  Error 42: Symbol Undefined 
 __init_3std1c7windows7windows16WIN32_FIND_DATAW
 phobos.lib(file)
  Error 42: Symbol Undefined 
 __init_3std1c7windows7windows15WIN32_FIND_DATA
 phobos.lib(gcx)
  Error 42: Symbol Undefined __init_3std1c7windows7windows7CONTEXT
 phobos.lib(syserror)
  Error 42: Symbol Undefined _MAKELANGID 8


 
 Ok, these are the static initializers for the struct, and a function for 
 MAKELANGID. My next question is what's the issue with linking with 
 windows.obj?


Linking with the windows module present in Phobos isn't a big deal.  But 
the Windows header project Don has been working on is quite expansive, 
and simply importing windows.d ends up pulling in a substantial chunk of 
the windows modules.  This slows compilation noticeably and increases 
binary size significantly--simply importing Don's windows.d in my test 
app made the executable size increase from ~90K to over 150K.  This 
appears to be related at least in part to what appears to be a fixed 
minimum for importing a module using DMD of approximately 500 bytes, 
which I assume is the ModuleInfo data and similar things.

What I find confusing, however, is that I don't appear to need to link 
in the standard C modules I've defined in order to use the structs 
defined there.  I performed a quick test using struct tm and no 
dependency problems came up simply by compiling the test app itself and 
linking against a library without std.c.time in it.  Assuming I didn't 
mess up the test somehow, can you explain the difference between this 
situation and the Windows one?  If it's simply a matter of having the 
static initializers available, I'm fine with building everything, but 
the inconsistency doesn't seem to make sense.


Sean
Jun 08 2006
parent reply Walter Bright <newshound digitalmars.com> writes:
Sean Kelly wrote:

 Linking with the windows module present in Phobos isn't a big deal.  But 
 the Windows header project Don has been working on is quite expansive, 
 and simply importing windows.d ends up pulling in a substantial chunk of 
 the windows modules.  This slows compilation noticeably and increases 
 binary size significantly--simply importing Don's windows.d in my test 
 app made the executable size increase from ~90K to over 150K.  This 
 appears to be related at least in part to what appears to be a fixed 
 minimum for importing a module using DMD of approximately 500 bytes, 
 which I assume is the ModuleInfo data and similar things.


It's pretty easy to see what's in a .obj file by running obj2asm on it.


 What I find confusing, however, is that I don't appear to need to link 
 in the standard C modules I've defined in order to use the structs 
 defined there.  I performed a quick test using struct tm and no 
 dependency problems came up simply by compiling the test app itself and 
 linking against a library without std.c.time in it.  Assuming I didn't 
 mess up the test somehow, can you explain the difference between this 
 situation and the Windows one?  If it's simply a matter of having the 
 static initializers available, I'm fine with building everything, but 
 the inconsistency doesn't seem to make sense.


You shouldn't need to link in the static initializer for a struct if the 
static initializer is all 0's.
Jun 09 2006
parent reply Sean Kelly <sean f4.ca> writes:
Walter Bright wrote:

 Sean Kelly wrote:

 Linking with the windows module present in Phobos isn't a big deal.  
 But the Windows header project Don has been working on is quite 
 expansive, and simply importing windows.d ends up pulling in a 
 substantial chunk of the windows modules.  This slows compilation 
 noticeably and increases binary size significantly--simply importing 
 Don's windows.d in my test app made the executable size increase from 
 ~90K to over 150K.  This appears to be related at least in part to 
 what appears to be a fixed minimum for importing a module using DMD of 
 approximately 500 bytes, which I assume is the ModuleInfo data and 
 similar things.


 
 It's pretty easy to see what's in a .obj file by running obj2asm on it.


Done that.  And for the smaller modules, the bulk of this does indeed 
appear to be ModuleInfo data.


 What I find confusing, however, is that I don't appear to need to link 
 in the standard C modules I've defined in order to use the structs 
 defined there.  I performed a quick test using struct tm and no 
 dependency problems came up simply by compiling the test app itself 
 and linking against a library without std.c.time in it.  Assuming I 
 didn't mess up the test somehow, can you explain the difference 
 between this situation and the Windows one?  If it's simply a matter 
 of having the static initializers available, I'm fine with building 
 everything, but the inconsistency doesn't seem to make sense.


 
 You shouldn't need to link in the static initializer for a struct if the 
 static initializer is all 0's.


Ah, that explains the dependencies then, as I believe all of the structs 
I listed have char arrays of some sort in them.  I do still think it's a 
tad odd that the program size would increase by 50K simply for linking 
in the Windows dependencies for a "hello world" program though.  For 
reference, this is the "hello.d" sample from Mango I've been referencing:

http://svn.dsource.org/projects/mango/trunk/example/hello.d

But perhaps it is merely that the Windows headers are a vast 
interdependent mess and this won't be an issue elsewhere.

As a side note, this is also my first experience with using Derek's 
Build program and I have seen reports of it producing unusually large 
executables in the past.  Is it possible that optlink can be tricked to 
not always discard all unnecessary data?  I do find it strange that 
Build could compile an application using DMD and produce a different 
result than if the traditional library method were used.  To this end, 
it does seem possible that a program with more information (ie. the 
compiler) could do a better job at this.  I don't suppose you've 
considered a full compilation approach for DMD?  If dependent modules 
must be opened and parsed anyway, it seems a fairly trivial extension to 
simply compile them as well.  This would actually address a problem I 
noticed with Build in that it isn't able to detect if a module is a 
"header" module and will compile and link the code in anyway, which 
produces "previous definition different" errors at link time.


Sean
Jun 09 2006
parent reply Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Sean Kelly wrote:

  This would actually address a problem I 
 noticed with Build in that it isn't able to detect if a module is a 
 "header" module and will compile and link the code in anyway, which 
 produces "previous definition different" errors at link time.
 
 
 Sean


Hum, Have you used build exclusions (option -X) in order not to compile 
and link a module?

-- 
Bruno Medeiros - CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 11 2006
parent reply "Derek Parnell" <derek psych.ward> writes:
On Mon, 12 Jun 2006 00:31:42 +1000, Bruno Medeiros  
<brunodomedeirosATgmail SPAM.com> wrote:


 Sean Kelly wrote:

  This would actually address a problem I noticed with Build in that it  
 isn't able to detect if a module is a "header" module and will compile  
 and link the code in anyway, which produces "previous definition  
 different" errors at link time.
   Sean


 Hum, Have you used build exclusions (option -X) in order not to compile  
 and link a module?


I think that the best option for now is to use

   version(build) pragma(nolink);

This will ensure that the file containing this pragma is not compiled or  
linked.


-- 
Derek Parnell
Melbourne, Australia
Jun 11 2006
parent reply Sean Kelly <sean f4.ca> writes:
Derek Parnell wrote:

 On Mon, 12 Jun 2006 00:31:42 +1000, Bruno Medeiros 
 <brunodomedeirosATgmail SPAM.com> wrote:
 

 Sean Kelly wrote:

  This would actually address a problem I noticed with Build in that 
 it isn't able to detect if a module is a "header" module and will 
 compile and link the code in anyway, which produces "previous 
 definition different" errors at link time.
   Sean


 Hum, Have you used build exclusions (option -X) in order not to 
 compile and link a module?


 
 I think that the best option for now is to use
 
   version(build) pragma(nolink);
 
 This will ensure that the file containing this pragma is not compiled or 
 linked.


Yup.  This is what I've been doing.


Sean
Jun 11 2006
parent reply Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Sean Kelly wrote:

 Derek Parnell wrote:

 On Mon, 12 Jun 2006 00:31:42 +1000, Bruno Medeiros 
 <brunodomedeirosATgmail SPAM.com> wrote:


 Sean Kelly wrote:

  This would actually address a problem I noticed with Build in that 
 it isn't able to detect if a module is a "header" module and will 
 compile and link the code in anyway, which produces "previous 
 definition different" errors at link time.
   Sean


 Hum, Have you used build exclusions (option -X) in order not to 
 compile and link a module?


 I think that the best option for now is to use

   version(build) pragma(nolink);

 This will ensure that the file containing this pragma is not compiled 
 or linked.


 
 Yup.  This is what I've been doing.
 
 
 Sean


Then what you meant about Build "isn't able to detect if a module is a 
"header" module" ?


-- 
Bruno Medeiros - CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 11 2006
parent reply Sean Kelly <sean f4.ca> writes:
Bruno Medeiros wrote:

 Sean Kelly wrote:

 Derek Parnell wrote:

 On Mon, 12 Jun 2006 00:31:42 +1000, Bruno Medeiros 
 <brunodomedeirosATgmail SPAM.com> wrote:


 Sean Kelly wrote:

  This would actually address a problem I noticed with Build in that 
 it isn't able to detect if a module is a "header" module and will 
 compile and link the code in anyway, which produces "previous 
 definition different" errors at link time.
   Sean


 Hum, Have you used build exclusions (option -X) in order not to 
 compile and link a module?


 I think that the best option for now is to use

   version(build) pragma(nolink);

 This will ensure that the file containing this pragma is not compiled 
 or linked.


 Yup.  This is what I've been doing.


 
 Then what you meant about Build "isn't able to detect if a module is a 
 "header" module" ?


It isn't able to by itself--you have to tell it.  And while I'm not 
certain DMD could do better, I'd like to believe so.  That was my only 
point.  That in the long term it may be more useful to have such 
features built directly into the compiler.  In my brief experience with 
Build it's a darn sight better than generating C-style libraries.  And I 
suspect it will get around the template linking issues I've run into to 
boot.


Sean
Jun 11 2006
next sibling parent Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Sean Kelly wrote:

 Bruno Medeiros wrote:

 Then what you meant about Build "isn't able to detect if a module is a 
 "header" module" ?


 
 It isn't able to by itself--you have to tell it.  And while I'm not 
 certain DMD could do better, I'd like to believe so.  That was my only 
 point.  That in the long term it may be more useful to have such 
 features built directly into the compiler.  In my brief experience with 
 Build it's a darn sight better than generating C-style libraries.  And I 
 suspect it will get around the template linking issues I've run into to 
 boot.
 
 
 Sean


How could Build, or *any* tool at all, know by itself if a D file is to 
be compiled or not? I don't think it's possible, unless the file 
extension is ".di".

-- 
Bruno Medeiros - CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 11 2006
prev sibling parent reply "Derek Parnell" <derek psych.ward> writes:
On Mon, 12 Jun 2006 05:55:18 +1000, Sean Kelly <sean f4.ca> wrote:



  Then what you meant about Build "isn't able to detect if a module is a  
 "header" module" ?


 It isn't able to by itself--you have to tell it.  And while I'm not  
 certain DMD could do better, I'd like to believe so.


The problem as I see it is how any anyone or anything know that a module  
is a 'hesder' module. The only two clear cut signals are a module that has  
*zero* implementation details or the module is presented as a .di file.  
The first signal could be detectable by Build and the second one currently  
is. But that doesn't handle other 'header' modules that don't fit into  
these two categories. That is why I adopted the simple method of having  
the coder put in the pragma to identify a module as a 'header' type.

Can you give me the algorithm to use to detect that a module is a 'header'  
or not? Maybe if I had such an algorithm I might be able to implement it.
-- 
Derek Parnell
Melbourne, Australia
Jun 11 2006
parent Sean Kelly <sean f4.ca> writes:
Derek Parnell wrote:

 
 Can you give me the algorithm to use to detect that a module is a 
 'header' or not? Maybe if I had such an algorithm I might be able to 
 implement it.


It probably isn't possible to come up with a general solution.  But I 
imagined that if the compiler could determine that no object level code 
generation were required (ie. the module contained only type definitions 
and inlinable code) then it could be assumed to be a header.  But of 
course what's inlined varies from compiler to compiler, and then there's 
the issue of static ctors for structs and such.  Being aware of .di 
files as build is now is probably the best and easiest solution.


Sean
Jun 11 2006
prev sibling parent Sean Kelly <sean f4.ca> writes:
Sean Kelly wrote:

 The standard C headers I've defined don't need to be linked against my 
 app in any circumstances I've encountered so far--they serve as pure 
 import modules.  However, I've recently discovered that this does not 
 hold true for the Windows headers I use, which seem to require me to 
 link against them if I reference any struct they declare.  Is it 
 possible that extern (C) declarations tell the compiler that the struct 
 will be available elsewhere and extern (Windows) doesn't?  It's worth 
 noting that the link errors I get display a D-style mangled name for the 
 struct definitions rather than a C-style name as I'd expect.  Should 
 these structs all be declared as extern (C)?


By the way, I ask this because the executable size increases by a 
minimum of 500 bytes for every module included, even if they contain 
just declarations.  The full Windows header set being what it is (thanks 
to Don and crew's efforts), this means more than a 50K increase in 
program size just for a "hello world" application if Build actually 
links all the Windows headers referenced.


Sean
Jun 08 2006