digitalmars.D - Big problem with Small programs
- kris (11/11) Jan 23 2007 I have a helloworld.d program. I compile it on both Win32 and on linux.
- Walter Bright (3/18) Jan 23 2007 Try using enums instead of const variables, they don't take up any space...
- Sean Kelly (7/25) Jan 23 2007 That shaved off maybe 30k. I think the remaining 40k are mostly static
- Frits van Bommel (8/17) Jan 23 2007 I'm not sure about Windows, but I just tried it on Linux. It seems
- kris (25/47) Jan 23 2007 Thanks, but that makes only a dent in the overhead.
- kris (7/65) Jan 23 2007 For example (off the top of my head to get things rolling), what would
- Todor Totev (43/47) Jan 23 2007 Actually using enums instead of consts have a very nice side effect.
- kris (5/70) Jan 23 2007 Certainly, and I think you'll find many who agree, along with some who
- Todor Totev (1/6) Jan 23 2007
- Sean Kelly (5/64) Jan 23 2007 I agree that this is a great feature of D. And if you're willing to
- Don Clugston (4/72) Jan 24 2007 As far as possible, we did that, for the ones that actually are enums.
- Sean Kelly (10/28) Jan 23 2007 For reference, the "hello world" app test was compiled with Bud, which
- Sean Kelly (75/103) Jan 23 2007 I'm beginning to suspect a compiler/linker bug. Here is my first test
- Walter Bright (5/9) Jan 23 2007 I suggest linking with /MAP, which will generate a .map file listing all...
- kris (6/18) Jan 23 2007 Done that, Walter.
- Sean Kelly (82/92) Jan 23 2007 hello_small.map:
- Walter Bright (3/9) Jan 23 2007 Please be sure you are linking with the /MAP switch, as it will give all...
- Sean Kelly (14/24) Jan 23 2007 Oops, I missed the leading slash. The output is obviously quite large
- Walter Bright (6/10) Jan 23 2007 That means that, in the library based version, nothing in the explicitly...
- Sean Kelly (6/17) Jan 24 2007 If that's true then nothing in either version referenced them. The
- Sean Kelly (51/64) Jan 25 2007 Just some further discussion. After exhaustive experimentation
I have a helloworld.d program. I compile it on both Win32 and on linux. On linux, the executable is 72KB whereas on Win32 it is 141KB. Why is this? Turns out that D programs importing comprehensive Win32 D headers will wind up with executable space occupied for *all* constants and *all* struct initializers for the darned Win32 decls; where none of them are actually used. That amounts to ~70KB of junk in the executable -- almost a full 100% increase in size beyond what it should be. Does OptLink remove this? I've had no success with it. Walter? Can you help with this? - Kris
Jan 23 2007
kris wrote:I have a helloworld.d program. I compile it on both Win32 and on linux. On linux, the executable is 72KB whereas on Win32 it is 141KB. Why is this? Turns out that D programs importing comprehensive Win32 D headers will wind up with executable space occupied for *all* constants and *all* struct initializers for the darned Win32 decls; where none of them are actually used. That amounts to ~70KB of junk in the executable -- almost a full 100% increase in size beyond what it should be. Does OptLink remove this? I've had no success with it. Walter? Can you help with this? - KrisTry using enums instead of const variables, they don't take up any space in the object file.
Jan 23 2007
Walter Bright wrote:kris wrote:That shaved off maybe 30k. I think the remaining 40k are mostly static struct initializers, all of which are unused in the final app. As optlink is an optimizing linker, shouldn't it discard whatever is unused by the application? Or does that only apply to certain classes of information? SeanI have a helloworld.d program. I compile it on both Win32 and on linux. On linux, the executable is 72KB whereas on Win32 it is 141KB. Why is this? Turns out that D programs importing comprehensive Win32 D headers will wind up with executable space occupied for *all* constants and *all* struct initializers for the darned Win32 decls; where none of them are actually used. That amounts to ~70KB of junk in the executable -- almost a full 100% increase in size beyond what it should be. Does OptLink remove this? I've had no success with it. Walter? Can you help with this?Try using enums instead of const variables, they don't take up any space in the object file.
Jan 23 2007
Frits van Bommel <fvbommel REMwOVExCAPSs.nl> Sean Kelly wrote:Walter Bright wrote:I'm not sure about Windows, but I just tried it on Linux. It seems struct init data is stored in .rodata instead of some separate section[1]. That means if *any* read-only data from a module is referenced, all of its struct initializers are kept, presuming optlink works similarly to 'ld --gc-sections'. [1]: Like .gnu.linkonce.*, where TypeInfo init data is stored (as well as functions).Try using enums instead of const variables, they don't take up any space in the object file.That shaved off maybe 30k. I think the remaining 40k are mostly static struct initializers, all of which are unused in the final app. As optlink is an optimizing linker, shouldn't it discard whatever is unused by the application? Or does that only apply to certain classes of information?
Jan 23 2007
Walter Bright wrote:kris wrote:Thanks, but that makes only a dent in the overhead. One might assume that dmd and optlink have no facilities to remove unreferenced entities such as these? If so, doesn't this seem to be a notable weak point in the toolchain? Why is this not an issue with C/C++ ? 1) C used true compile-time constants via #define. From your comment above, it would appear enum is the equivalent in D, rather than const int. Probably worth a note in the doc about that? 2) C does not have struct inititalizers. Why has this not come to light before? Well, the Phobos Win32 headers are pretty darned slim compared with the real mcCoy. There's been a lot of clamour for 'real' win32 headers for years. Now that we have them, we're penalized for using 'em :p Surely there must be some way for the linker to identify and discard unused entities? Another approach would be to make the win32 headers be .di files, such that no obj files are generated. However, this breaks quickly whenever one uses a struct requiring an initializer (link errors). Enum is only a small partial resolution here, and we need to complete the picture somehow. Any other suggestions, Walter? Can you tag the structs internally somehow to make the linker discard the unused ones? Can you explain exactly what's going on, such that others might be able to suggest something? - KrisI have a helloworld.d program. I compile it on both Win32 and on linux. On linux, the executable is 72KB whereas on Win32 it is 141KB. Why is this? Turns out that D programs importing comprehensive Win32 D headers will wind up with executable space occupied for *all* constants and *all* struct initializers for the darned Win32 decls; where none of them are actually used. That amounts to ~70KB of junk in the executable -- almost a full 100% increase in size beyond what it should be. Does OptLink remove this? I've had no success with it. Walter? Can you help with this? - KrisTry using enums instead of const variables, they don't take up any space in the object file.
Jan 23 2007
kris wrote:Walter Bright wrote:For example (off the top of my head to get things rolling), what would be the ramifications of placing each struct initializer into its own data-segment? Would the linker manage to remove the unused ones then? If so, could a compiler flag be used to enable that, or could it be done behind the scenes? Other suggestions?kris wrote:Thanks, but that makes only a dent in the overhead. One might assume that dmd and optlink have no facilities to remove unreferenced entities such as these? If so, doesn't this seem to be a notable weak point in the toolchain? Why is this not an issue with C/C++ ? 1) C used true compile-time constants via #define. From your comment above, it would appear enum is the equivalent in D, rather than const int. Probably worth a note in the doc about that? 2) C does not have struct inititalizers. Why has this not come to light before? Well, the Phobos Win32 headers are pretty darned slim compared with the real mcCoy. There's been a lot of clamour for 'real' win32 headers for years. Now that we have them, we're penalized for using 'em :p Surely there must be some way for the linker to identify and discard unused entities? Another approach would be to make the win32 headers be .di files, such that no obj files are generated. However, this breaks quickly whenever one uses a struct requiring an initializer (link errors). Enum is only a small partial resolution here, and we need to complete the picture somehow. Any other suggestions, Walter? Can you tag the structs internally somehow to make the linker discard the unused ones? Can you explain exactly what's going on, such that others might be able to suggest something?I have a helloworld.d program. I compile it on both Win32 and on linux. On linux, the executable is 72KB whereas on Win32 it is 141KB. Why is this? Turns out that D programs importing comprehensive Win32 D headers will wind up with executable space occupied for *all* constants and *all* struct initializers for the darned Win32 decls; where none of them are actually used. That amounts to ~70KB of junk in the executable -- almost a full 100% increase in size beyond what it should be. Does OptLink remove this? I've had no success with it. Walter? Can you help with this? - KrisTry using enums instead of const variables, they don't take up any space in the object file.
Jan 23 2007
Walter Bright wrote:=Try using enums instead of const variables, they don't take up any =Actually using enums instead of consts have a very nice side effect. Consider this declaration from win32: HANDLE CreateFileW(LPCWSTR, DWORD, DWORD, LPSECURITY_ATTRIBUTES, DWORD, = = DWORD, HANDLE); Now, which invocation is correct? CreateFile("a", OPEN_EXISTING, FILE_SHARE_READ, null, GENERIC_READ ...) or CreateFile("a", OPEN_EXISTING, GENERIC_READ, null, FILE_SHARE_READ ...) or even CreateFile("a", GENERIC_READ, OPEN_EXISTING, null, FILE_SHARE_READ ...)?= The compiler happily accepts all of them. Using enums, we have: enum FILE_SHARE { READ =3D 1, WRITE =3D 2, BOTH =3D READ | WRITE } enum DISPOSITION { CREATE_ALWAYS =3D 1, OPEN_EXISTING =3D 2 } void CreateFile(char[] fileName, DISPOSITION disposition, FILE_SHARE = share) {} int main() { // compiler allows unknown flags CreateFile("filename", cast(DISPOSITION)4, FILE_SHARE.READ); // when we swap the arguments by mistake CreateFile("filename", FILE_SHARE.READ, DISPOSITION.CREATE_ALWAYS); return 0; } This way if I make an error the compiler will very helpfully tell me what exactly is happening. Consider: CreateFont has 14 parameters, if the IDE does not help me i'd make an error when I use it, trust me On the other hand, when a new version of Windows come with expanded opti= ons I can just use the new numbers without breaking anything while the d = package is updated. The dotnet framework uses enums and I really like the idea. Regards, Todorspace in the object file.Thanks, but that makes only a dent in the overhead.
Jan 23 2007
Todor Totev wrote:Certainly, and I think you'll find many who agree, along with some who can't bear to type a few extra characters :) But that's a different type of issue than the one we're facing. I wish you had a fix for that one :pWalter Bright wrote:Actually using enums instead of consts have a very nice side effect. Consider this declaration from win32: HANDLE CreateFileW(LPCWSTR, DWORD, DWORD, LPSECURITY_ATTRIBUTES, DWORD, DWORD, HANDLE); Now, which invocation is correct? CreateFile("a", OPEN_EXISTING, FILE_SHARE_READ, null, GENERIC_READ ...) or CreateFile("a", OPEN_EXISTING, GENERIC_READ, null, FILE_SHARE_READ ...) or even CreateFile("a", GENERIC_READ, OPEN_EXISTING, null, FILE_SHARE_READ ...)? The compiler happily accepts all of them. Using enums, we have: enum FILE_SHARE { READ = 1, WRITE = 2, BOTH = READ | WRITE } enum DISPOSITION { CREATE_ALWAYS = 1, OPEN_EXISTING = 2 } void CreateFile(char[] fileName, DISPOSITION disposition, FILE_SHARE share) {} int main() { // compiler allows unknown flags CreateFile("filename", cast(DISPOSITION)4, FILE_SHARE.READ); // when we swap the arguments by mistake CreateFile("filename", FILE_SHARE.READ, DISPOSITION.CREATE_ALWAYS); return 0; } This way if I make an error the compiler will very helpfully tell me what exactly is happening. Consider: CreateFont has 14 parameters, if the IDE does not help me i'd make an error when I use it, trust me On the other hand, when a new version of Windows come with expanded options I can just use the new numbers without breaking anything while the d package is updated. The dotnet framework uses enums and I really like the idea. Regards, TodorTry using enums instead of const variables, they don't take up any space in the object file.Thanks, but that makes only a dent in the overhead.
Jan 23 2007
Certainly, and I think you'll find many who agree, along with some who can't bear to type a few extra characters :)YepBut that's a different type of issue than the one we're facing. I wish you had a fix for that one :p No, nada, nein, не, όχι :-(
Jan 23 2007
Todor Totev wrote:I agree that this is a great feature of D. And if you're willing to adapt the Win32 headers to use this convention then I would gladly accept them :-) SeanWalter Bright wrote:Actually using enums instead of consts have a very nice side effect. Consider this declaration from win32: HANDLE CreateFileW(LPCWSTR, DWORD, DWORD, LPSECURITY_ATTRIBUTES, DWORD, DWORD, HANDLE); Now, which invocation is correct? CreateFile("a", OPEN_EXISTING, FILE_SHARE_READ, null, GENERIC_READ ...) or CreateFile("a", OPEN_EXISTING, GENERIC_READ, null, FILE_SHARE_READ ...) or even CreateFile("a", GENERIC_READ, OPEN_EXISTING, null, FILE_SHARE_READ ...)? The compiler happily accepts all of them. Using enums, we have: enum FILE_SHARE { READ = 1, WRITE = 2, BOTH = READ | WRITE } enum DISPOSITION { CREATE_ALWAYS = 1, OPEN_EXISTING = 2 } void CreateFile(char[] fileName, DISPOSITION disposition, FILE_SHARE share) {} int main() { // compiler allows unknown flags CreateFile("filename", cast(DISPOSITION)4, FILE_SHARE.READ); // when we swap the arguments by mistake CreateFile("filename", FILE_SHARE.READ, DISPOSITION.CREATE_ALWAYS); return 0; } This way if I make an error the compiler will very helpfully tell me what exactly is happening. Consider: CreateFont has 14 parameters, if the IDE does not help me i'd make an error when I use it, trust me On the other hand, when a new version of Windows come with expanded options I can just use the new numbers without breaking anything while the d package is updated. The dotnet framework uses enums and I really like the idea.Try using enums instead of const variables, they don't take up any space in the object file.Thanks, but that makes only a dent in the overhead.
Jan 23 2007
Sean Kelly wrote:Todor Totev wrote:As far as possible, we did that, for the ones that actually are enums. The problem is, most of those things aren't actually enums! They are complicated bitfields that get ORed together.I agree that this is a great feature of D. And if you're willing to adapt the Win32 headers to use this convention then I would gladly accept them :-) SeanWalter Bright wrote:Actually using enums instead of consts have a very nice side effect. Consider this declaration from win32: HANDLE CreateFileW(LPCWSTR, DWORD, DWORD, LPSECURITY_ATTRIBUTES, DWORD, DWORD, HANDLE); Now, which invocation is correct? CreateFile("a", OPEN_EXISTING, FILE_SHARE_READ, null, GENERIC_READ ...) or CreateFile("a", OPEN_EXISTING, GENERIC_READ, null, FILE_SHARE_READ ...) or even CreateFile("a", GENERIC_READ, OPEN_EXISTING, null, FILE_SHARE_READ ...)? The compiler happily accepts all of them. Using enums, we have: enum FILE_SHARE { READ = 1, WRITE = 2, BOTH = READ | WRITE } enum DISPOSITION { CREATE_ALWAYS = 1, OPEN_EXISTING = 2 } void CreateFile(char[] fileName, DISPOSITION disposition, FILE_SHARE share) {} int main() { // compiler allows unknown flags CreateFile("filename", cast(DISPOSITION)4, FILE_SHARE.READ); // when we swap the arguments by mistake CreateFile("filename", FILE_SHARE.READ, DISPOSITION.CREATE_ALWAYS); return 0; } This way if I make an error the compiler will very helpfully tell me what exactly is happening. Consider: CreateFont has 14 parameters, if the IDE does not help me i'd make an error when I use it, trust me On the other hand, when a new version of Windows come with expanded options I can just use the new numbers without breaking anything while the d package is updated. The dotnet framework uses enums and I really like the idea.Try using enums instead of const variables, they don't take up any space in the object file.Thanks, but that makes only a dent in the overhead.
Jan 24 2007
Walter Bright wrote:kris wrote:For reference, the "hello world" app test was compiled with Bud, which basically just recursively compiled all .d files and links the objects together. However, I just tried creating a library out of the Win32 modules and renamed them to .di files to keep Bud from building them, and the resulting app was 90k instead of 144k. I don't suppose you can explain how linking a lib and linking individual object files can have such glaringly different effects? I'm going to try a manual build without Bud and see if the app is 144k now too. SeanI have a helloworld.d program. I compile it on both Win32 and on linux. On linux, the executable is 72KB whereas on Win32 it is 141KB. Why is this? Turns out that D programs importing comprehensive Win32 D headers will wind up with executable space occupied for *all* constants and *all* struct initializers for the darned Win32 decls; where none of them are actually used. That amounts to ~70KB of junk in the executable -- almost a full 100% increase in size beyond what it should be. Does OptLink remove this? I've had no success with it. Walter? Can you help with this?Try using enums instead of const variables, they don't take up any space in the object file.
Jan 23 2007
Sean Kelly wrote:Walter Bright wrote:I'm beginning to suspect a compiler/linker bug. Here is my first test supplying all modules on one line: -------------------------------------------------------------------------------- C:\code\src\d\test\appsize>dmd -inline -release -O hello.d \code\import\tango\tango\io\Console.d \code\import\tango\tango\sys\Common.d \code\import\tango\tango\sys\win32\Types.d \code\import\tango\tango\sys\win32\Common.d \code\import\tango\tango\io\Buffer.d \code\import\tango\tango\io\Exception.d \code\import\tango\tango\io\model\IBuffer.d \code\import\tango\tango\io\model\IConduit.d \code\import\tango\tango\io\DeviceConduit.d \code\import\tango\tango\io\Conduit.d c:\bin\dmd\bin\..\..\dm\bin\link.exe hello+Console+Common+Types+Common+Buffer+Ex ception+IBuffer+IConduit+DeviceConduit+Conduit,,,user32+kernel32/noi; C:\code\src\d\test>dir hello.exe Volume in drive C is System Volume Serial Number is 58B0-B8C5 Directory of C:\code\src\d\test\appsize 01/23/2007 12:05 PM 141,852 hello.exe -------------------------------------------------------------------------------- And my second test compiling all modules separately and then linking into an app: -------------------------------------------------------------------------------- dmd -inline -release -O -c hello.d dmd -inline -release -O -c \code\import\tango\tango\io\Console.d dmd -inline -release -O -c \code\import\tango\tango\sys\Common.d dmd -inline -release -O -c \code\import\tango\tango\sys\win32\Types.d dmd -inline -release -O -c \code\import\tango\tango\sys\win32\Common.d -ofCommonWin32.obj dmd -inline -release -O -c \code\import\tango\tango\io\Buffer.d dmd -inline -release -O -c \code\import\tango\tango\io\Exception.d dmd -inline -release -O -c \code\import\tango\tango\io\model\IBuffer.d dmd -inline -release -O -c \code\import\tango\tango\io\model\IConduit.d dmd -inline -release -O -c \code\import\tango\tango\io\DeviceConduit.d dmd -inline -release -O -c \code\import\tango\tango\io\Conduit.d C:\code\src\d\test\appsize>dmd hello *.obj c:\bin\dmd\bin\..\..\dm\bin\link.exe hello+*,,,user32+kernel32/noi; C:\code\src\d\test\appsize>dir hello.exe Volume in drive C is System Volume Serial Number is 58B0-B8C5 Directory of C:\code\src\d\test\appsize 01/23/2007 12:13 PM 141,340 hello.exe -------------------------------------------------------------------------------- And now my third test where tango/sys/win32/*.* is compiled into a library (win32.lib) and the lib is linked instead of the module object files: -------------------------------------------------------------------------------- C:\code\src\d\test\appsize>lib -c -n Win32.lib CommonWin32.obj Types.obj Digital Mars Librarian Version 8.00n Copyright (C) Digital Mars 2000-2002 All Rights Reserved www.digitalmars.com Digital Mars Librarian complete. C:\code\src\d\test\appsize>del CommonWin32.obj Types.obj C:\code\src\d\test\appsize>dmd hello *.obj Win32.lib c:\bin\dmd\bin\..\..\dm\bin\link.exe hello+*,,,Win32.lib+user32+kernel32/noi; C:\code\src\d\test\appsize>dir hello.exe Volume in drive C is System Volume Serial Number is 58B0-B8C5 Directory of C:\code\src\d\test\appsize 01/23/2007 12:20 PM 90,140 hello.exe -------------------------------------------------------------------------------- As you can see, simply putting the Win32 object files into a library prior to linking reduced the application size by 51,100 bytes, but separate compilation had the same (bad) result as compiling all modules on one line. I grant that 51k may not be a huge issue for the average Windows app, but it could be a killer for handheld (WinCE) applications. Could you please explain the size difference if this is indeed not a bug? I don't mind making these files .di files and adding a pragma(lib, "Win32.lib"), but this seems like something that must be addressed in a more general manner at some point. Seankris wrote:For reference, the "hello world" app test was compiled with Bud, which basically just recursively compiled all .d files and links the objects together. However, I just tried creating a library out of the Win32 modules and renamed them to .di files to keep Bud from building them, and the resulting app was 90k instead of 144k. I don't suppose you can explain how linking a lib and linking individual object files can have such glaringly different effects? I'm going to try a manual build without Bud and see if the app is 144k now too.I have a helloworld.d program. I compile it on both Win32 and on linux. On linux, the executable is 72KB whereas on Win32 it is 141KB. Why is this? Turns out that D programs importing comprehensive Win32 D headers will wind up with executable space occupied for *all* constants and *all* struct initializers for the darned Win32 decls; where none of them are actually used. That amounts to ~70KB of junk in the executable -- almost a full 100% increase in size beyond what it should be. Does OptLink remove this? I've had no success with it. Walter? Can you help with this?Try using enums instead of const variables, they don't take up any space in the object file.
Jan 23 2007
Sean Kelly wrote:As you can see, simply putting the Win32 object files into a library prior to linking reduced the application size by 51,100 bytes, but separate compilation had the same (bad) result as compiling all modules on one line.I suggest linking with /MAP, which will generate a .map file listing all the modules linked in, and all the global symbols linked in. It's a lot easier to see what's happening with that than guessing based on the file size.
Jan 23 2007
Walter Bright wrote:Sean Kelly wrote:Done that, Walter. The difference is entirey Win32 struct inits, consts and so on. Remove the int consts, and you still have the struct inits ... almost enough to hand out at Christmas :) What shall we do next?As you can see, simply putting the Win32 object files into a library prior to linking reduced the application size by 51,100 bytes, but separate compilation had the same (bad) result as compiling all modules on one line.I suggest linking with /MAP, which will generate a .map file listing all the modules linked in, and all the global symbols linked in. It's a lot easier to see what's happening with that than guessing based on the file size.
Jan 23 2007
Walter Bright wrote:Sean Kelly wrote:hello_small.map: Start Length Name Class 0002:00000000 00011570H _TEXT CODE 32-bit 0002:00011570 00000198H ICODE ICODE 32-bit 0003:00000000 00000004H .CRT$XIA DATA 32-bit 0003:00000010 00000004H .CRT$XIZ DATA 32-bit 0003:00000020 00000004H .CRT$XCA DATA 32-bit 0003:00000030 00000004H .CRT$XCZ DATA 32-bit 0003:00000040 00000004H .CRT$XPA DATA 32-bit 0003:00000050 00000004H .CRT$XPZ DATA 32-bit 0003:00000060 00000004H .CRT$XTA DATA 32-bit 0003:00000070 00000004H .CRT$XTZ DATA 32-bit 0003:00000074 00000000H IMP__DATA IMP__DATA 32-bit 0003:00000080 00005100H _DATA DATA 32-bit 0003:00005180 00000000H FMB DATA 32-bit 0003:00005180 00000024H FM DATA 32-bit 0003:000051A4 00000000H FME DATA 32-bit 0003:000051A4 00000000H XIB DATA 32-bit 0003:000051A4 0000001CH XI DATA 32-bit 0003:000051C0 00000000H XIE DATA 32-bit 0003:000051C0 00000000H XCB DATA 32-bit 0003:000051C0 00000014H XC DATA 32-bit 0003:000051D4 00000000H XCE DATA 32-bit 0003:000051D4 00000000H XIFCB DATA 32-bit 0003:000051D4 00000004H XIFU DATA 32-bit 0003:000051D8 00000000H XIFL DATA 32-bit 0003:000051D8 00000004H XIFM DATA 32-bit 0003:000051DC 00000000H XIFCE DATA 32-bit 0003:000051E0 00000000H CONST CONST 32-bit 0003:000051E0 00000000H EEND ENDBSS 32-bit 0003:000051E0 000016ECH _BSS BSS 32-bit 0003:000068CC 00000000H XOB BSS 32-bit 0003:000068CC 00000004H XO BSS 32-bit 0003:000068D0 00000000H XOE BSS 32-bit 0003:000068D0 00000000H XOFB BSS 32-bit 0003:000068D0 00000108H XOF BSS 32-bit 0003:000069D8 00000000H XOFE BSS 32-bit 0003:000069E0 00000419H c_common BSS 32-bit 0003:00006E00 00000000H STACK STACK 32-bit Program entry point at 0000A2AC -------------------------------------------------------------------------------- hello_large.map: Start Length Name Class 0002:00000000 00011570H _TEXT CODE 32-bit 0002:00011570 00000198H ICODE ICODE 32-bit 0003:00000000 00000004H .CRT$XIA DATA 32-bit 0003:00000010 00000004H .CRT$XIZ DATA 32-bit 0003:00000020 00000004H .CRT$XCA DATA 32-bit 0003:00000030 00000004H .CRT$XCZ DATA 32-bit 0003:00000040 00000004H .CRT$XPA DATA 32-bit 0003:00000050 00000004H .CRT$XPZ DATA 32-bit 0003:00000060 00000004H .CRT$XTA DATA 32-bit 0003:00000070 00000004H .CRT$XTZ DATA 32-bit 0003:00000074 00000000H IMP__DATA IMP__DATA 32-bit 0003:00000080 00011660H _DATA DATA 32-bit 0003:000116E0 00000000H FMB DATA 32-bit 0003:000116E0 00000024H FM DATA 32-bit 0003:00011704 00000000H FME DATA 32-bit 0003:00011704 00000000H XIB DATA 32-bit 0003:00011704 0000001CH XI DATA 32-bit 0003:00011720 00000000H XIE DATA 32-bit 0003:00011720 00000000H XCB DATA 32-bit 0003:00011720 00000014H XC DATA 32-bit 0003:00011734 00000000H XCE DATA 32-bit 0003:00011734 00000000H XIFCB DATA 32-bit 0003:00011734 00000004H XIFU DATA 32-bit 0003:00011738 00000000H XIFL DATA 32-bit 0003:00011738 00000004H XIFM DATA 32-bit 0003:0001173C 00000000H XIFCE DATA 32-bit 0003:00011740 00000000H CONST CONST 32-bit 0003:00011740 00000000H EEND ENDBSS 32-bit 0003:00011740 0000499CH _BSS BSS 32-bit 0003:000160DC 00000000H XOB BSS 32-bit 0003:000160DC 00000004H XO BSS 32-bit 0003:000160E0 00000000H XOE BSS 32-bit 0003:000160E0 00000000H XOFB BSS 32-bit 0003:000160E0 00000108H XOF BSS 32-bit 0003:000161E8 00000000H XOFE BSS 32-bit 0003:000161F0 00000419H c_common BSS 32-bit 0003:00016610 00000000H STACK STACK 32-bit Program entry point at 0000A2ACAs you can see, simply putting the Win32 object files into a library prior to linking reduced the application size by 51,100 bytes, but separate compilation had the same (bad) result as compiling all modules on one line.I suggest linking with /MAP, which will generate a .map file listing all the modules linked in, and all the global symbols linked in. It's a lot easier to see what's happening with that than guessing based on the file size.
Jan 23 2007
Sean Kelly wrote:Please be sure you are linking with the /MAP switch, as it will give all the symbols, not just the segments.I suggest linking with /MAP, which will generate a .map file listing all the modules linked in, and all the global symbols linked in. It's a lot easier to see what's happening with that than guessing based on the file size.hello_small.map:
Jan 23 2007
Walter Bright wrote:Sean Kelly wrote:Oops, I missed the leading slash. The output is obviously quite large so I'm just going to note some of the larger blocks that are present in hello_large.map that are missing from hello_small.map: 0003:0001172C _D5tango3sys5win325Types10ACE_HEADER6__initZ 0042372C ... 0003:00002C08 _D5tango3sys5win325Types9cSTARTDOCi 00414C08 0003:00001074 _D5tango3sys5win325Types10RASCS_DONEi 00413074 ... 0003:00014604 _D5tango3sys5win325Types10WINDOWINFO6__initZ 00426604 The first block goes from line 410 to line 6719 in the map file, and the second block goes from line 8047 to line 14559 in the map file. All told, that's 12821 separate constants or initializers in the manually linked version that seem absent from the library-based version.Please be sure you are linking with the /MAP switch, as it will give all the symbols, not just the segments.I suggest linking with /MAP, which will generate a .map file listing all the modules linked in, and all the global symbols linked in. It's a lot easier to see what's happening with that than guessing based on the file size.hello_small.map:
Jan 23 2007
Sean Kelly wrote:The first block goes from line 410 to line 6719 in the map file, and the second block goes from line 8047 to line 14559 in the map file. All told, that's 12821 separate constants or initializers in the manually linked version that seem absent from the library-based version.That means that, in the library based version, nothing in the explicitly linked .obj files referenced them. If you want to find out what reference is pulling in a particular library module, use lib to remove that library module, relink, and note the undefined reference.
Jan 23 2007
Walter Bright wrote:Sean Kelly wrote:If that's true then nothing in either version referenced them. The application is identical in both cases but for some of the objects living in a library in the small case.The first block goes from line 410 to line 6719 in the map file, and the second block goes from line 8047 to line 14559 in the map file. All told, that's 12821 separate constants or initializers in the manually linked version that seem absent from the library-based version.That means that, in the library based version, nothing in the explicitly linked .obj files referenced them.If you want to find out what reference is pulling in a particular library module, use lib to remove that library module, relink, and note the undefined reference.See above. Sean
Jan 24 2007
Sean Kelly wrote:Walter Bright wrote:Just some further discussion. After exhaustive experimentation yesterday with Tango the following seems to be true: Using the Bud/Build approach of compiling all modules to object files and then linking them always works, however on Win32 it may result in a surplus of data in the executable. This may actually be legitimate behavior based on how the linker works and how the OMF blocks are segmented in the object files at link time, but it seems odd. Using the library approach of compiling all library code into a static library and then linking it does not always work, but it results in much leaner executables on Win32. Little or no surplus data is added to the executable, including data that is added using the Bud/Build method, even for the exact same application. This may again be legitimate behavior based on how the lib tool splits/joins/moves OMF blocks when assembling the library (assuming it does so), or it may have something to do with the linker having "all the information at its fingertips" when assembling the application. Where the library approach fails is if the library contains template code. Results are inconsistent, but yesterday we were seeing link errors in some cases and GPFs in others just for different library modules using the same template code, all of which work perfectly using the Bud/Build method. For the link errors, as near as we could tell the object files contained all the relevant functions (many of the link errors referred to class TypeInfo instances) but for some reason they didn't appear to be available at link time. It is possible that the lib tool is throwing them out for some reason, but I could not speculate on why. This leaves us in an awkward situation for releasing Tango. Suggesting everyone use Bud/Build seems the most reliable but some may not be happy with large executables or with having to use Bud/Build at all. Assembling everything into a single static library seems unreliable at best, but it will undoubtedly be preferred by many people. At the moment, I suspect we will adopt a hybrid approach out of necessity, placing some of the larger packages (like the Win32 package) in static libraries and linking them either explicitly or implicitly via pragma(lib) (which is problematic in itself since it doesn't work if placed in include files, at least with DMD--Bud may be different, I have yet to test this and plan to do so today). I am hoping that we can produce a small test case for the template issues so we can move to a full library approach once the problems have been addressed. Alternately, perhaps we can find a way to address linking so that less unnecessary data is included while linking objects directly. I'll admit that at this point I'm of half a mind to simply write an OMF linker that takes as long as necessary to produce a working application and be done with it. Out of curiosity, I suppose it would be a substantial amount of work for DMD (and I suppose DMC) to produce COFF object files instead of OMF? And I don't suppose that some work in this area may be necessary for generating 64-bit executables? If there's a chance we could move to a newer object file format for Win32, we would at least gain access to a broader array of tools, even if COFF itself is a bit problematic. SeanSean Kelly wrote:If that's true then nothing in either version referenced them. The application is identical in both cases but for some of the objects living in a library in the small case.The first block goes from line 410 to line 6719 in the map file, and the second block goes from line 8047 to line 14559 in the map file. All told, that's 12821 separate constants or initializers in the manually linked version that seem absent from the library-based version.That means that, in the library based version, nothing in the explicitly linked .obj files referenced them.
Jan 25 2007