www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - shared library size

reply boolangery <eliott.dumeix gmail.com> writes:
Hi,

Why shared library size is so big ?

I tried to build https://github.com/etcimon/libasync as a shared 
library:


build/libasync.so: ELF 64-bit LSB shared object, x86-64, version 
1 (SYSV), dynamically linked, 
BuildID[sha1]=4df6b6d12da6d6cf2108178a14b61fc9453a8e9e, stripped

2,6M	build/libasync.so

ldd build/libasync.so
	linux-vdso.so.1 (0x00007ffd65d54000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f5181e94000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f5181c7d000)
	libphobos2-ldc.so.74 => /lib64/libphobos2-ldc.so.74 
(0x00007f51815b3000)
	libdruntime-ldc.so.74 => /lib64/libdruntime-ldc.so.74 
(0x00007f51812a3000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f518109f000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5180e81000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f5180b36000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f5180780000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f5182527000)
	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f5180569000)

Its just 16K lines of codes, why 2.6M ? Do you have tips for 
reduce this size ?

Thanks
Jul 02 2018
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 2 July 2018 at 09:35:53 UTC, boolangery wrote:
 Hi,

 Why shared library size is so big ?

 I tried to build https://github.com/etcimon/libasync as a 
 shared library:


 build/libasync.so: ELF 64-bit LSB shared object, x86-64, 
 version 1 (SYSV), dynamically linked, 
 BuildID[sha1]=4df6b6d12da6d6cf2108178a14b61fc9453a8e9e, stripped

 2,6M	build/libasync.so

 ldd build/libasync.so
 	linux-vdso.so.1 (0x00007ffd65d54000)
 	librt.so.1 => /lib64/librt.so.1 (0x00007f5181e94000)
 	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f5181c7d000)
 	libphobos2-ldc.so.74 => /lib64/libphobos2-ldc.so.74 
 (0x00007f51815b3000)
 	libdruntime-ldc.so.74 => /lib64/libdruntime-ldc.so.74 
 (0x00007f51812a3000)
 	libdl.so.2 => /lib64/libdl.so.2 (0x00007f518109f000)
 	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5180e81000)
 	libm.so.6 => /lib64/libm.so.6 (0x00007f5180b36000)
 	libc.so.6 => /lib64/libc.so.6 (0x00007f5180780000)
 	/lib64/ld-linux-x86-64.so.2 (0x00007f5182527000)
 	libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f5180569000)

 Its just 16K lines of codes, why 2.6M ? Do you have tips for 
 reduce this size ?

 Thanks
Hello, On POSIX by default, all the symbols in a shared library are exported. That makes a lot of symbols that can't be stripped automatically by the linker! And it won't change anytime soon (https://github.com/ldc-developers/ldc/issues/2431). The one option you have with LDC to have smaller shared libraries is to use: $ ldc2 -exported_symbols_list list-of-symbols.lst -dead_strip <rest-of-cmdline> Good luck!
Jul 02 2018
parent reply boolangery <eliott.dumeix gmail.com> writes:
On Monday, 2 July 2018 at 12:21:04 UTC, Guillaume Piolat wrote:
 On POSIX by default, all the symbols in a shared library are 
 exported. That makes a lot of symbols that can't be stripped 
 automatically by the linker!

 And it won't change anytime soon 
 (https://github.com/ldc-developers/ldc/issues/2431).

 The one option you have with LDC to have smaller shared 
 libraries is to use:

 $ ldc2 -exported_symbols_list list-of-symbols.lst -dead_strip 
 <rest-of-cmdline>

 Good luck!
hmmm doesn't seem to work for me ldc2: Unknown command line argument '-exported_symbols_list'. Try: 'ldc2 -help' ldc2: Did you mean '-import-instr-limit'? ldc2: Unknown command line argument '-dead_strip'. Try: 'ldc2 -help' ldc2: Did you mean '-defaultlib'?
Jul 02 2018
parent reply kinke <kinke libero.it> writes:
On Monday, 2 July 2018 at 13:06:12 UTC, boolangery wrote:
 On Monday, 2 July 2018 at 12:21:04 UTC, Guillaume Piolat wrote:
 On POSIX by default, all the symbols in a shared library are 
 exported. That makes a lot of symbols that can't be stripped 
 automatically by the linker!

 And it won't change anytime soon 
 (https://github.com/ldc-developers/ldc/issues/2431).

 The one option you have with LDC to have smaller shared 
 libraries is to use:

 $ ldc2 -exported_symbols_list list-of-symbols.lst -dead_strip 
 <rest-of-cmdline>

 Good luck!
hmmm doesn't seem to work for me ldc2: Unknown command line argument '-exported_symbols_list'. Try: 'ldc2 -help' ldc2: Did you mean '-import-instr-limit'? ldc2: Unknown command line argument '-dead_strip'. Try: 'ldc2 -help' ldc2: Did you mean '-defaultlib'?
Those are linker (ld) flags, so use the `-L` prefix with LDC: `-L-exported_symbols_list -L<file.lst> -L-dead_strip`.
Jul 02 2018
parent reply boolangery <eliott.dumeix gmail.com> writes:
On Monday, 2 July 2018 at 15:56:56 UTC, kinke wrote:
 Those are linker (ld) flags, so use the `-L` prefix with LDC: 
 `-L-exported_symbols_list -L<file.lst> -L-dead_strip`.
Thanks you both, is there a way to automate symbol list creation with D public symbols ?
Jul 02 2018
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Tuesday, 3 July 2018 at 06:39:58 UTC, boolangery wrote:
 On Monday, 2 July 2018 at 15:56:56 UTC, kinke wrote:
 Those are linker (ld) flags, so use the `-L` prefix with LDC: 
 `-L-exported_symbols_list -L<file.lst> -L-dead_strip`.
Thanks you both, is there a way to automate symbol list creation with D public symbols ?
No. Each of our products has 3 of these symbol lists for POSIX, whereas for Windows you don't need anything. That's precisely why https://github.com/ldc-developers/ldc/issues/2431 would be helpful.
Jul 03 2018
parent reply boolangery <eliott.dumeix gmail.com> writes:
On Tuesday, 3 July 2018 at 10:38:01 UTC, Guillaume Piolat wrote:
 No.
 Each of our products has 3 of these symbol lists for POSIX, 
 whereas for Windows you don't need anything.
 That's precisely why 
 https://github.com/ldc-developers/ldc/issues/2431 would be 
 helpful.
So I achieved to reduce library size to around 1M, it still too big for my target. An equivalent library in C++ is about 100K. I noticed that the .text EFL section of the D one is so huge compared to the C++ one.
Jul 03 2018
parent kinke <kinke libero.it> writes:
On Tuesday, 3 July 2018 at 16:45:19 UTC, boolangery wrote:
 So I achieved to reduce library size to around 1M, it still too 
 big for my target.

 An equivalent library in C++ is about 100K.

 I noticed that the .text EFL section of the D one is so huge 
 compared to the C++ one.
Unless DMD generates a smaller lib (unlikely IMO), I suggest moving this discussion to somewhere appropriate (general D or libasync if it contains template bloat). On Monday, 2 July 2018 at 12:21:04 UTC, Guillaume Piolat wrote:
 And it won't change anytime soon 
 (https://github.com/ldc-developers/ldc/issues/2431).
As always, all it takes is someone to step up and either contribute directly by PR or be willing to pay someone else to do it.
Jul 03 2018