D.gnu - Final steps for shared library support
- Johannes Pfau (74/74) May 22 2016 I've opened this pull request to finish the shared library support:
- Johannes Pfau (34/71) May 23 2016 OTOH the LDC approach doesn't sound too bad either. There probably
I've opened this pull request to finish the shared library support: https://github.com/D-Programming-GDC/GDC/pull/214 These are the remaining problems: 1) We have to find all ModuleInfos in any shared library or in the main executable (in the main executable even for statically linked libdruntime). We can use a nice binutils feature for this: If we place all symbols into a named section _not starting with a dot_ binutils will provide __start_SECTION and __stop_SECTION symbols for us: https://sourceware.org/binutils/docs-2.26/ld/Orphan-Sections.html#Orphan-Sections https://github.com/jpf91/GDC/blob/shared3/libphobos/libdruntime/__dshared.c#L12 This should work for all targets using the binutils linker, not just for linux. OTOH there are targets without binutils where we could also output ModuleInfos into a special section and use runtime functions to find the section (e.g. OSX): https://github.com/dlang/druntime/blob/master/src/rt/sections_osx_x86_64.d#L131 I guess we want to keep our platform-independent C constructor based approach for finding ModuleInfos for all targets not supported directly by the druntime section code. So the difficult question we have to answer now is: When does the compiler place the ModuleInfo into a section, when does it use the generic constructor code? I think this is the best solution: In the druntime configure script, run a small test to check for __start/stop_minfo symbols. These could be provided by the binutils feature or by explicit linker scripts. Then set a variable in gcc.config accordingly. In the compiler, search for a useModuleInfoSection manifest constant in rt.sections. If rt.sections does not exists or useModuleInfoSection is not defined, default to the old constructor based code. Otherwise choose accordingly to useModuleInfoSection. useModuleInfoSection is then set using static if & CTFE according to information from gcc.config and OS information (= true if gcc.config.minfoStartStop || version(OSX) ...) 2) We have to call _d_dso_registry once from every shared library and from the main executable: * DMD emits a weak function _d_dso_init into every object file. Additionally it adds a weak reference to this into the .ctors section for every object. We could probably do this manually. But this is not really portable and I don't think we can get the portable DECL_CONSTRUCTOR to emit weak references. * LDC also emits the weak _d_dso_init function. It emits normal constructors so _d_dso_init will be called multiple times. It uses a static variable in _d_dso_init to detect repeated calls. We could probably do this. Disadvantages: many useless constructors, the source code for _d_dso_init is hardcoded in the compiler and we have to hardcode the OS detection (i.e. emit _d_dso_init for linux, freebsd, ...) * We use the 'GCC way': The way C constructors are implemented is using startup files. These are special object files linked once into every shared library and executable. I've implemented & tested this approach in the pull request: If we link against libphobos, load https://github.com/jpf91/GDC/blob/shared3/libphobos/libdruntime/libdruntime.spec libdruntime.spec will then add __dshared.o to the start files, if the file exists. This means: * When linking with nophoboslib, __dshared.o and libdruntime.spec are not used and do not have to exist (good for bare metal code) * The spec can be redefined using GCCs spec file mechanism. * If __dshared.o does not exist (maybe not needed for some OSs) linking is done as usual. * We can add any code we want to __dshared.o/c/d. It will be linked once into every shared library and executable. OS detection can be done in C or D source code instead of in the compiler. * Main drawback: Linking needs to be done with gdc. If a user needs to link with gcc or g++ this is still possible using -specs=libdruntime.spec * As we need to link phobos/druntime with -nophoboslib we also have to use -specs=libdruntime.spec here. add a flag to ignore aggressive-loop-optimizations errors when building libdruntime? 4) Unit tests & test suite pass for shared libraries, except for one codegen/linker issue in the test suite. OTOH as this does not affect static linking we can handle this even after merging the initial shared library support code.
May 22 2016
Johannes Pfau <nospam example.com>
Am Sun, 22 May 2016 20:41:03 +0200
schrieb Johannes Pfau <nospam example.com>:
2) We have to call _d_dso_registry once from every shared library and
from the main executable:
* DMD emits a weak function _d_dso_init into every object file.
Additionally it adds a weak reference to this into the .ctors
section for every object. We could probably do this manually. But
this is not really portable and I don't think we can get the
portable DECL_CONSTRUCTOR to emit weak references.
* LDC also emits the weak _d_dso_init function. It emits normal
constructors so _d_dso_init will be called multiple times. It
uses a static variable in _d_dso_init to detect repeated calls.
We could probably do this. Disadvantages: many useless
constructors, the source code for _d_dso_init is hardcoded in the
compiler and we have to hardcode the OS detection (i.e. emit
_d_dso_init for linux, freebsd, ...)
* We use the 'GCC way': The way C constructors are implemented is
using startup files. These are special object files linked once
into every shared library and executable. I've implemented &
tested this approach in the pull request:
If we link against libphobos, load
https://github.com/jpf91/GDC/blob/shared3/libphobos/libdruntime/libdruntime.spec
libdruntime.spec will then add __dshared.o to the start files, if
the file exists. This means:
* When linking with nophoboslib, __dshared.o and libdruntime.spec
are not used and do not have to exist (good for bare metal
code)
* The spec can be redefined using GCCs spec file mechanism.
* If __dshared.o does not exist (maybe not needed for some OSs)
linking is done as usual.
* We can add any code we want to __dshared.o/c/d. It will be
linked once into every shared library and executable. OS detection can
be done in C or D source code instead of in the compiler.
* Main drawback: Linking needs to be done with gdc. If a user
needs to link with gcc or g++ this is still possible using
-specs=libdruntime.spec
* As we need to link phobos/druntime with -nophoboslib we also
have to use -specs=libdruntime.spec here.
OTOH the LDC approach doesn't sound too bad either. There probably
won't be more than ~1000 modules in a shared lib and the ctor stubs we
call should be pretty cheap. I don't want the OS detection in the
compiler though. Right now the function is quite simple, but who knows
what it will look like once we support more OSs. We could probably adapt
the __entrypoint.di approach: We need to add support for
attribute(constructor, destructor, visibility) first. Then we can
translate __dshared.c into D. We'd just ship the source file and
compile it into every object file (except for -betterC).
We also have to make sure the compiler does not emit ModuleInfo for
this module. Then we can simply do this:
__dshared.d:
// No ModuleInfo for this module
module sharedinclude;
import rt.sections : useDSORegistry;
static if (useDSORegistry)
{
weak hidden void __d_dso_init() {_d_dso_registry(...)}
weak hidden __gshared bool started, stopped;
// Have to somehow make sure this function does not clash with the
// same function in other object files (i.e. C-like static)
private ctor weak void startDSO()
{
if(started)
return;
started = true;
__d_dso_init();
}
[...]
}
I think both solutions are OK. Iain which one do you
prefer?
May 23 2016