www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Special behaviour for _Dmodule_ref and _d_run_main?

reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
Hi!

I am trying to build app & druntime from sources with -flto 
enabled and faced with this linker error:

ld.lld: error: undefined symbol: _Dmodule_ref

 referenced by app.d
               lto.tmp:(.text._D3app16__moduleinfoCtorZ+0x0)
 referenced by app.d
               lto.tmp:(_D3app16__moduleinfoCtorZ)







ld.lld: error: undefined symbol: _d_run_main

 referenced by entrypoint.d:35 
 (/usr/lib/ldc/x86_64-linux-gnu/include/d/core/internal/entrypoint.d:35)






It seems that link-time optimization discards these characters as 
unused.

I tried to add  assumeUsed into definitions, but it is not helps.

What can be ways to solve this problem?

Maybe LDC implements some special behaviour for _Dmodule_ref and 
_d_run_main?
May 11 2020
parent reply "David Nadlinger" <code klickverbot.at> writes:
On 11 May 2020, at 10:52, Denis Feklushkin via digitalmars-d-ldc wrote:

 ld.lld: error: undefined symbol: _Dmodule_ref

 referenced by app.d
               lto.tmp:(.text._D3app16__moduleinfoCtorZ+0x0)
 referenced by app.d
               lto.tmp:(_D3app16__moduleinfoCtorZ)








Which version of LDC/druntime are you trying to use? _Dmodule_ref 
shouldn't have been in use on x86(_64) Linux for a few years now.

  — David
May 11 2020
next sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 11:08:51 UTC, David Nadlinger wrote:

 On 11 May 2020, at 10:52, Denis Feklushkin via 
 digitalmars-d-ldc wrote:

 ld.lld: error: undefined symbol: _Dmodule_ref

 referenced by app.d
               lto.tmp:(.text._D3app16__moduleinfoCtorZ+0x0)
 referenced by app.d
               lto.tmp:(_D3app16__moduleinfoCtorZ)








 Which version of LDC/druntime are you trying to use?


LDC - the LLVM D compiler (1.20.1):
   based on DMD v2.090.1 and LLVM 9.0.1

Compiling for ARM (--mtriple=thumbv7m-unknown-none-eabi)


 _Dmodule_ref shouldn't have been in use on x86(_64) Linux for a 
 few years now.


LDC fork of druntime still uses it.

$ llvm-nm-9 -a libdruntime.a says that _Dmodule_ref and 
_Dmodule_ref is defined in libdruntime.a
May 11 2020
next sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 12:00:40 UTC, Denis Feklushkin wrote:


 $ llvm-nm-9 -a libdruntime.a says that _Dmodule_ref and 
 _Dmodule_ref is defined in libdruntime.a


fix: _Dmodule_ref and _d_run_main is defined in libdruntime.a
May 11 2020
prev sibling parent reply kinke <kinke gmx.net> writes:
On Monday, 11 May 2020 at 12:00:40 UTC, Denis Feklushkin wrote:

 _Dmodule_ref shouldn't have been in use on x86(_64) Linux for 
 a few years now.


 LDC fork of druntime still uses it.


It's not used for Linux; the legacy _Dmodule_ref linked list is 
only used for unknown OS IIRC. There have been undefined 
_Dmodule_ref symbol issues for WebAssembly too IIRC.


 $ llvm-nm-9 -a libdruntime.a says that _Dmodule_ref and 
 _Dmodule_ref is defined in libdruntime.a


Just for completeness, what are you linking against, an LTO 
druntime or a regular druntime?

Unknown OS isn't on my priorities list, especially not if there's 
only LTO troubles, so you'll probably have to look into this 
yourself if you wanna get it working with LTO.
May 11 2020
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 12:14:23 UTC, kinke wrote:

 On Monday, 11 May 2020 at 12:00:40 UTC, Denis Feklushkin wrote:

 _Dmodule_ref shouldn't have been in use on x86(_64) Linux for 
 a few years now.


 LDC fork of druntime still uses it.


 It's not used for Linux; the legacy _Dmodule_ref linked list is 
 only used for unknown OS IIRC. There have been undefined 
 _Dmodule_ref symbol issues for WebAssembly too IIRC.


 $ llvm-nm-9 -a libdruntime.a says that _Dmodule_ref and 
 _Dmodule_ref is defined in libdruntime.a


 Just for completeness, what are you linking against, an LTO 
 druntime or a regular druntime?


Linking against LTO libdruntime.a


 Unknown OS isn't on my priorities list,


The lack of an operating system, more precisely :-)


 especially not if there's only LTO troubles, so you'll probably 
 have to look into this yourself if you wanna get it working 
 with LTO.


There is some special behavior for libdruntime during linking?

_d_run_main and _Dmodule_ref placed not in core.internal.* 
(unlike of core.internal.entrypoint._d_cmain, for example), so it 
looks like there shouldn't be any problems... but problems is 
here.
May 11 2020
parent reply "David Nadlinger" <code klickverbot.at> writes:
On 11 May 2020, at 13:30, Denis Feklushkin via digitalmars-d-ldc wrote:

 [bare metal]


Ah, right – I had assumed you were building for x86_64-linux-gnu as 
per the path you pasted.


 There is some special behavior for libdruntime during linking?

 _d_run_main and _Dmodule_ref placed not in core.internal.* (unlike of 
 core.internal.entrypoint._d_cmain, for example), so it looks like 
 there shouldn't be any problems... but problems is here.


How are you currently invoking the linker, especially regarding ordering 
of libraries/object files on the command line? LDC doesn't have any 
special handling for druntime (beyond the defaultlib selection logic).

  — David
May 11 2020
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 12:48:20 UTC, David Nadlinger wrote:


 How are you currently invoking the linker, especially regarding 
 ordering of libraries/object files on the command line?


I run lld from clang by Meson. But I don’t think that this should 
influence anything: all discussed symbols are external in C 
format:

sections_ldc.d:
extern (C) __gshared ModuleReference* _Dmodule_ref;   // start of 
linked list

dmain2.d:
extern (C) int _d_run_main(int argc, char** argv, MainFunc 
mainFunc)

And also all works ok if "-flto" isn't used.

If I start linking by ldc2 it throws error at first passed object 
file:

firmware.elf exe/meson-generated_gpio.d.o: file not recognized: 
file format not recognized

llvm-nm-9 confirms that this is object file.

This issues occurs only if -flto= enabled.


 LDC doesn't have any special handling for druntime (beyond the 
 defaultlib selection logic).


It's good. I tried define some external(C) symbol near of 
_Dmodule_ref and then use it from main() - linking with this 
symbol works ok. So, this problem only with "hidden"  calls 
invoked by compiler.
May 11 2020
next sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 13:29:51 UTC, Denis Feklushkin wrote:


 If I start linking by ldc2 it throws error at first passed 
 object file:

 firmware.elf exe/meson-generated_gpio.d.o: file not recognized: 
 file format not recognized


Not related to topic: I forgot to add --gcc=arm-none-eabi-gcc 
while linking.
May 11 2020
prev sibling parent reply kinke <kinke gmx.net> writes:
On Monday, 11 May 2020 at 13:29:51 UTC, Denis Feklushkin wrote:

 It's good. I tried define some external(C) symbol near of 
 _Dmodule_ref and then use it from main() - linking with this 
 symbol works ok. So, this problem only with "hidden"  calls 
 invoked by compiler.


_d_run_main isn't invoked by magic compiler code. - What linker 
are you using (linker, not gcc linker driver)? We default to gold 
(-linker=gold) for Linux because there have been numerous LTO 
issues with default bfd in the past (and the LLVM plugin is 
(primarily?) for gold, not bfd).
May 11 2020
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 15:25:24 UTC, kinke wrote:


 - What linker are you using (linker, not gcc linker driver)? We 
 default to gold (-linker=gold) for Linux because there have 
 been numerous LTO issues with default bfd in the past (and the 
 LLVM plugin is (primarily?) for gold, not bfd).


I use clang for linking and it calls ld.lld for this purpose.
May 11 2020
parent reply "David Nadlinger" <code klickverbot.at> writes:
On 11 May 2020, at 16:44, Denis Feklushkin via digitalmars-d-ldc wrote:

 I use clang for linking and it calls ld.lld for this purpose.


Could you please create an issue on GitHub with instructions to 
reproduce, or at least the full compiler/linker command lines? This 
discussion is getting quite hard to follow…

  — David
May 11 2020
next sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 15:58:13 UTC, David Nadlinger wrote:

 On 11 May 2020, at 16:44, Denis Feklushkin via 
 digitalmars-d-ldc wrote:

 I use clang for linking and it calls ld.lld for this purpose.


 Could you please create an issue on GitHub with instructions to 
 reproduce, or at least the full compiler/linker command lines? 
 This discussion is getting quite hard to follow…


Yes, but it will not be fast - the project is quite large, 
discussed command line is huge size (generated by Meson)

Linking comand line (LFs added to convient reading):

clang  -o firmware.elf
   'firmware.elf exe/meson-generated_gpio.d.o'
   'firmware.elf exe/meson-generated_freertos.d.o'
   [... huge amount of obj files skip ...]
   'firmware.elf exe/subprojects_freertos_timers.c.o'
   'firmware.elf exe/subprojects_libunwind_src_Unwind-seh.cpp.o'
   
'firmware.elf exe/subprojects_libunwind_src_UnwindRegistersSave.S.o'
   -L/usr/lib/gcc/arm-none-eabi/8.3.1/thumb/v7-m/nofp/ 
-Wl,--as-needed -Wl,--no-undefined
   -Xlinker --fatal-warnings -flto=thin -target 
thumbv7m-unknown-none-eabi
   --no-standard-libraries -Wl,
   --start-group subprojects/picolibc/semihost/libsemihost.a 
d/libdruntime.a
   subprojects/libopencm3/libopencm3_stm32f1.a 
subprojects/libopencm3/libreset_handler.a
   -Xlinker --script=linker_script.ld -lgcc -Wl,--end-group
   
'-Wl,-rpath,$ORIGIN/subprojects/picolibc/semihost:$ORIGIN/d:$ORIGIN/subprojects/libopencm3'
   
-Wl,-rpath-link,/home/denizzz/Dev/d_c_arm_test/builddir/subprojects/picolibc/semihost
   -Wl,-rpath-link,/home/denizzz/Dev/d_c_arm_test/builddir/d
   
-Wl,-rpath-link,/home/denizzz/Dev/d_c_arm_test/builddir/subprojects/libopencm3
ld.lld: error: undefined symbol: _Dmodule_ref

 referenced by gpio.d
               lto.tmp:(.text._D4gpio16__moduleinfoCtorZ+0x0)
 referenced by gpio.d
               lto.tmp:(_D4gpio16__moduleinfoCtorZ)
 referenced by freertos.d
               
 lto.tmp:(.text._D8freertos16__moduleinfoCtorZ+0x0)
 referenced by freertos.d
               lto.tmp:(_D8freertos16__moduleinfoCtorZ)
 referenced by app.d
               lto.tmp:(.text._D3app16__moduleinfoCtorZ+0x0)
 referenced by app.d
               lto.tmp:(_D3app16__moduleinfoCtorZ)






ld.lld: error: undefined symbol: _d_run_main

 referenced by entrypoint.d:35 
 (/usr/lib/ldc/x86_64-linux-gnu/include/d/core/internal/entrypoint.d:35)
               lto.tmp:(main)






clang: error: ld.lld command failed with exit code 1 (use -v to 
see invocation)
May 11 2020
prev sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 15:58:13 UTC, David Nadlinger wrote:

 On 11 May 2020, at 16:44, Denis Feklushkin via 
 digitalmars-d-ldc wrote:

 I use clang for linking and it calls ld.lld for this purpose.


 Could you please create an issue on GitHub with instructions to 
 reproduce, or at least the full compiler/linker command lines? 
 This discussion is getting quite hard to follow…


Here is this project, ~master branch:
https://github.com/denizzzka/d_c_arm_test/
May 11 2020
parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
Problem solved!

It was not related to LDC. It was lead from mess-up of archievers 
ar, llvm-ar-7 and llvm-ar-9, which lead to creation of .a files 
what can't be readed properly by clang

Thanks to all who tried to help! :-)
May 11 2020
prev sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 11 May 2020 at 11:08:51 UTC, David Nadlinger wrote:

 On 11 May 2020, at 10:52, Denis Feklushkin via 
 digitalmars-d-ldc wrote:

 ld.lld: error: undefined symbol: _Dmodule_ref

 referenced by app.d
               lto.tmp:(.text._D3app16__moduleinfoCtorZ+0x0)
 referenced by app.d
               lto.tmp:(_D3app16__moduleinfoCtorZ)








 Which version of LDC/druntime are you trying to use? 
 _Dmodule_ref shouldn't have been in use on x86(_64) Linux for a 
 few years now.


Got it. I used src/rt/sections_ldc.d as the easiest for support 
in bare metal. It provides _Dmodule_ref.
May 11 2020