www.digitalmars.com         C & C++   DMDScript  

D.gnu - Including GDC in nixpkgs

reply Stefan Frijters <sfrijters gmail.com> writes:
As a fan of both D and the nixpkgs package manager I wanted to 
try and help include GDC in nixpkgs, but I have been running into 
some problems.

If anyone could help by taking a look at my pull request at 
https://github.com/NixOS/nixpkgs/pull/69144 that would be greatly 
appreciated.

Current issues include:

* When compiling without forcing --disable-shared, there are 
linker errors about needing to compile with -fPIC. It seems that 
some files are built using that flag, but not all, and I'm not 
sure how to go about forcing it for the rest.
* When invoking nix-build without specifying --cores 1 there are 
problems when building libphobos; it looks like object files are 
moved before they are generated. This problem goes away when I 
build on a single core, but of course everything takes a very 
long time to try out.
* I have disabled bootstrapping for now to speed up the debugging 
process, hopefully it can be enabled again later on.

I have taken the compiler flags to be used from 
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=gdc-git, 
which was linked in another thread on this forum, but maybe this 
is too specific for Arch?

Thanks in advance.
Sep 20 2019
parent reply Johannes Pfau <nospam example.com> writes:
Hi Stefan,

Am Fri, 20 Sep 2019 10:29:10 +0000 schrieb Stefan Frijters:

 As a fan of both D and the nixpkgs package manager I wanted to try and
 help include GDC in nixpkgs, but I have been running into some problems.
 
Thank you a lot for looking at this and helping the D ecosystem!
 If anyone could help by taking a look at my pull request at
 https://github.com/NixOS/nixpkgs/pull/69144 that would be greatly
 appreciated.
 
 Current issues include:
 
 * When compiling without forcing --disable-shared, there are linker
 errors about needing to compile with -fPIC. It seems that some files are
 built using that flag, but not all, and I'm not sure how to go about
 forcing it for the rest.
This should not happen, -fPIC should be added automatically where required. Maybe it's caused by --enable-default-pie or any of the other flags which are not really needed?
 * When invoking nix-build without specifying --cores 1 there are
 problems when building libphobos; it looks like object files are moved
 before they are generated. This problem goes away when I build on a
 single core, but of course everything takes a very long time to try out.
Is this a problem during the make step or during make install? I don't think make install is multi-thread safe, but make should actually be fine.
 * I have disabled bootstrapping for now to speed up the debugging
 process, hopefully it can be enabled again later on.
 
 I have taken the compiler flags to be used from
 https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=gdc-git, which
 was linked in another thread on this forum, but maybe this is too
 specific for Arch?
Yes, these are actually arch and not gdc specific. You do not need any special flags for gdc, just add d to --enable-languages= and you should be fine. --with-target-system-zlib=yes is the only D specific flag in your github pull request, and it forces libphobos to link against the system zlib instead of including a copy. You'll have to decide whether this is the behavior you want. -- Johannes
Sep 20 2019
parent reply Stefan Frijters <sfrijters gmail.com> writes:
Hi Johannes,

Thank you for the feedback. I've posted some more below so we can 
try to find out what I'm doing wrong. I've added some blank lines 
here and there to make things more readable, but it's all very 
spammy I'm afraid.

On Friday, 20 September 2019 at 17:18:20 UTC, Johannes Pfau wrote:
 Hi Stefan,

 Am Fri, 20 Sep 2019 10:29:10 +0000 schrieb Stefan Frijters:

 As a fan of both D and the nixpkgs package manager I wanted to 
 try and help include GDC in nixpkgs, but I have been running 
 into some problems.
 
Thank you a lot for looking at this and helping the D ecosystem!
 If anyone could help by taking a look at my pull request at 
 https://github.com/NixOS/nixpkgs/pull/69144 that would be 
 greatly appreciated.
 
 Current issues include:
 * I have disabled bootstrapping for now to speed up the 
 debugging process, hopefully it can be enabled again later on.
 
 I have taken the compiler flags to be used from 
 https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=gdc-git, which was
linked in another thread on this forum, but maybe this is too specific for Arch?
Yes, these are actually arch and not gdc specific. You do not need any special flags for gdc, just add d to --enable-languages= and you should be fine. --with-target-system-zlib=yes is the only D specific flag in your github pull request, and it forces libphobos to link against the system zlib instead of including a copy. You'll have to decide whether this is the behavior you want.
Ok, I actually started out with only adding d as an enabled language, but then I just started adding other flags hoping that they would to the trick and at least let the build get to the end. --with-target-system-zlib=yes seems to be necessary though for my situation. This is what I get from config.log if I remove all the extra flags (except --disable-shared and --disable-bootstrap), but keeping whatever nixpkgs gives me by default: $ ../gcc-9.2.0/configure --prefix=/nix/store/crh7b7jill2hh8gg1i59i34qvmiq2mfa-gdc-9.2.0 --with-gmp-include=/nix/store/jljw9i7377gi55gbnipsqanqirdzs41p-g p-6.1.2-dev/include --with-gmp-lib=/nix/store/j2p1qjbajqfb95ba3dqgjbskni knikk-gmp-6.1.2/lib --with-mpfr-include=/nix/store/676gyak02v4zsgjx23f5arhlnd3j02ny-mp r-4.0.2-dev/include --with-mpfr-lib=/nix/store/n5m42hdi36gn9yvbcjyihj4fxjn bgjf-mpfr-4.0.2/lib --with-mpc=/nix/store/4abiapxk8wng6viks962wyr3a 04xnz4-libmpc-1.1.0 --with-libelf=/nix/store/hgx581smqcl2zi76lqzyk1j9pv cf85p-libelf-0.8.13 --with-native-system-header-dir=/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-gl bc-2.27-dev/include --enable-lto --disable-libstdcxx-pch --without-included-gettext --with-system-zlib --enable-static --enable-languages=d --disable-multilib --enable-plugin --with-isl=/nix/store/wak0rkw1xvpzh01hk9a77h56cx2czhgi-isl-0.17.1 --disable-bootstrap --with-target-system-zlib=yes --disable-shared --build=x86_64-unknown-linux-gnu --host=x86_64-unknown-linux-gnu Does any of that look weird to you? And just to check: is it okay to build GDC9 with GCC8, or should I use GCC9?
 * When compiling without forcing --disable-shared, there are 
 linker errors about needing to compile with -fPIC. It seems 
 that some files are built using that flag, but not all, and 
 I'm not sure how to go about forcing it for the rest.
This should not happen, -fPIC should be added automatically where required. Maybe it's caused by --enable-default-pie or any of the other flags which are not really needed?
Without all the other flags I still get this problem (it was pretty much the reason I started adding things in the first place). Does this mean it's 'just' libphobos itself that's missing the -fPIC flag? libtool: link: /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu 8.3.0/include-fixed -Wl,-rpath,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyv 2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7 drnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -shared -fversion=Shared core/.libs/atomic.o core/.libs/attribute.o core/.libs/bitop.o core/.libs/checkedint.o core/.libs/cpuid.o core/.libs/demangle.o core/.libs/exception.o core/internal/.libs/abort.o core/internal/.libs/arrayop.o core/internal/.libs/convert.o core/internal/.libs/hash.o core/internal/.libs/spinlock.o core/internal/.libs/string.o core/internal/.libs/traits.o core/.libs/math.o core/.libs/memory.o core/.libs/runtime.o core/.libs/simd.o core/stdc/.libs/assert_.o core/stdc/.libs/complex.o core/stdc/.libs/config.o core/stdc/.libs/ctype.o core/stdc/.libs/errno.o core/stdc/.libs/fenv.o core/stdc/.libs/float_.o core/stdc/.libs/inttypes.o core/stdc/.libs/limits.o core/stdc/.libs/locale.o core/stdc/.libs/math.o core/stdc/.libs/signal.o core/stdc/.libs/stdarg.o core/stdc/.libs/stddef.o core/stdc/.libs/stdint.o core/stdc/.libs/stdio.o core/stdc/.libs/stdlib.o core/stdc/.libs/string.o core/stdc/.libs/tgmath.o core/stdc/.libs/time.o core/stdc/.libs/wchar_.o core/stdc/.libs/wctype.o core/sync/.libs/barrier.o core/sync/.libs/condition.o core/sync/.libs/config.o core/sync/.libs/exception.o core/sync/.libs/mutex.o core/sync/.libs/rwmutex.o core/sync/.libs/semaphore.o core/.libs/thread.o core/.libs/time.o core/.libs/vararg.o gcc/.libs/attribute.o gcc/.libs/backtrace.o gcc/.libs/builtins.o gcc/.libs/deh.o gcc/.libs/emutls.o gcc/.libs/gthread.o gcc/sections/.libs/android.o gcc/sections/.libs/elf_shared.o gcc/sections/.libs/osx.o gcc/sections/.libs/package.o gcc/sections/.libs/win32.o gcc/sections/.libs/win64.o gcc/unwind/.libs/arm.o gcc/unwind/.libs/arm_common.o gcc/unwind/.libs/c6x.o gcc/unwind/.libs/generic.o gcc/unwind/.libs/package.o gcc/unwind/.libs/pe.o .libs/object.o rt/.libs/aApply.o rt/.libs/aApplyR.o rt/.libs/aaA.o rt/.libs/adi.o rt/.libs/arrayassign.o rt/.libs/arraycast.o rt/.libs/arraycat.o rt/.libs/cast_.o rt/.libs/config.o rt/.libs/critical_.o rt/.libs/deh.o rt/.libs/dmain2.o rt/.libs/invariant.o rt/.libs/lifetime.o rt/.libs/memory.o rt/.libs/minfo.o rt/.libs/monitor_.o rt/.libs/obj.o rt/.libs/qsort.o rt/.libs/sections.o rt/.libs/switch_.o rt/.libs/tlsgc.o rt/typeinfo/.libs/ti_Acdouble.o rt/typeinfo/.libs/ti_Acfloat.o rt/typeinfo/.libs/ti_Acreal.o rt/typeinfo/.libs/ti_Adouble.o rt/typeinfo/.libs/ti_Afloat.o rt/typeinfo/.libs/ti_Ag.o rt/typeinfo/.libs/ti_Aint.o rt/typeinfo/.libs/ti_Along.o rt/typeinfo/.libs/ti_Areal.o rt/typeinfo/.libs/ti_Ashort.o rt/typeinfo/.libs/ti_C.o rt/typeinfo/.libs/ti_byte.o rt/typeinfo/.libs/ti_cdouble.o rt/typeinfo/.libs/ti_cent.o rt/typeinfo/.libs/ti_cfloat.o rt/typeinfo/.libs/ti_char.o rt/typeinfo/.libs/ti_creal.o rt/typeinfo/.libs/ti_dchar.o rt/typeinfo/.libs/ti_delegate.o rt/typeinfo/.libs/ti_double.o rt/typeinfo/.libs/ti_float.o rt/typeinfo/.libs/ti_idouble.o rt/typeinfo/.libs/ti_ifloat.o rt/typeinfo/.libs/ti_int.o rt/typeinfo/.libs/ti_ireal.o rt/typeinfo/.libs/ti_long.o rt/typeinfo/.libs/ti_n.o rt/typeinfo/.libs/ti_ptr.o rt/typeinfo/.libs/ti_real.o rt/typeinfo/.libs/ti_short.o rt/typeinfo/.libs/ti_ubyte.o rt/typeinfo/.libs/ti_ucent.o rt/typeinfo/.libs/ti_uint.o rt/typeinfo/.libs/ti_ulong.o rt/typeinfo/.libs/ti_ushort.o rt/typeinfo/.libs/ti_void.o rt/typeinfo/.libs/ti_wchar.o rt/util/.libs/array.o rt/util/container/.libs/array.o rt/util/container/.libs/common.o rt/util/container/.libs/hashtab.o rt/util/container/.libs/treap.o rt/util/.libs/random.o rt/util/.libs/typeinfo.o rt/util/.libs/utf.o core/stdc/.libs/libgdruntime_la-errno_.o gc/.libs/bits.o gc/.libs/config.o gc/.libs/gcinterface.o gc/impl/conservative/.libs/gc.o gc/impl/manual/.libs/gc.o gc/.libs/os.o gc/.libs/pooltable.o gc/.libs/proxy.o core/sys/posix/.libs/aio.o core/sys/posix/arpa/.libs/inet.o core/sys/posix/.libs/config.o core/sys/posix/.libs/dirent.o core/sys/posix/.libs/dlfcn.o core/sys/posix/.libs/fcntl.o core/sys/posix/.libs/grp.o core/sys/posix/.libs/iconv.o core/sys/posix/.libs/inttypes.o core/sys/posix/.libs/libgen.o core/sys/posix/.libs/mqueue.o core/sys/posix/net/.libs/if_.o core/sys/posix/.libs/netdb.o core/sys/posix/netinet/.libs/in_.o core/sys/posix/netinet/.libs/tcp.o core/sys/posix/.libs/poll.o core/sys/posix/.libs/pthread.o core/sys/posix/.libs/pwd.o core/sys/posix/.libs/sched.o core/sys/posix/.libs/semaphore.o core/sys/posix/.libs/setjmp.o core/sys/posix/.libs/signal.o core/sys/posix/.libs/spawn.o core/sys/posix/.libs/stdio.o core/sys/posix/.libs/stdlib.o core/sys/posix/sys/.libs/filio.o core/sys/posix/sys/.libs/ioccom.o core/sys/posix/sys/.libs/ioctl.o core/sys/posix/sys/.libs/ipc.o core/sys/posix/sys/.libs/mman.o core/sys/posix/sys/.libs/msg.o core/sys/posix/sys/.libs/resource.o core/sys/posix/sys/.libs/select.o core/sys/posix/sys/.libs/shm.o core/sys/posix/sys/.libs/socket.o core/sys/posix/sys/.libs/stat.o core/sys/posix/sys/.libs/statvfs.o core/sys/posix/sys/.libs/time.o core/sys/posix/sys/.libs/ttycom.o core/sys/posix/sys/.libs/types.o core/sys/posix/sys/.libs/uio.o core/sys/posix/sys/.libs/un.o core/sys/posix/sys/.libs/utsname.o core/sys/posix/sys/.libs/wait.o core/sys/posix/.libs/syslog.o core/sys/posix/.libs/termios.o core/sys/posix/.libs/time.o core/sys/posix/.libs/ucontext.o core/sys/posix/.libs/unistd.o core/sys/posix/.libs/utime.o core/sys/linux/.libs/config.o core/sys/linux/.libs/dlfcn.o core/sys/linux/.libs/elf.o core/sys/linux/.libs/epoll.o core/sys/linux/.libs/errno.o core/sys/linux/.libs/execinfo.o core/sys/linux/.libs/fcntl.o core/sys/linux/.libs/ifaddrs.o core/sys/linux/.libs/link.o core/sys/linux/netinet/.libs/in_.o core/sys/linux/netinet/.libs/tcp.o core/sys/linux/.libs/sched.o core/sys/linux/.libs/stdio.o core/sys/linux/sys/.libs/auxv.o core/sys/linux/sys/.libs/eventfd.o core/sys/linux/sys/.libs/file.o core/sys/linux/sys/.libs/inotify.o core/sys/linux/sys/.libs/mman.o core/sys/linux/sys/netinet/.libs/tcp.o core/sys/linux/sys/.libs/prctl.o core/sys/linux/sys/.libs/signalfd.o core/sys/linux/sys/.libs/socket.o core/sys/linux/sys/.libs/sysinfo.o core/sys/linux/sys/.libs/time.o core/sys/linux/sys/.libs/xattr.o core/sys/linux/.libs/termios.o core/sys/linux/.libs/time.o core/sys/linux/.libs/timerfd.o core/sys/linux/.libs/tipc.o core/sys/linux/.libs/unistd.o config/x86/.libs/libgdruntime_la-switchcontext.o gcc/.libs/config.o gcc/.libs/libbacktrace.o -Wl,--whole-archive ../../libatomic/.libs/libatomic_convenience.a ../../libbacktrace/.libs/libbacktrace.a -Wl,--no-whole-archive -lm -lpthread -ldl -lz -Wl,-rpath -Wl,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -nophoboslib -dstartfiles -B../src -Bgcc -Wl,-rpath -Wl,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyvx2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -pthread -Wl,-soname -Wl,libgdruntime.so.76 -o .libs/libgdruntime.so.76.0.3 /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/atomic.o: relocation R_X86_64_32S against hidden symbol `gdc.dso_slot' can not be used when making a shared object /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/attribute.o: relocation R_X86_64_32S against hidden symbol `gdc.dso_slot' can not be used when making a shared object /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/bitop.o: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/checkedint.o: relocation R_X86_64_32S against hidden symbol `gdc.dso_slot' can not be used when making a shared object /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/cpuid.o: relocation R_X86_64_32S against symbol `_D4core5cpuid18getcacheinfoCPUID2FNbNiNeZ14decipherCpuid2MFNbNihZ3idsyG63h' can not be used when making a shared object; recompile with -fPIC /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/demangle.o: relocation R_X86_64_32 against symbol `_D11TypeInfo_xm6__initZ' can not be used when making a shared object; recompile with -fPIC /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/.libs/exception.o: relocation R_X86_64_32 against symbol `_D12TypeInfo_Aya6__initZ' can not be used when making a shared object; recompile with -fPIC /nix/store/cl1i6bfqnx48ipakj4px7pb1babzs23j-binutils-2.31.1/bin/ld: core/internal/.libs/abort.o: relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC [... and some more screens of similar errors ... ] Looks like the shared version is getting -fversion=Shared, but not -fPIC: /nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23/bin/bash ../libtool --tag=D --mode=compile /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu 8.3.0/include-fixed -Wl,-rpath,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyv 2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7 drnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c -o core/sys/linux/sys/socket.lo ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/socket.d libtool: compile: /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu 8.3.0/include-fixed -Wl,-rpath,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyv 2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7 drnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/socket.d -fversion=Shared d21: warning: command line option '-idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include' is valid for C/C++/Fortran/ObjC/ObjC++ but not for D d21: warning: command line option '-idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu/ .3.0/include-fixed' is valid for C/C++/Fortran/ObjC/ObjC++ but not for D libtool: compile: mv -f "socket.o" "core/sys/linux/sys/.libs/socket.o" libtool: compile: /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu 8.3.0/include-fixed -Wl,-rpath,/nix/store/dwmw9jilsa5fmvpj1z568kxbnchzyv 2-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7 drnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/socket.d >/dev/null 2>&1 libtool: compile: mv -f "socket.o" "core/sys/linux/sys/socket.o"
 * When invoking nix-build without specifying --cores 1 there 
 are problems when building libphobos; it looks like object 
 files are moved before they are generated. This problem goes 
 away when I build on a single core, but of course everything 
 takes a very long time to try out.
Is this a problem during the make step or during make install? I don't think make install is multi-thread safe, but make should actually be fine.
It's happening during an invocation of libtool - pasted an example of lines grepped for time\.[od] below, but I've seen it happen for other files as well (e.g. array.o). Also note that in the example for socket.o above, which was built using a single core, the compile / mv steps are nicely interleaved, unlike below: /nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23/bin/bash ../libtool --tag=D --mode=compile /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu 8.3.0/include-fixed -Wl,-rpath,/nix/store/1cfsf8mqph03gz3r9qh9fbqv99vdg0 y-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7 drnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c -o core/sys/linux/sys/time.lo ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/time.d /nix/store/l6h4ya0wzb4b8mr0y58k2gh2nhfql4sn-bash-4.4-p23/bin/bash ../libtool --tag=D --mode=compile /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu 8.3.0/include-fixed -Wl,-rpath,/nix/store/1cfsf8mqph03gz3r9qh9fbqv99vdg0 y-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7 drnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c -o core/sys/linux/time.lo ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/time.d libtool: compile: /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu 8.3.0/include-fixed -Wl,-rpath,/nix/store/1cfsf8mqph03gz3r9qh9fbqv99vdg0 y-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7 drnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/sys/time.d libtool: compile: /build/build/./gcc/gdc -B/build/build/./gcc/ -O2 -I/nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -B/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/ -idirafter /nix/store/7b67hqgxavm47qizrkya55qvyp1f65s4-glibc-2.27-dev/include -idirafter /nix/store/pcs8pq4a5rkym1hzibqz7da45fxkmig7-gcc-8.3.0/lib/gcc/x86_64-unknown-linux-gnu 8.3.0/include-fixed -Wl,-rpath,/nix/store/1cfsf8mqph03gz3r9qh9fbqv99vdg0 y-gdc-9.2.0-lib/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7 drnn-glibc-2.27/lib -Wl,-rpath -Wl,/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -Wl,-L/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib -O2 -g -nostdinc -I ../../../../gcc-9.2.0/libphobos/libdruntime -I . -c ../../../../gcc-9.2.0/libphobos/libdruntime/core/sys/linux/time.d libtool: compile: mv -f "time.o" "core/sys/linux/sys/time.o" libtool: compile: mv -f "time.o" "core/sys/linux/time.o" mv: cannot stat 'time.o': No such file or directory This is particulary annoying since I have a 16-core Ryzen so I lose a *lot* of time debugging with a single core... Thanks for helping!
Sep 22 2019
parent reply Stefan Frijters <sfrijters gmail.com> writes:
On Sunday, 22 September 2019 at 10:29:24 UTC, Stefan Frijters 
wrote:
 Hi Johannes,

 Thank you for the feedback. I've posted some more below so we 
 can try to find out what I'm doing wrong. I've added some blank 
 lines here and there to make things more readable, but it's all 
 very spammy I'm afraid.

 [...]
Well, I've made some progress: * I still don't know what causes libtool to misbehave in parallel, but I've forced libphobos to be built with -j1, so that's at least a working workaround that doesn't slow down the rest of the build. * The nixpkgs wrappers for gcc introduce the -idirafter flag, which is not accepted by gdc. It looks like this causes the check for the '-fPIC' flag to fail, and the flag to be dropped. This is the cause of the linker error for the shared libs. I've hacked around for a bit and got rid of the flag at the point of the check, so libphobos now builds correctly (but still with a bunch of warnings). * I've added some nix code to generate a wrapper for gdc, like the ones that are generated for the rest of the GNU compiler family. With these changes I can now build gdc + libraries and I've successfully built the example on the dlang.org front page from a nix shell. The PR will need some reviewing from nixpkgs folks for sure since I did some nasty things.
Sep 25 2019
parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 25 Sep 2019 at 21:30, Stefan Frijters via D.gnu
<d.gnu puremagic.com> wrote:
 On Sunday, 22 September 2019 at 10:29:24 UTC, Stefan Frijters
 wrote:
 Hi Johannes,

 Thank you for the feedback. I've posted some more below so we
 can try to find out what I'm doing wrong. I've added some blank
 lines here and there to make things more readable, but it's all
 very spammy I'm afraid.

 [...]
Well, I've made some progress: * I still don't know what causes libtool to misbehave in parallel, but I've forced libphobos to be built with -j1, so that's at least a working workaround that doesn't slow down the rest of the build. * The nixpkgs wrappers for gcc introduce the -idirafter flag, which is not accepted by gdc. It looks like this causes the check for the '-fPIC' flag to fail, and the flag to be dropped. This is the cause of the linker error for the shared libs. I've hacked around for a bit and got rid of the flag at the point of the check, so libphobos now builds correctly (but still with a bunch of warnings). * I've added some nix code to generate a wrapper for gdc, like the ones that are generated for the rest of the GNU compiler family.
How does it go about building Ada or Go? Surely those two frontends will run into the same problem also.
 With these changes I can now build gdc + libraries and I've
 successfully built the example on the dlang.org front page from a
 nix shell.

 The PR will need some reviewing from nixpkgs folks for sure since
 I did some nasty things.
If you need any other eyes to have a look over, don't be afraid to ping. -- Iain
Oct 21 2019
parent reply Stefan Frijters <sfrijters gmail.com> writes:
On Monday, 21 October 2019 at 19:26:15 UTC, Iain Buclaw wrote:
 On Wed, 25 Sep 2019 at 21:30, Stefan Frijters via D.gnu 
 <d.gnu puremagic.com> wrote:
[...]
How does it go about building Ada or Go? Surely those two frontends will run into the same problem also.
 [...]
If you need any other eyes to have a look over, don't be afraid to ping.
It took some time, but my PR to get gdc into nixpkgs has been approved and merged. It will take some more time to trickle down to the stable branch, but it's available at the bleeding edge at least. Stefan
May 24
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On 24/05/2020 13:02, Stefan Frijters via D.gnu wrote:
 On Monday, 21 October 2019 at 19:26:15 UTC, Iain Buclaw wrote:
 On Wed, 25 Sep 2019 at 21:30, Stefan Frijters via D.gnu <d.gnu puremagic.com>
wrote:
 [...]
How does it go about building Ada or Go?  Surely those two frontends will run into the same problem also.
 [...]
If you need any other eyes to have a look over, don't be afraid to ping.
It took some time, but my PR to get gdc into nixpkgs has been approved and merged. It will take some more time to trickle down to the stable branch, but it's available at the bleeding edge at least.
Thanks for your persistence. Iain.
May 25