digitalmars.D.learn - GDC fails to link with GSL and fortran code
- Andrew Brown (29/29) Mar 16 2015 Hi,
- ketmar (4/4) Mar 16 2015 On Mon, 16 Mar 2015 16:44:45 +0000, Andrew Brown wrote:
- Johannes Pfau (17/52) Mar 16 2015 You could try to do the linking with the local compiler:
- Andrew Brown (15/76) Mar 17 2015 Thank you very much for your replies, I now have 2 solutions to
- Johannes Pfau (32/53) Mar 17 2015 GCC's verbose output can indeed be quite confusing but if you know
- Andrew Brown (12/76) Mar 17 2015 I'm getting some idea how difficult compilers could be to
Hi, I'm trying to compile code which calls C and fortan routines from D on the linux cluster at work. I've managed to get it to work with all 3 compilers on my laptop, but LDC and GDC fail on the cluster (though DMD works perfectly). I'm using the precompiled compiler binaries on these systems, the cluster doesn't have the prerequistites for building them myself and I don't have admin rights. For GDC the commands I run are: gcc -c C_code.c Fortran_code.f gdc D_code.d C_code.o Fortran_code.f -lblas -lgsl -lgslcblas -lm -lgfortran -o out The error messages are: /software/lib64/libgsl.so: undefined reference to `memcpy GLIBC_2.14' /software/lib64/libgfortran.so.3: undefined reference to `clock_gettime GLIBC_2.17' /software/lib64/libgfortran.so.3: undefined reference to `secure_getenv GLIBC_2.17' collect2: error: ld returned 1 exit status I can remove the gsl messages by statically linking to libgsl.a, but this doesn't solve the gfortran issues. If anyone knows a way round these issues, I'd be very grateful. I'd also eventually like to find a way to easily share linux biniaries with people, so they can use this code without these kinds of headaches. If anyone has any advice for making this portable, that would also help me out a lot. Thanks very much Andrew
Mar 16 2015
On Mon, 16 Mar 2015 16:44:45 +0000, Andrew Brown wrote: can you please show output of `dmd -v` and `gdc -v`, so we can see the=20 actual commands they both using for linking? =
Mar 16 2015
Am Mon, 16 Mar 2015 16:44:45 +0000 schrieb "Andrew Brown" <aabrown24 hotmail.com>:Hi, I'm trying to compile code which calls C and fortan routines from D on the linux cluster at work. I've managed to get it to work with all 3 compilers on my laptop, but LDC and GDC fail on the cluster (though DMD works perfectly). I'm using the precompiled compiler binaries on these systems, the cluster doesn't have the prerequistites for building them myself and I don't have admin rights. For GDC the commands I run are: gcc -c C_code.c Fortran_code.f gdc D_code.d C_code.o Fortran_code.f -lblas -lgsl -lgslcblas -lm -lgfortran -o outYou could try to do the linking with the local compiler: gdc D_code.d gcc D_code.o C_code.o Fortran_code.o -lgphobos2 -lpthread -lblas -lgsl -lgslcblas -lm -L path/to/x86_64-gdcproject-linux-gnu/lib/The error messages are: /software/lib64/libgsl.so: undefined reference to `memcpy GLIBC_2.14' /software/lib64/libgfortran.so.3: undefined reference to `clock_gettime GLIBC_2.17' /software/lib64/libgfortran.so.3: undefined reference to `secure_getenv GLIBC_2.17' collect2: error: ld returned 1 exit statusSeems like the binary GDC toolchain somehow picks up a wrong libc. The toolchains are built with GLIBC 2.14. But IIRC we don't ship the libc in the binary packages (for native compilers) and it should pick up the local libc. Please run gdc with the '-v' and '-Wl,--verbose' options and post a link to the full output.I can remove the gsl messages by statically linking to libgsl.a, but this doesn't solve the gfortran issues. If anyone knows a way round these issues, I'd be very grateful. I'd also eventually like to find a way to easily share linux biniaries with people, so they can use this code without these kinds of headaches. If anyone has any advice for making this portable, that would also help me out a lot.Usually the best option is to compile on old linux systems. Binaries often run on newer systems but not on older ones. You could setup debian wheezy or an older version in a VM or using docker. Or you use docker.io ;-) I personally think the docker approach is kind of overkill but avoiding compatibility issues is one of docker's main selling points.
Mar 16 2015
Thank you very much for your replies, I now have 2 solutions to my problem! Both compiling on a virtual machine running debian wheezy, and using gcc to do the linking produced executables that would run on the cluster. Compiling with the verbose flags for linker and compiler produced the following output: failed gdc attempt: http://dpaste.com/0Z5V4PV successful dmd attempt: http://dpaste.com/0S5WKJ5 successful use of gcc to link: http://dpaste.com/0YYR39V It seems a bit of a mess, with various libraries in various places. I'll see if I can get to the bottom of it, I think it'll be a learning experience. Thanks again for the swift and useful help and guidance. Andrew On Monday, 16 March 2015 at 19:22:18 UTC, Johannes Pfau wrote:Am Mon, 16 Mar 2015 16:44:45 +0000 schrieb "Andrew Brown" <aabrown24 hotmail.com>:Hi, I'm trying to compile code which calls C and fortan routines from D on the linux cluster at work. I've managed to get it to work with all 3 compilers on my laptop, but LDC and GDC fail on the cluster (though DMD works perfectly). I'm using the precompiled compiler binaries on these systems, the cluster doesn't have the prerequistites for building them myself and I don't have admin rights. For GDC the commands I run are: gcc -c C_code.c Fortran_code.f gdc D_code.d C_code.o Fortran_code.f -lblas -lgsl -lgslcblas -lm -lgfortran -o outYou could try to do the linking with the local compiler: gdc D_code.d gcc D_code.o C_code.o Fortran_code.o -lgphobos2 -lpthread -lblas -lgsl -lgslcblas -lm -L path/to/x86_64-gdcproject-linux-gnu/lib/The error messages are: /software/lib64/libgsl.so: undefined reference to `memcpy GLIBC_2.14' /software/lib64/libgfortran.so.3: undefined reference to `clock_gettime GLIBC_2.17' /software/lib64/libgfortran.so.3: undefined reference to `secure_getenv GLIBC_2.17' collect2: error: ld returned 1 exit statusSeems like the binary GDC toolchain somehow picks up a wrong libc. The toolchains are built with GLIBC 2.14. But IIRC we don't ship the libc in the binary packages (for native compilers) and it should pick up the local libc. Please run gdc with the '-v' and '-Wl,--verbose' options and post a link to the full output.I can remove the gsl messages by statically linking to libgsl.a, but this doesn't solve the gfortran issues. If anyone knows a way round these issues, I'd be very grateful. I'd also eventually like to find a way to easily share linux biniaries with people, so they can use this code without these kinds of headaches. If anyone has any advice for making this portable, that would also help me out a lot.Usually the best option is to compile on old linux systems. Binaries often run on newer systems but not on older ones. You could setup debian wheezy or an older version in a VM or using docker. Or you use docker.io ;-) I personally think the docker approach is kind of overkill but avoiding compatibility issues is one of docker's main selling points.
Mar 17 2015
Am Tue, 17 Mar 2015 12:13:44 +0000 schrieb "Andrew Brown" <aabrown24 hotmail.com>:Thank you very much for your replies, I now have 2 solutions to my problem! Both compiling on a virtual machine running debian wheezy, and using gcc to do the linking produced executables that would run on the cluster. Compiling with the verbose flags for linker and compiler produced the following output: failed gdc attempt: http://dpaste.com/0Z5V4PV successful dmd attempt: http://dpaste.com/0S5WKJ5 successful use of gcc to link: http://dpaste.com/0YYR39V It seems a bit of a mess, with various libraries in various places. I'll see if I can get to the bottom of it, I think it'll be a learning experience. Thanks again for the swift and useful help and guidance. AndrewGCC's verbose output can indeed be quite confusing but if you know what to look for it's possible to find some useful information :-) In your case the linker messages hinted at a problem with libc. And as there were only a few errors it's likely a version compatibility problem. If you search for libc.so in these logs you'll find this: Failed GDC: attempt to open /usr/lib/../lib64/libc.so succeeded opened script file /usr/lib/../lib64/libc.so opened script file /usr/lib/../lib64/libc.so attempt to open /lib64/libc.so.6 succeeded /lib64/libc.so.6 GCC: attempt to open /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so succeeded opened script file /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so opened script file /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so attempt to open /software/lib64/libc.so.6 succeeded /software/lib64/libc.so.6 The binary gdc searches for libaries in the 'usual' places, including /usr/lib64. Your gcc doesn't search in /usr/lib64 but in /software. You seem to have an incompatible libc in /usr/lib64 which gets picked up by gdc. This is one reason why binary compiler releases are difficult to maintain and we usually recommend to compile gdc from source. DMD avoids this mess by simply calling the local gcc instead of ld to link. GCC unfortunately doesn't support this and forces us to always call the linker directly.
Mar 17 2015
I'm getting some idea how difficult compilers could be to maintain across distros, but I'm probably still a long way from knowing the true horror. Thank you for taking the time to help me out, and then when my immediate problem was solved, more time to help me learn something. I see there's a lot of discussion on these forums about the state of the documentation. Between googling historical questions here, and people's willingness to help on this forum, I've always found my answers. I think that's quite amazing. Andrew On Tuesday, 17 March 2015 at 21:00:46 UTC, Johannes Pfau wrote:Am Tue, 17 Mar 2015 12:13:44 +0000 schrieb "Andrew Brown" <aabrown24 hotmail.com>:Thank you very much for your replies, I now have 2 solutions to my problem! Both compiling on a virtual machine running debian wheezy, and using gcc to do the linking produced executables that would run on the cluster. Compiling with the verbose flags for linker and compiler produced the following output: failed gdc attempt: http://dpaste.com/0Z5V4PV successful dmd attempt: http://dpaste.com/0S5WKJ5 successful use of gcc to link: http://dpaste.com/0YYR39V It seems a bit of a mess, with various libraries in various places. I'll see if I can get to the bottom of it, I think it'll be a learning experience. Thanks again for the swift and useful help and guidance. AndrewGCC's verbose output can indeed be quite confusing but if you know what to look for it's possible to find some useful information :-) In your case the linker messages hinted at a problem with libc. And as there were only a few errors it's likely a version compatibility problem. If you search for libc.so in these logs you'll find this: Failed GDC: attempt to open /usr/lib/../lib64/libc.so succeeded opened script file /usr/lib/../lib64/libc.so opened script file /usr/lib/../lib64/libc.so attempt to open /lib64/libc.so.6 succeeded /lib64/libc.so.6 GCC: attempt to open /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so succeeded opened script file /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so opened script file /software/lib/gcc/x86_64-redhat-linux/4.9.1/../../../../lib64/libc.so attempt to open /software/lib64/libc.so.6 succeeded /software/lib64/libc.so.6 The binary gdc searches for libaries in the 'usual' places, including /usr/lib64. Your gcc doesn't search in /usr/lib64 but in /software. You seem to have an incompatible libc in /usr/lib64 which gets picked up by gdc. This is one reason why binary compiler releases are difficult to maintain and we usually recommend to compile gdc from source. DMD avoids this mess by simply calling the local gcc instead of ld to link. GCC unfortunately doesn't support this and forces us to always call the linker directly.
Mar 17 2015