www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ldc - Can't manage to build ldc, problem seems to be compiling

reply Andrew <aabrown24 hotmail.com> writes:
Hi

While I have no problems on my laptop, running make fails to 
build ldc on my university's cluster. I'm trying with the latest 
master and llvm 3.8.0. The cmake command I'm using is:

cmake -L 
-DLLVM_CONFIG="/home/abrown/software/llvm/bin/llvm-config" 
-DLIBCONFIG_LIBRARY="/home/abrown/software/libs/lib/libconfig.a" 
-DLIBCONFIG_INCLUDE_DIR="/home/abrown/software/libs/include" 
-DCMAKE_INSTALL_PREFIX="/home/abrown/software/bin_files" ..

The lines with errors are copied below:

/home/abrown/software/ldc/runtime/profile-rt/profile-rt-38/GCD
Profiling.c:221:23: error: ‘MAP_FILE’ undeclared (first use in this
function)
make[2]: *** 
[runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-3
/GCDAProfiling.c.o] Error 1
make[1]: *** [runtime/CMakeFiles/ldc-profile-rt.dir/all] Error 2
make: *** [all] Error 2

Any help would be greatly appreciated, is there any other 
information I should share when asking these questions?

Thanks very much

Andrew
Jul 23 2016
parent reply Johan Engelen <j j.nl> writes:
On Saturday, 23 July 2016 at 12:19:34 UTC, Andrew wrote:
 While I have no problems on my laptop, running make fails to 
 build ldc on my university's cluster.
What OS is used on the cluster?
Jul 23 2016
parent reply Andrew <aabrown24 hotmail.com> writes:
On Saturday, 23 July 2016 at 12:59:47 UTC, Johan Engelen wrote:
 On Saturday, 23 July 2016 at 12:19:34 UTC, Andrew wrote:
 While I have no problems on my laptop, running make fails to 
 build ldc on my university's cluster.
What OS is used on the cluster?
cat /etc/centos-release gives Red Hat Enterprise Linux Server release 6.5 (Santiago)
Jul 23 2016
parent reply Johan Engelen <j j.nl> writes:
On Saturday, 23 July 2016 at 13:02:34 UTC, Andrew wrote:
 On Saturday, 23 July 2016 at 12:59:47 UTC, Johan Engelen wrote:
 On Saturday, 23 July 2016 at 12:19:34 UTC, Andrew wrote:
 While I have no problems on my laptop, running make fails to 
 build ldc on my university's cluster.
What OS is used on the cluster?
cat /etc/centos-release gives Red Hat Enterprise Linux Server release 6.5 (Santiago)
MAP_FILE is a system #define. Perhaps an extra file must be #included. Does it work if you change `MAP_FILE | MAP_SHARED` to just `MAP_SHARED`? (i.e. can it find `MAP_SHARED`?)
Jul 23 2016
next sibling parent reply Johan Engelen <j j.nl> writes:
Can you check whether the file "sys/mman.h" contains MAP_FILE or 
not?
Jul 23 2016
parent reply Andrew <aabrown24 hotmail.com> writes:
On Saturday, 23 July 2016 at 13:34:14 UTC, Johan Engelen wrote:
 Can you check whether the file "sys/mman.h" contains MAP_FILE 
 or not?
Sorry, replied without seeing this message. In /usr/include/sys/mman.h there's no mention of MAP_FILE or MAP_SHARED
Jul 23 2016
parent reply Andrew <aabrown24 hotmail.com> writes:
On Saturday, 23 July 2016 at 14:04:22 UTC, Andrew wrote:
 On Saturday, 23 July 2016 at 13:34:14 UTC, Johan Engelen wrote:
 Can you check whether the file "sys/mman.h" contains MAP_FILE 
 or not?
Sorry, replied without seeing this message. In /usr/include/sys/mman.h there's no mention of MAP_FILE or MAP_SHARED
/usr/include/bits/mman.h has:
Jul 23 2016
parent reply Johan Engelen <j j.nl> writes:
On Saturday, 23 July 2016 at 14:07:20 UTC, Andrew wrote:
 On Saturday, 23 July 2016 at 14:04:22 UTC, Andrew wrote:
 On Saturday, 23 July 2016 at 13:34:14 UTC, Johan Engelen wrote:
 Can you check whether the file "sys/mman.h" contains MAP_FILE 
 or not?
Sorry, replied without seeing this message. In /usr/include/sys/mman.h there's no mention of MAP_FILE or MAP_SHARED
/usr/include/bits/mman.h has:
Where is MAP_SHARED defined? This is a bug in LLVM's compile-rt project, and it'd be good to find a proper fix for it. Perhaps it is adding the correct header file, or adding #ifndef MAP_FILE #endif
Jul 23 2016
parent reply Andrew <aabrown24 hotmail.com> writes:
On Saturday, 23 July 2016 at 15:06:12 UTC, Johan Engelen wrote:
 Where is MAP_SHARED defined?

 This is a bug in LLVM's compile-rt project, and it'd be good to 
 find a proper fix for it.
 Perhaps it is adding the correct header file, or adding
 #ifndef MAP_FILE

 #endif
MAP_SHARED is defined in /usr/include/bits/mman.h as well: #define MAP_SHARED 0x01 /* Share changes. */ and /usr/include/sys/mman.h has the line: #include <bits/mman.h> I tried the following code, cut and pasting the header lines from GCDAProfiling.c: #include <errno.h> #include <fcntl.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #if defined(_WIN32) #include "WindowsMMap.h" #else #include <sys/mman.h> #include <sys/file.h> #endif int main() { printf("%d,%d\n", MAP_FILE, MAP_SHARED); return 0; } and it outputted 0,1 I've also tried adding the lines #ifndef MAP_FILE #endif straight after the #include <sys/mman.h> line (reverting the previous change) and that compiles fine too. This isn't the first time this cluster has left me confused. Could it be that the problem is on this end, and the cluster has been set up in a funny way? Andrew
Jul 23 2016
next sibling parent Andrew <aabrown24 hotmail.com> writes:
On Saturday, 23 July 2016 at 15:45:13 UTC, Andrew wrote:

 I've also tried adding the lines

 #ifndef MAP_FILE

 #endif

 straight after the #include <sys/mman.h> line (reverting the 
 previous change) and that compiles fine too.
Sorry, I meant after the #include <sys/mman.h> paragraph (outside the if clause)
Jul 23 2016
prev sibling parent reply Johan Engelen <j j.nl> writes:
On Saturday, 23 July 2016 at 15:45:13 UTC, Andrew wrote:
 On Saturday, 23 July 2016 at 15:06:12 UTC, Johan Engelen wrote:
 Where is MAP_SHARED defined?

 This is a bug in LLVM's compile-rt project, and it'd be good 
 to find a proper fix for it.
 Perhaps it is adding the correct header file, or adding
 #ifndef MAP_FILE

 #endif
MAP_SHARED is defined in /usr/include/bits/mman.h as well: #define MAP_SHARED 0x01 /* Share changes. */ and /usr/include/sys/mman.h has the line: #include <bits/mman.h> I tried the following code, cut and pasting the header lines from GCDAProfiling.c: #include <errno.h> #include <fcntl.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #if defined(_WIN32) #include "WindowsMMap.h" #else #include <sys/mman.h> #include <sys/file.h> #endif int main() { printf("%d,%d\n", MAP_FILE, MAP_SHARED); return 0; } and it outputted 0,1
Huh? Perhaps a different compiler is used when building ldc-profile-rt ? Can you have a look at the commandline to build GCDAProfiling.c?
Jul 23 2016
parent reply Andrew <aabrown24 hotmail.com> writes:
On Saturday, 23 July 2016 at 15:57:01 UTC, Johan Engelen wrote:

 Huh?
 Perhaps a different compiler is used when building 
 ldc-profile-rt ?
 Can you have a look at the commandline to build GCDAProfiling.c?
Sorry, not sure if I completely understand you. cmake says: -- The C compiler identification is GNU 4.9.1 -- The CXX compiler identification is GNU 4.9.1 -- Check for working C compiler: /software/bin/cc -- Check for working C compiler: /software/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working CXX compiler: /software/bin/c++ -- Check for working CXX compiler: /software/bin/c++ -- works /software/bin/cc -c runtime/profile-rt/profile-rt-38/GCDAProfiling.c -o a.o compiles ok /software/bin/c++ -c runtime/profile-rt/profile-rt-38/GCDAProfiling.c -o a.o has errors to do with conversion from void pointers but nothing else. If I grep within the build folder for GCDAProfiling.c I get the following lines in build.make: runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-3 /GCDAProfiling.c.o: runtime/CMakeFiles/ldc-profile-rt.dir/flags.make runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-3 /GCDAProfiling.c.o: ../runtime/profile-rt/profile-rt-38/GCDAProfiling.c $(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green "Building C object runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o" cd /home/abrown/software/ldc/build/runtime && /software/bin/cc $(C_DEFINES) $(C_FLAGS) -o CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o -c /home/abrown/software/ldc/runtime/profile-rt/profile-rt-38/GCDAProfiling.c runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-3 /GCDAProfiling.c.i: cmake_force $(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green "Preprocessing C source to CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.i" cd /home/abrown/software/ldc/build/runtime && /software/bin/cc $(C_DEFINES) $(C_FLAGS) -E /home/abrown/software/ldc/runtime/profile-rt/profile-rt-38/GCDAProfiling.c > CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.i runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-3 /GCDAProfiling.c.s: cmake_force $(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --green "Compiling C source to assembly CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.s" cd /home/abrown/software/ldc/build/runtime && /software/bin/cc $(C_DEFINES) $(C_FLAGS) -S /home/abrown/software/ldc/runtime/profile-rt/profile-rt-38/GCDAProfiling.c -o CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.s runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o.requires: .PHONY : runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o.requires runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAPro iling.c.o.provides: runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o.requires $(MAKE) -f runtime/CMakeFiles/ldc-profile-rt.dir/build.make runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o.provides.build .PHONY : runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o.provides runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling c.o.provides.build: runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o "CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o" \ lib/libldc-profile-rt.a: runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o runtime/CMakeFiles/ldc-profile-rt.dir/requires: runtime/CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o.requires
Jul 23 2016
parent reply Johan Engelen <j j.nl> writes:
On Saturday, 23 July 2016 at 16:21:56 UTC, Andrew wrote:
 On Saturday, 23 July 2016 at 15:57:01 UTC, Johan Engelen wrote:

 Huh?
 Perhaps a different compiler is used when building 
 ldc-profile-rt ?
 Can you have a look at the commandline to build 
 GCDAProfiling.c?
Sorry, not sure if I completely understand you. cmake says:
`make VERBOSE=1`, `make -n` or `ninja -v` print out the commands used to build the files (so you can also see which compile flags are used)
 -- The C compiler identification is GNU 4.9.1
 -- The CXX compiler identification is GNU 4.9.1
 -- Check for working C compiler: /software/bin/cc
 -- Check for working C compiler: /software/bin/cc -- works
 -- Detecting C compiler ABI info
 -- Detecting C compiler ABI info - done
 -- Check for working CXX compiler: /software/bin/c++
 -- Check for working CXX compiler: /software/bin/c++ -- works

 /software/bin/cc -c 
 runtime/profile-rt/profile-rt-38/GCDAProfiling.c -o a.o

 compiles ok
Just to be sure: you are testing without any changes right (so with MAP_FILE in the source) ?
Jul 24 2016
parent reply Andrew <aabrown24 hotmail.com> writes:
On Sunday, 24 July 2016 at 08:42:57 UTC, Johan Engelen wrote:
 On Saturday, 23 July 2016 at 16:21:56 UTC, Andrew wrote:
 On Saturday, 23 July 2016 at 15:57:01 UTC, Johan Engelen wrote:

 Huh?
 Perhaps a different compiler is used when building 
 ldc-profile-rt ?
 Can you have a look at the commandline to build 
 GCDAProfiling.c?
Sorry, not sure if I completely understand you. cmake says:
`make VERBOSE=1`, `make -n` or `ninja -v` print out the commands used to build the files (so you can also see which compile flags are used)
 -- The C compiler identification is GNU 4.9.1
 -- The CXX compiler identification is GNU 4.9.1
 -- Check for working C compiler: /software/bin/cc
 -- Check for working C compiler: /software/bin/cc -- works
 -- Detecting C compiler ABI info
 -- Detecting C compiler ABI info - done
 -- Check for working CXX compiler: /software/bin/c++
 -- Check for working CXX compiler: /software/bin/c++ -- works

 /software/bin/cc -c 
 runtime/profile-rt/profile-rt-38/GCDAProfiling.c -o a.o

 compiles ok
Just to be sure: you are testing without any changes right (so with MAP_FILE in the source) ?
Thanks, I never knew about that flag. I am using the default file. The line that fails is this one: /software/bin/cc -DLDC_WITH_PGO -std=c11 -I/home/abrown/software/ldc/. -I/home/abrown/software/ldc/ddmd -I/home/abrown/software/ldc/ddmd/root -I/home/abrown/software/ldc/build/ddmd -I/home/abrown/software/ldc -isystem /Home/abrown/software/llvm/include -isystem /home/abrown/software/libs/include -fPIC -O3 -DCOMPILER_RT_HAS_ATOMICS=1 -DCOMPILER_RT_HAS_FCNTL_LCK=1 -o CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o -c /home/abrown/software/ldc/runtime/profile-rt/profile-rt-38/GCDAProfiling.c It fails when I call it myself, but succeeds if I drop the -std=c11 flag. This I find very puzzling, since I've set in my environment variables that CFLAGS='-std=c11' so shouldn't the flag also be set when I just call the compiler without options? Thanks again for all the help. Andrew
Jul 24 2016
parent reply Andrew <aabrown24 hotmail.com> writes:
On Sunday, 24 July 2016 at 11:32:56 UTC, Andrew wrote:
 /software/bin/cc  -DLDC_WITH_PGO -std=c11  
 -I/home/abrown/software/ldc/. -I/home/abrown/software/ldc/ddmd 
 -I/home/abrown/software/ldc/ddmd/root 
 -I/home/abrown/software/ldc/build/ddmd 
 -I/home/abrown/software/ldc -isystem 
 /Home/abrown/software/llvm/include -isystem 
 /home/abrown/software/libs/include     -fPIC -O3 
 -DCOMPILER_RT_HAS_ATOMICS=1 -DCOMPILER_RT_HAS_FCNTL_LCK=1 -o 
 CMakeFiles/ldc-profile-rt.dir/profile-rt/profile-rt-38/GCDAProfiling.c.o   -c
/home/abrown/software/ldc/runtime/profile-rt/profile-rt-38/GCDAProfiling.c

 It fails when I call it myself, but succeeds if I drop the 
 -std=c11 flag. This I find very puzzling, since I've set in my 
 environment variables that CFLAGS='-std=c11' so shouldn't the 
 flag also be set when I just call the compiler without options?

 Thanks again for all the help.

 Andrew
So this was definitely the problem, all I need to do is unset CFLAGS before I run make and everything works fine. Thanks very much, for all your help and your patience with me. Andrew
Jul 24 2016
parent reply Johan Engelen <j j.nl> writes:
On Sunday, 24 July 2016 at 12:58:00 UTC, Andrew wrote:
 So this was definitely the problem, all I need to do is unset 
 CFLAGS before I run make and everything works fine.

 Thanks very much, for all your help and your patience with me.
I'll still submit a patch to the LLVM guys working on compiler-rt. I think it is a latent bug because MAP_FILE is not strictly needed in the header file IIUC. Great that it works for you now. cheers, Johan
Jul 24 2016
parent Johan Engelen <j j.nl> writes:
On Sunday, 24 July 2016 at 13:54:07 UTC, Johan Engelen wrote:
 I'll still submit a patch to the LLVM guys working on 
 compiler-rt. I think it is a latent bug because MAP_FILE is not 
 strictly needed in the header file IIUC.
http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/profile/GCDAProfiling.c?view=diff&r1=276707&r2=276708&pathrev=276708
Jul 26 2016
prev sibling parent Andrew <aabrown24 hotmail.com> writes:
On Saturday, 23 July 2016 at 13:11:22 UTC, Johan Engelen wrote:
 On Saturday, 23 July 2016 at 13:02:34 UTC, Andrew wrote:
 On Saturday, 23 July 2016 at 12:59:47 UTC, Johan Engelen wrote:
 On Saturday, 23 July 2016 at 12:19:34 UTC, Andrew wrote:
 While I have no problems on my laptop, running make fails to 
 build ldc on my university's cluster.
What OS is used on the cluster?
cat /etc/centos-release gives Red Hat Enterprise Linux Server release 6.5 (Santiago)
MAP_FILE is a system #define. Perhaps an extra file must be #included. Does it work if you change `MAP_FILE | MAP_SHARED` to just `MAP_SHARED`? (i.e. can it find `MAP_SHARED`?)
That's lovely, thanks very much. With that change it just built. I only changed it in ldc/runtime/profile-rt/profile-rt-38/GCDAProfiling.c. Grep found a couple of other examples in other folders, but I just left those. I ran ctest and got the following results (core.time has always failed on this cluster): 99% tests passed, 3 tests failed out of 751 Total Test time (real) = 1325.92 sec The following tests FAILED: 21 - core.time (OTHER_FAULT) 390 - core.time-debug (OTHER_FAULT) 751 - lit-tests (Failed) So should I keep patching it in this way? Is there any better solution I could help with? Thanks again for your help Andrew
Jul 23 2016