D.gnu - Adding support for mingw-w64 and winpthread
- Vincent R (54/54) Mar 24 2016 Hi,
- Iain Buclaw via D.gnu (14/34) Mar 24 2016 How readily does Mingw support pthreads? What may be needed is simply a ...
- Vincent R (10/35) Mar 24 2016 winpthread is defined here:
- Johannes Pfau (47/75) Mar 24 2016 It supports most of the pthreads API IIRC but there are sometimes
- Vincent R (7/33) Mar 25 2016 Thanks for all this information and it seems your knowledge is
- Johannes Pfau (14/23) Mar 25 2016 Great! We've simply removed gthreads now:
- Vincent R (34/58) Mar 30 2016 I have some linker errors when I try to use the gdc compiler :
- Johannes Pfau (10/35) Mar 30 2016 Known problem, I'll probably push a fix to master for this soon.
- Vincent R (2/14) Mar 30 2016 Thanks the second option works.
- Vincent R (5/24) May 11 2016 Is there any progress for mingw ? Have you fixed what you wanted
- Johannes Pfau (12/39) May 12 2016 I posted a fix for the link order issue at
Hi, I would like to generate gdc compiler on msys2/mingw-w64 platform. The build system on msys2 uses archlinux build architecture and the build file is here: https://github.com/vrichomme/MINGW-packages/blob/newfastrelease/mingw-w64-gcc/PKGBUILD To generate it I followed the following steps: 1) Install https://msys2.github.io/ 2) $ update-core $ pacman -Su $ git clone https://github.com/vrichomme/MINGW-packages.git $ cd MINGW-packages $ git fetch origin $ git checkout -b newfastrelease origin/newfastrelease $ cd mingw-w64-gcc $ export MINGW_INSTALLS=mingw64 $ makepkg-mingw -sLf But of course I get an error because the platform is not supported: /home/Vincent/tmp/MINGW-packages/mingw-w64-gcc/src/build-x86_64-w 4-mingw32/./gcc/gdc -B/home/Vincent/tmp/MINGW-packages/mingw-w64-gcc/src/build-x86_6 -w64-mingw32/./gcc/ -L/mingw64/x86_64-w64-mingw32/lib -L/mingw64/lib -isystem /mingw64/x86_64-w64-mingw32/include -isystem /mingw64/include -B/mingw64/x86_64-w64-mingw32/bin/ -B/mingw64/x86_64-w64-mingw32/lib/ -isystem /mingw64/x86_64-w64-mingw32/include -isystem /mingw64/x86_64-w64-mingw32/sys-include -o gcc/gthreads/posix.o -Wall -g -frelease -O2 -nostdinc -pipe -Wno-deprecated -I ../../../../gcc-5.3.0/libphobos/libdruntime -I ./x86_64-w64-mingw32 -I . -c ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:29:1: error: undefined identifier pthread_key_t, did you mean alias gthread_key_t? alias gthread_key_t = pthread_key_t; ^ ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:30:1: error: undefined identifier pthread_once_t, did you mean alias gthread_once_t? alias gthread_once_t = pthread_once_t; ^ ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:31:1: error: undefined identifier pthread_mutex_t, did you mean alias gthread_mutex_t? alias gthread_mutex_t = pthread_mutex_t; ^ ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:32:1: error: undefined identifier pthread_mutex_t, did you mean alias gthread_mutex_t? alias gthread_recursive_mutex_t = pthread_mutex_t; ^ ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:34:27: error: undefined identifier PTHREAD_MUTEX_INITIALIZER enum GTHREAD_MUTEX_INIT = PTHREAD_MUTEX_INITIALIZER; ^ ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:35:27: error: undefined identifier PTHREAD_ONCE_INIT, did you mean variable GTHREAD_ONCE_INIT? enum GTHREAD_ONCE_INIT = PTHREAD_ONCE_INIT; ^ ../../../../gcc-5.3.0/libphobos/libdruntime/gcc/gthreads/posix.d:36:37: error: undefined identifier PTHREAD_MUTEX_INITIALIZER enum GTHREAD_RECURSIVE_MUTEX_INIT = PTHREAD_MUTEX_INITIALIZER; My question is how does Version is defined when compiling core.sys.posix.pthread ? Because I can see some block corresponding to each platform : version( linux ) { ... } else version( OSX ) { ... } But how is defined this value ? Is it based on uname function ? Can I trace the value of version when compiling the file ? Thanks
Mar 24 2016
On 24 March 2016 at 10:52, Vincent R via D.gnu <d.gnu puremagic.com> wrote:Hi, I would like to generate gdc compiler on msys2/mingw-w64 platform. The build system on msys2 uses archlinux build architecture and the build file is here: https://github.com/vrichomme/MINGW-packages/blob/newfastrelease/mingw-w64-gcc/PKGBUILD To generate it I followed the following steps: 1) Install https://msys2.github.io/ 2) $ update-core $ pacman -Su $ git clone https://github.com/vrichomme/MINGW-packages.git $ cd MINGW-packages $ git fetch origin $ git checkout -b newfastrelease origin/newfastrelease $ cd mingw-w64-gcc $ export MINGW_INSTALLS=mingw64 $ makepkg-mingw -sLf But of course I get an error because the platform is not supported:How readily does Mingw support pthreads? What may be needed is simply a new package for mingw-specific system functions (ie: core.sys.mingw.pthread). In the meantime, you should be able to get around by passing --enable-threads=win32 when configuring teh gcc build.But how is defined this value ? Is it based on uname function ? Can I trace the value of version when compiling the file ?What versions that are internally defined depend on the target triplet passed when configuring the build. This directly affects what target headers are included, which contain macros we call when initializing the compiler. It has been requested even in DMD to have some command-line option to print all turned on version conditions set by the compiler. Itself is a very trivial addition. In the absence of that though, in a debugger you can set a breakpoint on VersionCondition::addPredefinedGlobalIdent. Iain.
Mar 24 2016
Vincent R <lol dlang.org> On Thursday, 24 March 2016 at 10:12:20 UTC, Iain Buclaw wrote:How readily does Mingw support pthreads? What may be needed is simply a new package for mingw-specific system functions (ie: core.sys.mingw.pthread). In the meantime, you should be able to get around by passing --enable-threads=win32 when configuring teh gcc build.But how is defined this value ? Is it based on uname function ? Can I trace the value of version when compiling the file ?What versions that are internally defined depend on the target triplet passed when configuring the build. This directly affects what target headers are included, which contain macros we call when initializing the compiler. It has been requested even in DMD to have some command-line option to print all turned on version conditions set by the compiler. Itself is a very trivial addition. In the absence of that though, in a debugger you can set a breakpoint on VersionCondition::addPredefinedGlobalIdent. Iain.How readily does Mingw support pthreads? What may be needed is simply a new package for mingw-specific system functions (ie: core.sys.mingw.pthread). In the meantime, you should be able to get around by passing --enable-threads=win32 when configuring teh gcc build.winpthread is defined here: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/master/tree/mingw-w64-libraries/winpthreads/include/ So winpthread is just a normal pthread.h include file and mingw compiler include a corresponding library. Thanks for the information about --enable-threads=win32 and that's what I used to do but now I don't want to use that kind of trick and I would like to generate a posix version because as long as it doesn't support it, it will never be considered as a valid package from msys2/mingw-w64 maintainers.
Mar 24 2016
Am Thu, 24 Mar 2016 11:12:20 +0100 schrieb "Iain Buclaw via D.gnu" <d.gnu puremagic.com>:On 24 March 2016 at 10:52, Vincent R via D.gnu <d.gnu puremagic.com> wrote:It supports most of the pthreads API IIRC but there are sometimes complaints about performance of the wrapper code. It basically only wraps windows primitives to the pthread API. It's even legal to mix winpthreads and windows API code. Some new C++11 (or 14) features are only supported by libstdc++ with the pthread API. This is the main reason why MinGW compilers use the winpthread code, nobody has written a native std::thread implementation for MinGW yet. However, I think for D the best solution is to always use the windows API. Upstream druntime takes care of this anyway so this means less work for us.Hi, I would like to generate gdc compiler on msys2/mingw-w64 platform. The build system on msys2 uses archlinux build architecture and the build file is here: https://github.com/vrichomme/MINGW-packages/blob/newfastrelease/mingw-w64-gcc/PKGBUILD [...]How readily does Mingw support pthreads? What may be needed is simply a new package for mingw-specific system functions (ie: core.sys.mingw.pthread). In the meantime, you should be able to get around by passing --enable-threads=win32 when configuring teh gcc build.And the details are kinda complicated. In this specific case though, the interesting part is this: * --enable-threads=X sets the GNU_Thread_Model=X variable in gcc.config * For windows / posix, we use version(MinGW) or version(Windows) / version(Posix). So in this case you end up with the combinations: version (Windows) { import gcc.config; static if(GNU_Thread_Model == ThreadModel.Posix) //winpthreads else static if(GNU_Thread_Model == ThreadModel.Win32) //native } version(Posix) is not set and should not be set! I also had another look at MinGW-W64 some weeks ago but then got distracted :-) Here's a quick patch to get winpthreads working: https://paste.gnome.org/pl3pqzcke This only hooks up the gthreads API with winpthreads. But when implementing this quick workaround, I realized there are two open questions: * We could probably use the standard windows gthread implementation, even if the GCC model is set to pthreads. AFAICS the only place we use gthreads is in rt.monitor for the class monitors and it doesn't matter if we use the windows API directly or the wrapper. We could probably even merge gcc/gthreads into the rt module, much of gcc.gthreads isn't even used anymore. * This patch doesn't expose the fact that a pthread emulation layer is available to user code (i.e. you can't use the core.sys.posix.* pthread modules). I don't think we need to expose this as we have cross-platform wrappers in core.thread/core.sync anyway. So the main question is: can we simply ditch gcc.gthreads and inline the code into rt.monitor?But how is defined this value ? Is it based on uname function ? Can I trace the value of version when compiling the file ?What versions that are internally defined depend on the target triplet passed when configuring the build. This directly affects what target headers are included, which contain macros we call when initializing the compiler.
Mar 24 2016
On Thursday, 24 March 2016 at 13:07:00 UTC, Johannes Pfau wrote:I also had another look at MinGW-W64 some weeks ago but then got distracted :-) Here's a quick patch to get winpthreads working: https://paste.gnome.org/pl3pqzcke This only hooks up the gthreads API with winpthreads. But when implementing this quick workaround, I realized there are two open questions: * We could probably use the standard windows gthread implementation, even if the GCC model is set to pthreads. AFAICS the only place we use gthreads is in rt.monitor for the class monitors and it doesn't matter if we use the windows API directly or the wrapper. We could probably even merge gcc/gthreads into the rt module, much of gcc.gthreads isn't even used anymore. * This patch doesn't expose the fact that a pthread emulation layer is available to user code (i.e. you can't use the core.sys.posix.* pthread modules). I don't think we need to expose this as we have cross-platform wrappers in core.thread/core.sync anyway. So the main question is: can we simply ditch gcc.gthreads and inline the code into rt.monitor?Thanks for all this information and it seems your knowledge is far beyond mine so I cannot answer your last question but from what you explain maybe it could be a better option. What I can tel you for now is your patch works and allowed me to generate a gdc compilier. Now I let you decide what is the best option.
Mar 25 2016
Am Fri, 25 Mar 2016 09:15:06 +0000 schrieb Vincent R <lol dlang.org>:Great! We've simply removed gthreads now: https://github.com/D-Programming-GDC/GDC/commit/096991b5817ac21ff6c81ea3759ad4a253deff59 It might take a few days till this is merged into the gdc-X branches though. BTW: When I tested MinGW I found a serious garbage collector bug: TLS variables are currently not scanned correctly. This means if you have GC objects only referenced from thread local memory these might get collected. I know how to fix this but it's not simple so it will take some time. If you run into this bug you could use DMD or LDC instead (IIRC you ran into an optlink bug when using DMD, but you could probably use the -mscoff32 or m64 options to avoid optlink).So the main question is: can we simply ditch gcc.gthreads and inline the code into rt.monitor?Thanks for all this information and it seems your knowledge is far beyond mine so I cannot answer your last question but from what you explain maybe it could be a better option. What I can tel you for now is your patch works and allowed me to generate a gdc compilier. Now I let you decide what is the best option.
Mar 25 2016
On Friday, 25 March 2016 at 10:11:46 UTC, Johannes Pfau wrote:Am Fri, 25 Mar 2016 09:15:06 +0000 schrieb Vincent R <lol dlang.org>:I have some linker errors when I try to use the gdc compiler : Start a mingw64 shell, remove wxwidgets3 if installed, generate wxwidgets28 and installs it: $ pacman -R wxwidgets $ export MINGW_INSTALLS=mingw64 $ git clone https://github.com/vrichomme/MINGW-packages.git $ cd mingw-w64-wxwidgets28 $ makepkg-mingw -sLf $ pacman -U mingw-w64-x86_64-wxWidgets-2.8.12-1-any.pkg.tar.xz $ git clone https://github.com/vrichomme/wxd.git $ cd src $ make $ $ cd Samples/ $ make make -C Controls make[1]: Entering directory '/home/19013692/tmp/wxd/src/Samples/Controls' make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/home/19013692/tmp/wxd/src/Samples/Controls' make -C Dialogs make[1]: Entering directory '/home/19013692/tmp/wxd/src/Samples/Dialogs' gdc -o ../../bin/Dialogs Dialogs.o -L../.. -lwxd -lwxc `wx-config --libs` -lstdc++ -lws2_32 -lwsock32 C:/DEV/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/5.3.0/libgphobos2.a(socket.o): In function `_D3std6socket8Protocol17getProtocolByNameMFxAaZb': C:\DEV\msys64\home\19013692\tmp2\MINGW-packages\mingw-w64-gcc\src\build-x86_64-w64-mingw32\x86_64-w64-mingw32\libphobos\src/../../../../gcc-5.3.0/libphobos/s c/std/socket.d:461: undefined reference to `getprotobyname' C:/DEV/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/5.3.0/libgphobos2.a(socket.o): In function `_D3std6socket8Protocol17getProtocolByTypeMFE3std6socket12ProtocolTypeZb': C:\DEV\msys64\home\19013692\tmp2\MINGW-packages\mingw-w64-gcc\src\build-x86_64-w64-mingw32\x86_64-w64-mingw32\libphobos\src/../../../../gcc-5.3.0/libphobos/s c/std/socket.d:474: undefined reference to `getprotobynumber' ... As you can see all the missing symbols are from the socket library and that's why I have added locally -lws2_32 -lwsock32 to the config.gcc but it doesn't solve the problem...Great! We've simply removed gthreads now: https://github.com/D-Programming-GDC/GDC/commit/096991b5817ac21ff6c81ea3759ad4a253deff59 It might take a few days till this is merged into the gdc-X branches though. BTW: When I tested MinGW I found a serious garbage collector bug: TLS variables are currently not scanned correctly. This means if you have GC objects only referenced from thread local memory these might get collected. I know how to fix this but it's not simple so it will take some time. If you run into this bug you could use DMD or LDC instead (IIRC you ran into an optlink bug when using DMD, but you could probably use the -mscoff32 or m64 options to avoid optlink).So the main question is: can we simply ditch gcc.gthreads and inline the code into rt.monitor?Thanks for all this information and it seems your knowledge is far beyond mine so I cannot answer your last question but from what you explain maybe it could be a better option. What I can tel you for now is your patch works and allowed me to generate a gdc compilier. Now I let you decide what is the best option.
Mar 30 2016
Am Wed, 30 Mar 2016 08:27:27 +0000 schrieb Vincent R <lol dlang.org>:make[1]: Entering directory '/home/19013692/tmp/wxd/src/Samples/Controls' make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/home/19013692/tmp/wxd/src/Samples/Controls' make -C Dialogs make[1]: Entering directory '/home/19013692/tmp/wxd/src/Samples/Dialogs' gdc -o ../../bin/Dialogs Dialogs.o -L../.. -lwxd -lwxc `wx-config --libs` -lstdc++ -lws2_32 -lwsock32 C:/DEV/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/5.3.0/libgphobos2.a(socket.o): In function `_D3std6socket8Protocol17getProtocolByNameMFxAaZb': C:\DEV\msys64\home\19013692\tmp2\MINGW-packages\mingw-w64-gcc\src\build-x86_64-w64-mingw32\x86_64-w64-mingw32\libphobos\src/../../../../gcc-5.3.0/libphobos/src/std/socket.d:461: undefined reference to `getprotobyname' C:/DEV/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/5.3.0/libgphobos2.a(socket.o): In function `_D3std6socket8Protocol17getProtocolByTypeMFE3std6socket12ProtocolTypeZb': C:\DEV\msys64\home\19013692\tmp2\MINGW-packages\mingw-w64-gcc\src\build-x86_64-w64-mingw32\x86_64-w64-mingw32\libphobos\src/../../../../gcc-5.3.0/libphobos/src/std/socket.d:474: undefined reference to `getprotobynumber' ... As you can see all the missing symbols are from the socket library and that's why I have added locally -lws2_32 -lwsock32 to the config.gcc but it doesn't solve the problem...Known problem, I'll probably push a fix to master for this soon. Linking -lws2_32 -lwsock32 is the correct workaround, but the libraries must be specified in the correct order regarding libphobos IIRC. Try gdc -nophoboslib -lws2_32 -lwsock32 -lgphobos2 or gdc -nophoboslib -lgphobos2 -lws2_32 -lwsock32 One of these should work ;-)
Mar 30 2016
On Wednesday, 30 March 2016 at 11:44:41 UTC, Johannes Pfau wrote:Am Wed, 30 Mar 2016 08:27:27 +0000 schrieb Vincent R <lol dlang.org>:Thanks the second option works.[...]Known problem, I'll probably push a fix to master for this soon. Linking -lws2_32 -lwsock32 is the correct workaround, but the libraries must be specified in the correct order regarding libphobos IIRC. Try gdc -nophoboslib -lws2_32 -lwsock32 -lgphobos2 or gdc -nophoboslib -lgphobos2 -lws2_32 -lwsock32 One of these should work ;-)
Mar 30 2016
On Wednesday, 30 March 2016 at 12:05:27 UTC, Vincent R wrote:On Wednesday, 30 March 2016 at 11:44:41 UTC, Johannes Pfau wrote:Is there any progress for mingw ? Have you fixed what you wanted to about tls , link order, ... ? I am asking because as long as it's not done it cannot be push upstream and cannot be tested.Am Wed, 30 Mar 2016 08:27:27 +0000 schrieb Vincent R <lol dlang.org>:Thanks the second option works.[...]Known problem, I'll probably push a fix to master for this soon. Linking -lws2_32 -lwsock32 is the correct workaround, but the libraries must be specified in the correct order regarding libphobos IIRC. Try gdc -nophoboslib -lws2_32 -lwsock32 -lgphobos2 or gdc -nophoboslib -lgphobos2 -lws2_32 -lwsock32 One of these should work ;-)
May 11 2016
Am Wed, 11 May 2016 09:26:12 +0000 schrieb Vincent R <lol dlang.org>:On Wednesday, 30 March 2016 at 12:05:27 UTC, Vincent R wrote:I posted a fix for the link order issue at https://github.com/D-Programming-GDC/GDC/pull/181 (and https://github.com/D-Programming-GDC/GDC/pull/182 ). We're still looking for a better solution though, so it hasn't been merged yet. I'll push a fix for the TLS issue once we have finished the shared library support (as these problems are related). The next step for shared library support is pulling https://github.com/D-Programming-GDC/GDC/pull/192 BTW: We updated to a new druntime/phobos version (2.067) which broke some more MinGW stuff. So there's some work to do...On Wednesday, 30 March 2016 at 11:44:41 UTC, Johannes Pfau wrote:Is there any progress for mingw ? Have you fixed what you wanted to about tls , link order, ... ? I am asking because as long as it's not done it cannot be push upstream and cannot be tested.Am Wed, 30 Mar 2016 08:27:27 +0000 schrieb Vincent R <lol dlang.org>:Thanks the second option works.[...]Known problem, I'll probably push a fix to master for this soon. Linking -lws2_32 -lwsock32 is the correct workaround, but the libraries must be specified in the correct order regarding libphobos IIRC. Try gdc -nophoboslib -lws2_32 -lwsock32 -lgphobos2 or gdc -nophoboslib -lgphobos2 -lws2_32 -lwsock32 One of these should work ;-)
May 12 2016