www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Supporting musl libc

reply Jacob Carlborg <doob me.com> writes:
As an alternative to glibc there's a C standard library called musl [1]. 
This is the C standard library used by ELLCC [2], a cross-compiler based 
on Clang. This cross-compiler makes it very easy to target other 
platforms and can be used as the C compiler when building with LDC.

The issue is that musl doesn't support the functions defined by 
execinfo.h: backtrace, backtrace_symbols_fd and backtrace_symbols, since 
these are glibc extensions. As far as I can see, these functions are 
used in two places in druntime: src/rt/backtrace/dwarf.d [3] and 
src/core/runtime.d [4].

The imports of execinfo is guarded by version(CRuntime_Glibc). I see 
that CRuntime_Glibc is a predefined version identifier defined by the 
compiler on Linux.

I'm not sure how to best handle different C standard libraries when it 
comes to choosing which one to use. Is it best to choose that when 
building the compiler or when building druntime? Or can it be a runtime 
option?

[1] https://www.musl-libc.org
[2] http://ellcc.org
[3] 
https://github.com/dlang/druntime/blob/master/src/rt/backtrace/dwarf.d#L41
[4] 
https://github.com/dlang/druntime/blob/master/src/core/runtime.d#L433-L434

-- 
/Jacob Carlborg
May 17 2016
next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Tuesday, 17 May 2016 at 08:51:01 UTC, Jacob Carlborg wrote:

 I'm not sure how to best handle different C standard libraries 
 when it comes to choosing which one to use. Is it best to 
 choose that when building the compiler or when building 
 druntime? Or can it be a runtime option?


Shouldn't be runtime detected, but a linktime/compile option.
We introduced CRuntime_Glibc specifically to separate libc from 
kernel differences, e.g. to support android.
Have a look at how -mscoff32 and Android are implememted in dmd.
At best defining CRuntime_Glibc should move into dmd.conf, where 
you could also select a different phobos library to link.
May 17 2016
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 17 May 2016 at 09:26:33 UTC, Martin Nowak wrote:

 On Tuesday, 17 May 2016 at 08:51:01 UTC, Jacob Carlborg wrote:

 I'm not sure how to best handle different C standard libraries 
 when it comes to choosing which one to use. Is it best to 
 choose that when building the compiler or when building 
 druntime? Or can it be a runtime option?


 Shouldn't be runtime detected, but a linktime/compile option.


Jacob, when you said "runtime option," you meant when running the 
compiler, right?  That's how I took it: the C library is chosen 
either when building the compiler or running the compiler.  On 
Windows, it's when running the compiler.


 We introduced CRuntime_Glibc specifically to separate libc from 
 kernel differences, e.g. to support android.
 Have a look at how -mscoff32 and Android are implememted in dmd.


Android is not in dmd yet, as I mentioned, you can look at my 
small PR though:

https://github.com/dlang/dmd/pull/3643


 At best defining CRuntime_Glibc should move into dmd.conf, 
 where you could also select a different phobos library to link.


Man, I'm not looking forward to that debate, though I'm not sure 
I care that much about its outcome myself. ;)
May 17 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-05-17 11:32, Joakim wrote:


 Jacob, when you said "runtime option," you meant when running the
 compiler, right?


Exactly.


 That's how I took it: the C library is chosen either
 when building the compiler or running the compiler.  On Windows, it's
 when running the compiler.


 We introduced CRuntime_Glibc specifically to separate libc from kernel
 differences, e.g. to support android.
 Have a look at how -mscoff32 and Android are implememted in dmd.


 Android is not in dmd yet, as I mentioned, you can look at my small PR
 though:

 https://github.com/dlang/dmd/pull/3643


Looks like a "runtime" option. It's not hard to implement, just a 
question of which approach we would like to use.

-- 
/Jacob Carlborg
May 17 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2016-05-17 11:26, Martin Nowak wrote:


 Shouldn't be runtime detected, but a linktime/compile option.
 We introduced CRuntime_Glibc specifically to separate libc from kernel
 differences, e.g. to support android.
 Have a look at how -mscoff32 and Android are implememted in dmd.
 At best defining CRuntime_Glibc should move into dmd.conf, where you
 could also select a different phobos library to link.


In this case when I said "runtime", I meant runtime for the compiler. 
-mscoff32 is a different object format that just happens to use a 
different C runtime as well. But I guess we could add a new flag, 
-cruntime=musl or something.

-- 
/Jacob Carlborg
May 17 2016
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Tuesday, 17 May 2016 at 08:51:01 UTC, Jacob Carlborg wrote:

 As an alternative to glibc there's a C standard library called 
 musl [1]. This is the C standard library used by ELLCC [2], a 
 cross-compiler based on Clang. This cross-compiler makes it 
 very easy to target other platforms and can be used as the C 
 compiler when building with LDC.

 The issue is that musl doesn't support the functions defined by 
 execinfo.h: backtrace, backtrace_symbols_fd and 
 backtrace_symbols, since these are glibc extensions. As far as 
 I can see, these functions are used in two places in druntime: 
 src/rt/backtrace/dwarf.d [3] and src/core/runtime.d [4].

 The imports of execinfo is guarded by version(CRuntime_Glibc). 
 I see that CRuntime_Glibc is a predefined version identifier 
 defined by the compiler on Linux.

 I'm not sure how to best handle different C standard libraries 
 when it comes to choosing which one to use. Is it best to 
 choose that when building the compiler or when building 
 druntime? Or can it be a runtime option?

 [1] https://www.musl-libc.org
 [2] http://ellcc.org
 [3] 
 https://github.com/dlang/druntime/blob/master/src/rt/backtrace/dwarf.d#L41
 [4] 
 https://github.com/dlang/druntime/blob/master/src/core/runtime.d#L433-L434


It is a runtime option on Windows, when choosing between the 
Digital Mars C runtime and the MSVC runtime, which is why those 
are also predefined version identifiers.  The one for Glibc was 
added as an afterthought in that Windows PR, and Bionic was also 
reserved last fall, though my PR to actually define it has been 
in limbo for a year.  That's mostly my fault: I never got back to 
it when I focused on ldc and Android/ARM.

There was a lot of debate among the core team when even the 
Windows CRuntime support was finalized, with Walter wanting to 
minimize such predefined versions and some of the core team 
arguing against it even after it was merged.  Even the iOS 
predefined version took 5 months to get in, because there was 
debate whether Darwin should be allowed as a common version for 
all Apple platforms.

The reason I'm mentioning all this is that adding predefined 
versions is a matter of much debate, but that's how you'll likely 
implement it as a runtime option.  There is no good answer on 
this right now: I suggest you just implement it and worry about 
run-time versus compile-time when you're done.  Since you'll be 
doing the same work either way, ie adding or separating out a 
bunch of version blocks for musl in druntime, do it any way you 
like for now and you can worry about merging that last bit about 
compile-time vs runtime at the end.
May 17 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-05-17 11:27, Joakim wrote:


 It is a runtime option on Windows, when choosing between the Digital
 Mars C runtime and the MSVC runtime, which is why those are also
 predefined version identifiers.  The one for Glibc was added as an
 afterthought in that Windows PR, and Bionic was also reserved last fall,
 though my PR to actually define it has been in limbo for a year.  That's
 mostly my fault: I never got back to it when I focused on ldc and
 Android/ARM.

 There was a lot of debate among the core team when even the Windows
 CRuntime support was finalized, with Walter wanting to minimize such
 predefined versions and some of the core team arguing against it even
 after it was merged.  Even the iOS predefined version took 5 months to
 get in, because there was debate whether Darwin should be allowed as a
 common version for all Apple platforms.

 The reason I'm mentioning all this is that adding predefined versions is
 a matter of much debate, but that's how you'll likely implement it as a
 runtime option.  There is no good answer on this right now: I suggest
 you just implement it and worry about run-time versus compile-time when
 you're done.  Since you'll be doing the same work either way, ie adding
 or separating out a bunch of version blocks for musl in druntime, do it
 any way you like for now and you can worry about merging that last bit
 about compile-time vs runtime at the end.


It depends. One approach would be to add an option to the druntime 
makefile, which would add -version CRuntime_Musl when compiling. In that 
case version(CRuntime_Musl) needs to come before version(CRuntime_Glibc) 
when declaring the imports. It doesn't sound like a good idea, to depend 
on the order of the version statements, but it's still a possibility.

-- 
/Jacob Carlborg
May 17 2016
parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 17 May 2016 at 11:30:33 UTC, Jacob Carlborg wrote:

 On 2016-05-17 11:27, Joakim wrote:


 [...]


 It depends. One approach would be to add an option to the 
 druntime makefile, which would add -version CRuntime_Musl when 
 compiling. In that case version(CRuntime_Musl) needs to come 
 before version(CRuntime_Glibc) when declaring the imports. It 
 doesn't sound like a good idea, to depend on the order of the 
 version statements, but it's still a possibility.


I wouldn't take the existence of CRuntime_Glibc as etched in 
stone, but whatever happens, they will not both be defined at the 
same time, so I don't think you should worry about the order.  
You can modify dmd to replace Glibc with Musl for now, and worry 
about how it'll be defined officially later.
May 17 2016
prev sibling next sibling parent reply Wyatt <wyatt.epp gmail.com> writes:
On Tuesday, 17 May 2016 at 08:51:01 UTC, Jacob Carlborg wrote:

 The issue is that musl doesn't support the functions defined by 
 execinfo.h: backtrace, backtrace_symbols_fd and 
 backtrace_symbols, since these are glibc extensions.


It's worth noting that musl does support a lot of GNU-isms, but 
the debug stuff is probably not on the table.  Here's some 
background on that from Rich Felker himself:

http://www.openwall.com/lists/musl/2015/04/09/3

-Wyatt
May 17 2016
parent Jacob Carlborg <doob me.com> writes:
On 2016-05-17 15:42, Wyatt wrote:


 It's worth noting that musl does support a lot of GNU-isms, but the
 debug stuff is probably not on the table.  Here's some background on
 that from Rich Felker himself:

 http://www.openwall.com/lists/musl/2015/04/09/3


Yeah, I saw that post.

-- 
/Jacob Carlborg
May 17 2016
prev sibling next sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
On Tuesday, 17 May 2016 at 08:51:01 UTC, Jacob Carlborg wrote:

 As an alternative to glibc there's a C standard library called 
 musl [1]. This is the C standard library used by ELLCC [2], a 
 cross-compiler based on Clang. This cross-compiler makes it 
 very easy to target other platforms and can be used as the C 
 compiler when building with LDC.

 The issue is that musl doesn't support the functions defined by 
 execinfo.h: backtrace, backtrace_symbols_fd and 
 backtrace_symbols, since these are glibc extensions. As far as 
 I can see, these functions are used in two places in druntime: 
 src/rt/backtrace/dwarf.d [3] and src/core/runtime.d [4].

 The imports of execinfo is guarded by version(CRuntime_Glibc). 
 I see that CRuntime_Glibc is a predefined version identifier 
 defined by the compiler on Linux.

 I'm not sure how to best handle different C standard libraries 
 when it comes to choosing which one to use. Is it best to 
 choose that when building the compiler or when building 
 druntime? Or can it be a runtime option?

 [1] https://www.musl-libc.org
 [2] http://ellcc.org
 [3] 
 https://github.com/dlang/druntime/blob/master/src/rt/backtrace/dwarf.d#L41
 [4] 
 https://github.com/dlang/druntime/blob/master/src/core/runtime.d#L433-L434


What is the current status? Without support of musl-libc, I can 
not ad support for a Alpine linux distribution. It is a shame 
because they already have go and rust support.
Oct 08 2016
parent reply Jacob Carlborg <doob me.com> writes:
On 2016-10-08 20:47, Daniel Kozak wrote:


 What is the current status? Without support of musl-libc, I can not ad
 support for a Alpine linux distribution. It is a shame because they
 already have go and rust support.


I've not worked at this at all. For my use case it was easier to just 
build in Docker instead.

-- 
/Jacob Carlborg
Oct 09 2016
parent reply Daniel Kozak <kozzi11 gmail.com> writes:
On Sunday, 9 October 2016 at 13:38:33 UTC, Jacob Carlborg wrote:

 On 2016-10-08 20:47, Daniel Kozak wrote:


 What is the current status? Without support of musl-libc, I 
 can not ad
 support for a Alpine linux distribution. It is a shame because 
 they
 already have go and rust support.


 I've not worked at this at all. For my use case it was easier 
 to just build in Docker instead.


I solved this by using libexecinfo library from freebsd
Oct 09 2016
parent reply openwrt <openwrt openwrt.com> writes:
On Sunday, 9 October 2016 at 15:48:49 UTC, Daniel Kozak wrote:

 On Sunday, 9 October 2016 at 13:38:33 UTC, Jacob Carlborg wrote:

 On 2016-10-08 20:47, Daniel Kozak wrote:


 What is the current status? Without support of musl-libc, I 
 can not ad
 support for a Alpine linux distribution. It is a shame 
 because they
 already have go and rust support.


 I've not worked at this at all. For my use case it was easier 
 to just build in Docker instead.


 I solved this by using libexecinfo library from freebsd


openwrt also use musl,  and I can not run my d code on it(unless 
I rebuild everythings with glic).
Oct 10 2016
parent reply Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
Dne 10.10.2016 v 15:27 openwrt via Digitalmars-d napsal(a):


 On Sunday, 9 October 2016 at 15:48:49 UTC, Daniel Kozak wrote:

 On Sunday, 9 October 2016 at 13:38:33 UTC, Jacob Carlborg wrote:

 On 2016-10-08 20:47, Daniel Kozak wrote:


 What is the current status? Without support of musl-libc, I can not ad
 support for a Alpine linux distribution. It is a shame because they
 already have go and rust support.


 I've not worked at this at all. For my use case it was easier to 
 just build in Docker instead.


 I solved this by using libexecinfo library from freebsd


 openwrt also use musl,  and I can not run my d code on it(unless I 
 rebuild everythings with glic).


Yes, even libexecinfo does not fixed all issues, we really need to add 
other C runtimes. In D runtime and phobos there is a lot of places 
which  wrongly use version(linux) instead of CRuntime_Glibc.
Oct 10 2016
parent reply Joakim <dlang joakim.fea.st> writes:
On Monday, 10 October 2016 at 15:19:00 UTC, Daniel Kozak wrote:

 Dne 10.10.2016 v 15:27 openwrt via Digitalmars-d napsal(a):


 On Sunday, 9 October 2016 at 15:48:49 UTC, Daniel Kozak wrote:

 On Sunday, 9 October 2016 at 13:38:33 UTC, Jacob Carlborg 
 wrote:

 On 2016-10-08 20:47, Daniel Kozak wrote:


 What is the current status? Without support of musl-libc, I 
 can not ad support for a Alpine linux distribution. It is a 
 shame because they already have go and rust support.


 I've not worked at this at all. For my use case it was 
 easier to just build in Docker instead.


 I solved this by using libexecinfo library from freebsd


 openwrt also use musl,  and I can not run my d code on 
 it(unless I rebuild everythings with glic).


 Yes, even libexecinfo does not fixed all issues, we really need 
 to add other C runtimes. In D runtime and phobos there is a lot 
 of places which  wrongly use version(linux) instead of 
 CRuntime_Glibc.


Sorry, didn't see your comment till this thread was just bumped.  
I was the one who separated out linux and Glibc in druntime, I 
noted then that some blocks were still mixed:

https://github.com/dlang/druntime/pull/1010#issuecomment-69815792

Also, it's possible others are still adding Glibc stuff as just 
linux, as that's happened a couple times since then, but I don't 
follow it and tell them to change it, as it usually doesn't 
affect me on Android.  If these are affecting you, submit a pull 
for druntime and ping me, I'll review it.
Jun 13 2017
parent reply Joseph Rushton Wakeling <joseph.wakeling sociomantic.com> writes:
On Tuesday, 13 June 2017 at 09:42:43 UTC, Joakim wrote:

 Also, it's possible others are still adding Glibc stuff as just 
 linux, as that's happened a couple times since then, but I 
 don't follow it and tell them to change it, as it usually 
 doesn't affect me on Android.  If these are affecting you, 
 submit a pull for druntime and ping me, I'll review it.


That sounds like something that automated testing ought to be 
able to catch ... ?
Jun 13 2017
parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 13 June 2017 at 10:10:38 UTC, Joseph Rushton Wakeling 
wrote:

 On Tuesday, 13 June 2017 at 09:42:43 UTC, Joakim wrote:

 Also, it's possible others are still adding Glibc stuff as 
 just linux, as that's happened a couple times since then, but 
 I don't follow it and tell them to change it, as it usually 
 doesn't affect me on Android.  If these are affecting you, 
 submit a pull for druntime and ping me, I'll review it.


 That sounds like something that automated testing ought to be 
 able to catch ... ?


Depends how deep you want it go, what did you have in mind?
Jun 13 2017
prev sibling next sibling parent Chad Joan <chadjoan gmail.com> writes:
On Tuesday, 17 May 2016 at 08:51:01 UTC, Jacob Carlborg wrote:

 As an alternative to glibc there's a C standard library called 
 musl [1]. [...]

 The issue is that musl doesn't support the functions defined by 
 execinfo.h: backtrace, backtrace_symbols_fd and 
 backtrace_symbols, since these are glibc extensions. As far as 
 I can see, these functions are used in two places in druntime: 
 src/rt/backtrace/dwarf.d [3] and src/core/runtime.d [4].

 The imports of execinfo is guarded by version(CRuntime_Glibc). 
 I see that CRuntime_Glibc is a predefined version identifier 
 defined by the compiler on Linux.

 [...]


I just ran into these problems while trying to get D running on a 
hardened Gentoo system using musl libc.

Using a patched gdc-4.9.4 I've been able to compile and run a 
simple D "Hello world".  That made me very happy!

Here are the binaries:
http://www.chadjoan.com/d/gdc-4.9.4-hardened-musl.tar.gz

I have forked the dlang overlay in the hope that it can provide 
the necessary information if someone tries to compile the thing 
for themselves:
https://github.com/chadjoan/dlang

This is pretty recent work, so I haven't had time to use it for 
anything besides "Hello world".  I'm also not sure how to run 
regression tests for this thing, if there even are any.  So this 
is all very untested.  I'm sharing a minimally useful milestone, 
and hopefully it is more than that.

Background:

Earlier, I modified the "dmd-2.067.1-r2" ebuild from the dlang 
overlay and actually managed to get it to compile and emit 
executables.  The BSD execinfo did not even manage to run its own 
test program on my system without segfaulting.  So I used 
libbacktrace for backtraces and had to do a bunch of other 
patching to get dmd/druntime/phobos to build on this system; it 
involved a lot of trial-and-error.  Although it emitted 
executables, these executables would segfault or overstep 
security bounds enforced by the kernel (to the point where 
removing all PAX restrictions would not help it).  I remember 
something about text relocations, but at this point it's been a 
while, so my memory of details is lacking.  Diving into codegen 
is far too time-consuming in my situation, so that was a dead-end.

I started over with gdc, because I knew that gcc could already 
generate working executables on the system.  Most of the pain in 
this process involved two things: (1) coercing portage into 
building gdc in its own directory so that it didn't take over my 
system compiler and (2) fixing gcc/gdc build scripts.  It was 
important to use portage and not just manually configure gdc, 
because I wanted to ensure that the resulting gdc build would be 
configured and patched in a way closely matching what my system 
compiler does (and thus having higher success chances).  Trying 
to configure and patch it by hand, thus repeating a lot of hard 
work already done by Gentoo maintainers, is... not practical.  As 
for the build scripts: I have no idea what conventions gcc 
maintainers use for these things, and thus how I might do basic 
things like use an environment variable from outside of the build 
process to affect what flags are passed to a compiler.  There be 
hacks in my patching!  As an example, it will unconditionally 
compile the D frontend with -fPIC; if you end up building this on 
another system and you didn't want that, then it's just too bad 
;)  So it took more trial-and-error, and maybe a little 
compromise, but eventually it got there.

I hope this helps someone.
Jun 12 2017
prev sibling parent Yuxuan Shui <yshuiv7 gmail.com> writes:
On Tuesday, 17 May 2016 at 08:51:01 UTC, Jacob Carlborg wrote:

 As an alternative to glibc there's a C standard library called 
 musl [1]. This is the C standard library used by ELLCC [2], a 
 cross-compiler based on Clang. This cross-compiler makes it 
 very easy to target other platforms and can be used as the C 
 compiler when building with LDC.

 The issue is that musl doesn't support the functions defined by 
 execinfo.h: backtrace, backtrace_symbols_fd and 
 backtrace_symbols, since these are glibc extensions. As far as 
 I can see, these functions are used in two places in druntime: 
 src/rt/backtrace/dwarf.d [3] and src/core/runtime.d [4].

 The imports of execinfo is guarded by version(CRuntime_Glibc). 
 I see that CRuntime_Glibc is a predefined version identifier 
 defined by the compiler on Linux.

 I'm not sure how to best handle different C standard libraries 
 when it comes to choosing which one to use. Is it best to 
 choose that when building the compiler or when building 
 druntime? Or can it be a runtime option?

 [1] https://www.musl-libc.org
 [2] http://ellcc.org
 [3] 
 https://github.com/dlang/druntime/blob/master/src/rt/backtrace/dwarf.d#L41
 [4] 
 https://github.com/dlang/druntime/blob/master/src/core/runtime.d#L433-L434


I tried to build dmd on musl, and it seems to work relatively 
well.

I need to build dmd 2.067 for bootstrapping. It doesn't build 
out-of-box, but there's patches floating around.

There're some missing symbols in druntime: a couple math related, 
backtrace() and backtrace_symbols(). The former ones can be 
workaround, but and proper solution is needed. The latter ones 
can be solved by linking in libbacktrace().
Dec 11 2017