www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Using D without libphobos

reply A. Nicholi <nich caveoforig.in> writes:
Hello,

I am working on a large cross-platform project, which will be 
written primarily in D, interfacing to C as necessary. To get 
finer control over memory safety, binary size, and use of the GC, 
we would like to disclude libphobos as a dependency in lieu of 
our own code. The project is compiled using LDC.

I am not sure if this is possible though, as it seems there are 
certain things in libphobos that are tightly coupled into the D 
runtime. There are several things in the core namespace that 
would be helpful for us (SIMD, C bindings, etc), but I am not 
sure if that is not also part of libphobos along with the other 
namespaces.

How do I remove libphobos as a runtime dependency with ld and 
MSVC’s link.exe? Is it possible to decouple core from other parts 
of the runtime, and if so, how?

Regards,
A. Nicholi
Apr 25 2018
next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/25/18 11:04 PM, A. Nicholi wrote:

 Hello,
 
 I am working on a large cross-platform project, which will be written 
 primarily in D, interfacing to C as necessary. To get finer control over 
 memory safety, binary size, and use of the GC, we would like to disclude 
 libphobos as a dependency in lieu of our own code. The project is 
 compiled using LDC.
 
 I am not sure if this is possible though, as it seems there are certain 
 things in libphobos that are tightly coupled into the D runtime. There 
 are several things in the core namespace that would be helpful for us 
 (SIMD, C bindings, etc), but I am not sure if that is not also part of 
 libphobos along with the other namespaces.
 
 How do I remove libphobos as a runtime dependency with ld and MSVC’s 
 link.exe? Is it possible to decouple core from other parts of the 
 runtime, and if so, how?
 
 Regards,
 A. Nicholi


It sounds like you want to simply link against druntime and not phobos.

In many cases, phobos and druntime are bundled together as a static 
library. If you just want druntime (core and rt) then you probably have 
to build druntime from source.

I'm generally working on dmd, so I don't know how it works for ldc. But 
I'm sure it's quite similar (building druntime with dmd is just one make 
command).

-Steve
Apr 25 2018
prev sibling next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Thursday, 26 April 2018 at 03:04:55 UTC, A. Nicholi wrote:


 I am working on a large cross-platform project, which will be 
 written primarily in D, interfacing to C as necessary. To get 
 finer control over memory safety, binary size, and use of the 
 GC, we would like to disclude libphobos as a dependency in lieu 
 of our own code. The project is compiled using LDC.

 I am not sure if this is possible though, as it seems there are 
 certain things in libphobos that are tightly coupled into the D 
 runtime. There are several things in the core namespace that 
 would be helpful for us (SIMD, C bindings, etc), but I am not 
 sure if that is not also part of libphobos along with the other 
 namespaces.

 How do I remove libphobos as a runtime dependency with ld and 
 MSVC’s link.exe? Is it possible to decouple core from other 
 parts of the runtime, and if so, how?


I suggest reading the following 2 items before digging deeper:
https://dlang.org/blog/2017/08/23/d-as-a-better-c/
https://dlang.org/changelog/2.079.0.html#minimal_runtime

Next you should realize that Phobos and DRuntime are actually 2 
different things that are, unfortunately, often compiled together 
into one "libphobos2" binary when the compiler is released.

Phobos is the standard library:
https://github.com/dlang/phobos

DRuntime contains C language bindings, OS bindings, and some D 
language feature implementations, including the GC.
https://github.com/dlang/druntime

It's quite unfortunate there so much hard coupling between all of 
these components, but that's the way it is.  Phobos depends on 
DRuntime.  DRuntime depends on C language bindings and OS 
bindings.

Compiling the simplest of programs with DMD 
(https://run.dlang.io/is/KMjKsJ) results in the following C 
compiler invocation:
------
cc onlineapp.o -o /tmp/dmd_run3nK4IS -g -m64 -Xlinker -v 
-L/dlang/dmd/linux/bin64/../lib64 -Xlinker --export-dynamic 
-Xlinker -Bstatic -lphobos2 -Xlinker -Bdynamic -lpthread -lm -lrt 
-ldl

Which in turn calls the following linker invocation:
------
GNU gold (GNU Binutils for Ubuntu 2.29.1) 1.14
collect2 version 7.2.0
/usr/bin/ld -plugin 
/usr/lib/gcc/x86_64-linux-gnu/7/liblto_plugin.so 
-plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/7/lto-wrapper 
-plugin-opt=-fresolution=/tmp/ccg3GKxT.res 
-plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s 
-plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc 
-plugin-opt=-pass-through=-lgcc_s --sysroot=/ --build-id 
--eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed 
-dynamic-linker /lib64/ld-linux-x86-64.so.2 -pie -z now -z relro 
-o /tmp/dmd_run3nK4IS 
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o 
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o 
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o 
-L/dlang/dmd/linux/bin64/../lib64 
-L/usr/lib/gcc/x86_64-linux-gnu/7 
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu 
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../../../lib 
-L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu 
-L/usr/lib/../lib -L/dlang/dmd/linux/lib64 
-L/usr/lib/gcc/x86_64-linux-gnu/7/../../.. onlineapp.o -v 
--export-dynamic -Bstatic -lphobos2 -Bdynamic -lpthread -lm -lrt 
-ldl -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc 
--as-needed -lgcc_s --no-as-needed 
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o 
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o

The compiler uses the C compiler (unfortunately again) to do its 
linking; and it becomes evident that D is in some ways a layer on 
top of C.

You can compile some D programs without linking libphobos2, but 
will require separate compilation and linking because the 
compiler itself actually hard-codes the call to the linker 
(actually the C compiler as demonstrated above).  Example 3 at 
https://dlang.org/changelog/2.079.0.html#minimal_runtime 
demonstrates this.

If you use that method, you won't be able to use certain features 
of D that have runtime implementations.  The obvious ones are 
classes, dynamic arrays, and exceptions.

I could go on, but I'd have to make some assumptions about what 
you're really after.  Feel free to ask more specific questions 
and I'll be happy to share what I know (or at least what I think 
I know; sometimes I'm wrong).

Mike
Apr 25 2018
parent reply A. Nicholi <nich caveoforig.in> writes:
On Thursday, 26 April 2018 at 03:53:54 UTC, Mike Franklin wrote:

 I suggest reading the following 2 items before digging deeper:
 https://dlang.org/blog/2017/08/23/d-as-a-better-c/
 https://dlang.org/changelog/2.079.0.html#minimal_runtime


I didn’t know D had begun offering serious decoupling like that. 
With something like this, we may very well be able to avoid 
writing C entirely, at least in-house! Thank you for bringing 
that up.

On Thursday, 26 April 2018 at 03:53:54 UTC, Mike Franklin wrote:

 The compiler uses the C compiler (unfortunately again) to do 
 its linking; and it becomes evident that D is in some ways a 
 layer on top of C.

 You can compile some D programs without linking libphobos2, but 
 will require separate compilation and linking because the 
 compiler itself actually hard-codes the call to the linker 
 (actually the C compiler as demonstrated above).  Example 3 at 
 https://dlang.org/changelog/2.079.0.html#minimal_runtime 
 demonstrates this.

 If you use that method, you won't be able to use certain 
 features of D that have runtime implementations.  The obvious 
 ones are classes, dynamic arrays, and exceptions.

 I could go on, but I'd have to make some assumptions about what 
 you're really after.  Feel free to ask more specific questions 
 and I'll be happy to share what I know (or at least what I 
 think I know; sometimes I'm wrong).

 Mike


So in a way, the D runtime is similar to libstdc++, providing 
implementations of runtime language features. But it is also like 
C++ in that those language features can be avoided, correct? At 
least with the use of minimal D, I mean. This means that as a 
language, there is enough granularity to theoretically provide as 
few or as many features as one desires for their use case, making 
the penning of new C and C++ code redundant? Do I get this right?
Apr 26 2018
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Thursday, 26 April 2018 at 09:24:19 UTC, A. Nicholi wrote:


 So in a way, the D runtime is similar to libstdc++, providing 
 implementations of runtime language features.


I would argue that Phobos is more analogous to libstc++, but 
there are some language features in C++ that are implemented in 
libstc++.  For example, if you try to compile the following code.

---main.cpp
int main()
{
     throw 1;
     return 0;
}

$g++ -c main.cpp
$ld main.o
ld: warning: cannot find entry symbol _start; defaulting to 
00000000004000e8
main.o: In function `main':
main.cpp:(.text+0xa): undefined reference to 
`__cxa_allocate_exception'
main.cpp:(.text+0x1c): undefined reference to `typeinfo for int'
main.cpp:(.text+0x24): undefined reference to `__cxa_throw'

So, yeah those "__cxa_..." functions are very similar to what 
you'll find in Druntim.  D calls them runtime hooks, and you can 
find an unmaintained list of them here:  
https://wiki.dlang.org/Runtime_Hooks

Here's similar experiment in D.  IMO D is actually better than 
C++ in this regard because it emits a compile-time error, instead 
of a linker error, when it can't find something in the runtime 
that it needs.

---object.d
module object;
// this useless object.d file is currently required, though I'm 
trying to
// find a way to get rid of such nonsense.
// See https://github.com/dlang/dmd/pull/7825

---main.d
void main()
{
     throw new Exception("whatever");
}

$dmd -conf= -defaultlib= main.d
main.d(3): Error: Cannot use throw statements because 
object.Throwable was not declared


 But it is also like C++ in that those language features can be 
 avoided, correct?


That was the goal with the "minimal runtime" features released in 
v2.079.  It's still a work in progress though.  See 
https://github.com/dlang/dmd/pull/8204

But, yes, as long as you avoid certain features that require 
runtime implementations in your code, you should still be able to 
write software in D.  And, if I am allowed to have may way, it 
should get even better.


 At least with the use of minimal D, I mean. This means that as 
 a language, there is enough granularity to theoretically 
 provide as few or as many features as one desires for their use 
 case, making the penning of new C and C++ code redundant? Do I 
 get this right?


Not sure if I totally understand that, but it sounds right on.  
The goal with the minimal runtime features is to allow one use D 
on any platform in a pay-as-you-go fashion.  That is, you only 
need to implement what you intend to use.  The primary use case I 
have in mind is using D for resource-constrained 
microcontrollers, though it would also be useful for libraries 
written in D, but intended to be linked in by applications 
written in other languages.  I'm there are also use cases that I 
haven't even thought of.

Mike
Apr 26 2018
prev sibling next sibling parent reply Radu <void null.pt> writes:
On Thursday, 26 April 2018 at 03:04:55 UTC, A. Nicholi wrote:

 Hello,

 I am working on a large cross-platform project, which will be 
 written primarily in D, interfacing to C as necessary. To get 
 finer control over memory safety, binary size, and use of the 
 GC, we would like to disclude libphobos as a dependency in lieu 
 of our own code. The project is compiled using LDC.

 I am not sure if this is possible though, as it seems there are 
 certain things in libphobos that are tightly coupled into the D 
 runtime. There are several things in the core namespace that 
 would be helpful for us (SIMD, C bindings, etc), but I am not 
 sure if that is not also part of libphobos along with the other 
 namespaces.

 How do I remove libphobos as a runtime dependency with ld and 
 MSVC’s link.exe? Is it possible to decouple core from other 
 parts of the runtime, and if so, how?

 Regards,
 A. Nicholi


LDC allows specifying default libs, something like 
`-defaultlib=phobos2-ldc,druntime-ldc`; you can remove phobos 
from the list.

I think you can use any symbol that is compile time only from 
phobos (like std.traits and std.meta), and you can take stuff 
from phobos and adapt it to your own needs. I use the compiler 
explorer (https://d.godbolt.org/) to check the generate code for 
symbols added.

LDC is flexible enough, from my experience, to allow certain 
degree of customization. You can override the default C compiler 
and add your own paths, or you can create a custom C compiler 
driver that overrides defaults. Check the ldc command line spec 
for inspiration https://wiki.dlang.org/Using_LDC

Something to remember though, if you intend to commit to serious 
commercial use, you should be prepared to dig in and fix stuff 
that doesn't work, or add new functionality, or sponsor the 
foundation for support. D and LDC are opensource projects, and 
they have mostly good quality implementation, but they are not 
commercial grade top of the line products.
Apr 25 2018
parent A. Nicholi <nich caveoforig.in> writes:
On Thursday, 26 April 2018 at 06:32:14 UTC, Radu wrote:

 LDC allows specifying default libs, something like 
 `-defaultlib=phobos2-ldc,druntime-ldc`; you can remove phobos 
 from the list.


Alright, that sounds simple enough. Thanks!

On Thursday, 26 April 2018 at 06:32:14 UTC, Radu wrote:

 I think you can use any symbol that is compile time only from 
 phobos (like std.traits and std.meta), and you can take stuff 
 from phobos and adapt it to your own needs. I use the compiler 
 explorer (https://d.godbolt.org/) to check the generate code 
 for symbols added.


Didn’t know Godbolt supported D. Thanks for sharing :)

On Thursday, 26 April 2018 at 06:32:14 UTC, Radu wrote:

 Something to remember though, if you intend to commit to 
 serious commercial use, you should be prepared to dig in and 
 fix stuff that doesn't work, or add new functionality, or 
 sponsor the foundation for support. D and LDC are opensource 
 projects, and they have mostly good quality implementation, but 
 they are not commercial grade top of the line products.


Right. We’re prepared for working around that, especially since 
we’ll be targeting platforms and architectures that LDC boasts 
only fleeting support for.
Apr 26 2018
prev sibling parent reply sarn <sarn theartofmachinery.com> writes:
On Thursday, 26 April 2018 at 03:04:55 UTC, A. Nicholi wrote:

 I am not sure if this is possible though


I think you've got the technical answer already (just don't link 
in phobos2) but I'll add my 2c that Phobosless programming isn't 
just possible but realistically doable.  It's a shame to go 
without Phobos because it's got so much higher-level 
functionality, but if you're okay with writing things from the 
ground up in C-like code, it works.  Most third-party D libraries 
depend on Phobos, though, so it's mostly DIY and using bindings 
to other languages.

Have a look at the standard library here:
https://dlang.org/phobos/index.html

Basically, all the std.* is Phobos, and most of the core.* stuff 
needs linking with the D runtime library. However, there's a 
bunch of bindings to standard C stuff in core.stdc.* that only 
need the C runtime and standard library, and these come in really 
handy when you're not using Phobos.  (There are more 
platform-specific C bindings in core.sys.* that aren't listed on 
that page.)  Also, there are some other things in core.bitop and 
core.atomic (probably more) that don't need linking to druntime.
Apr 26 2018
parent reply A. Nicholi <nich caveoforig.in> writes:
On Friday, 27 April 2018 at 00:21:51 UTC, sarn wrote:

 I think you've got the technical answer already (just don't 
 link in phobos2) but I'll add my 2c that Phobosless programming 
 isn't just possible but realistically doable.  It's a shame to 
 go without Phobos because it's got so much higher-level 
 functionality, but if you're okay with writing things from the 
 ground up in C-like code, it works.  Most third-party D 
 libraries depend on Phobos, though, so it's mostly DIY and 
 using bindings to other languages.


Right, that’s the plan for the program itself. Even without D we 
would be talking to services as directly as possible, and third 
party libraries would come in as C code and be tweaked to work 
in-house. Performance is paramount with this, which is why I was 
so pleased to find out about the ‘-betterC’ switch in 
development, even though we may have gotten by with a minimal D 
runtime. The ecosystem at work here is really something else :)

On Friday, 27 April 2018 at 00:21:51 UTC, sarn wrote:

 Have a look at the standard library here:
 https://dlang.org/phobos/index.html

 Basically, all the std.* is Phobos, and most of the core.* 
 stuff needs linking with the D runtime library. However, 
 there's a bunch of bindings to standard C stuff in core.stdc.* 
 that only need the C runtime and standard library, and these 
 come in really handy when you're not using Phobos.  (There are 
 more platform-specific C bindings in core.sys.* that aren't 
 listed on that page.)  Also, there are some other things in 
 core.bitop and core.atomic (probably more) that don't need 
 linking to druntime.


Right. So there isn’t anything in core.* that would be in 
libphobos, only D runtime? And std.* depends on both, that is 
correct? Just want to be sure there aren’t edge cases or 
exceptions, it would be a handy rule of thumb.
Apr 28 2018
parent sarn <sarn theartofmachinery.com> writes:
On Saturday, 28 April 2018 at 10:36:03 UTC, A. Nicholi wrote:

 Right. So there isn’t anything in core.* that would be in 
 libphobos, only D runtime? And std.* depends on both, that is 
 correct? Just want to be sure there aren’t edge cases or 
 exceptions, it would be a handy rule of thumb.


That's right, and I think it's intended to stay that way.  (I.e., 
you could file a bug report if you somehow found reverse 
dependencies from druntime to phobos2.)

Things from std.* are in the https://github.com/dlang/phobos 
repo, which builds libphobos2, and things from core.* are in 
https://github.com/dlang/druntime, which builds libdruntime.
Apr 28 2018