www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Minimal druntime?

reply Ethan <gooberman gmail.com> writes:
This is another thing I see in hushed tones around here. So all 
in one place, the information goes.

Here's the short story for my plans for the runtime component of 
Smithy:

extern( C++ ) all the things.

Shipping Quantum Break showed me one thing: Certification 
processes will stop you dead in your tracks if any given platform 
holder's code analysis tools can't understand what your code is 
doing.

This slots neatly in with my DMD/LDC on mobile post. If I extern( 
C++ ) all the things, and avoid core and std, then I should be 
able to write code that doesn't even need a heavy druntime 
initialisation. Perhaps not even an init at all. I just want to 
write code at that point that doesn't use typeinfo, moduleinfo, 
etc. Binderoo won't be linked for example, I intend on making a 
library that for all intents and purposes looks like it was 
written in C++ to every compiler in use on the platforms I'll be 
targeting.

Making the compiler not link phobos should be easy. But the 
compiler expects to find druntime so it can include all its 
little hooks for arrays etc.

So. Is there a working minimal druntime out there? Or a process 
that can be applied for any given DMD release?
Jul 27 2019
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, July 27, 2019 6:51:23 AM MDT Ethan via Digitalmars-d wrote:

 This is another thing I see in hushed tones around here. So all
 in one place, the information goes.

 Here's the short story for my plans for the runtime component of
 Smithy:

 extern( C++ ) all the things.

 Shipping Quantum Break showed me one thing: Certification
 processes will stop you dead in your tracks if any given platform
 holder's code analysis tools can't understand what your code is
 doing.

 This slots neatly in with my DMD/LDC on mobile post. If I extern(
 C++ ) all the things, and avoid core and std, then I should be
 able to write code that doesn't even need a heavy druntime
 initialisation. Perhaps not even an init at all. I just want to
 write code at that point that doesn't use typeinfo, moduleinfo,
 etc. Binderoo won't be linked for example, I intend on making a
 library that for all intents and purposes looks like it was
 written in C++ to every compiler in use on the platforms I'll be
 targeting.

 Making the compiler not link phobos should be easy. But the
 compiler expects to find druntime so it can include all its
 little hooks for arrays etc.

 So. Is there a working minimal druntime out there? Or a process
 that can be applied for any given DMD release?


It sounds to me like what you want is -betterC.

- Jonathan M Davis
Jul 27 2019
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 28/07/2019 1:11 AM, Jonathan M Davis wrote:

 It sounds to me like what you want is -betterC.
 
 - Jonathan M Davis



Indeed.

E.g.

extern(C) void main() {
     Foo foo = create!Foo;
     foo.func();
}

extern(C++) class Foo {
     int x;
     float y;

     this() {
     }

     this(int x) {
     	this.x = x;
     }

     void func() {
         import core.stdc.stdio : printf;

         printf("Hello!\n");
         printf("x = %d\n", x);
         printf("y = %f\n", y);
     }
}

T create(T, Args...)(Args args) {
     import core.stdc.stdlib : malloc;

     __gshared T Default = new T;
     enum Size = __traits(classInstanceSize, T);

     void* raw = cast(void*)Default;

     void* newRaw = malloc(Size);
     newRaw[0 .. Size] = raw[0 .. Size];

     T ret = cast(T)newRaw;

     static if (__traits(hasMember, T, "__ctor")) {
    	ret.__ctor(args);
     }

     return ret;
}
Jul 27 2019
prev sibling parent reply Ethan <gooberman gmail.com> writes:
On Saturday, 27 July 2019 at 13:11:23 UTC, Jonathan M Davis wrote:

 It sounds to me like what you want is -betterC.


It's kind of not though.

I don't want a subset of D, I want DMD to not inject things in to 
my library that I just plain won't use.

I'd already considered -betterC, and immediately decided it was 
going to be too much hassle after writing one line and finding 
out the compiler doesn't agree with my coding style. The code is 
going to compile with and without -betterC, which itself is a 
massive undertaking at this point.

I've got a related point about the Unity guys writing High 

This exact argument is what I have about -betterC. If D is so 
good, why do I need a subset of it?
Jul 27 2019
next sibling parent Kagamin <spam here.lot> writes:
I use this: http://dpaste.com/13HWHT0
This is posix version for ldc 1.4, no -betterC, not sure if it 
compiles with -betterC. Runtime was compiler specific until 
recently, not sure if it was fixed.
Jul 27 2019
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, July 27, 2019 8:20:52 AM MDT Ethan via Digitalmars-d wrote:

 On Saturday, 27 July 2019 at 13:11:23 UTC, Jonathan M Davis wrote:

 It sounds to me like what you want is -betterC.


 It's kind of not though.

 I don't want a subset of D, I want DMD to not inject things in to
 my library that I just plain won't use.

 I'd already considered -betterC, and immediately decided it was
 going to be too much hassle after writing one line and finding
 out the compiler doesn't agree with my coding style. The code is
 going to compile with and without -betterC, which itself is a
 massive undertaking at this point.

 I've got a related point about the Unity guys writing High

 This exact argument is what I have about -betterC. If D is so
 good, why do I need a subset of it?


Personally, I'd pretty much only use -betterC to help port a C/C++ library
or program to D. I sure wouldn't want to use a stripped down version of D
normally. But if you want to avoid stuff like TypeInfo, or you want your
library to just look like it's basically C/C++, then that's the direction
you're going. To fully use D's features, you need druntime with everything
that it does. In theory, druntime could be improved to be more pay-as-you-go
than it is, and work is slowly being done in that area, but what druntime
does really does provide language functionality, and trying to avoid parts
of it basically means avoiding language functionality. It's not like D's
features all come for free.

I actually think that -betterC is mostly a waste of time, because I don't
see enough benefit in using a stripped down version of D over just using C++
to bother, but it definitely can have benefits when porting code, and some
people seem to think that it's the way to go, much as I really don't
understand that.

However, D _is_ a general purpose programming language, and different people
have different constraints that they operate under. For the vast majority of
programs, what druntime does isn't a big deal at all. Complaints over stuff
like D using a GC are mostly unreasonable. But there are domains where it's
more problematic than is normally the case (games being one of those,
because they typically can't afford the delays from the stop-the-world GC),
and when you're in that kind of environment, you do potentially need to
avoid aspects of D (and would presumably have to do something similar with

around some of the downsides rather than just using the language like most
people would (e.g. having a separate thread that the GC has no control over
to deal with real-time stuff or specifically disabling the GC in certain
sections of code). And in general, when having multiple languages interact,
things can get a bit entertaining and potentially require avoiding aspects
of D.

- Jonathan M Davis
Jul 27 2019
next sibling parent IGotD- <nise nise.com> writes:
On Saturday, 27 July 2019 at 17:57:07 UTC, Jonathan M Davis wrote:

 On Saturday, July 27, 2019 8:20:52 AM MDT Ethan via 
 Digitalmars-d wrote:

 On Saturday, 27 July 2019 at 13:11:23 UTC, Jonathan M Davis 
 wrote:

 It sounds to me like what you want is -betterC.




 Personally, I'd pretty much only use -betterC to help port a 
 C/C++ library or program to D. I sure wouldn't want to use a 
 stripped down version of D normally. But if you want to avoid 
 stuff like TypeInfo, or you want your library to just look like 
 it's basically C/C++, then that's the direction you're going. 
 To fully use D's features, you need druntime with everything 
 that it does. In theory, druntime could be improved to be more 
 pay-as-you-go than it is, and work is slowly being done in that 
 area, but what druntime does really does provide language 
 functionality, and trying to avoid parts of it basically means 
 avoiding language functionality. It's not like D's features all 
 come for free.

 I actually think that -betterC is mostly a waste of time, 
 because I don't see enough benefit in using a stripped down 
 version of D over just using C++ to bother, but it definitely 
 can have benefits when porting code, and some people seem to 
 think that it's the way to go, much as I really don't 
 understand that.

 - Jonathan M Davis


I agree, why limit the D language. Always viewed -betterC as an 
intermediate step until the "pay for what you use" implementation 
is in place. For most embedded systems type/module info and so on 
isn't really much of a problem other those really memory limited 
targets. I regard the type/module info a great help actually even 
for bare metal targets.

What is the problem is the dependency to an operating system and 
how to make this more flexible. Hopefully in the future we can 
decouple the runtime language features (like classes) which 
doesn't really need OS support from things in the runtime that 
needs OS support.
Jul 27 2019
prev sibling parent JN <666total wp.pl> writes:
On Saturday, 27 July 2019 at 17:57:07 UTC, Jonathan M Davis wrote:

 I actually think that -betterC is mostly a waste of time, 
 because I don't see enough benefit in using a stripped down 
 version of D over just using C++ to bother, but it definitely 
 can have benefits when porting code, and some people seem to 
 think that it's the way to go, much as I really don't 
 understand that.


I think the main appeal of betterC right now is that it's very 
portable and offers new opportunities. Want to do web with D? 
Only BetterC can compile to WebAssembly at the moment. Want to do 
embedded? Usually only BetterC will be supported. Want to do 
mobile? In many cases it's betterC only.
Jul 28 2019
prev sibling next sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Saturday, 27 July 2019 at 14:20:52 UTC, Ethan wrote:

 On Saturday, 27 July 2019 at 13:11:23 UTC, Jonathan M Davis 
 wrote:

 It sounds to me like what you want is -betterC.


 It's kind of not though.

 I don't want a subset of D, I want DMD to not inject things in 
 to my library that I just plain won't use.

 I'd already considered -betterC, and immediately decided it was 
 going to be too much hassle after writing one line and finding 
 out the compiler doesn't agree with my coding style. The code 
 is going to compile with and without -betterC, which itself is 
 a massive undertaking at this point.

 I've got a related point about the Unity guys writing High 

 it? This exact argument is what I have about -betterC. If D is 
 so good, why do I need a subset of it?


For the same reason that C++ needs a C like subset when you want 
to extract the ultimate performance out of it.

Magical compiler optimizers able to generate the perfect Assembly 
and cache usage no matter what comes to the mind of the lazy 
developer are still a dream pipe.

Any programming language that cares about performance needs a 
performance oriented subset, for the use cases when the high 
level constructors don't deliver the desired performance and the 
optimizer just throws its hands up in the air.
Jul 28 2019
parent reply Ethan <gooberman gmail.com> writes:
On Sunday, 28 July 2019 at 07:32:37 UTC, Paulo Pinto wrote:

 For the same reason that C++ needs a C like subset when you 
 want to extract the ultimate performance out of it.


...are you talking about extern "C" there? Because that's not at 
all how extern "C" works in a C++ compiler.




was more to do with making sure engine logic and user-facing 
logic was written in the same language.

But it's also not really valid in relation to -betterC, which - 
as has been mentioned a few times in this thread - is more of a 
stand-in for "pay what you use of the D runtime" in lieu of DMD 
actually being able to do that. You don't need -betterC to  nogc 
and do completely manual memory management for example.
Jul 28 2019
parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Sunday, 28 July 2019 at 11:07:51 UTC, Ethan wrote:

 On Sunday, 28 July 2019 at 07:32:37 UTC, Paulo Pinto wrote:

 For the same reason that C++ needs a C like subset when you 
 want to extract the ultimate performance out of it.


 ...are you talking about extern "C" there? Because that's not 
 at all how extern "C" works in a C++ compiler.




 was more to do with making sure engine logic and user-facing 
 logic was written in the same language.

 But it's also not really valid in relation to -betterC, which - 
 as has been mentioned a few times in this thread - is more of a 
 stand-in for "pay what you use of the D runtime" in lieu of DMD 
 actually being able to do that. You don't need -betterC to 
  nogc and do completely manual memory management for example.


I am referring to the ISO C89 subset of the ISO C++98 programming 
language. which is now updated to the ISO C89 + ISO C11 libraries 
as of ISO C++17, and will be updated to some C90 language 
constructs like aggregate initializers in ISO C++20.
Jul 28 2019
parent reply Ethan <gooberman gmail.com> writes:
On Sunday, 28 July 2019 at 12:00:19 UTC, Paulo Pinto wrote:

 I am referring to the ISO C89 subset of the ISO C++98 
 programming language. which is now updated to the ISO C89 + ISO 
 C11 libraries as of ISO C++17, and will be updated to some C90 
 language constructs like aggregate initializers in ISO C++20.


Okay, but you understand that C++ was specifically designed as a 
superset of C, right? It's the exact opposite of my point. C++ 

respective languages.

Your point that C is what you use to write faster code than C++ 
is also misguided. I regularly write much faster code in C++ than 
I ever could in C thanks to heavy use of inlining and templates. 
This is even more true historically on (for example) 32-bit 
Windows systems, where the C ABI was very stack-heavy yet I could 
use things like __fastcall to call functions with parameters in 
registers and avoid the stack entirely.
Jul 28 2019
parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Sunday, 28 July 2019 at 16:13:10 UTC, Ethan wrote:

 On Sunday, 28 July 2019 at 12:00:19 UTC, Paulo Pinto wrote:

 I am referring to the ISO C89 subset of the ISO C++98 
 programming language. which is now updated to the ISO C89 + 
 ISO C11 libraries as of ISO C++17, and will be updated to some 
 C90 language constructs like aggregate initializers in ISO 
 C++20.


 Okay, but you understand that C++ was specifically designed as 
 a superset of C, right? It's the exact opposite of my point. 

 their respective languages.

 Your point that C is what you use to write faster code than C++ 
 is also misguided. I regularly write much faster code in C++ 
 than I ever could in C thanks to heavy use of inlining and 
 templates. This is even more true historically on (for example) 
 32-bit Windows systems, where the C ABI was very stack-heavy 
 yet I could use things like __fastcall to call functions with 
 parameters in registers and avoid the stack entirely.


So here goes another example, disabling exceptions and RTTI, 
which is considered UB as per ISO C++ standard, given that the 
standard assumes those features cannot be turned off and the 
standard library (as per ISO C++ document) does require them 
being present.

Or constraing to subsets like Embedded C++ used by Apple's IO Kit.

Disabling exceptions and RTTI, coupled with an alternative 
library like EASTL, or Embedded C++ existance, do follow the same
Jul 28 2019
parent reply Ethan <gooberman gmail.com> writes:
On Sunday, 28 July 2019 at 17:44:38 UTC, Paulo Pinto wrote:

 So here goes another example, disabling exceptions and RTTI, 
 which is considered UB as per ISO C++ standard, given that the 
 standard assumes those features cannot be turned off and the 
 standard library (as per ISO C++ document) does require them 
 being present.

 Or constraing to subsets like Embedded C++ used by Apple's IO 
 Kit.

 Disabling exceptions and RTTI, coupled with an alternative 
 library like EASTL, or Embedded C++ existance, do follow the 



Exceptions basically cost nothing on x64 these days. Structured 
exception handling hardware support, baybeeeeeee. Leave 
exceptions on. Either way, if you never try/throw/catch then the 
code isn't generated.

RTTI as well. Don't have virtuals in your objects? You won't get 
typeinfo code in your resulting binary. Don't dynamic cast? 
Compiler won't insert dynamic casts.

This is the "Pay as you go" thing we continually mention in here. 
Have all the features. Don't use them? Don't pay for them.


are a restrictive subset that for all intents and purposes is a 
different language made after the fact. Try to use a feature of 
the full language? Nope, not supported. No pay as you go. Just a 
compiler error.

EASTL? It does not follow the "pay as you go" spirit. EASTL is a 
drop-in replacement for STL that followed the API as defined, but 
replaced the awful memory patterns and added extra methods in to 
it. Extra methods that showed up in C++ 11 standard. Why are you 
bringing up EASTL? That thing's ancient history and has known 
bugs that never got fixed. Had to patch some stuff myself many 
years ago for a project I was shipping.
Jul 29 2019
parent Paulo Pinto <pjmlp progtools.org> writes:
On Monday, 29 July 2019 at 15:05:15 UTC, Ethan wrote:

 On Sunday, 28 July 2019 at 17:44:38 UTC, Paulo Pinto wrote:

 So here goes another example, disabling exceptions and RTTI, 
 which is considered UB as per ISO C++ standard, given that the 
 standard assumes those features cannot be turned off and the 
 standard library (as per ISO C++ document) does require them 
 being present.

 Or constraing to subsets like Embedded C++ used by Apple's IO 
 Kit.

 Disabling exceptions and RTTI, coupled with an alternative 
 library like EASTL, or Embedded C++ existance, do follow the 



 Exceptions basically cost nothing on x64 these days. Structured 
 exception handling hardware support, baybeeeeeee. Leave 
 exceptions on. Either way, if you never try/throw/catch then 
 the code isn't generated.

 RTTI as well. Don't have virtuals in your objects? You won't 
 get typeinfo code in your resulting binary. Don't dynamic cast? 
 Compiler won't insert dynamic casts.

 This is the "Pay as you go" thing we continually mention in 
 here. Have all the features. Don't use them? Don't pay for them.


 They are a restrictive subset that for all intents and purposes 
 is a different language made after the fact. Try to use a 
 feature of the full language? Nope, not supported. No pay as 
 you go. Just a compiler error.

 EASTL? It does not follow the "pay as you go" spirit. EASTL is 
 a drop-in replacement for STL that followed the API as defined, 
 but replaced the awful memory patterns and added extra methods 
 in to it. Extra methods that showed up in C++ 11 standard. Why 
 are you bringing up EASTL? That thing's ancient history and has 
 known bugs that never got fixed. Had to patch some stuff myself 
 many years ago for a project I was shipping.


To show that also C++ does not follow some high standard that 
doesn't do language subsets in specific domains.

Good luck having that speech in some C++ communities that use 
every possible way to "write C with C++ compiler", explicitly 
turning off compiler features and staying at an arm's length from 
the standard library.

C++Now, CppCon, NDC and Meeting C++ have plenty of talks how to 
advocate to those communities.
Jul 29 2019
prev sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 27 July 2019 at 14:20:52 UTC, Ethan wrote:


 I've got a related point about the Unity guys writing High 

 it? This exact argument is what I have about -betterC. If D is 
 so good, why do I need a subset of it?


I generally agree and have facetiously referred to betterC as 
worseD.

betterC is a janky feature added to the compiler because it was 
too much work to modify the compiler and runtime properly.  You 
can achieve almost the same think by importing druntime, but not 
link to it; with a few caveats, that's all betterC does.

Consider the following minimal D program:

--- main.d
void main() { }

Compile with `dmd -release main.d` and you get this:
$ size main
    text    data     bss     dec     hex filename
  526835   53168    2672  582675   8e413 main

Compare that with an equivalent C program:
$ size main
    text    data     bss     dec     hex filename
    1274     512       8    1794     702 main

Whatever D is adding to the binary is probably unnecessary.  In 
fact, if you modify the above main.d program as...

--- main.d
extern(C) void _d_dso_registry() {}

extern(C) void main() { }

... and then compile and link separate, you get a much more 
reasonable result:
$ dmd -release -c main.d
$ gcc test.o -o main
$ size main
    text    data     bss     dec     hex filename
    1430     544      16    1990     7c6 main


Seems to be a lot of room for improvement to achieve a 
pay-as-you-go experience.

There are some of us working to improve the compiler-runtime 
interface so, hopefully someday, you'll just be able to opt-in to 
any feature of the language by simply choosing to use it or not.  
I welcome others to join in and help.

Mike
Jul 28 2019
parent Mike Franklin <slavo5150 yahoo.com> writes:
On Monday, 29 July 2019 at 00:30:03 UTC, Mike Franklin wrote:


 can achieve almost the same think by importing druntime, but


Yeah, if only computers could actually think.  Of course, I meant 
thing.


 $ gcc test.o -o main


Should have been `gcc main.o -o main`.
Jul 28 2019
prev sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 27 July 2019 at 12:51:23 UTC, Ethan wrote:


 So. Is there a working minimal druntime out there? Or a process 
 that can be applied for any given DMD release?


This is the most minimal runtime you can use:

object.d
```
module object;
```

That's right, an empty object.d file.  Unfortunately, the 
compiler still requires an object.d file to exist in order to get 
a build.  I've tried a few pull requests to change that, but 
they've all failed scrutiny.

Assuming you use the above minimal runtime, you can create the 
following minimal hello world program on Linux.

main.d
```
private extern(C) void __d_sys_exit(long arg1)
{
     asm
     {
         mov RAX, 60;
         mov RDI, arg1;
         syscall;
     }
}

private long __d_sys_write(long arg1, in void* arg2, long arg3)
{
     long result;

     asm
     {
         mov RAX, 1;
         mov RDI, arg1;
         mov RSI, arg2;
         mov RDX, arg3;
         syscall;
     }

     return result;
}

void write(immutable(char)[] text)
{
     __d_sys_write(2, text.ptr, text.length);
}

private extern(C) void _start()
{
     main();
     __d_sys_exit(0);
}

void main()
{
     write("Hello, World!\n");
}
```

See it in action at https://run.dlang.io/is/Ix22Eo

 From there, the world's your oyster.  Start programming in D, and 
if the compiler or linker can't find something, either copy it 
from druntime, or implement your own.

I suspect that's not much use to you, but it is indeed the most 
"minimal runtime".  The motivation for using a minimal runtime is 
to incrementally port D to a new platform and implement only what 
is needed in a pay-as-you-go fashion.

Some of us are slowly improving the experience, but it still has 
a long way to go.  It's MUCH better than it was a few years ago, 
though.

Mike
Jul 27 2019
parent Ethan <gooberman gmail.com> writes:
On Saturday, 27 July 2019 at 14:06:51 UTC, Mike Franklin wrote:

 This is the most minimal runtime you can use:

 object.d
 ```
 module object;
 ```

 That's right, an empty object.d file.  Unfortunately, the 
 compiler still requires an object.d file to exist in order to 
 get a build.  I've tried a few pull requests to change that, 
 but they've all failed scrutiny.


Cheers. That's still a good start.

I also like the way the compiler is going, by templatising all 
this stuff. Which should mean that when the job is done it's just 
a matter of providing your own templates and customising the code 
gen that way. That'll be a massive help for getting ARC at a 
compiler level instead of a library solution (assuming it's being 
done the way I imagine).
Jul 27 2019