www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Linker-hacking out the D runtime

reply sarn <sarn theartofmachinery.com> writes:
As it stands, the -betterC flag is still immature and only 
removes a bit of the D runtime.  I've been playing around a bit 
to see what could be possible.  To do that, I've had to do some 
linker hacking to make code that's completely free of D runtime 
dependencies.

I thought I'd write something up to help other people experiment 
with this stuff:
https://theartofmachinery.com/2016/12/18/d_without_runtime.html

Let's make a better -betterC :)
Dec 17 2016
next sibling parent Iain Buclaw via Digitalmars-d-announce writes:
On 18 December 2016 at 01:04, sarn via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 As it stands, the -betterC flag is still immature and only removes a bit of
 the D runtime.


-betterC removes module info and module helpers, not the D runtime.
You will find it in gdc with the more appropriately named command line
switch -fno-moduleinfo.  ;-)
Dec 17 2016
prev sibling parent reply Mike <none none.com> writes:
On Sunday, 18 December 2016 at 00:04:54 UTC, sarn wrote:


 I thought I'd write something up to help other people 
 experiment with this stuff:
 https://theartofmachinery.com/2016/12/18/d_without_runtime.html


Thanks for this.

I abandoned D sometime ago largely because of 
https://issues.dlang.org/show_bug.cgi?id=14758 (but there were 
other reasons), so your blog post is interesting to me.  It is 
unfortunate that we have to resort to such hackery, but its nice 
to have such tools at our disposal regardless.

I proposed another idea for giving users more control over D 
Runtime by moving runtime hook definitions to .di header files.  
If you're interested, you can read about it here:  
http://forum.dlang.org/post/psssnzurlzeqeneagora forum.dlang.org. 
  I'd much rather have something like that over a -betterC; you 
can read more about some disadvantages to expanding on -betterC 
(e.g. removing RTTI) here: 
http://forum.dlang.org/post/nevipjrkdqxivoerftlw forum.dlang.org.

I've largely embraced Rust now for its "no runtime" and "no 
dependencies libcore" features (and a few other safety/robustness 
features), but I miss the modeling power and compile-time 
features of D greatly.

Anyway, thanks for the post; it's given me a few ideas.

Mike
Dec 17 2016
next sibling parent sarn <sarn theartofmachinery.com> writes:
On Sunday, 18 December 2016 at 02:37:22 UTC, Mike wrote:

 I abandoned D sometime ago largely because of 
 https://issues.dlang.org/show_bug.cgi?id=14758 (but there were 
 other reasons), so your blog post is interesting to me.  It is 
 unfortunate that we have to resort to such hackery, but its 
 nice to have such tools at our disposal regardless.


Yeah, the TypeInfo spam is the biggest pain point.


 I proposed another idea for giving users more control over D 
 Runtime by moving runtime hook definitions to .di header files.
  If you're interested, you can read about it here:  
 http://forum.dlang.org/post/psssnzurlzeqeneagora forum.dlang.org.
  I'd much rather have something like that over a -betterC; you 
 can read more about some disadvantages to expanding on -betterC 
 (e.g. removing RTTI) here: 
 http://forum.dlang.org/post/nevipjrkdqxivoerftlw forum.dlang.org.


I think D can still be very usable without TypeInfo (especially 
if the unnecessary language dependence on it improves).

But I'm also wary of "solving" the problem with a hundred 
compiler flags and causing fragmentation.


 Anyway, thanks for the post; it's given me a few ideas.


Thanks for letting me know.  I wasn't totally sure anyone would 
be interested in that hack.
Dec 18 2016
prev sibling parent reply Iain Buclaw via Digitalmars-d-announce writes:
On 18 December 2016 at 03:37, Mike via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Sunday, 18 December 2016 at 00:04:54 UTC, sarn wrote:


 I thought I'd write something up to help other people experiment with this
 stuff:
 https://theartofmachinery.com/2016/12/18/d_without_runtime.html



 Thanks for this.

 I abandoned D sometime ago largely because of
 https://issues.dlang.org/show_bug.cgi?id=14758 (but there were other
 reasons), so your blog post is interesting to me.  It is unfortunate that we
 have to resort to such hackery, but its nice to have such tools at our
 disposal regardless.


As a response to my last message in that thread, the changes for
classinfo generation is now ready to go in master.  However I already
have ideas for future implementation that is not so naive when it
comes to looking up fields to assign data.  How does a kind of tagging
system sound?

i.e:

class TypeInfo_Class
{
     name
    string classname;

     init
    byte[] initializer;
}
Dec 18 2016
next sibling parent reply Mike <none none.com> writes:
On Sunday, 18 December 2016 at 12:57:08 UTC, Iain Buclaw wrote:


 As a response to my last message in that thread, the changes 
 for classinfo generation is now ready to go in master.


Ok, I'll give it a test in the next week or so.


 However I already have ideas for future implementation that is 
 not so naive when it comes to looking up fields to assign data. 
  How does a kind of tagging system sound?

 i.e:

 class TypeInfo_Class
 {
      name
     string classname;

      init
     byte[] initializer;
 }


I'm assuming this idea addresses the issue with porting druntime 
and leaving some features unimplemented.  If I understand your 
intention, this looks akin to Rust's Lang Items feature 
(https://doc.rust-lang.org/book/lang-items.html).  Please correct 
me if I'm wrong.

I'm not sure what you're envisioning here, so I'd like to hear 
more.  Would the user get compile-time errors if they used a D 
feature that required a druntime implementation, but the 
implementation did not exist?

Mike
Dec 18 2016
parent reply Iain Buclaw via Digitalmars-d-announce writes:
On 19 December 2016 at 01:36, Mike via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Sunday, 18 December 2016 at 12:57:08 UTC, Iain Buclaw wrote:


 As a response to my last message in that thread, the changes for classinfo
 generation is now ready to go in master.



 Ok, I'll give it a test in the next week or so.


 However I already have ideas for future implementation that is not so
 naive when it comes to looking up fields to assign data.  How does a kind of
 tagging system sound?

 i.e:

 class TypeInfo_Class
 {
      name
     string classname;

      init
     byte[] initializer;
 }



 I'm assuming this idea addresses the issue with porting druntime and leaving
 some features unimplemented.  If I understand your intention, this looks
 akin to Rust's Lang Items feature
 (https://doc.rust-lang.org/book/lang-items.html).  Please correct me if I'm
 wrong.


Heh, I've never looked into Rust, so I didn't know that.  But in a
way, yes, it is quite similar.


 I'm not sure what you're envisioning here, so I'd like to hear more.  Would
 the user get compile-time errors if they used a D feature that required a
 druntime implementation, but the implementation did not exist?

 Mike


The compiler doesn't actually generate any code that peeks inside
TypeInfo.  It only generates the reference to the right typeinfo to
pass to library runtime functions -  or on request via typeid().  It
doesn't actually care about the data stored inside.  And that's the
angle I've taken when laying out the actual data - if you provide the
fields we want to populate, then we'll populate them.  If you omit a
few, then the compiler won't bother with them.  Because at the end of
the day, it's druntime library that uses and makes sense of the
TypeInfo data provided.  The compile just says: "Well, this is as much
as I'm willing to tell you about the type."

Iain
Dec 19 2016
parent reply Mike <none none.com> writes:
On Monday, 19 December 2016 at 19:53:06 UTC, Iain Buclaw wrote:


 The compiler doesn't actually generate any code that peeks 
 inside TypeInfo.  It only generates the reference to the right 
 typeinfo to pass to library runtime functions -  or on request 
 via typeid().  It doesn't actually care about the data stored 
 inside.  And that's the angle I've taken when laying out the 
 actual data - if you provide the fields we want to populate, 
 then we'll populate them.  If you omit a few, then the compiler 
 won't bother with them.  Because at the end of the day, it's 
 druntime library that uses and makes sense of the TypeInfo data 
 provided.  The compile just says: "Well, this is as much as I'm 
 willing to tell you about the type."


Ok, that's interesting, but what if you don't want TypeInfo at 
all?  Can you omit the TypeInfo structure entirely from object.d? 
  Or perhaps you still need to declare it in object.d, but since 
the compiler doesn't find any fields to populate, it results in 
an empty struct?  I'd really hate to have to add empty 
TypeInfo_XXX classes to my object.d though.

I'm still wondering about what the programmer would see if they 
tried to do a dyamic cast (for example) and there was no (or an 
incomplete) TypeInfo in the runtime.

You see, when I started with D, I had a grand plan that I could 
create a product like Arduino. I would provide a PCB and an 
accompanying API/SDK to my customers, and they would do the 
programming.  I would have wanted them to have a polished 
experience when programming my product, so if they were doing 
something that wasn't supported by the platform, they would 
receive a friendly compiler message telling them so.

I thought that this would be possible if the compiler resolved 
druntime symbols at compile-time in the same way it does any 
other library: by reading .di header files.   If a symbol was not 
found, they would get a compiler error telling them so.  I could 
also add templates, static-ifs, and static asserts to the .di 
files for, not only generating highly optimized code, but also to 
notify the programmer with more helpful and precise compile-time 
messages.

I also envisioned products with very high resource constraints 
(like ARM Cortex-M0) without threading, but for high performance 
microcontrollers (like ARM Cortex-M4/7) I could leverage D's 
built-in understanding of thead as my RTOS.  Each product would, 
therefore, have very different druntime implementations.

I think what you're proposing here by tagging fields is certainly 
better than what we currently have, and would be desirable for 
other D users (especially those of late), but does it scale well 
if I want to deliver a polished programming experience (with 
optimized code generation) to my customers when my platform only 
has a subset of D's rich feature set?

Mike
Dec 20 2016
parent reply Iain Buclaw via Digitalmars-d-announce writes:
On 20 December 2016 at 12:24, Mike via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Monday, 19 December 2016 at 19:53:06 UTC, Iain Buclaw wrote:


 The compiler doesn't actually generate any code that peeks inside
 TypeInfo.  It only generates the reference to the right typeinfo to pass to
 library runtime functions -  or on request via typeid().  It doesn't
 actually care about the data stored inside.  And that's the angle I've taken
 when laying out the actual data - if you provide the fields we want to
 populate, then we'll populate them.  If you omit a few, then the compiler
 won't bother with them.  Because at the end of the day, it's druntime
 library that uses and makes sense of the TypeInfo data provided.  The
 compile just says: "Well, this is as much as I'm willing to tell you about
 the type."



 Ok, that's interesting, but what if you don't want TypeInfo at all?  Can you
 omit the TypeInfo structure entirely from object.d?  Or perhaps you still
 need to declare it in object.d, but since the compiler doesn't find any
 fields to populate, it results in an empty struct?  I'd really hate to have
 to add empty TypeInfo_XXX classes to my object.d though.


Yeah, there are two logical steps that need ratifying.

1) -fno-rtti should be a flag that is honoured by the compiler.
2) Runtime functions that "lower" to a C function call should not
generate RTTI if all information about the type is known at
compile-time.

There are ideas floating around to improve [2], but for the time being
however, marking all functions as  nogc catches almost all cases where
this happens.
Jan 08 2017
parent reply Mike <none none.com> writes:
On Sunday, 8 January 2017 at 22:14:36 UTC, Iain Buclaw wrote:


 1) -fno-rtti should be a flag that is honoured by the compiler.


The more I think about it the more I dislike the whole idea of 
-fno-rtti.  All I've ever wanted from the D compiler is to not 
put code in my binary that has not chance of ever being used.  
Or, to generate it in a way that can be garbage collected by the 
linker.

If that dream came true, I could avoid RTTI by simply avoiding 
any feature that needed it (REALLY needed it!) and -fno-rtti 
would be of no value, except to enforce policy.
Jan 09 2017
parent sarn <sarn theartofmachinery.com> writes:
On Monday, 9 January 2017 at 13:42:01 UTC, Mike wrote:

 On Sunday, 8 January 2017 at 22:14:36 UTC, Iain Buclaw wrote:


 1) -fno-rtti should be a flag that is honoured by the compiler.


 The more I think about it the more I dislike the whole idea of 
 -fno-rtti.  All I've ever wanted from the D compiler is to not 
 put code in my binary that has not chance of ever being used.  
 Or, to generate it in a way that can be garbage collected by 
 the linker.


I've actually come to the same conclusion.  Different 
combinations of compiler flags for different codebases is just a 
recipe for pain.  We already have things like nothrow, and I'm 
pretty sure we can solve at least most of the problem just by 
exploiting language features to do what you said.
Jan 09 2017
prev sibling parent reply Mike <none none.com> writes:
On Sunday, 18 December 2016 at 12:57:08 UTC, Iain Buclaw wrote:

 On 18 December 2016 at 03:37, Mike via Digitalmars-d-announce 
 <digitalmars-d-announce puremagic.com> wrote:

 On Sunday, 18 December 2016 at 00:04:54 UTC, sarn wrote:


 I thought I'd write something up to help other people 
 experiment with this
 stuff:
 https://theartofmachinery.com/2016/12/18/d_without_runtime.html



 Thanks for this.

 I abandoned D sometime ago largely because of 
 https://issues.dlang.org/show_bug.cgi?id=14758 (but there were 
 other reasons), so your blog post is interesting to me.  It is 
 unfortunate that we have to resort to such hackery, but its 
 nice to have such tools at our disposal regardless.


 As a response to my last message in that thread, the changes 
 for classinfo generation is now ready to go in master.


Just built GDC from the gdc-6 branch.  The commit history says 
master was merged in 24 hours ago, so I'm assuming your ClassInfo 
changes are there.

Anyway the build produces the same result:  TypeInfo strings for 
each an every type in the .rodata section.  The binary should be 
about 6K, but is instead about 600K.  Perhaps I'll try again at a 
later date.

Mike
Dec 19 2016
parent Iain Buclaw via Digitalmars-d-announce writes:
On 19 December 2016 at 12:41, Mike via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Sunday, 18 December 2016 at 12:57:08 UTC, Iain Buclaw wrote:

 On 18 December 2016 at 03:37, Mike via Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:

 On Sunday, 18 December 2016 at 00:04:54 UTC, sarn wrote:


 I thought I'd write something up to help other people experiment with
 this
 stuff:
 https://theartofmachinery.com/2016/12/18/d_without_runtime.html




 Thanks for this.

 I abandoned D sometime ago largely because of
 https://issues.dlang.org/show_bug.cgi?id=14758 (but there were other
 reasons), so your blog post is interesting to me.  It is unfortunate that we
 have to resort to such hackery, but its nice to have such tools at our
 disposal regardless.


 As a response to my last message in that thread, the changes for classinfo
 generation is now ready to go in master.



 Just built GDC from the gdc-6 branch.  The commit history says master was
 merged in 24 hours ago, so I'm assuming your ClassInfo changes are there.

 Anyway the build produces the same result:  TypeInfo strings for each an
 every type in the .rodata section.  The binary should be about 6K, but is
 instead about 600K.  Perhaps I'll try again at a later date.

 Mike


Yeah, I'm just wrapping up the last touches, PR will land for master
in days, or tonight maybe...

May be a delay before it gets merged into the gdc-6 branch.
Dec 19 2016