www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - druntime redesign

reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
Hi!

This post is inspired by: 
https://forum.dlang.org/thread/scklbdrgjiypknuephfj forum.dlang.org

On Saturday, 22 February 2020 at 13:20:40 UTC, IGotD- wrote:

 Will there be a future improvement of this so that OS specific 
 support can be moved to separate files and selected on module 
 level instead? For example.

 void DSpecificLibraryFunction(...)
 {
     OSDependentLibraryFunction(...)
 }

 so that there are predefined OS specific functions that are 
 selected on module level. Also version() switches where 
 everything is consolidated into one file makes it more 
 difficult to merge.


More specific proposal how to achieve this step by step:

Сommunity may agree to add druntime module "core.sys.abstract" 
and everywhere in druntime code what references to a specific 
OSes and architecures (all these thousands of "version(){} else 
version(){}") will add also obtain code like:

version(Abstract_Platform){ core.sys.abstract.some_call(); }

This "core.sys.abstract" code for every exotic uncommon (or just 
"every"?) platform will be placed outside of druntime "src" dir 
and will be used for compilation only if appropriate compiler 
flag "-I" is set and version=Abstract_Platform is defined.

Then some uncommon platforms can be moved to "abstract" 
implementation (for purpose of code coverage for this new 
"abstract subsystem").

And at next step maybe every platforms can be implemented as 
abstract (and "core.sys.abstract" will be renamed into 
"core.sys").

For me it looks like it can be done gradually and at the same 
time not to break anything.

How do you like this idea? Do you see missings/flaws in it?
Feb 24 2020
next sibling parent reply IGotD- <nise nise.com> writes:
On Tuesday, 25 February 2020 at 00:05:05 UTC, Denis Feklushkin 
wrote:

 And at next step maybe every platforms can be implemented as 
 abstract (and "core.sys.abstract" will be renamed into 
 "core.sys").

 For me it looks like it can be done gradually and at the same 
 time not to break anything.

 How do you like this idea? Do you see missings/flaws in it?


This really opens up a second question, with your proposal with 
an abstract interface we can start to reduce the dependency on 
the C library. The C library is just there as baggage in order to 
give basic OS functionality. With an abstract interface we can 
use the OS primitives directly without going through any C 
library. This is a huge task but this offers a gradual approach.

We can have the C library as an option through the abstract 
interface if we still need it.

I see a double benefit of this.
Feb 24 2020
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 25 February 2020 at 00:22:19 UTC, IGotD- wrote:


 This really opens up a second question, with your proposal with 
 an abstract interface we can start to reduce the dependency on 
 the C library. The C library is just there as baggage in order 
 to give basic OS functionality. With an abstract interface we 
 can use the OS primitives directly without going through any C 
 library. This is a huge task but this offers a gradual approach.

 We can have the C library as an option through the abstract 
 interface if we still need it.


Yes!

Perhaps  need to introduce option DRUNTIME_LIBC_AVAILABLE for 
druntime so that we can gradually get rid of all libc's symbols 
like "errno" inside of new abstract implementations by wrapping 
some druntime code pieces into version() branches.
Feb 24 2020
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 25 February 2020 at 01:21:33 UTC, Denis Feklushkin 
wrote:

 More specific proposal how to achieve this step by step:



 Сommunity may agree to add druntime module "core.sys.abstract" 
 and everywhere in druntime code what references to a specific 
 OSes and architecures (all these thousands of "version(){} else 
 version(){}") will add also obtain code like


Probably I seem to know easiest way:

Many modules are scattered by version() keywords that implement 
common calls necessary when using any platform.
For brief example look into: core/sync/event.d

We can improve Makefiles (and cmake in case of LDC fork of 
druntime) to include only needed platform-dependend subdirs what 
will contain these modules but without "versions", just code for 
each supported platform.
(Do not confuse with core.sys.platform_name section!)

(And I couldn’t do it myself right now because I don’t know well 
enough make and cmake systems. I am need "initial sample 
template" to do things like in it.)

After such replacement it will be very easy to clear out 
remaining libc calls (if it will be need - all supported 
platforms actually have libc interfaces), and also support for 
new arbitrary platforms will be ready - for supporting any new 
platform will be enough to replace platform-specific section with 
your own at build stage by build system.

What do you think about this? Is that good idea?
Mar 15 2020
next sibling parent reply IGotD- <nise nise.com> writes:
On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin wrote:

 On Tuesday, 25 February 2020 at 01:21:33 UTC, Denis Feklushkin 
 wrote:

 Probably I seem to know easiest way:

 Many modules are scattered by version() keywords that implement 
 common calls necessary when using any platform.
 For brief example look into: core/sync/event.d

 We can improve Makefiles (and cmake in case of LDC fork of 
 druntime) to include only needed platform-dependend subdirs 
 what will contain these modules but without "versions", just 
 code for each supported platform.
 (Do not confuse with core.sys.platform_name section!)

 (And I couldn’t do it myself right now because I don’t know 
 well enough make and cmake systems. I am need "initial sample 
 template" to do things like in it.)

 After such replacement it will be very easy to clear out 
 remaining libc calls (if it will be need - all supported 
 platforms actually have libc interfaces), and also support for 
 new arbitrary platforms will be ready - for supporting any new 
 platform will be enough to replace platform-specific section 
 with your own at build stage by build system.

 What do you think about this? Is that good idea?


If take src/core/sync/semaphore.d (class Semaphore) for example 
this is an example with a lot of version switches that could be a 
good candidate for an abstract interface. I'm not sure what type 
of abstraction it should be, inherited SemaphoreImpl class, mixin 
member of a name given by OS module, fixed member name?

There is alot of confusion with the OS and clib as OS support 
functionality is mixed together with clib functionality in the 
code. This is very confusing. Instead if an abstract OS want to 
use clib, it should do so separately. For example Windows should 
use Microsoft Clib, Linux could use a variety of clib like glibc, 
musl etc. Clib support should be under OS support and not mixed 
into the generic code.

Another thing I've noticed is how elf code is duplicated for each 
OS. As elf is a standard we don't need to duplicate this and we 
can have a generic solution instead. We should have an ABI for 
this as well.

First I think we need to determine what type of "polymorphism" to 
use ie. how to select the OS specific implementation, then start 
to implement an ABI. Much of the ABI is straight forward I think.

BTW, isn't it time to retire make files and go for cmake only? 
Perhaps OT, I should start a new thread.
Mar 15 2020
parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Sunday, 15 March 2020 at 12:08:54 UTC, IGotD- wrote:

 On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin

 What do you think about this? Is that good idea?


 If take src/core/sync/semaphore.d (class Semaphore) for example 
 this is an example with a lot of version switches that could be 
 a good candidate for an abstract interface. I'm not sure what 
 type of abstraction it should be, inherited SemaphoreImpl 
 class, mixin member of a name given by OS module, fixed member 
 name?


It’s definitely not a class because in this case it is not 
required to provide polymorphism - there is only one platform 
will be presented in result binary. (If someone occasionally 
wants to make a multi-platform binaries it will still be more 
difficult than just several implementations of these base class.)

Just fixed member is preferred, as for me.


 There is alot of confusion with the OS and clib as OS support 
 functionality is mixed together with clib functionality in the 
 code. This is very confusing. Instead if an abstract OS want to 
 use clib, it should do so separately. For example Windows 
 should use Microsoft Clib, Linux could use a variety of clib 
 like glibc, musl etc. Clib support should be under OS support 
 and not mixed into the generic code.


My investigation showed me that there is not so much of this libc 
code. I even think that it will be possible to introduce  
temporary abstract libc version(CRuntime_AbstractLibc) to cover 
this case.


 Another thing I've noticed is how elf code is duplicated for 
 each OS. As elf is a standard we don't need to duplicate this 
 and we can have a generic solution instead. We should have an 
 ABI for this as well.


And, probably, not all platforms need ELF support in druntime.


 First I think we need to determine what type of "polymorphism" 
 to use ie. how to select the OS specific implementation, then 
 start to implement an ABI. Much of the ABI is straight forward 
 I think.


Just improve makefiles to be able select needed subdirectories 
like in
posix.mak/win64.mak and LDC's druntime CMakeLists.txt:


file(GLOB_RECURSE DRUNTIME_D_BIONIC  
${RUNTIME_DIR}/src/core/sys/bionic/*.d)
file(GLOB_RECURSE DRUNTIME_D_DARWIN  
${RUNTIME_DIR}/src/core/sys/darwin/*.d)
file(GLOB_RECURSE DRUNTIME_D_DRAGONFLYBSD 
${RUNTIME_DIR}/src/core/sys/dragonflybsd/*.d)
file(GLOB_RECURSE DRUNTIME_D_FREEBSD 
${RUNTIME_DIR}/src/core/sys/freebsd/*.d)
file(GLOB_RECURSE DRUNTIME_D_LINUX   
${RUNTIME_DIR}/src/core/sys/linux/*.d)
file(GLOB_RECURSE DRUNTIME_D_NETBSD  
${RUNTIME_DIR}/src/core/sys/netbsd/*.d)
file(GLOB_RECURSE DRUNTIME_D_OPENBSD 
${RUNTIME_DIR}/src/core/sys/openbsd/*.d)
file(GLOB_RECURSE DRUNTIME_D_POSIX   
${RUNTIME_DIR}/src/core/sys/posix/*.d)
file(GLOB_RECURSE DRUNTIME_D_SOLARIS 
${RUNTIME_DIR}/src/core/sys/solaris/*.d)
file(GLOB_RECURSE DRUNTIME_D_WINDOWS 
${RUNTIME_DIR}/src/core/sys/windows/*.d)


 BTW, isn't it time to retire make files and go for cmake only? 
 Perhaps OT, I should start a new thread.


Do you know a mind state when you recognize a new tool and it 
starts to seem perfect and you want to use it everywhere and 
always? So for me now this tool is Meson build system. Therefore, 
I am biased.

Maybe we should consider switching to it. It is ideal for D in 
terms of the ability to simplified link programs written in 
various languages.

In fact, I'm not going to use druntime with existing built-in 
make/cmake scripts in my Dlang MCU projects because it are too 
complex and it is very difficult to configure to support dozens 
of all kinds of options that microcontroller binaries need.
Mar 15 2020
prev sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin wrote:

 What do you think about this? Is that good idea?


Sounds like what Walter wanted to do in the first place:

https://github.com/dlang/druntime/pull/2616#issuecomment-496312793
https://github.com/dlang/druntime/pull/2616#issuecomment-503503846
Mar 15 2020
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Sunday, 15 March 2020 at 16:31:35 UTC, Vladimir Panteleev 
wrote:

 On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin 
 wrote:

 What do you think about this? Is that good idea?


 Sounds like what Walter wanted to do in the first place:

 https://github.com/dlang/druntime/pull/2616#issuecomment-496312793
 https://github.com/dlang/druntime/pull/2616#issuecomment-503503846


Yes, it seems like that.

Just in case, to exclude discrepancies, I detail purpose of my 
proposal:
My proposal is about exclude any version() from 
platform-independent code. It is too hard to maintain all 
versions in one module file because it is too many types of 
platforms is available. (Really, I think we have near of infinity 
number of them, if we treat as version various sets of types 
aliases, etc.)
Mar 15 2020
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Sunday, 15 March 2020 at 18:05:50 UTC, Denis Feklushkin wrote:

 On Sunday, 15 March 2020 at 16:31:35 UTC, Vladimir Panteleev 
 wrote:

 On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin 
 wrote:

 What do you think about this? Is that good idea?






https://github.com/dlang/druntime/pull/2994
Mar 16 2020
parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Monday, 16 March 2020 at 22:30:05 UTC, Denis Feklushkin wrote:

 On Sunday, 15 March 2020 at 18:05:50 UTC, Denis Feklushkin 
 wrote:

 On Sunday, 15 March 2020 at 16:31:35 UTC, Vladimir Panteleev 
 wrote:

 On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin 
 wrote:

 What do you think about this? Is that good idea?






 https://github.com/dlang/druntime/pull/2994


Problem:

It is need to set default import directory depending on the 
platform. (Like currently compiler does default import for 
Phobos.) Otherwise Phobos and all other software isn't builds 
because platform specific code isn't imported.
Mar 16 2020
parent reply Johan <j j.nl> writes:
On Tuesday, 17 March 2020 at 03:08:41 UTC, Denis Feklushkin wrote:

 On Monday, 16 March 2020 at 22:30:05 UTC, Denis Feklushkin 
 wrote:

 On Sunday, 15 March 2020 at 18:05:50 UTC, Denis Feklushkin 
 wrote:

 On Sunday, 15 March 2020 at 16:31:35 UTC, Vladimir Panteleev 
 wrote:

 On Sunday, 15 March 2020 at 10:29:11 UTC, Denis Feklushkin 
 wrote:

 What do you think about this? Is that good idea?






 https://github.com/dlang/druntime/pull/2994


 Problem:

 It is need to set default import directory depending on the 
 platform. (Like currently compiler does default import for 
 Phobos.) Otherwise Phobos and all other software isn't builds 
 because platform specific code isn't imported.


Rather than setting the import directory, I think a much nicer 
solution is something like this:
https://github.com/weka-io/mecca/blob/master/src/mecca/platform/os/package.d
This works for different OS types, different CPU types, etc. And 
it removes the need to specify a whole list of specific import 
directories for each OS/cpu combination in the config file 
(please keep LDC crosscompilation in mind).

-Johan
Mar 17 2020
parent reply IGotD- <nise nise.com> writes:
On Tuesday, 17 March 2020 at 16:25:22 UTC, Johan wrote:

 Rather than setting the import directory, I think a much nicer 
 solution is something like this:
 https://github.com/weka-io/mecca/blob/master/src/mecca/platform/os/package.d
 This works for different OS types, different CPU types, etc. 
 And it removes the need to specify a whole list of specific 
 import directories for each OS/cpu combination in the config 
 file (please keep LDC crosscompilation in mind).

 -Johan


How is this different from using version?
Mar 17 2020
next sibling parent Johan <j j.nl> writes:
On Tuesday, 17 March 2020 at 17:12:39 UTC, IGotD- wrote:

 On Tuesday, 17 March 2020 at 16:25:22 UTC, Johan wrote:

 Rather than setting the import directory, I think a much nicer 
 solution is something like this:
 https://github.com/weka-io/mecca/blob/master/src/mecca/platform/os/package.d
 This works for different OS types, different CPU types, etc. 
 And it removes the need to specify a whole list of specific 
 import directories for each OS/cpu combination in the config 
 file (please keep LDC crosscompilation in mind).

 -Johan


 How is this different from using version?


It isn't. But it puts `version` in only a few places, instead of 
spread across many different files.
We shouldn't be replacing version, but utilize it.

-Johan
Mar 17 2020
prev sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 17 March 2020 at 17:12:39 UTC, IGotD- wrote:

 On Tuesday, 17 March 2020 at 16:25:22 UTC, Johan wrote:

 Rather than setting the import directory, I think a much nicer 
 solution is something like this:




Yes, already implemented in new PR: 
https://github.com/dlang/druntime/pull/2997


 https://github.com/weka-io/mecca/blob/master/src/mecca/platform/os/package.d
 This works for different OS types, different CPU types, etc. 
 And it removes the need to specify a whole list of specific 
 import directories for each OS/cpu combination in the config 
 file (please keep LDC crosscompilation in mind).

 -Johan


 How is this different from using version?


I used version (CRuntime_Abstract) to import platform dependent 
code.

This strategy looks good, but again there is a problem that is 
not directly related to the code: changing one module required 
changing only 8 lines in the code, but 15 lines in build scripts. 
And this is not all of needed changes - LDC uses its own cmake 
build scripts.

And each module will require approximately same changes. It will 
be like a disaster. Thus, at first stage we need to clear up 
build scripts.
Mar 17 2020
next sibling parent reply IGotD- <nise nise.com> writes:
On Tuesday, 17 March 2020 at 17:38:41 UTC, Denis Feklushkin wrote:

 I used version (CRuntime_Abstract) to import platform dependent 
 code.

 This strategy looks good, but again there is a problem that is 
 not directly related to the code: changing one module required 
 changing only 8 lines in the code, but 15 lines in build 
 scripts. And this is not all of needed changes - LDC uses its 
 own cmake build scripts.

 And each module will require approximately same changes. It 
 will be like a disaster. Thus, at first stage we need to clear 
 up build scripts.


This is where something like

import core.sys.<compiletime identifier>.semaphore;

would be an absolute great help.
Mar 17 2020
next sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 17 March 2020 at 19:34:45 UTC, IGotD- wrote:


 And each module will require approximately same changes. It 
 will be like a disaster. Thus, at first stage we need to clear 
 up build scripts.


 This is where something like

 import core.sys.<compiletime identifier>.semaphore;

 would be an absolute great help.


This is not fundamental difference with version()
Mar 17 2020
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2020-03-17 20:34, IGotD- wrote:


 This is where something like
 
 import core.sys.<compiletime identifier>.semaphore;
 
 would be an absolute great help.


Can be easily handled with a string mixin and a utility function:

version (linux)
     enum platform = "linux";
else version (Windows)
     enum platform = "windows";

string import_(string[] imports ...)
{
     string result;

     foreach (i, imp; imports)
     {
         if (i != 0)
             result ~= '.';

         result ~= imp;
     }

     return "import " ~ result ~ ';';
}

mixin(import_("core", "sys", platform, "semaphore"));

-- 
/Jacob Carlborg
Mar 17 2020
next sibling parent IGotD- <nise nise.com> writes:
On Tuesday, 17 March 2020 at 20:37:04 UTC, Jacob Carlborg wrote:

 On 2020-03-17 20:34, IGotD- wrote:


 This is where something like
 
 import core.sys.<compiletime identifier>.semaphore;
 
 would be an absolute great help.


 Can be easily handled with a string mixin and a utility 
 function:

 version (linux)
     enum platform = "linux";
 else version (Windows)
     enum platform = "windows";

 string import_(string[] imports ...)
 {
     string result;

     foreach (i, imp; imports)
     {
         if (i != 0)
             result ~= '.';

         result ~= imp;
     }

     return "import " ~ result ~ ';';
 }

 mixin(import_("core", "sys", platform, "semaphore"));


Thank you, didn't know that mixins even works for imports. This 
greatly would help a generic ABI as the import would just select 
the specific files for the particular OS. The OS implementations 
could then have dedicated names that like SemapholeImpl as type 
and dedicated methods like SemapholeImpl.wait(). This really open 
ups the degrees of freedom.
Mar 17 2020
prev sibling parent reply IGotD- <nise nise.com> writes:
On Tuesday, 17 March 2020 at 20:37:04 UTC, Jacob Carlborg wrote:

 Can be easily handled with a string mixin and a utility 
 function:

 version (linux)
     enum platform = "linux";
 else version (Windows)
     enum platform = "windows";

 string import_(string[] imports ...)
 {
     string result;

     foreach (i, imp; imports)
     {
         if (i != 0)
             result ~= '.';

         result ~= imp;
     }

     return "import " ~ result ~ ';';
 }

 mixin(import_("core", "sys", platform, "semaphore"));


I see many occurrences like this insinde the version blocks:

import core.sys.platform.subfunctionality : subFuncImpl;
alias functionality = subFuncImpl;

We could provide with an abstract interface like this as you 
suggested.

mixin(import_("core", "sys", platform, "semaphore")) : 
SemaphoreAcquireImpl;
The Semaphore class can use SemaphoreAcquireImpl.

SemaphoreAcquireImpl would have a type like this where 
SemaphoreOSType also needs to be imported in similar fashion:
void SemaphoreAcquireImpl(SemaphoreOSType sem), but this function 
would be the same for every system. I think this type of 
interface could be repeated for almost all OS dependent 
functionality. The type of the semaphore descriptor must also be 
be provided, as some are a descriptor on certain systems or a 
pointer type on others.

Some functions can just be aliased
mixin(import_("core", "sys", platform, "process")) : getPidImpl;
alias getpid = getPidImpl;


This type of interface could just be added under one version 
block like:

version(druntimeAbstractInterface)
{
    ...
}



Something that we all can work on is a generic C library 
interface for "C library style OS" for compatibility. This of 
course would have limited functionality but could provide basic 
functionality. This also makes important to selectively choose 
what to support. A stub would be an alternative, that just goes 
through silently or raises a "not implemented exception". Nim has 
an --os:any option which means it will just use the C library.
May 06 2020
parent Jacob Carlborg <doob me.com> writes:
On 2020-05-06 14:18, IGotD- wrote:


 I see many occurrences like this insinde the version blocks:
 
 import core.sys.platform.subfunctionality : subFuncImpl;
 alias functionality = subFuncImpl;
 
 We could provide with an abstract interface like this as you suggested.
 
 mixin(import_("core", "sys", platform, "semaphore")) : 
 SemaphoreAcquireImpl;
 The Semaphore class can use SemaphoreAcquireImpl.


It doesn't need to be an abstract interface. Since the implementation 
will be selected at compile time the only thing that is required that 
all implementation implements the same API. The API can be functions, 
structs, classes, whatever. For an existing example see [1].

[1] https://github.com/dlang/druntime/blob/master/src/rt/sections.d

-- 
/Jacob Carlborg
May 06 2020
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2020-03-17 18:38, Denis Feklushkin wrote:

 On Tuesday, 17 March 2020 at 17:12:39 UTC, IGotD- wrote:

 On Tuesday, 17 March 2020 at 16:25:22 UTC, Johan wrote:

 Rather than setting the import directory, I think a much nicer 
 solution is something like this:




 
 Yes, already implemented in new PR: 
 https://github.com/dlang/druntime/pull/2997


That PR kind of misses the point. What I did with Mecca was to manage to 
remove, more or less, all version identifiers from outside of the 
`platform` package.

If you're doing this for druntime I recommend you have a package for 
each platform. Currently you still have all version identifiers in the 
file for Posix.

-- 
/Jacob Carlborg
Mar 17 2020
parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 17 March 2020 at 20:27:21 UTC, Jacob Carlborg wrote:

 On 2020-03-17 18:38, Denis Feklushkin wrote:

 On Tuesday, 17 March 2020 at 17:12:39 UTC, IGotD- wrote:

 On Tuesday, 17 March 2020 at 16:25:22 UTC, Johan wrote:

 Rather than setting the import directory, I think a much 
 nicer solution is something like this:




 
 Yes, already implemented in new PR: 
 https://github.com/dlang/druntime/pull/2997


 That PR kind of misses the point. What I did with Mecca was to 
 manage to remove, more or less, all version identifiers from 
 outside of the `platform` package.

 If you're doing this for druntime I recommend you have a 
 package for each platform. Currently you still have all version 
 identifiers in the file for Posix.


This is not a problem as long as all of these platforms are 
supported by druntime.

But yes, some another Posix files can be divided into smallest 
platform parts, but such small file as posix's time.d can be left 
and so - there will no be problem.

I didn’t think about staying on this state with splitted 
core.stdc.time. I wanted to divide core.time too and some of its 
parts maybe become divided to smallest sys modules as you say.
But now I stuck into these Makefiles and don't know how to solve 
this gracefully.
Mar 17 2020
prev sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 17 March 2020 at 17:38:41 UTC, Denis Feklushkin wrote:


 This strategy looks good, but again there is a problem that is 
 not directly related to the code: changing one module required 
 changing only 8 lines in the code, but 15 lines in build 
 scripts. And this is not all of needed changes - LDC uses its 
 own cmake build scripts.

 And each module will require approximately same changes. It 
 will be like a disaster. Thus, at first stage we need to clear 
 up build scripts.


This problem is solved by Rainer Schuetze, thank you!
https://github.com/dlang/druntime/pull/3026

Discussion related to this topic continues for a little in this 
PR:
https://github.com/dlang/druntime/pull/2997
Apr 26 2020
prev sibling next sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Tuesday, 25 February 2020 at 00:05:05 UTC, Denis Feklushkin 
wrote:

 Hi!

 This post is inspired by: 
 https://forum.dlang.org/thread/scklbdrgjiypknuephfj forum.dlang.org

 [...]


Somewhat related that might be of interest to you.  I wrote my 
own standard library that doesn't depend on libc, doesn't use 
druntime and is compatible with betterC.  
https://github.com/dragon-lang/mar
Feb 25 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 25 February 2020 at 16:42:04 UTC, Jonathan Marler 
wrote:

 On Tuesday, 25 February 2020 at 00:05:05 UTC, Denis Feklushkin 
 wrote:

 Hi!

 This post is inspired by: 
 https://forum.dlang.org/thread/scklbdrgjiypknuephfj forum.dlang.org

 [...]


 Somewhat related that might be of interest to you.  I wrote my 
 own standard library that doesn't depend on libc, doesn't use 
 druntime and is compatible with betterC.  
 https://github.com/dragon-lang/mar


This project is very interesting to me I believe it may be 
extremely valuable to D as a whole! I have seen it a few months 
ago, but back then I didn't have much time to check it out.

What is the current status?
Is it self-hosting? E.g. can you build dmd only with it?
What are your goals (in addition to minimal external 
dependencies)?
How much of Phobos and druntime do you want reimplement or offer 
alternative features to?

Threading, ranges, containers and algorithms, IO, std.math, etc.?
Feb 25 2020
parent Jonathan Marler <johnnymarler gmail.com> writes:
On Tuesday, 25 February 2020 at 22:39:54 UTC, Petar Kirov 
[ZombineDev] wrote:

 On Tuesday, 25 February 2020 at 16:42:04 UTC, Jonathan Marler 
 wrote:

 On Tuesday, 25 February 2020 at 00:05:05 UTC, Denis Feklushkin 
 wrote:

 Hi!

 This post is inspired by: 
 https://forum.dlang.org/thread/scklbdrgjiypknuephfj forum.dlang.org

 [...]


 Somewhat related that might be of interest to you.  I wrote my 
 own standard library that doesn't depend on libc, doesn't use 
 druntime and is compatible with betterC.  
 https://github.com/dragon-lang/mar


 This project is very interesting to me I believe it may be 
 extremely valuable to D as a whole! I have seen it a few months 
 ago, but back then I didn't have much time to check it out.

 What is the current status?
 Is it self-hosting? E.g. can you build dmd only with it?
 What are your goals (in addition to minimal external 
 dependencies)?
 How much of Phobos and druntime do you want reimplement or 
 offer alternative features to?

 Threading, ranges, containers and algorithms, IO, std.math, 
 etc.?


I wanted to be able to use D without relying on libc.  One use 
case I had for this was to write my own rootfs purely in D 
(https://github.com/marler8997/maros).  I was very glad I did 
this as it was instrumental in learning how most linux 
distributions work.

I want my standard library to work with or without GC, druntime, 
libc, phobos, and betterC.  I want the API to be as similar as 
possible whether or not I'm using these features.  I want to be 
able to enable/disable these like features without rewriting my 
code.  In order to do this, the standard library needs to work 
with or without them.

Also note that the "mar" library can be used alongside 
druntime/phobos as well.  It's in its own namespace so there's no 
issues in using them together.  I was able to gradually port code 
to use "mar" instead of druntime/phobos piece by piece by 
referencing both.

Another big feature is that "mar" tracks null-terminated strings 
in the type system.  Pretty much all OS APIs use null-terminated 
(aka sentinel) strings, and it seems like a good idea to enforce 
them at compile-time rather than runtime.

Anyway, there was alot of reasons I made it.  Even if all it 
becomes is an educational experience for me then I'm still glad I 
did it, and if other poeple can find more uses for it then even 
better.
Feb 25 2020
prev sibling parent reply IGotD- <nise nise.com> writes:
Trying to adapt druntime has been a cascade of version (this) {} 
else version (that). druntime and phobos is a ball of wool.

After porting druntime I realized that I need Phobos as well 
because there is a lot of simple stuff like converting int to 
string and so on. Converting an int to string doesn't rely on an 
operating system. However, I'm forced to go through this version 
cascade over again which is even bigger that druntime.

The situation is similar to the C library which also might have 
dependencies all over. Some C library implementations are more 
simple and OS support can be stubbed. What D needs is a stubbed 
ABI for druntime/phobos. This means no file support, no threading 
etc. This ties into the "pay as you go" goal of D. D should be 
able to compile with minimal OS support instead of now require a 
rich OS.

What we need is an ABI. Then we also need a stubbed 
implementation of the ABI. This gives any implementer a good 
overview what needs to be implemented as well as that the OS 
support can be done incrementally. Some OSes don't support 
everything like memory mapped files and we should support this 
possibility as well.

We should also try to isolate the OS independent functionality 
more. Convenience functions that doesn't need an OS should be 
broken out, or at least with switches be compiled isolated from 
the OS dependent code.

If we are able to do this, D will make a rocket career in 
embedded systems.
May 02 2020
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2020-05-02 22:24, IGotD- wrote:


 Some OSes don't support everything like memory mapped files and we 
 should support this possibility as well.


Yes, I noticed this the hard way when I added support for iOS. It was 
assumed to be Posix compatible but there are a couple of edge cases 
where some system calls are not available.


 We should also try to isolate the OS independent functionality more. 
 Convenience functions that doesn't need an OS should be broken out, or 
 at least with switches be compiled isolated from the OS dependent code.


I agree. Everything that is platform specific should be in its own 
package, with one sub-package for each platform. I managed to do this 
when I added support for macOS to Mecca [1], turned out quite nice. The 
only remaining version blocks outside the platform package was to select 
the correct platform specific import.

[1] https://github.com/weka-io/mecca/pull/12

-- 
/Jacob Carlborg
May 06 2020
parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Wednesday, 6 May 2020 at 07:54:31 UTC, Jacob Carlborg wrote:

 On 2020-05-02 22:24, IGotD- wrote:


 Some OSes don't support everything like memory mapped files 
 and we should support this possibility as well.


 Yes, I noticed this the hard way when I added support for iOS. 
 It was assumed to be Posix compatible but there are a couple of 
 edge cases where some system calls are not available.


POSIX has kind of lost relevance, hence why Microsoft went with 
Linux for their new new POSIX subsystem.

"POSIX Has Become Outdated"

http://www.cs.columbia.edu/~vatlidak/resources/POSIXmagazine.pdf
May 06 2020
parent reply Jacob Carlborg <doob me.com> writes:
On 2020-05-06 10:40, Paulo Pinto wrote:


 POSIX has kind of lost relevance, hence why Microsoft went with Linux 
 for their new new POSIX subsystem.


The Linux subsystem is now actually a proper Linux kernel, running side 
by side with the Windows kernel, and not a reimplementation of the Linux 
kernel API.

-- 
/Jacob Carlborg
May 06 2020
parent Paulo Pinto <pjmlp progtools.org> writes:
On Wednesday, 6 May 2020 at 17:38:40 UTC, Jacob Carlborg wrote:

 On 2020-05-06 10:40, Paulo Pinto wrote:


 POSIX has kind of lost relevance, hence why Microsoft went 
 with Linux for their new new POSIX subsystem.


 The Linux subsystem is now actually a proper Linux kernel, 
 running side by side with the Windows kernel, and not a 
 reimplementation of the Linux kernel API.


Indeed, which was the point I was making, Linux compatibility has 
become more valuable than plain old POSIX.

They are also not the first vendor to go for that approach, other 
UNIXes and mainframes also offer similar compatibility, through 
various means.
May 06 2020
prev sibling next sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Saturday, 2 May 2020 at 20:24:46 UTC, IGotD- wrote:

 Trying to adapt druntime has been a cascade of version (this) 
 {} else version (that). druntime and phobos is a ball of wool.


This fork of ldc version of druntime is close to success:

https://github.com/denizzzka/druntime/tree/non_posix

Not so much has been added there (look at the diff between ldc 
and this branch).


 The situation is similar to the C library which also might have 
 dependencies all over. Some C library implementations are more 
 simple and OS support can be stubbed. What D needs is a stubbed 
 ABI for druntime/phobos. This means no file support, no 
 threading etc. This ties into the "pay as you go" goal of D. D 
 should be able to compile with minimal OS support instead of 
 now require a rich OS.


It is important to understand that libc library is not actually a 
library for use C language.

Yes, it contains some C string and math functions, but OS API 
functions what provided by libc is convient wrapper for OS 
syscalls what used by everyone, because no one wants to support 
all number of OSes syscalls.

So, libc de facto is OS API.

It would be great to replace them. But this does not look as a 
priority, because you can always use parts of any libc as a 
static library for baremetal.

Perhaps it is better to wait for the "worldwide libc replacement" 
to appear? Personally, I don't like the concept of errno in libc.


 We should also try to isolate the OS independent functionality 
 more. Convenience functions that doesn't need an OS should be 
 broken out, or at least with switches be compiled isolated from 
 the OS dependent code.


$ grep -r " core.stdc" --include='*.d' druntime/src/core/ |grep 
import

displays that at least druntime contains not so much number of 
libc calls.
May 06 2020
parent reply Jacob Carlborg <doob me.com> writes:
On 2020-05-06 13:42, Denis Feklushkin wrote:


 It is important to understand that libc library is not actually a 
 library for use C language.
 
 Yes, it contains some C string and math functions, but OS API functions 
 what provided by libc is convient wrapper for OS syscalls what used by 
 everyone, because no one wants to support all number of OSes syscalls.


I'm not sure that I agree. I would say that libc is the library 
containing the functions and types defined by the C standard.

Then it just happens so that on most platforms they will bake in the OS 
APIs in the same library. I just checked, on macOS all programs are 
linked with libSystem. This is mostly an umbrella library that brings in 
other libraries. The library itself only contains 36 symbols, but it 
depends on 33 other libraries. One of them being libsystem_c, which 
seems to contain the actual C standard library. Then there's 
libsystem_kernel, which seems to contain the OS APIs.


 So, libc de facto is OS API.


That depends on the platform as well. On Linux the system calls provide 
a stable API. On macOS they don't. Apple explicitly specifies that the 
stable API is the syscall wrappers.


 It would be great to replace them. But this does not look as a priority, 
 because you can always use parts of any libc as a static library for 
 baremetal.


You cannot. glibc doesn't fully support static linking, not even on 
Linux. For example, DNS lookup does not work/is not fully supported in a 
fully statically linked binary.


 Perhaps it is better to wait for the "worldwide libc replacement" to 
 appear? Personally, I don't like the concept of errno in libc.


Yeah, I don't like it either.

-- 
/Jacob Carlborg
May 06 2020
next sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Wednesday, 6 May 2020 at 18:03:40 UTC, Jacob Carlborg wrote:


 It would be great to replace them. But this does not look as a 
 priority, because you can always use parts of any libc as a 
 static library for baremetal.


 You cannot. glibc doesn't fully support static linking, not 
 even on Linux. For example, DNS lookup does not work/is not 
 fully supported in a fully statically linked binary.



"any libc" -> "any suitable libc" :-)

(picolibc is my choise for this purposes)
May 06 2020
parent reply dangbinghoo <dangbinghoo gmail.com> writes:
On Wednesday, 6 May 2020 at 18:56:55 UTC, Denis Feklushkin wrote:

 On Wednesday, 6 May 2020 at 18:03:40 UTC, Jacob Carlborg wrote:


 It would be great to replace them. But this does not look as 
 a priority, because you can always use parts of any libc as a 
 static library for baremetal.


 You cannot. glibc doesn't fully support static linking, not 
 even on Linux. For example, DNS lookup does not work/is not 
 fully supported in a fully statically linked binary.



 "any libc" -> "any suitable libc" :-)

 (picolibc is my choise for this purposes)


for druntime to working with MCU platform, we probably can rely 
only on [newlib] or even [newlib-nano] style `libC`, and make all 
other things to OS APIs, although some RTOS has implemented 
several POSIX api, it's all only a subset of POSIX.

but the embedded programmer will happy if we support newlib-nano 
+ FreeRTOS or NuttX or Zephyr.
May 06 2020
parent dangbinghoo <dangbinghoo gmail.com> writes:
On Thursday, 7 May 2020 at 06:46:27 UTC, dangbinghoo wrote:

 On Wednesday, 6 May 2020 at 18:56:55 UTC, Denis Feklushkin 
 wrote:

 On Wednesday, 6 May 2020 at 18:03:40 UTC, Jacob Carlborg wrote:


 [...]


 You cannot. glibc doesn't fully support static linking, not 
 even on Linux. For example, DNS lookup does not work/is not 
 fully supported in a fully statically linked binary.



 "any libc" -> "any suitable libc" :-)

 (picolibc is my choise for this purposes)


 for druntime to working with MCU platform, we probably can rely 
 only on [newlib] or even [newlib-nano] style `libC`, and make 
 all other things to OS APIs, although some RTOS has implemented 
 several POSIX api, it's all only a subset of POSIX.

 but the embedded programmer will happy if we support 
 newlib-nano + FreeRTOS or NuttX or Zephyr.


sorry, I don't know [newlib-nano] is now called [picolibc]!  ^_^
May 07 2020
prev sibling parent IGotD- <nise nise.com> writes:
On Wednesday, 6 May 2020 at 18:03:40 UTC, Jacob Carlborg wrote:

 You cannot. glibc doesn't fully support static linking, not 
 even on Linux. For example, DNS lookup does not work/is not 
 fully supported in a fully statically linked binary.


Networking is not really part of the C library standard, despite 
it is often linked into libc. Supporting C library functionality 
gives you a limited system but at least a system that will be 
able to work on most system without any modifications, like 
lowest common denominator.
May 06 2020
prev sibling next sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Saturday, 2 May 2020 at 20:24:46 UTC, IGotD- wrote:


 After porting druntime I realized that I need Phobos as well 
 because there is a lot of simple stuff like converting int to 
 string and so on. Converting an int to string doesn't rely on 
 an operating system. However, I'm forced to go through this 
 version cascade over again which is even bigger that druntime.


I briefly looked (grep) which "versions" are used in Phobos. It 
seems to me that everything that is about different OSes support 
can be transferred to druntime without problem. (And in druntime 
can be added ability to select arbitrary OS code as repeatedly 
described above.)

...And then druntime could become "the new libc"
May 06 2020
prev sibling parent reply Denis Feklushkin <feklushkin.denis gmail.com> writes:
Ok, now my project runs druntime and can blink by LED. Binary 
size is 200Kb :-)

https://github.com/denizzzka/d_c_arm_test/

On Saturday, 2 May 2020 at 20:24:46 UTC, IGotD- wrote:

 Trying to adapt druntime has been a cascade of version (this) 
 {} else version (that). druntime and phobos is a ball of wool.


About Phobos: here is only one issue with it and supporting any 
platform for "blinking LED project":

file std/ascii.d:

/// Newline sequence for this system.
version (Windows)
     immutable newline = "\r\n";
else version (Posix)
     immutable newline = "\n";
else
     static assert(0, "Unsupported OS");

Compilation fails here because "OS" isn't Windows and isn't 
Posix. It is need to clarify way to pass external OS version into 
Phobos.
May 18 2020
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
https://dlang.org/spec/version.html#predefined-versions

FreeStanding

\n is probably correct for this version, but it probably should be 
discussed in a PR.
May 18 2020
parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 19 May 2020 at 05:10:01 UTC, rikki cattermole wrote:

 https://dlang.org/spec/version.html#predefined-versions

 FreeStanding

 \n is probably correct for this version, but it probably should 
 be discussed in a PR.


newline declaration can be moved into druntime, because druntime 
depends from OS.
May 19 2020
prev sibling parent Denis Feklushkin <feklushkin.denis gmail.com> writes:
On Tuesday, 19 May 2020 at 05:02:41 UTC, Denis Feklushkin wrote:


 file std/ascii.d:

 /// Newline sequence for this system.
 version (Windows)
     immutable newline = "\r\n";
 else version (Posix)
     immutable newline = "\n";
 else
     static assert(0, "Unsupported OS");


I just accidentally used dmd version of Phobos without LDC 
extension, this is already fixed for LDC fork
May 19 2020