www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - A betterC modular standard library?

reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
Hi,

Who is interested in betterC _modular_* standard library?
I am planing to make libmir org a community for it.
Thought and concerns?

We already have better `cpuid` and better `random` packages. The 
betterC std.range and std.algorithm analogs would be released 
with new ndslice implementation. Mir's algorithm would be faster 
then Phobos and will generate less template bloat. Then 
lightweight threads, multithread GLAS, matrix inversion. Fastest 
I/O and http2 ...

(they all are betterC modular libraries)

Andrei and probably 90% of existing D users don't want Phobos to 
move this direction.

In other hand I need a commercial attractive D infrastructure for 
large and heavy system projects. There is no commercial 
perspective for me to contribute to Phobos because:

1. Phobos version depends on compiler version. Delay with LDC 
release is too large. It should / can be one day.

2. We can not do hot fixes in Phobos without compiling and 
deploying patched libphobos. Mir's DUB package are much more 
flexible.

3. Dependencies should be clear. Modularity is a proper way for 
large std library. In phobos everything integrated with 
everything. DRuntime -> Phobos abstraction is weird for betterC 
because system modules can depends universal algorithms, but 
universal algorithm are more portable if they have not system 
dependencies.

4. Phobos does not provide and will not provide betterC 
guaranties. If something works for betterC mode now it may not 
work in the future release.

[1] https://github.com/libmir/
* - separate github/DUB libraries with clear dependencies.

Best regards,
Ilya
Dec 18 2016
next sibling parent reply Temtaime <temtaime gmail.com> writes:
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:
 Hi,

 Who is interested in betterC _modular_* standard library?
 I am planing to make libmir org a community for it.
 Thought and concerns?

 We already have better `cpuid` and better `random` packages. 
 The betterC std.range and std.algorithm analogs would be 
 released with new ndslice implementation. Mir's algorithm would 
 be faster then Phobos and will generate less template bloat. 
 Then lightweight threads, multithread GLAS, matrix inversion. 
 Fastest I/O and http2 ...

 (they all are betterC modular libraries)

 Andrei and probably 90% of existing D users don't want Phobos 
 to move this direction.

 In other hand I need a commercial attractive D infrastructure 
 for large and heavy system projects. There is no commercial 
 perspective for me to contribute to Phobos because:

 1. Phobos version depends on compiler version. Delay with LDC 
 release is too large. It should / can be one day.

 2. We can not do hot fixes in Phobos without compiling and 
 deploying patched libphobos. Mir's DUB package are much more 
 flexible.

 3. Dependencies should be clear. Modularity is a proper way for 
 large std library. In phobos everything integrated with 
 everything. DRuntime -> Phobos abstraction is weird for betterC 
 because system modules can depends universal algorithms, but 
 universal algorithm are more portable if they have not system 
 dependencies.

 4. Phobos does not provide and will not provide betterC 
 guaranties. If something works for betterC mode now it may not 
 work in the future release.

 [1] https://github.com/libmir/
 * - separate github/DUB libraries with clear dependencies.

 Best regards,
 Ilya
Let's write a D, phobos. Then we drop all the features and use a BetterC mode. Then we write all modules against betterC mode.
Dec 18 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 18 December 2016 at 11:06:58 UTC, Temtaime wrote:
 Let's write a D, phobos.
 Then we drop all the features and use a BetterC mode.
 Then we write all modules against betterC mode.
Sounds like a plan! --Ilya
Dec 18 2016
prev sibling next sibling parent reply Radu <void null.pt> writes:
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:
 Hi,

 Who is interested in betterC _modular_* standard library?
 I am planing to make libmir org a community for it.
 Thought and concerns?

 We already have better `cpuid` and better `random` packages. 
 The betterC std.range and std.algorithm analogs would be 
 released with new ndslice implementation. Mir's algorithm would 
 be faster then Phobos and will generate less template bloat. 
 Then lightweight threads, multithread GLAS, matrix inversion. 
 Fastest I/O and http2 ...

 (they all are betterC modular libraries)

 Andrei and probably 90% of existing D users don't want Phobos 
 to move this direction.

 In other hand I need a commercial attractive D infrastructure 
 for large and heavy system projects. There is no commercial 
 perspective for me to contribute to Phobos because:

 1. Phobos version depends on compiler version. Delay with LDC 
 release is too large. It should / can be one day.

 2. We can not do hot fixes in Phobos without compiling and 
 deploying patched libphobos. Mir's DUB package are much more 
 flexible.

 3. Dependencies should be clear. Modularity is a proper way for 
 large std library. In phobos everything integrated with 
 everything. DRuntime -> Phobos abstraction is weird for betterC 
 because system modules can depends universal algorithms, but 
 universal algorithm are more portable if they have not system 
 dependencies.

 4. Phobos does not provide and will not provide betterC 
 guaranties. If something works for betterC mode now it may not 
 work in the future release.

 [1] https://github.com/libmir/
 * - separate github/DUB libraries with clear dependencies.

 Best regards,
 Ilya
I hope this doesn't turn into a new Tango vs. Phobos war. If you have ideas on how to improve things without messing the entire community with ego wars and such, then it sounds good. Otherwise thumbs down! The first sign of faith will be that your modules will always work with druntime and phobos - i.e. you wont require a special runtime for your stuff to work. As for Andrei and Walter - I think you need to see why people (talented people!) want to side walk the phobos & druntime normal contribution routes. Something clearly is wrong there!
Dec 18 2016
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 18 December 2016 at 14:37:55 UTC, Radu wrote:
 On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
 wrote:
 Hi,

 Who is interested in betterC _modular_* standard library?
 I am planing to make libmir org a community for it.
 Thought and concerns?

 We already have better `cpuid` and better `random` packages. 
 The betterC std.range and std.algorithm analogs would be 
 released with new ndslice implementation. Mir's algorithm 
 would be faster then Phobos and will generate less template 
 bloat. Then lightweight threads, multithread GLAS, matrix 
 inversion. Fastest I/O and http2 ...

 (they all are betterC modular libraries)

 Andrei and probably 90% of existing D users don't want Phobos 
 to move this direction.

 In other hand I need a commercial attractive D infrastructure 
 for large and heavy system projects. There is no commercial 
 perspective for me to contribute to Phobos because:

 1. Phobos version depends on compiler version. Delay with LDC 
 release is too large. It should / can be one day.

 2. We can not do hot fixes in Phobos without compiling and 
 deploying patched libphobos. Mir's DUB package are much more 
 flexible.

 3. Dependencies should be clear. Modularity is a proper way 
 for large std library. In phobos everything integrated with 
 everything. DRuntime -> Phobos abstraction is weird for 
 betterC because system modules can depends universal 
 algorithms, but universal algorithm are more portable if they 
 have not system dependencies.

 4. Phobos does not provide and will not provide betterC 
 guaranties. If something works for betterC mode now it may not 
 work in the future release.

 [1] https://github.com/libmir/
 * - separate github/DUB libraries with clear dependencies.

 Best regards,
 Ilya
I hope this doesn't turn into a new Tango vs. Phobos war. If you have ideas on how to improve things without messing the entire community with ego wars and such, then it sounds good. Otherwise thumbs down!
Maybe I will be able to propose them on DConf.
 The first sign of faith will be that your modules will always 
 work with druntime and phobos - i.e. you wont require a special 
 runtime for your stuff to work.
Yes/No in the same time. 1. It would be always possible to use Mir and Phobos together, 100% guarantee. Furthermore, some Mir libraries are going to be infrastructure for _all_, not only for D. For example Mir GLAS will target the Julia language after D. 2. In the same time many thing is not possible to create with Phobos/Druntime. The simplest example is numeric libraries. I can create a multithread GLAS with core.thread, but if I do so GLAS will never be accepted as default BLAS implementation for Julia. So, i need to create my own low level thread library. Intel MKL, OpenBLAS have their own thread management: this is common practice. Cheers, Ilya
Dec 18 2016
parent reply Radu <void null.pt> writes:
On Sunday, 18 December 2016 at 15:01:55 UTC, Ilya Yaroshenko 
wrote:
 On Sunday, 18 December 2016 at 14:37:55 UTC, Radu wrote:
 [...]
Maybe I will be able to propose them on DConf.
 [...]
Yes/No in the same time. 1. It would be always possible to use Mir and Phobos together, 100% guarantee. Furthermore, some Mir libraries are going to be infrastructure for _all_, not only for D. For example Mir GLAS will target the Julia language after D. 2. In the same time many thing is not possible to create with Phobos/Druntime. The simplest example is numeric libraries. I can create a multithread GLAS with core.thread, but if I do so GLAS will never be accepted as default BLAS implementation for Julia. So, i need to create my own low level thread library. Intel MKL, OpenBLAS have their own thread management: this is common practice. Cheers, Ilya
I get point 2, but I should be able to use your new modules without giving up druntime/phobos, right?. They should not be mutual exclusive!
Dec 18 2016
next sibling parent bachmeier <no spam.net> writes:
On Sunday, 18 December 2016 at 15:20:48 UTC, Radu wrote:
 I get point 2, but I should be able to use your new modules 
 without giving up druntime/phobos, right?. They should not be 
 mutual exclusive!
I think that being able to use them the same as any other C library should be sufficient.
Dec 18 2016
prev sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 18 December 2016 at 15:20:48 UTC, Radu wrote:
 2. In the same time many thing is not possible to create with 
 Phobos/Druntime. The simplest example is numeric libraries. I 
 can create a multithread GLAS with core.thread, but if I do so 
 GLAS will never be accepted as default BLAS implementation for 
 Julia. So, i need to create my own low level thread library. 
 Intel MKL, OpenBLAS have their own thread management: this is 
 common practice.

 Cheers,
 Ilya
I get point 2, but I should be able to use your new modules without giving up druntime/phobos, right?. They should not be mutual exclusive!
Yes, you are able to use any Mir libs with Phobos/DRuntime --Ilya
Dec 18 2016
prev sibling parent qznc <qznc web.de> writes:
On Sunday, 18 December 2016 at 14:37:55 UTC, Radu wrote:
 On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko
 Who is interested in betterC _modular_* standard library?
 I am planing to make libmir org a community for it.
 Thought and concerns?
I hope this doesn't turn into a new Tango vs. Phobos war.
My first thought as well. Probably an irrational fear, but the question will pop up often.
Dec 18 2016
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 4:26 AM, Ilya Yaroshenko wrote:
 Hi,

 Who is interested in betterC _modular_* standard library?
 I am planing to make libmir org a community for it.
 Thought and concerns?

 We already have better `cpuid` and better `random` packages. The betterC
 std.range and std.algorithm analogs would be released with new ndslice
 implementation. Mir's algorithm would be faster then Phobos and will
 generate less template bloat. Then lightweight threads, multithread
 GLAS, matrix inversion. Fastest I/O and http2 ...

 (they all are betterC modular libraries)

 Andrei and probably 90% of existing D users don't want Phobos to move
 this direction.
Please do not misconstrue what I said. You have my support to improve the state of affairs with the D core runtime and standard library, and have had it from day one. With regard to new functionality that supplants existing pieces such as cpuid_v2 or random_v2, definitely. We can and should add that and many other great things to phobos. Go for it. With regard to avoiding to link druntime, we should build the argument better. Is all of druntime bad, or just parts of it? E.g. if something does not link the GC but does use vtables, is it good/bad, and to what extent? These issues are not entirely clear to me and it would be great if you explained them better. I definitely need to be better educated on this. What I do know is making "don't use druntime" a goal in and of itself is not the most useful way to frame the problem. With regard to changes that break the entire fabric of Phobos (such as: we must eliminate random-access ranges), this is definitely more like simple survival rather than things I want. With regard to properly attributing credit for good work, I believe this is important and I commit to do anything reasonable for making it happen. I kindly advise you to better understand what you are trying to do, and how to do it to maximize impact. It seems you are not entirely clear on what you want to do and what the best route is, as shown by you retiring your std.experimental.ndslice work after you've had full freedom on what should go in it.
 In other hand I need a commercial attractive D infrastructure for large
 and heavy system projects. There is no commercial perspective for me to
 contribute to Phobos because:

 1. Phobos version depends on compiler version. Delay with LDC release is
 too large. It should / can be one day.
Then a good way to help this is improve the LDC lag.
 2. We can not do hot fixes in Phobos without compiling and deploying
 patched libphobos. Mir's DUB package are much more flexible.
The reverse of that is less stability. The good part about merging your work in Phobos is you maximize impact and visibility. These seem to me artificial arguments. Yes, being preoccupied over quick deployment may make sense in the initial stages when there's no stability and the design may take any direction. That should get less necessary once some abstractions are in place. Organizations that are interested in patching minute fixes may build their own standard library (and sometimes compiler). This is routinely the case with C++ at Facebook, Google, and in all likelihood other large-scale tech shops. It would be tenuous to argue that this should be the default way to go about things for everyone. I don't understand why you consider it an essential matter.
 3. Dependencies should be clear. Modularity is a proper way for large
 std library. In phobos everything integrated with everything. DRuntime
 -> Phobos abstraction is weird for betterC because system modules can
 depends universal algorithms, but universal algorithm are more portable
 if they have not system dependencies.
I read this three times and cannot understand it. At any rate: DIP 1005, which is partially motivated by our discussions, should allow us to make dependencies as simple and clean as function-level.
 4. Phobos does not provide and will not provide betterC guaranties. If
 something works for betterC mode now it may not work in the future release.
Yes, we can definitely improve this. We (the Foundation) are all for making things better for achieving the things you want to achieve. At this point D needs unity and rallying under a shared vision, not balkanization in several special interest groups. There are so many great things you can do starting literally today to make things better for everybody using D, and why in the world would anyone want to stop you. I strongly believe your participation to the Standard D language and library will be vastly better for you personally and professionally. Andrei
Dec 18 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 18 December 2016 at 15:18:37 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/16 4:26 AM, Ilya Yaroshenko wrote:
 Hi,

 Who is interested in betterC _modular_* standard library?
 I am planing to make libmir org a community for it.
 Thought and concerns?

 We already have better `cpuid` and better `random` packages. 
 The betterC
 std.range and std.algorithm analogs would be released with new 
 ndslice
 implementation. Mir's algorithm would be faster then Phobos 
 and will
 generate less template bloat. Then lightweight threads, 
 multithread
 GLAS, matrix inversion. Fastest I/O and http2 ...

 (they all are betterC modular libraries)

 Andrei and probably 90% of existing D users don't want Phobos 
 to move
 this direction.
Please do not misconstrue what I said. You have my support to improve the state of affairs with the D core runtime and standard library, and have had it from day one. With regard to new functionality that supplants existing pieces such as cpuid_v2 or random_v2, definitely. We can and should add that and many other great things to phobos. Go for it. With regard to avoiding to link druntime, we should build the argument better. Is all of druntime bad, or just parts of it?
DRuntime is good for applications. But writing multi-language libraries with DRuntime is bad idea. The last paragraph in [1] describes why DRuntime can not be used for GLAS. I never said we should "remove GC" or "remove runtime". We just need to provide a clear way for betterC subset and modularise library. DRuntime and Phobos are well designed hight level libraries.
 E.g. if something does not link the GC but does use vtables, is 
 it good/bad, and to what extent?
Very good point. First, I don't see a real reason why we need DRuntime for vtables (classes and interfaces). LDC allows to use vtables without DRuntime (only with extern(C++), hehe). D classes can be constructed on stack or with allocators. The only problem is vtable initlization, which requires `typeid` (TypeInfo object). I am not a OOP expert, but it looks like we can have extern(C++) classes without DRuntime if we solve problem with initilization.
 These issues are not entirely clear to me and it would be great 
 if you explained them better. I definitely need to be better 
 educated on this. What I do know is making "don't use druntime" 
 a goal in and of itself is not the most useful way to frame the 
 problem.
0. Linking compatibility. Assume A has builded a betterC library AA using GDC version XXX and B has builded a betterC library BB using LDC version YYY. Question: how mister C can use BB and AA together in his C-library if both this libraries depends on different DRuntimes (compiler and version)? 1. Different mangling names for DMD and LDC. mir-cpuid provides extern(C) interface for all functionality except few very low level struct declarations. The general problem is that we do not have binary compatibility where we may have it. And it is not clear where we already have it. 2. Any function may throw an Error. 3. Any throwable is not a part of betterC. Phobos core team rejected approach to replace RangeErrors with asserts or another function declaration. Possible alternative is __range_assert() or a library function. 4. Threads are GC oriented: 1. They are classes (they can be structs like allocators) 2. They are integrated with GC. This requires additional computation time if I am not wrong. 3. They are not replaceable. Phobos multithread modules can not use something else. 5. Mutexes are GC oriented: ditto 1. and 3. 6. Intrinsics are compiler magic without specification. It is not clear what it is intrinsic, are they inlined, where their bodies are defined, how/if they works in CTFE. LLVM provides a list of intrinsics [3], this is very useful to write a betterC libraries. ... and many other ... They all are not clear for me too. ------------- A solution for mister C: Mister A and mister B should use a runtime which has either a small stable extern(C) interface or all its functions are inlined. All other functions can be a dub packages. ------------- Proof: cpuid: extern(C) interface with aliases. a fork of aligned mallocator for GLAS: all functions are always inlined.
 With regard to changes that break the entire fabric of Phobos 
 (such as: we must eliminate random-access ranges), this is 
 definitely more like simple survival rather than things I want.
I don't want eliminate R-A-R now. But we can change our Range concept and provide alternative module as dub package, which will be announced as a future replacement (say, after 5 years). But duplicated sorting implementation with cursors (lightweight iterators) in the same module is weird. It is better to continue this discussion after the cursor based implementation will be ready for your review.
 With regard to properly attributing credit for good work, I 
 believe this is important and I commit to do anything 
 reasonable for making it happen.

 I kindly advise you to better understand what you are trying to 
 do, and how to do it to maximize impact. It seems you are not 
 entirely clear on what you want to do and what the best route 
 is, as shown by you retiring your std.experimental.ndslice work 
 after you've had full freedom on what should go in it.

 In other hand I need a commercial attractive D infrastructure 
 for large
 and heavy system projects. There is no commercial perspective 
 for me to
 contribute to Phobos because:

 1. Phobos version depends on compiler version. Delay with LDC 
 release is
 too large. It should / can be one day.
Then a good way to help this is improve the LDC lag.
I mean lag between new Phobos PR and LDC release. This includes time for PR review, DMD release, LDC release. Libraries should be separated from a compiler, IMHO.
 2. We can not do hot fixes in Phobos without compiling and 
 deploying
 patched libphobos. Mir's DUB package are much more flexible.
The reverse of that is less stability.
A hot fix can be added in local repository without creating a PR. DUB allows to do it.
 The good part about merging your work in Phobos is you maximize 
 impact and visibility.
Yes, but merging existing libraries to Phobos will slowdown other projects as DCV. Also I will spend a lot efforts on stuff like repositories synchronisation. Plus it makes my users unhappy: they will need to switch from Phobos to Mir if they want to use recent addition.
 These seem to me artificial arguments. Yes, being preoccupied 
 over quick deployment may make sense in the initial stages when 
 there's no stability and the design may take any direction. 
 That should get less necessary once some abstractions are in 
 place.
C++ adds new APIs one time per few years. D add few changes every month. Phobos and DRuntime does not behave as traditional standard system language libraries. They are more like a scripting language heaps which are grows very fast (like scipy). I wan't a D Standard Library to be exist too, but it should be slim, has better backward compatibility, ideal backward binary compatibility, has good low level specification and documentation. Large hight level library like Phobos can not do it with community support only.
 Organizations that are interested in patching minute fixes may 
 build their own standard library (and sometimes compiler). This 
 is routinely the case with C++ at Facebook, Google, and in all 
 likelihood other large-scale tech shops. It would be tenuous to 
 argue that this should be the default way to go about things 
 for everyone. I don't understand why you consider it an 
 essential matter.
Good point. Why not to be more user friendly for large-scale tech companies then? If we can show that merging too LDC requires less efforts then upgrading from g++ to clang++ ...
 3. Dependencies should be clear. Modularity is a proper way 
 for large
 std library. In phobos everything integrated with everything. 
 DRuntime
 -> Phobos abstraction is weird for betterC because system 
 modules can
 depends universal algorithms, but universal algorithm are more 
 portable
 if they have not system dependencies.
I read this three times and cannot understand it. At any rate: DIP 1005, which is partially motivated by our discussions, should allow us to make dependencies as simple and clean as function-level.
The DIP 1005 idea is good. But this will fix few percentage of existing dependency ambiguity. Plus it does not solve the dependency problem, it just makes it slightly more visible.
 4. Phobos does not provide and will not provide betterC 
 guaranties. If
 something works for betterC mode now it may not work in the 
 future release.
Yes, we can definitely improve this. We (the Foundation) are all for making things better for achieving the things you want to achieve. At this point D needs unity and rallying under a shared vision, not balkanization in several special interest groups. There are so many great things you can do starting literally today to make things better for everybody using D, and why in the world would anyone want to stop you. I strongly believe your participation to the Standard D language and library will be vastly better for you personally and professionally. Andrei
I will continue in the answer to Walter. Thank you for the answer (it is hard to understand me because English and other reasons), Ilya [1] http://forum.dlang.org/post/jwmzvfashovivqzamxrq forum.dlang.org [2] http://forum.dlang.org/post/jneokbosxeatvkppbuvf forum.dlang.org [3] http://llvm.org/docs/LangRef.html#intrinsic-functions
Dec 18 2016
parent reply safety0ff <safety0ff.dev gmail.com> writes:
On Sunday, 18 December 2016 at 18:02:58 UTC, Ilya Yaroshenko 
wrote:
 Thank you for the answer (it is hard to understand me because 
 English and other reasons),

 Ilya
It was difficult to understand your vision until this post, now I think I grasp it. Let me try to summarize what I've understood: D as it stands, is not suitable for writing low level libraries as well as large scale software development because of compiler dependence of compiled code. Examples: If you have two software teams, and team A's software dependent on compiler X (e.g requires newer feature, requires bug/regression fix, etc) and team B's software depends on compiler Y to meet performance requirements, they get stuck. Also you want to create a low level library than can be easily distributed and linked from other languages (e.g. GLAS) extern (C) is the only viable option, but that can still lock in the D compiler used if you depend on phobos/druntime. So the proposal is to make binary compatibility possible in the near future by implementing "betterC" which provides a bare-bones language and removes the greatest sources of incompatibilities. Once this is done a community can form around it and create completely modular libraries. These can be used by all D and non-D users alike without compatibility problems. Since this is all predicated on "betterC", which isn't implemented. I think it is imperative to create a full specification. I look forward to seeing where this initiative goes. P.S.: I think Ilya writes "evaluates" where he means "evolves"
Dec 19 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 18:04:14 UTC, safety0ff wrote:
 On Sunday, 18 December 2016 at 18:02:58 UTC, Ilya Yaroshenko 
 wrote:
 Thank you for the answer (it is hard to understand me because 
 English and other reasons),

 Ilya
It was difficult to understand your vision until this post, now I think I grasp it. Let me try to summarize what I've understood: [...]
Yes, thank you for the clarification!
 P.S.: I think Ilya writes "evaluates" where he means "evolves"
Oh, yes! I mean "evolves"!
Dec 19 2016
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:
 We already have better `cpuid` and better `random` packages.
Great! Please PR them for Phobos.
 The betterC
 std.range and std.algorithm analogs would be released with new ndslice
 implementation. Mir's algorithm would be faster then Phobos and will generate
 less template bloat. Then lightweight threads, multithread GLAS, matrix
 inversion. Fastest I/O and http2 ...

 (they all are betterC modular libraries)
Propose them for Phobos!
 Andrei and probably 90% of existing D users don't want Phobos to move this
 direction.
I don't understand that point. What direction?
 In other hand I need a commercial attractive D infrastructure for large and
 heavy system projects. There is no commercial perspective for me to contribute
 to Phobos because:

 1. Phobos version depends on compiler version. Delay with LDC release is too
 large. It should / can be one day.
The language evolves over time, and the standard library must, too. It happens with every language.
 2. We can not do hot fixes in Phobos without compiling and deploying patched
 libphobos. Mir's DUB package are much more flexible.

 3. Dependencies should be clear. Modularity is a proper way for large std
 library. In phobos everything integrated with everything. DRuntime -> Phobos
 abstraction is weird for betterC because system modules can depends universal
 algorithms, but universal algorithm are more portable if they have not system
 dependencies.
Not sure what you mean. Algorithms in Phobos are not dependent on system idiosyncracies.
 4. Phobos does not provide and will not provide betterC guaranties. If
something
 works for betterC mode now it may not work in the future release.
I've done a fair amount of work to remove such things. Clearly more can and should be done. Generally, we do have a goal of making Phobos entirely "pay as you go" instead of being so interconnected (which is what betterC is really all about). We can definitely use help in improving that and helping it along.
Dec 18 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright wrote:
 On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:
 We already have better `cpuid` and better `random` packages.
Great! Please PR them for Phobos.
cpuid is used in Mir GLAS and it should be a betterC library. Without DRuntime dependency because DRuntime has not backward binary compatibility and has not compatibility between different compilers. From my reply to Andrei: ---- Linking compatibility. Assume A has builded a betterC library AA using GDC version XXX and B has builded a betterC library BB using LDC version YYY. Question: how mister C can use BB and AA together in his C-library if both this libraries depends on different DRuntimes (compiler and version)? ---- mir-glas and mir-cpuid are libraries with extern(C) interface for all languages. It is silly to build a BLAS implementation only for one language.
 The betterC
 std.range and std.algorithm analogs would be released with new 
 ndslice
 implementation. Mir's algorithm would be faster then Phobos 
 and will generate
 less template bloat. Then lightweight threads, multithread 
 GLAS, matrix
 inversion. Fastest I/O and http2 ...

 (they all are betterC modular libraries)
Propose them for Phobos!
First I need to implement them and announce for the community. Then community will say "What he is doing??? C++ iterators, really?" Hehe This is reason why ndslice is deprecated in Phobos. It requires another std.algorithm and std.range. I am not able to implement all this ideas in Phobos PRs: it is requires too much time. In addition, I can not use Phobos for other reasons.
 Andrei and probably 90% of existing D users don't want Phobos 
 to move this
 direction.
I don't understand that point. What direction?
1. Modularity: D should provide a very slim library (like std.traits and may be default GC with old core, which can be turned off or replaced). Other parts or Phobos/Druntime should be split into parts and the parts should have their own repositories. They should be dub packages with D Foundation review / control. 2. Low-level API first! A high level OOP/GC-oriented API can be provided later or can be provided in DUB libraries. 3. betterC guaranties / No problems with backward binary compatibility for betterC libs. This means that code should be templated / inlined. Some parts, like Exceptions and GC may have this problems, but Exceptions and GC should be optional language features and they are not a part of betterC mode.
 In other hand I need a commercial attractive D infrastructure 
 for large and
 heavy system projects. There is no commercial perspective for 
 me to contribute
 to Phobos because:

 1. Phobos version depends on compiler version. Delay with LDC 
 release is too
 large. It should / can be one day.
The language evolves over time, and the standard library must, too. It happens with every language.
Agreed. Because that it is more flexible to have a very slim std library with good set of community supported libraries. This is a reason why almost all portable multi-language numeric libraries are written in C. C runtime is slim and evaluates very-very slow.
 2. We can not do hot fixes in Phobos without compiling and 
 deploying patched
 libphobos. Mir's DUB package are much more flexible.

 3. Dependencies should be clear. Modularity is a proper way 
 for large std
 library. In phobos everything integrated with everything. 
 DRuntime -> Phobos
 abstraction is weird for betterC because system modules can 
 depends universal
 algorithms, but universal algorithm are more portable if they 
 have not system
 dependencies.
Not sure what you mean. Algorithms in Phobos are not dependent on system idiosyncracies.
They are, for example they depends on RangeErrors. See https://github.com/dlang/phobos/pull/4937
 4. Phobos does not provide and will not provide betterC 
 guaranties. If something
 works for betterC mode now it may not work in the future 
 release.
I've done a fair amount of work to remove such things. Clearly more can and should be done. Generally, we do have a goal of making Phobos entirely "pay as you go" instead of being so interconnected (which is what betterC is really all about). We can definitely use help in improving that and helping it along.
Great. Why not to go to the proposed direction: 1 Modularity, 2 Low-level API first, 3 betterC guaranties? Thank you, Ilya
Dec 18 2016
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:
 On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright wrote:
 On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:
 We already have better `cpuid` and better `random` packages.
Great! Please PR them for Phobos.
cpuid is used in Mir GLAS and it should be a betterC library.
It's this kind of imaginary dialog that I don't quite grok: Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my own." Andrei Alexandrescu: "OK, so what's different?" IY: "Mine has a few engineering improvements." AA: "Cool, why don't you merge them into core.cpuid?" IY: "Well mine doesn't have a shared static constructor, so it doesn't need a runtime to automatically call that library initialization function. User need to explicitly call an init function before using it." AA: "I understand. Great, so how about this - we add your code to core.cpuid_v2 to druntime. Then we make all in core.cpuid to forward to it so there's no duplication. It all works out!" IY: "No, I don't want to do that. It's still in druntime and I don't want druntime. I want betterC." AA: "But it will be compilable with betterC and we can add unittests to make that happen. YOU HAVE MY SUPPORT. Let's do it." IY: "No, I want to change it often. The deployment schedule of druntime is too slow." AA: "How often do you need to change it? Is it that unstable?" IY: "Um, not too often." AA: "Then what is the matter? Are you worried about the IP of the engineering improvements you are making? Are you licensing this differently?" IY: "No, it's for the most part similar to core.cpuid and the license will be also Boost." AA: "Then what is the matter? Do you want me to wait until you release a stable mir.cpuid and copy it over with credit, per the Boost license, to core.cpuid_v2?" IY: "..." It's this kind of stuff I need to have a better understanding of. Some technical arguments are meaningful, some others point to problems with obvious solutions that are somehow shunned, and yet others are like a hidden Markov model - I see the effects, but I don't see the causes. Andrei
Dec 18 2016
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 18 December 2016 at 19:28:24 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:
 On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright 
 wrote:
 On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:
 We already have better `cpuid` and better `random` packages.
Great! Please PR them for Phobos.
cpuid is used in Mir GLAS and it should be a betterC library.
It's this kind of imaginary dialog that I don't quite grok: Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my own." [...] IY: "..." It's this kind of stuff I need to have a better understanding of. Some technical arguments are meaningful, some others point to problems with obvious solutions that are somehow shunned, and yet others are like a hidden Markov model - I see the effects, but I don't see the causes. Andrei
My last argument was: ---- Linking compatibility. Assume A has builded a betterC library AA using GDC version XXX and B has builded a betterC library BB using LDC version YYY. Question: how mister C can use BB and AA together in his C-library if both this libraries depends on different DRuntimes (compiler and version)? ---- DRuntime is large and evaluates fast. We have not backward binary compatibility in DRuntime and we have not binary compatibility between different compiler versions. mir-glas is a betterC library for all languages. It depends in mir-cpuid. I want mir-glas to replace OpenBLAS. To do it I need the same portability. OK, assume we already have DRuntime for all targets including MIPS, POWER, armv7, aarch64, Loongson. Does this solves problem? No. The reason is very simple. All non-betterC D libraries must be compiled in lockstep: with the same compiler vendor and druntime version! This will just kill GLAS. How this can be packed for Ubuntu for example? It will depends on LDC libphobos == XXX. Another numeric library, say FFT, will depend on libphobos == YYY by GDC. How they can be linked if they require different DRuntime from different compilers? Or different druntime versions for the same compiler? Do you want to constraint D users to do not distribute binary libraries in popular repositories like Debian/Ubuntu? DRuntime incompatibility is a huge constraints for business. Do you want to force all companies to use identical version of compiler and identical vendor? Yes, C++ changes their API, but it few orders of magnitude slower then D. And they support multiple branches with bugs fixes. D supports only the recent one. This is why GLAS/cpuid need to be a betterC library. Ilya
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/2016 03:10 PM, Ilya Yaroshenko wrote:
 On Sunday, 18 December 2016 at 19:28:24 UTC, Andrei Alexandrescu wrote:
 On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:
 On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright wrote:
 On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:
 We already have better `cpuid` and better `random` packages.
Great! Please PR them for Phobos.
cpuid is used in Mir GLAS and it should be a betterC library.
It's this kind of imaginary dialog that I don't quite grok: Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my own." [...] IY: "..." It's this kind of stuff I need to have a better understanding of. Some technical arguments are meaningful, some others point to problems with obvious solutions that are somehow shunned, and yet others are like a hidden Markov model - I see the effects, but I don't see the causes. Andrei
My last argument was: ---- Linking compatibility. Assume A has builded a betterC library AA using GDC version XXX and B has builded a betterC library BB using LDC version YYY. Question: how mister C can use BB and AA together in his C-library if both this libraries depends on different DRuntimes (compiler and version)? ----
My understanding is that if cpuid_v2.d does not pull any dependency, it doesn't matter whether it sits in druntime or elsewhere. Is that the case?
 DRuntime is large and evaluates fast.
So is it moving too slow or too fast?
 We have not backward binary
 compatibility in DRuntime and we have not binary compatibility between
 different compiler versions.
Even if we don't, we can guarantee that core.cpuid_v2 adds no link-time dependency. We may offer it as a pure template library, or as C APIs. Whatever floats your boat. Is this enough for ensuring compatibility? Andrei
Dec 18 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 18 December 2016 at 20:16:11 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/2016 03:10 PM, Ilya Yaroshenko wrote:
 On Sunday, 18 December 2016 at 19:28:24 UTC, Andrei 
 Alexandrescu wrote:
 On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:
 On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright 
 wrote:
 On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:
 We already have better `cpuid` and better `random` 
 packages.
Great! Please PR them for Phobos.
cpuid is used in Mir GLAS and it should be a betterC library.
It's this kind of imaginary dialog that I don't quite grok: Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my own." [...] IY: "..." It's this kind of stuff I need to have a better understanding of. Some technical arguments are meaningful, some others point to problems with obvious solutions that are somehow shunned, and yet others are like a hidden Markov model - I see the effects, but I don't see the causes. Andrei
My last argument was: ---- Linking compatibility. Assume A has builded a betterC library AA using GDC version XXX and B has builded a betterC library BB using LDC version YYY. Question: how mister C can use BB and AA together in his C-library if both this libraries depends on different DRuntimes (compiler and version)? ----
My understanding is that if cpuid_v2.d does not pull any dependency, it doesn't matter whether it sits in druntime or elsewhere. Is that the case?
 DRuntime is large and evaluates fast.
So is it moving too slow or too fast?
It is too slow as library to contribute to and too fast as standard library to depend on.
 We have not backward binary
 compatibility in DRuntime and we have not binary compatibility 
 between
 different compiler versions.
Even if we don't, we can guarantee that core.cpuid_v2 adds no link-time dependency. We may offer it as a pure template library, or as C APIs. Whatever floats your boat. Is this enough for ensuring compatibility?
I need to understand how it is can be done because I did not find a valid solution. mir-cpuid has global symbols, they are accessed using extern(C) API. If multiple libraries trying to initialize it than it will be initialized only once. extern C API does not solve a problem with DRuntime because current DRuntime is not a betterC library and it is huge. A huge betterC library is a problem too because it is less portable. Anyway, if you find a proper solution please share it. How it can be implemented as template library? Does different libraries will have their own cpuid instantiations and __gshared symbols? How modularity can break existing Phobos? Why not to allow new packages like mir-cpuid and std.allocator to be a betterC library with their own repository? This will not break existing Phobos and Druntime but this will allow me to write server and numeric software without copy-pasting Phobos codes. I prefer to fork something rather then copypast. My plans to work on: 1. Low-level numeric libraries 2. Low-level fast async I/O adaptive generic event loop, asymmetric green threads, http2 3. Microelectromechanical systems They will require new threads, new timers, new mutexes, new event loop (existing C libs are slow), to be used. Existing one does not satisfy my requirements. I do not earn any money for this work. I am working on this projects because I believe in D and believe that my current and future clients will believe in D much more then now. I know what they need. Ilya
Dec 18 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/18/2016 1:02 PM, Ilya Yaroshenko wrote:
 I need to understand how it is can be done because I did not find a valid
solution.

 mir-cpuid has global symbols, they are accessed using extern(C) API. If
multiple
 libraries trying to initialize it than it will be initialized only once.

 extern C API does not solve a problem with DRuntime because current DRuntime is
 not a betterC library and it is huge. A huge betterC library is a problem too
 because it is less portable.
cpuid can be written in a manner that it is self-contained. If so, it doesn't matter how large druntime is, as it won't have references to any of the rest of it and so the rest won't be pulled in by the linker. https://github.com/dlang/druntime/blob/master/src/core/cpuid.d Currently, core.cpuid has no imports. It does have a static constructor, though, which references the startup code in druntime. This can be changed so it uses the static construction support in the C runtime library. Then it will be independent of druntime.
Dec 20 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 20 December 2016 at 12:00:43 UTC, Walter Bright wrote:
 On 12/18/2016 1:02 PM, Ilya Yaroshenko wrote:
 I need to understand how it is can be done because I did not 
 find a valid solution.

 mir-cpuid has global symbols, they are accessed using 
 extern(C) API. If multiple
 libraries trying to initialize it than it will be initialized 
 only once.

 extern C API does not solve a problem with DRuntime because 
 current DRuntime is
 not a betterC library and it is huge. A huge betterC library 
 is a problem too
 because it is less portable.
cpuid can be written in a manner that it is self-contained. If so, it doesn't matter how large druntime is, as it won't have references to any of the rest of it and so the rest won't be pulled in by the linker.
This is true for application but not for libraries. mir-glas depends on mir-cpuid. If I want GLAS to be ported for a new target then I will release a new CPUID version and will be able to use it the same day. With DRuntime (as solid project), I need to wait up to 6 months before it will be released with DMD and then with LDC. Workaround are possible, but they requires more efforts, more time, more code. Please do not force Mir to use DRuntime. We need a solid solution I do not force anyone to drop DRuntime: Mir libraries can be used with DRuntime/Phobos. We have full backward compatibility. DRuntime does not satisfy portability requirements (which was repeated multiple times in this thread). DRuntime (at least for now and future few years) is a big constraint for betterC libs to replace existing C analogs. Please do not force me to use DRuntime.
Dec 20 2016
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/20/2016 4:30 AM, Ilya Yaroshenko wrote:
 This is true for application but not for libraries. mir-glas depends on
 mir-cpuid. If I want GLAS to be ported for a new target then I will release a
 new CPUID version and will be able to use it the same day. With DRuntime (as
 solid project), I need to wait up to 6 months before it will be released with
 DMD and then with LDC. Workaround are possible, but they requires more efforts,
 more time, more code.

 Please do not force Mir to use DRuntime. We need a solid solution

 I do not force anyone to drop DRuntime: Mir libraries can be used with
 DRuntime/Phobos. We have full backward compatibility. DRuntime does not satisfy
 portability requirements (which was repeated multiple times in this thread).
 DRuntime (at least for now and future few years) is a big constraint for
betterC
 libs to replace existing C analogs.
Here's how anyone can override druntime's cpuid for their application without touching druntime: dmd -c mycpuid dmd myapp mycpuid.o I.e. the way linkers work is if the symbol's definition exists, it is not pulled in from the library.
 Please do not force me to use DRuntime.
I don't propose forcing anyone to use Druntime. What I do propose is reducing interdependencies among the object files in Druntime. Druntime is not a monolithic chunk of code, it is a library of object files. I don't see the need to rewrite cpuid because its sole dependency on other object files in Druntime is the static constructor, because the static constructor can be adjusted to not rely on anything else in Druntime. If there are other issues with cpuid, please elucidate.
Dec 20 2016
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 20 December 2016 at 12:49:53 UTC, Walter Bright wrote:

 I don't see the need to rewrite cpuid because its sole 
 dependency on other object files in Druntime is the static 
 constructor, because the static constructor can be adjusted to 
 not rely on anything else in Druntime.

 If there are other issues with cpuid, please elucidate.
1. https://issues.dlang.org/show_bug.cgi?id=16028 2. The code is very hard to maintain, fix, extend. This is a reason why mir-cpuid was written from scratch without any copy-pasting. 3. Library is x86 oriented. Some OSs has interface to fetch unified CPU information. 4. No TLB information provided. 5. Number of CPUs per cache is not know. ARM CPUs may have complex topology (plus 3.)
Dec 20 2016
parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/20/2016 5:35 AM, Ilya Yaroshenko wrote:
 On Tuesday, 20 December 2016 at 12:49:53 UTC, Walter Bright wrote:

 I don't see the need to rewrite cpuid because its sole dependency on other
 object files in Druntime is the static constructor, because the static
 constructor can be adjusted to not rely on anything else in Druntime.

 If there are other issues with cpuid, please elucidate.
1. https://issues.dlang.org/show_bug.cgi?id=16028 2. The code is very hard to maintain, fix, extend. This is a reason why mir-cpuid was written from scratch without any copy-pasting. 3. Library is x86 oriented. Some OSs has interface to fetch unified CPU information. 4. No TLB information provided. 5. Number of CPUs per cache is not know. ARM CPUs may have complex topology (plus 3.)
Thank you, but I meant issues that would require it to be separate from druntime, not bug fixes or enhancements to account for ever more things being added to the CPUID instruction. BTW, of course it is x86 oriented. Any use of cpuid must be. After all, what does "SSSE3.1" mean on ARM?
Dec 20 2016
prev sibling parent Mike <none none.com> writes:
On Tuesday, 20 December 2016 at 12:49:53 UTC, Walter Bright wrote:
 I don't propose forcing anyone to use Druntime.
Perhaps this is not part of Ilya's concern, but druntime is required to get a build. //----test.d--------- module test; void main() { } dmd -m64 -defaultlib= -debuglib= -conf= test.d Error: cannot find source code for runtime library file 'object.d' dmd might not be correctly installed. Run 'dmd -man' for installation instructions. config file: (null) Specify path to file 'object.d' with -I switch There's nothing here that really requires druntime, unless you want to count the plumbing required to call DMain. In that case, we can just call CMain directly: module test; extern(C) void main() { } Same Error: Error: cannot find source code for runtime library file 'object.d' Try with the -betterC switch: dmd -m64 -defaultlib= -debuglib= -conf= -betterC test.d Same Error: Error: cannot find source code for runtime library file 'object.d' Ok, fine, let's add an empty object.d //object.d module object; //test.d module test; extern(C) void main() { } dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d Error: undefined identifier 'Error' Error: undefined identifier 'Error' Ok, now I need something called 'Error'. I don't see anywhere in my code where that's necessary, but it must be important because it told me twice. //object.d module object; class Throwable { } class Error : Throwable { } //test.d module test; extern(C) void main() { } dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d object.d(3): Error: class object.Throwable missing or corrupt object.d Ok, that's weird. I just added it. Luckily I've done this before, so I know what to do. //object.d module object; class Object { } class Throwable { } class Error : Throwable { this(string x) { } } //test.d module test; extern(C) void main() { } dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d Error: no constructor for Error Error: no constructor for Error Hmm. Again with the double error messages. Ok, let's add a constructor. //object.d module object; class Object { } class Throwable { } class Error : Throwable { this(string x) { } } //test.d module test; extern(C) void main() { } dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d object.d(9): Error: undefined identifier 'string' At least its only telling me once. Easy fix. //object.d module object; alias immutable(char)[] string; class Object { } class Throwable { } class Error : Throwable { this(string x) { } } //test.d module test; extern(C) void main() { } dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d Error: TypeInfo not found. object.d may be incorrectly installed or corrupt, compile with -v switch I could go on, but I'm not going to. The final result to get a build looks like this, but you have to link with --gc-sections to avoid having to implement a buch of other stuff. module object; extern(C) void __dmd_personality_v0() { } extern(C) void _d_dso_registry() { } alias immutable(char)[] string; class Object { } class TypeInfo { } class TypeInfo_Class : TypeInfo { ubyte[136] ignore; } alias TypeInfo_Class ClassInfo; class Throwable { } class Error : Throwable { this(string x) { } } //test.d module test; extern(C) void main() { } dmd -m64 -defaultlib= -debuglib= -conf= -betterC object.d test.d -L=--gc-sections If you add structs or classes to your code, you have to add even more druntime stuff. The point I'm trying to make here is druntime is required just to get a build, even though your code doesn't need it. Mike
Dec 20 2016
prev sibling parent Andrey <avraliov gmail.com> writes:
On Sunday, 18 December 2016 at 19:28:24 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/2016 01:49 PM, Ilya Yaroshenko wrote:
 On Sunday, 18 December 2016 at 17:41:31 UTC, Walter Bright 
 wrote:
 On 12/18/2016 1:26 AM, Ilya Yaroshenko wrote:
 We already have better `cpuid` and better `random` packages.
Great! Please PR them for Phobos.
cpuid is used in Mir GLAS and it should be a betterC library.
It's this kind of imaginary dialog that I don't quite grok: Ilya Yaroshenko: "I can't use druntime's cpuid so I defined my own." Andrei Alexandrescu: "OK, so what's different?" IY: "Mine has a few engineering improvements." AA: "Cool, why don't you merge them into core.cpuid?" IY: "Well mine doesn't have a shared static constructor, so it doesn't need a runtime to automatically call that library initialization function. User need to explicitly call an init function before using it." AA: "I understand. Great, so how about this - we add your code to core.cpuid_v2 to druntime. Then we make all in core.cpuid to forward to it so there's no duplication. It all works out!" IY: "No, I don't want to do that. It's still in druntime and I don't want druntime. I want betterC." AA: "But it will be compilable with betterC and we can add unittests to make that happen. YOU HAVE MY SUPPORT. Let's do it." IY: "No, I want to change it often. The deployment schedule of druntime is too slow." AA: "How often do you need to change it? Is it that unstable?" IY: "Um, not too often." AA: "Then what is the matter? Are you worried about the IP of the engineering improvements you are making? Are you licensing this differently?" IY: "No, it's for the most part similar to core.cpuid and the license will be also Boost." AA: "Then what is the matter? Do you want me to wait until you release a stable mir.cpuid and copy it over with credit, per the Boost license, to core.cpuid_v2?" IY: "..." It's this kind of stuff I need to have a better understanding of. Some technical arguments are meaningful, some others point to problems with obvious solutions that are somehow shunned, and yet others are like a hidden Markov model - I see the effects, but I don't see the causes. Andrei
I think, the reason of misunderstanding is different skills in a subject. And even different subjects. Some experts are compilers developers, some are library developers. They all in different subject. In c++ community there is a group of compiler developers experts who desides, how to implement this or that feature, and the ability of implementation. They know how to develop compilers. It doesn't mean they are more clever than library developers, just different skills. And when Andrei asking to explain the reason of some proposal, he waits arguments in _terminology_ of comliler developer. _terminology_ understanding (in wide) means _experience_ (compiler developing in this case). If no experience in the subject, there is a way to get it, and the other way to ask a recommendation of experienced colleague.
Dec 18 2016
prev sibling parent reply Seb <seb wilzba.ch> writes:
On Sunday, 18 December 2016 at 18:49:33 UTC, Ilya Yaroshenko 
wrote:
 1. Modularity: D should provide a very slim library (like 
 std.traits and may be default GC with old core, which can be 
 turned off or replaced). Other parts or Phobos/Druntime should 
 be split into parts and the parts should have their own 
 repositories. They should be dub packages with D Foundation 
 review / control.
This would be awesome to have!! tl:dr: Phobos is very bloated. Let's do something about it! I absolutely agree that there's a lot of stuff that doesn't belong into Phobos. Also many parts are outdated or have ugly APIs, but they can only be replaced if a superior community solution has evolved. So why not "dubify" the optional parts of Phobos? Solved Problem: Versioning -------------------------- This also solves the versioning problem Andrei discussed last month nicely. For example when std.allocator was merged into Phobos a lot of builds broke because packages did can their dependency from dub to std.x.allocator, but many people were still compiling with an older compiler. Solved Problem: High quality community libraries ------------------------------------------------ Moreover, at the last DConf there were many voices that D is lacking good community libraries (aka the gold libraries). So the Phobos collection could be one. Btw one shouldn't forgot that people have already written a lot of replacements of Phobos modules and imho in most cases it made sense! A quick list ------------ (this list is incomplete and intended as an example to show that for most parts of Phobos better solutions co-exists) etc.c.sqlite (aka std.database) -> https://github.com/buggins/ddbc (and: https://wiki.dlang.org/Database_Libraries) etc.c.zlib / std.zip -> https://github.com/rcythr/archive std.base64 -> https://code.dlang.org/packages/base-d std.bigint -> https://github.com/andersonpd/eris/blob/master/integer/extended.d std.csv -> https://github.com/eBay/tsv-utils-dlang std.container -> https://github.com/economicmodeling/containers (uses allocators!) std.complex I am pretty sure this is on Ilya's list as well ;-) std.encoding: -> https://github.com/e10s/d-base32 std.getopt -> https://github.com/jasonwhite/darg -> https://github.com/SirTony/commando etc. std.json: -> https://github.com/s-ludwig/std_data_json std.math: -> https://code.dlang.org/packages/ctstdmath -> https://github.com/libmir/mir-math std.net.curl: -> https://github.com/ikod/dlang-requests std.net.isemail: -> https://github.com/anton-dutov/mail -> http://vibed.org/api/vibe.mail.smtp/Mail std.signals: -> https://code.dlang.org/packages/phobosx -> https://code.dlang.org/packages/observe std.socket: -> http://vibed.org/api/vibe.http.websockets/ -> http://vibed.org/api/vibe.core.net/TCPConnection std.stdio: -> https://github.com/jasonwhite/io -> https://github.com/schveiguy/iopipe -> https://github.com/rejectedsoftware/vibe.d/blob/master/stream/vibe/stream/stdio.d std.uri: -> https://github.com/rikkimax/alphaPhobos/blob/master/source/std/experimental/uri.d -> http://vibed.org/api/vibe.inet.url/ -> https://github.com/adamdruppe/arsd/blob/master/http2.d std.variant -> https://github.com/s-ludwig/taggedalgebraic std.xml -> https://github.com/lodo1995/experimental.xml -> https://github.com/jacob-carlborg/orange Also a lot of proposed Phobos modules already exist in the community: std.color https://github.com/TurkeyMan/color std.decimal https://github.com/andersonpd/eris/ https://github.com/jaypha/fixed std.events (planned?) -> https://github.com/etcimon/libasync Of course many of these libraries are one man projects and aren't considered "high-quality", but the reason they exist in the first place shows that a lot of things just shouldn't be "standardized", but more provided with a "practical variant". We should focus solely on modules that can be standardized easily (std.traits, std.ranges, ...) or are required to create a unified experience (e.g. a common allocation/logging interface). Imho there's really no need to ship sth. like SQLite as part of the standard library. Final remarks ------------- - From a technical point of view there shouldn't be a problem to have a unified documentation experience etc. - Thanks to Martin's awesome work, we already test a couple of "selected" DUB packages on every DMD/Druntime/Phobos commit
Dec 18 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/18/16 7:31 PM, Seb wrote:
 tl:dr: Phobos is very bloated.
How does the D standard library compare in size with some other languages (C++, Rust, Go, Java, Python)?
 Let's do something about it!
I've said this several times, and it bears repeating. We want to make D entirely pay-as-you go, and offer full support to anyone working on it. The best "something" anyone can do about it is do good engineering work in the D mainline. That will automatically benefit all users of the D language, instead of balkanizing the community. Granted, the lure of the clean slate is strong, and more care is needed to mind backwards compatibility, but that's where the real impact is. Andrei
Dec 18 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 01:52:21 UTC, Andrei Alexandrescu 
wrote:
 On 12/18/16 7:31 PM, Seb wrote:
 tl:dr: Phobos is very bloated.
How does the D standard library compare in size with some other languages (C++, Rust, Go, Java, Python)?
C++ and Rust are the best from this list in terms of low level programming. They have smaller libraries comparing with D. Go, Java, Python --- If I need a language like them I will you them, not D. Go - awesome GC, commercially supported, easy to write your own libs Java - awesome GC, commercially supported, Scala(!!!) Python - awesome scripting, commercially supported D - ??? awesome what ? What do we have better then Scala? Performance? NO. BRandomAccessRanges makes our performance less then C/C++. Clean and clever API in standard library? NO. We are placing our Range API whenever possible including RNGs. We have not a language standard for I/O. Half of Phobos is outdated to language standards. Best generics? NO. Scala is more powerful. ---------- What do we have awesome? Where D is the best? ---------- D is awesome for low-level system libraries in betterC mode. (and small text processing routines) This will make D great in other fields. ----------
 Let's do something about it!
I've said this several times, and it bears repeating. We want to make D entirely pay-as-you go, and offer full support to anyone working on it. The best "something" anyone can do about it is do good engineering work in the D mainline.
National D Foundation Party.
 That will automatically benefit all users of the D language, 
 instead of balkanizing the community.
...
 Granted, the lure of the clean slate is strong, and more care 
 is needed to mind backwards compatibility, but that's where the 
 real impact is.


 Andrei
How Mir projects breaks backward compatibility ? How proposed direction breaks backward compatibility? Lets use other prefix for the new libraries. Mir uses `mir.` And allow Phobos to be a sandbox for C++ as it is now. Ilya
Dec 18 2016
next sibling parent yawniek <yawniek srtnwz.com> writes:
On Monday, 19 December 2016 at 05:59:07 UTC, Ilya Yaroshenko 
wrote:
 D - ??? awesome what ? What do we have better then Scala?

 Performance? NO.
 BRandomAccessRanges makes our performance less then C/C++.

 Clean and clever API in standard library? NO.
 We are placing our Range API whenever possible including RNGs. 
 We have not a language standard for I/O. Half of Phobos is 
 outdated to language standards.

 Best generics? NO.
 Scala is more powerful.

 ----------
 What do we have awesome? Where D is the best?
 ----------
 D is awesome for low-level system libraries in betterC mode.
 (and small text processing routines)
 This will make D great in other fields.
while i mostly use D as a awesome high level language (with mediocre libs, and an inconsistent stdlib) i also think that D has an edge when it comes to building fast system level tools that used to be written in c: microservices, datatabases etc. however for that, as ilya mentioned it needs to be super user friendly to compile down a single static object/binary with no dependencies and a clear line on when phobos can/should/must be used and when not (e.g. create a view on phobos documentation that only shows things that work without druntime. add some official docs on how to use it) people turn to C to extend their language of choice because they need speed in a very narrow area. if that becomes as easy (or even easier, since there is dub!) as with C it would be a huge win for D, because people would start to extend their stuff with D. No other language is in a such good position as D, from what i see not that much needs to be done to make that work. i see a huge opportunity there
Dec 19 2016
prev sibling parent reply lurker <lurk lurking.org> writes:
On Monday, 19 December 2016 at 05:59:07 UTC, Ilya Yaroshenko 
wrote:
 On Monday, 19 December 2016 at 01:52:21 UTC, Andrei 
 Alexandrescu wrote:
 On 12/18/16 7:31 PM, Seb wrote:
 tl:dr: Phobos is very bloated.
How does the D standard library compare in size with some other languages (C++, Rust, Go, Java, Python)?
C++ and Rust are the best from this list in terms of low level programming. They have smaller libraries comparing with D. Go, Java, Python --- If I need a language like them I will you them, not D. Go - awesome GC, commercially supported, easy to write your own libs Java - awesome GC, commercially supported, Scala(!!!) Python - awesome scripting, commercially supported D - ??? awesome what ? What do we have better then Scala? Performance? NO. BRandomAccessRanges makes our performance less then C/C++. Clean and clever API in standard library? NO. We are placing our Range API whenever possible including RNGs. We have not a language standard for I/O. Half of Phobos is outdated to language standards. Best generics? NO. Scala is more powerful. ---------- What do we have awesome? Where D is the best? ---------- D is awesome for low-level system libraries in betterC mode. (and small text processing routines) This will make D great in other fields. ----------
 Let's do something about it!
I've said this several times, and it bears repeating. We want to make D entirely pay-as-you go, and offer full support to anyone working on it. The best "something" anyone can do about it is do good engineering work in the D mainline.
National D Foundation Party.
 That will automatically benefit all users of the D language, 
 instead of balkanizing the community.
...
 Granted, the lure of the clean slate is strong, and more care 
 is needed to mind backwards compatibility, but that's where 
 the real impact is.


 Andrei
How Mir projects breaks backward compatibility ? How proposed direction breaks backward compatibility? Lets use other prefix for the new libraries. Mir uses `mir.` And allow Phobos to be a sandbox for C++ as it is now. Ilya
<delurk> Thats why I asked for moderation in forums. <lurk>
Dec 19 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 21:00:57 UTC, lurker wrote:
 On Monday, 19 December 2016 at 05:59:07 UTC, Ilya Yaroshenko 
 wrote:
 [...]
<delurk> Thats why I asked for moderation in forums. <lurk>
+1, and ability to edit own posts.
Dec 19 2016
prev sibling next sibling parent reply Mike <none none.com> writes:
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:

 Who is interested in betterC _modular_* standard library?
What exactly do you mean by "betterC"? As far as I know, the `-betterC` switch just removes ModuleInfo generation. Do you actually mean, a standard library that has fewer dependencies on druntime features? Mike
Dec 18 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 01:33:42 UTC, Mike wrote:
 On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
 wrote:

 Who is interested in betterC _modular_* standard library?
What exactly do you mean by "betterC"? As far as I know, the `-betterC` switch just removes ModuleInfo generation.
Because -betterC was not implemented yet.
 Do you actually mean, a standard library that has fewer 
 dependencies on druntime features?
I mean this too Ilya
Dec 18 2016
parent Mike <none none.com> writes:
On Monday, 19 December 2016 at 05:22:10 UTC, Ilya Yaroshenko 
wrote:
 What exactly do you mean by "betterC"?  As far as I know, the 
 `-betterC` switch just removes ModuleInfo generation.
Because -betterC was not implemented yet.
I'm curious to know what a complete implementation entails.
 Do you actually mean, a standard library that has fewer 
 dependencies on druntime features?
I mean this too
I've been wanting to decouple from druntime since I started learning D in late 2013, though probably for different reasons than yours. There have been two recent posts [1] [2] wanting to do the same thing, and now your post envisioning a more modular druntime-less phobos. [1] - https://forum.dlang.org/post/drqdrobcoaanenocycgp forum.dlang.org [2] - https://forum.dlang.org/post/tmofjecvnqdthvetezfp forum.dlang.org It's encouraging to see this trend. I am interested in seeing D become more modular and pay-as-you-go, though I'd prefer to see changes in the compiler and druntime (e.g. [3] [4] [5]) before phobos. [3] - http://forum.dlang.org/post/mr71kt$2rgi$1 digitalmars.com [4] - http://forum.dlang.org/post/eiwalbqlbkipdrmsrfoh forum.dlang.org [5] - https://issues.dlang.org/show_bug.cgi?id=11666 Mike
Dec 18 2016
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
wrote:
 Hi,

 Who is interested in betterC _modular_* standard library?
 I am planing to make libmir org a community for it.
 Thought and concerns?

 We already have better `cpuid` and better `random` packages. 
 The betterC std.range and std.algorithm analogs would be 
 released with new ndslice implementation. Mir's algorithm would 
 be faster then Phobos and will generate less template bloat. 
 Then lightweight threads, multithread GLAS, matrix inversion. 
 Fastest I/O and http2 ...

 (they all are betterC modular libraries)

 Andrei and probably 90% of existing D users don't want Phobos 
 to move this direction.

 In other hand I need a commercial attractive D infrastructure 
 for large and heavy system projects. There is no commercial 
 perspective for me to contribute to Phobos because:
If I understand correctly, the only reaon not to disable the runtime instead of going betterC (what I call runtime-free vs runtime-less) is because Phobos and druntime aren't ported to every platform you need, but the libC is. That it calling for the question: "Isn't porting Phobos and druntime to those new platforms your clients need less work?" Maybe they only need ARM and x86-64, the effort should be minimal then? And with the runtime disabled, things like the GC just need to link, not actually work :) Tha makes porting a bit simpler. Things like std.algorithm, std.range, traits etc... took many man-years to do and can work as is. core.cpuid can work with little modifications. I'm not saying you can use a lot of Phobos and druntime without using the runtime, because currently you can't. But think about it: when not linking the runtime, there isn't even a way to call "emplace" or "destroy", dynamic casts won't work, etc. In the future more and more things will work without the runtime, exceptions, class creation and then you'll don't have to maintain a stdlib. Please reconsider reconsidering.
Dec 19 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 19:31:24 UTC, Guillaume Piolat 
wrote:
 On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
 wrote:
 [...]
If I understand correctly, the only reaon not to disable the runtime instead of going betterC (what I call runtime-free vs runtime-less) is because Phobos and druntime aren't ported to every platform you need, but the libC is.
No, 1. DRuntime has not fixed API 2. It has not backward binary compatibility 3. It has not cross-compiler binary compatibility. So, it can not be used.
Dec 19 2016
next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 19 December 2016 at 19:42:42 UTC, Ilya Yaroshenko 
wrote:
 No,
 1. DRuntime has not fixed API
Can't you static link with it? If you static link with it, your client don't deal with DRuntime.
 2. It has not backward binary compatibility
Ditto. If you static link with it, your client don't deal with binary compatibility.
 3. It has not cross-compiler binary compatibility.
Mandate that every D code in the project is built with the same compiler, which is realistic.
Dec 19 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 19:48:39 UTC, Guillaume Piolat 
wrote:
 On Monday, 19 December 2016 at 19:42:42 UTC, Ilya Yaroshenko 
 wrote:
 No,
 1. DRuntime has not fixed API
Can't you static link with it? If you static link with it, your client don't deal with DRuntime.
 2. It has not backward binary compatibility
Ditto. If you static link with it, your client don't deal with binary compatibility.
 3. It has not cross-compiler binary compatibility.
Mandate that every D code in the project is built with the same compiler, which is realistic.
How this can work for libraries? Please read this thread. This is not realistic. This a huge constraint for D libraries to be modular, replaceable and distributed in binary form.
Dec 19 2016
next sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 19:52:43 UTC, Ilya Yaroshenko 
wrote:
 On Monday, 19 December 2016 at 19:48:39 UTC, Guillaume Piolat 
 wrote:
 On Monday, 19 December 2016 at 19:42:42 UTC, Ilya Yaroshenko 
 wrote:
 No,
 1. DRuntime has not fixed API
Can't you static link with it? If you static link with it, your client don't deal with DRuntime.
 2. It has not backward binary compatibility
Ditto. If you static link with it, your client don't deal with binary compatibility.
 3. It has not cross-compiler binary compatibility.
Mandate that every D code in the project is built with the same compiler, which is realistic.
How this can work for libraries? Please read this thread. This is not realistic. This a huge constraint for D libraries to be modular, replaceable and distributed in binary form.
To be more precise: I do not build library exclusively for D. They should be easily interacted with other languages.
Dec 19 2016
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 19 December 2016 at 19:54:54 UTC, Ilya Yaroshenko 
wrote:
 To be more precise: I do not build library exclusively for D. 
 They should be easily interacted with other languages.
Non sequitur, you can enable the runtime/attah threads withing callbacks.
Dec 19 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 19:59:01 UTC, Guillaume Piolat 
wrote:
 On Monday, 19 December 2016 at 19:54:54 UTC, Ilya Yaroshenko 
 wrote:
 To be more precise: I do not build library exclusively for D. 
 They should be easily interacted with other languages.
Non sequitur, you can enable the runtime/attah threads withing callbacks.
No, DRuntime based numerical libraries will never replace C analogs.
Dec 19 2016
parent Guillaume Piolat <first.last gmail.com> writes:
On Monday, 19 December 2016 at 20:10:02 UTC, Ilya Yaroshenko 
wrote:
 On Monday, 19 December 2016 at 19:59:01 UTC, Guillaume Piolat 
 wrote:
 On Monday, 19 December 2016 at 19:54:54 UTC, Ilya Yaroshenko 
 wrote:
 To be more precise: I do not build library exclusively for D. 
 They should be easily interacted with other languages.
Non sequitur, you can enable the runtime/attah threads withing callbacks.
No, DRuntime based numerical libraries will never replace C analogs.
Going to bed. Please don't take it as attacks, I'm juste trying to help find the best possible solution, with the least amount of pain.
Dec 19 2016
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 19 December 2016 at 19:52:43 UTC, Ilya Yaroshenko 
wrote:
 How this can work for libraries? Please read this thread.
 This is not realistic. This a huge constraint for D libraries 
 to be modular, replaceable and distributed in binary form.
They can be at the MIR level. How this can work: Mir libraries are distributed as static and dynamic libs, whatever. Each one of these links statically with the same druntime and libphobos. When the user replace one MIR library it also replaces the others. I don't see what isn't realistic here. See also: CUDA, Intel Performance Primitives... People update CUDA in bunch, they don't use cuDNN from release 6.15 with cuRuntime from release 7.8
Dec 19 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 19:57:45 UTC, Guillaume Piolat 
wrote:
 On Monday, 19 December 2016 at 19:52:43 UTC, Ilya Yaroshenko 
 wrote:
 How this can work for libraries? Please read this thread.
 This is not realistic. This a huge constraint for D libraries 
 to be modular, replaceable and distributed in binary form.
They can be at the MIR level. How this can work: Mir libraries are distributed as static and dynamic libs, whatever. Each one of these links statically with the same druntime and libphobos. When the user replace one MIR library it also replaces the others. I don't see what isn't realistic here. See also: CUDA, Intel Performance Primitives... People update CUDA in bunch, they don't use cuDNN from release 6.15 with cuRuntime from release 7.8
Mir CPUID was created for Mir GLAS, which is BLAS implementation. How Mir GLAS be a default BLAS implementation in linux distributives if it is depends on DRuntime? What about other non Mir numerical library written in Dlang, say for FFT? How they should be linked together. This example is already in this thread multiple times. Do you propose to constraint Dlang library distribution in binary form? I do not write D libraries for Dlang. I wrote them for people, which uses different languages. Most of them do not know about D.
Dec 19 2016
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 19 December 2016 at 20:05:20 UTC, Ilya Yaroshenko 
wrote:
 What about other non Mir numerical library written in Dlang, 
 say for FFT?

 How they should be linked together.
If Mir libs are static libs this can indeed be a problem. If someone has made another D library like this, it will either have to be its own dynlib or come with a runtime with the same API. You can manage to use very little of druntime such as "emplace"/"destroy" and that's about it when it comes to API. I'll concede that my solution will not get you packaged on official package lists for Debian/Ubuntu, because shared libraries will get preferred there. But I just checked and libcublas is just put in the non-free packages. https://packages.debian.org/jessie/libcublas6.0 If it's good enough for CUDA then maybe it's good enough for MIR? Note that with MIR being several dynlibs everyone of Mir library can come with any runtime (and then use it or not) without interaction of these runtimes. When you use a CUDA dynamic library you don't know if it has a C runtime inside (or a X runtime, whatever).
 Do you propose to constraint Dlang library distribution in 
 binary form?
No. Not sure how that question ensues from the premises.
Dec 19 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 20:24:21 UTC, Guillaume Piolat 
wrote:
 On Monday, 19 December 2016 at 20:05:20 UTC, Ilya Yaroshenko 
 wrote:
 [...]
If Mir libs are static libs this can indeed be a problem. If someone has made another D library like this, it will either have to be its own dynlib or come with a runtime with the same API. You can manage to use very little of druntime such as "emplace"/"destroy" and that's about it when it comes to API. [...]
No, Mir is not a hardware manufacturer which forces its users to do something.
 Note that with MIR being several dynlibs everyone of Mir 
 library can come with any runtime (and then use it or not) 
 without interaction of these runtimes.

 [...]
Looks like bad design. Existing C libs would be better then.
Dec 19 2016
prev sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 19 December 2016 at 20:42, Ilya Yaroshenko via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 19 December 2016 at 19:31:24 UTC, Guillaume Piolat wrote:
 On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko wrote:
 [...]
If I understand correctly, the only reaon not to disable the runtime instead of going betterC (what I call runtime-free vs runtime-less) is because Phobos and druntime aren't ported to every platform you need, but the libC is.
No, 1. DRuntime has not fixed API 2. It has not backward binary compatibility 3. It has not cross-compiler binary compatibility. So, it can not be used.
You're confusing druntime with phobos. I haven't changed any bindings to druntime in years...
Dec 19 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Monday, 19 December 2016 at 20:22:29 UTC, Iain Buclaw wrote:
 On 19 December 2016 at 20:42, Ilya Yaroshenko via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:
 On Monday, 19 December 2016 at 19:31:24 UTC, Guillaume Piolat 
 wrote:
 On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko 
 wrote:
 [...]
If I understand correctly, the only reaon not to disable the runtime instead of going betterC (what I call runtime-free vs runtime-less) is because Phobos and druntime aren't ported to every platform you need, but the libC is.
No, 1. DRuntime has not fixed API 2. It has not backward binary compatibility 3. It has not cross-compiler binary compatibility. So, it can not be used.
You're confusing druntime with phobos. I haven't changed any bindings to druntime in years...
If someone build a library with GCC and another one with Clang they will work together. What about LDC and GDC?
Dec 19 2016
parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 19 December 2016 at 21:29, Ilya Yaroshenko via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 19 December 2016 at 20:22:29 UTC, Iain Buclaw wrote:
 On 19 December 2016 at 20:42, Ilya Yaroshenko via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Monday, 19 December 2016 at 19:31:24 UTC, Guillaume Piolat wrote:
 On Sunday, 18 December 2016 at 09:26:09 UTC, Ilya Yaroshenko wrote:
 [...]
If I understand correctly, the only reaon not to disable the runtime instead of going betterC (what I call runtime-free vs runtime-less) is because Phobos and druntime aren't ported to every platform you need, but the libC is.
No, 1. DRuntime has not fixed API 2. It has not backward binary compatibility 3. It has not cross-compiler binary compatibility. So, it can not be used.
You're confusing druntime with phobos. I haven't changed any bindings to druntime in years...
If someone build a library with GCC and another one with Clang they will work together. What about LDC and GDC?
Clang developers decided that they should be ABI compatible with GCC. I'm not sure if the LDC developers are doing the same thing. :-)
Dec 19 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: A betterC modular standard library?
References: <phexetutyelrssyrucvw forum.dlang.org>
In-Reply-To: <phexetutyelrssyrucvw forum.dlang.org>

--qhnfOq2CEv5mloVi7rH0OwT2qiGQnsjT4
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 12/18/2016 11:26 AM, Ilya Yaroshenko wrote:
 Hi,
=20
 Who is interested in betterC _modular_* standard library?
 I am planing to make libmir org a community for it.
 Thought and concerns?
I also consider Phobos a lost cause only suitable for scripting purposes. However what you try to do won't be acceptable for me either (see below).
 We already have better `cpuid` and better `random` packages. The better=
C
 std.range and std.algorithm analogs would be released with new ndslice
 implementation. Mir's algorithm would be faster then Phobos and will
 generate less template bloat. Then lightweight threads, multithread
 GLAS, matrix inversion. Fastest I/O and http2 ...
I consider only algorithms and interfaces suitable for standard library. GLAS, I/O, HTTP - this is exactly all the stuff I want to see removed into separate dub packages and maintained as such.
 (they all are betterC modular libraries)
=2E. and this "betterC" weirdness you use with your own meaning is definitely something I don't want to _ever_ see as standard library approach. It is similar to `malloc` vs `GC` debate for Phobos, because correct answer tends to be "neither", with rationale that any library utility that has to care about the difference is hardly generic enough for unconditional distribution. ------------------------ My own dream path for fixing Phobos may look like this: 1) Start with separating environment-agnostic core (algorithms and API's) from utilities 2) Rewrite all higher level utilities in a way that they don't have any inter-dependencies and only allowed to use algorithm/API modules 3) Move higher level stuff into own dub packages with a special "Phobos" category 4) When compiler is released, provide snapshot of latest up to date packages in that category as part of the distribution for those who don't want to bother with fetching manually This actually may be even possible to do within a linear deprecation process. On the other hand I doubt such drastic change to structure is possible to be sold to community. --qhnfOq2CEv5mloVi7rH0OwT2qiGQnsjT4--
Dec 20 2016
parent reply Paolo Invernizzi <paolo.invernizzi no.address> writes:
On Tuesday, 20 December 2016 at 08:15:35 UTC, Dicebot wrote:
 On 12/18/2016 11:26 AM, Ilya Yaroshenko wrote:>

 This actually may be even possible to do within a linear 
 deprecation process. On the other hand I doubt such drastic 
 change to structure is possible to be sold to community.
Consider that plan sold to me... --- Paolo
Dec 20 2016
parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
 ilya: regarding -betterC 's stated goal to bypass druntime:

besides static linking (which can be inefficient), why not use runtime
shared libraries to distribute mir (eg GLAS) to C clients ?

IIRC, from a single C++ program, I am able to dlopen & dlsym & run D
functions defined in shared libraries built with different compilers (eg
dmd and ldc) even if they use GC (and therefore dmd's and ldc's druntime) ;
pseudocode:

// main.cpp
```
void main(){
  dlopen, dlsym, and run "void fun1()" from libfun_dmd.so
  dlopen, dlsym, and run "void fun2()" from libfun_ldc.so
}
```

That would provide the needed isolation without suffering from
disadvantages of static libraries (eg bloat); compared to the betterC
alternative, you'd have access to all of druntime/phobos and not have to
reinvent the wheel


On Tue, Dec 20, 2016 at 1:06 AM, Paolo Invernizzi via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 20 December 2016 at 08:15:35 UTC, Dicebot wrote:

 On 12/18/2016 11:26 AM, Ilya Yaroshenko wrote:>

 This actually may be even possible to do within a linear deprecation
 process. On the other hand I doubt such drastic change to structure is
 possible to be sold to community.
Consider that plan sold to me... --- Paolo
Dec 20 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 20 December 2016 at 09:12:55 UTC, Timothee Cour wrote:
  ilya: regarding -betterC 's stated goal to bypass druntime:

 besides static linking (which can be inefficient), why not use 
 runtime
 shared libraries to distribute mir (eg GLAS) to C clients ?

 IIRC, from a single C++ program, I am able to dlopen & dlsym & 
 run D functions defined in shared libraries built with 
 different compilers (eg dmd and ldc) even if they use GC (and 
 therefore dmd's and ldc's druntime) ; pseudocode:

 // main.cpp
 ```
 void main(){
   dlopen, dlsym, and run "void fun1()" from libfun_dmd.so
   dlopen, dlsym, and run "void fun2()" from libfun_ldc.so
 }
 ```

 That would provide the needed isolation without suffering from 
 disadvantages of static libraries (eg bloat); compared to the 
 betterC alternative, you'd have access to all of 
 druntime/phobos and not have to reinvent the wheel
Because I do not have control on phobos, i can not distribute my copy of phobos with required patches like normal cache size estimation (core.cpuid is broken). Why not to fix core.cpuid? Its source code is not manageable comparing with mir-cpuid: it is faster to write a module from scratch for this case. DUB package appears when I want it, LDC release may appear after 6-10 months (depends on distribution).
Dec 20 2016
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 20/12/2016 10:32 PM, Ilya Yaroshenko wrote:
 On Tuesday, 20 December 2016 at 09:12:55 UTC, Timothee Cour wrote:
  ilya: regarding -betterC 's stated goal to bypass druntime:

 besides static linking (which can be inefficient), why not use runtime
 shared libraries to distribute mir (eg GLAS) to C clients ?

 IIRC, from a single C++ program, I am able to dlopen & dlsym & run D
 functions defined in shared libraries built with different compilers
 (eg dmd and ldc) even if they use GC (and therefore dmd's and ldc's
 druntime) ; pseudocode:

 // main.cpp
 ```
 void main(){
   dlopen, dlsym, and run "void fun1()" from libfun_dmd.so
   dlopen, dlsym, and run "void fun2()" from libfun_ldc.so
 }
 ```

 That would provide the needed isolation without suffering from
 disadvantages of static libraries (eg bloat); compared to the betterC
 alternative, you'd have access to all of druntime/phobos and not have
 to reinvent the wheel
Because I do not have control on phobos, i can not distribute my copy of phobos with required patches like normal cache size estimation (core.cpuid is broken). Why not to fix core.cpuid? Its source code is not manageable comparing with mir-cpuid: it is faster to write a module from scratch for this case. DUB package appears when I want it, LDC release may appear after 6-10 months (depends on distribution).
Of course you can bundle your own Phobos. If you put a std.* module in your path your build will be working against that instead. If you don't mess around with sc.ini it will override the default for that compilation.
Dec 20 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 20 December 2016 at 09:38:58 UTC, rikki cattermole 
wrote:
 On 20/12/2016 10:32 PM, Ilya Yaroshenko wrote:
 On Tuesday, 20 December 2016 at 09:12:55 UTC, Timothee Cour 
 wrote:
  ilya: regarding -betterC 's stated goal to bypass druntime:

 besides static linking (which can be inefficient), why not 
 use runtime
 shared libraries to distribute mir (eg GLAS) to C clients ?

 IIRC, from a single C++ program, I am able to dlopen & dlsym 
 & run D
 functions defined in shared libraries built with different 
 compilers
 (eg dmd and ldc) even if they use GC (and therefore dmd's and 
 ldc's
 druntime) ; pseudocode:

 // main.cpp
 ```
 void main(){
   dlopen, dlsym, and run "void fun1()" from libfun_dmd.so
   dlopen, dlsym, and run "void fun2()" from libfun_ldc.so
 }
 ```

 That would provide the needed isolation without suffering from
 disadvantages of static libraries (eg bloat); compared to the 
 betterC
 alternative, you'd have access to all of druntime/phobos and 
 not have
 to reinvent the wheel
Because I do not have control on phobos, i can not distribute my copy of phobos with required patches like normal cache size estimation (core.cpuid is broken). Why not to fix core.cpuid? Its source code is not manageable comparing with mir-cpuid: it is faster to write a module from scratch for this case. DUB package appears when I want it, LDC release may appear after 6-10 months (depends on distribution).
Of course you can bundle your own Phobos. If you put a std.* module in your path your build will be working against that instead. If you don't mess around with sc.ini it will override the default for that compilation.
CPUID should be precompiled. So this does not work. If GLAS do not need anything but CPUID, what the reason to depend on DRuntime? Please stop to force me to use DRuntime because you think it is better for Mir projects. This looks like DRuntime religion. A betterC library can be used with DRuntime. So I do not force anyone to drop DRuntime. Please do not force me to use it. GLAS already works without DRuntime and I am happy about that.
Dec 20 2016
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 20 December 2016 at 09:56:34 UTC, Ilya Yaroshenko 
wrote:
 Of course you can bundle your own Phobos.
 If you put a std.* module in your path your build will be 
 working against that instead. If you don't mess around with 
 sc.ini it will override the default for that compilation.
CPUID should be precompiled. So this does not work. If GLAS do not need anything but CPUID, what the reason to depend on DRuntime? Please stop to force me to use DRuntime because you think it is better for Mir projects. This looks like DRuntime religion. A betterC library can be used with DRuntime. So I do not force anyone to drop DRuntime. Please do not force me to use it. GLAS already works without DRuntime and I am happy about that.
What does "you can bundle your own phobos" have to do with that? No one's forcing you to do anything. You're suggesting something radical*, other people are suggesting that maybe there are good compromises where everybody wins, avoiding fragmentation. Then you say that "looks like DRuntime religion" and claim you're being forced to modify your code to add extra dependencies. It doesn't make sense. Your technical arguments have good content, in my opinion everyone would benefit from you writing them up with sufficient context for people who don't know what you know and without hyperbole (none of that "D will fail if we don't do X" or "phobos is bloated and useless" stuff, it doesn't help communicate your points). Developing the case study / thought experiment of getting two libraries (blas and fft) in to a traditional linux distro would be a great central point. See https://github.com/dlang/DIPs/pull/51 where Andrei describes not only how he thinks things should be, but also explains in detail why other approaches won't work. Now consider that what you're talking about is a lot more important and different/disruptive than a new import syntax, so it deserves at least comparably good description. * D used to have 2 standard libraries. That was not a happy time for the community. People are wary partly because of that. P.S. a good example of the sort of statement that needs more explanation, at least for me:
 The betterC
 std.range and std.algorithm analogs would be released with new 
 ndslice
 implementation. Mir's algorithm would be faster then Phobos and 
 will
 generate less template bloat."
How? Why? Is ndslice actually capable of the same level of flexibility as std.algorithm and std.range? You're effectively saying "all that stuff that Andrei (and others) designed and wrote, I can do better" without really showing anyone why they should believe that.
Dec 20 2016
next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Tuesday, 20 December 2016 at 12:33:27 UTC, John Colvin wrote:
 * D used to have 2 standard libraries. That was not a happy 
 time for the community. People are wary partly because of that.
To bring some cool to the discussion, let's remember that they also weren't compatible. Of course it's a concern but this case is different and deserve full consideration. let's say the thread title was "A runtime-less modular library?"
Dec 20 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 20 December 2016 at 12:45:13 UTC, Guillaume Piolat 
wrote:
 On Tuesday, 20 December 2016 at 12:33:27 UTC, John Colvin wrote:
 * D used to have 2 standard libraries. That was not a happy 
 time for the community. People are wary partly because of that.
To bring some cool to the discussion, let's remember that they also weren't compatible. Of course it's a concern but this case is different and deserve full consideration. let's say the thread title was "A runtime-less modular library?"
Thanks! Good point
Dec 20 2016
prev sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 20 December 2016 at 12:33:27 UTC, John Colvin wrote:

Mir uses deprecated native complex numbers because we can not 
spend days explaining why they are better then std.complex (in 
short: std.complex breaks compiler optimization logic, function 
inlining works, but it breaks optimization logic). I have already 
spend 4 hours for forum discussions and PRs about this issue 
without any result.

We are doing the best things we can do with current D. If 
something will be improved in future we will upgrade codebase.

 P.S. a good example of the sort of statement that needs more 
 explanation, at least for me:
 The betterC
 std.range and std.algorithm analogs would be released with new 
 ndslice
 implementation. Mir's algorithm would be faster then Phobos 
 and will
 generate less template bloat."
How? Why? Is ndslice actually capable of the same level of flexibility as std.algorithm and std.range? You're effectively saying "all that stuff that Andrei (and others) designed and wrote, I can do better" without really showing anyone why they should believe that.
The explanation of new idioms will appear after its code. I did not say "better". I said "betterC" plus "faster". Yes, i can say that for DCV library new idioms are much better then Phobos. But new idioms have another issues, for example D language is not capable to infer their safety for many of them. Other issues is that there a lot of fast non CTFE-able types and their CTFE-able analogs. One good things for safety and CTFE is allow multiple return value. In combination with `auto ref` it is _very_ powerful: ---- auto ref front() { // Returns 2 values, each value is returned by reference if possible return (a.front, b.front); } ---- This would be interesting to discuss and to have it in D in CTFE and safe mode. But I am not good in DIPs. This sorts of features is very desirable.
Dec 20 2016