www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [GSoC 2019] Interested in Baremetal D Runtime and project ideas

reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
[GSoC 2019] Interested in Baremetal D Runtime and project ideas

Hello,

I'm Stefanos Baziotis and I'm both new to D and GSoC.
I'm an undergraduate computer science student in Greece, and
I would like to contribute to the D programming language.

Unfortunately, probably I came late on the GSoC as I learned 
about it just
2 days before, so any info about it and/or D's selection process 
would be greatly appreciated.

I heard about D probably a month ago from a friend, and I thought 
I would
give it a try as I am not 100% satisfied with C, C++. I have to 
say, that was
a really fun month. :)
After about 3000 lines (of mostly writing a not-yet-finished 
compiler front-end), it was very exciting!

I'm mainly interested on the Baremetal D runtime project, as I am 
mostly comfortable
with low-level C programming. I have developed some allocators [1]
and have some SIMD experience as I have written a small image 
convolution MPI
program that uses AVX extensions [2].

I'm interested in other projects as well, specifically 
compiler-related and the persistent data structures.

I'm very enthusiastic to learn a lot about any of these projects, 
but I would like to
ask: What is the expected / required experience / knowledge / 
skills?
Also, is this the best place to discuss about this subject?

Best regards,
Stefanos Baziotis


[1] https://github.com/baziotis/Allocators
[2] https://github.com/baziotis/2D-Image-Convolution-MPI-SIMD
Apr 04
next sibling parent reply Seb <seb wilzba.ch> writes:
On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project ideas

 Hello,

 [...]
Hi Stefanos, Thanks a lot for your interest in D! You can find answers to some of your questions here: https://forum.dlang.org/post/aoxlhtmvnzcgkkssayti forum.dlang.org Probably the best place to ask further questions is here.
Apr 04
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Thursday, 4 April 2019 at 14:06:43 UTC, Seb wrote:
 Probably the best place to ask further questions is here.
Thanks! I just read it. Just for clarification, by "here" you mean this post or the FAQ post? It's just that the questions that are left are specific to the project, so I try to find the best place to "discuss your ideas publicly", as the FAQ states.
Apr 04
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2019-04-04 15:56, Stefanos Baziotis wrote:

 Unfortunately, probably I came late on the GSoC as I learned about it just
 2 days before, so any info about it and/or D's selection process would 
 be greatly appreciated.
 
 I'm mainly interested on the Baremetal D runtime project, as I am mostly 
 comfortable
 with low-level C programming. I have developed some allocators [1]
 and have some SIMD experience as I have written a small image 
 convolution MPI
 program that uses AVX extensions [2].
There's another potential GSoC student that has started a thread related to baremetal [1].
 I'm interested in other projects as well, specifically compiler-related 
 and the persistent data structures.
Same for persistent data structures [2][3]. Here are some compiler and GSoC related threads [4][5].
 I'm very enthusiastic to learn a lot about any of these projects, but I 
 would like to
 ask: What is the expected / required experience / knowledge / skills?
 Also, is this the best place to discuss about this subject?
Make sure you check the wiki page as well [6]. [1] https://forum.dlang.org/thread/cxntuuqepymkugvvrxhz forum.dlang.org [2] https://forum.dlang.org/thread/wvhuvdydvwjkwukawtmk forum.dlang.org [3] https://forum.dlang.org/thread/ygdbtgfablwkdxhhvbsy forum.dlang.org [4] https://forum.dlang.org/thread/ersuicbdvzxsliafvpjv forum.dlang.org [5] https://forum.dlang.org/thread/iwvtxsecerbshhagaunj forum.dlang.org [6] https://wiki.dlang.org/GSOC_2019_Ideas -- /Jacob Carlborg
Apr 04
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Thursday, 4 April 2019 at 18:07:03 UTC, Jacob Carlborg wrote:
 [1] 
 https://forum.dlang.org/thread/cxntuuqepymkugvvrxhz forum.dlang.org
 [2] 
 https://forum.dlang.org/thread/wvhuvdydvwjkwukawtmk forum.dlang.org
 [3] 
 https://forum.dlang.org/thread/ygdbtgfablwkdxhhvbsy forum.dlang.org
 [4] 
 https://forum.dlang.org/thread/ersuicbdvzxsliafvpjv forum.dlang.org
 [5] 
 https://forum.dlang.org/thread/iwvtxsecerbshhagaunj forum.dlang.org
 [6] https://wiki.dlang.org/GSOC_2019_Ideas
On Thursday, 4 April 2019 at 18:07:03 UTC, Jacob Carlborg wrote: Thank you! I have seen post [1]. It seems that this guy chose a different project idea in the end. Also, I came across [2] and [3], from which I infer that I should contact Andrei. [4] is very interesting, but I think I'm not so competent to propose for this but [5] actually seems to have some good ideas. :) About [6], yeah I have checked it multiple times to be sure I'm not missing anything, thanks. - Stefanos
Apr 04
prev sibling next sibling parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project ideas
I have submitted a proposal (before 2 days) for D Baremetal Runtime. Any feedback is very welcome. :)
Apr 06
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 6 April 2019 at 16:23:28 UTC, Stefanos Baziotis 
wrote:
 On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
 wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project ideas
I have submitted a proposal (before 2 days) for D Baremetal Runtime. Any feedback is very welcome. :)
Where can one find this proposal? Mike
Apr 06
next sibling parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Saturday, 6 April 2019 at 17:51:31 UTC, Mike Franklin wrote:
 On Saturday, 6 April 2019 at 16:23:28 UTC, Stefanos Baziotis 
 wrote:
 On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
 wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project 
 ideas
I have submitted a proposal (before 2 days) for D Baremetal Runtime. Any feedback is very welcome. :)
Where can one find this proposal? Mike
Unfortunately, I know too little about how the GSoC works from the organization-side, but from what I know, it should be available for potential mentors through the Summer of Code platform (if it's not, it is very unfortunate as I'm waiting 2 days :P ). I suppose I can send it here too, as I've seen another guy do it. In any case, this is the proposal: https://docs.google.com/document/d/1B3YbjY70iVgGT7hq_16tBriLuCAUtjFR0gf0xkxmQ5c/edit?usp=sharing Any comments are welcome!
Apr 06
next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 6 April 2019 at 18:56:42 UTC, Stefanos Baziotis 
wrote:

 Unfortunately, I know too little about how the GSoC works from 
 the organization-side, but
 from what I know, it should be available for potential mentors 
 through the Summer of Code platform (if it's not, it is very 
 unfortunate as I'm waiting 2 days :P ).
 I suppose I can send it here too, as I've seen another guy do 
 it.
 In any case, this is the proposal: 
 https://docs.google.com/document/d/1B3YbjY70iVgGT7hq_16tBriLuCAUtjFR0gf0xkxmQ5c/edit?usp=sharing

 Any comments are welcome!
I think the Bare Metal Runtime suggestion on the GSoC ideas page conflated a few separate projects into one. My other post in this thread elaborates on that. I don't have much, if anything, to offer, as I've never participated in GSoC or written a similar proposal. It looks like an awful lot of work, but I hope the project gets selected and is a success. I think it would be great for D in the long run. I'm JinShil, the author of memcpyD and some of the other projects linked on the GSoC ideas page. My interest is mostly in microcontrollers and embedded systems, and I've only dabbled in memcpy and malloc/free implementations, but you're welcome to contact me if you have anything you'd like to discuss. Good Luck! Mike
Apr 06
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Saturday, 6 April 2019 at 20:07:34 UTC, Mike Franklin wrote:
 I think the Bare Metal Runtime suggestion on the GSoC ideas 
 page conflated a few separate projects into one.  My other post 
 in this thread elaborates on that.

 I don't have much, if anything, to offer, as I've never 
 participated in GSoC or written a similar proposal.  It looks 
 like an awful lot of work, but I hope the project gets selected 
 and is a success.  I think it would be great for D in the long 
 run.
I think too and I would like to work on that. Sadly, there's still no proposed mentor.
 I'm JinShil, the author of memcpyD
It provided a lot of inspiration mate!
 you're welcome to contact me if you have anything you'd like to 
 discuss.

 Good Luck!

 Mike
I hope we talk soon. Thanks for the help.
Apr 06
prev sibling parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Saturday, 6 April 2019 at 18:56:42 UTC, Stefanos Baziotis 
wrote:
 In any case, this is the proposal: 
 https://docs.google.com/document/d/1B3YbjY70iVgGT7hq_16tBriLuCAUtjFR0gf0xkxmQ5c/edit?usp=sharing

 Any comments are welcome!
Here's a new proposal I made. I think this clarifies what the project will try to solve better. https://docs.google.com/document/d/1kLV9PA1QN3KTqGQDrEvnSELbp2eHtL7GImCQS9rqNLU/edit?usp=sharing And of course, any comments are welcome!
Apr 08
prev sibling parent reply Seb <seb wilzba.ch> writes:
On Saturday, 6 April 2019 at 17:51:31 UTC, Mike Franklin wrote:
 On Saturday, 6 April 2019 at 16:23:28 UTC, Stefanos Baziotis 
 wrote:
 On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
 wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project 
 ideas
I have submitted a proposal (before 2 days) for D Baremetal Runtime. Any feedback is very welcome. :)
Where can one find this proposal? Mike
You need to be a GSoC "Dementor" to see the proposal. I just sent you an invitation ;-)
Apr 06
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Saturday, 6 April 2019 at 22:11:51 UTC, Seb wrote:
 You need to be a GSoC "Dementor" to see the proposal. I just 
 sent you an invitation ;-)
Seb, do you know if anyone will mentor this?
Apr 06
prev sibling next sibling parent reply IGotD- <nise nise.com> writes:
On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project ideas

 Hello,

 [...]
What is in the runtime we need for baremetal projects that betterC doesn't provide?
Apr 06
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Saturday, 6 April 2019 at 17:50:39 UTC, IGotD- wrote:
 On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
 wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project ideas

 Hello,

 [...]
What is in the runtime we need for baremetal projects that betterC doesn't provide?
There are a number of routines for which D currently calls the C standard library equivalent. For example, memcpy, memmove etc. and also malloc(), free() etc. The idea is that these should be implemented in D to provide a more baremetal environment for D, for limited resources (aka embedded systems). Note that the C standard library provides a lot of benefits for the general case, mainly that it has been used and tested for years. https://wiki.dlang.org/GSOC_2019_Ideas#Baremetal_D_runtime
Apr 06
parent reply IGotD- <nise nise.com> writes:
On Saturday, 6 April 2019 at 18:52:04 UTC, Stefanos Baziotis 
wrote:
 On Saturday, 6 April 2019 at 17:50:39 UTC, IGotD- wrote:
 On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
 wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project 
 ideas

 Hello,

 [...]
What is in the runtime we need for baremetal projects that betterC doesn't provide?
There are a number of routines for which D currently calls the C standard library equivalent. For example, memcpy, memmove etc. and also malloc(), free() etc. The idea is that these should be implemented in D to provide a more baremetal environment for D, for limited resources (aka embedded systems). Note that the C standard library provides a lot of benefits for the general case, mainly that it has been used and tested for years. https://wiki.dlang.org/GSOC_2019_Ideas#Baremetal_D_runtime
memcpy, memmove and such are heavily architecture dependent and needs an implementation for each architecture (even on sub architecture level) in assembler in order to be optimized. At least this what I've seen for the C/C++ compilers. We can have a fallback in D though. Correct me if I'm wrong, doesn't LLVM provide with such primitives, at least partly? When it comes to malloc and free, I would suggest we have just hooks for these. Baremetal people have very different opinions what allocator suits their needs. Example allocators are of course welcome but being able to change to whatever allocator you want is essential for baremetal.
Apr 06
next sibling parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Saturday, 6 April 2019 at 21:31:19 UTC, IGotD- wrote:
 memcpy, memmove and such are heavily architecture dependent and 
 needs an implementation for each architecture (even on sub 
 architecture level) in assembler in order to be optimized. At 
 least this what I've seen for the C/C++ compilers. We can have 
 a fallback in D though. Correct me if I'm wrong, doesn't LLVM 
 provide with such primitives, at least partly?
I believe that you are correct about LLVM [1]. It seems though that it calls the C standard library. I would be glad to listen to the creators of the project idea about the clear goals of this project, but this is my take: 1) Independency in the C standard library. Being dependent on the C standard library means that users have to have a C toolchain. 2) It tries to target limited resources systems and as such the focus will be in those archictures. 3) C versions are nowadays quite complex, so they might want simpler versions. 4) As Mike states in github, take advantage of the templates and compile-time execution for potentialy more efficient code and also the strong type system to leverage features such as safe.
 When it comes to malloc and free, I would suggest we have just 
 hooks for these. Baremetal people have very different opinions 
 what allocator suits their needs. Example allocators are of 
 course welcome but being able to change to whatever allocator 
 you want is essential for baremetal.
I think that the reasons to implement malloc() and free() are the same with those of memcpy etc. Any enhancement on the reasons is welcome. [1] https://llvm.org/docs/LangRef.html#standard-c-library-intrinsics - Stefanos
Apr 06
prev sibling parent Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 6 April 2019 at 21:31:19 UTC, IGotD- wrote:

 memcpy, memmove and such are heavily architecture dependent and 
 needs an implementation for each architecture (even on sub 
 architecture level) in assembler in order to be optimized. At 
 least this what I've seen for the C/C++ compilers.
This has been my observation as well. However, DMD only supports Intel architectures. There's still a wide range of feature differences in the Intel family, but those can all be managed with information like https://dlang.org/phobos/core_cpuid.html I assume that this GSoC project would focus only on Intel architectures that DMD supports, and even then it could always fall back to C's `memcpy`, `memcmp`, etc. for those architectures that the implementation has not been optimized for. For LLVM and GDC, they'll probably have to do their own thing for all the architectures they support, but I believe DMD could set a precedent. I encourage this project to focus on the fact that this D, not C, and there's no reason to follow the C way. For example, `memcpy` in D would probably manifest itself as a template like `memcpy(T)(T* dest, const T* src)` or something like that. `void*` should go the way of the dodo if at all possible. Recognize also that users will likely never need to call `memcpy` directly in D. They would use the wealth of features already in the D language, and the compiler should lower those features to the `memcpy` implementation on behalf of the user.
 When it comes to malloc and free, I would suggest we have just 
 hooks for these. Baremetal people have very different opinions 
 what allocator suits their needs. Example allocators are of 
 course welcome but being able to change to whatever allocator 
 you want is essential for baremetal.
The implementations in druntime need to be ported to bare-metal with facilities like `version(X86_64)` and `version(FreeStanding)`, both of which already exist at https://dlang.org/spec/version.html This could be the default implementation, and it should be possible to build something in that would allow the user to override the default. Mike
Apr 06
prev sibling next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
wrote:

 I'm mainly interested on the Baremetal D runtime project, as I 
 am mostly comfortable
 with low-level C programming. I have developed some allocators 
 [1]
 and have some SIMD experience as I have written a small image 
 convolution MPI
 program that uses AVX extensions [2].
I'm sorry for responding late; I didn't see this thread until just a short while ago. I think there are probably a few different ways to define a "bare metal runtime in D". The way I define it is "port the existing D runtime to a processor that is not running an operating system". The problem with that is D's runtime expects an underlying operating system. So, one is left with a bit of a paradox: which comes first the operating system or the runtime. I would argue that if one is porting the existing D runtime to a processor without an operating system, the runtime becomes the operating system. Therefore, one needs to implement dynamic memory allocation, threads, exceptions, thread-local storage, the garbage collector, etc. all in the runtime, and all unique to a specific processor or processor family. Basically, one needs to build an operating system into the D runtime. In a way, that's pretty cool, as D itself would then come with an operating system included, but is that really practical? Perhaps what users are actually looking for is a minimal subset of D that doesn't require a runtime. That would likely mean no dynamic memory allocation, no threads, no thread-local storage, no GC, no classes, no exceptions, or some variant of that. That's more or less what -betterC provides. Perhaps one could use -betterC to build the OS on which the full D runtime would be ported to. What I would like to see is neither of those, but what Andrei recently proposed with this comment:
 Oh, and druntime must go.

 The whole distinction between the runtime library and the 
 standard library is clowny and has more to do with poorly 
 ported tradition from other languages, than with anything else.

 We need one standard library that is entirely pay-as-you-go 
 (e.g. empty main() means no library file is ever needed for 
 linking) and that offers an opt-in continuum.
In my opinion, that is smack dab right on and said better and more succinctly than I ever could. But, how do we get there? As I see it, we actually have the vast majority of it already implemented, but with all the dependencies on C, C++, operating system APIs, and a design philosophy the operating system and C/C++ toolchain would always be there from the beginning, we have a tangled mess. So, we need to untangle everything and reduce the implementations down to individual, modular components with minimal dependencies that can be imported without bringing in everything else. Here's Andrei delegating a few tasks to me: https://www.youtube.com/watch?v=uUQeQ7E8hOU A great help will be if Dan Printzell succeeds in eliminating the dependency of many runtime hooks on TypeInfo. Since TypeInfo is a class, his project will help eliminate the dependency on classes, and since classes more-or-less require the GC, it removes the dependency of those runtime hooks on the GC as well. It will also decouple the compiler from the runtime allowing it to generate code with fewer dependencies. However, those runtime hooks still require implementations in terms of memcpy, memcmp, malloc, free, etc. Having those written in D as strongly-typed templates would further reduce dependencies on a C/C++ toolchain and standard library. With Dan's work completed, the benefits of having memcpy, malloc and friends implemented in D would become much more apparent, as the new runtime template lowerings could then be implemented in D all the way down the stack utilizing CTFE, design by introspection, and all of D's compiler guarantees such a safety and purity. Finally, I think it would help greatly if we could detangle and pull out those features of Phobos that don't require any runtime support (e.g. std.traits, std.meta) that would provide a base, lightweight, no-dependency utility library for all D platforms including bare metal. Mike
Apr 06
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Saturday, 6 April 2019 at 19:48:42 UTC, Mike Franklin wrote:
 I'm sorry for responding late; I didn't see this thread until 
 just a short while ago.
No problem at all, I responded late as well.
 Basically, one needs to build an operating system into the D 
 runtime.
 In a way, that's pretty cool, as D itself would then come with 
 an operating system included, but is that really practical?
While that would be a very interesting hobby project, it is as you say not very practical and most of all, it would probably not help the community that much.
 As I see it, we actually have the vast majority of it already 
 implemented, but with all the dependencies on C, C++, operating 
 system APIs, and a design philosophy the operating system and 
 C/C++ toolchain would always be there from the beginning, we 
 have a tangled mess.
I totally agree with all of these. What I roughly got from the project description was to achieve what Andrei said but without the dependencies to so much existing C/C++ code.
 A great help will be if Dan Printzell succeeds in eliminating 
 the dependency of many runtime hooks on TypeInfo.  Since 
 TypeInfo is a class, his project will help eliminate the 
 dependency on classes, and since classes more-or-less require 
 the GC, it removes the dependency of those runtime hooks on the 
 GC as well. It will also decouple the compiler from the runtime 
 allowing it to generate code with fewer dependencies.
Actually Dan started on the Baremetal project at first (you probably know that) and then moved to that project after discussion in the forum. The fact that Dan works on that makes me doubly excited to work on this. I think the result would much better than simply -betterC with a couple more utilities.
 However, those runtime hooks still require implementations in 
 terms of memcpy, memcmp, malloc, free, etc.  Having those 
 written in D as strongly-typed templates would further reduce 
 dependencies on a C/C++ toolchain and standard library.  With 
 Dan's work completed, the benefits of having memcpy, malloc and 
 friends implemented in D would become much more apparent, as 
 the new runtime template lowerings could then be implemented in 
 D all the way down the stack utilizing CTFE, design by 
 introspection, and all of D's compiler guarantees such a safety 
 and purity.

 Finally, I think it would help greatly if we could detangle and 
 pull out those features of Phobos that don't require any 
 runtime support (e.g. std.traits, std.meta) that would provide 
 a base, lightweight, no-dependency utility library for all D 
 platforms including bare metal.

 Mike
Great proposals Mike. My answers might seem short but really you solved most of my quetsions. So what do you believe is the best way I can help? - Stefanos
Apr 06
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Saturday, 6 April 2019 at 20:22:19 UTC, Stefanos Baziotis 
wrote:

 I totally agree with all of these. What I roughly got from the 
 project
 description was to achieve what Andrei said but without the 
 dependencies to
 so much existing C/C++ code.
I don't see that Andrei objects much to the C/C++ dependency. Afterall, he recently approved adding the C++ standard library to the druntime: https://github.com/dlang/druntime/pull/2286 In fact, I find I'm in the minority when it comes to wanting to replace `memcpy` and friends with templates. It's hard to advocate for it right now because there isn't actually a way to utilize it. That's why Dan's project is so important. Once Dan is done and the other runtime hooks are converted to templates the benefit of replacing `memcpy` and friend will become more apparent. Until that work happens, we can't really utilize the templatized versions of `memcpy` and friends and so it will be difficult to measure real-world benefits. Once done however, could replace `memcpy` and friends in the runtime and run some real-world benchmarks on the DMD compiler, Phobos, and other D projects to measure real benefit. And that's not to mention turtles-all-the-way-down safety and purity guarantees provided by the compiler. I'm not sure who posted the original GSoC idea, so they'd have to speak for themselves, but it appears to be a synthesis of things I have been advocating for for a long time. It's not really about creating a bare metal runtime, it's mostly just about being able to use D for bare-metal programming in a pay-as-you-go fashion.
 So what do you believe is the best way I can help?
Where to begin? There's so much work that needs to be done, I'm not even sure where to start. At the moment, I'm just trying to clean up some unfinished business in the compiler like converting the last remaining C files in DMD to D, following through on deprecations so we can be liberated from some technical debt, and hopefully getting around to finishing https://github.com/dlang/dmd/pull/9068 which is a prerequisite to refactoring druntime enabling much of what I've been talking about. Probably the best thing you can do is focus on your proposal and make it great. I hope those reviewing the proposals will be able to see the relationship and natural progression with Dan's proposal and yours and both will be selected. For the past 5 years this has been my experience working on D: https://www.youtube.com/watch?v=AbSehcT19u0 If you're anxious to get your hands dirty, you can start by choosing a not-yet-converted runtime hook from https://wiki.dlang.org/Runtime_Hooks and have at it. See https://github.com/dlang/druntime/pull/2508 for another recent example of another contributor getting busy. https://github.com/dlang/dmd/pull/9068 is a low barrier to entry task that would help us out a lot. You can also start stripping out utilities from Phobos (e.g. std.traits, std.meta, std.conv, etc.) to create a no-dependency utility library. Basically my "Suggestion 2" here: https://forum.dlang.org/post/qooasdrdlrqdhghjwmus forum.dlang.org It could potentially serve as a starting point for std.v2 and the opt-in continuum. See https://forum.dlang.org/post/mailman.7799.1553797857.29801.digital ars-d puremagic.com for the thread where that was first proposed. Mike
Apr 06
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Sunday, 7 April 2019 at 04:07:43 UTC, Mike Franklin wrote:
 I don't see that Andrei objects much to the C/C++ dependency.  
 Afterall, he recently approved adding the C++ standard library 
 to the druntime: https://github.com/dlang/druntime/pull/2286
Yes, I don't mean Andrei specifically, I mean the ones who proposed the idea. And even them, not for generally but for embedded systems.
 And that's not to mention turtles-all-the-way-down safety and 
 purity guarantees provided by the compiler.

 I'm not sure who posted the original GSoC idea, so they'd have 
 to speak for themselves, but it appears to be a synthesis of 
 things I have been advocating for for a long time.  It's not 
 really about creating a bare metal runtime, it's mostly just 
 about being able to use D for bare-metal programming in a 
 pay-as-you-go fashion.
Mike, you give informative answers. To be honest, the more I discuss this project idea, the more unclear what the goals are becomes.
 Probably the best thing you can do is focus on your proposal 
 and make it great.  I hope those reviewing the proposals will 
 be able to see the relationship and natural progression with 
 Dan's proposal and yours and both will be selected.
Thanks, but frankly I don't see any interest from the community. That's ok as I (and all of us I guess) will be better off if I work on a project that the community is enthusiastic about.
 For the past 5 years this has been my experience working on D:  
 https://www.youtube.com/watch?v=AbSehcT19u0
Hahahahahahahahaha.. Honestly, that resonated and made me laugh more than it should. If I had a penny for every time I got into a project to do something other than memory management but the memory management was so bad, that now instead of doing the thing I was supposed to do, I had to redo most of the mem management -.-
 If you're anxious to get your hands dirty, you can start by 
 choosing a not-yet-converted runtime hook from 
 https://wiki.dlang.org/Runtime_Hooks and have at it.  See 
 https://github.com/dlang/druntime/pull/2508 for another recent 
 example of another contributor getting busy.
That will probably be my first start.
 https://github.com/dlang/dmd/pull/9068 is a low barrier to 
 entry task that would help us out a lot.

 You can also start stripping out utilities from Phobos (e.g. 
 std.traits, std.meta, std.conv, etc.) to create a no-dependency 
 utility library.  Basically my "Suggestion 2" here: 
 https://forum.dlang.org/post/qooasdrdlrqdhghjwmus forum.dlang.org
  It could potentially serve as a starting point for std.v2 and 
 the opt-in continuum.  See 
 https://forum.dlang.org/post/mailman.7799.1553797857.29801.digital
ars-d puremagic.com for the thread where that was first proposed.
I will have to ask again about this library, but surely, after I finish with the GSoC proposals, I will give it a try. - Stefanos
Apr 07
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Sunday, 7 April 2019 at 10:56:51 UTC, Stefanos Baziotis wrote:

 Mike, you give informative answers. To be honest, the more
 I discuss this project idea, the more unclear what the goals
 are becomes.
These are the goals as I see them: 1. Allow for the opt-in continuum described here: https://forum.dlang.org/post/q7j4sl$17pe$1 digitalmars.com 2. Remove artificial dependencies. IMO the current dependency D has on C is completely artificial. D does not need C (https://forum.dlang.org/post/kemnbymcuwycglevtbox forum.dlang.org). It's simply utilizing C as a matter of convenience and expediency. Many of C's implementations are not type and memory safe, and that is a disadvantage for D. C's implementations can't be evaluated at compile-time, that is a disadvantage for D. 3. Detangle dependencies. e.g. many of the runtime hooks don't need runtime TypeInfo; that information is available at compile-time. This also applies to modules in Phobos. e.g. std.traits, std.meta, std.conv, shouldn't require runtime support. 4. Embrace design by introspection (https://www.youtube.com/watch?v=tcyb1lpEHm0) all the way down the call stack. 5. Embrace D's compiler guarantees (type safety, memory safety, purity, etc.) all the way down the call stack It is the opt-in continuum that enables bare-metal programming in D. (2)~(5) are just things that need to be done to get there. We don't need -betterC; users can simply opt-in to the features they want by importing the modules they need, and no more. Currently, the compiler assumes an OS is present. It assumes the *entire* druntime library is complete and available at both compile-time and link time. Well, at least it used to; v2.079 introduced some improvements (https://dlang.org/changelog/2.079.0.html#minimal_runtime). We need to continue that work until (1) becomes a reality. Converting C's software building-blocks to D is not absolutely necessary for bare-metal programming, but it will make for a more polished experience and will truly embrace the zen of D if the conversion were done. I'm under the impression, also, that once converted to D, some D programs will out-perform their C counterparts. I think the GSoC idea was ported from a number of different ideas proposed at the Symmetry Autumn of Code here: https://wiki.dlang.org/SAOC_2018_ideas#Re-implement_Software_Building_Blocks_.28e.g._memcpy.2C_memset.2C_memcmp.2C_malloc.2C_free.2C_etc..29_in_D Don't worry about what's posted at GSoC ideas page. Understand why converting these software building blocks to D would be advantageous and make the case for it in your proposal, and why you should be the one to do it. If it's difficult to see why converting them to D would be advantageous, well, that's probably my fault. I don't know how else to say it. As I mentioned previously, once the runtime hooks are converted to templates, it should become much more apparent to all without me having to advocate for it. Mike
Apr 07
next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Monday, 8 April 2019 at 02:13:30 UTC, Mike Franklin wrote:

 If it's difficult to see why converting them to D would be 
 advantageous, well, that's probably my fault.  I don't know how 
 else to say it.  As I mentioned previously, once the runtime 
 hooks are converted to templates, it should become much more 
 apparent to all without me having to advocate for it.
Here's a little data I gathered very quickly to illustrate why there might be performance benefits. Below is a table of all the calls to `memcpy` that DMD makes when compiling Phobos and druntime. What's disappointing to me is the extremely larger number of calls to `memcpy` to copy 8-bytes of data on a 64-bit machine (not to mention the few to for 0 bytes!). Now imagine if that were a template like this (https://github.com/JinShil/memcpyD/blob/56dfe65cbae4407fa584e53dac54c105d433e95 /memcpyd.d#L17-L22) that could be inlined. I think there would be some performance benefits. It's actually quite alarming how many calls there are to memcpy for such small amounts of data. Phobos: count fn size_in_bytes 48420 memcpy 8 26298 memcpy 2 26298 memcpy 1 6484 memcpy 33 4766 memcpy 136 4250 memcpy 25 2308 memcpy 32 1994 memcpy 35 1444 memcpy 31 1272 memcpy 34 1000 memcpy 37 928 memcpy 28 816 memcpy 41 614 memcpy 42 560 memcpy 44 458 memcpy 43 388 memcpy 40 388 memcpy 30 372 memcpy 36 324 memcpy 39 304 memcpy 45 272 memcpy 29 268 memcpy 38 254 memcpy 20 248 memcpy 46 192 memcpy 49 192 memcpy 47 176 memcpy 48 104 memcpy 27 84 memcpy 51 72 memcpy 50 72 memcpy 26 64 memcpy 63 64 memcpy 52 48 memcpy 0 28 memcpy 53 26 memcpy 16 22 memcpy 24 20 memcpy 58 20 memcpy 57 20 memcpy 54 16 memcpy 73 16 memcpy 55 12 memcpy 82 12 memcpy 70 12 memcpy 56 12 memcpy 22 8 memcpy 92 8 memcpy 83 8 memcpy 81 8 memcpy 76 8 memcpy 72 8 memcpy 68 8 memcpy 65 8 memcpy 64 8 memcpy 61 8 memcpy 60 8 memcpy 19 4 memcpy 98 4 memcpy 90 4 memcpy 89 4 memcpy 85 4 memcpy 79 4 memcpy 78 4 memcpy 74 4 memcpy 71 4 memcpy 69 4 memcpy 67 4 memcpy 66 4 memcpy 168 4 memcpy 104 2 memcpy 8192 2 memcpy 57344 2 memcpy 109 druntime: 3285 memcpy 8 447 memcpy 136 130 memcpy 16 10 memcpy 8192 10 memcpy 32 2 memcpy 24 2 memcpy 0
Apr 08
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Monday, 8 April 2019 at 10:47:17 UTC, Mike Franklin wrote:

 It's actually quite alarming how many calls there are to memcpy 
 for such small amounts of data.

 Phobos:
   count fn     size_in_bytes
   48420 memcpy 8
   26298 memcpy 2
   26298 memcpy 1
I actually neglected to consider something here. I think the compiler actually recognizes calls to memcpy and replaces them so it may not actually be making function calls for each of these. Nevertheless, if this were implemented as a template in the runtime, there would be no need any compiler magic, which would make the compiler just that much simpler to implement and maintain. Mike
Apr 08
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Tuesday, 9 April 2019 at 00:14:09 UTC, Mike Franklin wrote:
 I actually neglected to consider something here.  I think the 
 compiler actually recognizes calls to memcpy and replaces them 
 so it may not actually be making function calls for each of 
 these.  Nevertheless, if this were implemented as a template in 
 the runtime, there would be no need any compiler magic, which 
 would make the compiler just that much simpler to implement and 
 maintain.

 Mike
How do we know that? I guess yes the compiler could have done that but don't we need a look in asm? I get the general point though, good call. - Stefanos
Apr 09
parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Tuesday, 9 April 2019 at 14:52:52 UTC, Stefanos Baziotis wrote:

 How do we know that? I guess yes the compiler could have done 
 that but
 don't we need a look in asm?
Yes, looking at the asm would be a good way to confirm it. I haven't done that, and I'm mostly relying on rumor, but here's some evidence: https://github.com/dlang/dmd/blob/e3cb9363bff55170624667fb819289aceccdc90d/src/dmd/backend/cod2.d#L3395-L3399 Mike
Apr 09
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Wednesday, 10 April 2019 at 00:05:16 UTC, Mike Franklin wrote:
 Yes, looking at the asm would be a good way to confirm it.  I 
 haven't done that, and I'm mostly relying on rumor, but here's 
 some evidence:  
 https://github.com/dlang/dmd/blob/e3cb9363bff55170624667fb819289aceccdc90d/src/dmd/backend/cod2.d#L3395-L3399
Yes, it's interesting. Well, unfortunately it needs a lot of time to understand the backend enough to track whether it actually replaces it. However, I made a simple test case: import core.stdc.string; extern(C) void main() { int a = 5; int b = 5; memcmp(&a, &b, int.sizeof); } which is a test case that should definitely be inlined. Compiled with -betterC -g -O -release -inline, there is an actual call: lea rsi,[rbp-0x4] lea rdi,[rbp-0x8] call 0x400640 <memcmp plt> and the code inside the call is what I would expect for a full memcmp. I used -betterC because it makes smaller executables and makes it easy to inspect on GDB. Maybe without it, dmd inlines it in some way although I doubt it. Also, maybe LDC and GDC do something clever, but I don't have available right now either of them.
Apr 10
parent reply Radu <void null.pt> writes:
On Wednesday, 10 April 2019 at 12:25:55 UTC, Stefanos Baziotis 
wrote:
 On Wednesday, 10 April 2019 at 00:05:16 UTC, Mike Franklin 
 wrote:
 [...]
Yes, it's interesting. Well, unfortunately it needs a lot of time to understand the backend enough to track whether it actually replaces it. [...]
LDC does this https://godbolt.org/z/y0WiI9
Apr 10
parent reply Radu <void null.pt> writes:
On Wednesday, 10 April 2019 at 12:40:47 UTC, Radu wrote:
 On Wednesday, 10 April 2019 at 12:25:55 UTC, Stefanos Baziotis 
 wrote:
 On Wednesday, 10 April 2019 at 00:05:16 UTC, Mike Franklin 
 wrote:
 [...]
Yes, it's interesting. Well, unfortunately it needs a lot of time to understand the backend enough to track whether it actually replaces it. [...]
LDC does this https://godbolt.org/z/y0WiI9
Actually, this is what you might look for: https://godbolt.org/z/Zvyc3G Notice that at -O the memcmp is inlined.
Apr 10
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Wednesday, 10 April 2019 at 12:56:39 UTC, Radu wrote:
 Actually, this is what you might look for:

 https://godbolt.org/z/Zvyc3G

 Notice that at -O the memcmp is inlined.
Yes, thanks Radu, as I said in the previous post, I don't have LDC on my machine right now to test but I also didn't know that godbolt supports D. That's good. GDC interestingly does not inline it.
Apr 10
next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 10 Apr 2019 at 15:10, Stefanos Baziotis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Wednesday, 10 April 2019 at 12:56:39 UTC, Radu wrote:
 Actually, this is what you might look for:

 https://godbolt.org/z/Zvyc3G

 Notice that at -O the memcmp is inlined.
Yes, thanks Radu, as I said in the previous post, I don't have LDC on my machine right now to test but I also didn't know that godbolt supports D. That's good. GDC interestingly does not inline it.
You need to cast(bool), or memcmp() == 0. -- Iain
Apr 10
prev sibling next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Wed, 10 Apr 2019 at 16:07, Iain Buclaw <ibuclaw gdcproject.org> wrote:
 On Wed, 10 Apr 2019 at 15:10, Stefanos Baziotis via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 On Wednesday, 10 April 2019 at 12:56:39 UTC, Radu wrote:
 Actually, this is what you might look for:

 https://godbolt.org/z/Zvyc3G

 Notice that at -O the memcmp is inlined.
Yes, thanks Radu, as I said in the previous post, I don't have LDC on my machine right now to test but I also didn't know that godbolt supports D. That's good. GDC interestingly does not inline it.
You need to cast(bool), or memcmp() == 0.
If you are interested in the number returned from memcmp(), I guess the reason why it wouldn't inline might be to do with the endianess, and that the order memcmp compares is in the other direction - I think you'll need to bswap all values before you compare. -- Iain
Apr 10
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Wednesday, 10 April 2019 at 14:12:02 UTC, Iain Buclaw wrote:
 You need to cast(bool), or memcmp() == 0.
Yes, thanks, didn't catch it.
 I guess the reason why it wouldn't inline might be to do with 
 the endianess, and that the order memcmp compares is in the 
 other direction - I think you'll need to bswap all values 
 before you compare.
From what I know, endianess can cause differences in the result but how does it change the inlining of the compiler?
Apr 10
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 10 April 2019 at 13:09:19 UTC, Stefanos Baziotis 
wrote:
 [snip]

 Yes, thanks Radu, as I said in the previous post, I don't have 
 LDC on my machine
 right now to test but I also didn't know that godbolt supports 
 D. That's good.
 GDC interestingly does not inline it.
I'm a big fan of run.dlang.org
Apr 10
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Wednesday, 10 April 2019 at 14:38:34 UTC, jmh530 wrote:
 I'm a big fan of run.dlang.org
I have to use from the first time I learned D. I didn't know it supports LDC. It's very usefult that you can see the LLVM IR! I like Godbolt better for asm, because of the coloring for each line. :)
Apr 10
prev sibling parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Monday, 8 April 2019 at 02:13:30 UTC, Mike Franklin wrote:
 Don't worry about what's posted at GSoC ideas page.  Understand 
 why converting these software building blocks to D would be 
 advantageous and make the case for it in your proposal, and why 
 you should be the one to do it.

 If it's difficult to see why converting them to D would be 
 advantageous, well, that's probably my fault.  I don't know how 
 else to say it.  As I mentioned previously, once the runtime 
 hooks are converted to templates, it should become much more 
 apparent to all without me having to advocate for it.

 Mike
Thanks for the stats Mike. Actually the points were pretty self-explanatory. I had a rough idea about 2 and thanks for the rest. I was headed in the wrong direction because this project idea needs a proposal that answers "why" questions and not "how/when" questions. I think you know your insight was key in this discussion. It's an interesting topic and given that I will have time, I will tackle it GSoC or not so wait for further contact :). I think this thread is a good place to report progress.
Apr 08
prev sibling next sibling parent 9il <ilyayaroshenko gmail.com> writes:
On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project ideas

 Hello,

 I'm Stefanos Baziotis and I'm both new to D and GSoC.
 I'm an undergraduate computer science student in Greece, and
 I would like to contribute to the D programming language.

 Unfortunately, probably I came late on the GSoC as I learned 
 about it just
 2 days before, so any info about it and/or D's selection 
 process would be greatly appreciated.

 I heard about D probably a month ago from a friend, and I 
 thought I would
 give it a try as I am not 100% satisfied with C, C++. I have to 
 say, that was
 a really fun month. :)
 After about 3000 lines (of mostly writing a not-yet-finished 
 compiler front-end), it was very exciting!

 I'm mainly interested on the Baremetal D runtime project, as I 
 am mostly comfortable
 with low-level C programming. I have developed some allocators 
 [1]
 and have some SIMD experience as I have written a small image 
 convolution MPI
 program that uses AVX extensions [2].

 I'm interested in other projects as well, specifically 
 compiler-related and the persistent data structures.

 I'm very enthusiastic to learn a lot about any of these 
 projects, but I would like to
 ask: What is the expected / required experience / knowledge / 
 skills?
 Also, is this the best place to discuss about this subject?

 Best regards,
 Stefanos Baziotis


 [1] https://github.com/baziotis/Allocators
 [2] https://github.com/baziotis/2D-Image-Convolution-MPI-SIMD
Hi Stefanos, Very interesting, it would be nice to have SIMD convolution algorithms in D. SIMD extensions actively used in mir-glas (experimental D BLAS implementation), a D's FFT library should use SIMD as well. The same for low-level BetterC projects. Many Mir libraries were created to be used from C/C++/Python. We need a few D language extensions: 1. multiple auto ref values 2. more clever alias syntax for templates (it is required for data science libraries in D) 3. opApply and foreach API design rework (requires 1.) Each of them requires a DIP (D Improvement Proposal) and Pull Request to DMD with required changes. As a small bonus, I will pay an additional $400 for each D language extension merged to DMD. https://github.com/libmir Let me know if you would like to work during GSoC on one of the related directions and especially DIPs. ilyayaroshenko at google mai Best regards, Ilya
Apr 07
prev sibling parent reply 9il <ilyayaroshenko gmail.com> writes:
On Thursday, 4 April 2019 at 13:56:08 UTC, Stefanos Baziotis 
wrote:
 [GSoC 2019] Interested in Baremetal D Runtime and project ideas

 Hello,

 I'm Stefanos Baziotis and I'm both new to D and GSoC.
 I'm an undergraduate computer science student in Greece, and
 I would like to contribute to the D programming language.

 Unfortunately, probably I came late on the GSoC as I learned 
 about it just
 2 days before, so any info about it and/or D's selection 
 process would be greatly appreciated.

 I heard about D probably a month ago from a friend, and I 
 thought I would
 give it a try as I am not 100% satisfied with C, C++. I have to 
 say, that was
 a really fun month. :)
 After about 3000 lines (of mostly writing a not-yet-finished 
 compiler front-end), it was very exciting!

 I'm mainly interested on the Baremetal D runtime project, as I 
 am mostly comfortable
 with low-level C programming. I have developed some allocators 
 [1]
 and have some SIMD experience as I have written a small image 
 convolution MPI
 program that uses AVX extensions [2].

 I'm interested in other projects as well, specifically 
 compiler-related and the persistent data structures.

 I'm very enthusiastic to learn a lot about any of these 
 projects, but I would like to
 ask: What is the expected / required experience / knowledge / 
 skills?
 Also, is this the best place to discuss about this subject?

 Best regards,
 Stefanos Baziotis


 [1] https://github.com/baziotis/Allocators
 [2] https://github.com/baziotis/2D-Image-Convolution-MPI-SIMD
Hi Stefanos, Very interesting, it would be nice to have SIMD convolution algorithms in D. SIMD extensions actively used in mir-glas (experimental D BLAS implementation), a D's FFT library should use SIMD as well. The same for low-level BetterC projects. Many Mir libraries were created to be used from C/C++/Python. We need a few D language extensions: 1. multiple auto ref values 2. more clever alias syntax for templates (it is required for data science libraries in D) 3. opApply and foreach API design rework (requires 1.) Each of them requires a DIP (D Improvement Proposal) and Pull Request to DMD with required changes. As a small bonus, I will pay an additional $400 for each D language extension merged to DMD. https://github.com/libmir Let me know if you would like to work during GSoC on one of the related directions and especially DIPs. ilyayaroshenko at google mai Best regards, Ilya
Apr 07
next sibling parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Sunday, 7 April 2019 at 07:27:01 UTC, 9il wrote:

 Very interesting, it would be nice to have SIMD convolution 
 algorithms in D.
I'm not an expert in image convolution, but any proposals are welcome.
 We need a few D language extensions:
  1. multiple auto ref values
  2. more clever alias syntax for templates (it is required for 
 data science libraries in D)
  3. opApply and foreach API design rework (requires 1.)
Can you please explain more 1. and 2. ?
 As a small bonus, I will pay an additional $400 for each D 
 language extension merged to DMD.
That's kind :). For me, certainly not needed though.
 Let me know if you would like to work during GSoC on one of the 
 related directions and especially DIPs.
Yes.
 ilyayaroshenko at google mai
I will reach you by mail.
Apr 07
parent 9il <ilyayaroshenko gmail.com> writes:
On Sunday, 7 April 2019 at 11:10:02 UTC, Stefanos Baziotis wrote:
 On Sunday, 7 April 2019 at 07:27:01 UTC, 9il wrote:

 Very interesting, it would be nice to have SIMD convolution 
 algorithms in D.
I'm not an expert in image convolution, but any proposals are welcome.
 We need a few D language extensions:
  1. multiple auto ref values
  2. more clever alias syntax for templates (it is required for 
 data science libraries in D)
  3. opApply and foreach API design rework (requires 1.)
Can you please explain more 1. and 2. ?
 As a small bonus, I will pay an additional $400 for each D 
 language extension merged to DMD.
That's kind :). For me, certainly not needed though.
 Let me know if you would like to work during GSoC on one of 
 the related directions and especially DIPs.
Yes.
 ilyayaroshenko at google mail
I will reach you by mail.
Copy-pasted the answer here, maybe someone has good ideas for syntax. 1. Multiple auto ref values. D allows returning a single value from a function. To return multiple values few workarounds can be used: a. Add additional arguments as ref/out parameters. b. Use std.type.Tuple Both of them don't allow to return multiple values by reference: for example, references on array/container elements. To return multiple values by reference we can use c. Add additional arguments as ref/out pointer parameters d. Use mir.functional.RefTuple Both of them don't well fit to modern D, Mir, and future generic containers. For example, Mir's Series consist as pair of two arrays (keys and values), keys are sorted. It would be awesome to use D Range syntax (front, popFront, empty) to iterate Series by reference. Also, it would be very good for performance, for D GC-free reference-counted libraries. The possible (but may not be the best) syntax is: auto ref fun(int[] a) { return (a[0], a[1] * 3.4); // returns ref int and double } void handle(ref int a, double b) { a = cast(int) (b * b); } int twice(int a) { a * 2; } void main() { int[] ar = [1, 2]; handle(fun(ar)); auto (i : &$0, f, e : $0 + $1, f : $1.twice) = fun(ar); // i is a pointer to ar[0], type of int* // d stores a values of a[1] * 3.4, type of double // e stores value ar[0] + a[1] * 3.4, type of double // f stores value ar[0] * 2, type of int } 2. This DIP for clever alias syntax is the fix for the following issues: https://issues.dlang.org/show_bug.cgi?id=16486 https://issues.dlang.org/show_bug.cgi?id=16465 The brief DIP idea is that the code like below should work: alias PackedUpperTriangularMatrix(T) = Slice!(StairsIterator!(T*, "-")); // fails, issue 16486 auto foo(T)(PackedUpperTriangularMatrix!T m) { } // Current workaround: it is too crazy for users to // know what is StairsIterator!(T*, "-")). auto foo(T)(Slice!(StairsIterator!(T*, "-")) m) { } Currently used Slice types in Lubeck / Production code Slice!(double*) - D slice analog Slice!(double*, 1, Universal) - BLAS vector, used in mir-lapack and mir-blas. Slice!(double*, 2) - Contiguous matrix, that has an efficient loop for iteration over elements, see mir.algorithm.iteration sources. Slice!(double*, 2, Canonical) - BLAS/LAPACK matrix representation, used in mir-lapack and mir.blas Slice!(double*, N, Universal) - zero copy view to work with ndarray in numpy, see also low level API bindgins, and high level bindings Slice!(StairsIterator!(double*, "+")) and ... Slice!(StairsIterator!(double*, "-")) - packed storage for triangular matrixes, for BLAS/LAPACK Slice!(ChopIterator!(size_t*, uint*)); - Memory efficient graph representation without labels. Possible future Slice types (2019?): Slice!(double*, 1, Contiguous, string*) - like Pandas Series Slice!(double*, 2, Contiguous, LabelT1*, LabelT2*) - like Pandas DataFrame Slice!(double*, 2, Contiguous, LabelT1*, LabelT2*, LabelT3*) - like Pandas Panel Slice!(ChopIterator!(size_t*, uint*), 1, Contiguous, string*); - Memory efficient graph representation with labels. Slice!(ChopIterator!(size_t*, Slice!(double*, 1, Contiguous, uint*))) - Sparse Matrix representation that can be used to interact with existing C/C++/Fortran libraries
Apr 07
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Sunday, 7 April 2019 at 07:27:01 UTC, 9il wrote:
 

 We need a few D language extensions:
  1. multiple auto ref values
  2. more clever alias syntax for templates (it is required for 
 data science libraries in D)
  3. opApply and foreach API design rework (requires 1.)

 Each of them requires a DIP (D Improvement Proposal) and Pull 
 Request to DMD with required changes.

 As a small bonus, I will pay an additional $400 for each D 
 language extension merged to DMD.
I'm in for an extra $100 at least for #2, could be persuaded on the others.
Apr 08
parent reply Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Monday, 8 April 2019 at 13:34:04 UTC, jmh530 wrote:
 I'm in for an extra $100 at least for #2, could be persuaded on 
 the others.
Haha, it is exciting that the community cares about those things a lot. I think they need a separate thread, in which anyone is welcome to propose how one could go about solving them (a thing I'm trying to figure out too).
Apr 08
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Monday, 8 April 2019 at 15:58:10 UTC, Stefanos Baziotis wrote:
 On Monday, 8 April 2019 at 13:34:04 UTC, jmh530 wrote:
 I'm in for an extra $100 at least for #2, could be persuaded 
 on the others.
Haha, it is exciting that the community cares about those things a lot. I think they need a separate thread, in which anyone is welcome to propose how one could go about solving them (a thing I'm trying to figure out too).
You could start here: https://forum.dlang.org/post/jyzgzxqgaggltgifwnxx forum.dlang.org
Apr 08
parent Stefanos Baziotis <sdi1600105 di.uoa.gr> writes:
On Monday, 8 April 2019 at 16:20:39 UTC, jmh530 wrote:
 You could start here:
 https://forum.dlang.org/post/jyzgzxqgaggltgifwnxx forum.dlang.org
Thanks, I'm discussing with Ilya about something related to the DataFrame project (specifically, the points that he wrote in the previous post). DataFrame project is rough.. I will certainly sum up our discussion in a separate thread so that anyone can see and add.
Apr 08