digitalmars.D - DMD supports ranges, but druntime can't see them?
- Manu (12/12) May 22 2019 I noticed a bizarre situation just now; while DMD has magic in the
- Mike Franklin (30/54) May 22 2019 I think Andrei said it best here:
- Manu (7/36) May 23 2019 Right... but we're a decade away from that vision, we should do
- Mike Franklin (6/7) May 23 2019 Take your best guess, submit a PR and see what the reviewers say.
- Mike Franklin (9/16) May 23 2019 A more conservative approach would be to put them in
- Seb (6/24) May 23 2019 Yes, let's go with core.internal.range.
- Jonathan M Davis (14/41) May 23 2019 Anything along those lines that gets done in druntime at this point need...
- Timo Sintonen (12/28) May 23 2019 We actually do need a runtime library. It should contain code for
- Mike Franklin (38/42) May 23 2019 I agree. But consider that that runtime library is also written
- jmh530 (5/19) May 23 2019 I like the sound of this, though I think it would be helpful to
- Atila Neves (6/24) May 23 2019 These are good questions.
- Mike Franklin (11/13) May 23 2019 Yes, my understanding is that the final binary would only have a
- Andrei Alexandrescu (2/24) May 23 2019 Ideally all of those would be small function templates.
I noticed a bizarre situation just now; while DMD has magic in the compiler to understand ranges and lower foreach loops to call range primitives and such, the material to determine any range API is not available to druntime, it's all the way out in std.range.primitives. Why is there no core.range with said primitives in it? I think it's fine that chain and zip and iota all that jazz is in phobos, but surely isInputRange, isRandomAccessRange, hasLength, etc should all be in the lowest level runtime possible? It's kinda the same deal as those select traits which are practically part of the language, they don't generate any code, and it's virtually impossible to do anything useful without them. Should they be moved?
May 22 2019
On Thursday, 23 May 2019 at 06:16:08 UTC, Manu wrote:I noticed a bizarre situation just now; while DMD has magic in the compiler to understand ranges and lower foreach loops to call range primitives and such, the material to determine any range API is not available to druntime, it's all the way out in std.range.primitives. Why is there no core.range with said primitives in it? I think it's fine that chain and zip and iota all that jazz is in phobos, but surely isInputRange, isRandomAccessRange, hasLength, etc should all be in the lowest level runtime possible?I think Andrei said it best here: https://forum.dlang.org/post/q7j4sl$17pe$1 digitalmars.com ---Oh, and druntime must go. The whole distinction between the runtime library and the standard library is clowny and has more to do with poorly ported tradition from other languages, than with anything else. We need one standard library that is entirely pay-as-you-go (e.g. empty main() means no library file is ever needed for linking) and that offers an opt-in continuum.--- So, the divide between druntime and phobos shouldn't even exist. But what does a united druntime/Phobos look like? How do we organize it logically, and modularly with clear boundaries, so it doesn't turn into a tangled mess and users only pay for what they use? I don't know, but I'm trying to figure that out. Andrei and I have a difference of opinion on how to achieve that which is apparent in this conversation: https://github.com/dlang/dmd/pull/9814 I'm trying to engage with him via e-mail and other PRs (e.g. https://github.com/JinShil/utiliD/pull/7) to incrementally work something out and get a clearer vision of what the future should look like. There are also some prerequisites that need to be implemented first before some of this stuff will work right (e.g. converting some runtime hooks to templates). That's also being worked on.It's kinda the same deal as those select traits which are practically part of the language, they don't generate any code, and it's virtually impossible to do anything useful without them. Should they be moved?For now, yeah, perhaps they should just be moved to `core.range` or something like that until a better vision of what a united, pay-as-you-go druntime/phobos is worked out. I'm mentoring a GSoC project this year and we could actually benefit from having some of that range stuff in druntime. And this isn't unique to the range stuff. There's quite a bit in Phobos (std.traits, std.conv, std.meta, etc...) that really needs to be available to all D programs including druntime and the compiler itself. Mike
May 22 2019
On Wed, May 22, 2019 at 11:45 PM Mike Franklin via Digitalmars-d <digitalmars-d puremagic.com> wrote:On Thursday, 23 May 2019 at 06:16:08 UTC, Manu wrote:Right... but we're a decade away from that vision, we should do something for the time being.I noticed a bizarre situation just now; while DMD has magic in the compiler to understand ranges and lower foreach loops to call range primitives and such, the material to determine any range API is not available to druntime, it's all the way out in std.range.primitives. Why is there no core.range with said primitives in it? I think it's fine that chain and zip and iota all that jazz is in phobos, but surely isInputRange, isRandomAccessRange, hasLength, etc should all be in the lowest level runtime possible?I think Andrei said it best here: https://forum.dlang.org/post/q7j4sl$17pe$1 digitalmars.comIndeed. Although only subsets of each of those files for my money, and they feel improperly organised to me. So, std.range.primitives -> core.range ?It's kinda the same deal as those select traits which are practically part of the language, they don't generate any code, and it's virtually impossible to do anything useful without them. Should they be moved?For now, yeah, perhaps they should just be moved to `core.range` or something like that until a better vision of what a united, pay-as-you-go druntime/phobos is worked out. I'm mentoring a GSoC project this year and we could actually benefit from having some of that range stuff in druntime. And this isn't unique to the range stuff. There's quite a bit in Phobos (std.traits, std.conv, std.meta, etc...) that really needs to be available to all D programs including druntime and the compiler itself.
May 23 2019
On Thursday, 23 May 2019 at 07:17:19 UTC, Manu wrote:So, std.range.primitives -> core.range ?Take your best guess, submit a PR and see what the reviewers say. If I had to make a decision, I'd just mimic what's in Phobos (i.e. replace `std` with `core`), unless there's a good reason not to. Mike
May 23 2019
On Thursday, 23 May 2019 at 07:26:34 UTC, Mike Franklin wrote:On Thursday, 23 May 2019 at 07:17:19 UTC, Manu wrote:A more conservative approach would be to put them in `core.internal`. `core.internal.traits` is already there. My understanding is that anything in `core.internal` is not intended for public consumption, so we could start there without committing to a public interface that we'd have to maintain. It can always be exposed publicly later when the future becomes more clear. MikeSo, std.range.primitives -> core.range ?Take your best guess, submit a PR and see what the reviewers say. If I had to make a decision, I'd just mimic what's in Phobos (i.e. replace `std` with `core`), unless there's a good reason not to.
May 23 2019
On Thursday, 23 May 2019 at 07:40:20 UTC, Mike Franklin wrote:On Thursday, 23 May 2019 at 07:26:34 UTC, Mike Franklin wrote:Yes, let's go with core.internal.range. Motivation: when we can the package name, we have the unique opportunity of getting rid of some historic range missteps like auto-decoding for strings or using a special save() constructor for ForwardRanges to support class ranges.On Thursday, 23 May 2019 at 07:17:19 UTC, Manu wrote:A more conservative approach would be to put them in `core.internal`. `core.internal.traits` is already there. My understanding is that anything in `core.internal` is not intended for public consumption, so we could start there without committing to a public interface that we'd have to maintain. It can always be exposed publicly later when the future becomes more clear. MikeSo, std.range.primitives -> core.range ?Take your best guess, submit a PR and see what the reviewers say. If I had to make a decision, I'd just mimic what's in Phobos (i.e. replace `std` with `core`), unless there's a good reason not to.
May 23 2019
Jonathan M Davis <newsgroup.d jmdavisprog.com> On Thursday, May 23, 2019 1:45:23 AM MDT Seb via Digitalmars-d wrote:On Thursday, 23 May 2019 at 07:40:20 UTC, Mike Franklin wrote:Anything along those lines that gets done in druntime at this point needs to not be part of the public API, because if we're really going to do some kind of phobos v2 and fix some of the mistakes with ranges, we want to get it right. And if we have something in core which is public, that will get in the way of that. Either way, it won't work to copy what we have in std.range.primitives right now (at least not for strings), because that relies on std.utf. However, since actually redesigning ranges is a much larger discussion, it really wouldn't be appropriate to just throw a new API in druntime either. So, I don't know that it's appropriate to put much in the way of range stuff in druntime at the moment. Not much there should even need it either. Regardless, anything that is done needs to be completely internal for now. - Jonathan M DavisOn Thursday, 23 May 2019 at 07:26:34 UTC, Mike Franklin wrote:Yes, let's go with core.internal.range. Motivation: when we can the package name, we have the unique opportunity of getting rid of some historic range missteps like auto-decoding for strings or using a special save() constructor for ForwardRanges to support class ranges.On Thursday, 23 May 2019 at 07:17:19 UTC, Manu wrote:A more conservative approach would be to put them in `core.internal`. `core.internal.traits` is already there. My understanding is that anything in `core.internal` is not intended for public consumption, so we could start there without committing to a public interface that we'd have to maintain. It can always be exposed publicly later when the future becomes more clear. MikeSo, std.range.primitives -> core.range ?Take your best guess, submit a PR and see what the reviewers say. If I had to make a decision, I'd just mimic what's in Phobos (i.e. replace `std` with `core`), unless there's a good reason not to.
May 23 2019
On Thursday, 23 May 2019 at 06:41:04 UTC, Mike Franklin wrote:We actually do need a runtime library. It should contain code for compiler generated runtime function calls. Nothing else. Users should have no need to import it or even have access to it. The rest should be moved. User callable code might go to phobos. Files that have only bindings and definitions might be in phobos or in utiliD. Then there is code and imports that are not for users but used internally in the library. I am not sure where they should be but utiliD might be a good place. What is left in runtime should not depend on phobos or utiliD. Some libc calls are always needed but they shoud be used as little as possible.Oh, and druntime must go. The whole distinction between the runtime library and the standard library is clowny and has more to do with poorly ported tradition from other languages, than with anything else. We need one standard library that is entirely pay-as-you-go (e.g. empty main() means no library file is ever needed for linking) and that offers an opt-in continuum.--- So, the divide between druntime and phobos shouldn't even exist. But what does a united druntime/Phobos look like? How do we organize it logically, and modularly with clear boundaries, so it doesn't turn into a tangled mess and users only pay for what they use? I don't know, but I'm trying to figure that out.
May 23 2019
On Thursday, 23 May 2019 at 09:46:14 UTC, Timo Sintonen wrote:We actually do need a runtime library. It should contain code for compiler generated runtime function calls. Nothing else.I agree. But consider that that runtime library is also written in D, and can have some complex constructs. That is why I want to build something like utiliD, so it can be imported by that runtime library and used to make the runtime library's implementation more idiomatic D. It can also be used by applications that aren't using any runtime features (e.g. some bare-metal platforms, and consuming D libraries from other languages such as C).Users should have no need to import it or even have access to it.This is where I disagree because I'm trying to make the language opt-in and pay-as-you-go. What I'm envisioning is a "Language as a Library". If one wants classes they `import classes;`. If one wants exceptions, they `import exceptions;` etc. The modules one imports contain the compiler lowerings and other constructs to support the feature. The compiler will emit an error if it can't find something that's not imported, including the compiler hooks, (just like it does today if one tries to use a symbol that isn't imported) Users opt-in to a feature by importing the feature. Note that this behavior is predicated on work being done to move all runtime lowerings into the semantic phase of the compiler instead of the IR stage like it is today. That work is currently underway. Walter and Andrei don't appear to like that idea. What they appear to propose is to make *all* runtime constructs templates. The runtime library is automatically imported, but because it's all templates, there's nothing to link. Users opt-in to a feature by simply using it. Either the instantiate one of the runtime templates explicitly, or by using the feature, the compiler lowers an expression to a runtime template implicitly. I'm skeptical of that. For example, do we need to declare `class Object() {}` instead of `class Object {}` in the runtime template library? And think how that would need to scale all the way down the call stack if each runtime implementation, and constructs it uses, and each construct that those constructs use all need to be templated. And what would that do to compile-times? Each module would have to re-compile all the runtime stuff they utilize. And what about the potential for template bloat? Mike
May 23 2019
On Thursday, 23 May 2019 at 11:00:13 UTC, Mike Franklin wrote:[snip] This is where I disagree because I'm trying to make the language opt-in and pay-as-you-go. What I'm envisioning is a "Language as a Library". If one wants classes they `import classes;`. If one wants exceptions, they `import exceptions;` etc. The modules one imports contain the compiler lowerings and other constructs to support the feature. The compiler will emit an error if it can't find something that's not imported, including the compiler hooks, (just like it does today if one tries to use a symbol that isn't imported) Users opt-in to a feature by importing the feature. Note that this behavior is predicated on work being done to move all runtime lowerings into the semantic phase of the compiler instead of the IR stage like it is today. That work is currently underway.I like the sound of this, though I think it would be helpful to have some sense of the size of the impact (on compile-times, run-times, and memory, at least) when those features are used (and not just when they are not used).
May 23 2019
On Thursday, 23 May 2019 at 11:00:13 UTC, Mike Franklin wrote:On Thursday, 23 May 2019 at 09:46:14 UTC, Timo Sintonen wrote: Walter and Andrei don't appear to like that idea. What they appear to propose is to make *all* runtime constructs templates. The runtime library is automatically imported, but because it's all templates, there's nothing to link. Users opt-in to a feature by simply using it. Either the instantiate one of the runtime templates explicitly, or by using the feature, the compiler lowers an expression to a runtime template implicitly.I would also favour this approach.I'm skeptical of that. For example, do we need to declare `class Object() {}` instead of `class Object {}` in the runtime template library? And think how that would need to scale all the way down the call stack if each runtime implementation, and constructs it uses, and each construct that those constructs use all need to be templated. And what would that do to compile-times? Each module would have to re-compile all the runtime stuff they utilize. And what about the potential for template bloat?These are good questions. I've never understood complaints about template bloat however. If the linker does its job, does it matter that individual object files are larger?
May 23 2019
On Thursday, 23 May 2019 at 13:01:58 UTC, Atila Neves wrote:If the linker does its job, does it matter that individual object files are larger?Yes, my understanding is that the final binary would only have a single copy of whatever templates were instantiated. However, there is a danger for template bloat if their are too many template permutations based on type or other compile-time factor. For example, if the authors of druntime create large functions templated on type without factoring out the type-independent code into smaller functions. Then template bloat could become a problem. It's just something that needs to be carefully monitored if everything in the runtime is to be a template. Mike
May 23 2019
On 5/23/19 5:46 AM, Timo Sintonen wrote:On Thursday, 23 May 2019 at 06:41:04 UTC, Mike Franklin wrote:Ideally all of those would be small function templates.We actually do need a runtime library. It should contain code for compiler generated runtime function calls. Nothing else.Oh, and druntime must go. The whole distinction between the runtime library and the standard library is clowny and has more to do with poorly ported tradition from other languages, than with anything else. We need one standard library that is entirely pay-as-you-go (e.g. empty main() means no library file is ever needed for linking) and that offers an opt-in continuum.--- So, the divide between druntime and phobos shouldn't even exist. But what does a united druntime/Phobos look like? How do we organize it logically, and modularly with clear boundaries, so it doesn't turn into a tangled mess and users only pay for what they use? I don't know, but I'm trying to figure that out.
May 23 2019