www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Motive behind !empty() with front() instead of Optional front()

reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

instead of having front returning an optional/maybe type with 
enforced pattern matching?

Lack of builtin Optional type?

Choosing the Optional path would have avoided the need for 
putting error diagnostics such as

https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

in the standard library of such an alternative solution.
Mar 24 2021
next sibling parent reply Meta <jared771 gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.


Most/all of the range stuff was Andrei's idea, and his C++ 
background heavily informed the design. Returning Optional<T> to 
represent that a calculation might fail is not common in C++, or 
at least wasn't back then. In general, more functional-oriented 
techniques have only recently started diffusing into the C++ 
culture (or at least it feels that way to me; I don't really 
follow C++ closely anymore).
Mar 24 2021
parent reply IGotD- <nise nise.com> writes:
On Wednesday, 24 March 2021 at 19:32:22 UTC, Meta wrote:

 Most/all of the range stuff was Andrei's idea, and his C++ 
 background heavily informed the design. Returning Optional<T> 
 to represent that a calculation might fail is not common in 
 C++, or at least wasn't back then. In general, more 
 functional-oriented techniques have only recently started 
 diffusing into the C++ culture (or at least it feels that way 
 to me; I don't really follow C++ closely anymore).


That it is because of historical reasons makes sense. From a 
programming point of view, it doesn't really matter since you 
must check your Optional<T> return value anyway and you still get 
a check regardless. Performance wise it doesn't matter.

Another motive would be that it would force to check if the 
return value is not empty, however you might not want that. For 
example in a loop and you know the amount of elements in advance 
then the check is unnecessary.
Mar 24 2021
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 24 March 2021 at 21:09:52 UTC, IGotD- wrote:

 For example in a loop and you know the amount of elements in 
 advance then the check is unnecessary.


Are you talking about a foreach loop? If so, the element type 
will never have to be an optional type. Regardless of whether 
`front` is wrapped in an Optional or not.
Mar 24 2021
parent IGotD- <nise nise.com> writes:
On Wednesday, 24 March 2021 at 21:20:40 UTC, Per Nordlöw wrote:

 Are you talking about a foreach loop? If so, the element type 
 will never have to be an optional type. Regardless of whether 
 `front` is wrapped in an Optional or not.


Could be a foreach loop but also a counted loop, you want to go 
through a certain amount of elements and you know for sure with 
check before that the elements exist. I'm not sure if I 
understand your statement, but wrapping such a loop with 
Optional<T> would lead to an extra overhead and since it is 
inside a loop it is not insignificant. I wouldn't trust an 
optimizer would understand how to remove such check.

Now the possibility is to have to versions of front, one that 
returns the element and another that returns Optional<T> (why do 
I write C++ syntax?). Not sure if that would lead to ambiguous 
case problems.
Mar 24 2021
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Wednesday, 24 March 2021 at 21:09:52 UTC, IGotD- wrote:

 Another motive would be that it would force to check if the 
 return value is not empty, however you might not want that. For 
 example in a loop and you know the amount of elements in 
 advance then the check is unnecessary.


If you know the Optional isn't empty you can just call `.unwrap` 
(or whatever it ends up being called) to get the value without 
checking.
Mar 24 2021
prev sibling next sibling parent Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.


Is optional not having a value a good way to know you are at the 
end of a list? I know that Nullable can use a flag, but if you 
return an optional with a value of null, feels like you are 
defeating the purpose of optional.
Mar 24 2021
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.


The big question is, what if I want a range of Optional!T ?
Mar 25 2021
next sibling parent Rumbu <rumbu rumbu.ro> writes:
On Thursday, 25 March 2021 at 09:35:40 UTC, deadalnix wrote:

 On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.


 The big question is, what if I want a range of Optional!T ?


You have to check on Optional!(Optional!T).
Mar 25 2021
prev sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 25 March 2021 at 09:35:40 UTC, deadalnix wrote:

 The big question is, what if I want a range of Optional!T ?


That might lead to confusing code, yes. Aliases or subtypes could 
come of use in such a situation. Better ask the Rust forums...
Mar 25 2021
prev sibling next sibling parent reply Piotr Mitana <piotr.mitana gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.


Actually changing front() to return optionals would probably 
break many things. However, it could be handy to add a method 
called nullableFront or frontOrNull (and nullableBack or 
backOrNull for bidirectional ranges) to std.range.
Mar 26 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 08:49:30 UTC, Piotr Mitana wrote:

 Actually changing front() to return optionals would probably 
 break many things. However, it could be handy to add a method 
 called nullableFront or frontOrNull (and nullableBack or 
 backOrNull for bidirectional ranges) to std.range.


Agreed. Alternative namings could be maybeFront or optionalFront.
Mar 26 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 09:10:28 UTC, Per Nordlöw wrote:

 On Friday, 26 March 2021 at 08:49:30 UTC, Piotr Mitana wrote:

 Actually changing front() to return optionals would probably 
 break many things. However, it could be handy to add a method 
 called nullableFront or frontOrNull (and nullableBack or 
 backOrNull for bidirectional ranges) to std.range.


 Agreed. Alternative namings could be maybeFront or 
 optionalFront.


A D specific solution could be to add compiler diagnostics that 
checks whether a call to `x.front` and x.popFront() is guarded by 
an `if (!x.empty)`. Provided no mutation has happened since the 
call to !empty and `x` is not accessible via other threads.
Mar 26 2021
parent reply Nick Treleaven <nick geany.org> writes:
On Friday, 26 March 2021 at 13:23:25 UTC, Per Nordlöw wrote:

 A D specific solution could be to add compiler diagnostics that 
 checks whether a call to `x.front` and x.popFront() is guarded 
 by an `if (!x.empty)`. Provided no mutation has happened since 
 the call to !empty and `x` is not accessible via other threads.


// in  safe code
scope p = &r;
if (!r.empty) {
   p.popFront;
   r.front.writeln; // oops
}

I don't think it's practical for the compiler to detect more 
complicated variants of the above. The beauty of languages with 
compile-time checked optional types is that the value has to 
exist in order to unwrap the optional.
Mar 26 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 15:02:27 UTC, Nick Treleaven wrote:

 compile-time checked optional types is that the value has to 
 exist in order to unwrap the optional.


Can you elaborate on what you mean by "compile-time checked 
optional types"?
Mar 26 2021
parent Nick Treleaven <nick geany.org> writes:
On Friday, 26 March 2021 at 15:51:41 UTC, Per Nordlöw wrote:

 Can you elaborate on what you mean by "compile-time checked 
 optional types"?


Like `if let` in Swift:
https://www.hackingwithswift.com/sixty/10/2/unwrapping-optionals

This form ensures at compile time that a runtime check has made 
in code that accesses the value held by the optional.

(There's also a `guard let` form to unwrap into a variable in the 
current scope that requires you to terminate the current scope 
when the optional is empty).
Mar 26 2021
prev sibling next sibling parent reply vitoroak <carvalhogvm gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.


I think one reason is that unlike Rust, D doesn't have a safe way 
to return Optional!(ref T) so we need front and empty so Ranges 
can return the items by reference.
Mar 26 2021
next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 13:38:01 UTC, vitoroak wrote:

 I think one reason is that unlike Rust, D doesn't have a safe 
 way to return Optional!(ref T) so we need front and empty so 
 Ranges can return the items by reference.


Ahh, thanks.
Mar 26 2021
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 26 March 2021 at 13:38:01 UTC, vitoroak wrote:

 I think one reason is that unlike Rust, D doesn't have a safe 
 way to return Optional!(ref T) so we need front and empty so 
 Ranges can return the items by reference.


I believe with DIP 1000 it should be possible to return something 
like `Optional!(Ref!T)`, where `Ref!T` is a safe wrapper around a 
`T*`.

Of course the ideal would be to make `ref` itself into a type 
qualifier (`ref(T)`).
Mar 26 2021
parent reply Q. Schroll <qs.il.paperinik gmail.com> writes:
On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:

 On Friday, 26 March 2021 at 13:38:01 UTC, vitoroak wrote:

 I think one reason is that unlike Rust, D doesn't have a safe 
 way to return `Optional!(ref T)` so we need front and empty so 
 Ranges can return the items by reference.


 I believe with DIP 1000 it should be possible to return 
 something like `Optional!(Ref!T)`, where `Ref!T` is a safe 
 wrapper around a `T*`.


When I experimented with `Ref` improving on 
[`Final`](https://dlang.org/phobos/std_experimental_typecons.html#.Final) for
pointers (`Ref!T` is `Final!(T*)`), I found it near useless. The only thing you
cannot do with a `Ref!T` but with a `T*` is reassigning it and pointer
arithmetic. But ` safe` already protects you against accidental pointer
arithmetic. Accidental assignment also isn't really an issue in D. I'm all in
for `final` as a type constructor (meaning head-`const`), but it probably won't
lift its weight.


 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).


If you look at all the exceptions C++ has for its reference type 
constructor, you'll immediately see why Walter created `ref` as a 
storage class and not a type constructor. C++ reference types in 
many cases don't compose; e.g. `vector<int&>` isn't a thing.
If you really think about it, you don't need `ref` as a type 
constructor. Although allowing `ref` local variables wouldn't be 
harmful, I guess.
Apr 05 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 6 April 2021 at 01:47:47 UTC, Q. Schroll wrote:

 On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:

 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).


 If you look at all the exceptions C++ has for its reference 
 type constructor, you'll immediately see why Walter created 
 `ref` as a storage class and not a type constructor. C++ 
 reference types in many cases don't compose; e.g. 
 `vector<int&>` isn't a thing.
 If you really think about it, you don't need `ref` as a type 
 constructor. Although allowing `ref` local variables wouldn't 
 be harmful, I guess.


Here's what the generic identity function currently looks like in 
D:

     auto ref T identity(T)(auto ref T arg)
     {
         import core.lifetime: forward;
         return forward!arg;
     }

Here's what the generic identity function *would* look like if 
`ref` were a type qualifier:

     T identity(T)(T arg)
     {
         return arg;
     }

I rest my case.
Apr 05 2021
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:

 Here's what the generic identity function *would* look like if 
 `ref` were a type qualifier:

     T identity(T)(T arg)
     {
         return arg;
     }

 I rest my case.


So what was the motive for not making it a type qualifier in D?

And what are the obstacles for making it that?
Apr 06 2021
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 April 2021 at 20:34:08 UTC, Per Nordlöw wrote:

 So what was the motive for not making it a type qualifier in D?

 And what are the obstacles for making it that?


See 
https://forum.dlang.org/post/mailman.7903.1554072555.29801.digitalmars-d puremagic.com
Apr 06 2021
prev sibling next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:

     auto ref T identity(T)(auto ref T arg)
     {
         import core.lifetime: forward;
         return forward!arg;
     }


When is the call to forward needed in this case?
Apr 06 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 6 April 2021 at 20:47:31 UTC, Per Nordlöw wrote:

 On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:

     auto ref T identity(T)(auto ref T arg)
     {
         import core.lifetime: forward;
         return forward!arg;
     }


 When is the call to forward needed in this case?


For non-copyable types. It's actually needed in both cases--we 
would need DIP 1040 (or something similar) to get rid of it.
Apr 06 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 6 April 2021 at 21:09:13 UTC, Paul Backus wrote:

 For non-copyable types. It's actually needed in both cases--we 
 would need DIP 1040 (or something similar) to get rid of it.


So let's help Walter getting DIP-1040 accepted then. :)

What else is forward needed for? The doc says

"Forwards function arguments while keeping `out`, `ref`, and 
`lazy` on
the parameters."

Why can't the compiler do that for us?
Apr 06 2021
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/6/21 5:14 PM, Per Nordlöw wrote:

 On Tuesday, 6 April 2021 at 21:09:13 UTC, Paul Backus wrote:

 For non-copyable types. It's actually needed in both cases--we would 
 need DIP 1040 (or something similar) to get rid of it.


 
 So let's help Walter getting DIP-1040 accepted then. :)
 
 What else is forward needed for? The doc says
 
 "Forwards function arguments while keeping `out`, `ref`, and `lazy` on
 the parameters."
 
 Why can't the compiler do that for us?


Because sometimes you want the usual semantics, i.e. create a copy of 
the argument.

These things are difficult to automate. I don't think a simple solution 
exists.
Apr 07 2021
parent Max Haughton <maxhaton gmail.com> writes:
On Thursday, 8 April 2021 at 02:16:31 UTC, Andrei Alexandrescu 
wrote:

 On 4/6/21 5:14 PM, Per Nordlöw wrote:

 On Tuesday, 6 April 2021 at 21:09:13 UTC, Paul Backus wrote:

 For non-copyable types. It's actually needed in both 
 cases--we would need DIP 1040 (or something similar) to get 
 rid of it.


 
 So let's help Walter getting DIP-1040 accepted then. :)
 
 What else is forward needed for? The doc says
 
 "Forwards function arguments while keeping `out`, `ref`, and 
 `lazy` on
 the parameters."
 
 Why can't the compiler do that for us?


 Because sometimes you want the usual semantics, i.e. create a 
 copy of the argument.

 These things are difficult to automate. I don't think a simple 
 solution exists.


I definitely agree that the "simple solution" probably doesn't 
exist, however I am kind of partial to just leaving it to the 
caller (especially for `out`, I've found in some places). It 
conflicts inside me however, because I like very plastic 
interfaces - however it kills resolution questions like you 
raised (lvalue vs. reference to lvalue) above.
Apr 07 2021
prev sibling next sibling parent reply Q. Schroll <qs.il.paperinik gmail.com> writes:
On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:

 On Tuesday, 6 April 2021 at 01:47:47 UTC, Q. Schroll wrote:

 On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:

 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).


 If you look at all the exceptions C++ has for its reference 
 type constructor, you'll immediately see why Walter created 
 `ref` as a storage class and not a type constructor. C++ 
 reference types in many cases don't compose; e.g. 
 `vector<int&>` isn't a thing.
 If you really think about it, you don't need `ref` as a type 
 constructor. Although allowing `ref` local variables wouldn't 
 be harmful, I guess.


 Here's what the generic identity function currently looks like 
 in D:

     auto ref T identity(T)(auto ref T arg)
     {
         import core.lifetime: forward;
         return forward!arg;
     }

 Here's what the generic identity function *would* look like if 
 `ref` were a type qualifier:

     T identity(T)(T arg)
     {
         return arg;
     }

 I rest my case.


In C++, the identity function is
```C++
template<typename T>
T&& identitiy(T&& arg)
{
     return std::forward<T>(arg);
}
```
if you want it to work like `identity(arg)`.

**TL;DR:** C++ doesn't do it the simple way for a reason.

If you do
```C++
template<typename T>
T identitiy(T arg)
{
     return arg;
}
```
you get copies unless you call it `identity<int&>(arg)` and 
`identity<int&&>(1)` (which defeats the purpose, I guess.

Template type deduction alone (C++ has 4 different rule sets 
about type deduction) is complicated. One reason for stripping 
`ref` from the argument types unless they explicitly bind to 
`ref` parameters is that references should act like aliases:
```C++
int i;
int& r = i;
f(i);
f(r);
```
Here, `f(r)` should act the same as `f(i)` irrespective of how 
`f` is defined. `f` could take `int`, `int&`, `const int&`, or 
`T`, `const T&`, or `T&&` for some `T` that can implicitly 
constructed from `int`. For `int&&` or `T&`it wouldn't compile. 
References as type constructors are weird. And please note, this 
is what I *immediately* could come up with and I verified. 
There's stuff with function types and static array bounds that I 
remember have shenanigans with references, but I'd have to do 
some search for details.

The only thing D does wrong in this regard is not having 
`forward` and `move` in `object.d`.
Apr 07 2021
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Wednesday, 7 April 2021 at 21:31:38 UTC, Q. Schroll wrote:

 Template type deduction alone (C++ has 4 different rule sets 
 about type deduction) is complicated. One reason for stripping 
 `ref` from the argument types unless they explicitly bind to 
 `ref` parameters is that references should act like aliases:
 ```C++
 int i;
 int& r = i;
 f(i);
 f(r);
 ```
 Here, `f(r)` should act the same as `f(i)` irrespective of how 
 `f` is defined. `f` could take `int`, `int&`, `const int&`, or 
 `T`, `const T&`, or `T&&` for some `T` that can implicitly 
 constructed from `int`. For `int&&` or `T&`it wouldn't compile. 
 References as type constructors are weird.


References as type constructors are weird *in C++* because of 
specific choices made by the designers of C++ (such as: 
"references should act like aliases"). There is nothing inherent 
to the concept of a reference type that implies this kind of 
weirdness. D does not have to make the same choices as C++, and 
`ref(T)` in D does not have to suffer from the issues that 
references suffer from in C++.
Apr 07 2021
parent reply Q. Schroll <qs.il.paperinik gmail.com> writes:
On Wednesday, 7 April 2021 at 21:44:52 UTC, Paul Backus wrote:

 On Wednesday, 7 April 2021 at 21:31:38 UTC, Q. Schroll wrote:

 Template type deduction alone (C++ has 4 different rule sets 
 about type deduction) is complicated. One reason for stripping 
 `ref` from the argument types unless they explicitly bind to 
 `ref` parameters is that references should act like aliases:
 ```C++
 int i;
 int& r = i;
 f(i);
 f(r);
 ```
 Here, `f(r)` should act the same as `f(i)` irrespective of how 
 `f` is defined. `f` could take `int`, `int&`, `const int&`, or 
 `T`, `const T&`, or `T&&` for some `T` that can implicitly 
 constructed from `int`. For `int&&` or `T&`it wouldn't 
 compile. References as type constructors are weird.


 References as type constructors are weird *in C++* because of 
 specific choices made by the designers of C++ (such as: 
 "references should act like aliases"). There is nothing 
 inherent to the concept of a reference type that implies this 
 kind of weirdness. D does not have to make the same choices as 
 C++, and `ref(T)` in D does not have to suffer from the issues 
 that references suffer from in C++.


As Andrei said, in C++, apart from the `decltype` exception 
(which is justified), references behave as aliases. In D, it's 
the same, apart from `__trais(isRef)` which is also justified. 
What else do you expect from a reference object? Maybe I'm 
biased, but I don't see how other solutions than C++'s would be 
better.
Apr 09 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 9 April 2021 at 20:40:54 UTC, Q. Schroll wrote:

 As Andrei said, in C++, apart from the `decltype` exception 
 (which is justified), references behave as aliases. In D, it's 
 the same, apart from `__trais(isRef)` which is also justified. 
 What else do you expect from a reference object? Maybe I'm 
 biased, but I don't see how other solutions than C++'s would be 
 better.


Hypothetically, if `ref` were a type qualifier, I'd expect it to 
behave the same way as D's other type qualifiers. We don't strip 
`const` or `shared` from types during template instantiation (the 
special case of `const(T[])` → `const(T)[]` notwithstanding), so 
my baseline expectation is that we wouldn't strip `ref` either.

Maybe there's an argument to be made that this is a bad idea, and 
that you should always be able to replace an lvalue with a 
reference without any effect on the program's behavior. But I 
don't think you can take it as given that that's *obviously* the 
correct way to do references, just because that's what C++ does.
Apr 09 2021
parent Q. Schroll <qs.il.paperinik gmail.com> writes:
On Friday, 9 April 2021 at 21:17:03 UTC, Paul Backus wrote:

 On Friday, 9 April 2021 at 20:40:54 UTC, Q. Schroll wrote:

 As Andrei said, in C++, apart from the `decltype` exception 
 (which is justified), references behave as aliases. In D, it's 
 the same, apart from `__trais(isRef)` which is also justified. 
 What else do you expect from a reference object? Maybe I'm 
 biased, but I don't see how other solutions than C++'s would 
 be better.


 Hypothetically, if `ref` were a type qualifier, I'd expect it 
 to behave the same way as D's other type qualifiers. We don't 
 strip `const` or `shared` from types during template 
 instantiation (the special case of `const(T[])` → `const(T)[]` 
 notwithstanding), so my baseline expectation is that we 
 wouldn't strip `ref` either.

 Maybe there's an argument to be made that this is a bad idea, 
 and that you should always be able to replace an lvalue with a 
 reference without any effect on the program's behavior. But I 
 don't think you can take it as given that that's *obviously* 
 the correct way to do references, just because that's what C++ 
 does.


The "special case of `const`" is exactly the point: If you have a 
`const(T[]) ts` object and you do
```D
auto ts2 = ts;
```
then, `typeof(ts2)` is `const(T)[]`. The whole point of making a 
copy is that it (or at least the first layer) becomes independent 
and mutable. I think references should behave the same: Assigning 
a `ref(T)` to an `auto` variable should create a copy that (or at 
least the first layer) is independent of the source. Creating 
another reference is near-useless.
When it comes to parameter passing, binding by copy is expected 
to be the default and binding by reference a specialty. One 
reason being that binding by copy is more flexible, e.g. it 
allows (implicit) conversions.

I don't think it's obviously the right way because C++ does it 
that way, but it's a hint. I came to the conclusion myself; 
thinking of various alternatives, I could see obvious problems. 
C++'s references have (obvious?) problems, so it's not like it's 
perfect. IMO, `ref` as a storage class is the Right Thing. D 
falls short in some places:
* Allowing `ref` for local variables would be valuable at times. 
There's no actual gain in not allowing them. Some stuff like 
postfix operators even lower to `ref` local variables.
* One cannot directly express `ref` returning delegate or 
function pointer types on many occasions, e.g. neither `is(DG == 
ref int delegate())` nor `is(DG == int delegate() ref)` compile. 
Also, one cannot directly specify `ref` returning delegate or 
function pointer types in parameter lists. In both cases, an 
alias has to be defined.

If `ref` were a type constructor, these would be non-issues for 
sure. But we're nowhere near *requiring* it to be one to solve 
these.
Apr 09 2021
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/7/21 5:31 PM, Q. Schroll wrote:

 On Tuesday, 6 April 2021 at 02:14:17 UTC, Paul Backus wrote:

 On Tuesday, 6 April 2021 at 01:47:47 UTC, Q. Schroll wrote:

 On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:

 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).


 If you look at all the exceptions C++ has for its reference type 
 constructor, you'll immediately see why Walter created `ref` as a 
 storage class and not a type constructor. C++ reference types in many 
 cases don't compose; e.g. `vector<int&>` isn't a thing.
 If you really think about it, you don't need `ref` as a type 
 constructor. Although allowing `ref` local variables wouldn't be 
 harmful, I guess.


 Here's what the generic identity function currently looks like in D:

     auto ref T identity(T)(auto ref T arg)
     {
         import core.lifetime: forward;
         return forward!arg;
     }

 Here's what the generic identity function *would* look like if `ref` 
 were a type qualifier:

     T identity(T)(T arg)
     {
         return arg;
     }

 I rest my case.


 
 In C++, the identity function is
 ```C++
 template<typename T>
 T&& identitiy(T&& arg)
 {
     return std::forward<T>(arg);
 }
 ```
 if you want it to work like `identity(arg)`.
 
 **TL;DR:** C++ doesn't do it the simple way for a reason.
 
 If you do
 ```C++
 template<typename T>
 T identitiy(T arg)
 {
     return arg;
 }
 ```
 you get copies unless you call it `identity<int&>(arg)` and 
 `identity<int&&>(1)` (which defeats the purpose, I guess.
 
 Template type deduction alone (C++ has 4 different rule sets about type 
 deduction) is complicated. One reason for stripping `ref` from the 
 argument types unless they explicitly bind to `ref` parameters is that 
 references should act like aliases:
 ```C++
 int i;
 int& r = i;
 f(i);
 f(r);
 ```
 Here, `f(r)` should act the same as `f(i)` irrespective of how `f` is 
 defined. `f` could take `int`, `int&`, `const int&`, or `T`, `const T&`, 
 or `T&&` for some `T` that can implicitly constructed from `int`. For 
 `int&&` or `T&`it wouldn't compile. References as type constructors are 
 weird. And please note, this is what I *immediately* could come up with 
 and I verified. There's stuff with function types and static array 
 bounds that I remember have shenanigans with references, but I'd have to 
 do some search for details.
 
 The only thing D does wrong in this regard is not having `forward` and 
 `move` in `object.d`.


Very well put, thank you.
Apr 07 2021
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/21 10:14 PM, Paul Backus wrote:

 On Tuesday, 6 April 2021 at 01:47:47 UTC, Q. Schroll wrote:

 On Friday, 26 March 2021 at 15:12:04 UTC, Paul Backus wrote:

 Of course the ideal would be to make `ref` itself into a type 
 qualifier (`ref(T)`).


 If you look at all the exceptions C++ has for its reference type 
 constructor, you'll immediately see why Walter created `ref` as a 
 storage class and not a type constructor. C++ reference types in many 
 cases don't compose; e.g. `vector<int&>` isn't a thing.
 If you really think about it, you don't need `ref` as a type 
 constructor. Although allowing `ref` local variables wouldn't be 
 harmful, I guess.


 
 Here's what the generic identity function currently looks like in D:
 
     auto ref T identity(T)(auto ref T arg)
     {
         import core.lifetime: forward;
         return forward!arg;
     }
 
 Here's what the generic identity function *would* look like if `ref` 
 were a type qualifier:
 
     T identity(T)(T arg)
     {
         return arg;
     }


I'm not so sure. Complications arise when you pass an lvalue to identity 
- is it to be considered of type T or ref T? There are pros and cons to 
each, which is why C++ chose to allow both; its unprecedented 
decltype(auto) was defined especially to be part of function return type 
when it is to transport refness out.

Same class of problems have caused other anomalies in C++, such as 
decltype(identifier) and decltype((identifier)) having different types.

The case is open and hot.
Apr 07 2021
parent Paul Backus <snarwin gmail.com> writes:
On Thursday, 8 April 2021 at 02:10:55 UTC, Andrei Alexandrescu 
wrote:

 I'm not so sure. Complications arise when you pass an lvalue to 
 identity - is it to be considered of type T or ref T? There are 
 pros and cons to each, which is why C++ chose to allow both; 
 its unprecedented decltype(auto) was defined especially to be 
 part of function return type when it is to transport refness 
 out.


In the hypothetical world where `ref` is a type qualifier, `T` 
and `ref(T)` are distinct types. So if the lvalue's type were `T` 
you'd get `identity!T`, and if its type were `ref(T)`, you'd get 
`identity!(ref(T))`. If you had a non-`ref` lvalue that you 
wanted to pass by reference to a template function that takes an 
argument not declared as `ref` (or `auto ref`), you'd add an 
explicit conversion to `ref(T)` at the call site. None of this 
requires any special cases or complications.

Unfortunately, there's no way to get to this world without a 
fairly substantial breaking change. Currently, it's possible to 
use by-value and by-reference overloads to distinguish between 
rvalues and lvalues:

     void fun(int n) { writeln("rvalue"); }
     void fun(ref int n) { writeln("lvalue"); }

     int a = 123;
     fun(a); // lvalue
     fun(456); // rvalue

Making `ref` into a type qualifier would cause the rvalue 
overload to be called in both cases, since it would be an exact 
match vs. a match by implicit conversion.

Realistically, this is probably too disruptive a change for too 
little benefit, so I expect the best we'll get is a library 
`Ref!T` type, like C++'s `std::reference_wrapper`. Fortunately D 
has the tools to make such a type fairly ergonomic to use.
Apr 07 2021
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/24/21 3:23 PM, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing
 
      if (!empty)
      {
          // use front or back
      }
 
 instead of having front returning an optional/maybe type with enforced 
 pattern matching?
 
 Lack of builtin Optional type?
 
 Choosing the Optional path would have avoided the need for putting error 
 diagnostics such as
 
 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a0445
0c4e6912a13cb5ac694 
 
 
 in the standard library of such an alternative solution.


I would say separation of concerns. Consider:

int someExpensiveCalc(int);

auto r = someArr.map!someExpensiveCalc;

r.empty is simply a forward to someArr.empty. However, r.front calls 
someExpensiveCalc (if not empty).

So code that looks specifically at emptiness might not want to trigger 
the expensive call unnecessarily. e.g. canFind.

Since empty is a distinct abstraction, it's the simplest building block. 
You can actually MAKE a "range-like" thing that uses empty to produce 
what you want. It's problematic the other way around.

That being said, there is definitely a case for straight-input-ranges to 
have a different API (maybe one without empty), since iterating a 
non-forward range can possibly destroy it, and if you change the API to 
simply an Optional!T next(), then you don't need to cache the element 
(currently I think the biggest wart in the range system).

But such a thing would have to possibly be supported in the language 
(ranges are a language feature in that they hook to foreach, I would 
expect the same for such input ranges).

Regarding the question about the github commit, that's an entirely 
different thing. Notice that it's an assert, which means it can be 
removed in correctly-written programs. If you return an Optional, 
emptiness MUST be checked on every access to an element, even when you 
know (or have proven) it's not empty.

I would support a new kind of range that uses `Optional!T next()` as its 
API. I would not support `Optional!T front(); void popFront()`.

-Steve
Mar 26 2021
next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 26 March 2021 at 15:53:39 UTC, Steven Schveighoffer 
wrote:

 ...
 I would support a new kind of range that uses `Optional!T 
 next()` as its API. I would not support `Optional!T front(); 
 void popFront()`.


Interesting.

For reference, note that Rust also has

https://doc.rust-lang.org/std/iter/struct.Peekable.html

alongside

https://doc.rust-lang.org/core/iter/trait.Iterator.html
Mar 26 2021
prev sibling parent reply Nick Treleaven <nick geany.org> writes:
On Friday, 26 March 2021 at 15:53:39 UTC, Steven Schveighoffer 
wrote:

 Notice that it's an assert, which means it can be removed in 
 correctly-written programs. If you return an Optional, 
 emptiness MUST be checked on every access to an element, even 
 when you know (or have proven) it's not empty.


An optional would have an `unwrap` method for this purpose, which 
also only asserts it is not empty and returns a value.
Mar 26 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/26/21 3:20 PM, Nick Treleaven wrote:

 On Friday, 26 March 2021 at 15:53:39 UTC, Steven Schveighoffer wrote:

 Notice that it's an assert, which means it can be removed in 
 correctly-written programs. If you return an Optional, emptiness MUST 
 be checked on every access to an element, even when you know (or have 
 proven) it's not empty.


 
 An optional would have an `unwrap` method for this purpose, which also 
 only asserts it is not empty and returns a value.


That could be possible if the Optional type lazily fetches the 
value/empty status, in which case now you are dealing with delegates 
and/or closures. I don't think this is the right path.

-Steve
Mar 26 2021
parent reply Nick Treleaven <nick geany.org> writes:
On Saturday, 27 March 2021 at 01:06:37 UTC, Steven Schveighoffer 
wrote:

 That could be possible if the Optional type lazily fetches the 
 value/empty status, in which case now you are dealing with 
 delegates and/or closures. I don't think this is the right path.


You are of course right that returning an optional must eagerly 
fetch a value. I was just responding to you saying you have to 
check an optional is empty before accessing the value - you don't.
Mar 27 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/27/21 5:50 AM, Nick Treleaven wrote:

 On Saturday, 27 March 2021 at 01:06:37 UTC, Steven Schveighoffer wrote:

 That could be possible if the Optional type lazily fetches the 
 value/empty status, in which case now you are dealing with delegates 
 and/or closures. I don't think this is the right path.


 
 You are of course right that returning an optional must eagerly fetch a 
 value. I was just responding to you saying you have to check an optional 
 is empty before accessing the value - you don't.


I meant on every call to `front`. With the current system, you can avoid 
this by disabling asserts. With the proposed system, in order to 
properly construct the Optional, emptiness must be checked. The only way 
to defer this check to the user is to have Optional be lazily constructed.

-Steve
Mar 27 2021
prev sibling next sibling parent uranuz <neuranuz gmail.com> writes:
On Wednesday, 24 March 2021 at 19:23:21 UTC, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing

     if (!empty)
     {
         // use front or back
     }

 instead of having front returning an optional/maybe type with 
 enforced pattern matching?

 Lack of builtin Optional type?

 Choosing the Optional path would have avoided the need for 
 putting error diagnostics such as

 https://github.com/dlang/phobos/commit/9bd2f2ba8ff1124a044560c4e6912a13cb5ac694

 in the standard library of such an alternative solution.


Hello! It is a good idea what are you talking about. But I agree 
that it's better to implement it as some method using UFCS. So 
this will not break anything.

Also I want to say that for me most of the times I check for *non 
empty* rather than *empty*. So every time I need to write 
exclamation mark before it. It is not very comfortable. But the 
problem is that I can't say antonym for the word *empty*. It 
actually shall not be antonym, but the word that say that this 
range should have at least one element. My variants are: hasNext, 
hasFront, hasMore, hasAny or smth like that.
The problemme with *empty* is that it has some *negative* 
meaning. But I read somewhere recommendations to prefer 
*positive* words for boolean flags except for rare cases where 
*negative* flags are simplier to work with, because of domain 
area where it's used.
But I believe that range is not such a case. And I explained by 
my own experience that mostly I test for *hasMore* then for 
*empty*.

Sorry for slightly off-topic post ;-)
Mar 29 2021
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/24/21 3:23 PM, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing
 
      if (!empty)
      {
          // use front or back
      }
 
 instead of having front returning an optional/maybe type with enforced 
 pattern matching?


Efficiency. It would be impossible to iterate an array as a range 
without copying each and every element thereof.

We investigated a few other possibilities, such as returning a pointer 
to the next element or null. But that has problems related to safety and 
escaping pointers.
Mar 29 2021
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
wrote:

 We investigated a few other possibilities, such as returning a 
 pointer to the next element or null. But that has problems 
 related to safety and escaping pointers.


Does -preview=dip1000 help at all with these issues? If so, might 
be worth revisiting.
Mar 30 2021
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 30 March 2021 at 16:05:09 UTC, Paul Backus wrote:

 On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
 wrote:

 We investigated a few other possibilities, such as returning a 
 pointer to the next element or null. But that has problems 
 related to safety and escaping pointers.


 Does -preview=dip1000 help at all with these issues? If so, 
 might be worth revisiting.


DIP1000 is not sufficient, and unsound by itself.
Mar 30 2021
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/30/21 12:05 PM, Paul Backus wrote:

 On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu wrote:

 We investigated a few other possibilities, such as returning a pointer 
 to the next element or null. But that has problems related to safety 
 and escaping pointers.


 
 Does -preview=dip1000 help at all with these issues? If so, might be 
 worth revisiting.


A good point.
Mar 30 2021
prev sibling next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
wrote:

 We investigated a few other possibilities, such as returning a 
 pointer to the next element or null. But that has problems 
 related to safety and escaping pointers.


I'm eager to experiment with compiler diagnostics that, for a 
growing number of cases, detects calls to X.front/back and 
X.popFront/popBack when X is `empty`.

What criteria would need to be fulfilled on `X` and its range 
interface members in order for such a diagnostics to kick in?
Mar 30 2021
prev sibling next sibling parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
wrote:

 Efficiency. It would be impossible to iterate an array as a 
 range without copying each and every element thereof.


Most likely affects performance in non-release mode, at least.

I wonder what the generated assembly looks like in Rust for a 
similar case...
Mar 30 2021
prev sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 30 March 2021 at 00:51:44 UTC, Andrei Alexandrescu 
wrote:

 Efficiency. It would be impossible to iterate an array as a 
 range without copying each and every element thereof.


On an efficiency-related note, it's worth remembering that the 
`empty` property can be a compile-time constant (as it is e.g. 
with pseudo-random number generators), which means that in some 
cases it should be possible (at least in principle) to elide any 
`empty` checks at runtime.

I imagine there is a way to achieve the same effective outcome 
with an appropriately designed Option type or trait, but I don't 
know if that would be as straightforward as with the `empty` 
property.


 We investigated a few other possibilities, such as returning a 
 pointer to the next element or null. But that has problems 
 related to safety and escaping pointers.


This is probably a factor in Rust's decision to use Option<T>: 
its iterators over containers do use references (i.e. smart 
pointers), and this is easier option for them because of the 
stronger reference safety constraints.
Mar 31 2021
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2021-03-24 20:23, Per Nordlöw wrote:

 What's the motive behinds D's range design choice of needing
 
     if (!empty)
     {
         // use front or back
     }
 
 instead of having front returning an optional/maybe type with enforced 
 pattern matching?


In my opinion, the best way to interact with an Optional type is not to 
use pattern matching but to use algorithms like `map`, `each` and so on. 
That means an Optional type needs to implement the range API. Might be a 
bit confusioning if `front` then returns an Optional.

-- 
/Jacob Carlborg
Mar 31 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 31 March 2021 at 07:27:18 UTC, Jacob Carlborg wrote:
way to interact with an Optional type

 is not to use pattern matching but to use algorithms like 
 `map`, `each` and so on.


Interesting. Can you show some pseudocode of this? Can those 
algorithms be implemented as non-member functions?
Apr 05 2021
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 5 April 2021 at 20:01:46 UTC, Per Nordlöw wrote:

 On Wednesday, 31 March 2021 at 07:27:18 UTC, Jacob Carlborg 
 wrote:
 way to interact with an Optional type

 is not to use pattern matching but to use algorithms like 
 `map`, `each` and so on.


 Interesting. Can you show some pseudocode of this? Can those 
 algorithms be implemented as non-member functions?


The `optional` package on dub [1] has some examples in its 
documentation.

[1] https://code.dlang.org/packages/optional
Apr 05 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 5 April 2021 at 20:11:07 UTC, Paul Backus wrote:

 The `optional` package on dub [1] has some examples in its 
 documentation.


Thanks. What about adding these algorithms to std.sumtype?
Apr 05 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 5 April 2021 at 20:33:44 UTC, Per Nordlöw wrote:

 On Monday, 5 April 2021 at 20:11:07 UTC, Paul Backus wrote:

 The `optional` package on dub [1] has some examples in its 
 documentation.


 Thanks. What about adding these algorithms to std.sumtype?


Optional implements the standard range interface, so the 
algorithms it uses are the existing ones in std.algorithm.
Apr 05 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 5 April 2021 at 20:38:33 UTC, Paul Backus wrote:

 Optional implements the standard range interface, so the 
 algorithms it uses are the existing ones in std.algorithm.


Ah, of course. Are you planning on adding Optional to Phobos 
aswell?
Apr 05 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 5 April 2021 at 20:46:07 UTC, Per Nordlöw wrote:

 On Monday, 5 April 2021 at 20:38:33 UTC, Paul Backus wrote:

 Optional implements the standard range interface, so the 
 algorithms it uses are the existing ones in std.algorithm.


 Ah, of course. Are you planning on adding Optional to Phobos 
 aswell?


No, because it's not my package, it's Ali Akhtarzada's.
Apr 05 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 5 April 2021 at 21:43:30 UTC, Paul Backus wrote:

 No, because it's not my package, it's Ali Akhtarzada's.


Is there a place for it in Phobos?
Apr 05 2021
parent Paul Backus <snarwin gmail.com> writes:
On Monday, 5 April 2021 at 22:21:39 UTC, Per Nordlöw wrote:

 On Monday, 5 April 2021 at 21:43:30 UTC, Paul Backus wrote:

 No, because it's not my package, it's Ali Akhtarzada's.


 Is there a place for it in Phobos?


I think there's a place for something like it in Phobos. Whether 
that's `Optional` itself, an improved version of 
`std.typecons.Nullable`, or something else entirely is an open 
question. And of course, Atila has the final say.
Apr 05 2021
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2021-04-05 22:01, Per Nordlöw wrote:


 Interesting. Can you show some pseudocode of this? Can those algorithms 
 be implemented as non-member functions?


Here's an article about idiomatic Scala and optional types [1]. I think 
it applies to D as well.

[1] 
https://www.originate.com/thinking/idiomatic-scala-your-options-do-not-match

-- 
/Jacob Carlborg
Apr 07 2021