www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Optional and orElse: design feedback/critique?

reply aliak <something something.com> writes:
Hi,

Can I ask for feedback on what people expect an optional/maybe 
type to have, and how to behave. There have been a number of 
discussions on the forums about it so far, and attempted PRs in 
phobos. And I currently maintain one that I've been using 
in-house very happily (backend server with vibe-d). I want to 
nail it down further and get to a version 1.0.0 so feedback and 
comments would be highly appreciated.

Optional semantics:
===================

So first, the semantics of Optional!T (these may or may not 
already be implemented):
* Is at least a forward range
* Equality with itself, T, and sentinel-type "none", which checks 
if optional is empty or not
* Assignment to T, `none`, or Optional!T
* Includes type constructors some(x) and no!T
* Forwards operators to T if T implements those operators
   E.g. some(1)++ and no!int++ both work
   E.g. If T has an opCall(...), then some(x)(...) and no!T()(...) 
both work
* Null pointers are valid objects, so some!(int*)(null) is non 
empty
* Null classses and interfaces are empty, so some!Class(null) is 
empty

Optional utilities:
===================

Two utilities are included. Optional chaining and a match 
function. Optional chaining allows you to go through an object's 
hierarchy and get an optional back at the end:

oc(obj).property.function()

If obj is none, then the end result is no!T where T is the return 
type of function. Else it has the actual value if the chain goes 
all the way though. This will also work if property is in turn an 
Optional itself. Secondly, the chaining function is also provided 
for NullableT and reference type.

The match function takes two handlers (lambda aliases) and calls 
the correct one based on if the Optional!T is empty or not.

orElse semantics:
=================

orElse will either get the front of a range or the range itself 
depending on the alternative value and also works on reference 
types. So:

Range!int range1, range2;
range1.orElse(range2) // returns range1 if range1 is not empty, 
else range2
range1.orElse(8) // returns front or range1 if non empty, else 8.

In essence, it also coalesces. I was on the fence on this, but 
it's turning out (again in our project) to be very convenient. 
I'm considering doing this for primitive types like int, float, 
or anything that can be cast to bool (while taking things like 
NaN in to account for e.g.)

So orElse works on a number of types and these are each types's 
semantics:

* If T is a reference type, val1.orElse(val2) will return val1 if 
(val1 !is null), else val2
* If T is a range, the example above shows the semantics
* If T is an Optional!U then it is treated separately in order to 
work with const Optional!U. The value returned by orElse is 
determined by the alternative value. If it is a U then 
optional.front will be returned, if it is an Optional!U then an 
optional will be returned. So the same semantics of range.
* If T is Nullable!U, then isNull is checked. If it is false and 
alternative value is a U, then nullable.get is returned. If T is 
another Nullable, then the return type is a Nullable.

I may have forgotten some stuff but I think that's everything.

Thanks in advance!

PS: I realize an obvious first comment is "what about 
Nullable!T". A number of reasons: 1) it doesn't have a range 
interface 2) the reference semantics are not desirable for me and 
writing generic code is awkward 
(https://github.com/aliak00/optional#what-about-stdtypeconsnullable), 3) alias
this, although deprecated, breaks it completely, 4) maintaining a dub package
feels much more productive.
Jul 27 2019
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 Hi,

 Can I ask for feedback on what people expect an optional/maybe 
 type to have, and how to behave. There have been a number of 
 discussions on the forums about it so far, and attempted PRs in 
 phobos. And I currently maintain one that I've been using 
 in-house very happily (backend server with vibe-d). I want to 
 nail it down further and get to a version 1.0.0 so feedback and 
 comments would be highly appreciated.


I think it is an awesome library and I an happy someone is 
spearheading this. Before this I used Optional, but it ain't as 
nice.


 Optional utilities:
 ===================

 Two utilities are included. Optional chaining and a match 
 function. Optional chaining allows you to go through an 
 object's hierarchy and get an optional back at the end:

 oc(obj).property.function()


In Scala you have to map constantly. This is way much better.


 orElse semantics:
 =================

 orElse will either get the front of a range or the range itself 
 depending on the alternative value and also works on reference 
 types.


Nice addition.
Jul 27 2019
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 Hi,

 Can I ask for feedback on what people expect an optional/maybe 
 type to have, and how to behave. There have been a number of 
 discussions on the forums about it so far, and attempted PRs in 
 phobos.


Would be nice to have functionality similar to Optional from java 
8+, such as orElseThrow, orElseGet, ifPresent etc. Also would be 
nice if it worked perfectly with immutable/const data, and when 
Optional itself is immutable/const. Another nice feature would be 
conversion from immutable to const optional, and from 
immutable/const optional to mutable Optional with const payload 
through probably copy constructor and opCast methods.

Best regards,
Alexandru
Jul 27 2019
parent reply aliak <something something.com> writes:
On Saturday, 27 July 2019 at 14:46:55 UTC, Alexandru Ermicioi 
wrote:

 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 Hi,

 Can I ask for feedback on what people expect an optional/maybe 
 type to have, and how to behave. There have been a number of 
 discussions on the forums about it so far, and attempted PRs 
 in phobos.


 Would be nice to have functionality similar to Optional from 
 java 8+, such as orElseThrow, orElseGet, ifPresent etc. Also


A lot of the java stuff is covered with std.algorithm and orElse

auto a = some(3);
auto gotValue = a.orElse(7); // orElseGet
a.each!(a => writeln(a)); // ifPresent

orElseThrow on the other hand would have to be added separately. 
I use that quite often as well but I can't work it in to the 
provided orElse because "throw blah" is not an expression. I have 
another experimental thing where that's available though -> 
https://aliak00.github.io/ddash/ddash/utils/orelse/orElseThrow.html (but that
library is not the most stable for now)

One other way would be to add an expression throw:

off top of me head:

auto throw_(T : Throwable)(lazy T e) { throw e(); }

Then this would work:

a.orElse(throw_(new Exception("")));


 would be nice if it worked perfectly with immutable/const data, 
 and when Optional itself is immutable/const. Another nice


You can currently wrap cost/immutable data. I try to test to 
ensure things work with qualified optionals [0], but it's tricky 
with the const system in D, and with how inout behaves with 
wrapper objects [1, 2]. The other problem here is that const 
ranges are not ranges. So maybe a decay function on an optional 
type?

const a = some(3);
a.decay.map...

[0]: 
https://github.com/aliak00/optional/blob/de223a7d63d346386b0d02d119cf4bdd79366299/tests/optional.d#L11
[1]: https://issues.dlang.org/show_bug.cgi?id=19126
[2]: https://issues.dlang.org/show_bug.cgi?id=19125


 feature would be conversion from immutable to const optional, 
 and from immutable/const optional to mutable Optional with 
 const payload through probably copy constructor and opCast 
 methods.


Hmm. Yes. The latter can probably be handled with the 
"decay-like" function. Isn't the former handled by D anyway? 
immutable is implicitly convertible to const?

immutable a = some(3);
const Optional!int b = a;


 Best regards,
 Alexandru


Thanks for the feedback!
Jul 27 2019
next sibling parent reply Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 18:26 schrieb aliak:

 
 A lot of the java stuff is covered with std.algorithm and orElse
 
 auto a = some(3);
 auto gotValue = a.orElse(7); // orElseGet
 a.each!(a => writeln(a)); // ifPresent


I still think it might be valuable to have separate names for these
things. As mentioned in my other answer, I think that orElse is
overloaded too much. And while sometimes it is really nice to think of
an optional as a range, it is not always straight forwards and in that
case using something like "each" is a bit awkward. It has the benefit of
working in combination with other range algorithms though, so maybe we
should implement "ifPresent" etc. to also work in ranges (in the same
way you did for "orElse", i.e. only using the first element). Another
utility that would be very nice is ifPresentOrElse. It is very similar
to match, but does never return anything and for that reason, the two
template parameters do not need to have the same return type:

```
int x = 0;

void foo(ref int value)
{
    value = -1;
}

some(1).match!(
    value => x += value,
    () => foo(x) // won't compile because the return types differ
);
```

The solution is to make the return types the same, but this prevents us
from using the error notation for lambdas, which is kind of annoying:

```
int x = 0;

void foo(ref int value)
{
    value = -1;
}

some(1).match!(
    (value) {x += value;},
    () => foo(x) // won't compile because the return types differ
);
```

ifPresentOrElse would basically do this for us:

```
void ifPresentOrElse(alias fun1, alias fun2, T)(inout auto ref
Optional!T opt)
{
	opt.match!(
    		(value) { fun1(value); },
    		() { fun2(); }
	);
}

int x = 0;

void foo(ref int value)
{
    value = -1;
}

some(1).ifPresentOrElse!(
    value => x += value,
    () => foo(x) // everything fine now :)
);
```
Jul 27 2019
parent Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 18:58 schrieb Johannes Loher:

 The solution is to make the return types the same, but this prevents us
 from using the error notation for lambdas, which is kind of annoying:


This should read: "[...] prevents us from using the _arrow_ notation [...]".
Jul 27 2019
prev sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 27 July 2019 at 16:26:26 UTC, aliak wrote:

 On Saturday, 27 July 2019 at 14:46:55 UTC, Alexandru Ermicioi 
 wrote:

 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 Hi,

 Can I ask for feedback on what people expect an 
 optional/maybe type to have, and how to behave. There have 
 been a number of discussions on the forums about it so far, 
 and attempted PRs in phobos.


 Would be nice to have functionality similar to Optional from 
 java 8+, such as orElseThrow, orElseGet, ifPresent etc. Also


 A lot of the java stuff is covered with std.algorithm and orElse

 auto a = some(3);
 auto gotValue = a.orElse(7); // orElseGet
 a.each!(a => writeln(a)); // ifPresent


Haven't thought this way. This is something I like in D, 
everytime you discover something hidden in it.


 orElseThrow on the other hand would have to be added 
 separately. I use that quite often as well but I can't work it 
 in to the provided orElse because "throw blah" is not an 
 expression. I have another experimental thing where that's 
 available though -> 
 https://aliak00.github.io/ddash/ddash/utils/orelse/orElseThrow.html (but that
library is not the most stable for now)

 One other way would be to add an expression throw:

 off top of me head:

 auto throw_(T : Throwable)(lazy T e) { throw e(); }

 Then this would work:

 a.orElse(throw_(new Exception("")));


Hmm trying to use orElse for throwing I'd say is kinda over 
engineering. A simple orElseThrow (throw_) on optional itself 
would be best to have, plus autocompletion would work and it is 
easier to discover by newbies to lib itself.


 would be nice if it worked perfectly with immutable/const 
 data, and when Optional itself is immutable/const. Another nice


 You can currently wrap cost/immutable data. I try to test to 
 ensure things work with qualified optionals [0], but it's 
 tricky with the const system in D, and with how inout behaves 
 with wrapper objects [1, 2]. The other problem here is that 
 const ranges are not ranges. So maybe a decay function on an 
 optional type?

 const a = some(3);
 a.decay.map...

 [0]: 
 https://github.com/aliak00/optional/blob/de223a7d63d346386b0d02d119cf4bdd79366299/tests/optional.d#L11
 [1]: https://issues.dlang.org/show_bug.cgi?id=19126
 [2]: https://issues.dlang.org/show_bug.cgi?id=19125


 feature would be conversion from immutable to const optional, 
 and from immutable/const optional to mutable Optional with 
 const payload through probably copy constructor and opCast 
 methods.


 Hmm. Yes. The latter can probably be handled with the 
 "decay-like" function. Isn't the former handled by D anyway? 
 immutable is implicitly convertible to const?

 immutable a = some(3);
 const Optional!int b = a;


Yep immutable and mutable optional is implicitly convertible to 
const, still we'd need copy constructor for const case accepting 
inout. For later opCast should be perfect match, and we'd also 
need copy constructors matching opCast.

Also it is interesting if it is possible to make whole optional 
 safe. Emsi containers for example templatized all functions in 
order to have attribute autodeduction, which is kinda think is 
overkill in my opinion plus it doesn't allow to have nice 
interfaces similar to java collections. Maybe some of optional 
methods could be marked as safe, such as front or empty provided 
they avoid any invocation of overloaded operators in payload 
itself, as they could be  system.

Regards,
Alexandru.
Jul 27 2019
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 * Null pointers are valid objects, so some!(int*)(null) is non 
 empty
 * Null classses and interfaces are empty, so some!Class(null) 
 is empty


I think it would be better to avoid weird special cases like 
this. The optional package already includes a NotNull type, so 
users can write `Optional!(NotNull!Class)` to opt into your 
proposed semantics explicitly.


 orElse semantics:
 =================

 [...]

 So orElse works on a number of types and these are each types's 
 semantics:

 * If T is a reference type, val1.orElse(val2) will return val1 
 if (val1 !is null), else val2
 * If T is a range, the example above shows the semantics
 * If T is an Optional!U then it is treated separately in order 
 to work with const Optional!U. The value returned by orElse is 
 determined by the alternative value. If it is a U then 
 optional.front will be returned, if it is an Optional!U then an 
 optional will be returned. So the same semantics of range.
 * If T is Nullable!U, then isNull is checked. If it is false 
 and alternative value is a U, then nullable.get is returned. If 
 T is another Nullable, then the return type is a Nullable.


My first impression, upon reading this, is that I'm going to have 
to refer to the documentation every time I use `orElse` in order 
to be sure I'm doing it right. To me, this is a smell: if it 
takes me this long to describe what ought to be a very simple 
utility function, I can tell I've made it too complicated.

As far as concrete advice: the overloads for reference types and 
ranges seem completely unrelated to Optional, so it seems like an 
odd choice to include them in the "optional" package. The 
overloads for Optional and Nullable make sense to include (though 
the Nullable one really *ought* to be in Phobos), but 
special-casing them to wrap the return value is too "magic" for 
my tastes. Much better to just let people write 
`some(val1.orElse(val2))` if that's what they want--it's only a 
few more characters, and it makes it obvious at a glance what the 
code is doing.
Jul 27 2019
next sibling parent reply aliak <something something.com> writes:
On Saturday, 27 July 2019 at 15:07:56 UTC, Paul Backus wrote:

 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 * Null pointers are valid objects, so some!(int*)(null) is non 
 empty
 * Null classses and interfaces are empty, so some!Class(null) 
 is empty


 I think it would be better to avoid weird special cases like 
 this. The optional package already includes a NotNull type, so 
 users can write `Optional!(NotNull!Class)` to opt into your 
 proposed semantics explicitly.


Egad! I just (i.e. today/yesterday) removed NotNull as I could 
not figure out how to guarantee NotNull semantics without it just 
being awkward to use. I'm not sure it's generally used either. 
But that's based on what I've been using myself, and issues and 
prs in the repo - so not a very wide view.

But maybe you're right in making it the same. I'm a bit weary 
because null classes inside optional values has bitten me and 
colleagues multiple times in Swift, Scala, and Kotlin over the 
past few years. Swift got rid of the issue in version 2 or 
something. Scala and Kotlin still have it because of necessary 
Java interop.


 orElse semantics:
 =================

 [...]

 So orElse works on a number of types and these are each 
 types's semantics:

 * If T is a reference type, val1.orElse(val2) will return val1 
 if (val1 !is null), else val2
 * If T is a range, the example above shows the semantics
 * If T is an Optional!U then it is treated separately in order 
 to work with const Optional!U. The value returned by orElse is 
 determined by the alternative value. If it is a U then 
 optional.front will be returned, if it is an Optional!U then 
 an optional will be returned. So the same semantics of range.
 * If T is Nullable!U, then isNull is checked. If it is false 
 and alternative value is a U, then nullable.get is returned. 
 If T is another Nullable, then the return type is a Nullable.


 My first impression, upon reading this, is that I'm going to 
 have to refer to the documentation every time I use `orElse` in 
 order to be sure I'm doing it right. To me, this is a smell: if 
 it takes me this long to describe what ought to be a very 
 simple utility function, I can tell I've made it too 
 complicated.


Very fair point. What if the description was simplified?

The thing is I want it to "just work" based on the call site. So 
if a user does a.orElse(literal) on a container type, it's pretty 
obvious what they want. If a user does a.orElse(container), it's 
again (I think) obvious what they want. But essentially those are 
the only two cases.  And since reference types are not 
containers, then only the second case applies. Does that make it 
simpler?

So make coalescing separate from orElse-ing is basically the gist 
of this right?


 As far as concrete advice: the overloads for reference types 
 and ranges seem completely unrelated to Optional, so it seems 
 like an odd choice to include them in the "optional" package. 
 The overloads for Optional and Nullable make sense to include 
 (though the Nullable one really *ought* to be in Phobos), but 
 special-casing them to wrap the return value is too "magic" for


This is true. They are kinda unrelated and just there for 
convenience because I didn't know where else to get the 
functionality from. I actually played around with having the 
orelse in a completely separate package, and providing a hook:

auto ref orElse(alias elseValue, T)(auto ref T value) {
   static if (hasMember!(T, "hookOrElse") {
     return value.hookOrElse!elseValue;
   }
   ...
}

And then implementing the hook in the optional package. But this 
bit me: 
https://forum.dlang.org/thread/lddwnnzlktszwspldxqu forum.dlang.org


 my tastes. Much better to just let people write 
 `some(val1.orElse(val2))` if that's what they want--it's only a 
 few more characters, and it makes it obvious at a glance what 
 the code is doing.


Sorry, what was the `some(val1.orElse(val2))`? Did you mean that 
instead of range1.orElse(range2) or range1.orElse(elementOfRange)?

And thanks for the feedback!
Jul 27 2019
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 July 2019 at 16:39:21 UTC, aliak wrote:

 Egad! I just (i.e. today/yesterday) removed NotNull as I could 
 not figure out how to guarantee NotNull semantics without it 
 just being awkward to use. I'm not sure it's generally used 
 either. But that's based on what I've been using myself, and 
 issues and prs in the repo - so not a very wide view.

 But maybe you're right in making it the same. I'm a bit weary 
 because null classes inside optional values has bitten me and 
 colleagues multiple times in Swift, Scala, and Kotlin over the 
 past few years. Swift got rid of the issue in version 2 or 
 something. Scala and Kotlin still have it because of necessary 
 Java interop.


As suggested elsewhere in this thread, treating both null 
references and null pointers as empty would also be a reasonable 
decision. The inconsistency is the real issue here.


 The thing is I want it to "just work" based on the call site. 
 So if a user does a.orElse(literal) on a container type, it's 
 pretty obvious what they want. If a user does 
 a.orElse(container), it's again (I think) obvious what they 
 want. But essentially those are the only two cases.  And since 
 reference types are not containers, then only the second case 
 applies. Does that make it simpler?


To me, it's obvious that the expression `a.orElse(b)` should 
return either a or b. If I ask the computer to choose between two 
alternatives for me, and instead it gives me a third thing that I 
didn't ask for (e.g., `a.front`), I am going to be very confused.

For Optional, there's a one-to-one correspondence between the 
container and the value inside it, so eliding the distinction 
isn't a big deal (though strictly speaking, `valueOrElse` would 
be a more accurate name). On the other hand, I'd definitely be 
scratching my head if I saw something like `[1, 2, 3].orElse(4)`. 
An array, or else a single int? How does that even type-check? 
Compare to `[1, 2, 3].frontOrElse(4)`, which is so obvious that 
it needs no explanation.

I understand the allure of this kind of "just works", "do what I 
mean" design, but following that philosophy is what gives us 
features like autodecoding--mostly harmless, but incredibly 
frustrating when they get in your way. After all, when someone 
iterates over a range of UTF-8 code units, it's "pretty obvious" 
that what they want is code points, right?


 So make coalescing separate from orElse-ing is basically the 
 gist of this right?


As I understand it, coalescing means flattening or unwrapping 
multiple layers of Optional at once. So, yes, orElse definitely 
shouldn't be doing that. But I'm guessing that's not what you 
meant to say.


 Sorry, what was the `some(val1.orElse(val2))`? Did you mean 
 that instead of range1.orElse(range2) or 
 range1.orElse(elementOfRange)?


Sorry, that was an incorrect example. What I was trying to say 
was, I don't think Optional.orElse should have two different 
return types depending on its arguments. If you want to have a 
version that unwraps its first argument and a version that 
doesn't, it would be better to give them different names, so it's 
always obvious which is being used. For example:

Optional!int a = no!int;
Optional!int b = some(123);

int x = a.valueOrElse(123);   // unwraps
Optional!int y = a.orElse(b); // doesn't unwrap

This is how they do it in Rust: the non-unwrapping version is 
called `or`, and the unwrapping version is called `unwrap_or`.
Jul 27 2019
parent aliak <something something.com> writes:
On Sunday, 28 July 2019 at 00:05:19 UTC, Paul Backus wrote:

 As suggested elsewhere in this thread, treating both null 
 references and null pointers as empty would also be a 
 reasonable decision. The inconsistency is the real issue here.


Aye, I agree.


 The thing is I want it to "just work" based on the call site. 
 So if a user does a.orElse(literal) on a container type, it's




[...]

 or me, and instead it gives me a third thing that I didn't ask 
 for (e.g., `a.front`), I am going to be very confused.


Ha! Touche. I am convinced.


 For Optional, there's a one-to-one correspondence between the 
 container and the value inside it, so eliding the distinction


[...]

g when they get in your way. After all, when someone
 iterates over a range of UTF-8 code units, it's "pretty 
 obvious" that what they want is code points, right?


 So make coalescing separate from orElse-ing is basically the 
 gist of this right?


 As I understand it, coalescing means flattening or unwrapping 
 multiple layers of Optional at once. So, yes, orElse definitely 
 shouldn't be doing that. But I'm guessing that's not what you 
 meant to say.


I was thinking in terms of the null coalescing operator found in 
some languages. So a ?? b will give you a if it's not null and b 
otherwise, which is what orElse was doing if b was a 
range/optional.


 Sorry, what was the `some(val1.orElse(val2))`? Did you mean 
 that instead of range1.orElse(range2) or 
 range1.orElse(elementOfRange)?


 Sorry, that was an incorrect example. What I was trying to say 
 was, I don't think Optional.orElse should have two different 
 return types depending on its arguments. If you want to have a 
 version that unwraps its first argument and a version that 
 doesn't, it would be better to give them different names, so 
 it's always obvious which is being used. For example:

 Optional!int a = no!int;
 Optional!int b = some(123);

 int x = a.valueOrElse(123);   // unwraps
 Optional!int y = a.orElse(b); // doesn't unwrap

 This is how they do it in Rust: the non-unwrapping version is 
 called `or`, and the unwrapping version is called `unwrap_or`.


Thanks for the feedback. It looks like I'll be going with 
frontOrElse, orElse, and making the null semantics consistent.
Jul 27 2019
prev sibling parent reply SimonN <eiderdaus gmail.com> writes:
On Saturday, 27 July 2019 at 15:07:56 UTC, Paul Backus wrote:

 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 * Null pointers are valid objects, so some!(int*)(null) is non 
 empty
 * Null classses and interfaces are empty, so some!Class(null) 
 is empty


 I think it would be better to avoid weird special cases like 
 this. The optional package already includes a NotNull type, so 
 users can write `Optional!(NotNull!Class)` to opt into your 
 proposed semantics explicitly.


It's surprising that pointers and references are special-cased 
differently, right.

But I suggest that null is the same as empty, both for pointers 
and for references. I.e., the original post's proposal should 
change its pointer semantics, some!(int*)(null) == no!(int*).

What was the original reason to treat pointers and class 
references differently?

Reason to treat null as empty, always: We want at most one layer 
of nullability/emptiness, and usually no nullability at all, 
because we want to avoid nesting of "if it exists, do stuff, 
otherwise do nothing". All possible nullability/emptiness should 
be handled at once.

This usual case (at most one layer of nullability/emptiness) 
should be quickest to write. If Optional!Class introduced 
emptiness different from nullability, then 
Optional!(NotNull!Class) would be standard usage because people 
want at most one layer of emptiness, but then standard usage 
would not be succinct.

I accept that Optional!(NotNull!Class) is more explicit, but this 
is only because of D's lack of non-nullable references, which is 
the problem in the first place. NotNull is an abstraction 
inversion: It builds a non-nullable type (the less complex type 
because it allows fewer values and is crash-free to call methods) 
from a nullable type (which instead should be a sum type of the 
nonexisting non-nullable type and empty).

I already use Optional!Class for single layer of nullability, and 
Class for non-null by convention. 95 % of the time, I want 
non-nullable. Exported methods, then, need in/out contracts to 
check for non-nullability. These contracts should ideally be 
inserted by the compiler on seeing a language-builtin 
non-nullable reference.

-- Simon
Jul 27 2019
parent aliak <something something.com> writes:
On Saturday, 27 July 2019 at 17:01:14 UTC, SimonN wrote:

 It's surprising that pointers and references are special-cased 
 differently, right.

 But I suggest that null is the same as empty, both for pointers 
 and for references. I.e., the original post's proposal should 
 change its pointer semantics, some!(int*)(null) == no!(int*).

 What was the original reason to treat pointers and class 
 references differently?


I only have a fuzzy recollection of something along the lines of: 
"null can be a valid return type of a function, and maybe there's 
an error and I want to return nothing". But, I can see how that 
thought is a bit... unconvincing right now.

So yeah, bottom line, I'll fix the inconsistency as you suggest 
as I think that sounds more reasonable then what it currently is.

Chur!
Jul 27 2019
prev sibling next sibling parent reply Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 15:17 schrieb aliak:

 Hi,
 
 Can I ask for feedback on what people expect an optional/maybe type to
 have, and how to behave. There have been a number of discussions on the
 forums about it so far, and attempted PRs in phobos. And I currently
 maintain one that I've been using in-house very happily (backend server
 with vibe-d). I want to nail it down further and get to a version 1.0.0
 so feedback and comments would be highly appreciated.
 
 Optional semantics:
 ===================
 
 So first, the semantics of Optional!T (these may or may not already be
 implemented):
 * Is at least a forward range
 * Equality with itself, T, and sentinel-type "none", which checks if
 optional is empty or not
 * Assignment to T, `none`, or Optional!T
 * Includes type constructors some(x) and no!T
 * Forwards operators to T if T implements those operators
   E.g. some(1)++ and no!int++ both work
   E.g. If T has an opCall(...), then some(x)(...) and no!T()(...) both work
 * Null pointers are valid objects, so some!(int*)(null) is non empty
 * Null classses and interfaces are empty, so some!Class(null) is empty
 
 Optional utilities:
 ===================
 
 Two utilities are included. Optional chaining and a match function.
 Optional chaining allows you to go through an object's hierarchy and get
 an optional back at the end:
 
 oc(obj).property.function()
 
 If obj is none, then the end result is no!T where T is the return type
 of function. Else it has the actual value if the chain goes all the way
 though. This will also work if property is in turn an Optional itself.
 Secondly, the chaining function is also provided for NullableT and
 reference type.
 
 The match function takes two handlers (lambda aliases) and calls the
 correct one based on if the Optional!T is empty or not.
 
 orElse semantics:
 =================
 
 orElse will either get the front of a range or the range itself
 depending on the alternative value and also works on reference types. So:
 
 Range!int range1, range2;
 range1.orElse(range2) // returns range1 if range1 is not empty, else range2
 range1.orElse(8) // returns front or range1 if non empty, else 8.
 
 In essence, it also coalesces. I was on the fence on this, but it's
 turning out (again in our project) to be very convenient. I'm
 considering doing this for primitive types like int, float, or anything
 that can be cast to bool (while taking things like NaN in to account for
 e.g.)
 
 So orElse works on a number of types and these are each types's semantics:
 
 * If T is a reference type, val1.orElse(val2) will return val1 if (val1
 !is null), else val2
 * If T is a range, the example above shows the semantics
 * If T is an Optional!U then it is treated separately in order to work
 with const Optional!U. The value returned by orElse is determined by the
 alternative value. If it is a U then optional.front will be returned, if
 it is an Optional!U then an optional will be returned. So the same
 semantics of range.
 * If T is Nullable!U, then isNull is checked. If it is false and
 alternative value is a U, then nullable.get is returned. If T is another
 Nullable, then the return type is a Nullable.
 
 I may have forgotten some stuff but I think that's everything.
 
 Thanks in advance!
 
 PS: I realize an obvious first comment is "what about Nullable!T". A
 number of reasons: 1) it doesn't have a range interface 2) the reference
 semantics are not desirable for me and writing generic code is awkward
 (https://github.com/aliak00/optional#what-about-stdtypeconsnullable), 3)
 alias this, although deprecated, breaks it completely, 4) maintaining a
 dub package feels much more productive.


orElse currently has an overload that takes a callable that returns a
fallback value as template parameter. I think something like this is
needed, but it might be better to give it a different name (in Java,
Scala etc. this is called orElseGet). Other convenience utilities might
be nice, e.g. ifPresent, ifPresentOrElse, orElseThrow etc. (take Java's
Optional or Scala's Option as reference). These are all very easy to
implement and I have already done so several times when using your library.

I also think that your current orElse semantics are a bit too
complicated: There is simply too much stuff (with slighty differing
behavior) in that one single function (I know, it is several overloads,
but the user does not care). E.g. I think the overload that takes
another range (or Optional or Nullable) should be a seperate function
because it actually has different semantics. All overloads should have
the same general behavior, in my opinion. In Java, this function is
called "or".

Otherwise, I actually quite like the semantics of your library (and I
use it regularly). Thanks for your work!
Jul 27 2019
parent reply aliak <something something.com> writes:
On Saturday, 27 July 2019 at 16:27:53 UTC, Johannes Loher wrote:

 orElse currently has an overload that takes a callable that 
 returns a
 fallback value as template parameter. I think something like 
 this is
 needed, but it might be better to give it a different name (in 
 Java,
 Scala etc. this is called orElseGet). Other convenience 
 utilities might
 be nice, e.g. ifPresent, ifPresentOrElse, orElseThrow etc. 
 (take Java's
 Optional or Scala's Option as reference). These are all very 
 easy to
 implement and I have already done so several times when using 
 your library.


How would you find using .each over ifPresent? As for 
ifPresentOrElse, would this suffice?:

no!int.orElse!(() => log("nothing there"));

Would you prefer an explicit orElseThrow or a throw_ expression 
as proposed in a previous post?


 I also think that your current orElse semantics are a bit too 
 complicated: There is simply too much stuff (with slighty 
 differing behavior) in that one single function (I know, it is 
 several overloads, but the user does not care). E.g. I think 
 the overload that takes another range (or Optional or Nullable) 
 should be a seperate function because it actually has different 
 semantics. All overloads should have the same general behavior, 
 in my opinion. In Java, this function is called "or".


Alright, more for separation of orElse's functions :) I must say 
that I quite like the coalesce functionality. But I think maybe 
you and Paul might be right to separate that and make it a 
different function. Or just not do the coalescing and the user 
can be a bit more explicit as I think Paul suggested.


 Otherwise, I actually quite like the semantics of your library 
 (and I use it regularly). Thanks for your work!


You're welcome!
Jul 27 2019
parent reply Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 19:15 schrieb aliak:

 
 How would you find using .each over ifPresent?


See my other answer. Though I don't care _that_ much. "each" might be a
bit awkward at times, but maybe it is good enough.

 As for ifPresentOrElse, would this suffice?:
 
 no!int.orElse!(() => log("nothing there"));


I don't see how that has anything to do with ifPresentOrElse. Could you
please explain? In my other answer I also wrote a bit about why
ifPresentOrElse would be nice (and why match does not completely replace
it).


 Would you prefer an explicit orElseThrow or a throw_ expression as
 proposed in a previous post?


I don't actually care that much, but I think I slightly prefer
orElseThrow (probably because I'm used to it from Java). This is what I
have been doing in the past:

```
no!int.orElse!(function int() { throw new Exception(""); });
```

It is slightly annoying that you have to write the return type
explictly, but it's not that bad. I think I also wrote my own orElseThrow:

```
import optional;

auto orElseThrow(alias fun, T)(auto ref Optional!T opt)
{
    return opt.orElse!(delegate T() { throw fun(); });
}

void main()
{
    no!int.orElseThrow!(() => new Exception(""));
}
```
(you could make some improvements by checking (with static ifs) if a
delegate is actually needed, or a function suffices)


 Alright, more for separation of orElse's functions :) I must say that I
 quite like the coalesce functionality. But I think maybe you and Paul
 might be right to separate that and make it a different function. Or
 just not do the coalescing and the user can be a bit more explicit as I
 think Paul suggested.


Most of the time, I don't actually have a need for the overloads that
take a range, optional, nullable, so I'd be kind of OK with simply
removing it. I'd also totally be OK with it being a separate function.

What Paul suggested is not a good solution imo because it is besides the
point. Wrapping with optional after calling orElse is something totally
different than providing another optional as fallback, because in the
first case, you need to know that the fallback value is actually present
beforehand (in which case you should simply use the regular orElse
anyways). The value that the additional orElse overload currently
provides is that you can provide another fallback value of which you do
not know if it present as well and you can also chain this in a neat way:

/* ... */
enum fallback = 0;
return no!int
    .orElse(dbRequestThatMightOrMightNotReturnSomething())
    .orElse(httpRequestThatMightOrMightNotReturnSomething())
    .orElse(fallback);
/* ... */

As mentioned above, I do not use this that often, but I think it is
still a valid usecase and Pauls suggestion is not a substitute for it.
Jul 27 2019
next sibling parent Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 19:59 schrieb Johannes Loher:

 It is slightly annoying that you have to write the return type
 explictly, but it's not that bad. I think I also wrote my own orElseThrow:
 
 ```
 import optional;
 
 auto orElseThrow(alias fun, T)(auto ref Optional!T opt)
 {
     return opt.orElse!(delegate T() { throw fun(); });
 }
 
 void main()
 {
     no!int.orElseThrow!(() => new Exception(""));
 }
 ```
 (you could make some improvements by checking (with static ifs) if a
 delegate is actually needed, or a function suffices)


By the way, in this situation it would be kind of useful to have a
bottom type or an (inferred)  noreturn annotation. Then in addition
callables that return something of type T, you could explictily allow
passing in callables that never return.
Jul 27 2019
prev sibling next sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 27 July 2019 at 17:59:19 UTC, Johannes Loher wrote:

 Am 27.07.19 um 19:15 schrieb aliak:



 ```
 import optional;

 auto orElseThrow(alias fun, T)(auto ref Optional!T opt)
 {
     return opt.orElse!(delegate T() { throw fun(); });
 }

 void main()
 {
     no!int.orElseThrow!(() => new Exception(""));
 }
 ```



Lazy arguments could fit perfectly for orElseThrow statement:

```D
import optional;

auto orElseThrow(T)(auto ref Optional!T opt, lazy Exception 
throwable)
{
     if (opt.empty) {
         throw throwable;
     }

     return opt.front;
}

void main()
{
     no!int.orElseThrow(new Exception("")); // maybe instead of 
no, just optional!int? using just negation is kinda ambiguous.
}
```
Jul 27 2019
prev sibling parent aliak <something something.com> writes:
On Saturday, 27 July 2019 at 17:59:19 UTC, Johannes Loher wrote:

 Am 27.07.19 um 19:15 schrieb aliak:

 
 How would you find using .each over ifPresent?


 See my other answer. Though I don't care _that_ much. "each" 
 might be a bit awkward at times, but maybe it is good enough.

 As for ifPresentOrElse, would this suffice?:
 
 no!int.orElse!(() => log("nothing there"));


 I don't see how that has anything to do with ifPresentOrElse. 
 Could you please explain? In my other answer I also wrote a bit 
 about why ifPresentOrElse would be nice (and why match does not 
 completely replace it).


Sorry, I misunderstood. So basically:

if (a.empty)
   fun();
else
   value += 1;

So Scala provides this with match out of the box by promoting 
(demoting?) the types to Any, and indeed the current match can be 
made to do the "same" (if the return types are incompatible, then 
there's no return, i.e. void). Should work no? If you mess up 
types the compiler will shout so it sounds, sound.

Btw, this reminds me of an optional vs matching cheat sheet that 
I was pointed to some time ago and thought it was great! -> 
https://www.originate.com/thinking/stories/idiomatic-scala-your-options-do-not-match/


 Would you prefer an explicit orElseThrow or a throw_ 
 expression as proposed in a previous post?


 I don't actually care that much, but I think I slightly prefer 
 orElseThrow (probably because I'm used to it from Java). This 
 is what I have been doing in the past:

 ```
 no!int.orElse!(function int() { throw new Exception(""); });
 ```

 It is slightly annoying that you have to write the return type
 explictly, but it's not that bad. I think I also wrote my own 
 orElseThrow:

 ```
 import optional;

 auto orElseThrow(alias fun, T)(auto ref Optional!T opt)
 {
     return opt.orElse!(delegate T() { throw fun(); });
 }

 void main()
 {
     no!int.orElseThrow!(() => new Exception(""));
 }
 ```
 (you could make some improvements by checking (with static ifs) 
 if a
 delegate is actually needed, or a function suffices)


Yes. I've done this as well. So I think I'll add this in.


 Alright, more for separation of orElse's functions :) I must 
 say that I quite like the coalesce functionality. But I think 
 maybe you and Paul might be right to separate that and make it 
 a different function. Or just not do the coalescing and the 
 user can be a bit more explicit as I think Paul suggested.


 Most of the time, I don't actually have a need for the 
 overloads that take a range, optional, nullable, so I'd be kind 
 of OK with simply removing it. I'd also totally be OK with it 
 being a separate function.


👌


 What Paul suggested is not a good solution imo because it is 
 besides the point. Wrapping with optional after calling orElse 
 is something totally different than providing another optional 
 as fallback, because in the first case, you need to know that 
 the fallback value is actually present beforehand (in which 
 case you should simply use the regular orElse anyways). The 
 value that the additional orElse overload currently provides is 
 that you can provide another fallback value of which you do not 
 know if it present as well and you can also chain this in a 
 neat way:

 /* ... */
 enum fallback = 0;
 return no!int
     .orElse(dbRequestThatMightOrMightNotReturnSomething())
     .orElse(httpRequestThatMightOrMightNotReturnSomething())
     .orElse(fallback);
 /* ... */

 As mentioned above, I do not use this that often, but I think 
 it is still a valid usecase and Pauls suggestion is not a 
 substitute for it.


Yes, I agree that's a nice use-case (which I use). I think I'll 
probably go with frontOrElse and orElse to separate the 
functionality. Something is screaming flatMap to me right now but 
I just can't put my finger on it ;)
Jul 27 2019
prev sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 PS: I realize an obvious first comment is "what about 
 Nullable!T". A number of reasons: 1) it doesn't have a range 
 interface 2) the reference semantics are not desirable for me 
 and writing generic code is awkward 
 (https://github.com/aliak00/optional#what-about-stdtypeconsnullable), 3) alias
this, although deprecated, breaks it completely, 4) maintaining a dub package
feels much more productive.Skimmed a bit through main optional module, and have
some questions/suggestions.


Just wondering whats the reason of treating not null as being a 
value? Per my knowledge the meaning of null basically translates 
to "no value", does it not make sense to have null be treated as 
empty optional, and consequently any null pointer?


Saw you're using from pattern to import symbols, wouldn't it make 
a template bloat and increase compile time given big projects?

As for aliasing what is the problem with it? Would not it be 
better than introspecting and defining operator overloads in 
Optional? Existing design would introduce overhead on compilation 
time due to introspection of payload.

I see you can assign value to optional payload wouldn't it be 
better to have Optional contents pseudo immutable, and have any 
changes defined through copies of it?

Regarding toString, a sink version would be nice to have as it 
allows nogc use of it.

Json conversion could be outside of Optional in a separate module 
or even project, since some people might want serialization not 
only to vibe d.

If we'll treat null values as empty then won't be any need in 
none type, and optional == null would be enough.

`some` for constructor of nullable is too generic name, maybe 
rename it to optional/asOptional, it will make more sense and be 
consistent with toOptional present in project.

toNullable could be moved as opCast maybe.

Best regards,
Alexandru.
Jul 27 2019
parent aliak <something something.com> writes:
On Saturday, 27 July 2019 at 19:00:13 UTC, Alexandru Ermicioi 
wrote:

 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:

 PS: I realize an obvious first comment is "what about 
 Nullable!T". A number of reasons: 1) it doesn't have a range 
 interface 2) the reference semantics are not desirable for me 
 and writing generic code is awkward 
 (https://github.com/aliak00/optional#what-about-stdtypeconsnullable), 3) alias
this, although deprecated, breaks it completely, 4) maintaining a dub package
feels much more productive.Skimmed a bit through main optional module, and have
some questions/suggestions.


 Just wondering whats the reason of treating not null as being a 
 value? Per my knowledge the meaning of null basically 
 translates to "no value", does it not make sense to have null 
 be treated as empty optional, and consequently any null pointer?


I think you're right here. Will be changing this.


 Saw you're using from pattern to import symbols, wouldn't it 
 make a template bloat and increase compile time given big 
 projects?


I would assume so. Maybe. It also at the same time reduces 
compile time by lazily importing modules only if needed, i.e. 
when instantiations happen. Though, since you brought it to 
attention, I do see that it's being used in some places that 
don't make any sense (at declaration scope of the Optional for 
e.g.) So thanks for that!


 As for aliasing what is the problem with it? Would not it be 
 better than introspecting and defining operator overloads in 
 Optional? Existing design would introduce overhead on 
 compilation time due to introspection of payload.


I could move the operator introspection to the optional chain. I 
don't know how much overhead on compilation time this will gain. 
Have you had experience with this and do you think it's 
significant?


 I see you can assign value to optional payload wouldn't it be 
 better to have Optional contents pseudo immutable, and have any 
 changes defined through copies of it?


I guess so? Hmm... this one I'll have to sit on for a while


 Regarding toString, a sink version would be nice to have as it 
 allows nogc use of it.


Agreed!


 Json conversion could be outside of Optional in a separate 
 module or even project, since some people might want 
 serialization not only to vibe d.


For vibe-d it has to be inside unfortunately. But it's only 
compiled in if you're using vibe-d so I think that should be ok.


 If we'll treat null values as empty then won't be any need in 
 none type, and optional == null would be enough.


Indeed. That might be hard with opAssign present and T being null 
assignable.


 `some` for constructor of nullable is too generic name, maybe 
 rename it to optional/asOptional, it will make more sense and 
 be consistent with toOptional present in project.


indeed, also from your other post, maybe some! and no! can just 
be both replaced with optional! ... I always just liked the 
wording though.


 toNullable could be moved as opCast maybe.


It could. If I overload an opX explicitly for the Optional type, 
then I'll have to move all the forwarding opXs to somewhere else 
as well. Might be the way to go.


 Best regards,
 Alexandru.


Thanks for all the comments. It sounds like you actually went 
through the code! Much appreciated.
Jul 27 2019