www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Optional and orElse: design feedback/critique?

reply aliak <something something.com> writes:
Hi,

Can I ask for feedback on what people expect an optional/maybe 
type to have, and how to behave. There have been a number of 
discussions on the forums about it so far, and attempted PRs in 
phobos. And I currently maintain one that I've been using 
in-house very happily (backend server with vibe-d). I want to 
nail it down further and get to a version 1.0.0 so feedback and 
comments would be highly appreciated.

Optional semantics:
===================

So first, the semantics of Optional!T (these may or may not 
already be implemented):
* Is at least a forward range
* Equality with itself, T, and sentinel-type "none", which checks 
if optional is empty or not
* Assignment to T, `none`, or Optional!T
* Includes type constructors some(x) and no!T
* Forwards operators to T if T implements those operators
   E.g. some(1)++ and no!int++ both work
   E.g. If T has an opCall(...), then some(x)(...) and no!T()(...) 
both work
* Null pointers are valid objects, so some!(int*)(null) is non 
empty
* Null classses and interfaces are empty, so some!Class(null) is 
empty

Optional utilities:
===================

Two utilities are included. Optional chaining and a match 
function. Optional chaining allows you to go through an object's 
hierarchy and get an optional back at the end:

oc(obj).property.function()

If obj is none, then the end result is no!T where T is the return 
type of function. Else it has the actual value if the chain goes 
all the way though. This will also work if property is in turn an 
Optional itself. Secondly, the chaining function is also provided 
for NullableT and reference type.

The match function takes two handlers (lambda aliases) and calls 
the correct one based on if the Optional!T is empty or not.

orElse semantics:
=================

orElse will either get the front of a range or the range itself 
depending on the alternative value and also works on reference 
types. So:

Range!int range1, range2;
range1.orElse(range2) // returns range1 if range1 is not empty, 
else range2
range1.orElse(8) // returns front or range1 if non empty, else 8.

In essence, it also coalesces. I was on the fence on this, but 
it's turning out (again in our project) to be very convenient. 
I'm considering doing this for primitive types like int, float, 
or anything that can be cast to bool (while taking things like 
NaN in to account for e.g.)

So orElse works on a number of types and these are each types's 
semantics:

* If T is a reference type, val1.orElse(val2) will return val1 if 
(val1 !is null), else val2
* If T is a range, the example above shows the semantics
* If T is an Optional!U then it is treated separately in order to 
work with const Optional!U. The value returned by orElse is 
determined by the alternative value. If it is a U then 
optional.front will be returned, if it is an Optional!U then an 
optional will be returned. So the same semantics of range.
* If T is Nullable!U, then isNull is checked. If it is false and 
alternative value is a U, then nullable.get is returned. If T is 
another Nullable, then the return type is a Nullable.

I may have forgotten some stuff but I think that's everything.

Thanks in advance!

PS: I realize an obvious first comment is "what about 
Nullable!T". A number of reasons: 1) it doesn't have a range 
interface 2) the reference semantics are not desirable for me and 
writing generic code is awkward 
(https://github.com/aliak00/optional#what-about-stdtypeconsnullable), 3) alias
this, although deprecated, breaks it completely, 4) maintaining a dub package
feels much more productive.
Jul 27
next sibling parent Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 Hi,

 Can I ask for feedback on what people expect an optional/maybe 
 type to have, and how to behave. There have been a number of 
 discussions on the forums about it so far, and attempted PRs in 
 phobos. And I currently maintain one that I've been using 
 in-house very happily (backend server with vibe-d). I want to 
 nail it down further and get to a version 1.0.0 so feedback and 
 comments would be highly appreciated.
I think it is an awesome library and I an happy someone is spearheading this. Before this I used Optional, but it ain't as nice.
 Optional utilities:
 ===================

 Two utilities are included. Optional chaining and a match 
 function. Optional chaining allows you to go through an 
 object's hierarchy and get an optional back at the end:

 oc(obj).property.function()
In Scala you have to map constantly. This is way much better.
 orElse semantics:
 =================

 orElse will either get the front of a range or the range itself 
 depending on the alternative value and also works on reference 
 types.
Nice addition.
Jul 27
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 Hi,

 Can I ask for feedback on what people expect an optional/maybe 
 type to have, and how to behave. There have been a number of 
 discussions on the forums about it so far, and attempted PRs in 
 phobos.
Would be nice to have functionality similar to Optional from java 8+, such as orElseThrow, orElseGet, ifPresent etc. Also would be nice if it worked perfectly with immutable/const data, and when Optional itself is immutable/const. Another nice feature would be conversion from immutable to const optional, and from immutable/const optional to mutable Optional with const payload through probably copy constructor and opCast methods. Best regards, Alexandru
Jul 27
parent reply aliak <something something.com> writes:
On Saturday, 27 July 2019 at 14:46:55 UTC, Alexandru Ermicioi 
wrote:
 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 Hi,

 Can I ask for feedback on what people expect an optional/maybe 
 type to have, and how to behave. There have been a number of 
 discussions on the forums about it so far, and attempted PRs 
 in phobos.
Would be nice to have functionality similar to Optional from java 8+, such as orElseThrow, orElseGet, ifPresent etc. Also
A lot of the java stuff is covered with std.algorithm and orElse auto a = some(3); auto gotValue = a.orElse(7); // orElseGet a.each!(a => writeln(a)); // ifPresent orElseThrow on the other hand would have to be added separately. I use that quite often as well but I can't work it in to the provided orElse because "throw blah" is not an expression. I have another experimental thing where that's available though -> https://aliak00.github.io/ddash/ddash/utils/orelse/orElseThrow.html (but that library is not the most stable for now) One other way would be to add an expression throw: off top of me head: auto throw_(T : Throwable)(lazy T e) { throw e(); } Then this would work: a.orElse(throw_(new Exception("")));
 would be nice if it worked perfectly with immutable/const data, 
 and when Optional itself is immutable/const. Another nice
You can currently wrap cost/immutable data. I try to test to ensure things work with qualified optionals [0], but it's tricky with the const system in D, and with how inout behaves with wrapper objects [1, 2]. The other problem here is that const ranges are not ranges. So maybe a decay function on an optional type? const a = some(3); a.decay.map... [0]: https://github.com/aliak00/optional/blob/de223a7d63d346386b0d02d119cf4bdd79366299/tests/optional.d#L11 [1]: https://issues.dlang.org/show_bug.cgi?id=19126 [2]: https://issues.dlang.org/show_bug.cgi?id=19125
 feature would be conversion from immutable to const optional, 
 and from immutable/const optional to mutable Optional with 
 const payload through probably copy constructor and opCast 
 methods.
Hmm. Yes. The latter can probably be handled with the "decay-like" function. Isn't the former handled by D anyway? immutable is implicitly convertible to const? immutable a = some(3); const Optional!int b = a;
 Best regards,
 Alexandru
Thanks for the feedback!
Jul 27
next sibling parent reply Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 18:26 schrieb aliak:
 
 A lot of the java stuff is covered with std.algorithm and orElse
 
 auto a = some(3);
 auto gotValue = a.orElse(7); // orElseGet
 a.each!(a => writeln(a)); // ifPresent
I still think it might be valuable to have separate names for these things. As mentioned in my other answer, I think that orElse is overloaded too much. And while sometimes it is really nice to think of an optional as a range, it is not always straight forwards and in that case using something like "each" is a bit awkward. It has the benefit of working in combination with other range algorithms though, so maybe we should implement "ifPresent" etc. to also work in ranges (in the same way you did for "orElse", i.e. only using the first element). Another utility that would be very nice is ifPresentOrElse. It is very similar to match, but does never return anything and for that reason, the two template parameters do not need to have the same return type: ``` int x = 0; void foo(ref int value) { value = -1; } some(1).match!( value => x += value, () => foo(x) // won't compile because the return types differ ); ``` The solution is to make the return types the same, but this prevents us from using the error notation for lambdas, which is kind of annoying: ``` int x = 0; void foo(ref int value) { value = -1; } some(1).match!( (value) {x += value;}, () => foo(x) // won't compile because the return types differ ); ``` ifPresentOrElse would basically do this for us: ``` void ifPresentOrElse(alias fun1, alias fun2, T)(inout auto ref Optional!T opt) { opt.match!( (value) { fun1(value); }, () { fun2(); } ); } int x = 0; void foo(ref int value) { value = -1; } some(1).ifPresentOrElse!( value => x += value, () => foo(x) // everything fine now :) ); ```
Jul 27
parent Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 18:58 schrieb Johannes Loher:
 The solution is to make the return types the same, but this prevents us
 from using the error notation for lambdas, which is kind of annoying:
This should read: "[...] prevents us from using the _arrow_ notation [...]".
Jul 27
prev sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 27 July 2019 at 16:26:26 UTC, aliak wrote:
 On Saturday, 27 July 2019 at 14:46:55 UTC, Alexandru Ermicioi 
 wrote:
 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 Hi,

 Can I ask for feedback on what people expect an 
 optional/maybe type to have, and how to behave. There have 
 been a number of discussions on the forums about it so far, 
 and attempted PRs in phobos.
Would be nice to have functionality similar to Optional from java 8+, such as orElseThrow, orElseGet, ifPresent etc. Also
A lot of the java stuff is covered with std.algorithm and orElse auto a = some(3); auto gotValue = a.orElse(7); // orElseGet a.each!(a => writeln(a)); // ifPresent
Haven't thought this way. This is something I like in D, everytime you discover something hidden in it.
 orElseThrow on the other hand would have to be added 
 separately. I use that quite often as well but I can't work it 
 in to the provided orElse because "throw blah" is not an 
 expression. I have another experimental thing where that's 
 available though -> 
 https://aliak00.github.io/ddash/ddash/utils/orelse/orElseThrow.html (but that
library is not the most stable for now)

 One other way would be to add an expression throw:

 off top of me head:

 auto throw_(T : Throwable)(lazy T e) { throw e(); }

 Then this would work:

 a.orElse(throw_(new Exception("")));
Hmm trying to use orElse for throwing I'd say is kinda over engineering. A simple orElseThrow (throw_) on optional itself would be best to have, plus autocompletion would work and it is easier to discover by newbies to lib itself.
 would be nice if it worked perfectly with immutable/const 
 data, and when Optional itself is immutable/const. Another nice
You can currently wrap cost/immutable data. I try to test to ensure things work with qualified optionals [0], but it's tricky with the const system in D, and with how inout behaves with wrapper objects [1, 2]. The other problem here is that const ranges are not ranges. So maybe a decay function on an optional type? const a = some(3); a.decay.map... [0]: https://github.com/aliak00/optional/blob/de223a7d63d346386b0d02d119cf4bdd79366299/tests/optional.d#L11 [1]: https://issues.dlang.org/show_bug.cgi?id=19126 [2]: https://issues.dlang.org/show_bug.cgi?id=19125
 feature would be conversion from immutable to const optional, 
 and from immutable/const optional to mutable Optional with 
 const payload through probably copy constructor and opCast 
 methods.
Hmm. Yes. The latter can probably be handled with the "decay-like" function. Isn't the former handled by D anyway? immutable is implicitly convertible to const? immutable a = some(3); const Optional!int b = a;
Yep immutable and mutable optional is implicitly convertible to const, still we'd need copy constructor for const case accepting inout. For later opCast should be perfect match, and we'd also need copy constructors matching opCast. Also it is interesting if it is possible to make whole optional safe. Emsi containers for example templatized all functions in order to have attribute autodeduction, which is kinda think is overkill in my opinion plus it doesn't allow to have nice interfaces similar to java collections. Maybe some of optional methods could be marked as safe, such as front or empty provided they avoid any invocation of overloaded operators in payload itself, as they could be system. Regards, Alexandru.
Jul 27
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 * Null pointers are valid objects, so some!(int*)(null) is non 
 empty
 * Null classses and interfaces are empty, so some!Class(null) 
 is empty
I think it would be better to avoid weird special cases like this. The optional package already includes a NotNull type, so users can write `Optional!(NotNull!Class)` to opt into your proposed semantics explicitly.
 orElse semantics:
 =================

 [...]

 So orElse works on a number of types and these are each types's 
 semantics:

 * If T is a reference type, val1.orElse(val2) will return val1 
 if (val1 !is null), else val2
 * If T is a range, the example above shows the semantics
 * If T is an Optional!U then it is treated separately in order 
 to work with const Optional!U. The value returned by orElse is 
 determined by the alternative value. If it is a U then 
 optional.front will be returned, if it is an Optional!U then an 
 optional will be returned. So the same semantics of range.
 * If T is Nullable!U, then isNull is checked. If it is false 
 and alternative value is a U, then nullable.get is returned. If 
 T is another Nullable, then the return type is a Nullable.
My first impression, upon reading this, is that I'm going to have to refer to the documentation every time I use `orElse` in order to be sure I'm doing it right. To me, this is a smell: if it takes me this long to describe what ought to be a very simple utility function, I can tell I've made it too complicated. As far as concrete advice: the overloads for reference types and ranges seem completely unrelated to Optional, so it seems like an odd choice to include them in the "optional" package. The overloads for Optional and Nullable make sense to include (though the Nullable one really *ought* to be in Phobos), but special-casing them to wrap the return value is too "magic" for my tastes. Much better to just let people write `some(val1.orElse(val2))` if that's what they want--it's only a few more characters, and it makes it obvious at a glance what the code is doing.
Jul 27
next sibling parent reply aliak <something something.com> writes:
On Saturday, 27 July 2019 at 15:07:56 UTC, Paul Backus wrote:
 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 * Null pointers are valid objects, so some!(int*)(null) is non 
 empty
 * Null classses and interfaces are empty, so some!Class(null) 
 is empty
I think it would be better to avoid weird special cases like this. The optional package already includes a NotNull type, so users can write `Optional!(NotNull!Class)` to opt into your proposed semantics explicitly.
Egad! I just (i.e. today/yesterday) removed NotNull as I could not figure out how to guarantee NotNull semantics without it just being awkward to use. I'm not sure it's generally used either. But that's based on what I've been using myself, and issues and prs in the repo - so not a very wide view. But maybe you're right in making it the same. I'm a bit weary because null classes inside optional values has bitten me and colleagues multiple times in Swift, Scala, and Kotlin over the past few years. Swift got rid of the issue in version 2 or something. Scala and Kotlin still have it because of necessary Java interop.
 orElse semantics:
 =================

 [...]

 So orElse works on a number of types and these are each 
 types's semantics:

 * If T is a reference type, val1.orElse(val2) will return val1 
 if (val1 !is null), else val2
 * If T is a range, the example above shows the semantics
 * If T is an Optional!U then it is treated separately in order 
 to work with const Optional!U. The value returned by orElse is 
 determined by the alternative value. If it is a U then 
 optional.front will be returned, if it is an Optional!U then 
 an optional will be returned. So the same semantics of range.
 * If T is Nullable!U, then isNull is checked. If it is false 
 and alternative value is a U, then nullable.get is returned. 
 If T is another Nullable, then the return type is a Nullable.
My first impression, upon reading this, is that I'm going to have to refer to the documentation every time I use `orElse` in order to be sure I'm doing it right. To me, this is a smell: if it takes me this long to describe what ought to be a very simple utility function, I can tell I've made it too complicated.
Very fair point. What if the description was simplified? The thing is I want it to "just work" based on the call site. So if a user does a.orElse(literal) on a container type, it's pretty obvious what they want. If a user does a.orElse(container), it's again (I think) obvious what they want. But essentially those are the only two cases. And since reference types are not containers, then only the second case applies. Does that make it simpler? So make coalescing separate from orElse-ing is basically the gist of this right?
 As far as concrete advice: the overloads for reference types 
 and ranges seem completely unrelated to Optional, so it seems 
 like an odd choice to include them in the "optional" package. 
 The overloads for Optional and Nullable make sense to include 
 (though the Nullable one really *ought* to be in Phobos), but 
 special-casing them to wrap the return value is too "magic" for
This is true. They are kinda unrelated and just there for convenience because I didn't know where else to get the functionality from. I actually played around with having the orelse in a completely separate package, and providing a hook: auto ref orElse(alias elseValue, T)(auto ref T value) { static if (hasMember!(T, "hookOrElse") { return value.hookOrElse!elseValue; } ... } And then implementing the hook in the optional package. But this bit me: https://forum.dlang.org/thread/lddwnnzlktszwspldxqu forum.dlang.org
 my tastes. Much better to just let people write 
 `some(val1.orElse(val2))` if that's what they want--it's only a 
 few more characters, and it makes it obvious at a glance what 
 the code is doing.
Sorry, what was the `some(val1.orElse(val2))`? Did you mean that instead of range1.orElse(range2) or range1.orElse(elementOfRange)? And thanks for the feedback!
Jul 27
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 July 2019 at 16:39:21 UTC, aliak wrote:
 Egad! I just (i.e. today/yesterday) removed NotNull as I could 
 not figure out how to guarantee NotNull semantics without it 
 just being awkward to use. I'm not sure it's generally used 
 either. But that's based on what I've been using myself, and 
 issues and prs in the repo - so not a very wide view.

 But maybe you're right in making it the same. I'm a bit weary 
 because null classes inside optional values has bitten me and 
 colleagues multiple times in Swift, Scala, and Kotlin over the 
 past few years. Swift got rid of the issue in version 2 or 
 something. Scala and Kotlin still have it because of necessary 
 Java interop.
As suggested elsewhere in this thread, treating both null references and null pointers as empty would also be a reasonable decision. The inconsistency is the real issue here.
 The thing is I want it to "just work" based on the call site. 
 So if a user does a.orElse(literal) on a container type, it's 
 pretty obvious what they want. If a user does 
 a.orElse(container), it's again (I think) obvious what they 
 want. But essentially those are the only two cases.  And since 
 reference types are not containers, then only the second case 
 applies. Does that make it simpler?
To me, it's obvious that the expression `a.orElse(b)` should return either a or b. If I ask the computer to choose between two alternatives for me, and instead it gives me a third thing that I didn't ask for (e.g., `a.front`), I am going to be very confused. For Optional, there's a one-to-one correspondence between the container and the value inside it, so eliding the distinction isn't a big deal (though strictly speaking, `valueOrElse` would be a more accurate name). On the other hand, I'd definitely be scratching my head if I saw something like `[1, 2, 3].orElse(4)`. An array, or else a single int? How does that even type-check? Compare to `[1, 2, 3].frontOrElse(4)`, which is so obvious that it needs no explanation. I understand the allure of this kind of "just works", "do what I mean" design, but following that philosophy is what gives us features like autodecoding--mostly harmless, but incredibly frustrating when they get in your way. After all, when someone iterates over a range of UTF-8 code units, it's "pretty obvious" that what they want is code points, right?
 So make coalescing separate from orElse-ing is basically the 
 gist of this right?
As I understand it, coalescing means flattening or unwrapping multiple layers of Optional at once. So, yes, orElse definitely shouldn't be doing that. But I'm guessing that's not what you meant to say.
 Sorry, what was the `some(val1.orElse(val2))`? Did you mean 
 that instead of range1.orElse(range2) or 
 range1.orElse(elementOfRange)?
Sorry, that was an incorrect example. What I was trying to say was, I don't think Optional.orElse should have two different return types depending on its arguments. If you want to have a version that unwraps its first argument and a version that doesn't, it would be better to give them different names, so it's always obvious which is being used. For example: Optional!int a = no!int; Optional!int b = some(123); int x = a.valueOrElse(123); // unwraps Optional!int y = a.orElse(b); // doesn't unwrap This is how they do it in Rust: the non-unwrapping version is called `or`, and the unwrapping version is called `unwrap_or`.
Jul 27
parent aliak <something something.com> writes:
On Sunday, 28 July 2019 at 00:05:19 UTC, Paul Backus wrote:
 As suggested elsewhere in this thread, treating both null 
 references and null pointers as empty would also be a 
 reasonable decision. The inconsistency is the real issue here.
Aye, I agree.
 The thing is I want it to "just work" based on the call site. 
 So if a user does a.orElse(literal) on a container type, it's
[...]
 or me, and instead it gives me a third thing that I didn't ask 
 for (e.g., `a.front`), I am going to be very confused.
Ha! Touche. I am convinced.
 For Optional, there's a one-to-one correspondence between the 
 container and the value inside it, so eliding the distinction
[...]
g when they get in your way. After all, when someone
 iterates over a range of UTF-8 code units, it's "pretty 
 obvious" that what they want is code points, right?

 So make coalescing separate from orElse-ing is basically the 
 gist of this right?
As I understand it, coalescing means flattening or unwrapping multiple layers of Optional at once. So, yes, orElse definitely shouldn't be doing that. But I'm guessing that's not what you meant to say.
I was thinking in terms of the null coalescing operator found in some languages. So a ?? b will give you a if it's not null and b otherwise, which is what orElse was doing if b was a range/optional.
 Sorry, what was the `some(val1.orElse(val2))`? Did you mean 
 that instead of range1.orElse(range2) or 
 range1.orElse(elementOfRange)?
Sorry, that was an incorrect example. What I was trying to say was, I don't think Optional.orElse should have two different return types depending on its arguments. If you want to have a version that unwraps its first argument and a version that doesn't, it would be better to give them different names, so it's always obvious which is being used. For example: Optional!int a = no!int; Optional!int b = some(123); int x = a.valueOrElse(123); // unwraps Optional!int y = a.orElse(b); // doesn't unwrap This is how they do it in Rust: the non-unwrapping version is called `or`, and the unwrapping version is called `unwrap_or`.
Thanks for the feedback. It looks like I'll be going with frontOrElse, orElse, and making the null semantics consistent.
Jul 27
prev sibling parent reply SimonN <eiderdaus gmail.com> writes:
On Saturday, 27 July 2019 at 15:07:56 UTC, Paul Backus wrote:
 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 * Null pointers are valid objects, so some!(int*)(null) is non 
 empty
 * Null classses and interfaces are empty, so some!Class(null) 
 is empty
I think it would be better to avoid weird special cases like this. The optional package already includes a NotNull type, so users can write `Optional!(NotNull!Class)` to opt into your proposed semantics explicitly.
It's surprising that pointers and references are special-cased differently, right. But I suggest that null is the same as empty, both for pointers and for references. I.e., the original post's proposal should change its pointer semantics, some!(int*)(null) == no!(int*). What was the original reason to treat pointers and class references differently? Reason to treat null as empty, always: We want at most one layer of nullability/emptiness, and usually no nullability at all, because we want to avoid nesting of "if it exists, do stuff, otherwise do nothing". All possible nullability/emptiness should be handled at once. This usual case (at most one layer of nullability/emptiness) should be quickest to write. If Optional!Class introduced emptiness different from nullability, then Optional!(NotNull!Class) would be standard usage because people want at most one layer of emptiness, but then standard usage would not be succinct. I accept that Optional!(NotNull!Class) is more explicit, but this is only because of D's lack of non-nullable references, which is the problem in the first place. NotNull is an abstraction inversion: It builds a non-nullable type (the less complex type because it allows fewer values and is crash-free to call methods) from a nullable type (which instead should be a sum type of the nonexisting non-nullable type and empty). I already use Optional!Class for single layer of nullability, and Class for non-null by convention. 95 % of the time, I want non-nullable. Exported methods, then, need in/out contracts to check for non-nullability. These contracts should ideally be inserted by the compiler on seeing a language-builtin non-nullable reference. -- Simon
Jul 27
parent aliak <something something.com> writes:
On Saturday, 27 July 2019 at 17:01:14 UTC, SimonN wrote:
 It's surprising that pointers and references are special-cased 
 differently, right.

 But I suggest that null is the same as empty, both for pointers 
 and for references. I.e., the original post's proposal should 
 change its pointer semantics, some!(int*)(null) == no!(int*).

 What was the original reason to treat pointers and class 
 references differently?
I only have a fuzzy recollection of something along the lines of: "null can be a valid return type of a function, and maybe there's an error and I want to return nothing". But, I can see how that thought is a bit... unconvincing right now. So yeah, bottom line, I'll fix the inconsistency as you suggest as I think that sounds more reasonable then what it currently is. Chur!
Jul 27
prev sibling next sibling parent reply Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 15:17 schrieb aliak:
 Hi,
 
 Can I ask for feedback on what people expect an optional/maybe type to
 have, and how to behave. There have been a number of discussions on the
 forums about it so far, and attempted PRs in phobos. And I currently
 maintain one that I've been using in-house very happily (backend server
 with vibe-d). I want to nail it down further and get to a version 1.0.0
 so feedback and comments would be highly appreciated.
 
 Optional semantics:
 ===================
 
 So first, the semantics of Optional!T (these may or may not already be
 implemented):
 * Is at least a forward range
 * Equality with itself, T, and sentinel-type "none", which checks if
 optional is empty or not
 * Assignment to T, `none`, or Optional!T
 * Includes type constructors some(x) and no!T
 * Forwards operators to T if T implements those operators
   E.g. some(1)++ and no!int++ both work
   E.g. If T has an opCall(...), then some(x)(...) and no!T()(...) both work
 * Null pointers are valid objects, so some!(int*)(null) is non empty
 * Null classses and interfaces are empty, so some!Class(null) is empty
 
 Optional utilities:
 ===================
 
 Two utilities are included. Optional chaining and a match function.
 Optional chaining allows you to go through an object's hierarchy and get
 an optional back at the end:
 
 oc(obj).property.function()
 
 If obj is none, then the end result is no!T where T is the return type
 of function. Else it has the actual value if the chain goes all the way
 though. This will also work if property is in turn an Optional itself.
 Secondly, the chaining function is also provided for NullableT and
 reference type.
 
 The match function takes two handlers (lambda aliases) and calls the
 correct one based on if the Optional!T is empty or not.
 
 orElse semantics:
 =================
 
 orElse will either get the front of a range or the range itself
 depending on the alternative value and also works on reference types. So:
 
 Range!int range1, range2;
 range1.orElse(range2) // returns range1 if range1 is not empty, else range2
 range1.orElse(8) // returns front or range1 if non empty, else 8.
 
 In essence, it also coalesces. I was on the fence on this, but it's
 turning out (again in our project) to be very convenient. I'm
 considering doing this for primitive types like int, float, or anything
 that can be cast to bool (while taking things like NaN in to account for
 e.g.)
 
 So orElse works on a number of types and these are each types's semantics:
 
 * If T is a reference type, val1.orElse(val2) will return val1 if (val1
 !is null), else val2
 * If T is a range, the example above shows the semantics
 * If T is an Optional!U then it is treated separately in order to work
 with const Optional!U. The value returned by orElse is determined by the
 alternative value. If it is a U then optional.front will be returned, if
 it is an Optional!U then an optional will be returned. So the same
 semantics of range.
 * If T is Nullable!U, then isNull is checked. If it is false and
 alternative value is a U, then nullable.get is returned. If T is another
 Nullable, then the return type is a Nullable.
 
 I may have forgotten some stuff but I think that's everything.
 
 Thanks in advance!
 
 PS: I realize an obvious first comment is "what about Nullable!T". A
 number of reasons: 1) it doesn't have a range interface 2) the reference
 semantics are not desirable for me and writing generic code is awkward
 (https://github.com/aliak00/optional#what-about-stdtypeconsnullable), 3)
 alias this, although deprecated, breaks it completely, 4) maintaining a
 dub package feels much more productive.
orElse currently has an overload that takes a callable that returns a fallback value as template parameter. I think something like this is needed, but it might be better to give it a different name (in Java, Scala etc. this is called orElseGet). Other convenience utilities might be nice, e.g. ifPresent, ifPresentOrElse, orElseThrow etc. (take Java's Optional or Scala's Option as reference). These are all very easy to implement and I have already done so several times when using your library. I also think that your current orElse semantics are a bit too complicated: There is simply too much stuff (with slighty differing behavior) in that one single function (I know, it is several overloads, but the user does not care). E.g. I think the overload that takes another range (or Optional or Nullable) should be a seperate function because it actually has different semantics. All overloads should have the same general behavior, in my opinion. In Java, this function is called "or". Otherwise, I actually quite like the semantics of your library (and I use it regularly). Thanks for your work!
Jul 27
parent reply aliak <something something.com> writes:
On Saturday, 27 July 2019 at 16:27:53 UTC, Johannes Loher wrote:
 orElse currently has an overload that takes a callable that 
 returns a
 fallback value as template parameter. I think something like 
 this is
 needed, but it might be better to give it a different name (in 
 Java,
 Scala etc. this is called orElseGet). Other convenience 
 utilities might
 be nice, e.g. ifPresent, ifPresentOrElse, orElseThrow etc. 
 (take Java's
 Optional or Scala's Option as reference). These are all very 
 easy to
 implement and I have already done so several times when using 
 your library.
How would you find using .each over ifPresent? As for ifPresentOrElse, would this suffice?: no!int.orElse!(() => log("nothing there")); Would you prefer an explicit orElseThrow or a throw_ expression as proposed in a previous post?
 I also think that your current orElse semantics are a bit too 
 complicated: There is simply too much stuff (with slighty 
 differing behavior) in that one single function (I know, it is 
 several overloads, but the user does not care). E.g. I think 
 the overload that takes another range (or Optional or Nullable) 
 should be a seperate function because it actually has different 
 semantics. All overloads should have the same general behavior, 
 in my opinion. In Java, this function is called "or".
Alright, more for separation of orElse's functions :) I must say that I quite like the coalesce functionality. But I think maybe you and Paul might be right to separate that and make it a different function. Or just not do the coalescing and the user can be a bit more explicit as I think Paul suggested.
 Otherwise, I actually quite like the semantics of your library 
 (and I use it regularly). Thanks for your work!
You're welcome!
Jul 27
parent reply Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 19:15 schrieb aliak:
 
 How would you find using .each over ifPresent?
See my other answer. Though I don't care _that_ much. "each" might be a bit awkward at times, but maybe it is good enough.
 As for ifPresentOrElse, would this suffice?:
 
 no!int.orElse!(() => log("nothing there"));
I don't see how that has anything to do with ifPresentOrElse. Could you please explain? In my other answer I also wrote a bit about why ifPresentOrElse would be nice (and why match does not completely replace it).
 Would you prefer an explicit orElseThrow or a throw_ expression as
 proposed in a previous post?
I don't actually care that much, but I think I slightly prefer orElseThrow (probably because I'm used to it from Java). This is what I have been doing in the past: ``` no!int.orElse!(function int() { throw new Exception(""); }); ``` It is slightly annoying that you have to write the return type explictly, but it's not that bad. I think I also wrote my own orElseThrow: ``` import optional; auto orElseThrow(alias fun, T)(auto ref Optional!T opt) { return opt.orElse!(delegate T() { throw fun(); }); } void main() { no!int.orElseThrow!(() => new Exception("")); } ``` (you could make some improvements by checking (with static ifs) if a delegate is actually needed, or a function suffices)
 Alright, more for separation of orElse's functions :) I must say that I
 quite like the coalesce functionality. But I think maybe you and Paul
 might be right to separate that and make it a different function. Or
 just not do the coalescing and the user can be a bit more explicit as I
 think Paul suggested.
Most of the time, I don't actually have a need for the overloads that take a range, optional, nullable, so I'd be kind of OK with simply removing it. I'd also totally be OK with it being a separate function. What Paul suggested is not a good solution imo because it is besides the point. Wrapping with optional after calling orElse is something totally different than providing another optional as fallback, because in the first case, you need to know that the fallback value is actually present beforehand (in which case you should simply use the regular orElse anyways). The value that the additional orElse overload currently provides is that you can provide another fallback value of which you do not know if it present as well and you can also chain this in a neat way: /* ... */ enum fallback = 0; return no!int .orElse(dbRequestThatMightOrMightNotReturnSomething()) .orElse(httpRequestThatMightOrMightNotReturnSomething()) .orElse(fallback); /* ... */ As mentioned above, I do not use this that often, but I think it is still a valid usecase and Pauls suggestion is not a substitute for it.
Jul 27
next sibling parent Johannes Loher <johannes.loher fg4f.de> writes:
Am 27.07.19 um 19:59 schrieb Johannes Loher:
 It is slightly annoying that you have to write the return type
 explictly, but it's not that bad. I think I also wrote my own orElseThrow:
 
 ```
 import optional;
 
 auto orElseThrow(alias fun, T)(auto ref Optional!T opt)
 {
     return opt.orElse!(delegate T() { throw fun(); });
 }
 
 void main()
 {
     no!int.orElseThrow!(() => new Exception(""));
 }
 ```
 (you could make some improvements by checking (with static ifs) if a
 delegate is actually needed, or a function suffices)
By the way, in this situation it would be kind of useful to have a bottom type or an (inferred) noreturn annotation. Then in addition callables that return something of type T, you could explictily allow passing in callables that never return.
Jul 27
prev sibling next sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 27 July 2019 at 17:59:19 UTC, Johannes Loher wrote:
 Am 27.07.19 um 19:15 schrieb aliak:
 ```
 import optional;

 auto orElseThrow(alias fun, T)(auto ref Optional!T opt)
 {
     return opt.orElse!(delegate T() { throw fun(); });
 }

 void main()
 {
     no!int.orElseThrow!(() => new Exception(""));
 }
 ```
Lazy arguments could fit perfectly for orElseThrow statement: ```D import optional; auto orElseThrow(T)(auto ref Optional!T opt, lazy Exception throwable) { if (opt.empty) { throw throwable; } return opt.front; } void main() { no!int.orElseThrow(new Exception("")); // maybe instead of no, just optional!int? using just negation is kinda ambiguous. } ```
Jul 27
prev sibling parent aliak <something something.com> writes:
On Saturday, 27 July 2019 at 17:59:19 UTC, Johannes Loher wrote:
 Am 27.07.19 um 19:15 schrieb aliak:
 
 How would you find using .each over ifPresent?
See my other answer. Though I don't care _that_ much. "each" might be a bit awkward at times, but maybe it is good enough.
 As for ifPresentOrElse, would this suffice?:
 
 no!int.orElse!(() => log("nothing there"));
I don't see how that has anything to do with ifPresentOrElse. Could you please explain? In my other answer I also wrote a bit about why ifPresentOrElse would be nice (and why match does not completely replace it).
Sorry, I misunderstood. So basically: if (a.empty) fun(); else value += 1; So Scala provides this with match out of the box by promoting (demoting?) the types to Any, and indeed the current match can be made to do the "same" (if the return types are incompatible, then there's no return, i.e. void). Should work no? If you mess up types the compiler will shout so it sounds, sound. Btw, this reminds me of an optional vs matching cheat sheet that I was pointed to some time ago and thought it was great! -> https://www.originate.com/thinking/stories/idiomatic-scala-your-options-do-not-match/
 Would you prefer an explicit orElseThrow or a throw_ 
 expression as proposed in a previous post?
I don't actually care that much, but I think I slightly prefer orElseThrow (probably because I'm used to it from Java). This is what I have been doing in the past: ``` no!int.orElse!(function int() { throw new Exception(""); }); ``` It is slightly annoying that you have to write the return type explictly, but it's not that bad. I think I also wrote my own orElseThrow: ``` import optional; auto orElseThrow(alias fun, T)(auto ref Optional!T opt) { return opt.orElse!(delegate T() { throw fun(); }); } void main() { no!int.orElseThrow!(() => new Exception("")); } ``` (you could make some improvements by checking (with static ifs) if a delegate is actually needed, or a function suffices)
Yes. I've done this as well. So I think I'll add this in.
 Alright, more for separation of orElse's functions :) I must 
 say that I quite like the coalesce functionality. But I think 
 maybe you and Paul might be right to separate that and make it 
 a different function. Or just not do the coalescing and the 
 user can be a bit more explicit as I think Paul suggested.
Most of the time, I don't actually have a need for the overloads that take a range, optional, nullable, so I'd be kind of OK with simply removing it. I'd also totally be OK with it being a separate function.
👌
 What Paul suggested is not a good solution imo because it is 
 besides the point. Wrapping with optional after calling orElse 
 is something totally different than providing another optional 
 as fallback, because in the first case, you need to know that 
 the fallback value is actually present beforehand (in which 
 case you should simply use the regular orElse anyways). The 
 value that the additional orElse overload currently provides is 
 that you can provide another fallback value of which you do not 
 know if it present as well and you can also chain this in a 
 neat way:

 /* ... */
 enum fallback = 0;
 return no!int
     .orElse(dbRequestThatMightOrMightNotReturnSomething())
     .orElse(httpRequestThatMightOrMightNotReturnSomething())
     .orElse(fallback);
 /* ... */

 As mentioned above, I do not use this that often, but I think 
 it is still a valid usecase and Pauls suggestion is not a 
 substitute for it.
Yes, I agree that's a nice use-case (which I use). I think I'll probably go with frontOrElse and orElse to separate the functionality. Something is screaming flatMap to me right now but I just can't put my finger on it ;)
Jul 27
prev sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 PS: I realize an obvious first comment is "what about 
 Nullable!T". A number of reasons: 1) it doesn't have a range 
 interface 2) the reference semantics are not desirable for me 
 and writing generic code is awkward 
 (https://github.com/aliak00/optional#what-about-stdtypeconsnullable), 3) alias
this, although deprecated, breaks it completely, 4) maintaining a dub package
feels much more productive.Skimmed a bit through main optional module, and have
some questions/suggestions.
Just wondering whats the reason of treating not null as being a value? Per my knowledge the meaning of null basically translates to "no value", does it not make sense to have null be treated as empty optional, and consequently any null pointer? Saw you're using from pattern to import symbols, wouldn't it make a template bloat and increase compile time given big projects? As for aliasing what is the problem with it? Would not it be better than introspecting and defining operator overloads in Optional? Existing design would introduce overhead on compilation time due to introspection of payload. I see you can assign value to optional payload wouldn't it be better to have Optional contents pseudo immutable, and have any changes defined through copies of it? Regarding toString, a sink version would be nice to have as it allows nogc use of it. Json conversion could be outside of Optional in a separate module or even project, since some people might want serialization not only to vibe d. If we'll treat null values as empty then won't be any need in none type, and optional == null would be enough. `some` for constructor of nullable is too generic name, maybe rename it to optional/asOptional, it will make more sense and be consistent with toOptional present in project. toNullable could be moved as opCast maybe. Best regards, Alexandru.
Jul 27
parent aliak <something something.com> writes:
On Saturday, 27 July 2019 at 19:00:13 UTC, Alexandru Ermicioi 
wrote:
 On Saturday, 27 July 2019 at 13:17:32 UTC, aliak wrote:
 PS: I realize an obvious first comment is "what about 
 Nullable!T". A number of reasons: 1) it doesn't have a range 
 interface 2) the reference semantics are not desirable for me 
 and writing generic code is awkward 
 (https://github.com/aliak00/optional#what-about-stdtypeconsnullable), 3) alias
this, although deprecated, breaks it completely, 4) maintaining a dub package
feels much more productive.Skimmed a bit through main optional module, and have
some questions/suggestions.
Just wondering whats the reason of treating not null as being a value? Per my knowledge the meaning of null basically translates to "no value", does it not make sense to have null be treated as empty optional, and consequently any null pointer?
I think you're right here. Will be changing this.
 Saw you're using from pattern to import symbols, wouldn't it 
 make a template bloat and increase compile time given big 
 projects?
I would assume so. Maybe. It also at the same time reduces compile time by lazily importing modules only if needed, i.e. when instantiations happen. Though, since you brought it to attention, I do see that it's being used in some places that don't make any sense (at declaration scope of the Optional for e.g.) So thanks for that!
 As for aliasing what is the problem with it? Would not it be 
 better than introspecting and defining operator overloads in 
 Optional? Existing design would introduce overhead on 
 compilation time due to introspection of payload.
I could move the operator introspection to the optional chain. I don't know how much overhead on compilation time this will gain. Have you had experience with this and do you think it's significant?
 I see you can assign value to optional payload wouldn't it be 
 better to have Optional contents pseudo immutable, and have any 
 changes defined through copies of it?
I guess so? Hmm... this one I'll have to sit on for a while
 Regarding toString, a sink version would be nice to have as it 
 allows nogc use of it.
Agreed!
 Json conversion could be outside of Optional in a separate 
 module or even project, since some people might want 
 serialization not only to vibe d.
For vibe-d it has to be inside unfortunately. But it's only compiled in if you're using vibe-d so I think that should be ok.
 If we'll treat null values as empty then won't be any need in 
 none type, and optional == null would be enough.
Indeed. That might be hard with opAssign present and T being null assignable.
 `some` for constructor of nullable is too generic name, maybe 
 rename it to optional/asOptional, it will make more sense and 
 be consistent with toOptional present in project.
indeed, also from your other post, maybe some! and no! can just be both replaced with optional! ... I always just liked the wording though.
 toNullable could be moved as opCast maybe.
It could. If I overload an opX explicitly for the Optional type, then I'll have to move all the forwarding opXs to somewhere else as well. Might be the way to go.
 Best regards,
 Alexandru.
Thanks for all the comments. It sounds like you actually went through the code! Much appreciated.
Jul 27