digitalmars.D - Proposing std.typecons : Optional (with PR)
- FeepingCreature (19/19) Jun 11 2019 Since my rfc regarding deprecating `alias get this` in Nullable
- aliak (8/17) Jun 11 2019 If we're going to put an optional type there, I would consider
- FeepingCreature (10/17) Jun 11 2019 I've looked at that and I specifically disagree with the decision
- aliak (17/34) Jun 11 2019 The explanation is a start :)
- FeepingCreature (13/41) Jun 11 2019 Right, my point is that in functional languages things like
- aliak (14/57) Jun 11 2019 It does actually. I use it like a range all the time:
- FeepingCreature (9/19) Jun 11 2019 That's a fundamental problem with D though. Null *is* a valid
- Nick Treleaven (20/31) Jun 11 2019 It is accepted but it is not valid. There is no object there, you
- Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= (5/10) Jun 11 2019 I don't disagree, but one often end up creating a Nobody subclass
- Aliak (5/15) Jun 11 2019 If there’s a possibility your class can be null then it should
- Johannes Loher (16/20) Jun 11 2019 Java has the exact same problem with all non primitive types being
- Jacob Carlborg (4/5) Jun 12 2019 And the rest of the algorithms?
- FeepingCreature (7/10) Jun 12 2019 As the saying goes, "PRs welcome." Personally I get along fine
- aliak (3/16) Jun 12 2019 These algorithms are already there if it's implemented as a
- aliak (2/3) Jun 12 2019 ... I would hope.
- FeepingCreature (7/9) Jun 12 2019 For instance, `orElse` is the very clear and understandable
- Jacob Carlborg (4/14) Jun 13 2019 As I mentioned in the PR, I expect a "orElse" function to be available.
- Johannes Loher (45/48) Jun 11 2019 Yes, ranges are not a general replacement for the concept of a monad,
- Marco de Wild (17/38) Jun 11 2019 I agree that Nullable has behaviour that is unwanted. However,
- FeepingCreature (27/44) Jun 11 2019 My impression is that the distinction turns on whether the null
- Johannes Loher (4/18) Jun 11 2019 Exactly. This is why I'd expect a "real" Optional type to provide a lot
Since my rfc regarding deprecating `alias get this` in Nullable (see https://github.com/dlang/phobos/pull/7060 ) encountered no meaningful criticism (as I'm tactfully choosing to interpret the total absence of reaction), but somebody mentioned that a new type may have more success, since it doesn't have to justify itself in the same way as a change to existing behavior, I present `Optional`: https://github.com/dlang/phobos/pull/7065 . To reiterate the difference: `Nullable` is the standard type for "adding an undefined value state to an existing type". It's being abused as an `Optional` type. Why "abused"? Well: 1. `get` is **undefined** if Nullable is null 2. `get` is called implicitly. This is not good behavior for an `Optional` type! Since my attempt to turn `Nullable` into an `Optional` type encountered zero enthusiasm and mild resistance, I've started a second parallel attempt to introduce an `Optional` type from scratch. (Translation: I've copypasted `Nullable` and changed a few names.) Opinion? Anyone? Anyone at all? I'm so alone...
Jun 11 2019
On Tuesday, 11 June 2019 at 07:46:38 UTC, FeepingCreature wrote:Since my rfc regarding deprecating `alias get this` in Nullable (see https://github.com/dlang/phobos/pull/7060 ) encountered no meaningful criticism (as I'm tactfully choosing to interpret the total absence of reaction), but somebody mentioned that a new type may have more success, since it doesn't have to justify itself in the same way as a change to existing behavior, I present `Optional`: https://github.com/dlang/phobos/pull/7065 . [...]If we're going to put an optional type there, I would consider not copy pasting nullable as it has some other issues as well, and maybe nail the interface down properly? E.g. having an optional that's a range would be rather awesome. Have you looked at: https://code.dlang.org/packages/optional. It has a lot of other added features as well but those can be removed if they're too much.
Jun 11 2019
On Tuesday, 11 June 2019 at 09:58:48 UTC, aliak wrote:If we're going to put an optional type there, I would consider not copy pasting nullable as it has some other issues as well, and maybe nail the interface down properly? E.g. having an optional that's a range would be rather awesome. Have you looked at: https://code.dlang.org/packages/optional. It has a lot of other added features as well but those can be removed if they're too much.I've looked at that and I specifically disagree with the decision to make it a range; that's why I didn't just internally switch to dub optional. Ranges are not monads. We may wish we had a concept of monads but we don't; ranges are not a general replacement for any conceivable container. `Optional.front` is just *weird* and unintuitive at first glance. It's not a bad decision in isolation, but it doesn't fit what I consider "the D style" of type design. Sorry if that explanation is too fuzzy.
Jun 11 2019
On Tuesday, 11 June 2019 at 10:01:18 UTC, FeepingCreature wrote:On Tuesday, 11 June 2019 at 09:58:48 UTC, aliak wrote:The explanation is a start :) Ranges are not monads in the strictly mathematical sense you mean or? Why do you say this? You can add a .get or .value to a range as well to not have the weirdness. Though ranges and their .front are a central concept in D so it should be understood *if* option is defined as a range. Plus, whether or not you agree ranges are/can be monads or not is a tangential issue to seeing an optional as a monad or a collection (it's not mutually exclusive). In scala for e.g. it's a collection (also a monad), rust implements FromIterator and IntoIterator, haskell adheres to foldable, applicative, traversable and i guess others... I'm not saying it has to be a range either, but you lose out on functional composition if it's not. Or you re-implement all the stuff from std.range/algorithm you want as part of option's interface to get them.If we're going to put an optional type there, I would consider not copy pasting nullable as it has some other issues as well, and maybe nail the interface down properly? E.g. having an optional that's a range would be rather awesome. Have you looked at: https://code.dlang.org/packages/optional. It has a lot of other added features as well but those can be removed if they're too much.I've looked at that and I specifically disagree with the decision to make it a range; that's why I didn't just internally switch to dub optional. Ranges are not monads. We may wish we had a concept of monads but we don't; ranges are not a general replacement for any conceivable container. `Optional.front` is just *weird* and unintuitive at first glance. It's not a bad decision in isolation, but it doesn't fit what I consider "the D style" of type design. Sorry if that explanation is too fuzzy.
Jun 11 2019
On Tuesday, 11 June 2019 at 11:56:01 UTC, aliak wrote:On Tuesday, 11 June 2019 at 10:01:18 UTC, FeepingCreature wrote:Right, my point is that in functional languages things like monads or generic interfaces in general are used to compose reusable containers into larger processing chains. However, I see `Optional` less as a container and more as a metaphor. As a metaphor, it's not "a range of elements with a length of either 0 or 1", it's "a type that may be either of those type's values plus 'unset'". That's not something that easily lends itself to the set of verbs that are used to manipulate ranges. D is not a functional language, it's a multiparadigm language. It shouldn't be afraid to have a type that deviates from the functional paradigm if the functional paradigm is a bad metaphoric fit for the semantic message that the type expresses.I've looked at that and I specifically disagree with the decision to make it a range; that's why I didn't just internally switch to dub optional. Ranges are not monads. We may wish we had a concept of monads but we don't; ranges are not a general replacement for any conceivable container. `Optional.front` is just *weird* and unintuitive at first glance. It's not a bad decision in isolation, but it doesn't fit what I consider "the D style" of type design. Sorry if that explanation is too fuzzy.The explanation is a start :) Ranges are not monads in the strictly mathematical sense you mean or? Why do you say this? You can add a .get or .value to a range as well to not have the weirdness. Though ranges and their .front are a central concept in D so it should be understood *if* option is defined as a range. Plus, whether or not you agree ranges are/can be monads or not is a tangential issue to seeing an optional as a monad or a collection (it's not mutually exclusive). In scala for e.g. it's a collection (also a monad), rust implements FromIterator and IntoIterator, haskell adheres to foldable, applicative, traversable and i guess others... I'm not saying it has to be a range either, but you lose out on functional composition if it's not. Or you re-implement all the stuff from std.range/algorithm you want as part of option's interface to get them.
Jun 11 2019
On Tuesday, 11 June 2019 at 12:01:34 UTC, FeepingCreature wrote:On Tuesday, 11 June 2019 at 11:56:01 UTC, aliak wrote:It does actually. I use it like a range all the time: arrayOfStuff .map!maybeParseThing .each!processThing; Your argument against it being a range sounds a bit too philosophical. I think we should consider the actual technical advantages/disadvantages of having it as a range or not. And if not, at least getting functions like map/flatmap/each/whatever as usable with optional. I basically think a bit of thought should be put in to putting an optional type in the standard library instead of copying nullable (as mentioned, the range thing is not the only problem - another big one is treating null as a valid value for classes).On Tuesday, 11 June 2019 at 10:01:18 UTC, FeepingCreature wrote:Right, my point is that in functional languages things like monads or generic interfaces in general are used to compose reusable containers into larger processing chains. However, I see `Optional` less as a container and more as a metaphor. As a metaphor, it's not "a range of elements with a length of either 0 or 1", it's "a type that may be either of those type's values plus 'unset'". That's not something that easily lends itself to the set of verbs that are used to manipulate ranges.I've looked at that and I specifically disagree with the decision to make it a range; that's why I didn't just internally switch to dub optional. Ranges are not monads. We may wish we had a concept of monads but we don't; ranges are not a general replacement for any conceivable container. `Optional.front` is just *weird* and unintuitive at first glance. It's not a bad decision in isolation, but it doesn't fit what I consider "the D style" of type design. Sorry if that explanation is too fuzzy.The explanation is a start :) Ranges are not monads in the strictly mathematical sense you mean or? Why do you say this? You can add a .get or .value to a range as well to not have the weirdness. Though ranges and their .front are a central concept in D so it should be understood *if* option is defined as a range. Plus, whether or not you agree ranges are/can be monads or not is a tangential issue to seeing an optional as a monad or a collection (it's not mutually exclusive). In scala for e.g. it's a collection (also a monad), rust implements FromIterator and IntoIterator, haskell adheres to foldable, applicative, traversable and i guess others... I'm not saying it has to be a range either, but you lose out on functional composition if it's not. Or you re-implement all the stuff from std.range/algorithm you want as part of option's interface to get them.
Jun 11 2019
On Tuesday, 11 June 2019 at 12:30:10 UTC, aliak wrote:Your argument against it being a range sounds a bit too philosophical. I think we should consider the actual technical advantages/disadvantages of having it as a range or not. And if not, at least getting functions like map/flatmap/each/whatever as usable with optional.We have "map for Optional/Nullable", that's `apply`.I basically think a bit of thought should be put in to putting an optional type in the standard library instead of copying nullable (as mentioned, the range thing is not the only problem - another big one is treating null as a valid value for classes).That's a fundamental problem with D though. Null *is* a valid value for classes. Making a class type optional just means that you're adding *another* type that also means "absent" at a different level of abstraction. Was it the right decision to allow null in class types? Who knows. (No.) But I don't think that's a decision that should be patched in library types.
Jun 11 2019
On Tuesday, 11 June 2019 at 13:55:52 UTC, FeepingCreature wrote:On Tuesday, 11 June 2019 at 12:30:10 UTC, aliak wrote:It is accepted but it is not valid. There is no object there, you can't dereference it (unless null is a valid memory address, which it is in some cases). Null is a gaping hole in any static type system. Rant: IMO the worst thing Walter ever said was "Non-nullable pointers are just plugging one hole in a cheese grater": https://twitter.com/walterbright/status/338821357762654208 Optional types with language enforcement can allow a static guarantee that any data is constructed with particular values, returning an optional that is none when the initializer was outside those values. I'd actually rather have language-assisted optional types than features like function overloading (a convenience feature, sometimes abused), because it means *the type system can model runtime checks as a static type guarantee*. They are a game changer.(as mentioned, the range thing is not the only problem - another big one is treating null as a valid value for classes).That's a fundamental problem with D though. Null *is* a valid value for classes.Making a class type optional just means that you're adding *another* type that also means "absent" at a different level of abstraction.Re-use null to mean absent.Was it the right decision to allow null in class types? Who knows. (No.) But I don't think that's a decision that should be patched in library types.Then I vote against including your Optional.
Jun 11 2019
On Tuesday, 11 June 2019 at 18:01:32 UTC, Nick Treleaven wrote:I'd actually rather have language-assisted optional types than features like function overloading (a convenience feature, sometimes abused), because it means *the type system can model runtime checks as a static type guarantee*. They are a game changer.I don't disagree, but one often end up creating a Nobody subclass singleton that is used as a replacement for null. Although that does have benefits (e.g. if you call a nobody.name() you get "nobody" rather than a crash).
Jun 11 2019
On Tuesday, 11 June 2019 at 19:02:12 UTC, Ola Fosheim Grøstad wrote:On Tuesday, 11 June 2019 at 18:01:32 UTC, Nick Treleaven wrote:If there’s a possibility your class can be null then it should probably be an optional. Or lazy. Have not seen this pattern you describe in swift scala or Kotlin.I'd actually rather have language-assisted optional types than features like function overloading (a convenience feature, sometimes abused), because it means *the type system can model runtime checks as a static type guarantee*. They are a game changer.I don't disagree, but one often end up creating a Nobody subclass singleton that is used as a replacement for null. Although that does have benefits (e.g. if you call a nobody.name() you get "nobody" rather than a crash).
Jun 11 2019
Am 11.06.19 um 20:01 schrieb Nick Treleaven:Java has the exact same problem with all non primitive types being nullable. They also solved it on the library level with Optional. In Java, you can never extract null from an Optional: If you try to constrcut an Optional from null using Optional.of, you will get an exception and if you use Optional.ofNullable, the result will simply be an empty Optional (which technically probably holds the null reference inside, but you can never retrieve it because trying to access it will throw an exception). This approach seems to have worked out quite well for Java, so I suggest we try a similar approach. Of course the situation is a bit easier for Java than for D because they only need to care about class types and we have to make everything work for pointers, arrays, associstive arrays, ..., as well, and the desired semantics are not that clear for those... But at least for classes, I think the route that Java took is the correct one.Making a class type optional just means that you're adding *another* type that also means "absent" at a different level of abstraction.Re-use null to mean absent.
Jun 11 2019
On 2019-06-11 15:55, FeepingCreature wrote:We have "map for Optional/Nullable", that's `apply`.And the rest of the algorithms? -- /Jacob Carlborg
Jun 12 2019
On Wednesday, 12 June 2019 at 09:53:03 UTC, Jacob Carlborg wrote:On 2019-06-11 15:55, FeepingCreature wrote:As the saying goes, "PRs welcome." Personally I get along fine without them, so I didn't see the need to include them. For instance, we have a local orElse implementation hanging around. The point of having `Optional` in the stdlib is that the rest of the ecosystem can standardize around the data structure. There is none such need for algorithms, especially oneliners.We have "map for Optional/Nullable", that's `apply`.And the rest of the algorithms?
Jun 12 2019
On Wednesday, 12 June 2019 at 10:52:22 UTC, FeepingCreature wrote:On Wednesday, 12 June 2019 at 09:53:03 UTC, Jacob Carlborg wrote:These algorithms are already there if it's implemented as a range. There's no need to reimplement them.On 2019-06-11 15:55, FeepingCreature wrote:As the saying goes, "PRs welcome." Personally I get along fine without them, so I didn't see the need to include them. For instance, we have a local orElse implementation hanging around. The point of having `Optional` in the stdlib is that the rest of the ecosystem can standardize around the data structure. There is none such need for algorithms, especially oneliners.We have "map for Optional/Nullable", that's `apply`.And the rest of the algorithms?
Jun 12 2019
On Wednesday, 12 June 2019 at 11:56:57 UTC, aliak wrote:There's no need to reimplement them.... I would hope.
Jun 12 2019
On Wednesday, 12 June 2019 at 11:56:57 UTC, aliak wrote:These algorithms are already there if it's implemented as a range. There's no need to reimplement them.For instance, `orElse` is the very clear and understandable `5.optional.chain(7.only).front`. Which, if you are at all interested in readability, you'll `alias orElse = (a, b) => a.chain(b.only).front;` anyways. At which point you might as well `alias orElse = (a, b) a.present ? a.value : b;`
Jun 12 2019
On 2019-06-12 17:58, FeepingCreature wrote:On Wednesday, 12 June 2019 at 11:56:57 UTC, aliak wrote:As I mentioned in the PR, I expect a "orElse" function to be available. -- /Jacob CarlborgThese algorithms are already there if it's implemented as a range. There's no need to reimplement them.For instance, `orElse` is the very clear and understandable `5.optional.chain(7.only).front`. Which, if you are at all interested in readability, you'll `alias orElse = (a, b) => a.chain(b.only).front;` anyways. At which point you might as well `alias orElse = (a, b) a.present ? a.value : b;`
Jun 13 2019
Am 11.06.19 um 12:01 schrieb FeepingCreature:Ranges are not monads. We may wish we had a concept of monads but we don't; ranges are not a general replacement for any conceivable container.Yes, ranges are not a general replacement for the concept of a monad, but the concept "range" is basically a monad: Type constructor: For any type T, a range of T is something for that isInputRange!T is true (yes, technically that's not a single "type", but rather something like a typeclass, but it is close enough). Type converter (unit / return): unit: T -> Range T, x -> only(x) Combinator (bind / >>=): (Range T, T → Range U) → Range U, (r,f) -> r.map!f.joiner Obviously, using these definitions, unit is both a left- and right-identity for bind and bind is essentially associative. So aside from the fact that there is no "real" type constructor because there is no actual general range type, I believe this qualifies the concept of a range as a monad. All of this aside, I agree that ranges are not a good general replacement for every container. I do however disagree with you in this specific case. I personally have to use Java a lot at work and I am regularly annoyed by the fact, that Optionals in Java are not Iterables (or Streams...). I like the aproach of Scala's and Vavr's "Option" much more, which implement the corresponding Iterable interfaces. It makes a lot of things just that much easier, e.g. List.of(Optional.of(42), Optional.of(1234)).stream() .filter(Optional::isPresent) .map(Optional::get) becomes List.of(Optional.of(42), Optional.of(1234)).stream() .flatMap(o -> o) or in D: only(optional(42), optional(1234)) .filter!(o -> o.present) .map!(o -> o.value) becomes only(optional(42), optional(1234)) .joiner Of course it is still debatable if we want this, but in my opinion this is a great improvement. If we decide against Optional being a "real" optional type as used in other more functional languages, I'm actually against adding it to the standard library. I'd much prefer to fix the `alias get this` issue of Nullable in that case. Don't get me wrong however, I'd love a real Optional type. But so far, the `optional` dub package works reasonably well for me. I like the pattern matching functionality in particular, but it also provides other goodies you'd expect from a real optional type, too, e.g. `orElse`. Your suggested implementation is lacking these unfortunately.
Jun 11 2019
On Tuesday, 11 June 2019 at 07:46:38 UTC, FeepingCreature wrote:Since my rfc regarding deprecating `alias get this` in Nullable (see https://github.com/dlang/phobos/pull/7060 ) encountered no meaningful criticism (as I'm tactfully choosing to interpret the total absence of reaction), but somebody mentioned that a new type may have more success, since it doesn't have to justify itself in the same way as a change to existing behavior, I present `Optional`: https://github.com/dlang/phobos/pull/7065 . To reiterate the difference: `Nullable` is the standard type for "adding an undefined value state to an existing type". It's being abused as an `Optional` type. Why "abused"? Well: 1. `get` is **undefined** if Nullable is null 2. `get` is called implicitly. This is not good behavior for an `Optional` type! Since my attempt to turn `Nullable` into an `Optional` type encountered zero enthusiasm and mild resistance, I've started a second parallel attempt to introduce an `Optional` type from scratch. (Translation: I've copypasted `Nullable` and changed a few names.) Opinion? Anyone? Anyone at all? I'm so alone...I agree that Nullable has behaviour that is unwanted. However, does this really warrant a new type in the standard library that contains the same functionality but with different implementation details? From the perspective of someone who is not too familiar with the standard library, which one do they need to choose? Why would they pick Optional over Nullable? Why would they pick Nullable over Optional? If Optional coexists with Nullable, IMO it needs to have a distinct use case to prevent confusion. I would prefer to fix Nullable and have one consistent way of dealing with this problem in the standard library, rather than a (so it seems) "battle" between Nullable and Optional fighting for attention and usage. I don't know if any Phobos' functions use Nullable, but those would need to be updated to work with Optional as well (for said consistency). I didn't comment earlier, but I am in favour of your earlier pull request.
Jun 11 2019
On Tuesday, 11 June 2019 at 11:02:34 UTC, Marco de Wild wrote:I agree that Nullable has behaviour that is unwanted. However, does this really warrant a new type in the standard library that contains the same functionality but with different implementation details? From the perspective of someone who is not too familiar with the standard library, which one do they need to choose? Why would they pick Optional over Nullable? Why would they pick Nullable over Optional? If Optional coexists with Nullable, IMO it needs to have a distinct use case to prevent confusion. I would prefer to fix Nullable and have one consistent way of dealing with this problem in the standard library, rather than a (so it seems) "battle" between Nullable and Optional fighting for attention and usage. I don't know if any Phobos' functions use Nullable, but those would need to be updated to work with Optional as well (for said consistency). I didn't comment earlier, but I am in favour of your earlier pull request.My impression is that the distinction turns on whether the null case can be expected to occur in "normal use" of the type. As `Nullable` says, accessing `get` while the `Nullable` `isNull` is *undefined*; that is, at the level of accessing missing array keys. (Except you can access them implicitly.) From this I infer that `Nullable` is mostly supposed to be non-null, maybe to enable delayed initialization or the like, which would justify the `alias get this` and match the unittests. This is a *very* different thing from an `Optional` type! An `Optional` type is expected to be unset in normal operation. (That's why my implementation throws an Exception on access, not an Error.) That is, the user is expected to regularly encounter `Optional`s whose value is absent, and handle this state in his ordinary control flow. Hence no implicit alias, and no AssertError on access. I too, would prefer to get `Nullable` into a state where it can serve as an Optional type. However, given the amount of forum interest in this has been an approximate **zero**, I've chosen to take any and every avenue that may at some point lead to a sane Optional type in the standard library. I am **sick** of spending hours debugging `alias get this` issues, or being told that a coworker spent hours debugging an `alias get this` issue. Whatever gets me rid of them, I will do. I don't care which of these PRs gets attention. The `Optional` PR does not obsolete the `deprecate alias get this` PR. If either of them gets traction, I will be glad of it.
Jun 11 2019
Am 11.06.19 um 13:47 schrieb FeepingCreature:My impression is that the distinction turns on whether the null case can be expected to occur in "normal use" of the type. As `Nullable` says, accessing `get` while the `Nullable` `isNull` is *undefined*; that is, at the level of accessing missing array keys. (Except you can access them implicitly.) From this I infer that `Nullable` is mostly supposed to be non-null, maybe to enable delayed initialization or the like, which would justify the `alias get this` and match the unittests. This is a *very* different thing from an `Optional` type! An `Optional` type is expected to be unset in normal operation. (That's why my implementation throws an Exception on access, not an Error.) That is, the user is expected to regularly encounter `Optional`s whose value is absent, and handle this state in his ordinary control flow. Hence no implicit alias, and no AssertError on access.Exactly. This is why I'd expect a "real" Optional type to provide a lot more utilities to actually work with it, such as pattern matching, orElse, iterating, mapping, flatMapping, filtering, etc.
Jun 11 2019