www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - expectations 0.1.0

reply Paul Backus <snarwin gmail.com> writes:
expectations is an error-handling library that lets you bundle 
exceptions together with return values. It is based on Rust's 
Result<T, E> [1] and C++'s proposed std::expected. [2] If you're 
not familiar with those, Andrei's NDC Oslo talk, "Expect the 
Expected" [3], explains the advantages of this approach to error 
handling in considerable detail.

Features:
- `Expected` values can be treated as either return codes or 
exceptions.
- Functions that return `Expected` values can be composed easily 
using a monadic interface (`andThen`).
- `Expected!void` is valid and (hopefully) works the way you'd 
expect.
- Everything, except for `opEquals` (which depends on 
`Object.opEquals`), works in  safe code.

This is very much a work in progress; all comments and feedback 
are welcome.

Documentation: 
https://pbackus.github.io/expectations/expectations.html
DUB: https://code.dlang.org/packages/expectations
Code: https://github.com/pbackus/expectations

[1] https://doc.rust-lang.org/std/result/enum.Result.html
[2] https://wg21.link/p0323r7
[3] https://www.youtube.com/watch?v=nVzgkepAg5Y
Sep 01 2018
next sibling parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:

 expectations is an error-handling library that lets you bundle 
 exceptions together with return values. It is based on Rust's 
 Result<T, E> [1] and C++'s proposed std::expected. [2] If 
 you're not familiar with those, Andrei's NDC Oslo talk, "Expect 
 the Expected" [3], explains the advantages of this approach to 
 error handling in considerable detail.


Incidentally, I've already proposed `Expected` into Phobos 
std.experimental.typecons here

https://github.com/dlang/phobos/pull/6686

Is it ok if I try to merge your effort into this pull request?
Sep 02 2018
parent Paul Backus <snarwin gmail.com> writes:
On Sunday, 2 September 2018 at 23:38:41 UTC, Per Nordlöw wrote:

 On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:

 expectations is an error-handling library that lets you bundle 
 exceptions together with return values. It is based on Rust's 
 Result<T, E> [1] and C++'s proposed std::expected. [2] If 
 you're not familiar with those, Andrei's NDC Oslo talk, 
 "Expect the Expected" [3], explains the advantages of this 
 approach to error handling in considerable detail.


 Incidentally, I've already proposed `Expected` into Phobos 
 std.experimental.typecons here

 https://github.com/dlang/phobos/pull/6686

 Is it ok if I try to merge your effort into this pull request?


I don't think expectations has reached a high enough level of 
quality yet to be ready for inclusion in Phobos. However, if 
you'd like to submit it for comments as a WIP, or use it as a 
reference for your own implementation, that's completely fine, 
and I'd be happy to help you.
Sep 02 2018
prev sibling next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:

 expectations is an error-handling library that lets you bundle 
 exceptions together with return values. It is based on Rust's 
 Result<T, E> [1] and C++'s proposed std::expected. [2] If 
 you're not familiar with those, Andrei's NDC Oslo talk, "Expect 
 the Expected" [3], explains the advantages of this approach to 
 error handling in considerable detail.


Sorry, I didn't watch the talk, but this sounds like something 
that's been on my mind for a while.

There are generally two classic approaches to error handling:

- Error codes. Plus: no overhead. Minus: you need to remember to 
check them. Some languages force you to check them, but it 
results in very noisy code in some cases (e.g. 
https://stackoverflow.com/a/3539342/21501).

- Exceptions. Plus: simple to use. Minus: unnecessary (and 
sometimes considerable) overhead when failure is not exceptional.

Now, Rust's Result works because it forces you to check the error 
code, and provides some syntax sugar to pass the result up the 
stack or abort if an error occurred. D, however, has nothing to 
force checking the return value of a function (except for pure 
functions, which is inapplicable for things like I/O).

Please correct me if I'm wrong, but from looking at the code, 
given e.g.:

Expected!void copyFile(string from, string to);

nothing prevents me from writing:

void main() { copyFile("nonexistent", "target"); }

The success value is silently discarded, so we end up with a "ON 
ERROR RESUME NEXT" situation again, like badly written C code.

One way we could improve on this in theory is to let functions 
return a successfulness value, which is converted into a thrown 
exception IFF the function failed AND the caller didn't check if 
an error occurred.

Draft implementation:

struct Success(E : Exception)
{
	private E _exception;
	private bool checked = false;
	 property E exception() { checked = true; return _exception; }
	 property ok() { return exception is null; }
	 disable this(this);
	~this() { if (_exception && !checked) throw _exception; }
}
Success!E failure(E)(E e) { return Success!E(e); }

Success!Exception copyFile(string from, string to)
{
	// dummy
	if (from == "nonexistent")
		return failure(new Exception("EEXIST"));
	else
		return typeof(return)();
}

void main()
{
	import std.exception;

	copyFile("existent", "target");
	assert(!copyFile("nonexistent", "target").ok);
	assertThrown!Exception({ copyFile("nonexistent", "target"); }());
}

This combines some of the advantages of the two approaches above. 
In the above draft I used a real exception object for the 
payload, constructing which still has a significant overhead (at 
least over an error code), though we do get rid of a try/catch if 
an error is not an exceptional situation. The advantage of using 
a real exception object is that its stack trace is generated when 
the exception is instantiated, and not when it's thrown, which 
means that the error location inside the copyFile implementation 
is recorded; but the same general idea would work with a 
numerical error code payload too.

Any thoughts?
Sep 02 2018
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev 
wrote:

 Please correct me if I'm wrong, but from looking at the code, 
 given e.g.:

 Expected!void copyFile(string from, string to);

 nothing prevents me from writing:

 void main() { copyFile("nonexistent", "target"); }

 The success value is silently discarded, so we end up with a 
 "ON ERROR RESUME NEXT" situation again, like badly written C 
 code.


This is definitely a big weakness of `Expected!void`, and one I 
hadn't considered when writing the code. With a normal 
`Expected!T`, the fact that you care about the return value is 
what forces you to check for the error, but that doesn't apply 
when the return value is `void`.

I'm not sure at this point if it's better to leave 
`Expected!void` in for the sake of completeness, or remove it so 
that nobody's tempted to shoot themself in the foot. Definitely 
something to think about.


 One way we could improve on this in theory is to let functions 
 return a successfulness value, which is converted into a thrown 
 exception IFF the function failed AND the caller didn't check 
 if an error occurred.

 Draft implementation:

 struct Success(E : Exception)
 {
 	private E _exception;
 	private bool checked = false;
 	 property E exception() { checked = true; return _exception; }
 	 property ok() { return exception is null; }
 	 disable this(this);
 	~this() { if (_exception && !checked) throw _exception; }
 }


This is a really clever technique. As you said, hard to say 
whether it's worth it compared to just throwing an exception 
normally, but still, really clever.
Sep 02 2018
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 09/02/2018 11:23 PM, Paul Backus wrote:

 
 This is a really clever technique. As you said, hard to say whether it's 
 worth it compared to just throwing an exception normally, but still, 
 really clever.


IMO, it's worth it. First of all, it decreases the asymmetry between 
`Expected!void` and other `Expected!T`. But more than that, there's one 
of the core benefits of of expected:

What's awesome about expected is that by providing only one function, 
the caller can decide whether they want a `foo()` that throws, or a 
`tryFoo()` that lets them manually handle the case where it doesn't work 
(and is potentially nothrow).

Note that the above has *nothing* to do with retrieving a value. 
Retrieving a value is merely used by the implementation as a trigger to 
lazily decide whether the caller wants `foo` or `tryFoo`. Going out of 
scope without making the choice could also be considered another trigger 
point. In fact, this "out-of-scope without being checked" could even be 
used as an additional trigger for even the non-void variety. After all: 
what if an error occurs, but the caller checks *neither* value nor hasValue?

There's only one possible downside I see:

What if the caller *intentionally* wants to ignore the error condition? 
Yes, that's generally bad practice, and signifies maybe it shouldn't be 
an exception in the first place. But consider Scriptlike: It has 
functions like `tryMkdir` and `tryRmdir` with the deliberate purpose 
letting people say "Unlike Phobos's mkdir/rmdir, I don't care whether 
the directory already exists or not, just MAKE SURE it exists (or 
doesn't) and don't bother me with the details!"

I suppose for cases like those, it's perfectly worth leaving it up to 
expectation's user to design, create and document a "Don't worry about 
the failure" variant, should they so choose. Probably safer that way, 
anyway.
Sep 02 2018
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 3 September 2018 at 04:49:40 UTC, Nick Sabalausky 
(Abscissa) wrote:

 Note that the above has *nothing* to do with retrieving a 
 value. Retrieving a value is merely used by the implementation 
 as a trigger to lazily decide whether the caller wants `foo` or 
 `tryFoo`. Going out of scope without making the choice could 
 also be considered another trigger point. In fact, this 
 "out-of-scope without being checked" could even be used as an 
 additional trigger for even the non-void variety. After all: 
 what if an error occurs, but the caller checks *neither* value 
 nor hasValue?


The thing is, triggering on explicit access lets you handle 
errors lazily, whereas triggering at the end of the scope forces 
you to handle them eagerly.

Vladimir's `Success` type is, essentially, a way for a function 
to send something back up the stack that its caller is forced to 
acknowledge. Throwing an exception is *also* a way for a function 
to send something back up the stack that its caller is forced to 
acknowledge. The exact details are different, but when it comes 
to overall control-flow semantics, they are basically equivalent.

By contrast, a function that returns an `Expected!T` does *not* 
force its caller to acknowledge it. If an error occurs, and the 
caller never checks value or hasValue...nothing happens. That's 
what being lazy means: if you never open the box, it doesn't 
matter whether the cat is alive or dead.

The problem, when it comes to `Expected!void`, is that there's no 
good way to express what we *actually* care about--the function's 
side effect--as a value. If we could write `Expected!(IO!Unit)` 
like in Haskell, everything would be fine.

To me, the only acceptable choices are for `Expected!void` to 
have the same lazy semantics as `Expected!T`, or for 
`Expected!void` to be removed altogether. Having one 
specialization be lazy and one be eager would be a nightmare for 
anyone trying to use the library. For the reasons Vladimir 
brought up, I'm leaning toward removal--without something like 

likely to do more harm than good.
Sep 02 2018
next sibling parent aliak <something something.com> writes:
On Monday, 3 September 2018 at 06:49:41 UTC, Paul Backus wrote:

 To me, the only acceptable choices are for `Expected!void` to 
 have the same lazy semantics as `Expected!T`, or for 
 `Expected!void` to be removed altogether. Having one 
 specialization be lazy and one be eager would be a nightmare 
 for anyone trying to use the library. For the reasons Vladimir 
 brought up, I'm leaning toward removal--without something like 

 is likely to do more harm than good.


I'm leaning on agreeing with removal of Expected!void as well

When we get opPostMove then maybe Expected!void can throw on a 
move or a copy if the result was a failure. This would also not 
allow the error to be ignored as it'd throw.

Or... can it throw in ~this() if it was not checked?
Sep 03 2018
prev sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 09/03/2018 02:49 AM, Paul Backus wrote:

 On Monday, 3 September 2018 at 04:49:40 UTC, Nick Sabalausky (Abscissa) 
 wrote:

 Note that the above has *nothing* to do with retrieving a value. 
 Retrieving a value is merely used by the implementation as a trigger 
 to lazily decide whether the caller wants `foo` or `tryFoo`. Going out 
 of scope without making the choice could also be considered another 
 trigger point. In fact, this "out-of-scope without being checked" 
 could even be used as an additional trigger for even the non-void 
 variety. After all: what if an error occurs, but the caller checks 
 *neither* value nor hasValue?


 
 The thing is, triggering on explicit access lets you handle errors 
 lazily, whereas triggering at the end of the scope forces you to handle 
 them eagerly.
 
 Vladimir's `Success` type is, essentially, a way for a function to send 
 something back up the stack that its caller is forced to acknowledge.


Yes, that's correct.


 Throwing an exception is *also* a way for a function to send something 
 back up the stack that its caller is forced to acknowledge.


Yes, but it's heavier-weight AND prevents the callee from being nothrow.


 but when it comes to overall control-flow 
 semantics, they are basically equivalent.


Control-flow semantics, sure, but as I pointed out in my previous 
sentence, there's more relevant things involved here than just control 
flow semantics.


 By contrast, a function that returns an `Expected!T` does *not* force 
 its caller to acknowledge it. If an error occurs, and the caller never 
 checks value or hasValue...nothing happens.


That's called squelching an error, and its EXACTLY the same problem as 
using non-Expect return values to indicate errors. I'd regard that as 
very notable hole in any Expected design as it breaks one of the core 
points of using Expect vs returning plain error codes: The user can 
still accidentally (or deliberately) squelch an error.

To clarify: If the caller never checks value or hasValue, that does NOT 
mean the caller has carefully and deliberately chosen to disregard the 
error. It *could* mean that, but it could also mean they simply messed 
up. Deliberately squeching an error should NEVER be implicit, it should 
always require something like:

catch(...) { /+ Do nothing +/ }

or

if(!x.hasValue) { /+ Do nothing +/ }


 That's what being lazy 
 means: if you never open the box, it doesn't matter whether the cat is 
 alive or dead.


I don't see the laziness here as being the core point. The laziness is 
the "how", not the raison d'etre. The laziness is simply a tool being 
used to achieve the real goals of:

- Allowing the caller the decide between foo/tryFoo versions without the 
API duplication.

- Decreasing exception-related overhead and increasing utility of nothrow.


 
 Having one specialization be lazy and one be eager 
 would be a nightmare for anyone trying to use the library.


Vladimir's Success vs Expect!T is NOT an example of "eager vs lazy". In 
BOTH cases, the callee treats errors lazily. And in BOTH cases, the 
caller (or whomever the caller passes it off to) is expected to, at some 
point, make a deliberate, explicit choice between "handle or throw". And 
as I said before, allowing the caller to accidentally (or implicitly) 
squelch the error is a fundamental breakage in the whole point behind 
Except.
Sep 03 2018
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 3 September 2018 at 21:55:57 UTC, Nick Sabalausky 
(Abscissa) wrote:

 By contrast, a function that returns an `Expected!T` does 
 *not* force its caller to acknowledge it. If an error occurs, 
 and the caller never checks value or hasValue...nothing 
 happens.


 That's called squelching an error, and its EXACTLY the same 
 problem as using non-Expect return values to indicate errors. 
 I'd regard that as very notable hole in any Expected design as 
 it breaks one of the core points of using Expect vs returning 
 plain error codes: The user can still accidentally (or 
 deliberately) squelch an error.


If you receive an `Expected!T`, you have the following choices 
available to you:

1. Handle the success case locally, and the failure case 
non-locally (i.e. use `value` directly).
2. Handle both the success case and the failure case locally 
(i.e. check `hasValue`).
3. Handle both the success case and the failure case non-locally 
(i.e., pass the `Expected!T` along untouched).

The difference between `Expected`, on the one hand, and both 
`Success` and plain-old exceptions, on the other, is that 



success and failure follow the same code path. That makes 
functions that use `Expected` much easier to compose than ones 
that throw exceptions. For example, if you throw an exception in 
the middle of a range pipeline, the entire thing comes crashing 
down--but an `Expected!T` will pass right through, and let you 
handle it when it comes out the other end.

Now, you will probably object--and rightly so--that there is an 
implicit assumption being made here, which is that "handle the 
success case" is equivalent to "use the return value." Clearly, 
this equivalence does not always hold in the presence of side 
effects. That's why `Expected!void` is so problematic. 
Nevertheless, I think it holds in enough cases to make `Expected` 
useful in practice. In particular, it is guaranteed to hold for 
strongly-pure functions, and will also hold for functions whose 
side effects are visible only through the return value (e.g., 
`readln`).


 I don't see the laziness here as being the core point. The 
 laziness is the "how", not the raison d'etre. The laziness is 
 simply a tool being used to achieve the real goals of:

 - Allowing the caller the decide between foo/tryFoo versions 
 without the API duplication.

 - Decreasing exception-related overhead and increasing utility 
 of nothrow.


The laziness (on the part of the caller, i.e., the code that 
*receives* the `Expected!T`) is important because it's what makes
Sep 03 2018
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 09/04/2018 12:05 AM, Paul Backus wrote:

 On Monday, 3 September 2018 at 21:55:57 UTC, Nick Sabalausky (Abscissa) 
 wrote:

 By contrast, a function that returns an `Expected!T` does *not* force 
 its caller to acknowledge it. If an error occurs, and the caller 
 never checks value or hasValue...nothing happens.


 That's called squelching an error, and its EXACTLY the same problem as 
 using non-Expect return values to indicate errors. I'd regard that as 
 very notable hole in any Expected design as it breaks one of the core 
 points of using Expect vs returning plain error codes: The user can 
 still accidentally (or deliberately) squelch an error.


 
 If you receive an `Expected!T`, you have the following choices available 
 to you:
 
 1. Handle the success case locally, and the failure case non-locally 
 (i.e. use `value` directly).
 2. Handle both the success case and the failure case locally (i.e. check 
 `hasValue`).
 3. Handle both the success case and the failure case non-locally (i.e., 
 pass the `Expected!T` along untouched).
 
 The difference between `Expected`, on the one hand, and both `Success` 
 and plain-old exceptions, on the other, is that `Expected` gives you 

 


I think you may be getting hung up on a certain particular detail of 
Vladimir's exact "draft" implementation of Success, whereas I'm focusing 
more on Success's more general point of "Once the object is no longer 
around, guarantee the error doesn't get implicitly squelched."

You're right that, *in the draft implementation as-is*, it can be 
awkward for the caller to then pass the Success along to some other code 
(another function call, or something higher up the stack). *Although*, 

awkward...

But reference counting would be enough to fix that. (Or a 
compiler-supported custom datatype that's automatically pass-by-moving, 
but that's of course not something D has).

And you haven't actually directly addressed the issue I've raised about 
failing to guarantee errors aren't implicitly squelched.



 failure follow the same code path. That makes functions that use 
 `Expected` much easier to compose than ones that throw exceptions. For 
 example, if you throw an exception in the middle of a range pipeline, 
 the entire thing comes crashing down--but an `Expected!T` will pass 
 right through, and let you handle it when it comes out the other end.


Right. And as described above, I'm advocating an approach that preserves 
that (even for void) while *also* improving Expect so it can not 
*merely* improve things "in most cases", but would actually *guarantee* 
errors are not implicitly squelched in ALL cases where Expect!whatever 
is used.


 I don't see the laziness here as being the core point. The laziness is 
 the "how", not the raison d'etre. The laziness is simply a tool being 
 used to achieve the real goals of:

 - Allowing the caller the decide between foo/tryFoo versions without 
 the API duplication.

 - Decreasing exception-related overhead and increasing utility of 
 nothrow.


 
 The laziness (on the part of the caller, i.e., the code that *receives* 

 possible. It's an essential part of the design.


Again, what I'm proposing still preserves that.
Sep 04 2018
parent Paul Backus <snarwin gmail.com> writes:
On Tuesday, 4 September 2018 at 22:08:48 UTC, Nick Sabalausky 
(Abscissa) wrote:

 I think you may be getting hung up on a certain particular 
 detail of Vladimir's exact "draft" implementation of Success, 
 whereas I'm focusing more on Success's more general point of 
 "Once the object is no longer around, guarantee the error 
 doesn't get implicitly squelched."

 You're right that, *in the draft implementation as-is*, it can 
 be awkward for the caller to then pass the Success along to 
 some other code (another function call, or something higher up 

 eliminated, it's simply made awkward...

 But reference counting would be enough to fix that. (Or a 
 compiler-supported custom datatype that's automatically 
 pass-by-moving, but that's of course not something D has).


Ok, I think I understand what you're proposing now--basically, 

for taking the time to explain. I agree that that would be a nice 
feature for `Expected` to have.

The thing is, D already has a mechanism for signalling failures 
that can't be ignored: exceptions. So adding that functionality 
to `Expected`, while convenient, doesn't actually let you 
accomplish anything you couldn't already.

Now, if it were easy to implement, then sure, no problem. But 
it's not. Reference counting in particular is so problematic that 
Walter and Andrei have proposed *multiple* new language features 
(copy constructors, __mutable) to make it work cleanly. As things 
currently stand, making `Expected` reference-counted would mean 
at the very least giving up compatibility with `const` and 
`immutable`, which makes `Expected` a worse fit for strongly-pure 
functions (currently its *best* use-case).

It's a shame that D forces us to make this tradeoff, but given 
the options in front of me, I would rather have `Expected` shine 
in the area where it has a comparative advantage, even if that 
means making it less universally-applicable as an error-handling 
mechanism.
Sep 05 2018
prev sibling parent reply Thomas Mader <thomas.mader gmail.com> writes:
On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev 
wrote:

 There are generally two classic approaches to error handling:


std::expected is not the only thing on this topic going on in C++.
There is also the proposal from Herb Sutter [1].
It's not a library solution and changes even the ABI but it's an 
interesting approach.
He also tries to get compatibility into C via an extension. (See 
4.6.11 in [1])

[1] 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf
Sep 02 2018
parent reply aliak <something something.com> writes:
On Monday, 3 September 2018 at 06:00:06 UTC, Thomas Mader wrote:

 On Monday, 3 September 2018 at 00:52:39 UTC, Vladimir Panteleev 
 wrote:

 There are generally two classic approaches to error handling:


 std::expected is not the only thing on this topic going on in 
 C++.
 There is also the proposal from Herb Sutter [1].
 It's not a library solution and changes even the ABI but it's 
 an interesting approach.
 He also tries to get compatibility into C via an extension. 
 (See 4.6.11 in [1])

 [1] 
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf


This would be great to have in D. Swift [0] has something 
similar, and personally after using it for a few years, I can say 
that I've seen next to no unhandled exception errors in iOS code 
at least.

[0] 
https://www.mikeash.com/pyblog/friday-qa-2017-08-25-swift-error-handling-implementation.html
Sep 03 2018
parent Thomas Mader <thomas.mader gmail.com> writes:
On Monday, 3 September 2018 at 13:00:05 UTC, aliak wrote:

 This would be great to have in D.


Indeed, if it's really going into C++ D needs to think about how 
to handle that anyway if it wants to offer C++ ABI interfacing.


 Swift [0] has something similar, and personally after using it 
 for a few years, I can say that I've seen next to no unhandled 
 exception errors in iOS code at least.


Thanks, didn't know that Swift is already using something like 
this.
Sep 03 2018
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Sunday, 2 September 2018 at 06:59:20 UTC, Paul Backus wrote:

 expectations is an error-handling library that lets you bundle 
 exceptions together with return values. It is based on Rust's 
 Result<T, E> [1] and C++'s proposed std::expected. [2] If 
 you're not familiar with those, Andrei's NDC Oslo talk, "Expect 
 the Expected" [3], explains the advantages of this approach to 
 error handling in considerable detail.


expectations 0.2.0 is now available, with the following updates
- `hasValue`, `value`, and `exception` now work for const and 
immutable `Expected` objects.
- `Expected!void` has been removed.
- `map` and `andThen` can now be partially applied to functions, 
"lifting" them into the Expected monad.
- The documentation has been improved based on the feedback given 
in this thread.
Sep 04 2018