www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to use exceptions

reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
Dear d-lang experts,

lets say in general I am quite happy with exceptions.
Recently though I stumbled upon two problems with them:
1. Its quite simple to loose valuable information
2. Its hard to present the exception messages to end users of your
    program

Let me elaborate on those:
Lets take a simple config parser example (pseudocode, the real code is 
in the end):
```d
auto parseConfig(string filename)
{
     return s
         .readText
         .parseJSON;
}
..
void main()
{
     ...
     auto config = parseConfig("config.json");
     run(config);
     ...
}
```
Lets look at what we would get in terms of error messages for the user 
of this program (besides that a full stacktrace is printed, which is 
nice for the developer of the program, but perhaps not so nice for the 
end user)
- file is not there: config.json: No such file or directory
   nice ... its almost human readable and one could guess that something
   with the config file is amiss
- file is not readable for the user: config.json: Permission denied
   nice ... quite a good explanation as well
- file with invalid UTF-8: Invalid UTF-8 sequence (at index 1)
   not so nice ... something is wrong with some UTF-8 in the program,
   but where and in which area of the program
- file with broken json: Illegal control character. (Line 4:0)
   not nice ... line of which file, illegal control character does
   not even sound like json processing.

Arguably readText could behave a little better and not throw away
information about the file its working with, but for parseJSON the
problem is "bigger" as it is only working with a string in memory not
with a file anymore, so it really needs some help of the application
developer I would say. When browsing through phobos I stumbled upon the
genius std.exception.ifThrown function, that allows for very nice
fallbacks in case of recoverable exceptions. Building on that I came up
with the idea (parhaps similar to Rusts error contexts) to use this
mechanism to wrap the exceptions in better explaining exceptions. This
would allow to provide exceptions with the information that might
otherwise be lost aswell as lift the error messages onto a end user 
consumable level (at least if only the topmost exception message is 
looked at).


```d

import std;

auto contextWithString(T)(lazy scope T expression, string s)
{
     try
     {
         return expression();
     }
     catch (Exception e)
     {
         throw new Exception("%s\n%s".format(s, e.msg));
     }
     assert(false);
}

auto contextWithException(T)(lazy scope T expression, Exception 
delegate(Exception) handler)
{
     Exception newException;
     try
     {
         return expression();
     }
     catch (Exception e)
     {
         newException = handler(e);
     }
     throw newException;
}

// plain version, no special error handling
JSONValue readConfig1(string s)
{
     // dfmt off
     return s
         .readText
         .parseJSON;
     // dfmt.on
}

// wraps all exceptions with a description whats going on
JSONValue readConfig2(string s)
{
     // dfmt off
     return s
         .readText
         .parseJSON
         .contextWithString("Cannot process config file %s".format(s));
     // dfmt on
}

// tries to deduplicate the filename from the exception messages
// but misses on utf8 errors
JSONValue readConfig3(string s)
{
     // dfmt off
     auto t = s
         .readText;
     return t
         .parseJSON
         .contextWithString("Cannot process config file %s".format(s));
     // dfmt on
}

// same as 3 just different api
JSONValue readConfig4(string s)
{
     // dfmt off
     auto t = s
         .readText;
     return t
         .parseJSON
         .contextWithException((Exception e) {
             return new Exception("Cannot process config file%s\n 
%s".format(s, e.msg));
         });
     // dfmt on
}

void main()
{
     foreach (file; [
         "normal.txt",
         "missing.txt",
         "broken_json.txt",
         "not_readable.txt",
         "invalid_utf8.txt",
     ])
     {
 
writeln("=========================================================================");
         size_t idx = 0;
         foreach (kv; [
             tuple("readConfig1", &readConfig1),
	    tuple("readConfig2", &readConfig2),
	    tuple("readConfig3", &readConfig3),
             tuple("readConfig4", &readConfig4),
         ])
         {
             auto f = kv[1];
             try
             {
                 if (idx++ > 0) 
writeln("-------------------------------------------------------------------------");
                 writeln("Working on ", file, " with ", kv[0]);
                 f("testfiles/%s".format(file));
             }
             catch (Exception e)
             {
                 writeln(e.msg);
             }
         }
     }
}
```

What do you guys think about that?
Full dub project available at 
git github.com:gizmomogwai/d-exceptions.git ... If you want to play with 
it, please run ./setup-files.sh first (which will create a file that is 
not readable).

One thing I am especially interested in would be how to annotate only
one part of a callchain with additional error information.
e.g. given the chain
```d
     someObject
          .method1
          .method2
          .method3
```
I would like to have the options to only add additional information to 
`method2` calls, without having to introduce intermediate variables (and
breaking the chain with it).

Also is there a way to do this kind of error handling without exceptions
(I think its hard to do this with optionals as they get verbose without
a special quick return feature).


Thanks in advance,
Christian
Aug 11 2022
parent reply Adam D Ruppe <destructionator gmail.com> writes:
You might find my recent blog post interesting too:

http://dpldocs.info/this-week-in-d/Blog.Posted_2022_08_01.html#exception-template-concept

and a draft of some more concepts:
http://arsd-official.dpldocs.info/source/arsd.exception.d.html


I also find the lack of information disturbing, but I also hate 
formatting it all into strings, so I wanted to experiment with 
other options there that avoid the strings.
Aug 11 2022
next sibling parent =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 12.08.22 01:06, Adam D Ruppe wrote:

 You might find my recent blog post interesting too:
 
 http://dpldocs.info/this-week-in-d/Blog.Posted_2022_08_01.html#exception-template-concept
 
 and a draft of some more concepts:
 http://arsd-official.dpldocs.info/source/arsd.exception.d.html
 
 
 I also find the lack of information disturbing, but I also hate 
 formatting it all into strings, so I wanted to experiment with other 
 options there that avoid the strings.


Yes ... I already read your blog post. In general I am with you there as
it makes sense to have either dedicated exception classes or something 
more along the lines of what you describe in your post.
Aug 11 2022
prev sibling parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Thu, Aug 11, 2022 at 11:06:45PM +0000, Adam D Ruppe via Digitalmars-d-learn
wrote:

 You might find my recent blog post interesting too:
 
 http://dpldocs.info/this-week-in-d/Blog.Posted_2022_08_01.html#exception-template-concept
 
 and a draft of some more concepts:
 http://arsd-official.dpldocs.info/source/arsd.exception.d.html
 
 
 I also find the lack of information disturbing, but I also hate
 formatting it all into strings, so I wanted to experiment with other
 options there that avoid the strings.


I think the OP's idea is somewhat different: adding contextual
information to a propagating exception that the throwing code may not
have access to.

I've often encountered this situation: for example, I have a program
that's driven by script, and contains a bunch of code that does
different computations which are ultimately called by the script parser.
There may be an error deep inside a computation, e.g., a vector was zero
where it shouldn't be, for example, and this may be nested inside some
deep expression tree walker. So it throws an exception.  But the
resulting message is unhelpful: it merely says some obscure vector is
unexpectedly zero, but not where the error occurred, because the
expression walk code doesn't know where in the script it is. And it
*shouldn't* know -- the filename/line of a script is completely
orthogonal to evaluating expressions; for all it knows, it could be
invoked from somewhere else *not* from the input script, in which case
it would be meaningless to try to associate a filename/line with the
exception.  Expression evaluation code should be decoupled from parser
code; it shouldn't need to know about things only the parser knows.

And besides, if the expression code wasn't invoked from the parser, the
exception shouldn't include filename/line information where there isn't
any. It should be the parser that tacks on this information when the
exception propagates up the call stack from the lower-level code. In
fact, the exception should acquire *different* contextual information on
its way up the call stack, depending on what triggered the upper-level
call. If the expression evaluation was invoked, say, by network code,
then the exception when it arrives at the catch block ought to carry
network source IP information, for example.  If it was invoked by
simulation code, then it should carry information about the current
state of the simulation.

In the other direction, if the expression evaluator code calls, say,
std.conv.to at some point, and .to throws a conversion error, then the
outgoing exception should carry some information about where in the
exception the problem happened. There is no way std.conv could know
about this information (and it shouldn't know anyway); the expression
code should be the one tacking this information on.  And as the
exception propagates upwards, it should gather more higher-level
contextual information, each coming from its respective level of
abstraction.

The OP's idea of wrapping throwing code with a function that tacks on
extra information is a good idea.  Perhaps the use of strings isn't
ideal, but in principle I like his idea of exceptions acquiring
higher-level information as it propagates up the call stack.


T

-- 
PNP = Plug 'N' Pray
Aug 11 2022
next sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Thursday, 11 August 2022 at 23:50:58 UTC, H. S. Teoh wrote:

 I think the OP's idea is somewhat different: adding contextual 
 information to a propagating exception that the throwing code 
 may not have access to.


Yeah, but you can use the mechanism again: you'd catch the one 
then throw a new one with the old one tacked on the back.


 The OP's idea of wrapping throwing code with a function that 
 tacks on extra information is a good idea.


Yeah, that is good. I also kinda wish that scope(failure) could 
do it so you could tack on info with a more convenient syntax... 
i have some more wild ideas brewing now lol
Aug 11 2022
parent reply "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Aug 12, 2022 at 12:12:13AM +0000, Adam D Ruppe via Digitalmars-d-learn
wrote:

 On Thursday, 11 August 2022 at 23:50:58 UTC, H. S. Teoh wrote:

 I think the OP's idea is somewhat different: adding contextual
 information to a propagating exception that the throwing code may
 not have access to.


 
 Yeah, but you can use the mechanism again: you'd catch the one then
 throw a new one with the old one tacked on the back.


True.  But having to insert try/catch syntax wrapping around every
potential abstraction level that might want to add contextual
information is a pain.  If we could use a wrapper that can be inserted
at strategic entry points, that'd be much better.



 The OP's idea of wrapping throwing code with a function that tacks
 on extra information is a good idea.


 
 Yeah, that is good. I also kinda wish that scope(failure) could do it
 so you could tack on info with a more convenient syntax... i have some
 more wild ideas brewing now lol


Hmm! That gets me thinking.  Maybe something like this?

	// Totally fantastical, hypothetical syntax :P
	auto myFunc(Args...)(Args args) {
		int additionalInfo = 123;
		scope(failure, additionalInfo);

		return runUnreliableOperation(args);
	}


T

-- 
"No, John.  I want formats that are actually useful, rather than over-featured
megaliths that address all questions by piling on ridiculous internal links in
forms which are hideously over-complex." -- Simon St. Laurent on xml-dev
Aug 11 2022
parent Adam D Ruppe <destructionator gmail.com> writes:
On Friday, 12 August 2022 at 00:40:48 UTC, H. S. Teoh wrote:

 Hmm! That gets me thinking.  Maybe something like this?


aye. and scopes share dtors which means we can do it in the lib 
today:

---
struct AdditionalInfo {
         static string[] info;

         this(string info) {
                 AdditionalInfo.info ~= info;
         }

         ~this() {
                 AdditionalInfo.info = AdditionalInfo.info[0 .. $ 
- 1];

         }

          disable this(this);
}

class AdditionalInfoException : Exception {
         this(string t) {
                 import std.string;
                 super(t ~ "\n" ~ AdditionalInfo.info.join(" "));
         }
}

void bar() {
         with(AdditionalInfo("zone 1")) {
                 with(AdditionalInfo("zone 2")) {
                 }

                 throw new AdditionalInfoException("info");
         }
}

void main() {
         bar();
}
---

the throw site needs to cooperate with this but you could still 
try/catch an operation as a whole too and attach the original 
exception in a new one.

needs a bit more thought but this might work. biggest problem is 
still being stringly typed, ugh.

with compiler help tho we could possibly attach info on function 
levels right in the EH metadata, so it looks it up as it does the 
stack trace generation. but that would be limited to per-function 
i believe... but eh there's nested functions.

tbh i think op's delegate is a better plan at this point but 
still my brain is running some concepts.
Aug 11 2022
prev sibling parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 12.08.22 01:50, H. S. Teoh wrote:

 ...

 The OP's idea of wrapping throwing code with a function that tacks on
 extra information is a good idea.  Perhaps the use of strings isn't
 ideal, but in principle I like his idea of exceptions acquiring
 higher-level information as it propagates up the call stack.


Thanks for the kind words ... I am still thinking how to put this
into a nicer API ... as it is in my current demo, its not much to type
out so I am happy with that. Actually having read the source of
ifThrown, its amazing how lazy makes wrapping easy!
Aug 12 2022
parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 12.08.22 23:05, Christian Köstlin wrote:

 On 12.08.22 01:50, H. S. Teoh wrote:

 ...


  >

 The OP's idea of wrapping throwing code with a function that tacks on
 extra information is a good idea.  Perhaps the use of strings isn't
 ideal, but in principle I like his idea of exceptions acquiring
 higher-level information as it propagates up the call stack.


 
 Thanks for the kind words ... I am still thinking how to put this
 into a nicer API ... as it is in my current demo, its not much to type
 out so I am happy with that. Actually having read the source of
 ifThrown, its amazing how lazy makes wrapping easy!


One thing that can be done is to templateize the exception handler so
that only exceptions of a certain type are handled.
```d
auto contextWithException(T, E)(lazy scope T expression, Exception 
delegate(E) handler)
{
     Exception newException;
     try
     {
         return expression();
     }
     catch (E e)
     {
         newException = handler(e);
     }
     throw newException;
}
```

which would enable something like

```d
     return  s
         .readText
         .parseJSON
         .contextWithException((UTFException e) {
             return new Exception("Cannot process UTF-8 in config 
file%s\n  %s".format(s, e.msg), e);
         })
         .contextWithException((FileException e) {
             return new Exception("Cannot process config file%s\n 
%s".format(s, e.msg), e);
         });
```

Not sure if that makes it any better though, as I have the feeling that
most exceptions need to be wrapped at least once on their way to the end
user.

kind regards,
Christian
Aug 12 2022
parent reply kdevel <kdevel vogtner.de> writes:
On Friday, 12 August 2022 at 21:41:25 UTC, Christian Köstlin 
wrote:


 which would enable something like

 ```d
     return  s
         .readText
         .parseJSON
         .contextWithException((UTFException e) {
             return new Exception("Cannot process UTF-8 in 
 config file%s\n  %s".format(s, e.msg), e);
         })
         .contextWithException((FileException e) {
             return new Exception("Cannot process config 
 file%s\n %s".format(s, e.msg), e);
         });
 ```


This is not as DRY as it could be. Furthermore I would try 
implement the error handling completely outside the main 
execution path, ideally in a wrapper around a the old main 
function (renamed to main_). This approach becomes problematic if 
exceptions of the same class can be thrown from two functions of 
the chain.

Your code is printing e.msg. How to you localize that string?
Aug 13 2022
parent reply =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 13.08.22 15:00, kdevel wrote:

 On Friday, 12 August 2022 at 21:41:25 UTC, Christian Köstlin wrote:
 

 which would enable something like

 ```d
     return  s
         .readText
         .parseJSON
         .contextWithException((UTFException e) {
             return new Exception("Cannot process UTF-8 in config 
 file%s\n  %s".format(s, e.msg), e);
         })
         .contextWithException((FileException e) {
             return new Exception("Cannot process config file%s\n 
 %s".format(s, e.msg), e);
         });
 ```


 
 This is not as DRY as it could be. Furthermore I would try implement the 
 error handling completely outside the main execution path, ideally in a 
 wrapper around a the old main function (renamed to main_). This approach 
 becomes problematic if exceptions of the same class can be thrown from 
 two functions of the chain. >
 Your code is printing e.msg. How to you localize that string?


Those 3 points are exactly touching the problems (I am ignoring the DRY 
thing for now, as this is a side battle):
1. error handling in main path: exactly thats what I would like todo. 
but for that the errors that raise need to have meaningful information. 
this is exactly what I am trying in those context* functions ... they do 
not do error handling, but more something like error enhancement (by 
fixing up the error information).
2. yes ... for that the chain would need to be broken up, then you can 
react on the same exception classes of different members of the chain 
differently ... for that I do not see a nice way to write it in d.
3. localization is a fantastic example of data that needs to be added to 
almost every exception that raises from below. my example with the 
strings is just an example, it could also be that by conventions of your 
application framework all context* functions raise 
LocalizableExceptions, that somehow can then be mapped to different 
languages with their own api. the generalalized main function then would 
handle those exception classes differently from "normal" exceptions.

hope that makes sense.

kind regards,
christian
Aug 13 2022
parent reply kdevel <kdevel vogtner.de> writes:
On Saturday, 13 August 2022 at 13:36:08 UTC, Christian Köstlin 
wrote:

 [...]
 1. error handling in main path: exactly thats what I would like 
 todo. but for that the errors that raise need to have 
 meaningful information. this is exactly what I am trying in 
 those context* functions ... they do not do error handling, but 
 more something like error enhancement (by fixing up the error 
 information).


"Exception enrichment" would be my wording which is supported by 
google [1].
There is also the notion of "exception context" [2] and 
"contexted exception" [3].


 2. yes ... for that the chain would need to be broken up, then 
 you can react on the same exception classes of different 
 members of the chain differently ... for that I do not see a 
 nice way to write it in d.


The current filename imposes a context on its processing. The 
exception thrown from readText knows that filename and propagates 
it within the exception msg. Unfortunately exceptions are 
frequently only "stringly typed", e.g.

394: throw new JSONException("JSONValue is not a number type");
406: throw new JSONException("JSONValue is not a an integral 
type");

This should have been made into an exception hierarchy with the 
faulty value beeing a data member of the respective class. That 
would nicely support the crafting of a message for the user in 
the presentation layer.

Of course the UTF8Exception must be enriched with the filename in 
the application code.


 3. localization is a fantastic example of data that needs to be 
 added to almost every exception that raises from below.


How would you handle a FileException.msg = "myconfig.cnf: 
Permission denied"? Run regular expressions over the msg? 
Something is fundamentally wrong if any kind of parsing of error 
strings is required.

Then there is another type of user who shall not be faced with 
any kind of specific error message: The client of a company which 
runs a web application. This case is easy to implement: Catch any 

message. Leave the code as it is and let the stack trace go into 
the error_log. The devops people of the company probably 
understand english messages.


[1] 
https://jenkov.com/tutorials/java-exception-handling/exception-enrichment.html

[2] 
https://docs.microsoft.com/en-us/dotnet/api/system.web.mvc.exceptioncontext?view=aspnet-mvc-5.2

[3] 
https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/exception/ContextedException.html
Aug 13 2022
parent =?UTF-8?Q?Christian_K=c3=b6stlin?= <christian.koestlin gmail.com> writes:
On 13.08.22 17:00, kdevel wrote:

 "Exception enrichment" would be my wording which is supported by google 
 [1].
 There is also the notion of "exception context" [2] and "contexted 
 exception" [3].


Thats really a good word! Especially it describes better what the java
guys are doing by adding information to an existing object and
rethrowing, without creating a new exception, covering the tracks of the
original exception by wrapping it and throwing this.
Can we have that for dlang exceptions as well (I mean as additional api
in the baseclass)? It would not break old code, as it would be
additional api?


 The current filename imposes a context on its processing. The exception 
 thrown from readText knows that filename and propagates it within the 
 exception msg. Unfortunately exceptions are frequently only "stringly 
 typed", e.g.


yes .. thats a shame. I think it is, because its much shorter than
defining a new member, a new constructor, a new message method in the
new exception subclass.


 
 394: throw new JSONException("JSONValue is not a number type");
 406: throw new JSONException("JSONValue is not a an integral type");
 
 This should have been made into an exception hierarchy with the faulty 
 value beeing a data member of the respective class. That would nicely 
 support the crafting of a message for the user in the presentation layer.


I do agree.


 Of course the UTF8Exception must be enriched with the filename in the 
 application code.


I do not agree. As for me it looks like `readText` works with kind of
two API's internally. One is the file API that can throw, and forwards
information nicely (about which file has the problem). And then some
UTF-8 processing based on data from that file (which should also forward
the information of the file, as the end user of this API did not even
touch a file at all (only a string representing a filename)).
But for sure there are different scenarios where the enrichment needs to
happen at call site.


 3. localization is a fantastic example of data that needs to be added 
 to almost every exception that raises from below.


 
 How would you handle a FileException.msg = "myconfig.cnf: Permission 
 denied"? Run regular expressions over the msg? Something is 
 fundamentally wrong if any kind of parsing of error strings is required.


Thats what I mean with "that raises from below". I think the
FileException is not supposed to be on the top of the exception stack,
but a more application level exception (e.g. Cannot process
configuration file (as Configuration file is something on application
level)).


 Then there is another type of user who shall not be faced with any kind 
 of specific error message: The client of a company which runs a web 
 application. This case is easy to implement: Catch any Exception and 

 as it is and let the stack trace go into the error_log. The devops 
 people of the company probably understand english messages.


One could argue, that every exception stacktrace that comes to
stdout/-err or a user interface is already a bug. It might be helpful
for developers and system administrators, but not for end users. So your
solution is perhaps inconvinient for developers, but good in general.



 [1] 
 https://jenkov.com/tutorials/java-exception-handling/exception-enrichment.html
 
 [2] 
 https://docs.microsoft.com/en-us/dotnet/api/system.web.mvc.exceptioncontext?view=aspnet-mvc-5.2
 
 [3] 
 https://commons.apache.org/proper/commons-lang/apidocs/org/apache/commons/lang3/exception/ContextedException.html


thanks again for the links ... nice read!!!

Kind regards,
Christian
Aug 13 2022