www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Ranges and Exception handling PR 2724

reply "Robert burner Schadek" <rburners gmail.com> writes:
This PR 
https://github.com/D-Programming-Language/phobos/pull/2724 adds 
an generic way of handling Exception in Range processing. 
quickfur and Dicebot ask me to start a thread here so the concept 
could be discussed.
Nov 14 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Nov 15, 2014 at 01:43:05AM +0000, Robert burner Schadek via
Digitalmars-d wrote:

 This PR https://github.com/D-Programming-Language/phobos/pull/2724
 adds an generic way of handling Exception in Range processing.
 quickfur and Dicebot ask me to start a thread here so the concept
 could be discussed.


Maybe you should give a more detail explanation here so that people know
what you're talking about.


From what I understand, this PR is proposing to add a range wrapper that


catches exceptions thrown from range primitives and passes them to a
user-specified handler. Seems to be a promising idea, but it's probably
better if people hash it out here first and work out the best API for
it, before we commit it to Phobos.


T

-- 
People say I'm indecisive, but I'm not sure about that. -- YHL, CONLANG
Nov 18 2014
parent reply "Robert burner Schadek" <rburners gmail.com> writes:
On Wednesday, 19 November 2014 at 05:49:55 UTC, H. S. Teoh via 
Digitalmars-d w

From what I understand, this PR is proposing to add a range 
wrapper that


 catches exceptions thrown from range primitives and passes them 
 to a
 user-specified handler. Seems to be a promising idea, but it's 
 probably


It is exactly that:

     auto s = "12,1337z32,54,2,7,9,1z,6,8";

     auto r = s.splitter(',')
         .map!(a => to!int(a))
         .handleBack!(ConvException, (e, r) => 0)
         .array;

     assert(equal(h, [12, 0, 54, 2, 7, 9, 0, 6, 8]));
Nov 20 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Nov 20, 2014 at 11:57:41AM +0000, Robert burner Schadek via
Digitalmars-d wrote:

 On Wednesday, 19 November 2014 at 05:49:55 UTC, H. S. Teoh via Digitalmars-d
 w

From what I understand, this PR is proposing to add a range wrapper
that catches exceptions thrown from range primitives and passes them
to a user-specified handler. Seems to be a promising idea, but it's
probably


 
 It is exactly that:
 
     auto s = "12,1337z32,54,2,7,9,1z,6,8";
 
     auto r = s.splitter(',')
         .map!(a => to!int(a))
         .handleBack!(ConvException, (e, r) => 0)
         .array;
 
     assert(equal(h, [12, 0, 54, 2, 7, 9, 0, 6, 8]));


Unfortunately, it looks like people are more interested in arguing about
signed vs. unsigned instead of reviewing new Phobos features. *sigh* :-(


T

-- 
PNP = Plug 'N' Pray
Nov 20 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
H. S. Teoh:


 Unfortunately, it looks like people are more interested in 
 arguing about
 signed vs. unsigned instead of reviewing new Phobos features. 
 *sigh* :-(


Both kind of discussions are important.

Regarding this Phobos feature, I suggested something less general 
and more efficient (no exceptions are involved):
https://issues.dlang.org/show_bug.cgi?id=6840

See also:
https://issues.dlang.org/show_bug.cgi?id=6843

Bye,
bearophile
Nov 20 2014
parent "Robert burner Schadek" <rburners gmail.com> writes:
hm, the thing is there are ranges that will throw, making them 
nothrow is of course a very good idea, but some will still throw 
map(a => throw ...)

This handleXXX ranges deal with them.
Nov 20 2014
prev sibling next sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Saturday, 15 November 2014 at 01:43:07 UTC, Robert burner 
Schadek wrote:

 This PR 
 https://github.com/D-Programming-Language/phobos/pull/2724 adds 
 an generic way of handling Exception in Range processing. 
 quickfur and Dicebot ask me to start a thread here so the 
 concept could be discussed.


It's a small thing, but it might be nice if it were possible to 
provide a handler that takes only the exception as an argument, 
or maybe even just a default value with no arguments at all.  I 
like the general idea though.
Nov 20 2014
prev sibling next sibling parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Saturday, 15 November 2014 at 01:43:07 UTC, Robert burner 
Schadek wrote:

 This PR 
 https://github.com/D-Programming-Language/phobos/pull/2724 adds 
 an generic way of handling Exception in Range processing. 
 quickfur and Dicebot ask me to start a thread here so the 
 concept could be discussed.


I actually ran into this problem today when using the dirEntries 
function in std.file.  I was attempting to iterate all the files 
on my C drive and I got an Access Denied error which caused the 
DirIterator to throw an exception.  There's nothing I could do to 
catch the exception and continue.  I'm very glad people are aware 
of this problem and I'm glad you are trying to do something about 
it.

Correct me if I'm wrong, but it appears that the handleXXX 
methods allow the user to provide callback functions to "finish" 
the input range when an exception occurs.  "Finish" meaning it 
could restore the input range state or provide a whole new input 
range, whatever the user wants. Is this correct?  If so, I'm not 
sure how this could be used to solve the dirEntries case I ran 
into.  The logic to restore the DirIterator state after an 
exception could be quite complicated and error prone.

Maybe this is a naive solution but I thought I would share my 
idea to solve the dirEntries case in hopes it will help you solve 
the general case. The solution I thought of was to pass a 
callback function to dirEntries that would tell it how to handle 
errors.  An example of the callback could look like this:

// return true to throw an exception, false to skip the file and 
continue
alias FileErrorHandler = bool delegate(FileError error, 
const(char)[] filename) nothrow;

FileError could be an enum like:
enum FileError { accessDenied, ... }

So the dirEntries function could add an optional parameter

auto dirEntries(string path, SpanMode mode, bool followSymlink = 
true, FileErrorHandler errorHandler = null);

It looks "similar" to your solution with a key difference.  The 
InputRange is able to figure out how the error is suppose to be 
handled before it throws an exception and messes up any state it 
currently has (state including function stacks and the 
instruction pointer, etc).

I'm not sure how the API would look for the general case, but 
somehow the user will need to provide the input range with a 
callback.  Anyway, I don't know if this is helpful or not but 
good luck and I'll be waiting to see how this turns out.
Nov 21 2014
next sibling parent "Robert burner Schadek" <rburners gmail.com> writes:
Your idea designs an idiom on how to let ranges handle 
exceptions. My PR is about how to handle exceptions thrown by 
ranges. Both sort-of do the same thing but at different points. 
Your design idiom needs source access (needs to be programmed 
in). Mine can be bolted on later (an additional element in the 
range chain). Of course fixing an erroneous range might be tricky 
but than exception handling and recovering is not easy to being 
with.

Back to your problem: If you do the foreach by hand, can you 
place the part that throws (popFront, front or empty) in an try 
catch block and still iterate to the next element afterwards?
Nov 21 2014
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Fri, 21 Nov 2014 08:56:17 +0000
schrieb "Jonathan Marler" <johnnymarler gmail.com>:


 I actually ran into this problem today when using the dirEntries 
 function in std.file.  I was attempting to iterate all the files 
 on my C drive and I got an Access Denied error which caused the 
 DirIterator to throw an exception.  There's nothing I could do to 
 catch the exception and continue.  I'm very glad people are aware 
 of this problem and I'm glad you are trying to do something about 
 it.


Yep, that dirEntries Exception is quite the show stopper. You
need to be certain that you have access to all directories
that it may encounter, which makes it unusable for file system
roots, but also breaks way to easily with unreadable
directories in user directories when all you need is a list of
the _accessible_ files.

The bug reports so far:

std.file: dirEntries-range crashes, when hitting the system folder "System
Volume Information"
https://issues.dlang.org/show_bug.cgi?id=12513

DirEntries throws in foreach
https://issues.dlang.org/show_bug.cgi?id=12391

dirEntries throws when encountering a "long path" on windows
https://issues.dlang.org/show_bug.cgi?id=8967

-- 
Marco
Nov 21 2014
parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Friday, 21 November 2014 at 16:57:29 UTC, Marco Leise wrote:

 Am Fri, 21 Nov 2014 08:56:17 +0000
 schrieb "Jonathan Marler" <johnnymarler gmail.com>:


 I actually ran into this problem today when using the 
 dirEntries function in std.file.  I was attempting to iterate 
 all the files on my C drive and I got an Access Denied error 
 which caused the DirIterator to throw an exception.  There's 
 nothing I could do to catch the exception and continue.  I'm 
 very glad people are aware of this problem and I'm glad you 
 are trying to do something about it.


 Yep, that dirEntries Exception is quite the show stopper. You
 need to be certain that you have access to all directories
 that it may encounter, which makes it unusable for file system
 roots, but also breaks way to easily with unreadable
 directories in user directories when all you need is a list of
 the _accessible_ files.

 The bug reports so far:

 std.file: dirEntries-range crashes, when hitting the system 
 folder "System Volume Information"
 https://issues.dlang.org/show_bug.cgi?id=12513

 DirEntries throws in foreach
 https://issues.dlang.org/show_bug.cgi?id=12391

 dirEntries throws when encountering a "long path" on windows
 https://issues.dlang.org/show_bug.cgi?id=8967


Submitted my idea as a PR in phobos here: 
https://github.com/D-Programming-Language/phobos/pull/2655
Nov 24 2014
parent "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 25 November 2014 at 00:46:11 UTC, Jonathan Marler 
wrote:

 On Friday, 21 November 2014 at 16:57:29 UTC, Marco Leise wrote:

 Am Fri, 21 Nov 2014 08:56:17 +0000
 schrieb "Jonathan Marler" <johnnymarler gmail.com>:


 I actually ran into this problem today when using the 
 dirEntries function in std.file.  I was attempting to iterate 
 all the files on my C drive and I got an Access Denied error 
 which caused the DirIterator to throw an exception.  There's 
 nothing I could do to catch the exception and continue.  I'm 
 very glad people are aware of this problem and I'm glad you 
 are trying to do something about it.


 Yep, that dirEntries Exception is quite the show stopper. You
 need to be certain that you have access to all directories
 that it may encounter, which makes it unusable for file system
 roots, but also breaks way to easily with unreadable
 directories in user directories when all you need is a list of
 the _accessible_ files.

 The bug reports so far:

 std.file: dirEntries-range crashes, when hitting the system 
 folder "System Volume Information"
 https://issues.dlang.org/show_bug.cgi?id=12513

 DirEntries throws in foreach
 https://issues.dlang.org/show_bug.cgi?id=12391

 dirEntries throws when encountering a "long path" on windows
 https://issues.dlang.org/show_bug.cgi?id=8967


 Submitted my idea as a PR in phobos here: 
 https://github.com/D-Programming-Language/phobos/pull/2655


Woops wrong link, heres the right one: 
https://github.com/D-Programming-Language/phobos/pull/2768
Nov 24 2014
prev sibling parent reply "Robert burner Schadek" <rburners gmail.com> writes:
On Saturday, 15 November 2014 at 01:43:07 UTC, Robert burner 
Schadek wrote:

 This PR 
 https://github.com/D-Programming-Language/phobos/pull/2724 adds 
 an generic way of handling Exception in Range processing. 
 quickfur and Dicebot ask me to start a thread here so the 
 concept could be discussed.


We are currently searching for a name for an enum that is used to 
select which methods of the range to handle.

Current ideas are found in the PR. please have a look and give 
comments.
Dec 27 2014
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Saturday, 27 December 2014 at 22:14:41 UTC, Robert burner
Schadek wrote:

 On Saturday, 15 November 2014 at 01:43:07 UTC, Robert burner 
 Schadek wrote:

 This PR 
 https://github.com/D-Programming-Language/phobos/pull/2724 
 adds an generic way of handling Exception in Range processing. 
 quickfur and Dicebot ask me to start a thread here so the 
 concept could be discussed.


 We are currently searching for a name for an enum that is used 
 to select which methods of the range to handle.

 Current ideas are found in the PR. please have a look and give 
 comments.


I don't like RangeMethod because the term ‘Method’ is not used by
the language documentation, which only uses the term ‘member
function’. Some range primitives aren't necessarily functions
either. What about RangePrimitive?
Dec 27 2014
parent Mike Parker <aldacron gmail.com> writes:
On 12/28/2014 7:49 AM, Tobias Pankrath wrote:

 On Saturday, 27 December 2014 at 22:14:41 UTC, Robert burner
 Schadek wrote:

 On Saturday, 15 November 2014 at 01:43:07 UTC, Robert burner Schadek
 wrote:

 This PR https://github.com/D-Programming-Language/phobos/pull/2724
 adds an generic way of handling Exception in Range processing.
 quickfur and Dicebot ask me to start a thread here so the concept
 could be discussed.


 We are currently searching for a name for an enum that is used to
 select which methods of the range to handle.

 Current ideas are found in the PR. please have a look and give comments.


 I don't like RangeMethod because the term ‘Method’ is not used by
 the language documentation, which only uses the term ‘member
 function’. Some range primitives aren't necessarily functions
 either. What about RangePrimitive?


RangeAction?
Dec 27 2014