www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Plan for Exceptions and nogc?

reply "Jonathan Marler" <johnnymarler gmail.com> writes:
Is there a proposal for how D will support throwing Exceptions in 
 nogc code in the future?  I've searched the forums and found 
different proposals that involve things like pre-allocated 
exceptions, non-gc heap allocated exceptions or even stack 
allocated exceptions.  I don't want to debate the details of each 
solution, I'd just like to know if any of these proposals are 
deemed a "good idea", or if any of them are currently being 
worked on.  I personally think that using non-gc heap allocated 
exceptions in combination with the new scope semantics would be a 
great solution in many cases, but I don't know if there is any 
consensus or if this topic is just on the back-burner. Thanks.
Feb 16 2015
next sibling parent reply "philippecp" <philly.dilly gmail.com> writes:
I was wondering the same myself. I think there would be a lot of 
negative side effects to using alternate memory model other than 
GC (non-gc could introduce leaks, on stack has potential to be 
overwritten). For that reason, I think the only choices that make 
sense are that either  nogc functions are also nothrow, or that 
phobos has a set of preallocated immutable exceptions that can 
can thrown.

I personally think that  nogc should be relaxed to exclude 
allocating exceptions since there would be no gc allocation other 
than for exceptional situations. I suspect that having code that 
relies on exception for main case logic is far worse for 
performance than any gc overhead anyway and is a pattern everyone 
knows to avoid.
Feb 16 2015
parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 17 February 2015 at 05:52:23 UTC, philippecp wrote:

 I was wondering the same myself. I think there would be a lot 
 of negative side effects to using alternate memory model other 
 than GC (non-gc could introduce leaks, on stack has potential 
 to be overwritten). For that reason, I think the only choices 
 that make sense are that either  nogc functions are also 
 nothrow, or that phobos has a set of preallocated immutable 
 exceptions that can can thrown.

 I personally think that  nogc should be relaxed to exclude 
 allocating exceptions since there would be no gc allocation 
 other than for exceptional situations. I suspect that having 
 code that relies on exception for main case logic is far worse 
 for performance than any gc overhead anyway and is a pattern 
 everyone knows to avoid.


Relaxing the definition of  nogc to exclude exceptions could be 
an acceptable compromise.  However, the nature of an exception is 
that it is allocated when it is created, and deallocated after it 
is caught.  This model fits nicely with scope semantics.  The 
code catching the exception would catch a "scoped" reference 
which means it would be responsible for cleaning up the 
exception.  It would still be allocated on the heap, but it 
wouldn't be GC memory.  This is how I think exceptions should 
have been implemented in the first place, but back then the GC 
wasn't a big deal and scope didn't exist yet.
Feb 16 2015
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Tuesday, 17 February 2015 at 07:24:43 UTC, Jonathan Marler 
wrote:

 Relaxing the definition of  nogc to exclude exceptions could be 
 an acceptable compromise.  However, the nature of an exception 
 is that it is allocated when it is created, and deallocated 
 after it is caught.  This model fits nicely with scope 
 semantics.  The code catching the exception would catch a 
 "scoped" reference which means it would be responsible for 
 cleaning up the exception.  It would still be allocated on the 
 heap, but it wouldn't be GC memory.  This is how I think 
 exceptions should have been implemented in the first place, but 
 back then the GC wasn't a big deal and scope didn't exist yet.


This actually puts scope on the head. It's unique / ownership 
you're looking for (if at all).
Feb 17 2015
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 17 February 2015 at 09:19:55 UTC, Tobias Pankrath 
wrote:

 On Tuesday, 17 February 2015 at 07:24:43 UTC, Jonathan Marler 
 wrote:

 Relaxing the definition of  nogc to exclude exceptions could 
 be an acceptable compromise.  However, the nature of an 
 exception is that it is allocated when it is created, and 
 deallocated after it is caught.  This model fits nicely with 
 scope semantics.  The code catching the exception would catch 
 a "scoped" reference which means it would be responsible for 
 cleaning up the exception.  It would still be allocated on the 
 heap, but it wouldn't be GC memory.  This is how I think 
 exceptions should have been implemented in the first place, 
 but back then the GC wasn't a big deal and scope didn't exist 
 yet.


 This actually puts scope on the head. It's unique / ownership 
 you're looking for (if at all).


Right. But `scope` still has a place in it. It would either be 
necessary to allow throwing and catching the unique/owned types 
(instead of `Throwable`), but that would be quite a change to the 
language. Or the runtime manages the exceptions (freeing them as 
soon as they are no longer needed). In that case, the exceptions 
must not leave the `catch` blocks, which is what `scope` 
guarantees.
Feb 17 2015
parent reply "Matthias Bentrup" <matthias.bentrup googlemail.com> writes:
On Tuesday, 17 February 2015 at 12:41:51 UTC, Marc Schütz wrote:

 On Tuesday, 17 February 2015 at 09:19:55 UTC, Tobias Pankrath 
 wrote:

 On Tuesday, 17 February 2015 at 07:24:43 UTC, Jonathan Marler 
 wrote:

 Relaxing the definition of  nogc to exclude exceptions could 
 be an acceptable compromise.  However, the nature of an 
 exception is that it is allocated when it is created, and 
 deallocated after it is caught.  This model fits nicely with 
 scope semantics.  The code catching the exception would catch 
 a "scoped" reference which means it would be responsible for 
 cleaning up the exception.  It would still be allocated on 
 the heap, but it wouldn't be GC memory.  This is how I think 
 exceptions should have been implemented in the first place, 
 but back then the GC wasn't a big deal and scope didn't exist 
 yet.


 This actually puts scope on the head. It's unique / ownership 
 you're looking for (if at all).


 Right. But `scope` still has a place in it. It would either be 
 necessary to allow throwing and catching the unique/owned types 
 (instead of `Throwable`), but that would be quite a change to 
 the language. Or the runtime manages the exceptions (freeing 
 them as soon as they are no longer needed). In that case, the 
 exceptions must not leave the `catch` blocks, which is what 
 `scope` guarantees.


Maybe it is possible to have a separate ScopedThrowable exception 
class.

Those exceptions would be allocated on the stack and would be 
allowed to carry references to local/scoped data, but they live 
only for the duration of the corresponding exception handler.

The compiler should check that the exception and its payload 
don't escape the catch block, and of course the exception handler 
has to run before the stack unwinding is done.

The whole point is of course that ScopedThrowables could be 
thrown from  nogc functions.
Feb 17 2015
parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 17 February 2015 at 13:32:40 UTC, Matthias Bentrup 
wrote:

 Maybe it is possible to have a separate ScopedThrowable 
 exception class.

 Those exceptions would be allocated on the stack and would be 
 allowed to carry references to local/scoped data, but they live 
 only for the duration of the corresponding exception handler.

 The compiler should check that the exception and its payload 
 don't escape the catch block, and of course the exception 
 handler has to run before the stack unwinding is done.

 The whole point is of course that ScopedThrowables could be 
 thrown from  nogc functions.


In response to allocating an exception on the stack...It depends 
on when you allocate the exception.  If the "catcher" of the 
exception allocates it on the stack, then you are fine so long as 
you have a way to pass a reference to it to any children 
functions, however, if the thrower allocates it on the stack then 
the memory will be released when the exception gets thrown.  It 
may make sense in some cases for the "catcher" to allocate the 
exception but I don't think that would be the norm.

The reason you can't keep the "thrower's" stack memory around for 
the exception handler is because the exception handler may need 
that memory.  Once the exception is thrown the stack is unwound 
to the function that has the exception handler so all the memory 
gets released. In most cases the exception handler probably won't 
mess up the memory the exception is using, but that can't be 
guaranteed.

As far as preventing the exception from escaping the catch 
block...that's precisely what the scope keyword would ensure.

Exception myException;

try {
    SomethingBad();
} catch(scope Exception e)
{
     myException = e; // Nope. e is a scoped variable
     // Note: if you needed the exception outside this
     // block then you could copy the memory to another
     // location.
}
Feb 17 2015
parent reply "Matthias Bentrup" <matthias.bentrup googlemail.com> writes:
On Tuesday, 17 February 2015 at 17:38:20 UTC, Jonathan Marler 
wrote:

 The reason you can't keep the "thrower's" stack memory around 
 for the exception handler is because the exception handler may 
 need that memory.  Once the exception is thrown the stack is 
 unwound to the function that has the exception handler so all 
 the memory gets released. In most cases the exception handler 
 probably won't mess up the memory the exception is using, but 
 that can't be guaranteed.


The problem I see, is that if I program a  nogc function for 
performance reasons, I'll likely have some data in scoped memory 
that is useful for handling the exception. If the stack is 
unwound before the exception handler, the thrower has to copy it 
to non-scoped memory and the catcher has to deal with it whether 
it needs the data or not.

If the unwinding is done after the exception handler is left, the 
thrower can safely reference the data directly on the stack and 
the catcher can ignore any data it doesn't need. (It may copy the 
data to safety if it is needed later, but the catcher knows what 
it needs, whereas the thrower has to always assume the worst 
case.)
Feb 17 2015
next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
 If the unwinding is done after the exception handler is left


Cannot see how that should work.
Feb 17 2015
prev sibling parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 17 February 2015 at 18:04:53 UTC, Matthias Bentrup 
wrote:

 If the unwinding is done after the exception handler is left, 
 the thrower can safely reference the data directly on the stack 
 and the catcher can ignore any data it doesn't need. (It may 
 copy the data to safety if it is needed later, but the catcher 
 knows what it needs, whereas the thrower has to always assume 
 the worst case.)


That is a good idea, unfortunately this is impossible since the 
catch block needs to execute code. But this is a good thought 
process.  I thought of the same thing but then realized that it 
would be impossible to ensure that the catch block wouldn't stomp 
on that memory.  This leads to having a second stack...however, 
we already have a solution...it's called the heap :)  By the time 
you're done trying to resolve this issue you will have just 
redesigned the heap.

IMO, Allocating the exception on the non-gc heap and making the 
catch block responsible for freeing the memory is a pretty good 
solution.
Feb 17 2015
parent reply "Matthias Bentrup" <matthias.bentrup googlemail.com> writes:
On Tuesday, 17 February 2015 at 18:30:24 UTC, Jonathan Marler 
wrote:

 I thought of the same thing but then realized that it would be 
 impossible to ensure that the catch block wouldn't stomp on 
 that memory.


The catcher wouldn't stomp any more on the thrower's memory than 
a function stomps on the memory of its caller. All the data of 
the thrower is safe, because it is above the stack pointer. The 
unwinding hasn't been done at that point.
Feb 17 2015
next sibling parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Tuesday, 17 February 2015 at 18:40:51 UTC, Matthias Bentrup 
wrote:

 On Tuesday, 17 February 2015 at 18:30:24 UTC, Jonathan Marler 
 wrote:

 I thought of the same thing but then realized that it would be 
 impossible to ensure that the catch block wouldn't stomp on 
 that memory.


 The catcher wouldn't stomp any more on the thrower's memory 
 than a function stomps on the memory of its caller. All the 
 data of the thrower is safe, because it is above the stack 
 pointer. The unwinding hasn't been done at that point.


That would be a deep change in language semantics. Think 
scope(exit), scope(failure), destructors of structs.
Feb 17 2015
prev sibling parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 17 February 2015 at 18:40:51 UTC, Matthias Bentrup 
wrote:

 On Tuesday, 17 February 2015 at 18:30:24 UTC, Jonathan Marler 
 wrote:

 I thought of the same thing but then realized that it would be 
 impossible to ensure that the catch block wouldn't stomp on 
 that memory.


 The catcher wouldn't stomp any more on the thrower's memory 
 than a function stomps on the memory of its caller. All the 
 data of the thrower is safe, because it is above the stack 
 pointer. The unwinding hasn't been done at that point.


That would work if you didn't have to unwind the stack but 
unfortunately you do.  The catch block exists in the context of 
the function it is written in.  That means it assumes the stack 
pointer and stack variables are all in the context of it's 
defining function.  If you executed the catch code when the stack 
wasn't unwound, then it wouldn't know where any of the variables 
were.  Does that make sense?  Think about it for a minute.  You 
proposal suggests that the catch code can be executed no matter 
how many child functions have been added to the stack.  This is 
impossible since the catch code no longer knows where all of it's 
stack variables are.  Normally it uses an offset to the stack 
pointer but now it has been changed.  That's why you have to 
unwind the stack.

I like that you thought of this idea but you have to follow 
through with the details to see whether or not it would work.  I 
immediately thought of this idea and then realized that it was 
impossible.  A variation on the idea might work, but the idea as 
it is does not.  Keep thinking about it though...if nothing else 
it will give you a better understanding of the stack.  The more 
people that are intimate with the inner workings of how these 
things work the better.
Feb 17 2015
parent reply "Matthias Bentrup" <matthias.bentrup googlemail.com> writes:
On Tuesday, 17 February 2015 at 20:48:07 UTC, Jonathan Marler 
wrote:

 That would work if you didn't have to unwind the stack but 
 unfortunately you do.  The catch block exists in the context of 
 the function it is written in.  That means it assumes the stack 
 pointer and stack variables are all in the context of it's 
 defining function.  If you executed the catch code when the 
 stack wasn't unwound, then it wouldn't know where any of the 
 variables were.  Does that make sense?  Think about it for a 
 minute.  You proposal suggests that the catch code can be 
 executed no matter how many child functions have been added to 
 the stack.  This is impossible since the catch code no longer 
 knows where all of it's stack variables are.  Normally it uses 
 an offset to the stack pointer but now it has been changed.  
 That's why you have to unwind the stack.


So the catcher would have to behave like a delegate.
Feb 17 2015
parent "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 17 February 2015 at 21:30:00 UTC, Matthias Bentrup 
wrote:

 On Tuesday, 17 February 2015 at 20:48:07 UTC, Jonathan Marler 
 wrote:

 That would work if you didn't have to unwind the stack but 
 unfortunately you do.  The catch block exists in the context 
 of the function it is written in.  That means it assumes the 
 stack pointer and stack variables are all in the context of 
 it's defining function.  If you executed the catch code when 
 the stack wasn't unwound, then it wouldn't know where any of 
 the variables were.  Does that make sense?  Think about it for 
 a minute.  You proposal suggests that the catch code can be 
 executed no matter how many child functions have been added to 
 the stack.  This is impossible since the catch code no longer 
 knows where all of it's stack variables are.  Normally it uses 
 an offset to the stack pointer but now it has been changed.  
 That's why you have to unwind the stack.


 So the catcher would have to behave like a delegate.


Sure but then you're allocating GC memory again (compounding the 
problem you're trying to solve in the first place).  The best 
solution I can think of that should work in almost every case is 
to allocate the exception on the non-GC heap, then make the 
catcher cleans up the exception.  Simple, and makes sense when 
you think about the lifetime of an exception.  This would have 
been dangerous before but with the new scope semantics D can 
ensure that the exception does not escape the catch block (and 
therefore gets cleaned up properly).
Feb 17 2015
prev sibling next sibling parent reply "weaselcat" <weaselcat gmail.com> writes:
On Monday, 16 February 2015 at 23:17:03 UTC, Jonathan Marler 
wrote:

 Is there a proposal for how D will support throwing Exceptions 
 in  nogc code in the future?  I've searched the forums and 
 found different proposals that involve things like 
 pre-allocated exceptions, non-gc heap allocated exceptions or 
 even stack allocated exceptions.  I don't want to debate the 
 details of each solution, I'd just like to know if any of these 
 proposals are deemed a "good idea", or if any of them are 
 currently being worked on.  I personally think that using 
 non-gc heap allocated exceptions in combination with the new 
 scope semantics would be a great solution in many cases, but I 
 don't know if there is any consensus or if this topic is just 
 on the back-burner. Thanks.


Would RefCounted!T being  nogc help alleviate this issue?
Feb 16 2015
next sibling parent "Jonathan Marler" <johnnymarler gmail.com> writes:
On Tuesday, 17 February 2015 at 07:28:03 UTC, weaselcat wrote:

 Would RefCounted!T being  nogc help alleviate this issue?


I've heard of RefCounted and I understand the concept but I've 
never used it.  I'll do some reading a some experiments. Maybe 
this will work :)
Feb 16 2015
prev sibling parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On 17/02/2015 07:28, weaselcat wrote:

 Would RefCounted!T being  nogc help alleviate this issue?


I think Andrei once said that was a good solution for exceptions.

I think the stumbling block is that if T contains indirections, 
RefCounted calls GC.addRange on the T*, which currently violates  nogc. 
Perhaps addRange should be allowed even with  nogc, as it doesn't 
trigger collections as of itself.

Otherwise, we would need some way of controlling whether addRange should 
be called by smart pointers or not.
Feb 17 2015
parent Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On 17/02/2015 16:15, Nick Treleaven wrote:

 Would RefCounted!T being  nogc help alleviate this issue?


 I think Andrei once said that was a good solution for exceptions.


Or rather that refcounting would be appropriate for exceptions, not 
necessarily RefCounted. (The compiler can't generate code using Phobos 
symbols, only druntime).
Feb 17 2015
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/16/15 3:17 PM, Jonathan Marler wrote:

 Is there a proposal for how D will support throwing Exceptions in  nogc
 code in the future?


Nothing definite. We will get on to that right after 2.067. This is a 
good time to start discussions. -- Andrei
Feb 17 2015
next sibling parent reply "Chris Williams" <yoreanon-chrisw yahoo.co.jp> writes:
On Tuesday, 17 February 2015 at 15:54:17 UTC, Andrei Alexandrescu 
wrote:

 On 2/16/15 3:17 PM, Jonathan Marler wrote:

 Is there a proposal for how D will support throwing Exceptions 
 in  nogc
 code in the future?


 Nothing definite. We will get on to that right after 2.067. 
 This is a good time to start discussions. -- Andrei


Could someone give a description of the minutiae of why Exception 
throwing uses memory allocation as it is and why (for example) 
passing it back on the stack isn't an option?
Feb 17 2015
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
 Could someone give a description of the minutiae of why 
 Exception throwing uses memory allocation as it is and why (for 
 example) passing it back on the stack isn't an option?


The stack frame of the thrower is the first one to be rolled 
back. So you cannot allocate in the stack frame of the thrower, 
but a function cannot now, who (if anyone) is catching the 
exceptions it might throw. So I fear the stack is out.
Feb 17 2015
parent reply "Chris Williams" <yoreanon-chrisw yahoo.co.jp> writes:
On Tuesday, 17 February 2015 at 18:50:46 UTC, Tobias Pankrath 
wrote:

 Could someone give a description of the minutiae of why 
 Exception throwing uses memory allocation as it is and why 
 (for example) passing it back on the stack isn't an option?


 The stack frame of the thrower is the first one to be rolled 
 back. So you cannot allocate in the stack frame of the thrower, 
 but a function cannot now, who (if anyone) is catching the 
 exceptions it might throw. So I fear the stack is out.


Every throwable function call could be assumed to have a typed 
result (even void functions) and if, after the return, the caller 
checks the type and detects that it was an error, bubbles that 
up, then eventually you get to wherever the catcher is.

But so basically, the current ABI doesn't support it and there's 
no desire to change it? How do exceptions currently happen, if 
not via some official ABI declaration of how throwable methods 
interact with one another? The compiler/linker determines where 
the catcher is and inserts code to cut down the stack and perform 
a long jump all the way back? If so, how do scope statements work?
Feb 17 2015
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 17 February 2015 at 19:03:49 UTC, Chris Williams 
wrote:

 Every throwable function call could be assumed to have a typed 
 result (even void functions) and if, after the return, the 
 caller checks the type and detects that it was an error, 
 bubbles that up, then eventually you get to wherever the 
 catcher is.

 But so basically, the current ABI doesn't support it and 
 there's no desire to change it? How do exceptions currently 
 happen, if not via some official ABI declaration of how 
 throwable methods interact with one another? The 
 compiler/linker determines where the catcher is and inserts 
 code to cut down the stack and perform a long jump all the way 
 back? If so, how do scope statements work?


This kind of stunt is taxing on the fast path. It can be 
implemented as setjmp/longjmp but is more and more avoided in 
favor of libunwind based solutions for languages that have code 
running on unwinding.

libunwind based solution is slower to unwind, but is not very 
taxing in the fast path (only prevent some optimizations to be 
done like tail call).

This solution is a non starter perforamnce-wize for D.
Feb 17 2015
parent reply "Chris Williams" <yoreanon-chrisw yahoo.co.jp> writes:
On Wednesday, 18 February 2015 at 00:14:55 UTC, deadalnix wrote:

 On Tuesday, 17 February 2015 at 19:03:49 UTC, Chris Williams 
 wrote:

 Every throwable function call could be assumed to have a typed 
 result (even void functions) and if, after the return, the 
 caller checks the type and detects that it was an error, 
 bubbles that up, then eventually you get to wherever the 
 catcher is.

 But so basically, the current ABI doesn't support it and 
 there's no desire to change it? How do exceptions currently 
 happen, if not via some official ABI declaration of how 
 throwable methods interact with one another? The 
 compiler/linker determines where the catcher is and inserts 
 code to cut down the stack and perform a long jump all the way 
 back? If so, how do scope statements work?


 This kind of stunt is taxing on the fast path. It can be 
 implemented as setjmp/longjmp but is more and more avoided in 
 favor of libunwind based solutions for languages that have code 
 running on unwinding.

 libunwind based solution is slower to unwind, but is not very 
 taxing in the fast path (only prevent some optimizations to be 
 done like tail call).

 This solution is a non starter perforamnce-wize for D.


I didn't mean it as a solution. As said, I was just looking for 
an intro to the topic, so that I (and others) could meaningfully 
contribute or at least understand the options. I'll look up 
libunwind and, if that has enough info for me to grok it, create 
a wiki page on the subject.
Feb 17 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 18 February 2015 at 00:54:37 UTC, Chris Williams 
wrote:

 I didn't mean it as a solution. As said, I was just looking for 
 an intro to the topic, so that I (and others) could 
 meaningfully contribute or at least understand the options. 
 I'll look up libunwind and, if that has enough info for me to 
 grok it, create a wiki page on the subject.


It is a horrible solution developed for the Itanium VLIW 
architecture which is very sensitive to branching. IRRC it 
basically works by looking at the return address on the stack, 
then picking up stack frame information in a global table to 
unwind. It is language agnostic and the language provides a 
"personality function" to unwind correctly in a language 
dependent manner...

AFAIK, C++ exceptions are copied from the stack to a special 
memory region when unwinding to prevent the memory issues D 
suffers from.

I agree that a fast unwind with stack pointer reset or multiple 
return paths would be much better, but you need to rewrite the 
backend to support it. That's the main issue... the "fast path" 
argument is just a sorry excuse that literally means that 
exceptions are avoided for common failures in C++. As a result 
you get APIs that are nonuniform.
Feb 18 2015
next sibling parent reply "Matthias Bentrup" <matthias.bentrup googlemail.com> writes:
On Wednesday, 18 February 2015 at 08:13:35 UTC, Ola Fosheim 
Grøstad wrote:

 On Wednesday, 18 February 2015 at 00:54:37 UTC, Chris Williams 
 wrote:

 I didn't mean it as a solution. As said, I was just looking 
 for an intro to the topic, so that I (and others) could 
 meaningfully contribute or at least understand the options. 
 I'll look up libunwind and, if that has enough info for me to 
 grok it, create a wiki page on the subject.


 It is a horrible solution developed for the Itanium VLIW 
 architecture which is very sensitive to branching. IRRC it 
 basically works by looking at the return address on the stack, 
 then picking up stack frame information in a global table to 
 unwind. It is language agnostic and the language provides a 
 "personality function" to unwind correctly in a language 
 dependent manner...

 AFAIK, C++ exceptions are copied from the stack to a special 
 memory region when unwinding to prevent the memory issues D 
 suffers from.

 I agree that a fast unwind with stack pointer reset or multiple 
 return paths would be much better, but you need to rewrite the 
 backend to support it. That's the main issue... the "fast path" 
 argument is just a sorry excuse that literally means that 
 exceptions are avoided for common failures in C++. As a result 
 you get APIs that are nonuniform.


Windows SEH maintains a per-thread linked list of exception 
handlers, but the C++ runtime seems to install only one handler 
at the start of every function and resorts to lookup tables if 
there are multiply try{}s in the function.

If you want to avoid lookup tables, you can of course add/remove 
catchers dynamically whenever you enter/leave a try block, that 
would add a small cost to every try, but avoids the (larger) 
table lookup cost on the catch.
Feb 18 2015
next sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 18 February 2015 at 09:04:38 UTC, Matthias Bentrup 
wrote:

 Windows SEH maintains a per-thread linked list of exception 
 handlers, but the C++ runtime seems to install only one handler 
 at the start of every function and resorts to lookup tables if 
 there are multiply try{}s in the function.

 If you want to avoid lookup tables, you can of course 
 add/remove catchers dynamically whenever you enter/leave a try 
 block, that would add a small cost to every try, but avoids the 
 (larger) table lookup cost on the catch.


You want to do this as C++ introduce a ton of implicit finally 
blocks for destructors. If you would setup one everytime you need 
one, you would trash performance in the fast path.
Feb 18 2015
prev sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 18 February 2015 at 09:04:38 UTC, Matthias Bentrup 
wrote:

 If you want to avoid lookup tables, you can of course 
 add/remove catchers dynamically whenever you enter/leave a try 
 block, that would add a small cost to every try, but avoids the 
 (larger) table lookup cost on the catch.


There are many ways to get better performance than the current 
regime, both dynamic and static approaches, with little execution 
costs, but since most C++ programs don't rely on exceptions where 
speed matters there is little incentive to improve considering 
the complications in the backend. So I don't expect the C++ crowd 
to do anything about it.

As a result C++ programmers that need speed can keep pretending 
exceptions don't exist, and if you don't need speed, why are you 
using C++? :-). Thus status quo persists and nothing interesting 
happens.

D has made C/C++ compatibility a goal... so nothing is going to 
happen with D either...

In most programs the possible call trees at a given point are 
quite limited, like a regular expression, so with whole program 
optimization you only need to special case catch blocks where the 
program may stop unwinding. I.e. you could detect which of the 
possible call trees you are in and unwind "multiple stack frames" 
from the same code location. If only a few functions can call 
you, you most certainly don't need to look up anything, right? If 
the compiler is allowed to move beyond "the peephole separate 
compilation viewpoint"...
Feb 18 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 18 February 2015 at 08:13:35 UTC, Ola Fosheim 
Grøstad wrote:

 It is a horrible solution developed for the Itanium VLIW 
 architecture which is very sensitive to branching. IRRC it 
 basically works by looking at the return address on the stack, 
 then picking up stack frame information in a global table to 
 unwind. It is language agnostic and the language provides a 
 "personality function" to unwind correctly in a language 
 dependent manner...


Which is true, but would be as true without the horrible mention. 
Adjective do not constitute arguments.


 I agree that a fast unwind with stack pointer reset or multiple 
 return paths would be much better, but you need to rewrite the 
 backend to support it. That's the main issue... the "fast path" 
 argument is just a sorry excuse that literally means that 
 exceptions are avoided for common failures in C++. As a result 
 you get APIs that are nonuniform.


What you agree with is irrelevant if it do not come backed by 
facts.

You can qualify thing as "horrible", "sorry excuses" and so on, 
the only thing it is telling us is that you seems incapable of 
forming a compelling argument and rely on smear instead.
Feb 18 2015
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 18 February 2015 at 17:55:49 UTC, deadalnix wrote:

 Which is true, but would be as true without the horrible 
 mention. Adjective do not constitute arguments.


And your nonsensical point is?
Feb 18 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 18 February 2015 at 18:43:49 UTC, Ola Fosheim 
Grøstad wrote:

 On Wednesday, 18 February 2015 at 17:55:49 UTC, deadalnix wrote:

 Which is true, but would be as true without the horrible 
 mention. Adjective do not constitute arguments.


 And your nonsensical point is?


That you have no idea how to argue, and rely on sticking negative 
adjective close to things you don't like in hope that you'll fool 
someone.

This post I'm answering to is more evidence of that.

That's the last answer you'll get from me until you make any 
valuable argument.
Feb 18 2015
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 18 February 2015 at 18:59:03 UTC, deadalnix wrote:

 That's the last answer you'll get from me until you make any 
 valuable argument.


Excellent! I hope you keep your promise.
Feb 18 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 17 February 2015 at 19:03:49 UTC, Chris Williams 
wrote:

 Every throwable function call could be assumed to have a typed 
 result (even void functions) and if, after the return, the 
 caller checks the type and detects that it was an error, 
 bubbles that up, then eventually you get to wherever the 
 catcher is.

 But so basically, the current ABI doesn't support it and 
 there's no desire to change it? How do exceptions currently 
 happen, if not via some official ABI declaration of how 
 throwable methods interact with one another? The 
 compiler/linker determines where the catcher is and inserts 
 code to cut down the stack and perform a long jump all the way 
 back? If so, how do scope statements work?


No problem. in fact this kind of solution is a good fit for 
language where try blocks are rare (say OCaml for instance) as 
you can unwind much faster, while keeping the cost on the fast 
path small.

But that wouldn't be a good fit for D (or even worse for C++) 
because you have unwinding going on (destructor, scope 
statements, ...) which makes for a lot of implicit try blocks.
Feb 19 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 17 February 2015 at 15:54:17 UTC, Andrei Alexandrescu 
wrote:

 On 2/16/15 3:17 PM, Jonathan Marler wrote:

 Is there a proposal for how D will support throwing Exceptions 
 in  nogc
 code in the future?


 Nothing definite. We will get on to that right after 2.067. 
 This is a good time to start discussions. -- Andrei


There are various problem with exception, and  nogc is only one 
of them.

The other big issue is that exceptions bypass the type qualifier 
system.

I want to hammer once more the owned solution.
Feb 17 2015
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 16 February 2015 at 23:17:03 UTC, Jonathan Marler 
wrote:

 Is there a proposal for how D will support throwing Exceptions 
 in  nogc code in the future?  I've searched the forums and 
 found different proposals that involve things like 
 pre-allocated exceptions, non-gc heap allocated exceptions or 
 even stack allocated exceptions.  I don't want to debate the 
 details of each solution, I'd just like to know if any of these 
 proposals are deemed a "good idea", or if any of them are 
 currently being worked on.  I personally think that using 
 non-gc heap allocated exceptions in combination with the new 
 scope semantics would be a great solution in many cases, but I 
 don't know if there is any consensus or if this topic is just 
 on the back-burner. Thanks.


 From my POV best proposal from last lengthy discussion was to 
enable reference-counted non-gc-heap Exceptions. But that needs a 
language change because RefCounted!T is a struct and thus neither 
can be thrown nor can be part of Throwable class hierarchy.

Any concept that implies that exceptions an be deallocated in 
`catch` block is absolutely unacceptable because it conflicts 
with exception propagation from fibers - a very important piece 
of functionality.
Feb 18 2015
next sibling parent reply "Jonathan Marler" <johnnymarler gmail.com> writes:
On Wednesday, 18 February 2015 at 14:46:30 UTC, Dicebot wrote:

 From my POV best proposal from last lengthy discussion was to 
 enable reference-counted non-gc-heap Exceptions. But that needs 
 a language change because RefCounted!T is a struct and thus 
 neither can be thrown nor can be part of Throwable class 
 hierarchy.

 Any concept that implies that exceptions an be deallocated in 
 `catch` block is absolutely unacceptable because it conflicts 
 with exception propagation from fibers - a very important piece 
 of functionality.


RefCounted Exceptions would work quite nicely.  You are right 
that It will require work on the language to support them but I 
like that solution.

If you don't mind, could you explain why cleaning up exception in 
the catch block would break exception propogation from fibers.  
Are you saying that if you throw inside a catch block and save a 
reference to the exception (in the new exceptions "next" field), 
that it will create a dangling pointer?
Feb 18 2015
parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 18 February 2015 at 18:02:14 UTC, Jonathan Marler 
wrote:

 On Wednesday, 18 February 2015 at 14:46:30 UTC, Dicebot wrote:

 From my POV best proposal from last lengthy discussion was to 
 enable reference-counted non-gc-heap Exceptions. But that 
 needs a language change because RefCounted!T is a struct and 
 thus neither can be thrown nor can be part of Throwable class 
 hierarchy.

 Any concept that implies that exceptions an be deallocated in 
 `catch` block is absolutely unacceptable because it conflicts 
 with exception propagation from fibers - a very important 
 piece of functionality.


 RefCounted Exceptions would work quite nicely.  You are right 
 that It will require work on the language to support them but I 
 like that solution.


It will likely trigger whole load of other issues being totally 
new language concept but so far I am not aware of any better 
proposal.

Of course to truly address  nogc you need also some sort of 
object pool for exceptions instances where those get returned 
upon release - otherwise it would simply move allocations from GC 
heap to plain heap. Which you can do already and does not really 
fix the problem.

But that is a relatively simple library solution that can be 
built if reference counted exception become reality.


 If you don't mind, could you explain why cleaning up exception 
 in the catch block would break exception propogation from 
 fibers.  Are you saying that if you throw inside a catch block 
 and save a reference to the exception (in the new exceptions 
 "next" field), that it will create a dangling pointer?


Imagine application like vibe.d which uses fibers to imitate 
blocking API for async operations (we have something similar in 
Sociomantic projects too). Callbacks execute in the context of 
event loop but if any of those throws you want exception trace to 
point to original "blocking" call that registered callback.

Right now doing this is relatively simple. You catch all 
exception in callbacks and store exception instance in relevant 
fiber-local data. Upon resuming that fiber it checks if there are 
any pending exceptions and re-throws if there is one. There are 
probably more tricky details but basic principle should be like 
this.
Feb 18 2015
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 18 February 2015 at 14:46:30 UTC, Dicebot wrote:

 From my POV best proposal from last lengthy discussion was to 
 enable reference-counted non-gc-heap Exceptions. But that needs 
 a language change because RefCounted!T is a struct and thus 
 neither can be thrown nor can be part of Throwable class 
 hierarchy.


This is not satisfactory. First there is no safe way to refcount 
right now. Second, this has all kind of implication for the GC to 
scan the heap and so on. Finally this do not solve the type 
qualifier problem.

I don't think a language change to do this pay for itself. i 
understand the appeal, as this is probably simpler to implement 
than the alternative, but that is really building technical debt.


 Any concept that implies that exceptions an be deallocated in 
 `catch` block is absolutely unacceptable because it conflicts 
 with exception propagation from fibers - a very important piece 
 of functionality.


Exception propagate the same way between thread, which is even 
worse :)
Feb 18 2015
parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 18 February 2015 at 18:26:34 UTC, deadalnix wrote:

 On Wednesday, 18 February 2015 at 14:46:30 UTC, Dicebot wrote:

 From my POV best proposal from last lengthy discussion was to 
 enable reference-counted non-gc-heap Exceptions. But that 
 needs a language change because RefCounted!T is a struct and 
 thus neither can be thrown nor can be part of Throwable class 
 hierarchy.


 This is not satisfactory. First there is no safe way to 
 refcount right now. Second, this has all kind of implication 
 for the GC to scan the heap and so on. Finally this do not 
 solve the type qualifier problem.

 I don't think a language change to do this pay for itself. i 
 understand the appeal, as this is probably simpler to implement 
 than the alternative, but that is really building technical 
 debt.


Right now I don't care for full memory safety or type safety of 
any proposed solution. I will be glad to have any that actually 
works - and I have not heard of any idea that allows to do that 
without language changes. Your push for owned is also hardly 
relevant because exceptions are good examples of data with shared 
/ non-determenistic ownership and thus won't benefit from it.
Feb 18 2015
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 18 February 2015 at 20:25:07 UTC, Dicebot wrote:

 Right now I don't care for full memory safety or type safety of 
 any proposed solution. I will be glad to have any that actually 
 works - and I have not heard of any idea that allows to do that 
 without language changes. Your push for owned is also hardly 
 relevant because exceptions are good examples of data with 
 shared / non-determenistic ownership and thus won't benefit 
 from it.


I think it make sense to require that something thrown to be 
owned. That means ownership can be transferred from one thread to 
the other. That also mean that the compiler can free the 
exception when it goes out of scope, which is what is wanted for 
exception and  nogc to work together.

The invalid operation in  nogc would become promoting owned to 
Tl/shared/immutable rather than not allocating at all.

This has added benefit to allow for way more code to be  nogc, 
and in many cases transfers the GCness of something in the 
caller, which makes for  nogc compatible libraries that can be 
used in both  nogc and gc situation.

If we are gonna add something in the language, it'd better be 
worth it.
Feb 18 2015
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 18 February 2015 at 20:47:36 UTC, deadalnix wrote:

 On Wednesday, 18 February 2015 at 20:25:07 UTC, Dicebot wrote:

 Right now I don't care for full memory safety or type safety 
 of any proposed solution. I will be glad to have any that 
 actually works - and I have not heard of any idea that allows 
 to do that without language changes. Your push for owned is 
 also hardly relevant because exceptions are good examples of 
 data with shared / non-determenistic ownership and thus won't 
 benefit from it.


 I think it make sense to require that something thrown to be 
 owned. That means ownership can be transferred from one thread 
 to the other. That also mean that the compiler can free the 
 exception when it goes out of scope, which is what is wanted 
 for exception and  nogc to work together.

 The invalid operation in  nogc would become promoting owned to 
 Tl/shared/immutable rather than not allocating at all.

 This has added benefit to allow for way more code to be  nogc, 
 and in many cases transfers the GCness of something in the 
 caller, which makes for  nogc compatible libraries that can be 
 used in both  nogc and gc situation.

 If we are gonna add something in the language, it'd better be 
 worth it.


+1

Could you take the time to make a concrete proposal? I'll try to 
have another go at `scope`, which is of course deeply linked with 
ownership, and these two things need to work well together. I 
want to avoid proposing something that will later collide with 
your ideas.
Feb 19 2015
prev sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 18 February 2015 at 20:25:07 UTC, Dicebot wrote:

 On Wednesday, 18 February 2015 at 18:26:34 UTC, deadalnix wrote:

 On Wednesday, 18 February 2015 at 14:46:30 UTC, Dicebot wrote:

 From my POV best proposal from last lengthy discussion was to 
 enable reference-counted non-gc-heap Exceptions. But that 
 needs a language change because RefCounted!T is a struct and 
 thus neither can be thrown nor can be part of Throwable class 
 hierarchy.


 This is not satisfactory. First there is no safe way to 
 refcount right now. Second, this has all kind of implication 
 for the GC to scan the heap and so on. Finally this do not 
 solve the type qualifier problem.

 I don't think a language change to do this pay for itself. i 
 understand the appeal, as this is probably simpler to 
 implement than the alternative, but that is really building 
 technical debt.


 Right now I don't care for full memory safety or type safety of 
 any proposed solution. I will be glad to have any that actually 
 works - and I have not heard of any idea that allows to do that 
 without language changes. Your push for owned is also hardly 
 relevant because exceptions are good examples of data with 
 shared / non-determenistic ownership and thus won't benefit 
 from it.


Non-deterministic ownership is exactly what the owned concept 
helps with, because it allows to delay deciding on an actual 
management mechanism until it reaches the consumer. When someone 
catches an owned exception, they can then choose to store it in a 
non-owned variable (=> making it GC owned), in a ref-counted 
(which then takes ownership), or to not store it at all, in which 
case it will get released automatically when the catch block is 
left. Before that decision is made, it can still be used without 
losing the "owned" property, by passing it as a scope parameter 
(or storing it in a scope variable).
Feb 19 2015