www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Scope failure is not preventing application crush in case of uncaught

reply kerdemdemir <kerdemdemir gmail.com> writes:
While calling this function :

bool PublicMarketCall( ref Json result )
{
   string fullUrl = 
"https://bittrex.com/api/v1.1/public/getmarketsummaries";
   Json data;
   try
   {
   	requestHTTP(fullUrl,
   		(scope req) {
   			req.method = HTTPMethod.GET;
   		},
   		(scope res) {
   			data = parseJsonString(res.bodyReader.readAllUTF8());	
   		}
   	);
   }
   catch ( std.json.JSONException e )
   {
   	writeln("Exception was caught while making the public call: ", 
fullUrl);
   	return false;
   }

   scope(failure)
   	writeln("Exception was caught while public data: ", fullUrl);
   return false;
}

Very rarely (I am calling this function thousands of times in a 
day) like once in a day I am getting a crush while calling 
requestHTTP function :

object.Exception ../../../.dub/packages/vibe-d-0.8.1/vibe-d/core/vibe/core/drive
s/libevent2.d(375): Failed to connect to host 104.17.154.108:443: 7FF703F4F617
----------------
??:? [0x606cee]
??:? [0x60ceb2]
exception.d:421 [0x4644a3]
exception.d:388 [0x41396d]
libevent2.d:375 [0x59002b]
net.d:129 [0x5be726]
client.d:610 [0x4a8ed4]
client.d:528 [0x4a7224]
client.d:462 [0x4a5a3f]
client.d:157 [0x4a5819]
client.d:133 [0x4a52bf]
.... --> Some files belonging my function
app.d:41 [0x43b947]
??:? [0x612f3e]
??:? [0x612efd]
??:? [0x612df8]
__entrypoint.d:8 [0x43be54]
??:? __libc_start_main [0x3de282f]
??:? [0x406fb8]
Program exited with code 1

As far as I know scope(failure) should be collecting all failure 
cases. And my application shouldn't crush with uncaught 
exceptions. What I am missing?
Dec 16 2017
next sibling parent reply ketmar <ketmar ketmar.no-ip.org> writes:
kerdemdemir wrote:


 As far as I know scope(failure) should be collecting all failure cases.


nope. `failure` scope won't stop exception propagation, it is just called 
before exception leaves your function, to give you a last chance to do some 
immediate cleanup.
Dec 16 2017
parent reply kerdemdemir <kerdemdemir gmail.com> writes:
On Saturday, 16 December 2017 at 20:56:26 UTC, ketmar wrote:

 kerdemdemir wrote:


 As far as I know scope(failure) should be collecting all 
 failure cases.


 nope. `failure` scope won't stop exception propagation, it is 
 just called before exception leaves your function, to give you 
 a last chance to do some immediate cleanup.


Than do I still need to catch'em all? For me it doesn't matter 
the which exception it is. It is more important for me to not 
pollute my code with so many exception related lines. What is the 
most compact way for me to catch'em all?
Dec 16 2017
next sibling parent ketmar <ketmar ketmar.no-ip.org> writes:
kerdemdemir wrote:


 On Saturday, 16 December 2017 at 20:56:26 UTC, ketmar wrote:

 kerdemdemir wrote:


 As far as I know scope(failure) should be collecting all failure cases.


 nope. `failure` scope won't stop exception propagation, it is just 
 called before exception leaves your function, to give you a last chance 
 to do some immediate cleanup.


 Than do I still need to catch'em all? For me it doesn't matter the which 
 exception it is. It is more important for me to not pollute my code with 
 so many exception related lines. What is the most compact way for me to 
 catch'em all?


see collectException[0], for example. something like (WARNING! DON'T DO 
THIS! it's not wise to silence everything! try to be selective with 
`collectException!RequiredExceptionType` instead!)

	collectException(() {
		your code here
	}());

or just move your code to some function, and do

	collectException(myfunction);


[0] http://dpldocs.info/experimental-docs/std.exception.collectException.2.html
Dec 16 2017
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, December 16, 2017 21:22:47 kerdemdemir via Digitalmars-d-learn 
wrote:

 On Saturday, 16 December 2017 at 20:56:26 UTC, ketmar wrote:

 kerdemdemir wrote:

 As far as I know scope(failure) should be collecting all
 failure cases.


 nope. `failure` scope won't stop exception propagation, it is
 just called before exception leaves your function, to give you
 a last chance to do some immediate cleanup.


 Than do I still need to catch'em all? For me it doesn't matter
 the which exception it is. It is more important for me to not
 pollute my code with so many exception related lines. What is the
 most compact way for me to catch'em all?


The only way to catch an exception is with a catch block, and if you do

catch(Exception e)

then it will catch all exceptions derived from Exception (which is
everything that isn't an Error, though catching Errors is almost always a
bad idea).

You can also use something like std.exception.catchException like Ketmar
suggests, though that's just using a catch internally and returning the
Exception. There's nothing magical about it.

In general though, just eating exceptions is likely to cause problems. If an
exception is thrown, it's because there's a problem, and your program should
either be responding to it or exiting. Usually, you put try-catch statements
at the points in your program that are best suited for handling whatever
exceptions are likely to be thrown. If you feel the need to put tons of
try-catch statements in your program, you're probably doing something wrong.
Sometimes, lots of try-catch statements is appropriate, but usually, you
shouldn't need very many.

See Also:

http://ddili.org/ders/d.en/exceptions.html
http://ddili.org/ders/d.en/scope.html

- Jonathan M Davis
Dec 16 2017
parent reply Anonymouse <asdf asdf.net> writes:
On Saturday, 16 December 2017 at 21:56:49 UTC, Jonathan M Davis 
wrote:

 The only way to catch an exception is with a catch block, and 
 if you do


If you return inside a scopeguard though, the exception (or 
error!) is swallowed. https://run.dlang.io/is/GEtQ6D

void main()
{
     foo();
     writeln("no error!");
}

int foo()
{
     scope(failure) return -1;
     assert(0);
}
Dec 16 2017
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, December 17, 2017 00:10:27 Anonymouse via Digitalmars-d-learn 
wrote:

 On Saturday, 16 December 2017 at 21:56:49 UTC, Jonathan M Davis

 wrote:

 The only way to catch an exception is with a catch block, and
 if you do


 If you return inside a scopeguard though, the exception (or
 error!) is swallowed. https://run.dlang.io/is/GEtQ6D

 void main()
 {
      foo();
      writeln("no error!");
 }

 int foo()
 {
      scope(failure) return -1;
      assert(0);
 }


Which is arguably a bug. The problem stems from the fact that scope
statements get lowered to try-catch-finally blocks. But the entire purposed
of scope statements is to be able to do stuff while an exception is in
flight, not to actually handle the exception or affect it in any way.

- Jonathan M Davis
Dec 16 2017
prev sibling parent reply codephantom <me noyb.com> writes:
On Sunday, 17 December 2017 at 00:10:27 UTC, Anonymouse wrote:

 If you return inside a scopeguard though, the exception (or 
 error!) is swallowed. https://run.dlang.io/is/GEtQ6D


The scope guard will not 'swallow' it, if you use -release mode.

Which suggests to me, that assert(0) operates differently when 
'not' using -release mode.

My 'guess' is, that release mode issues a halt instruction, 
whereas in non-release mode it is just an exception like any 
other exception.
Dec 17 2017
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, December 17, 2017 08:10:06 codephantom via Digitalmars-d-learn 
wrote:

 On Sunday, 17 December 2017 at 00:10:27 UTC, Anonymouse wrote:

 If you return inside a scopeguard though, the exception (or
 error!) is swallowed. https://run.dlang.io/is/GEtQ6D


 The scope guard will not 'swallow' it, if you use -release mode.

 Which suggests to me, that assert(0) operates differently when
 'not' using -release mode.

 My 'guess' is, that release mode issues a halt instruction,
 whereas in non-release mode it is just an exception like any
 other exception.


assert(0) does indeed turn into a HLT instruction in -release mode (as does
any assertion that's known to be false at compile time). It's a special case
intended for marking code that's supposed to be unreachable. Without
-release, failed assertions throw AssertErrors, so they're Errors not
Exceptions, though I think that scope statements currently catch and then
rethrow Throwable rather than Exception, which is why AssertErrors would be
affected.

- Jonathan M Davis
Dec 17 2017
parent reply codephantom <me noyb.com> writes:
On Sunday, 17 December 2017 at 08:26:03 UTC, Jonathan M Davis 
wrote:

 assert(0) does indeed turn into a HLT instruction in -release 
 mode (as does any assertion that's known to be false at compile 
 time). It's a special case intended for marking code that's 
 supposed to be unreachable. Without -release, failed assertions 
 throw AssertErrors, so they're Errors not Exceptions, though I 
 think that scope statements currently catch and then rethrow 
 Throwable rather than Exception, which is why AssertErrors 
 would be affected.

 - Jonathan M Davis



It's a little confusing, because Andrei's book says that 
scope(failure) guarantees statement will be executed, if and only 
if the current scope is exited by throwing an 'exception'.

One could *mistakingly* take that as meaning, throwing an object 
rooted in the 'Exception' subclass of Throwable.

Of course assert actually throws an exception too, but it's a 
special kind of exception rooted in the 'Error' subclass of 
Throwable.

Perhaps, to be clearer, Andrei's book should have said: 
scope(failure) guarantees statement will be executed, if and only 
if the current scope is exited by throwing an object that 
inherits from type Throwable. That would include, objects rooted 
in the 'Exception' subclass, as well as objects rooted in 'Error' 
subclass.

In which case, there's no bug in that code. It's doing what it's 
meant to be doing.
Dec 17 2017
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, December 17, 2017 09:32:53 codephantom via Digitalmars-d-learn 
wrote:

 On Sunday, 17 December 2017 at 08:26:03 UTC, Jonathan M Davis

 wrote:

 assert(0) does indeed turn into a HLT instruction in -release
 mode (as does any assertion that's known to be false at compile
 time). It's a special case intended for marking code that's
 supposed to be unreachable. Without -release, failed assertions
 throw AssertErrors, so they're Errors not Exceptions, though I
 think that scope statements currently catch and then rethrow
 Throwable rather than Exception, which is why AssertErrors
 would be affected.

 - Jonathan M Davis


 It's a little confusing, because Andrei's book says that
 scope(failure) guarantees statement will be executed, if and only
 if the current scope is exited by throwing an 'exception'.

 One could *mistakingly* take that as meaning, throwing an object
 rooted in the 'Exception' subclass of Throwable.

 Of course assert actually throws an exception too, but it's a
 special kind of exception rooted in the 'Error' subclass of
 Throwable.

 Perhaps, to be clearer, Andrei's book should have said:
 scope(failure) guarantees statement will be executed, if and only
 if the current scope is exited by throwing an object that
 inherits from type Throwable. That would include, objects rooted
 in the 'Exception' subclass, as well as objects rooted in 'Error'
 subclass.

 In which case, there's no bug in that code. It's doing what it's
 meant to be doing.


Well, exception vs Exception can be confusing, but there is actually no
guarantee whatsoever spec-wise that scope statements work with Errors - the
same goes for any other clean-up code (such as destructors or finally
blocks). Some stuff like nothrow functions make it impossible, and Walter
has been adamant that trying to recover from an Error is an incredibly bad
idea and that programs in general are far better off exiting on an Error
than trying to do any recovery from it (since by definition, when an Error
is thrown, the program is in a buggy and unknown state). Given Walter's
stance on it, I'm honestly a bit surprised that Error is a Throwable rather
than being designed to just immediately kill the program right then an
there, though under _rare_ circumstances, when you know exactly what you're
doing aand catch the Error pretty much at the point it was thrown, you can
get away with it for certain types of Errors like OutOfMemoryError.

However, at some point a while back, someone went into druntime and made it
so that destructors, scope statements, and finally blocks run as much as
possible when Errors are thrown - which isn't always but is a lot of the
time. So, on some level, right now, clean-up code does get run when an Error
is thrown, but it's not guaranteed and can't be guaranteed. Given Walter's
stance on it, I'm surprised that druntime has never been changed so that it
only does clean-up on Exceptions, but for now at least, we have a bit of a
halfway mess with how Errors are handled.

Basically, code shouldn't be relying on Errors either triggering or not
triggering clean-up code such as scope statements. They are only guaranteed
to run when an Exception is thrown and may or may not run when any other
kind of Throwable is thrown.

- Jonathan M Davis
Dec 17 2017
prev sibling parent Anonymouse <asdf asdf.net> writes:
On Sunday, 17 December 2017 at 08:10:06 UTC, codephantom wrote:

 On Sunday, 17 December 2017 at 00:10:27 UTC, Anonymouse wrote:

 If you return inside a scopeguard though, the exception (or 
 error!) is swallowed. https://run.dlang.io/is/GEtQ6D


 The scope guard will not 'swallow' it, if you use -release mode.

 Which suggests to me, that assert(0) operates differently when 
 'not' using -release mode.

 My 'guess' is, that release mode issues a halt instruction, 
 whereas in non-release mode it is just an exception like any 
 other exception.


Apologies. I chose assert(0) just to show that it catches Errors 
too, so just replace it with throw new Exception and it behaves 
the same without opening that can of worms.
Dec 17 2017
prev sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 12/16/2017 12:51 PM, kerdemdemir wrote:


 I am getting a crush while calling requestHTTP
 function :


That sounds like "having a crush", which happens to me too but I think 
you meant "crash". :o)

Ali
Dec 16 2017