www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why dont dlang check NullPointer?

reply "zhmt" <zhmtzhmt qq.com> writes:
	class A
	{
		void test()
		{
			writeln("test");
		}
	}

	try
	{
		A a = null;
		a.test();
	}catch(Throwable t)
	{
		writeln(t.msg);
	}


The code above will not print a exception message, but crashes 
instead.


I dont think this a good choice for most scenes. For example,I 
developed many modules in my server, and add a new module now. If 
null exception happens, I hope the server continue running, not 
crash. If the server continue running, the functions offered by 
old modules can be used by users, only the new module stop 
serving.



In short words, I want to catch something like 
NullPointerException.

Is this possible?
Mar 26 2015
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Friday, 27 March 2015 at 03:46:49 UTC, zhmt wrote:

 The code above will not print a exception message, but crashes 
 instead.


It actually will throw on Windows and can be tricked into it on 
Linux; it is an operating system thing more than a language thing.

But...


 If null exception happens, I hope the server continue running, 
 not crash. If the server continue running, the functions 
 offered by old modules can be used by users, only the new 
 module stop serving.


The best way to do that is to separate the server modules into 
independent processes. Then if one crashes, the others keep 
running without fear of corruption.

So instead of server modules, try doing mini servers that 
communicate with the main server. This is how a lot of newer 
programs are written because of the reliability and security 
benefits it offers.
Mar 26 2015
parent reply "zhmt" <zhmtzhmt qq.com> writes:
 The best way to do that is to separate the server modules into 
 independent processes. Then if one crashes, the others keep 
 running without fear of corruption.

 So instead of server modules, try doing mini servers that 
 communicate with the main server. This is how a lot of newer 
 programs are written because of the reliability and security 
 benefits it offers.


But this will make the developement more difficult for me, or not 
acceptable.

Is there any other ways?
Mar 26 2015
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Friday, 27 March 2015 at 03:59:30 UTC, zhmt wrote:

 The best way to do that is to separate the server modules into 
 independent processes. Then if one crashes, the others keep 
 running without fear of corruption.

 So instead of server modules, try doing mini servers that 
 communicate with the main server. This is how a lot of newer 
 programs are written because of the reliability and security 
 benefits it offers.


 But this will make the developement more difficult for me, or 
 not acceptable.

 Is there any other ways?


http://www.deadalnix.me/2012/03/24/get-an-exception-from-a-segfault-on-linux-x86-and-x86_64-using-some-black-magic/

There is a hook in the runtime to enable this if you want.

BUT, null pointer exception or not, Adam is right. Have your 
stuff run in multiple process that you can restart. This is more 
reliable, this is more secure, this is easier to update without 
downtime, and so on... This is far superior solution for server 
stuff.
Mar 26 2015
next sibling parent reply "zhmt" <zhmtzhmt qq.com> writes:
On Friday, 27 March 2015 at 04:13:01 UTC, deadalnix wrote:

 On Friday, 27 March 2015 at 03:59:30 UTC, zhmt wrote:

 The best way to do that is to separate the server modules 
 into independent processes. Then if one crashes, the others 
 keep running without fear of corruption.

 So instead of server modules, try doing mini servers that 
 communicate with the main server. This is how a lot of newer 
 programs are written because of the reliability and security 
 benefits it offers.


 But this will make the developement more difficult for me, or 
 not acceptable.

 Is there any other ways?


 http://www.deadalnix.me/2012/03/24/get-an-exception-from-a-segfault-on-linux-x86-and-x86_64-using-some-black-magic/

 There is a hook in the runtime to enable this if you want.

 BUT, null pointer exception or not, Adam is right. Have your 
 stuff run in multiple process that you can restart. This is 
 more reliable, this is more secure, this is easier to update 
 without downtime, and so on... This is far superior solution 
 for server stuff.


multi-process means crashes are isolated by process, but isolated 
by thread may be more handy.



This can make dlang app developed by most developers more 
reliable.
Mar 26 2015
parent reply Shammah Chancellor <anonymous coward.com> writes:
On 2015-03-27 05:34:59 +0000, zhmt said:


 On Friday, 27 March 2015 at 04:13:01 UTC, deadalnix wrote:

 On Friday, 27 March 2015 at 03:59:30 UTC, zhmt wrote:

 
 The best way to do that is to separate the server modules into 
 independent processes. Then if one crashes, the others keep running 
 without fear of corruption.
 
 So instead of server modules, try doing mini servers that communicate 
 with the main server. This is how a lot of newer programs are written 
 because of the reliability and security benefits it offers.


 
 But this will make the developement more difficult for me, or not acceptable.
 
 Is there any other ways?


 
 http://www.deadalnix.me/2012/03/24/get-an-exception-from-a-segfault-on-linux-x86-and-x86_64-usi
g-some-black-magic/ 
 
 
 There is a hook in the runtime to enable this if you want.
 
 BUT, null pointer exception or not, Adam is right. Have your stuff run 
 in multiple process that you can restart. This is more reliable, this 
 is more secure, this is easier to update without downtime, and so on... 
 This is far superior solution for server stuff.


 
 multi-process means crashes are isolated by process, but isolated by 
 thread may be more handy.
 

 
 This can make dlang app developed by most developers more reliable.


All the languages you mention run in a VM.   In the case of a systems 
language like D, the operation system itself is intercepting the 
reference to invalid memory and sending a SIGSEG to the process.  The 
default handler causes the process to immediately terminate.   Having 
the D runtime do something different in the SIGSEG handler by default 
would be bad form.

-Shammah
Mar 27 2015
parent "deadalnix" <deadalnix gmail.com> writes:
On Friday, 27 March 2015 at 14:39:36 UTC, Shammah Chancellor 
wrote:

 All the languages you mention run in a VM.   In the case of a 
 systems language like D, the operation system itself is 
 intercepting the reference to invalid memory and sending a 
 SIGSEG to the process.  The default handler causes the process 
 to immediately terminate.   Having the D runtime do something 
 different in the SIGSEG handler by default would be bad form.

 -Shammah


Most VM use segfault trapping for null check.
Mar 27 2015
prev sibling next sibling parent "zhmt" <zhmtzhmt qq.com> writes:
 Is there any other ways?


 http://www.deadalnix.me/2012/03/24/get-an-exception-from-a-segfault-on-linux-x86-and-x86_64-using-some-black-magic/

 There is a hook in the runtime to enable this if you want.


I will try this black magic,Thanks.
Mar 26 2015
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 3/27/15 12:13 AM, deadalnix wrote:

 On Friday, 27 March 2015 at 03:59:30 UTC, zhmt wrote:

 The best way to do that is to separate the server modules into
 independent processes. Then if one crashes, the others keep running
 without fear of corruption.

 So instead of server modules, try doing mini servers that communicate
 with the main server. This is how a lot of newer programs are written
 because of the reliability and security benefits it offers.


 But this will make the developement more difficult for me, or not
 acceptable.

 Is there any other ways?


 http://www.deadalnix.me/2012/03/24/get-an-exception-from-a-segfault-on-linux-x86-and-x86_64-using-some-black-magic/


 There is a hook in the runtime to enable this if you want.

 BUT, null pointer exception or not, Adam is right. Have your stuff run
 in multiple process that you can restart. This is more reliable, this is
 more secure, this is easier to update without downtime, and so on...
 This is far superior solution for server stuff.


Please note, this is NOT a null pointer exception, it's a segfault 
exception. This can happen with corruption (absolutely should not 
continue) as well as forgetting to initialize a variable (dangerous if 
not handled correctly, but still feasible to continue). It may not be as 
black and white as if it's a null pointer that was dereferenced or not. 
I highly recommend terminating the process.

As for the original question (why does D do this?), it's because the 
system ALREADY catches null pointer access. To add additional checks 
would slow down the system. And as you can see, you can hook these 
mechanisms to actually throw an exception, but this is a relatively 
recent development.

In addition, as I mentioned, a seg fault can occur for a number of 
reasons, and D takes the position that you really should just terminate 
the process if this happens.

The reason using multiple processes is more secure and reliable is 
because a rogue thread (one that has segfaulted because of a memory 
corruption error) can corrupt data in all your other threads. A separate 
process cannot.

-Steve
Mar 27 2015
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 27 March 2015 at 11:17:54 UTC, Steven Schveighoffer 
wrote:

 Please note, this is NOT a null pointer exception, it's a 
 segfault exception. This can happen with corruption (absolutely 
 should not continue) as well as forgetting to initialize a 
 variable (dangerous if not handled correctly, but still 
 feasible to continue). It may not be as black and white as if 
 it's a null pointer that was dereferenced or not. I highly 
 recommend terminating the process.


A segfault can also be I/O error on a mmap'ed file, so 
termination is not always the right action.
Mar 31 2015
prev sibling parent reply Timothee Cour via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, Mar 26, 2015 at 9:13 PM, deadalnix via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Friday, 27 March 2015 at 03:59:30 UTC, zhmt wrote:


 The best way to do that is to separate the server modules into
 independent processes. Then if one crashes, the others keep running wit=






hout

 fear of corruption.

 So instead of server modules, try doing mini servers that communicate
 with the main server. This is how a lot of newer programs are written
 because of the reliability and security benefits it offers.


 But this will make the developement more difficult for me, or not
 acceptable.

 Is there any other ways?


 http://www.deadalnix.me/2012/03/24/get-an-exception-from-a-
 segfault-on-linux-x86-and-x86_64-using-some-black-magic/

 There is a hook in the runtime to enable this if you want.



You post only mentions linux, not OSX (and a comment showed:

"According to several people I talked to, it is possible to do a similar
stuff on BSD like systems (including OSX). But I=E2=80=99m really not a spe=
cialist
of such platforms, so I can=E2=80=99t really explain you how."

Could anyone post such a solution?

If there's no easy way to do it, there should be a way (eg a compiler
option) to throw an exception on null pointer access. Even if it's unsafe,
it would help for debugging (eg printing relevant application specific
context). This is made worse by the fact that stacktraces are not very
helpful on OSX (eg line number often missing etc).




 BUT, null pointer exception or not, Adam is right. Have your stuff run in
 multiple process that you can restart. This is more reliable, this is mor=


e

 secure, this is easier to update without downtime, and so on... This is f=


ar

 superior solution for server stuff.
Mar 28 2015
parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Saturday, 28 March 2015 at 22:53:54 UTC, Timothee Cour wrote:

 If there's no easy way to do it, there should be a way (eg a 
 compiler
 option) to throw an exception on null pointer access. Even if 
 it's unsafe,
 it would help for debugging (eg printing relevant application 
 specific
 context). This is made worse by the fact that stacktraces are 
 not very
 helpful on OSX (eg line number often missing etc).


In that case, it should be an Error, not an Exception, since 
it'll be disabled in release mode.
Mar 28 2015
parent reply "zhmt" <zhmtzhmt qq.com> writes:
Especially working with fibers, ability to catch 
NullPointerException is more important. If a NullPointerException 
is caught , only one fiber terminates, otherwise, the whole 
server crashes.

If the server is something like 
webserver(stateless),multi-process is ok. But I am aiming to 
develope a mmorpg server, it is stateful, so it is not allowed to 
crash entirely.

Maybe the solution is to make use of  a script engine (such as 
lua), but the benefit of choosing dlang is lost.
Mar 30 2015
parent reply "w0rp" <devw0rp gmail.com> writes:
On Tuesday, 31 March 2015 at 03:58:33 UTC, zhmt wrote:

 Especially working with fibers, ability to catch 
 NullPointerException is more important. If a 
 NullPointerException is caught , only one fiber terminates, 
 otherwise, the whole server crashes.

 If the server is something like 
 webserver(stateless),multi-process is ok. But I am aiming to 
 develope a mmorpg server, it is stateful, so it is not allowed 
 to crash entirely.

 Maybe the solution is to make use of  a script engine (such as 
 lua), but the benefit of choosing dlang is lost.


I argue that instead of catching NullPointerExceptions, you 
should make them never happen. This is where Option types come 
in. Rather than using T*, use an Option!T and force yourself to 
always check for null gracefully. Also use Some!T (a.k.a NotNull) 
and get classes and pointers which are verified via contracts 
that they are not null.

I have already implemented such a thing here.

https://w0rp.com/project/dstruct/dstruct/option/

If you don't like it, you can always implement a similar thing 
yourself.

Now at this point you might wish for this to gain some benefits 
from being a language feature. I think I will agree, but who 
knows if that will ever happen.
Mar 31 2015
next sibling parent "w0rp" <devw0rp gmail.com> writes:
I should say Option!(T*) and Some!(T*), as that's what is.
Mar 31 2015
prev sibling parent reply "zhmt" <zhmtzhmt qq.com> writes:
On Tuesday, 31 March 2015 at 07:48:08 UTC, w0rp wrote:


 I argue that instead of catching NullPointerExceptions, you 
 should make them never happen. This is where Option types come 
 in. Rather than using T*, use an Option!T and force yourself to 
 always check for null gracefully. Also use Some!T (a.k.a 
 NotNull) and get classes and pointers which are verified via 
 contracts that they are not null.

 I have already implemented such a thing here.

 https://w0rp.com/project/dstruct/dstruct/option/

 If you don't like it, you can always implement a similar thing 
 yourself.

 Now at this point you might wish for this to gain some benefits 
 from being a language feature. I think I will agree, but who 
 knows if that will ever happen.


Maybe this a good direction.

A pointer or reference is replaced by a struct (a pointer 
holder?), and the struct will check if the pointer is null.

It is a good idea, I could check the pointer myself now, and 
stopping worring about server crashing, that's enough.

Excellent job, Thank you!
Mar 31 2015
parent "w0rp" <devw0rp gmail.com> writes:
On Tuesday, 31 March 2015 at 08:17:29 UTC, zhmt wrote:

 On Tuesday, 31 March 2015 at 07:48:08 UTC, w0rp wrote:


 I argue that instead of catching NullPointerExceptions, you 
 should make them never happen. This is where Option types come 
 in. Rather than using T*, use an Option!T and force yourself 
 to always check for null gracefully. Also use Some!T (a.k.a 
 NotNull) and get classes and pointers which are verified via 
 contracts that they are not null.

 I have already implemented such a thing here.

 https://w0rp.com/project/dstruct/dstruct/option/

 If you don't like it, you can always implement a similar thing 
 yourself.

 Now at this point you might wish for this to gain some 
 benefits from being a language feature. I think I will agree, 
 but who knows if that will ever happen.


 Maybe this a good direction.

 A pointer or reference is replaced by a struct (a pointer 
 holder?), and the struct will check if the pointer is null.

 It is a good idea, I could check the pointer myself now, and 
 stopping worring about server crashing, that's enough.

 Excellent job, Thank you!


It's a commmon pattern now. I think I personally copied the way 
Scala does it. I used contracts for the checks, and my hope is 
that with the contracts off and the right optimisations, you can 
get roughly the same code generated as if you didn't use the 
option types at all.

I think my inclusion of opApply on the Option type was a mistake. 
I will probably remove that in future. I have a function for 
creating a range from them anyway. Feel free to try my types if 
you like. I haven't tested them enough to figure out how 
effective they are, or there are any bugs I missed.
Mar 31 2015
prev sibling next sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Friday, 27 March 2015 at 03:46:49 UTC, zhmt wrote:

 	class A
 	{
 		void test()
 		{
 			writeln("test");
 		}
 	}

 	try
 	{
 		A a = null;
 		a.test();
 	}catch(Throwable t)
 	{
 		writeln(t.msg);
 	}


 The code above will not print a exception message, but crashes 
 instead.


 I dont think this a good choice for most scenes. For example,I 
 developed many modules in my server, and add a new module now. 
 If null exception happens, I hope the server continue running, 
 not crash. If the server continue running, the functions 
 offered by old modules can be used by users, only the new 
 module stop serving.



 In short words, I want to catch something like 
 NullPointerException.

 Is this possible?


GCC has the switch -fno-non-call-exceptions, but not sure if GDC 
does. You could ask in the gdc forum?
Mar 27 2015
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
zhmt:


 In short words, I want to catch something like 
 NullPointerException.

 Is this possible?


One solution is to add null tests to D in nonrelease mode. A 
better solution is to modify D to remove all or most chances of 
dereferencing null pointers and class references.

Bye,
bearophile
Mar 27 2015
prev sibling next sibling parent "w0rp" <devw0rp gmail.com> writes:
I'd be tempted to go way back to the very root of the problem 
starting with Tony Hoare again. Eliminate null as a possibility. 
That's a whole other subject, though.
Mar 27 2015
prev sibling parent "Gary Willoughby" <dev nomad.so> writes:
On Friday, 27 March 2015 at 03:46:49 UTC, zhmt wrote:

 In short words, I want to catch something like 
 NullPointerException.

 Is this possible?


Yes on Linux use: etc.linux.memoryerror which is undocumented for 
some reason.

https://github.com/D-Programming-Language/druntime/blob/master/src/etc/linux/memoryerror.d
Apr 06 2015