www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Safe method wont check dangling pointer?

reply "lzzll" <ownrepos gmail.com> writes:
Looks like dangling point is not checked even in method mark as 
safe.
Example:
---
import std.stdio;

class A {
	int value;
	void set_value(int value)  safe {
		this.value = value;
	}
}

void test_safe(A a)  safe {
	a.set_value(1);
}

int main(string[] args) {
	A a = new A();
	test_safe(a);
	test_safe(null);
	test_safe(*(&a+100));
	
	writeln("done.");
	return 0;
}
---
test_safe(null);
and
test_safe(*(&a+100));
will cause segmentation fault.

I guess reason is check dangling pointer is very inefficient.
I found another post about this
http://forum.dlang.org/thread/llezieyytpcbcaoqeajz forum.dlang.org#post-miyvktgkczatvoguawda:40forum.dlang.org
null pointer is not a safety problem, but pointer like *(&a+100) 
maybe.

Regard.
Apr 14 2014
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Apr 2014 19:28:06 -0400, lzzll <ownrepos gmail.com> wrote:


 Looks like dangling point is not checked even in method mark as safe.
 Example:
 ---
 import std.stdio;

 class A {
 	int value;
 	void set_value(int value)  safe {
 		this.value = value;
 	}
 }

 void test_safe(A a)  safe {
 	a.set_value(1);
 }

 int main(string[] args) {
 	A a = new A();
 	test_safe(a);
 	test_safe(null);
 	test_safe(*(&a+100));
 	
 	writeln("done.");
 	return 0;
 }
 ---
 test_safe(null);
 and
 test_safe(*(&a+100));
 will cause segmentation fault.


Safe cannot verify its inputs. main() is not marked as safe, therefore it  
will not help.

But even so, dereferencing null is  safe, since it does not corrupt  
memory. Your *(&a + 100) will definitely not compile if main is marked  
 safe.

-Steve
Apr 14 2014
parent reply "lzzll" <ownrepos gmail.com> writes:
On Monday, 14 April 2014 at 23:44:47 UTC, Steven Schveighoffer 
wrote:

 On Mon, 14 Apr 2014 19:28:06 -0400, lzzll <ownrepos gmail.com> 
 wrote:


 Looks like dangling point is not checked even in method mark 
 as safe.
 Example:
 ---
 import std.stdio;

 class A {
 	int value;
 	void set_value(int value)  safe {
 		this.value = value;
 	}
 }

 void test_safe(A a)  safe {
 	a.set_value(1);
 }

 int main(string[] args) {
 	A a = new A();
 	test_safe(a);
 	test_safe(null);
 	test_safe(*(&a+100));
 	
 	writeln("done.");
 	return 0;
 }
 ---
 test_safe(null);
 and
 test_safe(*(&a+100));
 will cause segmentation fault.


 Safe cannot verify its inputs. main() is not marked as safe, 
 therefore it will not help.

 But even so, dereferencing null is  safe, since it does not 
 corrupt memory. Your *(&a + 100) will definitely not compile if 
 main is marked  safe.

 -Steve


That's correct.
Let me ask:
1. That's mean if I write a safe library and another guy use it 
in the wrong way, it still not really safe, right?
2. In the real world use, if I received a segmentation fault that 
mean I had to get the core dump and trace where is the problem, 
that's all right. But if I not received anything but actually the 
bad memory has been write or leak, that's the security issue.
3. I hope it will be truly safe in the future, prevent the access 
to dangling pointer, is there any plan or idea for this?
Apr 14 2014
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
lzzll:


 1. That's mean if I write a safe library and another guy use it 
 in the wrong way, it still not really safe, right?


Right.



 3. I hope it will be truly safe in the future, prevent the 
 access to dangling pointer, is there any plan or idea for this?


How do you suggest to design & implement it?

Bye,
bearophile
Apr 14 2014
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Apr 2014 21:10:57 -0400, lzzll <ownrepos gmail.com> wrote:


 Let me ask:
 1. That's mean if I write a safe library and another guy use it in the  
 wrong way, it still not really safe, right?


Garbage in, garbage out. The safe function must have reasonable  
expectations, and it's up to you to meet them. Is it "mean"? I don't think  
so. I think you have to adjust what you think  safe means.


 2. In the real world use, if I received a segmentation fault that mean I  
 had to get the core dump and trace where is the problem, that's all  
 right. But if I not received anything but actually the bad memory has  
 been write or leak, that's the security issue.


If it's for a null pointer, you will not have a memory corruption.


 3. I hope it will be truly safe in the future, prevent the access to  
 dangling pointer, is there any plan or idea for this?


This is actually impossible to implement.

-Steve
Apr 14 2014
parent reply "lzzll" <ownrepos gmail.com> writes:
Let me show some exmaple on c, and two common memory error detect 
tool.

example 1 (stack overflow):
---
int a = 100;
printf("%p\n", &a);

int *b = &a+1;
printf("%p\n", &b);

*b = 100;
---
valgrind: nothing detected
address sanitizer: ==1996== ERROR: AddressSanitizer: 
stack-buffer-overflow on address 0x7fffc976dbc4

example 2 (cross address)
---
int a = 100;
int b = 200;
printf("%p\n", &a);
printf("%p\n", &b);

int *c = &a+(&b-&a);
printf("%p\n", c);

*c = 100;
---
Of course it can't be detected.

example 3 (heap overflow)
---
int *a = (int*) malloc(sizeof(int));
printf("%p\n", a);

int *b = a + 1;
printf("%p\n", b);

*b = 100;
---
valgrind: Address 0x51f0044 is 0 bytes after a block of size 4 
alloc'd
address sanitizer: AddressSanitizer: heap-buffer-overflow on 
address 0x60040000dff4

It's possible to a certain extent.
Reference:
http://valgrind.org/docs/manual/mc-manual.html#mc-manual.vaddress
http://code.google.com/p/address-sanitizer/wiki/AddressSanitizerAlgorithm

I understand implemented this is hard and it need huge cost.
It still be useful if we only use it to detect memory error and 
trun it off when release.
I'll be glad if I can see it on D after some years.
Apr 14 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/14/2014 8:14 PM, lzzll wrote:

 I understand implemented this is hard and it need huge cost.
 It still be useful if we only use it to detect memory error and trun it off
when
 release.
 I'll be glad if I can see it on D after some years.


Valgrind is an incredibly useful tool, but programs run terribly slowly under
it.
Apr 14 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/14/14, 10:47 PM, Walter Bright wrote:

 On 4/14/2014 8:14 PM, lzzll wrote:

 I understand implemented this is hard and it need huge cost.
 It still be useful if we only use it to detect memory error and trun
 it off when
 release.
 I'll be glad if I can see it on D after some years.


 Valgrind is an incredibly useful tool, but programs run terribly slowly
 under it.


ASAN = like valgrind but a crapton faster. -- Andrei
Apr 15 2014
parent "lzzll" <ownrepos gmail.com> writes:
On Tuesday, 15 April 2014 at 07:09:34 UTC, Andrei Alexandrescu 
wrote:

 On 4/14/14, 10:47 PM, Walter Bright wrote:

 On 4/14/2014 8:14 PM, lzzll wrote:

 I understand implemented this is hard and it need huge cost.
 It still be useful if we only use it to detect memory error 
 and trun
 it off when
 release.
 I'll be glad if I can see it on D after some years.


 Valgrind is an incredibly useful tool, but programs run 
 terribly slowly
 under it.


 ASAN = like valgrind but a crapton faster. -- Andrei


I think if stuff like this implemented on D will be more faster,
because D have "normal pointer" and "raw pointer",
that mean we only need check them on convert.

example:
A a = new A(); //a is safe
A *b = *a+1; //from normal pointer to raw pointer, no need check
A c = *b; //from raw point to normal point, so check here

But it require the code of D itself is safe, and asan is much 
low-level.
Apr 15 2014
prev sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:


 Valgrind is an incredibly useful tool, but programs run 
 terribly slowly under it.


On the other hand the C/C++ world in the last years has seen
numerous advancements that D should keep an eye on. If you look
at the latest versions of LLVM-Clang and GCC you see various
"sanitizers" (available as built-in tools of the compiler) that
don't use too much memory, don't slow down your code too much,
and catch dangling or wrong pointers, integer overflows,
past-by-one errors, and more. One of those tools is less needed
by D (thanks to the good management of the array bounds), but the
others are nice.

Bye,
bearophile
Apr 15 2014
next sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 15 April 2014 at 09:33:19 UTC, bearophile wrote:

 Walter Bright:


 Valgrind is an incredibly useful tool, but programs run 
 terribly slowly under it.


 On the other hand the C/C++ world in the last years has seen
 numerous advancements that D should keep an eye on. If you look
 at the latest versions of LLVM-Clang and GCC you see various
 "sanitizers" (available as built-in tools of the compiler) that
 don't use too much memory, don't slow down your code too much,
 and catch dangling or wrong pointers, integer overflows,
 past-by-one errors, and more. One of those tools is less needed
 by D (thanks to the good management of the array bounds), but 
 the
 others are nice.

 Bye,
 bearophile


Except, as far as I am aware, they only work on GNU/Linux and Mac
OS X, leaving out all other operating systems out there.


--
Paulo
Apr 15 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Paulo Pinto:


 Except, as far as I am aware, they only work on GNU/Linux and 
 Mac OS X, leaving out all other operating systems out there.


Is "-fsanitize=integer" not available in the Windows version of 
Clang?

Perhaps it's a good idea to create something similar to the 
sanitizers (the memory one, etc) that work on all D compilers.

Bye,
bearophile
Apr 15 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/15/2014 4:05 AM, bearophile wrote:

 Perhaps it's a good idea to create something similar to the sanitizers (the
 memory one, etc) that work on all D compilers.


That's what  safe is for.
Apr 15 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:


 That's what  safe is for.


I think those sanitizers (but the integer-related one) are meant 
to help D programmers catch bugs in  system code.

Bye,
bearophile
Apr 15 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/15/2014 10:26 AM, bearophile wrote:

 Walter Bright:


 That's what  safe is for.


 I think those sanitizers (but the integer-related one) are meant to help D
 programmers catch bugs in  system code.


I understand that. I've written my own sanitizers in the past and used them 
heavily. The big advantage of  safe is that it offers a guarantee, sanitizers
do 
not.

Very little, however, of even a hardcore app needs to be  system. What little 
remains is often  system for performance reasons, where you'd turn off a 
sanitizer anyway.

To sum up, a sanitizer for D offers little incremental benefit, and has a 
substantial implementation cost. Such cost would take away from other 
improvements to D that would be far more valuable.
Apr 15 2014
next sibling parent "lzzll" <ownrepos gmail.com> writes:
On Tuesday, 15 April 2014 at 17:40:13 UTC, Walter Bright wrote:

 On 4/15/2014 10:26 AM, bearophile wrote:

 Walter Bright:


 That's what  safe is for.


 I think those sanitizers (but the integer-related one) are 
 meant to help D
 programmers catch bugs in  system code.


 I understand that. I've written my own sanitizers in the past 
 and used them heavily. The big advantage of  safe is that it 
 offers a guarantee, sanitizers do not.

 Very little, however, of even a hardcore app needs to be 
  system. What little remains is often  system for performance 
 reasons, where you'd turn off a sanitizer anyway.

 To sum up, a sanitizer for D offers little incremental benefit, 
 and has a substantial implementation cost. Such cost would take 
 away from other improvements to D that would be far more 
 valuable.


I agree about implementation cost, and I didn't have enough 
skills to do it.
I think  safe just a guide or helper, it doesn't offer a 
guarantee,
because phobos and druntime use a lot of pointer, but less test 
with them (or didn't release?).

Also we need pointer because many library, kernel of 
windows/linux/bsd, communication api from outer device is written 
in c/c++.
Free from pointer is a good dream although it is unrealistic.

Thanks that valgrind can use with d, so it's the only choise now.
Apr 15 2014
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:


 What little remains is often  system for performance reasons, 
 where you'd turn off a sanitizer anyway.


You are wrong. Both integer overflow sanitizers and those various 
modern memory/address sanitizers are not meant to be used in the 
final release of the code. They are meant to be used one at a 
time during testing or debugging (and the memory sanitizers slow 
down the code 2-3 times, so they can be used in many cases of 
debugging), and then later disabled.

Bye,
bearophile
Apr 15 2014
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Walter Bright:


 Such cost would take away from other improvements to D that
 would be far more valuable.


But I agree with this. In this moment there are more important 
things to do (like finishing synchronized, scope, SIMD, vector 
operations, operator
overloading, dynamic libraries, GC, and more).

Bye,
bearophile
Apr 15 2014
prev sibling parent Paulo Pinto <pjmlp progtools.org> writes:
Am 15.04.2014 13:05, schrieb bearophile:

 Paulo Pinto:


 Except, as far as I am aware, they only work on GNU/Linux and Mac OS
 X, leaving out all other operating systems out there.


 Is "-fsanitize=integer" not available in the Windows version of Clang?

 Perhaps it's a good idea to create something similar to the sanitizers
 (the memory one, etc) that work on all D compilers.

 Bye,
 bearophile


I had the idea that they require some kind of OS MMU API support, which 
neither Windows, nor commercial UNIX or embedded OS offer.

Maybe I am wrong as I just saw some LLVM presentations about it and 
currently only code C++ occasionally so I don't follow everything in C++ 
world.

--
Paulo
Apr 15 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/15/2014 2:33 AM, bearophile wrote:

 On the other hand the C/C++ world in the last years has seen
 numerous advancements that D should keep an eye on. If you look
 at the latest versions of LLVM-Clang and GCC you see various
 "sanitizers" (available as built-in tools of the compiler) that
 don't use too much memory, don't slow down your code too much,
 and catch dangling or wrong pointers, integer overflows,
 past-by-one errors, and more. One of those tools is less needed
 by D (thanks to the good management of the array bounds), but the
 others are nice.


Or you can use  safe, which doesn't slow your code down at all, at least for
the 
memory corruption problems.
Apr 15 2014
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 14 Apr 2014 23:14:06 -0400, lzzll <ownrepos gmail.com> wrote:


 Let me show some exmaple on c, and two common memory error detect tool.

 example 1 (stack overflow):
 ---
 int a = 100;
 printf("%p\n", &a);

 int *b = &a+1;
 printf("%p\n", &b);

 *b = 100;
 ---


import std.stdio;

void main()  safe
{
   int a = 100;
   writeln(&a); // line 6

   int *b = &a+1; // line 8
   *b = 100;
}

testsafe.d(6): Error: cannot take address of local a in  safe function main
testsafe.d(6): Error: safe function 'D main' cannot call system function  
'std.stdio.writeln!(int*).writeln' (wow, really?)
testsafe.d(8): Error: cannot take address of local a in  safe function main
testsafe.d(8): Error: pointer arithmetic not allowed in  safe functions

 safe prevents memory errors by preventing pointer arithmetic, and taking  
addresses of stack locals.

I was not aware that writeln is also unsafe! That must be a bug. Even  
writeln!int is unsafe.


 valgrind: nothing detected
 address sanitizer: ==1996== ERROR: AddressSanitizer:  
 stack-buffer-overflow on address 0x7fffc976dbc4

 example 2 (cross address)
 ---
 int a = 100;
 int b = 200;
 printf("%p\n", &a);
 printf("%p\n", &b);

 int *c = &a+(&b-&a);
 printf("%p\n", c);

 *c = 100;
 ---


illegal to use c's initializer in  safe code.


 Of course it can't be detected.


It's also not a memory corruption :)

optimizer will turn int *c = &a+(&b-&a) to int *c = &b;


 example 3 (heap overflow)
 ---
 int *a = (int*) malloc(sizeof(int));
 printf("%p\n", a);

 int *b = a + 1;
 printf("%p\n", b);

 *b = 100;
 ---


illegal to use b's initializer in  safe code. The malloc is allowed, but  
only GC.malloc. C's malloc is not  safe as it requires free.


 valgrind: Address 0x51f0044 is 0 bytes after a block of size 4 alloc'd
 address sanitizer: AddressSanitizer: heap-buffer-overflow on address  
 0x60040000dff4

 It's possible to a certain extent.
 Reference:
 http://valgrind.org/docs/manual/mc-manual.html#mc-manual.vaddress
 http://code.google.com/p/address-sanitizer/wiki/AddressSanitizerAlgorithm

 I understand implemented this is hard and it need huge cost.
 It still be useful if we only use it to detect memory error and trun it  
 off when release.
 I'll be glad if I can see it on D after some years.


It's impossible to have 100% coverage. Using sentinels and instrumenting  
can help find memory errors, but just not allowing memory-unsafe code is  
more efficient and guaranteed.

Note that valgrind should be able to instrument D code as well.

-Steve
Apr 15 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 15 April 2014 at 12:12:00 UTC, Steven Schveighoffer 
wrote:

 The malloc is allowed, but only GC.malloc. C's malloc is not 
  safe as it requires free.


I think the reason is that it returns `void*` (needs a cast and 
has no length) and points to uninitialized memory, not because 
you need to free it manually. A memory leak is still memory safe.
Apr 15 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/15/2014 6:55 AM, "Marc Schütz" <schuetzm gmx.net>" wrote:

 I think the reason is that it returns `void*` (needs a cast and has no length)
 and points to uninitialized memory, not because you need to free it manually.


Yes.


 A memory leak is still memory safe.


True, which is why we can guarantee memory safety with a GC.
Apr 15 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
I think  safe guarantees are much stronger if you _start_ with 
 safe main and only interact with system code via small  trusted 
gateways. If it is  system code calling  safe, low-level trickery 
it allows is just too powerful to avoid with reasonably fast 
checks.
Apr 15 2014