www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - array slice setting not bounds checked!

reply derick_eddington nospam.yashmoo.com writes:
import std.stdio;

void main ()
{
ubyte[] uba = new ubyte[123];

uba[-2 .. length+5] = 7;      // not bounds checked!

//uba[-2 .. length+5][] = 7;  // but this does throw ArrayBoundsError...

ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+5];

foreach (size_t i, ubyte ub; wtf)
writef("%.3d:%d ", i, ub);

writefln();
}
Apr 04 2005
parent reply derick_eddington nospam.yashmoo.com writes:
That should have been:
ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+7];  // not + 5

but same point...


In article <d2sa8a$518$1 digitaldaemon.com>, derick_eddington nospam.yashmoo.com
says...

import std.stdio;

void main ()
{
ubyte[] uba = new ubyte[123];

uba[-2 .. length+5] = 7;      // not bounds checked!

//uba[-2 .. length+5][] = 7;  // but this does throw ArrayBoundsError...

ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+5];

foreach (size_t i, ubyte ub; wtf)
writef("%.3d:%d ", i, ub);

writefln();
}
Apr 04 2005
next sibling parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

derick_eddington nospam.yashmoo.com schrieb am Mon, 4 Apr 2005 21:16:13 +0000
(UTC):

 That should have been:
 ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+7];  // not + 5

 but same point...


 In article <d2sa8a$518$1 digitaldaemon.com>,
derick_eddington nospam.yashmoo.com
 says...

import std.stdio;

void main ()
{
ubyte[] uba = new ubyte[123];

uba[-2 .. length+5] = 7;      // not bounds checked!

//uba[-2 .. length+5][] = 7;  // but this does throw ArrayBoundsError...

ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+5];

foreach (size_t i, ubyte ub; wtf)
writef("%.3d:%d ", i, ub);

writefln();
}




http://digitalmars.com/d/arrays.html


Unless the quote some lines lower is implemented, the quote above should
be changed to "can rely on" ?




Thomas


-----BEGIN PGP SIGNATURE-----

iD8DBQFCUbtJ3w+/yD4P9tIRAmSJAKDIq6UX/bEMpWl/Z36SEhZ5TYEw8ACgzleJ
vQo0cL7+DmGEBSttpuG7Pzw=
=w5eE
-----END PGP SIGNATURE-----
Apr 04 2005
parent reply derick_eddington place.com writes:
In article <9jq8i2-arr.ln1 lnews.kuehne.cn>, Thomas Kuehne says...

http://digitalmars.com/d/arrays.html


Unless the quote some lines lower is implemented, the quote above should
be changed to "can rely on" ?




Thomas


I remember reading that, but if bounds checking is turned on and the double
slice bounds checks, I think the other should.  I like the idea of having the
option of turning on and off bounds checking, but I would think bounds checking
should also be a feature that you can rely on to work comprehensively and
consistently when it is turned on and be a feature to use in release/production
code.  Why only go half way?
Apr 04 2005
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
derick_eddington place.com wrote:

 In article <9jq8i2-arr.ln1 lnews.kuehne.cn>, Thomas Kuehne says...
 

http://digitalmars.com/d/arrays.html


Unless the quote some lines lower is implemented, the quote above should
be changed to "can rely on" ?




By "may not rely on", it means that a program that needs ABC turned on 
in order to work correctly is incorrectly written.  As such, it will not 
work when compiled in release mode.








That switch is -release.  At least in the current DMD design.


 I remember reading that, but if bounds checking is turned on and the double
 slice bounds checks, I think the other should.  I like the idea of having the
 option of turning on and off bounds checking, but I would think bounds checking
 should also be a feature that you can rely on to work comprehensively and
 consistently when it is turned on


Entirely agreed.


 and be a feature to use in release/production code.  Why only go half way?


Disagreed.  ABC is left out of release builds for the same reason DBC is 
- because it is intended for checking that the program is working 
correctly, and having it in production code would slow the program down. 
  As such, programs are supposed to be written so that array indexes 
will never be out of bounds.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Apr 05 2005
parent reply derick_eddington yamoooo.com writes:
In article <d2uj5q$2hoh$1 digitaldaemon.com>, Stewart Gordon says...

derick_eddington place.com wrote:

 In article <9jq8i2-arr.ln1 lnews.kuehne.cn>, Thomas Kuehne says...
 

http://digitalmars.com/d/arrays.html


Unless the quote some lines lower is implemented, the quote above should
be changed to "can rely on" ?




By "may not rely on", it means that a program that needs ABC turned on 
in order to work correctly is incorrectly written.  As such, it will not 
work when compiled in release mode.








That switch is -release.  At least in the current DMD design.


 I remember reading that, but if bounds checking is turned on and the double
 slice bounds checks, I think the other should.  I like the idea of having the
 option of turning on and off bounds checking, but I would think bounds checking
 should also be a feature that you can rely on to work comprehensively and
 consistently when it is turned on


Entirely agreed.


 and be a feature to use in release/production code.  Why only go half way?


Disagreed.  ABC is left out of release builds for the same reason DBC is 
- because it is intended for checking that the program is working 
correctly, and having it in production code would slow the program down. 
  As such, programs are supposed to be written so that array indexes 
will never be out of bounds.

Stewart.


I think having the switch which disables ABC and DBC be called -release is fine.
What I meant was that if you want to have ABC turned on in your production code,
you should be able to rely on it; not in the sense of using it to control
program flow (like the arrays.html example) but rely on it to catch any and
every out-of-bounds (this would mean comphrensive ABC that could be used to
control program flow if you wanted, but I agree that's not good design).

Saying one should be a good enough programmer that you never allow overflow is a
great ideal and one I advocate but the plethora of overflow vulnerabilities says
to me it's not being widely achieved.  If you'd like to trade performance for
assurance, I think you should be able to; especially for programs where DBC and
ABC won't have a large impact on performance, and for important programs that
are exposed to a lot of untrusted input and use, I think not using -release for
your production version and being able to count on DBC and ABC would be a great
option and huge selling point for D.  If I'm running a financial app exposed to
anyone and I can afford more hardware to get whatever performance I need,
knowing I can rely on DBC and ABC would make D very attractive to me.
Apr 05 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 6 Apr 2005 00:51:58 +0000 (UTC), <derick_eddington yamoooo.com>  
wrote:

 In article <d2uj5q$2hoh$1 digitaldaemon.com>, Stewart Gordon says...

 derick_eddington place.com wrote:

 In article <9jq8i2-arr.ln1 lnews.kuehne.cn>, Thomas Kuehne says...


 http://digitalmars.com/d/arrays.html


 Unless the quote some lines lower is implemented, the quote above  
 should
 be changed to "can rely on" ?




 By "may not rely on", it means that a program that needs ABC turned on
 in order to work correctly is incorrectly written.  As such, it will not
 work when compiled in release mode.








 That switch is -release.  At least in the current DMD design.


 I remember reading that, but if bounds checking is turned on and the  
 double
 slice bounds checks, I think the other should.  I like the idea of  
 having the
 option of turning on and off bounds checking, but I would think bounds  
 checking
 should also be a feature that you can rely on to work comprehensively  
 and
 consistently when it is turned on


 Entirely agreed.


 and be a feature to use in release/production code.  Why only go half  
 way?


 Disagreed.  ABC is left out of release builds for the same reason DBC is
 - because it is intended for checking that the program is working
 correctly, and having it in production code would slow the program down.
  As such, programs are supposed to be written so that array indexes
 will never be out of bounds.

 Stewart.


 I think having the switch which disables ABC and DBC be called -release  
 is fine.
 What I meant was that if you want to have ABC turned on in your  
 production code,
 you should be able to rely on it; not in the sense of using it to control
 program flow (like the arrays.html example) but rely on it to catch any  
 and
 every out-of-bounds (this would mean comphrensive ABC that could be used  
 to
 control program flow if you wanted, but I agree that's not good design).

 Saying one should be a good enough programmer that you never allow  
 overflow is a
 great ideal and one I advocate but the plethora of overflow  
 vulnerabilities says
 to me it's not being widely achieved.  If you'd like to trade  
 performance for
 assurance, I think you should be able to; especially for programs where  
 DBC and
 ABC won't have a large impact on performance, and for important programs  
 that
 are exposed to a lot of untrusted input and use, I think not using  
 -release for
 your production version and being able to count on DBC and ABC would be  
 a great
 option and huge selling point for D.  If I'm running a financial app  
 exposed to
 anyone and I can afford more hardware to get whatever performance I need,
 knowing I can rely on DBC and ABC would make D very attractive to me.


See the other thread of this thread, I beleive you've found a bug (or a  
strange 'feature'). Making this thread of this thread err.. pointless?  
(perhaps?)

Regan
Apr 05 2005
prev sibling parent Stewart Gordon <smjg_1998 yahoo.com> writes:
derick_eddington yamoooo.com wrote:
<snip>

 Saying one should be a good enough programmer that you never allow
 overflow is a great ideal and one I advocate but the plethora of
 overflow vulnerabilities says to me it's not being widely achieved.


Exactly.  That's the point of ABC - it gives you the chance to fix the
bugs causing overflow.


 If you'd like to trade performance for assurance, I think you should 
 be able to; especially for programs where DBC and ABC won't have a 
 large impact on performance, and for important programs that are 
 exposed to a lot of untrusted input and use, I think not using 
 -release for your production version and being able to count on DBC 
 and ABC would be a great option and huge selling point for D.  If I'm 
 running a financial app exposed to anyone and I can afford more 
 hardware to get whatever performance I need, knowing I can rely on 
 DBC and ABC would make D very attractive to me.


If you want to release your programs with the ABC and DBC still in, 
nobody's going to stop you.  But do prepare to be thrown stones at by 
anyone who reads your code and finds that you rely on it.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on
the 'group where everyone may benefit.
Apr 06 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+7];  // not + 5


This is not a bug. Creating an array from a pointer does not do any bounds 
checking ever since a pointer does not store any bounds information. If you 
want to perform bounds checking you need to slice [a .. b] on an array and 
not a pointer. In fact one should even talk about slicing a pointer since 
that implies a pointer has a range - instead one should talk about defining 
an array.
Apr 04 2005
parent reply derick_eddington yahsoo.com writes:
What I consider the bug is the line:

uba[-2 .. length+5] = 7;      // not bounds checked!

which is allowing an overflowed slice-setting of a previously declared ubyte[].
wtf is for creating an array on the overflowed range to print it out and see
that 7 was indeed set before and after uba's correct range.  I understand that
slicing a pointer can't and shouldn't do bounds checking.

In article <d2sm9q$icv$1 digitaldaemon.com>, Ben Hinkle says...

 ubyte[] wtf = (uba.ptr - 2)[0 .. uba.length+7];  // not + 5


This is not a bug. Creating an array from a pointer does not do any bounds 
checking ever since a pointer does not store any bounds information. If you 
want to perform bounds checking you need to slice [a .. b] on an array and 
not a pointer. In fact one should even talk about slicing a pointer since 
that implies a pointer has a range - instead one should talk about defining 
an array.
Apr 05 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 6 Apr 2005 00:11:23 +0000 (UTC), <derick_eddington yahsoo.com>  
wrote:

 What I consider the bug is the line:

 uba[-2 .. length+5] = 7;      // not bounds checked!

 which is allowing an overflowed slice-setting of a previously declared  
 ubyte[].


Indeed. I believe you have found a bug (or at least something odd),  
another example:

import std.stdio;

void main()
{
	int[] test;
	int[] part;
	
	test.length = 10;
	test[] = 1;
	
	part = test[2..5];
	
	//part[-1] = 0;     //array bounds error
	part[-1 .. 7] = 2;  //no array bounds error;
	
	foreach(int i; test)
		writefln(i);
}

Output:
1
2
2
2
2
2
2
2
2
1

It appears not to check bounds for a slice with upper/lower bounds if the  
lower bound is negative. Is this intentional?

Regan
Apr 05 2005
parent derick_eddington yamoooo.com writes:
Slice-setting with a positive lower bound and out-of-bounds upper bound also
doesn't check, so I don't think it's having a negative lower bound that's the
reason.  I think it's a problem with slice-setting.

Also, if you "double slice-set" the out-of-bounds slice it does throw.
ie:  
int[] ia = new int[4];
ia[2 .. 7][] = 2;  // detects
which I think is because it's evaluating the accessing-slice expression first
which checks.



In article <opsosct1cv23k2f5 nrage.netwin.co.nz>, Regan Heath says...

On Wed, 6 Apr 2005 00:11:23 +0000 (UTC), <derick_eddington yahsoo.com>  
wrote:

 What I consider the bug is the line:

 uba[-2 .. length+5] = 7;      // not bounds checked!

 which is allowing an overflowed slice-setting of a previously declared  
 ubyte[].


Indeed. I believe you have found a bug (or at least something odd),  
another example:

import std.stdio;

void main()
{
	int[] test;
	int[] part;
	
	test.length = 10;
	test[] = 1;
	
	part = test[2..5];
	
	//part[-1] = 0;     //array bounds error
	part[-1 .. 7] = 2;  //no array bounds error;
	
	foreach(int i; test)
		writefln(i);
}

Output:
1
2
2
2
2
2
2
2
2
1

It appears not to check bounds for a slice with upper/lower bounds if the  
lower bound is negative. Is this intentional?

Regan
Apr 05 2005