www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Slicing static arrays should be system

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
I just thought that I should bring greater attention to

http://d.puremagic.com/issues/show_bug.cgi?id=8838

As it stands, I think that the slicing static arrays being considered  safe is 
a major hole in SafeD's safety, and I think that it's one that many of us 
aren't aware of. But there seems to be some resistance to outright making the 
slicing of static arrays  system within that bug report, and I recall similar 
resistance in the thread that prompted that bug report (unfortunately, I don't 
remember what thread it was - somewhere in D.Learn I think).

So, I was wondering what the general consensus on this was. Should slicing 
static arrays be considered  system? I honestly don't see how we could do 
otherwise without the compiler being way, way smarter at detecting escaping 
references than is ever going to happen.

Also, I really think that if we're agreed that this change needs to be made 
that it should then be made sooner rather than later, because it's going to 
break code which actually tries to use  safe.

- Jonathan M Davis
Nov 03 2012
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Jonathan M Davis:


 I honestly don't see how we could do otherwise
 without the compiler being way, way smarter at detecting 
 escaping references than is ever going
 to happen.


One question is how much work does it take to precisely keep 
track of such memory zones inside the static type system of D? :-)

Bye,
bearophile
Nov 03 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 04, 2012 06:37:57 bearophile wrote:

 One question is how much work does it take to precisely keep
 track of such memory zones inside the static type system of D? :-)


I don't think that the type system has any concept of that whatsoever.

- Jonathan M Davis
Nov 03 2012
prev sibling next sibling parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Sunday, 4 November 2012 at 05:31:41 UTC, Jonathan M Davis 
wrote:

 I just thought that I should bring greater attention to

 http://d.puremagic.com/issues/show_bug.cgi?id=8838

 As it stands, I think that the slicing static arrays being 
 considered  safe is
 a major hole in SafeD's safety, and I think that it's one that 
 many of us
 aren't aware of. But there seems to be some resistance to 
 outright making the
 slicing of static arrays  system within that bug report, and I 
 recall similar
 resistance in the thread that prompted that bug report 
 (unfortunately, I don't
 remember what thread it was - somewhere in D.Learn I think).

 So, I was wondering what the general consensus on this was. 
 Should slicing
 static arrays be considered  system? I honestly don't see how 
 we could do
 otherwise without the compiler being way, way smarter at 
 detecting escaping
 references than is ever going to happen.

 Also, I really think that if we're agreed that this change 
 needs to be made
 that it should then be made sooner rather than later, because 
 it's going to
 break code which actually tries to use  safe.

 - Jonathan M Davis


Just as a note, if slicing static arrays cannot be  safe, then 
type-safe variadic functions can't be  safe either, as it 
essentially does the same thing, just implicitly:

     void foo(int[] a...)
     {
         // 'a' is a slice of stack memory when foo *isn't* called 
with an explicit array.
     }

The compiler does perform its simple escape analysis on such 
parameters, but as we know, it's not very smart.

I guess another solution to this particular case is to make it 
generate a dynamic array when foo is  safe, but I don't think 
anyone is excited about this option...

Then there's the scope parameter storage class. The compiler 
isn't very smart about scope parameters either.

So what do we do about all these related issues?
Nov 03 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 04, 2012 06:48:15 Jakob Ovrum wrote:

 So what do we do about all these related issues?


I think that anything that the compiler can't absolutely gurantee is  safe 
must be  system. If that's annoying in some places, then that's life, because 
we can't compromise on SafeD just because a few things that we use a lot can't 
be  safe. Now, if we can make further improvements that make it so that the 
compiler _can_ determine that something is actually  safe when before it 
couldn't, then that's great. So, in at least some cases, compiler improvements 
could be used to reduce the annoyance factor, but I don't think that we can 
compromise on this.

- Jonathan M Davis
Nov 03 2012
parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Sunday, 4 November 2012 at 05:58:20 UTC, Jonathan M Davis 
wrote:

 I think that anything that the compiler can't absolutely 
 gurantee is  safe
 must be  system. If that's annoying in some places, then that's 
 life, because
 we can't compromise on SafeD just because a few things that we 
 use a lot can't
 be  safe. Now, if we can make further improvements that make it 
 so that the
 compiler _can_ determine that something is actually  safe when 
 before it
 couldn't, then that's great. So, in at least some cases, 
 compiler improvements
 could be used to reduce the annoyance factor, but I don't think 
 that we can
 compromise on this.

 - Jonathan M Davis


I completely agree, but I don't think we should decide what 
should be guaranteed and what cannot by the current state of the 
compiler. We need a properly specified goal that programmers can 
rely on, and implementers work towards.
Nov 03 2012
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/4/12 1:31 AM, Jonathan M Davis wrote:

 I just thought that I should bring greater attention to

 http://d.puremagic.com/issues/show_bug.cgi?id=8838

 As it stands, I think that the slicing static arrays being considered  safe is
 a major hole in SafeD's safety, and I think that it's one that many of us
 aren't aware of. But there seems to be some resistance to outright making the
 slicing of static arrays  system within that bug report, and I recall similar
 resistance in the thread that prompted that bug report (unfortunately, I don't
 remember what thread it was - somewhere in D.Learn I think).

 So, I was wondering what the general consensus on this was. Should slicing
 static arrays be considered  system? I honestly don't see how we could do
 otherwise without the compiler being way, way smarter at detecting escaping
 references than is ever going to happen.

 Also, I really think that if we're agreed that this change needs to be made
 that it should then be made sooner rather than later, because it's going to
 break code which actually tries to use  safe.


Slicing of static arrays is unsafe only if they're stack-allocated and 
the slice is subsequently escaped.

Andrei
Nov 04 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 04, 2012 08:59:10 Andrei Alexandrescu wrote:

 Slicing of static arrays is unsafe only if they're stack-allocated and
 the slice is subsequently escaped.


If we want to make it so that slicing a static array is  safe when the 
compiler can determine for sure that the slice isn't escaping, then fine, but 
in general, it can't do that. And taking the address of a local variable is 
already considered  system regardless of what you do with it. Taking the slice 
of a static array is really no different from taking the address of a local 
variable. It's just that it has a length along with the address. So, it would 
be completely in line with how taking the address of a local variable is 
treated to make taking the slice of a static array which is a local variable 
 system.

Regardless, the point is that there is a major hole at this point with how 
slicing static arrays is always considered  safe. Nothing which the compiler 
can't guarantee is  safe should be considered  safe. So, even if we treat 
slicing static arrays as  safe in some cases, they need to be cases where the 
compiler is able to prove that it's safe.

- Jonathan M Davis
Nov 04 2012
prev sibling parent reply "Daniel Murphy" <yebblies nospamgmail.com> writes:
"Jonathan M Davis" <jmdavisProg gmx.com> wrote in message 
news:mailman.1518.1352007101.5162.digitalmars-d puremagic.com...

I just thought that I should bring greater attention to

 http://d.puremagic.com/issues/show_bug.cgi?id=8838

 As it stands, I think that the slicing static arrays being considered 
  safe is
 a major hole in SafeD's safety, and I think that it's one that many of us
 aren't aware of. But there seems to be some resistance to outright making 
 the
 slicing of static arrays  system within that bug report, and I recall 
 similar
 resistance in the thread that prompted that bug report (unfortunately, I 
 don't
 remember what thread it was - somewhere in D.Learn I think).


Slicing static arrays on the stack is equivalent to taking the address of a 
local variable, which is already illegal in SafeD.
Nov 06 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, November 07, 2012 12:44:26 Daniel Murphy wrote:

 Slicing static arrays on the stack is equivalent to taking the address of a
 local variable, which is already illegal in SafeD.


Which is what I'm arguing, but a number of people seem to really not like the 
idea of making slicing static arrays on the stack  system even though it _is_ 
the same thing as taking the address of a local variable, which is  system.

- Jonathan M Davis
Nov 06 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/7/2012 5:54 AM, Jonathan M Davis пишет:

 On Wednesday, November 07, 2012 12:44:26 Daniel Murphy wrote:

 Slicing static arrays on the stack is equivalent to taking the address of a
 local variable, which is already illegal in SafeD.


 Which is what I'm arguing, but a number of people seem to really not like the
 idea of making slicing static arrays on the stack  system even though it _is_
 the same thing as taking the address of a local variable, which is  system.


Same here. No matter how convenient this loophole is it just can't be safe.
I recall SafeD's motto was along the lines of
"memory safety is not negotiable almost everything else is".


-- 
Dmitry Olshansky
Nov 08 2012