www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Discussion Thread: DIP 1035-- system Variables--Community Review Round

reply Mike Parker <aldacron gmail.com> writes:
This is the discussion thread for the second round of Community 
Review of DIP 1035, " system Variables":

https://github.com/dlang/DIPs/blob/c39f6ac62210e0604dcee99b0092c1930839f93a/DIPs/DIP1035.md

The review period will end at 11:59 PM ET on March 11, or when I 
make a post declaring it complete. Discussion in this thread may 
continue beyond that point.

Here in the discussion thread, you are free to discuss anything 
and everything related to the DIP. Express your support or 
opposition, debate alternatives, argue the merits, etc.

However, if you have any specific feedback on how to improve the 
proposal itself, then please post it in the feedback thread. The 
feedback thread will be the source for the review summary that I 
will write at the end of this review round. I will post a link to 
that thread immediately following this post. Just be sure to read 
and understand the Reviewer Guidelines before posting there:

https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md

And my blog post on the difference between the Discussion and 
Feedback threads:

https://dlang.org/blog/2020/01/26/dip-reviews-discussion-vs-feedback/

Please stay on topic here. I will delete posts that are 
completely off-topic.
Feb 25 2021
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:


 However, if you have any specific feedback on how to improve 
 the proposal itself, then please post it in the feedback 
 thread. The feedback thread will be the source for the review 
 summary that I will write at the end of this review round. I 
 will post a link to that thread immediately following this post.


The Feedback Thread is here:

https://forum.dlang.org/post/eivezuohaejnvvlhaeec forum.dlang.org
Feb 25 2021
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On 25.02.21 10:21, Mike Parker wrote:

 This is the discussion thread for the second round of Community Review 
 of DIP 1035, " system Variables":
 
 https://github.com/dlang/DIPs/blob/c39f6ac62210e0604dcee99b0092c1930839
93a/DIPs/DIP1035.md 


Big thanks to Dennis and Paul for championing this! I've only given the 
DIP a cursory read, but it looks great to me.
Feb 25 2021
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
 Example: User-Defined Slice


Since ptr and length are private, they are only accessible to member functions 
of IntSlice. I don't think there is any way to prove memory safety of the 
implementation of this.


  Instead, every function that touches ptr and length, including the  safe 


constructor, must be manually checked.

Right. So the idea is to minimize access to .ptr/.length for member functions
of 
IntSlice. This can be done with:

   struct IntSlice
   {
     private static struct Slice
     {
         private int* ptr;
         private size_t length;

        trusted:

         this(int[] src)
         {
              ptr = src.ptr;
              length = src.length;
         }

         int[] opSlice()
         {
              return ptr[0 .. length];
         }
      }

      // Now, the only access is via  trusted functions

     safe:

     private Slice slice;

     this(int[] src)
     {
         slice = Slice(src);
     }

     ref int opIndex(size_t i)
     {
         return slice[][i]; // note no assert(i < length) needed
     }

     // ... every function goes here ...
   }

So, the general idea is to restrict access to ` system` fields by encapsulating 
them separately, and providing a minimized  trusted interface to them.
Feb 26 2021
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 26.02.21 10:49, Walter Bright wrote:

  > Example: User-Defined Slice
 
 Since ptr and length are private, they are only accessible to member 
 functions of IntSlice. I don't think there is any way to prove memory 
 safety of the implementation of this.
 
  >  Instead, every function that touches ptr and length, including the 
  safe constructor, must be manually checked.
 
 Right. So the idea is to minimize access to .ptr/.length for member 
 functions of IntSlice. This can be done with:
 
    struct IntSlice
    {
      private static struct Slice
      {
          private int* ptr;
          private size_t length;
 
         trusted:
 
          this(int[] src)
          {
               ptr = src.ptr;
               length = src.length;
          }
 
          int[] opSlice()
          {
               return ptr[0 .. length];
          }
       }
 
       // Now, the only access is via  trusted functions
 
      safe:
 
      private Slice slice;
 
      this(int[] src)
      {
          slice = Slice(src);
      }
 
      ref int opIndex(size_t i)
      {
          return slice[][i]; // note no assert(i < length) needed
      }
 
      // ... every function goes here ...
    }
 
 So, the general idea is to restrict access to ` system` fields by 
 encapsulating them separately, and providing a minimized  trusted 
 interface to them.



Yes, that's the idea, but this is a convention. It's not possible now to 
make the compiler check your convention.  system fields address this 
limitation.
Feb 26 2021
prev sibling next sibling parent Paul Backus <snarwin gmail.com> writes:
On Friday, 26 February 2021 at 09:49:40 UTC, Walter Bright wrote:

     private static struct Slice
     {
         private int* ptr;
         private size_t length;

        trusted:

         this(int[] src)
         {
              ptr = src.ptr;
              length = src.length;
         }

         int[] opSlice()
         {
              return ptr[0 .. length];
         }
      }


This is equivalent to the example in the DIP, except that you 
have changed the constructor from  safe to  trusted by replacing 
`&src[0]` with `src.ptr`. If you had written it in the same was 
as the original, you would still have to manually check  safe 
code in order to prove that this interface is memory-safe.
Feb 26 2021
prev sibling parent reply Nick Treleaven <nick geany.org> writes:
On Friday, 26 February 2021 at 09:49:40 UTC, Walter Bright wrote:

 Example: User-Defined Slice


 Since ptr and length are private, they are only accessible to 
 member functions of IntSlice.


Er, doesn't private mean module access? This DIP would help 
verify long modules that might inadvertently access unsafe fields 
from outside the struct definition, at least in  safe code.
Feb 26 2021
parent reply Max Haughton <maxhaton gmail.com> writes:
On Friday, 26 February 2021 at 18:29:26 UTC, Nick Treleaven wrote:

 On Friday, 26 February 2021 at 09:49:40 UTC, Walter Bright 
 wrote:

 Example: User-Defined Slice


 Since ptr and length are private, they are only accessible to 
 member functions of IntSlice.


 Er, doesn't private mean module access? This DIP would help 
 verify long modules that might inadvertently access unsafe 
 fields from outside the struct definition, at least in  safe 
 code.


https://run.dlang.io/is/KalIzj

Yes - however code subject to is supposed to be consumed in a 
different module for the most part, so this could be a nice side 
effect but it shouldn't (de jure) guide the actual DIP itself.
Feb 26 2021
parent Nick Treleaven <nick geany.org> writes:
On Friday, 26 February 2021 at 22:29:09 UTC, Max Haughton wrote:

 On Friday, 26 February 2021 at 18:29:26 UTC, Nick Treleaven 
 wrote:

 Er, doesn't private mean module access? This DIP would help 
 verify long modules that might inadvertently access unsafe 
 fields from outside the struct definition, at least in  safe 
 code.


 https://run.dlang.io/is/KalIzj

 Yes - however code subject to is supposed to be consumed in a 
 different module for the most part, so this could be a nice 
 side effect but it shouldn't (de jure) guide the actual DIP 
 itself.


It's more than a nice side effect - it's actually necessary to 
enforce a  safe interface for structs that do unsafe things. 
Otherwise changing  safe code can cause memory-safety violations 
- that's contrary to the design goal of  safe. It also would in 
practice mean a programmer/reviewer would have to read every line 
of code in a module every time there was a change - that's not a 
sensible design.
Feb 27 2021
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
 Example: Short String


This example boils down to an invariant not being maintained if the struct is 
void-initialized. A couple simple solutions come to mind:

1. disallow void initialization in  safe code if the struct has a constructor

2. disallow void initialization if the struct has an invariant

I suggest that the ShortString struct is not properly written, because it has a 
hidden invariant that was not made clear with an explicit:

     invariant { assert(length <= data.length); }
Feb 26 2021
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2021 1:57 AM, Walter Bright wrote:

 2. disallow void initialization if the struct has an invariant


Thinking about this some more.

An invariant specifies a subset of values that the fields could have, while a 
void initialization means any random values are acceptable.

Therefore (2) makes sense.
Feb 26 2021
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 26.02.21 11:08, Walter Bright wrote:

 On 2/26/2021 1:57 AM, Walter Bright wrote:

 2. disallow void initialization if the struct has an invariant


 
 Thinking about this some more.
 
 An invariant specifies a subset of values that the fields could have, 
 while a void initialization means any random values are acceptable.
 
 Therefore (2) makes sense.



An invariant is anything that remains true throughout the execution, and 
it's not always easy to check. (2) makes sense, but it's not a 
substitute for  system fields.  safe code shouldn't be able to mess with 
unsafe values at all, not only through `void` initialization, and 
 system code should be able to define its own unsafe values.
Feb 26 2021
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 26 February 2021 at 09:57:03 UTC, Walter Bright wrote:

 Example: Short String


 This example boils down to an invariant not being maintained if 
 the struct is void-initialized.


Note that it's not just void initialization. Overlap in a union 
and reinterpreting casts can also create instances with broken 
invariants; e.g.,

union U
{
     int n;
     ShortString s;
}

 safe
void main()
{
     U u = { n: 0xDEADBEEF };
     writeln(u.s[]); // undefined behavior
}
Feb 26 2021
parent reply Walter Bright <newshound2 digitalmars.com> writes:
https://issues.dlang.org/show_bug.cgi?id=21665
Feb 26 2021
parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Saturday, 27 February 2021 at 05:59:23 UTC, Walter Bright 
wrote:

 https://issues.dlang.org/show_bug.cgi?id=21665


Would that be a compiler error or warning ignoring initialization 
if you tried to void init a struct having invariant?
Feb 27 2021
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/27/2021 12:38 AM, Imperatorn wrote:

 On Saturday, 27 February 2021 at 05:59:23 UTC, Walter Bright wrote:

 https://issues.dlang.org/show_bug.cgi?id=21665


 
 Would that be a compiler error or warning ignoring initialization if you tried 
 to void init a struct having invariant?


error
Feb 27 2021
prev sibling next sibling parent reply Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:

 This is the discussion thread for the second round of Community 
 Review of DIP 1035, " system Variables":

 [...]


Well done. Everything in the direction of safety if very 
important to our company. Great work.
Feb 26 2021
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 26.02.21 13:34, Imperatorn wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:

 This is the discussion thread for the second round of Community Review 
 of DIP 1035, " system Variables":

 [...]


 
 Well done. Everything in the direction of safety if very important to 
 our company. Great work.


Feel free to contribute to what is important to you instead of 
unproductively trolling discussion threads you do not care about. Thanks.

This is my last post in this subthread.
Feb 26 2021
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 26 February 2021 at 14:04:45 UTC, Timon Gehr wrote:

 On 26.02.21 13:34, Imperatorn wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker 
 wrote:

 This is the discussion thread for the second round of 
 Community Review of DIP 1035, " system Variables":

 [...]


 
 Well done. Everything in the direction of safety if very 
 important to our company. Great work.


 Feel free to contribute to what is important to you instead of 
 unproductively trolling discussion threads you do not care 
 about. Thanks.

 This is my last post in this subthread.


I don't think you understand the definition of trolling 
unfortunately.

Your behavior is just objectively wierd.
You have no idea what I care about.

If there are something happening in you life that make you 
socially underperformed or unable to have good manners in 
general, direct your last watt of energy towards fixing that 
instead instead of writing in the forums. Thanks.
Feb 26 2021
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 26.02.21 13:34, Imperatorn wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:

 This is the discussion thread for the second round of Community Review 
 of DIP 1035, " system Variables":

 [...]


 
 Well done. Everything in the direction of safety if very important to 
 our company. Great work.


On 26.02.21 14:04, Imperatorn wrote:

 
 I don't think you understand the definition of trolling unfortunately.


For some reason I read your original comment as sarcastic. I think I had 
confused you with another member of the forums who is also posting under 
a pseudonym. Sorry about that. Mea culpa.

On 26.02.21 14:04, Imperatorn wrote:

 If there are something happening in you life that make you socially 


underperformed or unable to have good manners in general, [...]

I am sorry for having provoked you into violating basic standards of 
decency.
Feb 26 2021
parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Friday, 26 February 2021 at 15:14:51 UTC, Timon Gehr wrote:

 On 26.02.21 13:34, Imperatorn wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker 
 wrote:

 This is the discussion thread for the second round of 
 Community Review of DIP 1035, " system Variables":

 [...]


 
 Well done. Everything in the direction of safety if very 
 important to our company. Great work.





 For some reason I read your original comment as sarcastic. I 
 think I had confused you with another member of the forums who 
 is also posting under a pseudonym. Sorry about that. Mea culpa.


I see. No worries. I actually just wanted to say that it was good 
work and that we care about that.

Misunderstandings happens sometimes. Peace
Feb 26 2021
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
A simple workaround is an unsafe wrapper:

struct Unsafe
{
	private T a;
	T get()  system { return a; }
}

struct IntSlice
{
	private Unsafe!(int*) ptr;
	private Unsafe!size_t length;
...
Feb 27 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 February 2021 at 08:23:38 UTC, Kagamin wrote:

 A simple workaround is an unsafe wrapper:

 struct Unsafe
 {
 	private T a;
 	T get()  system { return a; }
 }

 struct IntSlice
 {
 	private Unsafe!(int*) ptr;
 	private Unsafe!size_t length;
 ...


The compiler will still allow you to 
void-initialize/overlap/reinterpret-cast an Unsafe!size_t in 
 safe code, because it considers size_t to be a safe type.

You *can* get the compiler to treat any value as unsafe by 
overlapping it with a pointer; e.g.,

struct Unsafe(T)
{
     union Storage { T value; void* dummy; }
     Storage storage;
     ref T get() { return Storage.value; }
     /* ... */
}

But this introduces memory overhead for any T smaller than a 
pointer, which is not ideal.
Feb 27 2021
next sibling parent reply Kagamin <spam here.lot> writes:
On Saturday, 27 February 2021 at 13:32:24 UTC, Paul Backus wrote:

 The compiler will still allow you to 
 void-initialize/overlap/reinterpret-cast an Unsafe!size_t in 
  safe code, because it considers size_t to be a safe type.


The wrapped value still won't be accessible to safe code.
Feb 27 2021
parent Dennis <dkorpel gmail.com> writes:
On Saturday, 27 February 2021 at 15:19:14 UTC, Kagamin wrote:

 On Saturday, 27 February 2021 at 13:32:24 UTC, Paul Backus 
 wrote:

 The compiler will still allow you to 
 void-initialize/overlap/reinterpret-cast an Unsafe!size_t in 
  safe code, because it considers size_t to be a safe type.


 The wrapped value still won't be accessible to safe code.


You can access the wrapped value through a union in  safe code, 
and your  trusted code can't assume anything about your Unsafe!T, 
so it does not accomplish much.
Mar 02 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/27/21 8:32 AM, Paul Backus wrote:

 On Saturday, 27 February 2021 at 08:23:38 UTC, Kagamin wrote:

 A simple workaround is an unsafe wrapper:

 struct Unsafe
 {
     private T a;
     T get()  system { return a; }
 }

 struct IntSlice
 {
     private Unsafe!(int*) ptr;
     private Unsafe!size_t length;
 ...


 
 The compiler will still allow you to 
 void-initialize/overlap/reinterpret-cast an Unsafe!size_t in  safe code, 
 because it considers size_t to be a safe type.
 
 You *can* get the compiler to treat any value as unsafe by overlapping 
 it with a pointer; e.g.,
 
 struct Unsafe(T)
 {
      union Storage { T value; void* dummy; }
      Storage storage;
      ref T get() { return Storage.value; }
      /* ... */
 }
 
 But this introduces memory overhead for any T smaller than a pointer, 
 which is not ideal.


Better mark that get as  system, or it can be used in  safe code.

-Steve
Feb 27 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 February 2021 at 15:21:41 UTC, Steven 
Schveighoffer wrote:

 On 2/27/21 8:32 AM, Paul Backus wrote:

 You *can* get the compiler to treat any value as unsafe by 
 overlapping it with a pointer; e.g.,
 
 struct Unsafe(T)
 {
      union Storage { T value; void* dummy; }
      Storage storage;
      ref T get() { return Storage.value; }
      /* ... */
 }
 
 But this introduces memory overhead for any T smaller than a 
 pointer, which is not ideal.


 Better mark that get as  system, or it can be used in  safe 
 code.


It's in a template, so it will be inferred as  system.
Feb 27 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/27/21 10:55 AM, Paul Backus wrote:

 On Saturday, 27 February 2021 at 15:21:41 UTC, Steven Schveighoffer wrote:

 On 2/27/21 8:32 AM, Paul Backus wrote:

 You *can* get the compiler to treat any value as unsafe by 
 overlapping it with a pointer; e.g.,

 struct Unsafe(T)
 {
      union Storage { T value; void* dummy; }
      Storage storage;
      ref T get() { return Storage.value; }
      /* ... */
 }

 But this introduces memory overhead for any T smaller than a pointer, 
 which is not ideal.


 Better mark that get as  system, or it can be used in  safe code.


 
 It's in a template, so it will be inferred as  system.


D considers that  safe. It was a whole section of my Dconf online talk. 
I filed a bug report on it (in which you seem to think this is OK): 
https://issues.dlang.org/show_bug.cgi?id=21565

Even without templates:

struct DThinksThisIsSafe
{
    union Storage { int value; void *dummy; }
    Storage storage;
     safe ref int get() return { return storage.value; } // compiles 
just fine
}

-Steve
Feb 27 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 February 2021 at 19:05:01 UTC, Steven 
Schveighoffer wrote:

 D considers that  safe. It was a whole section of my Dconf 
 online talk. I filed a bug report on it (in which you seem to 
 think this is OK): 
 https://issues.dlang.org/show_bug.cgi?id=21565

 Even without templates:

 struct DThinksThisIsSafe
 {
    union Storage { int value; void *dummy; }
    Storage storage;
     safe ref int get() return { return storage.value; } // 
 compiles just fine
 }


Ah, right, because you're only accessing the int, not the 
pointer. Good catch.
Feb 27 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 February 2021 at 20:07:30 UTC, Paul Backus wrote:

 Ah, right, because you're only accessing the int, not the 
 pointer. Good catch.


...which means the whole approach doesn't actually work to begin 
with. D is perfectly within its rights to let you void-initialize 
the union (even though it currently doesn't), because  safe code 
can't access the pointer anyway, so it can never lead to 
undefined behavior.
Feb 27 2021
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/27/21 3:12 PM, Paul Backus wrote:

 On Saturday, 27 February 2021 at 20:07:30 UTC, Paul Backus wrote:

 Ah, right, because you're only accessing the int, not the pointer. 
 Good catch.




It's telling that you intuitively thought the system should prevent you 
from doing this (as you should!)


 
 ...which means the whole approach doesn't actually work to begin with. D 
 is perfectly within its rights to let you void-initialize the union 
 (even though it currently doesn't), because  safe code can't access the 
 pointer anyway, so it can never lead to undefined behavior.


D is perfectly within its rights to do whatever it wants for  safe code. 
It could let you write an array length without extending the array, and 
then only allow you accessing the single element pointed at. It could 
prevent dereferencing pointers, and still be considered memory-safe. But 
there is still the question of whether this is useful to programmers or not.

-Steve
Feb 27 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 27 February 2021 at 20:54:34 UTC, Steven 
Schveighoffer wrote:

 D is perfectly within its rights to do whatever it wants for 
  safe code. It could let you write an array length without 
 extending the array, and then only allow you accessing the 
 single element pointed at. It could prevent dereferencing 
 pointers, and still be considered memory-safe. But there is 
 still the question of whether this is useful to programmers or 
 not.


The general trend in the design of  safe, as exemplified by DIP 
25 and DIP 1000, has been to lift restrictions when the compiler 
can prove they are not necessary to prevent UB. And I would argue 
that this is exactly what one wants in a proof system: the 
strongest result (memory safety) from the fewest axioms 
(restrictions on  safe code).

Another way to look at it is that automatically extending an 
array when its length is changed allows the compiler to prove 
memory safety for programs that would otherwise require  trusted, 
so it is a useful restriction. On the other hand, forbidding 
access to safe members of unions does not make any 
previously- trusted programs  safe, so it is a useless 
restriction.

I understand from previous discussions that you have some 
less-rigorous ideas about what is "useful to programmers" and 
what is not, but I think this is an occasion where rigor is 
warranted.
Feb 27 2021
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/27/21 4:13 PM, Paul Backus wrote:

 I understand from previous discussions that you have some less-rigorous 
 ideas about what is "useful to programmers" and what is not, but I think 
 this is an occasion where rigor is warranted.


This is going quite off topic, but I wanted to say this is absolutely 
not the driver for my point of view. It is not a matter of rigor but a 
matter of what are the expectations of what  safe should imply.

My point of view is that if the semantic meaning of a a  safe union 
between an int and a pointer currently is that the pointer isn't usable 
even in  trusted code (even though the compiler doesn't prevent you from 
doing it), then this is going to violate the expectations of the 
programmer (why would he write a union, when you can only use one of the 
members).

We should not only focus in our concept of  safe in proving the memory 
safety of the rules we come up with, but in coming up with rules that 
make sense in the context of utility to the programmer. We can make up 
whatever rules we want, and based on those rules, we can prove safety, 
but if the result is "you can do this, but it NEVER is usable", I think 
we can do better.

-Steve
Feb 27 2021
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:

 This is the discussion thread for the second round of Community 
 Review of DIP 1035, " system Variables":

 [...]


"pointers, arrays, and other reference types are unsafe"

This confused me. Pointers aren't unsafe unless you got one from 
 system code. Could you clarify please?
Mar 01 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Monday, 1 March 2021 at 20:55:40 UTC, Atila Neves wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker 
 wrote:

 This is the discussion thread for the second round of 
 Community Review of DIP 1035, " system Variables":

 [...]


 "pointers, arrays, and other reference types are unsafe"

 This confused me. Pointers aren't unsafe unless you got one 
 from  system code. Could you clarify please?


The language here is sloppy, but the intent is to refer to 
pointer *types*, not pointer *values*. It should read, "pointer 
types, dynamic array types, and other reference types are unsafe."
Mar 01 2021
next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 2 March 2021 at 02:46:58 UTC, Paul Backus on the 
feedback thread wrote:

 On Monday, 1 March 2021 at 23:38:28 UTC, Dukc wrote:

 Which leads to, what are the values compiler considers to be 
 safe?


 The answer to this question is is already part of the language 
 spec:

 https://dlang.org/spec/function.html#safe-values


Yes, you can use that. But the dip needs to update that part of 
the spec, since what is going to be a safe struct instance / 
union instance / class ref value is changing. For instance, what 
the compiler will consider safe values for the `ShortString` 
struct provided in the example?
Mar 02 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 2 March 2021 at 11:48:09 UTC, Dukc wrote:

 On Tuesday, 2 March 2021 at 02:46:58 UTC, Paul Backus on the 
 feedback thread wrote:

 On Monday, 1 March 2021 at 23:38:28 UTC, Dukc wrote:

 Which leads to, what are the values compiler considers to be 
 safe?


 The answer to this question is is already part of the language 
 spec:

 https://dlang.org/spec/function.html#safe-values


 Yes, you can use that. But the dip needs to update that part of 
 the spec, since what is going to be a safe struct instance / 
 union instance / class ref value is changing. For instance, 
 what the compiler will consider safe values for the 
 `ShortString` struct provided in the example?


Good point. For user-defined unsafe types like ShortString, the 
compiler will have to assume that any value that can be obtained 
in  safe code is a safe value.
Mar 02 2021
parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 2 March 2021 at 14:09:01 UTC, Paul Backus wrote:

 Good point. For user-defined unsafe types like ShortString, the 
 compiler will have to assume that any value that can be 
 obtained in  safe code is a safe value.


But what about determining ` safe`ty of value given by ` system` 
initializer? Is it just not attempted for user-defined types? Or 
is the rule "`.init` value is ` safe`, everything else is 
` system`"? Or something more complicated?
Mar 02 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 2 March 2021 at 14:28:43 UTC, Dukc wrote:

 On Tuesday, 2 March 2021 at 14:09:01 UTC, Paul Backus wrote:

 Good point. For user-defined unsafe types like ShortString, 
 the compiler will have to assume that any value that can be 
 obtained in  safe code is a safe value.


 But what about determining ` safe`ty of value given by 
 ` system` initializer? Is it just not attempted for 
 user-defined types? Or is the rule "`.init` value is ` safe`, 
 everything else is ` system`"? Or something more complicated?


I think you may be confusing the safety of a *value* with the 
safety of a *variable*.

Consider the following example:

      safe ShortString makeShortString(string source) { /* ... */ }

     auto s = makeShortString("hello");

The value of `makeShortString("hello")` is *assumed* to be a safe 
value, because `makeShortString` is a  safe function. Therefore, 
the variable `s` is *determined* to be a  safe variable, because 
its initializer is a safe value.

Regarding `.init`: because default initialization of any type is 
allowed in  safe code, the `.init` value of any user-defined type 
must be a safe value. This is true today, and it will still be 
true if DIP 1035 is accepted. The fact that the compiler 
currently does not enforce this is a bug. [1]

[1] https://issues.dlang.org/show_bug.cgi?id=21675
Mar 02 2021
parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 2 March 2021 at 14:57:04 UTC, Paul Backus wrote:

 I think you may be confusing the safety of a *value* with the 
 safety of a *variable*.


No, I was thinking this piece:


 An exception to the above rules is made on unsafe types when
 the compiler knows the resulting value is safe.
 ```D
 int* getNull() pure  system {return null;}
 int* n = getNull(); // despite unsafe type with  system
 initialization expression, inferred as  safe
 ```


So the queastion is:

```D
ShortString getSString() pure  system {<...>}
//under what conditions, if any, is the initialization inferred 
as  safe here?
ShortString n = getSString();
```
Mar 02 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 2 March 2021 at 15:17:54 UTC, Dukc wrote:

 So the queastion is:

 ```D
 ShortString getSString() pure  system {<...>}
 //under what conditions, if any, is the initialization inferred 
 as  safe here?
 ShortString n = getSString();
 ```


In this example, `n` will always be inferred as a  system 
variable.
Mar 02 2021
parent Dukc <ajieskola gmail.com> writes:
On Tuesday, 2 March 2021 at 16:04:14 UTC, Paul Backus wrote:

 On Tuesday, 2 March 2021 at 15:17:54 UTC, Dukc wrote:

 So the queastion is:

 ```D
 ShortString getSString() pure  system {<...>}
 //under what conditions, if any, is the initialization 
 inferred as  safe here?
 ShortString n = getSString();
 ```


 In this example, `n` will always be inferred as a  system 
 variable.


Fair enough. Remember to mention that in the DIP - either at the 
place I just quoted or at the changes you propose to what is 
considered a safe value.
Mar 02 2021
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 1 March 2021 at 21:26:18 UTC, Paul Backus wrote:

 On Monday, 1 March 2021 at 20:55:40 UTC, Atila Neves wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker 
 wrote:

 This is the discussion thread for the second round of 
 Community Review of DIP 1035, " system Variables":

 [...]


 "pointers, arrays, and other reference types are unsafe"

 This confused me. Pointers aren't unsafe unless you got one 
 from  system code. Could you clarify please?


 The language here is sloppy, but the intent is to refer to 
 pointer *types*, not pointer *values*. It should read, "pointer 
 types, dynamic array types, and other reference types are 
 unsafe."


Pointer types *can* be unsafe, if the values came from  system 
code. Otherwise they're perfectly safe. Slices (dynamic arrays) 
are slightly different because of the necessity of bounds checks. 
But deferencing a pointer is fine in  safe code - the 
possibilities are:

* it came from the GC.
* is the address of a module-level variable.
* is a scoped address on the stack.
* is null.

Am I missing a case?
Mar 02 2021
next sibling parent ag0aep6g <anonymous example.com> writes:
On 02.03.21 21:46, Atila Neves wrote:

 Pointer types *can* be unsafe, if the values came from  system code. 


Pointer *values* can be unsafe, if they come from  system code. A type 
that can have unsafe values is an unsafe type (so defined in the DIP). 
So pointer types are unsafe types.
Mar 02 2021
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Tuesday, 2 March 2021 at 20:46:17 UTC, Atila Neves wrote:

 Pointer types *can* be unsafe, if the values came from  system 
 code. Otherwise they're perfectly safe. Slices (dynamic arrays) 
 are slightly different because of the necessity of bounds 
 checks. But deferencing a pointer is fine in  safe code - the 
 possibilities are:

 * it came from the GC.
 * is the address of a module-level variable.
 * is a scoped address on the stack.
 * is null.

 Am I missing a case?


You are conflating types and values. Pointer *values* can be 
either safe or unsafe, depending on what they point to. Pointer 
*types* are always unsafe, because they include both safe and 
unsafe values.
Mar 02 2021
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 2 March 2021 at 21:41:40 UTC, Paul Backus wrote:

 You are conflating types and values. Pointer *values* can be 
 either safe or unsafe, depending on what they point to. Pointer 
 *types* are always unsafe, because they include both safe and 
 unsafe values.


A type-system can be made to distinguish between safe and unsafe 
pointers.

A type is just a tag stuck onto a value.
Mar 02 2021
parent Paulo Pinto <pjmlp progtools.org> writes:
On Tuesday, 2 March 2021 at 22:06:30 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 2 March 2021 at 21:41:40 UTC, Paul Backus wrote:

 You are conflating types and values. Pointer *values* can be 
 either safe or unsafe, depending on what they point to. 
 Pointer *types* are always unsafe, because they include both 
 safe and unsafe values.


 A type-system can be made to distinguish between safe and 
 unsafe pointers.

 A type is just a tag stuck onto a value.



this pointer distinction.

Pointers are always safe, unless explicitly marked otherwise or 
used in unsafe contexts, converting between both types is also 
explicit.
Mar 02 2021
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 2 March 2021 at 21:41:40 UTC, Paul Backus wrote:

 On Tuesday, 2 March 2021 at 20:46:17 UTC, Atila Neves wrote:

 Pointer types *can* be unsafe, if the values came from  system 
 code. Otherwise they're perfectly safe. Slices (dynamic 
 arrays) are slightly different because of the necessity of 
 bounds checks. But deferencing a pointer is fine in  safe code 
 - the possibilities are:

 * it came from the GC.
 * is the address of a module-level variable.
 * is a scoped address on the stack.
 * is null.

 Am I missing a case?


 You are conflating types and values. Pointer *values* can be 
 either safe or unsafe, depending on what they point to. Pointer 
 *types* are always unsafe, because they include both safe and 
 unsafe values.


I don't think I am, but I think I understand where you're coming 
from. Let me restate my point and maybe then it will be clearer: 
if all the code in a program is  safe, then pointers are memory 
safe (with DIP1000).

I guess I'd argue that pointer types are safe unless the value 
was obtained from  system code. But throw  system code into the 
mix...

Anyway, the wording confused me.
Mar 04 2021
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/4/21 1:23 PM, Atila Neves wrote:

 On Tuesday, 2 March 2021 at 21:41:40 UTC, Paul Backus wrote:

 On Tuesday, 2 March 2021 at 20:46:17 UTC, Atila Neves wrote:

 Pointer types *can* be unsafe, if the values came from  system code. 
 Otherwise they're perfectly safe. Slices (dynamic arrays) are 
 slightly different because of the necessity of bounds checks. But 
 deferencing a pointer is fine in  safe code - the possibilities are:

 * it came from the GC.
 * is the address of a module-level variable.
 * is a scoped address on the stack.
 * is null.

 Am I missing a case?


 You are conflating types and values. Pointer *values* can be either 
 safe or unsafe, depending on what they point to. Pointer *types* are 
 always unsafe, because they include both safe and unsafe values.


 
 I don't think I am, but I think I understand where you're coming from. 
 Let me restate my point and maybe then it will be clearer: if all the 
 code in a program is  safe, then pointers are memory safe (with DIP1000).
 
 I guess I'd argue that pointer types are safe unless the value was 
 obtained from  system code. But throw  system code into the mix...
 
 Anyway, the wording confused me.
 


At the very top of the DIP:

"D's memory safety system distinguishes between safe values, which can 
be used freely in  safe code without causing undefined behavior, and 
unsafe values, which cannot. A type that has only safe values is a safe 
type; one that has both safe and unsafe values is an unsafe type."

Unsaid here is that it doesn't matter where it comes from ( safe or 
 system). This is how they are defining "safe types" and "unsafe types". 
Everything follows from that.

Given that definition, pointers are unsafe.

-Steve
Mar 04 2021
parent Atila Neves <atila.neves gmail.com> writes:
On Thursday, 4 March 2021 at 18:47:28 UTC, Steven Schveighoffer 
wrote:

 On 3/4/21 1:23 PM, Atila Neves wrote:

 [...]


 At the very top of the DIP:

 "D's memory safety system distinguishes between safe values, 
 which can be used freely in  safe code without causing 
 undefined behavior, and unsafe values, which cannot. A type 
 that has only safe values is a safe type; one that has both 
 safe and unsafe values is an unsafe type."

 Unsaid here is that it doesn't matter where it comes from 
 ( safe or  system). This is how they are defining "safe types" 
 and "unsafe types". Everything follows from that.

 Given that definition, pointers are unsafe.

 -Steve


That makes sense, but I'm not sure that's how we *should* define 
it, given that pointers are memory-safe in  safe code.
Mar 04 2021
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Thursday, 4 March 2021 at 18:23:00 UTC, Atila Neves wrote:

 On Tuesday, 2 March 2021 at 21:41:40 UTC, Paul Backus wrote:

 You are conflating types and values. Pointer *values* can be 
 either safe or unsafe, depending on what they point to. 
 Pointer *types* are always unsafe, because they include both 
 safe and unsafe values.


 I don't think I am, but I think I understand where you're 
 coming from. Let me restate my point and maybe then it will be 
 clearer: if all the code in a program is  safe, then pointers 
 are memory safe (with DIP1000).

 I guess I'd argue that pointer types are safe unless the value 
 was obtained from  system code. But throw  system code into the 
 mix...

 Anyway, the wording confused me.


I agree that the wording is confusing. In particular, the words 
"safe" and
"unsafe" are heavily overloaded. Unfortunately, "safe values" and 
" safe code"
are both official language-spec terms, so there's nothing the DIP 
can do about
those.

It would probably still be helpful to replace "safe type" and 
"unsafe type" with
something more distinct. Maybe "unrestricted type" and 
"restricted type"? E.g.,

     Pointer *values* can be either safe or unsafe, depending on 
what they point
     to. Pointer *types* are always restricted types, because they 
include both
     safe and unsafe values.

A restricted type is any type which has limits placed on what you 
are allowed to do with it in  safe code. Hopefully it is clear 
that a type must be restricted if and only if it includes unsafe 
values.
Mar 04 2021
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 2 March 2021 at 20:46:17 UTC, Atila Neves wrote:

 On Monday, 1 March 2021 at 21:26:18 UTC, Paul Backus wrote:

 On Monday, 1 March 2021 at 20:55:40 UTC, Atila Neves wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker 
 wrote:

 This is the discussion thread for the second round of 
 Community Review of DIP 1035, " system Variables":

 [...]


 "pointers, arrays, and other reference types are unsafe"

 This confused me. Pointers aren't unsafe unless you got one 
 from  system code. Could you clarify please?


 The language here is sloppy, but the intent is to refer to 
 pointer *types*, not pointer *values*. It should read, 
 "pointer types, dynamic array types, and other reference types 
 are unsafe."


 Pointer types *can* be unsafe, if the values came from  system 
 code. Otherwise they're perfectly safe.


The correct wording to this is: pointer *values* can be unsafe if 
they come from ` system` code, therefore pointer *types* are 
unsafe, as defined by this DIP. When the DIP talks about unsafe 
type, it really means only potentially unsafe type.


 Slices (dynamic arrays) are slightly different because of the 
 necessity of bounds checks.


They are not. `(ubyte*).init[0xBAA..0xBEE]` is just as unsafe as 
`cast(int*)0xDEADBEEF`, regardless of whether there are bound 
checks when accessing it. Bounds checked access is safe only if 
the value is safe, just as with pointer dereference.

But deferencing a pointer is fine in  safe code - the

 possibilities are:

 * it came from the GC.
 * is the address of a module-level variable.
 * is a scoped address on the stack.
 * is null.

 Am I missing a case?


These are all true. The intention of the DIP is that you could 
make user-defined types that behave the same way - that ` safe` 
would guarantee they will not have unsafe values.

Think about defining a 16-bit pointer for 32-bit or 64-bit 
architecture. If you now declare it as `struct ptr16{private 
short handle;}` you will not be able to make it work in  safe 
code, as it can be void-initialized with wrong values. You'll 
have to do something like `union ptr16{short handle; void[2] 
unsafeMarker;}` instead.
Mar 03 2021
parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 3 March 2021 at 12:54:53 UTC, Dukc wrote:

 On Tuesday, 2 March 2021 at 20:46:17 UTC, Atila Neves wrote:

 On Monday, 1 March 2021 at 21:26:18 UTC, Paul Backus wrote:

 On Monday, 1 March 2021 at 20:55:40 UTC, Atila Neves wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker 
 wrote:

 This is the discussion thread for the second round of 
 Community Review of DIP 1035, " system Variables":

 [...]











 Slices (dynamic arrays) are slightly different because of the 
 necessity of bounds checks.


 They are not. `(ubyte*).init[0xBAA..0xBEE]` is just as unsafe 
 as `cast(int*)0xDEADBEEF`, regardless of whether there are 
 bound checks when accessing it.


Yes, but not allowed in  safe code. My point is that  safe can 
still do:

auto arr = new int[7];
auto oops = arr[3_000_000];
Mar 04 2021
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 2 March 2021 at 13:56:28 UTC, Dennis in the feedback 
thread wrote:

 There is no disagreement, the rationale text you mention 
 describes the situation before DIP 1035, while the description 
 describes the situation after DIP 1035.


How is this piece from the rationale section describing the 
situation before DIP1035?

"Since the initialization expression cast(int*) 0xDEADBEEF would 
not be allowed in a  safe function, and since the initial value 
of y is unknown, the compiler should annotate variables x and y 
as possibly containing an unsafe value, so they cannot be 
accessed in a  safe function. Only z is known to have a safe 
initial value in this case, so the compiler could allow access to 
it in  safe code."
Mar 02 2021
parent reply Dennis <dkorpel gmail.com> writes:
On Tuesday, 2 March 2021 at 15:34:31 UTC, Dukc wrote:

 How is this piece from the rationale section describing the 
 situation before DIP1035?

 "Since the initialization expression cast(int*) 0xDEADBEEF 
 would not be allowed in a  safe function, and since the initial 
 value of y is unknown, the compiler should annotate variables x 
 and y as possibly containing an unsafe value, so they cannot be 
 accessed in a  safe function. Only z is known to have a safe 
 initial value in this case, so the compiler could allow access 
 to it in  safe code."


I don't understand the question. Is there a contradiction in that 
paragraph I'm supposed to see? Note that the description is about 
what the compiler _should_ do given the current language 
semantics, not what dmd actually does. The version of the DIP 
from the previous review round acknowledged existing holes in 
 safe and was critiqued for it, so this version tries to use the 
language specification as a base rather than the implementation. 
(Though that's not always easy, since the specification is 
lacking in certain areas as well)
Mar 03 2021
parent Dukc <ajieskola gmail.com> writes:
On Wednesday, 3 March 2021 at 10:05:34 UTC, Dennis wrote:

 On Tuesday, 2 March 2021 at 15:34:31 UTC, Dukc wrote:

 How is this piece from the rationale section describing the 
 situation before DIP1035?

 [snip]


 I don't understand the question.


Don't worry, I think you managed to answer it anyway.


 Is there a contradiction in that paragraph I'm supposed to see? 
 Note that the description is about what the compiler _should_ 
 do given the current language semantics, not what dmd actually 
 does.


That explains it. I thought that since the dip says "should" and 
dmd currently does not do it (interpreting `extern int*` as 
` system`), I thought it was explaining what the DIP will do. My 
misunderstanding, though changing a few words might be in order 
to avoid this confusion.
Mar 03 2021
prev sibling next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:

 This is the discussion thread for the second round of Community 
 Review of DIP 1035, " system Variables":


I don't have time to dig into this right now, and the current 
 system/ safe distinction is in need of being replaced by 
«something».

But, the problem with adding more stuff to this aspect of the 
type system is that the language is becoming increasingly 
convoluted.

At some point local improvements no longer improve the whole... 
and a redesign is the only reasonable approach.
Mar 02 2021
parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 2 March 2021 at 16:30:59 UTC, Ola Fosheim Grøstad 
wrote:

 But, the problem with adding more stuff to this aspect of the 
 type system is that the language is becoming increasingly 
 convoluted.

 At some point local improvements no longer improve the whole... 
 and a redesign is the only reasonable approach.


This particular improvement should not add that much complexity 
cost, as it is rather a completion of a previously added feature 
( safe/ trusted/ system) than a new feature. If you know how 
memory safety of D works right now, you can probably figure out 
what ` system extern int;` does. And if you have `struct 
mySlice{int* ptr;  system size_t length;}` it is not probably 
going to surprise you that you can't directly write to either of 
the slice variables.
Mar 02 2021
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 2 March 2021 at 19:32:38 UTC, Dukc wrote:

 This particular improvement should not add that much complexity 
 cost, as it is rather a completion of a previously added 
 feature ( safe/ trusted/ system) than a new feature.


Maybe, but one could probably say that for every individual 
feature. E.g. the various " safe" parameter mechanisms, scope, 
 live. It adds up.
Mar 02 2021
prev sibling next sibling parent reply Guillaume Piolat <first.name spam.org> writes:
On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:

 The review period will end at 11:59 PM ET on March 11, or when 
 I make a post declaring it complete. Discussion in this thread 
 may continue beyond that point.

 Here in the discussion thread, you are free to discuss anything 
 and everything related to the DIP. Express your support or 
 opposition, debate alternatives, argue the merits, etc.


So, in a few words, the merit of the DIP is that after this 
change, you have to review only  trusted functions to ensure a 
 safe function hasn't broken an invariant of the aggregate.

Which is strange to me since I was under the impression 
 safe/ trusted was only about memory safety. But in a way it 
makes sense to use it for "general bugs" since  trusted function 
will be audited. So, no real opinion here. Why not.
Mar 02 2021
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 02.03.21 23:15, Guillaume Piolat wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:

 The review period will end at 11:59 PM ET on March 11, or when I make 
 a post declaring it complete. Discussion in this thread may continue 
 beyond that point.

 Here in the discussion thread, you are free to discuss anything and 
 everything related to the DIP. Express your support or opposition, 
 debate alternatives, argue the merits, etc.


 
 So, in a few words, the merit of the DIP is that after this change, you 
 have to review only  trusted functions to ensure a  safe function hasn't 
 broken an invariant of the aggregate.
 
 Which is strange to me since I was under the impression  safe/ trusted 
 was only about memory safety. But in a way it makes sense to use it for 
 "general bugs" since  trusted function will be audited. So, no real 
 opinion here. Why not.


 system variables are still about memory safety.
Mar 02 2021
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 3/2/21 5:15 PM, Guillaume Piolat wrote:

 On Thursday, 25 February 2021 at 09:21:20 UTC, Mike Parker wrote:

 The review period will end at 11:59 PM ET on March 11, or when I make 
 a post declaring it complete. Discussion in this thread may continue 
 beyond that point.

 Here in the discussion thread, you are free to discuss anything and 
 everything related to the DIP. Express your support or opposition, 
 debate alternatives, argue the merits, etc.


 
 So, in a few words, the merit of the DIP is that after this change, you 
 have to review only  trusted functions to ensure a  safe function hasn't 
 broken an invariant of the aggregate.


I would change this to "you can make it possible" to only have to review 
 trusted functions. Without actually marking the variables  system, you 
would still have to review  safe code.


 Which is strange to me since I was under the impression  safe/ trusted 
 was only about memory safety. But in a way it makes sense to use it for 
 "general bugs" since  trusted function will be audited. So, no real 
 opinion here. Why not.


Inside a  trusted function, you can assume that any data entering the 
function is only set to values that obey the  safe rules. Currently, 
that means certain invariants cannot be realized (like that a length 
field identifies how many bytes are valid past a pointer field).

This DIP allows you to restrict the rules further so more assumptions 
can be made in the  trusted function. You can assume that  system 
variables never were modified directly by  safe functions, and 
therefore, it's only necessary to review  trusted and  system functions 
to ensure the invariants hold.

Given the theory that most code is  safe and should be marked or 
inferred that way, you can therefore vastly reduce the amount of code 
that needs review for memory safety.

-Steve
Mar 04 2021
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 2/25/21 4:21 AM, Mike Parker wrote:

 This is the discussion thread for the second round of Community Review 
 of DIP 1035, " system Variables":
 
 https://github.com/dlang/DIPs/blob/c39f6ac62210e0604dcee99b0092c1930839
93a/DIPs/DIP1035.md 
 
 


In the example for (2):

struct Handle {
      system int handle;
}

// struct with  system field is an unsafe type
 safe   Handle safeHandle = Handle(1);
 system Handle systemHandle = Handle(-1);

...

void main()  safe {
     Handle h0 = safeHandle;        // allowed,  safe variable
     Handle h1 = systemHandle;      // error, reading  system var of 
unsafe type
     ...
}

I'm concerned about the allowance of just declaring a  safe Handle.

If Handle(-1) is unsafe, what is stopping me from doing:

 safe Handle sneakyHandle = Handle(-1);

And can I just do this inside main():

    Handle h2 = Handle(-1);

I guess my biggest problem with this DIP is surrounding the allowance of 
initialization of  system variables without requiring a  system call. 
And/or the weird rules of "you can't do it if it's a  system variable, 
but perfectly fine if you type out the initializer"

Or maybe I'm misunderstanding something. Perhaps it would be good to 
specify how one prevents anything in  safe from using Handle(-1).

------

Another note, the ShortString example is unsafe, even with the DIP, as 
`s[]` will provide access to data that might move elsewhere.

Perhaps it would be good to restate the examples with the assumption the 
DIP is implemented, and show why they are now fully  safe.

-Steve
Mar 04 2021
parent Paul Backus <snarwin gmail.com> writes:
On Thursday, 4 March 2021 at 20:10:05 UTC, Steven Schveighoffer 
wrote:

 In the example for (2):

 struct Handle {
      system int handle;
 }


[...]

 If Handle(-1) is unsafe, what is stopping me from doing:

  safe Handle sneakyHandle = Handle(-1);


Nothing, because Handle(-1) isn't unsafe. If your next question 
is "then why use a  system variable?", the answer is that this 
isn't intended to be a realistic example. It's just an 
illustration of the semantics of  system variables.

If you actually wanted to maintain an invariant like "Handles 
can't be negative", you would have to write a constructor:

struct Handle {
      system int handle;
     this(int n)  safe {
         assert(n >= 0);
         handle = n;
     }
}


 Another note, the ShortString example is unsafe, even with the 
 DIP, as `s[]` will provide access to data that might move 
 elsewhere.


ShortString doesn't contain internal pointers, and the compiler 
won't let `s[]` outlive the ShortString's lexical scope, which 
means that the problem in issue 17448 [1] can't happen here. So 
it should be fine, unless there's something else I'm missing.

It does require DIP 1000, which means that opIndex should be 
annotated with `return`.

[1] https://issues.dlang.org/show_bug.cgi?id=17448
Mar 04 2021