www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Ranges and safe

reply SrMordred <patric.dexheimer gmail.com> writes:
I think that .front in ranges should'nt be safe by default.

 safe{ iota(0,0).front(); } //BOOM, but compiles on  safe.

I can put a  system on front method, but then
 safe{ foreach(v ; range){ ... } } //dont compile, its not safe.
But i think that this lowered code should at least be  trusted 
since the algorithm is using the range correctly and can´t do 
unsafe things (right?).

I can wrap the range in opApply and solve this, but its a dirty 
trick i think.
What's your ideas on this?

all the code i'm talking here:
https://run.dlang.io/is/yPy26j
Sep 22
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Sunday, 22 September 2019 at 23:03:32 UTC, SrMordred wrote:
 I think that .front in ranges should'nt be safe by default.

  safe{ iota(0,0).front(); } //BOOM, but compiles on  safe.
safe does not mean "does not have runtime errors", it means "does not corrupt memory". Accessing the front of an empty range may crash your program, but it shouldn't cause memory corruption (unless you disable bounds-checking).
Sep 22
parent SrMordred <patric.dexheimer gmail.com> writes:
On Sunday, 22 September 2019 at 23:18:12 UTC, Paul Backus wrote:
 On Sunday, 22 September 2019 at 23:03:32 UTC, SrMordred wrote:
 I think that .front in ranges should'nt be safe by default.

  safe{ iota(0,0).front(); } //BOOM, but compiles on  safe.
safe does not mean "does not have runtime errors", it means "does not corrupt memory". Accessing the front of an empty range may crash your program, but it shouldn't cause memory corruption (unless you disable bounds-checking).
Hm.. yes. Maybe i´m taking this on the wrong angle. Maybe safe should be more constraint then. I feel like code under safe should always be ok, and if any thing breaks you should look at trusted code to see where is the problem. You "box" unsafe code on specific places (like Rust do with unsafe)
Sep 22
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, September 22, 2019 5:03:32 PM MDT SrMordred via Digitalmars-d 
wrote:
 I think that .front in ranges should'nt be safe by default.

  safe{ iota(0,0).front(); } //BOOM, but compiles on  safe.

 I can put a  system on front method, but then
  safe{ foreach(v ; range){ ... } } //dont compile, its not safe.
 But i think that this lowered code should at least be  trusted
 since the algorithm is using the range correctly and cant do
 unsafe things (right?).

 I can wrap the range in opApply and solve this, but its a dirty
 trick i think.
 What's your ideas on this?

 all the code i'm talking here:
 https://run.dlang.io/is/yPy26j
You do not seem to understand what safe is for. safe is for checking for memory safety. safe code throw Exceptions. It can throw Errors. It can segfault. It can be extremely buggy. But it can't access invalid memory (e.g. by using an index which is out-of-bounds with an array or by accessing memory that points to an object that no longe exists). If code is marked as safe, the the compiler mechanically verifies for the programmer that no operations which are considered system are used within that code. Barring a compiler bug, code which the compiler has mechanically verified as being safe cannot access invalid memory and thus has no bugs related to memory safety. That's all that it guarantees. It doesn't guarantee that any other bugs don't exist within that code. If a function is marked as system, then the compiler considers that function to be system and thus potentially not memory safe. So, any code that calls will consider it system, and when the function itself is compiled, its memory safety is not checked by the compiler. So, you'll never get something like the compiler flagging a function that was marked as system as something that's actually memory safe. You'll just get the compiler flagging system operations within an safe function. You can mark a function as trusted to indicate that that function is safe without the compiler doing any checks. When you do that, you're telling the compiler that you know that the code is memory safe and that it doesn't have to check it. This is intended for cases where a function has system operations but where the programmer is able to verify that the code is actually safe with what it's doing with those system operations. For instance, pointer arithmetic is system, because the compiler cannot verify that you're accessing valid memory when you use pointer arithmetic. So, a function like int foo(int[] arr) safe { if(arr.length < 2) return int.min; auto ptr = arr.ptr; ++ptr; return *ptr; } would fail to compile, because it's doing pointer arithemtic. However, it's possible for the programmer to look at that code and see that the pointer arithmetic it's doing won't actually access invalid memory. So, they can mark it as trusted to inform the compiler that it can be considered safe. As such, int foo(int[] arr) trusted { if(arr.length < 2) return int.min; auto ptr = arr.ptr; ++ptr; return *ptr; } would compile, and any code that calls foo would consider it to be safe. As long as the programmer was correct that foo was actually safe, then any safe code that calls foo is guaranteed to not violate memory safety. However, if the programmer gets it wrong, then there's a bug in the program which could result in it accessing invalid memory, and the safe code isn't actually memory safe. So, with trusted, it's the programmer's responsibility to get it right, and the memory safety of any safe code that calls that trusted code relies on the programmer having gotten it right. Because the safety of templated functions often depends on the template arguments (e.g. std.algorithm's find is safe if the range that it's given is safe, but it wouldn't be safe if it was given a range where one or more of its range API functions were system), safe is inferred for templates. So, int foo()(int v) { return v; } would have its safety inferred by the compiler, and because foo contains no system operations, it's considered safe. Similarly, int foo()(int* ptr) { ++ptr; return *ptr; } would be inferred to be system, because it contains system operations. However, explicitly marking the function as safe, trusted, or system works exactly the same as it would on a non-templated function. safe is also inferred for functions whose return type is inferred. e.g. auo foo(int v) { return v; } would be inferred to be safe, whereas auto foo(int* ptr) { ++ptr; return *ptr; } would be inferred to be system. As with templated functions, explicitly marking the function with safe, trusted, or system overrides the attribute inference and safety is checked per the attribute that was provided just like with a normal function. As for ranges, the vast majority of range-based code is templated, so the vast majority of it uses attribute inference, but regardless, the compiler doesn't treat ranges as special in any way shape or form with regards to safety. Your example of iota(0, 0).front is perfectly safe. The compiler correctly infers iota's front to be safe, because it does nothing which violates memory safety. Any program that has iota(0, 0).front is buggy, because it's calling front on an empty range, and if that program is not compiled with -release, then an assertion in iota's front will fail, resulting in an AssertError being thrown, but none of that violates safety, because it doesn't do anything that can access invalid memory. It's buggy, and any program that does it needs to be fixed, but it's safe. As for your larger example, you've marked Range's front as system, so it will be treated as system even though it's not doing anything that violates memory safety, whereas you've marked its empty and popFront as trusted, so they'll be considered safe regardless of whether they violate memory safety (which they don't). As such, the code auto range = Range(2); foreach(v ; range ) writeln(v); //not safe! correctly gets flagged by the compiler as being system. It can't actually violate memory safety, but the compiler doesn't know that. It's just treating Range's front as system, because you told it to. You then marked RangeWrap's entire implementation as trusted. So, of course auto range = Range(2); auto range_wrap = RangeWrap(range); foreach(v ; range_wrap ) writeln(v); //fine, but its a trick. compiles, and it's not a bug. You told the compiler to treat RangeWrap's functions as trusted, so the compiler doesn't check them, and it's up to you to verify that they're actually safe. RangeWrap's opApply calls the system function front on Range, so it's doing an operation that the compiler considers system, but you marked RangeWrap's opApply as trusted, so if it's not memory safe to call Range's front, then it's up to you to catch that and fix it. There is no compiler bug here. Of course, none of this code is actually doing anything that isn't memory safe - presumably, you're just marking it the way you did to provide an example without having to have actual system operations in Range's front - but all of the checks that the compiler is or isn't doing based on those attributes are correct. Any time that you use trusted, you have to manually verify the code yourself for whether it violates memory safety. The compiler trusts you when you use trusted. So, don't expect it to catch any safety violations in trusted code. You told it that that code was safe when you used trusted. And again, safe is all about memory safety. Exceptions, Errors, and segfaults are all memory safe. Many, many bugs are memory safe. Don't expect safe to catch all of the various bugs in your program or even to prevent all crashes. It's just catching code that is potentially not memory safe, and when you use trusted, you're telling it to not check that code, so any memory safety bugs in that code are on you. - Jonathan M Davis
Sep 22
parent reply SrMordred <patric.dexheimer gmail.com> writes:
On Monday, 23 September 2019 at 00:16:01 UTC, Jonathan M Davis 
wrote:
 On Sunday, September 22, 2019 5:03:32 PM MDT SrMordred via 
 Digitalmars-d wrote:
Yes, thanks for the lengthy explanations :) I was coding betterC code with safe in mind , and since i was trying to make everything as safe as possible i just forgot about what safe is really about and started to see this "problems" on safe blocks. I think that what i wanted was a reallySafe flag :P
Sep 22
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 23 September 2019 at 00:33:02 UTC, SrMordred wrote:
 I think that what i wanted was a  reallySafe flag :P
You should always strive to process ranges as sets of ordered data that are potentially empty. In most cases you can prevent the use of the member empty() by instead using `foreach` or range-based algorithms.
Sep 23