www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Ranges and safe

reply SrMordred <patric.dexheimer gmail.com> writes:
I think that .front in ranges should'nt be safe by default.

 safe{ iota(0,0).front(); } //BOOM, but compiles on  safe.

I can put a  system on front method, but then
 safe{ foreach(v ; range){ ... } } //dont compile, its not safe.
But i think that this lowered code should at least be  trusted 
since the algorithm is using the range correctly and can´t do 
unsafe things (right?).

I can wrap the range in opApply and solve this, but its a dirty 
trick i think.
What's your ideas on this?

all the code i'm talking here:
https://run.dlang.io/is/yPy26j
Sep 22 2019
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Sunday, 22 September 2019 at 23:03:32 UTC, SrMordred wrote:

 I think that .front in ranges should'nt be safe by default.

  safe{ iota(0,0).front(); } //BOOM, but compiles on  safe.


 safe does not mean "does not have runtime errors", it means 
"does not corrupt memory". Accessing the front of an empty range 
may crash your program, but it shouldn't cause memory corruption 
(unless you disable bounds-checking).
Sep 22 2019
parent SrMordred <patric.dexheimer gmail.com> writes:
On Sunday, 22 September 2019 at 23:18:12 UTC, Paul Backus wrote:

 On Sunday, 22 September 2019 at 23:03:32 UTC, SrMordred wrote:

 I think that .front in ranges should'nt be safe by default.

  safe{ iota(0,0).front(); } //BOOM, but compiles on  safe.


  safe does not mean "does not have runtime errors", it means 
 "does not corrupt memory". Accessing the front of an empty 
 range may crash your program, but it shouldn't cause memory 
 corruption (unless you disable bounds-checking).


Hm.. yes.
Maybe i´m taking this on the wrong angle.
Maybe  safe should be more constraint then.

I feel like code under  safe should always be ok, and if any 
thing breaks you should look at
 trusted code to see where is the problem.
You "box" unsafe code on specific places (like Rust do with 
unsafe)
Sep 22 2019
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, September 22, 2019 5:03:32 PM MDT SrMordred via Digitalmars-d 
wrote:

 I think that .front in ranges should'nt be safe by default.

  safe{ iota(0,0).front(); } //BOOM, but compiles on  safe.

 I can put a  system on front method, but then
  safe{ foreach(v ; range){ ... } } //dont compile, its not safe.
 But i think that this lowered code should at least be  trusted
 since the algorithm is using the range correctly and can´t do
 unsafe things (right?).

 I can wrap the range in opApply and solve this, but its a dirty
 trick i think.
 What's your ideas on this?

 all the code i'm talking here:
 https://run.dlang.io/is/yPy26j


You do not seem to understand what  safe is for.  safe is for checking for
memory safety.  safe code throw Exceptions. It can throw Errors. It can
segfault. It can be extremely buggy. But it can't access invalid memory
(e.g. by using an index which is out-of-bounds with an array or by accessing
memory that points to an object that no longe exists).

If code is marked as  safe, the the compiler mechanically verifies for the
programmer that no operations which are considered  system are used within
that code. Barring a compiler bug, code which the compiler has mechanically
verified as being  safe cannot access invalid memory and thus has no bugs
related to memory safety. That's all that it guarantees. It doesn't
guarantee that any other bugs don't exist within that code.

If a function is marked as  system, then the compiler considers that
function to be  system and thus potentially not memory safe. So, any code
that calls will consider it  system, and when the function itself is
compiled, its memory safety is not checked by the compiler. So, you'll never
get something like the compiler flagging a function that was marked as
 system as something that's actually memory safe. You'll just get the
compiler flagging  system operations within an  safe function.

You can mark a function as  trusted to indicate that that function is  safe
without the compiler doing any checks. When you do that, you're telling the
compiler that you know that the code is memory safe and that it doesn't have
to check it. This is intended for cases where a function has  system
operations but where the programmer is able to verify that the code is
actually  safe with what it's doing with those  system operations.

For instance, pointer arithmetic is  system, because the compiler cannot
verify that you're accessing valid memory when you use pointer arithmetic.
So, a function like

int foo(int[] arr)  safe
{
    if(arr.length < 2)
        return int.min;
    auto ptr = arr.ptr;
    ++ptr;
    return *ptr;
}

would fail to compile, because it's doing pointer arithemtic. However, it's
possible for the programmer to look at that code and see that the pointer
arithmetic it's doing won't actually access invalid memory. So, they can
mark it as  trusted to inform the compiler that it can be considered  safe.
As such,

int foo(int[] arr)  trusted
{
    if(arr.length < 2)
        return int.min;
    auto ptr = arr.ptr;
    ++ptr;
    return *ptr;
}

would compile, and any code that calls foo would consider it to be  safe. As
long as the programmer was correct that foo was actually  safe, then any
 safe code that calls foo is guaranteed to not violate memory safety.
However, if the programmer gets it wrong, then there's a bug in the program
which could result in it accessing invalid memory, and the  safe code isn't
actually memory safe. So, with  trusted, it's the programmer's
responsibility to get it right, and the memory safety of any  safe code that
calls that  trusted code relies on the programmer having gotten it right.

Because the  safety of templated functions often depends on the template
arguments (e.g. std.algorithm's find is  safe if the range that it's given
is  safe, but it wouldn't be  safe if it was given a range where one or more
of its range API functions were  system),  safe is inferred for templates.
So,

int foo()(int v)
{
    return v;
}

would have its  safety inferred by the compiler, and because foo contains no
 system operations, it's considered  safe. Similarly,

int foo()(int* ptr)
{
    ++ptr;
    return *ptr;
}

would be inferred to be  system, because it contains  system operations.
However, explicitly marking the function as  safe,  trusted, or  system
works exactly the same as it would on a non-templated function.

 safe is also inferred for functions whose return type is inferred. e.g.

auo foo(int v)
{
    return v;
}

would be inferred to be  safe, whereas

auto foo(int* ptr)
{
    ++ptr;
    return *ptr;
}

would be inferred to be  system. As with templated functions, explicitly
marking the function with  safe,  trusted, or  system overrides the
attribute inference and  safety is checked per the attribute that was
provided just like with a normal function.

As for ranges, the vast majority of range-based code is templated, so the
vast majority of it uses attribute inference, but regardless, the compiler
doesn't treat ranges as special in any way shape or form with regards to
 safety.

Your example of iota(0, 0).front is perfectly  safe. The compiler correctly
infers iota's front to be  safe, because it does nothing which violates
memory safety. Any program that has iota(0, 0).front is buggy, because it's
calling front on an empty range, and if that program is not compiled with
-release, then an assertion in iota's front will fail, resulting in an
AssertError being thrown, but none of that violates  safety, because it
doesn't do anything that can access invalid memory. It's buggy, and any
program that does it needs to be fixed, but it's  safe.

As for your larger example, you've marked Range's front as  system, so it
will be treated as  system even though it's not doing anything that violates
memory safety, whereas you've marked its empty and popFront as  trusted, so
they'll be considered  safe regardless of whether they violate memory safety
(which they don't). As such, the code

    auto range = Range(2);
    foreach(v ; range ) writeln(v); //not safe!

correctly gets flagged by the compiler as being  system. It can't actually
violate memory safety, but the compiler doesn't know that. It's just
treating Range's front as  system, because you told it to.

You then marked RangeWrap's entire implementation as  trusted. So, of course

    auto range = Range(2);
    auto range_wrap = RangeWrap(range);
    foreach(v ; range_wrap ) writeln(v); //fine, but its a trick.

compiles, and it's not a bug. You told the compiler to treat RangeWrap's
functions as  trusted, so the compiler doesn't check them, and it's up to
you to verify that they're actually  safe. RangeWrap's opApply calls the
 system function front on Range, so it's doing an operation that the
compiler considers  system, but you marked RangeWrap's opApply as  trusted,
so if it's not memory safe to call Range's front, then it's up to you to
catch that and fix it. There is no compiler bug here.

Of course, none of this code is actually doing anything that isn't memory
safe - presumably, you're just marking it the way you did to provide an
example without having to have actual  system operations in Range's front -
but all of the checks that the compiler is or isn't doing based on those
attributes are correct. Any time that you use  trusted, you have to manually
verify the code yourself for whether it violates memory safety. The compiler
trusts you when you use  trusted. So, don't expect it to catch any  safety
violations in  trusted code. You told it that that code was  safe when you
used  trusted.

And again,  safe is all about memory safety. Exceptions, Errors, and
segfaults are all memory safe. Many, many bugs are memory safe. Don't expect
 safe to catch all of the various bugs in your program or even to prevent
all crashes. It's just catching code that is potentially not memory safe,
and when you use  trusted, you're telling it to not check that code, so
any memory safety bugs in that code are on you.

- Jonathan M Davis
Sep 22 2019
parent reply SrMordred <patric.dexheimer gmail.com> writes:
On Monday, 23 September 2019 at 00:16:01 UTC, Jonathan M Davis 
wrote:

 On Sunday, September 22, 2019 5:03:32 PM MDT SrMordred via 
 Digitalmars-d wrote:


Yes, thanks for the lengthy explanations :)

I was coding betterC code with  safe in mind , and since i was 
trying to make everything as safe as possible i just forgot about 
what  safe is really about and started to see this "problems" on 
safe blocks.

I think that what i wanted was a  reallySafe flag :P
Sep 22 2019
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Monday, 23 September 2019 at 00:33:02 UTC, SrMordred wrote:

 I think that what i wanted was a  reallySafe flag :P


You should always strive to process ranges as sets of ordered 
data that are potentially empty. In most cases you can prevent 
the use of the member empty() by instead using `foreach` or 
range-based algorithms.
Sep 23 2019