www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - `in` on function parameters: const, scope const, ref scope const ?

reply Mathias LANG <geod24 gmail.com> writes:
There's been a discussion recently about `-preview` switches 
(https://forum.dlang.org/thread/r627m8$2p6c$1 digitalmars.com).

It turns out one of the preview switch that is to be introduced 
next release is `-preview=in`, meant to make `in` on parameters 
(not to be confused with opIn) means `scope const` instead of 
just `const`.
Now I wasn't particularly fond of this PR, since it doesn't seem 
logical to me to have 2 switches for the same thing (`in` was 
changed to mean `const` when DIP1000 was a PR, aiming to reduce 
the breakage, but before it was behind a switch, which completely 
nullified the need for changing it in the first place).

But since leadership decided to go with the switch anyway, why 
not go all the way and make it means what it was actually 
supposed to mean in the first place ? My experience with D is 
that I had to stick quite a few `const ref scope` (in varying 
order) on my parameters to mean "this is an input parameter. I'm 
going to read it, not modify it. And I will not keep a reference 
to it, because I just need it for the processing of this 
function". However I couldn't use `in` because it would pass 
things by value, meaning my `struct`s would get blitted over and 
over.

I have submitted a PR for it: 
https://github.com/dlang/dmd/pull/11000 and would like to know 
how other users of `in` expect it to behave, or would want it to 
behave.
Bear in mind this is a `-preview` switch: it won't break your 
code.
One of the downside I'm aware of is passing rvalue. But luckily, 
we got another preview for this (`-preview=rvaluerefparam`).
Apr 02 2020
next sibling parent reply tsbockman <thomas.bockman gmail.com> writes:
On Friday, 3 April 2020 at 06:19:38 UTC, Mathias LANG wrote:

 I have submitted a PR for it: 
 https://github.com/dlang/dmd/pull/11000 and would like to know 
 how other users of `in` expect it to behave, or would want it 
 to behave.


Given the possibility of aliasing, adding `ref` to parameters 
that are merely `const`, but not `immutable`, could break 
existing code in very subtle ways. (I know there has been talk of 
disallowing aliasing of `ref` arguments, but I don't think D's 
tracking of such things is reliable enough to depend on to 
prevent code breakage yet. Also, I'm hoping that proposal isn't 
actually implemented, anyway, because I find intentionally 
aliasing `ref` parameters useful.) Less importantly, adding `ref` 
where it wasn't intended may cause minor performance problems.

However, if I ignore the code breakage issue, then `in` == `ref 
scope const` is the most aesthetically pleasing option for the 
symmetry with `out`. I'll also guess that by-value `scope const` 
is needed a lot less often than `ref scope const`, so this is 
probably the better long-term option to reduce the considerable 
verbosity of the language - IF we can avoid breaking too much 
existing code due the altered run-time semantics from adding 
`ref`.
Apr 04 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 5 April 2020 at 02:01:05 UTC, tsbockman wrote:

 On Friday, 3 April 2020 at 06:19:38 UTC, Mathias LANG wrote:

 I have submitted a PR for it: 
 https://github.com/dlang/dmd/pull/11000 and would like to know 
 how other users of `in` expect it to behave, or would want it 
 to behave.


 Given the possibility of aliasing, adding `ref` to parameters 
 that are merely `const`, but not `immutable`, could break 
 existing code in very subtle ways.


Interesting, I wasn't aware of this. Can you elaborate more?


 (I know there has been talk of disallowing aliasing of `ref`
 arguments, but I don't think D's tracking of such things is
 reliable enough to depend on to prevent code breakage yet. Also,
 I'm hoping that proposal isn't actually implemented, anyway,
 because I find intentionally aliasing `ref` parameters useful.)


Can you provide an example?


 However, if I ignore the code breakage issue, then `in` == `ref 
 scope const` is the most aesthetically pleasing option for the 
 symmetry with `out`.


I completely agree.


 I'll also guess that by-value `scope const` is needed a lot less
 often than `ref scope const`, so this is probably the better
 long-term option to reduce the considerable verbosity of the
 language - IF we can avoid breaking too much existing code due 
 the
 altered run-time semantics from adding `ref`.


I think the best option would be what  kinke proposed in that 
pull request:


 I think the optimal solution would be either `const scope` or 
 `const scope ref`, depending on the type and target ABI, for 
 the most efficient way to pass an input parameter. I.e., no ref 
 (extra indirection) for class references and small PODs (incl. 
 delegates and slices for most ABIs), but a ref for non-PODs and 
 large PODs. To be coupled with `-preview=rvaluerefparam` as 
 you've mentioned in the forum thread.


https://github.com/dlang/dmd/pull/11000#issuecomment-608382442
Apr 05 2020
parent tsbockman <thomas.bockman gmail.com> writes:
On Sunday, 5 April 2020 at 12:54:01 UTC, Petar Kirov [ZombineDev] 
wrote:

 On Sunday, 5 April 2020 at 02:01:05 UTC, tsbockman wrote:

 Given the possibility of aliasing, adding `ref` to parameters 
 that are merely `const`, but not `immutable`, could break 
 existing code in very subtle ways.


 Interesting, I wasn't aware of this. Can you elaborate more?


`const` marks a read-only view of some data, for which there 
*may* currently be mutable aliases, as well. `immutable` means 
that all current aliases for the data are read-only, and so it is 
guaranteed not to change as long as the type system is not 
subverted through unsafe casting.

import std.stdio;

bool isPositive(in int x, scope ref int callCount) pure  safe {
     callCount += 1;
     return (x > 0);
}

void main()  safe {
     int a = 0;
     writeln(isPositive(a, a));
}

The above should print "true" if `in` means `ref scope const`, 
and "false" if `in` means just `scope const`. This minimal 
example is completely contrived, of course, but the issue does 
appear in reasonable real world code, too.

Aliasing can occur via `ref`, but also via class references, 
pointers, delegates, array slices, and user-defined data 
structures containing or referencing any of those things at any 
level of indirection. And, those are just things that need to be 
tracked to detect aliasing in a program that is 100%  safe code; 
with  trusted, there is the possibility that additional aliases 
may be encoded, and correctly managed, in arbitrary data types 
that are not understood as pointer types by the semantic analyzer:

import std.stdio;

struct CustomRef {
     private size_t address;
     void opAssign(int* target) pure  trusted nothrow  nogc {
         address = cast(size_t) target;
     }
     ref inout(int) access() inout pure  trusted nothrow  nogc {
         return *cast(inout(int)*) address;
     }
}

bool isPositive(ref scope const int x, scope ref int callCount) 
pure  safe {
     callCount += 1;
     return (x > 0);
}

int a = 0;
CustomRef q;

void main()  safe {
     q = &a;
     writeln(isPositive(a, q.access));
}

Again, this is a contrived example, and simple enough that the 
semantic analyzer could no doubt be extended to catch it. But, 
there are valid practical applications for code like this, some 
of which may be far more difficult, or even impossible, to 
analyze automatically in the general case.

So, changing `in` to be pass-by-ref is, at least in theory, a 
very serious breaking change - *far* more so than adding `scope`, 
which was always planned anyway. If we're going to do this I 
think a long deprecation cycle is needed for the existing 
behavior, with aggressive warnings about the upcoming change.

I don't think it will break very much code in the real world, 
it's just that some of the code it does break will probably fail 
in such a subtle way, for certain inputs only, that it will be a 
real pain to debug.
Apr 05 2020
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 3 April 2020 at 06:19:38 UTC, Mathias LANG wrote:

 [..]


One more thought:

`in` needs be its own parameter storage class, like `out` and 
`ref` on the name-mangling / ABI level. This is the only way to 
change its meaning while preserving it in compiler-generated *.di 
headers and preventing subtle errors while linking different 
libraries compiled with and without the preview switch.
Apr 05 2020
parent tsbockman <thomas.bockman gmail.com> writes:
On Sunday, 5 April 2020 at 12:59:23 UTC, Petar Kirov [ZombineDev] 
wrote:

 `in` needs be its own parameter storage class, like `out` and 
 `ref` on the name-mangling / ABI level.


If we go with  kinke 's suggestion, then yes `in` definitely 
needs to be its own storage class, and we'll need good 
meta-programming tools for working with it.
Apr 05 2020