www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Should 'in' Imply 'ref' as Well for Value Types?

reply Vijay Nayar <madric gmail.com> writes:
While working on a library built for high efficiency, avoiding 
unnecessary copies of structs became an issue.  I had assumed 
that `in` was doing this, but a bit of experimentation revealed 
that it does not.  However, `ref in` works great.

My question is, should `in` by default also imply `ref` for value 
types like structs?  Is there a reason not to do this?

This is the test program I used for reference:

```
import std.stdio;

struct Bob {
     int a;
     this(this) {
       writeln("<Bob copied>");
     }
}

void main()
{
     Bob b = Bob(3);
     writeln("                &b = ", &b);
     void showAddrIn(in Bob b) {
         writeln("(showAddrIn)    &b = ", &b);
     }
     showAddrIn(b);
     void showAddrRefIn(ref in Bob b) {
         writeln("(showAddrRefIn) &b = ", &b);
     }
     showAddrRefIn(b);
}
```

The output is as follows:

```
                 &b = 7FFD9F526AD0
<Bob copied>
(showAddrIn)    &b = 7FFD9F526AB0
(showAddrRefIn) &b = 7FFD9F526AD0
```
May 04 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, May 04, 2018 09:15:57 Vijay Nayar via Digitalmars-d wrote:
 While working on a library built for high efficiency, avoiding
 unnecessary copies of structs became an issue.  I had assumed
 that `in` was doing this, but a bit of experimentation revealed
 that it does not.  However, `ref in` works great.

 My question is, should `in` by default also imply `ref` for value
 types like structs?  Is there a reason not to do this?
ref only accepts lvalues and as such can be extremely annoying to use. There's no question that it has its place, but in most cases, you don't want it, because it makes it so that you then mostly can't chain function calls and are forced to store all of the function results on the stack in order to call the next function. And addition to being annoying, it's usually less less efficient, because if you pass an rvalue to a function, then the value is moved to the parameter, which can't be done if you store the value on the stack. It's actually not infrequent now that in C++, you want to pass stuf by value rather than const& precisely because move semantics can be used to avoid copies. So, it's not at all necessarily the case that passing by ref is the efficient thing to do. It's heavily dependent on the code in question. So, even if we were designing in from scratch, making ref would be a questionable choice. However, even if we all agreed that it would be desirable, such a change would break a large percentage of code that currently uses in, because any code that passes an rvalue to a function taking its argument as in would then break. So, any attempt to change in to include ref would have to have some sort of deprecation path to avoid code breakage. But given how annoying it would be to have in be ref by default due to how that affects rvalues, I'm willing to bet that most programmers would not be at all happy with such a change, regardless of how well the transitition was handled. - Jonathan M Davis
May 04 2018
parent reply Bolpat <qs.il.paperinik gmail.com> writes:
On Friday, 4 May 2018 at 09:34:14 UTC, Jonathan M Davis wrote:
 [...]
 It's actually not infrequent now that in C++, you want to pass 
 stuf by value rather than const& precisely because move 
 semantics can be used to avoid copies. So, it's not at all 
 necessarily the case that passing by ref is the efficient thing 
 to do. It's heavily dependent on the code in question.
I once proposed that `in` can mean `const scope ref` that also binds rvalues. https://github.com/dlang/DIPs/pull/111#issuecomment-381911140 We could make `in` be something similar to `inline`. The compiler can implement it as stated above (assign the expression to temporary, reference it), or use copy if copy is cheaper than referencing.
 So, even if we were designing in from scratch, making ref would 
 be a questionable choice. However, even if we all agreed that 
 it would be desirable, such a change would break a large 
 percentage of code that currently uses in, because any code 
 that passes an rvalue to a function taking its argument as in 
 would then break. So, any attempt to change in to include ref 
 would have to have some sort of deprecation path to avoid code 
 breakage. But given how annoying it would be to have in be ref 
 by default due to how that affects rvalues, I'm willing to bet 
 that most programmers would not be at all happy with such a 
 change, regardless of how well the transitition was handled.
Yes. Therefore the idea above. It will break much less code if any.
May 05 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, May 05, 2018 15:22:04 Bolpat via Digitalmars-d wrote:
 On Friday, 4 May 2018 at 09:34:14 UTC, Jonathan M Davis wrote:
 [...]
 It's actually not infrequent now that in C++, you want to pass
 stuf by value rather than const& precisely because move
 semantics can be used to avoid copies. So, it's not at all
 necessarily the case that passing by ref is the efficient thing
 to do. It's heavily dependent on the code in question.
I once proposed that `in` can mean `const scope ref` that also binds rvalues. https://github.com/dlang/DIPs/pull/111#issuecomment-381911140 We could make `in` be something similar to `inline`. The compiler can implement it as stated above (assign the expression to temporary, reference it), or use copy if copy is cheaper than referencing.
Having ref of any kind accept rvalues is a highly controversial issue, and it may or may not ever be in the language. If it's added, then at that point, whether it makes sense to make in imply ref could be re-examined, and maybe at that point, doing so would make great sense. But as long as ref does not accept rvalues, it really doesn't make sense. It would break too much code and would be far too annoying to use in many, many cases where it is currently commonly used. - Jonathan M Davis
May 05 2018
parent reply Q. Schroll <qs.il.paperinik gmail.com> writes:
On Saturday, 5 May 2018 at 15:39:19 UTC, Jonathan M Davis wrote:
 On Saturday, May 05, 2018 15:22:04 Bolpat via Digitalmars-d 
 wrote:
 On Friday, 4 May 2018 at 09:34:14 UTC, Jonathan M Davis wrote:
 [...]
 It's actually not infrequent now that in C++, you want to 
 pass
 stuf by value rather than const& precisely because move
 semantics can be used to avoid copies. So, it's not at all
 necessarily the case that passing by ref is the efficient 
 thing
 to do. It's heavily dependent on the code in question.
I once proposed that `in` can mean `const scope ref` that also binds rvalues. https://github.com/dlang/DIPs/pull/111#issuecomment-381911140 We could make `in` be something similar to `inline`. The compiler can implement it as stated above (assign the expression to temporary, reference it), or use copy if copy is cheaper than referencing.
Having ref of any kind accept rvalues is a highly controversial issue, and it may or may not ever be in the language. If it's added, then at that point, whether it makes sense to make in imply ref could be re-examined, and maybe at that point, doing so would make great sense. But as long as ref does not accept rvalues, it really doesn't make sense. It would break too much code and would be far too annoying to use in many, many cases where it is currently commonly used.
I never suggested some spelled out `ref` should bind rvalues. Having explicit `ref` bind rvalues is a mistake C++ did and it confuses the hell out of people. For clarification: `in` should mean for the called function that it may only "look at" the information for decisions, but not "touch" it. In this sense, not only modifying, also copying or leaking it are a forms of touching. Note that objects of a non-copyable types can be looked at. This is how I came to what `in` naturally must mean. It must mean `const`. It must mean `scope`. It must mean referencing, but not in the restrictive way of `ref`, but in the permissive interpretation, so it may bind rvalues, too. For the caller, `in` basically is `const scope` with guaranteed no copying. So, `in` should not imply `ref`, it should imply referencing, which is not the same thing. `in` and `ref` could be combined, so that the restrictive character of `ref` does its job, but I'd favor not to allow that (similarly `out ref` is not allowed). If you want `const scope ref`, spell it out. I'd assume the cases where you want to allow lvalues only are rare. They well may exist, so it's good to be able to express them.
May 06 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, May 06, 2018 21:26:32 Q. Schroll via Digitalmars-d wrote:
 On Saturday, 5 May 2018 at 15:39:19 UTC, Jonathan M Davis wrote:
 On Saturday, May 05, 2018 15:22:04 Bolpat via Digitalmars-d

 wrote:
 On Friday, 4 May 2018 at 09:34:14 UTC, Jonathan M Davis wrote:
 [...]
 It's actually not infrequent now that in C++, you want to
 pass
 stuf by value rather than const& precisely because move
 semantics can be used to avoid copies. So, it's not at all
 necessarily the case that passing by ref is the efficient
 thing
 to do. It's heavily dependent on the code in question.
I once proposed that `in` can mean `const scope ref` that also binds rvalues. https://github.com/dlang/DIPs/pull/111#issuecomment-381911140 We could make `in` be something similar to `inline`. The compiler can implement it as stated above (assign the expression to temporary, reference it), or use copy if copy is cheaper than referencing.
Having ref of any kind accept rvalues is a highly controversial issue, and it may or may not ever be in the language. If it's added, then at that point, whether it makes sense to make in imply ref could be re-examined, and maybe at that point, doing so would make great sense. But as long as ref does not accept rvalues, it really doesn't make sense. It would break too much code and would be far too annoying to use in many, many cases where it is currently commonly used.
I never suggested some spelled out `ref` should bind rvalues. Having explicit `ref` bind rvalues is a mistake C++ did and it confuses the hell out of people. For clarification: `in` should mean for the called function that it may only "look at" the information for decisions, but not "touch" it. In this sense, not only modifying, also copying or leaking it are a forms of touching. Note that objects of a non-copyable types can be looked at. This is how I came to what `in` naturally must mean. It must mean `const`. It must mean `scope`. It must mean referencing, but not in the restrictive way of `ref`, but in the permissive interpretation, so it may bind rvalues, too. For the caller, `in` basically is `const scope` with guaranteed no copying. So, `in` should not imply `ref`, it should imply referencing, which is not the same thing. `in` and `ref` could be combined, so that the restrictive character of `ref` does its job, but I'd favor not to allow that (similarly `out ref` is not allowed). If you want `const scope ref`, spell it out. I'd assume the cases where you want to allow lvalues only are rare. They well may exist, so it's good to be able to express them.
in has always copied if it was not also used with ref, and changing that would break code. It's also never actually really meant scope, much as that was the intention (mostly because aside from delegates, scope has never done anything). -dip1000 is finally doing something with scope, but in the process, it looks likely that in will then not mean scope due to the amount of code that would break if it did, but we'll have to see how that goes. Historically though, in has been pretty much identical to const. Regardless, you seem to be describing something that has nothing to do with how any type qualifier or storage class currently acts. If you want to anything like that, you're going to have to create a DIP for it, and if you're looking for in to mean anything drastically different like it seems like you want to, you're going to have to present good arguments for why such a change would make good sense, what the resulting code breakage would be, and why it would be worth it. And honestly, much as you say that you don't want to have ref bind rvalues, what you're saying sounds a lot like you're arguing for an equivalent to const& with the addition that it's also scope, and Andrei has repeatedly shot down anything similar to C++'s const&. So, you'll need solid arguments for why what you want is a good idea. - Jonathan M Davis
May 06 2018
prev sibling parent kinke <noone nowhere.com> writes:
On Saturday, 5 May 2018 at 15:22:04 UTC, Bolpat wrote:
 I once proposed that `in` can mean `const scope ref` that also 
 binds rvalues.
 https://github.com/dlang/DIPs/pull/111#issuecomment-381911140
 We could make `in` be something similar to `inline`. The 
 compiler can implement it as stated above (assign the 
 expression to temporary, reference it), or use copy if copy is 
 cheaper than referencing.
I remember, and I still like that proposal a lot, as it'd allow the compiler to tune generic code to the targeted platform and its ABI and free the dev from having to worry about how to pass a read-only input argument in the most efficient way. So if `in` semantics are ever to be redefined, `const [scope ref]` (depending on type and target ABI) are the only ones I'd happily agree with. [And I'd be extremely happy if rvalues could finally bind to ref params, not just as prerequisite for this.]
May 05 2018
prev sibling parent Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
Personally I would have expected the compiler to be free to 
choose what was needed. I'm sure that has complications with 
separate compilation.
May 04 2018