www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Must ranges have value semantics?

reply Chris Wright <dhasenan gmail.com> writes:
I noticed that some methods in Phobos will have very different behavior 
with forward ranges that have reference semantics and those that have 
value semantics.

Example:

auto range = getSomeRange();
auto count = range.walkLength;
foreach (element; range) { writeln(element); }

If getSomeRange returns a forward range that is a reference type with 
no .length property, walkLength will exhaust the range. The iteration 
after that will never enter the loop body.

However, if getSomeRange returns a range with value semantics with 
no .length property, then iteration is still possible.

I haven't found documentation about how ranges are intended to be used in 
D written by people who maintain Phobos. Is it normal and expected that I 
should have to call .save everywhere, manually? Was there discussion on 
whether this should be the case or a published document containing the 
reasoning behind the decision?
Dec 15 2015
next sibling parent "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Tue, Dec 15, 2015 at 05:36:52PM +0000, Chris Wright via Digitalmars-d-learn
wrote:

 I noticed that some methods in Phobos will have very different
 behavior with forward ranges that have reference semantics and those
 that have value semantics.
 
 Example:
 
 auto range = getSomeRange();
 auto count = range.walkLength;
 foreach (element; range) { writeln(element); }
 
 If getSomeRange returns a forward range that is a reference type with
 no .length property, walkLength will exhaust the range. The iteration
 after that will never enter the loop body.
 
 However, if getSomeRange returns a range with value semantics with no
 .length property, then iteration is still possible.
 
 I haven't found documentation about how ranges are intended to be used
 in D written by people who maintain Phobos. Is it normal and expected
 that I should have to call .save everywhere, manually? Was there
 discussion on whether this should be the case or a published document
 containing the reasoning behind the decision?


This is one of the "undefined" or poorly-defined areas of D ranges. :-(
Basically, in generic code, you should always use .save where you expect
to reuse the range after iterating over it.  In non-generic code, of
course, you already know what the range semantics are and you don't need
to use .save, but in generic code, where the incoming range type may or
may not have reference semantics, always call .save to be on the safe
side. Value-type ranges simply return `this` in their .save method
anyway, so it doesn't hurt.

(There have been (and possibly still are) places in Phobos where .save
wasn't used, leading to subtle bugs that only manifest themselves when
user code starts passing reference-semantics ranges around. Phobos
unittests in general, unfortunately, tend to only test value-type
ranges, so this problem often gets missed. There have been attempts to
remedy this, but AFAIK there are still many unittests out there that
don't adequately verify their target functions with reference-type
ranges. Or, on a related note, anything other than arrays, which often
leads to hiding places for bugs that only show up with non-array
ranges.)

There was a discussion recently that .save may have been a
miscalculation in the design of ranges, and that forward ranges should
have done this in postblit instead of a separate method. (One should
understand, however, that at the time ranges were first defined, we
didn't have postblit semantics the way we have them today, so at that
time a .save method was possibly the best compromise given the state of
the language then.)


T

-- 
Never ascribe to malice that which is adequately explained by incompetence. --
Napoleon Bonaparte
Dec 15 2015
prev sibling parent Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Tuesday, 15 December 2015 at 17:36:52 UTC, Chris Wright wrote:

 I noticed that some methods in Phobos will have very different 
 behavior with forward ranges that have reference semantics and 
 those that have value semantics.

 Example:

 auto range = getSomeRange();
 auto count = range.walkLength;
 foreach (element; range) { writeln(element); }

 If getSomeRange returns a forward range that is a reference 
 type with no .length property, walkLength will exhaust the 
 range. The iteration after that will never enter the loop body.

 However, if getSomeRange returns a range with value semantics 
 with no .length property, then iteration is still possible.

 I haven't found documentation about how ranges are intended to 
 be used in D written by people who maintain Phobos. Is it 
 normal and expected that I should have to call .save 
 everywhere, manually? Was there discussion on whether this 
 should be the case or a published document containing the 
 reasoning behind the decision?


Unfortunately you must make no assumption about the ranges state 
in generic code after calling a function that takes the range by 
value. If you want to perform other operations after the call to 
walklength, you'll need to have a new range created with save() 
before that call.

The value vs reference rage creates challenges, but in practice 
it will be something you occasionally hit.

I kind of feel like a range should always be taken by reference 
so that it is predictable that the range will be modified if it 
is value or reference semantics.
Dec 15 2015