www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Ranges seem awkward to work with

reply Hasen Judy <hasen.judy gmail.com> writes:
Is this is a common beginner issue? I remember using an earlier 
version of D some long time ago and I don't remember seeing this 
concept.

Now, a lot of library functions seem to expect ranges as inputs 
and return ranges as output.

Even parsing a csv line returns a range. And the funny thing is, 
once you loop over it, it's done. You've basically consumed it.

For example, I was having some trouble with the api of the 
std.csv module, so to help me debug, I printed the result of the 
csv. ok, the result seems good. Now I try to use it, for example:

     auto name = row[1];

And to my surprise there's a runtime error, something about range 
something something. I don't even remember what the error was. 
The thing is, it wasn't clear what was going on.

The line was actually more like:

         auto some_var = 
some_function(row[1].some_other_library_method!template_variable);

So because I was calling several library methods on the same 
line, I thought the problem might have something to do with the 
range not exactly matching what the library was expecting. I 
thought maybe row[1] also returned some range instead of a string 
and that range had something wrong with it.

Well, it turned out that my earlier attempt to print the parsed 
csv row resulted in the row being "consumed" and now the row is 
an empty range(!).

Is there a straight forward way to convert a Range to a list 
other than manually doing a foreach?

     string[] items;
     foreach(item; someRangeThing) {
         items ~= item;
     }

I feel like that is a bit of an overkill.
Sep 11 2017
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:

 Is this is a common beginner issue? I remember using an earlier 
 version of D some long time ago and I don't remember seeing 
 this concept.

 Now, a lot of library functions seem to expect ranges as inputs 
 and return ranges as output.

 Even parsing a csv line returns a range. And the funny thing 
 is, once you loop over it, it's done. You've basically consumed 
 it.

 For example, I was having some trouble with the api of the 
 std.csv module, so to help me debug, I printed the result of 
 the csv. ok, the result seems good. Now I try to use it, for 
 example:

     auto name = row[1];

 And to my surprise there's a runtime error, something about 
 range something something. I don't even remember what the error 
 was. The thing is, it wasn't clear what was going on.

 The line was actually more like:

         auto some_var = 
 some_function(row[1].some_other_library_method!template_variable);

 So because I was calling several library methods on the same 
 line, I thought the problem might have something to do with the 
 range not exactly matching what the library was expecting. I 
 thought maybe row[1] also returned some range instead of a 
 string and that range had something wrong with it.

 Well, it turned out that my earlier attempt to print the parsed 
 csv row resulted in the row being "consumed" and now the row is 
 an empty range(!).

 Is there a straight forward way to convert a Range to a list 
 other than manually doing a foreach?

     string[] items;
     foreach(item; someRangeThing) {
         items ~= item;
     }

 I feel like that is a bit of an overkill.


if `range.save` works use that, otherwise

`range.dup` will duplicate the range

or `range.array` (from std.array) will enumerate the range into 
an array (which is pretty much the same as your solution above.
Sep 11 2017
parent Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
On Tuesday, 12 September 2017 at 01:18:21 UTC, Nicholas Wilson 
wrote:

 On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:

 Is this is a common beginner issue?


 if `range.save` works use that, otherwise


std.csv does not, IIRC.


 `range.dup` will duplicate the range


That isn't a range property.



 or `range.array` (from std.array) will enumerate the range into 
 an array (which is pretty much the same as your solution above.


This. Ranges are great in part because they are often lazy, a 
sequence of operations can be defined but only executed when 
used. Some operations need the whole collection, eg sort.
Sep 11 2017
prev sibling next sibling parent Moritz Maxeiner <moritz ucworks.org> writes:
On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:

 Is this is a common beginner issue? I remember using an earlier 
 version of D some long time ago and I don't remember seeing 
 this concept.


D's ranges can take getting used to, so if you haven't already, 
these two articles are worth the read to get familiar with them 
imho [1][2].
One way to look at it is that input ranges (empty,front,popFront) 
model iteration of the elements of some data source (another is 
that they model a monotonic advancing data source).

[1] 
http://www.drdobbs.com/architecture-and-design/component-programming-in-d/240008321
[2] https://wiki.dlang.org/Component_programming_with_ranges
Sep 11 2017
prev sibling next sibling parent Mike Parker <aldacron gmail.com> writes:
On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:

 Is this is a common beginner issue? I remember using an earlier 
 version of D some long time ago and I don't remember seeing 
 this concept.

 Now, a lot of library functions seem to expect ranges as inputs 
 and return ranges as output.

 Even parsing a csv line returns a range. And the funny thing 
 is, once you loop over it, it's done. You've basically consumed 
 it.

 For example, I was having some trouble with the api of the 
 std.csv module, so to help me debug, I printed the result of 
 the csv. ok, the result seems good. Now I try to use it, for 
 example:

     auto name = row[1];

 And to my surprise there's a runtime error, something about 
 range something something. I don't even remember what the error 
 was. The thing is, it wasn't clear what was going on.

 The line was actually more like:

         auto some_var = 
 some_function(row[1].some_other_library_method!template_variable);

 So because I was calling several library methods on the same 
 line, I thought the problem might have something to do with the 
 range not exactly matching what the library was expecting. I 
 thought maybe row[1] also returned some range instead of a 
 string and that range had something wrong with it.

 Well, it turned out that my earlier attempt to print the parsed 
 csv row resulted in the row being "consumed" and now the row is 
 an empty range(!).

 Is there a straight forward way to convert a Range to a list 
 other than manually doing a foreach?

     string[] items;
     foreach(item; someRangeThing) {
         items ~= item;
     }

 I feel like that is a bit of an overkill.


Don't think of ranges as persistent containers. They aren't. They 
are transient entities, temporary views of data. If you have a 
container and want a consumable view of it that doesn't mutate 
the container itself, produce a range. If you receive a range and 
want to transform it from a transient entity into something 
concrete, use std.array.array or a range-based container API. If 
it helps, Chapter 6: Understanding Ranges from 'Learning D' is 
available (in somewhat mutilated form) as an article at the 
publisher's site:

https://www.packtpub.com/books/content/understanding-ranges
Sep 11 2017
prev sibling parent reply Azi Hassan <azi.hassan live.fr> writes:
On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:

 Now, a lot of library functions seem to expect ranges as inputs 
 and return ranges as output.


Unless I'm mistaken, it was done on purpose to reduce the amount 
of memory allocations in the standard library so that it becomes 
 nogc-friendly. But yes, you can use std.array.array to turn it 
into an array, but keep in mind that it does allocate so you 
might need to watch out if you're dealing with large CSV files.
Sep 12 2017
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, September 12, 2017 13:47:47 Azi Hassan via Digitalmars-d-learn 
wrote:

 On Tuesday, 12 September 2017 at 01:13:29 UTC, Hasen Judy wrote:

 Now, a lot of library functions seem to expect ranges as inputs
 and return ranges as output.


 Unless I'm mistaken, it was done on purpose to reduce the amount
 of memory allocations in the standard library so that it becomes
  nogc-friendly. But yes, you can use std.array.array to turn it
 into an array, but keep in mind that it does allocate so you
 might need to watch out if you're dealing with large CSV files.


Ranges were a big thing in Phobos years before  nogc came along. They're
used because they're an extremely powerful idiom. I don't even recall much
in the way of discussions related to their efficiency vs arrays early on.
That did come later, and ranges have significantly helped the memory
efficiency of D programs regardless of  nogc, but that really wasn't the
motivating factor. And even now, a lot of the basic uses of ranges _do_
allocate, because they use lambdas that end up with closures being allocated
for them (which uses far less memory than allocating a dynamic array but is
not  nogc). That can be avoided by doing stuff like using static nested
functions or functors, but your average rangd-based code doesn't tend to
jump through those hoops.

Ranges do take some getting used to, but we have them precisely because
they're so flexible and powerful.

- Jonathan M Davis
Sep 12 2017