www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Consume an entire range

reply "Brad Anderson" <eco gnuk.net> writes:
Is there a simple way to consume a range apart from 
std.array.array?  I don't need to result of the range stored in 
an array, I just need a lazy map to evaluate completely.
May 29 2013
parent reply "Brad Anderson" <eco gnuk.net> writes:
On Thursday, 30 May 2013 at 03:50:52 UTC, Brad Anderson wrote:

 Is there a simple way to consume a range apart from 
 std.array.array?  I don't need to result of the range stored in 
 an array, I just need a lazy map to evaluate completely.


Obviously I could just popFront.  To be more clear, I want 
something like eat() or consume() or exhaust() that I can tack on 
the end of my chained algorithm calls.
May 29 2013
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, May 30, 2013 05:53:02 Brad Anderson wrote:

 On Thursday, 30 May 2013 at 03:50:52 UTC, Brad Anderson wrote:

 Is there a simple way to consume a range apart from
 std.array.array?  I don't need to result of the range stored in
 an array, I just need a lazy map to evaluate completely.


 
 Obviously I could just popFront.  To be more clear, I want
 something like eat() or consume() or exhaust() that I can tack on
 the end of my chained algorithm calls.


If you specifically want to iterate over it, then I think that you need to 
repeatedly call popFront (or callPopFrontN if it hasLength). However, if what 
you want is for the resultant range to be empty, you can use 
std.range.takeNone. If it can, it'll return the same range type as the 
original, and if it can't, it'll return takeExactly(range, 0).

- Jonathan M Davis
May 29 2013
parent reply "Brad Anderson" <eco gnuk.net> writes:
On Thursday, 30 May 2013 at 04:00:39 UTC, Jonathan M Davis wrote:

 On Thursday, May 30, 2013 05:53:02 Brad Anderson wrote:

 On Thursday, 30 May 2013 at 03:50:52 UTC, Brad Anderson wrote:

 Is there a simple way to consume a range apart from
 std.array.array?  I don't need to result of the range stored 
 in
 an array, I just need a lazy map to evaluate completely.


 
 Obviously I could just popFront.  To be more clear, I want
 something like eat() or consume() or exhaust() that I can tack 
 on
 the end of my chained algorithm calls.


 If you specifically want to iterate over it, then I think that 
 you need to
 repeatedly call popFront (or callPopFrontN if it hasLength). 
 However, if what
 you want is for the resultant range to be empty, you can use
 std.range.takeNone. If it can, it'll return the same range type 
 as the
 original, and if it can't, it'll return takeExactly(range, 0).

 - Jonathan M Davis


There was a filter in the change so I had no length.  The tailing 
map had a void element type which mean foreach didn't work on it.

I ended up with:

void eat(R)(R r) { while(!r.empty) { r.front; r.popFront(); } }

I was actually just playing around reimplementing Andrei's 
example from his InformIT article[1] using std.algorithm and 
friends.  It went from:

import std.stdio, std.string;

void main() {
    uint[string] dic;
    foreach (line; stdin.byLine) {
       // Break sentence into words
       string[] words = split(strip(line));
       // Add each word in the sentence to the vocabulary
       foreach (word; words) {
          if (word in dic) continue; // nothing to do
          uint newID = dic.length;
          dic[word] = newID;
          writeln(newID, '\t', word);
       }
    }
}

to:

import std.stdio, std.algorithm, std.array;

void eat(R)(R r) { while(!r.empty) { r.front; r.popFront; } }

void main() {
    size_t[dstring] dic;
    stdin.byLine
	   .joiner(" ")
	   .array
	   .splitter(' ')
	   .filter!(w => !w.empty && w !in dic)
	   .map!(w => writeln(dic[w.idup] = dic.length, '\t', w))
	   .eat;
}

I would have prefered to not use joiner() but working with ranges 
of ranges of ranges (splitter() on each line) got a bit weird and 
confusing.

splitter() needed array slicing and length which joiner() doesn't 
have so I had to use array(). I was confused about why I was 
suddenly getting dchar[] out the other end but found the 
StackOverflow question [2] in which you explain why joiner does 
that (you really kick ass at answering StackOverflow questions, 
Jonathan).  Having a variant of splitter that ignores empty 
tokens would be nice to have too.

1. 
http://www.informit.com/articles/article.aspx?p=1381876&seqNum=4
2. 
http://stackoverflow.com/questions/12288465/std-algorithm-joinerstring-string-why-result-elements-are-dchar-and-not-ch
May 29 2013
next sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Thursday, 30 May 2013 at 04:30:11 UTC, Brad Anderson wrote:

 [snip]
 import std.stdio, std.algorithm, std.array;

 void eat(R)(R r) { while(!r.empty) { r.front; r.popFront; } }

 void main() {
    size_t[dstring] dic;
    stdin.byLine
 	   .joiner(" ")
 	   .array
 	   .splitter(' ')
 	   .filter!(w => !w.empty && w !in dic)
 	   .map!(w => writeln(dic[w.idup] = dic.length, '\t', w))
 	   .eat;
 }

 I would have prefered to not use joiner() but working with 
 ranges of ranges of ranges (splitter() on each line) got a bit 
 weird and confusing.


Ok, I guess it's not that weird after all:

void main() {
    size_t[string] dic;
    stdin.byLine
	   .map!(l => l.splitter(' ')
	   			   .filter!(w => !w.empty && w !in dic)
	   			   .map!(w => writeln(dic[w.idup] = dic.length, '\t', 
w)).eat).eat;
}
May 29 2013
prev sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Brad Anderson:


 import std.stdio, std.algorithm, std.array;

 void eat(R)(R r) { while(!r.empty) { r.front; r.popFront; } }

 void main() {
    size_t[dstring] dic;
    stdin.byLine
 	   .joiner(" ")
 	   .array
 	   .splitter(' ')
 	   .filter!(w => !w.empty && w !in dic)
 	   .map!(w => writeln(dic[w.idup] = dic.length, '\t', w))
 	   .eat;
 }

 I would have prefered to not use joiner() but working with 
 ranges of ranges of ranges (splitter() on each line) got a bit 
 weird and confusing.


Maybe here it's better to work on lines. Alternatively I don't 
know if you can read the whole input there.

It's usually better to give only pure functions to filter/map, 
because in Bugzilla I've shown those higher order functions don't 
work well otherwise.

So I prefer a terminal function that takes an impure function and 
returns nothing, something like:

...
.filter!(w => !w.empty && w !in dic)
.forEach!((w) { writeln(dic[w.idup] = dic.length, '\t', w); });


Bye,
bearophile
May 30 2013
parent reply "Brad Anderson" <eco gnuk.net> writes:
On Thursday, 30 May 2013 at 11:38:58 UTC, bearophile wrote:

 Brad Anderson:


 import std.stdio, std.algorithm, std.array;

 void eat(R)(R r) { while(!r.empty) { r.front; r.popFront; } }

 void main() {
   size_t[dstring] dic;
   stdin.byLine
 	   .joiner(" ")
 	   .array
 	   .splitter(' ')
 	   .filter!(w => !w.empty && w !in dic)
 	   .map!(w => writeln(dic[w.idup] = dic.length, '\t', w))
 	   .eat;
 }

 I would have prefered to not use joiner() but working with 
 ranges of ranges of ranges (splitter() on each line) got a bit 
 weird and confusing.


 Maybe here it's better to work on lines. Alternatively I don't 
 know if you can read the whole input there.


I posted a version that worked on lines instead.  I was having 
trouble but the trouble was actually the same problem I was 
having with the joined version (my map wasn't being consumed).


 It's usually better to give only pure functions to filter/map, 
 because in Bugzilla I've shown those higher order functions 
 don't work well otherwise.


Have any links?  I considered using sort and uniq to avoid the 
closure around dic but then the order would be different and I 
wanted to keep it using a similar technique to the original.


 So I prefer a terminal function that takes an impure function 
 and returns nothing, something like:

 ...
 .filter!(w => !w.empty && w !in dic)
 .forEach!((w) { writeln(dic[w.idup] = dic.length, '\t', w); });


Is forEach real?  I sought it out because it'd be a better fit 
but came up empty (it's not in std.range or std.algorithm).


 Bye,
 bearophile
May 30 2013
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, May 30, 2013 19:58:45 Brad Anderson wrote:

 Is forEach real? I sought it out because it'd be a better fit
 but came up empty (it's not in std.range or std.algorithm).


No. The normal thing would be to just use an actual foreach loop.

- Jonathan M Davis
May 30 2013
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Brad Anderson:


 Have any links?


http://d.puremagic.com/issues/show_bug.cgi?id=9674



 Is forEach real?


It's just an idea.

Bye,
bearophile
May 30 2013
prev sibling parent reply "Diggory" <diggsey googlemail.com> writes:
On Thursday, 30 May 2013 at 03:53:06 UTC, Brad Anderson wrote:

 On Thursday, 30 May 2013 at 03:50:52 UTC, Brad Anderson wrote:

 Is there a simple way to consume a range apart from 
 std.array.array?  I don't need to result of the range stored 
 in an array, I just need a lazy map to evaluate completely.


 Obviously I could just popFront.  To be more clear, I want 
 something like eat() or consume() or exhaust() that I can tack 
 on the end of my chained algorithm calls.


There's "walkLength"?
May 29 2013
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, May 30, 2013 06:12:51 Diggory wrote:

 On Thursday, 30 May 2013 at 03:53:06 UTC, Brad Anderson wrote:

 On Thursday, 30 May 2013 at 03:50:52 UTC, Brad Anderson wrote:

 Is there a simple way to consume a range apart from
 std.array.array?  I don't need to result of the range stored
 in an array, I just need a lazy map to evaluate completely.


 
 Obviously I could just popFront.  To be more clear, I want
 something like eat() or consume() or exhaust() that I can tack
 on the end of my chained algorithm calls.


 
 There's "walkLength"?


Ah, I should have thought of that, though that won't work if the range defines 
length. Of course, if it defines length, you can always use popFrontN with 
length.

- Jonathan M Davis
May 29 2013
parent "Diggory" <diggsey googlemail.com> writes:
On Thursday, 30 May 2013 at 04:18:14 UTC, Jonathan M Davis wrote:

 On Thursday, May 30, 2013 06:12:51 Diggory wrote:

 On Thursday, 30 May 2013 at 03:53:06 UTC, Brad Anderson wrote:

 On Thursday, 30 May 2013 at 03:50:52 UTC, Brad Anderson 
 wrote:

 Is there a simple way to consume a range apart from
 std.array.array?  I don't need to result of the range stored
 in an array, I just need a lazy map to evaluate completely.


 
 Obviously I could just popFront.  To be more clear, I want
 something like eat() or consume() or exhaust() that I can 
 tack
 on the end of my chained algorithm calls.


 
 There's "walkLength"?


 Ah, I should have thought of that, though that won't work if 
 the range defines
 length. Of course, if it defines length, you can always use 
 popFrontN with
 length.

 - Jonathan M Davis


Do you think it would be worth adding a "consume" method to 
std.range or std.algorithm?
May 29 2013
prev sibling parent "Brad Anderson" <eco gnuk.net> writes:
On Thursday, 30 May 2013 at 04:12:56 UTC, Diggory wrote:

 On Thursday, 30 May 2013 at 03:53:06 UTC, Brad Anderson wrote:

 On Thursday, 30 May 2013 at 03:50:52 UTC, Brad Anderson wrote:

 Is there a simple way to consume a range apart from 
 std.array.array?  I don't need to result of the range stored 
 in an array, I just need a lazy map to evaluate completely.


 Obviously I could just popFront.  To be more clear, I want 
 something like eat() or consume() or exhaust() that I can tack 
 on the end of my chained algorithm calls.


 There's "walkLength"?


I tried that. There is an isInputRange && !isInfinite template 
constraint but I don't see how my stuff didn't meet that 
requirement.
May 29 2013