www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Creating InputRanges from strings, files etc.

reply Vinay Sajip <vinay_sajip yahoo.co.uk> writes:
Excuse my ignorance, but from looking at the documentation on 
std.range and a quick skim of the guides mentioned there near the 
top, I can't see what the simple way is of creating an 
InputRange!(ubyte) from strings, files etc. I would have expected 
to find something in the DLang Tour about this, but couldn't find 
anything. Please can someone tell me how to do it? Just to be 
clear, I want to create an object which is an InputRange!(ubyte) 
and pass that around, rather than e.g. iterate over a string or a 
file.
Nov 08 2018
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 09/11/2018 2:58 AM, Vinay Sajip wrote:

 Excuse my ignorance, but from looking at the documentation on std.range 
 and a quick skim of the guides mentioned there near the top, I can't see 
 what the simple way is of creating an InputRange!(ubyte) from strings, 
 files etc. I would have expected to find something in the DLang Tour 
 about this, but couldn't find anything. Please can someone tell me how 
 to do it? Just to be clear, I want to create an object which is an 
 InputRange!(ubyte) and pass that around, rather than e.g. iterate over a 
 string or a file.


TLDR of how to write an input range:

struct MyInputRange {
	ubyte[] input;

	 property {
		ubyte front() {
			return this.input[0];
		}

		bool empty() {
			return this.input.length == 0;
		}
	}

	void popFront() {
		this.input = this.input[1 .. $];
	}
}

import std.stdio;

void main() {
     foreach(b; MyInputRange([1, 2, 3])) {
     	writeln(b);
     }
}
Nov 08 2018
prev sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Thursday, 8 November 2018 at 13:58:55 UTC, Vinay Sajip wrote:

 Excuse my ignorance, but from looking at the documentation on 
 std.range and a quick skim of the guides mentioned there near 
 the top, I can't see what the simple way is of creating an 
 InputRange!(ubyte) from strings, files etc. I would have 
 expected to find something in the DLang Tour about this, but 
 couldn't find anything. Please can someone tell me how to do 
 it? Just to be clear, I want to create an object which is an 
 InputRange!(ubyte) and pass that around, rather than e.g. 
 iterate over a string or a file.


You can iterate through a file one ubyte at a time using 
`byChunk` and `joiner`:

     auto r1 = stdin.byChunk(1024).joiner;
     assert(is(typeof(r1.front) == ubyte));

You can iterate through a string one ubyte at a time using 
`representation`:

     auto r2 = "To be or not to be".representation;
     assert(is(typeof(r2.front) == immutable(ubyte)));

To pass these ranges around using the `InputRange` interface, use 
`inputRangeObject` to wrap them:

     InputRange!ubyte r3 = inputRangeObject(r1);
     InputRange!(immutable(ubyte)) r4 = inputRangeObject(r2);
Nov 08 2018
next sibling parent Vinay Sajip <vinay_sajip yahoo.co.uk> writes:
On Thursday, 8 November 2018 at 14:38:37 UTC, Paul Backus wrote:

 You can iterate through a file one ubyte at a time using 
 `byChunk` and `joiner`:

     auto r1 = stdin.byChunk(1024).joiner;
     assert(is(typeof(r1.front) == ubyte));

 You can iterate through a string one ubyte at a time using 
 `representation`:

     auto r2 = "To be or not to be".representation;
     assert(is(typeof(r2.front) == immutable(ubyte)));

 To pass these ranges around using the `InputRange` interface, 
 use `inputRangeObject` to wrap them:

     InputRange!ubyte r3 = inputRangeObject(r1);
     InputRange!(immutable(ubyte)) r4 = inputRangeObject(r2);


Aha - inputRangeObject was the thing I was missing. Thanks!
Nov 08 2018
prev sibling parent reply Vinay Sajip <vinay_sajip yahoo.co.uk> writes:
On Thursday, 8 November 2018 at 14:38:37 UTC, Paul Backus wrote:

 To pass these ranges around using the `InputRange` interface, 
 use `inputRangeObject` to wrap them:

     InputRange!ubyte r3 = inputRangeObject(r1);
     InputRange!(immutable(ubyte)) r4 = inputRangeObject(r2);


I did a bit more digging, and it seems to work for strings but 
not for files: The program

import std.algorithm.iteration;
import std.format;
import std.range;
import std.stdio;
import std.string;

void somefn(InputRange!(immutable(ubyte)) r) {
     writeln(format!"%s"(r));
}

void main()
{
     auto a = "Hello, world!";
     auto b = inputRangeObject(a.representation);
     somefn(b);
     auto c = stdin.byChunk(1024).joiner;
     auto d = inputRangeObject(c);
     //somefn(d);
}

compiles as given above, but if the somefn(d) line is 
uncommented, I get an error:

function onlineapp.somefn(InputRange!(immutable(ubyte)) r) is not 
callable using argument types (InputRangeObject!(Result))
onlineapp.d(18):        cannot pass argument d of type 
std.range.interfaces.InputRangeObject!(Result) to parameter 
InputRange!(immutable(ubyte)) r

Do I need to do an explicit cast? If so, can someone tell me the 
precise incantation? How come it doesn't figure out that the 
underlying range is a ubyte range, or is it to do with 
immutability, or something else altogether?
Nov 08 2018
next sibling parent Alex <sascha.orlov gmail.com> writes:
On Thursday, 8 November 2018 at 16:15:25 UTC, Vinay Sajip wrote:

 On Thursday, 8 November 2018 at 14:38:37 UTC, Paul Backus wrote:

 To pass these ranges around using the `InputRange` interface, 
 use `inputRangeObject` to wrap them:

     InputRange!ubyte r3 = inputRangeObject(r1);
     InputRange!(immutable(ubyte)) r4 = inputRangeObject(r2);


 I did a bit more digging, and it seems to work for strings but 
 not for files: The program

 import std.algorithm.iteration;
 import std.format;
 import std.range;
 import std.stdio;
 import std.string;

 void somefn(InputRange!(immutable(ubyte)) r) {
     writeln(format!"%s"(r));
 }

 void main()
 {
     auto a = "Hello, world!";
     auto b = inputRangeObject(a.representation);
     somefn(b);
     auto c = stdin.byChunk(1024).joiner;
     auto d = inputRangeObject(c);
     //somefn(d);
 }

 compiles as given above, but if the somefn(d) line is 
 uncommented, I get an error:

 function onlineapp.somefn(InputRange!(immutable(ubyte)) r) is 
 not callable using argument types (InputRangeObject!(Result))
 onlineapp.d(18):        cannot pass argument d of type 
 std.range.interfaces.InputRangeObject!(Result) to parameter 
 InputRange!(immutable(ubyte)) r

 Do I need to do an explicit cast? If so, can someone tell me 
 the precise incantation? How come it doesn't figure out that 
 the underlying range is a ubyte range, or is it to do with 
 immutability, or something else altogether?


you could use a template for somefn definition:

´´´
void somefn(T)(T r) {
     writeln(format!"%s"(r));
}
´´´
Nov 08 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/8/18 11:15 AM, Vinay Sajip wrote:

 On Thursday, 8 November 2018 at 14:38:37 UTC, Paul Backus wrote:

 To pass these ranges around using the `InputRange` interface, use 
 `inputRangeObject` to wrap them:

     InputRange!ubyte r3 = inputRangeObject(r1);
     InputRange!(immutable(ubyte)) r4 = inputRangeObject(r2);


 
 I did a bit more digging, and it seems to work for strings but not for 
 files: The program
 
 import std.algorithm.iteration;
 import std.format;
 import std.range;
 import std.stdio;
 import std.string;
 
 void somefn(InputRange!(immutable(ubyte)) r) {
      writeln(format!"%s"(r));
 }
 
 void main()
 {
      auto a = "Hello, world!";
      auto b = inputRangeObject(a.representation);
      somefn(b);
      auto c = stdin.byChunk(1024).joiner;
      auto d = inputRangeObject(c);
      //somefn(d);
 }
 
 compiles as given above, but if the somefn(d) line is uncommented, I get 
 an error:
 
 function onlineapp.somefn(InputRange!(immutable(ubyte)) r) is not 
 callable using argument types (InputRangeObject!(Result))
 onlineapp.d(18):        cannot pass argument d of type 
 std.range.interfaces.InputRangeObject!(Result) to parameter 
 InputRange!(immutable(ubyte)) r
 
 Do I need to do an explicit cast? If so, can someone tell me the precise 
 incantation? How come it doesn't figure out that the underlying range is 
 a ubyte range, or is it to do with immutability, or something else 
 altogether?


A cool feature of D is to have it tell you something about your code at 
compile time.

I did this in a run.dlang.org playground:

pragma(msg, ElementType!(typeof(b)));
pragma(msg, ElementType!(typeof(d)));

I get:
immutable(ubyte)
ubyte

Which means they aren't the same type, and they don't define the same 
interface (InputRange!(ubyte) is not the same as 
InputRange!(immutable(ubyte)) ).

Other than simply using compile-time functions, and dropping the object 
interface as Alex suggests, the easiest thing I can recommend is 
wrapping representation into a casting input range such as map:

auto b = inputRangeObject(a.representation.map!(b => ubyte(b)));

You can see all this here:

https://run.dlang.io/is/1E6Uqj

-Steve
Nov 08 2018
parent Vinay Sajip <vinay_sajip yahoo.co.uk> writes:
On Thursday, 8 November 2018 at 16:41:50 UTC, Steven 
Schveighoffer wrote:

 I did this in a run.dlang.org playground:

 pragma(msg, ElementType!(typeof(b)));
 pragma(msg, ElementType!(typeof(d)));

 I get:
 immutable(ubyte)
 ubyte

 Which means they aren't the same type, and they don't define 
 the same interface (InputRange!(ubyte) is not the same as 
 InputRange!(immutable(ubyte)) ).

 Other than simply using compile-time functions, and dropping 
 the object interface as Alex suggests, the easiest thing I can 
 recommend is wrapping representation into a casting input range 
 such as map:

 auto b = inputRangeObject(a.representation.map!(b => ubyte(b)));

 You can see all this here:

 https://run.dlang.io/is/1E6Uqj

 -Steve


Thanks, guys, those are helpful pointers.
Nov 08 2018