www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Using arrays with functions taking ranges

reply "Mu" <invalid_email address.int> writes:
The code below works as is with memory mapped files' casted 
opSlice's.

However, I can't figure out how to test it with simple arrays.
Its unittest fails.

Question:
How to implement the function correctly?
Otherwise what is the correct way to test it?

Thank you.

Code:
----------------
Range2 caesarCipher(Range1, Range2)(Range1 input, Range2 output, 
int shift)
	if (isInputRange!Range1 && isOutputRange!(Range2, ubyte))
{
	auto rotAb = lowercase.dtext.dup; // rotated alphabet

	shift %= lowercase.length.to!int; // bring the shift within the 
length of the alphabet

	if (shift < 0)
		bringToFront(rotAb[0 .. $ + shift], rotAb[$ + shift .. $]);
	else
		bringToFront(rotAb[0 .. shift], rotAb[shift .. $]);

	foreach (i, ref o; output)
	{
		const char c = input[i];

		if (isAlpha(c))
			if (isUpper(c))
				o = toUpper(rotAb[lowercase.countUntil(toLower(c))]).to!ubyte;
			else
				o = rotAb[lowercase.countUntil(c)].to!ubyte;
		else
			o = c;
	}

	return output;
}

unittest
{
	ubyte[] uba;

	assert(caesarCipher("Exxego ex srgi!", uba, -56).to!string == 
"Attack at once!");
}
Dec 13 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Mu:


 	foreach (i, ref o; output)


But isn't output empty?

Bye,
bearophile
Dec 13 2012
parent reply "Mu" <invalid_email address.int> writes:
 	foreach (i, ref o; output)


 But isn't output empty?


Indeed it is, when output is a new array.

Then how can I approach this to work for both cases:
1) output is empty?
2) output is the size of input? (MmFile's)
Dec 13 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Mu:


 Then how can I approach this to work for both cases:
 1) output is empty?
 2) output is the size of input? (MmFile's)



You have to tell those cases apart with a run time test, so your 
output range should support empty, or even length. If you want to 
overwrite the already allocated space, then I think you need the 
output range to be assignable, or to have items usable by ref. 
Other people should be able to give you a better answer.

Bye,
bearophile
Dec 13 2012
parent reply "Mu" <invalid_email address.int> writes:
Thank you for your suggestion, bearophile.
I ended up checking if the range is empty, and if it was, I'd 
increment it as elements were added.

Honestly, I much dislike this method.
It does not feel right. The C++ version is more consistent 
because of the iterators.

Please, if anyone has a better approach, share it.
http://pastebin.com/MRB5L44M

Thank you.
Dec 14 2012
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 14 December 2012 at 10:20:38 UTC, Mu wrote:

 Thank you for your suggestion, bearophile.
 I ended up checking if the range is empty, and if it was, I'd 
 increment it as elements were added.

 Honestly, I much dislike this method.
 It does not feel right. The C++ version is more consistent 
 because of the iterators.


No it isn't. Iterators, like ranges, have no notion of the 
underlying container. You CANNOT modify the underlying container 
via a range or an iterator. the operation "++output.length" is 
NOT a valid range operation, and there is no equivalent in C++ 
either.

You have to use an output range, and use its "put" primitive. 
Note that currently, the definition of "isOutputRange" is a bit 
flawed. It'll answer "true" on arrays, which, arguably, are not 
output ranges. Ideally, you'd use a "true" "outputRange" or 
"sink", such as [Ref]Appender.

Appender will use your "input" _slice_ as a starting point, but 
will not actually modify your slice.
RefAppender will modify your slice.

Furthermore, note there is a bug in your code:
When you write "to!string", this will transform your 
representation into a string, NOT re-interpret into a string. It 
will LITERALLY generate the string:
"[65, 116, 116, 97, 99, 107, 32, 97, 116, 32, 111, 110, 99, 101, 
33]"

Here is your program, tweaked and with extra tests to show you 
all that together.

http://dpaste.dzfl.pl/b61fe4c5

Hope that helps. Please feel free to ask if there are more doubts.
Dec 14 2012
parent reply "Mu" <invalid_email address.int> writes:
Thank you, monarch_dodra. This looks like what I wanted.

I have a question:
How come the function works with MmFile.opSlice's without 
appender(&)?
And is this reliable behavior?

caesarCipher(cast(ubyte[]) inputFile.opSlice, cast(ubyte[]) 
outputFile.opSlice, 13);

Trying to use appender(&) on the casted opSlice's yields a "is 
not an lvalue" error.
Dec 14 2012
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 14 December 2012 at 13:09:15 UTC, Mu wrote:

 Thank you, monarch_dodra. This looks like what I wanted.

 I have a question:
 How come the function works with MmFile.opSlice's without 
 appender(&)?
 And is this reliable behavior?

 caesarCipher(cast(ubyte[]) inputFile.opSlice, cast(ubyte[]) 
 outputFile.opSlice, 13);


It "works" because in theory, all mutable ranges verify the "is 
output range" trait. However, they are not "sinks", so if you 
write too much into them, you'll get an out of index exception. 
Does it work at runtime, and do you get the correct behavior?

In this case, I don't think appender would work, because it would 
just re-allocate to a GC-allocated slice, and not your MM.

Reading the MM doc, I don't think it offers output range 
interface.

In this specific case, I think you are supposed to reserve the 
correct amount of space beforehand, and copy using input range 
interface. You'll have to make sure there is enough room first 
though. That's basically what you'd do with iterators mind you.


 Trying to use appender(&) on the casted opSlice's yields a "is 
 not an lvalue" error.


Yes, that's because when you are casting to ubyte[], you are 
creating a new *slice*. It refers to the same data as your 
"outputFile.opSlice" object, but the slice itself is a new 
object, so you cannot extract its address.

You might as well just use Appender instead of RefAppender: they 
both do exactly the same thing. The difference is that 
RefAppender will always re-assign your slice to the current 
buffer, so you can "see" the updates as it goes.
Dec 14 2012
parent reply "Mu" <invalid_email address.int> writes:
 It "works" because in theory, all mutable ranges verify the "is 
 output range" trait. However, they are not "sinks", so if you 
 write too much into them, you'll get an out of index exception. 
 Does it work at runtime, and do you get the correct behavior?


 From what I tested, yes it works correctly, but I don't 
understand why.
If put() is used, and the opSlice has a length different from 
zero, how come the data is filled in starting at opSlice's first 
element?

If my questions are becoming trivial, please point me to the 
relevant documentation.
Thank you.
Dec 14 2012
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 14 December 2012 at 14:56:45 UTC, Mu wrote:

 It "works" because in theory, all mutable ranges verify the 
 "is output range" trait. However, they are not "sinks", so if 
 you write too much into them, you'll get an out of index 
 exception. Does it work at runtime, and do you get the correct 
 behavior?


 From what I tested, yes it works correctly, but I don't 
 understand why.
 If put() is used, and the opSlice has a length different from 
 zero, how come the data is filled in starting at opSlice's 
 first element?

 If my questions are becoming trivial, please point me to the 
 relevant documentation.
 Thank you.


No, the question isn't trivial at all.

It works because "put" is defined for all input ranges as "write 
to first element and popFront".

Basically, in C++ terms, it's the same as "*(it++) = value".

The "problem" in this case is that you have to make sure *before 
hand*, that there is enough room to do this. If you were to 
"accidently" stuff into your input range more than it can take, 
you'll error out. EG. the same as going past it_end.

I don't have the context to your program, so I can't give you a 
perfect answer. It sounds like you are doing the right thing.

--------
Long story short:
*Input range: It is pre-allocated, and you can only put stuff up 
to its capacity.
*(appender-style) Output range: Has only "put", and grows as you 
stuff it.
Dec 14 2012
parent reply "Mu" <invalid_email address.int> writes:
 It works because "put" is defined for all input ranges as 
 "write to first element and popFront".


That makes more sense, thanks.
So what happens is that for "regular" ranges std.range.put() gets 
used, while for arrays in RefAppender std.array.put() gets used, 
right?

(I noticed that the unittest fails if the destination char[] 
isn't empty.)


 The "problem" in this case is that you have to make sure 
 *before hand*, that there is enough room to do this. If you 
 were to "accidently" stuff into your input range more than it 
 can take, you'll error out. EG. the same as going past it_end.


I think it's OK. The output MmFile is constructed to have the 
size of the input MmFile. So their opSlice's will have the same 
length.
Dec 14 2012
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 14 December 2012 at 15:59:48 UTC, Mu wrote:

 It works because "put" is defined for all input ranges as 
 "write to first element and popFront".


 That makes more sense, thanks.
 So what happens is that for "regular" ranges std.range.put() 
 gets used, while for arrays in RefAppender std.array.put() gets 
 used, right?


Nope, you call the member function. std.array.RefAppender.put.

Link to how "put" resolves. 
http://dlang.org/phobos/std_range.html#put
Dec 14 2012