www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Idiomatic way to process const/immutable arrays as ranges

reply "Dicebot" <m.strashun gmail.com> writes:
This question is so stupid I am just struck with trying to chose 
the "good" solution.

Consider this code:

string func(in string[] args)
{
     return args.join(" "); // compile error, args is not an input 
range
}

It is somewhat expected as you can hardly popFront on a const 
range. But then question is: how can I wrap const slice into a 
mutable range without copying data (using standard 
language/library solution)? Of course, I can do .dup.join, but 
there is no real need to do copy, even a shallow one.
Feb 11 2013
next sibling parent reply "Namespace" <rswhite4 googlemail.com> writes:
On Monday, 11 February 2013 at 14:46:55 UTC, Dicebot wrote:

 This question is so stupid I am just struck with trying to 
 chose the "good" solution.

 Consider this code:

 string func(in string[] args)
 {
     return args.join(" "); // compile error, args is not an 
 input range
 }

 It is somewhat expected as you can hardly popFront on a const 
 range. But then question is: how can I wrap const slice into a 
 mutable range without copying data (using standard 
 language/library solution)? Of course, I can do .dup.join, but 
 there is no real need to do copy, even a shallow one.


Try:
string func(string[] args)
without 'in'.
Feb 11 2013
parent "Dicebot" <m.strashun gmail.com> writes:
On Monday, 11 February 2013 at 14:53:17 UTC, Namespace wrote:

 On Monday, 11 February 2013 at 14:46:55 UTC, Dicebot wrote:

 This question is so stupid I am just struck with trying to 
 chose the "good" solution.

 Consider this code:

 string func(in string[] args)
 {
    return args.join(" "); // compile error, args is not an 
 input range
 }

 It is somewhat expected as you can hardly popFront on a const 
 range. But then question is: how can I wrap const slice into a 
 mutable range without copying data (using standard 
 language/library solution)? Of course, I can do .dup.join, but 
 there is no real need to do copy, even a shallow one.


 Try:
 string func(string[] args)
 without 'in'.


I have "in" there for a reason. There is no rationale in removing 
it. My request is perfectly valid - creating a mutable range for 
immutable array traversal.
Feb 11 2013
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 11 Feb 2013 09:46:54 -0500, Dicebot <m.strashun gmail.com> wrote:


 This question is so stupid I am just struck with trying to chose the  
 "good" solution.

 Consider this code:

 string func(in string[] args)
 {
      return args.join(" "); // compile error, args is not an input range
 }

 It is somewhat expected as you can hardly popFront on a const range. But  
 then question is: how can I wrap const slice into a mutable range  
 without copying data (using standard language/library solution)? Of  
 course, I can do .dup.join, but there is no real need to do copy, even a  
 shallow one.


What you are looking for is a tail-const array:

string func(const(string)[] args)

This will allow traversal, and not allow modification of the elements.

-Steve
Feb 11 2013
parent reply "Dicebot" <m.strashun gmail.com> writes:
On Monday, 11 February 2013 at 15:06:26 UTC, Steven Schveighoffer 
wrote:

 On Mon, 11 Feb 2013 09:46:54 -0500, Dicebot 
 <m.strashun gmail.com> wrote:


 This question is so stupid I am just struck with trying to 
 chose the "good" solution.

 Consider this code:

 string func(in string[] args)
 {
     return args.join(" "); // compile error, args is not an 
 input range
 }

 It is somewhat expected as you can hardly popFront on a const 
 range. But then question is: how can I wrap const slice into a 
 mutable range without copying data (using standard 
 language/library solution)? Of course, I can do .dup.join, but 
 there is no real need to do copy, even a shallow one.


 What you are looking for is a tail-const array:

 string func(const(string)[] args)

 This will allow traversal, and not allow modification of the 
 elements.

 -Steve


No, I do not look for tail const. I don't want to allow 
head-mutability for parameter, but I want to make a mutable 
slice-range for it later. That does not break any const 
guarantees and should be safely possible.
Feb 11 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 11 Feb 2013 10:26:40 -0500, Dicebot <m.strashun gmail.com> wrote:


 On Monday, 11 February 2013 at 15:06:26 UTC, Steven Schveighoffer wrote:

 On Mon, 11 Feb 2013 09:46:54 -0500, Dicebot <m.strashun gmail.com>  
 wrote:


 This question is so stupid I am just struck with trying to chose the  
 "good" solution.

 Consider this code:

 string func(in string[] args)
 {
     return args.join(" "); // compile error, args is not an input range
 }

 It is somewhat expected as you can hardly popFront on a const range.  
 But then question is: how can I wrap const slice into a mutable range  
 without copying data (using standard language/library solution)? Of  
 course, I can do .dup.join, but there is no real need to do copy, even  
 a shallow one.


 What you are looking for is a tail-const array:

 string func(const(string)[] args)

 This will allow traversal, and not allow modification of the elements.

 -Steve


 No, I do not look for tail const. I don't want to allow head-mutability  
 for parameter, but I want to make a mutable slice-range for it later.  
 That does not break any const guarantees and should be safely possible.


in means "const scope".  scope is a no-op, const makes the array const,  
including the pointer length.

const(T)[] means, the ELEMENTS are const, but the pointer and length can  
be changed.  This makes it a valid input range.

Since your function is making a copy of the data, this should be fine.

Your explanation is difficult to understand, I'm basically going on what  
your code does.  If you change "in string[]" to "const(string)[]", the  
function should compile.

-Steve
Feb 11 2013
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, February 11, 2013 11:35:22 Steven Schveighoffer wrote:

 in means "const scope". scope is a no-op, const makes the array const,
 including the pointer length.


I would point out that that may change in the future. scope is supposed to 
stop references escaping in general but is only partially implemented 
(currently only for delegates - and I'm not even sure that it's fully 
implemented there). That being the case, using scope or in on an array could 
have a major effect on what you can do with it in the future (e.g. returning a 
slice of it would become illegal), which is one reason why I pretty much 
always tell people never to use in. Too many people take a liking to in on the 
theory that it's the opposite of out, but it hides the fact that scope is 
involved, and a lot of code is going to break once scope is fully implemented. 
in is one thing that I would _love_ to have eliminated from the language, but 
it's a carryover from D1 that was badly translated to D2.


 const(T)[] means, the ELEMENTS are const, but the pointer and length can
 be changed. This makes it a valid input range.
 
 Since your function is making a copy of the data, this should be fine.
 
 Your explanation is difficult to understand, I'm basically going on what
 your code does. If you change "in string[]" to "const(string)[]", the
 function should compile.


The const(T)[] cannot alter the original array at all, so I concur with Steven 
in that the complaints about not wanting to use tail-const make no sense. 
Maybe this article will help clear things up:

http://dlang.org/d-array-article.html

- Jonathan M Davis
Feb 11 2013
parent reply "Dicebot" <m.strashun gmail.com> writes:
On Monday, 11 February 2013 at 16:54:04 UTC, Jonathan M Davis 
wrote:

 The const(T)[] cannot alter the original array at all, so I 
 concur with Steven
 in that the complaints about not wanting to use tail-const make 
 no sense.
 Maybe this article will help clear things up:


And I exactly want to prohibit altering original array. What I 
want is to copy slice (without deep copying of data) and alter 
this fresh copy. I can't see how it contradicts immutability 
guarantees or D array design.
Feb 11 2013
parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Mon, 11 Feb 2013 17:13:21 -0000, Dicebot <m.strashun gmail.com> wrote:


 On Monday, 11 February 2013 at 16:54:04 UTC, Jonathan M Davis wrote:

 The const(T)[] cannot alter the original array at all, so I concur with  
 Steven
 in that the complaints about not wanting to use tail-const make no  
 sense.
 Maybe this article will help clear things up:


 And I exactly want to prohibit altering original array. What I want is  
 to copy slice (without deep copying of data) and alter this fresh copy.  
 I can't see how it contradicts immutability guarantees or D array design.


This *does* prohibit altering the original array ("test" passed by main  
below) without deep copying the data, and allows you to alter the fresh  
copy ("arr" inside the function).

Example:

import std.stdio;

void foo(const(string)[] arr)
{
   // arr[0] = "aa";  // Error: arr[0] isn't mutable
   arr.length = 10;
   writefln("foo arr.length = %d", arr.length);
}

void main()
{
   const(string[]) test = ["a", "b"];

   // test[0] = "aa";   // Error: test[0] isn't mutable
   // test.length = 10; // Error: variable ****.test cannot modify const
   foo(test);
   writefln("test.length = %d", test.length);
}

What you have to realise is that when you pass an array/slice in D, you  
get a copy of that array/slice in the function.  Just like when you pass  
an 'int' or any other "value" type.

Think of the array/slice as this struct:

struct array
{
   int length;
   void *ptr;
}

Passing that to a function would copy the entire struct into the function  
parameter, altering that parameter would not alter the original.   
array/slices are the same.

The data referred to by 'ptr' is not copied, and neither are the elements  
of an array/slice.

If you did want to alter the original, you'd pass the array/slice by  
'ref'erence.

R

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Feb 11 2013
parent "Dicebot" <m.strashun gmail.com> writes:
I know all of this. When I speak about "original" array, I mean 
function parameter itself, not one passed to function. Actually, 
nevermind, Jonathan has kind of answered my question.
Feb 11 2013
prev sibling parent reply "Dicebot" <m.strashun gmail.com> writes:
On Monday, 11 February 2013 at 16:35:22 UTC, Steven Schveighoffer 
wrote:

 in means "const scope".  scope is a no-op, const makes the 
 array const, including the pointer length.

 const(T)[] means, the ELEMENTS are const, but the pointer and 
 length can be changed.  This makes it a valid input range.

 Since your function is making a copy of the data, this should 
 be fine.

 Your explanation is difficult to understand, I'm basically 
 going on what your code does.  If you change "in string[]" to 
 "const(string)[]", the function should compile.


I know all of this. And I need guarantees that initial slice will 
always start at the very same point and will never be consumed by 
any part of this function. Thus, I do not need tail const and in 
is exactly what I want. But I see no reason why I can't copy 
slice pointers to create a new tail const range to consume.

To sum up: I want to maintain full const for parameter 
range/slice but be able to create a tail const range from it 
every time I want to actually consume it. Without copying data 
itself.
Feb 11 2013
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 11 Feb 2013 12:01:53 -0500, Dicebot <m.strashun gmail.com> wrote:


 On Monday, 11 February 2013 at 16:35:22 UTC, Steven Schveighoffer wrote:

 in means "const scope".  scope is a no-op, const makes the array const,  
 including the pointer length.

 const(T)[] means, the ELEMENTS are const, but the pointer and length  
 can be changed.  This makes it a valid input range.

 Since your function is making a copy of the data, this should be fine.

 Your explanation is difficult to understand, I'm basically going on  
 what your code does.  If you change "in string[]" to "const(string)[]",  
 the function should compile.


 I know all of this. And I need guarantees that initial slice will always  
 start at the very same point and will never be consumed by any part of  
 this function. Thus, I do not need tail const and in is exactly what I  
 want. But I see no reason why I can't copy slice pointers to create a  
 new tail const range to consume.

 To sum up: I want to maintain full const for parameter range/slice but  
 be able to create a tail const range from it every time I want to  
 actually consume it. Without copying data itself.


Ah, ok.  So that isn't the complete function.

The answer is, yes you can make a tail const variable from a const  
variable:

string func(in string[] args)
{
     const(string)[] tmpargs = args;
     return tmpargs.join(" ");
}

Hm... trying this out, it doesn't work.  The assignment to tmpargs works  
alright, it's the join call that doesn't.  Even my original suggestion  
doesn't work.  The problem is that const(string) is not an input range.   
join requires each element from its input range to also be an input  
range.  But of course, const(string) (which is not an input range)  
implicitly casts to string (which is an input range).  So join simply  
isn't taking that step.

We seriously need to fix this kind of problem.  I think someone had  
proposed at some point that tail-const/tail-immutable versions are always  
preferred in all cases, but I guess that never went through.

Maybe there are other ways to fix this.

-Steve
Feb 11 2013
parent reply "Dicebot" <m.strashun gmail.com> writes:
Ye, looks like this is the root of problem, I tweaked at a bit 
and have come to the same conclusion :( Sad, I really hoped I 
simply have overlooked some common idiom.
Feb 11 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 11 Feb 2013 13:59:41 -0500, Dicebot <m.strashun gmail.com> wrote:


 Ye, looks like this is the root of problem, I tweaked at a bit and have  
 come to the same conclusion :( Sad, I really hoped I simply have  
 overlooked some common idiom.


This is a bug, joinImpl compiles with the given type, it's just join that  
doesn't.

Note that I performed this change to the template constraint on join:

if(isInputRange!RoR &&
        (isInputRange!(ElementType!RoR) ||  
isDynamicArray!(ElementType!RoR)) &&
        isForwardRange!R &&
        is(Unqual!(ElementType!(ElementType!RoR)) ==  
Unqual!(ElementType!R)))

Note the isInputRange || isDynamicArray part.

This check was already present on joinImpl.

With that change, then you can compile my original suggestion, or my later  
suggestion.

-Steve
Feb 11 2013
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 11 Feb 2013 14:52:51 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:



 With that change, then you can compile my original suggestion, or my  
 later suggestion.


Actually, it seems that even args.join(" ") works, without changing to the  
tail-const type...

That is technically correct, but I'm not sure why.  I would have expected  
isInputRange!(const string[]) to be false.

-Steve
Feb 11 2013
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 11 Feb 2013 14:56:29 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 On Mon, 11 Feb 2013 14:52:51 -0500, Steven Schveighoffer  
 <schveiguy yahoo.com> wrote:



 With that change, then you can compile my original suggestion, or my  
 later suggestion.


 Actually, it seems that even args.join(" ") works, without changing to  
 the tail-const type...

 That is technically correct, but I'm not sure why.  I would have  
 expected isInputRange!(const string[]) to be false.


Ah, it is false.  But IFTI is removing the head-constness of it (that was  
the feature I was thinking of).

But it doesn't remove the head-constness of the element type!  I don't  
know if there is a correct way to do this.  We probably need to add a new  
template constraint that examines the most permissive legal copy of a  
range is.

-Steve
Feb 11 2013
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, February 11, 2013 14:56:29 Steven Schveighoffer wrote:

 On Mon, 11 Feb 2013 14:52:51 -0500, Steven Schveighoffer
 
 <schveiguy yahoo.com> wrote:

 With that change, then you can compile my original suggestion, or my
 later suggestion.


 
 Actually, it seems that even args.join(" ") works, without changing to the
 tail-const type...
 
 That is technically correct, but I'm not sure why. I would have expected
 isInputRange!(const string[]) to be false.


It _is_ false.

--------
import std.range;

void main()
{
 assert(!isInputRange!(const string[]));
}
--------

But remember that templated functions which are given array always instantiate 
with the tail-const type of an array (assuming that you use IFTI rather than 
explicit instantiaton), so that may be where the confusion is coming from.

- Jonathan M Davis
Feb 11 2013
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, February 11, 2013 18:01:53 Dicebot wrote:

 On Monday, 11 February 2013 at 16:35:22 UTC, Steven Schveighoffer
 
 wrote:

 in means "const scope". scope is a no-op, const makes the
 array const, including the pointer length.
 
 const(T)[] means, the ELEMENTS are const, but the pointer and
 length can be changed. This makes it a valid input range.
 
 Since your function is making a copy of the data, this should
 be fine.
 
 Your explanation is difficult to understand, I'm basically
 going on what your code does. If you change "in string[]" to
 "const(string)[]", the function should compile.


 
 I know all of this. And I need guarantees that initial slice will
 always start at the very same point and will never be consumed by
 any part of this function. Thus, I do not need tail const and in
 is exactly what I want.


Actually, it sounds like tail-const is _exactly_ what you want. If the 
function accepts const(string)[], then _nothing_ you do to that parameter will 
alter the original array, and no deep copies will be performed. Making the 
whole array const buys you _nothing_. You're just making the local variable 
const, making it so that you can't use it as a range. It has _zero_ effect on 
the original array either way. I really don't understand what you're trying to 
gain by making the whole array const.


 But I see no reason why I can't copy
 slice pointers to create a new tail const range to consume.
 
 To sum up: I want to maintain full const for parameter
 range/slice but be able to create a tail const range from it
 every time I want to actually consume it. Without copying data
 itself.


If you want a tail-const slice, then just slice it.

auto newSlice = orig[];

- Jonathan M Davis
Feb 11 2013
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Dicebot:


 string func(in string[] args)
 {
     return args.join(" "); // compile error, args is not an 
 input range
 }


join() should to able to join a const array of strings.

Bye,
bearophile
Feb 11 2013
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11-Feb-2013 18:46, Dicebot пишет:

 This question is so stupid I am just struck with trying to chose the
 "good" solution.

 Consider this code:

 string func(in string[] args)
 {
      return args.join(" "); // compile error, args is not an input range
 }


I might be wrong but you can slice it e.g.:

args[].join(" ");


 It is somewhat expected as you can hardly popFront on a const range. But
 then question is: how can I wrap const slice into a mutable range
 without copying data (using standard language/library solution)? Of
 course, I can do .dup.join, but there is no real need to do copy, even a
 shallow one.



-- 
Dmitry Olshansky
Feb 11 2013
parent reply "Dicebot" <m.strashun gmail.com> writes:
On Monday, 11 February 2013 at 16:25:35 UTC, Dmitry Olshansky 
wrote:

 I might be wrong but you can slice it e.g.:

 args[].join(" ");


[], as well as [0.$] does return the very same qualified slice, 
have tried it.
Feb 11 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, February 11, 2013 17:42:13 Dicebot wrote:

 On Monday, 11 February 2013 at 16:25:35 UTC, Dmitry Olshansky
 
 wrote:

 I might be wrong but you can slice it e.g.:
 
 args[].join(" ");


 
 [], as well as [0.$] does return the very same qualified slice,
 have tried it.


For arrays, [] returns a tail-const slice of the array. If it doesn't, it's a 
bug.

- Jonathan M Davis
Feb 11 2013
parent "Dicebot" <m.strashun gmail.com> writes:
On Monday, 11 February 2013 at 16:53:53 UTC, Jonathan M Davis 
wrote:

 On Monday, February 11, 2013 17:42:13 Dicebot wrote:

 On Monday, 11 February 2013 at 16:25:35 UTC, Dmitry Olshansky
 
 wrote:

 I might be wrong but you can slice it e.g.:
 
 args[].join(" ");


 
 [], as well as [0.$] does return the very same qualified slice,
 have tried it.


 For arrays, [] returns a tail-const slice of the array. If it 
 doesn't, it's a
 bug.

 - Jonathan M Davis


Actually I was wrong, beg my pardon. It does return tail-const 
slice, but join fails to accept even tail-const one. Will debug 
further and report.
Feb 11 2013