www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - to! converting 1D to 2D array

reply "ed" <growlercab gmail.com> writes:
Hi All,

I am trying to convert a 1D array to a 2D array using to! and was 
wondering if someone could explain to me why this first example 
works but the second example does not compile:

Example 1:
~~~
import std.conv;
long[4] a=[1,2,3,4];
int[2][2] b = to!(int[])(a);
assert(b == [[1,2],[3,4]]);
~~~~

Example 2:
~~~
import std.conv;
long[4] a=[1,2,3,4];
int[2][2] b;
b = to!(int[])(a);
assert(b == [[1,2],[3,4]]);
~~~~
Error: cannot implicitly convert expression (to(a[])) of type 
int[][] to int[2][].

It's like the ReturnType in to!()() for Example 1 and Example 2 
is different. Is  this the case and why would it be so? If not, 
what is happening here?

Thanks,
ed
Mar 11 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
ed:


 I am trying to convert a 1D array to a 2D array


If you have a dynamic array (1D), you can convert it to a dynamic 
array of dynamic arrays (2D) using chunks:


void main() {
     import std.stdio, std.range, std.algorithm;

     int[] a = [1, 2, 3, 4, 5, 6];
     int[][] b = a.chunks(2).array;
     b.writeln;
}


Output:

[[1, 2], [3, 4], [5, 6]]

Your problems are caused by mixing fixed-size arrays (that are 
values, allocated in-place), with dynamic arrays (that are little 
length+pointer structs that often point to heap-allocated memory).

Bye,
bearophile
Mar 11 2014
parent reply "ed" <growlercab gmail.com> writes:
On Wednesday, 12 March 2014 at 02:14:45 UTC, bearophile wrote:

 ed:


 I am trying to convert a 1D array to a 2D array


 If you have a dynamic array (1D), you can convert it to a 
 dynamic array of dynamic arrays (2D) using chunks:


 void main() {
     import std.stdio, std.range, std.algorithm;

     int[] a = [1, 2, 3, 4, 5, 6];
     int[][] b = a.chunks(2).array;
     b.writeln;
 }


 Output:

 [[1, 2], [3, 4], [5, 6]]

 Your problems are caused by mixing fixed-size arrays (that are 
 values, allocated in-place), with dynamic arrays (that are 
 little length+pointer structs that often point to 
 heap-allocated memory).

 Bye,
 bearophile


Thanks for explaining this, it makes sense what you said. But I'm 
still not sure why my original Example 1 worked.

~~~
// This works OK and converts long[4] to int[] then implicitly to 
int[2][2]
long[4] a=[1,2,3,4];
int[2][2] b = to!(int[])(a);

// Why does this not work the same way?
long[4] a=[1,2,3,4];
int[2][2] b;
b = to!(int[])(a);
~~~

My understanding of your explanation is that it shouldn't work.


Cheers,
ed
Mar 11 2014
next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 12 March 2014 at 03:37:49 UTC, ed wrote:

 Thanks for explaining this, it makes sense what you said. But 
 I'm still not sure why my original Example 1 worked.

 ~~~
 // This works OK and converts long[4] to int[] then implicitly 
 to int[2][2]
 long[4] a=[1,2,3,4];
 int[2][2] b = to!(int[])(a);

 // Why does this not work the same way?
 long[4] a=[1,2,3,4];
 int[2][2] b;
 b = to!(int[])(a);
 ~~~

 My understanding of your explanation is that it shouldn't work.


 Cheers,
 ed


I *believe* it's really just that you are allowed to *initialize* 
a static array (of any depth) from a dynamic array.

However, for assignment, it doesn't work that way:

     int[] a = [0, 1, 2, 3];
     int[2][1][2][1] b = a; //OK!
     b = a; //ERROR

If you want do want to do the assignment, you'll have to 
re-interpret your array's layout:

     int[] a = [0, 1, 2, 3];
     int[2][1][2][1] b;
     *cast(int[4]*)&b = a;
     assert(b == [[[[0, 1]], [[2, 3]]]]);
Mar 11 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
monarch_dodra:


     int[] a = [0, 1, 2, 3];
     int[2][1][2][1] b;
     *cast(int[4]*)&b = a;
     assert(b == [[[[0, 1]], [[2, 3]]]]);


Those pointers are not needed:

cast(int[4])b = a;

Bye,
bearoophile
Mar 12 2014
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Wednesday, 12 March 2014 at 07:43:46 UTC, bearophile wrote:

 monarch_dodra:


    int[] a = [0, 1, 2, 3];
    int[2][1][2][1] b;
    *cast(int[4]*)&b = a;
    assert(b == [[[[0, 1]], [[2, 3]]]]);


 Those pointers are not needed:

 cast(int[4])b = a;

 Bye,
 bearoophile


I'm *never* actually sure if these kinds of casts are value 
casts, or re-interpreting casts.

Thanks.
Mar 12 2014
prev sibling parent "ed" <growlercab gmail.com> writes:
On Wednesday, 12 March 2014 at 07:43:46 UTC, bearophile wrote:

 monarch_dodra:


    int[] a = [0, 1, 2, 3];
    int[2][1][2][1] b;
    *cast(int[4]*)&b = a;
    assert(b == [[[[0, 1]], [[2, 3]]]]);


 Those pointers are not needed:

 cast(int[4])b = a;

 Bye,
 bearoophile


Thanks for this, the casts work perfectly albeit a little ugly :D

Cheers,
ed
Mar 12 2014
prev sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 12 March 2014 at 06:53:19 UTC, monarch_dodra wrote:

 I *believe* it's really just that you are allowed to 
 *initialize* a static array (of any depth) from a dynamic array.

 However, for assignment, it doesn't work that way:

     int[] a = [0, 1, 2, 3];
     int[2][1][2][1] b = a; //OK!
     b = a; //ERROR


Is this intentional, or is it a bug?
Mar 13 2014
prev sibling parent reply "Chris Williams" <yoreanon-chrisw yahoo.co.jp> writes:
On Wednesday, 12 March 2014 at 03:37:49 UTC, ed wrote:

 My understanding of your explanation is that it shouldn't work.


It shouldn't and probably isn't working. If nothing else, when 
you use to!(x)(y), "x" should be the type that you're trying to 
convert into. So I would expect your code to be 
to!(int[2][2])(a). I would still be impressed if that worked, 
since std.conv is intended more for parsing strings into ints and 
stuff, not for splitting and grouping arrays.

Your assertion probably succeeds because in memory a two 
dimensional array is arranged with the first row followed by the 
second row. So a flat array of four items and a 2-D array of two 
arrays of two items end up laying out the same in memory. Some 
how it's going straight to memory and comparing them that way 
(maybe).
Mar 12 2014
parent reply "ed" <growlercab gmail.com> writes:
On Thursday, 13 March 2014 at 00:15:19 UTC, Chris Williams wrote:
[snip]

 It shouldn't and probably isn't working.


It is working and in fact it is in a "const pure  safe" function. 
So I will trust it  :-)


 If nothing else, when you use to!(x)(y), "x" should be the type 
 that you're trying to convert into. So I would expect your code 
 to be to!(int[2][2])(a).


Yes, I'm aware of this, the conversion I was doing is as follows:

long[4] -> to! -> int[] ->implicit conv. -> int[2][2].

it is the implicit conversion that was failing, not the to! 
function. I didn't realise this until after bearophile's reply :D

Thanks,
ed
Mar 12 2014
parent reply "Chris Williams" <yoreanon-chrisw yahoo.co.jp> writes:
On Thursday, 13 March 2014 at 03:31:09 UTC, ed wrote:

 On Thursday, 13 March 2014 at 00:15:19 UTC, Chris Williams 
 wrote:
 [snip]

 It shouldn't and probably isn't working.


 It is working and in fact it is in a "const pure  safe" 
 function. So I will trust it  :-)


Well it's like a broken watch being correct twice a day. The 
correct result doesn't mean that something is working.

I'm glad you've found a solution though.
Mar 13 2014
parent reply "ed" <growlercab gmail.com> writes:
On Thursday, 13 March 2014 at 18:17:03 UTC, Chris Williams wrote:

 On Thursday, 13 March 2014 at 03:31:09 UTC, ed wrote:

 On Thursday, 13 March 2014 at 00:15:19 UTC, Chris Williams 
 wrote:
 [snip]

 It shouldn't and probably isn't working.


 It is working and in fact it is in a "const pure  safe" 
 function. So I will trust it  :-)


 Well it's like a broken watch being correct twice a day. The 
 correct result doesn't mean that something is working.


But this is perfectly valid and safe D code, is it not?

long[4] a=[1,2,3,4];
int[2][2] b = a.to!(int[]);

which is no different to this:

int[] a_int = a.to!(int[]);
int[2][2] b = a_int;

Am I missing something here?

Thanks,
ed
Mar 13 2014
parent reply "ed" <growlercab gmail.com> writes:
On Friday, 14 March 2014 at 04:18:18 UTC, ed wrote:

 On Thursday, 13 March 2014 at 18:17:03 UTC, Chris Williams 
 wrote:

 On Thursday, 13 March 2014 at 03:31:09 UTC, ed wrote:

 On Thursday, 13 March 2014 at 00:15:19 UTC, Chris Williams 
 wrote:
 [snip]

 It shouldn't and probably isn't working.


 It is working and in fact it is in a "const pure  safe" 
 function. So I will trust it  :-)


 Well it's like a broken watch being correct twice a day. The 
 correct result doesn't mean that something is working.


 But this is perfectly valid and safe D code, is it not?

 long[4] a=[1,2,3,4];
 int[2][2] b = a.to!(int[]);

 which is no different to this:

 int[] a_int = a.to!(int[]);
 int[2][2] b = a_int;

 Am I missing something here?

 Thanks,
 ed


Looking at the disassembly I'm convinced the code is benign and 
valid D. It just goes through the D runtime which does the full 
array bounds checking.

As to whether or not this should work:

int[4] a=[1,2,3,4];
int[2][2] b;
b=a;

is up to the D language gurus. I think it should... but I'm no 
language developer, there may be other side-effects I haven't 
thought about.

Cheers,
ed
Mar 13 2014
parent reply "Chris Williams" <yoreanon-chrisw yahoo.co.jp> writes:
On Friday, 14 March 2014 at 04:36:27 UTC, ed wrote:

 As to whether or not this should work:

 int[4] a=[1,2,3,4];
 int[2][2] b;
 b=a;

 is up to the D language gurus. I think it should... but I'm no 
 language developer, there may be other side-effects I haven't 
 thought about.

 Cheers,
 ed


In C, any array is just a starting address in memory. Accessing 
indexes is accomplished during compile time, where the compiler 
does some math based on the size of the objects in the array and 
how many dimensions the array has, then more-or-less hardcodes an 
offset to add to the starting address. All arrays are mutually 
exchangeable because they're just a pointer.

In D, an array is a struct (struct Array), with an address and a 
length value. A multi-dimensional array is an Array with an 
address pointing to an array of Arrays. So with an int[2][2] 
array, you have a layout like:

 1000 Array(address=1016, length=2)
 1016 [Array(address=1048, length=2),Array(address=1056, 
length=2)]
 1048 [1,2]
 1056 [3,4]

In this particular case, the data at 1056 is directly following 
the data at 1048. There's no gap between them, so considering the 
buffer at 1048 to be a single array of 4 or two arrays of two is 
inconsequential. But that's no guarantee. Those two arrays could 
be off in entirely separate chunks of RAM. In that case, setting 
a=b would force a copy to occur, since a requires the items to be 
continguous, where b=a results in both variables pointing to the 
same underlying data (changing one changes the other).

Now that's assuming that the compiler is actually trying to 
convert one into the other.

There's the other option of considering a and b to both be of 
type Array. As such, you can simply copy the values in one over 
to the other.

a=b; // equivalent to a.address=b.address; a.length=b.length;
b=a; // equivalent to b.address=a.address; b.length=a.length;

This makes complete sense, other than it trashes the settee's 
type. What was int[2][2] effectively becomes int[4] (or 
vise-versa), which should then make an access to b[0][1] fail, 
since the value at entry [0] isn't an Array struct.

Personally, I don't like the inconsistency of the former 
strategy. People should be forced to implement their own strategy 
for converting array types (whether to create a copy or point to 
the same underlying data). For the latter strategy, changing an 
lvalue's type by setting into it seems like it should only be 
allowed if you cast. Though since it's the lvalue changing, it 
would be the lvalue that you would have to cast, and usually a 
cast only lasts for that line not the rest of time, which 
wouldn't be the case here so....

cast(int[4])b = a; // ???

Overall, I don't think the compiler should allow the original 
code. Even if the specification specifies what should happen, the 
minutiae of it seems prone to creating bugs.
Mar 14 2014
next sibling parent "ed" <growlercab gmail.com> writes:
On Friday, 14 March 2014 at 19:24:21 UTC, Chris Williams wrote:

[snip]

 address pointing to an array of Arrays. So with an int[2][2] 
 array, you have a layout like:

  1000 Array(address=1016, length=2)
  1016 [Array(address=1048, length=2),Array(address=1056, 
 length=2)]
  1048 [1,2]
  1056 [3,4]

 In this particular case, the data at 1056 is directly following 
 the data at 1048. There's no gap between them, so considering 
 the buffer at 1048 to be a single array of 4 or two arrays of 
 two is inconsequential. But that's no guarantee.


Actually this is guaranteed for static rectangular arrays:

http://dlang.org/arrays#static-arrays (see Rectangular Arrays)
http://wiki.dlang.org/Dense_multidimensional_arrays (Static 
Arrays)

This code below is safe. It is nothing more than a check for 
conformity followed by a memcpy:

int[] a = [1,2,3,4];
int[2][2] b = a;


I might raise a new question asking why this doesn't work as I 
expect:

int[2][2] b;
b=a;


Thanks for your help on this one. It has forced me to drill into 
the internals a bit more, which is always a good thing :)

Cheers,
ed
Mar 14 2014
prev sibling next sibling parent "ed" <growlercab gmail.com> writes:
On Friday, 14 March 2014 at 19:24:21 UTC, Chris Williams wrote:

It looks like you might be right after all about the code being 
invalid D. It could be a bug when it compiles without the cast.

I filed a bug report about it, see where it leads :D

Thanks,
ed
Mar 14 2014
prev sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 14 March 2014 at 19:24:21 UTC, Chris Williams wrote:

 In D, an array is a struct (struct Array), with an address and 
 a length value. A multi-dimensional array is an Array with an 
 address pointing to an array of Arrays. So with an int[2][2] 
 array, you have a layout like:

  1000 Array(address=1016, length=2)
  1016 [Array(address=1048, length=2),Array(address=1056, 
 length=2)]
  1048 [1,2]
  1056 [3,4]


This is not true; you need to distinguish between fixed-size and 
dynamically sized arrays. It's better to call the latter "slices" 
to avoid the confusion.

Fixed-size arrays are value types, there are no 
references/addresses involved in storing them. So this:

int[2][2] a;

always has the following layout:

 1000 [1,2]
 1008 [3,4]

You layout would be correct for this example:

int[][] = [[1,2],[3,4]];
Mar 22 2014