www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - array operations and ranges

reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
Array operations are super cool, and I'm using ranges (which kinda
look and feel like arrays) more and more these days, but I can't help
but feel like their incompatibility with the standard array operations
is a massive loss.

Let's say I want to assign one range to another: b[] = a[];
It's not clear to me why this should fall down if I want to apply a
lazy operation for instance: b[] = a.map!(e=>e*2)[];
... or something to that effect.

I find that my lazy ranges often end up on the stack, but I can't
assign/initialise directly: float[] a = b.transform[];
Instead I need to: float[] a;  b.transform.copy(a[]);

The fact that they don't mix with array expressions or operators means
as soon as a lazy range finds it wants to enter existing array code,
it needs to be converted to a series of map()'s.

There must be years of thoughts and work on this sort of thing?
It seems arrays and ranges are unnecessarily distanced from
eachother... what are the reasons for this?
Apr 26 2015
next sibling parent reply "Laeeth Isharc" <nospamlaeeth nospam.laeeth.com> writes:
On Sunday, 26 April 2015 at 10:17:59 UTC, Manu wrote:
 Array operations are super cool, and I'm using ranges (which 
 kinda
 look and feel like arrays) more and more these days, but I 
 can't help
 but feel like their incompatibility with the standard array 
 operations
 is a massive loss.

 Let's say I want to assign one range to another: b[] = a[];
 It's not clear to me why this should fall down if I want to 
 apply a
 lazy operation for instance: b[] = a.map!(e=>e*2)[];
 ... or something to that effect.

 I find that my lazy ranges often end up on the stack, but I 
 can't
 assign/initialise directly: float[] a = b.transform[];
 Instead I need to: float[] a;  b.transform.copy(a[]);

 The fact that they don't mix with array expressions or 
 operators means
 as soon as a lazy range finds it wants to enter existing array 
 code,
 it needs to be converted to a series of map()'s.

 There must be years of thoughts and work on this sort of thing?
 It seems arrays and ranges are unnecessarily distanced from
 eachother... what are the reasons for this?
Vlad Levenfeld has been doing some interesting work on just this sort of thing. I will see if he is around.
Apr 26 2015
parent reply "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
 On Sunday, 26 April 2015 at 10:17:59 UTC, Manu wrote:
 I find that my lazy ranges often end up on the stack, but I 
 can't
 assign/initialise directly: float[] a = b.transform[];
 Instead I need to: float[] a;  b.transform.copy(a[]);
To enable the first line, builtin arrays would need to be able to recognize arbitrary range types and do the right thing. You can always do this: float[] a = b[].transform.array; I've got a lib to enable this syntax: Array!float a = b[].transform_1; a[i..j] = c[x..y].transform_2; for arbitrary user-defined or composed n-dimensional range types. here: https://github.com/evenex/autodata where ranges are made more interoperable by common mixin templates which also cut down on a lot of boilerplate. (see examples in: https://github.com/evenex/autodata/tree/master/source/spaces)
Apr 26 2015
parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Sunday, 26 April 2015 at 18:48:15 UTC, Vlad Levenfeld wrote:
 I've got a lib to enable this syntax:

   Array!float a = b[].transform_1;
   a[i..j] = c[x..y].transform_2;
Phobos containers already support the first line, and it would be a natural extension to make them support the second.
Apr 26 2015
parent reply "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
 Phobos containers already support the first line, and it would 
 be a natural extension to make them support the second.
Sure, it's not complicated. It's something I had done in this other code and showing for example.
Apr 26 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 27 April 2015 at 15:58, Vlad Levenfeld via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 Phobos containers already support the first line, and it would be a
 natural extension to make them support the second.
Sure, it's not complicated. It's something I had done in this other code and showing for example.
Yeah, see I don't feel making a simple thing like an array into something more complex by wrapping it in templates is ever a good thing to do. I just think it's a missed opportunity that the compiler doesn't support any of this in the language. It would appear at face value to be a great opportunity for lowering. Assignment can lower to .copy(), operators can lower to map!(...) I can tell you, if I tried to explain to my colleagues that we should wrap an array in a template so assignment works, they would laugh at me, then ridicule me, and then they would dismiss D. Better to say it's not supported than to show them that approach.
Apr 26 2015
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 27 April 2015 at 06:52:11 UTC, Manu wrote:
 On 27 April 2015 at 15:58, Vlad Levenfeld via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Phobos containers already support the first line, and it 
 would be a
 natural extension to make them support the second.
Sure, it's not complicated. It's something I had done in this other code and showing for example.
Yeah, see I don't feel making a simple thing like an array into something more complex by wrapping it in templates is ever a good thing to do. I just think it's a missed opportunity that the compiler doesn't support any of this in the language. It would appear at face value to be a great opportunity for lowering. Assignment can lower to .copy(), operators can lower to map!(...)
It's similar to foreach, which already recognizes input ranges, so why not...
Apr 27 2015
prev sibling next sibling parent "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
On Monday, 27 April 2015 at 06:52:11 UTC, Manu wrote:
 On 27 April 2015 at 15:58, Vlad Levenfeld via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Phobos containers already support the first line, and it 
 would be a
 natural extension to make them support the second.
Sure, it's not complicated. It's something I had done in this other code and showing for example.
Yeah, see I don't feel making a simple thing like an array into something more complex by wrapping it in templates is ever a good thing to do. I just think it's a missed opportunity that the compiler doesn't support any of this in the language. It would appear at face value to be a great opportunity for lowering. Assignment can lower to .copy(), operators can lower to map!(...) I can tell you, if I tried to explain to my colleagues that we should wrap an array in a template so assignment works, they would laugh at me, then ridicule me, and then they would dismiss D. Better to say it's not supported than to show them that approach.
Yeah, it is an excessive solution. I guess just wanted to talk about my thing and didn't think it through. I think laughing at and ridiculing are the same thing though. Anyway, lowering to copy built-in to the language, at least, would be pretty great. Especially if it worked like foreach and picked up local symbols, so that custom copy and map implementations could be used wherever they were defined.
Apr 27 2015
prev sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 27 April 2015 at 06:52:11 UTC, Manu wrote:
 On 27 April 2015 at 15:58, Vlad Levenfeld via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Phobos containers already support the first line, and it 
 would be a
 natural extension to make them support the second.
Sure, it's not complicated. It's something I had done in this other code and showing for example.
Yeah, see I don't feel making a simple thing like an array into something more complex by wrapping it in templates is ever a good thing to do. I just think it's a missed opportunity that the compiler doesn't support any of this in the language. It would appear at face value to be a great opportunity for lowering. Assignment can lower to .copy(), operators can lower to map!(...)
builtin slicesopSliceAssign (and opSliceOpAssign) understanding ranges as source operands is a good idea. I might even see if I can implement it. Lowering array operations to lazy ranges seems like a huge can of worms. Not so keen. Lazy array ops are great, but I don't see it working out as a builtin feature unless it had it's own syntax.
Apr 27 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 28 April 2015 at 06:42, John Colvin via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Monday, 27 April 2015 at 06:52:11 UTC, Manu wrote:
 On 27 April 2015 at 15:58, Vlad Levenfeld via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:
 Phobos containers already support the first line, and it would be a
 natural extension to make them support the second.
Sure, it's not complicated. It's something I had done in this other code and showing for example.
Yeah, see I don't feel making a simple thing like an array into something more complex by wrapping it in templates is ever a good thing to do. I just think it's a missed opportunity that the compiler doesn't support any of this in the language. It would appear at face value to be a great opportunity for lowering. Assignment can lower to .copy(), operators can lower to map!(...)
builtin slicesopSliceAssign (and opSliceOpAssign) understanding ranges as source operands is a good idea. I might even see if I can implement it. Lowering array operations to lazy ranges seems like a huge can of worms. Not so keen. Lazy array ops are great, but I don't see it working out as a builtin feature unless it had it's own syntax.
The only reason I considered it workable was because an operator expression is generally meaningless without an assignment to the left; "a[] = b[]*2;", so the mul could conceivably be expressed in a lazy sense, since it will be evaluated immediately by the assignment. Array operations passed as arguments to functions are already not allowed in the usual case, so I think it would be fine in this instance... it's impossibly for an array operation to have any meaning without a final assignment (I think?)
Apr 27 2015
prev sibling next sibling parent "Vlad Levenfeld" <vlevenfeld gmail.com> writes:
Manu, I just saw your other post clarifying the code was float[N] 
a = ..., not float[] a. That changes things a bit.

I just implemented a static array type in the lib (1-d only for 
now) which can do the following:

unittest {
	import std.range: only;

	StaticArray!(int, 2) x;

	assert (x[] == [0, 0]);

	x[0..2] = only (5, 6);

	assert (x[] == [5, 6]);

	x[] += 5;

	assert (x[] == [10, 11]);

	x[0..1] -= 5;

	assert (x[] == [5, 11]);

	StaticArray!(int, 4) y = only (1,2,3,4);

	assert (y[] == [1, 2, 3, 4]);

	auto z = only (9,8,7).static_array!([3]);

	assert (z[] == [9, 8, 7]);
}


Assertions are thrown if the assigned range doesn't match the 
static array length.

https://github.com/evenex/autodata/blob/master/source/spaces/array.d
Apr 26 2015
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/26/2015 3:17 AM, Manu via Digitalmars-d wrote:
 Array operations are super cool, and I'm using ranges (which kinda
 look and feel like arrays) more and more these days, but I can't help
 but feel like their incompatibility with the standard array operations
 is a massive loss.

 Let's say I want to assign one range to another: b[] = a[];
 It's not clear to me why this should fall down if I want to apply a
 lazy operation for instance: b[] = a.map!(e=>e*2)[];
 ... or something to that effect.

 I find that my lazy ranges often end up on the stack, but I can't
 assign/initialise directly: float[] a = b.transform[];
 Instead I need to: float[] a;  b.transform.copy(a[]);

 The fact that they don't mix with array expressions or operators means
 as soon as a lazy range finds it wants to enter existing array code,
 it needs to be converted to a series of map()'s.

 There must be years of thoughts and work on this sort of thing?
 It seems arrays and ranges are unnecessarily distanced from
 eachother... what are the reasons for this?
I think you've got the start of some dazz ideas. I want to think about this more.
Apr 30 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 May 2015 at 15:10, Walter Bright via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On 4/26/2015 3:17 AM, Manu via Digitalmars-d wrote:
 There must be years of thoughts and work on this sort of thing?
 It seems arrays and ranges are unnecessarily distanced from
 eachother... what are the reasons for this?
I think you've got the start of some dazz ideas. I want to think about this more.
Cool. I'm finding myself using ranges more and more in my code these days. My current projects are almost exclusively range based code. The most problem cases are: float[100] stackArray = range.map!(e => e*2)[]; // <- I often have a lazy operation that needs to be resolved to the stack. It's sad to break this on to another line and use .copy() Other cases come when I have a standard array operation, for instance: a[] = b[]*2; I often want to apply an operation that is expressed via a lazy range, ie, map!() or something. As soon as you do that, you need to rearrange the code: a[] = b.amplitude[]*2; // <- where amplitude is a function that returns map!(e=>20*log10(abs(e))) Or should it be: a[] = b[].amplitude*2;// ? Either way, it must be refactored to keep the compiler happy: b[].amplitude.map!(e=>e*2).copy(a); I think that's a lot less easy to follow. I would try to avoid that compared to the line above, but that creates a mutual exclusion with the use of ranges and simple, readable code. My experience is revealing to me that about 60% of my lines (in range-related code) are map!(e=>...), and that's a lot of boilerplate surrounding '...', obscuring the actual operation. I also find the paren nesting gets pretty deep (for instance, the map above) and the code is harder to follow than a traditional foreach loop. If we can do normal array operations on ranges, and have them lower to sensible things (like map), I think that sugar will make range programming a lot more succinct. Perhaps my 'amplitude' function above could just be a scalar function? I have one of them too, there are duplicates of most things because I need to supply a range based version in parallel to the scalar versions :/ Using UFCS; range.scalarFunction could be recognised as a range-vs-scalar operation, the same as range*2. Ie, if there are no proper overload/template matches, the compiler could try and lower to map(e=>e.scalarFunction), assuming ElementType!Range is compatible with the scalar function. That would eliminate most instances of map!(), which are the highest contributor to pointless visual noise in my current code. Anyway, I think there's a lot of low-hanging fruit in this area, and I think range based programming is proving to be one of the most compelling (and surprisingly awesome) parts of D. With a little polish it could become a major selling point.
May 02 2015