www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - More powerful foreach statements

reply Ben Phillips <Ben_member pathlink.com> writes:
While D's foreach is a much appreciated improvement over C++'s template version,
I feel there are still some ways in which it can be improved. I listed two ideas
below, though if you're in a hurry you can just skip to the second one because
thats the most interesting one (imo).

First proposal: an index tracker
Recently there was a suggestion to allow "first" and "last" blocks in a foreach
statement. This would allow things such as special initialization to get done
during the first iteration through the foreach and possible clean-up work to get
done during the last iteration. The current solution is to either go with a
conventional "for" loop or to create a counter of some sort that counts what
iteration the foreach statement is in (Though I'm not sure how one would detect
the last iteration with a counter).

It would convenient if a foreach statement automatically kept track of the
iteration number.
Example:
foreach(Foo f; ManyFoos mf)
{
if (_iteration == 0)
// special stuff on first iteration here
.. 
}

The _iteration variable should follow normal scoping rules so nested foreach
statements can still access their own iteration counters.

Second proposal: opApply variants
This proposal is more radical than the last. One of the largest problems I have
with foreach statements is that [afaik] iteration can only be done in one
direction. For example, it is easy to use a foreach statement to iterate through
an array starting at index zero but impossible to do it in reverse. The only way
I know to work around this is the following:
class Collection
{
private bool reverse = false;
public Collection reverse()
{
// either sets reverse = true (therefore requiring it to get set back) or
// duplicates this whole collection and sets the reverse flag in that one
// (very expensive)
}
}

I believe a possible solution to this would be to allow one to write different
variants of opApply.
Example:
class Collection(T)
{
// normal opApply
int opApply(int delegate(inout T) dg) { ... }
// reverse order
int opApply|reverse|(int delegate(inout T) dg) { ... }
// some weird iteration pattern
int opApply|weird|(int delegate(inout T) dg) { ... }
}

Usage:
Collection!(int) collection = new Collection!(int)();
..
// to do a reverse order iteration
foreach|reverse|(int element; collection) { ... }
// a weird order one
foreach|weird|(int element, collection) { ... }

The idea is that within the function "opApply" there can be different variants
that can created through use of another string placed inbetween pipe characters
("|"). This string can be whatever the class author feels is appropriate. This
allows for easy implementation of different types of iteration that can be used
with a foreach statement, while still allowing the compiler to recognize the
"opApply" function.

Technically the idea above could also be applied to other functions, however its
usefulness degrades since most functions can just be renamed for different
variants while opApply cannot.
Jul 20 2006
parent reply kris <foo bar.com> writes:
Ben Phillips wrote:

 While D's foreach is a much appreciated improvement over C++'s template
version,
 I feel there are still some ways in which it can be improved. I listed two
ideas
 below, though if you're in a hurry you can just skip to the second one because
 thats the most interesting one (imo).
 
 First proposal: an index tracker
 Recently there was a suggestion to allow "first" and "last" blocks in a foreach
 statement. This would allow things such as special initialization to get done
 during the first iteration through the foreach and possible clean-up work to
get
 done during the last iteration. The current solution is to either go with a
 conventional "for" loop or to create a counter of some sort that counts what
 iteration the foreach statement is in (Though I'm not sure how one would detect
 the last iteration with a counter).
 
 It would convenient if a foreach statement automatically kept track of the
 iteration number.
 Example:
 foreach(Foo f; ManyFoos mf)
 {
 if (_iteration == 0)
 // special stuff on first iteration here
 .. 
 }
 
 The _iteration variable should follow normal scoping rules so nested foreach
 statements can still access their own iteration counters.
 
 Second proposal: opApply variants
 This proposal is more radical than the last. One of the largest problems I have
 with foreach statements is that [afaik] iteration can only be done in one
 direction. For example, it is easy to use a foreach statement to iterate
through
 an array starting at index zero but impossible to do it in reverse. The only
way
 I know to work around this is the following:
 class Collection
 {
 private bool reverse = false;
 public Collection reverse()
 {
 // either sets reverse = true (therefore requiring it to get set back) or
 // duplicates this whole collection and sets the reverse flag in that one
 // (very expensive)
 }
 }
 
 I believe a possible solution to this would be to allow one to write different
 variants of opApply.
 Example:
 class Collection(T)
 {
 // normal opApply
 int opApply(int delegate(inout T) dg) { ... }
 // reverse order
 int opApply|reverse|(int delegate(inout T) dg) { ... }
 // some weird iteration pattern
 int opApply|weird|(int delegate(inout T) dg) { ... }
 }
 
 Usage:
 Collection!(int) collection = new Collection!(int)();
 ..
 // to do a reverse order iteration
 foreach|reverse|(int element; collection) { ... }
 // a weird order one
 foreach|weird|(int element, collection) { ... }
 
 The idea is that within the function "opApply" there can be different variants
 that can created through use of another string placed inbetween pipe characters
 ("|"). This string can be whatever the class author feels is appropriate. This
 allows for easy implementation of different types of iteration that can be used
 with a foreach statement, while still allowing the compiler to recognize the
 "opApply" function.
 
 Technically the idea above could also be applied to other functions, however
its
 usefulness degrades since most functions can just be renamed for different
 variants while opApply cannot.
 
 



Both can be done right now:

1)  foreach (int i, Foo f; ManyFooInstance)
              if (i is 0)
                  ....

Or somthing like that.


2) foreach (int element; collection.reverse)
	    ...


some kind of Iterator construct. IIRC, someone already has a framework 
for such things? Was it Oskar?
Jul 20 2006
next sibling parent reply Ben Phillips <Ben_member pathlink.com> writes:
In article <e9ogh7$b1b$1 digitaldaemon.com>, kris says...

1)  foreach (int i, Foo f; ManyFooInstance)
              if (i is 0)
                  ....

Or somthing like that.


I tested it and it works, AND its documented. My fault.


2) foreach (int element; collection.reverse)
	    ...


some kind of Iterator construct. IIRC, someone already has a framework 
for such things? Was it Oskar?


I remember seeing stuff like this in the MinTL code, but having to create a
separate type for each type of iteration that you want to support seems rather
silly (though adding a new language feature to do so may be equally silly as
well).

Also, that still doesn't allow one to go reverse order through a builtin array
with a foreach, unless of course you call ".reverse", which is needlessly
expensive.
Jul 20 2006
next sibling parent "Andrei Khropov" <andkhropov nospam_mtu-net.ru> writes:
Ben Phillips wrote:


 In article <e9ogh7$b1b$1 digitaldaemon.com>, kris says...

 
 1)  foreach (int i, Foo f; ManyFooInstance)
              if (i is 0)
                  ....
 
 Or somthing like that.
 


 
 I tested it and it works, AND its documented. My fault.


And you can save you some typing if you like by omitting types:

------------------------------------------------------------------
foreach ( i,foo; ManyFooInstance)
   if (i is 0)
------------------------------------------------------------------

-- 
AKhropov
Jul 20 2006
prev sibling parent Oskar Linde <oskar.lindeREM OVEgmail.com> writes:
Ben Phillips wrote:

 In article <e9ogh7$b1b$1 digitaldaemon.com>, kris says...

 1)  foreach (int i, Foo f; ManyFooInstance)
              if (i is 0)
                  ....

 Or somthing like that.


 
 I tested it and it works, AND its documented. My fault.
 

 2) foreach (int element; collection.reverse)
 	    ...


 some kind of Iterator construct. IIRC, someone already has a framework 
 for such things? Was it Oskar?


 
 I remember seeing stuff like this in the MinTL code, but having to create a
 separate type for each type of iteration that you want to support seems rather
 silly (though adding a new language feature to do so may be equally silly as
 well).
 
 Also, that still doesn't allow one to go reverse order through a builtin array
 with a foreach, unless of course you call ".reverse", which is needlessly
 expensive.


Yes it does. Something like this:

//Converts static arrays to dynamic arrays, leaving other types as
//they are
template MakeDynamic(T) {
         static if( is(typeof(Declare!(T).ptr[0]) E))
                 static if (is(T:E[]))
                         alias E[] MakeDynamic;
                 else
                         alias T MakeDynamic;
         else
                 alias T MakeDynamic;
}


struct ArrayReverseIterator(Array) {
	MakeDynamic!(Array) array;

	alias elementType!(Array) Element;

	int opApply(int delegate(inout Element) dg) {
		for (int i = array.length-1; i >= 0; i--) {
			if (auto status = dg(array[i]))
				return status;
		}
		return 0;
	}
}

ArrayReverseIterator!(Array) reverseIterator(Array)(Array array) {
	ArrayReverseIterator!(Array) iter;
	iter.array = array;
	return iter;
}


Usage:

foreach(x; "abcd".reverseIterator())
	writefln("x = %s",x);

/Oskar
Jul 21 2006
prev sibling parent reply pragma <pragma_member pathlink.com> writes:
In article <e9ogh7$b1b$1 digitaldaemon.com>, kris says...

Ben Phillips wrote:

 While D's foreach is a much appreciated improvement over C++'s template
version,
 I feel there are still some ways in which it can be improved. I listed two
ideas
 below,




[snipped foreach proposal]

Both can be done right now:

1)  foreach (int i, Foo f; ManyFooInstance)
              if (i is 0)
                  ....

Or somthing like that.


Agreed.  The only variation that is missing is for AA's:

foreach(key, value; myAA){
}

Being able to get at the current iteration would be nice, but its certainly
sugar.  The only way I can see this single case being covered, is with a third
foreach arg, like so:

foreach(uint i,key,value; myAA){
}

.. as this makes a rather nice, logical extension to things IMO.  But I'm not
holding my breath for its inclusion into D. ;)

A variation would be to use a specialized iterator:

uint i;
foreach(key,value; myAA.forward(&i){
//i is incremented for each call of opApply()
}

.. or to iterate over the keys and lookup on each pass:

foreach(i,key; myAA.keys){
auto value = myAA[key];
}

I don't have a clue if doing an explicit lookup for 'value' is less efficent
than letting the compiler figure it out what 'value' is via the foreach()
expression. ::shrug::


2) foreach (int element; collection.reverse)
	    ...


some kind of Iterator construct. 


Also agreed.  There are simply too many kinds of variants for iteration, that
anything outside the standard "storage order" (analagous to "database order" for
SQL) iteration we have now should be left up to the developer.  Granted, having
a set of typical iterators like "reverse" would be a nice to have, but that's
probably best left to a library.


IIRC, someone already has a framework 
for such things? Was it Oskar?


I remember seeing a heavily templated Array library posted here, that Andrew
made a while back, could that be it?  It was drafted well before implicit
templates were available, so maybe its worth reviewing again.

- EricAnderton at yahoo
Jul 20 2006
parent reply Oskar Linde <oskar.lindeREM OVEgmail.com> writes:
pragma wrote:

 In article <e9ogh7$b1b$1 digitaldaemon.com>, kris says...

 Ben Phillips wrote:

 While D's foreach is a much appreciated improvement over C++'s template
version,
 I feel there are still some ways in which it can be improved. I listed two
ideas
 below,




 [snipped foreach proposal]
 Being able to get at the current iteration would be nice, but its certainly
 sugar.  The only way I can see this single case being covered, is with a third
 foreach arg, like so:
 
 foreach(uint i,key,value; myAA){
 }
 
 .. as this makes a rather nice, logical extension to things IMO.  But I'm not
 holding my breath for its inclusion into D. ;)
 
 A variation would be to use a specialized iterator:
 
 uint i;
 foreach(key,value; myAA.forward(&i){
 //i is incremented for each call of opApply()
 }


or why not:
uint i = -1;
foreach(key,value; myAA) {
	i++;
	...
}

:)


 IIRC, someone already has a framework 
 for such things? Was it Oskar?


 
 I remember seeing a heavily templated Array library posted here, that Andrew
 made a while back, could that be it?  It was drafted well before implicit
 templates were available, so maybe its worth reviewing again.


I also posted a templated array library suggestion earlier. It was 
drafted just after the inclusion of implicit function template 
instantiation and was designed to work with the current limited ifti 
support. The code would be a lot nicer if we got a more complete ifti 
support implemented though.

The implementation and (ugly) ddoc:

http://www.csc.kth.se/~ol/array.d
http://www.csc.kth.se/~ol/array.html

It doesn't currently contain any iterator functions. I've been toying 
with that a little though and have things like these working:

// pensionCosts is 4 % of the net salary of employees older than 28 yrs
// assume Employee[] employees;
double pensionCosts = 0;
foreach(e; employees.select((Employee e) { return e.age >= 28; }))
	pensionCosts += employee.salary * 0.04;

Reverse iterators: (I haven't currently implemented reverse utf 
conversions, like foreach(dchar c; "abcd"c.reverseIterator()). It is a 
lot of work and I'm not sure they are worth it.)

foreach(x; "abcd".reverseIterator())
	writefln("x = %s",x);

Python style ranges:

foreach(x; range(5))
	... x is 0,1,2,3,4
foreach(x; range(1, 6))
	... x is 1,2,3,4,5
foreach(x; range(1.5, 0, -0.25)) // bad idea to support doubles?
	... x is 1.5, 1.25, 1.0, 0.75, 0.5, 0.25

I've not found a good iterator design yet though. Only supporting 
foreach style iteration is a bit too limited.

I would also like array views to work as iterators and also support 
common array operations without having to generate temporary arrays.

Currently:

double pensionCosts = employees
          .filter((Employee e) { return e.age >= 28; })
          .map((Employee e) { return e.salary * 0.04; })
          .sum();

Will generate a temporary Employee[] array and a temporary double[] array.

Using array views one could get around the temporaries:

// give all employees over 55 a 3 % raise:

employees.select((Employee e) { return e.age >= 55; })
          .update((Employee e) { e.salary *= 1.03; });

//(No temporaries created.)

/Oskar
Jul 21 2006
next sibling parent reply "Andrei Khropov" <andkhropov nospam_mtu-net.ru> writes:
Oskar Linde wrote:


 Python style ranges:
 
 foreach(x; range(5))
 	... x is 0,1,2,3,4
 foreach(x; range(1, 6))
 	... x is 1,2,3,4,5
 foreach(x; range(1.5, 0, -0.25)) // bad idea to support doubles?
 	... x is 1.5, 1.25, 1.0, 0.75, 0.5, 0.25


Mmm, I would like slices to be first-class types and some synactic sugar:

-------------------------------------------------------
foreach(i; 0..5)
	... i is 0,1,2,3,4


slice   a   = 1..5;
int[5]  arr = 0..5; // same as [0,1,2,3,4]

foreach(x; arr[a])
	... x is arr[1], arr[2], ...   arr[4]
-------------------------------------------------------

But it's for D 1.0+ I think.


 
 I've not found a good iterator design yet though. Only supporting foreach
 style iteration is a bit too limited.
 
 I would also like array views to work as iterators and also support common
 array operations without having to generate temporary arrays.


Yes, this is a definitely needed functionality.


 
 Currently:
 
 double pensionCosts = employees
          .filter((Employee e) { return e.age >= 28; })
          .map((Employee e) { return e.salary * 0.04; })
          .sum();
 
 Will generate a temporary Employee[] array and a temporary double[] array.
 
 Using array views one could get around the temporaries:
 
 // give all employees over 55 a 3 % raise:
 
 employees.select((Employee e) { return e.age >= 55; })
          .update((Employee e) { e.salary *= 1.03; });
 
 //(No temporaries created.)
 
 /Oskar


Such things should naturally belong to DTL I think.

And I would also like to see type inference for delegate params by argument
type in parameter declaration, e.g. :

-------------------------------------------------------
employees.select( (e) { return e.age >= 55; })
         .update( (e) { e.salary *= 1.03; });
-------------------------------------------------------

e is inferred to be Employee in both cases because select and update are
declared as
-------------------------------------------------------
... select( delegate bool(Employee e) )
... update( delegate void(Employee e) )
-------------------------------------------------------

if select is overloaded with different types then this would be ambiguous of
course and we have to explicitly specify arg type.

-- 
AKhropov
Jul 21 2006
parent Oskar Linde <oskar.lindeREM OVEgmail.com> writes:
Andrei Khropov wrote:

 Oskar Linde wrote:
 

 Python style ranges:

 foreach(x; range(5))
 	... x is 0,1,2,3,4
 foreach(x; range(1, 6))
 	... x is 1,2,3,4,5
 foreach(x; range(1.5, 0, -0.25)) // bad idea to support doubles?
 	... x is 1.5, 1.25, 1.0, 0.75, 0.5, 0.25


 
 Mmm, I would like slices to be first-class types and some synactic sugar:
 
 -------------------------------------------------------
 foreach(i; 0..5)
 	... i is 0,1,2,3,4
 
 
 slice   a   = 1..5;
 int[5]  arr = 0..5; // same as [0,1,2,3,4]
 
 foreach(x; arr[a])
 	... x is arr[1], arr[2], ...   arr[4]
 -------------------------------------------------------
 
 But it's for D 1.0+ I think.


Yes. A slice type would be extremely nice, and would also be needed to 
support slicing of user defined multidimensional arrays.

I would love to see

opSlice(int start, int end)

replaced with:

opIndex(Range r);

which would allow multidimensional mixed indexing and slicing like:

arr[a..b, c, d..e, f..g, h, i];

via some template hackery. This could be implemented without breaking 
existing code by only calling opIndex(Range r) if no opSlice exists, so 
there is no rush to get this before D 1.0.


 I've not found a good iterator design yet though. Only supporting foreach
 style iteration is a bit too limited.

 I would also like array views to work as iterators and also support common
 array operations without having to generate temporary arrays.


 
 Yes, this is a definitely needed functionality.
 

 Currently:

 double pensionCosts = employees
          .filter((Employee e) { return e.age >= 28; })
          .map((Employee e) { return e.salary * 0.04; })
          .sum();

 Will generate a temporary Employee[] array and a temporary double[] array.

 Using array views one could get around the temporaries:

 // give all employees over 55 a 3 % raise:

 employees.select((Employee e) { return e.age >= 55; })
          .update((Employee e) { e.salary *= 1.03; });

 //(No temporaries created.)

 /Oskar


 
 Such things should naturally belong to DTL I think.


Yeah. It's a pity D is so UDT hostile.


 And I would also like to see type inference for delegate params by argument
 type in parameter declaration, e.g. :
 
 -------------------------------------------------------
 employees.select( (e) { return e.age >= 55; })
          .update( (e) { e.salary *= 1.03; });
 -------------------------------------------------------
 
 e is inferred to be Employee in both cases because select and update are
 declared as
 -------------------------------------------------------
 ... select( delegate bool(Employee e) )
 ... update( delegate void(Employee e) )
 -------------------------------------------------------
 
 if select is overloaded with different types then this would be ambiguous of
 course and we have to explicitly specify arg type.


Yes, This makes D's delegates very close to Ruby's:
{|e| e.salary = e.salary * 1.03 }.
Exactly the same process of type inference exists already with foreach:

foreach(x; myObj)
// typeof(x) is inferred from the argument type of myObj.opApply()

/Oskar
Jul 21 2006
prev sibling next sibling parent reply pragma <pragma_member pathlink.com> writes:
In article <e9qe1i$1m5c$3 digitaldaemon.com>, Oskar Linde says...

pragma wrote:

 In article <e9ogh7$b1b$1 digitaldaemon.com>, kris says...

 Ben Phillips wrote:

 While D's foreach is a much appreciated improvement over C++'s template
version,
 I feel there are still some ways in which it can be improved. I listed two
ideas
 below,




 [snipped foreach proposal]
 Being able to get at the current iteration would be nice, but its certainly
 sugar.  The only way I can see this single case being covered, is with a third
 foreach arg, like so:
 
 foreach(uint i,key,value; myAA){
 }
 
 .. as this makes a rather nice, logical extension to things IMO.  But I'm not
 holding my breath for its inclusion into D. ;)
 
 A variation would be to use a specialized iterator:
 
 uint i;
 foreach(key,value; myAA.forward(&i){
 //i is incremented for each call of opApply()
 }


or why not:
uint i = -1;
foreach(key,value; myAA) {
	i++;
	...
}

:)


Oh sure.  If you want to do things the old-fashioned way... might as well use
'goto' while you're at it. ;p



 IIRC, someone already has a framework 
 for such things? Was it Oskar?


 
 I remember seeing a heavily templated Array library posted here, that Andrew
 made a while back, could that be it?  It was drafted well before implicit
 templates were available, so maybe its worth reviewing again.


I also posted a templated array library suggestion earlier. It was 
drafted just after the inclusion of implicit function template 
instantiation and was designed to work with the current limited ifti 
support. The code would be a lot nicer if we got a more complete ifti 
support implemented though.

The implementation and (ugly) ddoc:

http://www.csc.kth.se/~ol/array.d
http://www.csc.kth.se/~ol/array.html

It doesn't currently contain any iterator functions. I've been toying 
with that a little though and have things like these working:

// pensionCosts is 4 % of the net salary of employees older than 28 yrs
// assume Employee[] employees;
double pensionCosts = 0;
foreach(e; employees.select((Employee e) { return e.age >= 28; }))
	pensionCosts += employee.salary * 0.04;

Reverse iterators: (I haven't currently implemented reverse utf 
conversions, like foreach(dchar c; "abcd"c.reverseIterator()). It is a 
lot of work and I'm not sure they are worth it.)

foreach(x; "abcd".reverseIterator())
	writefln("x = %s",x);

Python style ranges:

foreach(x; range(5))
	... x is 0,1,2,3,4
foreach(x; range(1, 6))
	... x is 1,2,3,4,5
foreach(x; range(1.5, 0, -0.25)) // bad idea to support doubles?
	... x is 1.5, 1.25, 1.0, 0.75, 0.5, 0.25

I've not found a good iterator design yet though. Only supporting 
foreach style iteration is a bit too limited.

I would also like array views to work as iterators and also support 
common array operations without having to generate temporary arrays.

Currently:

double pensionCosts = employees
          .filter((Employee e) { return e.age >= 28; })
          .map((Employee e) { return e.salary * 0.04; })
          .sum();

Will generate a temporary Employee[] array and a temporary double[] array.

Using array views one could get around the temporaries:

// give all employees over 55 a 3 % raise:

employees.select((Employee e) { return e.age >= 55; })
          .update((Employee e) { e.salary *= 1.03; });

//(No temporaries created.)

/Oskar


IMO, I think it's all *very* worth it.  It kind of fills this gap between what
phobos gives us and what STL's iterators and algorithms has to offer. 

I think that if you have an 'array view' widget/template that supports
opApply(), length(), and opIndex() for all the view types you return, then
you're pretty much set.  That will cover both the random-access and
sequential-access modes that arrays already enjoy.  That way, you can support
for() iteration as well, and still not need any temporaries. ;)

As for ranges, I wouldn't throw those out.  Ruby also has them, and thanks to
Rails, prototype.js now supports them now too.  I'd be willing to bet that
someone will come along to the DNG and say "Where's the range support?!" - and
we can all just point them to your library.  

I'd imagine ranges being an extension of your 'array view' concept.

I think something like the following would be workable:

/**/ struct ReverseArrayView(ArrayT){
/**/   ElemType!(ArrayT) opIndex(uint idx);
/**/   ArrayT opSlice(uint start,uint end); //creates temporary or another view?
/**/   int opApply(int delegate(inout ElemType!(ArrayT)) dg);
/**/   uint length();
/**/   static opCall(ArrayT arr);
/**/ }
/**/
/**/ template reverse(ArrayT){
/**/   ReverseArrayView!(ArrayT) reverse(ArrayT arr){ 
/**/     return ReverseArrayView!(ArrayT)(arr);
/**/   }
/**/ }

The trick is to get ArrayView instances to jive with your templates like
ElemType!() such that they get treated like first-class arrays where possible. I
think static if() could be used to check for a static property on ArrayViews
like "isArrayView=true", such that standard arrays will fail the test.

- EricAnderton at yahoo
Jul 21 2006
parent Oskar Linde <oskar.lindeREM OVEgmail.com> writes:
pragma wrote:

 In article <e9qe1i$1m5c$3 digitaldaemon.com>, Oskar Linde says...

 I've not found a good iterator design yet though. Only supporting 
 foreach style iteration is a bit too limited.

 I would also like array views to work as iterators and also support 
 common array operations without having to generate temporary arrays.

 Currently:

 double pensionCosts = employees
          .filter((Employee e) { return e.age >= 28; })
          .map((Employee e) { return e.salary * 0.04; })
          .sum();

 Will generate a temporary Employee[] array and a temporary double[] array.

 Using array views one could get around the temporaries:

 // give all employees over 55 a 3 % raise:

 employees.select((Employee e) { return e.age >= 55; })
          .update((Employee e) { e.salary *= 1.03; });

 //(No temporaries created.)

 /Oskar


 
 IMO, I think it's all *very* worth it.  It kind of fills this gap between what
 phobos gives us and what STL's iterators and algorithms has to offer. 
 
 I think that if you have an 'array view' widget/template that supports
 opApply(), length(), and opIndex() for all the view types you return, then
 you're pretty much set.  That will cover both the random-access and
 sequential-access modes that arrays already enjoy.  That way, you can support
 for() iteration as well, and still not need any temporaries. ;)


Yes, that is probably the right way do to it. If the views support the 
same operations as arrays do, they will be usable just as arrays, and 
views-of-views and similar would work transparently.


 As for ranges, I wouldn't throw those out.  Ruby also has them, and thanks to
 Rails, prototype.js now supports them now too.  I'd be willing to bet that
 someone will come along to the DNG and say "Where's the range support?!" - and
 we can all just point them to your library.  
 
 I'd imagine ranges being an extension of your 'array view' concept.


Yes, ideally, a range should work just like a (read only) array, but 
without allocating any memory. Silly example:

range(100).select((int x) { return x % 5 == 0 }).sum()
//returns 950


 I think something like the following would be workable:
 
 /**/ struct ReverseArrayView(ArrayT){
 /**/   ElemType!(ArrayT) opIndex(uint idx);
 /**/   ArrayT opSlice(uint start,uint end); //creates temporary or another
view?
 /**/   int opApply(int delegate(inout ElemType!(ArrayT)) dg);
 /**/   uint length();
 /**/   static opCall(ArrayT arr);
 /**/ }
 /**/
 /**/ template reverse(ArrayT){
 /**/   ReverseArrayView!(ArrayT) reverse(ArrayT arr){ 
 /**/     return ReverseArrayView!(ArrayT)(arr);
 /**/   }
 /**/ }
 
 The trick is to get ArrayView instances to jive with your templates like
 ElemType!() such that they get treated like first-class arrays where possible.
I
 think static if() could be used to check for a static property on ArrayViews
 like "isArrayView=true", such that standard arrays will fail the test.


Yes, that is definitely doable. The only problem is that there is no way 
to "inject" methods for anything but built-in arrays. I'd have to add 
wrappers to all views for all functions (a mixin will probably be 
enough), or live with calling them as ordinary functions:

update(select(employees,
              (Employee e) { return e.age >= 55; }),
       (Employee e) { e.salary *= 1.03; });

instead of:

employees.select((Employee e) { return e.age >= 55; })
          .update((Employee e) { e.salary *= 1.03; });

It would be nice to have a way to inject methods/define extension 
methods for other types than arrays. Maybe something like what has been 
proposed before:

int squared(this int x) { return x*x; }
and called:
5.squared();

Just add ifti template support to this and the array functions would be 
applicable as methods of any type that implements the needed iterator 
semantics (opApply or opIndex and length()).

/Oskar
Jul 21 2006
prev sibling parent Kirk McDonald <kirklin.mcdonald gmail.com> writes:
Oskar Linde wrote:

 Python style ranges:
 
 foreach(x; range(5))
     ... x is 0,1,2,3,4
 foreach(x; range(1, 6))
     ... x is 1,2,3,4,5
 foreach(x; range(1.5, 0, -0.25)) // bad idea to support doubles?
     ... x is 1.5, 1.25, 1.0, 0.75, 0.5, 0.25
 


For what it's worth, writing a Python 'xrange'-style construct in D 
isn't that hard. I've posted one previously to the group:

http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/38808

Some follow-ups to that post had templated versions.

-- 
Kirk McDonald
Pyd: Wrapping Python with D
http://dsource.org/projects/pyd/wiki
Jul 21 2006