www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Empowering foreach with multiple aggregates

reply "Janice Caron" <caron800 googlemail.com> writes:
There are some things that iterators can do that foreach can't. I
/don't/ want to add iterators to D. Instead, I'd like to see foreach
enhanced, so that it can do the things it currently can't.

Here's my first example.

 void tweak(int[] dst, int[] src)
 {
     assert(dst.length == src.length);
     for (int i=0; i<dst.length; ++i)
     {
         dst[i] = src[i] + 1;
     }
 }

How much nicer it would be to be able to say

 void tweak(int[] dst, int[] src)
 {
     foreach (d; dst; s; src)
     {
         d = s + 1;
     }
 }

The ability to walk through two collections simultaneously is such a
common thing to want to do that I'm suprised there's no language
support for it. Of course, it needs some extra code if dst.length and
src.length are /not/ equal, but that would also be true of the
iterator way. Using the foreach way, you'd end up with something like:

 void tweak(int[] dst, int[] src)
 {
     int len = dst.length < src.length ? dst.length : src.length;
     foreach (d; dst[0..len]; s; src[0..len])
     {
         d = s + 1;
     }
     if (dst.length > src.length) foreach(d; dst[len..$])
     {
         d = 1;
     }
 }

There might be some array operations planned that will do something
similar, but the point is that not all collections are arrays. You
might want to iterate through any arbitrary collections, and that's
where iterators show their strength. Why not let foreach have that
additional strength?
Sep 09 2007
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Janice Caron wrote:

 [snip]
 
  void tweak(int[] dst, int[] src)
  {
      foreach (d; dst; s; src)
      {
          d = s + 1;
      }
  }


I believe Walter and Andrei have talked about this before.  The syntax
they proposed at the time was something like:

foreach(ref d ; dst)(s ; src)
{
    d = s + 1
}

I think the main problem with doing this is related to how foreach is
actually implemented.  If we assume that dst and src are actually user
types, then this loop:

foreach(ref T d ; dst)
  d = 1;

Becomes:

dst.opApply((ref T d) {
  d = 1;
  return 0;
});

This is a problem if you want to do iteration over multiple aggregates;
you'd essentially need to call two functions simultaneously, then have
those call into a third function passing half of the arguments each.

Adding this to D would probably require completely rethinking how custom
aggregates are created, which isn't necessarily a bad thing, but
certainly a difficult thing.

	-- Daniel
Sep 09 2007
next sibling parent "Janice Caron" <caron800 googlemail.com> writes:
 foreach(ref T d ; dst)
   d = 1;

 Becomes:

 dst.opApply((ref T d) {
   d = 1;
   return 0;
 });

 This is a problem if you want to do iteration over multiple aggregates;


Yes, I see. I had not realised that. Thanks.

I guess you could do it by creating some sort of hybrid collection -
but I don't know if that could be done at a language level (and if it
can't there'd be no point). By this I mean, given a collection C c,
and collection D d, where (c.length == d.length), generate a third
collection A a such that a[n] is a Tuple!(ref c[n], d[n]). Then you
could do:

 foreach(a; AggregatePair!(dst, src))
 {
     a[0] = a[1] + 1;
 }

But I don't have enough experience with Tuples to know if that's even
possible. I mean, AggregatePair!'s opApply() method would have to be
really clever.




 you'd essentially need to call two functions simultaneously, then have
 those call into a third function passing half of the arguments each.


Could this AggregatePair! idea do that? Or is that just too not possible?



 Adding this to D would probably require completely rethinking how custom
 aggregates are created, which isn't necessarily a bad thing, but
 certainly a difficult thing.


Oh well!
C'est la vie.
Sep 09 2007
prev sibling parent BCS <ao pathlink.com> writes:
Reply to Daniel,


 This is a problem if you want to do iteration over multiple
 aggregates; you'd essentially need to call two functions
 simultaneously, then have those call into a third function passing
 half of the arguments each.
 


stack threads?


 
 -- Daniel
Sep 09 2007
prev sibling parent reply Downs <default_357-line yahoo.de> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

This works, right now, but it requires a kind of abuse of the foreach
callback system. :D

xt4100 ~ $ cat foo2.d && gdc foo2.d -o foo2 && ./foo2
import std.stdio;

struct pairiter(T, U) {
  T[] first; U[] second;
  int opApply(int delegate(ref T, ref U) dg) {
    for (int i=0; i<first.length; ++i) {
      auto res=dg(first[i], second[i]);
      if (res) return res;
    }
    return 0;
  }
}

pairiter!(T, U) pair(T, U)(T[] a, U[] b) {
  assert(a.length==b.length);
  return pairiter!(T, U)(a, b);
}

void main() {
  char[] dst=new char[6]; char[] src="foobar";
  foreach (ref d, ref s; pair(dst, src)) {
    d=s+1;
  }
  writefln(dst);
}
gppcbs


Here's a version that works with arbitrary foreachable things (uses
scrapple.tools.stackthreads):

xt4100 ~ $ cat foo2.d && rebuild foo2 && ./foo2
module foo2;
import std.stdio;
import tools.stackthreads;

template isArray(T) { const bool isArray=false; }
template isArray(T: T[]) { const bool isArray=true; }

import std.traits;
template IT(T) { // itertype
  static if(isArray!(T)) alias typeof(T[0]) IT;
  else alias ParameterTypeTuple!(
    ParameterTypeTuple!(
      typeof(&T.init.opApply)
    )[0]
  )[$-1] IT;
}

struct pairiter(T, U) {
  T first; U second;
  int opApply(int delegate(ref IT!(T), ref IT!(U)) dg) {
    auto gen=new class Generator!(IT!(T)*) { void generate() {
      foreach (ref elem; first) yield (&elem);
    } };
    foreach (elem2; second) {
      auto elem1=gen();
      auto res=dg(*elem1, elem2);
      if (res) return res;
    }
    return 0;
  }
}

pairiter!(T, U) pair(T, U)(T a, U b) {
  return pairiter!(T, U)(a, b);
}

struct strng {
  string st;
  int opApply(int delegate(ref char) dg) {
    foreach (ref ch; st) {
      auto res=dg(ch); if (res) return res;
    }
    return 0;
  }
}

void main() {
  char[] dst=new char[6]; char[] src="foobar";
  foreach (ref d, ref s; pair(dst, strng(src))) {
    d=s+1;
  }
  writefln(dst);
}
gppcbs

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG49m9pEPJRr05fBERApmEAJ9KwyXbsEOZ5plZwDqjw+BXvNU/8gCbB2b9
37UFe1OLUIalwF4gSNlfV0k=
=eYwJ
-----END PGP SIGNATURE-----
Sep 09 2007
parent reply "Janice Caron" <caron800 googlemail.com> writes:
Wow! You're good!




   static if(isArray!(T)) alias typeof(T[0]) IT;
   else alias ParameterTypeTuple!(
     ParameterTypeTuple!(
       typeof(&T.init.opApply)
     )[0]
   )[$-1] IT;


Sorry to jump threads here, but this is a really good example of why
alias dst=src; would be a good thing. It took me a long time and a lot
of counting parantheses to figure out that IT was being defined!

Anyway - that is excellent.

I guess the next question would be, can this trick be built into the
language in an easy-to-use way. Or in Phobos?
Sep 09 2007
parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Janice Caron wrote:

 Wow! You're good!


He's downs.


   static if(isArray!(T)) alias typeof(T[0]) IT;
   else alias ParameterTypeTuple!(
     ParameterTypeTuple!(
       typeof(&T.init.opApply)
     )[0]
   )[$-1] IT;


 
 Sorry to jump threads here, but this is a really good example of why
 alias dst=src; would be a good thing. It took me a long time and a lot
 of counting parantheses to figure out that IT was being defined!
 
 Anyway - that is excellent.
 
 I guess the next question would be, can this trick be built into the
 language in an easy-to-use way. Or in Phobos?


I suspect not unless Walter adds stackthreads/coroutines to it first.

Then again, since it's so clean to use:

  foreach (ref d, ref s; pair(dst, strng(src)))

It seems fine as a library method.

P.S. to downs: did you have that one lying around, or is this old work? :P

	-- Daniel
Sep 09 2007
parent Downs <default_357-line yahoo.de> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Daniel Keep wrote:

 
 Janice Caron wrote:

 Wow! You're good!


 
 He's downs.
 


Hehe .. thanks :)


   static if(isArray!(T)) alias typeof(T[0]) IT;
   else alias ParameterTypeTuple!(
     ParameterTypeTuple!(
       typeof(&T.init.opApply)
     )[0]
   )[$-1] IT;


 Sorry to jump threads here, but this is a really good example of why
 alias dst=src; would be a good thing. It took me a long time and a lot
 of counting parantheses to figure out that IT was being defined!

 Anyway - that is excellent.

 I guess the next question would be, can this trick be built into the
 language in an easy-to-use way. Or in Phobos?


 
 I suspect not unless Walter adds stackthreads/coroutines to it first.
 
 Then again, since it's so clean to use:
 
   foreach (ref d, ref s; pair(dst, strng(src)))
 
 It seems fine as a library method.
 
 P.S. to downs: did you have that one lying around, or is this old work? :P
 
 	-- Daniel


The tools.stackthreads impl is relatively new (I wrote it a few days ago
with lots of help from #d), though there's an older (and better) one in
the stackthreads lib on http://assertfalse.com/projects.shtml (swhere I
got the idea).
The pair function was whipped up in about half an hour, most of it spent
hunting template bugs. I'm familiar with the pattern of functions that
return structs with extended behavior, because I use it a lot in
tools.iter. Also, I was looking for a good application of
tools.stackthreads for a while now :)

The problem with pair is that it's probably not extensible to triplets,
and there's no stackthreads in phobos. Maybe Walter could add them?
 ..
Hah. I wish. :)
 --downs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG4+vGpEPJRr05fBERAvnmAJ965+lpuWRluXob0OrqzqKEZSPbTgCeN7qE
fU2g+SNCbsIaMaZ1PmXLyKM=
=i7YA
-----END PGP SIGNATURE-----
Sep 09 2007