www.digitalmars.com         C & C++   DMDScript  

D - DMD 0.71 release

reply "Walter" <walter digitalmars.com> writes:
This includes the new foreach construct, which is the building block for
container classes and iterations. I'll work on the bug reports next!

http://www.digitalmars.com/d/changelog.html
Sep 03 2003
next sibling parent reply And <And_member pathlink.com> writes:
In article <bj5pa5$156u$1 digitaldaemon.com>, Walter says...

This includes the new foreach construct, which is the building block for
container classes and iterations. I'll work on the bug reports next!

http://www.digitalmars.com/d/changelog.html


Fixed bug with linux file.read() and file.append() functions.

I believe you mean write and append.

Ant
Sep 03 2003
parent "Walter" <walter digitalmars.com> writes:
"And" <And_member pathlink.com> wrote in message
news:bj6ds0$212l$1 digitaldaemon.com...

 I believe you mean write and append.


Yes, thanks.
Sep 03 2003
prev sibling next sibling parent reply Mike Wynn <mike l8night.co.uk> writes:
Walter wrote:

 This includes the new foreach construct, which is the building block for
 container classes and iterations. I'll work on the bug reports next!
 
 http://www.digitalmars.com/d/changelog.html
 


while I like the inclusion of foreach,
would it not be a little nicer if it was

foreach( <type> <name> (in|inout) <expr>) <statement>
i.e.

void func( char[] str ) {
	foreach( char c in str ) {
		// modifying c does not effect str[i]
	}
	foreach( char c inout str ) {
		modifying c modifies str[i];
	}
}

the apply member:
why is it not
void apply( bit delegate( in T ) ) { .. } // foreach in
void apply( bit delegate( inout T ) ) { .. } // foreach inout.

the apply delegate returns true if the iteration should continue, I 
don't see the need for apply to return 0 or for the delegate to return 
an int

implemented as
class canForeach {
	char[] str;
	void apply( bit delegate( in char ) applyfunc ) {
		for( int i = 0; i < str.length; i++ ) {	
			if ( !applyfunc( str[i] ) ) { return; }
		}
	}
}
Sep 04 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Mike Wynn" <mike l8night.co.uk> wrote in message
news:bj7gn4$gtb$1 digitaldaemon.com...

 while I like the inclusion of foreach,
 would it not be a little nicer if it was

 foreach( <type> <name> (in|inout) <expr>) <statement>
 i.e.

 void func( char[] str ) {
 foreach( char c in str ) {
 // modifying c does not effect str[i]
 }
 foreach( char c inout str ) {
 modifying c modifies str[i];
 }
 }


I thought about making it compatible with function parameter syntax that
uses inout.


 the apply member:
 why is it not
 void apply( bit delegate( in T ) ) { .. } // foreach in
 void apply( bit delegate( inout T ) ) { .. } // foreach inout.

 the apply delegate returns true if the iteration should continue, I
 don't see the need for apply to return 0 or for the delegate to return
 an int

 implemented as
 class canForeach {
 char[] str;
 void apply( bit delegate( in char ) applyfunc ) {
 for( int i = 0; i < str.length; i++ ) {
 if ( !applyfunc( str[i] ) ) { return; }
 }
 }
 }


The need for it to return an int rather than bit is because of the compiler
'magic' involved with making this work. The foreach body is transformed into
a nested function (!). The return value is needed to handle the various ways
of exiting the foreach body, such as goto, break label, return, etc. The
foreach itself is replaced with a switch statement calling the apply
function. It sounds a bit hairy, but that's all hidden from the programmer.
The neato thing is there is no need to write state-preserving iterators, a
rather tricky thing to get right for containers that are recursive or
otherwise need state-preserving iterators.
Sep 04 2003
next sibling parent reply "Sean L. Palmer" <palmer.sean verizon.net> writes:
"Walter" <walter digitalmars.com> wrote in message
news:bj84jb$1e41$1 digitaldaemon.com...


 The need for it to return an int rather than bit is because of the


compiler

 'magic' involved with making this work. The foreach body is transformed


into

 a nested function (!). The return value is needed to handle the various


ways

 of exiting the foreach body, such as goto, break label, return, etc. The
 foreach itself is replaced with a switch statement calling the apply
 function. It sounds a bit hairy, but that's all hidden from the


programmer.

 The neato thing is there is no need to write state-preserving iterators, a
 rather tricky thing to get right for containers that are recursive or
 otherwise need state-preserving iterators.


This is pretty cool, Walter.  It sounds to me as if you're halfway to making
general inplace anonymous functions!

Will it inline these calls to the nested function if everything is final?

And you can, say, break out of a foreach?  This (behind the scenes) returns
a certain value from the nested function?

One nice thing about having actual iterators is that you can form a range
over which to iterate.  It doesn't sound like this setup allows that... it
just allows sequentially iterating from the beginning until the Apply()
function decides to return a "stop" value or it hits the end, correct?
Granted, 90% of the cases would want to iterate the entire container, but in
the other 10% it may prove a bit constrictive.

Combined with not being able to overload operator [] means it's very hard to
make a good usable array class that looks and feels like a builtin.
Everybody will do it differently.  Usually with array-like things you can
use operator [].  So by providing operator [] and a size() function you can
emulate an array, and by providing the iterator functions you can mimic a
list or generic forward-iterating container.  foreach would try to use the
iterators if available and if not, default to indexing 0 .. size-1 and using
operator []?

How is foreach going to work with user-defined containers?

Sean
Sep 05 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <palmer.sean verizon.net> wrote in message
news:bj9eod$aps$1 digitaldaemon.com...

 Will it inline these calls to the nested function if everything is final?


No, since it is done as a pointer to a function.


 And you can, say, break out of a foreach?  This (behind the scenes)


returns

 a certain value from the nested function?


Yes.


 One nice thing about having actual iterators is that you can form a range
 over which to iterate.  It doesn't sound like this setup allows that... it
 just allows sequentially iterating from the beginning until the Apply()
 function decides to return a "stop" value or it hits the end, correct?
 Granted, 90% of the cases would want to iterate the entire container, but


in

 the other 10% it may prove a bit constrictive.


The secret to modifying this for specific cases is to override/modify the
apply(). Here's a little template technique to run an array in reverse:


template ReverseIterator(T : T[])
{
    class RI
    {
 T[] a;

 this(T[] a)
 {
     this.a = a;
 }

 int apply(int delegate(inout T v) dg)
 {   int result;

     for (int i = a.length; i; i--)
     { result = dg(a[i - 1]);
  if (result)
      break;
     }
     return result;
 }
    }
}

void main()
{
    int[3] a;

    a[0] = 1;
    a[1] = 2;
    a[2] = 3;

    foreach (int i; new instance ReverseIterator(int[]).RI(a))
    {
 printf("i = %d\n", i);
    }
}



 Combined with not being able to overload operator [] means it's very hard


to

 make a good usable array class that looks and feels like a builtin.


I need to provide [] overloads, no question about it.


 Everybody will do it differently.  Usually with array-like things you can
 use operator [].  So by providing operator [] and a size() function you


can

 emulate an array, and by providing the iterator functions you can mimic a
 list or generic forward-iterating container.  foreach would try to use the
 iterators if available and if not, default to indexing 0 .. size-1 and


using

 operator []?
 How is foreach going to work with user-defined containers?


All the container needs is an apply() function - no need for iterators. I
have never particularly liked the iterator paradigm, as it just involves
lots of cruft to write. Things get really ugly when trying to write an
iterator for a recursive structure, articles on iterators tend to ignore

trying to get around this by using coroutines, but the complexities of that
are just too awful. There hasn't been a foreach in D before because I just
could never figure out a solution that I liked, and then finally had a
brainwave about using a magic nested function.

The more I use nested functions, it turns out, the more useful they become
<g>.
Sep 05 2003
parent reply "Sean L. Palmer" <palmer.sean verizon.net> writes:
"Walter" <walter digitalmars.com> wrote in message
news:bj9ha5$eff$1 digitaldaemon.com...

 "Sean L. Palmer" <palmer.sean verizon.net> wrote in message
 news:bj9eod$aps$1 digitaldaemon.com...

 Will it inline these calls to the nested function if everything is




final?

 No, since it is done as a pointer to a function.


Can you make it a referece to a function instead so future compilers can
inline it if they want?  This is the great thing about STL's for_each.


 One nice thing about having actual iterators is that you can form a




range

 over which to iterate.  It doesn't sound like this setup allows that...




it

 just allows sequentially iterating from the beginning until the Apply()
 function decides to return a "stop" value or it hits the end, correct?
 Granted, 90% of the cases would want to iterate the entire container,




but in

 the other 10% it may prove a bit constrictive.


 The secret to modifying this for specific cases is to override/modify the
 apply(). Here's a little template technique to run an array in reverse:


So to iterate over a range you can just make a slice and iterate that, I
guess.  ;)


 All the container needs is an apply() function - no need for iterators. I
 have never particularly liked the iterator paradigm, as it just involves
 lots of cruft to write. Things get really ugly when trying to write an
 iterator for a recursive structure, articles on iterators tend to ignore

 trying to get around this by using coroutines, but the complexities of


that

 are just too awful. There hasn't been a foreach in D before because I just
 could never figure out a solution that I liked, and then finally had a
 brainwave about using a magic nested function.


I'm glad you figured it out!  It seems like a pretty good solution.


 The more I use nested functions, it turns out, the more useful they become
 <g>.


Awesome!

One thing you might think about is the name apply().  Most functional
languages would call this feature "map".  With D having associative arrays
built in, there's not much need for STL-style map container, so this may be
an ok name.

Sean
Sep 06 2003
parent "Daniel Yokomiso" <daniel_yokomiso yahoo.com.br> writes:
"Sean L. Palmer" <palmer.sean verizon.net> escreveu na mensagem
news:bjcuo2$2c19$1 digitaldaemon.com...

[snip]


 One thing you might think about is the name apply().  Most functional
 languages would call this feature "map".  With D having associative arrays
 built in, there's not much need for STL-style map container, so this may


be

 an ok name.

 Sean


This isn't correct. In functional languages map is a function that transform
one kind of collection in another. As a foreach can do several different
things (e.g. sum the elements) it's just a matter of finding the correct
name. For example in DTL I defined both a foreach function (a command to be
executed for each item) and an apply function (a transformation to be
applied for each array entry):


public void foreach(T[] array, void function(T) operation);
public void apply(inout T[] array, T function(T) operation);


IMO we should call this operation "forEach", but I'm ok with apply (I'll
change DTL "apply" to "transform").

    Best regards,
    Daniel Yokomiso.

"Boss: "D?" What happened to C? Or that other one, C++, that you're always
harping about?
Me: Well, D is a further refinement of C++, and ...
Boss: Weren't you just telling me that "J" language was supposed to be the
next big thing? Isn't J further along than D?
Me: Yes and no ...

[suddenly, a shot rang out]"
 - sunwukong at /.


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.514 / Virus Database: 312 - Release Date: 29/8/2003
Sep 06 2003
prev sibling parent reply Mike Wynn <mike l8night.co.uk> writes:
Walter wrote:

 
 The need for it to return an int rather than bit is because of the compiler
 'magic' involved with making this work. The foreach body is transformed into
 a nested function (!). The return value is needed to handle the various ways
 of exiting the foreach body, such as goto, break label, return, etc. The
 foreach itself is replaced with a switch statement calling the apply
 function. It sounds a bit hairy, but that's all hidden from the programmer.
 The neato thing is there is no need to write state-preserving iterators, a
 rather tricky thing to get right for containers that are recursive or
 otherwise need state-preserving iterators.
 


why not use a local like this ..

void func( SomeObj ar ) {
outer: {
   ......
	foreach( int a; ar ) {
		printf( "%d ", a );
		if ( a == '\n' ) { break outer; }
	}
   } // outer loop
}
... becomes ...
void func( SomeObj ar ) {
  outer: {
      .....
	{ // foreach
		int __iter_000_state = 0;
		bit irf( int a ) {
			printf( "%d", a );
			if ( a == '\n' ) {
				__iter_000_state = BREAK_000;
			// update parent stack frame
				return false;
			}
			return true;
		} // end iterator func
		ar.apply( irf );
		switch( __iter_000_state ) {
		default: break;
		BREAK_000: break outer;
		}
	}
    } // outer
}

simple break is just return false (stop iterating)
simple continue is just return true (stop current iteration, but continue)

I agree this is a little slower, BUT removes the "magic" from the 
programmers view, I feel the int magic although a little faster does 
expose the inner workings a little too much and is a bug waiting to happen.
Sep 05 2003
next sibling parent reply "Walter" <walter digitalmars.com> writes:
"Mike Wynn" <mike l8night.co.uk> wrote in message
news:bjam8u$21if$1 digitaldaemon.com...

 I agree this is a little slower, BUT removes the "magic" from the
 programmers view, I feel the int magic although a little faster does
 expose the inner workings a little too much and is a bug waiting to


happen.

That would work just as well, but I don't think there's any real difference.
Sep 05 2003
parent reply Mike Wynn <mike l8night.co.uk> writes:
Walter wrote:

 "Mike Wynn" <mike l8night.co.uk> wrote in message
 news:bjam8u$21if$1 digitaldaemon.com...
 

I agree this is a little slower, BUT removes the "magic" from the
programmers view, I feel the int magic although a little faster does
expose the inner workings a little too much and is a bug waiting to


 
 happen.
 
 That would work just as well, but I don't think there's any real difference.
 


the inner workings of the block->nested_function/delgate is hidden from 
the programmer.
the apply method can not cause unexpected errors by returning a silly value.
to experienced D/C developers maybe there is no difference, but to those 
new to D, I think the need to keep the return value from the delgate and 
pass it back may seem odd, (and who reads the manual?)

to me the difference is that its more robust, no "you need to know this 
to use X", and the types used reflect their use explicitly.
Sep 05 2003
parent reply "Walter" <walter digitalmars.com> writes:
"Mike Wynn" <mike l8night.co.uk> wrote in message
news:bjb0en$2gk9$1 digitaldaemon.com...

 Walter wrote:

 "Mike Wynn" <mike l8night.co.uk> wrote in message
 news:bjam8u$21if$1 digitaldaemon.com...

I agree this is a little slower, BUT removes the "magic" from the
programmers view, I feel the int magic although a little faster does
expose the inner workings a little too much and is a bug waiting to


 happen.
 That would work just as well, but I don't think there's any real




difference.

 the inner workings of the block->nested_function/delgate is hidden from
 the programmer.
 the apply method can not cause unexpected errors by returning a silly


value.

 to experienced D/C developers maybe there is no difference, but to those
 new to D, I think the need to keep the return value from the delgate and
 pass it back may seem odd, (and who reads the manual?)
 to me the difference is that its more robust, no "you need to know this
 to use X", and the types used reflect their use explicitly.


Ok, I see your point, although am not convinced. If experience shows you to
be correct, I'll change it. I'm also a bit reluctant to take the speed
sacrifice, even though it is only a 3 or 4 instructions. This needs to be
speed competitive with C++.
Sep 06 2003
parent Mike Wynn <mike l8night.co.uk> writes:
Walter wrote:

 "Mike Wynn" <mike l8night.co.uk> wrote in message
 news:bjb0en$2gk9$1 digitaldaemon.com...
 

the inner workings of the block->nested_function/delgate is hidden from
the programmer.
the apply method can not cause unexpected errors by returning a silly


 
 value.
 

to experienced D/C developers maybe there is no difference, but to those
new to D, I think the need to keep the return value from the delgate and
pass it back may seem odd, (and who reads the manual?)
to me the difference is that its more robust, no "you need to know this
to use X", and the types used reflect their use explicitly.


 
 
 Ok, I see your point, although am not convinced. If experience shows you to
 be correct, I'll change it. I'm also a bit reluctant to take the speed
 sacrifice, even though it is only a 3 or 4 instructions. This needs to be
 speed competitive with C++.
 


I did think about that, I'm not convinced that it will be slower, EAX 
will be free as a temp until the return value is assign, instead of it 
having to retain the value or be spilled to the stack.

on other platforms the same may be true too, ARM for instance uses 
R0..R3 as the first 4 params and also as the return/temp regs, (I think 
MIPS is the same) which would force you to save R0 on each loop.

I know you only need to save it if its not 0, in which case you should 
be exiting the loop so want it as a return value, but the iterator might 
call another function (hence it need storing across the func call on 
almost all archs,  sparc with reg windows is about the only one i can 
thnk of that might not be effected.

this has to be done inside the iterating loop, where as saving the 
"enum" to the parent frame is only done once at the time that it needed 
(just before exiting prematurly) it seems to me it might actually be faster.
Sep 06 2003
prev sibling parent "Sean L. Palmer" <palmer.sean verizon.net> writes:
"Mike Wynn" <mike l8night.co.uk> wrote in message
news:bjam8u$21if$1 digitaldaemon.com...

 Walter wrote:

 The need for it to return an int rather than bit is because of the




compiler

 'magic' involved with making this work. The foreach body is transformed




into

 a nested function (!). The return value is needed to handle the various




ways

 of exiting the foreach body, such as goto, break label, return, etc. The
 foreach itself is replaced with a switch statement calling the apply
 function. It sounds a bit hairy, but that's all hidden from the




programmer.

 The neato thing is there is no need to write state-preserving iterators,




a

 rather tricky thing to get right for containers that are recursive or
 otherwise need state-preserving iterators.


...
 simple break is just return false (stop iterating)
 simple continue is just return true (stop current iteration, but continue)

 I agree this is a little slower, BUT removes the "magic" from the
 programmers view, I feel the int magic although a little faster does
 expose the inner workings a little too much and is a bug waiting to


happen.

I agree... at very least consider making it an enum instead of a plain int.

Sean
Sep 06 2003
prev sibling parent reply "Sean L. Palmer" <palmer.sean verizon.net> writes:
Yup, that's pretty much what I had in mind.

Is there a way to obtain the 'type' of the elements to be iterated over
automatically?  It should be possible to infer the type.

Sean

"Walter" <walter digitalmars.com> wrote in message
news:bj5pa5$156u$1 digitaldaemon.com...

 This includes the new foreach construct, which is the building block for
 container classes and iterations. I'll work on the bug reports next!

 http://www.digitalmars.com/d/changelog.html
Sep 04 2003
parent "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <palmer.sean verizon.net> wrote in message
news:bj7pc7$tie$1 digitaldaemon.com...

 Yup, that's pretty much what I had in mind.

 Is there a way to obtain the 'type' of the elements to be iterated over
 automatically?  It should be possible to infer the type.


No. It sounds like a good idea, though.
Sep 04 2003