www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Prettier iterator implementations in D?

reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Forgetting about 'foreach' for the moment, I'd like to talk about the 
other end of things: opApply().

This code in D:
     int opApply(int delegate(inout uint) dg)
     {   int result = 0;

	for (int i = 0; i < array.length; i++)
	{
	    result = dg(array[i]);
	    if (result)
		break;
	}
	return result;
     }

seems to be rendered in Ruby-ish D as something like:

     void opApply( ) |inout uint|
     {
	for (int i = 0; i < array.length; i++)
	{
	    yield array[i];
	}
     }

Is there some way to make the real D one look as clean as the Rubified 
one?

Main differences:
1) need to explicitly declare a delegate,
2) need to fiddle with return values.

For 1) I'm thinking if there were only some more compact way to declare 
a delegate it would be golden.

How about this: with function literals, if you leave out the keyword 
'delegate' or 'function' you get a delegate.  I.e.

      int delegate(long c) { return 6 + b; }

is equivalent to

      int (long c) { return 6 + b; }

So it seems logical that as a variable declaration:

      int delegate(long c) dg;

could also be made to be equivalent to

      int (long c) dg;

In that case we could say in D:

     void opApply(void (inout uint) dg)
     {
	for (int i = 0; i < array.length; i++)
	{
	    dg(array[i]);
	}
     }

(assuming that pesky 'int return' business can also be sorted out).

The void can also be omitted frequently in D.  So for a void-returning 
delegate that could even be shortened to:

     void opApply( (inout uint) block)
     {
	for (int i = 0; i < array.length; i++)
	{
	    block(array[i]);
	}
     }

Now *that* is something I wouldn't mind staring at at 3am after a long 
day coding. :-)

--bb
Oct 19 2006
next sibling parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
Shorter and more beautiful is good, and has merits indeed...

But sometimes clarity comes as a price, and that's not so good.  If I 
were new to D, this:

void opApply( (inout uint) block)

Would look like a funky sort of cast.  That said, this:

void opApply( ) |inout uint|

Is at least clear, if not horrendously ugly to my eyes.

-[Unknown]



 Forgetting about 'foreach' for the moment, I'd like to talk about the 
 other end of things: opApply().
 
 This code in D:
     int opApply(int delegate(inout uint) dg)
     {   int result = 0;
 
     for (int i = 0; i < array.length; i++)
     {
         result = dg(array[i]);
         if (result)
         break;
     }
     return result;
     }
 
 seems to be rendered in Ruby-ish D as something like:
 
     void opApply( ) |inout uint|
     {
     for (int i = 0; i < array.length; i++)
     {
         yield array[i];
     }
     }
 
 Is there some way to make the real D one look as clean as the Rubified one?
 
 Main differences:
 1) need to explicitly declare a delegate,
 2) need to fiddle with return values.
 
 For 1) I'm thinking if there were only some more compact way to declare 
 a delegate it would be golden.
 
 How about this: with function literals, if you leave out the keyword 
 'delegate' or 'function' you get a delegate.  I.e.
 
      int delegate(long c) { return 6 + b; }
 
 is equivalent to
 
      int (long c) { return 6 + b; }
 
 So it seems logical that as a variable declaration:
 
      int delegate(long c) dg;
 
 could also be made to be equivalent to
 
      int (long c) dg;
 
 In that case we could say in D:
 
     void opApply(void (inout uint) dg)
     {
     for (int i = 0; i < array.length; i++)
     {
         dg(array[i]);
     }
     }
 
 (assuming that pesky 'int return' business can also be sorted out).
 
 The void can also be omitted frequently in D.  So for a void-returning 
 delegate that could even be shortened to:
 
     void opApply( (inout uint) block)
     {
     for (int i = 0; i < array.length; i++)
     {
         block(array[i]);
     }
     }
 
 Now *that* is something I wouldn't mind staring at at 3am after a long 
 day coding. :-)
 
 --bb
Oct 19 2006
parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Unknown W. Brackets wrote:

 Shorter and more beautiful is good, and has merits indeed...
 
 But sometimes clarity comes as a price, and that's not so good.  If I 
 were new to D, this:
 
 void opApply( (inout uint) block)
 
 Would look like a funky sort of cast.  


Well if you're new to D then there's a whole *lot* of things that are 
going to look funky.  Like  foo!(int)() -- looks like some kind of 
negation of a cast of nothing.  Funky!


 That said, this:
 
 void opApply( ) |inout uint|
 
 Is at least clear, if not horrendously ugly to my eyes.


I agree that it's not so ugly, but I don't think it's any clearer unless 
you've been reading Ruby code.  And it doesn't have any analogues 
anywhere else in D.  At least leaving out the keyword 'delegate' and the 
keyword 'void' is already sanctioned practice elsewhere in D.  Being 
able to leave it out on variable declarations actually makes things 
*more* consistent.


--bb
Oct 19 2006
prev sibling next sibling parent reply Alexander Panek <a.panek brainsware.org> writes:
I know Ruby, and it's loops are very cool, indeed. But this is D, and as
a full blown *system and application programming language*, you can't
compare it to Ruby. I don't think syntactic sugar should be added too
hasty. Apart from that I kinda like the delegate / function ptr syntax
as it is.

Alex


On Fri, 2006-10-20 at 12:41 +0900, Bill Baxter wrote:

 Forgetting about 'foreach' for the moment, I'd like to talk about the 
 other end of things: opApply().
 
 This code in D:
      int opApply(int delegate(inout uint) dg)
      {   int result = 0;
 
 	for (int i = 0; i < array.length; i++)
 	{
 	    result = dg(array[i]);
 	    if (result)
 		break;
 	}
 	return result;
      }
 
 seems to be rendered in Ruby-ish D as something like:
 
      void opApply( ) |inout uint|
      {
 	for (int i = 0; i < array.length; i++)
 	{
 	    yield array[i];
 	}
      }
 
 Is there some way to make the real D one look as clean as the Rubified 
 one?
 
 Main differences:
 1) need to explicitly declare a delegate,
 2) need to fiddle with return values.
 
 For 1) I'm thinking if there were only some more compact way to declare 
 a delegate it would be golden.
 
 How about this: with function literals, if you leave out the keyword 
 'delegate' or 'function' you get a delegate.  I.e.
 
       int delegate(long c) { return 6 + b; }
 
 is equivalent to
 
       int (long c) { return 6 + b; }
 
 So it seems logical that as a variable declaration:
 
       int delegate(long c) dg;
 
 could also be made to be equivalent to
 
       int (long c) dg;
 
 In that case we could say in D:
 
      void opApply(void (inout uint) dg)
      {
 	for (int i = 0; i < array.length; i++)
 	{
 	    dg(array[i]);
 	}
      }
 
 (assuming that pesky 'int return' business can also be sorted out).
 
 The void can also be omitted frequently in D.  So for a void-returning 
 delegate that could even be shortened to:
 
      void opApply( (inout uint) block)
      {
 	for (int i = 0; i < array.length; i++)
 	{
 	    block(array[i]);
 	}
      }
 
 Now *that* is something I wouldn't mind staring at at 3am after a long 
 day coding. :-)
 
 --bb
Oct 19 2006
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Alexander Panek wrote:

 I know Ruby, and it's loops are very cool, indeed. But this is D, and as
 a full blown *system and application programming language*, you can't
 compare it to Ruby. 


What is D if not "the speed of C++ with the ease of Ruby"?


 I don't think syntactic sugar should be added too
 hasty. 


I agree.  That's why plenty of discussion is needed.  So start poking holes!


 Apart from that I kinda like the delegate / function ptr syntax
 as it is.


And you could still use it as is if you like, because the suggestion is 
to make 'delegate' optional, not mandate it's removal.

About whether sugar is warranted here:  it seems Walter sees this 
delegate mechanism as becoming *the* primary technique for iteration in 
D.  That being the case, it should be as easy to read and write as 
possible.   Being easier to use than C++ iterators is the primary reason 
he gives for liking it, in fact.  So let's figure out how to make 
absolutely as simple as possible.

Right now, say you want to write a generic array iterator factory 
function 'traverser', which you do because it allows you to do this:

     foreach(int i; iarray.traverser()) {
       ...
     }
to iterate in some custom way over the elements of any array.

Here is the signature for that now (returns a delegate that takes a 
delegate parameter):

   int delegate(int delegate(inout typeof(ArrayT[0])))
      traverser(ArrayT)(inout ArrayT array)
   {
   }
   ...


The int delegate(int delegate ...) business is just too verbose to grok 
easily.   If I change the ints to voids and drop the 'delegate':

(( inout typeof(ArrayT[0]) )) reversed(ArrayT)(inout ArrayT array)

I think that's easier to look at if for no other reason than being 
shorter.  But I'll admit it probably is more mind boggling at first. 
But it's not a stretch to say most everybody could get used to reading

   (int) somefunc() {...}

as a function that returns a void-returning, int-taking delegate.  Just 
think of the (int) as a lone argument list ["takes an int"] and it's 
pretty clear.
And in that light it's not hard to see the extra sets of parens (( )) 
mean 'a void returning delegate that takes a void returning delegate'. 
At some point (( type )) just becomes second nature as the basic return 
signature for an iterator factory.  It's easy to see once you know to 
look for it, because its so short and (( )) stands out if spaced properly.

Another alternative is standard aliases for those complicated types:

template Types(ArgT) {
     static if( is (ArgT[0]) ) {
	alias typeof(ArgT[0]) elem_t;
     }
     else static if( is(ArgT.elem_t) ) {
	alias ArgT.elem_t elem_t;
     }
     else {
         alias ArgT elem_t;
     }
     alias int delegate(int delegate(inout elem_t)) iter_t;
     // this works for anything that is either array-like or has an
     // elem_t alias.
}

Then you can use those aliases like:

   Types!(ArrayT).iter_t  reversed(ArrayT)(inout ArrayT array)
   { ... }

that still looks kinda klunky :-/, but maybe better.  At least it's 
short enough to fit on one line.  But it just replaces having to know 
what (( )) means with having to know what Types!().iter_t means.  So I'm 
not sure it's really better.

Well at least it is something that works now!

--bb
Oct 20 2006
parent Alexander Panek <a.panek brainsware.org> writes:
On Fri, 2006-10-20 at 16:50 +0900, Bill Baxter wrote:

 Alexander Panek wrote:

 I know Ruby, and it's loops are very cool, indeed. But this is D, and as
 a full blown *system and application programming language*, you can't
 compare it to Ruby. 


 
 What is D if not "the speed of C++ with the ease of Ruby"?


D has a C like syntax, Ruby does definitely not. :)


 

 I don't think syntactic sugar should be added too
 hasty. 


 
 I agree.  That's why plenty of discussion is needed.  So start poking holes!
 

 Apart from that I kinda like the delegate / function ptr syntax
 as it is.


 
 And you could still use it as is if you like, because the suggestion is 
 to make 'delegate' optional, not mandate it's removal.


Wouldn't it be better to solve such things with a little library (as you
suggested)? I do like the constructs of higher level languages that have
been added to D, but the meaning of each should be recognizable on the
first sight. So when you look at a big fat nested arguments list you'd
use your syntax to shorten it - but would it be readable for others?

What D has made possible [imho] is to make code more readable. You have
explicitly named or styled syntax sugar, that is mostly not used as is
in other languages, and thus can't really be misread through confusion.
I wouldn't want this to be changed, actually.


 
 About whether sugar is warranted here:  it seems Walter sees this 
 delegate mechanism as becoming *the* primary technique for iteration in 
 D.  That being the case, it should be as easy to read and write as 
 possible.   Being easier to use than C++ iterators is the primary reason 
 he gives for liking it, in fact.  So let's figure out how to make 
 absolutely as simple as possible.


alias! :P


 
 Right now, say you want to write a generic array iterator factory 
 function 'traverser', which you do because it allows you to do this:
 
      foreach(int i; iarray.traverser()) {
        ...
      }
 to iterate in some custom way over the elements of any array.
 
 Here is the signature for that now (returns a delegate that takes a 
 delegate parameter):
 
    int delegate(int delegate(inout typeof(ArrayT[0])))
       traverser(ArrayT)(inout ArrayT array)
    {
    }
    ...
 
 
 The int delegate(int delegate ...) business is just too verbose to grok 
 easily.   If I change the ints to voids and drop the 'delegate':
 
 (( inout typeof(ArrayT[0]) )) reversed(ArrayT)(inout ArrayT array)
 
 I think that's easier to look at if for no other reason than being 
 shorter.  But I'll admit it probably is more mind boggling at first. 
 But it's not a stretch to say most everybody could get used to reading
 
    (int) somefunc() {...}
 
 as a function that returns a void-returning, int-taking delegate.  Just 
 think of the (int) as a lone argument list ["takes an int"] and it's 
 pretty clear.
 And in that light it's not hard to see the extra sets of parens (( )) 
 mean 'a void returning delegate that takes a void returning delegate'. 
 At some point (( type )) just becomes second nature as the basic return 
 signature for an iterator factory.  It's easy to see once you know to 
 look for it, because its so short and (( )) stands out if spaced properly.
 
 Another alternative is standard aliases for those complicated types:
 
 template Types(ArgT) {
      static if( is (ArgT[0]) ) {
 	alias typeof(ArgT[0]) elem_t;
      }
      else static if( is(ArgT.elem_t) ) {
 	alias ArgT.elem_t elem_t;
      }
      else {
          alias ArgT elem_t;
      }
      alias int delegate(int delegate(inout elem_t)) iter_t;
      // this works for anything that is either array-like or has an
      // elem_t alias.
 }
 
 Then you can use those aliases like:
 
    Types!(ArrayT).iter_t  reversed(ArrayT)(inout ArrayT array)
    { ... }
 
 that still looks kinda klunky :-/, but maybe better.  At least it's 
 short enough to fit on one line.  But it just replaces having to know 
 what (( )) means with having to know what Types!().iter_t means.  So I'm 
 not sure it's really better.


I doesn't look klunky to me, actually. You can still use standard
aliases to 'beautify' it, for example if you make a library of such
things.


 
 Well at least it is something that works now!
 
 --bb
 


Yes :)

Alex
Oct 21 2006
prev sibling parent Reiner Pope <reiner.pope REMOVE.THIS.gmail.com> writes:
Alexander Panek wrote:

 I know Ruby, and it's loops are very cool, indeed. But this is D, and as
 a full blown *system and application programming language*, you can't
 compare it to Ruby. 


The fundamental difference between D and Ruby is static typing. This 
leads on to D being more suitable for compilation than Ruby, and thus 
its speed, type-safety and meta-programming, and D not as concisely 
handling some aspects of reflection. Other than that, they are the 
leading languages in their fields (interpreted and compiled languages), 
and can learn a lot from each other.

Cheers,

Reiner
Oct 20 2006
prev sibling next sibling parent reply Reiner Pope <reiner.pope REMOVE.THIS.gmail.com> writes:
The proposal looks good (as long as it creates no syntactical 
ambiguities) but if we had variadic template parameters (see 
http://www.generic-programming.org/~dgregor/cpp/variadic-templates.html 
for a good version for C++) then you could do it much more easily with 
templates:

template IterFunc(type ...)
{
     alias int delegate(type ...) IterFunc;
}

and then you could just declare your iterators:

int opApply(IterFunc!(int) dg)

The best thing about this is you can easily modify it:

template IndexedIterFunc(type ...)
{
     alias int delegate(size_t index, type ...) IndexedIterFunc;
}

or if the return status was instead the first function parameter (to 
allow for return values, not void):

template NewIterFunc(retVal, type ...)
{
   alias retVal delegate(out int status, type ...) NewIterFunc;
}

Unfortunately, this wouldn't handle 'inout' parameters, since you can't 
instantiate a template with an inout parameter...

Cheers,

Reiner
Oct 20 2006
parent Bill Baxter <wbaxter gmail.com> writes:
Reiner Pope wrote:

 The proposal looks good (as long as it creates no syntactical 
 ambiguities) but if we had variadic template parameters (see 
 http://www.generic-programming.org/~dgregor/cpp/variadic-templates.html 
 for a good version for C++) then you could do it much more easily with 
 templates:
 
 template IterFunc(type ...)
 {
     alias int delegate(type ...) IterFunc;
 }


Ooh, I missed this "Implicit Template Properties" thing.  That's an 
improvement over what I had even without the variadic bit.  Just means I 
need an IterFunc1, IterFunc2,... etc.


 
 and then you could just declare your iterators:
 
 int opApply(IterFunc!(int) dg)
 [...]
 Unfortunately, this wouldn't handle 'inout' parameters, since you can't 
 instantiate a template with an inout parameter...


Ah, that's a pity.  Is that a spec limitation or just something still on 
the "todo" list?

--bb
Oct 20 2006
prev sibling next sibling parent reply David Medlock <noone nowhere.com> writes:
Bill Baxter wrote:

 Forgetting about 'foreach' for the moment, I'd like to talk about the 
 other end of things: opApply().
 <snip>


You could always use this handy template which automatically exposes any 
array or assoc-array in a class you want, with a single mixin.



class Foo
{
   Bar[]  bar;

   mixin applyThis!(Bar,array);
}


Bar b = new Bar();
foreach( Bar b; foo ) {...}

-DavidM
Oct 20 2006
parent Bill Baxter <wbaxter gmail.com> writes:
David Medlock wrote:

 Bill Baxter wrote:
 

 Forgetting about 'foreach' for the moment, I'd like to talk about the 
 other end of things: opApply().
 <snip>


 
 
 You could always use this handy template which automatically exposes any 
 array or assoc-array in a class you want, with a single mixin.
 [...]


Wow.  That is very cool.  D is spiffy. :-)
But I still think the core syntax should be clean.
Makes me wonder, though, if C++ had mixins like that, maybe proper 
iterators wouldn't be so hard to do correctly there either?  Just a 
matter of adding in a line:
   iteratorThis!(...)


--bb
Oct 20 2006
prev sibling parent Reiner Pope <reiner.pope REMOVE.THIS.gmail.com> writes:
We could also make the syntax cleaner with type inference. Consider your 
example:

Bill Baxter wrote:

 This code in D:
     int opApply(int delegate(inout uint) dg)
     {   int result = 0;
 
     for (int i = 0; i < array.length; i++)
     {
         result = dg(array[i]);
         if (result)
         break;
     }
     return result;
     }


The compiler knows the type of array[i], so it can infer what the type 
of the parameter is. This inference should never be wrong, as long as 
the delegate is called at least once.

The return type can't be inferred, however, so we would combine your 
syntax with type inference to get some hybrid like this:

   int opApply(int (auto) dg)

To handle inout parameters, you simply need inout to be allowed at the 
call-site:

   result = dg(inout array[i]);


optional in D -- why shouldn't it?

The only problem I can see with this is that it moves the type of the 
delegate out of the function prototype and into the function. Never mind 
-- many people from functional languages with Hindley-Milner type 
inference have been doing that for years and not had problems with it; 
the saving is more than the gain.
Oct 20 2006