www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 5660] New: yield syntax sugar

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660

           Summary: yield syntax sugar
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: bearophile_hugs eml.cc



Ranges are flexible and useful, but my practice with Python has shown me that
many times you just want something simple that yields items lazily. To do this
in D there is opApply() (that currently doesn't play well with most Phobos.
Only std.array.array() and few other things are able to use them), but its
syntax is awful, and even after years of usage I can't remember it and I need
to look for an example of opApply() usage to copy & modify.

opApply() requires a significant amount of boilerplate code that makes code
quite longer, obfuscates the purpose of the code, and is bug-prone. This is a
simple example (D2 code, works with dmd 2.052):



import std.stdio;

/// Sequence of moves to solve Towers of Hanoi OEIS A001511
immutable final class hanoiTower {
    static opCall() {
        return new typeof(this)();
    }

    int opApply(int delegate(ref int) dg) {
        int result;
        int y = 1;

        result = dg(y);
        if (result)
            return result;

        foreach (x; hanoiTower()) {
            y = x + 1;
            result = dg(y);
            if (result)
                return result;

            y = 1;
            result = dg(y);
            if (result)
                return result;
        }

        return result;
    }

    int opApply(int delegate(ref int, ref int) dg) {
        int result;
        int i, y = 1;

        result = dg(i, y);
        i++;
        if (result)
            return result;

        foreach (x; hanoiTower()) {
            y = x + 1;
            result = dg(i, y);
            i++;
            if (result)
                return result;

            y = 1;
            result = dg(i, y);
            i++;
            if (result)
                return result;
        }

        return result;
    }
}


class genChar {
    char stop;

    this(char stop_) {
        stop = stop_;
    }

    static opCall(char stop_) {
        return new typeof(this)(stop_);
    }

    int opApply(int delegate(ref char) dg) {
        int result;
        for (char c = 'a'; c < stop; c++) {
            result = dg(c);
            if (result)
                return result;
        }
        return result;
    }

    int opApply(int delegate(ref int, ref char) dg) {
        int result, i;
        for (char c = 'a'; c < stop; c++) {
            result = dg(i, c);
            i++;
            if (result)
                return result;
        }
        return result;
    }
}

void main() {
    foreach (i, move; hanoiTower()) {
        writeln(i, " ", move);
        if (i >= 20)
            break;
    }
    writeln();

    foreach (i, c; genChar('h'))
        writeln(i, " ", c);
}


So I suggest a syntax like this, that's just syntax sugar for the precedent
code:


import std.stdio;

yield(int) hanoiTower() {
    yield(1);
    foreach (x; hanoiTower()) {
        yield(x + 1);
        yield(1);
    }
}

yield(auto) genChar(char stop) {
    for (char c = 'a'; c < stop; c++)
        yield(c);
}

void main() {
    foreach (i, move; hanoiTower()) {
        writeln(i, " ", move);
        if (i >= 20)
            break;
    }
    writeln();

    foreach (i, c; genChar('h'))
        writeln(i, " ", c);
}




to those other three languages.)

The normal syntax of opApply() is of course kept in D, this is just an additive
change, that acts at syntax level (so just a lowering is needed to implement
it).

I suggest the generators with yield to be classes that define a "static
opCall()" because their semantics is more flexible than structs. This causes a
heap allocation, but situations where performance is so important are less
common, and in such situation the programmer may just write the normal struct
code with opApply(), or use a struct with more inlinable range protocol
methods, or use something more lower level.

The syntax uses yield(...) as return type to denote such iterable classes, and
yield(auto) too supported, and it acts like auto return type for functions (all
yields in a generator must yield the same type).

Some disadvantages:
- "yield" may need to become a kind of keyword.
- Currently opApply() doesn't work well with std.algorithm and Phobos in
general. This is true for the normal current usage of opApply(), but this
proposal makes their usage more attractive and probably more widespread.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 27 2011
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660


David Simcha <dsimcha yahoo.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dsimcha yahoo.com



opApply plus fibers should do what you need.  It's inefficient, but so is
pretty much any coroutine/yield-based way of doing things, including Python's. 
I was thinking a while back that something like this belonged in std.range, but 
I wanted to handle ref properly, so Bug 2443 got in my way.

Example (Warning:  Untested.):

/**
This must be a class or otherwise have reference/heap-allocated semantics.
*/
class OpApplyToRange(Iterable) {
    Fiber fiber;
    ForeachType!Iterable _front;
    bool _empty;
    Iterable iterable;

    void doLoop() {
        foreach(elem; iterable) {
            _front = elem;
            Fiber.yield();
        }

        _empty = true;
    }

    this(Iterable iterable) {
        this.iterable = iterable;
        fiber = new Fiber(&doLoop);
        fiber.call();
    }

    void popFront() {
        fiber.call();
    }

    ForeachType!Iterable front()  property { return _front; }

    bool empty()  property { return _empty; }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 27 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660




More programming in D shows me that syntax sugar like this one will be very
useful to me:

yield(int) foo() {
    yield(1);
}


So, is it possible to automatically convert such kind of code into a Range
(istead of opApply and fibers)?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 03 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660




Just as an example, this is Python2.6 code:


def process((i, j), a, b):
  if i == 0: yield (a, j)
  if j == 0: yield (i, b)
  if i == a: yield (0, j)
  if j == b: yield (i, 0)
  if j != b: yield (i + j - b, b) if (b < i + j) else (0, i + j)
  if i != a: yield (a, i + j - a) if (a < i + j) else (i + j, 0)



And this is how ShedSkin compiles it to C++ (a bit edited):


class __gen_process : public __iter<tuple2<int, int> *> {
public:
  tuple2<int, int> *__2, *__3;
  int a, b, i, j;

  int __last_yield;

  __gen_process(tuple2<int, int> *__2,int a,int b) {
    this->__2 = __2;
    this->a = a;
    this->b = b;
    __last_yield = -1;
  }

  tuple2<int, int> * __get_next() {
    switch(__last_yield) {
      case 0: goto __after_yield_0;
      case 1: goto __after_yield_1;
      case 2: goto __after_yield_2;
      case 3: goto __after_yield_3;
      case 4: goto __after_yield_4;
      case 5: goto __after_yield_5;
      default: break;
    }
    if ((i==0)) {
      __last_yield = 0;
      __result = (new tuple2<int, int>(2,a,j));
      return __result;
      __after_yield_0:;
    }
    if ((j==0)) {
      __last_yield = 1;
      __result = (new tuple2<int, int>(2,i,b));
      return __result;
      __after_yield_1:;
    }
    if ((i==a)) {
      __last_yield = 2;
      __result = (new tuple2<int, int>(2,0,j));
      return __result;
      __after_yield_2:;
    }
    if ((j==b)) {
      __last_yield = 3;
      __result = (new tuple2<int, int>(2,i,0));
      return __result;
      __after_yield_3:;
    }
    if ((j!=b)) {
      __last_yield = 4;
      __result = (((b<(i+j)))?((new tuple2<int, int>(2,((i+j)-b),b))):((new
tuple2<int, int>(2,0,(i+j)))));
      return __result;
      __after_yield_4:;
    }
    if ((i!=a)) {
      __last_yield = 5;
      __result = (((a<(i+j)))?((new tuple2<int, int>(2,a,((i+j)-a)))):((new
tuple2<int, int>(2,(i+j),0))));
      return __result;
      __after_yield_5:;
    }
    __stop_iteration = true;
  }
};

__iter<tuple2<int, int> *> *process(tuple2<int, int> *__2, int a, int b) {
  return new __gen_process(__2,a,b);
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Aug 03 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660




Alternative syntax (here inside the generator yield is used like the "return"
statement):


yield(int) hanoiTower() {
    yield(1);
    foreach (x; hanoiTower()) {
        yield x + 1;
        yield 1;
    }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 12 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660




Alternative syntax (here inside the generator yield is used like the "return"
statement):


yield(int) hanoiTower() {
    yield 1;
    foreach (x; hanoiTower()) {
        yield x + 1;
        yield 1;
    }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Sep 12 2011
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660


Witold Baryluk <baryluk smp.if.uj.edu.pl> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |baryluk smp.if.uj.edu.pl



21:18:51 PST ---
I have library for advanced iterators using Fibers. Take look here 
https://github.com/baryluk/iterators_2  (it is not update, but I use it very
often, and locally have many updates to it, not just commited).

Library was created before D ranges was introduced, so few things should be
improved, but it still works.

But any way - no need to syntax sugars.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 14 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660







 But any way - no need to syntax sugars.


This was a request for syntax sugar for opApply. No need for Fibers here
(fibers are useful, but here I am not asking for them).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Feb 15 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660


Adam D. Ruppe <destructionator gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |destructionator gmail.com



07:51:55 PDT ---
// this is your sugar!
string yield(string what) { return `if(auto result = dg(`~what~`)) return
result;`; }


Now, every time you would write

        result = dg(y);
        if (result)
            return result;

instead write:

mixin(yield("y"));

Or mixin(yield("i, y")); if you have multiple arguments.


And the boilerplate is way down. Since you keep result around,
you'll want to make it just if(result) instead of if(auto result),
but the idea is the same.


We might be able to make a class template or a mixn for
yield(int) hanoiTower() too, but writing int opApply(...)
isn't too bad.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 29 2012
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=5660


Adrian Matoga <epi atari8.info> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |epi atari8.info



Upvoted! Sometimes the yield syntax is the most natural and gives the most
readable code. And it should be lowered to an input range, as this is often the
only kind of iteration accepted by Phobos.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 12 2012