www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - parameter pack to inputRange

reply Erik Smith <erik cruiserhouse.com> writes:
Is there an existing way to adapt a parameter pack to an input 
range? I would like to construct an array with it.  Example:

void run(A...) (A args) {
      Array!int a(toInputRange(args));
}
May 05 2016
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 05/05/2016 10:00 PM, Erik Smith wrote:

 Is there an existing way to adapt a parameter pack to an input range? I
 would like to construct an array with it.  Example:

 void run(A...) (A args) {
       Array!int a(toInputRange(args));
 }


Just initialize an array with the arguments:

void run(A...) (A args) {
     writeln([args]);
}

Ali
May 05 2016
parent reply Erik Smith <erik cruiserhouse.com> writes:
On Friday, 6 May 2016 at 05:20:50 UTC, Ali Çehreli wrote:

 On 05/05/2016 10:00 PM, Erik Smith wrote:

 Is there an existing way to adapt a parameter pack to an input 
 range? I
 would like to construct an array with it.  Example:

 void run(A...) (A args) {
       Array!int a(toInputRange(args));
 }


 Just initialize an array with the arguments:

 void run(A...) (A args) {
     writeln([args]);
 }

 Ali


That's allocating a dynamic array though, right?  It seems like 
there should be a non-GC adaptor for this purpose (especially 
since I'm using std.container.array).

erik
May 05 2016
parent reply Dicebot <public dicebot.lv> writes:
Unless parameter list is very (very!) long, I'd suggest to simply 
copy it into a stack struct. Something like this:

auto toInputRange (T...) (T args)
{
     struct Range
     {
         T args;
         size_t index;

         T[0] front () { return args[index]; }
         void popFront () { ++index; }
         bool empty () { return index >= args.length; }
     }

     return Range(args, 0);
}
May 05 2016
next sibling parent Dicebot <public dicebot.lv> writes:
On Friday, 6 May 2016 at 06:08:24 UTC, Dicebot wrote:

 Unless parameter list is very (very!) long, I'd suggest to 
 simply copy it into a stack struct. Something like this:

 auto toInputRange (T...) (T args)
 {
     struct Range
     {
         T args;
         size_t index;

         T[0] front () { return args[index]; }
         void popFront () { ++index; }
         bool empty () { return index >= args.length; }
     }

     return Range(args, 0);
 }


(note that you can actually make it random access range and not 
just input range, I simply have not defined required methods)
May 05 2016
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 05/05/2016 11:08 PM, Dicebot wrote:

 Unless parameter list is very (very!) long, I'd suggest to simply copy
 it into a stack struct. Something like this:

 auto toInputRange (T...) (T args)
 {
      struct Range
      {
          T args;
          size_t index;

          T[0] front () { return args[index]; }
          void popFront () { ++index; }
          bool empty () { return index >= args.length; }
      }

      return Range(args, 0);
 }


I wanted this syntax to work but when I tested I saw that T does not 
expand to struct members.

I like Alex Parrill's only() solution but it allocates a dynamic array 
as well by doing the equivalent of [args] in the guts of its implementation.

As Dicebot said, unless there are tons of arguments, I think the 
following is the best as it is  nogc. It executes a switch statement for 
each front() call though. And I like the CommonType!T idea there.

import std.stdio;
import std.string;

/* Support empty Args? */
 nogc pure nothrow
auto toInputRange(Args...)() {
     struct Range {
         size_t index;

         bool empty() {
             return index >= Args.length;
         }

         void popFront() {
             ++index;
         }

         import std.traits : CommonType;
         alias E = CommonType!Args;

         E front() {
             final switch (index) {
                 /* static */ foreach (i, arg; Args) {
                     case i:
                         return arg;
                 }
             }
         }
     }

     return Range();
}

unittest {
     import std.traits;
     import std.range;

     static assert(isInputRange!(ReturnType!(toInputRange!(1))));
}

void main() {
     auto r = toInputRange!(1, 2.5, 3);
     writeln(r);
}

Ali
May 08 2016
next sibling parent Erik Smith <erik cruiserhouse.com> writes:
On Sunday, 8 May 2016 at 14:11:31 UTC, Ali Çehreli wrote:

 On 05/05/2016 11:08 PM, Dicebot wrote:

 Unless parameter list is very (very!) long, I'd suggest to


 simply copy

 it into a stack struct. Something like this:

 auto toInputRange (T...) (T args)
 {
      struct Range
      {
          T args;
          size_t index;

          T[0] front () { return args[index]; }
          void popFront () { ++index; }
          bool empty () { return index >= args.length; }
      }

      return Range(args, 0);
 }


 I wanted this syntax to work but when I tested I saw that T 
 does not expand to struct members.

 I like Alex Parrill's only() solution but it allocates a 
 dynamic array as well by doing the equivalent of [args] in the 
 guts of its implementation.

 As Dicebot said, unless there are tons of arguments, I think 
 the following is the best as it is  nogc. It executes a switch 
 statement for each front() call though. And I like the 
 CommonType!T idea there.

 import std.stdio;
 import std.string;

 /* Support empty Args? */
  nogc pure nothrow
 auto toInputRange(Args...)() {
     struct Range {
         size_t index;

         bool empty() {
             return index >= Args.length;
         }

         void popFront() {
             ++index;
         }

         import std.traits : CommonType;
         alias E = CommonType!Args;

         E front() {
             final switch (index) {
                 /* static */ foreach (i, arg; Args) {
                     case i:
                         return arg;
                 }
             }
         }
     }

     return Range();
 }

 unittest {
     import std.traits;
     import std.range;

     static assert(isInputRange!(ReturnType!(toInputRange!(1))));
 }

 void main() {
     auto r = toInputRange!(1, 2.5, 3);
     writeln(r);
 }

 Ali


I like this solution and it's exactly what I was looking for.   
However, I'm having an issue when I try to apply it with an 
actual variadic function.  This seems like a DMD bug.

static void foo(Args...)(Args args) {
     import std.container.array;
     auto array = Array!int(toInputRange!(args));

     foreach(a; array) {
         writeln("e: ", a);
     }
}

foo(1,2,3);

e:1431827808
e:32767
e:1254116144
May 08 2016
prev sibling next sibling parent Dicebot <public dicebot.lv> writes:
On Sunday, 8 May 2016 at 14:11:31 UTC, Ali Çehreli wrote:

 On 05/05/2016 11:08 PM, Dicebot wrote:

 Unless parameter list is very (very!) long, I'd suggest to


 simply copy

 it into a stack struct. Something like this:

 auto toInputRange (T...) (T args)
 {
      struct Range
      {
          T args;
          size_t index;

          T[0] front () { return args[index]; }
          void popFront () { ++index; }
          bool empty () { return index >= args.length; }
      }

      return Range(args, 0);
 }


 I wanted this syntax to work but when I tested I saw that T


It does (and it must). I have just tested it and noticed another 
issue though, one I have not foreseen initially. When accessing 
variadic variable list, one must use compile-time known index 
because variables are not guaranteed to be of same size. It can 
be fixed by using static array (and CommonType is a right call of 
course):

auto toInputRange (T...) (T args)  nogc
{
     import std.traits : CommonType;
     alias E = CommonType!T;

     struct Range
     {
         E[T.length] args;
         size_t index;

         E front () { return args[index]; }
         void popFront () { ++index; }
         bool empty () { return index >= args.length; }
     }

     Range range;
     foreach (i, ref arg; args)
         range.args[i] = arg;
     return range;
}

void main ( )
{
     import std.stdio;
     writeln(toInputRange(1, 2, 3));
}

This version is checked to work on my machine.
May 08 2016
prev sibling next sibling parent reply Dicebot <public dicebot.lv> writes:
On Sunday, 8 May 2016 at 14:11:31 UTC, Ali Çehreli wrote:

         E front() {
             final switch (index) {
                 /* static */ foreach (i, arg; Args) {
                     case i:
                         return arg;
                 }
             }
         }


AFAIK, this will do funny things with referencing stack if 
arguments are variables and not literals.
May 08 2016
parent reply Erik Smith <erik cruiserhouse.com> writes:
On Sunday, 8 May 2016 at 22:37:44 UTC, Dicebot wrote:

 On Sunday, 8 May 2016 at 14:11:31 UTC, Ali Çehreli wrote:

         E front() {
             final switch (index) {
                 /* static */ foreach (i, arg; Args) {
                     case i:
                         return arg;
                 }
             }
         }


 AFAIK, this will do funny things with referencing stack if 
 arguments are variables and not literals.


Thanks! The static array version works for me too.   It would be 
good to understand more about what is going on.  It looks like 
the cost of the static array is an extra copy for each element.  
Maybe there is still a way to avoid that.
May 08 2016
next sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 05/08/2016 04:48 PM, Erik Smith wrote:

 On Sunday, 8 May 2016 at 22:37:44 UTC, Dicebot wrote:

 On Sunday, 8 May 2016 at 14:11:31 UTC, Ali Çehreli wrote:

         E front() {
             final switch (index) {
                 /* static */ foreach (i, arg; Args) {
                     case i:
                         return arg;
                 }
             }
         }


 AFAIK, this will do funny things with referencing stack if arguments
 are variables and not literals.


 Thanks! The static array version works for me too.   It would be good to
 understand more about what is going on.  It looks like the cost of the
 static array is an extra copy for each element. Maybe there is still a
 way to avoid that.


I had to change one line of your test code. Dicebot's code work with it:

auto toInputRange (T...) (T args)  nogc
{
     import std.traits : CommonType;
     alias E = CommonType!T;

     struct Range
     {
         E[T.length] args;
         size_t index;

         E front () { return args[index]; }
         void popFront () { ++index; }
         bool empty () { return index >= args.length; }
     }

     Range range;
     foreach (i, ref arg; args)
         range.args[i] = arg;
     return range;
}

static void foo(Args...)(Args args) {
     import std.container.array;
     auto array = Array!int(toInputRange(args));  // <-- HERE

     foreach(a; array) {
         import std.stdio : writeln;
         writeln("e: ", a);
     }
}

void main ( )
{
     import std.stdio;
     writeln(toInputRange(1, 2, 3));

     foo(1,2,3);
}

It used to be

     toInputRange!(args)

Ali
May 08 2016
parent Erik Smith <erik cruiserhouse.com> writes:
On Sunday, 8 May 2016 at 23:49:40 UTC, Ali Çehreli wrote:

 On 05/08/2016 04:48 PM, Erik Smith wrote:

 On Sunday, 8 May 2016 at 22:37:44 UTC, Dicebot wrote:

 On Sunday, 8 May 2016 at 14:11:31 UTC, Ali Çehreli wrote:

         E front() {
             final switch (index) {
                 /* static */ foreach (i, arg; Args) {
                     case i:
                         return arg;
                 }
             }
         }


 AFAIK, this will do funny things with referencing stack if 
 arguments
 are variables and not literals.


 Thanks! The static array version works for me too.   It would 
 be good to
 understand more about what is going on.  It looks like the 
 cost of the
 static array is an extra copy for each element. Maybe there is 
 still a
 way to avoid that.


 I had to change one line of your test code. Dicebot's code work 
 with it:

 auto toInputRange (T...) (T args)  nogc
 {
     import std.traits : CommonType;
     alias E = CommonType!T;

     struct Range
     {
         E[T.length] args;
         size_t index;

         E front () { return args[index]; }
         void popFront () { ++index; }
         bool empty () { return index >= args.length; }
     }

     Range range;
     foreach (i, ref arg; args)
         range.args[i] = arg;
     return range;
 }

 static void foo(Args...)(Args args) {
     import std.container.array;
     auto array = Array!int(toInputRange(args));  // <-- HERE

     foreach(a; array) {
         import std.stdio : writeln;
         writeln("e: ", a);
     }
 }

 void main ( )
 {
     import std.stdio;
     writeln(toInputRange(1, 2, 3));

     foo(1,2,3);
 }

 It used to be

     toInputRange!(args)

 Ali


I did notice that but forgot to mention it - thanks for 
clarifying.  Again it definitely works but it would be nice to 
find a non-copy solution.  I tried to form a static array of 
pointers to the args (see below).  It compiled but the output was 
bad.  I would expect that taking the address of the arguments 
should be safe as long as you are in the called function.


auto toInputRange (T...) (T args)  nogc {
     import std.traits : CommonType;
     alias E = CommonType!T;

     struct Range {
         alias P = E*;
         P[T.length] args;
         size_t index;

         E front () { return *args[index]; }
         void popFront () { ++index; }
         bool empty () { return index >= args.length; }
     }

     Range range;
     foreach (i, ref arg; args) range.args[i] = &arg;
     return range;
}
May 08 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
On Sunday, 8 May 2016 at 23:48:01 UTC, Erik Smith wrote:

 Thanks! The static array version works for me too.   It would 
 be good to understand more about what is going on.  It looks 
 like the cost of the static array is an extra copy for each 
 element.  Maybe there is still a way to avoid that.


The difference is quite simple: my latest version does save copy 
of arguments in static array which means you can freely pass it 
around with referenced data being valid. But it may become 
expensive to copy with large argument lists if compiler can't 
optimize it away. Ali version generates a compile-time switch for 
index that returns either literals or external variables by 
referencing their symbol directly. If this gets used to reference 
external stack variables which later get out of scope, range may 
either start iterating over garbage memory or silently allocate a 
closure (don't remember which is the case for latest compiler).

In practice, if you use good inlining compiler, like LDC, I'd say 
static array version is strictly better. It is guaranteed to be 
safe and RVO allows compiler to completely get rid of additional 
stack copies if usage scope is limited.
May 09 2016
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 05/09/2016 03:44 AM, Dicebot wrote:


 Ali version generates a compile-time switch for index


I think it's a run-time switch, generated by a compile-time foreach:

E front() {
     final switch (index) {                    // <-- RUNTIME
         /* static */ foreach (i, arg; Args) { // <-- COMPILE TIME
             case i:                           // <-- RUNTIME
                 return arg;                   // <-- RUNTIME
         }
     }
}

Quite confusing. :)


 that returns either literals or external variables by
 referencing their symbol directly.


I see. So, a parameter pack (AliasSeq) uses 'alias' template parameters 
for symbols. Makes sense...

Ali
May 09 2016
parent Dicebot <public dicebot.lv> writes:
On Monday, 9 May 2016 at 12:36:00 UTC, Ali Çehreli wrote:

 On 05/09/2016 03:44 AM, Dicebot wrote:


 Ali version generates a compile-time switch for index


 I think it's a run-time switch, generated by a compile-time 
 foreach:

 E front() {
     final switch (index) {                    // <-- RUNTIME
         /* static */ foreach (i, arg; Args) { // <-- COMPILE 
 TIME
             case i:                           // <-- RUNTIME
                 return arg;                   // <-- RUNTIME
         }
     }
 }

 Quite confusing. :)


It becomes even more confusing if you keep in mind that it is a 
final switch on an index range known at compile-time - which 
means that as long as whole template can be inlined and actual 
index is know at CT, compiler can eliminate all runtime indexing 
checks and just replace it with matching literal/symbol.
May 09 2016
prev sibling parent Alex Parrill <initrd.gz gmail.com> writes:
On Sunday, 8 May 2016 at 14:11:31 UTC, Ali Çehreli wrote:

 I like Alex Parrill's only() solution but it allocates a 
 dynamic array as well by doing the equivalent of [args] in the 
 guts of its implementation.


No it does not.

The constructor does `this.data = [values];`, but `this.data` is 
a fixed-sized array, which is stored in the structure itself. No 
allocation needs to happen (it should be the same as the foreach 
loop in your implementation).

You guys are just re-inventing `only`.
May 09 2016
prev sibling next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 05/05/2016 10:00 PM, Erik Smith wrote:

 Is there an existing way to adapt a parameter pack to an input range? I
 would like to construct an array with it.  Example:

 void run(A...) (A args) {
       Array!int a(toInputRange(args));
 }


Inspired by my DConf 2016 talk ;) here is a fiber-based InputRange solution:

import std.stdio;
import std.concurrency;
import std.container;

alias GeneratorFiber = std.concurrency.Generator;

void run(A...) (A args) {
     void asInputRange() {
         /* static */ foreach (arg; args) {
             yield(arg);
         }
     }

     auto range =  new GeneratorFiber!int(&asInputRange);
     auto a = Array!int(range);
     writeln(a[]);
}

void main() {
     run(1, 2, 3);
}

Ali
May 05 2016
prev sibling parent Alex Parrill <initrd.gz gmail.com> writes:
On Friday, 6 May 2016 at 05:00:48 UTC, Erik Smith wrote:

 Is there an existing way to adapt a parameter pack to an input 
 range? I would like to construct an array with it.  Example:

 void run(A...) (A args) {
      Array!int a(toInputRange(args));
 }


Use std.range.only: http://dlang.org/phobos/std_range.html#only

void run(A...)(A args) {
     auto rng = only(args);
     // ...
}
May 06 2016