www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Variadic template arguments unpacking

reply "Max Strakhov" <monnoroch gmail.com> writes:
Suppose, in c++ i've got a variadic template function F, and i 
want to convert all the arguments in some individual manner by 
function conv and pass all the converted arguments as a variadic 
list to another variadic template function G.
Then i just do the following:

template<typename T>
struct R;

template<typename T>
R<T>::Type conv(T&& val);

template<typename ... Args>
void F(Args&& ... args) {
     G(conv(std::forward<Args>(args))...);
}

Is there any way to do the same kind of thing in D? I couldn't 
find any unpacking operator or a library function to do so...
The problem is that i have to pass converted lits to a C-like 
variadic function, for example printf, so it would be like:

template<typename T>
struct R;

template<typename T>
R<T>::Type conv(T&& val);

template<typename ... Args>
void MyPrintf(const char * format, Args&& ... args) {
     printf(convFormat(format), conv(std::forward<Args>(args))...);
}

I hope, my point is clear enough.
Jul 05 2013
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/05/2013 05:41 PM, Max Strakhov wrote:

 Suppose, in c++ i've got a variadic template function F, and i want to
 convert all the arguments in some individual manner by function conv and
 pass all the converted arguments as a variadic list to another variadic
 template function G.
 Then i just do the following:

 template<typename T>
 struct R;

 template<typename T>
 R<T>::Type conv(T&& val);

 template<typename ... Args>
 void F(Args&& ... args) {
      G(conv(std::forward<Args>(args))...);
 }

 Is there any way to do the same kind of thing in D? I couldn't find any
 unpacking operator or a library function to do so...
 The problem is that i have to pass converted lits to a C-like variadic
 function, for example printf, so it would be like:

 template<typename T>
 struct R;

 template<typename T>
 R<T>::Type conv(T&& val);

 template<typename ... Args>
 void MyPrintf(const char * format, Args&& ... args) {
      printf(convFormat(format), conv(std::forward<Args>(args))...);
 }

 I hope, my point is clear enough.


Here is one that works:

import core.stdc.stdio;
import std.string;

string convFormat(string s)
{
     return s ~ '\n';
}

void myPrintf(T...)(string s, T args)
{
     printf(convFormat(s).toStringz, args);
}

void main()
{
     myPrintf("format %d %f %c %s", 42, 1.5, 'a', "hello".toStringz);
}

Ideally, the caller of MyPrintf should not have to call toStringz on 
"hello", because myPrintf is a proper D function. However, I don't know 
how to manipulate args to call toStringz on string arguments.

More information is at

   http://dlang.org/template.html#TemplateTupleParameter

Ali
Jul 05 2013
next sibling parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/05/2013 06:35 PM, Ali Çehreli wrote:


 More information is at

    http://dlang.org/template.html#TemplateTupleParameter


And the definitive document on templates is the following:

   https://github.com/PhilippeSigaud/D-templates-tutorial

Ali
Jul 05 2013
prev sibling parent "Max Strakhov" <monnoroch gmail.com> writes:
On Saturday, 6 July 2013 at 01:35:28 UTC, Ali Çehreli wrote:

 On 07/05/2013 05:41 PM, Max Strakhov wrote:
 Here is one that works:

 import core.stdc.stdio;
 import std.string;

 string convFormat(string s)
 {
     return s ~ '\n';
 }

 void myPrintf(T...)(string s, T args)
 {
     printf(convFormat(s).toStringz, args);
 }

 void main()
 {
     myPrintf("format %d %f %c %s", 42, 1.5, 'a', 
 "hello".toStringz);
 }


You only converted the format string with was not in the 
question. I need to convert all args by a template function, but 
your example is actually better than mine.
Suppose, i want my own string representation of primitive types, 
then in c++ i would write those two functions:
convFormat("format %d %f %c %s") -> std::string("format %s %s %s 
%s")
conv(42) -> std::string("int(42)")
conv(1.5)-> std::string("float(1.5)")
etc.
and the resulting printf:
template<typename ... Args>
void myPrintf(const char * format, Args&& ... args) {
    printf(convFormat(format).c_str(), 
conv(std::forward<Args>(args)).c_str()...);
}
And that would do the job. Can i somehow do the same thing in D?
Jul 06 2013
prev sibling parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 07/06/13 02:41, Max Strakhov wrote:

 Suppose, in c++ i've got a variadic template function F, and i want to convert
all the arguments in some individual manner by function conv and pass all the
converted arguments as a variadic list to another variadic template function G.


http://forum.dlang.org/thread/yiougbmcegjlsrynlvyt forum.dlang.org?page=2#post-mailman.1205.1371553020.13711.digitalmars-d-learn:40puremagic.com

artur
Jul 06 2013
parent reply "Max Strakhov" <monnoroch gmail.com> writes:
Artur, if i use your solution like

printf("...", ForEach!(val => conv(val).c_str())(values).tuple);

Than i would get a crash, because all the tuple elements would be 
char*'s, pointing to already freed memory, as std::string's 
destructor gets called each time right after alias function 
exits. Ths is what c++ unpacking operator for: i actually return 
a packed list of std::string's, apply .c_str() to each and than 
unpack the list to current context.
Jul 06 2013
next sibling parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 07/06/13 14:21, Max Strakhov wrote:

 Artur, if i use your solution like
 
 printf("...", ForEach!(val => conv(val).c_str())(values).tuple);
 
 Than i would get a crash, because all the tuple elements would be char*'s,
pointing to already freed memory, as std::string's destructor gets called each
time right after alias function exits. Ths is what c++ unpacking operator for:
i actually return a packed list of std::string's, apply .c_str() to each and
than unpack the list to current context.


Can you show some simple code that exhibits the problem?

artur
Jul 06 2013
next sibling parent reply "Max Strakhov" <monnoroch gmail.com> writes:
Thanx for all your help, i just solved the problem with mixins 
like this:

string res = "";
for(i, rev v; args)
     res ~= string.format(", f(args[%d]).val()");
mixin("return G(" ~ res ~ ");");

Only without any actual looping and variables, so compiler could 
deal with it.
Jul 06 2013
parent reply "TommiT" <tommitissari hotmail.com> writes:
On Saturday, 6 July 2013 at 16:15:52 UTC, Max Strakhov wrote:

 Thanx for all your help, i just solved the problem with mixins 
 like this:

 string res = "";
 for(i, rev v; args)
     res ~= string.format(", f(args[%d]).val()");
 mixin("return G(" ~ res ~ ");");

 Only without any actual looping and variables, so compiler 
 could deal with it.


printf("...", ForEach!(val => conv(val).c_str())(values).tuple);

...would have done exactly the same as C++:

printf("...", conv(values).c_str()...);

...does (barring it makes a temporary struct which holds one 
field for each of those converted const char pointers, but the 
compiler can optimize it away).
Jul 06 2013
parent "TommiT" <tommitissari hotmail.com> writes:
On Saturday, 6 July 2013 at 16:40:25 UTC, TommiT wrote:

 ...does (barring it makes a temporary struct which holds one 
 field for each of those converted const char pointers, ...


Should be "...one field per each of those..."
Jul 06 2013
prev sibling next sibling parent reply "TommiT" <tommitissari hotmail.com> writes:
On Saturday, 6 July 2013 at 15:23:40 UTC, Artur Skawina wrote:

 On 07/06/13 14:21, Max Strakhov wrote:

 Artur, if i use your solution like
 
 printf("...", ForEach!(val => 
 conv(val).c_str())(values).tuple);
 
 Than i would get a crash, because all the tuple elements would 
 be char*'s, pointing to already freed memory, as std::string's 
 destructor gets called each time right after alias function 
 exits. Ths is what c++ unpacking operator for: i actually 
 return a packed list of std::string's, apply .c_str() to each 
 and than unpack the list to current context.


 Can you show some simple code that exhibits the problem?

 artur


import std.stdio;

template _TypeMap(alias MAP, size_t N, TS...) {
   static if (N<TS.length)
       alias _TypeMap = _TypeMap!(MAP,
                                  N+1,
                                  TS[0..N],
                                  typeof(MAP(TS[N].init)),
                                  TS[N+1..$]);
   else
       alias _TypeMap = TS;
}
template TypeMap(alias MAP, TS...) {
  alias TypeMap = _TypeMap!(MAP, 0, TS);
}

struct _ForEach(alias MAP, TS...)
{
   TypeMap!(MAP, TS) tuple;

   this(TS values)
   {
       foreach (i, ref v; values)
           tuple[i] = MAP(v);
   }
}

auto ForEach(alias MAP, TS...)(TS ts)
{
   return _ForEach!(MAP, TS)(ts);
}


int[2] getArray(int n)
{
     int[2] data = n;
     return data;
}

void foo(R...)(R ranges)
{
     foreach (range; ranges)
         foreach (value; range)
             write(value, " ");
}

void main()
{
     // Expecting this to print: 1 1 2 2 3 3
     foo(ForEach!(a => getArray(a)[])(1, 2, 3).tuple);
     // ... but it prints random garbage
}

Notice that there were a couple of typos in _TypeMap.
Jul 06 2013
next sibling parent "Max Strakhov" <monnoroch gmail.com> writes:
 There were a couple of typos in _TypeMap.


Yeah, i fixed them myself. Still, there were some problems.
I like my solution better anyway, because it behaves exactly like 
c++ ... operator and just expands some code for all items in 
tuple.
Jul 06 2013
prev sibling parent reply Artur Skawina <art.08.09 gmail.com> writes:
On 07/06/13 19:42, TommiT wrote:

                                  TS[N+1..$]);



 Notice that there were a couple of typos in _TypeMap.


Thanks for catching that.



 int[2] getArray(int n)
 {
     int[2] data = n;
     return data;
 }
 
 void foo(R...)(R ranges)
 {
     foreach (range; ranges)
         foreach (value; range)
             write(value, " ");
 }
 
 void main()
 {
     // Expecting this to print: 1 1 2 2 3 3
     foo(ForEach!(a => getArray(a)[])(1, 2, 3).tuple);
     // ... but it prints random garbage
 }


This is slicing a local static array and returning that slice,
which points to the stack and lives only until the lambda
returns. Moving the data to the heap or simply returning it
directly (ie by value) will work:


     foo(ForEach!(a => getArray(a).dup[])(1, 2, 3).tuple);
     foo(ForEach!(a => getArray(a))(1, 2, 3).tuple);



I like Max's solution better than mine too, btw. Why didn't i
think of that... :)

artur
Jul 06 2013
parent reply "TommiT" <tommitissari hotmail.com> writes:
On Saturday, 6 July 2013 at 18:26:28 UTC, Artur Skawina wrote:

 This is slicing a local static array and returning that slice,
 which points to the stack and lives only until the lambda
 returns.


Yes, but certainly simple parameter pack expansion shouldn't be 
this brittle. I want to be able to write:

foo(getArray(v)[]...); // v is a parameter pack (1, 2, 3)

And be confident in that it gets simply re-written as:

foo(getArray(v[0])[], getArray(v[1])[], getArray(v[2])[]);

It's easy to conceptualise and easy to get it right.
Jul 06 2013
parent reply "Max Strakhov" <monnoroch gmail.com> writes:
 foo(getArray(v[0])[], getArray(v[1])[], getArray(v[2])[]);

 It's easy to conceptualise and easy to get it right.


I wrote a template, that can do something like that:

template Expand(string code, string Args, string args, Ts...) {
	const hasType = std.string.indexOf(code, "%Arg") != -1;
	const hasVal  = std.string.indexOf(code, "%arg") != -1;

	static if(hasType)
		const code1 = std.array.replace(code, "%Arg", Args ~ "[%d]");
	else
		const code1 = code;

	static if(hasVal)
		const code2 = std.array.replace(code1, "%arg", args ~ "[%d]");
	else
		const code2 = code1;

	template Fun(size_t i, T) {
		static if(!hasType && !hasVal)
			alias Fun = TypeTuple!(code2);
		else static if(hasType && hasVal)
			alias Fun = TypeTuple!(std.string.format(code2, i, i));
		else
			alias Fun = TypeTuple!(std.string.format(code2, i));
	}

	template ConcatArgs() {
		const Val = "";
	}

	template ConcatArgs(T, As...) if(!is(T == string)) {
		static assert(T);
	}

	template ConcatArgs(string s, As...) if(As.length == 0) {
		const Val = s;
	}

	template ConcatArgs(string s, As...) if(As.length != 0) {
		const Val = s ~ ", " ~ ConcatArgs!As.Val;
	}

	const Val = ConcatArgs!(staticIndexedMap!(Fun, Ts)).Val;
}

Usage:

writeln(Expand!("getArray(%arg)[]", "", "v", string, int, 
int).Val); // == getArray(v[0])[], getArray(v[1])[], 
getArray(v[2])[]

If anyine has any ideas how make this implementation simplier, 
bring it on :)
Jul 06 2013
parent Artur Skawina <art.08.09 gmail.com> writes:
On 07/06/13 21:39, Max Strakhov wrote:

 
 If anyine has any ideas how make this implementation simplier, bring it on :)


Well, you could just use the pseudo-code version that you posted,
almost verbatim:


 string res = "";
 for(i, rev v; args)
     res ~= string.format(", f(args[%d]).val()");
 mixin("return G(" ~ res ~ ");");
 
 Only without any actual looping and variables, so compiler could deal with it.


   auto F(A...)(string format, A args) {
      mixin({
         string res;
         foreach (I, _; A)
             res ~= (I?", ":"") ~ "conv(args["~I.stringof~"])";
         return "return G(convformat(format), " ~ res ~ ");";
      }());
   }

The compiler can deal with this, no need to make it more complicated.

It's a good idea for non-obvious text-as-code manipulations to be
explicit, so I'd actually do it like that.

A problem when manipulating source code directly is that not
everything can be referred to by name - a non-local type or
symbol which isn't available in the current scope will work when
it's a template parameter or via typeof(), but you can't directly
name it and then mix it back in. Locals, including function args,
will work though. 

For cases where the above restriction isn't a problem, we could
use something more generic, that will make it possible to solve
your problem with a one-liner. It will work with not just
"argument-packs" (ie arg-tuples), but also some symbols, types
and statically evaluable expressions. The "oops" line shows what
does *not* work and why these kinds of mixin tricks should
be used only when you have control over all args and users. 

   template evalExpMap(string C, string F, A...) {
      enum evalExpMap = {
         import std.array;
         string s;
         static if (is(typeof(A))) alias B = typeof(A);
                else               alias B = A;
         foreach (I, _; B)
            s ~= (I?", ":"") ~ replace(F, "%", A[I].stringof);
         return replace(C, "%", s);
      }();
   }
   
   import std.stdio;

   void main() {
       auto c = mixin (evalExpMap!(q{ foo(%) }, q{ getArray(%)[] }, 1, "two",
3.14));
       b1(c, 3_000_000_000u, 2.14, -43L);
       auto s = S!(int, float, "/4", "blah")();
       writeln(typeof(s.data).stringof, typeof(s.blah).stringof);
       //auto oops = S!(typeof(as()), float, "/4", "blah")();
   }

   auto getArray(T)(T n) { T[2] data = n; return data; }

   auto foo(A...)(A args) {
       int c;
       foreach (rs; args) foreach (v; rs) ++c, write(v, " ");
       return c;
   }
   
   void b1(T...)(T values) {
       mixin (evalExpMap!(q{b2('\n',1,2,3,%);}, q{%+1}, values));
   }
   void b2(T...)(T vs) { foreach (v; vs) writeln(v); }
   
   template Q(A...) { alias Q=A; }
   struct S(A...) {
      mixin (evalExpMap!(q{alias T=Q!(%);}, q{%[16]}, A[0..$-2]));
      T data;
      mixin (evalExpMap!(q{Q!(%) }~A[3]~q{;}, q{%[16}~A[$-2]~q{]}, const A[1],
shared(A[0])*));
   }
   
   auto as() { struct An {} return An(); }


artur
Jul 07 2013
prev sibling parent reply "TommiT" <tommitissari hotmail.com> writes:
On Saturday, 6 July 2013 at 15:23:40 UTC, Artur Skawina wrote:

 ...


Can you tell me why this isn't working though?

import std.stdio, std.typetuple;

template _TypeMap(alias MAP, size_t N, TS...) {
   static if (N<TS.length)
       alias _TypeMap = _TypeMap!(MAP,
                                  N+1,
                                  TS[0..N],
                                  typeof(MAP(TS[N].init)),
                                  TS[N+1..$]);
   else
       alias _TypeMap = TS;
}
template TypeMap(alias MAP, TS...) {
  alias TypeMap = _TypeMap!(MAP, 0, TS);
}

struct _ForEach(alias MAP, TS...)
{
   TypeMap!(MAP, TS) tuple;

   this(TS values)
   {
       foreach (i, v; values)
           tuple[i] = MAP(v);
   }
}

auto ForEach(alias MAP, TS...)(TS ts)
{
   return _ForEach!(MAP, TS)(ts);
}


int[2] getArray(int n)
{
     int[2] data = n;
     return data;
}

void foo(R...)(R ranges)
{
     foreach (range; ranges)
         foreach (value; range)
             write(value, " ");
}

void main()
{
     alias pack = TypeTuple!(1, 2, 3);

     // prints random garbage
     foo(ForEach!(tmp => tmp[])
                 (ForEach!(a => getArray(a))(pack).tuple).tuple);
}
Jul 06 2013
parent "TommiT" <tommitissari hotmail.com> writes:
On Saturday, 6 July 2013 at 18:17:34 UTC, TommiT wrote:

 Can you tell me why this isn't working though?


I can get this so far that it prints: 1 1 2 0 <random> <random>

import std.stdio, std.typetuple;

template _TypeMap(alias MAP, size_t N, TS...) {
   static if (N<TS.length)
       alias _TypeMap = _TypeMap!(MAP,
                                  N+1,
                                  TS[0..N],
                                  typeof(MAP([TS[N]][0])),
                                  TS[N+1..$]);
   else
       alias _TypeMap = TS;
}
template TypeMap(alias MAP, TS...) {
  alias TypeMap = _TypeMap!(MAP, 0, TS);
}

struct _ForEach(alias MAP, TS...)
{
   TypeMap!(MAP, TS) tuple;

   this(TS values)
   {
       foreach (i, v; values)
           tuple[i] = MAP(v);
   }
}

auto ForEach(alias MAP, TS...)(TS ts)
{
   return _ForEach!(MAP, TS)(ts);
}


int[2] getArray(int n)
{
     int[2] data = n;
     return data;
}

void foo(R...)(R ranges)
{
     foreach (range; ranges)
         foreach (value; range)
             write(value, " ");
}

void main()
{
     alias pack = TypeTuple!(1, 2, 3);

     // prints random garbage
     foo(ForEach!((ref int[2] tmp) { return tmp[]; })
                 (ForEach!(a => getArray(a))(pack).tuple).tuple);
}
Jul 06 2013
prev sibling parent reply "TommiT" <tommitissari hotmail.com> writes:
On Saturday, 6 July 2013 at 12:21:29 UTC, Max Strakhov wrote:

 Artur, if i use your solution like

 printf("...", ForEach!(val => conv(val).c_str())(values).tuple);

 Than i would get a crash, because all the tuple elements would 
 be char*'s, pointing to already freed memory, as std::string's 
 destructor gets called each time right after alias function 
 exits. Ths is what c++ unpacking operator for: i actually 
 return a packed list of std::string's, apply .c_str() to each 
 and than unpack the list to current context.


I thought about this a bit more and realised you're right. The 
following should work though:

printf( "...", ForEach!(tmp => tmp.c_str())(ForEach!(val => 
conv(val))(values).tuple).tuple );

It's a defect in ForEach though, and gives me more reason to ask 
for the C++ style ellipsis expansion operator for D.
Jul 06 2013
parent reply "Max Strakhov" <monnoroch gmail.com> writes:
 The following should work though:

 printf( "...", ForEach!(tmp => tmp.c_str())(ForEach!(val => 
 conv(val))(values).tuple).tuple );


I had the same idea too, but couldn't make it work: compiler 
complains about failed CTFE or something. On the other hand i 
might do somethong wrong.


 It's a defect in ForEach though, and gives me more reason to 
 ask for the C++ style ellipsis expansion operator for D.


Yeap, i personally find (as * 2)... awesome feature of c++.
Jul 06 2013
parent "TommiT" <tommitissari hotmail.com> writes:
On Saturday, 6 July 2013 at 17:29:36 UTC, Max Strakhov wrote:

 The following should work though:

 printf( "...", ForEach!(tmp => tmp.c_str())(ForEach!(val => 
 conv(val))(values).tuple).tuple );


 I had the same idea too, but couldn't make it work: compiler 
 complains about failed CTFE or something. On the other hand i 
 might do somethong wrong.


There were a couple of typos in _TypeMap. That's a correct one:

template _TypeMap(alias MAP, size_t N, TS...) {
   static if (N<TS.length)
       alias _TypeMap = _TypeMap!(MAP, N+1, TS[0..N], 
typeof(MAP(TS[N].init)), TS[N+1..$]);
   else
       alias _TypeMap = TS;
}


 Yeap, i personally find (as * 2)... awesome feature of c++.


Me too.
Jul 06 2013