www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Templates and stringof...

reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
  This is moved/copied from the D.learning group. I need opinions 
on the matter, and thoughts perhaps from Walter or Andrei.

  Here are the highlights

---

While working on bitfields code I've found a unique scenario that 
poses some annoyances when generating the code.

template XYZ(alias x) {
     enum XYZ = x.stringof ~ "=100;";
}

struct I { int i;}

I i_num;
int n;

mixin(XYZ!(i_num.i)); //cannot find variable i
mixin(XYZ!(n));

the mixins become:
i=100;
n=100;

  Now, how do I get the template's stringof to print out 'i_num.i' 
and not 'i'?

On Friday, 3 August 2012 at 21:19:08 UTC, David Nadlinger wrote:


 You don't. Using .stringof in conjunction with string mixins is 
 The Wrong Thing (tm) in virtually all cases.

 What do you want to achieve? Why can't you pass a string 
 instead?


  Because a string doesn't hold it's type information for size 
checking.

On Friday, 3 August 2012 at 22:47:52 UTC, David Nadlinger wrote:

 On Friday, 3 August 2012 at 22:23:23 UTC, Era Scarecrow wrote:

 Seems like an ugly hack though (to get this done). Why not 
 have another method of fullpathStringof or something similar? 
 Then again if this is one of the few cases that could benefit 
 from it, then maybe we should make it ugly so no one else will 
 use it.


 This can't work in general. What should such a function return? 
 The fully qualified name, I.e. including packages and modules? 
 What is if the referred to entity is a nested function/local 
 variable? What is if it is defined in a module which is not 
 imported in the place where the string is mixed in?


  I would think, the exact 'text' of the way the variable was 
called, in my example it would be "inum.i", and since it would be 
at the local level (where the mixin was called) it should have 
access to everything that it did at that time.


On Friday, 3 August 2012 at 22:50:54 UTC, David Nadlinger wrote:

 No, it shouldn't be. There are almost no legit use cases for 
 it, and having it in the library encourages abuse of string 
 mixins/stringof in cases like this one.


  Then doesn't it seem like we're missing a potentially important 
piece of the puzzle for mixins and templates? very likely my 
modified template will include you including the same variable 
twice, but if someone gets lazy then it may not work.


  True I can add asserts as part of the output code, but as 
mentioned hopefully the constraints could do that work, plus 
adding extra checks as part of a mixin seems a little...  
excessive, dirty and ugly.

  Let's assume we use 'localStringof' or 'callingStringof' and 
that it returns the string of how the variable is 
called/referenced; Give me some examples how it would be abused 
or used wrongly? Aliased variables are pretty much perfect 
forwarded in templates (Unless my understanding is off) so they 
would carry that information forward.
Aug 03 2012
next sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 4 August 2012 at 03:07:21 UTC, Era Scarecrow wrote:

 Then doesn't it seem like we're missing a potentially important 
 piece of the puzzle for mixins and templates? very likely my 
 modified template will include you including the same variable 
 twice, but if someone gets lazy then it may not work.


  Well I have a working work-around that I need to test further, 
however it does the job; Although for other applications it may 
fail miserably on, and has very limited application.

template bitfieldsOn(string storeName, T...) {
   enum bitfieldsOn = "mixin(bitfieldsOn_b!(\"" ~ storeName ~ 
"\"," ~ storeName ~ TupleToString!(T) ~ "));";
}

template TupleToString(T...) {
   static if (T.length)
     enum TupleToString = "," ~ T[0].stringof ~ TupleToString!(T[1 
.. $]);
   else
     enum TupleToString = "";
}

so this makes...
   mixin(bitfieldsOn!("bits",
     ulong, "back",  32,
     ulong, "", 32));

and becomes:
   mixin(bitfieldsOn_b!("bits",bits,ulong,"back",32,ulong,"",32));
Aug 03 2012
prev sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Saturday, 4 August 2012 at 03:07:21 UTC, Era Scarecrow wrote:

 This can't work in general. What should such a function 
 return? The fully qualified name, I.e. including packages and 
 modules? What is if the referred to entity is a nested 
 function/local variable? What is if it is defined in a module 
 which is not imported in the place where the string is mixed 
 in?


  I would think, the exact 'text' of the way the variable was 
 called, in my example it would be "inum.i", and since it would 
 be at the local level (where the mixin was called) it should 
 have access to everything that it did at that time.


How do you determine what the »local level« is? A string mixin 
isn't something you can »call«, just a compile-time constant 
string, which can be »evaluated« in a completely different 
place than it can be constructed. Maybe some heuristic could be 
used, but it's not worth the trouble, in my opinion.


  True I can add asserts as part of the output code, but as 
 mentioned hopefully the constraints could do that work, plus 
 adding extra checks as part of a mixin seems a little...  
 excessive, dirty and ugly.


Why would it be dirty? String mixins are just compile-time code 
generation. Also, using template constraints has almost zero 
benefits if overload resolution is not involved anyway – 
remember, this is just about the helper function/template 
generating the string to be mixed in.


  Then doesn't it seem like we're missing a potentially 
 important piece of the puzzle for mixins and templates? very 
 likely my modified template will include you including the same 
 variable twice, but if someone gets lazy then it may not work.


As I said, there are almost always ways to achieve what you want 
without resorting to getting the string representation of an 
alias parameter. For example:

---
mixin template BitfieldsOn(alias target, <…>) if 
(isIntegral!(typeof(target))) {
     mixin({
         string code;
         // Generate code using "target" as identifier.
         return code;
     }());
}

mixin BitfieldsOn!(foo, <…>);
---

David
Aug 04 2012
next sibling parent reply "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 4 August 2012 at 08:53:20 UTC, David Nadlinger wrote:

 How do you determine what the »local level« is? A string 
 mixin isn't something you can »call«, just a compile-time 
 constant string, which can be »evaluated« in a completely 
 different place than it can be constructed. Maybe some 
 heuristic could be used, but it's not worth the trouble, in my 
 opinion.


Mmm I would define it as the first one to call on any templates, 
at least if you plan on mixing stuff at that spot, further 
called/aliased stuff carries over the same information regardless 
how many levels deep it goes. If it began inside a template or 
from another function then I really don't know, probably holds 
the local scope from that level (or none at all).


 Why would it be dirty? String mixins are just compile-time code 
 generation. Also, using template constraints has almost zero 
 benefits if overload resolution is not involved anyway – 
 remember, this is just about the helper function/template 
 generating the string to be mixed in.


Depending on how many asserts I need to add in the string output 
seems a bit much if there may be more for the checks after the 
generation than before, or the ratio of checks in the returned 
string outweigh the actual code you want; Or so I would think. If 
all the checks are static than it won't show up in the 
executable, but if you need to output it out to glance over the 
generated code it would be quite messy to sift through it, 
depending on how complex it got.

But I'm probably wrong.


 As I said, there are almost always ways to achieve what you 
 want without resorting to getting the string representation of 
 an alias parameter. For example:



 ---
 mixin template BitfieldsOn(alias target, <…>) if 
 (isIntegral!(typeof(target))) {
     mixin({
         string code;
         // Generate code using "target" as identifier.
         return code;
     }());
 }

 mixin BitfieldsOn!(foo, <…>);


  I really don't see how that is an improvement. If the only way 
to use the mixin properly is the full variable location as a 
string (and stringof won't do) than the only acceptable input is 
a string; Since strings won't hold the type information...

  It ends up with a mixin calling another mixin.
Aug 04 2012
parent reply "David Nadlinger" <see klickverbot.at> writes:
On Saturday, 4 August 2012 at 10:08:34 UTC, Era Scarecrow wrote:

 ---
 mixin template BitfieldsOn(alias target, <…>) if 
 (isIntegral!(typeof(target))) {
    mixin({
        string code;
        // Generate code using "target" as identifier.
        return code;
    }());
 }

 mixin BitfieldsOn!(foo, <…>);


  I really don't see how that is an improvement.


It's a sketch of an implementation for your template which 
actually works with foo being passed only as an alias parameter. 
It's not a string mixin, but I tend to avoid exposing string 
mixins to user code as much as possible anyway.

I'm not sure what you mean with »[…] the only acceptable input 
is a string; Since strings won't hold the type information...« 
– the input to BitfieldsOn is _not_ a string, it is an alias. 
The trick is that you can refer to it as »target« inside the 
mixin template, so the problem of obtaining the »real name« of 
something doesn't even arise.

David
Aug 04 2012
next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sat, Aug 4, 2012 at 1:23 PM, David Nadlinger <see klickverbot.at> wrote:

 On Saturday, 4 August 2012 at 10:08:34 UTC, Era Scarecrow wrote:

 ---
 mixin template BitfieldsOn(alias target, <=E2=80=A6>) if
 (isIntegral!(typeof(target))) {
    mixin({
        string code;
        // Generate code using "target" as identifier.
        return code;
    }());
 }

 mixin BitfieldsOn!(foo, <=E2=80=A6>);



  I really don't see how that is an improvement.



 It's a sketch of an implementation for your template which actually works
 with foo being passed only as an alias parameter. It's not a string mixin=


,

 but I tend to avoid exposing string mixins to user code as much as possib=


le

 anyway.

 I'm not sure what you mean with =C2=BB[=E2=80=A6] the only acceptable inp=


ut is a string;

 Since strings won't hold the type information...=C2=AB =E2=80=93 the inpu=


t to BitfieldsOn

 is _not_ a string, it is an alias. The trick is that you can refer to it =


as

 =C2=BBtarget=C2=AB inside the mixin template, so the problem of obtaining=


 the =C2=BBreal

 name=C2=AB of something doesn't even arise.


Yes, it's a design I like:

- using a standard template gives you problem when you try to 'send'
the name in another module, as David demonstrated.
- using a string mixin is indeed a little less 'clean', as the use
might get his grubby hands on the string and then, who knows what may
happen.
- but using a mixin template that incorporates a string-generating
block is good, as you get the full access to types and symbols and can
still incorporate any D code inside the delegate/block.

Who here knows that D blocks are also void delegate()'s?
Aug 04 2012
prev sibling parent "Era Scarecrow" <rtcvb32 yahoo.com> writes:
On Saturday, 4 August 2012 at 11:23:07 UTC, David Nadlinger wrote:

 I'm not sure what you mean with »[…] the only acceptable 
 input is a string; Since strings won't hold the type 
 information...« – the input to BitfieldsOn is _not_ a 
 string, it is an alias. The trick is that you can refer to it 
 as »target« inside the mixin template, so the problem of 
 obtaining the »real name« of something doesn't even arise.


  Honestly i don't understand. In my code i was referring to the 
variable as 'target'; But when came time to return a string it 
needs the full name/location to refer to (as a string) to inject 
the code, which then the 'stringof' didn't help.

  I've already done a workaround that resolves the problem, we can 
refer back to this another time if another solution presents 
itself.
Aug 04 2012
prev sibling parent reply Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sat, Aug 4, 2012 at 10:53 AM, David Nadlinger <see klickverbot.at> wrote=
:


 example:

 ---
 mixin template BitfieldsOn(alias target, <=E2=80=A6>) if
 (isIntegral!(typeof(target))) {
     mixin({
         string code;
         // Generate code using "target" as identifier.
         return code;
     }());
 }

 mixin BitfieldsOn!(foo, <=E2=80=A6>);
 ---


Yes, I recently rediscovered this trick of having a mixin integrate an
anonymous delegate where code-as-string is generated. This is a very
nice interaction between different parts of D. This should be
explained somewhere, as it's a tool anyone using templates to do
metaprogramming should be aware of.

I'll update my tutorial on templates.

Last example in line, I needed to have a gensym, a 'fresh' symbol,
guaranteed not to be aleady defined in the local scope. That is, user
code uses:

mixin(gensym)

to get a fresh identifier. D templates are becoming powerful enough
that something like Lisp macros is becoming possible and, hence,
hygiene problems. (Hygiene, for those reading this, is having a
template use an already-defined variable and breaking local code, it's
not a quip on the personal habits of Lisp hackers)

Anyway, so, this gensym uses the exact same pattern.
Aug 04 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/04/2012 01:35 PM, Philippe Sigaud wrote:

 On Sat, Aug 4, 2012 at 10:53 AM, David Nadlinger<see klickverbot.at>  wrote:


 example:

 ---
 mixin template BitfieldsOn(alias target,<…>) if
 (isIntegral!(typeof(target))) {
      mixin({
          string code;
          // Generate code using "target" as identifier.
          return code;
      }());
 }

 mixin BitfieldsOn!(foo,<…>);
 ---


 Yes, I recently rediscovered this trick of having a mixin integrate an
 anonymous delegate where code-as-string is generated. This is a very
 nice interaction between different parts of D. This should be
 explained somewhere, as it's a tool anyone using templates to do
 metaprogramming should be aware of.


http://d.puremagic.com/issues/show_bug.cgi?id=7653
Aug 04 2012
parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sat, Aug 4, 2012 at 3:04 PM, Timon Gehr <timon.gehr gmx.ch> wrote:


 This should be
 explained somewhere, as it's a tool anyone using templates to do
 metaprogramming should be aware of.


 http://d.puremagic.com/issues/show_bug.cgi?id=7653


Yes. There is a workaround, but we lose the elegance of the previous solution.

mixin template AddMembers()
{
    private static string __AddMembers()
    {
        string r;
        foreach(x;["a","b","c"]) r~="int "~x~";";
        return r;
    }
    mixin(__AddMembers());
}

struct S
{
    mixin AddMembers;
}

void main()
{
    S s;
    writeln(s.a);
}

The limit is of course this introduce a new symbol in the local scope
(here, __AddMembers, made private and static).
Not that the collision risk with user code is great, the catch is if a
second instantiation of an equivalent template is done, thus trying to
create in the scope an already defined symbol.

Hence my need for a gensym template I alluded to in a previous
message. AddMembers would first create a fresh, non-already-used-here
symbol and sue that as its helper function's name.

And... drat, my gensym use the same trick, so back to the starting
point. I'll vote this bug up, one never knows.
Aug 04 2012