www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - strange CFTE issue

reply Inquie <Inquie data1.com> writes:
If I do something like

enum X = Methods!(C);

foreach(x; X)
{
     mixin(x);
}

I get an error about x not being a compile time variable. (code 
above is simplified, the error has nothing to do with the form 
but of the foreach(x )

but if I wrap it in a function it works

string foo()
{
enum X = Methods!(C);
string y = "";
foreach(x; X)
{
    y ~= x;
}
  return y;
}

mixin(y);

The only diff, of course, is the foreach in the first case mixes 
in on each iteration, while in the second it doesn't... but it 
shouldn't matter. in both cases x is the same.. and it definitely 
is a compile time constant in both. (Methods is just a wrapper on 
__traits(allMembers) but does some manipulation (stole the code 
from somewhere on the forum))
Mar 14 2017
parent reply ag0aep6g <anonymous example.com> writes:
On 03/15/2017 03:01 AM, Inquie wrote:

 If I do something like

 enum X = Methods!(C);

 foreach(x; X)
 {
     mixin(x);
 }

 I get an error about x not being a compile time variable. (code above is
 simplified, the error has nothing to do with the form but of the
 foreach(x )


"Compile time variable" may be misleading here. The compiler does not 
try to figure out what values are actually constant at compile time. 
Rather, the language defines some specific cases where it's guaranteed 
that a value is a compile-time constant. Only then can you use it in 
static contexts such as mixins. Other values are rejected, even if they 
would turn out be constant on a closer look.

`foreach` is mostly a run-time feature. There is a special case when you 
iterate over a "compile-time list" [1] (aka AliasSeq, formerly 
TypeTuple). In that case, the loop variable is recognized as a constant. 
That variant of `foreach` is also dubbed a "static foreach" (even though 
the `static` keyword is not used).

Note that it's not a "static foreach" when you iterate over an array, no 
matter if that array is constant at compile time or not.

So this works:

     import std.meta: AliasSeq;
     foreach (x; AliasSeq!("int foo;", "double bar;")) mixin(x);

But this doesn't:

     foreach (x; ["int foo;", "double bar;"]) mixin(x);

This is expected and works as intended. Without the definition of your 
`Methods` template, I can't say for sure if you're hitting this or if 
something else is going on. But if your `X` is an array, this is it.


 but if I wrap it in a function it works

 string foo()
 {
 enum X = Methods!(C);
 string y = "";
 foreach(x; X)
 {
    y ~= x;
 }
  return y;
 }

 mixin(y);


(I'm assuming that last line should be `mixin(foo());`.)

The return value of `foo` is recognized as a compile-time constant 
there, because you call it in a context that forces it. This is called CTFE.

Note that you cannot assign the result of `foo()` to a variable first:

    string y = foo(); /* no CTFE */
    mixin(y); /* no go */

That's because `y` is a normal variable, which is not a recognized 
compile-time constant. The value could of course be evaluated at 
compile-time, but the compiler doesn't attempt CTFE opportunistically.


 The only diff, of course, is the foreach in the first case mixes in on
 each iteration, while in the second it doesn't... but it shouldn't
 matter. in both cases x is the same.. and it definitely is a compile
 time constant in both.


To the compiler it's not a "compile-time constant" in either of them (in 
a rather specific sense of the term "compile-time"). During CTFE, 
run-time rules apply. So in `foo`, `x` is a normal variable. The same 
rules apply as for actual run-time variables. Only the return value of 
`foo` is seen as a compile-time value by the compiler.

You're not the first one who stumbles over this meaning of "compile 
time". CTFE happens at compile time, and has "compile time" in the name, 
but during CTFE you're actually dealing with "run time" values that 
might never see the actual run time. Maybe these things could use some 
better names.


[1] https://dlang.org/ctarguments.html
Mar 14 2017
parent reply Inquie <Inquie data1.com> writes:
On Wednesday, 15 March 2017 at 03:40:42 UTC, ag0aep6g wrote:

 On 03/15/2017 03:01 AM, Inquie wrote:

 If I do something like

 enum X = Methods!(C);

 foreach(x; X)
 {
     mixin(x);
 }

 I get an error about x not being a compile time variable. 
 (code above is
 simplified, the error has nothing to do with the form but of 
 the
 foreach(x )


 "Compile time variable" may be misleading here. The compiler 
 does not try to figure out what values are actually constant at 
 compile time. Rather, the language defines some specific cases 
 where it's guaranteed that a value is a compile-time constant. 
 Only then can you use it in static contexts such as mixins. 
 Other values are rejected, even if they would turn out be 
 constant on a closer look.

 `foreach` is mostly a run-time feature. There is a special case 
 when you iterate over a "compile-time list" [1] (aka AliasSeq, 
 formerly TypeTuple). In that case, the loop variable is 
 recognized as a constant. That variant of `foreach` is also 
 dubbed a "static foreach" (even though the `static` keyword is 
 not used).

 Note that it's not a "static foreach" when you iterate over an 
 array, no matter if that array is constant at compile time or 
 not.

 So this works:

     import std.meta: AliasSeq;
     foreach (x; AliasSeq!("int foo;", "double bar;")) mixin(x);

 But this doesn't:

     foreach (x; ["int foo;", "double bar;"]) mixin(x);

 This is expected and works as intended. Without the definition 
 of your `Methods` template, I can't say for sure if you're 
 hitting this or if something else is going on. But if your `X` 
 is an array, this is it.


 but if I wrap it in a function it works

 string foo()
 {
 enum X = Methods!(C);
 string y = "";
 foreach(x; X)
 {
    y ~= x;
 }
  return y;
 }

 mixin(y);


 (I'm assuming that last line should be `mixin(foo());`.)

 The return value of `foo` is recognized as a compile-time 
 constant there, because you call it in a context that forces 
 it. This is called CTFE.

 Note that you cannot assign the result of `foo()` to a variable 
 first:

    string y = foo(); /* no CTFE */
    mixin(y); /* no go */

 That's because `y` is a normal variable, which is not a 
 recognized compile-time constant. The value could of course be 
 evaluated at compile-time, but the compiler doesn't attempt 
 CTFE opportunistically.


 The only diff, of course, is the foreach in the first case 
 mixes in on
 each iteration, while in the second it doesn't... but it 
 shouldn't
 matter. in both cases x is the same.. and it definitely is a 
 compile
 time constant in both.


 To the compiler it's not a "compile-time constant" in either of 
 them (in a rather specific sense of the term "compile-time"). 
 During CTFE, run-time rules apply. So in `foo`, `x` is a normal 
 variable. The same rules apply as for actual run-time 
 variables. Only the return value of `foo` is seen as a 
 compile-time value by the compiler.

 You're not the first one who stumbles over this meaning of 
 "compile time". CTFE happens at compile time, and has "compile 
 time" in the name, but during CTFE you're actually dealing with 
 "run time" values that might never see the actual run time. 
 Maybe these things could use some better names.


 [1] https://dlang.org/ctarguments.html


Thanks, it explains it, but there is one difference. The array is 
assigned to an enum, so surely the compiler can figure that out? 
It should be similar to AliasSeq. I could probably wrap AliasSeq 
around it and it work or convert it to
an AliasSeq. Seems like an issue with the compiler.

Remember, it's


 enum X = Methods!(C);

 foreach(x; X)
 {
     mixin(x);
 }





and not


 string X = Methods!(C);

 foreach(x; X)
 {
     mixin(x);
 }




that should be quite difference and the static foreach in the 
first case and non-static foreach in the second.

Essentially what you are saying is that if I were to have 
Methods! return an AliasSeq then it would work. It is a CTFE that 
returns an array. If there is a function that can convert the the 
array to an AliasSeq of tuples there should be no problem, 
although I don't see how to do that, it should be possible?

But, this all then seems to be skirting the fact that the loop is 
still over a compile time constant(enum or AliasSeq, shouldn't 
matter) and should work.
Mar 15 2017
parent reply ag0aep6g <anonymous example.com> writes:
On 03/15/2017 02:00 PM, Inquie wrote:

 Thanks, it explains it, but there is one difference. The array is
 assigned to an enum, so surely the compiler can figure that out? It
 should be similar to AliasSeq.


The enum array is similar to an AliasSeq in that you can them both in 
contexts that require compile-time constants. But that doesn't mean that 
every context in which you put them gets evaluated at compile time.

Different example: If you write `if (true)` that's a run-time 
conditional even though `true` is a constant. By the language rules, the 
check is done at run time. It may of course be optimized out, but the 
language doesn't care about that. There's also a `static if`. If you 
write `static if (true)`, that check is done at compile time, by the 
language rules. That means you can only put compile-time constants as 
the condition of a `static if`. In other words, the `static if` forces 
compile-time evaluation of its condition.

This "forcing of compile-time evaluation" is how things work. 
Compile-time evaluation is only attempted when it's forced by the 
context. It's not attempted when run-time evaluation is possible. 
`enum`, `mixin`, and `static if` force compile-time evaluation. With 
`foreach` it's only forced when looping over an AliasSeq, because you 
can't do that at run-time.

As for why this is so: Consider that a run-time loop can take a long 
time (hours, days, ...) even if it's over a constant array. It's 
expected that the generated program runs for a long time. It would be 
surprising if the compilation were to take that long time, just because 
the compiler saw that it's possible.

By the way, I'd love to see an actual `static foreach`, with the 
`static` keyword. I'd have it work similar to `static if` and force 
compile-time semantics. Then (slowly) deprecate using plain `foreach` on 
AliasSeqs. Would make D code clearer, in my opinion. But this might have 
non-obvious issues. I haven't thought too hard about it.

[...]

 Remember, it's


 enum X = Methods!(C);

 foreach(x; X)
 {
     mixin(x);
 }





 and not


 string X = Methods!(C);

 foreach(x; X)
 {
     mixin(x);
 }




 that should be quite difference and the static foreach in the first case
 and non-static foreach in the second.


It would make a difference with the hypothetical `static foreach`, which 
would accept the first one but reject the second one.

With plain `foreach` it doesn't matter. Ignoring the body, the loop can 
be done at run-time in both versions, so that's what the compiler tries 
to do. Compile-time evaluation must be requested by the programmer. The 
compiler doesn't attempt it eagerly.


 Essentially what you are saying is that if I were to have Methods!
 return an AliasSeq then it would work.


Yup.


 It is a CTFE that returns an
 array.


You might use the term "CTFE" correctly here, but you also might use it 
incorrectly. Just to clarify the term, if `Methods` looks this:

     template Methods(T) { enum Methods = /* ... whatever ... */; }

or like this:

     enum Methods(T) = /* ... whatever ... */;

then it's not a function, but a template. CTFE may happen in the 
"whatever" part, but `Methods` itself does not go through CTFE as it's 
not a function.

But if `Methods` looks like this:

     string[] Methods(T)() { /* ... whatever ... */ }

then it is a function and does go through CTFE.


 If there is a function that can convert the the array to an
 AliasSeq of tuples there should be no problem, although I don't see how
 to do that, it should be possible?


Phobos has it: std.meta.aliasSeqOf "converts an input range [...] to an 
alias sequence." [1]

Applied to your code:

     enum X = ["int foo;", "double bar;"];
     foreach(x; aliasSeqOf!X)
     {
         mixin(x); /* works */
     }


 But, this all then seems to be skirting the fact that the loop is still
 over a compile time constant(enum or AliasSeq, shouldn't matter) and
 should work.


Yeah, no. A compile-time constant being used doesn't mean anything. It's 
the context in which it is used that forces compile-time evaluation.
Mar 15 2017
parent reply ag0aep6g <anonymous example.com> writes:
 Phobos has it: std.meta.aliasSeqOf "converts an input range [...] to an
 alias sequence." [1]


Woops, forgot to give the URL for that "[1]". Here it is:

http://dlang.org/phobos/std_meta.html#aliasSeqOf
Mar 15 2017
parent reply Boris-Barboris <ismailsiege gmail.com> writes:
On Wednesday, 15 March 2017 at 17:27:35 UTC, ag0aep6g wrote:

 Phobos has it: std.meta.aliasSeqOf "converts an input range 
 [...] to an
 alias sequence." [1]


 Woops, forgot to give the URL for that "[1]". Here it is:

 http://dlang.org/phobos/std_meta.html#aliasSeqOf


Hello, I have a similar problem. For the life of me I can't make 
CTFE work while manipulating collections. Source:

import std.meta;
import std.traits;

template isFunctionField(OwnerType, string field_name)
{
	enum isFunctionField =
		isFunction(mixin(OwnerType.stringof ~ "." ~ field_name));
}

string[] sfilter(T)(string[] fields)
{
	string[] result;
	foreach (f; fields)
	{
		enum isfunc = isFunctionField!(T, f);
// variable f cannot be read at compile time.
// Even when I change isFunctionField from template to function 
that returns
// bool and takes "string field_name".
		if (isfunc)
			result ~= f;
	}
	return result;
}

string[] TypeFields(T)() pure
{
	enum field_names = [__traits(allMembers, T)];
	pragma(msg, typeof(field_names)); // prints string[]
	pragma(msg, field_names);         // prints correct member names
	enum filtered = sfilter!(T)(field_names);
	return filtered;
}

//then I call it by:
enum fields = TypeFields!(SomeStruct)();

   I can get string array of class members allright. Whenever I 
try to do any logic with it, I fail. I get that "enum field_names 
= " is not lvalue. How should I pass it to sfilter? When I change 
foreach loop into for loop, I can't index "string[] fields" array 
since "variable fields cannot be read at compile time".
   Should I pass field_names string array as some other type, 
AliasSeq or something?

   The thing is, on some level inside TypeFields, I will need to 
mutate array, I can't stick to tuples. I tried declaring array 
parameters immutable, but CTFE forbids casting from mutable into 
immutable. I'm kinda out of ideas.
Apr 09 2017
parent reply ag0aep6g <anonymous example.com> writes:
On 04/09/2017 04:36 PM, Boris-Barboris wrote:

 Hello, I have a similar problem. For the life of me I can't make CTFE
 work while manipulating collections. Source:


This post is only loosely related to the thread. Generally, please make 
a new thread for a new problem.


 import std.meta;
 import std.traits;

 template isFunctionField(OwnerType, string field_name)
 {
     enum isFunctionField =
         isFunction(mixin(OwnerType.stringof ~ "." ~ field_name));


Aside: You forgot an exclamation mark here. isFunction is a template, 
not a function.


 }

 string[] sfilter(T)(string[] fields)
 {
     string[] result;
     foreach (f; fields)
     {
         enum isfunc = isFunctionField!(T, f);
 // variable f cannot be read at compile time.
 // Even when I change isFunctionField from template to function that
 returns
 // bool and takes "string field_name".
         if (isfunc)
             result ~= f;
     }
     return result;
 }

 string[] TypeFields(T)() pure
 {
     enum field_names = [__traits(allMembers, T)];
     pragma(msg, typeof(field_names)); // prints string[]
     pragma(msg, field_names);         // prints correct member names
     enum filtered = sfilter!(T)(field_names);
     return filtered;
 }

 //then I call it by:
 enum fields = TypeFields!(SomeStruct)();

   I can get string array of class members allright. Whenever I try to do
 any logic with it, I fail. I get that "enum field_names = " is not
 lvalue.


As far as I see, lvalue vs rvalue doesn't matter here. It's compile-time 
constant vs variable. Or maybe "static value" vs "dynamic value".


 How should I pass it to sfilter?


Via a template parameter. Function parameters are never considered 
static values (or compile-time constants), even during CTFE. It's 
important to understand and to keep in mind that CTFE follows the exact 
same rules as normal run-time evaluation (plus some more restrictions 
that apply only to CTFE). You can't define an enum with true run-time 
data, so you can't define it with a variable in CTFE either.

You can make `fields` a template sequence parameter [1]:

----
string[] sfilter(T, fields ...)()
{
     /* ... body as you have it ... */
}
----

Or you can make it a template value parameter [2] with type `string[]`:

----
string[] sfilter(T, string[] fields)()
{
     string[] result;
     foreach (f; aliasSeqOf!fields)
     {
         /* ... loop body as you have it ... */
     }
     return result;
}
----

Both times, the `foreach` operates on a compile-time list [3] (aka alias 
sequence aka AliasSeq). This is a special case of `foreach` dubbed 
"static foreach". It means that the loop variable `f` is a static value. 
It can be passed as a template argument.

As an alternative to the "static foreach", you could use recursion. And 
then you could make it just a template instead of a function template. 
CTFE wouldn't be involved. That's how std.meta.Filter [4] works. You can 
use it instead of writing your own filter function/template:

----
     enum field_names = __traits(allMembers, T);
     enum pred(string field_name) = isFunctionField!(T, field_name);
     enum filtered = [Filter!(pred, field_names)];
----


 When I change foreach loop into
 for loop, I can't index "string[] fields" array since "variable fields
 cannot be read at compile time".


Unlike `foreach`, `for` does not have a "static for" special case. The 
loop variable of a `for` loop is always dynamic, never static. You can't 
ever use it in a context that needs a static value.


   Should I pass field_names string array as some other type, AliasSeq or
 something?


Aside: AliasSeq is not a type.


   The thing is, on some level inside TypeFields, I will need to mutate
 array, I can't stick to tuples.


It's not obvious to me why this rules out compile-time lists (aka 
tuples). Note that you can convert both ways between a compile-time list 
(aka tuple) and an enum array. aliasSeqOf goes one way, wrapping in 
square brackets goes the other.


 I tried declaring array parameters
 immutable, but CTFE forbids casting from mutable into immutable. I'm
 kinda out of ideas.


I suppose you tried `immutable` to tell the compiler that the values are 
constant? But `immutable` doesn't mean "compile-time constant". 
Dynamically computed values can be immutable. Function parameters are 
never usable in static contextsZ.


[1] https://dlang.org/spec/template.html#variadic-templates
[2] https://dlang.org/spec/template.html#template_value_parameter
[3] https://dlang.org/ctarguments.html
Apr 09 2017
parent Boris-Barboris <ismailsiege gmail.com> writes:
On Sunday, 9 April 2017 at 16:44:41 UTC, ag0aep6g wrote:


 Or you can make it a template value parameter [2] with type 
 `string[]`:

 ----
 string[] sfilter(T, string[] fields)()
 {
     string[] result;
     foreach (f; aliasSeqOf!fields)
     {
         /* ... loop body as you have it ... */
     }
     return result;
 }
 ----


Thank you, I didn't even expect it was possible, array as a 
template parameter is quite foreign to me after C++.


 Function parameters are never usable in static contexts.


   Good to know, thank you. So, there are no exceptions to this, 
even when parameter is very simple numeric type value?
Apr 09 2017