www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - static foreach considered

reply "Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> writes:
Walter and I are looking at ways to implement it. Here's a 
baseline without static foreach - a "trace" function that prints 
function calls before they are made:

http://dpaste.dzfl.pl/762c83c7fe30

If the function is overloaded, that won't work. In such cases, 
static foreach might be helpful. Here's code from the cycle "I 
have a dream":

http://dpaste.dzfl.pl/82a70c809210

I'm trying to collect together motivating examples and to figure 
out the semantics of the feature.


Andrei
Jun 08 2015
next sibling parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu wrote:
 Walter and I are looking at ways to implement it. Here's a 
 baseline without static foreach - a "trace" function that 
 prints function calls before they are made:

 http://dpaste.dzfl.pl/762c83c7fe30

 If the function is overloaded, that won't work. In such cases, 
 static foreach might be helpful. Here's code from the cycle "I 
 have a dream":

 http://dpaste.dzfl.pl/82a70c809210

 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.


 Andrei
How will scoping work? Similar to mixin templates? It would be nice together with this feature to be able to mixin identifiers: static foreach (ident; ["foo", "bar"]) { auto mixin(ident)() { // code for foo/bar } } Otherwise, other than overloads and template instantiations this won't be much better then generating code strings with CTFE...
Jun 08 2015
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, 8 June 2015 at 20:16:53 UTC, Idan Arye wrote:
 On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu 
 wrote:
 Walter and I are looking at ways to implement it. Here's a 
 baseline without static foreach - a "trace" function that 
 prints function calls before they are made:

 http://dpaste.dzfl.pl/762c83c7fe30

 If the function is overloaded, that won't work. In such cases, 
 static foreach might be helpful. Here's code from the cycle "I 
 have a dream":

 http://dpaste.dzfl.pl/82a70c809210

 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.


 Andrei
How will scoping work? Similar to mixin templates? It would be nice together with this feature to be able to mixin identifiers: static foreach (ident; ["foo", "bar"]) { auto mixin(ident)() { // code for foo/bar } } Otherwise, other than overloads and template instantiations this won't be much better then generating code strings with CTFE...
I would assume that it would be pretty much the same as doing foreach(T; TypeTuple!(...)) { ... } except that you're not forced to shove everything in a TypeTuple. - Jonathan M Davis
Jun 08 2015
next sibling parent "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 8 June 2015 at 21:14:46 UTC, Jonathan M Davis wrote:
 I would assume that it would be pretty much the same as doing

 foreach(T; TypeTuple!(...))
 {
     ...
 }

 except that you're not forced to shove everything in a 
 TypeTuple.

 - Jonathan M Davis
If that was the case, A library solution for converting a compile-time range to a TypeTuple would have sufficed(http://dpaste.dzfl.pl/7eb30f5e1156 - this compiles in 2.67). The problem with regular `foreach` over type tuple is that declarations inside the foreach's body are invisible from the outside. If `static foreach` had this limitation, Andrei's example wouldn't work since `trace` would be local to the body of the `static foreach`. This essentially renders the main usecase of this feature(declaring stuff) and leaves us with a loop unrolling optimization...
Jun 08 2015
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 6/8/15 2:14 PM, Jonathan M Davis wrote:
 I would assume that it would be pretty much the same as doing

 foreach(T; TypeTuple!(...))
 {
      ...
 }

 except that you're not forced to shove everything in a TypeTuple.
I'm even fine with the type tuple, just allow at declaration level and don't introduce a scope. Would be a great start. -- Andrei
Jun 08 2015
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/08/2015 10:16 PM, Idan Arye wrote:
 On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu wrote:
 Walter and I are looking at ways to implement it. Here's a baseline
 without static foreach - a "trace" function that prints function calls
 before they are made:

 http://dpaste.dzfl.pl/762c83c7fe30

 If the function is overloaded, that won't work. In such cases, static
 foreach might be helpful. Here's code from the cycle "I have a dream":

 http://dpaste.dzfl.pl/82a70c809210

 I'm trying to collect together motivating examples and to figure out
 the semantics of the feature.


 Andrei
How will scoping work? Similar to mixin templates? ...
I think the body should have access to a scope that is hidden from the outside which contains the loop variable, but declarations should be inserted into the enclosing scope like for static if.
 It would be nice together with this feature to be able to mixin
 identifiers:

      static foreach (ident; ["foo", "bar"])
      {
          auto mixin(ident)()
          {
              // code for foo/bar
          }
      }

 Otherwise, other than overloads and template instantiations this won't
 be much better then generating code strings with CTFE...
+1. Other use cases: auto mixin(ident) = x; identifier.list.mixin(ident).foo(); import std.mixin(ident);
Jun 08 2015
next sibling parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 8 June 2015 at 21:32:52 UTC, Timon Gehr wrote:
 I think the body should have access to a scope that is hidden 
 from the outside which contains the loop variable, but 
 declarations should be inserted into the enclosing scope like 
 for static if.
This would require some syntax to mark the declarations we want to expose. Maybe `out`? This is far better than the mixin template approach, since it'll alert us early about conflicts: static foreach (ident; ["a", "b", "a"]) { int mixin(ident ~ "1"); out int mixin(ident ~ "2"); } `a1` is created twice, but that's OK since it isn't marked with `out`. `a2` is declared twice and raises a compilation error because it's marked with `out`. This will ensure these kind of errors are detected early and the compilation error points to the exact place of declaration.
Jun 08 2015
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06/09/2015 12:12 AM, Idan Arye wrote:
 On Monday, 8 June 2015 at 21:32:52 UTC, Timon Gehr wrote:
 I think the body should have access to a scope that is hidden from the
 outside which contains the loop variable, but declarations should be
 inserted into the enclosing scope like for static if.
This would require some syntax to mark the declarations we want to expose. Maybe `out`? This is far better than the mixin template approach, since it'll alert us early about conflicts: static foreach (ident; ["a", "b", "a"]) { int mixin(ident ~ "1"); out int mixin(ident ~ "2"); } `a1` is created twice, but that's OK since it isn't marked with `out`. `a2` is declared twice and raises a compilation error because it's marked with `out`. This will ensure these kind of errors are detected early and the compilation error points to the exact place of declaration.
I actually intended all declarations in the body to be inserted into the enclosing scope, at least by default.
Jun 08 2015
parent "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 8 June 2015 at 22:16:50 UTC, Timon Gehr wrote:
 On 06/09/2015 12:12 AM, Idan Arye wrote:
 On Monday, 8 June 2015 at 21:32:52 UTC, Timon Gehr wrote:
 I think the body should have access to a scope that is hidden 
 from the
 outside which contains the loop variable, but declarations 
 should be
 inserted into the enclosing scope like for static if.
This would require some syntax to mark the declarations we want to expose. Maybe `out`? This is far better than the mixin template approach, since it'll alert us early about conflicts: static foreach (ident; ["a", "b", "a"]) { int mixin(ident ~ "1"); out int mixin(ident ~ "2"); } `a1` is created twice, but that's OK since it isn't marked with `out`. `a2` is declared twice and raises a compilation error because it's marked with `out`. This will ensure these kind of errors are detected early and the compilation error points to the exact place of declaration.
I actually intended all declarations in the body to be inserted into the enclosing scope, at least by default.
What about helper declarations that repeat in each static iteration? It can work like with mixin templates, where declarations hide each other(http://dpaste.dzfl.pl/c173395eb0cd), but that means that if there is a repeat in the declaration you do want to expose, the compiler will simply hide it without issuing an error. You will get an error when you try to access that declaration from somewhere else, but this error message is distant from the root cause both in time - you might only write the code that access the declaration created by that particular iteration much later in the development process - and space - the error will point to the point of usage, not the point of duplicate declaration. Also, if the point of usage is inside a template and depends on the template instantiation this kind of error is much harder to debug... As for exposing the declaration by default - unless there is a backward compatibility issue, it's usually best to make the most restrictive and contained version the default one. If not exposing is the default and someone neglects to mark the exposed declaration, it will fail immediately when they try to access it(and they will. Immediately. Because that's the code they are writing right now) and they can just add the annotation. But if exposing is the default, and someone neglects to mark the internal helpers as non-exposed, well - they better hope that there are duplications that'll expose their mistake. This is not always trivial: struct Foo(Types...) { static foreach (Type; Types) { static if (isSomeString!Type) { // I forgot to mark this as non-exposed void stringHelper() { // helper for strings } void doSomething(Type arg) { // Something that uses stringHelper } } else { void doSomething(Type arg) { // The non-string versio } } } } unittest { alias MyFoo = Foo!(int, float, string); // some tests with MyFoo } Since I only have one string in MyFoo's types list, `stringHelper` is only declared once. A month from now, when I try to create `Foo!(string, wstring)`, it'll create two `stringHelper`s and result in compilation error. Having an error show up a month later is not fun. It's much less fun when it it pops for someone else that now needs to figure out what you were trying to do... That's why I think not exposing should be the default. In that case, since `doSomething` is not marked as exposed, this will fail early, because we can safely assume the exposed functionality is being tested - even if I don't write a proper unit test, since `doSomething` is part of `Foo`'s API I will try to use it(if you write API without at least trying it out you deserve whatever method of torture the people that use that API can think of), so the bug will pop early.
Jun 08 2015
prev sibling next sibling parent reply "Tofu Ninja" <emmons0 purdue.edu> writes:
On Monday, 8 June 2015 at 22:12:06 UTC, Idan Arye wrote:
 On Monday, 8 June 2015 at 21:32:52 UTC, Timon Gehr wrote:
 I think the body should have access to a scope that is hidden 
 from the outside which contains the loop variable, but 
 declarations should be inserted into the enclosing scope like 
 for static if.
This would require some syntax to mark the declarations we want to expose. Maybe `out`? This is far better than the mixin template approach, since it'll alert us early about conflicts: static foreach (ident; ["a", "b", "a"]) { int mixin(ident ~ "1"); out int mixin(ident ~ "2"); } `a1` is created twice, but that's OK since it isn't marked with `out`. `a2` is declared twice and raises a compilation error because it's marked with `out`. This will ensure these kind of errors are detected early and the compilation error points to the exact place of declaration.
I think it would make more sense to behave similar to how static if currently does.
Jun 08 2015
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06/09/2015 12:36 AM, Tofu Ninja wrote:
 On Monday, 8 June 2015 at 22:12:06 UTC, Idan Arye wrote:
 On Monday, 8 June 2015 at 21:32:52 UTC, Timon Gehr wrote:
 I think the body should have access to a scope that is hidden from
 the outside which contains the loop variable, but declarations should
 be inserted into the enclosing scope like for static if.
This would require some syntax to mark the declarations we want to expose. Maybe `out`? This is far better than the mixin template approach, since it'll alert us early about conflicts: static foreach (ident; ["a", "b", "a"]) { int mixin(ident ~ "1"); out int mixin(ident ~ "2"); } `a1` is created twice, but that's OK since it isn't marked with `out`. `a2` is declared twice and raises a compilation error because it's marked with `out`. This will ensure these kind of errors are detected early and the compilation error points to the exact place of declaration.
I think it would make more sense to behave similar to how static if currently does.
No, static if should be fixed, and then it should behave similarly to the fixed version: void main(){ static if(is(int T==int)){} T x; // wtf? }
Jun 08 2015
prev sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 8 June 2015 at 22:12:06 UTC, Idan Arye wrote:
 On Monday, 8 June 2015 at 21:32:52 UTC, Timon Gehr wrote:
 I think the body should have access to a scope that is hidden 
 from the outside which contains the loop variable, but 
 declarations should be inserted into the enclosing scope like 
 for static if.
This would require some syntax to mark the declarations we want to expose. Maybe `out`? This is far better than the mixin template approach, since it'll alert us early about conflicts: static foreach (ident; ["a", "b", "a"]) { int mixin(ident ~ "1"); out int mixin(ident ~ "2"); } `a1` is created twice, but that's OK since it isn't marked with `out`. `a2` is declared twice and raises a compilation error because it's marked with `out`. This will ensure these kind of errors are detected early and the compilation error points to the exact place of declaration.
static foreach (ident; ["a", "b", "a"]) { private int mixin(ident ~ "1"); int mixin(ident ~ "2"); }
Jun 09 2015
prev sibling parent Nick Treleaven <nick geany.org> writes:
On Monday, 8 June 2015 at 21:32:52 UTC, Timon Gehr wrote:
 On 06/08/2015 10:16 PM, Idan Arye wrote:
 It would be nice together with this feature to be able to mixin
 identifiers:

      static foreach (ident; ["foo", "bar"])
      {
          auto mixin(ident)()
          {
              // code for foo/bar
          }
      }
+1. Other use cases: auto mixin(ident) = x; identifier.list.mixin(ident).foo(); import std.mixin(ident);
Browsing bugzilla, I've discovered Idan's feature was actually proposed in 2009, but with different syntax: int __ident(name)() {...} This seems better syntax, see: https://issues.dlang.org/show_bug.cgi?id=2698#c3
May 14 2018
prev sibling next sibling parent "Daniel N" <ufo orbiting.us> writes:
On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu wrote:
 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.


 Andrei
Hmmm, codegen based on UDA? struct magic { interface readonly; private static struct hidden { readonly enum a = 0; } static foreach(id; __traits(allMembers, hidden)) { // Inspect UDA and generate code accordingly, // Ex fields with public ReadOnly and private ReadWrite } }
Jun 08 2015
prev sibling next sibling parent reply "rsw0x" <anonymous anonymous.com> writes:
On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu wrote:
 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.
maybe not completely related, but I made a blog post on using CTFE to unroll foreach at compiletime https://rsw0x.github.io/post/switch-unrolling/ I find myself often writing recursive templates for compile-time generation of constructs that could be done cleaner with static foreach.
Jun 08 2015
parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Monday, 8 June 2015 at 22:15:32 UTC, rsw0x wrote:
 On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu 
 wrote:
 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.
maybe not completely related, but I made a blog post on using CTFE to unroll foreach at compiletime https://rsw0x.github.io/post/switch-unrolling/ I find myself often writing recursive templates for compile-time generation of constructs that could be done cleaner with static foreach.
I also use this method alot, and sometimes encounter this "bug": http://dpaste.dzfl.pl/16af3c5dad73 The break inside the `foreach` is breaking from the `foreach`, not from the `switch`, so it continues to execute the `default` clause. This is not really a bug - `foreach` unrolling is more of a loop unrolling optimization that we hijack, so it makes sense `break` inside it will act like it's inside a regular `foreach`. With `static foreach`, we might want `break`(and `continue`) to operate on the containing, runtime control structure.
Jun 08 2015
parent reply "rsw0x" <anonymous anonymous.com> writes:
On Tuesday, 9 June 2015 at 00:01:07 UTC, Idan Arye wrote:
 On Monday, 8 June 2015 at 22:15:32 UTC, rsw0x wrote:
 On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu 
 wrote:
 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.
maybe not completely related, but I made a blog post on using CTFE to unroll foreach at compiletime https://rsw0x.github.io/post/switch-unrolling/ I find myself often writing recursive templates for compile-time generation of constructs that could be done cleaner with static foreach.
I also use this method alot, and sometimes encounter this "bug": http://dpaste.dzfl.pl/16af3c5dad73 The break inside the `foreach` is breaking from the `foreach`, not from the `switch`, so it continues to execute the `default` clause. This is not really a bug - `foreach` unrolling is more of a loop unrolling optimization that we hijack, so it makes sense `break` inside it will act like it's inside a regular `foreach`. With `static foreach`, we might want `break`(and `continue`) to operate on the containing, runtime control structure.
I knew there was something I was forgetting in that short example, thanks for the reminder. Interestingly, the assembly generated with `break` and `break label` with a label on the switch is exactly the same. I don't have time right now to go review the spec, so I have no idea if that's correct.
Jun 08 2015
parent "Idan Arye" <GenericNPC gmail.com> writes:
On Tuesday, 9 June 2015 at 00:48:34 UTC, rsw0x wrote:
 On Tuesday, 9 June 2015 at 00:01:07 UTC, Idan Arye wrote:
 On Monday, 8 June 2015 at 22:15:32 UTC, rsw0x wrote:
 On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu 
 wrote:
 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.
maybe not completely related, but I made a blog post on using CTFE to unroll foreach at compiletime https://rsw0x.github.io/post/switch-unrolling/ I find myself often writing recursive templates for compile-time generation of constructs that could be done cleaner with static foreach.
I also use this method alot, and sometimes encounter this "bug": http://dpaste.dzfl.pl/16af3c5dad73 The break inside the `foreach` is breaking from the `foreach`, not from the `switch`, so it continues to execute the `default` clause. This is not really a bug - `foreach` unrolling is more of a loop unrolling optimization that we hijack, so it makes sense `break` inside it will act like it's inside a regular `foreach`. With `static foreach`, we might want `break`(and `continue`) to operate on the containing, runtime control structure.
I knew there was something I was forgetting in that short example, thanks for the reminder. Interestingly, the assembly generated with `break` and `break label` with a label on the switch is exactly the same. I don't have time right now to go review the spec, so I have no idea if that's correct.
Why wouldn't they? If we neglect RAII(for simplicity), `break` jumps to the first instruction after the `foreach`, and `break label` with a label on the `switch` jumps to the first instruction after the `switch`. Since there is nothing in the `switch` after the `foreach`, the first instruction after the `foreach` is also the first instruction after the `switch`, so the command to jump to that instruction is the same, and the assembly is the same.
Jun 08 2015
prev sibling next sibling parent Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 06/08/15 22:02, Andrei Alexandrescu via Digitalmars-d wrote:
 Walter and I are looking at ways to implement it. Here's a baseline without
static foreach - a "trace" function that prints function calls before they are
made:
 
 http://dpaste.dzfl.pl/762c83c7fe30
 
 If the function is overloaded, that won't work. In such cases, static foreach
might be helpful. Here's code from the cycle "I have a dream":
 
 http://dpaste.dzfl.pl/82a70c809210
The goal should be more like: template trace(alias fun) { static foreach (O; Overloads!fun) auto ref trace(Parameters!O args) { write("Tracing: ", __traits(identifier, fun), "("); foreach (i, arg; args) { if (i) write(", "); write(arg); } writeln(")"); return fun(args); } } And, btw, one of your asserts in that dreamy example has an off-by-one bug. Of course the only reason I spotted it is because I actually ran the code. ;) [1] artur [1] My old static-foreach hack makes it look like this: template trace(alias fun) { alias OS = Overloads!fun; #foreach (I, _; OS) { auto ref trace(Parameters!(OS[$I]) args) { write("Tracing: ", __traits(identifier, fun), "("); foreach (i, arg; args) { if (i) write(", "); write(arg); } writeln(")"); return fun(args); } } } which obviously isn't that pretty, but has the advantage of working right now. But w/o a built-in implementation it (a) is a bit too verbose, and (b) doesn't nest very well; a built-in version wouldn't have these problems.
Jun 08 2015
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu wrote:
 Walter and I are looking at ways to implement it. Here's a 
 baseline without static foreach - a "trace" function that 
 prints function calls before they are made:

 http://dpaste.dzfl.pl/762c83c7fe30

 If the function is overloaded, that won't work. In such cases, 
 static foreach might be helpful. Here's code from the cycle "I 
 have a dream":

 http://dpaste.dzfl.pl/82a70c809210

 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.


 Andrei
There was also http://wiki.dlang.org/DIP57 and relevant discussion thread
Jun 08 2015
prev sibling next sibling parent reply "Daniel N" <ufo orbiting.us> writes:
On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu wrote:
 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.


 Andrei
Could something like this fly? struct weird { void xxx()(T val) if(T.sizeof <= 8); void xxx()(const auto ref T val) if(T.sizeof > 8); // would require something similar to getOverloads but for templates...? static foreach(decl; __traits(getTemplates, typeof(this), "xxx")) { // impl xxx only once } } Or is there a better way to accomplish the above? Daniel
Jun 16 2015
parent reply "Daniel N" <ufo orbiting.us> writes:
On Tuesday, 16 June 2015 at 20:39:44 UTC, Daniel N wrote:
 Or is there a better way to accomplish the above?

 Daniel
Sorry for the noise, ofc there's a better way... struct weird(T) { static if(T.sizeof<=8) { void proto_val(T val); alias U = ParameterTypeTuple!proto_val; } else { void proto_ref(const ref T val); alias U = ParameterTypeTuple!proto_ref; } void zzz(U x) { } } Daniel
Jun 16 2015
parent "Daniel N" <ufo orbiting.us> writes:
On Tuesday, 16 June 2015 at 21:21:26 UTC, Daniel N wrote:
 On Tuesday, 16 June 2015 at 20:39:44 UTC, Daniel N wrote:
 Or is there a better way to accomplish the above?
Yet another snippet. static foreach(param; ParameterTypeTuple!((ref int _1) {})) { // in my dream it remains 'ref int', doesn't decay into 'int'. void fun(param p) {} }
Jun 17 2015
prev sibling parent "jmh530" <john.michael.hall gmail.com> writes:
On Monday, 8 June 2015 at 20:02:11 UTC, Andrei Alexandrescu wrote:
 I'm trying to collect together motivating examples and to 
 figure out the semantics of the feature.
I was just playing around with something and I thought being able to do a loop over an enum at compile time would be convenient. The motivation is when you have a function that works a little differently for arrays than for scalers, but it works the same way for all sizes of arrays (i.e. 1d/2d/3d all the same). template GenArrayMathFunction(string function_name, string array_string) { const char[] GenArrayMathFunction = "real" ~ array_string ~ " " ~ function_name ~"(real" ~ array_string ~ " x) {\n" ~ "\treal" ~ array_string ~" result = x.map!(a => " ~ function_name ~ "(a)).array;\n" ~ "\treturn result;\n" ~ "}"; } private enum array_dimension { Dim1 = "[]", Dim2 = "[][]", Dim3 = "[][][]" } static foreach(i; array_dimension) { mixin(GenArrayMathFunction!(f, i)); } Basically, the first part generates a string representing a function that uses map to apply a function to some input. The type of the input depends on a string representing the kind of array. The enum represents a string for 1/2/3-dimensional arrays. Finally, I would want to loop through the enums and use mixin to create a function for each. The foreach loop doesn't work right now. Right now, I've just written out each of the three functions and put in array_dimension.Dim1, etc.
Jun 16 2015