www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is old style compile-time foreach redundant?

reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
Is 'static foreach' sufficient for all needs or is there any value for 
regular foreach over compile-time sequences?

Code unrelated to the question:

import std.stdio;

void main() {
     // Old style compile-time foreach. This still works
     // when 'static' is uncommented below.
     import std.meta : AliasSeq;
     /* static */ foreach (i; AliasSeq!(1, "hello", 2)) {
         writeln(i);
     }

     // Proper 'static foreach'.
     import std.range : iota;
     import std.algorithm : map;
     static foreach (i; 3.iota.map!(a => a * 10)) {
         writeln(i);
     }
}

Ali
Jan 06 2018
next sibling parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Saturday, 6 January 2018 at 23:25:58 UTC, Ali Çehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any 
 value for regular foreach over compile-time sequences?


There is, as far as I've seen, there's nothing old 
foreach-over-tuple can do that new-style static foreach can't. 
The big difference is static foreach doesn't introduce a scope, 
which is great when that's what you want, and only requires an 
additional set of braces otherwise.

In some cases, especially when combined with mixins, this new 
behavior might manifest bugs at some later point. Simplified 
example:

mixin template declareInts(names...) {
     static foreach (name; names) {
         mixin("int "~name~";");
     }
}

As long as you use it with non-duplicate names, this will work 
great. It will fail the moment a junior programmer decides he 
wants two ints with the same name. Real-world examples will of 
course be more involved.

Old foreach is however not going away any time soon - there's far 
too many codebases out there that use the feature.

--
   Simen
Jan 06 2018
prev sibling next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Saturday, 6 January 2018 at 23:25:58 UTC, Ali Çehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any 
 value for regular foreach over compile-time sequences?

 Code unrelated to the question:

 import std.stdio;

 void main() {
     // Old style compile-time foreach. This still works
     // when 'static' is uncommented below.
     import std.meta : AliasSeq;
     /* static */ foreach (i; AliasSeq!(1, "hello", 2)) {
         writeln(i);
     }

     // Proper 'static foreach'.
     import std.range : iota;
     import std.algorithm : map;
     static foreach (i; 3.iota.map!(a => a * 10)) {
         writeln(i);
     }
 }

 Ali


No it's not.
When you can use the old style do so. Since it puts less stress 
on the compiler in the general case.
Jan 06 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Jan 07, 2018 at 12:55:27AM +0000, Stefan Koch via Digitalmars-d-learn
wrote:

 On Saturday, 6 January 2018 at 23:25:58 UTC, Ali ehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any value
 for regular foreach over compile-time sequences?




[...]

 No it's not.
 When you can use the old style do so. Since it puts less stress on the
 compiler in the general case.


Really? Based on a recent post by Jonathan Davis, the new static foreach
actually runs faster in certain use cases.


T

-- 
No! I'm not in denial!
Jan 06 2018
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Sunday, 7 January 2018 at 01:08:44 UTC, H. S. Teoh wrote:

 On Sun, Jan 07, 2018 at 12:55:27AM +0000, Stefan Koch via 
 Digitalmars-d-learn wrote:

 On Saturday, 6 January 2018 at 23:25:58 UTC, Ali Çehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any 
 value for regular foreach over compile-time sequences?




 [...]

 No it's not.
 When you can use the old style do so. Since it puts less 
 stress on the
 compiler in the general case.


 Really? Based on a recent post by Jonathan Davis, the new 
 static foreach actually runs faster in certain use cases.


 T


That might be true.
If you are hitting some constant factor, however the big-o for 
static foreach is worse then for tuple foreach.
Jan 06 2018
parent Seb <seb wilzba.ch> writes:
On Sunday, 7 January 2018 at 02:17:02 UTC, Stefan Koch wrote:

 On Sunday, 7 January 2018 at 01:08:44 UTC, H. S. Teoh wrote:

 On Sun, Jan 07, 2018 at 12:55:27AM +0000, Stefan Koch via 
 Digitalmars-d-learn wrote:

 On Saturday, 6 January 2018 at 23:25:58 UTC, Ali Çehreli 
 wrote:

 Is 'static foreach' sufficient for all needs or is there 
 any value for regular foreach over compile-time sequences?




 [...]

 No it's not.
 When you can use the old style do so. Since it puts less 
 stress on the
 compiler in the general case.


 Really? Based on a recent post by Jonathan Davis, the new 
 static foreach actually runs faster in certain use cases.


 T


 That might be true.
 If you are hitting some constant factor, however the big-o for 
 static foreach is worse then for tuple foreach.



FWIW there isn't any measurable performance penalty / difference.
We moved to use static foreach on Phobos recently: 
https://github.com/dlang/phobos/pull/5989
Jan 12 2018
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/06/2018 04:55 PM, Stefan Koch wrote:


 When you can use the old style do so. Since it puts less stress on the
 compiler in the general case.


My question is related to how to update my book. If the difference is 
mainly about performance, I think I will cover 'static foreach' but also 
mention old style foreach as a note.

Ali
Jan 06 2018
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 7 January 2018 at 01:52:10 UTC, Ali Çehreli wrote:

 On 01/06/2018 04:55 PM, Stefan Koch wrote:


 When you can use the old style do so. Since it puts less


 stress on the

 compiler in the general case.


 My question is related to how to update my book. If the 
 difference is mainly about performance, I think I will cover 
 'static foreach' but also mention old style foreach as a note.

 Ali


How about doing sth. similar like for DIP1003?

https://dlang.org/spec/contracts.html#pre_post_contracts


 Since DIP1003 has been applied the actual function body starts 
 with do. In the past, body was used, and could still be 
 encountered in old code bases.
Jan 06 2018
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/06/2018 06:20 PM, Seb wrote:


 How about doing sth. similar like for DIP1003?
 
 https://dlang.org/spec/contracts.html#pre_post_contracts


Already done! :)

 
https://bitbucket.org/acehreli/ddili/commits/2f10c048c2940a49263319d0c23b0ad661449f3e

Ali
Jan 06 2018
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 01/06/2018 10:53 PM, Ali Çehreli wrote:

 On 01/06/2018 06:20 PM, Seb wrote:
 

 How about doing sth. similar like for DIP1003?

 https://dlang.org/spec/contracts.html#pre_post_contracts


 
 Already done! :)
 
 
 https://bitbucket.org/acehreli/ddili/commits/2f10c048c2940a492633
9d0c23b0ad661449f3e 


What I meant is, yes, I will do it similar to the body-do change as you 
suggested.

Ali
Jan 06 2018
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/6/18 6:25 PM, Ali Çehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any value for 
 regular foreach over compile-time sequences?


If you use continues or breaks, then you need to switch to gotos if 
using static foreach, as it does not support them directly.

-Steve
Jan 07 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Jan 07, 2018 at 10:39:19PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:

 On 1/6/18 6:25 PM, Ali ehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any value
 for regular foreach over compile-time sequences?


 
 If you use continues or breaks, then you need to switch to gotos if
 using static foreach, as it does not support them directly.


[...]

Are you sure?  I was under te impression that it does support continues
and breaks -- but only if they are labelled, because of a syntactic
ambiguity otherwise.


T

-- 
"Hi." "'Lo."
Jan 08 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/8/18 9:27 AM, H. S. Teoh wrote:

 On Sun, Jan 07, 2018 at 10:39:19PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:

 On 1/6/18 6:25 PM, Ali Çehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any value
 for regular foreach over compile-time sequences?


 If you use continues or breaks, then you need to switch to gotos if
 using static foreach, as it does not support them directly.


 [...]
 
 Are you sure?  I was under te impression that it does support continues
 and breaks -- but only if they are labelled, because of a syntactic
 ambiguity otherwise.


I thought it only worked for constructs outside the static foreach (like 
switch).

testing...

Nope, doesn't work. The ambiguity is that if you have a breakable or 
continuable construct outside a static foreach (e.g. switch), then you 
may believe that the break statement is affecting the foreach (in fact, 
that is how tuple-foreach works), but you are actually affecting the 
outer construct. The extra requirement is to help you realize the 
implication. It may be removed in the future.

I may have been misleading when I made my first comment. What I mean is 
that you *can't* break or continue a static foreach, even with labels. 
However, you *can* do it to a standard foreach over a tuple. This may be 
one reason you want to use a tuple-foreach over a static foreach.

-Steve
Jan 09 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 09, 2018 at 10:57:03AM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:

 On 1/8/18 9:27 AM, H. S. Teoh wrote:

 On Sun, Jan 07, 2018 at 10:39:19PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:

 On 1/6/18 6:25 PM, Ali Çehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any
 value for regular foreach over compile-time sequences?


 
 If you use continues or breaks, then you need to switch to gotos
 if using static foreach, as it does not support them directly.


 [...]
 
 Are you sure?  I was under te impression that it does support
 continues and breaks -- but only if they are labelled, because of a
 syntactic ambiguity otherwise.


 
 I thought it only worked for constructs outside the static foreach
 (like switch).
 
 testing...
 
 Nope, doesn't work.


Grrr... I thought it did, but you're right, attempting to break the
static foreach with a label gets this compile error:

-----
test.d(7): Error: enclosing label FE for break not found
-----



 The ambiguity is that if you have a breakable or continuable construct
 outside a static foreach (e.g. switch), then you may believe that the
 break statement is affecting the foreach (in fact, that is how
 tuple-foreach works), but you are actually affecting the outer
 construct.


Yes, that's the ambiguity I was referring to. :-)



 The extra requirement is to help you realize the implication. It may
 be removed in the future.


I vaguely remember Timon mentioning something about implementing static
break / static continue, and somehow I thought the labelled break /
labelled continue was supposed to be it. Or at least, they are stand-ins
until static break/continue are implemented.  Is that no longer on the
table?



 I may have been misleading when I made my first comment. What I mean
 is that you *can't* break or continue a static foreach, even with
 labels. However, you *can* do it to a standard foreach over a tuple.
 This may be one reason you want to use a tuple-foreach over a static
 foreach.


[...]

Actually, that's wrong too. Tuple-foreach does not interpret
break/continue either. Here's a proof:

	alias Seq(A...) = A;
	foreach (i; Seq!(0, 1, 2, 3)) {
		static if (i==2)
			break;
		static assert(i < 3); // will fail on the 4th iteration
	}

What actually happens is that all iterations are unrolled, then the
unreachable iterations are elided by the optimizer during codegen. The
foreach itself is not affected by break/continue at all.


T

-- 
Дерево держится корнями, а человек -
друзьями.
Jan 09 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/9/18 11:35 AM, H. S. Teoh wrote:

 On Tue, Jan 09, 2018 at 10:57:03AM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:

 I may have been misleading when I made my first comment. What I mean
 is that you *can't* break or continue a static foreach, even with
 labels. However, you *can* do it to a standard foreach over a tuple.
 This may be one reason you want to use a tuple-foreach over a static
 foreach.


 [...]
 
 Actually, that's wrong too. Tuple-foreach does not interpret
 break/continue either. Here's a proof:
 
 	alias Seq(A...) = A;
 	foreach (i; Seq!(0, 1, 2, 3)) {
 		static if (i==2)
 			break;
 		static assert(i < 3); // will fail on the 4th iteration
 	}
 
 What actually happens is that all iterations are unrolled, then the
 unreachable iterations are elided by the optimizer during codegen. The
 foreach itself is not affected by break/continue at all.


A break or continue is simply a goto underneath. A goto in an unrolled 
loop isn't much different than a goto in a um... rolled loop :) It's 
just that there are copies of each loop body, and the gotos need copies 
of the labels.

So no, it's not "interpreted" by the foreach statement, but the foreach 
statement provides the anchors for the goto label targets.

e.g.:

int x;

foreach(i; Seq!(0, 1, 2, 3)) {
    x += i;
    static if(i % 2) continue;
    x *= i;
}

=>

int x;
{
    x += 0;
    x *= 0;
}
{
    x += 1;
    goto label1;
    x *= 1;
}
{
label1:
    x += 2;
    x *= 2;
}
{
    x += 3;
    goto label2;
    x *= 3;
}
label2:

And then of course, the optimizer weeds out the unreachable statements. 
Doing this with static foreach wouldn't be as pleasant. You'd have to 
branch the entire loop body, or use a goto in the case of a break.

-Steve
Jan 09 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 09, 2018 at 02:24:11PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
[...]

 A break or continue is simply a goto underneath. A goto in an unrolled
 loop isn't much different than a goto in a um... rolled loop :) It's
 just that there are copies of each loop body, and the gotos need
 copies of the labels.
 
 So no, it's not "interpreted" by the foreach statement, but the
 foreach statement provides the anchors for the goto label targets.


True.


[...]

 And then of course, the optimizer weeds out the unreachable
 statements.  Doing this with static foreach wouldn't be as pleasant.
 You'd have to branch the entire loop body, or use a goto in the case
 of a break.


[...]

If there were a hypothetical `static continue` or `static break` that's
recognized by the static foreach unroller, we could in theory automate
this branching in the compiler itself, e.g., by deleting the AST nodes
that would be skipped.  Of course, the syntax need not be `static
continue`; if there were a way to overload `break LABEL;` for the same
purpose, i.e., have the static foreach unroller inspect the label to see
if it is referring to the static foreach itself, then this would work.

But I don't know the static foreach implementation enough to be able to
tell whether this is actually possible at the time static foreach is
processed, or whether there may be some chicken-and-egg problem with
inspecting the target of a break/continue before semantic or whatever.


T

-- 
Государство делает вид, что платит нам
зарплату, а мы делаем вид, что работаем.
Jan 09 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/9/18 2:31 PM, H. S. Teoh wrote:

 On Tue, Jan 09, 2018 at 02:24:11PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 [...]

 A break or continue is simply a goto underneath. A goto in an unrolled
 loop isn't much different than a goto in a um... rolled loop :) It's
 just that there are copies of each loop body, and the gotos need
 copies of the labels.

 So no, it's not "interpreted" by the foreach statement, but the
 foreach statement provides the anchors for the goto label targets.


 
 True.
 
 
 [...]

 And then of course, the optimizer weeds out the unreachable
 statements.  Doing this with static foreach wouldn't be as pleasant.
 You'd have to branch the entire loop body, or use a goto in the case
 of a break.


 [...]
 
 If there were a hypothetical `static continue` or `static break` that's
 recognized by the static foreach unroller, we could in theory automate
 this branching in the compiler itself, e.g., by deleting the AST nodes
 that would be skipped.  Of course, the syntax need not be `static
 continue`; if there were a way to overload `break LABEL;` for the same
 purpose, i.e., have the static foreach unroller inspect the label to see
 if it is referring to the static foreach itself, then this would work.
 
 But I don't know the static foreach implementation enough to be able to
 tell whether this is actually possible at the time static foreach is
 processed, or whether there may be some chicken-and-egg problem with
 inspecting the target of a break/continue before semantic or whatever.


Yeah, I think in terms of static foreach, it's not a straightforward 
problem. Because the compiler may not know enough information at the 
time to figure out whether it should still keep generating code.

For example:

static foreach(i; 0 .. 5)
{
    if(i == 3) static break;
    static assert(i < 3);
}

How does it know whether the static break should be "executed" at 
compile-time if it hasn't evaluated the if-statement? The code would 
have to have no runtime branches to make sure that static break can be 
evaluated at compile-time.

And this still puts it at a disadvantage when compared to tuple-foreach, 
at least as far as break/continue are concerned.

-Steve
Jan 09 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 09, 2018 at 03:26:32PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:

 On 1/9/18 2:31 PM, H. S. Teoh wrote:


[...]

 If there were a hypothetical `static continue` or `static break`
 that's recognized by the static foreach unroller, we could in theory
 automate this branching in the compiler itself, e.g., by deleting
 the AST nodes that would be skipped.  Of course, the syntax need not
 be `static continue`; if there were a way to overload `break LABEL;`
 for the same purpose, i.e., have the static foreach unroller inspect
 the label to see if it is referring to the static foreach itself,
 then this would work.
 
 But I don't know the static foreach implementation enough to be able
 to tell whether this is actually possible at the time static foreach
 is processed, or whether there may be some chicken-and-egg problem
 with inspecting the target of a break/continue before semantic or
 whatever.


 
 Yeah, I think in terms of static foreach, it's not a straightforward
 problem. Because the compiler may not know enough information at the
 time to figure out whether it should still keep generating code.
 
 For example:
 
 static foreach(i; 0 .. 5)
 {
    if(i == 3) static break;
    static assert(i < 3);
 }
 
 How does it know whether the static break should be "executed" at
 compile-time if it hasn't evaluated the if-statement? The code would
 have to have no runtime branches to make sure that static break can be
 evaluated at compile-time.


[...]

Static foreach does not (and should not!) evaluate a runtime branch,
because this is before CTFE even happens. CTFE cannot happen until the
static foreach has been fully unrolled, so it doesn't even make sense to
talk about evaluating the if-statement at this point. For your example
to make sense, you'd have to use static if, then the break would be
possible, and non-problematic.

Of course, that still doesn't solve the problem of what static break is
supposed to do from inside a runtime branch. I'm tempted to say that
static break should mean "delete all subsequent nodes from the AST that
follows this node in depth-first traversal order", so your example above
would be transformed into:

	if (0 == 3) {}
	// all subsequent iterations deleted

because the static break is unconditionally compiled (it has nothing to
do with the runtime branch).  You'd have to use static if to make it
conditionally-compiled and thus not instantly aborting the loop.

Such semantics would be logically consistent, but unfortunately rather
counterintuitive at first glance.


T

-- 
If the comments and the code disagree, it's likely that *both* are wrong. --
Christopher
Jan 09 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09.01.2018 22:04, H. S. Teoh wrote:

 	if (0 == 3) {}
 	// all subsequent iterations deleted
 
 because the static break is unconditionally compiled (it has nothing to
 do with the runtime branch).  You'd have to use static if to make it
 conditionally-compiled and thus not instantly aborting the loop.
 
 Such semantics would be logically consistent, but unfortunately rather
 counterintuitive at first glance.


I think "if (0 == 3) { static break; }" should be a compile-time error.
Jan 09 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Jan 10, 2018 at 12:18:46AM +0100, Timon Gehr via Digitalmars-d-learn
wrote:

 On 09.01.2018 22:04, H. S. Teoh wrote:

 	if (0 == 3) {}
 	// all subsequent iterations deleted
 
 because the static break is unconditionally compiled (it has nothing
 to do with the runtime branch).  You'd have to use static if to make
 it conditionally-compiled and thus not instantly aborting the loop.
 
 Such semantics would be logically consistent, but unfortunately
 rather counterintuitive at first glance.


 
 I think "if (0 == 3) { static break; }" should be a compile-time error.


That's also a possible solution, perhaps a better solution than what I
described.  Make it so that static break/continue cannot be nested
inside runtime conditionals.  That should exclude all of the
pathological cases, hopefully.


T

-- 
The diminished 7th chord is the most flexible and fear-instilling chord. Use it
often, use it unsparingly, to subdue your listeners into submission!
Jan 09 2018
parent Seb <seb wilzba.ch> writes:
On Tuesday, 9 January 2018 at 23:27:42 UTC, H. S. Teoh wrote:

 On Wed, Jan 10, 2018 at 12:18:46AM +0100, Timon Gehr via 
 Digitalmars-d-learn wrote:

 On 09.01.2018 22:04, H. S. Teoh wrote:

 [...]


 
 I think "if (0 == 3) { static break; }" should be a 
 compile-time error.


 That's also a possible solution, perhaps a better solution than 
 what I described.  Make it so that static break/continue cannot 
 be nested inside runtime conditionals.  That should exclude all 
 of the pathological cases, hopefully.


 T


FWIW I recently bumped into a problem where `static break` would 
be _really_ useful:

https://github.com/dlang/dmd/pull/7577#discussion_r159175229
Jan 09 2018
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, January 08, 2018 06:27:12 H. S. Teoh via Digitalmars-d-learn 
wrote:

 On Sun, Jan 07, 2018 at 10:39:19PM -0500, Steven Schveighoffer via 


Digitalmars-d-learn wrote:

 On 1/6/18 6:25 PM, Ali ehreli wrote:

 Is 'static foreach' sufficient for all needs or is there any value
 for regular foreach over compile-time sequences?


 If you use continues or breaks, then you need to switch to gotos if
 using static foreach, as it does not support them directly.


 [...]

 Are you sure?  I was under te impression that it does support continues
 and breaks -- but only if they are labelled, because of a syntactic
 ambiguity otherwise.


It does support them if they're labeled. I did it just the other day.
Originally, that code was not using static foreach (though in both cases, it
was over an AliasSeq of template arguments), and I'd used a normal break
(which compiles just fine with a non-static foreach) and been very confused
about why my tests were failing. since for whetever reason, it didn't occur
to me when I was writing it that break would break out of the foreach
(probably since I was thinking of it as a compile-time thing and not as a
loop that would be run, but I don't know; it seems kind of dumb in
retrospect either way). Since my tests failed miserably, I realized
something was wrong and fixed it (though man was it confusing to begin
with), but I was later able to switch that foreach to static foreach without
any breakage. However, if I remove the label now, then it won't compile,
since it's a static foreach. So, if I'd used a static foreach from the
get-go, it would have actually caught a bug for me before I even ran my
tests.

But regardless, labeled break definitely works within a static foreach, and
I expect that a labeled continue does as well, but I haven't tried it.

- Jonathan M Davis
Jan 08 2018
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 1/8/18 3:07 PM, Jonathan M Davis wrote:

 But regardless, labeled break definitely works within a static foreach, and
 I expect that a labeled continue does as well, but I haven't tried it.


I didn't mean it that way, see my reply to H.

-Steve
Jan 09 2018