• Quirin Schroll (8/11) Mar 09 2022 Why? This will rarely be wanted or be used intentionally. When a
• Timon Gehr (3/14) Mar 09 2022 It's a consequence of how such a foreach loop is lowered. Looks like one...
• bauss (2/18) Mar 09 2022 Surely we could work around it?
• Quirin Schroll (50/70) Mar 10 2022 The issue is not the complexity of the workaround (example
• Adam D Ruppe (6/20) Mar 10 2022 __traits(parameters) always gives the parameters of the function
• Timon Gehr (6/15) Mar 10 2022 It is not used inside a nested function. It is used within a foreach
• Adam Ruppe (5/6) Mar 10 2022 I don't really feel strongly about it. As it is, this will
• Timon Gehr (5/12) Mar 10 2022 Fair enough. I think what should have been done is:
• Paul Backus (7/21) Mar 10 2022 From [the PR discussion][1], the rationale given for the current
• Adam D Ruppe (2/3) Mar 10 2022 objection, prejudiced language
• Timon Gehr (8/13) Mar 10 2022 Pardon? It's pretty clear that it should not behave this way. The point
• Steven Schveighoffer (11/12) Mar 10 2022 Yep. Which is too bad.
• H. S. Teoh (8/10) Mar 10 2022 [...]
• Timon Gehr (6/14) Mar 10 2022 Not with me. I think it's pretty useful. (It's a limited form of call/cc...
• Timon Gehr (10/16) Mar 10 2022 Maybe not, but it's death by a thousand little cuts with those things. I...
• Steven Schveighoffer (18/29) Mar 10 2022 Yes, I agree.
• Elronnd (10/11) Mar 10 2022 I was a bit on the fence regarding the issue, but this is what
• MoonlightSentinel (2/3) Mar 10 2022 Proposed fix: https://github.com/dlang/dmd/pull/13798
• max haughton (3/7) Mar 10 2022 LGTM
• Timon Gehr (5/9) Mar 10 2022 Thanks! That's probably going to be a lot more helpful than prolonged

Quirin Schroll <quirin.schroll mgm-tp.com> writes:

In the changelog it says:
 When used inside a foreach using an overloaded `opApply`, the 
 trait yields the parameters to the delegate and not the 
 function the foreach appears within.

Why? This will rarely be wanted or be used intentionally. When a 
programmer uses `__traits(parameters)` in a `foreach` loop, it 
will for certain happen to someone not aware of this. The 
iteration implemention (`opApply` or range functions) is a detail 
one should not really have to care about on the usage side. This 
complicates the language. Morally, this is a bug. Please 
reconsider this design decision before it sticks.

Timon Gehr <timon.gehr gmx.ch> writes:

On 3/9/22 11:04, Quirin Schroll wrote:
 In the changelog it says:
 When used inside a foreach using an overloaded `opApply`, the trait 
 yields the parameters to the delegate and not the function the foreach 
 appears within.

 
 Why? This will rarely be wanted or be used intentionally. When a 
 programmer uses `__traits(parameters)` in a `foreach` loop, it will for 
 certain happen to someone not aware of this. The iteration implemention 
 (`opApply` or range functions) is a detail one should not really have to 
 care about on the usage side. This complicates the language. Morally, 
 this is a bug. Please reconsider this design decision before it sticks.

It's a consequence of how such a foreach loop is lowered. Looks like one 
of those cases where a bug was resolved by changing the specification.

bauss <jj_1337 live.dk> writes:

On Wednesday, 9 March 2022 at 19:02:27 UTC, Timon Gehr wrote:
 On 3/9/22 11:04, Quirin Schroll wrote:
 In the changelog it says:
 When used inside a foreach using an overloaded `opApply`, the 
 trait yields the parameters to the delegate and not the 
 function the foreach appears within.

 
 Why? This will rarely be wanted or be used intentionally. When 
 a programmer uses `__traits(parameters)` in a `foreach` loop, 
 it will for certain happen to someone not aware of this. The 
 iteration implemention (`opApply` or range functions) is a 
 detail one should not really have to care about on the usage 
 side. This complicates the language. Morally, this is a bug. 
 Please reconsider this design decision before it sticks.

 It's a consequence of how such a foreach loop is lowered. Looks 
 like one of those cases where a bug was resolved by changing 
 the specification.

Surely we could work around it?

Quirin Schroll <quirin.schroll mgm-tp.com> writes:

On Thursday, 10 March 2022 at 07:21:23 UTC, bauss wrote:
 On Wednesday, 9 March 2022 at 19:02:27 UTC, Timon Gehr wrote:
 On 3/9/22 11:04, Quirin Schroll wrote:
 In the changelog it says:
 When used inside a foreach using an overloaded `opApply`, 
 the trait yields the parameters to the delegate and not the 
 function the foreach appears within.

 
 Why? This will rarely be wanted or be used intentionally. 
 When a programmer uses `__traits(parameters)` in a `foreach` 
 loop, it will for certain happen to someone not aware of 
 this. The iteration implemention (`opApply` or range 
 functions) is a detail one should not really have to care 
 about on the usage side. This complicates the language. 
 Morally, this is a bug. Please reconsider this design 
 decision before it sticks.

 It's a consequence of how such a foreach loop is lowered. 
 Looks like one of those cases where a bug was resolved by 
 changing the specification.

 Surely we could work around it?

The issue is not the complexity of the workaround (example 
below), but that it is in practice hard to know when it applies 
and it is *hard to teach* (cf. Scott Meyers: Language is good == 
Features are easy to explain). You have to remember to be careful 
in `foreach` loops when you use `__traits(parameters)` . You have 
to remember not because the case is truly an odd-one-out, but 
because … Well, I don’t know how to finish that sentence without 
being rude. A justified special case is: “If we used the simple 
rule, reasonable expectations would break. Therefore we have a 
complicated rule.” — Here, we have it backwards: “If we used the 
simple rule, we’d have a complicated compiler implementation. 
Therefore we break expectations.” The authors did not even have 
the mercy to make it an error so that it is guaranteed that 
programmers work around it. (The error message could claim it is 
amibiguous what you mean. It morally is not, but technically it 
is.)

The compiler does a bunch of stuff to properly lower `return`, 
`break`, `continue`, and `goto` statments in the `foreach` body 
when given to `opApply`. Why the authors of this feature decided 
not require it in this case is beyond me.


Say you wanted to access the function's parameters in a `foreach` 
loop. Say you are in a meta-programming context where you don't 
know the type of the range.
```D
void f(R, Ts...)(R range, int param, Ts args)
{
     foreach (auto ref x; range)
     {
         // some amount of code
         g!(Ts[1..$])(x, __traits(parameters)[1..$]);
     }
}
```
This is the idiomatic way to write the call to `g`, but it might 
not work correctly depending on the details of the iteration of 
`range`. Worse, it might compile and silently do unexpected 
stuff. The always-correct way:
```D
void f(R, Ts...)(R range, int param, Ts args)
{
     alias relevantParams = __traits(parameters)[1..$]; // Why?
     foreach (auto ref x; range)
     {
         // some amount of code
         g!(Ts[1..$])(x, relevantParams);
     }
}
```

Adam D Ruppe <destructionator gmail.com> writes:

On Thursday, 10 March 2022 at 10:18:09 UTC, Quirin Schroll wrote:
 The issue is not the complexity of the workaround (example 
 below), but that it is in practice hard to know when it applies 
 and it is *hard to teach*


__traits(parameters) always gives the parameters of the function 
in which it is used *even if it is used inside a nested function*.


 Say you wanted to access the function's parameters in a 
 `foreach` loop. Say you are in a meta-programming context where 
 you don't know the type of the range.
 ```D
 void f(R, Ts...)(R range, int param, Ts args)
 {
     foreach (auto ref x; range)
     {
         // some amount of code
         g!(Ts[1..$])(x, __traits(parameters)[1..$]);

Why wouldn't you just use `args` here? The __traits(parameters) 
actually doesn't give any real value when you already have a 
variadic.

Timon Gehr <timon.gehr gmx.ch> writes:

On 10.03.22 13:27, Adam D Ruppe wrote:
 On Thursday, 10 March 2022 at 10:18:09 UTC, Quirin Schroll wrote:
 The issue is not the complexity of the workaround (example below), but 
 that it is in practice hard to know when it applies and it is *hard to 
 teach*

 
 
 __traits(parameters) always gives the parameters of the function in 
 which it is used *even if it is used inside a nested function*.
 

It is not used inside a nested function. It is used within a foreach 
loop. Different rules apply. Or are you arguing that (like Steve pointed 
out already), return statements in opApply foreach loop bodies should 
return from the delegate because "return always returns from the 
function in which it is used *even if it is used inside a nested function*?

Adam Ruppe <destructionator gmail.com> writes:

On Thursday, 10 March 2022 at 22:13:30 UTC, Timon Gehr wrote:
 *snip*

I don't really feel strongly about it. As it is, this will 
deliver value to me. What it does inside a foreach loop is 
immaterial to my use cases (and I suspect to just about any that 
can't be easier done with preexisting features).

Timon Gehr <timon.gehr gmx.ch> writes:

On 10.03.22 23:47, Adam Ruppe wrote:
 On Thursday, 10 March 2022 at 22:13:30 UTC, Timon Gehr wrote:
 *snip*

 
 I don't really feel strongly about it. As it is, this will deliver value 
 to me. What it does inside a foreach loop is immaterial to my use cases 
 (and I suspect to just about any that can't be easier done with 
 preexisting features).

Fair enough. I think what should have been done is:

- Pull the buggy feature, comment out the test.
- Put an issue in bugzilla to be picked up by someone who actually needs 
this to work in foreach loops.

Paul Backus <snarwin gmail.com> writes:

On Wednesday, 9 March 2022 at 10:04:57 UTC, Quirin Schroll wrote:
 In the changelog it says:
 When used inside a foreach using an overloaded `opApply`, the 
 trait yields the parameters to the delegate and not the 
 function the foreach appears within.

 Why? This will rarely be wanted or be used intentionally. When 
 a programmer uses `__traits(parameters)` in a `foreach` loop, 
 it will for certain happen to someone not aware of this. The 
 iteration implemention (`opApply` or range functions) is a 
 detail one should not really have to care about on the usage 
 side. This complicates the language. Morally, this is a bug. 
 Please reconsider this design decision before it sticks.

 From [the PR discussion][1], the rationale given for the current 
behavior was:

 I'm not sure whether this was desirable or not, but it's not 
 really along the lines of how this feature is intended to be 
 used so I don't see any utility any different behaviour.

So there is no real justification for it; the interaction with 
`foreach` was just not considered important enough to be worth 
the effort of handling correctly.

[1]: https://github.com/dlang/dmd/pull/13071

Adam D Ruppe <destructionator gmail.com> writes:

On Thursday, 10 March 2022 at 12:49:57 UTC, Paul Backus wrote:
 handling correctly.

objection, prejudiced language

Timon Gehr <timon.gehr gmx.ch> writes:

On 10.03.22 13:57, Adam D Ruppe wrote:
 On Thursday, 10 March 2022 at 12:49:57 UTC, Paul Backus wrote:
 handling correctly.

 
 objection, prejudiced language
 

Pardon? It's pretty clear that it should not behave this way. The point 
of opApply is to overload on foreach syntax, where you can transparently 
exchange the type of aggregate. This is a headache for generic code that 
can loop either over an array or a struct with an opApply. There is zero 
justification for the meaning of the foreach loop body to change based 
on how the aggregate is expressed.

I guess __traits(parent, {}) and similar are also affected.

Steven Schveighoffer <schveiguy gmail.com> writes:

On 3/10/22 5:21 PM, Timon Gehr wrote:
 I guess __traits(parent, {}) and similar are also affected.

Yep. Which is too bad.

I tried some other things out. `__FUNCTION__` prints the delegate name, 
shadowing variable names is allowed but deprecated.

It would be nice if we could eliminate any real indications that this is 
a lambda (which really is an implementation detail).

On the other hand, opApply is pretty out of favor now, so maybe it's not 
going to be as bad of an effect. This really is one of those difficult 
to explain pieces to new users, and really difficult to get correct 
within generic code.

-Steve

"H. S. Teoh" <hsteoh quickfur.ath.cx> writes:

On Thu, Mar 10, 2022 at 05:37:03PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
[...]
 On the other hand, opApply is pretty out of favor now, so maybe it's
 not going to be as bad of an effect.

[...]

Parallel foreach still depends on opApply, though.  Is it really "out of
favor"?


T

-- 
"Outlook not so good." That magic 8-ball knows everything! I'll ask about
Exchange Server next. -- (Stolen from the net)

Timon Gehr <timon.gehr gmx.ch> writes:

On 10.03.22 23:51, H. S. Teoh wrote:
 On Thu, Mar 10, 2022 at 05:37:03PM -0500, Steven Schveighoffer via
Digitalmars-d wrote:
 [...]
 On the other hand, opApply is pretty out of favor now, so maybe it's
 not going to be as bad of an effect.

 [...]
 
 Parallel foreach still depends on opApply, though.  Is it really "out of
 favor"?

Not with me. I think it's pretty useful. (It's a limited form of call/cc.)

Interestingly, parallel foreach barely uses the nicest parts of opApply:
"std.parallelism.ParallelForeachError (0): Cannot break from a parallel 
foreach loop using break, return, labeled break/continue or goto 
statements."

Timon Gehr <timon.gehr gmx.ch> writes:

On 10.03.22 23:37, Steven Schveighoffer wrote:
 
 On the other hand, opApply is pretty out of favor now, so maybe it's not 
 going to be as bad of an effect.

Maybe not, but it's death by a thousand little cuts with those things. I 
really dislike the stance of "let's document the wart, then it won't 
technically be a bug". This has been pretty rampant. It's so much better 
to have a known bug than to enshrine a known bad design in the spec...

 This really is one of those difficult 
 to explain pieces to new users, and really difficult to get correct 
 within generic code.

Yah. :( I can work around those issues, but they are rather embarrassing 
to explain to new developers on a project who are not even PL geeks and 
already a bit confused as to why D has been used in the first place. 
("Why are you using D again if it can't get basic things like this 
right?") I do use opApply pretty extensively to implement tree traversals.

Steven Schveighoffer <schveiguy gmail.com> writes:

On 3/9/22 5:04 AM, Quirin Schroll wrote:
 In the changelog it says:
 When used inside a foreach using an overloaded `opApply`, the trait 
 yields the parameters to the delegate and not the function the foreach 
 appears within.

 
 Why? This will rarely be wanted or be used intentionally. When a 
 programmer uses `__traits(parameters)` in a `foreach` loop, it will for 
 certain happen to someone not aware of this. The iteration implemention 
 (`opApply` or range functions) is a detail one should not really have to 
 care about on the usage side. This complicates the language. Morally, 
 this is a bug. Please reconsider this design decision before it sticks.

Yes, I agree.

```d
string s = "WTF";

alias p = typeof(__traits(parameters));
foreach(x; s) {
    static assert(__traits(isSame, p, typeof(__traits(parameters))));
}

foreach(dchar x; s) {
    static assert(!__traits(isSame, p, typeof(__traits(parameters))));
}
```

Lowering should be invisible to the plain language. This is similar to 
issues where lowered code has errors and you get error messages on code 
that you never wrote.

Imagine if inside an opApply delegate, a return statement just returned 
the delegate. This is equivalent to what you are saying here.

-Steve

Elronnd <elronnd elronnd.net> writes:

On Thursday, 10 March 2022 at 15:08:11 UTC, Steven Schveighoffer 
wrote:
 *snip*

I was a bit on the fence regarding the issue, but this is what 
convinced me.

The issue is primarily one of abstraction and modularity.  It 
should be an implementation detail whether a given object uses 
opApply or range APIs.  It should be possible to switch from one 
to the other without breaking client code.  I should not have to 
care how the object I'm iterating over arranges to be iterated 
over.

MoonlightSentinel <moonlightsentinel disroot.org> writes:

On Wednesday, 9 March 2022 at 10:04:57 UTC, Quirin Schroll wrote:
 Please reconsider this design decision before it sticks.

Proposed fix: https://github.com/dlang/dmd/pull/13798

max haughton <maxhaton gmail.com> writes:

On Thursday, 10 March 2022 at 23:58:36 UTC, MoonlightSentinel 
wrote:
 On Wednesday, 9 March 2022 at 10:04:57 UTC, Quirin Schroll 
 wrote:
 Please reconsider this design decision before it sticks.

 Proposed fix: https://github.com/dlang/dmd/pull/13798

LGTM

Timon Gehr <timon.gehr gmx.ch> writes:

On 11.03.22 00:58, MoonlightSentinel wrote:
 On Wednesday, 9 March 2022 at 10:04:57 UTC, Quirin Schroll wrote:
 Please reconsider this design decision before it sticks.

 
 Proposed fix: https://github.com/dlang/dmd/pull/13798

Thanks! That's probably going to be a lot more helpful than prolonged 
bickering on the forums. ;)

Note that there are some other __traits that have the same problem, 
notably __traits(parent, ...).