www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Discussion Thread: DIP 1033--Implicit Conversion of Expressions to

reply Mike Parker <aldacron gmail.com> writes:
This is the discussion thread for the first round of Community 
Review of DIP 1033, "Implicit Conversion of Expressions to 
Delegates":

https://github.com/dlang/DIPs/blob/7b61411cb6cf8db05d9b8e1df5d2d9bae53a5f1e/DIPs/DIP1033.md

The review period will end at 11:59 PM ET on May 6, or when I 
make a post declaring it complete. Discussion in this thread may 
continue beyond that point.

Here in the discussion thread, you are free to discuss anything 
and everything related to the DIP. Express your support or 
opposition, debate alternatives, argue the merits, etc.

However, if you have any specific feedback on how to improve the 
proposal itself, then please post it in the feedback thread. The 
feedback thread will be the source for the review summary I write 
at the end of this review round. I will post a link to that 
thread immediately following this post. Just be sure to read and 
understand the Reviewer Guidelines before posting there:

https://github.com/dlang/DIPs/blob/master/docs/guidelines-reviewers.md

And my blog post on the difference between the Discussion and 
Feedback threads:

https://dlang.org/blog/2020/01/26/dip-reviews-discussion-vs-feedback/

Please stay on topic here. I will delete posts that are 
completely off-topic.
Apr 22 2020
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Wednesday, 22 April 2020 at 07:41:12 UTC, Mike Parker wrote:


 Here in the discussion thread, you are free to discuss anything 
 and everything related to the DIP. Express your support or 
 opposition, debate alternatives, argue the merits, etc.

 However, if you have any specific feedback on how to improve 
 the proposal itself, then please post it in the feedback 
 thread. The feedback thread will be the source for the review 
 summary I write at the end of this review round. I will post a 
 link to that thread immediately following this post.


The Feedback Thread is here:

https://forum.dlang.org/post/nxahrsukobybkezibcsm forum.dlang.org
Apr 22 2020
prev sibling next sibling parent reply kinke <noone nowhere.com> writes:
I don't think this is a good idea. One of the problems with 
`lazy` IMO is that one needs good IDE support to somehow 
highlight that the argument expression isn't evaluated before the 
call, and might not be evaluated at all:

// in some other module:
void foo(lazy int i) {}

void main()
{
     int i = 1;
     foo(++i);
     assert(i == 1); // what? not incremented above? ah yeah, lazy 
and not evaluated at all by foo
}

The proposal would extend that problem to regular delegate params 
too. So I find the explicit version (`() => ++i`) better to make 
that clear. I don't think the 'overhead' of putting an additional 
`() =>` in front of the expression justifies introducing the 
proposed implicit convertibility.
Apr 22 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/22/20 7:56 AM, kinke wrote:

 I don't think this is a good idea. One of the problems with `lazy` IMO 
 is that one needs good IDE support to somehow highlight that the 
 argument expression isn't evaluated before the call, and might not be 
 evaluated at all:
 
 // in some other module:
 void foo(lazy int i) {}
 
 void main()
 {
      int i = 1;
      foo(++i);
      assert(i == 1); // what? not incremented above? ah yeah, lazy and 
 not evaluated at all by foo
 }


This is more of a problem of lazy parameters that mutate something. Most 
of the time, that isn't the use case of lazy -- it's to prevent calling 
some expensive thing when it's not needed. Most definitely this DIP 
would make that problem worse.

What could be something that helps is enforcing that expressions to 
delegate implicit conversions require no mutation. I'm not sure how that 
could be enforced, but possibly requiring only pure function calls on 
const data (including the context).

It wouldn't be a breaking change, because currently expressions don't 
bind to delegates, so we can enforce that rule there, then deprecate 
mutating expressions on lazy parameters, and finally remove lazy.


 
 The proposal would extend that problem to regular delegate params too. 
 So I find the explicit version (`() => ++i`) better to make that clear. 
 I don't think the 'overhead' of putting an additional `() =>` in front 
 of the expression justifies introducing the proposed implicit 
 convertibility.


I've never had much of an issue with the verbosity of delegates, 
especially since the () => form came about.

What I HAVE had an issue with is not being able to pass a delegate into 
a lazy parameter. e.g.:

void foo(lazy int x);

foo(() {some; large; calculation; return result; }); // no good.
foo(() {...}()); // works but, is a delegate that calls a delegate, ugh.

To that end, would it be possible to add lazy binding to a delegate in 
the same DIP? The more we make lazy behave like a delegate, the easier 
it will be to remove lazy.

-Steve
Apr 22 2020
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 22 April 2020 at 12:30:52 UTC, Steven Schveighoffer 
wrote:

 This is more of a problem of lazy parameters that mutate 
 something. Most of the time, that isn't the use case of lazy -- 
 it's to prevent calling some expensive thing when it's not 
 needed.


I disagree - `foo(myData.recomputeFromScratch())` has the same 
problem, it's not obvious from the call site alone that the 
expensive operation is deferred and might be skipped.


 To that end, would it be possible to add lazy binding to a 
 delegate in the same DIP? The more we make lazy behave like a 
 delegate, the easier it will be to remove lazy.


That might be a migration path towards removing lazy, so that the 
call sites can switch to delegates already, and the lazy params 
transformed to explicit delegates in a 2nd step. No idea if such 
a two-step approach would be worth it though.
Apr 22 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/22/20 10:04 AM, kinke wrote:

 On Wednesday, 22 April 2020 at 12:30:52 UTC, Steven Schveighoffer wrote:

 This is more of a problem of lazy parameters that mutate something. 
 Most of the time, that isn't the use case of lazy -- it's to prevent 
 calling some expensive thing when it's not needed.


 
 I disagree - `foo(myData.recomputeFromScratch())` has the same problem, 
 it's not obvious from the call site alone that the expensive operation 
 is deferred and might be skipped.


But if myData.recomputeFromScratch() has no side effects, what does it 
matter if not used or called?

For example, if you have:

auto foo(T)(bool cond, T trueValue, T falseValue)
{
    return cond ? trueValue : falseValue;
}

And you call like:

auto x = foo(cond, val, expensivePureCalc());

A sufficiently intelligent compiler is going to avoid calling 
expensivePureCalc when cond is false. Is that a problem?

Really the only use of lazy I use is in enforce. The lazy message might 
do GC allocations to generate a string that is thrown away if not used. 
But the string generation doesn't mutate anything, so it's not going to 
have an effect on the operation of the program if not called.

What I'm saying is if the DIP *enforced* that expression-to-delegate 
happens only for these cases, then there is no worry about whether the 
expression is evaluated or not.

-Steve
Apr 22 2020
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/22/20 11:15 AM, Steven Schveighoffer wrote:

 A sufficiently intelligent compiler is going to avoid calling 
 expensivePureCalc when cond is false. Is that a problem?


ugh, when cond is *true* obviously ;)

-Steve
Apr 22 2020
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 22 April 2020 at 15:15:24 UTC, Steven Schveighoffer 
wrote:

 Really the only use of lazy I use is in enforce.


I'd say that object.d has some pretty good use cases in addition 
to enforce:

inout(V) get(K, V)(inout(V[K]) aa, K key, lazy inout(V) 
defaultValue):
https://github.com/dlang/druntime/blob/v2.091.0/src/object.d#L2576

ref V require(K, V)(ref V[K] aa, K key, lazy V value = V.init):
https://github.com/dlang/druntime/blob/v2.091.0/src/object.d#L2605
Apr 22 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/22/20 12:50 PM, Petar Kirov [ZombineDev] wrote:

 On Wednesday, 22 April 2020 at 15:15:24 UTC, Steven Schveighoffer wrote:

 Really the only use of lazy I use is in enforce.


 
 I'd say that object.d has some pretty good use cases in addition to 
 enforce:
 
 inout(V) get(K, V)(inout(V[K]) aa, K key, lazy inout(V) defaultValue):
 https://github.com/dlang/druntime/blob/v2.091.0/src/object.d#L2576
 
 ref V require(K, V)(ref V[K] aa, K key, lazy V value = V.init):
 https://github.com/dlang/druntime/blob/v2.091.0/src/object.d#L2605
 
 
 


Right, and I still think doing mutations in those expressions would be 
confusing.

-Steve
Apr 22 2020
prev sibling parent reply aliak <something something.com> writes:
On Wednesday, 22 April 2020 at 11:56:44 UTC, kinke wrote:

 I don't think this is a good idea. One of the problems with 
 `lazy` IMO is that one needs good IDE support to somehow 
 highlight that the argument expression isn't evaluated before 
 the call, and might not be evaluated at all:

 // in some other module:
 void foo(lazy int i) {}

 void main()
 {
     int i = 1;
     foo(++i);
     assert(i == 1); // what? not incremented above? ah yeah, 
 lazy and not evaluated at all by foo
 }

 The proposal would extend that problem to regular delegate 
 params too. So I find the explicit version (`() => ++i`) better 
 to make that clear. I don't think the 'overhead' of putting an 
 additional `() =>` in front of the expression justifies 
 introducing the proposed implicit convertibility.


I agree with this. Implicit conversion has always been argued 
against in D - why is it ok now? And this is not the good kind of 
implicit conversion where the user explicitly asks for it, but 
the bad kind where the language decides it should happen and the 
user doesn't know it can even happen.

Thinking about it more now feels like the real rationale for the 
DIP is "in order to deprecate lazy, we need an alternative" ... 
???
Apr 23 2020
parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Thursday, 23 April 2020 at 09:33:35 UTC, aliak wrote:

 On Wednesday, 22 April 2020 at 11:56:44 UTC, kinke wrote:

 I don't think this is a good idea. One of the problems with 
 `lazy` IMO is that one needs good IDE support to somehow 
 highlight that the argument expression isn't evaluated before 
 the call, and might not be evaluated at all:

 // in some other module:
 void foo(lazy int i) {}

 void main()
 {
     int i = 1;
     foo(++i);
     assert(i == 1); // what? not incremented above? ah yeah, 
 lazy and not evaluated at all by foo
 }

 The proposal would extend that problem to regular delegate 
 params too. So I find the explicit version (`() => ++i`) 
 better to make that clear. I don't think the 'overhead' of 
 putting an additional `() =>` in front of the expression 
 justifies introducing the proposed implicit convertibility.


 I agree with this. Implicit conversion has always been argued 
 against in D - why is it ok now? And this is not the good kind 
 of implicit conversion where the user explicitly asks for it, 
 but the bad kind where the language decides it should happen 
 and the user doesn't know it can even happen.

 Thinking about it more now feels like the real rationale for 
 the DIP is "in order to deprecate lazy, we need an alternative" 
 ... ???


I always thought that lazy should also be marked at the call site 
as it is a very different semantic on the caller and on the 
callee. As it is now, it is true that the example above is 
unfortunate and a real source of unpleasant surprizes.


void foo(lazy int i) {}

  void main()
  {
      int i = 1;
      foo(lazy ++i);
      assert(i == 1); // No surprize here, the required annotation 
tells it.
  }
Apr 23 2020
next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
wrote:

 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above is 
 unfortunate and a real source of unpleasant surprizes.


+1

Step 1: lazy must be used at the call site
Step 2: deprecate lazy
Step 3: remove this terrible, unreadable idea that was done 
hastily years ago with no vote whatsoever
Apr 23 2020
parent reply Meta <jared771 gmail.com> writes:
On Thursday, 23 April 2020 at 13:20:08 UTC, Guillaume Piolat 
wrote:

 On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
 wrote:

 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above is 
 unfortunate and a real source of unpleasant surprizes.


 +1

 Step 1: lazy must be used at the call site


That would play havoc with generic code and would significantly 
increase the complexity of standard phobos functions like map, 
filter, etc.
Apr 23 2020
parent reply aliak <something something.com> writes:
On Thursday, 23 April 2020 at 14:49:53 UTC, Meta wrote:

 On Thursday, 23 April 2020 at 13:20:08 UTC, Guillaume Piolat 
 wrote:

 On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
 wrote:

 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above 
 is unfortunate and a real source of unpleasant surprizes.


 +1

 Step 1: lazy must be used at the call site


 That would play havoc with generic code and would significantly 
 increase the complexity of standard phobos functions like map, 
 filter, etc.


Can you elaborate? How so?
Apr 23 2020
next sibling parent aliak <something something.com> writes:
On Thursday, 23 April 2020 at 20:15:19 UTC, aliak wrote:

 On Thursday, 23 April 2020 at 14:49:53 UTC, Meta wrote:

 On Thursday, 23 April 2020 at 13:20:08 UTC, Guillaume Piolat 
 wrote:

 On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
 wrote:

 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above 
 is unfortunate and a real source of unpleasant surprizes.


 +1

 Step 1: lazy must be used at the call site


 That would play havoc with generic code and would 
 significantly increase the complexity of standard phobos 
 functions like map, filter, etc.


 Can you elaborate? How so?


Nevermind. Got it!

I wonder if phobos generic code should even account for that 🤔
Apr 23 2020
prev sibling parent Meta <jared771 gmail.com> writes:
On Thursday, 23 April 2020 at 20:15:19 UTC, aliak wrote:

 On Thursday, 23 April 2020 at 14:49:53 UTC, Meta wrote:

 On Thursday, 23 April 2020 at 13:20:08 UTC, Guillaume Piolat 
 wrote:

 On Thursday, 23 April 2020 at 11:13:34 UTC, Patrick Schluter 
 wrote:

 I always thought that lazy should also be marked at the call 
 site as it is a very different semantic on the caller and on 
 the callee. As it is now, it is true that the example above 
 is unfortunate and a real source of unpleasant surprizes.


 +1

 Step 1: lazy must be used at the call site


 That would play havoc with generic code and would 
 significantly increase the complexity of standard phobos 
 functions like map, filter, etc.


 Can you elaborate? How so?


This is not how those functions are implemented, but it should 
illustrate my point:

auto function map(alias fun)(Range!T range)
{
     T[] result;
     foreach (i, item; range)
         result[i] = fun(item);

     return result;
}

int evenicize(lazy int n)
{
     if (n % 2 == 1)
         return n * 2;
}

auto odds = [1, 3, 5, 7];

//If the call site needs an explicit "lazy" for arguments to lazy 
parameters:
auto evens = odds.map!evenicize(); //Error: must specify "lazy" 
for parameter n when calling function "evenicize"

So then map has to account add a bunch of extra machinery to 
detect whether "fun" has any lazy parameters:

template hasLazyParameter(alias fun)
{
     //Some implementation
}

auto function map(alias fun)(Range!T range)
{
     //Note, if you had to write this as "fun(lazy T.init)", 
typeof would fail here
     typeof(fun(T.init))[] result;
     foreach (i, item; range)
         static if (hasLazyParameter!fun)
             result[i] = fun(lazy item);
         else
             result[i] = fun(item);

     return result;
}

Which actually is not that bad for map, as the function you pass 
to it only takes 1 parameter (although IMO, this is already too 
onerous).

But what about something like std.functional.reverseArgs, which 
has to be able to handle functions with arbitrary arity?
Apr 23 2020
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/23/20 7:13 AM, Patrick Schluter wrote:

 On Thursday, 23 April 2020 at 09:33:35 UTC, aliak wrote:

 On Wednesday, 22 April 2020 at 11:56:44 UTC, kinke wrote:

 I don't think this is a good idea. One of the problems with `lazy` 
 IMO is that one needs good IDE support to somehow highlight that the 
 argument expression isn't evaluated before the call, and might not be 
 evaluated at all:

 // in some other module:
 void foo(lazy int i) {}

 void main()
 {
     int i = 1;
     foo(++i);
     assert(i == 1); // what? not incremented above? ah yeah, lazy and 
 not evaluated at all by foo
 }

 The proposal would extend that problem to regular delegate params 
 too. So I find the explicit version (`() => ++i`) better to make that 
 clear. I don't think the 'overhead' of putting an additional `() =>` 
 in front of the expression justifies introducing the proposed 
 implicit convertibility.


 I agree with this. Implicit conversion has always been argued against 
 in D - why is it ok now? And this is not the good kind of implicit 
 conversion where the user explicitly asks for it, but the bad kind 
 where the language decides it should happen and the user doesn't know 
 it can even happen.

 Thinking about it more now feels like the real rationale for the DIP 
 is "in order to deprecate lazy, we need an alternative" ... ???


 
 I always thought that lazy should also be marked at the call site as it 
 is a very different semantic on the caller and on the callee. As it is 
 now, it is true that the example above is unfortunate and a real source 
 of unpleasant surprizes.
 
 
 void foo(lazy int i) {}
 
   void main()
   {
       int i = 1;
       foo(lazy ++i);
       assert(i == 1); // No surprize here, the required annotation tells 
 it.
   }
 


If lazy binds to delegates then this just becomes

foo(() => ++i);

And then we can require lazy to require a delegate at the call site, and 
then get rid of lazy.

-Steve
Apr 23 2020
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Wednesday, 22 April 2020 at 07:41:12 UTC, Mike Parker wrote:

 Here in the discussion thread, you are free to discuss anything 
 and everything related to the DIP. Express your support or 
 opposition, debate alternatives, argue the merits, etc.


I don't like it. I'm actively avoiding lazy, and delegates are 
generally not very usable in nothrow  nogc -betterC modes. More 
easy syntax for delegates sounds not that desirable to me. I'd 
prefer if lazy was removed.
Apr 22 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 22 April 2020 at 16:37:28 UTC, Guillaume Piolat 
wrote:

 On Wednesday, 22 April 2020 at 07:41:12 UTC, Mike Parker wrote:

 Here in the discussion thread, you are free to discuss 
 anything and everything related to the DIP. Express your 
 support or opposition, debate alternatives, argue the merits, 
 etc.


 I don't like it. I'm actively avoiding lazy, and delegates are 
 generally not very usable in nothrow  nogc -betterC modes. More 
 easy syntax for delegates sounds not that desirable to me. I'd 
 prefer if lazy was removed.


If this DIP is accepted, `lazy` could be deprecated, as it could 
be fully replaced by scope delegates, which are nothrow  nogc 
-betterC compatible.
Apr 22 2020
parent reply kinke <noone nowhere.com> writes:
On Wednesday, 22 April 2020 at 16:53:09 UTC, Petar Kirov 
[ZombineDev] wrote:

 If this DIP is accepted, `lazy` could be deprecated, as it 
 could be fully replaced by scope delegates, which are nothrow 
  nogc -betterC compatible.


We don't need the DIP for that. It would just allow not having to 
touch the call sites (just the lazy *params*), but as said, I 
think the arguments should be forced to be converted to regular 
delegates too, for legibility and less confusion.
Apr 22 2020
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 22 April 2020 at 17:48:35 UTC, kinke wrote:

 On Wednesday, 22 April 2020 at 16:53:09 UTC, Petar Kirov 
 [ZombineDev] wrote:

 If this DIP is accepted, `lazy` could be deprecated, as it 
 could be fully replaced by scope delegates, which are nothrow 
  nogc -betterC compatible.


 We don't need the DIP for that.


We need it in order to avoid API breaking changes. If we 
deprecate `lazy`, without this DIP this code will break:

   const result = cache.require(key, calculateValue(key));


 It would just allow not having to touch the call sites (just 
 the lazy *params*), but as said, I think the arguments should 
 be forced to be converted to regular delegates too, for 
 legibility and less confusion.


Reasonable people may disagree. Just like C programmers like the 
explicitness of passing a pointer to a local variable, while `T&` 
is being touted as one of the big improvements of C++, compared 
to C.
Apr 22 2020
parent kinke <noone nowhere.com> writes:
On Wednesday, 22 April 2020 at 19:08:08 UTC, Petar Kirov 
[ZombineDev] wrote:

 We need it in order to avoid API breaking changes. If we 
 deprecate `lazy`, without this DIP this code will break:

   const result = cache.require(key, calculateValue(key));


I thought these cases could later be simplified by overloading 
these few functions, taking either a value directly or a factory 
delegate. But after scanning for lazy params in Phobos, it looks 
like its use is reasonably restricted to a few places where the 
shortcut syntax is indeed nice (enforce, assertThrown, 
logging...).

But I still think this DIP would be a step in the wrong 
direction; instead, I'd prefer `lazy` to keep designating params 
where arguments are eligible for this shortcut syntax / implicit 
conversion of expressions to delegates. The problems with lazy 
('being an oddity means it is hard to reason about, especially 
with the proliferation of parameter attributes') might be solved 
by transitioning to requiring a delegate type for lazy 
parameters, e.g.:

void write(string);

alias Factory(T) = scope T delegate();
void log(lazy Factory!bool condition, lazy Factory!string msg)
{
     if (condition())
         write(msg());
}

void foo(int p)
{
     log(true, "bla");
     log(() => ++p, "blub"); // enable optional delegate syntax too
}

lazy is used very sparingly in druntime/Phobos, and AFAIK, nobody 
has requested the proposed implicit conversion for other 
expressions so far. So instead of trying out something completely 
new and opening the door for abuse and confusion, I think we 
should rather focus on the few sprinkled usages of lazy params.
Apr 22 2020
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
Followup discussion to Walter's response to my feedback:


 int delegate() = 3;

 doesn't use any globals. Why shouldn't this work?

 int function() = 3;


 
 Because it seems kinda pointless to support function pointers only with
constants.
 
 While it is not impossible to support function pointers with this, I can't
think of a compelling use case. 


Walter, you literally are using int delegate() = 3 as a use case in the 
DIP. How is a delegate that returns a constant more useful (or 
different) than a function pointer that returns a constant?

The statement "the expression can only consist of globals" is just 
factually incorrect. It can consist of globals, new memory, literals, 
enums, thread local data, the result of some other function call with 
those types of parameters.

I tend to think that it doesn't make a whole lot of sense to exclude 
function pointers in this DIP, but even if you disagree, the text fails 
to convey a valid rationale of why they shouldn't be included.

-Steve
Apr 22 2020