www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - nothrow/ nogc inference troubles with emplace/move?

reply Manu <turkeyman gmail.com> writes:
I'm getting a lot of nothrow/ nogc errors where I would have expected the
attributed to be inferred...
In particular; core.lifetime stuff, I can't live without that, but I'm
finding that move and emplace are both often not inferring the nothrow and
 nogc attributes correctly, and so they break my code.

Is this a known issue? Should I expect attribute inference to work
comprehensively?

I've copy-pasted core.lifetime into my own library, stripped it bare and
marked everything `nothrow  nogc`, and that works... but I'm not really
keen to do this.
Aug 26
parent reply Dennis <dkorpel gmail.com> writes:
On Tuesday, 27 August 2024 at 05:17:18 UTC, Manu wrote:

 Is this a known issue? Should I expect attribute inference to 
 work comprehensively?s.


The error message for calling a function that failed attribute 
inference should include the reason the attribute failed to be 
inferred. If it doesn't, you're likely hitting this classic:

[Issue 7205 - Function attribute inference fails in case of 
mutual dependencies 
](https://issues.dlang.org/show_bug.cgi?id=7205)
Aug 27
parent reply Manu <turkeyman gmail.com> writes:
On Tue, 27 Aug 2024 at 21:10, Dennis via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Tuesday, 27 August 2024 at 05:17:18 UTC, Manu wrote:

 Is this a known issue? Should I expect attribute inference to
 work comprehensively?s.


 The error message for calling a function that failed attribute
 inference should include the reason the attribute failed to be
 inferred. If it doesn't, you're likely hitting this classic:

 [Issue 7205 - Function attribute inference fails in case of
 mutual dependencies
 ](https://issues.dlang.org/show_bug.cgi?id=7205)


Oh wow... that's bad. Is there any plan to fix this?
Aug 28
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 28 August 2024 at 08:59:49 UTC, Manu wrote:

 Oh wow... that's bad. Is there any plan to fix this?


I want to, but with dmd's current architecture, I don't know how. 
The tricky case is:

```D
// example of  safe inference, but the same applies to pure 
nothrow and  nogc

void systemFunc()  system;

void fun1()()
{
     fun2();
     systemFunc();
}

void fun2()()
{
     fun1();
}

void main0()  system
{
     fun1();
}

void main1()  safe
{
     fun2(); // should error
}
```

fun1 gets analyzed first, which gets interrupted when it sees the 
call to fun2.
Then fun2 gets analyzed, but that sees a call to fun1, which at 
that point is still in the process of inferring attributes. The 
current implementation gives up here and infers fun2 as ` system`.

So I tried replacing that pessimistic assumption with an 
optimistic assumption, but in this case, `fun1` will turn out to 
be ` system` because of the `systemFunc()` call. But at the time 
`fun2` ends its analysis, this is completely unknown. Until 
`fun1` ends its analysis, the needed information isn't there.

I could start by inferring `fun2` as ` safe` and then retract 
that once `fun1` finishes analysis, but currently, the compiler 
assumes a function type to be final after its body was analyzed, 
so mutating the type later is going to mess up things.

So without re-architecturing dmd's semantic analysis, I don't see 
a way out.
Aug 28
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Wed, 28 Aug 2024 at 20:11, Dennis via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Wednesday, 28 August 2024 at 08:59:49 UTC, Manu wrote:

 Oh wow... that's bad. Is there any plan to fix this?


 I want to, but with dmd's current architecture, I don't know how.
 The tricky case is:

 ```D
 // example of  safe inference, but the same applies to pure
 nothrow and  nogc

 void systemFunc()  system;

 void fun1()()
 {
      fun2();
      systemFunc();
 }

 void fun2()()
 {
      fun1();
 }

 void main0()  system
 {
      fun1();
 }

 void main1()  safe
 {
      fun2(); // should error
 }
 ```

 fun1 gets analyzed first, which gets interrupted when it sees the
 call to fun2.
 Then fun2 gets analyzed, but that sees a call to fun1, which at
 that point is still in the process of inferring attributes. The
 current implementation gives up here and infers fun2 as ` system`.

 So I tried replacing that pessimistic assumption with an
 optimistic assumption, but in this case, `fun1` will turn out to
 be ` system` because of the `systemFunc()` call. But at the time
 `fun2` ends its analysis, this is completely unknown. Until
 `fun1` ends its analysis, the needed information isn't there.

 I could start by inferring `fun2` as ` safe` and then retract
 that once `fun1` finishes analysis, but currently, the compiler
 assumes a function type to be final after its body was analyzed,
 so mutating the type later is going to mess up things.

 So without re-architecturing dmd's semantic analysis, I don't see
 a way out.


I reckon while parsing a function X, you could gather any evidence
available that invalidates the inference, and in lieu of any invalidation
when you encounter Y that's not itself resolved, place a token on X that
it's waiting on Y (and another token that it's waiting on Z, etc), then
also place a token on Y and Z that says when it's finished it's own
resolution it should poke the result back to X.
What will happen then is Y may run inference and be invalidated, in which
case it reports the invalidation result back to X, and that may invalidate
a cascade of pending inferences... or it may itself not be finalised
waiting optimistically on the inference of X (or some other cycle).
At the end when everything's had a go, all outstanding tokens must be
involved in optimistic cycles since there was nothing in any of those
functions that invalidated their inferences; and since they're all
outstanding on an optimistic cycle, then I think they naturally all satisfy
eachother in the optimistic case. So just close out all outstanding tokens
in the optimistic case... does that sound right?
Aug 28
parent reply Dennis <dkorpel gmail.com> writes:
On Wednesday, 28 August 2024 at 11:20:43 UTC, Manu wrote:

 At the end when everything's had a go
 (...)
 So just close out all outstanding tokens
 in the optimistic case... does that sound right?


That's what I tried to implement, with each function keeping an 
array of callers of that function, so it could propagate an 
attribute violation to functions that had 'outstanding tokens' 
from calling that function.

But we don't have the luxury to wait, because D code may inspect 
function types and demand an answer. Here's a (contrived) example:

```D
import std.traits;

 system void systemFunc();

void fun1()()
{
     alias T = typeof(fun2()); // is fun2  safe? need an answer 
for T now!

     // let's make a contradiction!
     static if (hasFunctionAttributes!(fun2!(), " safe"))
         systemFunc();
}

void fun2()()
{
     fun1();
}

void main0()  system
{
     fun1();
}

void main()  safe
{
     fun2(); // not system!
}
```

We don't need to support such contrived examples of course, but 
even for normal code, there are many places where the compiler 
inspects the type of a function. Making it account everywhere for 
the possibility that the function's attributes can still change, 
even after semantic analysis has finished, is a complex task.
Aug 28
parent reply Manu <turkeyman gmail.com> writes:
On Wed, 28 Aug 2024 at 22:11, Dennis via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On Wednesday, 28 August 2024 at 11:20:43 UTC, Manu wrote:

 At the end when everything's had a go
 (...)
 So just close out all outstanding tokens
 in the optimistic case... does that sound right?


 That's what I tried to implement, with each function keeping an
 array of callers of that function, so it could propagate an
 attribute violation to functions that had 'outstanding tokens'
 from calling that function.

 But we don't have the luxury to wait, because D code may inspect
 function types and demand an answer. Here's a (contrived) example:

 ```D
 import std.traits;

  system void systemFunc();

 void fun1()()
 {
      alias T = typeof(fun2()); // is fun2  safe? need an answer
 for T now!

      // let's make a contradiction!
      static if (hasFunctionAttributes!(fun2!(), " safe"))
          systemFunc();
 }

 void fun2()()
 {
      fun1();
 }

 void main0()  system
 {
      fun1();
 }

 void main()  safe
 {
      fun2(); // not system!
 }
 ```

 We don't need to support such contrived examples of course, but
 even for normal code, there are many places where the compiler
 inspects the type of a function. Making it account everywhere for
 the possibility that the function's attributes can still change,
 even after semantic analysis has finished, is a complex task.


Yeah I get that... it's complex because there's not enough passes; a lot is
done in one or a few passes, so there's no way to be confident that you can
establish some compile-time facts before you expect they could be used...

I notice that all of your inferred functions are templates, and that
reminded me a thought I had the other night... I don't see any reason
non-templates couldn't be subject to attribute inferrence too; just that
they shouldn't have it mangled into the name. If it carries a silent
attribute that it inferred, it could be super convenient since there are so
many attributes to juggle these days.
It would also mean people who didn't write code aggressively pursuing
attributes may inadvertently write libraries that still CAN be used by more
context sensitive users. I've noticed on my current project that I have to
throw basically every library every written out the window... including
druntime, etc. Heaps of druntime is compatible with my code, but because
someone didn't put the attribute on it, I can't call it. It would be good
if it inferred the full set of attributes silently, and then I could
actually use the libs.
Aug 28
next sibling parent Dennis <dkorpel gmail.com> writes:
On Wednesday, 28 August 2024 at 12:57:24 UTC, Manu wrote:

 Yeah I get that... it's complex because there's not enough 
 passes; a lot is done in one or a few passes, so there's no way 
 to be confident that you can establish some compile-time facts 
 before you expect they could be used...


I wish attribute checking were a separate pass that was 
completely subtractive. I.e. it rejects programs with violations, 
but doesn't affect program semantics otherwise. But that's not 
the case with D's current feature set.


 I don't see any reason
 non-templates couldn't be subject to attribute inferrence too;


Inference for all came up in the last DLF monthly as something we 
want to pursue.


 just that
 they shouldn't have it mangled into the name.


That's an interesting aspect that didn't come up. It could be 
confusing when the listed function type differs from the de-facto 
type with respect to attributes, but worth considering.


 druntime, etc. Heaps of druntime is compatible with my code, 
 but because
 someone didn't put the attribute on it, I can't call it.


druntime and Phobos should be relatively well-annotated, but I've 
stumbled on missing `nothrow` and ` nogc` in Windows bindings 
before. If you have any specifics, please post them on bugzilla.
Aug 28
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/28/2024 5:57 AM, Manu wrote:

 Heaps of druntime is compatible with my code, but because someone didn't put
the 
 attribute on it, I can't call it.


You can add the attribute and submit a PR for it! Or submit a bugzilla issue.
Or 
email Dennis or myself. Or post in the n.g.

But we can't fix any of these because you don't report them!

There's no reason to suffer from this.
Aug 28
parent reply Manu <turkeyman gmail.com> writes:
On Thu, 29 Aug 2024 at 11:16, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 On 8/28/2024 5:57 AM, Manu wrote:

 Heaps of druntime is compatible with my code, but because someone didn't


 put the

 attribute on it, I can't call it.


 You can add the attribute and submit a PR for it! Or submit a bugzilla
 issue. Or
 email Dennis or myself. Or post in the n.g.

 But we can't fix any of these because you don't report them!

 There's no reason to suffer from this.



Sure, but the point extends past druntime. 3rd party libs often just don't
make any attempt. Don't brush off the problem.
Aug 28
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/28/2024 7:31 PM, Manu wrote:

 Sure, but the point extends past druntime. 3rd party libs often just don't
make 
 any attempt. Don't brush off the problem.


I have no control over 3rd party libraries. But druntime, I can fix. Please
post 
the issues there.
Aug 30
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 8/31/24 01:10, Walter Bright wrote:

 On 8/28/2024 7:31 PM, Manu wrote:

 Sure, but the point extends past druntime. 3rd party libs often just 
 don't make any attempt. Don't brush off the problem.


 
 I have no control over 3rd party libraries. But druntime, I can fix.


You can't just add the attributes. That will break someone else's code.
Aug 31
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Excellent explanatory cases like this should be added to the bug report.
Aug 28