digitalmars.dip.ideas - Delegates and qualifier transitivity
- Dukc (64/64) Dec 05 2025 D type qualifiers - `const`, `immutable`, `shared` and `inout` -
- Dukc (6/8) Dec 08 2025 Duh, there is a hole in my proposal as [Timon noted in a DMD
- Dukc (5/7) Dec 24 2025 Update: I have not given up. I have spent time reading the spec
- Dukc (34/36) Dec 29 2025 Well, investigation finally done. Fortunately I don't think the
- Quirin Schroll (81/119) Jan 15 First, let me say that this is a tricky area and even I struggle
- Dukc (50/90) Jan 16 It definitely involves thinking really hard and slow for me.
- Dukc (6/16) Dec 29 2025 I was mistaken on this point. In my example, `imm` is typed as
- Dukc (5/13) Dec 29 2025 I still think this point stands, but changing it isn't necessary
D type qualifiers - `const`, `immutable`, `shared` and `inout` -
are supposed to be transitive.
Delegates are supposed to work like structs that contain a
function pointer and an untyped context pointer. In my opinion at
least, this means the context pointer should have the same
qualifiers as the delegate itself. In other words,
`immutable(void delegate())` should be the same as
`immutable(void delegate() immutable)` (it's still fine if the
return type is mutable.).
Currently the compiler seems so buggy in this regard that I'm not
sure what it tries to do. While testing, I even managed to get a
declaration like `immutable del = &immutableStruct.immutableFun;`
to have the compiler say "Error: cannot implicitly convert
expression `&immutableStruct.immutableFun` of type `int
delegate() immutable safe` to `immutable(int delegate() safe)`"
Which leads to implicit conversions between delegates being wrong:
```D
struct S
{ int field;
safe mutableFun(){}
safe constFun() const{}
safe immutableFun() immutable{}
}
safe void main()
{ void delegate() safe mut;
void delegate() safe const con;
void delegate() safe immutable imm;
S s1;
immutable S s2;
mut = &s1.mutableFun; //Compiles, as it should
mut = &s1.constFun; //Compiles as it should
mut = &s2.immutableFun; //Doesn't compile but should
con = &s1.mutableFun; //Doesn't compile and neither should
con = &s1.constFun; //Compiles as it should
con = &s2.immutableFun; //Doesn't compile but should
imm = &s1.mutableFun; //Doesn't compile and neither should
imm = &s1.constFun; //Compiles but shouldn't
imm = &s2.immutableFun; //Compiles, as it should
}
```
You might be wondering why `mut = &s2.immutableFun;` should be
allowed in safe code - a context pointer to mutable even when the
struct is immutable. Well, the pointer is typed as `void*`. You
can't mutate through that in ` safe` code, and calling the
delegate will not do so either since the referred function
actually takes the context pointer as immutable. The compiler
guarantees it does, as it rejects delegates like `&s2.mutableFun`.
On the other hand, `imm = &s1.constFun;` is dangerous. While
`constFun` itself wont mutate the struct, immutability assumes
no-one will mutate the data. The compiler is free to assume the
context of the delegate will remain untouched by other things
done in the calling function, yet clearly the assumption could be
easily broken by mutating `s1`, being a mutable struct. Also, an
immutable delegate could be stored to thread-shared memory, and
the results of calling it would depend on a thread-local context
- the very issue `immutable` and `shared` are supposed to prevent.
Changing the behaviour will in all likelihood lead to fairly
large breakage, which means it must be done over an edition
switch. But first, before I go on to write an actual DIP (and
maybe the proposed compiler changes also), I'd like to ask you to
check my assessment. Is there a reason other than just plain
misdesign why it works how it works right now? If not, the fix it
pictured above seems clear to me, but is it really? Is there
another way to solve this that should be considered?
Dec 05 2025
On Friday, 5 December 2025 at 16:37:35 UTC, Dukc wrote:If not, the fix it pictured above seems clear to me, but is it really?Duh, there is a hole in my proposal as [Timon noted in a DMD issue suggesting part of what I did](https://github.com/dlang/dmd/issues/19131#issuecomment-2541990651). Yet the way how it works now is also still clearly wrong. I have to rethink this. I suspect the solution will have to be quite hairy to be sound. Ideas, anyone?
Dec 08 2025
Dukc <ajieskola gmail.com> On Monday, 8 December 2025 at 20:55:29 UTC, Dukc wrote:I have to rethink this. I suspect the solution will have to be quite hairy to be sound. Ideas, anyone?Update: I have not given up. I have spent time reading the spec and DMD source, but had too much else to do lately. I should get this proposal fixed within Christmastide, probably within this year.
Dec 24 2025
On Monday, 8 December 2025 at 20:55:29 UTC, Dukc wrote:I have to rethink this. I suspect the solution will have to be quite hairy to be sound. Ideas, anyone?Well, investigation finally done. Fortunately I don't think the solution is going to be as hairy as I feared, if it gets accepted. First, I should clarify how the compiler currently thinks about delegate qualifiers as this isn't written in the spec or anything, and unless I was mistaken in my initial testing there are also bugs that further confound what's the intended scheme. It does not think about them transitively. `qualifier(void delegate())` and `qualifier(void delegate() qualifier)` are treated as different types. When considering whether a delegate `T delegate() qualifierA` can be assigned to `T delegate() qualifierB`, the answer is the the same as whether `T function(qualifierA void*)` can be assigned to `T function(qualifierB void*)`. Meaning, `void delegate() const` can be assigned to `void delegate()`, but not vice-versa. What about qualifiers to the whole delegate? Converting between them is not restricted. At all. You can convert between `yourDelegate`, `immutable(yourDelegate)`, `const(yourDelegate)`, `inout(yourDelegate)` and even `shared(yourDelegate)` just as freely as if you replaced `yourDelegate` with `int`. Obviously, this breaks the type system assumptions about qualifier transitivity hard. What to do about this? I still the delegate qualifiers should also apply to the `this` argument. You probably don't want a delegate with an `immutable` context that can't actually be called with one, and if you really do you can accomplish that manually with a function pointer/context pointer pair. Instead, qualifier conversions between delegates should be restricted in ` safe` code. You would be able to safely cast `qualifier(T delegate(args))` and `T delegate(args) qualifier` to each other, but just like you can't cast `immutable(int**)` to `int**` you couldn't cast `immutable int delegate()` to `int delegate`. This also needs to apply to pure factory function casts of mutables to `const` or `immutable`.
Dec 29 2025
First, let me say that this is a tricky area and even I struggle at times with it despite having a mathematics degree. It seemed Timon Gehr and I were the only ones who cared about getting this problem solved. On Monday, 29 December 2025 at 17:51:46 UTC, Dukc wrote:On Monday, 8 December 2025 at 20:55:29 UTC, Dukc wrote:This is not the right analysis. A delegate is *implemented* very similar to a struct with a `void*` context and a function pointer, but that doesn’t suffice for what you’re trying. A delegate also has an important invariant: The function pointer and the context match, i.e. it is valid to call the function pointer “on” the context. That is checked (ideally) statically when creating a delegate. Thus, the components of a delegate cannot be assigned individually in ` safe` code. For that reason, all qualifiers, including `immutable` and `shared`, as member function attributes on delegate types, only provide outward guarantees and don’t make inward demands after the delegate is formed. **That leads to the following diagram:** ``` immutable (←) function ↓ const inout shared → const shared / inout shared → shared ↓ ↓ ↓ ↓ const inout → const / inout → (mutable) ``` **The rules:** 1. `immutable` converts to anything. 2. everything converts to *mutable.* 3. `const`, `shared`, and `inout` can be removed at will. 4. `const`, `shared`, and `inout` cannot be added. In the diagram, `/` means that there are two separate diagrams, one with the left-hand sides and one with the right-hand sides. The conversion indicated by `(←)` means that it’s not a reference conversion, but a value conversion, very much like `int` to `long`. **The rule for delegate formation:** If the qualifier member function attributes are `quals`, for all captured variables `var`, the assignment `ref quals(typeof(var)) _ = var` must be valid. Inference for lambdas etc. should attempt to apply all viable member function attributes (`immutable`, `const`, `shared`, and, in an `inout` context, also `inout`), very much like `pure`, `nothrow`, ` nogc`, and ` safe`. It might be worth adding the keyword `function` as an attribute for delegate types specifically to encode the fact that there is no context at all.I have to rethink this. I suspect the solution will have to be quite hairy to be sound. Ideas, anyone?Well, investigation finally done. Fortunately I don't think the solution is going to be as hairy as I feared, if it gets accepted. First, I should clarify how the compiler currently thinks about delegate qualifiers as this isn't written in the spec or anything, and unless I was mistaken in my initial testing there are also bugs that further confound what's the intended scheme. It does not think about them transitively. `qualifier(void delegate())` and `qualifier(void delegate() qualifier)` are treated as different types. When considering whether a delegate `T delegate() qualifierA` can be assigned to `T delegate() qualifierB`, the answer is the the same as whether `T function(qualifierA void*)` can be assigned to `T function(qualifierB void*)`. Meaning, `void delegate() const` can be assigned to `void delegate()`, but not vice-versa.What about qualifiers to the whole delegate? Converting between them is not restricted. At all. You can convert between `yourDelegate`, `immutable(yourDelegate)`, `const(yourDelegate)`, `inout(yourDelegate)` and even `shared(yourDelegate)` just as freely as if you replaced `yourDelegate` with `int`. Obviously, this breaks the type system assumptions about qualifier transitivity hard. What to do about this? I still the delegate qualifiers should also apply to the `this` argument. You probably don't want a delegate with an `immutable` context that can't actually be called with one, and if you really do you can accomplish that manually with a function pointer/context pointer pair. Instead, qualifier conversions between delegates should be restricted in ` safe` code. You would be able to safely cast `qualifier(T delegate(args))` and `T delegate(args) qualifier` to each other, but just like you can't cast `immutable(int**)` to `int**` you couldn't cast `immutable int delegate()` to `int delegate`. This also needs to apply to pure factory function casts of mutables to `const` or `immutable`.Now, what about outside qualifiers? They can change what the object can do. A delegate can only be invoked, so the question is: When can a qualified delegate type be invoked? Here, `immutable` equals `immutable const inout shared`. **The rule:** The delegate type `q₁(R delegate(…) q₂)` can be invoked if: 1. `q₁` is a subset of `q₂`, or 2. `q₂` includes `const` and `shared`. Clause 1 expresses that the function pointer guarantees all that is asked of it. Clause 2 works because if the delegate guarantees it’s not going to incur changes through the context (it’s `const`) and guarantees that it’s doing it in a thread-safe manner, the invocation is valid. What about `immutable(int delegate() const)`? It doesn’t match Clause 1 or Clause 2, so it’s not valid to invoke. That is because `immutable` is implicitly `shared`. That would allow passing a callable delegate to different threads that could invoke it concurrently despite it not supporting that. Outside qualifiers don’t matter for `pure` factory delegates at all. Only `R delegate(…) pure immutable` provides enough guarantees to make a unique object, and it can always be invoked, no matter what outside qualifiers you apply to it. The next weaker qualification is `pure const inout shared` and it’s not enough: `inout` makes no guarantees (it could be *mutable*), and `const` plus `shared` isn’t enough as mutable indirections might exist. Also, why shouldn’t you be able to convert an `immutable(R delegate(…) const shared)` to `shared(R delegate(…) const shared)`? By copy, that is. An obvious principle is: If a delegate type can’t be invoked, it definitely can’t be converted to something that can be invoked. While a delegate is a reference type, removing `inout`/`immutable` from the context is fine while retaining `const` and `shared` because the context is opaque and the function pointer is tightly coupled.
Jan 15
On Thursday, 15 January 2026 at 23:53:20 UTC, Quirin Schroll wrote:First, let me say that this is a tricky area and even I struggle at times with it despite having a mathematics degree. It seemed Timon Gehr and I were the only ones who cared about getting this problem solved.It definitely involves thinking really hard and slow for me. Thanks for reading despite the difficulty!This is not the right analysis. A delegate is *implemented* very similar to a struct with a `void*` context and a function pointer, but that doesn’t suffice for what you’re trying. A delegate also has an important invariant: The function pointer and the context match, i.e. it is valid to call the function pointer “on” the context. That is checked (ideally) statically when creating a delegate. Thus, the components of a delegate cannot be assigned individually in ` safe` code.Here, I was describing how the language *currently* works. I agree the invariant you write about should hold at all times but currently, it does not.For that reason, all qualifiers, including `immutable` and `shared`, as member function attributes on delegate types, only provide outward guarantees and don’t make inward demands after the delegate is formed.That right, and I don't think that parts needs to change. It is why `void delegate() const` can be assigned to `void delegate()`. `void delegate()` does not have to actually be able to mutate it's parameter, it just needs to work with variables that might be mutated by others, just like `void delegate() const`.**The rules:** 1. `immutable` converts to anything. 2. everything converts to *mutable.* 3. `const`, `shared`, and `inout` can be removed at will. 4. `const`, `shared`, and `inout` cannot be added.Yes, I agree with allowing all of these as far as I see. I was just writing that I'd be more liberal I don't see an issue converting `R delegate() const` to `R delegate immutable`, which is also allowed right now, because the context pointer isn't immutable so you don't end up actually breaking the assumption of the function you're pointing to. But I just realised I also proposed that `R delegate immutable` and `immutable(R delegate)` would freely convert to each other, which means this would be possible: ```D S var; void delegate() immutable del1 = &var.constMemFun; immutable void delegate del2 = del1; // context of del2 modified -> undefined behaviour var.field++; ``` So I guess we need to stick with what you rule here, although I'll need to think this over once more another day before I feel I have an informed opinion.Inference for lambdas etc. should attempt to apply all viable member function attributes (`immutable`, `const`, `shared`, and, in an `inout` context, also `inout`), very much like `pure`, `nothrow`, ` nogc`, and ` safe`.So also for member functions that have auto-inference on I presume. I'm not sure this is a good idea, or at least not when bundled with the delegate qualifier fix DIP we're discussing. Auto inference is a separate language feature after all.It might be worth adding the keyword `function` as an attribute for delegate types specifically to encode the fact that there is no context at all.Do you mean that we could write `int function(int) safe` alternatively as `int delegate(int) safe function`? I think this is also off scope for the DIP I'm thinking.**The rule:** The delegate type `q₁(R delegate(…) q₂)` can be invoked if: 1. `q₁` is a subset of `q₂`, or 2. `q₂` includes `const` and `shared`.This would be slightly more powerful than what I'm proposing, but also quite a bit more complex and annoying to type, since you couldn't shorten `immutable R delegate() immutable` to `immutable R delegate()`. Would the fact your `immutable(R delegate() const shared)` needs to be `const shared(R delegate())` instead be such a major issue that the complications are worth it?Outside qualifiers don’t matter for `pure` factory delegates at all. Only `R delegate(…) pure immutable` provides enough guarantees to make a unique object, and it can always be invoked, no matter what outside qualifiers you apply to it. The next weaker qualification is `pure const inout shared` and it’s not enough: `inout` makes no guarantees (it could be *mutable*), and `const` plus `shared` isn’t enough as mutable indirections might exist.The issue is not using the delegate itself as the pure factory function. The issue is that a mutable delegate with a mutable context could be *returned* from a pure factory function and then converted to an immutable delegate with an "immutable" context. Check [the issue](https://github.com/dlang/dmd/issues/19131#issuecomment-2541990651) and you'll understand.
Jan 16
On Friday, 5 December 2025 at 16:37:35 UTC, Dukc wrote:On the other hand, `imm = &s1.constFun;` is dangerous. While `constFun` itself wont mutate the struct, immutability assumes no-one will mutate the data. The compiler is free to assume the context of the delegate will remain untouched by other things done in the calling function, yet clearly the assumption could be easily broken by mutating `s1`, being a mutable struct. Also, an immutable delegate could be stored to thread-shared memory, and the results of calling it would depend on a thread-local context - the very issue `immutable` and `shared` are supposed to prevent.I was mistaken on this point. In my example, `imm` is typed as the function taking an immutable context pointer, but the delegate itself is not immutable and therefore the context pointer is neither. Therefore the issues I write about don't apply and this is safe.
Dec 29 2025
On Friday, 5 December 2025 at 16:37:35 UTC, Dukc wrote:You might be wondering why `mut = &s2.immutableFun;` should be allowed in safe code - a context pointer to mutable even when the struct is immutable. Well, the pointer is typed as `void*`. You can't mutate through that in ` safe` code, and calling the delegate will not do so either since the referred function actually takes the context pointer as immutable. The compiler guarantees it does, as it rejects delegates like `&s2.mutableFun`.I still think this point stands, but changing it isn't necessary to fix the type system hole. My actual focus is in qualifier transitivity and disallowing the willy-nilly top-level conversions between them.
Dec 29 2025