www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Help improve error message

reply ryuukk_ <ryuukk.dev gmail.com> writes:
First of all, a little quiz


What should the error message be for the following code?

```D
struct Something{}

void main()
{
     Something test;
     test.get();
}
```


Well.. it's not what you expect..



```
onlineapp.d(6): Error: none of the overloads of template 
`object.get` are callable using argument types `!()(Something)`
/dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3442):  
       Candidates are: `get(K, V)(inout(V[K]) aa, K key, lazy 
inout(V) defaultValue)`
/dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3449):  
                       `get(K, V)(inout(V[K])* aa, K key, lazy 
inout(V) defaultValue)`
```

There is the actual error message, surprising right?

``Something`` is not an associative array, so why AA's functions 
are in the candidate list?

What ``Something`` has to do with ``object.get``??


The correct and desirable error message should be:

```
Error: no property `get` for `test` of type `Something`
```

Not something confusing and misleading.. that's hard to debug

Do you agree with that?

If you do and want to improve D, then join me and try to fix that 
behavior!


2 possible solutions that i can think of:

- functions in ``object.d`` should be cleaned, everything that 
has to do with ``AA`` should only work with ``AA``, and 
everything that is generic, should be renamed so it doesn't 
produce conflicts

- DMD should be smarter and the logic for resolving the function 
should reject these functions, so it can produce the proper error 
message


I am obviously clueless, so my 1st attempt was to remove the 
functions plain and simple, it works for me, but that's not the 
proper solution right..

I suspect that the problem lies here: 
https://github.com/dlang/dmd/blob/54b6a2f2b0a129fa3c4a994658d9f99e3a6d4ee3/compiler/src/dmd/func.d#L3265-L3485

Join me and let's make D better (join the IRC channel so we can 
chat about it)
May 06 2023
next sibling parent reply max haughton <maxhaton gmail.com> writes:
On Saturday, 6 May 2023 at 20:02:30 UTC, ryuukk_ wrote:

 First of all, a little quiz


 What should the error message be for the following code?

 ```D
 struct Something{}

 void main()
 {
     Something test;
     test.get();
 }
 ```


 Well.. it's not what you expect..



 ```
 onlineapp.d(6): Error: none of the overloads of template 
 `object.get` are callable using argument types `!()(Something)`
 /dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3442):       
Candidates are: `get(K, V)(inout(V[K]) aa, K key, lazy inout(V) defaultValue)`
 /dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3449):              
         `get(K, V)(inout(V[K])* aa, K key, lazy inout(V) defaultValue)`
 ```

 There is the actual error message, surprising right?

 ``Something`` is not an associative array, so why AA's 
 functions are in the candidate list?

 What ``Something`` has to do with ``object.get``??


 The correct and desirable error message should be:

 ```
 Error: no property `get` for `test` of type `Something`
 ```

 Not something confusing and misleading.. that's hard to debug

 Do you agree with that?

 If you do and want to improve D, then join me and try to fix 
 that behavior!


 2 possible solutions that i can think of:

 - functions in ``object.d`` should be cleaned, everything that 
 has to do with ``AA`` should only work with ``AA``, and 
 everything that is generic, should be renamed so it doesn't 
 produce conflicts

 - DMD should be smarter and the logic for resolving the 
 function should reject these functions, so it can produce the 
 proper error message


This wouldn't be that hard to do but then you have a tradeoff 
between making this cleaner and making other contexts more 
confusing.

On the compiler side it would be interesting to make it output 
some kind of prose based on some kind of type pattern matching / 
unification e.g. rather than saying "Cannot call X with Y", we'd 
have a much more contextual error message that uses the 
information in the template signature (e.g. for `(K, 
V)(inout(V[K])` to be matched, it must satisfy the pattern, so 
must be an AA of type V[K]).

There's a dlang/projects entry for error messages, this would 
make a good thing to be done in that project.

As for the runtime, I'm not a big fan of these free functions 
either.
May 06 2023
parent RazvanN <razvan.nitu1305 gmail.com> writes:
On Saturday, 6 May 2023 at 23:28:58 UTC, max haughton wrote:

 On Saturday, 6 May 2023 at 20:02:30 UTC, ryuukk_ wrote:

 [...]


 This wouldn't be that hard to do but then you have a tradeoff 
 between making this cleaner and making other contexts more 
 confusing.

 On the compiler side it would be interesting to make it output 
 some kind of prose based on some kind of type pattern matching 
 / unification e.g. rather than saying "Cannot call X with Y", 
 we'd have a much more contextual error message that uses the 
 information in the template signature (e.g. for `(K, 
 V)(inout(V[K])` to be matched, it must satisfy the pattern, so 
 must be an AA of type V[K]).

 There's a dlang/projects entry for error messages, this would 
 make a good thing to be done in that project.

 As for the runtime, I'm not a big fan of these free functions 
 either.


Yes, I think that `get` should not be directly callable but 
rather the compiler should insert a call to it. That way you can 
call it something else in druntime (__getAAKey or somrthing like 
that) and the compiler can lower to it in case the lhs of the dot 
expression is an associative array. It adds a bit of complexity 
in the compiler, but it gets rid of these weird error messages.
May 08 2023
prev sibling next sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Saturday, 6 May 2023 at 20:02:30 UTC, ryuukk_ wrote:

 ...
 it's not what you expect..

 ```
 onlineapp.d(6): Error: none of the overloads of template 
 `object.get` are callable using argument types `!()(Something)`
 /dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3442):       
Candidates are: `get(K, V)(inout(V[K]) aa, K key, lazy inout(V) defaultValue)`
 /dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3449):              
         `get(K, V)(inout(V[K])* aa, K key, lazy inout(V) defaultValue)`
 ```

 There is the actual error message, surprising right?


The interesting thing is that we encountered this error in 
previous versions (eg. v2.087). It hasn't been tried in all 
versions, but I'm sure this error will appear at some version. I 
encountered this once as well.

Now I tried with the following code. Yes, even with the latest 
version I got the same error. However, DMD has such an advanced 
error and suggestion mechanism that it guides you how to use 
opAssign correctly.

```d
struct Something {
   void opAssign(int) {

   }
}

void main() {
      Something test;/* = 1; /*
      `test = 1` is the first assignment of `test` therefore it 
represents its initialization
      `opAssign` methods are not used for initialization, but for 
subsequent assignments//*/

       test = 1;/*
       // Error: no overload matches for `get`
       test.get();//*/
       writeln(__VERSION__*.001); // 2.087
}
```

SDB 79
May 07 2023
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/6/23 4:02 PM, ryuukk_ wrote:

 First of all, a little quiz
 
 
 What should the error message be for the following code?
 
 ```D
 struct Something{}
 
 void main()
 {
      Something test;
      test.get();
 }
 ```
 
 
 Well.. it's not what you expect..


Depends ;) I've been using D long enough that it's exactly what I expected.


 
 There is the actual error message, surprising right?
 
 ``Something`` is not an associative array, so why AA's functions are in 
 the candidate list?
 
 What ``Something`` has to do with ``object.get``??
 
 
 The correct and desirable error message should be:
 
 ```
 Error: no property `get` for `test` of type `Something`
 ```
 
 Not something confusing and misleading.. that's hard to debug
 
 Do you agree with that?
 
 If you do and want to improve D, then join me and try to fix that behavior!
 
 
 2 possible solutions that i can think of:
 
 - functions in ``object.d`` should be cleaned, everything that has to do 
 with ``AA`` should only work with ``AA``, and everything that is 
 generic, should be renamed so it doesn't produce conflicts
 
 - DMD should be smarter and the logic for resolving the function should 
 reject these functions, so it can produce the proper error message


So the thing about this -- you know what you meant. The compiler 
doesn't. In cases where you thought you were using an AA, but weren't, 
this message is very helpful.

When faced with an ambiguity, the compiler has to say something. It says 
what it thinks you wanted it to do, and why that doesn't work. It can't 
read your mind and understand what you thought was going to happen. It 
can't think about what a reasonable person might have done, because it 
doesn't have that power.

If we make it say something different, it's going to confuse another 
developer with another situation. This is why we have so much structure 
around code -- to give the compiler anchor points and delineations to 
try and make it easier to understand what is being asked of it. We do 
the best we can think of to make error messages reflect the predicted 
behavior of people. But it can't be perfect.

UFCS greatly expands the universe of "what you might have meant", and so 
we have the sometimes-bizarre error cases that show up. My 
recommendation is just to get used to what that means. I don't see a 
good way to fix this.

I'll give you another example, one which I just recently fixed (will be 
soon deprecated, and then eventually removed): core.time has a `to` 
function. If you want to use the regular `to` and forgot to import 
`std.conv`, but had imported `core.time`, or imported `core.time` in a 
way that overrides `std.conv` (i.e. locally in a function), you would 
see a bizarre message:

```d
import core.time;

void main()
{
   auto x = "12345".to!int;
}
```

```
onlineapp.d(5): Error: none of the overloads of template `core.time.to` 
are callable using argument types `!(int)(string)`
/dlang/dmd/linux/bin64/../../src/druntime/import/core/time.d(1776): 
   Candidate is: `to(string units, T, D)(D td)`
```

However, if you don't import `core.time`, you get the better error message:

```
onlineapp.d(5): Error: no property `to` for type `string`, perhaps 
`import std.conv;` is needed?
```

Turns out core.time.to is only used for a completely deprecated type 
`TickDuration`, and so we can remove it completely. There's the fix for 
that. We can't do the same for `object.get`.

-Steve
May 07 2023
next sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Sunday, 7 May 2023 at 14:19:49 UTC, Steven Schveighoffer wrote:

 
 There's the fix for that. We can't do the same for `object.get`.


Why?

I do not accept this. Because this problem has a reason! 
Something happened after D Compiler v2.096! If you import the 
std.net.curl module, you get the error code we want.

For example you can try here:
https://wandbox.org/permlink/0I2x7W9kM6e94vl2

SDB 79
May 07 2023
prev sibling next sibling parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Sunday, 7 May 2023 at 14:19:49 UTC, Steven Schveighoffer wrote:


 Well.. it's not what you expect..


 Depends ;) I've been using D long enough that it's exactly what 
 I expected.



See.. that's were my expectation of this language seems to be 
non-compatible with the majority of you, and that's super 
unfortunate because there is nothing i can do to change things

``Something`` is a struct, and you are telling me that it is 
normal for D to treat it as a associative array

With that kind of expectation, i now understand better why the 
.enum DIP got shut down, language is stuck in the past with no 
way to improve anything, including this type identity crisis
May 07 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/7/23 8:50 PM, ryuukk_ wrote:

 On Sunday, 7 May 2023 at 14:19:49 UTC, Steven Schveighoffer wrote:
 

 Well.. it's not what you expect..


 Depends ;) I've been using D long enough that it's exactly what I 
 expected.


 
 
 See.. that's were my expectation of this language seems to be 
 non-compatible with the majority of you, and that's super unfortunate 
 because there is nothing i can do to change things
 
 ``Something`` is a struct, and you are telling me that it is normal for 
 D to treat it as a associative array


No, it's normal that the name `get` when not defined on a type then 
tries UFCS. And because I know `get` is a UFCS function in `object.d`, 
it will pick that up instead of saying "no such function".

What if `Something` had a function called `get`, but it didn't take the 
parameters you sent it? Would it be acceptable to you if it said "no 
function named `get`" instead of detailing why the parameters don't 
match? That's basically what you are asking.


 With that kind of expectation, i now understand better why the .enum DIP 
 got shut down, language is stuck in the past with no way to improve 
 anything, including this type identity crisis


This is unrelated. UFCS is a fabulous feature of the language, and 
understanding the way it works is crucial in order to understand the 
confusing error messages you might see. Not that it's always ideal, but 
I'll take confusing messages and UFCS over the alternative without UFCS 
any day.

FWIW, I was in favor of the enum DIP and think the reasons for rejection 
were invalid. But I've also been here long enough to know when arguing 
over the dead horse isn't worth my time.

-Steve
May 07 2023
next sibling parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Monday, 8 May 2023 at 02:55:57 UTC, Steven Schveighoffer wrote:

 On 5/7/23 8:50 PM, ryuukk_ wrote:

 On Sunday, 7 May 2023 at 14:19:49 UTC, Steven Schveighoffer 
 wrote:
 

 Well.. it's not what you expect..


 Depends ;) I've been using D long enough that it's exactly 
 what I expected.


 
 
 See.. that's were my expectation of this language seems to be 
 non-compatible with the majority of you, and that's super 
 unfortunate because there is nothing i can do to change things
 
 ``Something`` is a struct, and you are telling me that it is 
 normal for D to treat it as a associative array


 No, it's normal that the name `get` when not defined on a type 
 then tries UFCS. And because I know `get` is a UFCS function in 
 `object.d`, it will pick that up instead of saying "no such 
 function".

 What if `Something` had a function called `get`, but it didn't 
 take the parameters you sent it? Would it be acceptable to you 
 if it said "no function named `get`" instead of detailing why 
 the parameters don't match? That's basically what you are 
 asking.


 With that kind of expectation, i now understand better why the 
 .enum DIP got shut down, language is stuck in the past with no 
 way to improve anything, including this type identity crisis


 This is unrelated. UFCS is a fabulous feature of the language, 
 and understanding the way it works is crucial in order to 
 understand the confusing error messages you might see. Not that 
 it's always ideal, but I'll take confusing messages and UFCS 
 over the alternative without UFCS any day.

 FWIW, I was in favor of the enum DIP and think the reasons for 
 rejection were invalid. But I've also been here long enough to 
 know when arguing over the dead horse isn't worth my time.

 -Steve


You don't understand, my complain isn't about UFCS, it's about 
the error message that is misleading when trying to debug a 
problem in your project, the compiler never gives you the hint 
that ``Something`` doesn't have a ``get`` function, it tells you 
that in some other places it doesn't match any overload, it's 
perhaps time saved when you were looking to use an UFCS function, 
but it is time wasted when you were looking to call a function 
from `Something`


I perhaps expect the following error message:


```
onlineapp.d(6): Error: no property `get` for `test` of type 
`Something`
onlineapp.d(6): Note: The following UFCS functions where found:

     
/dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3442):
         `get(K, V)(inout(V[K]) aa, K key, lazy inout(V) 
defaultValue)`
     
/dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3449):
         `get(K, V)(inout(V[K])* aa, K key, lazy inout(V) 
defaultValue)`

onlineapp.d(6): Error: none of the overloads of template 
`object.get` are callable using argument types `!()(Something)`
```

If your language has good feature, diagnostics should match, 
otherwise it's like with this global that can't be debugged, you 
perhaps learned to live with it, i haven't and refuse to do so
May 07 2023
next sibling parent Salih Dincer <salihdb hotmail.com> writes:
On Monday, 8 May 2023 at 03:31:10 UTC, ryuukk_ wrote:

 ... my complain isn't about UFCS, it's about the error message 
 that is misleading when trying to debug a problem in your 
 project, the compiler never gives you the hint...


I put signature under what you say! Obviously, UFCS doesn't get 
along well with DMD. There is an extra get() function as you will 
see below.  This causes the error message [¹] to change 
completely.

```d
struct S {}

void main() {
   S s;
   s.get();
}

void get() {}

```
I don't think there can be a sluggish compiler like DMD! It's 
bullshit. Definitely it needs to be recoded. It's like a rag bag 
all over.

[¹] **onlineapp.d(5): Error: function `onlineapp.get()` is not 
callable using**
**argument types `(S)`**
**onlineapp.d(5):        expected 0 argument(s), not 1**


SDB 79
May 07 2023
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/7/23 11:31 PM, ryuukk_ wrote:


 
 You don't understand, my complain isn't about UFCS, it's about the error 
 message that is misleading when trying to debug a problem in your 
 project, the compiler never gives you the hint that ``Something`` 
 doesn't have a ``get`` function, it tells you that in some other places 
 it doesn't match any overload, it's perhaps time saved when you were 
 looking to use an UFCS function, but it is time wasted when you were 
 looking to call a function from `Something`


I take the "no overloads match for function in unrelated module" to mean 
the function doesn't exist as a method. If there was even one `get` 
method, then it would not even look for other functions.

This is somewhat par for the course in languages that have overload 
resolution. It found something by that name, but couldn't make it work. 
The unique part about D is that those functions can be outside the type 
even though they look like they are method calls.


 
 
 I perhaps expect the following error message:
 
 
 ```
 onlineapp.d(6): Error: no property `get` for `test` of type `Something`
 onlineapp.d(6): Note: The following UFCS functions where found:
 
 /dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3442):
          `get(K, V)(inout(V[K]) aa, K key, lazy inout(V) defaultValue)`
 /dlang/dmd/linux/bin64/../../src/druntime/import/object.d(3449):
          `get(K, V)(inout(V[K])* aa, K key, lazy inout(V) defaultValue)`
 
 onlineapp.d(6): Error: none of the overloads of template `object.get` 
 are callable using argument types `!()(Something)`
 ```


Once you get used to UFCS, you will understand these error messages easier.

But the compiler can always use clearer error messages.


 If your language has good feature, diagnostics should match, otherwise 
 it's like with this global that can't be debugged, you perhaps learned 
 to live with it, i haven't and refuse to do so


Not being able to debug the global is an actual bug though, not a 
language design feature.

My comment on it was that I don't really use debuggers, even for other 
languages. I didn't "learn to live with it", I just never needed it.

-Steve
May 08 2023
prev sibling parent reply Dennis <dkorpel gmail.com> writes:
On Monday, 8 May 2023 at 03:31:10 UTC, ryuukk_ wrote:

 I perhaps expect the following error message:


 ```
 onlineapp.d(6): Error: no property `get` for `test` of type 
 `Something`
 onlineapp.d(6): Note: The following UFCS functions where found:
 ```


https://github.com/dlang/dmd/pull/15208
May 09 2023
parent ryuukk_ <ryuukk.dev gmail.com> writes:
On Tuesday, 9 May 2023 at 09:37:37 UTC, Dennis wrote:

 On Monday, 8 May 2023 at 03:31:10 UTC, ryuukk_ wrote:

 I perhaps expect the following error message:


 ```
 onlineapp.d(6): Error: no property `get` for `test` of type 
 `Something`
 onlineapp.d(6): Note: The following UFCS functions where found:
 ```


 https://github.com/dlang/dmd/pull/15208


Thanks a lot!
May 09 2023
prev sibling parent Max Samukha <maxsamukha gmail.com> writes:
On Monday, 8 May 2023 at 02:55:57 UTC, Steven Schveighoffer wrote:

 UFCS is a fabulous feature of the language,


It would be fabulous if:
1. it worked with operators
2. overload sets of member and global functions were supported
3. there are other issues with it, but I can't remember what
May 08 2023
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 7 May 2023 at 14:19:49 UTC, Steven Schveighoffer wrote:

 [..]


I completely agree about you in the general case. However, just 
like you highlighted the interaction between `std.conv.to` and 
`core.time.to`, I believe we do have something we can do to 
improve the status quo in this specific case, although not easily 
feasible in the current language - if the compiler lowered `V[K]` 
types to a fully library defined type (say 
`core.associative_array.Hash(Key, Value)`), then all the 
AA-related free functions we currently have in `object.d` would 
become member functions and then the compiler wouldn't consider 
them as overload candidates.

Speaking of this, I now remembered that you created this project: 
https://github.com/schveiguy/newaa

What language changes are necessary before we can replace the 
built-in AAs with a completely library-based solution?
May 08 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/8/23 5:53 AM, Petar Kirov [ZombineDev] wrote:

 On Sunday, 7 May 2023 at 14:19:49 UTC, Steven Schveighoffer wrote:

 [..]


 
 I completely agree about you in the general case. However, just like you 
 highlighted the interaction between `std.conv.to` and `core.time.to`, I 
 believe we do have something we can do to improve the status quo in this 
 specific case, although not easily feasible in the current language - if 
 the compiler lowered `V[K]` types to a fully library defined type (say 
 `core.associative_array.Hash(Key, Value)`), then all the AA-related free 
 functions we currently have in `object.d` would become member functions 
 and then the compiler wouldn't consider them as overload candidates.


That possibly could improve the situation. UFCS "properties" for AAs are 
kinda sucky.

But that still leaves normal array properties, like `.dup`. I don't see 
us ever changing slices to templated structs.


 
 Speaking of this, I now remembered that you created this project: 
 https://github.com/schveiguy/newaa
 
 What language changes are necessary before we can replace the built-in 
 AAs with a completely library-based solution?


Two problems. One is straightforward, but requires some language 
updates: Having some mechanism to tell `opIndex` that the result should 
be inserted if it doesn't exist. Used in cases like:
```d
int[int][int] aa;

++aa[42][24]; // currently compiles
```

The second is not as straightforward. There are a few implicit casts 
that have no analog for structs. Without some way to say e.g. `Foo!(T)` 
can be casted implicitly to `Foo!(const(T))`, I don't know how we can 
figure that one.

Aside from that, there are a lot of things that can be done to newaa 
that will bring it closer to usable. I just haven't done them yet.

The biggest use case right now is building an AA at compile time, and 
using it at runtime.

-Steve
May 08 2023
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 8 May 2023 at 16:08:32 UTC, Steven Schveighoffer wrote:

 [..]

 That possibly could improve the situation. UFCS "properties" 
 for AAs are kinda sucky.

 But that still leaves normal array properties, like `.dup`. I 
 don't see us ever changing slices to templated structs.


Hmm, that doesn't sound like the worst idea. From implementation 
point of view, we much be extra careful to avoid any sort of 
pessimization (e.g. template code bloat), since slices are used 
everywhere, but other than that, why not?

In some languages, even built-in integer types are 
"library-defined", see e.g.:

Swift:
* 
[`stdlib/public/core/IntegerTypes.swift.gyb`](https://github.com/apple/swift/blob/main/stdlib/public/core/IntegerTypes.swift.gyb)
Rust:
* 
[`library/core/src/ops/arith.rs`](https://github.com/rust-lang/rust/blob/a0111af531178e48375f14f838d7a2298524067c/library/core/src/ops/arith.rs#L94-L109)
* 
[`library/core/src/num/int_macros.rs`](https://github.com/rust-lang/rust/blob/a0111af531178e48375f14f838d7a2298524067c/library/core/src/num/int_macros.rs)
* 
[`library/core/src/num/mod.rs`](https://github.com/rust-lang/rust/blob/a0111af531178e48375f14f838d7a2298524067c/library/core/src/num/mod.rs#L245-L264)

Anyway, let's not get ahead of ourselves.


 
 Speaking of this, I now remembered that you created this 
 project: https://github.com/schveiguy/newaa
 
 What language changes are necessary before we can replace the 
 built-in AAs with a completely library-based solution?


 Two problems. One is straightforward, but requires some 
 language updates: Having some mechanism to tell `opIndex` that 
 the result should be inserted if it doesn't exist. Used in 
 cases like:
 ```d
 int[int][int] aa;

 ++aa[42][24]; // currently compiles
 ```


Yeah, the so-called 
[autovivification](https://en.wikipedia.org/wiki/Autovivification). For a
moment I was confused as to what's the problem with simply:
1. Having `opIndex` return by `ref`, so you can mutate the value 
in-place.
2. Having `opIndex` construct the value on first access, 
something along the lines of:
   `if (key !in impl) return impl.at(key).emplace!Value();`

But then I remembered that `aa[missingKey]` actually throws a 
`RangeError` and we wouldn't be able to replicate the behaviour 
if we did 2.

I wonder if the compiler simply could lower:

```d
int[int][int] aa;
++aa[42][24];
```

to:

```d
int[int][int] aa;
++aa.require(42).require(24);
```

Would that address this issue?


 The second is not as straightforward. There are a few implicit 
 casts that have no analog for structs. Without some way to say 
 e.g. `Foo!(T)` can be casted implicitly to `Foo!(const(T))`, I 
 don't know how we can figure that one.


Thanks for reminding me, I think this was discussed many times in 
the past, IIRC it was most recently brought up by deadalnix in 
the sumtypes thread.

If we consider this issue unsolvable for the time being, a 
pragmatic way forward would be to consider the following:
The compiler already special cases `T[]` and `V[K]` types with 
regards to type qualifiers. Perhaps we can keep this code around 
and update it to work with the new AA implementation. That would 
mean that the implicit type qualifier conversion won't work if 
one directly imports the AA type: 
`core.associative_array.Hash!(K, V)`, however existing code that 
uses the `V[K]` syntax would continue working fine.


 Aside from that, there are a lot of things that can be done to 
 newaa that will bring it closer to usable. I just haven't done 
 them yet.

 The biggest use case right now is building an AA at compile 
 time, and using it at runtime.


Nevertheless, I think you have very solid progress so far! 
Hopefully, it won't be too long before we incorporate it in some 
form in Druntime!
May 08 2023
next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 5/8/23 6:12 PM, Petar Kirov [ZombineDev] wrote:

 On Monday, 8 May 2023 at 16:08:32 UTC, Steven Schveighoffer wrote:

 [..]

 That possibly could improve the situation. UFCS "properties" for AAs 
 are kinda sucky.

 But that still leaves normal array properties, like `.dup`. I don't 
 see us ever changing slices to templated structs.


 
 Hmm, that doesn't sound like the worst idea. From implementation point 
 of view, we much be extra careful to avoid any sort of pessimization 
 (e.g. template code bloat), since slices are used everywhere, but other 
 than that, why not?
 
 In some languages, even built-in integer types are "library-defined", 
 see e.g.:
 
 Swift:
 * 
 [`stdlib/public/core/IntegerTypes.swift.gyb`](https://github.com/apple/swift/blob/main/stdlib/public/core/IntegerTypes.swift.gyb)
 Rust:
 * 
 [`library/core/src/ops/arith.rs`](https://github.com/rust-lang/rust/blob/a0111af531178e48375f14f838d7a2298524067c/library/core/src/ops/arith.rs#L94-L109)
 * 
 [`library/core/src/num/int_macros.rs`](https://github.com/rust-lang/rust/blob/a0111af531178e48375f14f838d7a2298524067c/library/core/src/num/int_macros.rs)
 * 
 [`library/core/src/num/mod.rs`](https://github.com/rust-lang/rust/blob/a0111af531178e48375f14f838d7a2298524067c/library/core/src/num/mod.rs#L245-L264)
 
 Anyway, let's not get ahead of ourselves.]


Yeah, not touching that ;) To be frank, I've used swift's different 
integer types, and it sucks compared to D.


 

 Speaking of this, I now remembered that you created this project: 
 https://github.com/schveiguy/newaa

 What language changes are necessary before we can replace the 
 built-in AAs with a completely library-based solution?


 Two problems. One is straightforward, but requires some language 
 updates: Having some mechanism to tell `opIndex` that the result 
 should be inserted if it doesn't exist. Used in cases like:
 ```d
 int[int][int] aa;

 ++aa[42][24]; // currently compiles
 ```


 
 Yeah, the so-called 
 [autovivification](https://en.wikipedia.org/wiki/Autovivification). For 
 a moment I was confused as to what's the problem with simply:
 1. Having `opIndex` return by `ref`, so you can mutate the value in-place.
 2. Having `opIndex` construct the value on first access, something along 
 the lines of:
    `if (key !in impl) return impl.at(key).emplace!Value();`
 
 But then I remembered that `aa[missingKey]` actually throws a 
 `RangeError` and we wouldn't be able to replicate the behaviour if we 
 did 2.
 
 I wonder if the compiler simply could lower:
 
 ```d
 int[int][int] aa;
 ++aa[42][24];
 ```
 
 to:
 
 ```d
 int[int][int] aa;
 ++aa.require(42).require(24);
 ```
 
 Would that address this issue?


What is needed is to have the compiler try a different hook when this is 
expected. This is actually exactly what the compiler does for AAs -- a 
different runtime call is used when the result of the opIndex is 
expected to be filled in if absent.

I would not use `require`, but rather, another opIndex flavor. This 
shouldn't be tuned specifically to AA, we should do it right for all 
custom aggregates.



 

 The second is not as straightforward. There are a few implicit casts 
 that have no analog for structs. Without some way to say e.g. 
 `Foo!(T)` can be casted implicitly to `Foo!(const(T))`, I don't know 
 how we can figure that one.


 
 Thanks for reminding me, I think this was discussed many times in the 
 past, IIRC it was most recently brought up by deadalnix in the sumtypes 
 thread.
 
 If we consider this issue unsolvable for the time being, a pragmatic way 
 forward would be to consider the following:
 The compiler already special cases `T[]` and `V[K]` types with regards 
 to type qualifiers. Perhaps we can keep this code around and update it 
 to work with the new AA implementation. That would mean that the 
 implicit type qualifier conversion won't work if one directly imports 
 the AA type: `core.associative_array.Hash!(K, V)`, however existing code 
 that uses the `V[K]` syntax would continue working fine.


No, if we fix it, we should fix it once and for all. I don't think 
making AA into a library type is the only goal. The more D can allow 
people to write library types that mimic the builtins, the better off we 
will be.


 

 Aside from that, there are a lot of things that can be done to newaa 
 that will bring it closer to usable. I just haven't done them yet.

 The biggest use case right now is building an AA at compile time, and 
 using it at runtime.


 
 Nevertheless, I think you have very solid progress so far! Hopefully, it 
 won't be too long before we incorporate it in some form in Druntime!


Thanks! I think it's all pretty straightforward stuff -- implementing 
all the various AA things (like `require`, etc.), and being able to 
print the thing properly.

-Steve
May 08 2023
prev sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Mon, May 08, 2023 at 10:12:53PM +0000, Petar via Digitalmars-d wrote:
[...]

 Two problems. One is straightforward, but requires some language
 updates: Having some mechanism to tell `opIndex` that the result
 should be inserted if it doesn't exist. Used in cases like:
 ```d
 int[int][int] aa;
 
 ++aa[42][24]; // currently compiles
 ```


 
 On Monday, 8 May 2023 at 16:08:32 UTC, Steven Schveighoffer wrote:
 Yeah, the so-called
 [autovivification](https://en.wikipedia.org/wiki/Autovivification).
 For a moment I was confused as to what's the problem with simply:
 1. Having `opIndex` return by `ref`, so you can mutate the value
 in-place.
 2. Having `opIndex` construct the value on first access, something
 along the lines of:
   `if (key !in impl) return impl.at(key).emplace!Value();`
 
 But then I remembered that `aa[missingKey]` actually throws a
 `RangeError` and we wouldn't be able to replicate the behaviour if we
 did 2.


Sigh: https://issues.dlang.org/show_bug.cgi?id=7753


T

-- 
Democracy: The triumph of popularity over principle. -- C.Bond
May 08 2023