www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Do you want to help me fix static AA initialization?

reply Steven Schveighoffer <schveiguy gmail.com> writes:
Hi everyone,

I have a fix for static associative array initialization. Basically, I 
have a project called `newaa` which is a library-based copy of D's 
associative arrays. The url is here: https://github.com/schveiguy/newaa

This allows static initialization of AAs, e.g.:

```d

Hash!(string, string) ctaa = [
    "name" : "Steve",
    "username" : "schveiguy"
];

void main()
{
    string[string] aa = ctaa.asAA;
}

```

If you look at the project, you can see that `asAA` actually is just a 
reinterpret cast of the internals, since the layout is exactly the same. 
The actual AA is built at compile time, and just used at runtime, 
without any issues (and yes, the mutability is fine).

I want to make this part of the language/library, such that you can just 
straight-up initialize AA's at compile time and use them at runtime.

However, I need the compiler's help. Why? Because CTFE does not let you 
retinterpret cast, plus it doesn't have a distinct type for "runtime AA" 
vs. "compile time AA" (these are distinct types). So I need some kind of 
hook to cross the barrier, where the compiler ignores the return type 
not being an AA and just bludgeons it into an AA location. I also need 
it to call this hook *only* when going to convert the AA from CT to RT, 
including if it is a member in some CT structure.

Does anyone have the skills to help me? Hit me here, or on discord, or 
send me an email: schveiguy gmail.com

BTW, this fix in principle is tentatively approved by Walter for 
inclusion (assuming everything we thought of is kosher with the 
language). As long as it doesn't break any existing code, and just 
allows currently non-working code to just work.

-Steve
Jun 29 2023
next sibling parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 29 June 2023 at 19:48:40 UTC, Steven Schveighoffer 
wrote:

 Hi everyone,

 I have a fix for static associative array initialization. 
 Basically, I have a project called `newaa` which is a 
 library-based copy of D's associative arrays. The url is here: 
 https://github.com/schveiguy/newaa

 [...]


I have copied the druntime implementation of AAs into the dmd 
compiler and allowed it to build static array literals.
It needs some work to make it really generic but it would be 
something to look at:
https://github.com/dlang/dmd/compare/master...UplinkCoder:dmd:aaConstantLiteral
Jul 03 2023
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/3/23 10:18 AM, Stefan Koch wrote:

 On Thursday, 29 June 2023 at 19:48:40 UTC, Steven Schveighoffer wrote:

 Hi everyone,

 I have a fix for static associative array initialization. Basically, I 
 have a project called `newaa` which is a library-based copy of D's 
 associative arrays. The url is here: https://github.com/schveiguy/newaa

 [...]


 
 I have copied the druntime implementation of AAs into the dmd compiler 
 and allowed it to build static array literals.
 It needs some work to make it really generic but it would be something 
 to look at:
 https://github.com/dlang/dmd/compare/master...UplinkCoder:dmd:aaConstantLiteral


Thanks for the link, but I think this is the wrong approach. The 
compiler shouldn't have a separate identical implementation internally, 
that means we have to maintain 2 identical implementations, and more 
unspoken dependencies from compiler to library (someone could make a 
minor change in the library version and forget to update the compiler, 
resulting in odd behaviors or crashes). What we need is a library 
implementation that can do this work during CTFE.

However, I appreciate the link, because maybe I can actually figure out 
where to properly add the right calls.

-Steve
Jul 03 2023
prev sibling next sibling parent reply Chris Piker <chris hoopjump.com> writes:
On Thursday, 29 June 2023 at 19:48:40 UTC, Steven Schveighoffer 
wrote:

 I have a fix for static associative array initialization.


Cool!


 This allows static initialization of AAs, e.g.:

 ```d

 Hash!(string, string) ctaa = [
    "name" : "Steve",
    "username" : "schveiguy"
 ];

 void main()
 {
    string[string] aa = ctaa.asAA;
 }

 ```


Dumb user question:  Presuming this is fixed would it allow code 
that looked like this:
```d
import std.stdio: writeln;

auto aa = ["one":1, "two":2];

void main(){
     writeln(aa["one"]);
}
```
to work, or would the end result require `Hash!(string, string)` 
as the type definition?
Jul 20 2023
parent reply Quirin Schroll <qs.il.paperinik gmail.com> writes:
On Friday, 21 July 2023 at 06:24:09 UTC, Chris Piker wrote:

 Dumb user question:  Presuming this is fixed would it allow 
 code that looked like this: `auto aa = ["one":1, "two":2];`
 to work, or would the end result require `Hash!(string, 
 string)` as the type definition?


As far as I understand, it would be the former. It’s only not in 
Steven’s examples because his `Hash` template is not in the 
runtime and of course not special-cased by the compiler. He asks 
for help to get exactly there.
Jul 21 2023
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/21/23 4:55 AM, Quirin Schroll wrote:

 On Friday, 21 July 2023 at 06:24:09 UTC, Chris Piker wrote:

 Dumb user question:  Presuming this is fixed would it allow code that 
 looked like this: `auto aa = ["one":1, "two":2];`
 to work, or would the end result require `Hash!(string, string)` as 
 the type definition?


 
 As far as I understand, it would be the former. It’s only not in 
 Steven’s examples because his `Hash` template is not in the runtime and 
 of course not special-cased by the compiler. He asks for help to get 
 exactly there.
 


Yes, you are correct.

The goal is not to have a `Hash` type, but to have a mechanism for the 
compiler to convert a CTFE AA to a runtime AA. The Hash type is just a 
demonstration that it can be done.

BTW, fun stuff, I've learned that the CTFE AA is actually implemented 
using 2 CTFE arrays. So maybe... it might be also better to just replace 
it with a CTFE-compatible library type. But there are other tricks that 
need to be figured out (AAs have some magic properties).

-Steve
Jul 22 2023
prev sibling parent reply RazvanN <razvan.nitu1305 gmail.com> writes:
On Thursday, 29 June 2023 at 19:48:40 UTC, Steven Schveighoffer 
wrote:

 Hi everyone,

 I have a fix for static associative array initialization. 
 Basically, I have a project called `newaa` which is a 
 library-based copy of D's associative arrays. The url is here: 
 https://github.com/schveiguy/newaa

 This allows static initialization of AAs, e.g.:

 ```d

 Hash!(string, string) ctaa = [
    "name" : "Steve",
    "username" : "schveiguy"
 ];

 void main()
 {
    string[string] aa = ctaa.asAA;
 }

 ```

 If you look at the project, you can see that `asAA` actually is 
 just a reinterpret cast of the internals, since the layout is 
 exactly the same. The actual AA is built at compile time, and 
 just used at runtime, without any issues (and yes, the 
 mutability is fine).

 I want to make this part of the language/library, such that you 
 can just straight-up initialize AA's at compile time and use 
 them at runtime.

 However, I need the compiler's help. Why? Because CTFE does not 
 let you retinterpret cast, plus it doesn't have a distinct type 
 for "runtime AA" vs. "compile time AA" (these are distinct 
 types). So I need some kind of hook to cross the barrier, where 
 the compiler ignores the return type not being an AA and just 
 bludgeons it into an AA location. I also need it to call this 
 hook *only* when going to convert the AA from CT to RT, 
 including if it is a member in some CT structure.

 Does anyone have the skills to help me? Hit me here, or on 
 discord, or send me an email: schveiguy gmail.com

 BTW, this fix in principle is tentatively approved by Walter 
 for inclusion (assuming everything we thought of is kosher with 
 the language). As long as it doesn't break any existing code, 
 and just allows currently non-working code to just work.

 -Steve


Maybe this has been addressed in previous discussions, but why 
not rewrite:

```d
int[string] b = ["mimi" : 2];
```

to

```d
int[string] b;

     static this()
     {
         b = ["mimi" : 2];
     }
     ```

I know that we have problems with cycle detection in module 
constructors, but assuming this will be fixed by Adam's proposal, 
this seems like a lot easier to implement and does not stress 
compilation time.

RazvanN
Jul 25 2023
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/25/23 4:45 AM, RazvanN wrote:

 
 Maybe this has been addressed in previous discussions, but why not rewrite:
 
 ```d
 int[string] b = ["mimi" : 2];
 ```
 
 to
 
 ```d
 int[string] b;
 
      static this()
      {
          b = ["mimi" : 2];
      }
      ```
 
 I know that we have problems with cycle detection in module 
 constructors, but assuming this will be fixed by Adam's proposal, this 
 seems like a lot easier to implement and does not stress compilation time.
 


Yes, this can be done. We can even mark the static ctor as standalone, 
so it doesn't participate in cycles.

And then in the future if we can find a way to do it during CTFE, then 
it can be seamlessly changed.

There are some hairy aspects -- like what you do if the initialized 
value is done in a function call or the AA is a member of a type instead 
of the thing being initialized, but those are surmountable.

Great idea!

-Steve
Jul 25 2023
parent RazvanN <razvan.nitu1305 gmail.com> writes:
On Tuesday, 25 July 2023 at 13:24:32 UTC, Steven Schveighoffer 
wrote:

 On 7/25/23 4:45 AM, RazvanN wrote:

 
 Maybe this has been addressed in previous discussions, but why 
 not rewrite:
 
 ```d
 int[string] b = ["mimi" : 2];
 ```
 
 to
 
 ```d
 int[string] b;
 
      static this()
      {
          b = ["mimi" : 2];
      }
      ```
 
 I know that we have problems with cycle detection in module 
 constructors, but assuming this will be fixed by Adam's 
 proposal, this seems like a lot easier to implement and does 
 not stress compilation time.
 


 Yes, this can be done. We can even mark the static ctor as 
 standalone, so it doesn't participate in cycles.

 And then in the future if we can find a way to do it during 
 CTFE, then it can be seamlessly changed.

 There are some hairy aspects -- like what you do if the 
 initialized value is done in a function call or the AA is a 
 member of a type instead of the thing being initialized, but 
 those are surmountable.

 Great idea!

 -Steve


There are some issues with this proposal though:

- accepting the static initializer leaves the impression that the 
initializer code is CTFEable when actually it could be not.
- immutable AAs should be usable at compile time, whereas with 
this proposal they would not be because they are initialized at 
runtime (we could just say you can slap `enum` instead of 
`immutable` to make it work, but that's not how language design 
should be fone.)
- there is no way that I know where static variables inside 
function scopes can be accessed from the outside.

Overall, this starts to look like a hack.
Jul 27 2023