www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Self-referential struct mixins

reply rbscott <tjonsies818 yahoo.com> writes:
Hello,

I recently started using D, and I am exploring some of the 
functionality in CTFE. I ran into a case that I can't get past. 
The goal is to take a JSON API Schema and convert it into D types 
using CTFE. The problem is, the types can be self-referential.

A simplified form of the language would look like:

{ "A" : { "type": struct, "fields" : { "b": { "type": "string" }, 
"a" : { "type": "A" }}}

This would yield a type "A" that has two fields, "a" and "b". 
Where "a" is a string and "b" is of type "A". Or in D:

struct A {
   string b;
   A* a;
}

This is a simplified example, but imagine this is highly nested 
to generate arbitrarily complex types that could reference on one 
another.

I got this working in the simple case if none of the fields are 
self-referential, but it breaks as soon as there is a loop.

Here is a gist which demonstrates the issue: 
https://gist.github.com/rbscott/ee0f3ba94296f9c8224a8c4c13c2f026.

I think I can workaround this by with string mixins everywhere, 
but the structure generation logic is complex enough that I would 
prefer not to do that. This approach could be totally off, so any 
suggestions would be greatly appreciated!

thanks,
rbscott
Nov 16 2019
parent reply Claude <claudemr live.fr> writes:
On Sunday, 17 November 2019 at 07:13:52 UTC, rbscott wrote:

 Hello,

 I recently started using D, and I am exploring some of the 
 functionality in CTFE. I ran into a case that I can't get past. 
 The goal is to take a JSON API Schema and convert it into D 
 types using CTFE. The problem is, the types can be 
 self-referential.

 A simplified form of the language would look like:

 { "A" : { "type": struct, "fields" : { "b": { "type": "string" 
 }, "a" : { "type": "A" }}}

 This would yield a type "A" that has two fields, "a" and "b". 
 Where "a" is a string and "b" is of type "A". Or in D:

 struct A {
   string b;
   A* a;
 }

 This is a simplified example, but imagine this is highly nested 
 to generate arbitrarily complex types that could reference on 
 one another.

 I got this working in the simple case if none of the fields are 
 self-referential, but it breaks as soon as there is a loop.

 Here is a gist which demonstrates the issue: 
 https://gist.github.com/rbscott/ee0f3ba94296f9c8224a8c4c13c2f026.

 I think I can workaround this by with string mixins everywhere, 
 but the structure generation logic is complex enough that I 
 would prefer not to do that. This approach could be totally 
 off, so any suggestions would be greatly appreciated!

 thanks,
 rbscott


But "a" is not strictly of type A, but rather A*, which makes a 
big difference.

Anyway, upon the mixin template instantiation, you can try to 
'declare' all your struct's on top and then properly 'define' 
them afterwards:

struct A;
struct B;

struct A {
   B* b;
}

struct B {
   A* a;
}


Have you tried that?
Nov 17 2019
parent reply Elie Morisse <syniurge gmail.com> writes:
On Sunday, 17 November 2019 at 07:13:52 UTC, rbscott wrote:

 I got this working in the simple case if none of the fields are 
 self-referential, but it breaks as soon as there is a loop.

 Here is a gist which demonstrates the issue: 
 https://gist.github.com/rbscott/ee0f3ba94296f9c8224a8c4c13c2f026.


This is a DMD bug, you should open an issue.

It's part of a bigger family of issues caused by how DMD's 
semantic analysis is architectured. I tried to tackle it some 
time ago but didn't see it through the end :

https://github.com/dlang/dmd/pull/7018
https://forum.dlang.org/post/auvbfyzmqjxtfhkvmkxu forum.dlang.org

As I see it the proper fix requires rewritting a large part of 
the compiler frontend, but this is definitely something worth 
doing at some point to make DMD less brittle esp. regarding 
advanced usage of mixins/CTFE (I don't have time anymore, and 
IMHO someone should be paid 2 months to work full-time on 
re-architecturing DMD for this)

On Sunday, 17 November 2019 at 09:16:19 UTC, Claude wrote:

 Anyway, upon the mixin template instantiation, you can try to 
 'declare' all your struct's on top and then properly 'define' 
 them afterwards:

 struct A;
 struct B;

 struct A {
   B* b;
 }

 struct B {
   A* a;
 }


 Have you tried that?


Forward declaring structs isn't allowed in D.
Nov 17 2019
parent reply rbscott <tjonsies818 yahoo.com> writes:
On Sunday, 17 November 2019 at 17:35:07 UTC, Elie Morisse wrote:

 It's part of a bigger family of issues caused by how DMD's 
 semantic analysis is architectured. I tried to tackle it some 
 time ago but didn't see it through the end :

 https://github.com/dlang/dmd/pull/7018
 https://forum.dlang.org/post/auvbfyzmqjxtfhkvmkxu forum.dlang.org

 As I see it the proper fix requires rewritting a large part of 
 the compiler frontend, but this is definitely something worth 
 doing at some point to make DMD less brittle esp. regarding 
 advanced usage of mixins/CTFE (I don't have time anymore, and 
 IMHO someone should be paid 2 months to work full-time on 
 re-architecturing DMD for this)


Thanks for pointing that out. Given what you understand about the 
issue, do you have any recommendations for working around this? I 
am pretty sure a string mixin will work, but hopefully there is 
an alternative. Maybe a different compiler will work?


 Forward declaring structs isn't allowed in D.


I did try forward declarations. I thought because of "Opaque 
Pointers", this would work, but it generates a compiler error.
Nov 17 2019
parent Elie Morisse <syniurge gmail.com> writes:
On Sunday, 17 November 2019 at 19:32:58 UTC, rbscott wrote:

 Thanks for pointing that out. Given what you understand about 
 the issue, do you have any recommendations for working around 
 this? I am pretty sure a string mixin will work, but hopefully 
 there is an alternative. Maybe a different compiler will work?


DMD, GDC, LDC all share the same frontend so you'd get the same 
errors.

As a workaround a single string mixin to generate the full body 
of Structures would work yes. I can't think of other 
alternatives..
Nov 18 2019