www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Compiler-assisted code reflection for IDEs

reply SealabJaster <sealabjaster gmail.com> writes:
A common flaw with things like code-d and (in my limited 
experience) Visual-D, these plugins are unable to resolve even 
simple things like:

```d
struct S(T)
{
     T a;
}

...

S!int.init.// No autocompletion from this point.
```

I was wondering if functionality can be added onto the compiler 
(DMD at least, due to its speed) which allows for external 
tooling to gain accurate information about the code?

Take the above struct for example, maybe when the plugin detects 
the following:

```d
S!int myS;
```

It could make a call to the compiler like: `dmd myapp.d -reflect 
"myapp.S!(int)"`, where the compiler would return all the info it 
can (It's a struct, it has a member of int called a, etc.)

It's just a rough idea, but I hope you can see what I'm trying to 
suggest, and why I feel it's at least worth asking about.

In other words, build some much needed language service 
functionality into the compiler, to allow the actual language 
servers to provide a much higher quality experience. Naively 
assuming it's as simple as that :3
Jul 16 2021
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Friday, 16 July 2021 at 10:02:06 UTC, SealabJaster wrote:

 I was wondering if functionality can be added onto the compiler 
 (DMD at least, due to its speed) which allows for external 
 tooling to gain accurate information about the code?

 Take the above struct for example, maybe when the plugin 
 detects the following:

 ```d
 S!int myS;
 ```

 It could make a call to the compiler like: `dmd myapp.d 
 -reflect "myapp.S!(int)"`, where the compiler would return all 
 the info it can (It's a struct, it has a member of int called 
 a, etc.)

 It's just a rough idea, but I hope you can see what I'm trying 
 to suggest, and why I feel it's at least worth asking about.

 In other words, build some much needed language service 
 functionality into the compiler, to allow the actual language 
 servers to provide a much higher quality experience. Naively 
 assuming it's as simple as that :3


That's absolutely possible. DMD is available as a library. You 
basically take the source code feed it to the engine, apply 
whatever compiler passes you want, and accept the results using a 
visitor of your choice. There are already visitors available; I'd 
suggest starting with `SemanticTimeTransitiveVisitor` because 
it's a transitive visitor (less boiler-plate) that should be 
applied after all the semantic phases has been applied.

I suggest you first get in contact with the developers of 
Visual-D (Rainer Schutze) code-d, DCD (D completion daemon) and 
others to see what's already been done. They likely have similar 
visions.

Good luck!
Jul 16 2021
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 17/07/2021 2:38 AM, Per Nordlöw wrote:

 I suggest you first get in contact with the developers of Visual-D 
 (Rainer Schutze) code-d, DCD (D completion daemon) and others to see 
 what's already been done. They likely have similar visions.


VisualD is the only one of importance in terms of discussion as it is 
the only one that uses dmd-fe for completion.

Everything else that is active pretty much only uses DCD and we have 
known for ages it needs a rewrite to use dmd-fe. But once DCD is 
rewritten, all existing IDE's that support it will just work in terms of 
switching over.
Jul 16 2021
prev sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Friday, 16 July 2021 at 14:38:07 UTC, Per Nordlöw wrote:

 ...


I wasn't quite suggesting doing it myself. While I could *try* I 
know I wouldn't be able to get anywhere, at least for quite a 
long time.

I was more seeking discussion on the possibility and silently 
hoping someone with more experience would take up the mantle >:D


 But once DCD is rewritten, all existing IDE's that support it 
 will just work in > terms of switching over.


So an ETA of roughly 5 years minimum then. Doesn't dmd's frontend 
have the issue of poorly managing its memory (issue for longer 
running tools), or is that mostly fixed now?
Jul 16 2021
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jul 16, 2021 at 10:11:37PM +0000, SealabJaster via Digitalmars-d wrote:
[...]

 So an ETA of roughly 5 years minimum then. Doesn't dmd's frontend have
 the issue of poorly managing its memory (issue for longer running
 tools), or is that mostly fixed now?


There's an option to turn on the GC for DMDFE.  But the fact remains
that the code was written with single execution in mind, meaning that
once you call it to compile a code snippet, you cannot call it again
without running into problems. There's global state and stuff that
doesn't get cleaned up (this is part of why dmd is so fast: it doesn't
bother cleaning up anything, just does its job ASAP then lets the OS
clean up afterwards).

So you have to reboot the DMDFE component for each compilation request
anyway, meaning memory management is mostly moot (just load it up as a
subprocess, do its thing, then let the OS reclaim the subprocess
memory).  It was never meant to be a long-running process.

The code may have been updated more recently to make it more friendly
for reuse, but AFAIK the bulk of the code still assumes single
invocation only.


T

-- 
Today's society is one of specialization: as you grow, you learn more and more
about less and less. Eventually, you know everything about nothing.
Jul 16 2021
prev sibling parent reply SealabJaster <sealabjaster gmail.com> writes:
On Friday, 16 July 2021 at 22:11:37 UTC, SealabJaster wrote:


Spent a good half a day learning and testing out dmd-fe.

T_T My head hurts slightly. I feel I'm definitely not the right 
person to try and do this.



 Jul 16 2021



  parent reply russhy <russhy gmail.com>  writes:


That's one of the issue with IDEs in D, everything works nice and 
great, including debugging, but when you play with generics, then 
nothing works (code completion)

Generics/Templates is one of D's strength, lot to gain from 
making sure that works too, setting up a work group to tackle it 
would a great investment imo

But it's hard to find people with expertise in the subject, and 
it is harder to motivate people to do the work, because it's not 
an easy job and requires allocating significant amount of free 
time



 Jul 18 2021



 next sibling parent  SealabJaster <sealabjaster gmail.com>  writes:


On Sunday, 18 July 2021 at 22:14:50 UTC, russhy wrote:

 But it's hard to find people with expertise in the subject, and 
 it is harder to motivate people to do the work, because it's 
 not an easy job and requires allocating significant amount of 
 free time


I could only wish to be even a fraction as educated and 
knowledgeable as a lot of the people on this forums.

Sadly though, that'd likely mean I'd no longer have the free time 
to do anything!
... except I don't have the expertise to really do anything 
either :D!



 Jul 20 2021



prev sibling  parent  JN <666total wp.pl>  writes:


On Sunday, 18 July 2021 at 22:14:50 UTC, russhy wrote:

 But it's hard to find people with expertise in the subject, and 
 it is harder to motivate people to do the work, because it's 
 not an easy job and requires allocating significant amount of 
 free time


I think expertise might not be enough. C++ has been trying to 
solve this problem for decades with much more manpower thrown at 
it and IDEs still mostly give up when encountering templates.



 Jul 22 2021