www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - SAOC LLDB D integration: 5th Weekly Update

reply =?ISO-8859-1?Q?Lu=EDs?= Ferreira <contact lsferreira.net> writes:
Hi D community!

I'm here again, to describe what I've done during the fifth week of
Symmetry
Autumn of Code.



During the discussion on some patches, two other patches have been
raised,
refactoring the usage of string utilities shared among other
demanglers.

- https://reviews.llvm.org/D111948
- https://reviews.llvm.org/D111947



This week I started working on the second milestone, mostly covering
DWARF
issues that I discovered. The initial idea is to research a bit about
what
debug information is provided by each compiler and understand what can
be done,
not only on the LLDB side but also improve information generated by the
compilers.

During this week I also took a look at the DWARF5 standard to
understand what
to improve. I discovered very interesting tags that can help generalize
the
arrays and strings interaction with the debug info.

To give a bit of context, and more deeply into the DWARF standard,
`DW_TAG_array_type` exists to represent array types. Similarly, there
is also a
tag to specifically represent string types. These tags can be used with
`DW_TAG_subrange_type` to represent a so-called "slice" and attributes
like
`DW_AT_count` or `DW_AT_upper_bound` can be used to represent the slice
length.
Interestingly those attributes can refer to runtime variables and
therefore
create debug information for a "dynamic array". I checked other
compilers and
only C VLAs are using it.

I wrote a document describing some of the issues and comparing the
DWARF
capabilities of the 3 most known D compilers. You can read it
[here](https://gist.github.com/ljmf00/0194d03f6f30303feec9f58eb9f0c7df)
.

Some of those issues I managed to fix in the compiler backend, others I
filed a
bug. Here is a list of patches and bug reports I created:

- https://github.com/dlang/dmd/pull/13196
- https://issues.dlang.org/show_bug.cgi?id=3D22423
- https://github.com/dlang/dmd/pull/13200
- https://issues.dlang.org/show_bug.cgi?id=3D22426
- https://github.com/dlang/dmd/pull/13202
- https://github.com/dlang/dmd/pull/13204
- https://bugs.llvm.org/show_bug.cgi?id=3D52223

I have an upcoming patch to the DMD backend that I'm currently working
on to
refactor some of the DMD DWARF generator, specifically to the
abbreviation
code. My motivation for the refactor is the fact that I spent a lot of
time

realized,
when looking into the DWARF standard, that DWARF abbreviations are all
LEB128
encoded and the code is not respecting that. I believe some of the
errors
produced by the compiler is due to that.



To have pretty-printing capabilities I will need to either implement
some of
those missing DWARF tags or/and write an LLDB language plugin. The last
bug
report on the list above triggers a discussion in that regard. Very
soon I'm
going to write a thread on the LLVM mailing list describing the goals
of the
plugin and discuss the possibility of upstreaming it, however, I'm not
too
confident that will happen, because Rust among other language plugins
being
removed from the upstream for "lacking maintainability".

I already started working on some D integration specifically on the
LLDB side,
although I haven't pushed any of this code, since some stuff is still
janky
experiments and not properly split up.



Next week, now that I have some scope about DWARF, I'm going to tackle
majorly
on the LLDB integration, since that is what the milestone is focused
on. I'm
going to clean up the idea I have of the language plugin and hopefully
show
some interesting practical results.

You can also read this on my blog,
[here](https://lsferreira.net/posts/d-saoc-2021-05).

--=20
Sincerely,
Lu=C3=ADs Ferreira   lsferreira.net
Oct 20 2021
parent reply WebFreak001 <d.forum webfreak.org> writes:
On Thursday, 21 October 2021 at 01:48:32 UTC, Luís Ferreira wrote:

 Hi D community!

 [...]


I love it! Are you planning to do more D related stuff after SAoC 
too?

When I made my LLDB pretty printer plugin I implemented the array 
and string types, which will probably become obsolete once your 
suggested DWARF changes are in the compilers. Additionally if you 
are planning to support D as an LLDB plugin, I think the Phobos 
types would be a great addition in the future (JSONValue, 
Variant, sumtype, DateTime, SysTime, Duration, Appender...) and 
especially if it's somehow possible to detect ranges and evaluate 
them.
Oct 21 2021
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Thursday, 21 October 2021 at 08:46:12 UTC, WebFreak001 wrote:

 On Thursday, 21 October 2021 at 01:48:32 UTC, Luís Ferreira 
 wrote:

 Hi D community!

 [...]


 I love it! Are you planning to do more D related stuff after 
 SAoC too?

 When I made my LLDB pretty printer plugin I implemented the 
 array and string types, which will probably become obsolete 
 once your suggested DWARF changes are in the compilers. 
 Additionally if you are planning to support D as an LLDB 
 plugin, I think the Phobos types would be a great addition in 
 the future (JSONValue, Variant, sumtype, DateTime, SysTime, 
 Duration, Appender...) and especially if it's somehow possible 
 to detect ranges and evaluate them.


What are the planned DWARF changes?
Oct 21 2021
next sibling parent =?ISO-8859-1?Q?Lu=EDs?= Ferreira <contact lsferreira.net> writes:
On Thu, 2021-10-21 at 20:50 +0000, deadalnix via Digitalmars-d wrote:

 On Thursday, 21 October 2021 at 08:46:12 UTC, WebFreak001 wrote:

 On Thursday, 21 October 2021 at 01:48:32 UTC, Lu=C3=ADs Ferreira=20
 wrote:

 Hi D community!
=20
 [...]


=20
 I love it! Are you planning to do more D related stuff after=20
 SAoC too?
=20
 When I made my LLDB pretty printer plugin I implemented the=20
 array and string types, which will probably become obsolete=20
 once your suggested DWARF changes are in the compilers.=20
 Additionally if you are planning to support D as an LLDB=20
 plugin, I think the Phobos types would be a great addition in=20
 the future (JSONValue, Variant, sumtype, DateTime, SysTime,=20
 Duration, Appender...) and especially if it's somehow possible=20
 to detect ranges and evaluate them.


=20
 What are the planned DWARF changes?


You can briefly check them on the document I mentioned. Most of the
things I'm going to try to implement on the DMD, which is things
implemented by either GDC or LDC.

The other things are just possibilities. We need to study them to see
what is the best fit. I'm specifically talking about the arrays and
strings. DWARF has a type for arrays and strings. I describe that on
the post as well.

--=20
Sincerely,
Lu=C3=ADs Ferreira   lsferreira.net
Oct 21 2021
prev sibling parent WebFreak001 <d.forum webfreak.org> writes:
On Thursday, 21 October 2021 at 20:50:22 UTC, deadalnix wrote:

 On Thursday, 21 October 2021 at 08:46:12 UTC, WebFreak001 wrote:

 On Thursday, 21 October 2021 at 01:48:32 UTC, Luís Ferreira 
 wrote:

 Hi D community!

 [...]


 I love it! Are you planning to do more D related stuff after 
 SAoC too?

 When I made my LLDB pretty printer plugin I implemented the 
 array and string types, which will probably become obsolete 
 once your suggested DWARF changes are in the compilers. 
 Additionally if you are planning to support D as an LLDB 
 plugin, I think the Phobos types would be a great addition in 
 the future (JSONValue, Variant, sumtype, DateTime, SysTime, 
 Duration, Appender...) and especially if it's somehow possible 
 to detect ranges and evaluate them.


 What are the planned DWARF changes?


I was referring to the linked github gist: 
https://gist.github.com/ljmf00/0194d03f6f30303feec9f58eb9f0c7df
Oct 21 2021
prev sibling parent =?ISO-8859-1?Q?Lu=EDs?= Ferreira <contact lsferreira.net> writes:
On Thu, 2021-10-21 at 08:46 +0000, WebFreak001 via Digitalmars-d wrote:

 On Thursday, 21 October 2021 at 01:48:32 UTC, Lu=C3=ADs Ferreira wrote:

 Hi D community!
=20
 [...]


=20
 I love it! Are you planning to do more D related stuff after SAoC=20
 too?
=20


Yes. I have some ideas to improve D ecosystem and also some proposals
in mind for D itself.

- One of the things I'm writing is a DIP to be able to ignore coverage.
The idea is to be able to mark code as not intended to be covered, like
dummy statements. I'm not sure if it is needed a forma DIP, but I'm
doing it anyway to have some rationale.
- Related to coverage I recently created a library/tool to convert
various coverage formats, ready to adapt on a coverage service. I want
to expand it for other well known formats and implement a C API.
- Have a stalled DIP implementation on the compiler about throw STC.
- I'm restructuring the runtime demangler to be faster and exception-
free.
- Have plans to work on a 'silly' package based test runner to support
BetterC and fancy colored diffs with green and red color diffs when
comparing actual and expected sides and syntatic sugar assert
variations for that purpose.
- Have a stale implementation of parser combinators library CTFE ready
to parse protobufs and generically other things more easily.
- I would like to, at some point, contribute to libdparse and dscanner
to implement more code smell and add the possibility of ignoring false
positives or "trust" certain code.

I have planty of other projects that I want to contribute/do but either
my time is limited by other priorities or sometimes not motivated to do
some of them. Normally I do all of them at the same time, and therefore
suffer from the problem of implementing none.

I have my TODO list open-source, so you can check it out:
https://tasks.lsferreira.net/ . I also plan to dramatically improve the
interface of it, and probably D will be involved on that :P


 When I made my LLDB pretty printer plugin I implemented the array=20
 and string types, which will probably become obsolete once your=20
 suggested DWARF changes are in the compilers. Additionally if you=20
 are planning to support D as an LLDB plugin, I think the Phobos=20
 types would be a great addition in the future (JSONValue,=20
 Variant, sumtype, DateTime, SysTime, Duration, Appender...) and=20
 especially if it's somehow possible to detect ranges and evaluate=20
 them.


About that, I think implementing support for array and strings is
straightforward and acceptable, because the memory layout of them are
specified on the D ABI spec. Things like associative arrays and other
data structures are not, and hardcoding the fields is ok but at some
point in time it can break, unfortunetly. I think if we plan to
upstream the plugin too (which I think will not going to happen,
because Rust created a fork due to being denied by the upstream) we
should not include those "ambiguous" additions, although, would be cool
to find another way to export information for the debuggers. If this
end up not being upstreamed, I think we can think about syncing that
with the reference compiler/runtime/stdlib implementation and possibly
support other major implementations. Another possibility is to
standardize the memory layout of those data structures, but I think
that is difficult and restrictive.

But yeah, for now, I'm going to focus on the "achievable" things and
then improve it when I have some "stable" implementation.

--=20
Sincerely,
Lu=C3=ADs Ferreira   lsferreira.net
Oct 21 2021