www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Debugging mixins - we need to do something

reply Manu <turkeyman gmail.com> writes:
TL;DR, debugging is critical and neglected. Mixin is worst offender.

So, string mixin breaks the debugger. All of a sudden there is code
that's executing that doesn't exist anywhere. We need a solution to
this problem...

As I and others have suggested before; mixin expansion should emit a
`[sourcefile].d.mixin` file to the object directory, this file should
accumulate mixin instantiations, and perhaps it would be ideal to also
emit surrounding code for context.

The debuginfo should point into that file when compiling mixin code,
so that it's possible to step and inspect within the mixin.

I initially imagined an 'expanded' copy of the file would be cool, but
that doesn't work when mixins are driven by template args, since the
expansion may be different for every instantiation of the surrounding
code.

There has been lots of chatter in the past, can we prioritise it?
It would help me a lot.


Some background on this issue: (TL;DR, don't read)

D's biggest draw for me is it's powerful meta, and ability to
obliterate a huge quantity of typical C++ boilerplate. As C++
continues to close the gap, this becomes more and more the defining
feature, and probably the primary basis for an argument that you might
make to switch a C++ workflow to D. (closely followed by modules and
build-times)

Most uninterested and/or unopinionated people I've interacted with
recently have agreed that D feels 'pretty good', and they'd be willing
to explore it further if there was a compelling argument.

The focus of resistance in my recent experience has been related 90%
to tooling. In particular, IDE integration, and debug experience. (the
other 10% being implementation issues in extern(C++), which I've been
working to improve)

From my own experience over the last decade, there is a critical
tension between this core value proposition; which is meta and boilerplate-reduction vs debugging experience, which is also critical (and neglected). I think the state of this tension is the most likely point where a case for D will fail with my particular set of colleagues. While I might balance that boilerplate reduction and code simplicity is worth the trade, they will argue (and I think they will win) that the fact you practically lose the ability to debug your code when there is meta nearby is very bad indeed. I think this is a high-importance ticket. I have absolutely no idea where to start with this, and I don't have anywhere near enough time to put towards my current initiatives (extern(C++) stuff). Is anyone interested in this issue?
Sep 08 2018
next sibling parent Peter Alexander <peter.alexander.au gmail.com> writes:
On Saturday, 8 September 2018 at 22:01:23 UTC, Manu wrote:
 As I and others have suggested before; mixin expansion should 
 emit a `[sourcefile].d.mixin` file to the object directory, 
 this file should accumulate mixin instantiations, and perhaps 
 it would be ideal to also emit surrounding code for context.
This implies also emitting all template instantiation instances, right? I agree this is an important problem.
Sep 08 2018
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 8 September 2018 at 22:01:23 UTC, Manu wrote:
 TL;DR, debugging is critical and neglected. Mixin is worst 
 offender.

 So, string mixin breaks the debugger. All of a sudden there is 
 code that's executing that doesn't exist anywhere. We need a 
 solution to this problem...

 As I and others have suggested before; mixin expansion should 
 emit a `[sourcefile].d.mixin` file to the object directory, 
 this file should accumulate mixin instantiations, and perhaps 
 it would be ideal to also emit surrounding code for context.

 The debuginfo should point into that file when compiling mixin 
 code, so that it's possible to step and inspect within the 
 mixin.
How is this done in C/C++ w.r.t macros? Point to within the macro?
 I initially imagined an 'expanded' copy of the file would be 
 cool, but that doesn't work when mixins are driven by template 
 args, since the expansion may be different for every 
 instantiation of the surrounding code.

 There has been lots of chatter in the past, can we prioritise 
 it? It would help me a lot.

 I think this is a high-importance ticket. I have absolutely no 
 idea
 where to start with this, and I don't have anywhere near enough 
 time
 to put towards my current initiatives (extern(C++) stuff).

 Is anyone interested in this issue?
Obligatory "Bugzilla issue?". I don't think is should be too difficult to come up with a really dumb solution, just dump CompileStatement's string to a file, set the Lexer's loc to that file and let regular debug info generation do its thing. I'm busy for the next few days, but I'll take a crack at it if no-one beats me to it.
Sep 08 2018
parent reply tide <tide tide.tide> writes:
On Sunday, 9 September 2018 at 00:58:15 UTC, Nicholas Wilson 
wrote:
 Obligatory "Bugzilla issue?".
Obligatory "it already exists and has exited for X amount of years" (4 in this case). https://issues.dlang.org/show_bug.cgi?id=12790 As with most issues in bugzilla, no one has so much as made a comment on it or any other kind of activity. Oh another one from 2008, 10 years ago. https://issues.dlang.org/show_bug.cgi?id=1870 At least this one has someone adding to the issue, is Bill Baxter still around? Then 6 years later, someone says they want the feature and 2 years after that the guy that made the other issue adds a comment. Followed by 4 years of nothing. This just about describes Dlang's bug tracking as accurately as it gets.
Sep 08 2018
parent Neia Neutuladh <neia ikeran.org> writes:
On Sunday, 9 September 2018 at 03:02:41 UTC, tide wrote:
 Oh another one from 2008, 10 years ago.

 https://issues.dlang.org/show_bug.cgi?id=1870
Oh hey, a wild me appears. So, yeah, the tricky bit with this is where to emit the source code. If you just want a best-effort aid, you could specify a file manually to record string mixins in, and then error messages within string mixins would refer to locations within that file. (Which should just be a file number offset. There's already special handling, so that shouldn't be a huge amount of refactoring.) No attempt to append to an existing file; it's going to blow away the existing contents each time. This is a partial solution that gets a fair amount of value, and it's where I'd probably start if I were trying to solve this. But if you want to support this well enough for debug information to point at it, that's harder. Consider: --- module sourcelib; template Mixin(string toMixIn) { mixin(toMixIn); } --- module foo; import sourcelib; Mixin!`static assert(1 == 1);`; --- module bar; import sourcelib; Mixin!`static assert(2 == 2);`; --- Compile this with: dmd -c foo.d dmd -c bar.d Now sourcelib.d-mixin.d contains: static assert(1 == 1); static assert(2 == 2); You recompile foo.d and now it's got: static assert(1 == 1); static assert(2 == 2); static assert(1 == 1); Keep doing incremental builds and the length of this file is only limited by your disk space. Plus your debug info has to point at lines in this file, and the only way to determine that is by reading the entire file, so that's going to slow down your build. Okay, let's include the name of the file you're compiling. Now we have two mixin logs: foo.d-sourcelib.d-mixin.d: static assert(1 == 1); bar.d-sourcelib.d-mixin.d: static assert(2 == 2); Good, yes? Except now we change it so foo imports bar. What do we see now? foo.d-sourcelib.d-mixin.d: static assert(2 == 2); static assert(1 == 1); bar.d-sourcelib.d-mixin.d: static assert(2 == 2); So we end up with very large mixin logs whenever we compile nontrivial projects. Cue the command-line options for ignoring mixins based on the name of the module doing the mixing in. What if there are more layers of indirection? You need to walk up the chain of templates to find the ultimate source to see if it's listed on the command line. Or, in a more forward-looking way of doing things, you can record that in whatever context object you might have during template instantiation. Faster but potentially more buggy. It's also not terribly readable. We could add template stacktraces to it: foo.d-sourcelib.d-mixin.d: // bar.d:3 // sourcelib.d:3 static assert(2 == 2); // foo.d:3 // sourcelib.d:3 static assert(1 == 1); bar.d-sourcelib.d-mixin.d: // bar.d:3 // sourcelib.d:3 static assert(2 == 2); I think this would work (not certain), but it's not simple. That's why I'd prefer the best-effort version.
Sep 08 2018
prev sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 8 September 2018 at 22:01:23 UTC, Manu wrote:
 [snip]
 Is anyone interested in this issue?
https://github.com/dlang/dmd/pull/8677
Sep 09 2018