digitalmars.D.ide - Christmas request! VD closure debugging
- Michelle Long (33/33) Dec 13 2018 I've mentioned this before but I am really struggling with this
- Petar Kirov [ZombineDev] (29/63) Dec 14 2018 Have you had the chance to compare the debug information that LDC
- Rainer Schuetze (5/41) Dec 14 2018 Have you tried the latest release with dmd master/nightly? I have added
- Michelle Long (11/17) Dec 14 2018 Cool, I tried but the link was down:
- Rainer Schuetze (9/33) Dec 14 2018 Not sure why this doesn't work, but the release archive still works:
- Michelle Long (81/81) Dec 15 2018 And, of course, it is not working in my production code when a
- Rainer Schuetze (7/130) Dec 16 2018 The debug info for captured variables is not generated for OMF object
- Michelle Long (83/120) Dec 18 2018 Ok, I installed the latest dmd,rebuilt some libraries and the
- Rainer Schuetze (21/121) Dec 20 2018 That's a bug in the mago extension that hides the captured variables if
- Michelle Long (19/56) Dec 20 2018 Yeah. Maybe a better approach or another approach is to provide a
I've mentioned this before but I am really struggling with this issue and it is probably the biggest problem I have with VD + D in general. Again, any nesting of stuff (such as functions) or opApply overrides of foreach and other cases lose scope of the locals above it(it's globals) are lost in the locals, auto, watch, etc.... This makes it tremendously difficult to debug unless hard aliases are created... which uglify the code and must be maintained to prevent having to constantly re-add them and remove them. You told me that dmd and walter do not think it is a big issue and that it is a problem with dmd. If this is true surely you can use your clout to get him to change his mind. He doens't realize the headache this causes in debugging. The whole point of being able to see the contents of variables is at the essence of debugging. Without it the debugging experience would be rather pointless. I'm not sure what is required and how it can be done much less of the exact cause so I implore you to try to convince Walter it is necessary for VD to function properly. Surely some compromise could be had. D is a very nested language and many times a simple nesting of a function and or struct can make all the difference in a problem. But even since opApply exhibits this problem, it is very common. I run across it even when I am using library code or anything that overrides foreach. I actually though I remembered not having this problem a while back or I'm sure I would have complained about it from the get go... but I could be wrong or just not experienced it much at the time. I don't think it would be a hard problem to solve since it really is just keeping track of the variables addresses and which ones that need to be exposed. But maybe this is not so easy to get working?
Dec 13 2018
Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> On Friday, 14 December 2018 at 03:21:58 UTC, Michelle Long wrote:I've mentioned this before but I am really struggling with this issue and it is probably the biggest problem I have with VD + D in general. Again, any nesting of stuff (such as functions) or opApply overrides of foreach and other cases lose scope of the locals above it(it's globals) are lost in the locals, auto, watch, etc.... This makes it tremendously difficult to debug unless hard aliases are created... which uglify the code and must be maintained to prevent having to constantly re-add them and remove them. You told me that dmd and walter do not think it is a big issue and that it is a problem with dmd. If this is true surely you can use your clout to get him to change his mind. He doens't realize the headache this causes in debugging. The whole point of being able to see the contents of variables is at the essence of debugging. Without it the debugging experience would be rather pointless. I'm not sure what is required and how it can be done much less of the exact cause so I implore you to try to convince Walter it is necessary for VD to function properly. Surely some compromise could be had. D is a very nested language and many times a simple nesting of a function and or struct can make all the difference in a problem. But even since opApply exhibits this problem, it is very common. I run across it even when I am using library code or anything that overrides foreach. I actually though I remembered not having this problem a while back or I'm sure I would have complained about it from the get go... but I could be wrong or just not experienced it much at the time. I don't think it would be a hard problem to solve since it really is just keeping track of the variables addresses and which ones that need to be exposed. But maybe this is not so easy to get working?Have you had the chance to compare the debug information that LDC produces with that of DMD? Since LDC follows relatively closely dmd, it shouldn't be too much work to switch between the compilers back and forth. My understanding is that since it uses a different API internally (that of LLVM) its output can different and better than that of dmd. Also it can benefit from improvements that the broader LLVM community provides, independently of the LDC/DMD devs. Walter himself is not against debug experience improvements (just the contrary), it's just more likely that he doesn't use a source-level debugger himself and has higher priority issues on his list. But as someone who participates (when I have a little free time) in the development, I can assure you that he's not against any well documented and implemented improvement. I myself like meta programming, and the complete lack of debugging has taught me to write my code in more easily unit testable manner. I don't know your codebase, so I can't make any informed recommendations, but I have often found that splitting complex code in smaller more general building blocks helps me a lot. code with async/await, LINQ, nested lambdas, etc. I know how much D debuggers can improve. You can have look at the debug info tests here: https://github.com/ldc-developers/ldc/tree/master/tests/debuginfo to get a sense of the level of support. Often, if you follow a commit that made a change to those files, you would find insightful discussions in the pull request that introduced them.
Dec 14 2018
On 14/12/2018 04:21, Michelle Long wrote:I've mentioned this before but I am really struggling with this issue and it is probably the biggest problem I have with VD + D in general. Again, any nesting of stuff (such as functions) or opApply overrides of foreach and other cases lose scope of the locals above it(it's globals) are lost in the locals, auto, watch, etc.... This makes it tremendously difficult to debug unless hard aliases are created... which uglify the code and must be maintained to prevent having to constantly re-add them and remove them. You told me that dmd and walter do not think it is a big issue and that it is a problem with dmd. If this is true surely you can use your clout to get him to change his mind. He doens't realize the headache this causes in debugging. The whole point of being able to see the contents of variables is at the essence of debugging. Without it the debugging experience would be rather pointless. I'm not sure what is required and how it can be done much less of the exact cause so I implore you to try to convince Walter it is necessary for VD to function properly. Surely some compromise could be had. D is a very nested language and many times a simple nesting of a function and or struct can make all the difference in a problem. But even since opApply exhibits this problem, it is very common. I run across it even when I am using library code or anything that overrides foreach. I actually though I remembered not having this problem a while back or I'm sure I would have complained about it from the get go... but I could be wrong or just not experienced it much at the time. I don't think it would be a hard problem to solve since it really is just keeping track of the variables addresses and which ones that need to be exposed. But maybe this is not so easy to get working?Have you tried the latest release with dmd master/nightly? I have added appropriate debug information to be generated by dmd and added support for these in mago. It should show variables captured from an outer function as local variables.
Dec 14 2018
On Friday, 14 December 2018 at 18:40:47 UTC, Rainer Schuetze wrote:On 14/12/2018 04:21, Michelle Long wrote:Cool, I tried but the link was down: https://downloads.dlang.org/nightlies/dmd-master/dmd.master.windows.7z I assume I'll have to download the VD nightly too? Is there a link to that? https://github.com/dlang/visuald/releases It would be nice if there were similar links on the D download page for Visual D or on the github page for the nightly, if it exists. Thanks for the work you have done! I appreciate it![...]Have you tried the latest release with dmd master/nightly? I have added appropriate debug information to be generated by dmd and added support for these in mago. It should show variables captured from an outer function as local variables.
Dec 14 2018
On 15/12/2018 01:45, Michelle Long wrote:On Friday, 14 December 2018 at 18:40:47 UTC, Rainer Schuetze wrote:Not sure why this doesn't work, but the release archive still works: http://downloads.dlang.org/nightlies/dmd-master-2018-12-15/On 14/12/2018 04:21, Michelle Long wrote:Cool, I tried but the link was down: https://downloads.dlang.org/nightlies/dmd-master/dmd.master.windows.7z[...]Have you tried the latest release with dmd master/nightly? I have added appropriate debug information to be generated by dmd and added support for these in mago. It should show variables captured from an outer function as local variables.I assume I'll have to download the VD nightly too?No, version 0.48 already has that.Is there a link to that?Development mostly happens on my fork and is tested by https://ci.appveyor.com/project/rainers/visuald. The resulting installers are slightly different than actual releases due to limitations of the images provided by Appveyor (i.e. not all versions of VS installed in the same image).https://github.com/dlang/visuald/releases It would be nice if there were similar links on the D download page for Visual D or on the github page for the nightly, if it exists. Thanks for the work you have done! I appreciate it!
Dec 14 2018
And, of course, it is not working in my production code when a
simple example works:
auto foo()
{
int x = 0;
auto bar()
{
int y = 0;
x++;
y++;
}
bar();
}
int main()
{
foo();
return 0;
}
My code is more like
class X
{
auto foo()
{
int x = 0;
auto bar()
{
int y = 0;
x++;
y++;
}
bar();
}
}
int main()
{
X x = new X();
x.foo();
return 0;
}
which when done gives a crap value for x in the auto's and watch.
(in my production code the variables are not found)
Note that making foo static (and changing to X.foo()) works...
My my production code isn't showing any of the variables is
beyond me since it is pretty much the above case almost exactly,
in fact, here is the code structure:
class A { }
class X
{
A[] a;
static auto foo(string s, bool b = false)
{
int x = 0;
struct v;
auto r(T, bool s = true)(int l = 1)
{
static if (s)
{
ulong q = 0;
x += T.sizeof*l;
}
return x;
}
r!(char, true)();
}
}
int main()
{
X x = new X();
X.foo("asdf");
return 0;
}
which does show X just fine(I simplified the code above removing
all "unnecessary" things but seems it isn't so unnecessary.
Building under x64 does work although the auto's are not shown...
I can still watch the (global)variables and they give the correct
values.
So, this offers a solution until the issue can be sorted. As long
as I can get at the globals then I can properly debug. Why my
code is having issues with the x86 build, but when reduced works
fine, but also works partially with the x64 build is beyond me...
Thanks...
Dec 15 2018
On 16/12/2018 05:13, Michelle Long wrote:And, of course, it is not working in my production code when a simple example works: auto foo() { int x = 0; auto bar() { int y = 0; x++; y++; } bar(); } int main() { foo(); return 0; } My code is more like class X { auto foo() { int x = 0; auto bar() { int y = 0; x++; y++; } bar(); } } int main() { X x = new X(); x.foo(); return 0; } which when done gives a crap value for x in the auto's and watch.Should be fixed by https://github.com/dlang/dmd/pull/9092(in my production code the variables are not found)The debug info for captured variables is not generated for OMF object files as it crashes optlink. You have to use COFF output.Note that making foo static (and changing to X.foo()) works... My my production code isn't showing any of the variables is beyond me since it is pretty much the above case almost exactly, in fact, here is the code structure: class A { } class X { A[] a; static auto foo(string s, bool b = false) { int x = 0; struct v; auto r(T, bool s = true)(int l = 1) { static if (s) { ulong q = 0; x += T.sizeof*l; } return x; } r!(char, true)(); } } int main() { X x = new X(); X.foo("asdf"); return 0; } which does show X just fine(I simplified the code above removing all "unnecessary" things but seems it isn't so unnecessary.This seems to work with the patch above, too. Sometimes I don't see all captured variables listed as locals, will have to investigate it.Building under x64 does work although the auto's are not shown... I can still watch the (global)variables and they give the correct values. So, this offers a solution until the issue can be sorted. As long as I can get at the globals then I can properly debug. Why my code is having issues with the x86 build, but when reduced works fine, but also works partially with the x64 build is beyond me... Thanks...
Dec 16 2018
On Saturday, 15 December 2018 at 07:33:30 UTC, Rainer Schuetze wrote:On 15/12/2018 01:45, Michelle Long wrote:Ok, I installed the latest dmd,rebuilt some libraries and the source for it then tried some code I had and the parent variables did not show up: The code was effectively auto foo() { int x = 0; auto bar() { int y = 0; x++; } } I put a BP on the `int y = 0;` line and the variable x is not being displayed. x is not showing up in the locals but I did add it as a watch and it seemed to show it just fine. This might just be a simple display bug for locals. It is also showing up for autos! So this is actually good and seems to be working. I was mistakenly thinking they should be locals. I tend to use only locals rather than auto's and watches because of the screen real estate required. Not sure if there is any feasible solution. Would it be possible to have, say, a variable in the locals called "globals"(or some better name) that contained all the globals to the scope or just list the globals inline(like autos does) when there are just a few or when they are just one level above(which would get most needed cases)? Possibly prefix the this case. Again, the main idea is to save screen space and time by not having to juggle windows. Locals tend to cover most cases fine(biggest problem I have with it is that I tend to have trouble finding a specific variable by name since the list is in some order that makes no sense to me... definitely not alphabetical and VS doesn't allow sorting... I'd actually preferred them sorted, or at least have it as an option since then I can locate the variables much quicker). Possible solutions: 1. + Globals x ... y [Just list all global variables to the current scope in a globals tree] 2. y + Globals x Globals ... [Show the globals in a tree if that info is available] 3. x y [If there are just a few globals or parent variables then inline them] 4. 1 x y [using prefix, 1 is the ancestor level of the variable and is just stating it is global] Optional's: 1. Allow different sorting of variable names in the locals, autos, watches, etc. Not sure why VS doesn't have this built in, maybe a switch is required or something must be setup? Most programs that display any type of information in a column will can't recall. 2. case scopes seem to capture(show in locals) all the variables outside it. That is, a case doesn't seem to create it's own local scope. case 1: int x; case 2: int y; x is shown while debugging in y... x is not needed to be seen though. 3. List types from derived to base instead of base to derived(already discussed before). You can do all this stuff when ever you feel like it of course... Just pointing out some improvements. I haven't tested the globals fix yet but if it works well then I will just switch to autos or use a watch when I need to.On Friday, 14 December 2018 at 18:40:47 UTC, Rainer Schuetze wrote:Not sure why this doesn't work, but the release archive still works: http://downloads.dlang.org/nightlies/dmd-master-2018-12-15/On 14/12/2018 04:21, Michelle Long wrote:Cool, I tried but the link was down: https://downloads.dlang.org/nightlies/dmd-master/dmd.master.windows.7z[...]Have you tried the latest release with dmd master/nightly? I have added appropriate debug information to be generated by dmd and added support for these in mago. It should show variables captured from an outer function as local variables.I assume I'll have to download the VD nightly too?No, version 0.48 already has that.Is there a link to that?Development mostly happens on my fork and is tested by https://ci.appveyor.com/project/rainers/visuald. The resulting installers are slightly different than actual releases due to limitations of the images provided by Appveyor (i.e. not all versions of VS installed in the same image).https://github.com/dlang/visuald/releases It would be nice if there were similar links on the D download page for Visual D or on the github page for the nightly, if it exists. Thanks for the work you have done! I appreciate it!
Dec 18 2018
On 18/12/2018 21:09, Michelle Long wrote:Ok, I installed the latest dmd,rebuilt some libraries and the source for it then tried some code I had and the parent variables did not show up: The code was effectively auto foo() { int x = 0; auto bar() { int y = 0; x++; } } I put a BP on the `int y = 0;` line and the variable x is not being displayed. x is not showing up in the locals but I did add it as a watch and it seemed to show it just fine. This might just be a simple display bug for locals. It is also showing up for autos! So this is actually good and seems to be working. I was mistakenly thinking they should be locals.That's a bug in the mago extension that hides the captured variables if they are from a stack capture, not an allocated closure. Fixed in the next version. BTW: if you enable showing internal symbols in the mago options, you can see __capture as the actual local with the captured variables as its struct fields.I tend to use only locals rather than auto's and watches because of the screen real estate required. Not sure if there is any feasible solution.Would it be possible to have, say, a variable in the locals called "globals"(or some better name) that contained all the globals to the scope or just list the globals inline(like autos does) when there are just a few or when they are just one level above(which would get most to signify it is a global in this case. Again, the main idea is to save screen space and time by not having to juggle windows. Locals tend to cover most cases fine(biggest problem I have with it is that I tend to have trouble finding a specific variable by name since the list is in some order that makes no sense to me... definitely not alphabetical and VS doesn't allow sorting... I'd actually preferred them sorted, or at least have it as an option since then I can locate the variables much quicker). Possible solutions: 1. + Globals x ... y [Just list all global variables to the current scope in a globals tree] 2. y + Globals x Globals ... [Show the globals in a tree if that info is available] 3. x y [If there are just a few globals or parent variables then inline them] 4. 1 x y [using prefix, 1 is the ancestor level of the variable and is just stating it is global]Globals are a bit problematic because there is no debug information about what's currently imported, so name lookup can be ambiguous (and using the language service will likely be too slow). There is some support to figure the current scope of a function if it is extern(D), so that statics and globals in the same module can be resolved. I wouldn't want to add globals to the locals window as it could slow it down, but having some special symbol to explore them in a tree (built from fully qualified names) in the watch window might be a feasible solution.Optional's: 1. Allow different sorting of variable names in the locals, autos, watches, etc. Not sure why VS doesn't have this built in, maybe a switch is required or something must be setup? Most programs that display any type of information in a column will allow it to be sorted. I think thatSorting the list by clicking on the column header is not under control of the extension, but for locals an option could be added. These are listed as they are found in the debug information.2. case scopes seem to capture(show in locals) all the variables outside it. That is, a case doesn't seem to create it's own local scope. case 1: int x; case 2: int y; x is shown while debugging in y... x is not needed to be seen though.The compiler will have to generate proper scope information. Maybe this already works if there are conflicting declarations as with other scopes?3. List types from derived to base instead of base to derived(already discussed before). You can do all this stuff when ever you feel like it of course... Just pointing out some improvements. I haven't tested the globals fix yet but if it works well then I will just switch to autos or use a watch when I need to.
Dec 20 2018
By globals I mean anything global to the current scope. I don't know if there is already a name for these variables.[If there are just a few globals or parent variables then inline them] 4. 1 x y [using prefix, 1 is the ancestor level of the variable and is just stating it is global]Globals are a bit problematic because there is no debug information about what's currently imported, so name lookup can be ambiguous (and using the language service will likely be too slow). There is some support to figure the current scope of a function if it is extern(D), so that statics and globals in the same module can be resolved.I wouldn't want to add globals to the locals window as it could slow it down, but having some special symbol to explore them in a tree (built from fully qualified names) in the watch window might be a feasible solution.Yeah. Maybe a better approach or another approach is to provide a single "global access" that allows one to see any variable at any point in time that is available to the compiler by digging down a tree. Lua sorta has this with _G. The idea is that if something is not being show or accessed one can go find it if necessary, that way there always is some way to see a variable. It would simply have to deal displaying the information correctly(threading, availability, etc).Sorting the list by clicking on the column header is not under control of the extension, but for locals an option could be added. These are listed as they are found in the debug information.ok.That would be something I wouldn't know about. The compiler thinks case statements shadow variables so it doesn't seem to be aware or treat case statements as their own scope. If the debugger can determine it is in a switch statement and get the line info of the cases then it could brute force it but this sounds like to much work for something that probably could be done in a few lines of code.2. case scopes seem to capture(show in locals) all the variables outside it. That is, a case doesn't seem to create it's own local scope. case 1: int x; case 2: int y; x is shown while debugging in y... x is not needed to be seen though.The compiler will have to generate proper scope information. Maybe this already works if there are conflicting declarations as with other scopes?
Dec 20 2018