www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.ide - Christmas request! VD closure debugging

reply Michelle Long <HappyDance321 gmail.com> writes:
I've mentioned this before but I am really struggling with this 
issue and it is probably the biggest problem I have with VD + D 
in general.

Again, any nesting of stuff (such as functions) or opApply 
overrides of foreach and other cases lose scope of the locals 
above it(it's globals) are lost in the locals, auto, watch, 
etc....

This makes it tremendously difficult to debug unless hard aliases 
are created... which uglify the code and must be maintained to 
prevent having to constantly re-add them and remove them.


You told me that dmd and walter do not think it is a big issue 
and that it is a problem with dmd. If this is true surely you can 
use your clout to get him to change his mind. He doens't realize 
the headache this causes in debugging. The whole point of being 
able to see the contents of variables is at the essence of 
debugging. Without it the debugging experience would be rather 
pointless.

I'm not sure what is required and how it can be done much less of 
the exact cause so I implore you to try to convince Walter it is 
necessary for VD to function properly. Surely some compromise 
could be had. D is a very nested language and many times a simple 
nesting of a function and or struct can make all the difference 
in a problem.

But even since opApply exhibits this problem, it is very common. 
I run across it even when I am using library code or anything 
that overrides foreach. I actually though I remembered not having 
this problem a while back or I'm sure I would have complained 
about it from the get go... but I could be wrong or just not 
experienced it much at the time.

I don't think it would be a hard problem to solve since it really 
is just keeping track of the variables addresses and which ones 
that need to be exposed. But maybe this is not so easy to get 
working?
Dec 13 2018
next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 14 December 2018 at 03:21:58 UTC, Michelle Long wrote:
 I've mentioned this before but I am really struggling with this 
 issue and it is probably the biggest problem I have with VD + D 
 in general.

 Again, any nesting of stuff (such as functions) or opApply 
 overrides of foreach and other cases lose scope of the locals 
 above it(it's globals) are lost in the locals, auto, watch, 
 etc....

 This makes it tremendously difficult to debug unless hard 
 aliases are created... which uglify the code and must be 
 maintained to prevent having to constantly re-add them and 
 remove them.


 You told me that dmd and walter do not think it is a big issue 
 and that it is a problem with dmd. If this is true surely you 
 can use your clout to get him to change his mind. He doens't 
 realize the headache this causes in debugging. The whole point 
 of being able to see the contents of variables is at the 
 essence of debugging. Without it the debugging experience would 
 be rather pointless.

 I'm not sure what is required and how it can be done much less 
 of the exact cause so I implore you to try to convince Walter 
 it is necessary for VD to function properly. Surely some 
 compromise could be had. D is a very nested language and many 
 times a simple nesting of a function and or struct can make all 
 the difference in a problem.

 But even since opApply exhibits this problem, it is very 
 common. I run across it even when I am using library code or 
 anything that overrides foreach. I actually though I remembered 
 not having this problem a while back or I'm sure I would have 
 complained about it from the get go... but I could be wrong or 
 just not experienced it much at the time.

 I don't think it would be a hard problem to solve since it 
 really is just keeping track of the variables addresses and 
 which ones that need to be exposed. But maybe this is not so 
 easy to get working?
Have you had the chance to compare the debug information that LDC produces with that of DMD? Since LDC follows relatively closely dmd, it shouldn't be too much work to switch between the compilers back and forth. My understanding is that since it uses a different API internally (that of LLVM) its output can different and better than that of dmd. Also it can benefit from improvements that the broader LLVM community provides, independently of the LDC/DMD devs. Walter himself is not against debug experience improvements (just the contrary), it's just more likely that he doesn't use a source-level debugger himself and has higher priority issues on his list. But as someone who participates (when I have a little free time) in the development, I can assure you that he's not against any well documented and implemented improvement. I myself like meta programming, and the complete lack of debugging has taught me to write my code in more easily unit testable manner. I don't know your codebase, so I can't make any informed recommendations, but I have often found that splitting complex code in smaller more general building blocks helps me a lot. code with async/await, LINQ, nested lambdas, etc. I know how much D debuggers can improve. You can have look at the debug info tests here: https://github.com/ldc-developers/ldc/tree/master/tests/debuginfo to get a sense of the level of support. Often, if you follow a commit that made a change to those files, you would find insightful discussions in the pull request that introduced them.
Dec 14 2018
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 14/12/2018 04:21, Michelle Long wrote:
 I've mentioned this before but I am really struggling with this issue
 and it is probably the biggest problem I have with VD + D in general.
 
 Again, any nesting of stuff (such as functions) or opApply overrides of
 foreach and other cases lose scope of the locals above it(it's globals)
 are lost in the locals, auto, watch, etc....
 
 This makes it tremendously difficult to debug unless hard aliases are
 created... which uglify the code and must be maintained to prevent
 having to constantly re-add them and remove them.
 
 
 You told me that dmd and walter do not think it is a big issue and that
 it is a problem with dmd. If this is true surely you can use your clout
 to get him to change his mind. He doens't realize the headache this
 causes in debugging. The whole point of being able to see the contents
 of variables is at the essence of debugging. Without it the debugging
 experience would be rather pointless.
 
 I'm not sure what is required and how it can be done much less of the
 exact cause so I implore you to try to convince Walter it is necessary
 for VD to function properly. Surely some compromise could be had. D is a
 very nested language and many times a simple nesting of a function and
 or struct can make all the difference in a problem.
 
 But even since opApply exhibits this problem, it is very common. I run
 across it even when I am using library code or anything that overrides
 foreach. I actually though I remembered not having this problem a while
 back or I'm sure I would have complained about it from the get go... but
 I could be wrong or just not experienced it much at the time.
 
 I don't think it would be a hard problem to solve since it really is
 just keeping track of the variables addresses and which ones that need
 to be exposed. But maybe this is not so easy to get working?
 
 
Have you tried the latest release with dmd master/nightly? I have added appropriate debug information to be generated by dmd and added support for these in mago. It should show variables captured from an outer function as local variables.
Dec 14 2018
parent reply Michelle Long <HappyDance321 gmail.com> writes:
On Friday, 14 December 2018 at 18:40:47 UTC, Rainer Schuetze 
wrote:
 On 14/12/2018 04:21, Michelle Long wrote:
 [...]
Have you tried the latest release with dmd master/nightly? I have added appropriate debug information to be generated by dmd and added support for these in mago. It should show variables captured from an outer function as local variables.
Cool, I tried but the link was down: https://downloads.dlang.org/nightlies/dmd-master/dmd.master.windows.7z I assume I'll have to download the VD nightly too? Is there a link to that? https://github.com/dlang/visuald/releases It would be nice if there were similar links on the D download page for Visual D or on the github page for the nightly, if it exists. Thanks for the work you have done! I appreciate it!
Dec 14 2018
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 15/12/2018 01:45, Michelle Long wrote:
 On Friday, 14 December 2018 at 18:40:47 UTC, Rainer Schuetze wrote:
 On 14/12/2018 04:21, Michelle Long wrote:
 [...]
Have you tried the latest release with dmd master/nightly? I have added appropriate debug information to be generated by dmd and added support for these in mago. It should show variables captured from an outer function as local variables.
Cool, I tried but the link was down: https://downloads.dlang.org/nightlies/dmd-master/dmd.master.windows.7z
Not sure why this doesn't work, but the release archive still works: http://downloads.dlang.org/nightlies/dmd-master-2018-12-15/
 
 I assume I'll have to download the VD nightly too? 
No, version 0.48 already has that.
 Is there a link to that?
Development mostly happens on my fork and is tested by https://ci.appveyor.com/project/rainers/visuald. The resulting installers are slightly different than actual releases due to limitations of the images provided by Appveyor (i.e. not all versions of VS installed in the same image).
 https://github.com/dlang/visuald/releases
 
 It would be nice if there were similar links on the D download page for
 Visual D or on the github page for the nightly, if it exists.
 
 Thanks for the work you have done! I appreciate it!
Dec 14 2018
next sibling parent reply Michelle Long <HappyDance321 gmail.com> writes:
And, of course, it is not working in my production code when a 
simple example works:

auto foo()
{
     int x = 0;
     auto bar()
     {
		int y = 0;
		x++;

		y++;
     }

	bar();
}


int main()
{

foo();


     return 0;
}


My code is more like

class X
{
	auto foo()
	{
		int x = 0;
		auto bar()
		{
			int y = 0;
			x++;

			y++;
		}

		bar();
	}
}

int main()
{

	X x = new X();
	x.foo();


     return 0;
}


which when done gives a crap value for x in the auto's and watch. 
(in my production code the variables are not found)


Note that making foo static (and changing to X.foo()) works...

My my production code isn't showing any of the variables is 
beyond me since it is pretty much the above case almost exactly, 
in fact, here is the code structure:

class A { }

class X
{
	A[] a;

	static auto foo(string s, bool b = false)
	{

		int x = 0;

		struct v;

		auto r(T, bool s = true)(int l = 1)
		{
			static if (s)
			{
				ulong q = 0;
				x += T.sizeof*l;
			}

			return x;
		}

		r!(char, true)();

	}
}

int main()
{

	X x = new X();
	X.foo("asdf");


     return 0;
}

which does show X just fine(I simplified the code above removing 
all "unnecessary" things but seems it isn't so unnecessary.

Building under x64 does work although the auto's are not shown... 
I can still watch the (global)variables and they give the correct 
values.

So, this offers a solution until the issue can be sorted. As long 
as I can get at the globals then I can properly debug.  Why my 
code is having issues with the x86 build, but when reduced works 
fine, but also works partially with the x64 build is beyond me...

Thanks...
Dec 15 2018
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 16/12/2018 05:13, Michelle Long wrote:
 And, of course, it is not working in my production code when a simple
 example works:
 
 auto foo()
 {
     int x = 0;
     auto bar()
     {
         int y = 0;
         x++;
 
         y++;
     }
 
     bar();
 }
 
 
 int main()
 {
 
 foo();
 
 
     return 0;
 }
 
 
 My code is more like
 
 class X
 {
     auto foo()
     {
         int x = 0;
         auto bar()
         {
             int y = 0;
             x++;
 
             y++;
         }
 
         bar();
     }
 }
 
 int main()
 {
 
     X x = new X();
     x.foo();
 
 
     return 0;
 }
 
 
 which when done gives a crap value for x in the auto's and watch. 
Should be fixed by https://github.com/dlang/dmd/pull/9092
 (in my
 production code the variables are not found)
The debug info for captured variables is not generated for OMF object files as it crashes optlink. You have to use COFF output.
 
 
 Note that making foo static (and changing to X.foo()) works...
 
 My my production code isn't showing any of the variables is beyond me
 since it is pretty much the above case almost exactly, in fact, here is
 the code structure:
 
 class A { }
 
 class X
 {
     A[] a;
 
     static auto foo(string s, bool b = false)
     {
 
         int x = 0;
 
         struct v;
 
         auto r(T, bool s = true)(int l = 1)
         {
             static if (s)
             {
                 ulong q = 0;
                 x += T.sizeof*l;
             }
 
             return x;
         }
 
         r!(char, true)();
 
     }
 }
 
 int main()
 {
 
     X x = new X();
     X.foo("asdf");
 
 
     return 0;
 }
 
 which does show X just fine(I simplified the code above removing all
 "unnecessary" things but seems it isn't so unnecessary.
This seems to work with the patch above, too. Sometimes I don't see all captured variables listed as locals, will have to investigate it.
 
 Building under x64 does work although the auto's are not shown... I can
 still watch the (global)variables and they give the correct values.
 
 So, this offers a solution until the issue can be sorted. As long as I
 can get at the globals then I can properly debug.  Why my code is having
 issues with the x86 build, but when reduced works fine, but also works
 partially with the x64 build is beyond me...
 
 Thanks...
 
 
 
Dec 16 2018
prev sibling parent reply Michelle Long <HappyDance321 gmail.com> writes:
On Saturday, 15 December 2018 at 07:33:30 UTC, Rainer Schuetze 
wrote:
 On 15/12/2018 01:45, Michelle Long wrote:
 On Friday, 14 December 2018 at 18:40:47 UTC, Rainer Schuetze 
 wrote:
 On 14/12/2018 04:21, Michelle Long wrote:
 [...]
Have you tried the latest release with dmd master/nightly? I have added appropriate debug information to be generated by dmd and added support for these in mago. It should show variables captured from an outer function as local variables.
Cool, I tried but the link was down: https://downloads.dlang.org/nightlies/dmd-master/dmd.master.windows.7z
Not sure why this doesn't work, but the release archive still works: http://downloads.dlang.org/nightlies/dmd-master-2018-12-15/
 
 I assume I'll have to download the VD nightly too?
No, version 0.48 already has that.
 Is there a link to that?
Development mostly happens on my fork and is tested by https://ci.appveyor.com/project/rainers/visuald. The resulting installers are slightly different than actual releases due to limitations of the images provided by Appveyor (i.e. not all versions of VS installed in the same image).
 https://github.com/dlang/visuald/releases
 
 It would be nice if there were similar links on the D download 
 page for Visual D or on the github page for the nightly, if it 
 exists.
 
 Thanks for the work you have done! I appreciate it!
Ok, I installed the latest dmd,rebuilt some libraries and the source for it then tried some code I had and the parent variables did not show up: The code was effectively auto foo() { int x = 0; auto bar() { int y = 0; x++; } } I put a BP on the `int y = 0;` line and the variable x is not being displayed. x is not showing up in the locals but I did add it as a watch and it seemed to show it just fine. This might just be a simple display bug for locals. It is also showing up for autos! So this is actually good and seems to be working. I was mistakenly thinking they should be locals. I tend to use only locals rather than auto's and watches because of the screen real estate required. Not sure if there is any feasible solution. Would it be possible to have, say, a variable in the locals called "globals"(or some better name) that contained all the globals to the scope or just list the globals inline(like autos does) when there are just a few or when they are just one level above(which would get most needed cases)? Possibly prefix the this case. Again, the main idea is to save screen space and time by not having to juggle windows. Locals tend to cover most cases fine(biggest problem I have with it is that I tend to have trouble finding a specific variable by name since the list is in some order that makes no sense to me... definitely not alphabetical and VS doesn't allow sorting... I'd actually preferred them sorted, or at least have it as an option since then I can locate the variables much quicker). Possible solutions: 1. + Globals x ... y [Just list all global variables to the current scope in a globals tree] 2. y + Globals x Globals ... [Show the globals in a tree if that info is available] 3. x y [If there are just a few globals or parent variables then inline them] 4. 1 x y [using prefix, 1 is the ancestor level of the variable and is just stating it is global] Optional's: 1. Allow different sorting of variable names in the locals, autos, watches, etc. Not sure why VS doesn't have this built in, maybe a switch is required or something must be setup? Most programs that display any type of information in a column will can't recall. 2. case scopes seem to capture(show in locals) all the variables outside it. That is, a case doesn't seem to create it's own local scope. case 1: int x; case 2: int y; x is shown while debugging in y... x is not needed to be seen though. 3. List types from derived to base instead of base to derived(already discussed before). You can do all this stuff when ever you feel like it of course... Just pointing out some improvements. I haven't tested the globals fix yet but if it works well then I will just switch to autos or use a watch when I need to.
Dec 18 2018
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 18/12/2018 21:09, Michelle Long wrote:
 Ok, I installed the latest dmd,rebuilt some libraries and the source for
 it then tried some code I had and the parent variables did not show up:
 
 The code was effectively
 
 auto foo()
 {
     int x = 0;
     auto bar()
     {
        int y = 0;
        x++;
     }
 }
 
 I put a BP on the `int y = 0;` line and the variable x is not being
 displayed.
 
 x is not showing up in the locals but I did add it as a watch and it
 seemed to show it just fine. This might just be a simple display bug for
 locals. It is also showing up for autos! So this is actually good and
 seems to be working. I was mistakenly thinking they should be locals.
That's a bug in the mago extension that hides the captured variables if they are from a stack capture, not an allocated closure. Fixed in the next version. BTW: if you enable showing internal symbols in the mago options, you can see __capture as the actual local with the captured variables as its struct fields.
 
 I tend to use only locals rather than auto's and watches because of the
 screen real estate required.
 
 Not sure if there is any feasible solution.
 
 Would it be possible to have, say, a variable in the locals called
 "globals"(or some better name) that contained all the globals to the
 scope or just list the globals inline(like autos does) when there are
 just a few or when they are just one level above(which would get most

 to signify it is a global in this case.
 
 Again, the main idea is to save screen space and time by not having to
 juggle windows. Locals tend to cover most cases fine(biggest problem I
 have with it is that I tend to have trouble finding a specific variable
 by name since the list is in some order that makes no sense to me...
 definitely not alphabetical and VS doesn't allow sorting... I'd actually
 preferred them sorted, or at least have it as an option since then I can
 locate the variables much quicker).
 
 
 Possible solutions:
 
 1. + Globals
        x
        ...
    y
 [Just list all global variables to the current scope in a globals tree]
 
 2. y
    + Globals
         x
         Globals
         ...
 [Show the globals in a tree if that info is available]
 
 3. x
    y
 [If there are just a few globals or parent variables then inline them]
 
 4. 1 x
    y
 [using prefix, 1 is the ancestor level of the variable and   is just
 stating it is global]
Globals are a bit problematic because there is no debug information about what's currently imported, so name lookup can be ambiguous (and using the language service will likely be too slow). There is some support to figure the current scope of a function if it is extern(D), so that statics and globals in the same module can be resolved. I wouldn't want to add globals to the locals window as it could slow it down, but having some special symbol to explore them in a tree (built from fully qualified names) in the watch window might be a feasible solution.
 
 
 
 
 Optional's:
 1. Allow different sorting of variable names in the locals, autos,
 watches, etc. Not sure why VS doesn't have this built in, maybe a switch
 is required or something must be setup? Most programs that display any
 type of information in a column will allow it to be sorted. I think that

Sorting the list by clicking on the column header is not under control of the extension, but for locals an option could be added. These are listed as they are found in the debug information.
 
 2. case scopes seem to capture(show in locals) all the variables outside
 it. That is, a case doesn't seem to create it's own local scope.
 
 case 1:
    int x;
 case 2:
    int y;
 
 x is shown while debugging in y... x is not needed to be seen though.
The compiler will have to generate proper scope information. Maybe this already works if there are conflicting declarations as with other scopes?
 3. List types from derived to base instead of base to derived(already
 discussed before).
 
 
 You can do all this stuff when ever you feel like it of course... Just
 pointing out some improvements. I haven't tested the globals fix yet but
 if it works well then I will just switch to autos or use a watch when I
 need to.
 
 
 
 
 
Dec 20 2018
parent Michelle Long <HappyDance321 gmail.com> writes:
 [If there are just a few globals or parent variables then 
 inline them]
 
 4. 1 x
    y
 [using prefix, 1 is the ancestor level of the variable and   
 is just
 stating it is global]
Globals are a bit problematic because there is no debug information about what's currently imported, so name lookup can be ambiguous (and using the language service will likely be too slow). There is some support to figure the current scope of a function if it is extern(D), so that statics and globals in the same module can be resolved.
By globals I mean anything global to the current scope. I don't know if there is already a name for these variables.
 I wouldn't want to add globals to the locals window as it could 
 slow it down, but having some special symbol to explore them in 
 a tree (built from fully qualified names) in the watch window 
 might be a feasible solution.
Yeah. Maybe a better approach or another approach is to provide a single "global access" that allows one to see any variable at any point in time that is available to the compiler by digging down a tree. Lua sorta has this with _G. The idea is that if something is not being show or accessed one can go find it if necessary, that way there always is some way to see a variable. It would simply have to deal displaying the information correctly(threading, availability, etc).
 Sorting the list by clicking on the column header is not under 
 control of the extension, but for locals an option could be 
 added. These are listed as they are found in the debug 
 information.
ok.
 
 2. case scopes seem to capture(show in locals) all the 
 variables outside it. That is, a case doesn't seem to create 
 it's own local scope.
 
 case 1:
    int x;
 case 2:
    int y;
 
 x is shown while debugging in y... x is not needed to be seen 
 though.
The compiler will have to generate proper scope information. Maybe this already works if there are conflicting declarations as with other scopes?
That would be something I wouldn't know about. The compiler thinks case statements shadow variables so it doesn't seem to be aware or treat case statements as their own scope. If the debugger can determine it is in a switch statement and get the line info of the cases then it could brute force it but this sounds like to much work for something that probably could be done in a few lines of code.
Dec 20 2018