www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Ambiguous mangling of symbols declared in nested scopes

reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
Today I was investigating this bug:

	https://issues.dlang.org/show_bug.cgi?id=10619

and found that the problem appears to be an ambiguous mangling of local
variables declared in nested scopes. For example, given this code:

	import std.stdio;
	void myFunc(alias Sym)()
	{
		writeln(Sym);
	}
	void main()
	{
		{
			{
				int x = 789;
				myFunc!x();
			}
	
			int x = 456;
			myFunc!x();
		}
	
		int x = 123;
		myFunc!x();
	}

The expected output is:

	789
	456
	123

However, the actual output is:

	789
	789
	789

Checking the compiler output, myFunc is only instantiated once, with the
mangled name:

	_D4test31__T6myFuncS17_D4test4mainFZ1xiZ6myFuncMFZv

This name is ambiguous, since there is no way to distinguish between the
three different instances of 'x' in main(). So it seems that the
compiler conflates them into a single function, even though they should
be 3 distinct ones.

Interestingly enough, moving the nested blocks to after the outer
block's 'x' is declared will trigger a variable shadowing error, but in
the above code, there is actually no shadowing since the inner 'x' is
already out of scope by the time the outer 'x' is declared, so it's
ostensibly legal code.

The question then is, how should we fix this bug? Outlaw the above code?
Or change the mangling scheme to uniquely identify local variables
declared in nested blocks?


P.S. Hmph, looks like we're going to have to go with the second route,
because the following clearly-legal code is broken:

	import std.stdio;
	
	void myFunc(alias Sym)()
	{
		pragma(msg, myFunc.mangleof);
		writeln(Sym);
	}
	
	void main()
	{
		foreach (i; 0..3) {
			myFunc!i();
		}
	
		foreach (i; 5..8) {
			myFunc!i();
		}
	}

Output:

	0
	1
	2
	2
	2
	2

Notice how the second invocation of myFunc appears to reference the
first, out-of-scope, 'i'.  Moreover, 'i' is local to both scopes, so we
*can't* outlaw this usage!

So looks like we'll have to change the mangling scheme of templates with
alias arguments to local variables.


T

-- 
"No, John.  I want formats that are actually useful, rather than
over-featured megaliths that address all questions by piling on
ridiculous internal links in forms which are hideously over-complex." --
Simon St. Laurent on xml-dev
Jul 16 2014
next sibling parent reply "Ivan Kazmenko" <gassa mail.ru> writes:
On Wednesday, 16 July 2014 at 17:24:27 UTC, H. S. Teoh via 
Digitalmars-d wrote:
 Today I was investigating this bug:

 https://issues.dlang.org/show_bug.cgi?id=10619

 and found that the problem appears to be an ambiguous mangling 
 of local variables declared in nested scopes. <...>
Amazing how such a bug survived until now! Can all (meaningful) scopes of a module be numbered internally to distinguish them? Some way, like lambdas are. Or is it too much of a change? If they can, the scope's unique ID can then go into the mangled name. Ivan Kazmenko.
Jul 16 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jul 16, 2014 at 05:36:45PM +0000, Ivan Kazmenko via Digitalmars-d wrote:
 On Wednesday, 16 July 2014 at 17:24:27 UTC, H. S. Teoh via Digitalmars-d
 wrote:
Today I was investigating this bug:

https://issues.dlang.org/show_bug.cgi?id=10619

and found that the problem appears to be an ambiguous mangling of
local variables declared in nested scopes. <...>
Amazing how such a bug survived until now!
Well, I found out about it last year and filed a bug, but it seems to have escaped everyone's attention. Prior to that, a friend of mine who occasionally dabbles in D has seen the problem, I don't know how long ago. It makes me wonder if others may have seen it too, but only haven't spoken up about it. :-/ In any case, I've been wanting to familiarize myself more with dmd code, so this is as good an opportunity as any other. Plus, I'm tired of waiting around for somebody more familiar with dmd to take notice, so I decided that I'm gonna do something about it myself, even if it means taking longer 'cos right now I've no idea what I'm doing. :-P
 Can all (meaningful) scopes of a module be numbered internally to
 distinguish them?  Some way, like lambdas are.  Or is it too much of a
 change?
 If they can, the scope's unique ID can then go into the mangled name.
[...] Ultimately we'll have to do something like that, I think, since I can't think of any other way to disambiguate between all these local variables. But maybe some of the DMD experts can speak up about this. ;-) T -- Freedom: (n.) Man's self-given right to be enslaved by his own depravity.
Jul 16 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"H. S. Teoh via Digitalmars-d"  wrote in message 
news:mailman.4230.1405534833.2907.digitalmars-d puremagic.com...

 But maybe some of the DMD experts can speak up about this. ;-)
This is fairly well known, the same thing exists for static variables and nested functions etc. I think there is even a bounty on one of those bugs. The solution is also fairly straightforward - give each local declaration a unique name (for mangling only). It probably requires some minor structural changes in dmd.
Jul 17 2014
parent reply "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 17 July 2014 at 10:03:08 UTC, Daniel Murphy wrote:
 The solution is also fairly straightforward - give each local 
 declaration a unique name (for mangling only).  It probably 
 requires some minor structural changes in dmd.
As an aside, these names should be independent of import/instantiation order for things like forceinline to work properly (see https://github.com/jerro/ldc/compare/inlining, last commit). David
Jul 17 2014
parent "Sean Kelly" <sean invisibleduck.org> writes:
I opened a number of bugs regarding ambiguities while writing 
core.demangle. They're all tagged with some searchable keyword 
too, though I can't recall what it is.
Jul 17 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
Slight correction : it is not a mangling of variable that is a 
problem (local variables don't have any mangling) but mangling of 
the function itself (it naively uses variable name for mangled 
name generation). Should use something like unique id instead.
Jul 17 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 17, 2014 at 10:15:56AM +0000, Dicebot via Digitalmars-d wrote:
 Slight correction : it is not a mangling of variable that is a problem
 (local variables don't have any mangling) but mangling of the function
 itself (it naively uses variable name for mangled name generation).
 Should use something like unique id instead.
Hmm. How to generate a unique id? It must be module-wide, but not program-wide, because otherwise you get a different mangling depending on the order of imports. It must not increment within the same scope: void myFunc(alias A)() { ... } void main() { { int i; // All 3 instances below should refer to the // same instantiation of myFunc. myFunc!i(); myFunc!i(); myFunc!i(); } { int i, j; myFunc!i(); // but this one should be different { myFunc!i(); // this one should be same as previous line myFunc!j(); // this one should be different } myFunc!j(); // same as previous line } Also, it should resolve ambiguities between identically-named inner functions declared in nested scopes: void delegate() dg, dg2; { void inner() { ... } myFunc!inner(); myFunc!inner(); // same as previous line dg = &inner; } { void inner() { ... } myFunc!inner(); // different from previous scope dg2 = &inner; } assert(dg !is dg2); } So looks like we need to index scopes inside function bodies, perhaps by number of scopes in preceding lexical position, and any inner function's mangled name should somehow incorporate this index. Alias parameters should also incorporate this index in the mangling of the resulting template. T -- It only takes one twig to burn down a forest.
Jul 17 2014
next sibling parent "Dicebot" <public dicebot.lv> writes:
Something like reduced hash from mangled name of parent scope + 
incremental ID for each new stack instance.
Jul 17 2014
prev sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"H. S. Teoh via Digitalmars-d"  wrote in message 
news:mailman.4366.1405642929.2907.digitalmars-d puremagic.com...

 Hmm. How to generate a unique id? It must be module-wide, but not
 program-wide, because otherwise you get a different mangling depending
 on the order of imports. It must not increment within the same scope:
An easy option is to just have a counter inside the enclosing fd and increment it each time a local is created. In D I'd do it with an int[string] to track each name independently, but it's probably not worth the trouble in C++. These mangled names are not externally visible and do not need to be stable, just unique.
Jul 17 2014