www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Ambiguous mangling of symbols declared in nested scopes

reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
Today I was investigating this bug:

	https://issues.dlang.org/show_bug.cgi?id=10619

and found that the problem appears to be an ambiguous mangling of local
variables declared in nested scopes. For example, given this code:

	import std.stdio;
	void myFunc(alias Sym)()
	{
		writeln(Sym);
	}
	void main()
	{
		{
			{
				int x = 789;
				myFunc!x();
			}
	
			int x = 456;
			myFunc!x();
		}
	
		int x = 123;
		myFunc!x();
	}

The expected output is:

	789
	456
	123

However, the actual output is:

	789
	789
	789

Checking the compiler output, myFunc is only instantiated once, with the
mangled name:

	_D4test31__T6myFuncS17_D4test4mainFZ1xiZ6myFuncMFZv

This name is ambiguous, since there is no way to distinguish between the
three different instances of 'x' in main(). So it seems that the
compiler conflates them into a single function, even though they should
be 3 distinct ones.

Interestingly enough, moving the nested blocks to after the outer
block's 'x' is declared will trigger a variable shadowing error, but in
the above code, there is actually no shadowing since the inner 'x' is
already out of scope by the time the outer 'x' is declared, so it's
ostensibly legal code.

The question then is, how should we fix this bug? Outlaw the above code?
Or change the mangling scheme to uniquely identify local variables
declared in nested blocks?


P.S. Hmph, looks like we're going to have to go with the second route,
because the following clearly-legal code is broken:

	import std.stdio;
	
	void myFunc(alias Sym)()
	{
		pragma(msg, myFunc.mangleof);
		writeln(Sym);
	}
	
	void main()
	{
		foreach (i; 0..3) {
			myFunc!i();
		}
	
		foreach (i; 5..8) {
			myFunc!i();
		}
	}

Output:

	0
	1
	2
	2
	2
	2

Notice how the second invocation of myFunc appears to reference the
first, out-of-scope, 'i'.  Moreover, 'i' is local to both scopes, so we
*can't* outlaw this usage!

So looks like we'll have to change the mangling scheme of templates with
alias arguments to local variables.


T

-- 
"No, John.  I want formats that are actually useful, rather than
over-featured megaliths that address all questions by piling on
ridiculous internal links in forms which are hideously over-complex." --
Simon St. Laurent on xml-dev
Jul 16 2014
next sibling parent reply "Ivan Kazmenko" <gassa mail.ru> writes:
On Wednesday, 16 July 2014 at 17:24:27 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 Today I was investigating this bug:

 https://issues.dlang.org/show_bug.cgi?id=10619

 and found that the problem appears to be an ambiguous mangling 
 of local variables declared in nested scopes. <...>


Amazing how such a bug survived until now!

Can all (meaningful) scopes of a module be numbered internally to 
distinguish them?  Some way, like lambdas are.  Or is it too much 
of a change?

If they can, the scope's unique ID can then go into the mangled 
name.

Ivan Kazmenko.
Jul 16 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jul 16, 2014 at 05:36:45PM +0000, Ivan Kazmenko via Digitalmars-d wrote:

 On Wednesday, 16 July 2014 at 17:24:27 UTC, H. S. Teoh via Digitalmars-d
 wrote:

Today I was investigating this bug:

https://issues.dlang.org/show_bug.cgi?id=10619

and found that the problem appears to be an ambiguous mangling of
local variables declared in nested scopes. <...>


 
 Amazing how such a bug survived until now!


Well, I found out about it last year and filed a bug, but it seems to
have escaped everyone's attention. Prior to that, a friend of mine who
occasionally dabbles in D has seen the problem, I don't know how long
ago. It makes me wonder if others may have seen it too, but only haven't
spoken up about it. :-/

In any case, I've been wanting to familiarize myself more with dmd code,
so this is as good an opportunity as any other. Plus, I'm tired of
waiting around for somebody more familiar with dmd to take notice, so I
decided that I'm gonna do something about it myself, even if it means
taking longer 'cos right now I've no idea what I'm doing. :-P



 Can all (meaningful) scopes of a module be numbered internally to
 distinguish them?  Some way, like lambdas are.  Or is it too much of a
 change?
 If they can, the scope's unique ID can then go into the mangled name.


[...]

Ultimately we'll have to do something like that, I think, since I can't
think of any other way to disambiguate between all these local
variables.

But maybe some of the DMD experts can speak up about this. ;-)


T

-- 
Freedom: (n.) Man's self-given right to be enslaved by his own depravity.
Jul 16 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"H. S. Teoh via Digitalmars-d"  wrote in message 
news:mailman.4230.1405534833.2907.digitalmars-d puremagic.com...


 But maybe some of the DMD experts can speak up about this. ;-)


This is fairly well known, the same thing exists for static variables and 
nested functions etc.  I think there is even a bounty on one of those bugs.

The solution is also fairly straightforward - give each local declaration a 
unique name (for mangling only).  It probably requires some minor structural 
changes in dmd.
Jul 17 2014
parent reply "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 17 July 2014 at 10:03:08 UTC, Daniel Murphy wrote:

 The solution is also fairly straightforward - give each local 
 declaration a unique name (for mangling only).  It probably 
 requires some minor structural changes in dmd.


As an aside, these names should be independent of 
import/instantiation order for things like forceinline to work 
properly (see https://github.com/jerro/ldc/compare/inlining, last 
commit).

David
Jul 17 2014
parent "Sean Kelly" <sean invisibleduck.org> writes:
I opened a number of bugs regarding ambiguities while writing 
core.demangle. They're all tagged with some searchable keyword 
too, though I can't recall what it is.
Jul 17 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
Slight correction : it is not a mangling of variable that is a 
problem (local variables don't have any mangling) but mangling of 
the function itself (it naively uses variable name for mangled 
name generation). Should use something like unique id instead.
Jul 17 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 17, 2014 at 10:15:56AM +0000, Dicebot via Digitalmars-d wrote:

 Slight correction : it is not a mangling of variable that is a problem
 (local variables don't have any mangling) but mangling of the function
 itself (it naively uses variable name for mangled name generation).
 Should use something like unique id instead.


Hmm. How to generate a unique id? It must be module-wide, but not
program-wide, because otherwise you get a different mangling depending
on the order of imports. It must not increment within the same scope:

	void myFunc(alias A)() { ... }
	void main() {
		{
			int i;

			// All 3 instances below should refer to the
			// same instantiation of myFunc.
			myFunc!i();
			myFunc!i();
			myFunc!i();
		}
		{
			int i, j;
			myFunc!i(); // but this one should be different
			{
				myFunc!i(); // this one should be same as previous line
				myFunc!j(); // this one should be different
			}
			myFunc!j(); // same as previous line
		}

Also, it should resolve ambiguities between identically-named inner
functions declared in nested scopes:

		void delegate() dg, dg2;
		{
			void inner() { ... }
			myFunc!inner();
			myFunc!inner(); // same as previous line
			dg = &inner;
		}
		{
			void inner() { ... }
			myFunc!inner(); // different from previous scope
			dg2 = &inner;
		}
		assert(dg !is dg2);
	}

So looks like we need to index scopes inside function bodies, perhaps by
number of scopes in preceding lexical position, and any inner function's
mangled name should somehow incorporate this index. Alias parameters
should also incorporate this index in the mangling of the resulting
template.


T

-- 
It only takes one twig to burn down a forest.
Jul 17 2014
next sibling parent "Dicebot" <public dicebot.lv> writes:
Something like reduced hash from mangled name of parent scope + 
incremental ID for each new stack instance.
Jul 17 2014
prev sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"H. S. Teoh via Digitalmars-d"  wrote in message 
news:mailman.4366.1405642929.2907.digitalmars-d puremagic.com...


 Hmm. How to generate a unique id? It must be module-wide, but not
 program-wide, because otherwise you get a different mangling depending
 on the order of imports. It must not increment within the same scope:


An easy option is to just have a counter inside the enclosing fd and 
increment it each time a local is created.  In D I'd do it with an 
int[string] to track each name independently, but it's probably not worth 
the trouble in C++.  These mangled names are not externally visible and do 
not need to be stable, just unique.
Jul 17 2014