www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Identifier resolution, the great implementation defined mess.

reply "deadalnix" <deadalnix gmail.com> writes:
So, as I implement more and more features of D, the question of 
identifier resolution pops up more and more, to the point it 
starts to become a problem.

Each feature that introduce new lookup is usually defined, but 
there is no definition of what are the priorities in between 
them, and how the should interact.

When resolving an identifier in an object, if local lookups 
fails, here are the possible fallback places where you look into:
  - parent class/interface (1)
  - alias this (1)
  - __outer (1)
  - UFCS (2)

That is for local symbols. This get even more tricky you put into 
the equation that one most likely want to register symbol from 
parent class/interface into the child scope as you wouldn't want 
an overridden method to resolve differently than a a non override 
one, but not other symbols.

Now let's consider the free symbol scenario. It can fallback in 
various ways as well:
  - this (1)
  - context (2)
  - with statement (2)
  - import (2)

And as for local symbols, fallback can recursively fallback into 
each others.

Here are the possible fallback:
  1 - parent class/interface
    - alias this
    - __outer
  2 - context
    - with statement
    - import
    - this

Sounds like we need to define some priorities here.
Sep 16 2014
parent reply "Dicebot" <public dicebot.lv> writes:
I had impression that general rule is "most inner scope takes 
priority" (with base classes being one "imaginary" scope above 
the current one). Are there any actual inconsistencies you have 
noticed or it just a matter of lacking matching spec entry?
Sep 17 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 17 September 2014 at 16:25:57 UTC, Dicebot wrote:

 I had impression that general rule is "most inner scope takes 
 priority" (with base classes being one "imaginary" scope above 
 the current one). Are there any actual inconsistencies you have 
 noticed or it just a matter of lacking matching spec entry?


There is no inconsistencies because there is no spec.
Sep 17 2014
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Wednesday, 17 September 2014 at 22:42:27 UTC, deadalnix wrote:

 On Wednesday, 17 September 2014 at 16:25:57 UTC, Dicebot wrote:

 I had impression that general rule is "most inner scope takes 
 priority" (with base classes being one "imaginary" scope above 
 the current one). Are there any actual inconsistencies you 
 have noticed or it just a matter of lacking matching spec 
 entry?


 There is no inconsistencies because there is no spec.


Maybe in this case it is best to just look at what dmd does and 
add that to the spec (assuming what dmd does is sound, and makes 
sense).
Sep 18 2014
next sibling parent reply "Brian Schott" <briancschott gmail.com> writes:
On Thursday, 18 September 2014 at 21:31:26 UTC, Peter Alexander 
wrote:

 Maybe in this case


And in every case. DMD's behavior is correct because it 
consistent with DMD.
Sep 18 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/19/2014 12:06 AM, Brian Schott wrote:

 On Thursday, 18 September 2014 at 21:31:26 UTC, Peter Alexander wrote:

 Maybe in this case


 And in every case. DMD's behavior is correct because it consistent with
 DMD.


???
Sep 18 2014
parent "Brian Rogoff" <brogoff gmail.com> writes:
On Thursday, 18 September 2014 at 23:14:41 UTC, Timon Gehr wrote:

 On 09/19/2014 12:06 AM, Brian Schott wrote:

 On Thursday, 18 September 2014 at 21:31:26 UTC, Peter 
 Alexander wrote:

 Maybe in this case


 And in every case. DMD's behavior is correct because it 
 consistent with
 DMD.


 ???


http://en.wikipedia.org/wiki/Gallows_humor
Sep 18 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 18 September 2014 at 21:31:26 UTC, Peter Alexander
wrote:

 On Wednesday, 17 September 2014 at 22:42:27 UTC, deadalnix 
 wrote:

 On Wednesday, 17 September 2014 at 16:25:57 UTC, Dicebot wrote:

 I had impression that general rule is "most inner scope takes 
 priority" (with base classes being one "imaginary" scope 
 above the current one). Are there any actual inconsistencies 
 you have noticed or it just a matter of lacking matching spec 
 entry?


 There is no inconsistencies because there is no spec.


 Maybe in this case it is best to just look at what dmd does and 
 add that to the spec (assuming what dmd does is sound, and 
 makes sense).


Yeah this is exactly what I was asking about. I assumed that
deadlnix has done some research about it and found some specific
inconsistencies / issues - after all, it is not the only
implementation-defined feature he must have encountered :)
Sep 19 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Friday, 19 September 2014 at 10:59:24 UTC, Dicebot wrote:

 Yeah this is exactly what I was asking about. I assumed that
 deadlnix has done some research about it and found some specific
 inconsistencies / issues - after all, it is not the only
 implementation-defined feature he must have encountered :)


DMD does very bizarre things. I think I should write a DIP, but 
time is always running low...

Free goodie: when you import, all symbol are resolved via the 
expected import resolution mechanism. All ? No, the root package 
is imported in the local scope.

foo(int a) {
   import a.b.c;
   // a is now a package and not the parameter a anymore.
}
Sep 20 2014
next sibling parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com> writes:
Am 21.09.2014 07:29, schrieb deadalnix:

 On Friday, 19 September 2014 at 10:59:24 UTC, Dicebot wrote:

 Yeah this is exactly what I was asking about. I assumed that
 deadlnix has done some research about it and found some specific
 inconsistencies / issues - after all, it is not the only
 implementation-defined feature he must have encountered :)


 DMD does very bizarre things. I think I should write a DIP, but time is
 always running low...

 Free goodie: when you import, all symbol are resolved via the expected
 import resolution mechanism. All ? No, the root package is imported in
 the local scope.

 foo(int a) {
    import a.b.c;
    // a is now a package and not the parameter a anymore.
 }


There are also funny little things like this one:
http://dpaste.dzfl.pl/aac84d5ffae8
Sep 20 2014
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 21 September 2014 at 06:41:25 UTC, Sönke Ludwig wrote:

 There are also funny little things like this one:
 http://dpaste.dzfl.pl/aac84d5ffae8


HAHAHAHA, that is retarded XD
Sep 20 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sun, Sep 21, 2014 at 06:49:22AM +0000, deadalnix via Digitalmars-d wrote:

 On Sunday, 21 September 2014 at 06:41:25 UTC, Sönke Ludwig wrote:

There are also funny little things like this one:
http://dpaste.dzfl.pl/aac84d5ffae8


 
 HAHAHAHA, that is retarded XD


Wow, that is truly messed up. XD  That's a perfect WAT candidate. :D


T

-- 
Ignorance is bliss... until you suffer the consequences!
Sep 21 2014
prev sibling parent Daniel Kozak via Digitalmars-d <digitalmars-d puremagic.com> writes:
V Sun, 21 Sep 2014 08:41:26 +0200
Sönke Ludwig via Digitalmars-d <digitalmars-d puremagic.com> napsáno:


 Am 21.09.2014 07:29, schrieb deadalnix:

 On Friday, 19 September 2014 at 10:59:24 UTC, Dicebot wrote:

 Yeah this is exactly what I was asking about. I assumed that
 deadlnix has done some research about it and found some specific
 inconsistencies / issues - after all, it is not the only
 implementation-defined feature he must have encountered :)


 DMD does very bizarre things. I think I should write a DIP, but
 time is always running low...

 Free goodie: when you import, all symbol are resolved via the
 expected import resolution mechanism. All ? No, the root package is
 imported in the local scope.

 foo(int a) {
    import a.b.c;
    // a is now a package and not the parameter a anymore.
 }


 
 There are also funny little things like this one:
 http://dpaste.dzfl.pl/aac84d5ffae8
 


Yep, this is reason why I do not use local imports for all symbols.
When I need local import of something, then I import just what I really
need with alias:

http://dpaste.dzfl.pl/0fe84e060f718
Sep 20 2014
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/21/2014 07:29 AM, deadalnix wrote:

 Free goodie: when you import, all symbol are resolved via the expected
 import resolution mechanism. All ? No, the root package is imported in
 the local scope.

 foo(int a) {
    import a.b.c;
    // a is now a package and not the parameter a anymore.
 }


For local imports, DMD imports _all_ symbols into the local scope, 
shadowing anything that was there, which is plain broken (as SÓ§nke's 
example shows). BTW: how do you suggest to treat the root package? I 
think importing into the local scope is fine, but the above example 
should emit an error because the parameter and the package conflict.

(The following code does not compile either:
int std;
import std.conv;)
Sep 21 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sun, Sep 21, 2014 at 02:55:47PM +0200, Timon Gehr via Digitalmars-d wrote:

 On 09/21/2014 07:29 AM, deadalnix wrote:

Free goodie: when you import, all symbol are resolved via the
expected import resolution mechanism. All ? No, the root package is
imported in the local scope.

foo(int a) {
   import a.b.c;
   // a is now a package and not the parameter a anymore.
}


 
 For local imports, DMD imports _all_ symbols into the local scope,
 shadowing anything that was there, which is plain broken (as SÓ§nke's
 example shows).  BTW: how do you suggest to treat the root package? I
 think importing into the local scope is fine, but the above example
 should emit an error because the parameter and the package conflict.
 
 (The following code does not compile either:
 int std;
 import std.conv;)


I also think it's OK to import symbols into the local scope, but I don't
think conflicts should cause errors immediately, only when you actually
try to reference an ambiguous symbol. I.e., this should work:

	string foo(string text) {
		import std.conv; // includes std.conv.text
		return "";	// but `text` is never referenced
	}

but this should emit an error:

	string foo(string text) {
		import std.conv; // includes std.conv.text
		return text;	// error: parameter `text` conflicts with std.conv.text
	}


T

-- 
In order to understand recursion you must first understand recursion.
Sep 21 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/21/2014 03:53 PM, H. S. Teoh via Digitalmars-d wrote:

 I.e., this should work:

 	string foo(string text) {
 		import std.conv; // includes std.conv.text
 		return "";	// but `text` is never referenced
 	}

 but this should emit an error:

 	string foo(string text) {
 		import std.conv; // includes std.conv.text
 		return text;	// error: parameter `text` conflicts with std.conv.text
 	}


Why? That's inconsistent with how imports work at module scope. In the 
second example, 'text' should refer to the parameter.
Sep 21 2014
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/21/14, 5:55 AM, Timon Gehr wrote:

 For local imports, DMD imports _all_ symbols into the local scope,
 shadowing anything that was there, which is plain broken (as SÓ§nke's
 example shows).


Has this been bugzillized yet? -- Andrei
Sep 21 2014
parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 21 September 2014 at 15:11:22 UTC, Andrei Alexandrescu
wrote:

 On 9/21/14, 5:55 AM, Timon Gehr wrote:

 For local imports, DMD imports _all_ symbols into the local 
 scope,
 shadowing anything that was there, which is plain broken (as 
 SÓ§nke's
 example shows).


 Has this been bugzillized yet? -- Andrei


AFAIR it is closely related to famous
https://issues.dlang.org/show_bug.cgi?id=314
Sep 21 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2014 5:55 AM, Timon Gehr wrote:

 For local imports, DMD imports _all_ symbols into the local scope, shadowing
 anything that was there, which is plain broken (as SÓ§nke's example shows).
BTW:
 how do you suggest to treat the root package? I think importing into the local
 scope is fine, but the above example should emit an error because the parameter
 and the package conflict.

 (The following code does not compile either:
 int std;
 import std.conv;)


Of course it shouldn't, any more than:

   int std;
   double std;

should compile.
Sep 21 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/21/2014 09:54 PM, Walter Bright wrote:

 On 9/21/2014 5:55 AM, Timon Gehr wrote:

 For local imports, DMD imports _all_ symbols into the local scope,
 shadowing
 anything that was there, which is plain broken (as SÓ§nke's example
 shows). BTW:
 how do you suggest to treat the root package? I think importing into
 the local
 scope is fine, but the above example should emit an error because the
 parameter
 and the package conflict.

 (The following code does not compile either:
 int std;
 import std.conv;)


 Of course it shouldn't, ...


(That was my point.)
Sep 21 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2014 12:58 PM, Timon Gehr wrote:

 On 09/21/2014 09:54 PM, Walter Bright wrote:

 On 9/21/2014 5:55 AM, Timon Gehr wrote:

 For local imports, DMD imports _all_ symbols into the local scope,
 shadowing
 anything that was there, which is plain broken (as SÓ§nke's example
 shows). BTW:
 how do you suggest to treat the root package? I think importing into
 the local
 scope is fine, but the above example should emit an error because the
 parameter
 and the package conflict.

 (The following code does not compile either:
 int std;
 import std.conv;)


 Of course it shouldn't, ...


 (That was my point.)


Parameters are not in the same scope as local variables.
Sep 21 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/21/2014 10:08 PM, Walter Bright wrote:

 On 9/21/2014 12:58 PM, Timon Gehr wrote:

 On 09/21/2014 09:54 PM, Walter Bright wrote:

 On 9/21/2014 5:55 AM, Timon Gehr wrote:

 For local imports, DMD imports _all_ symbols into the local scope,
 shadowing
 anything that was there, which is plain broken (as SÓ§nke's example
 shows). BTW:
 how do you suggest to treat the root package? I think importing into
 the local
 scope is fine, but the above example should emit an error because the
 parameter
 and the package conflict.

 (The following code does not compile either:
 int std;
 import std.conv;)


 Of course it shouldn't, ...


 (That was my point.)


 Parameters are not in the same scope as local variables.


I know, and you will know that it makes no practical difference since 
identifier shadowing is disallowed (deprecated) within a function.
Sep 21 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2014 2:33 PM, Timon Gehr wrote:

 On 09/21/2014 10:08 PM, Walter Bright wrote:

 Parameters are not in the same scope as local variables.


 I know, and you will know that it makes no practical difference since
identifier
 shadowing is disallowed (deprecated) within a function.


The difference does become apparent when imports are done.

Also, the parameter scope matters for things like template function constraints.
Sep 21 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/20/2014 10:29 PM, deadalnix wrote:

 DMD does very bizarre things. I think I should write a DIP, but time is always
 running low...

 Free goodie: when you import, all symbol are resolved via the expected import
 resolution mechanism. All ? No, the root package is imported in the local
scope.

 foo(int a) {
    import a.b.c;
    // a is now a package and not the parameter a anymore.
 }


What's bizarre about it? You declared a package symbol 'a' in the local scope 
with the import declaration.

The way imports (and mixin templates) work for symbol lookup is completely 
consistent.

You could reasonably argue that since package 'a' shadows parameter 'a' in the 
same way that this issues an error:

   foo(int a) {
     double a;
   }

but, again, there is nothing bizarre about the import name lookup.


Lookup rules are straightforward:

   scope is current scope
   do {
       look up name in scope
       if name is found, done!
       look up name in imports imported into scope
       if name is found, done!
       set scope to enclosing scope
    } while scope exists

I don't know what mental model people have for how lookups work, but the above 
algorithm is how it actually works.
Sep 21 2014
next sibling parent "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 21 September 2014 at 20:05:57 UTC, Walter Bright wrote:

 I don't know what mental model people have for how lookups 
 work, but the above algorithm is how it actually works.


My mental model for local imports is "it's the same as module 
level imports, except the symbols are only available in this 
scope". I wouldn't expect a module symbol to shadow a local 
symbol.
Sep 21 2014
prev sibling next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 21 Sep 2014 13:04:49 -0700
Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 I don't know what mental model people have for how lookups work, but
 the above algorithm is how it actually works.


i believe that people expect this:

  void foo (int a) {
    import a;
    a.bar(); // here we use 'a' module
    xyzzy(); // it's actually 'a.xyzzy', and 'a as module' is implicit
    writeln(a); // here we use 'int a' argument
  }

i.e. symbol resolver will try argument/local name first, and only if it
failed tries to search in module.

or, more complicated sample:

  struct A { int bar; int baz; }
  void foo (in A a) {
    import a;
    a.bar(); // KABOOM, conflicting names
    xyzzy(); // it's actually 'a.xyzzy', and 'a as module' is implicit
    writeln(a); // KABOOM (both module and arg are complex types
    writeln(a.baz); // it's ok until module 'a' doesn't have 'baz'
  }

i'm not saying that this is how things *must* work, but this is what
one excepts, i think.
Sep 21 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2014 2:17 PM, ketmar via Digitalmars-d wrote:

 On Sun, 21 Sep 2014 13:04:49 -0700
 Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 I don't know what mental model people have for how lookups work, but
 the above algorithm is how it actually works.


 i believe that people expect this:

    void foo (int a) {
      import a;
      a.bar(); // here we use 'a' module
      xyzzy(); // it's actually 'a.xyzzy', and 'a as module' is implicit
      writeln(a); // here we use 'int a' argument


Context dependent lookups? That's an awful lot more complex than the existing
rules.


    }

 i.e. symbol resolver will try argument/local name first, and only if it
 failed tries to search in module.


That's how it does work. It's just that parameters are in an enclosing scope.



 or, more complicated sample:

    struct A { int bar; int baz; }
    void foo (in A a) {
      import a;
      a.bar(); // KABOOM, conflicting names
      xyzzy(); // it's actually 'a.xyzzy', and 'a as module' is implicit
      writeln(a); // KABOOM (both module and arg are complex types
      writeln(a.baz); // it's ok until module 'a' doesn't have 'baz'
    }

 i'm not saying that this is how things *must* work, but this is what
 one excepts, i think.


I have no idea how to even write such rules, let alone what kind of error 
messages to generate when the user does it wrong.

I believe it is far better to have simple rules, easy to explain, and have a
few 
awkward edge cases than having a terribly complex setup with special cases that 
nobody understands.

For example, probably 3 people on the planet understand C++ overloading rules 
(pages and pages of trivia). The rest just try things at random until it
appears 
to work.
Sep 21 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 22 September 2014 at 06:05:42 UTC, Walter Bright wrote:

 On 9/21/2014 2:17 PM, ketmar via Digitalmars-d wrote:

 On Sun, 21 Sep 2014 13:04:49 -0700
 Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:


 I don't know what mental model people have for how lookups 
 work, but
 the above algorithm is how it actually works.


 i believe that people expect this:

   void foo (int a) {
     import a;
     a.bar(); // here we use 'a' module
     xyzzy(); // it's actually 'a.xyzzy', and 'a as module' is 
 implicit
     writeln(a); // here we use 'int a' argument


 Context dependent lookups? That's an awful lot more complex 
 than the existing rules.


   }

 i.e. symbol resolver will try argument/local name first, and 
 only if it
 failed tries to search in module.


 That's how it does work. It's just that parameters are in an 
 enclosing scope.



 or, more complicated sample:

   struct A { int bar; int baz; }
   void foo (in A a) {
     import a;
     a.bar(); // KABOOM, conflicting names
     xyzzy(); // it's actually 'a.xyzzy', and 'a as module' is 
 implicit
     writeln(a); // KABOOM (both module and arg are complex 
 types
     writeln(a.baz); // it's ok until module 'a' doesn't have 
 'baz'
   }

 i'm not saying that this is how things *must* work, but this 
 is what
 one excepts, i think.


 I have no idea how to even write such rules, let alone what 
 kind of error messages to generate when the user does it wrong.

 I believe it is far better to have simple rules, easy to 
 explain, and have a few awkward edge cases than having a 
 terribly complex setup with special cases that nobody 
 understands.

 For example, probably 3 people on the planet understand C++ 
 overloading rules (pages and pages of trivia). The rest just 
 try things at random until it appears to work.


+1

We should simply do a lookup for local symbol, and if that fail, 
imported symbols.

In that case, a should resolve as the parameter, all the time.
Sep 21 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2014 11:09 PM, deadalnix wrote:

 We should simply do a lookup for local symbol, and if that fail, imported
symbols.


That's what it does now, i.e. lookup in the current scope, and if that fails, 
look in imports, if that fails, go to the enclosing scope.


 In that case, a should resolve as the parameter, all the time.


Parameters are in an uplevel enclosing scope.
Sep 21 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 22 September 2014 at 06:59:14 UTC, Walter Bright wrote:

 On 9/21/2014 11:09 PM, deadalnix wrote:

 We should simply do a lookup for local symbol, and if that 
 fail, imported symbols.


 That's what it does now, i.e. lookup in the current scope, and 
 if that fails, look in imports, if that fails, go to the 
 enclosing scope.


Can't this be made depth first ? That would seem more sensible to 
me, and apparently to other in this thread. After all, it seems 
legitimate to resolve what is under your nose than what is 
imported (and with lazy imports, it may even allow the compiler 
to process some imports).


 In that case, a should resolve as the parameter, all the time.


 Parameters are in an uplevel enclosing scope.


Yes I know, there is good reason for that. But from the 
programmer perspective, not the implementer, that do not look 
right.
Sep 22 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/22/2014 12:02 AM, deadalnix wrote:

 On Monday, 22 September 2014 at 06:59:14 UTC, Walter Bright wrote:

 On 9/21/2014 11:09 PM, deadalnix wrote:

 We should simply do a lookup for local symbol, and if that fail, imported
 symbols.


 That's what it does now, i.e. lookup in the current scope, and if that fails,
 look in imports, if that fails, go to the enclosing scope.


 Can't this be made depth first ? That would seem more sensible to me, and
 apparently to other in this thread. After all, it seems legitimate to resolve
 what is under your nose than what is imported (and with lazy imports, it may
 even allow the compiler to process some imports).


It is depth first. It starts at the innermost scope, which is the current
scope. 
Somehow, we don't seem to be talking the same language :-(



 In that case, a should resolve as the parameter, all the time.


 Parameters are in an uplevel enclosing scope.


 Yes I know, there is good reason for that. But from the programmer perspective,
 not the implementer, that do not look right.


I don't know of a better rule.
Sep 22 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Monday, 22 September 2014 at 09:17:16 UTC, Walter Bright wrote:

 It is depth first. It starts at the innermost scope, which is 
 the current scope. Somehow, we don't seem to be talking the 
 same language :-(


Depth first in the sense, go from the inner to the outer scope 
and look for local symbols. If that fails, go from the inner to 
the outer and look for imported symbols.
Sep 22 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/22/2014 10:27 PM, deadalnix wrote:

 On Monday, 22 September 2014 at 09:17:16 UTC, Walter Bright wrote:

 It is depth first. It starts at the innermost scope, which is the
 current scope. Somehow, we don't seem to be talking the same language :-(


 Depth first in the sense, go from the inner to the outer scope and look
 for local symbols. If that fails, go from the inner to the outer and
 look for imported symbols.


That sounds almost right, but it still suffers from hijacking issues, 
because more nested (non-explicitly!) imported identifiers would hide 
less nested ones.

What about:

Go from nested to outer scopes and look for local symbols. If that 
fails, look up the symbol simultaneously in all modules that are 
imported in scopes enclosing the lookup.

I currently think this would be the most sane behaviour for imports. It 
would need to be determined what to do about template mixins.
Sep 22 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/22/2014 1:27 PM, deadalnix wrote:

 On Monday, 22 September 2014 at 09:17:16 UTC, Walter Bright wrote:

 It is depth first. It starts at the innermost scope, which is the current
 scope. Somehow, we don't seem to be talking the same language :-(


 Depth first in the sense, go from the inner to the outer scope and look for
 local symbols. If that fails, go from the inner to the outer and look for
 imported symbols.


I would find that to be surprising behavior, as I'd expect inner declarations
to 
override outer ones.
Sep 22 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, Sep 22, 2014 at 02:58:33PM -0700, Walter Bright via Digitalmars-d wrote:

 On 9/22/2014 1:27 PM, deadalnix wrote:

On Monday, 22 September 2014 at 09:17:16 UTC, Walter Bright wrote:

It is depth first. It starts at the innermost scope, which is the
current scope. Somehow, we don't seem to be talking the same
language :-(


Depth first in the sense, go from the inner to the outer scope and
look for local symbols. If that fails, go from the inner to the outer
and look for imported symbols.


 
 I would find that to be surprising behavior, as I'd expect inner
 declarations to override outer ones.


I think what makes the situation under discussion stick out like a sore
thumb, is the fact that a local import statement inserts symbols into
the current inner scope "implicitly". That is to say, when you write:

	struct S {
		int x;
		void method(int y) {
			foreach (z; 0..10) {
				import somemodule;
			}
		}
	}

the statement "import somemodule" can potentially pull in arbitrary
symbols into the inner scope, even though none of these symbols are
explicitly named in the code. If we were to write, instead:

	struct S {
		int x;
		void method(int y) {
			foreach (z; 0..10) {
				import somemodule : x, y, z;
			}
		}
	}

then it would not be surprising that x, y, z, refer to the symbols from
somemodule, rather than the loop variable, method parameter, or member
variable.

However, in the former case, the user is left at the mercy of somemodule
(keep in mind that this can be a 3rd party module over which the user
has little control) what symbols will be introduced into the inner
scope. If somemodule defines a symbol 'x', then it silently overrides
S.x in the inner scope. But since 'x' is implicit from the import
statement, a casual perusal of the code would give the wrong impression
that 'x' refers to S.x, whereas it actually refers to somemodule.x.

This problem is intrinsically the same problem exhibited by shadowing of
local variables:

	void func(int x) {
		int x;
		{
			int x;
			x++;
		}
	}

which is rejected by the compiler. Both result in code that is difficult
to reason about and error-prone because of symbol ambiguity.

Arguably, we should reject such import statements if it would introduce
this kind of symbol shadowing. But regardless of whether we decide to do
that or not, it's becoming clear that unqualified local imports are a
bad idea, and I, for one, will avoid using it for the above reasons.
Instead, I'd recommend only using qualified local imports:

	void func(int x, int z) {
		import submodule : x, y;
		// I explicitly want submodule.x to shadow parameter x;
		// I also name submodule.y so that in the event that
		// submodule actually contains a symbol 'z', it does not
		// accidentally shadow parameter z.
	}

leaving unqualified imports to the module global scope. In an ideal
world, the language would enforce this usage.


T

-- 
Life would be easier if I had the source code. -- YHL
Sep 22 2014
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Sunday, 21 September 2014 at 20:05:57 UTC, Walter Bright wrote:

 On 9/20/2014 10:29 PM, deadalnix wrote:

 DMD does very bizarre things. I think I should write a DIP, 
 but time is always
 running low...

 Free goodie: when you import, all symbol are resolved via the 
 expected import
 resolution mechanism. All ? No, the root package is imported 
 in the local scope.

 foo(int a) {
   import a.b.c;
   // a is now a package and not the parameter a anymore.
 }


 What's bizarre about it? You declared a package symbol 'a' in 
 the local scope with the import declaration.


Because this is not consistent with module level import and that 
is not consistent with how local shadowing works.

IMO you should fallback to import if local lookup failed.
Sep 21 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/22/2014 01:37 AM, deadalnix wrote:

 On Sunday, 21 September 2014 at 20:05:57 UTC, Walter Bright wrote:

 On 9/20/2014 10:29 PM, deadalnix wrote:

 DMD does very bizarre things. I think I should write a DIP, but time
 is always
 running low...

 Free goodie: when you import, all symbol are resolved via the
 expected import
 resolution mechanism. All ? No, the root package is imported in the
 local scope.

 foo(int a) {
   import a.b.c;
   // a is now a package and not the parameter a anymore.
 }


 What's bizarre about it? You declared a package symbol 'a' in the
 local scope with the import declaration.


 Because this is not consistent with module level import


Yes it is.


 and that is not consistent with how local shadowing works.


Indeed.


 IMO you should fallback to import if local lookup failed.


The package symbol is not imported, it is introduced by imports. This is 
true in module scopes and local scopes alike.
Sep 21 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2014 4:37 PM, deadalnix wrote:

 On Sunday, 21 September 2014 at 20:05:57 UTC, Walter Bright wrote:

 On 9/20/2014 10:29 PM, deadalnix wrote:

 DMD does very bizarre things. I think I should write a DIP, but time is always
 running low...

 Free goodie: when you import, all symbol are resolved via the expected import
 resolution mechanism. All ? No, the root package is imported in the local
scope.

 foo(int a) {
   import a.b.c;
   // a is now a package and not the parameter a anymore.
 }


 What's bizarre about it? You declared a package symbol 'a' in the local scope
 with the import declaration.


 Because this is not consistent with module level import


Yes, it is.


 and that is not consistent with how local shadowing works.


You can argue that 'a' should be an error to shadow the parameter 'a'. But 
extending this to the contents of the import is not consistent.



 IMO you should fallback to import if local lookup failed.


That's exactly how it does work. It's the SAME code that implements it.

Your misunderstanding appears to be that:

    foo(int a) { int b; }

'a' and 'b' are in the same scope. They are NOT in the same scope.
Sep 21 2014
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 21 Sep 2014 23:07:26 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:


 Your misunderstanding appears to be that:
 
     foo(int a) { int b; }
 
 'a' and 'b' are in the same scope. They are NOT in the same scope.


But quite understandable that people expect them to be in the
same scope, seeing as there is only one set of {}. Adding some
shadowing warnings should deal with that, so that the earlier
example with `text` being hijacked somehow errors out.

-- 
Marco
Sep 21 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2014 11:44 PM, Marco Leise wrote:

 But quite understandable that people expect them to be in the
 same scope, seeing as there is only one set of {}.


{ } introduce a new nested scope, they do not extend an existing one.



 Adding some
 shadowing warnings should deal with that, so that the earlier
 example with `text` being hijacked somehow errors out.


Are how the current lookup rules work (for better or worse) clear to you or are 
they still mysterious?
Sep 22 2014
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 22 Sep 2014 00:05:09 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:


 On 9/21/2014 11:44 PM, Marco Leise wrote:

 But quite understandable that people expect them to be in the
 same scope, seeing as there is only one set of {}.


 
 { } introduce a new nested scope, they do not extend an existing one.


Of course, but

void foo(int x)
{
	int y;
}

appears to the uninitiated as two variables that will only be
visible inside of foo. The notion of a sourrounding parameter
scope only comes up when you dig deep into the language. It
is almost safe to say, without reading the specs or the
compiler source one wouldn't know. (I didn't :) )
 

 Adding some
 shadowing warnings should deal with that, so that the earlier
 example with `text` being hijacked somehow errors out.


 
 Are how the current lookup rules work (for better or worse) clear to you or
are 
 they still mysterious?


Now that I understand that the parameter scope exists I
understand how the lookup happens.
I could say the same about the famous JavaScript WAT video.
Once you understand all the language rules it becomes sane.
That is exactly what a WAT is about: It is a logical
consequence of some hidden context inside the language that
yields totally unexpected results in innocent looking code.

-- 
Marco
Sep 23 2014
parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 23 September 2014 at 19:04:55 UTC, Marco Leise wrote:

 Of course, but

 void foo(int x)
 {
 	int y;
 }

 appears to the uninitiated as two variables that will only be
 visible inside of foo. The notion of a sourrounding parameter
 scope only comes up when you dig deep into the language. It
 is almost safe to say, without reading the specs or the
 compiler source one wouldn't know. (I didn't :) )
 


It is obvious when you think about it. Consider contract for
instance.


 Now that I understand that the parameter scope exists I
 understand how the lookup happens.
 I could say the same about the famous JavaScript WAT video.
 Once you understand all the language rules it becomes sane.
 That is exactly what a WAT is about: It is a logical
 consequence of some hidden context inside the language that
 yields totally unexpected results in innocent looking code.


Yes, there is obviously a reason. Nobody explicitly decided that
these construct would yield completely idiotic results. But they
do.
Sep 23 2014
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 21/09/14 22:04, Walter Bright wrote:


 Lookup rules are straightforward:

    scope is current scope
    do {
        look up name in scope
        if name is found, done!
        look up name in imports imported into scope
        if name is found, done!
        set scope to enclosing scope
     } while scope exists

 I don't know what mental model people have for how lookups work, but the
 above algorithm is how it actually works.


You better write down the scope rules as well. It gets complicated with 
base classes, template mixins and all features available in D.

-- 
/Jacob Carlborg
Sep 21 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/21/2014 11:36 PM, Jacob Carlborg wrote:

 You better write down the scope rules as well. It gets complicated with base
 classes, template mixins and all features available in D.


Sure, I had thought they were.

BTW, template mixins work exactly like imports. See "Mixin Scope" here:
https://dlang.org/template-mixin
Sep 22 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/22/14, 12:09 AM, Walter Bright wrote:

 On 9/21/2014 11:36 PM, Jacob Carlborg wrote:

 You better write down the scope rules as well. It gets complicated
 with base
 classes, template mixins and all features available in D.


 Sure, I had thought they were.

 BTW, template mixins work exactly like imports. See "Mixin Scope" here:
 https://dlang.org/template-mixin


D lookup rules are logical and relatively simple.

In the case of local imports however, there's definitely an element of 
surprise, and also an issue akin to hijacking. Consider:

void main(string[] args) {
    import some_module;
    ... use args ...
}

Let's assume module "some_module" does not define a name "args". All 
works fine.

Time goes by, some_module gets updated to also define the name "args". 
Now this application is recompiled and may actually compile 
successfully, but the semantics has changed - it doesn't use the "args" 
in the parameter list, but instead the symbol exported by some_module.

That's clearly something difficult to ignore. It's one of those cases in 
which logical and relatively simple doesn't fit the bill the same way 
that a pair of pants built out of simple shapes like cylinders and 
hemispheres won't be a good fit.

We must look into this.


Andrei
Sep 22 2014
parent "deadalnix" <deadalnix gmail.com> writes:
On Monday, 22 September 2014 at 08:06:11 UTC, Andrei Alexandrescu 
wrote:

 D lookup rules are logical and relatively simple.


If you look at my first post, you'll notice that the discussion 
so far touched only a fraction of the issue (and I forgot to 
mention opDispatch in there).

That being said, I'm fairly sure we can come up with something 
logical. The problem is right now, if put aside the surprise of 
the local import, that various methods of resolutions are defined 
independently, but not how they interact.
Sep 22 2014