www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - module madness

reply John Stoneham <captnjameskirk moc.oohay> writes:
I hope I'm not retreading old ground. I've been browsing the archives 
for a while on this issue and I haven't seen this particular problem 
addressed. If I've overlooked it, I apologize.

------begin stuff.d-----------
module stuff;

private import std.stdio;

[...]
-------end stuff.d-----------

-------begin main.d----------
private import stuff.d

void main()
{
   std.stdio.writefln("Hello crazy world!");
}
--------end main.d-----------

Correct me if I'm wrong, but doesn't "private import std.stdio;" in 
stuff.d make the import of std.stdio private to that module? That *is* 
the way it's supposed to work, right? So how am I able to reference 
std.stdio in the above code without an error?

This behavior led me to a somewhat frustrating experience. I'm working 
on a project that has about 10 modules. One of these modules contains 
global variables (not classes) that several other modules need (and also 
need to sometimes change). A couple of those modules also reference each 
other, and every import in every module is "private import".

However, in the main program file I was qualifying the globals as 
globals.whatever and didn't realize that I hadn't "private import"ed the 
globals module there at all; but since another module had imported it 
and that module was imported in the main file, I could access it using 
qualified notation.

And now for the really weird part: When the main file doesn't import the 
globals module, some of the other modules which "private import" it 
complain that it now conflicts with the "private import" of that globals 
module in another module. But it shouldn't conflict at all! All imports 
are "private import" and there are no name clashes in any of the modules 
in any case. And simply importing the globals module in the main file 
causes the other conflicts to be resolved somehow, although no other 
change was made to any module. I can give a small example using 3 very 
short modules if my explanation here wasn't clear.

This behavior makes no sense to me! Can someone please explain why this 
is happening? I realize that the private/public module namespace issue 
has been done to death, but I thought I understood it until this problem.

--John
Feb 10 2006
next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
John Stoneham wrote:


 I hope I'm not retreading old ground. I've been browsing the archives
 for a while on this issue and I haven't seen this particular problem
 addressed. If I've overlooked it, I apologize.
 
 ------begin stuff.d-----------
 module stuff;
 
 private import std.stdio;
 
 [...]
 -------end stuff.d-----------
 
 -------begin main.d----------
 private import stuff.d
 
 void main()
 {
    std.stdio.writefln("Hello crazy world!");
 }
 --------end main.d-----------
 
 Correct me if I'm wrong, but doesn't "private import std.stdio;" in
 stuff.d make the import of std.stdio private to that module? That *is*
 the way it's supposed to work, right? So how am I able to reference
 std.stdio in the above code without an error?
 
 This behavior led me to a somewhat frustrating experience. I'm working
 on a project that has about 10 modules. One of these modules contains
 global variables (not classes) that several other modules need (and also
 need to sometimes change). A couple of those modules also reference each
 other, and every import in every module is "private import".
 
 However, in the main program file I was qualifying the globals as
 globals.whatever and didn't realize that I hadn't "private import"ed the
 globals module there at all; but since another module had imported it
 and that module was imported in the main file, I could access it using
 qualified notation.
 
 And now for the really weird part: When the main file doesn't import the
 globals module, some of the other modules which "private import" it
 complain that it now conflicts with the "private import" of that globals
 module in another module. But it shouldn't conflict at all! All imports
 are "private import" and there are no name clashes in any of the modules
 in any case. And simply importing the globals module in the main file
 causes the other conflicts to be resolved somehow, although no other
 change was made to any module. I can give a small example using 3 very
 short modules if my explanation here wasn't clear.
 
 This behavior makes no sense to me! Can someone please explain why this
 is happening? I realize that the private/public module namespace issue
 has been done to death, but I thought I understood it until this problem.
 
 --John


Sounds like something I reported with testcases 2 years ago :P Don't
remember the details anymore though.
Feb 10 2006
prev sibling parent reply John Stoneham <captnjameskirk moc.oohay> writes:
There is definitely a bug in the module namespace parsing. The code 
below demonstrates the bug, which is first evident as being able to 
access a module's variables/functions even though it hasn't been 
imported. In the code below 3 modules are defined and used in the main 
program. The first module, notes, simply declares an enum which is used 
by the other modules. The second, printer, contains one function which 
returns a string representation of the enum. The third, compose, 
contains one function which combines 3 of the string representations of 
the enum into one longer string, using the function defined in printer 
to return a string. (Don't worry about whether this is "good code", I 
just threw it together as a quick example to demonstrate the bug.)

In the main file, 3 lines are commented out and numbered with a comment 
at the end of the line. When the 3 lines are commented out, the program 
compiles fine, even though in main.d there is no import of notes and the 
enum defined there is accessed using qualified notation.

THIS IS A BUG AND SHOULD BE GENERATE AN ERROR WHEN COMPILED. According 
the documentation:

     If the import is private, such as:

     module abc;
     private import def;

     then def is not searched when another module imports abc.

So when main imports compose, notes should not be searched (according to 
the spec) and main should know NOTHING about the definitions in notes. 
Trying to access a name local to notes when notes is not imported should 
generate a compile-time error.

The problem exposes itself further when the lines (1) and (3) are 
uncommented. When this is done, the compiler generates an error stating 
"compose.d(4): import compose.notes conflicts with printer.notes at 
printer.d(3)"

THERE IS NO CONFLICT. Both modules use "private import" to import notes. 
When line 1 is uncommented and both compose and printer are imported 
into main, each one has "private import notes" and the compiler should 
not "search" notes twice. If it is not an error to allow qualified 
notation to access a sub-module privately-imported by an imported module 
(although it should be), then there is a bug in the way the compiler 
parses the module namespaces and claiming a conflict which does not exist.

When line 2 is uncommented the program compiles fine. Notice that no 
change was made to any of the modules to fix the compile errors once 
they appeared. The only change was importing into main the module that 
the other modules also import.

-----begin notes.d-----
module notes;

enum { Do, Re, Mi, Fa, So, La, Ti }
------end notes.d------

-----begin printer.d-----
module printer;

private import notes;

char[] print_note(uint a)
{
  static char[][7] notes_str = ["Do", "Re", "Mi", "Fa",
                                "So", "La", "Ti"];
  // range check omitted
   return notes_str[a];
}
------end printer.d------

-----begin compose.d-----
module compose;

private import printer;
private import notes;

char[] chord(uint a, uint b, uint c)
{
   if (a <= Ti && b <= Ti && c <= Ti)
     return print_note(a) ~ "+" ~ print_note(b) ~ "+" ~ print_note(c);
   else
     return "";
}
------end compose.d------

-----begin main.d-----
import std.stdio;

private import compose;
//private import printer; // (1)
//private import notes; // (2)

void main()
{
   writefln("chord of Do+Mi+So = %s",
            chord(notes.Do, notes.Mi, notes.So));
   //writefln("note %d = %s", notes.Mi, print_note(notes.Mi)); // (3)
}
------end main.d------
Feb 12 2006
parent reply =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= <jmjmak utu.fi.invalid> writes:
John Stoneham wrote:

 There is definitely a bug in the module namespace parsing. The code
 below demonstrates the bug, which is first evident as being able to
 access a module's variables/functions even though it hasn't been
 imported. In the code below 3 modules are defined and used in the main
 program. The first module, notes, simply declares an enum which is used
 by the other modules. The second, printer, contains one function which
 returns a string representation of the enum. The third, compose,
 contains one function which combines 3 of the string representations of
 the enum into one longer string, using the function defined in printer
 to return a string. (Don't worry about whether this is "good code", I
 just threw it together as a quick example to demonstrate the bug.)
 


<snip>

Thank you John for the "practical" example. This is the biggest problem
I've had with D since my very first "big" project in 2003. I like the
way it is written in the docs, but this implementation is terribly
broken. It's practically impossible to create a project with >= 3
modules without using some quirks with imports (to avoid namespace
conflicts). I think it shouldn't be allowed to access privately imported
module members at all.

Of course it improves flexibility to have access to every possible
symbol everywhere in the program, but what's the point with visibility
keywords then? Walter, can you please finally fix this?

-- 
Jari-Matti
Feb 15 2006
parent =?ISO-8859-1?Q?Jari-Matti_M=E4kel=E4?= <jmjmak utu.fi.invalid> writes:
Jari-Matti Mäkelä wrote:

 John Stoneham wrote:

 There is definitely a bug in the module namespace parsing. The code
 below demonstrates the bug, which is first evident as being able to
 access a module's variables/functions even though it hasn't been
 imported. In the code below 3 modules are defined and used in the main
 program. The first module, notes, simply declares an enum which is used
 by the other modules. The second, printer, contains one function which
 returns a string representation of the enum. The third, compose,
 contains one function which combines 3 of the string representations of
 the enum into one longer string, using the function defined in printer
 to return a string. (Don't worry about whether this is "good code", I
 just threw it together as a quick example to demonstrate the bug.)


 <snip>
 
 Thank you John for the "practical" example. This is the biggest problem
 I've had with D since my very first "big" project in 2003. I like the
 way it is written in the docs, but this implementation is terribly
 broken. It's practically impossible to create a project with >= 3
 modules without using some quirks with imports (to avoid namespace
 conflicts). I think it shouldn't be allowed to access privately imported
 module members at all.
 
 Of course it improves flexibility to have access to every possible
 symbol everywhere in the program, but what's the point with visibility
 keywords then? Walter, can you please finally fix this?
 


Here's a solution proposal:

I think the problem is that the compiler only keeps track of the
"latest" module in the import chain. If the same symbol is imported
using >= 2 different paths, the compiler regards these symbols as
completely separate. Is there any possibility to include some sort of
additional information that tells the "origin" of the symbol so that the
compiler would be able to merge these identical symbols as one.

"origin" = where the symbol was last declared/(re)implemented in one
import chain

The only chance of conflict I can think of is when the symbol contents
are "tampered" along the way and two symbols with different
implementations are introduced in the same module. Should we use some
kind of simplified c++ multiple inheritance rules here then? Simplified,
because D modules cannot inherit.

I think this would be an elegant solution since it would not only allow
the compiler to "pollute" the symbol namespace with private symbols and
thus giving more precise error messages than C/C++ with the help of the
symbol table, but also prevent conflicts with these "intelligent"
merging rules.

-- 
Jari-Matti
Feb 16 2006