www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [frustration, sorry] Import conflict madness.

reply Dawid =?UTF-8?B?Q2nEmcW8YXJraWV3aWN6?= <dawid.ciezarkiewicz gmail.com> writes:
You write your great soft in D. You're coding with smile because everything
in seems to be so good. You added few new files, new classes. Porting your
project from C++ to D seems to be so good idea. You type "make".

And then out of nowhere: BANG!!!

battle.d(31): import battle.map conflicts with cell.cell.map at
cell/cell.d(29)

GAME OVER!

You're stuck with this nothing telling you error message. You start
wondering what is wrong. Imports probably, but ... why? It takes 15 minutes
of frustrating checking your files. You seek help on #D irc channel (nice
people BTW) - but they can not help you. They can only tell "It might be
that in some unrelated file you are trying to use map without importing map
but are importing files that do import map" which is great help but still
will require you to spend great amount of time finding this place.

Please Walter - fix this. It is sooooooo frustrating and everyone confirms
that. It's something that all D people hits once or more.
Jun 17 2006
next sibling parent reply Deewiant <deewiant.doesnotlike.spam gmail.com> writes:
Dawid Ciężarkiewicz wrote:

 Imports probably, but ... why? It takes 15 minutes of frustrating checking
 your files. You seek help on #D irc channel (nice people BTW) - but they can
 not help you. They can only tell "It might be that in some unrelated file you
 are trying to use map without importing map but are importing files that do
 import map" which is great help but still will require you to spend great
 amount of time finding this place.
 


Seems to me that using "private import" instead of "public import" (the default)
might help in locating this. When using "private import A;" in a module B, any
other module that imports B does not automatically import A.

I have long felt that private should be the default.
Jun 17 2006
next sibling parent reply "Derek Parnell" <derek psych.ward> writes:
On Sun, 18 Jun 2006 03:49:44 +1000, Deewiant  
<deewiant.doesnotlike.spam gmail.com> wrote:


 Dawid Ciężarkiewicz wrote:

 Imports probably, but ... why? It takes 15 minutes of frustrating  
 checking
 your files. You seek help on #D irc channel (nice people BTW) - but  
 they can
 not help you. They can only tell "It might be that in some unrelated  
 file you
 are trying to use map without importing map but are importing files  
 that do
 import map" which is great help but still will require you to spend  
 great
 amount of time finding this place.


 Seems to me that using "private import" instead of "public import" (the  
 default)
 might help in locating this. When using "private import A;" in a module  
 B, any
 other module that imports B does not automatically import A.

 I have long felt that private should be the default.


I have yet to find a *need* for public imports.

-- 
Derek Parnell
Melbourne, Australia
Jun 17 2006
next sibling parent BCS <BCS pathlink.com> writes:
Derek Parnell wrote:

 On Sun, 18 Jun 2006 03:49:44 +1000, Deewiant  
 <deewiant.doesnotlike.spam gmail.com> wrote:


[...]

 Seems to me that using "private import" instead of "public import" 
 (the  default) might help in locating this. When using "private 
 import A;" in a module  B, any other module that imports B does
 not automatically import A. I have long felt that private should 
 be the default.


  
 I have yet to find a *need* for public imports.
 


A single-point-of-access module for a lib might count as a use. OTOH I 
have taken to doing all imports as private.

import myBigLib;

<code file="myBigLib.d>
public import theFirstFile;
public import theSecondFile;
public import theThirdFile;
...
</code>
Jun 17 2006
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
Derek Parnell wrote:

 
 I have yet to find a *need* for public imports.


I have.  Implementing C headers it's common for an alias to be declared 
in one header that must be visible in other headers.  Sure, you could 
require the user to manually import both headers, but that's 
non-intuitive and a tad annoying.  And the alternative (private import 
combined with aliasing symbols to expose them piecemeal) results in 
symbol collision problems.


Sean
Jun 17 2006
parent "Derek Parnell" <derek psych.ward> writes:
On Sun, 18 Jun 2006 04:30:32 +1000, Sean Kelly <sean f4.ca> wrote:


 Derek Parnell wrote:

  I have yet to find a *need* for public imports.


 I have.  Implementing C headers it's common for an alias to be declared  
 in one header that must be visible in other headers.  Sure, you could  
 require the user to manually import both headers, but that's  
 non-intuitive and a tad annoying.  And the alternative (private import  
 combined with aliasing symbols to expose them piecemeal) results in  
 symbol collision problems.


And yet I find it intuitive and helpful. People differ.

For me, if 'foo' is declared in A, and B needs to access 'foo' then B  
should import A. Now if C needs to access something in B and access 'foo'  
it should import both A and B. This seems natural to me. To do otherwise  
means that C's access to 'A.foo' becomes dependant on the implementation  
of B. This is too much cohesion in my books.

And yes, it can result in the need to disambiguate public symbol  
references in C, but I see that as a good thing. It help document the code  
and reduces the chance of calling the wrong function.

-- 
Derek Parnell
Melbourne, Australia
Jun 17 2006
prev sibling parent reply Jari-Matti =?UTF-8?B?TcOka2Vsw6Q=?= <jmjmak utu.fi.invalid> writes:
Deewiant wrote:


 Dawid Ciężarkiewicz wrote:

 Imports probably, but ... why? It takes 15 minutes of frustrating
 checking your files. You seek help on #D irc channel (nice people BTW) -
 but they can not help you. They can only tell "It might be that in some
 unrelated file you are trying to use map without importing map but are
 importing files that do import map" which is great help but still will
 require you to spend great amount of time finding this place.
 


 
 Seems to me that using "private import" instead of "public import" (the
 default) might help in locating this. When using "private import A;" in a
 module B, any other module that imports B does not automatically import A.


I'm not that sure about that. This might be an oversimplified case, but I
think the problem comes up, when (A,B,C,D are modules here) B and C
(privately) import A, and D imports both B and C. Now DMD regards function
foo() in module A as B.foo() and C.foo() through the import chains. When D
imports both B and C and a name conflict comes up, since they can't both
become D.foo().

Ok, how do we fix this problem? A proper way to handle these would be IMO to
save full info about the symbols and their origins in the symbol table. I'm
not a compiler writer and haven't had much time to find out how this is
done at the moment. But as a workaround you can (private) import B in
module C. After that there's only one route to the symbol source.



 I have long felt that private should be the default.


Yes. But imports should work properly, too.

-- 
Jari-Matti
Jun 17 2006
parent Deewiant <deewiant.doesnotlike.spam gmail.com> writes:
Jari-Matti Mäkelä wrote:

 Deewiant wrote:

 I have long felt that private [import] should be the default.


 
 Yes. But imports should work properly, too.
 


Quite correct.

<rant>
Hell, "private" should work properly! As Sean Kelly posted to this thread, C++
has its (arguable) reasons for why "private members are inaccessible but not
invisible", but D doesn't (seem to) have any. It's quite an annoyance, if not
worse.
</rant>
Jun 17 2006
prev sibling next sibling parent reply "Chris Miller" <chris dprogramming.com> writes:
Here's a simple example of the issue:


// -- foo.d --
private import std.stdio;


// -- bar.d --
private import std.stdio;


// -- test.d --

import foo, bar;

void main()
{
     char[] hi = std.string.toString(3);
}


DMD 0.160:
    foo.d(1): import foo.std conflicts with bar.std at bar.d(1)


Of course, the code in test.d is a mistake, but look how unhelpful the  
error message is. It will blame a library's code for a mistake the  
library-user made.

Also notice that it's considering *private* imports, which shouldn't even  
be available to test.d.
Jun 17 2006
next sibling parent Jari-Matti =?UTF-8?B?TcOka2Vsw6Q=?= <jmjmak utu.fi.invalid> writes:
Chris Miller wrote:


 Here's a simple example of the issue:
 
 
 // -- foo.d --
 private import std.stdio;
 
 
 // -- bar.d --
 private import std.stdio;
 
 
 // -- test.d --
 
 import foo, bar;
 
 void main()
 {
      char[] hi = std.string.toString(3);
 }
 
 
 DMD 0.160:
     foo.d(1): import foo.std conflicts with bar.std at bar.d(1)
 
 
 Of course, the code in test.d is a mistake, but look how unhelpful the
 error message is. It will blame a library's code for a mistake the
 library-user made.
 
 Also notice that it's considering *private* imports, which shouldn't even
 be available to test.d.


The compiler should say something like "function module1.foo is private and
thus cannot be used in module2", when it's not accessible.

The annoying thing is that when there's a function name conflict and the
first conflicting function is privately imported and the other is publicly
imported, the public version does not override the private one. And when
the same symbol is imported using 2+ import routes, there's the same old
bogus naming conflict again.

-- 
Jari-Matti
Jun 17 2006
prev sibling next sibling parent reply "Derek Parnell" <derek psych.ward> writes:
On Sun, 18 Jun 2006 04:15:11 +1000, Chris Miller <chris dprogramming.com>  
wrote:


 DMD 0.160:
     foo.d(1): import foo.std conflicts with bar.std at bar.d(1)


 Of course, the code in test.d is a mistake, but look how unhelpful the  
 error message is. It will blame a library's code for a mistake the  
 library-user made.


Maybe a message more like ...

     foo.d(6): std.string is not accessible

might be more useful for coders.


 Also notice that it's considering *private* imports, which shouldn't  
 even be available to test.d.


This has been broken for a *long* time now. Both "private" and "package"  
are just ignored by DMD for at least the last 5 or 6 releases. It used to  
work but now it doesn't.

-- 
Derek Parnell
Melbourne, Australia
Jun 17 2006
next sibling parent reply Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Derek Parnell wrote:

 

 Also notice that it's considering *private* imports, which shouldn't 
 even be available to test.d.


 
 This has been broken for a *long* time now. Both "private" and "package" 
 are just ignored by DMD for at least the last 5 or 6 releases. It used 
 to work but now it doesn't.
 
 --Derek Parnell
 Melbourne, Australia


5 or 6 releases? Far more than that, this bug 
(http://d.puremagic.com/bugzilla/show_bug.cgi?id=48) was reported for 
DMD .149 and it was broken I don't know how many releases even before that.
You said it used to work? When?

-- 
Bruno Medeiros - CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 17 2006
parent "Derek Parnell" <derek psych.ward> writes:
On Sun, 18 Jun 2006 10:58:27 +1000, Bruno Medeiros  
<brunodomedeirosATgmail SPAM.com> wrote:


 Derek Parnell wrote:

 Also notice that it's considering *private* imports, which shouldn't  
 even be available to test.d.


  This has been broken for a *long* time now. Both "private" and  
 "package" are just ignored by DMD for at least the last 5 or 6  
 releases. It used to work but now it doesn't.
  --Derek Parnell
 Melbourne, Australia


 5 or 6 releases? Far more than that, this bug  
 (http://d.puremagic.com/bugzilla/show_bug.cgi?id=48) was reported for  
 DMD .149 and it was broken I don't know how many releases even before  
 that.


I was being generous and polite ;-)


 You said it used to work? When?


The 'private' used to work at one stage because I tripped up on it. What  
one has to remember is that private restricts access but not visiblity.  
The 'package' qualifier has never worked AFAIK.



-- 
Derek Parnell
Melbourne, Australia
Jun 18 2006
prev sibling parent reply Georg Wrede <georg.wrede nospam.org> writes:
Derek Parnell wrote:

 On Sun, 18 Jun 2006 04:15:11 +1000, Chris Miller 
 <chris dprogramming.com>  wrote:
 

 DMD 0.160:
     foo.d(1): import foo.std conflicts with bar.std at bar.d(1)


 Of course, the code in test.d is a mistake, but look how unhelpful 
 the  error message is. It will blame a library's code for a mistake 
 the  library-user made.


 
 
 Maybe a message more like ...
 
     foo.d(6): std.string is not accessible
 
 might be more useful for coders.
 

 Also notice that it's considering *private* imports, which shouldn't  
 even be available to test.d.


 
 
 This has been broken for a *long* time now. Both "private" and 
 "package"  are just ignored by DMD for at least the last 5 or 6 
 releases. It used to  work but now it doesn't.


How is anyone new to D supposed to even consider using it when stuff 
like _private_ and _package_ are broken????????????????????????

This is both *elementary* and *essential* to any nontrivial project. It 
simply *has* to work.

"Would you buy a 3-wheel Ferrari?"

The first guys we lose this way are those who were considering D for 
real work (as opposed to hobby or tinkering).
Jun 18 2006
parent reply kris <foo bar.com> writes:
Georg Wrede wrote:

 Derek Parnell wrote:

 This has been broken for a *long* time now. Both "private" and 
 "package"  are just ignored by DMD for at least the last 5 or 6 
 releases. It used to  work but now it doesn't.


 
 
 How is anyone new to D supposed to even consider using it when stuff 
 like _private_ and _package_ are broken????????????????????????
 
 This is both *elementary* and *essential* to any nontrivial project. It 
 simply *has* to work.
 
 "Would you buy a 3-wheel Ferrari?"
 
 The first guys we lose this way are those who were considering D for 
 real work (as opposed to hobby or tinkering).


Dig dat, bro' ...

Apparently *so* fundamental that there's not even a stress-test for this 
kind of thing
Jun 18 2006
parent Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
kris wrote:

 Georg Wrede wrote:

 Derek Parnell wrote:

 This has been broken for a *long* time now. Both "private" and 
 "package"  are just ignored by DMD for at least the last 5 or 6 
 releases. It used to  work but now it doesn't.



 How is anyone new to D supposed to even consider using it when stuff 
 like _private_ and _package_ are broken????????????????????????

 This is both *elementary* and *essential* to any nontrivial project. 
 It simply *has* to work.

 "Would you buy a 3-wheel Ferrari?"

 The first guys we lose this way are those who were considering D for 
 real work (as opposed to hobby or tinkering).


 
 Dig dat, bro' ...
 
 Apparently *so* fundamental that there's not even a stress-test for this 
 kind of thing
 
 


What do you mean? There are lot of testcases in DStress for protection 
attributes:
http://dstress.kuehne.cn/www/dstress.html#private_03


-- 
Bruno Medeiros - CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 19 2006
prev sibling parent Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Chris Miller wrote:

 Here's a simple example of the issue:
 
 
 // -- foo.d --
 private import std.stdio;
 
 
 // -- bar.d --
 private import std.stdio;
 
 
 // -- test.d --
 
 import foo, bar;
 
 void main()
 {
     char[] hi = std.string.toString(3);
 }
 
 
 DMD 0.160:
    foo.d(1): import foo.std conflicts with bar.std at bar.d(1)
 
 
 Of course, the code in test.d is a mistake, but look how unhelpful the 
 error message is. It will blame a library's code for a mistake the 
 library-user made.
 
 Also notice that it's considering *private* imports, which shouldn't 
 even be available to test.d.


It's considering private imports because of bug 48, but this issue is 
not the same as bug 48, it is a separate bug, because it also happens 
with public imports.
Submitted as bug 209.


-- 
Bruno Medeiros - CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 19 2006
prev sibling next sibling parent reply Sean Kelly <sean f4.ca> writes:
Dawid Ciężarkiewicz wrote:

 You write your great soft in D. You're coding with smile because everything
 in seems to be so good. You added few new files, new classes. Porting your
 project from C++ to D seems to be so good idea. You type "make".
 
 And then out of nowhere: BANG!!!
 
 battle.d(31): import battle.map conflicts with cell.cell.map at
 cell/cell.d(29)


For what it's worth, the symbol resolution rules styled after the C++ 
class-scope rules.  On the surface, this does seem to provide solid 
support for the approach: it's a sticky issue and an established 
language has solved the problem this way, so why not do that?  However, 
I think the reasons that this rule was chosen for C++ may be related to 
language syntax that D simply doesn't have, thus reducing the strength 
of the correlation.

I decided to go looking for information on why the C++ rules are the way 
they are, and oddly enough the first hit was the C++ Module proposal 
(available here: 
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n1964.pdf) 
Section 4.3 reads thus:

   4.3 Shielding from private members

   The fact that private members are inaccessible but not invisible
   regularly surprises incidental programmers. Like macros, seemingly
   unrelated declarations interfere with subsequent code. Unfortunately,
   there are good reasons for this state of affair: Without it, private
   out-of-class member declarations become impractical to parse in the
   general case.

   Module namespaces appear to be an ideal boundary for making the
   private member fully invisible: Within the module the implementer has
   full control over naming conventions and can therefore easily avoid
   interference, while outside the module the client will never
   have to implement private members. (Note that this also addresses the
   concerns of N1602 "Class Scope Using Declarations & private Members"
   by Francis Glassborow; the extension proposed therein is then no
   longer needed.)

It's worth noting that the issue discussed in the first paragraph: 
out-of-class declarations, is not supported in D.  All members must be 
declared in class scope.  The second paragraph goes on to make an 
argument for why private symbols should not be considered across modules 
for lookup resolution.  The only issue not addressed is package 
visibility, but I believe the same approach could be done at the package 
level as at the module level.  After all, there's a file separation 
either way.

I was unable to find older discussions relating why these rules were 
chosen for C++, but if there are other arguments against the approach 
which apply to D, either for current or future features, I would love to 
hear them.  The current state of affairs is annoying and somewhat 
confusing to new users, but I'd prefer an annoying system that works for 
all corner-cases to a cleaner one that doesn't.


Sean
Jun 17 2006
parent reply Bruno Medeiros <brunodomedeirosATgmail SPAM.com> writes:
Sean Kelly wrote:

 Dawid Ciężarkiewicz wrote:

 You write your great soft in D. You're coding with smile because 
 everything
 in seems to be so good. You added few new files, new classes. Porting 
 your
 project from C++ to D seems to be so good idea. You type "make".

 And then out of nowhere: BANG!!!

 battle.d(31): import battle.map conflicts with cell.cell.map at
 cell/cell.d(29)


 
 For what it's worth, the symbol resolution rules styled after the C++ 
 class-scope rules.  On the surface, this does seem to provide solid 
 support for the approach: it's a sticky issue and an established 
 language has solved the problem this way, so why not do that?  However, 
 I think the reasons that this rule was chosen for C++ may be related to 
 language syntax that D simply doesn't have, thus reducing the strength 
 of the correlation.


I can't figure out what kind of parallel or comparison could be made of 
symbol resolution with a language that has no structured module system. 
What would be the analogous case in C++ for the situation mentioned 
above? What are the C++ class-scope rules?

-- 
Bruno Medeiros - CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Jun 18 2006
parent Sean Kelly <sean f4.ca> writes:
Bruno Medeiros wrote:

 Sean Kelly wrote:

 Dawid Ciężarkiewicz wrote:

 You write your great soft in D. You're coding with smile because 
 everything
 in seems to be so good. You added few new files, new classes. Porting 
 your
 project from C++ to D seems to be so good idea. You type "make".

 And then out of nowhere: BANG!!!

 battle.d(31): import battle.map conflicts with cell.cell.map at
 cell/cell.d(29)


 For what it's worth, the symbol resolution rules styled after the C++ 
 class-scope rules.  On the surface, this does seem to provide solid 
 support for the approach: it's a sticky issue and an established 
 language has solved the problem this way, so why not do that?  
 However, I think the reasons that this rule was chosen for C++ may be 
 related to language syntax that D simply doesn't have, thus reducing 
 the strength of the correlation.


 
 I can't figure out what kind of parallel or comparison could be made of 
 symbol resolution with a language that has no structured module system. 
 What would be the analogous case in C++ for the situation mentioned 
 above? What are the C++ class-scope rules?


C++ class member symbol lookup is roughly similar to module-level lookup 
in D at a conceptual level, so Walter modeled the D lookup rules on this 
aspect of C++.  If you pretend a C++ class is a D module and use 
multiple inheritance instead of import you'll see identical behavior for 
symbol lookup.


Sean
Jun 18 2006
prev sibling next sibling parent reply Dawid =?UTF-8?B?Q2nEmcW8YXJraWV3aWN6?= <dawid.ciezarkiewicz gmail.com> writes:
Dawid Ciężarkiewicz wrote:

 And then out of nowhere: BANG!!!
 
 battle.d(31): import battle.map conflicts with cell.cell.map at
 cell/cell.d(29)


OK. After some time I've finally discovered where the problem was and I see
that my previous post is not very informative. Instead it gives good look
how such "details" can frustrate people.

The problem is (if I understand right):
When you use identifier which was imported privately by something you
imported in current module you get this missleading "import X conficts with
Y" instead of info where it happens.

battle.d(31) == "private import map";
cell/cell/d(29) == "private import map";

As you can see your IDE will take you to battle.d, line 31 and you'll be
confused. You don't know which line of code triggered error. If you are
lucky and smart enough, you'll try fixing later errors and find which part
of the code triggered this one. If you're not - you'll be siting and
screaming knowing not what to do.

The good error msg would look like:

$file($line): triggered import conflict with X ($X:$Xline) and Y ($Y:$Yline)

Where $file($line) is place where compiler found that this two modules are
conflicting. This would be quite informative IMO. 

Another thing is that I do "private import $i" for purpose - I don't want
names from module be _anyhow_ visible - not triggering any conflicts.
Jun 17 2006
parent Jari-Matti =?UTF-8?B?TcOka2Vsw6Q=?= <jmjmak utu.fi.invalid> writes:
Dawid Ciężarkiewicz wrote:

 The good error msg would look like:
 
 $file($line): triggered import conflict with X ($X:$Xline) and Y
 ($Y:$Yline)
 
 Where $file($line) is place where compiler found that this two modules are
 conflicting. This would be quite informative IMO.


Yes.


 
 Another thing is that I do "private import $i" for purpose - I don't want
 names from module be _anyhow_ visible - not triggering any conflicts.


I think they should be visible enough to trigger a proper error message,
when you don't have any public symbols with the same name. But when you do
have, the compiler should be able to mask the private ones then. Only if
you have at least two public symbols with the same name, a name conflict
should appear.

-- 
Jari-Matti
Jun 17 2006
prev sibling parent reply I hear ya <I_member pathlink.com> writes:
You're write it is awful.  A useful error message would make this a hundred
times easier.

I've also noticed this always happens strictly with phobos, has anyone had these
conflicts with their own modules ?


In article <e71f1s$31fj$1 digitaldaemon.com>, Dawid
=?UTF-8?B?Q2nEmcW8YXJraWV3aWN6?= says...

You write your great soft in D. You're coding with smile because everything
in seems to be so good. You added few new files, new classes. Porting your
project from C++ to D seems to be so good idea. You type "make".

And then out of nowhere: BANG!!!

battle.d(31): import battle.map conflicts with cell.cell.map at
cell/cell.d(29)

GAME OVER!

You're stuck with this nothing telling you error message. You start
wondering what is wrong. Imports probably, but ... why? It takes 15 minutes
of frustrating checking your files. You seek help on #D irc channel (nice
people BTW) - but they can not help you. They can only tell "It might be
that in some unrelated file you are trying to use map without importing map
but are importing files that do import map" which is great help but still
will require you to spend great amount of time finding this place.

Please Walter - fix this. It is sooooooo frustrating and everyone confirms
that. It's something that all D people hits once or more.
Jun 17 2006
parent Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
I hear ya wrote:

 You're write it is awful.  A useful error message would make this a hundred
 times easier.
 
 I've also noticed this always happens strictly with phobos, has anyone had
these
 conflicts with their own modules ?


Ummmm yeah... a lot...

Walter, please fix it, it will save my hair ;D


-- 
Tomasz Stachowiak  /+ a.k.a. h3r3tic +/
Jun 17 2006