www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Some (big) problems with modules right now (long post)

reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Working on a large(r) project in D, I've started to notice some.. =
irritation and clumsiness with modules.  The spec for modules is not =
very extensive, either.  So for the past day or so, I've been working on =
figuring out just how the modules work (or don't).  Here are the =
problems I've come across.



Naming

=20

The name deduction process for modules seems to go like this, as defined =
in the spec:

            if file has a "module" declaration

            {

                        use "module" declaration name

            }

            else

            {

                        ambiguous?  spec says:

                                    "the module name is taken to be the =
same name (stripped of path                                              =
 and extension)"

                        and at the same time:

                                    "The packages correspond to =
directory names in the source file                                       =
          path."

            }

=20

This can be confusing.  I could have a module with a filename of =
/modules/math/vector.d, but have a module declaration in the file that =
calls it "3dmath.vectors" or something.  I understand why this is =
necessary - some file systems are case-sensitive, and it'd be a pain to =
have different cases for the directory name and the "module" name (and =
in fact, there may be computers which don't have the concept of =
directories, making the directory system worthless there). =20

=20

Perhaps the directory system should be removed, if for nothing else but =
consistency.  That is, all modules must have a module declaration for =
them to be importable.  This would still allow modules to be named =
differently from their directories, but it'd be consistent - you'd have =
to look in a module to see what it's called.  No more guessing whether =
to call it by its directory name or its "module" name.

=20

Lastly, the directory naming ambiguity is described in the next =
section..

=20

Importing

=20

I've come across two rather confusing errors with this naming stuff as =
well as some other.. issues.  My directory tree looks like this:

=20

/dtest

+---dtest.d

+---/mod

+---+---mod1.d

=20

The contents of the files:

=20

dtest.d

=20

import std.stdio;

import mod1;

=20

void main()

{
     writefln(x);

}

=20

mod1.d

=20

int x=3D5;

=20

This compiles and runs.  I'm kind of confused why it doesn't require me =
to write import mod.mod1;, as it should be using the directory naming =
system (and no, /dtest/mod is not in my include path).  In fact, I can =
nest the mod1.d in as many layers of directories as I want and I'll =
still be able to import it as just mod1.

=20

In fact, if I try to use import mod.mod1, I get the following, confusing =
error:

dtest.d(2): module mod1 is in multiply defined

What on earth is that supposed to mean?  Is the "in" just a typo?  If =
so, where is the module multiply defined?  Why can't I import the module =
like this, if the directory structure is like so?

=20

If, however, I remove mod1.d from the compiler command line, I must use =
import mod.mod1.  Of course, the linker then complains about x being an =
unresolved reference.  Why are there two systems of import naming =
depending on whether or not I'm compiling the module?  Yet another =
reason to question the directory naming system.

=20

Even stranger: if I change my import to import mod.mod1, and add a =
module declaration to the module called module mod1, I get yet another =
incomprehensible error:

dtest.d(2): module mod1 is in multiple packages mod1

If I use import mod1, on the other hand, it works fine. =20

=20

And yet again, if the module is not being compiled, I must explicitly =
qualify the module name as mod.mod1. =20

=20

Oh, and this one takes the cake:

=20

dtest.d

import std.stdio;

// notice what I call it here!

import mod.mod1;

=20

void main()

{

     writefln(x);

}

=20

mod1.d

// notice the name is now mod2!

module mod2;

int x=3D5;

=20

This compiles and runs when mod1 is not being compiled.  The linker =
whines of course.  Apparently, the "module" declaration is completely =
ignored if the module is just being lexed!  If I compile the module as =
well, I must import mod2, not mod.mod1 in dtest.d.  Very confusing.

=20

Then you start getting into multiple modules and modules which import =
one another.  Consider that there is a mod2.d, which does something =
else, who cares what.  Say mod1 privately imports mod2.  When all three =
files are compiled, the modules can be imported just by their name.  =
However, take the modules out of the command line, and suddenly, mod1 =
can't import mod2 by just name anymore!  You have to actually modify =
mod1 by changing the import mod2 to import mod.mod2!  How is that easy =
to use?

=20

Protection Attributes

=20

By default, everything is public, and that works fine.  Then there are =
package and private.

=20

Package, to put it simply, doesn't seem to do ANYTHING.  I've tried =
putting modules in so many different directory trees, naming them =
different ways, compiling them or not, etc.  And I can still, very =
easily, import a module and access its package members.  Perhaps it has =
something to do with the module's namespace being merged with the =
current namespace when it is imported?  I can't seem to figure it out.

=20

Private works fine for variables, arrays, and functions, but beyond =
that, it does nothing.  Enums, structs, interfaces, unions, and classes =
can all be declared private, but it doesn't stop you from using them =
outside the module.  Note that it's just definitions that don't work.  =
if I do this in the module:

=20

private class C
{
     int x;
}



C c;

=20

I will be able to create Cs outside the module, access C's public static =
members etc.  But I will not be allowed to access the instance "c."

=20

I posted this as a possible bug, and someone said that perhaps it's just =
making the definition private.  Well what good does that do?  It's not =
like we can modify the definition outside of the file, and it would make =
more sense and be more useful if the entire class were inaccessible =
outside the module.  The same applies for structs, interfaces, unions, =
and enums. =20

=20

Conclusion

=20

I really think this mess needs to be cleaned up before 1.0.  Modules =
seem to be a convenient way to organize your code, but I swear, there =
are just as many caveats and irritation as using a header/cpp pair in =
C++. =20
Jan 24 2005
next sibling parent reply "Walter" <newshound digitalmars.com> writes:
On your first example -

o Please turn off html in your message format. html not only makes it
difficult to embed replies to your long message, I (and many others) have
html turned off in my newsreader as it is a security risk. Please use
plaintext only.

o Please, for your first example, show exactly which directories dtest.d and
mod1.d are in (I am not positive what is meant by +----+----mod1.d), what
directory was the default when you ran the compiler, and what -I switches
were used to compile. All of these have a large influence over the results.
Jan 24 2005
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
 o Please turn off html in your message format. html not only makes it
 difficult to embed replies to your long message, I (and many others) have
 html turned off in my newsreader as it is a security risk. Please use
 plaintext only.


Yes, Sahib ;)


 o Please, for your first example, show exactly which directories dtest.d


and

 mod1.d are in (I am not positive what is meant by +----+----mod1.d), what
 directory was the default when you ran the compiler, and what -I switches
 were used to compile. All of these have a large influence over the


results.

/dtest/dtest.d
/dtest/mod/mod1.d

I was trying to do some kind of ASCII representation of the directory
structure, but I guess it didn't work out.  All subsequent examples use the
same directory structure (and mod2.d is in /dtest/mod as well).

The default directory is /dtest.

The only -I switch in effect is the phobos include path (/dmd/src/phobos).
Jan 24 2005
parent reply "Walter" <newshound digitalmars.com> writes:
"Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message
news:ct3eic$1rhu$1 digitaldaemon.com...

 /dtest/dtest.d
 /dtest/mod/mod1.d

 I was trying to do some kind of ASCII representation of the directory
 structure, but I guess it didn't work out.  All subsequent examples use


the

 same directory structure (and mod2.d is in /dtest/mod as well).

 The default directory is /dtest.

 The only -I switch in effect is the phobos include path (/dmd/src/phobos).


I can't reproduce the difficulty you're having with it. Here's what I have,
which gives the expected error:

---------------------------------------------------
C:\dtest>type dtest.d
import std.stdio;
import mod1;

void main()
{
     writefln(x);
}

C:\dtest>type mod\mod1.d
int x = 5;

C:\dtest>dir
 Volume in drive C has no label.
 Volume Serial Number is C83C-5D49

 Directory of C:\dtest

01/25/2005  12:27 AM    <DIR>          mod
01/25/2005  12:28 AM                73 dtest.d
01/25/2005  12:28 AM    <DIR>          ..
01/25/2005  12:28 AM    <DIR>          .
               1 File(s)             73 bytes
               3 Dir(s)  10,404,511,744 bytes free

C:\dtest>dir mod
 Volume in drive C has no label.
 Volume Serial Number is C83C-5D49

 Directory of C:\dtest\mod

01/25/2005  12:27 AM    <DIR>          ..
01/25/2005  12:27 AM                12 mod1.d
01/25/2005  12:27 AM    <DIR>          .
               1 File(s)             12 bytes
               2 Dir(s)  10,404,511,744 bytes free

C:\dtest>dmd dtest
Error: Error reading file 'mod1.d'


C:\dtest>
------------------------------------------
Jan 25 2005
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
 I can't reproduce the difficulty you're having with it. Here's what I


have,

 which gives the expected error:


I wasn't quite clear, but in the first trial, both dtest.d and mod/mod1.d
are in the command line, and the code you posted (which is what I posted)
compiles and runs.  However:

"If, however, I remove mod1.d from the compiler command line, I must use
import mod.mod1."

From the original post.  Sorry about that misunderstanding.
Jan 25 2005
parent reply "Walter" <newshound digitalmars.com> writes:
"Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message
news:ct6pol$1k1$1 digitaldaemon.com...

 I can't reproduce the difficulty you're having with it. Here's what I


 have,

 which gives the expected error:


 I wasn't quite clear, but in the first trial, both dtest.d and mod/mod1.d
 are in the command line, and the code you posted (which is what I posted)
 compiles and runs.  However:

 "If, however, I remove mod1.d from the compiler command line, I must use
 import mod.mod1."

 From the original post.  Sorry about that misunderstanding.


What is the exact command you are using?
Jan 26 2005
parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
 What is the exact command you are using?


...

dmd dtest.d mod/mod1.d

If I use "import mod.mod1" with that command line, I get the error

dtest.d(2): module mod1 is in multiply defined

I must use "import mod1".

IF, however, I remove mod/mod1.d from the command line, and use "import
mod1", it can't find mod1.  I must use "import mod.mod1."
Jan 26 2005
parent "Walter" <newshound digitalmars.com> writes:
"Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message
news:ct950q$2tls$1 digitaldaemon.com...

 What is the exact command you are using?


 ...

 dmd dtest.d mod/mod1.d

 If I use "import mod.mod1" with that command line, I get the error

 dtest.d(2): module mod1 is in multiply defined

 I must use "import mod1".

 IF, however, I remove mod/mod1.d from the command line, and use "import
 mod1", it can't find mod1.  I must use "import mod.mod1."


Ok, I see now. I'll work on it.

BTW, putting:

    module mod.mod1;

in mod\mod1.d will fix it.
Jan 29 2005
prev sibling next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
 "Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message


news:ct39ut$1kqi$1 digitaldaemon.com...
[snip]

 Conclusion

 I really think this mess needs to be cleaned up before 1.0.  Modules seem


to be a convenient way to organize your code, but I swear, there  are just
as many caveats and irritation as using a header/cpp pair in C++.

I think the module/package/import system is pretty much like Java's (and I

given a module name while in Java there is no default mechanism. There
probably are a few other differences but not much I think. Maybe some better
error messages would make it easier to use.
Jan 24 2005
next sibling parent reply "Charles" <no email.com> writes:
 One difference is that files without a module declaration are
 given a module name while in Java there is no default mechanism


Question about the frontend :

When there is no module declaration present, the toChars() method of Module
returns garbage , strlen and strcmp dont work on it , so how can I know when
a  ( Module ) has no declared 'module foo;' ?  ( Module->ident->toChars()
not working either )

Thx,
C


"Ben Hinkle" <bhinkle mathworks.com> wrote in message
news:ct3eq5$1rqv$1 digitaldaemon.com...

 "Jarrett Billingsley" <kb3ctd2 yahoo.com> wrote in message


 news:ct39ut$1kqi$1 digitaldaemon.com...
 [snip]

 Conclusion

 I really think this mess needs to be cleaned up before 1.0.  Modules




seem

 to be a convenient way to organize your code, but I swear, there  are just
 as many caveats and irritation as using a header/cpp pair in C++.

 I think the module/package/import system is pretty much like Java's (and I



are

 given a module name while in Java there is no default mechanism. There
 probably are a few other differences but not much I think. Maybe some


better

 error messages would make it easier to use.
Jan 24 2005
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"Charles" <no email.com> wrote in message
news:ct3fo6$1t4s$1 digitaldaemon.com...

 One difference is that files without a module declaration are
 given a module name while in Java there is no default mechanism


 Question about the frontend :

 When there is no module declaration present, the toChars() method of


Module

 returns garbage , strlen and strcmp dont work on it , so how can I know


when

 a  ( Module ) has no declared 'module foo;' ?  ( Module->ident->toChars()
 not working either )

 Thx,
 C


It looks like the module declaration is used at line 393 of module.c in the
frontend source. After parsing the module if there is a ModuleDeclaration
then it uses the name from that instead of the file name. I'm not sure if
this helps answer your question but I'd poke around there.

As a side note the typo in the "is in multiply defined" error Jarrett was
referring to is on line 416 of module.c

-Ben
Jan 24 2005
parent Charlie <Charlie_member pathlink.com> writes:
That got ( ModuleDeclaration md ).  Thx!

Chuck D

In article <ct3ged$1ttf$1 digitaldaemon.com>, Ben Hinkle says...

"Charles" <no email.com> wrote in message
news:ct3fo6$1t4s$1 digitaldaemon.com...

 One difference is that files without a module declaration are
 given a module name while in Java there is no default mechanism


 Question about the frontend :

 When there is no module declaration present, the toChars() method of


Module

 returns garbage , strlen and strcmp dont work on it , so how can I know


when

 a  ( Module ) has no declared 'module foo;' ?  ( Module->ident->toChars()
 not working either )

 Thx,
 C


It looks like the module declaration is used at line 393 of module.c in the
frontend source. After parsing the module if there is a ModuleDeclaration
then it uses the name from that instead of the file name. I'm not sure if
this helps answer your question but I'd poke around there.

As a side note the typo in the "is in multiply defined" error Jarrett was
referring to is on line 416 of module.c

-Ben
Jan 25 2005
prev sibling parent reply Georg Wrede <georg.wrede nospam.org> writes:
I really think this mess needs to be cleaned up before 1.0.  Modules seem


 
 to be a convenient way to organize your code, but I swear, there  are just
 as many caveats and irritation as using a header/cpp pair in C++.
 
 I think the module/package/import system is pretty much like Java's (and I

 given a module name while in Java there is no default mechanism. There
 probably are a few other differences but not much I think. Maybe some better
 error messages would make it easier to use.


[removed the names above, since I'm not directing this to them...]

Whether the "module/package/import system" is like that of Java or not 
is not the issue here. What counts is the end result. If a smart 
programmer new to the language can't figure it out easily, by just 
trying out a few examples (I know, "don't ask the compiler, check the 
specs!", but, hey, we live in the real world), then the system is 
unobvious. This should be remedied.
Jan 24 2005
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"Georg Wrede" <georg.wrede nospam.org> wrote in message
news:41F54AFD.4020702 nospam.org...

I really think this mess needs to be cleaned up before 1.0.  Modules






seem

 to be a convenient way to organize your code, but I swear, there  are




just

 as many caveats and irritation as using a header/cpp pair in C++.

 I think the module/package/import system is pretty much like Java's (and




I






are

 given a module name while in Java there is no default mechanism. There
 probably are a few other differences but not much I think. Maybe some




better

 error messages would make it easier to use.


 [removed the names above, since I'm not directing this to them...]

 Whether the "module/package/import system" is like that of Java or not
 is not the issue here. What counts is the end result.


I agree. I mentioned Java to point out another fairly successful system that
used a similar mechanism. Compared to C++'s #define mechanism D's imports
might seem wierd but compared to Java's import it seems very natural.


 If a smart
 programmer new to the language can't figure it out easily, by just
 trying out a few examples (I know, "don't ask the compiler, check the
 specs!", but, hey, we live in the real world), then the system is
 unobvious. This should be remedied.


Yup, agreed. I think better error messages and perhaps more doc would help
since the relation between module declaration and the actual directory
structure and file names is not clear. And I agree with the OP's point about
"private class" declarations not doing anything when they probably should.
But I'm not so quick to agree that the module and import behavior needs
fundamental changes or is a mess.
Jan 24 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 24 Jan 2005 14:57:02 -0500, Ben Hinkle <bhinkle mathworks.com>  
wrote:

<snip>


 And I agree with the OP's point about "private class" declarations not  
 doing anything when they probably should.


<snip>

To make a class private I have always done:






It seems to me that:






doesn't do anything at the moment, I couldn't even get it to deny me  
access to an instance as the OP did.

So, my vote would be that "private class X" means the  
definition/declaration of class X is private, meaning it behaves as  
private instances do.

Regan
Jan 24 2005
prev sibling parent Derek Parnell <derek psych.ward> writes:
On Mon, 24 Jan 2005 12:08:23 -0500, Jarrett Billingsley wrote:


 Working on a large(r) project in D, I've started to notice some.. irritation
 and clumsiness with modules.  The spec for modules is not very extensive,
either.
  So for the past day or so, I've been working on figuring out just how the
 modules work (or don't).


[snip]

I didn't see this post 'cos I was on annual leave at the time.

Here is what I've learned to do re modules ...

** I always put a module statement in each source file. The module
statement states the module name and the complete package name. For example
...

  module ui.core;

** I always call a module the same name as its source file name. 

** I always have the source files in the correct directory, based on the
package name that is in their module statement. For example ...

The file above would be called 'core.d' it would be placed in the 'ui/'
directory.

** I always give the fully specified path/file to the DMD compiler (via the
"build" utility).

** I always use "private import", though I can see myself relaxing this at
odd times, if especially exceptional circumstances came up.

** I always use fully specified package and module names on the import
statements.


When I do all this, I don't have problems.

BTW, the 'multiplied defined module' message is very misleading.

Hope this helps you work around D's way of doing things.


-- 
Derek
Melbourne, Australia
16/02/2005 2:53:34 PM
Feb 15 2005