www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Module/Package "Best Practices"

reply "news.digitalmars.com" <dbarrett quinthar.com> writes:
What's the "best practice" surrounding modules and packages?

I want to have a library of reusable code that I can import into an
app-specific .d file with a single statement.  This library is composed of
dozens of classes, each in its own file/module.

In C++ I'd create a single global.h file that includes the .h files for all
library classes.  In this way, the app-specific code can include the entire
library with a single #include statement.

In D I *think* I can do the same thing: just create a global.d file that
imports all the .d files of the library.  Thus the app-specific code just
imports the library.global module, and it gets the whole thing.

Is this the way it's supposed to be done?

I'd prefer some kind of wildcard import, like:





But that doesn't appear to work.

What have people been doing to organize and separate their reusable and
app-specific code?

-david
Jul 02 2004
next sibling parent Charlie <Charlie_member pathlink.com> writes:
For libraries thats what ive done, create module project.api; which contains all
nessecary imports.  

For most other imports its usually necessary to import them at the class level
to avoid forward reference problems, which for large projects can be a show
stopper ( DWT I know has struggled with it ).

Chuck

In article <cc5ipd$22jv$1 digitaldaemon.com>, news.digitalmars.com says...

What's the "best practice" surrounding modules and packages?

I want to have a library of reusable code that I can import into an
app-specific .d file with a single statement.  This library is composed of
dozens of classes, each in its own file/module.

In C++ I'd create a single global.h file that includes the .h files for all
library classes.  In this way, the app-specific code can include the entire
library with a single #include statement.

In D I *think* I can do the same thing: just create a global.d file that
imports all the .d files of the library.  Thus the app-specific code just
imports the library.global module, and it gets the whole thing.

Is this the way it's supposed to be done?

I'd prefer some kind of wildcard import, like:





But that doesn't appear to work.

What have people been doing to organize and separate their reusable and
app-specific code?

-david
Jul 03 2004
prev sibling parent reply Andy Friesen <andy ikagames.com> writes:
news.digitalmars.com wrote:


 What's the "best practice" surrounding modules and packages?
 
 I want to have a library of reusable code that I can import into an
 app-specific .d file with a single statement.  This library is composed of
 dozens of classes, each in its own file/module.
 
 In D I *think* I can ... just create a global.d file that
 imports all the .d files of the library.  Thus the app-specific code just
 imports the library.global module, and it gets the whole thing.
 
 Is this the way it's supposed to be done?
 
 I'd prefer some kind of wildcard import, like:
 



 
 But that doesn't appear to work.
 
 What have people been doing to organize and separate their reusable and
 app-specific code?


I just make a module of the same name as the package.  It's a tad 
awkward to say

     import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.  It'd be nice to have an explicit convention for this sort of 
thing, but I'm happy as long as it says what it is. :)

  -- andy
Jul 03 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...


I just make a module of the same name as the package.  It's a tad 
awkward to say

     import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.


Yeah, that's what I do too. Hence etc.bigint.bigint, etc.crypto.crypto and
etc.unicode.unicode. (The latter two aren't actually there yet).



It'd be nice to have an explicit convention for this sort of 
thing, but I'm happy as long as it says what it is. :)


Well, I borrowed the idea from std.windows.windows and std.linux.linux. I guess
if Phobos can do it, that gotta at least count towards a convention.

Arcane Jill
Jul 03 2004
parent reply Ant <Ant_member pathlink.com> writes:
In article <cc76jk$1i2p$1 digitaldaemon.com>, Arcane Jill says...

In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...


I just make a module of the same name as the package.  It's a tad 
awkward to say

     import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.


 etc.bigint.bigint
 etc.crypto.crypto 
 etc.unicode.unicode
 std.windows.windows
 std.linux.linux


I don't like it, it's not really descriptive.
both all.d and api.d look better to me.

DUI has "All.d" but it will change to "all.d"

but this is a minor issue.

Ant
Jul 03 2004
parent reply Sam McCall <tunah.d tunah.net> writes:
Ant wrote:

 In article <cc76jk$1i2p$1 digitaldaemon.com>, Arcane Jill says...
 

In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...



I just make a module of the same name as the package.  It's a tad 
awkward to say

    import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.


etc.bigint.bigint
etc.crypto.crypto 
etc.unicode.unicode
std.windows.windows
std.linux.linux


 
 
 I don't like it, it's not really descriptive.
 both all.d and api.d look better to me.
 
 DUI has "All.d" but it will change to "all.d"
 
 but this is a minor issue.
 
 Ant
 
 


<Java>
Walter, can we have * added to the valid identifier characters? ;-)
</Java>

Sam
Jul 04 2004
parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <ccaoqp$fli$1 digitaldaemon.com>, Sam McCall says...


<Java>
Walter, can we have * added to the valid identifier characters? ;-)
</Java>


Is there any reason why the import statement can't be made to allow:



meaning, import all .d files in directory .../abc/def?
(just like Java does)

Arcane Jill
Jul 05 2004
next sibling parent reply Gold Dragon <dragonwing dragonu.net> writes:
I was thinking the same thing, but then again maybe you wouldn't want to 
include all of the modules. What Java does mostly with the classes that 
Sun created are built in to the VM so that when you compile the code, 
you aren't actually including any of those classes. Just opcodes of the 
classes so that the VM can tell what class you were using. This allows 
you to use the '*' without having bloat. D would only include a module 
once in the program but would include/import ALL of the modules if you 
use them or not.

If it can tell if you used a module or not then I would agree that it 
would be useful.


 Is there any reason why the import statement can't be made to allow:
 

 
 meaning, import all .d files in directory .../abc/def?
 (just like Java does)
 
 Arcane Jill
Jul 05 2004
parent Sam McCall <tunah.d tunah.net> writes:
Gold Dragon wrote:


 I was thinking the same thing, but then again maybe you wouldn't want to 
 include all of the modules. What Java does mostly with the classes that 
 Sun created are built in to the VM so that when you compile the code, 
 you aren't actually including any of those classes. Just opcodes of the 
 classes so that the VM can tell what class you were using.  This allows
 you to use the '*' without having bloat.


This is done for all classes, not just rt.jar.
This is also how D always works.
Sam
Jul 06 2004
prev sibling parent reply Benji Smith <dlanguage xxagg.com> writes:
On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
<Arcane_member pathlink.com> wrote:


Is there any reason why the import statement can't be made to allow:



meaning, import all .d files in directory .../abc/def?
(just like Java does)


Actually, I really dislike Java's wildcard imports. It makes it
non-obvious which sub-packages are being imported. This can be a pain
when using a tool like grep to search for source files which import a
particular package or module.

As a matter of style, when I'm writing Java code, I always include a
separate import statement for each distinct package/module that I'll
be using. Not only does this provide for clearer, more precise code,
it also prevents enormous amounts of runtime bloat that would
otherwise result from loading classes that won't be used during
program execution. (I imagine that a similar amount of bloat is
avoided in D by getting rid of the wildcard imports.)

That's my 2c, anyhow.

--Benji Smith
Jul 05 2004
next sibling parent Arcane Jill <Arcane_member pathlink.com> writes:
In article <g6sje0po6iulbho0t8j5fab8gga4nr1kkb 4ax.com>, Benji Smith says...

On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
<Arcane_member pathlink.com> wrote:


Is there any reason why the import statement can't be made to allow:



meaning, import all .d files in directory .../abc/def?
(just like Java does)


As a matter of style, when I'm writing Java code, I always include a
separate import statement for each distinct package/module that I'll
be using.


Understood, and actually I agree with you. However, that's not really what we're
talking about. Suppose I write a "distinct package" comprising two hundred
individual files. It would makes sense for me to provide a single importable
module which imports all the others. The original question on this thread was
"What should that module be called"?

Phobos (and now Deimos) uses:



Others prefer:



Of course, what we'd ALL prefer is:



But D doesn't allow that (module and package with same name). And so, as a kind
of "second choice", I ventured that:



might be an acceptable alternative. I guess what we really need here is a
proclaimation in the style-guide. I'm not hung up on ANY particular convention,
but I do want "what people expect" to "just work", and what people expect is
entirely dependent on consistency across multiple projects.

Arcane Jill
Jul 06 2004
prev sibling parent reply Sam McCall <tunah.d tunah.net> writes:
Benji Smith wrote:


 On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
 <Arcane_member pathlink.com> wrote:
 
 

Is there any reason why the import statement can't be made to allow:



meaning, import all .d files in directory .../abc/def?
(just like Java does)


 
 
 Actually, I really dislike Java's wildcard imports. It makes it
 non-obvious which sub-packages are being imported.


No, because java doesn't have sub-packages ;-)
Technically, java.awt.image (for example), doesn't have any magic 
relationship with java.awt, they're two unrelated packages.


 This can be a pain
 when using a tool like grep to search for source files which import a
 particular package 


You can't import a package, unless you mean all the modules (D) or 
class/interfaces (java) in that package, in which case search for 
'import package.[^.]+;' for "any module in a package", or
'import package.*;' for "all modules in a package".
If * was available, sure, you couldn't search for "all modules in a 
package, imported separately", but you can't do that now and it's pretty 
meaningless.


 or module.


Search for 'import package.(modulename|\*);'


 Not only does this provide for clearer, more precise code.


A matter of opinion IMO, as the compiler will error on any unintentional 
ambiguities.


 it also prevents enormous amounts of runtime bloat that would
 otherwise result from loading classes that won't be used during
 program execution.


Huh? Unless i'm _severely_ mistaken, imports are completely on the 
symbollic level, and only needed code is linked in. If that's not the 
case, then we've got big problems anyway.

Arcane Jill wrote:

 Phobos (and now Deimos) uses:

 Others prefer:

 Of course, what we'd ALL prefer is:

 But D doesn't allow that (module and package with same name).


What about if
import abc.def;
(where abc.def is a package) imported the module abc.def.package, or if 
that wasn't found, all modules in abc.def (but not subpackages of abc.def)?

(I say not subpackages because subpackages could be used for internal 
implementation stuff).

Seems fairly simple and useful.

Arguments for/against?

Sam
Jul 06 2004
parent reply Regan Heath <regan netwin.co.nz> writes:
On Tue, 06 Jul 2004 21:12:50 +1200, Sam McCall <tunah.d tunah.net> wrote:

 Benji Smith wrote:


 On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
 <Arcane_member pathlink.com> wrote:



 Is there any reason why the import statement can't be made to allow:



 meaning, import all .d files in directory .../abc/def?
 (just like Java does)



 Actually, I really dislike Java's wildcard imports. It makes it
 non-obvious which sub-packages are being imported.


 No, because java doesn't have sub-packages ;-)
 Technically, java.awt.image (for example), doesn't have any magic 
 relationship with java.awt, they're two unrelated packages.


 This can be a pain
 when using a tool like grep to search for source files which import a
 particular package


 You can't import a package, unless you mean all the modules (D) or 
 class/interfaces (java) in that package, in which case search for 
 'import package.[^.]+;' for "any module in a package", or
 'import package.*;' for "all modules in a package".
 If * was available, sure, you couldn't search for "all modules in a 
 package, imported separately", but you can't do that now and it's pretty 
 meaningless.


 or module.


 Search for 'import package.(modulename|\*);'


 Not only does this provide for clearer, more precise code.


 A matter of opinion IMO, as the compiler will error on any unintentional 
 ambiguities.


 it also prevents enormous amounts of runtime bloat that would
 otherwise result from loading classes that won't be used during
 program execution.


 Huh? Unless i'm _severely_ mistaken, imports are completely on the 
 symbollic level, and only needed code is linked in. If that's not the 
 case, then we've got big problems anyway.

 Arcane Jill wrote:
  > Phobos (and now Deimos) uses:

  > Others prefer:

  > Of course, what we'd ALL prefer is:

  > But D doesn't allow that (module and package with same name).


I think having something like this is a good idea. I have just done some 
hashing routines, the modules include:

etc.crypto.hash.md2
etc.crypto.hash.md4
etc.crypto.hash.md5
etc.crypto.hash.sha0
etc.crypto.hash.sha1
etc.crypto.hash.tiger

which are intended for public importation, and

etc.crypto.hash.tigerboxes
etc.crypto.hash.misc
etc.crypto.hash.sha
etc.crypto.hash.md

which are used internally. I have created a

etc.crypto.hash.hash

module which imports the ones intended for public import, so you can 
simply import that and get them all.


What would be better is being able to say either:

import etc.crypto.hash

or

import etc.crypto.hash.*

to import everything in the etc.crypto.hash module level, but, not the 
ones below, thus I could move the internal ones to:

etc.crypto.hash.sys.tigerboxes
etc.crypto.hash.sys.misc
etc.crypto.hash.sys.sha
etc.crypto.hash.sys.md

so as they are not imported by the above import directives. I think I'll 
do this anyway, what does everyone think of the 'sys' package level name 
for internal modules? anyone got a better name? 'internal' maybe, I was 
hoping for a more terse suggestion :)


 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package


This would not be obvious when you read "import abc.def;" so I think it's 
a bad idea.


 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?


I like this.


 (I say not subpackages because subpackages could be used for internal 
 implementation stuff).


Yeah :)


 Seems fairly simple and useful.

 Arguments for/against?


I only have the 1 against that .package idea.

Regan

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 06 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <opsapm5e0b5a2sq9 digitalmars.com>, Regan Heath says...


so as they are not imported by the above import directives. I think I'll 
do this anyway, what does everyone think of the 'sys' package level name 
for internal modules? anyone got a better name? 'internal' maybe, I was 
hoping for a more terse suggestion :)


So, generalizing this, you mean...

If a package exists, (previously) called abc.def, containing modules abc.def.m1,
abc.def.m2 and abc.def.m3, then, instead of doing this (which is what we do
now):



We instead relocate m1, m2 and m3 to:





and use



to import the above? I don't think that will work, because you still have a
module and package with the same name. I think I may have misunderstood you
though.

Of course, you could instead relocate m1, m2 and m3 to:





or even:





(Simplest convention I can think of - replace the last dot with an underscore).

Once upon a time, I did something like that with the Int class. My subdirectory
was called bigint_files. It seemed rather ugly, so I changed it.

The thing is, though conventions only become conventions if used by a whole
bunch of people, and I'd rather "join in" (do what everyone else does) than
"cheerlead" (push for a particular convention).

With one exception, of course. I'd like to push for being able to import the
module:



which in turn imports





This is currently disallowed (module and package with same name), but I /really/
don't understand why this should be disallowed. The only argument against I've
heard so far is Walter saying something like "it wouldn't work", which, as
reasoned arguments go, I didn't find entirely convincing.

I'd be really interested in hearing other people's ideas on this one.

Arcane Jill
Jul 06 2004
parent Regan Heath <regan netwin.co.nz> writes:
On Tue, 6 Jul 2004 11:28:17 +0000 (UTC), Arcane Jill 
<Arcane_member pathlink.com> wrote:

 In article <opsapm5e0b5a2sq9 digitalmars.com>, Regan Heath says...


 so as they are not imported by the above import directives. I think I'll
 do this anyway, what does everyone think of the 'sys' package level name
 for internal modules? anyone got a better name? 'internal' maybe, I was
 hoping for a more terse suggestion :)


 So, generalizing this, you mean...

 If a package exists, (previously) called abc.def, containing modules 
 abc.def.m1,
 abc.def.m2 and abc.def.m3, then, instead of doing this (which is what we 
 do
 now):


 m3

 We instead relocate m1, m2 and m3 to:





 and use



 to import the above?


No. That's not what I meant to say :)

I was simply saying that import abc.def should import abc.def.* BUT not 
abc.def.*.*
soo..

abc.def.m1;
abc.def.m2;
abc.def.m3;

would all be imported by import abc.def;

if you did not want m3 to be imported you'd move it to somewhere i.e.

abc.def.sys.m3;

so that import abc.def would not get it.



 I don't think that will work, because you still have a
 module and package with the same name. I think I may have misunderstood 
 you
 though.

 Of course, you could instead relocate m1, m2 and m3 to:





 or even:





 (Simplest convention I can think of - replace the last dot with an 
 underscore).

 Once upon a time, I did something like that with the Int class. My 
 subdirectory
 was called bigint_files. It seemed rather ugly, so I changed it.

 The thing is, though conventions only become conventions if used by a 
 whole
 bunch of people, and I'd rather "join in" (do what everyone else does) 
 than
 "cheerlead" (push for a particular convention).

 With one exception, of course. I'd like to push for being able to import 
 the
 module:



 which in turn imports





 This is currently disallowed (module and package with same name), but I 
 /really/
 don't understand why this should be disallowed. The only argument 
 against I've
 heard so far is Walter saying something like "it wouldn't work", which, 
 as
 reasoned arguments go, I didn't find entirely convincing.

 I'd be really interested in hearing other people's ideas on this one.

 Arcane Jill




-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 06 2004
prev sibling parent reply Sam McCall <tunah.d tunah.net> writes:
Regan Heath wrote:


 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package


 
 
 This would not be obvious when you read "import abc.def;" so I think 
 it's a bad idea.


Firstly, the intention of "abc.def.package" would just be to import all 
relevant stuff, and possibly provide some nice aliases. If this just 
consisted of importing every module, it would be omitted.
Even so, I agree, that's exactly what I thought at first. However my 
question is: Does it need to be obvious?
In the code you write yourself, the distinction between a module and a 
package is clear: a module is a compilation unit/file/bunch of code, and 
a package is a namespace to store logically grouped code.
However when you're using someone else's library, how they split up 
their modules is less important. A module is (small) bunch of logically 
grouped code that you want to import, and a package is a (large) bunch 
of logically grouped code you want to import (and you usually achieve 
this by importing a "magic" module).
In view of this, importing packages and modules with the same syntax is 
a feature, not a bug. It hides the organisation details of the library 
code and allows library authors to switch between the two at will and 
according to preference.
At the moment, monolithic modules IMO are a pain to edit but easy to 
import and split up packages are nicer to edit but a pain for importing. 
Removing the syntactic difference should give us the best of both worlds.


 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?


 I like this.


I think there needs to be a way to override it. Sometimes you _do_ want 
your subpackages pulled in (and obviously, sometimes you don't).
Also, people may have private implementation packages directly in the 
package. IMO, the language shouldn't force people to use subpackages.


 so as they are not imported by the above import directives. I think
 I'll do this anyway, what does everyone think of the 'sys' package
 level name for internal modules? anyone got a better name? 'internal' 
 maybe, I was hoping for a more terse suggestion  :)


What about "impl"? (standing for implementation). At least in java, it's 
common as a classname suffix and I've seen it used for packages.

Sam
Jul 06 2004
next sibling parent reply Arcane Jill <Arcane_member pathlink.com> writes:
In article <cce9r5$2v52$1 digitaldaemon.com>, Sam McCall says...


In view of this, importing packages and modules with the same syntax is 
a feature, not a bug.


Well, it would be, if you could do it. But so far as I am aware, importing
packages is currently not possible with /any/ syntax.

If you tell me I'm wrong, I'll be really happy.



It hides the organisation details of the library 
code and allows library authors to switch between the two at will and 
according to preference.


Yes of course. But are you describing an existing feature or a wished-for
feature?


Arcane Jill
Jul 06 2004
parent Sam McCall <tunah.d tunah.net> writes:
Arcane Jill wrote:

 Yes of course. But are you describing an existing feature or a wished-for
 feature?


The latter, I'm afraid, I was trying to address Regan's concerns about a 
suggestion I made:

I wrote:

 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package, or
 if that wasn't found, all modules in abc.def (but not subpackages of
 abc.def)?
 (I say not subpackages because subpackages could be used for internal
 implementation stuff).

 Seems fairly simple and useful.


Sam
Jul 06 2004
prev sibling parent Regan Heath <regan netwin.co.nz> writes:
On Wed, 07 Jul 2004 01:38:38 +1200, Sam McCall <tunah.d tunah.net> wrote:

 Regan Heath wrote:


 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package



 This would not be obvious when you read "import abc.def;" so I think 
 it's a bad idea.


 Firstly, the intention of "abc.def.package" would just be to import all 
 relevant stuff, and possibly provide some nice aliases. If this just 
 consisted of importing every module, it would be omitted.
 Even so, I agree, that's exactly what I thought at first. However my 
 question is: Does it need to be obvious?


It makes it easier to learn. If anyone can look at it and make a guess at 
what it does and be right most of the time, that's a good thing.

I don't like hidden behaviour, to me suggesting that "import abc.def;" 
looks for abc.def.package is describing hidden behaviour. I'd prefer 
"import abc.def;" simply imported everything in that package i.e. 
abc.def.* but not abc.def.*.*


 In the code you write yourself, the distinction between a module and a 
 package is clear: a module is a compilation unit/file/bunch of code, and 
 a package is a namespace to store logically grouped code.
 However when you're using someone else's library, how they split up 
 their modules is less important. A module is (small) bunch of logically 
 grouped code that you want to import, and a package is a (large) bunch 
 of logically grouped code you want to import (and you usually achieve 
 this by importing a "magic" module).
 In view of this, importing packages and modules with the same syntax is 
 a feature, not a bug.


I agree. I say:

import x.y;

should import module 'y' in package 'x'. OR all modules in package 'y', in 
package 'x'. depending on whether y is a module or a package.

The person doing this import does not and should not care whether y is a 
module or a package.


 It hides the organisation details of the library code and allows library 
 authors to switch between the two at will and according to preference.


Agreed.


 At the moment, monolithic modules IMO are a pain to edit but easy to 
 import and split up packages are nicer to edit but a pain for importing. 
 Removing the syntactic difference should give us the best of both worlds.


 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?


 I like this.


 I think there needs to be a way to override it. Sometimes you _do_ want 
 your subpackages pulled in (and obviously, sometimes you don't).
 Also, people may have private implementation packages directly in the 
 package. IMO, the language shouldn't force people to use subpackages.


I'm on the fence here, on one hand I think it *should* force them to have 
sub-packages, as that forces a consistent organisation to the code. On the 
other, I don't like being told what to do.

The other alternative is to allow module declarations like:

   public module a.foobar;

which simply means it *is* imported when the package 'a' is imported, or 
alternately

   private module a.foobar;

which means it *is not* imported when the package 'a' is imported.

I think I prefer forcing people to organise the internals in sub-packages.


  > so as they are not imported by the above import directives. I think
  > I'll do this anyway, what does everyone think of the 'sys' package
  > level name for internal modules? anyone got a better name? 'internal' 
  > maybe, I was hoping for a more terse suggestion  :)
 What about "impl"? (standing for implementation). At least in java, it's 
 common as a classname suffix and I've seen it used for packages.


I like it.

Regan



-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 06 2004