Digital Mars - digitalmars.D - Module/Package "Best Practices"

↑ ↓ ← → "news.digitalmars.com" <dbarrett quinthar.com> writes:

What's the "best practice" surrounding modules and packages?

I want to have a library of reusable code that I can import into an
app-specific .d file with a single statement.  This library is composed of
dozens of classes, each in its own file/module.

In C++ I'd create a single global.h file that includes the .h files for all
library classes.  In this way, the app-specific code can include the entire
library with a single #include statement.

In D I *think* I can do the same thing: just create a global.d file that
imports all the .d files of the library.  Thus the app-specific code just
imports the library.global module, and it gets the whole thing.

Is this the way it's supposed to be done?

I'd prefer some kind of wildcard import, like:

#
# import library.*;
#

But that doesn't appear to work.

What have people been doing to organize and separate their reusable and
app-specific code?

-david

Jul 02 2004

↑ ↓ ← → Charlie <Charlie_member pathlink.com> writes:

For libraries thats what ive done, create module project.api; which contains all
nessecary imports.  

For most other imports its usually necessary to import them at the class level
to avoid forward reference problems, which for large projects can be a show
stopper ( DWT I know has struggled with it ).

Chuck

In article <cc5ipd$22jv$1 digitaldaemon.com>, news.digitalmars.com says...
What's the "best practice" surrounding modules and packages?

I want to have a library of reusable code that I can import into an
app-specific .d file with a single statement.  This library is composed of
dozens of classes, each in its own file/module.

In C++ I'd create a single global.h file that includes the .h files for all
library classes.  In this way, the app-specific code can include the entire
library with a single #include statement.

In D I *think* I can do the same thing: just create a global.d file that
imports all the .d files of the library.  Thus the app-specific code just
imports the library.global module, and it gets the whole thing.

Is this the way it's supposed to be done?

I'd prefer some kind of wildcard import, like:

#
# import library.*;
#

But that doesn't appear to work.

What have people been doing to organize and separate their reusable and
app-specific code?

-david

Jul 03 2004

↑ ↓ ← → Andy Friesen <andy ikagames.com> writes:

news.digitalmars.com wrote:

 What's the "best practice" surrounding modules and packages?
 
 I want to have a library of reusable code that I can import into an
 app-specific .d file with a single statement.  This library is composed of
 dozens of classes, each in its own file/module.
 
 In D I *think* I can ... just create a global.d file that
 imports all the .d files of the library.  Thus the app-specific code just
 imports the library.global module, and it gets the whole thing.
 
 Is this the way it's supposed to be done?
 
 I'd prefer some kind of wildcard import, like:
 
 #
 # import library.*;
 #
 
 But that doesn't appear to work.
 
 What have people been doing to organize and separate their reusable and
 app-specific code?


I just make a module of the same name as the package.  It's a tad 
awkward to say

     import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.  It'd be nice to have an explicit convention for this sort of 
thing, but I'm happy as long as it says what it is. :)

  -- andy

Jul 03 2004

↑ ↓ ← → Arcane Jill <Arcane_member pathlink.com> writes:

In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...

I just make a module of the same name as the package.  It's a tad 
awkward to say

     import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.


Yeah, that's what I do too. Hence etc.bigint.bigint, etc.crypto.crypto and
etc.unicode.unicode. (The latter two aren't actually there yet).


It'd be nice to have an explicit convention for this sort of 
thing, but I'm happy as long as it says what it is. :)


Well, I borrowed the idea from std.windows.windows and std.linux.linux. I guess
if Phobos can do it, that gotta at least count towards a convention.

Arcane Jill

Jul 03 2004

↑ ↓ ← → Ant <Ant_member pathlink.com> writes:

In article <cc76jk$1i2p$1 digitaldaemon.com>, Arcane Jill says...
In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...

I just make a module of the same name as the package.  It's a tad 
awkward to say

     import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.


 etc.bigint.bigint
 etc.crypto.crypto 
 etc.unicode.unicode
 std.windows.windows
 std.linux.linux


I don't like it, it's not really descriptive.
both all.d and api.d look better to me.

DUI has "All.d" but it will change to "all.d"

but this is a minor issue.

Ant

Jul 03 2004

↑ ↓ ← → Sam McCall <tunah.d tunah.net> writes:

Ant wrote:
 In article <cc76jk$1i2p$1 digitaldaemon.com>, Arcane Jill says...
 
In article <cc6qas$112n$1 digitaldaemon.com>, Andy Friesen says...


I just make a module of the same name as the package.  It's a tad 
awkward to say

    import libfork.libfork;

But I think the meaning is clear enough.  I've seen 'all.d' used as 
well.


etc.bigint.bigint
etc.crypto.crypto 
etc.unicode.unicode
std.windows.windows
std.linux.linux


 
 I don't like it, it's not really descriptive.
 both all.d and api.d look better to me.
 
 DUI has "All.d" but it will change to "all.d"
 
 but this is a minor issue.
 
 Ant
 
 


Walter, can we have * added to the valid identifier characters? ;-)
</Java>

Sam

Jul 04 2004

↑ ↓ ← → Gold Dragon <dragonwing dragonu.net> writes:

I was thinking the same thing, but then again maybe you wouldn't want to 
include all of the modules. What Java does mostly with the classes that 
Sun created are built in to the VM so that when you compile the code, 
you aren't actually including any of those classes. Just opcodes of the 
classes so that the VM can tell what class you were using. This allows 
you to use the '*' without having bloat. D would only include a module 
once in the program but would include/import ALL of the modules if you 
use them or not.

If it can tell if you used a module or not then I would agree that it 
would be useful.

 Is there any reason why the import statement can't be made to allow:
 
 #    import abc.def.*;
 
 meaning, import all .d files in directory .../abc/def?
 (just like Java does)
 
 Arcane Jill

Jul 05 2004

↑ ↓ ← → Benji Smith <dlanguage xxagg.com> writes:

On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
<Arcane_member pathlink.com> wrote:

Is there any reason why the import statement can't be made to allow:

#    import abc.def.*;

meaning, import all .d files in directory .../abc/def?
(just like Java does)


Actually, I really dislike Java's wildcard imports. It makes it
non-obvious which sub-packages are being imported. This can be a pain
when using a tool like grep to search for source files which import a
particular package or module.

As a matter of style, when I'm writing Java code, I always include a
separate import statement for each distinct package/module that I'll
be using. Not only does this provide for clearer, more precise code,
it also prevents enormous amounts of runtime bloat that would
otherwise result from loading classes that won't be used during
program execution. (I imagine that a similar amount of bloat is
avoided in D by getting rid of the wildcard imports.)

That's my 2c, anyhow.

--Benji Smith

Jul 05 2004

↑ ↓ ← → Arcane Jill <Arcane_member pathlink.com> writes:

In article <g6sje0po6iulbho0t8j5fab8gga4nr1kkb 4ax.com>, Benji Smith says...
On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
<Arcane_member pathlink.com> wrote:

Is there any reason why the import statement can't be made to allow:

#    import abc.def.*;

meaning, import all .d files in directory .../abc/def?
(just like Java does)


As a matter of style, when I'm writing Java code, I always include a
separate import statement for each distinct package/module that I'll
be using.


Understood, and actually I agree with you. However, that's not really what we're
talking about. Suppose I write a "distinct package" comprising two hundred
individual files. It would makes sense for me to provide a single importable
module which imports all the others. The original question on this thread was
"What should that module be called"?

Phobos (and now Deimos) uses:

#    import abc.def.def;

Others prefer:

#    import abc.def.all;

Of course, what we'd ALL prefer is:

#    import abc.def;

But D doesn't allow that (module and package with same name). And so, as a kind
of "second choice", I ventured that:

#    import abc.def.*;

might be an acceptable alternative. I guess what we really need here is a
proclaimation in the style-guide. I'm not hung up on ANY particular convention,
but I do want "what people expect" to "just work", and what people expect is
entirely dependent on consistency across multiple projects.

Arcane Jill

Jul 06 2004

↑ ↓ ← → Sam McCall <tunah.d tunah.net> writes:

Benji Smith wrote:

 On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
 <Arcane_member pathlink.com> wrote:
 
 
Is there any reason why the import statement can't be made to allow:

#    import abc.def.*;

meaning, import all .d files in directory .../abc/def?
(just like Java does)


 
 Actually, I really dislike Java's wildcard imports. It makes it
 non-obvious which sub-packages are being imported.


Technically, java.awt.image (for example), doesn't have any magic 
relationship with java.awt, they're two unrelated packages.

 This can be a pain
 when using a tool like grep to search for source files which import a
 particular package 


class/interfaces (java) in that package, in which case search for 
'import package.[^.]+;' for "any module in a package", or
'import package.*;' for "all modules in a package".
If * was available, sure, you couldn't search for "all modules in a 
package, imported separately", but you can't do that now and it's pretty 
meaningless.

 or module.



 Not only does this provide for clearer, more precise code.


ambiguities.

 it also prevents enormous amounts of runtime bloat that would
 otherwise result from loading classes that won't be used during
 program execution.


symbollic level, and only needed code is linked in. If that's not the 
case, then we've got big problems anyway.

Arcane Jill wrote:
 Phobos (and now Deimos) uses:
 #    import abc.def.def;
 Others prefer:
 #    import abc.def.all;
 Of course, what we'd ALL prefer is:
 #    import abc.def;
 But D doesn't allow that (module and package with same name).


import abc.def;
(where abc.def is a package) imported the module abc.def.package, or if 
that wasn't found, all modules in abc.def (but not subpackages of abc.def)?

(I say not subpackages because subpackages could be used for internal 
implementation stuff).

Seems fairly simple and useful.

Arguments for/against?

Sam

Jul 06 2004

↑ ↓ ← → Regan Heath <regan netwin.co.nz> writes:

On Tue, 06 Jul 2004 21:12:50 +1200, Sam McCall <tunah.d tunah.net> wrote:
 Benji Smith wrote:

 On Mon, 5 Jul 2004 20:40:37 +0000 (UTC), Arcane Jill
 <Arcane_member pathlink.com> wrote:

 Is there any reason why the import statement can't be made to allow:

 #    import abc.def.*;

 meaning, import all .d files in directory .../abc/def?
 (just like Java does)

 Actually, I really dislike Java's wildcard imports. It makes it
 non-obvious which sub-packages are being imported.

 Technically, java.awt.image (for example), doesn't have any magic 
 relationship with java.awt, they're two unrelated packages.

 This can be a pain
 when using a tool like grep to search for source files which import a
 particular package

 class/interfaces (java) in that package, in which case search for 
 'import package.[^.]+;' for "any module in a package", or
 'import package.*;' for "all modules in a package".
 If * was available, sure, you couldn't search for "all modules in a 
 package, imported separately", but you can't do that now and it's pretty 
 meaningless.

 or module.

 Not only does this provide for clearer, more precise code.

 ambiguities.

 it also prevents enormous amounts of runtime bloat that would
 otherwise result from loading classes that won't be used during
 program execution.

 symbollic level, and only needed code is linked in. If that's not the 
 case, then we've got big problems anyway.

 Arcane Jill wrote:
  > Phobos (and now Deimos) uses:
  > #    import abc.def.def;
  > Others prefer:
  > #    import abc.def.all;
  > Of course, what we'd ALL prefer is:
  > #    import abc.def;
  > But D doesn't allow that (module and package with same name).

I think having something like this is a good idea. I have just done some 
hashing routines, the modules include:

etc.crypto.hash.md2
etc.crypto.hash.md4
etc.crypto.hash.md5
etc.crypto.hash.sha0
etc.crypto.hash.sha1
etc.crypto.hash.tiger

which are intended for public importation, and

etc.crypto.hash.tigerboxes
etc.crypto.hash.misc
etc.crypto.hash.sha
etc.crypto.hash.md

which are used internally. I have created a

etc.crypto.hash.hash

module which imports the ones intended for public import, so you can 
simply import that and get them all.

What would be better is being able to say either:

import etc.crypto.hash

or

import etc.crypto.hash.*

to import everything in the etc.crypto.hash module level, but, not the 
ones below, thus I could move the internal ones to:

etc.crypto.hash.sys.tigerboxes
etc.crypto.hash.sys.misc
etc.crypto.hash.sys.sha
etc.crypto.hash.sys.md

so as they are not imported by the above import directives. I think I'll 
do this anyway, what does everyone think of the 'sys' package level name 
for internal modules? anyone got a better name? 'internal' maybe, I was 
hoping for a more terse suggestion :)

 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package

This would not be obvious when you read "import abc.def;" so I think it's 
a bad idea.

 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?

I like this.

 (I say not subpackages because subpackages could be used for internal 
 implementation stuff).

Yeah :)

 Seems fairly simple and useful.

 Arguments for/against?

I only have the 1 against that .package idea.

Regan

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

Jul 06 2004

↑ ↓ ← → Arcane Jill <Arcane_member pathlink.com> writes:

In article <opsapm5e0b5a2sq9 digitalmars.com>, Regan Heath says...

so as they are not imported by the above import directives. I think I'll 
do this anyway, what does everyone think of the 'sys' package level name 
for internal modules? anyone got a better name? 'internal' maybe, I was 
hoping for a more terse suggestion :)


So, generalizing this, you mean...

If a package exists, (previously) called abc.def, containing modules abc.def.m1,
abc.def.m2 and abc.def.m3, then, instead of doing this (which is what we do
now):

# Create a module abc.def.def (or abc.def.all) which imports m1, m2 and m3

We instead relocate m1, m2 and m3 to:

#    abc.def.sys.m1
#    abc.def.sys.m2
#    abc.def.sys.m3

and use

#    abc.def

to import the above? I don't think that will work, because you still have a
module and package with the same name. I think I may have misunderstood you
though.

Of course, you could instead relocate m1, m2 and m3 to:

#    abc.def_sys.m1
#    abc.def_sys.m2
#    abc.def_sys.m3

or even:

#    abc.def_m1
#    abc.def_m2
#    abc.def_m3

(Simplest convention I can think of - replace the last dot with an underscore).

Once upon a time, I did something like that with the Int class. My subdirectory
was called bigint_files. It seemed rather ugly, so I changed it.

The thing is, though conventions only become conventions if used by a whole
bunch of people, and I'd rather "join in" (do what everyone else does) than
"cheerlead" (push for a particular convention).

With one exception, of course. I'd like to push for being able to import the
module:

#    abc.def

which in turn imports

#    abc.def.m1
#    abc.def.m2
#    abc.def.m3

This is currently disallowed (module and package with same name), but I /really/
don't understand why this should be disallowed. The only argument against I've
heard so far is Walter saying something like "it wouldn't work", which, as
reasoned arguments go, I didn't find entirely convincing.

I'd be really interested in hearing other people's ideas on this one.

Arcane Jill

Jul 06 2004

↑ ↓ ← → Regan Heath <regan netwin.co.nz> writes:

On Tue, 6 Jul 2004 11:28:17 +0000 (UTC), Arcane Jill 
<Arcane_member pathlink.com> wrote:
 In article <opsapm5e0b5a2sq9 digitalmars.com>, Regan Heath says...

 so as they are not imported by the above import directives. I think I'll
 do this anyway, what does everyone think of the 'sys' package level name
 for internal modules? anyone got a better name? 'internal' maybe, I was
 hoping for a more terse suggestion :)


 So, generalizing this, you mean...

 If a package exists, (previously) called abc.def, containing modules 
 abc.def.m1,
 abc.def.m2 and abc.def.m3, then, instead of doing this (which is what we 
 do
 now):

 # Create a module abc.def.def (or abc.def.all) which imports m1, m2 and 
 m3

 We instead relocate m1, m2 and m3 to:

 #    abc.def.sys.m1
 #    abc.def.sys.m2
 #    abc.def.sys.m3

 and use

 #    abc.def

 to import the above?


No. That's not what I meant to say :)

I was simply saying that import abc.def should import abc.def.* BUT not 
abc.def.*.*
soo..

abc.def.m1;
abc.def.m2;
abc.def.m3;

would all be imported by import abc.def;

if you did not want m3 to be imported you'd move it to somewhere i.e.

abc.def.sys.m3;

so that import abc.def would not get it.


 I don't think that will work, because you still have a
 module and package with the same name. I think I may have misunderstood 
 you
 though.

 Of course, you could instead relocate m1, m2 and m3 to:

 #    abc.def_sys.m1
 #    abc.def_sys.m2
 #    abc.def_sys.m3

 or even:

 #    abc.def_m1
 #    abc.def_m2
 #    abc.def_m3

 (Simplest convention I can think of - replace the last dot with an 
 underscore).

 Once upon a time, I did something like that with the Int class. My 
 subdirectory
 was called bigint_files. It seemed rather ugly, so I changed it.

 The thing is, though conventions only become conventions if used by a 
 whole
 bunch of people, and I'd rather "join in" (do what everyone else does) 
 than
 "cheerlead" (push for a particular convention).

 With one exception, of course. I'd like to push for being able to import 
 the
 module:

 #    abc.def

 which in turn imports

 #    abc.def.m1
 #    abc.def.m2
 #    abc.def.m3

 This is currently disallowed (module and package with same name), but I 
 /really/
 don't understand why this should be disallowed. The only argument 
 against I've
 heard so far is Walter saying something like "it wouldn't work", which, 
 as
 reasoned arguments go, I didn't find entirely convincing.

 I'd be really interested in hearing other people's ideas on this one.

 Arcane Jill




-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

Jul 06 2004

↑ ↓ ← → Sam McCall <tunah.d tunah.net> writes:

Regan Heath wrote:

 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package


 
 This would not be obvious when you read "import abc.def;" so I think 
 it's a bad idea.


relevant stuff, and possibly provide some nice aliases. If this just 
consisted of importing every module, it would be omitted.
Even so, I agree, that's exactly what I thought at first. However my 
question is: Does it need to be obvious?
In the code you write yourself, the distinction between a module and a 
package is clear: a module is a compilation unit/file/bunch of code, and 
a package is a namespace to store logically grouped code.
However when you're using someone else's library, how they split up 
their modules is less important. A module is (small) bunch of logically 
grouped code that you want to import, and a package is a (large) bunch 
of logically grouped code you want to import (and you usually achieve 
this by importing a "magic" module).
In view of this, importing packages and modules with the same syntax is 
a feature, not a bug. It hides the organisation details of the library 
code and allows library authors to switch between the two at will and 
according to preference.
At the moment, monolithic modules IMO are a pain to edit but easy to 
import and split up packages are nicer to edit but a pain for importing. 
Removing the syntactic difference should give us the best of both worlds.

 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?




your subpackages pulled in (and obviously, sometimes you don't).
Also, people may have private implementation packages directly in the 
package. IMO, the language shouldn't force people to use subpackages.

 so as they are not imported by the above import directives. I think
 I'll do this anyway, what does everyone think of the 'sys' package
 level name for internal modules? anyone got a better name? 'internal' 
 maybe, I was hoping for a more terse suggestion  :)


common as a classname suffix and I've seen it used for packages.

Sam

Jul 06 2004

↑ ↓ ← → Arcane Jill <Arcane_member pathlink.com> writes:

In article <cce9r5$2v52$1 digitaldaemon.com>, Sam McCall says...

In view of this, importing packages and modules with the same syntax is 
a feature, not a bug.


Well, it would be, if you could do it. But so far as I am aware, importing
packages is currently not possible with /any/ syntax.

If you tell me I'm wrong, I'll be really happy.


It hides the organisation details of the library 
code and allows library authors to switch between the two at will and 
according to preference.


Yes of course. But are you describing an existing feature or a wished-for
feature?


Arcane Jill

Jul 06 2004

↑ ↓ ← → Regan Heath <regan netwin.co.nz> writes:

On Wed, 07 Jul 2004 01:38:38 +1200, Sam McCall <tunah.d tunah.net> wrote:
 Regan Heath wrote:

 What about if
 import abc.def;
 (where abc.def is a package) imported the module abc.def.package

 This would not be obvious when you read "import abc.def;" so I think 
 it's a bad idea.

 relevant stuff, and possibly provide some nice aliases. If this just 
 consisted of importing every module, it would be omitted.
 Even so, I agree, that's exactly what I thought at first. However my 
 question is: Does it need to be obvious?

It makes it easier to learn. If anyone can look at it and make a guess at 
what it does and be right most of the time, that's a good thing.

I don't like hidden behaviour, to me suggesting that "import abc.def;" 
looks for abc.def.package is describing hidden behaviour. I'd prefer 
"import abc.def;" simply imported everything in that package i.e. 
abc.def.* but not abc.def.*.*

 In the code you write yourself, the distinction between a module and a 
 package is clear: a module is a compilation unit/file/bunch of code, and 
 a package is a namespace to store logically grouped code.
 However when you're using someone else's library, how they split up 
 their modules is less important. A module is (small) bunch of logically 
 grouped code that you want to import, and a package is a (large) bunch 
 of logically grouped code you want to import (and you usually achieve 
 this by importing a "magic" module).
 In view of this, importing packages and modules with the same syntax is 
 a feature, not a bug.

I agree. I say:

import x.y;

should import module 'y' in package 'x'. OR all modules in package 'y', in 
package 'x'. depending on whether y is a module or a package.

The person doing this import does not and should not care whether y is a 
module or a package.

 It hides the organisation details of the library code and allows library 
 authors to switch between the two at will and according to preference.

Agreed.

 At the moment, monolithic modules IMO are a pain to edit but easy to 
 import and split up packages are nicer to edit but a pain for importing. 
 Removing the syntactic difference should give us the best of both worlds.

 , or if that wasn't found, all modules in abc.def (but not subpackages 
 of abc.def)?

 your subpackages pulled in (and obviously, sometimes you don't).
 Also, people may have private implementation packages directly in the 
 package. IMO, the language shouldn't force people to use subpackages.

I'm on the fence here, on one hand I think it *should* force them to have 
sub-packages, as that forces a consistent organisation to the code. On the 
other, I don't like being told what to do.

The other alternative is to allow module declarations like:

   public module a.foobar;

which simply means it *is* imported when the package 'a' is imported, or 
alternately

   private module a.foobar;

which means it *is not* imported when the package 'a' is imported.

I think I prefer forcing people to organise the internals in sub-packages.

  > so as they are not imported by the above import directives. I think
  > I'll do this anyway, what does everyone think of the 'sys' package
  > level name for internal modules? anyone got a better name? 'internal' 
  > maybe, I was hoping for a more terse suggestion  :)
 What about "impl"? (standing for implementation). At least in java, it's 
 common as a classname suffix and I've seen it used for packages.

I like it.

Regan

-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

Jul 06 2004