www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Proposal for a standard for D library naming

reply Gregor Richards <Richards codu.org> writes:
(I'm reposting this, since my last post had some issues)

Since there aren't many libraries for D (that is, builds that produce 
.so files or the ilk), there's no standard on file naming for libraries. 
  I'd like to interject a standard before it starts getting chaotic (as 
with every other language there is).

Please respond with any opinions.


A Modest Proposal for Standardization in Naming of D Libraries
--------------------------------------------------------------
(by Gregor Richards)


Outline
-------

At present, there is to my knowledge no standard for naming D
libraries (that is, .a, .lib, .so and .dll files). There's no
immediate problem posed, as there is very little D code that forms
libraries - but with any luck, the amount of code doing so will
increase.

For background on me and why I'm interested in this, both my job and
several of my personal projects involve compilation and linking of
software in UNIX environments, so I'm constantly exposed to
inconsistent libraries.

My experience with the D community has shown that few people at the
moment seem to care about the creation of libraries, but I believe
that desire will increase as popularity does. I consider the fact
that there is no standard for naming to be a significant problem, as
a lack of a clear naming convention will likely make the entire
compilation process difficult (think of autoconf).

I intend to outline in this document a naming convention for library
files which will be easily parseable by both humans and software
(such as [d]build).

Note that this document does not currently target Windows .dll and
.lib files, as I know nothing whatsoever about them. If anyone would
like to step in and make modifications, feel free.


Don't Break from Convention
---------------------------

(This section applies only to UNIX)

The UNIX convention for naming of libraries is:

lib<name>.so.<major>.<minor>.<revision>

with symlinks for each of:

lib<name>.so.<major>.<minor>
lib<name>.so.<major>

and also a symlink of:

lib<name>.so

if the library has its headers installed. There is no compelling
reason to break from this standard, so it is left as such in this
document.

For .a files, the file name is more simple:

lib<name>.a


Packages and Libraries
----------------------

Package names should be directly reflected in the file name of the
library. Furthermore, the fact that this library is written in D
should also be in the name of the library. To this end, the basic
form is:

libD.<package name>.<extension>

Of course, packages are more complex than that, with subpackages,
etc. The division becomes more complex for packages such as:

a.*
a.b.*
a.c.*
a.d.*

Each library must only be inclusive of /modules/ in the package, not
subpackages (though it may be either at the creator's whimsy). One
way to comply would be to simply have a single file:

libD.a.<extension>

However, if one wanted to subdivide, it could also have several:

libD.a.b.<extension>
libD.a.c.<extension>
libD.a.d.<extension>

Now let's imagine the situation of the package 'a' having a module
named 'foo'. a.foo doesn't belong in its own library (we've already
determined that libraries correspond to packages), so it will go into
its own file:

libD.a.<extension>

With this subdivision, we now have four files:

libD.a.<extension>
libD.a.b.<extension>
libD.a.c.<extension>
libD.a.d.<extension>

Now, in all likelihood, one or all of the subpackages depend on
modules in the superpackage. Luckily, this is trivial to detect in a
build system. Since all of the imports will be iterated over, a list
of packages to test for can be built that is guaranteed to capture
everything (so long as the installation is complete).


Version Hell
------------

(Or DLL Hell as it's affectionately known on Windows)

Version Hell isn't too bad with UNIX-style .so files, so long as
people comply to this simple standard:

1) Only increase the major version if you've made non-backwards-
    compatible changes.
2) Increase the minor version when you've added but not removed
    symbols.
3) Increase the revision for all other changes.
4) Don't worry about D ABI changes - they're well beyond the scope of
    library naming convention.

I would recommend that a .DLL version of this spec incorporate some
sort of versioning similar to UNIX.


Build Tools
-----------

With this setup, a build tool (such as build) has a very trivial task
to generate a library name from an import. It simply needs to,
starting with the most specific and expanding to the most general,
check whether files corresponding to the name exist. So, if an import
for a.b.c.d.e.f.g had been specified and the imported file was not in
the included source, it would search (in this order):

libD.a.b.c.d.e.f.g.so
libD.a.b.c.d.e.f.g.a
libD.a.b.c.d.e.f.so
libD.a.b.c.d.e.f.a
libD.a.b.c.d.e.so
libD.a.b.c.d.e.a
libD.a.b.c.d.so
libD.a.b.c.d.a
libD.a.b.c.so
libD.a.b.c.a
libD.a.b.so
libD.a.b.a
libD.a.so
libD.a.a

If none of them were found, it fails. Because each file carries its
own dependencies, the build tool only needs to link against the one
it found (and know how to read .add files) to get all of the symbols
resolved.

The generation of these files is a bit more complex, but any build
tool would only need a tiny bit of input from the user (exactly where
to subdivide) to be able to produce them.


Questions?
----------

If anything is confusing here, please respond, and I will attempt to
clarify. It's all quite clear in my head :)
Sep 18 2006
next sibling parent Derek Parnell <derek nomail.afraid.org> writes:
On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:

I was wondering if this sort of thing might be better on the D Wiki 

    http://www.prowiki.org/wiki4d/wiki.cgi?FrontPage

instead of this newsgroup. You could always put a link to the wiki entry on
the newsgroup, but the wiki is a good place to host discussions.



-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocrity!"
19/09/2006 3:00:48 PM
Sep 18 2006
prev sibling next sibling parent Gregor Richards <Richards codu.org> writes:
Per Derek's suggestion, I've put this on wiki4d:

http://www.prowiki.org/wiki4d/wiki.cgi?LibraryNamingConvention

Feel free to respond via either medium.

  - Gregor Richards
Sep 18 2006
prev sibling next sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Gregor Richards wrote:


 (This section applies only to UNIX)
 
 The UNIX convention for naming of libraries is:
 
 lib<name>.so.<major>.<minor>.<revision>
 
 with symlinks for each of:
 
 lib<name>.so.<major>.<minor>
 lib<name>.so.<major>
 
 and also a symlink of:
 
 lib<name>.so
 
 if the library has its headers installed. There is no compelling
 reason to break from this standard, so it is left as such in this
 document.


Not to rain on your parade, but Darwin libraries use:

lib<name>.<major>.<minor>.<revision>.dylib
lib<name>.<major>.<minor>.dylib ->
lib<name>.<major>.dylib ->
lib<name>.dylib ->

So you might want to make the "so" name flexible... ?

Using the -L and -l flags works as expected, though.
(With GDC that is, DMD uses those flags differently)

Everything else applies.
--anders


SIDE NOTE:
However, most Mac OS X libraries are instead packaged
as "frameworks" which include *both* libs and includes.
(Similar to how the Mac OS X applications are bundled up
as "apps" which include both executables and resources)

Frameworks are linked with the -framework flag, and the
headers are automatically searched for in the system dirs.
Don't think that the D libraries will ever be packaged as
frameworks, so it only applies to libs like OpenGL and SDL.
Sep 19 2006
parent reply Gregor Richards <Richards codu.org> writes:
Anders F Björklund wrote:

 Gregor Richards wrote:
 

 (This section applies only to UNIX)

 The UNIX convention for naming of libraries is:

 lib<name>.so.<major>.<minor>.<revision>

 with symlinks for each of:

 lib<name>.so.<major>.<minor>
 lib<name>.so.<major>

 and also a symlink of:

 lib<name>.so

 if the library has its headers installed. There is no compelling
 reason to break from this standard, so it is left as such in this
 document.


 
 
 Not to rain on your parade, but Darwin libraries use:
 
 lib<name>.<major>.<minor>.<revision>.dylib
 lib<name>.<major>.<minor>.dylib ->
 lib<name>.<major>.dylib ->
 lib<name>.dylib ->
 
 So you might want to make the "so" name flexible... ?
 
 Using the -L and -l flags works as expected, though.
 (With GDC that is, DMD uses those flags differently)
 
 Everything else applies.
 --anders
 
 
 SIDE NOTE:
 However, most Mac OS X libraries are instead packaged
 as "frameworks" which include *both* libs and includes.
 (Similar to how the Mac OS X applications are bundled up
 as "apps" which include both executables and resources)
 


I didn't mean to imply that these extensions were the only ones 
possible, I used them because they were the ones I knew about.



 Frameworks are linked with the -framework flag, and the
 headers are automatically searched for in the system dirs.
 Don't think that the D libraries will ever be packaged as
 frameworks, so it only applies to libs like OpenGL and SDL.


But it /doesn't/ apply to libraries like OpenGL and SDL, they're not 
written in D, and would still need to be manually included.



 Why does the fact that it uses D have to be reflected in the name ?


So that a unique namespace can be guaranteed - remember, these library 
names are to be not only human-understandable but machine-generatable.



 I don't see any C++ libraries advertising that they are using that,
 so I still need to remember to link them to libstdc++ or get errors.


This is a good thing?



 Don't think it would be that bad to avoid prefixing them with "D.",
 and instead get similar link errors from Phobos (and the D runtime).


But what's the advantage? The D prefix is more of a note that a D 
package name is coming than something to say "this is written in D"

eg if you wrote a library in D that exports a C interface, you may be 
inclined to name it in a more traditional way, as that library is for 
use by C (or D), so the D build tool does not necessarily need to 
generate the paths in the same way.



 So I would rather continue to name them as lib<name>.a/<name>.lib...


The entire idea behind my naming proposal is that it makes everything 
100% computer and human generatable while adding very little confusion. 
  Having things named lib<name>.<extension> makes associating packages 
and libraries difficult.

Because C/++ don't have such a nice, structured package style, they 
don't have a naming convention for library files (there really couldn't 
be one), but I think that's a terrible reason to not use a naming 
convention for D.  It would building, even against libraries, 
ridiculously easy.



 --anders


  - Gregor Richards
Sep 19 2006
next sibling parent reply Gregor Richards <Richards codu.org> writes:
Gregor Richards wrote:

 Anders F Björklund wrote:
  > I don't see any C++ libraries advertising that they are using that,
  > so I still need to remember to link them to libstdc++ or get errors.
 
 This is a good thing?
 
 


Another reason for the D prefix is for bindings.  Many bindings will 
have package names similar to their C library names - so similar in fact 
that the names could overlap if there was nothing to uniquely identify 
the D one.  And if the names overlapped, it would need to break the 
naming convention, which would defeat the whole purpose of having a 
naming convention in the first place :)

  - Gregor Richards
Sep 19 2006
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Gregor Richards wrote:


 Another reason for the D prefix is for bindings.  Many bindings will 
 have package names similar to their C library names - so similar in fact 
 that the names could overlap if there was nothing to uniquely identify 
 the D one.  And if the names overlapped, it would need to break the 
 naming convention, which would defeat the whole purpose of having a 
 naming convention in the first place :)


Yes, this is a very valid reason. I used a d suffix, for same reason.

--anders
Sep 19 2006
prev sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Gregor Richards wrote:


 So you might want to make the "so" name flexible... ?


 
 I didn't mean to imply that these extensions were the only ones 
 possible, I used them because they were the ones I knew about.


No problem, just thought I'd add to the little list since
Mac OS X is identified by GDC as being a "Unix" platform...


 So I would rather continue to name them as lib<name>.a/<name>.lib...


 
 The entire idea behind my naming proposal is that it makes everything 
 100% computer and human generatable while adding very little confusion. 
  Having things named lib<name>.<extension> makes associating packages 
 and libraries difficult.


Think I missed the "auto-generated" part of the proposal... :-)

Currently I have the "wx" modules defined in a libwxd.a (or wxd.lib),
partly since wx is already the C++ library and wxc are the C wrappers.

Guess I could change that to libD.wx.a, but it looks somewhat strange.


 Because C/++ don't have such a nice, structured package style, they 
 don't have a naming convention for library files (there really couldn't 
 be one), but I think that's a terrible reason to not use a naming 
 convention for D.  It would building, even against libraries, 
 ridiculously easy.


Actually I don't find it that hard, and e.g. the difference in
which flags to use between DMD and GDC to be more "trouble"...

--anders
Sep 19 2006
parent reply Gregor Richards <Richards codu.org> writes:
Anders F Björklund wrote:

 Gregor Richards wrote:

 The entire idea behind my naming proposal is that it makes everything 
 100% computer and human generatable while adding very little 
 confusion.  Having things named lib<name>.<extension> makes 
 associating packages and libraries difficult.


 
 
 Think I missed the "auto-generated" part of the proposal... :-)
 
 Currently I have the "wx" modules defined in a libwxd.a (or wxd.lib),
 partly since wx is already the C++ library and wxc are the C wrappers.
 
 Guess I could change that to libD.wx.a, but it looks somewhat strange.
 


Yes, but if you've ever seen the naming convention, it would be easily 
recognizeable.

Also, I wouldn't be particularly opposed to moving the 'D' to the end, I 
just think that makes the suffixing a but confusing *shrugs*



 Because C/++ don't have such a nice, structured package style, they 
 don't have a naming convention for library files (there really 
 couldn't be one), but I think that's a terrible reason to not use a 
 naming convention for D.  It would building, even against libraries, 
 ridiculously easy.


 
 
 Actually I don't find it that hard, and e.g. the difference in
 which flags to use between DMD and GDC to be more "trouble"...
 


Remember, the entire purpose of this is for a build tool (say, build) to 
conform to a standard.  Only the build tool needs to figure out the 
flags for DMD/GDC, the person compiling should get the libraries "for free."



 --anders


  - Gregor Richards
Sep 19 2006
parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Gregor Richards wrote:


 Also, I wouldn't be particularly opposed to moving the 'D' to the end, I 
 just think that makes the suffixing a but confusing *shrugs*


I think there is a Windows standard that libraries with d suffix
means "debugging" versions, so a D prefix might actually be better.


 Remember, the entire purpose of this is for a build tool (say, build) to 
 conform to a standard.  Only the build tool needs to figure out the 
 flags for DMD/GDC, the person compiling should get the libraries "for 
 free."


I haven't gotten Build to work for me, so I am still using Make.
To make things easier, I am using "gcc" to link - even for DMD.

--anders
Sep 19 2006
parent reply Sean Kelly <sean f4.ca> writes:
Anders F Björklund wrote:

 Gregor Richards wrote:
 

 Also, I wouldn't be particularly opposed to moving the 'D' to the end, 
 I just think that makes the suffixing a but confusing *shrugs*


 
 I think there is a Windows standard that libraries with d suffix
 means "debugging" versions, so a D prefix might actually be better.


This is true, and I agree.


 Remember, the entire purpose of this is for a build tool (say, build) 
 to conform to a standard.  Only the build tool needs to figure out the 
 flags for DMD/GDC, the person compiling should get the libraries "for 
 free."


 
 I haven't gotten Build to work for me, so I am still using Make.
 To make things easier, I am using "gcc" to link - even for DMD.


I didn't use Build until recently, and now I wouldn't do without it. 
Build makes using D similar to using Java but without the recompilation 
issues--ie. it "just works".  If Build could be made to work with 
different library designs in a clean and simple manner it would feel 
like cheating.


Sean
Sep 19 2006
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Sean Kelly wrote:


 I didn't use Build until recently, and now I wouldn't do without it. 
 Build makes using D similar to using Java but without the recompilation 
 issues--ie. it "just works".  If Build could be made to work with 
 different library designs in a clean and simple manner it would feel 
 like cheating.


Build is like that cool power tool, that refuses to start when I try it.
Make is like that old rusty grandpa tool, that eventually does the job.

I'll switch, eventually. Meanwhile, make will do the job just fine. ;-)
--anders
Sep 19 2006
prev sibling next sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Gregor Richards wrote:


 Package names should be directly reflected in the file name of the
 library. Furthermore, the fact that this library is written in D
 should also be in the name of the library. To this end, the basic
 form is:
 
 libD.<package name>.<extension>


Why does the fact that it uses D have to be reflected in the name ?

I don't see any C++ libraries advertising that they are using that,
so I still need to remember to link them to libstdc++ or get errors.
Don't think it would be that bad to avoid prefixing them with "D.",
and instead get similar link errors from Phobos (and the D runtime).

So I would rather continue to name them as lib<name>.a/<name>.lib...

--anders
Sep 19 2006
prev sibling next sibling parent reply =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Gregor Richards wrote:


 A Modest Proposal for Standardization in Naming of D Libraries
 --------------------------------------------------------------
 (by Gregor Richards)


[...]

 If anything is confusing here, please respond, and I will attempt to
 clarify. It's all quite clear in my head :)


Just to check I understood this, would these be standard names:

/usr/include/d/sdl/*
/usr/lib/libD.sdl.a

/usr/include/d/gl/*
/usr/lib/libD.gl.a

/usr/include/d/wx/*
/usr/lib/libD.wx.a

For the modules under "sdl." and "gl." and "wx." , respectively ?

--anders
Sep 19 2006
parent reply Gregor Richards <Richards codu.org> writes:
Anders F Björklund wrote:

 Gregor Richards wrote:
 

 A Modest Proposal for Standardization in Naming of D Libraries
 --------------------------------------------------------------
 (by Gregor Richards)


 [...]

 If anything is confusing here, please respond, and I will attempt to
 clarify. It's all quite clear in my head :)


 
 Just to check I understood this, would these be standard names:
 
 /usr/include/d/sdl/*
 /usr/lib/libD.sdl.a
 
 /usr/include/d/gl/*
 /usr/lib/libD.gl.a
 
 /usr/include/d/wx/*
 /usr/lib/libD.wx.a
 
 For the modules under "sdl." and "gl." and "wx." , respectively ?
 
 --anders


That would be one means of dividing it.

Mind you, I'm not inclusive of include files, they are not in the scope 
of the text.

Furthermore, you could subdivide, if you wanted.  For example, if you 
have this:

/usr/include/d/sdl/amazingsdlplugina/*
/usr/include/d/sdl/*

You can do either one library:

libD.sdl.<extension>

Or more:

libD.sdl.<extension>
libD.sdl.amazingsdlplugina.<extension>

, so long as a module can always be found in the most specific, extant 
library.

  - Gregor Richards
Sep 19 2006
next sibling parent reply Sean Kelly <sean f4.ca> writes:
Gregor Richards wrote:

 Anders F Björklund wrote:

 Gregor Richards wrote:


 A Modest Proposal for Standardization in Naming of D Libraries
 --------------------------------------------------------------
 (by Gregor Richards)


 [...]

 If anything is confusing here, please respond, and I will attempt to
 clarify. It's all quite clear in my head :)


 Just to check I understood this, would these be standard names:

 /usr/include/d/sdl/*
 /usr/lib/libD.sdl.a

 /usr/include/d/gl/*
 /usr/lib/libD.gl.a

 /usr/include/d/wx/*
 /usr/lib/libD.wx.a

 For the modules under "sdl." and "gl." and "wx." , respectively ?

 --anders


 
 That would be one means of dividing it.
 
 Mind you, I'm not inclusive of include files, they are not in the scope 
 of the text.
 
 Furthermore, you could subdivide, if you wanted.  For example, if you 
 have this:
 
 /usr/include/d/sdl/amazingsdlplugina/*
 /usr/include/d/sdl/*


For D import files I settled on using /usr/import instead of 
/usr/include.  It seemed to provide a cleaner separation than sticking 
C/C++/D files all in the same tree.


Sean
Sep 19 2006
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Sean Kelly wrote:


 For D import files I settled on using /usr/import instead of 
 /usr/include.  It seemed to provide a cleaner separation than sticking 
 C/C++/D files all in the same tree.


Sure, but /usr/import is not in the Filesystem Hierarchy Standard (FHS).

And I don't really have a problem with combining C/C++/D together. But
if it is ever important, "import" can be a symlink over to "include/d".

--anders
Sep 19 2006
prev sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Gregor Richards wrote:


 That would be one means of dividing it.
 
 Mind you, I'm not inclusive of include files, they are not in the scope 
 of the text.


I was just trying to make it more concrete, since those are
the libraries that I am using for SDL, OpenGL and wxWidgets.

In the RPM packages, the includes and libs travel together.


 Furthermore, you could subdivide, if you wanted. 


I really don't :-)

A minor problem is that wxD is now two libraries (wxc/wxd),
but I guess those objects could be stuffed into a libD.wx.a

--anders
Sep 19 2006
prev sibling next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Gregor Richards wrote:


 Please respond with any opinions.


Sounds good.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource & #D: larsivi
Sep 19 2006
prev sibling next sibling parent reply Gregor Richards <Richards codu.org> writes:
Since there seems to be a lot of confusion as per the purpose of this 
library naming convention, I've added the following section:

Purpose

As there is much misunderstanding of the purpose of this library naming 
convention, it's outlined here:

     * The Library Naming Convention has no bearing whatsoever over the 
names or arrangement of source files.
     * The Library Naming Convention does not intend to replace 
build-style 'build everything from source with include paths' building, 
it intends to be a superior alternative.
     * The Library Naming Convention is a convention. That is, no one is 
required to conform to it, but (were it to be accepted) it would be 
inconveniencing not to.
     * The Library Naming Convention allows the source maintainer to 
choose what packages to make into libraries, with what level of 
division, etc. It has few rules over what is and what isn't in given 
libraries, as its purpose is the naming of libraries, not the content.
           o The only rule is that any module must be in the most 
specifically-named library corresponding to that module. That is, if you 
have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in 
libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous anyway.



http://www.prowiki.org/wiki4d/wiki.cgi?LibraryNamingConvention
Sep 20 2006
parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Gregor Richards wrote:

         o The only rule is that any module must be in the most 
 specifically-named library corresponding to that module. That is, if you 
 have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in 
 libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous 
 anyway.
 
 
 


Why so? Why should a given library be splited across multiple so/dll 
files according to package, instead of being compiled in just one file?


-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Sep 23 2006
parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Bruno Medeiros wrote:


 Gregor Richards wrote:

         o The only rule is that any module must be in the most
 specifically-named library corresponding to that module. That is, if you
 have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in
 libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous
 anyway.
 
 
 


 
 Why so? Why should a given library be splited across multiple so/dll
 files according to package, instead of being compiled in just one file?
 
 


You totally missed the point, if the library _is_ split, then there should
be a standardized way to decide which part a module is in.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource & #D: larsivi
Sep 23 2006
parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Lars Ivar Igesund wrote:

 Bruno Medeiros wrote:
 

 Gregor Richards wrote:

         o The only rule is that any module must be in the most
 specifically-named library corresponding to that module. That is, if you
 have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in
 libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous
 anyway.


 Why so? Why should a given library be splited across multiple so/dll
 files according to package, instead of being compiled in just one file?


 
 You totally missed the point, if the library _is_ split, then there should
 be a standardized way to decide which part a module is in.
 


I see.

Well, But then I think it is a bad code "convention" to have 
subdivisions (packages) are not at the same level. That means that one 
should either have:
   a.*   // a lib with 'a' and all subpackages and modules
or have:
   // three related libs, each with it's subpackages and modules.
   a.b.*
   a.c.*
   a.d.*
This means that in the second option, package 'a' cannot have 
sub-modules, only sub-packages.

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Sep 26 2006
parent reply Gregor Richards <Richards codu.org> writes:
Bruno Medeiros wrote:

 Lars Ivar Igesund wrote:
 

 Bruno Medeiros wrote:


 Gregor Richards wrote:


         o The only rule is that any module must be in the most
 specifically-named library corresponding to that module. That is, if 
 you
 have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in
 libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous
 anyway.


 Why so? Why should a given library be splited across multiple so/dll
 files according to package, instead of being compiled in just one file?


 You totally missed the point, if the library _is_ split, then there 
 should
 be a standardized way to decide which part a module is in.


 
 I see.
 
 Well, But then I think it is a bad code "convention" to have 
 subdivisions (packages) are not at the same level. That means that one 
 should either have:
   a.*   // a lib with 'a' and all subpackages and modules
 or have:
   // three related libs, each with it's subpackages and modules.
   a.b.*
   a.c.*
   a.d.*
 This means that in the second option, package 'a' cannot have 
 sub-modules, only sub-packages.
 


No it doesn't - the rule is that a module is in the /most specific/ 
library.  So, if you wanted to split out a.b, a.c and a.d, but had a 
module a.mod, there would be a library libD.a.{so,a,lib,whatever}, but 
it would only contain the module a.mod, no subpackages.  A library is 
not assumed to be fully inclusive - it is assumed to be inclusive of 
everything that isn't in a more specifically-named library.

  - Gregor Richards

PS: Yes, I thought this all through :)
Sep 26 2006
parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Gregor Richards wrote:

 Bruno Medeiros wrote:

 Lars Ivar Igesund wrote:


 Bruno Medeiros wrote:


 Gregor Richards wrote:


         o The only rule is that any module must be in the most
 specifically-named library corresponding to that module. That is, 
 if you
 have libD.a.b.so.0 and libD.a.so.0, the module a.b.c should be in
 libD.a.b.so.0, not libD.a.so.0. Doing otherwise is fairly ridiculous
 anyway.


 Why so? Why should a given library be splited across multiple so/dll
 files according to package, instead of being compiled in just one file?


 You totally missed the point, if the library _is_ split, then there 
 should
 be a standardized way to decide which part a module is in.


 I see.

 Well, But then I think it is a bad code "convention" to have 
 subdivisions (packages) are not at the same level. That means that one 
 should either have:
   a.*   // a lib with 'a' and all subpackages and modules
 or have:
   // three related libs, each with it's subpackages and modules.
   a.b.*
   a.c.*
   a.d.*
 This means that in the second option, package 'a' cannot have 
 sub-modules, only sub-packages.


 
 No it doesn't - the rule is that a module is in the /most specific/ 
 library.  So, if you wanted to split out a.b, a.c and a.d, but had a 
 module a.mod, there would be a library libD.a.{so,a,lib,whatever}, but 
 it would only contain the module a.mod, no subpackages.  A library is 
 not assumed to be fully inclusive - it is assumed to be inclusive of 
 everything that isn't in a more specifically-named library.
 
  - Gregor Richards
 
 PS: Yes, I thought this all through :)


It doesn't have to in *your* rule. But I was not describing your rule, I 
was describing mine. That is, I was saying that I would prefer a rule 
where a lib named "foo.{so|a|lib|whatever}" would mean that the lib 
"foo" contains all sub-packages of 'foo', not just the sub-modules.

Since that lib is almost surely dependent on that packages. No, it is 
just not dependent one those packages, it is *made by* those packages. 
And what if those packages change between versions, can you imagine the 
confusion? Consider these two versions of the same lib:

lib.foo.1.so
lib.foo.bar.1.so
lib.foo.gok.1.so

lib.foo.2.so
lib.foo.bar.2.so
lib.foo.newgok.2.so
lib.foo.zap.2.so

Now see how they look sorted in a lib dir:

lib.foo.1.so
lib.foo.2.so
lib.foo.bar.1.so
lib.foo.bar.2.so
lib.foo.gok.1.so
lib.foo.newgok.2.so
lib.foo.zap.2.so

It's quite a mess.

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Sep 28 2006
parent reply Gregor Richards <Richards codu.org> writes:
Bruno Medeiros wrote:

 It doesn't have to in *your* rule. But I was not describing your rule, I 
 was describing mine. That is, I was saying that I would prefer a rule 
 where a lib named "foo.{so|a|lib|whatever}" would mean that the lib 
 "foo" contains all sub-packages of 'foo', not just the sub-modules.


That was my original thought.

But that doesn't work.


 
 Since that lib is almost surely dependent on that packages.


In the general case, module a.b.c is dependent on module a.c, NOT the 
reverse.  Dependency within a tree usually goes up or across the tree, 
not down.  Across is simply interlibrary dependency, up is where our 
point of contention is.


 No, it is 
 just not dependent one those packages, it is *made by* those packages. 
 And what if those packages change between versions, can you imagine the 
 confusion? Consider these two versions of the same lib:
 
 lib.foo.1.so
 lib.foo.bar.1.so
 lib.foo.gok.1.so


This is not the standard way of naming UNIX library versions, but OK, 
continuing ...


 
 lib.foo.2.so
 lib.foo.bar.2.so
 lib.foo.newgok.2.so
 lib.foo.zap.2.so
 
 Now see how they look sorted in a lib dir:
 
 lib.foo.1.so
 lib.foo.2.so
 lib.foo.bar.1.so
 lib.foo.bar.2.so
 lib.foo.gok.1.so
 lib.foo.newgok.2.so
 lib.foo.zap.2.so


Makes sense to me.


 
 It's quite a mess.
 


I see no mess.

  - Gregor Richards
Sep 28 2006
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Gregor Richards wrote:

 Bruno Medeiros wrote:

 It doesn't have to in *your* rule. But I was not describing your rule, 
 I was describing mine. That is, I was saying that I would prefer a 
 rule where a lib named "foo.{so|a|lib|whatever}" would mean that the 
 lib "foo" contains all sub-packages of 'foo', not just the sub-modules.


 
 That was my original thought.
 
 But that doesn't work.
 


Wouldn't work? Why not?


 Since that lib is almost surely dependent on that packages.


 
 In the general case, module a.b.c is dependent on module a.c, NOT the 
 reverse.  Dependency within a tree usually goes up or across the tree, 
 not down.  Across is simply interlibrary dependency, up is where our 
 point of contention is.
 


I'm uncertain of that. Ultimately it depends on how one organizes his 
code structure and modules, but I think an organization where modules 
are usually dependent on submodules/subpackages(and not parent ones) is 
more logical.
(In any case it doesn't affect my point about the lib file)


 Now see how they look sorted in a lib dir:

 lib.foo.1.so
 lib.foo.2.so
 lib.foo.bar.1.so
 lib.foo.bar.2.so
 lib.foo.gok.1.so
 lib.foo.newgok.2.so
 lib.foo.zap.2.so


 
 Makes sense to me.
 

 It's quite a mess.


 
 I see no mess.
 
  - Gregor Richards


Well then, we've hit a subjective point(opinion), so not much to argue 
then. (I like the files as clean and tidy as possible)


-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Oct 02 2006
prev sibling next sibling parent Frank Benoit <keinfarbton nospam.xyz> writes:
I like the idea of having this naming schema.

I want to add the suggestion of using the unique package naming with
reversed domain names. Like e.g. mango from dsource.org

Package: org.dsource.mango in Version 1.2.3

and the resulting lib should be in /usr/lib/

lib: libD.org.dsource.mango.so.1.2.3
lnk: libD.org.dsource.mango.so.1.2
lnk: libD.org.dsource.mango.so.1
lnk: libD.org.dsource.mango.so

I think the include directory hierarchy is close related to this. I
would suggest to generate a header tree like this:

in /usr/include/d/
dir: org.dsource.mango.1.2.3
lnk: org.dsource.mango.1.2
lnk: org.dsource.mango.1
lnk: org.dsource.mango
src: org.dsource.mango.1.2.3/org/dsource/mango/....

Perhaps, in future, it is possible to pack and deploy the headers in
some kind of jar file.

Frank
Sep 23 2006
prev sibling parent reply Derek Parnell <derek psyc.ward> writes:
On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:


 Since there aren't many libraries for D (that is, builds that produce 
 .so files or the ilk)


In fact, are there any? I vaguely remember reading that D does not yet
support the creation of .so files. Is that still the case?

Anyhow, let's break Gregor's proposal into two parts:
(a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
*nix)
(b) using shared libraries in the build process

I will defer (a) until I know how to do it :-)

With (b) I'm willing to add this feature to Build but with idea that
failure to locate a library from a package name is not a failure to build.
I'm not convinced that Gregor's idea that every (top-level) package must
also be a library is a valid idea. I can imagine people still wanting to do
static linking of object files.

-- 
Derek Parnell
Melbourne, Australia
"Down with mediocrity!"
Sep 23 2006
parent reply Gregor Richards <Richards codu.org> writes:
Derek Parnell wrote:

 On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:
 
 

Since there aren't many libraries for D (that is, builds that produce 
.so files or the ilk)


 
 
 In fact, are there any? I vaguely remember reading that D does not yet
 support the creation of .so files. Is that still the case?
 
 Anyhow, let's break Gregor's proposal into two parts:
 (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
 *nix)
 (b) using shared libraries in the build process
 
 I will defer (a) until I know how to do it :-)
 
 With (b) I'm willing to add this feature to Build but with idea that
 failure to locate a library from a package name is not a failure to build.
 I'm not convinced that Gregor's idea that every (top-level) package must
 also be a library is a valid idea. I can imagine people still wanting to do
 static linking of object files.
 


This last paragraph makes me think that I've miscommunicated my concept 
*urk*

1) Failure to locate an appropriately-named library is /not/ a failure, 
just a failure in that subsystem (if indeed the symbol can't be found 
anywhere, that'd be a failure).  This would probably be a 
report-but-continue type warning.
2) My creation of libraries is slightly more flexible than what you're 
suggesting, but less flexible than what you'd like I think.  Any 
/package/ can be a library (whether top-level or not), only making 
libraries of top-level packages is just a /default/.  The only reason 
for making this restriction is, to not place a reasonable restriction on 
what modules are contained in a library makes the naming exponentially 
complex or impossible.  Urk.

As per making .so files, DMD can't do it but GDC can.  For DMD to do it, 
it would need the equivalent of an -fPIC option, to produce 
position-independent code - the linking part is easy.  Of course, since 
either can create .a files, its implementable so long as DMD users are 
willing to surrender the ability to make .so files until it's capable. 
If build did this all in its automatic processing, the source maintainer 
wouldn't need to care whether the compiler being used is capable of 
producing libraries of any sort.

(Actually, DMD may be producing position independent code already for 
all I know, in which case it's perfectly capable, just need -L-shared - 
unfortunately, this is hard to test, since much software will "just 
work" even if it's not made position independent)

Furthermore, this proposal is not shared-library specific, it just shows 
its greatest benefit there (since the reliability in naming is carried 
through every conceivable stage of making/running a program).

  - Gregor Richards
Sep 24 2006
next sibling parent Gregor Richards <Richards codu.org> writes:
Gregor Richards wrote:

 Derek Parnell wrote:
 

 On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:



 Since there aren't many libraries for D (that is, builds that produce 
 .so files or the ilk)




 In fact, are there any? I vaguely remember reading that D does not yet
 support the creation of .so files. Is that still the case?

 Anyhow, let's break Gregor's proposal into two parts:
 (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
 *nix)
 (b) using shared libraries in the build process

 I will defer (a) until I know how to do it :-)

 With (b) I'm willing to add this feature to Build but with idea that
 failure to locate a library from a package name is not a failure to 
 build.
 I'm not convinced that Gregor's idea that every (top-level) package must
 also be a library is a valid idea. I can imagine people still wanting 
 to do
 static linking of object files.


 
 This last paragraph makes me think that I've miscommunicated my concept 
 *urk*
 
 1) Failure to locate an appropriately-named library is /not/ a failure, 
 just a failure in that subsystem (if indeed the symbol can't be found 
 anywhere, that'd be a failure).  This would probably be a 
 report-but-continue type warning.
 2) My creation of libraries is slightly more flexible than what you're 
 suggesting, but less flexible than what you'd like I think.  Any 
 /package/ can be a library (whether top-level or not), only making 
 libraries of top-level packages is just a /default/.  The only reason 
 for making this restriction is, to not place a reasonable restriction on 
 what modules are contained in a library makes the naming exponentially 
 complex or impossible.  Urk.
 
 As per making .so files, DMD can't do it but GDC can.  For DMD to do it, 
 it would need the equivalent of an -fPIC option, to produce 
 position-independent code - the linking part is easy.  Of course, since 
 either can create .a files, its implementable so long as DMD users are 
 willing to surrender the ability to make .so files until it's capable. 
 If build did this all in its automatic processing, the source maintainer 
 wouldn't need to care whether the compiler being used is capable of 
 producing libraries of any sort.
 
 (Actually, DMD may be producing position independent code already for 
 all I know, in which case it's perfectly capable, just need -L-shared - 
 unfortunately, this is hard to test, since much software will "just 
 work" even if it's not made position independent)
 
 Furthermore, this proposal is not shared-library specific, it just shows 
 its greatest benefit there (since the reliability in naming is carried 
 through every conceivable stage of making/running a program).
 
  - Gregor Richards


According to DMD's docs, it has an -fPIC flag.  I see no compelling 
reason why it shouldn't be able to create .so files, given that.

To make a .so file (just a guess):

dmd <source files> -fPIC -L-shared -L-Wl-soname=libfoo.so.0 
-oflibfoo.so.0.0.0

(In case you don't get the discrepency in naming, it's all about how ELF 
.so files are versioned: the first number after .so is essentially the 
ABI version, so it should only change when non-backwards-compatible 
changes are made.  The name that programs keep in them has one number. 
The next is generally used for backwards-compatible ABI changes, the 
next for other changes)

If you are willing to implement this in build, I'd be glad to extend my 
help, especially on getting all the UNIX stuff named/installed as would 
be traditional for a UNIX system.  (And of course I've also offered to 
make the changes myself, but I'm under the distinctive impression that 
you'd prefer to make big changes like that yourself - perfectly 
understandable)

  - Gregor Richards
Sep 24 2006
prev sibling parent reply Derek Parnell <derek nomail.afraid.org> writes:
On Sun, 24 Sep 2006 02:24:06 -0700, Gregor Richards wrote:


 Derek Parnell wrote:

 On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:
 

Since there aren't many libraries for D (that is, builds that produce 
.so files or the ilk)


 
 In fact, are there any? I vaguely remember reading that D does not yet
 support the creation of .so files. Is that still the case?
 
 Anyhow, let's break Gregor's proposal into two parts:
 (a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
 *nix)
 (b) using shared libraries in the build process
 
 I will defer (a) until I know how to do it :-)
 
 With (b) I'm willing to add this feature to Build but with idea that
 failure to locate a library from a package name is not a failure to build.
 I'm not convinced that Gregor's idea that every (top-level) package must
 also be a library is a valid idea. I can imagine people still wanting to do
 static linking of object files.
 


 
 This last paragraph makes me think that I've miscommunicated my concept 
 *urk*


Oh, I wouldn't blame my failure to understand on your ability to
communicate ;-)


 1) Failure to locate an appropriately-named library is /not/ a failure, 
 just a failure in that subsystem (if indeed the symbol can't be found 
 anywhere, that'd be a failure).  This would probably be a 
 report-but-continue type warning.


You still seem to be saying that the *only* place that module ought to
reside in is in libraries. Whereas I maintain that they *could* also reside
in object files (.o/.obj) files. So if Build doesn't find a library that
matches the import name, why can't it use the object file that maps to the
import name - and if the object file doesn't exist then why can't I have
Build compile the module's source to create it? The fact that the library
doesn't exist is not a failure then - or must there be a switch to say that
only libraries are acceptable for this build process?

Example:
  import a.b.c;

Looks for 
   libD.a.b.so
   libD.a.b.a
   libD.a.so
   libD.a.a
   a.b.c.o

and places the first one it comes across onto the linker command line. If
it finds none of these, it will compile a.b.c.d to form an object file.

And then looks for
   a.b.c.di
   a.b.c.d

to see if the source file is more recent than the 'object' file found
earlier.

So a further complication is what should Build do if the module source
files are more recent than the matching library? Currently, it recompiles
the source files and uses the resulting object files in the linkage.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocrity!"
25/09/2006 9:41:49 AM
Sep 24 2006
next sibling parent reply Gregor Richards <Richards codu.org> writes:
Derek Parnell wrote:

 On Sun, 24 Sep 2006 02:24:06 -0700, Gregor Richards wrote:
 
 

Derek Parnell wrote:


On Mon, 18 Sep 2006 21:10:46 -0700, Gregor Richards wrote:



Since there aren't many libraries for D (that is, builds that produce 
.so files or the ilk)


In fact, are there any? I vaguely remember reading that D does not yet
support the creation of .so files. Is that still the case?

Anyhow, let's break Gregor's proposal into two parts:
(a) creating shared libraries (DLL in Windows, .so in Linux, ? in other
*nix)
(b) using shared libraries in the build process

I will defer (a) until I know how to do it :-)

With (b) I'm willing to add this feature to Build but with idea that
failure to locate a library from a package name is not a failure to build.
I'm not convinced that Gregor's idea that every (top-level) package must
also be a library is a valid idea. I can imagine people still wanting to do
static linking of object files.


This last paragraph makes me think that I've miscommunicated my concept 
*urk*


 
 
 Oh, I wouldn't blame my failure to understand on your ability to
 communicate ;-)
 
 

1) Failure to locate an appropriately-named library is /not/ a failure, 
just a failure in that subsystem (if indeed the symbol can't be found 
anywhere, that'd be a failure).  This would probably be a 
report-but-continue type warning.


 
 
 You still seem to be saying that the *only* place that module ought to
 reside in is in libraries.


Aha!  This is a core misunderstanding.  All of this library stuff is 
supposed to come well after failing to find it in an object file - the 
order in my head is: source files, object files, specified libraries, 
automatically-discovered libraries.  That is, it should FIRST look for 
source, then objects, and only if failing in both of those should it 
look at libraries - libraries are solely for things that are NOT 
included with the source!


 Whereas I maintain that they *could* also reside
 in object files (.o/.obj) files. So if Build doesn't find a library that
 matches the import name, why can't it use the object file that maps to the
 import name - and if the object file doesn't exist then why can't I have
 Build compile the module's source to create it?  The fact that the library
 doesn't exist is not a failure then - or must there be a switch to say that
 only libraries are acceptable for this build process?
 
 Example:
   import a.b.c;
 
 Looks for 
    libD.a.b.so
    libD.a.b.a
    libD.a.so
    libD.a.a
    a.b.c.o
 
 and places the first one it comes across onto the linker command line. If
 it finds none of these, it will compile a.b.c.d to form an object file.
 
 And then looks for
    a.b.c.di
    a.b.c.d
 
 to see if the source file is more recent than the 'object' file found
 earlier.
 
 So a further complication is what should Build do if the module source
 files are more recent than the matching library? Currently, it recompiles
 the source files and uses the resulting object files in the linkage.
 


The timing on source/object files has no reason to change - the only 
issue is with conflicts, and in my opinion source should /always/ trump 
libraries on conflicts.

  - Gregor Richards
Sep 24 2006
parent reply Derek Parnell <derek nomail.afraid.org> writes:
On Sun, 24 Sep 2006 17:14:06 -0700, Gregor Richards wrote:


 Derek Parnell wrote:

 You still seem to be saying that the *only* place that module ought to
 reside in is in libraries.


 
 Aha!  This is a core misunderstanding.  All of this library stuff is 
 supposed to come well after failing to find it in an object file - the 
 order in my head is: source files, object files, specified libraries, 
 automatically-discovered libraries.  That is, it should FIRST look for 
 source, then objects, and only if failing in both of those should it 
 look at libraries - libraries are solely for things that are NOT 
 included with the source!
 


Ok! So the example should be more like ...

Example:
   import a.b.c;
 
Looks for 
    a.b.c.o
    libD.a.b.so
    libD.a.b.a
    libD.a.so
    libD.a.a
 
and places the first one it comes across onto the linker command line. If
it finds none of these, or finds that the source is more recent, it will
compile a.b.c.d to form an object file.

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocrity!"
25/09/2006 10:17:17 AM
Sep 24 2006
next sibling parent Gregor Richards <Richards codu.org> writes:
Derek Parnell wrote:

 On Sun, 24 Sep 2006 17:14:06 -0700, Gregor Richards wrote:
 
 

Derek Parnell wrote:


You still seem to be saying that the *only* place that module ought to
reside in is in libraries.


Aha!  This is a core misunderstanding.  All of this library stuff is 
supposed to come well after failing to find it in an object file - the 
order in my head is: source files, object files, specified libraries, 
automatically-discovered libraries.  That is, it should FIRST look for 
source, then objects, and only if failing in both of those should it 
look at libraries - libraries are solely for things that are NOT 
included with the source!


 
 
 Ok! So the example should be more like ...
 
 Example:
    import a.b.c;
  
 Looks for 
     a.b.c.o
     libD.a.b.so
     libD.a.b.a
     libD.a.so
     libD.a.a
  
 and places the first one it comes across onto the linker command line. If
 it finds none of these, or finds that the source is more recent, it will
 compile a.b.c.d to form an object file.
 


Exactly!  Using libraries isn't supposed to /replace/ using source or 
already-compiled object files, it's supposed to be an adjunct - in many 
cases, it's just inconvenient to use object files or source, and of 
course there's the possibility of proprietary software in which a 
library is clearly the superior method.  Obviously, if you included the 
source by some means, you intend to use them - otherwise, libraries are 
probably more useful.

  - Gregor Richards
Sep 24 2006
prev sibling parent reply Miles <_______ _______.____> writes:
 Example:
    import a.b.c;
  
 Looks for 
     a.b.c.o


I think that DMD looks for a/b/c.o (a\b\c.obj on windows).
Sep 24 2006
parent Derek Parnell <derek nomail.afraid.org> writes:
On Sun, 24 Sep 2006 23:17:22 -0300, Miles wrote:


 Example:
    import a.b.c;
  
 Looks for 
     a.b.c.o


 
 I think that DMD looks for a/b/c.o (a\b\c.obj on windows).


Oops. Of course your right. 

-- 
Derek
(skype: derek.j.parnell)
Melbourne, Australia
"Down with mediocrity!"
25/09/2006 12:21:45 PM
Sep 24 2006
prev sibling parent Sean Kelly <sean f4.ca> writes:
Derek Parnell wrote:

 
 So a further complication is what should Build do if the module source
 files are more recent than the matching library? Currently, it recompiles
 the source files and uses the resulting object files in the linkage.


Shouldn't a module be recompiled if its object file is older than the 
module or any of its dependencies?  If -inline is specified at any rate.


Sean
Sep 24 2006