www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Namespace for a module defined by its import path

reply Jeff Thompson <nospam example.com> writes:
I have different versions of a library module in different 
folders. For example

/version1/lib.d
/version2/lib.d

In my application I need to be able to update the names of the 
different folders without needing to change the module files. 
That means the file cannot contain "module version1.lib;" since I 
may need to change it to "versionA".

Ideally, I want to omit the "module" declaration in the library 
file, but still import it with the desired file path as "import 
version1.lib;".  And I would also want to "import version2.lib;" 
without conflict.

Basically, I want the namespace of the identifiers in the library 
module to be defined by the path which is use to load them 
*without needing to put the path in the module file.*  Is there 
some way to do this?
Oct 24 2016
next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Monday, 24 October 2016 at 21:06:18 UTC, Jeff Thompson wrote:

 I have different versions of a library module in different 
 folders. For example

 /version1/lib.d
 /version2/lib.d

 In my application I need to be able to update the names of the 
 different folders without needing to change the module files. 
 That means the file cannot contain "module version1.lib;" since 
 I may need to change it to "versionA".

 Ideally, I want to omit the "module" declaration in the library 
 file, but still import it with the desired file path as "import 
 version1.lib;".  And I would also want to "import 
 version2.lib;" without conflict.

 Basically, I want the namespace of the identifiers in the 
 library module to be defined by the path which is use to load 
 them *without needing to put the path in the module file.*  Is 
 there some way to do this?


Not sure if I understand you correctly, but you're allowed to 
omit the module declaration, if that helps:

https://dlang.org/spec/module.html#module_declaration

"The ModuleDeclaration sets the name of the module and what 
package it belongs to. If absent, the module name is taken to be 
the same name (stripped of path and extension) of the source file 
name."
Oct 24 2016
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 24 October 2016 at 23:26:09 UTC, bitwise wrote:

 Not sure if I understand you correctly, but you're allowed to 
 omit the module declaration, if that helps:


It doesn't help, I strongly recommend you ALWAYS use the explicit 
module definition. If you leave it out, you'll regret it sooner 
or later.

The automatic module name never has a package component, it is 
just one word, and is thus highly likely to cause name conflicts 
later.
Oct 24 2016
prev sibling next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 24 October 2016 at 21:06:18 UTC, Jeff Thompson wrote:

 Basically, I want the namespace of the identifiers in the 
 library module to be defined by the path which is use to load 
 them *without needing to put the path in the module file.*  Is 
 there some way to do this?


That's not going to work... but why do you want this? If they are 
different versions of the file, editing the module definition 
should be no real problem. You can use renamed imports or a 
helper module with public import to keep user code more 
compatible.

You can also do the opposite easily: have several files all with 
the same module name, but pick the specific path you want on the 
command line. Just make two files, v1.d and v2.d, both with 
`module myproject.myfile;` and import it as `import 
myproject.myfile;`

When compiling, then just do `dmd yourcode.d v1.d` or `dmd 
yourcode.d v2.d` to pick the one you want.
Oct 24 2016
parent reply Jeff Thompson <nospam example.com> writes:
On Tuesday, 25 October 2016 at 00:42:59 UTC, Adam D. Ruppe wrote:

 On Monday, 24 October 2016 at 21:06:18 UTC, Jeff Thompson wrote:

 Basically, I want the namespace of the identifiers in the 
 library module to be defined by the path which is use to load 
 them *without needing to put the path in the module file.*  Is 
 there some way to do this?


 That's not going to work... but why do you want this? If they 
 are different versions of the file, editing the module 
 definition should be no real problem. You can use renamed 
 imports or a helper module with public import to keep user code 
 more compatible.


A variant of this is where the version name is the Git commit 
hash. Instead of "version1" and "version2" it is:

/commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/lib.d
/commit_df0741a84c8a967ea08674090afdff4e9a58d23e/lib.d

The file lib.d can't contain its own commit hash, so I can't put 
it in the module declaration. And different parts of the 
application rely on different versions so I need different 
application files to be able to import one or the other without 
conflict. I'm trying to escape the situation where a piece of 
code needs to know the path which the operating system uses to 
load it.
Oct 24 2016
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Tuesday, 25 October 2016 at 06:47:14 UTC, Jeff Thompson wrote:

 On Tuesday, 25 October 2016 at 00:42:59 UTC, Adam D. Ruppe 
 wrote:

 [...]


 A variant of this is where the version name is the Git commit 
 hash. Instead of "version1" and "version2" it is:

 /commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/lib.d
 /commit_df0741a84c8a967ea08674090afdff4e9a58d23e/lib.d

 The file lib.d can't contain its own commit hash, so I can't 
 put it in the module declaration. And different parts of the 
 application rely on different versions so I need different 
 application files to be able to import one or the other without 
 conflict. I'm trying to escape the situation where a piece of 
 code needs to know the path which the operating system uses to 
 load it.


Sorry if I didn't understand something, couldn't you do this?

/commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/lib.d
/commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/lib.d
Oct 25 2016
parent reply Jeff Thompson <nospam example.com> writes:
On Tuesday, 25 October 2016 at 19:54:42 UTC, Jonathan Marler 
wrote:

 On Tuesday, 25 October 2016 at 06:47:14 UTC, Jeff Thompson 
 wrote:

 [...]


 Sorry if I didn't understand something, couldn't you do this?

 /commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/lib.d
 /commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/lib.d


If I do that, can lib.d avoid the following module statement?
module commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94.version.lib
Oct 25 2016
parent reply Mike Parker <aldacron gmail.com> writes:
On Tuesday, 25 October 2016 at 22:25:51 UTC, Jeff Thompson wrote:

 On Tuesday, 25 October 2016 at 19:54:42 UTC, Jonathan Marler 
 wrote:

 On Tuesday, 25 October 2016 at 06:47:14 UTC, Jeff Thompson 
 wrote:

 [...]


 Sorry if I didn't understand something, couldn't you do this?

 /commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/lib.d
 /commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/lib.d


 If I do that, can lib.d avoid the following module statement?
 module 
 commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94.version.lib


dmd -I/commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/ 
main.d
dmd -I//commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/ 
main.d
Oct 25 2016
parent reply Jeff Thompson <nospam example.com> writes:
On Wednesday, 26 October 2016 at 01:15:02 UTC, Mike Parker wrote:

 On Tuesday, 25 October 2016 at 22:25:51 UTC, Jeff Thompson 
 wrote:

 On Tuesday, 25 October 2016 at 19:54:42 UTC, Jonathan Marler 
 wrote:

 On Tuesday, 25 October 2016 at 06:47:14 UTC, Jeff Thompson 
 wrote:

 [...]


 Sorry if I didn't understand something, couldn't you do this?

 /commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/lib.d
 /commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/lib.d


 If I do that, can lib.d avoid the following module statement?
 module 
 commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94.version.lib


 dmd -I/commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/ 
 main.d
 dmd 
 -I//commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/ 
 main.d


This will force the application to use either one version or the 
other. As I said, it's a large application. I different parts of 
the same application to be able to import the version they need.
Oct 26 2016
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Wednesday, 26 October 2016 at 07:14:30 UTC, Jeff Thompson 
wrote:


 dmd 
 -I/commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/ 
 main.d
 dmd 
 -I//commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/ 
 main.d


 This will force the application to use either one version or 
 the other. As I said, it's a large application. I different 
 parts of the same application to be able to import the version 
 they need.


A module named foo.bar is foo.bar. There's no way to have a 
foo.bar version1 and a foo.bar version2 in the same program. You 
compile and link one version or the other, period. If you want 
multiple versions in the same program, then they need to have 
distinct module names:

foo/bar1.d --> module foo.bar1;
foo/bar2.d --> module foo.bar2;

I can't see how you would expect it to work otherwise. How would 
the compiler know that a call to foo.bar.someFunc belongs to one 
version or another if both have the same fully qualified name?

Or maybe I'm misunderstanding you (and I'm apparently not the 
only one in this thread). What problem are you trying to solve? 
Why do you need different versions of the same module or package 
at the same time in the same program?
Oct 26 2016
parent reply Jeff Thompson <nospam example.com> writes:
On Wednesday, 26 October 2016 at 07:57:57 UTC, Mike Parker wrote:

 On Wednesday, 26 October 2016 at 07:14:30 UTC, Jeff Thompson 
 wrote:


 dmd 
 -I/commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/ 
 main.d
 dmd 
 -I//commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/ 
 main.d


 This will force the application to use either one version or 
 the other. As I said, it's a large application. I different 
 parts of the same application to be able to import the version 
 they need.


 A module named foo.bar is foo.bar. There's no way to have a 
 foo.bar version1 and a foo.bar version2 in the same program. 
 You compile and link one version or the other, period. If you 
 want multiple versions in the same program, then they need to 
 have distinct module names:

 foo/bar1.d --> module foo.bar1;
 foo/bar2.d --> module foo.bar2;

 I can't see how you would expect it to work otherwise. How 
 would the compiler know that a call to foo.bar.someFunc belongs 
 to one version or another if both have the same fully qualified 
 name?

 Or maybe I'm misunderstanding you (and I'm apparently not the 
 only one in this thread). What problem are you trying to solve? 
 Why do you need different versions of the same module or 
 package at the same time in the same program?


I need different versions of the same module at the same time in 
the same program to be able to use multiple libraries, some of 
which have been verified to work one version of the module and 
others to work with another version. It is to be able to have 
reliability in large scale programs where different libraries 
have slightly different dependencies.

I can do this in Node.js. Suppose I have the following files:

commit_b3bf5c7/utils.js
commit_df0741a/utils.js
somelib.js
anotherlib.js

In Node.js, somelib.js can have

var Utils = require('commit_b3bf5c7/utils.js').Utils;

And anotherlib.js can have

var Utils = require('commit_df0741a/utils.js').Utils;

with no conflict whatsoever. Each lib can use the version that it 
knows works correctly.  A file can even include both if it needs 
to:

var Utils1 = require('commit_b3bf5c7/utils.js').Utils;
var Utils2 = require('commit_df0741a/utils.js').Utils;

Again no conflict between using Utils1 and Utils2. The point is 
that the namespace of the identifiers of imported files is 
defined by the file that imports it. This allows much more 
scalability, instead of forcing every library and every library 
it depends on to use the same version of an imported file.
Oct 26 2016
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Wednesday, 26 October 2016 at 09:53:35 UTC, Jeff Thompson 
wrote:


 var Utils1 = require('commit_b3bf5c7/utils.js').Utils;
 var Utils2 = require('commit_df0741a/utils.js').Utils;


That works because require operates in terms of files, not 
modules. In D, module and package names may have a default 
relationship with the file system, but the modules and packages 
themselves are wholly constructs of the compiler. `import foo` 
knows nothing about any files called foo. You need a way for the 
commit_* directories to become a part of the package name so that 
you can have distinct namespaces and avoid symbol clashes. If you 
can make that happen, then you can do this:

import Utils1 = commit_b3bf5c7.utils;
import Utils2 = commit_df0741a.utils;

You also might be able to generate something at compile time with 
string imports and string mixins.

But really, statically compiled languages aren't fit for this 
sort of thing. If you can compiled the version-sensitive bits 
into shared libraries, define a common interface in the main 
program, then load the libraries manually at runtime... that's 
the only reasonable way I know of to handle something like this. 
You could have multiple versions of the interface implementations 
loaded without issue. But I don't believe D's shared library 
support is fully implemented across all platforms at the moment.
Oct 26 2016
prev sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Wednesday, 26 October 2016 at 09:53:35 UTC, Jeff Thompson 
wrote:

 On Wednesday, 26 October 2016 at 07:57:57 UTC, Mike Parker 
 wrote:

 On Wednesday, 26 October 2016 at 07:14:30 UTC, Jeff Thompson 
 wrote:


 dmd 
 -I/commit_b3bf5c7725c98ee3e49dfc4e47318162f138fe94/version/ 
 main.d
 dmd 
 -I//commit_df0741a84c8a967ea08674090afdff4e9a58d23e/version/ 
 main.d


 This will force the application to use either one version or 
 the other. As I said, it's a large application. I different 
 parts of the same application to be able to import the 
 version they need.


 A module named foo.bar is foo.bar. There's no way to have a 
 foo.bar version1 and a foo.bar version2 in the same program. 
 You compile and link one version or the other, period. If you 
 want multiple versions in the same program, then they need to 
 have distinct module names:

 foo/bar1.d --> module foo.bar1;
 foo/bar2.d --> module foo.bar2;

 I can't see how you would expect it to work otherwise. How 
 would the compiler know that a call to foo.bar.someFunc 
 belongs to one version or another if both have the same fully 
 qualified name?

 Or maybe I'm misunderstanding you (and I'm apparently not the 
 only one in this thread). What problem are you trying to 
 solve? Why do you need different versions of the same module 
 or package at the same time in the same program?


 I need different versions of the same module at the same time 
 in the same program to be able to use multiple libraries, some 
 of which have been verified to work one version of the module 
 and others to work with another version. It is to be able to 
 have reliability in large scale programs where different 
 libraries have slightly different dependencies.

 I can do this in Node.js. Suppose I have the following files:

 commit_b3bf5c7/utils.js
 commit_df0741a/utils.js
 somelib.js
 anotherlib.js

 In Node.js, somelib.js can have

 var Utils = require('commit_b3bf5c7/utils.js').Utils;

 And anotherlib.js can have

 var Utils = require('commit_df0741a/utils.js').Utils;

 with no conflict whatsoever. Each lib can use the version that 
 it knows works correctly.  A file can even include both if it 
 needs to:

 var Utils1 = require('commit_b3bf5c7/utils.js').Utils;
 var Utils2 = require('commit_df0741a/utils.js').Utils;

 Again no conflict between using Utils1 and Utils2. The point is 
 that the namespace of the identifiers of imported files is 
 defined by the file that imports it. This allows much more 
 scalability, instead of forcing every library and every library 
 it depends on to use the same version of an imported file.


I've thought about this and have concluded that you can't 
replicate these semantics in D.  Here's 2 solutions though:

1. Change the module name for each version ("module 
commit_b3bf5c7.util;").
    CON: when someone uses the util library in isolation, this 
will be odd.

2. If you know that you will only ever use 1 version of the 
library in each file, you can omit the module name in the library 
and import it with the version:
    "import commit_b3bf5c7.util;"
    However, this actual module name will be "util", so if you 
import more than one version in the same file, you will have a 
module name conflict because both will be named "util".  You 
could also use "import util" and select which version to use 
based on the import paths that are passed to the compiler.

Both of these scenarios have caveats (won't work the same as the 
NodeJS version in every case).  To support the NodeJS semantics, 
we would have to modify how modules work a little bit.

If I were to come up with a way to modify the module system to 
support this, I could imagine something like this:

pathimport <path> <module>;

Then the module system would keep track of the path that a module 
was imported by.  So you could do this:

pathimport commit_b3bfc70 util;
pathimport commit_df0741a util;

This would maintain the module name as "util" and also give the 
compiler something to differentiate modules with the same name 
(the path).  I believe if a feature like this was supported, you 
could replicate the same semantics you have in NodeJS, but alas, 
it doesn't exist.  There's probably also a better solution than 
this, but I figured I would try to show you what a potential 
solution looks like so you can understand what's missing.
Oct 26 2016
parent reply Jeff Thompson <nospam example.com> writes:
On Wednesday, 26 October 2016 at 17:12:30 UTC, Jonathan Marler 
wrote:

 On Wednesday, 26 October 2016 at 09:53:35 UTC, Jeff Thompson 
 wrote:

 [...]


 I've thought about this and have concluded that you can't 
 replicate these semantics in D.  Here's 2 solutions though:

 [...]


Right now, if a module doesn't have a package statement, the 
module name is the file name with the path and and extension 
stripped off. My problem would be solved by a compiler option to 
keep the path.
Nov 08 2016
parent reply Jeff Thompson <nospam example.com> writes:
On Tuesday, 8 November 2016 at 17:00:44 UTC, Jeff Thompson wrote:

 On Wednesday, 26 October 2016 at 17:12:30 UTC, Jonathan Marler 
 wrote:

 On Wednesday, 26 October 2016 at 09:53:35 UTC, Jeff Thompson 
 wrote:

 [...]


 I've thought about this and have concluded that you can't 
 replicate these semantics in D.  Here's 2 solutions though:

 [...]


 Right now, if a module doesn't have a package statement, the 
 module name is the file name with the path and and extension 
 stripped off. My problem would be solved by a compiler option 
 to keep the path.


... or is there a way to get the path at compile time? I could 
try some mixin magic to construct the package name including the 
path.
Nov 08 2016
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Tuesday, 8 November 2016 at 18:33:13 UTC, Jeff Thompson wrote:

 On Tuesday, 8 November 2016 at 17:00:44 UTC, Jeff Thompson 
 wrote:

 On Wednesday, 26 October 2016 at 17:12:30 UTC, Jonathan Marler 
 wrote:

 On Wednesday, 26 October 2016 at 09:53:35 UTC, Jeff Thompson 
 wrote:

 [...]


 I've thought about this and have concluded that you can't 
 replicate these semantics in D.  Here's 2 solutions though:

 [...]


 Right now, if a module doesn't have a package statement, the 
 module name is the file name with the path and and extension 
 stripped off. My problem would be solved by a compiler option 
 to keep the path.


 ... or is there a way to get the path at compile time? I could 
 try some mixin magic to construct the package name including 
 the path.


I guess this is what you're looking for:
Nov 08 2016
parent reply Jeff Thompson <nospam example.com> writes:
On Wednesday, 9 November 2016 at 07:21:28 UTC, ZombineDev wrote:

 On Tuesday, 8 November 2016 at 18:33:13 UTC, Jeff Thompson 
 wrote:

 On Tuesday, 8 November 2016 at 17:00:44 UTC, Jeff Thompson 
 wrote:

 On Wednesday, 26 October 2016 at 17:12:30 UTC, Jonathan 
 Marler wrote:

 [...]


 Right now, if a module doesn't have a package statement, the 
 module name is the file name with the path and and extension 
 stripped off. My problem would be solved by a compiler option 
 to keep the path.


 ... or is there a way to get the path at compile time? I could 
 try some mixin magic to construct the package name including 
 the path.


 I guess this is what you're looking for:



Wow, thanks! This looks like the keyword was just added last 
month. Is that right?
Nov 09 2016
parent Jeff Thompson <nospam example.com> writes:
On Wednesday, 9 November 2016 at 22:15:31 UTC, Jeff Thompson 
wrote:

 On Wednesday, 9 November 2016 at 07:21:28 UTC, ZombineDev wrote:

 On Tuesday, 8 November 2016 at 18:33:13 UTC, Jeff Thompson 
 wrote:

 On Tuesday, 8 November 2016 at 17:00:44 UTC, Jeff Thompson 
 wrote:

 [...]


 ... or is there a way to get the path at compile time? I 
 could try some mixin magic to construct the package name 
 including the path.


 I guess this is what you're looking for:



 Wow, thanks! This looks like the keyword was just added last 
 month. Is that right?


... Unfortunately it seems I can't use a mixin for a module 
declaration. Is that right?
Nov 09 2016
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Wednesday, 26 October 2016 at 07:14:30 UTC, Jeff Thompson 
wrote:

 This will force the application to use either one version or 
 the other. As I said, it's a large application. I different 
 parts of the same application to be able to import the version 
 they need.


Unless there are some part of the application that cannot be 
updated for some reason, I would strongly advice against this, in 
D or any other language.

The cost of upgrading some piece of code to use the new version 
of the library is far lower than the cost of ongoing maintenance 
of many version of that library.

In D specifically, I'm not sure what people will come up with, 
but at the end, if you have the same symbol defined in several 
version of the library, you are going to be very sorry.
Oct 26 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
 protected-headers="v1"
From: Dicebot <public dicebot.lv>
Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D
Subject: Re: Namespace for a module defined by its import path
References: <tdmehehrlstjehyhkmli forum.dlang.org>
In-Reply-To: <tdmehehrlstjehyhkmli forum.dlang.org>

--nu4PacOxMRElxb3krURb4iWNo7b7IXtLP
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On 10/24/2016 11:06 PM, Jeff Thompson wrote:

 ...


If I really had to do something like that and there was no other way,
I'd most likely resort to auto-generating such module names during build
pipeline.


--nu4PacOxMRElxb3krURb4iWNo7b7IXtLP--
Oct 26 2016