www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP16: Transparently substitute module with package

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Starting a new thread from one in announce:

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

Please comment, after which Walter will approve. Walter's approval means 
that he would approve a pull request implementing DIP16 (subject to 
regular correctness checks).


Destroy!

Andrei
Mar 30 2012
next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 30/03/2012 16:46, Andrei Alexandrescu a écrit :

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei


I was waiting for it :D

First, as all.d is already a convention in many D projects, why choose 
package.d ? I understand this is a keyword, but this will also be 
painfull for many developer.

Second, what the rule of a .d file and a folder existing, but in 
different path (think -I switch of the compiler).

In the first place, is the module declaration really usefull ?

About the lookup rule, I understand that if I import a.b and use the 
function a.b.c.foo , then i must be able to refers to it as a.b.foo, but 
why a.foo ? It seems to me like going too far in the modification for no 
benefit (it can only increase the number of collision, and has no 
benefit I can think of).
Mar 30 2012
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/30/2012 04:46 PM, Andrei Alexandrescu wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei



Those are two proposals.

I think the first one should be implemented.


I don't really like the second one.

1. It is an over-general solution, because it does not solve a general 
problem.

2. Library writers lose control. All symbol names in the package are 
reserved when the package is deployed. It will be a breaking change to 
introduce a sub-package that happens to have the same name as some 
symbol defined in any module in the same package hierarchy.

3. symbol lookup is already hard enough to get right, because compile 
time reflection and conditional code generation can introduce 
ambiguities and contradictions. DMD does not get it right. It is likely 
that this change would make fixing this in a general but 
not-too-conservative way considerably harder.


Maybe it would be better to just interpret foo.bar.baz as 
foo.bar.package.baz if foo.bar is a package that has been imported via 
the foo.bar.package rewrite? Of course, issue 2 probably would remain.
Mar 30 2012
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On second thought, issue 2 is probably not that much of a problem.
Mar 30 2012
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"Timon Gehr" <timon.gehr gmx.ch> wrote in message 
news:jl4jmg$2j1r$1 digitalmars.com...

 I don't really like the second one.

 1. It is an over-general solution, because it does not solve a general 
 problem.

 Maybe it would be better to just interpret foo.bar.baz as 
 foo.bar.package.baz if foo.bar is a package that has been imported via the 
 foo.bar.package rewrite?


That occurred to me, and I thought about proposing the same thing you're 
suggesting, but on second thought I wasn't so sure:

If I need to disambiguate between "std.algorithm.find" and 
"foo.bar.baz.find", it might be nice to be able to just say "Meh, 
just...that one in Phobos, ie 'std'". Or "Just go with that 'foo' one".

I could go either way, really.
Mar 30 2012
prev sibling next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 30/03/2012 15:46, Andrei Alexandrescu wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei


The proposal doesn't say what happens when package.d is not found but 
foo/bar/ exists.

Given that a lot of people will just use public import foo.bar.*; in 
that file, would it make sense for package.d missing to imply import 
foo.bar.*? That would save typing out every single file in there. Of 
course it would also be annoying if you wanted to import everything 
except one file, as you'd then have to type out every single import anyway.

The other option is to error, which is probably a more sane option.

-- 
Robert
http://octarineparrot.com/
Mar 30 2012
parent "Nick Sabalausky" <a a.a> writes:
"Robert Clipsham" <robert octarineparrot.com> wrote in message 
news:jl4l5t$2m62$1 digitalmars.com...

 On 30/03/2012 15:46, Andrei Alexandrescu wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei


 The proposal doesn't say what happens when package.d is not found but 
 foo/bar/ exists.

 Given that a lot of people will just use public import foo.bar.*; in that 
 file, would it make sense for package.d missing to imply import foo.bar.*? 
 That would save typing out every single file in there. Of course it would 
 also be annoying if you wanted to import everything except one file, as 
 you'd then have to type out every single import anyway.


That would effectively be the same as Java's "import foo.*" and a lot of 
people have issues with that.


 The other option is to error, which is probably a more sane option.


That's what I'd suggest doing. Just treat it like importing any other 
missing package.
Mar 30 2012
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means  
 that he would approve a pull request implementing DIP16 (subject to  
 regular correctness checks).


I think package.d should be allowed to specify module.  Otherwise,  
something like /usr/include/d/std/datetime/package.d, what is the assumed  
package?  If module is not useful here, it is not useful anywhere.

I think the omission should be strictly anything after the package  
directory.  For example, if you have std/algorithm/package.d and  
std/algorithm/sorting.d, where package.d imports sorting.d, you can refer  
to std.algorithm.sorting.sort by omitting sorting, but not by omitting  
algorithm or std.

Other than that, this is a good change.

-Steve
Mar 30 2012
prev sibling next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 Destroy!


"That means a program that imports std.algorithm may use "std.sort"
for the symbol "std.algorithm.sort"."

That's quite interesting. Would that also mean that you could do:
import std.algorithm;  // has indexOf
import std.string;  // has indexOf
void main() {
    string.indexOf("foo", "foo"); -> std.string.indexOf
}

That would ease usage of Phobos a little bit. OTOH 'string' already is
a keyword and things might get hairy..

Still this is one of the few proposals I like. My only caveat is the
comment: "except the file is not allowed to use the "module"
declaration.". Wouldn't it be better if we explicitly declared a
module as a package instead? In foo\bar\package.d:
package foo.bar;

Since the "module" declaration must be on the first line (or second
line after shebang), you could special-case DMD to allow the package
keyword to be used here. I know D likes to abuse a keyword for
multiple things (hello Mr. Static!), but I think we could live with
it.
Mar 30 2012
parent "Nick Sabalausky" <a a.a> writes:
"Andrej Mitrovic" <andrej.mitrovich gmail.com> wrote in message 
news:mailman.1240.1333130858.4860.digitalmars-d puremagic.com...

 Still this is one of the few proposals I like. My only caveat is the
 comment: "except the file is not allowed to use the "module"
 declaration.". Wouldn't it be better if we explicitly declared a
 module as a package instead? In foo\bar\package.d:
 package foo.bar;

 Since the "module" declaration must be on the first line (or second
 line after shebang), you could special-case DMD to allow the package
 keyword to be used here. I know D likes to abuse a keyword for
 multiple things (hello Mr. Static!), but I think we could live with
 it.


Or maybe just require the module name ends with ".package"
Mar 30 2012
prev sibling next sibling parent reply Brad Anderson <eco gnuk.net> writes:
On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <
andrej.mitrovich gmail.com> wrote:


 On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 Destroy!


 "That means a program that imports std.algorithm may use "std.sort"
 for the symbol "std.algorithm.sort"."

 That's quite interesting. Would that also mean that you could do:
 import std.algorithm;  // has indexOf
 import std.string;  // has indexOf
 void main() {
    string.indexOf("foo", "foo"); -> std.string.indexOf
 }


I was actually kind of surprised when I found out this doesn't work.  It
seems so natural to resolve ambiguity using as little context as necessary.



 That would ease usage of Phobos a little bit. OTOH 'string' already is
 a keyword and things might get hairy..

 Still this is one of the few proposals I like. My only caveat is the
 comment: "except the file is not allowed to use the "module"
 declaration.". Wouldn't it be better if we explicitly declared a
 module as a package instead? In foo\bar\package.d:
 package foo.bar;

 Since the "module" declaration must be on the first line (or second
 line after shebang), you could special-case DMD to allow the package
 keyword to be used here. I know D likes to abuse a keyword for
 multiple things (hello Mr. Static!), but I think we could live with
 it.
Mar 30 2012
parent "F i L" <witte2008 gmail.com> writes:
On Friday, 30 March 2012 at 18:15:57 UTC, Brad Anderson wrote:

 On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <
 andrej.mitrovich gmail.com> wrote:


 On 3/30/12, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:

 Destroy!


 "That means a program that imports std.algorithm may use 
 "std.sort"
 for the symbol "std.algorithm.sort"."

 That's quite interesting. Would that also mean that you could 
 do:
 import std.algorithm;  // has indexOf
 import std.string;  // has indexOf
 void main() {
    string.indexOf("foo", "foo"); -> std.string.indexOf
 }


 I was actually kind of surprised when I found out this doesn't 
 work.  It
 seems so natural to resolve ambiguity using as little context 
 as necessary.


Ya that was the behavior I expected as well. Would be great if it 
worked like that. Just back trace the reference until the 
ambiguity is resolved.


// -----

Also, I'm probably missing something here, but I never understood 
why importing a package doesn't work like it does in 
Actionscript/Java/others...

import foo.bar.*; // everything
import foo.bar.all; // custom

That makes a lot of sense to me.
Mar 30 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16


Btw, I bet with the help of hackers like e.g. Kenji Hara we'll have
this implemented in a matter of days (if it gets accepted). Compare
that to having a C++ committee that would spend the next 5 years
writing a 100-page spec on what is or isn't a package, heheh. Didn't
Bjarne say on Going Native that he doesn't really know what a module
is or should be? He should try D sometime. ;)
Mar 30 2012
prev sibling next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 09:46:19 Andrei Alexandrescu wrote:

 Starting a new thread from one in announce:
 
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16
 
 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).
 
 
 Destroy!


The first part with package.d seems like a good idea and certainly affects some 
of what I've been doing with std.datetime. In order to split it right now, you 
need a new package name, leaving the original for importing everything, and 
this provides a better way of dealing with that.

However, I'm very nervous about the second part. e.g. std.sort instead of 
std.algorithm.sort seems like a bad idea to me. It increases the odds of name 
conflicts for little benefit. Not to mention, it'll make it a lot more
confusing 
to find what modules stuff is actually in if people start doing stuff like

std.sort(arr);

In the case of sort, you may know where it's from - particularly since it's so 
common - but the less well-known the function is, the less likely that is at 
all obvious where it comes from, and if you're dealing with 3rd party 
software, then it wouldn't be at all obvious. For instance, how would you know 
that party.foo is really party.bar.foo? You wouldn't. Being so lax about 
importing could really harm code readibility (and maintainibility, since it 
increases the odds of name clashes). So, I'm inclined to say that that is a 
_bad_ idea.

I'd propose that we make it so that if a module publicly imports another 
module, then you could treat it as if it were in that module. So, because 
std.datetime.package publicly imports std.datetime.systime, you could use 
std.datetime.SysTime instead of std.datetime.systime.SysTime. The compiler 
would need to realize that they're exactly the same symbol (we've had bugs 
relating to importing with : and the like which ended up creating new symbols, 
and we don't want that here), but that shouldn't be all that hard. That gives 
you control over which symbols are able to be treated as if they were in a 
given package rather than affecting everything indiscriminitely (and if you use 
: with public imports, it should then be possible, to restrict it to the 
_exact_ set of symbols that you want if you don't want all of the symbols to 
be treated as if they were in that module).

Another question is how this affects the documentation. Does package.d generate 
a page just like the other modules do? The lack of a module declaration could 
make that difficult (not impossible, but it would probably require changes to 
ddoc in addition to the module stuff). Also, does that page get treated in any 
special manner in how the documentation is laid out, because it's for the 
package as a whole (probably more of a site question than a ddoc one though)? 
I'd like to be able to have a page describing the package as a whole for 
std.datetime in addition to having the individual pages rather than just 
splitting it up, and leaving the programmer to read each of the individual 
pages with no overview. And I think that however package.d works, it needs to 
enable that.

- Jonathan M Davis
Mar 30 2012
next sibling parent "Rene Zwanenburg" <renezwanenburg gmail.com> writes:
On Friday, 30 March 2012 at 18:39:44 UTC, Jonathan M Davis wrote:

 I'd propose that we make it so that if a module publicly 
 imports another
 module, then you could treat it as if it were in that module. 
 So, because
 std.datetime.package publicly imports std.datetime.systime, you 
 could use
 std.datetime.SysTime instead of std.datetime.systime.SysTime.


I'm not sure if that's a good idea. I'd prefer a new kind of 
import statement, perhaps something like:

// module std.datetime.package
alias import std.datetime.systime;

which is similar to a public alias of everything in that module?
Mar 30 2012
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 3/30/12 1:39 PM, Jonathan M Davis wrote:

 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds of name
 conflicts for little benefit.


Example?


 Not to mention, it'll make it a lot more confusing
 to find what modules stuff is actually in if people start doing stuff like

 std.sort(arr);

 In the case of sort, you may know where it's from - particularly since it's so
 common - but the less well-known the function is, the less likely that is at
 all obvious where it comes from, and if you're dealing with 3rd party
 software, then it wouldn't be at all obvious. For instance, how would you know
 that party.foo is really party.bar.foo? You wouldn't.


Why should you?


 Being so lax about
 importing could really harm code readibility (and maintainibility, since it
 increases the odds of name clashes). So, I'm inclined to say that that is a
 _bad_ idea.


Maybe if you produce a solid example, I'd be convinced.


Andrei
Mar 30 2012
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 14:33:58 Andrei Alexandrescu wrote:

 On 3/30/12 1:39 PM, Jonathan M Davis wrote:

 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds of
 name conflicts for little benefit.


 
 Example?


std.sort works because there's only one sort. If there are two, you get a 
conflict (e.g. if you had std.path.sort which sorted paths in some
path-specific 
manner). If std.path.sort existed now, then std.sort wouldn't work, and you'd 
be forced to specify std.algorithm.sort or std.path.sort, and that's fine. It 
would be similar to having to specify std.algorithm.indexOf when you've 
imported both std.string and std.algorithm. But the problem is when 
std.path.sort is added _later_.

All of a sudden, code which used std.sort and worked is now broken. The 
problem does currently exist in that if we added indexOf to another module - 
say std.array - then code which imported either std.string or std.algorithm as 
well as std.array would break with the addition of std.array.indexOf, but your 
proposal makes it worse. Not only does it provide another way in which adding 
a function could result in conflicts when existing code is recompiled, but it 
makes it so that if you add any function anywhere in the _entire standard 
library_ which has the name as an existing one, you get a conflict (if anyone 
uses std.x rather than x or the full import path).

D does a good job of providing ways to fix name conflicts, but it doesn't do a 
good job of preventing them when adding new symbols to a library (primarily 
because it doesn't use static imports by default), and your proposal makes 
that part of the problem worse. If std.x were to become common practice, then 
any time that you added a symbol to a library when that symbol was already 
used by another module, you'd create conflicts (combined with the fact that 
private doesn't hide symbol names but merely makes them inaccessible, this 
could result in a lot of symbol name conflicts).


 Not to mention, it'll make it a lot more confusing
 to find what modules stuff is actually in if people start doing stuff like
 
 std.sort(arr);
 
 In the case of sort, you may know where it's from - particularly since
 it's so common - but the less well-known the function is, the less likely
 that is at all obvious where it comes from, and if you're dealing with
 3rd party software, then it wouldn't be at all obvious. For instance, how
 would you know that party.foo is really party.bar.foo? You wouldn't.


 
 Why should you?


Do you know what the foo function does? If you don't, you're going to have to 
look it up. And if you don't know what module it comes from, you can't do 
that. You also have to know where foo is from if a foo function is added to 
another module and causes a conflict, because you're going to have to give the 
full import path to actually use it. That's currently true with just bare foo 
as well, but party.foo gives the illusion of specifying where foo is from 
without actually specifying where it's from. At least right now, if foo is 
used with its import path, you know that that's actually its import path.

Also, what happens if we want to add a module named sort later? The fact that 
people are using std.sort means that adding std.sort as a module will break 
code. Granted, it's not very likely that we're going to add a module named 
sort, but there are plenty of other symbol names that it could happen with. 
But then again, if we decided to provide a module with all of the major sort 
algorithms, then maybe we _would_ create a module named std.sort. Just because 
we don't see a need now doesn't mean that we won't later. In either case, by 
allowing std.x where x is a symbol in any sub-module of std, you're going to 
create conflicts any time that you add a module which has the same name as an 
existing symbol anywhere in the library.


 Being so lax about
 importing could really harm code readibility (and maintainibility, since
 it
 increases the odds of name clashes). So, I'm inclined to say that that is
 a
 _bad_ idea.


 
 Maybe if you produce a solid example, I'd be convinced.


Well, as I've pointed with a few examples here, your proposal will increase 
the chances of adding symbol conflicts any time that a symbol is added to a 
library all just so that you can do std.algorithm.sort instead of 
std.algorithm.submodule1.sort once sort has been moved to 
std.algorithm.sumodule1. And we could make it possible to do 
std.algorithm.sort without adding all of those possible conflicts.

The simplest solution would simply be to make it so that if std.algorithm.sort 
is used, and std.algorithm is a package with a std.algorithm.package module, 
then the compiler looks in all of the sub-modules of std.algorithm to find 
sort. That solves the problem right there without increasing the odds of 
symbol conflicts across the entire library like your proposal does.

But personally, I like the idea of making it so that publicly imported symbols 
can be accessed as if they were in the module that publicly imported them 
(with package.d being treated as if it had the same name as the package that 
it's in). That's essentially how it already works except when specifying the 
full import path for a symbol. And that way, you can specify in 
std.algorithm.package.d exactly what you want to be imported when 
std.algorithm is imported (including using : to restrict it to specific symbols 
in a module), and only those symbols will be treated as if they were part of 
std.algorithm - both for importing purposes and when specifying the import 
path when using a symbol. The library maintainer then has control over which 
symbols get used with which import paths.

- Jonathan M Davis
Mar 30 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/30/2012 11:35 PM, Jonathan M Davis wrote:

 But personally, I like the idea of making it so that publicly imported symbols
 can be accessed as if they were in the module that publicly imported them


+1. That is even better than treating the package module specially.
Mar 31 2012
parent reply "Martin Nowak" <dawg dawgfoto.de> writes:
On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:


 On 03/30/2012 11:35 PM, Jonathan M Davis wrote:

 But personally, I like the idea of making it so that publicly imported  
 symbols
 can be accessed as if they were in the module that publicly imported  
 them


 +1. That is even better than treating the package module specially.


That already works, doesn't it?
Mar 31 2012
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 03/31/2012 03:49 PM, Martin Nowak wrote:

 On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:


 On 03/30/2012 11:35 PM, Jonathan M Davis wrote:

 But personally, I like the idea of making it so that publicly
 imported symbols
 can be accessed as if they were in the module that publicly imported
 them


 +1. That is even better than treating the package module specially.


 That already works, doesn't it?


It already works indeed. My bad.
Mar 31 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, March 31, 2012 17:30:36 Timon Gehr wrote:

 On 03/31/2012 03:49 PM, Martin Nowak wrote:

 On Sat, 31 Mar 2012 13:06:36 +0200, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 03/30/2012 11:35 PM, Jonathan M Davis wrote:

 But personally, I like the idea of making it so that publicly
 imported symbols
 can be accessed as if they were in the module that publicly imported
 them


 
 +1. That is even better than treating the package module specially.


 
 That already works, doesn't it?


 
 It already works indeed. My bad.


Does it? I thought that std.range.replace wouldn't work (even though std.range 
publicly imports std.array), because replace isn't part of std.range. You 
don't need to import std.array if you import std.range, because std.range 
does, but you can't specificy it's path as if it were in std.range. I'll have 
to check...

Okay. You're right. std.range.replace _does_ work. So then the only issue is 
making it so that public imports in std.datetime.package are treated as if 
they were in std.datetime. That seems like the cleanest solution to me. It 
goes along quite nicely with making it so that anything in 
std.datetime.package is imported when importing std.datetime.

- Jonathan M Davis
Mar 31 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 30/03/2012 23:35, Jonathan M Davis a écrit :

 On Friday, March 30, 2012 14:33:58 Andrei Alexandrescu wrote:

 On 3/30/12 1:39 PM, Jonathan M Davis wrote:

 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds of
 name conflicts for little benefit.


 Example?


 std.sort works because there's only one sort. If there are two, you get a
 conflict (e.g. if you had std.path.sort which sorted paths in some
path-specific
 manner). If std.path.sort existed now, then std.sort wouldn't work, and you'd
 be forced to specify std.algorithm.sort or std.path.sort, and that's fine. It
 would be similar to having to specify std.algorithm.indexOf when you've
 imported both std.string and std.algorithm. But the problem is when
 std.path.sort is added _later_.


I mentionned that as being a ptotential issue, so this is a +1 .


 But personally, I like the idea of making it so that publicly imported symbols
 can be accessed as if they were in the module that publicly imported them
 (with package.d being treated as if it had the same name as the package that
 it's in). That's essentially how it already works except when specifying the
 full import path for a symbol. And that way, you can specify in
 std.algorithm.package.d exactly what you want to be imported when
 std.algorithm is imported (including using : to restrict it to specific symbols
 in a module), and only those symbols will be treated as if they were part of
 std.algorithm - both for importing purposes and when specifying the import
 path when using a symbol. The library maintainer then has control over which
 symbols get used with which import paths.

 - Jonathan M Davis


I did propose that in another thread, so this is a +1 too.
Mar 31 2012
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Mar 30, 2012 at 05:35:25PM -0400, Jonathan M Davis wrote:
[...]

 But personally, I like the idea of making it so that publicly imported
 symbols can be accessed as if they were in the module that publicly
 imported them (with package.d being treated as if it had the same name
 as the package that it's in). That's essentially how it already works
 except when specifying the full import path for a symbol. And that
 way, you can specify in std.algorithm.package.d exactly what you want
 to be imported when std.algorithm is imported (including using : to
 restrict it to specific symbols in a module), and only those symbols
 will be treated as if they were part of std.algorithm - both for
 importing purposes and when specifying the import path when using a
 symbol. The library maintainer then has control over which symbols get
 used with which import paths.


[...]

+1.

I also have my doubts about the wisdom of std.sort implicitly binding to
std.algorithm.sort. It's nice syntactic sugar in the short term, but as
Jonathan points out, it can cause headaches in the long term. I say we
should hold off on that part of the proposal.


T

-- 
Microsoft is to operating systems & security ... what McDonalds is to gourmet
cooking.
Mar 30 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 30/03/2012 21:33, Andrei Alexandrescu a écrit :

 On 3/30/12 1:39 PM, Jonathan M Davis wrote:

 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds
 of name
 conflicts for little benefit.


 Example?


 Not to mention, it'll make it a lot more confusing
 to find what modules stuff is actually in if people start doing stuff
 like

 std.sort(arr);

 In the case of sort, you may know where it's from - particularly since
 it's so
 common - but the less well-known the function is, the less likely that
 is at
 all obvious where it comes from, and if you're dealing with 3rd party
 software, then it wouldn't be at all obvious. For instance, how would
 you know
 that party.foo is really party.bar.foo? You wouldn't.


 Why should you?


 Being so lax about
 importing could really harm code readibility (and maintainibility,
 since it
 increases the odds of name clashes). So, I'm inclined to say that that
 is a
 _bad_ idea.


 Maybe if you produce a solid example, I'd be convinced.


 Andrei


You are reversing the logic. It have to be shown that it is not a 
problem to do the change. The other way around is flawed logic.
Mar 31 2012
prev sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
 However, I'm very nervous about the second part. e.g. std.sort instead of
 std.algorithm.sort seems like a bad idea to me. It increases the odds of  
 name
 conflicts for little benefit. Not to mention, it'll make it a lot more  
 confusing
 to find what modules stuff is actually in if people start doing stuff  
 like

 std.sort(arr);


You can turn that argument around, std.sort(arr) instead of sort(arr)
provides some extra context for disambiguation.
You'd be able to do that for public imports in the package.d module anyhow,
so it makes sense to support it for implicit packages as well.
Mar 31 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 20:06:57 Andrej Mitrovic wrote:

 On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 Destroy!


 
 "That means a program that imports std.algorithm may use "std.sort"
 for the symbol "std.algorithm.sort"."
 
 That's quite interesting. Would that also mean that you could do:
 import std.algorithm; // has indexOf
 import std.string; // has indexOf
 void main() {
 string.indexOf("foo", "foo"); -> std.string.indexOf
 }


No, I don't think so. If I understand the proposal correctly, it would enable 
std.indexOf (which doesn't help you at all in this case), not string.indexOf. 
It's trying to make it so that you can treat a symbol in a sub-module as it 
were in a higher module, and string.indexOf doesn't help with that at all.

- Jonathan M Davis
Mar 30 2012
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, March 30, 2012 12:15:44 Brad Anderson wrote:

 On Fri, Mar 30, 2012 at 12:06 PM, Andrej Mitrovic <
 
 andrej.mitrovich gmail.com> wrote:

 On 3/30/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 Destroy!


 
 "That means a program that imports std.algorithm may use "std.sort"
 for the symbol "std.algorithm.sort"."
 
 That's quite interesting. Would that also mean that you could do:
 import std.algorithm; // has indexOf
 import std.string; // has indexOf
 void main() {
 
 string.indexOf("foo", "foo"); -> std.string.indexOf
 
 }


 
 I was actually kind of surprised when I found out this doesn't work. It
 seems so natural to resolve ambiguity using as little context as necessary.


It would certainly be desirable in some cases, but I believe that the reason 
that it doesn't work is due to the ambiguities that it would create. I'd have 
to go dig up old discussions on it though to remember all of the details.

alias is supposed to solve the problem, but it doesn't really work all that 
well for it, since private doesn't hide symbols, it only makes them 
inaccessible (just like with C++). So, creating aliases in a module causes 
problems in other modules that import that module, even if the aliases are 
private. There are definitely some folks pushing for private to actually start 
hiding symbols (IIRC, there's even a pull request for it), but I don't know 
what the odds of convincing Walter are. If/Once that happens, alias will 
actually become usable for this sort of situation, and the inability to do 
string.indexOf won't be as big a deal.

- Jonathan M Davis
Mar 30 2012
prev sibling next sibling parent "Nick Sabalausky" <a a.a> writes:
My comments:

1. My first impression was that using "foo/bar/package.d" instead of 
"foo/bar.d" seemed a bit odd and messy. But I realize now that cleverly 
solves the issue where "foo/bar.d" would be considered to be inside a 
different package from "foo/bar/*.d". So I like that. Personally, I think I 
would have gone with "foo/bar/_.d" as that sorts much, much better, but 
naming debates can go on forever, and I can live with "package.d"

2. I don't understand any of this:

-------------------------------------------
When looking up the symbol "foo.bar.baz", currently an exact match is 
needed. However. when looking up ".baz" or simply "baz", a flexible lookup 
is used that has many advantages (less verbose, hijacking detection etc). 
Therefore we think similar flexibility should be imparted to "foo.bar.baz", 
as follows:

If a qualified symbol "foo.bar.baz" appears in code, the compiler considers 
"foo.bar" a prefix that sets the starting point of the lookup, and then 
proceeds with looking up "baz" from that starting point. That means a 
program that imports std.algorithm may use "std.sort" for the symbol 
"std.algorithm.sort".
-------------------------------------------

I *do* understand "a program that imports std.algorithm may use "std.sort" 
for the symbol 'std.algorithm.sort'", and I think that's a good idea. It 
solves a problem I hadn't even thought of. But I don't understand that stuff 
I quoted above. Perhaps you could reword/clarify?

3. Other than that stuff, I'm very much in favor of this. I'll have some of 
that!
Mar 30 2012
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-03-30 14:46:19 +0000, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 Destroy!


Since you're asking…

One thing the current system avoids is unresolvable symbols. If two 
symbol name clashes, you just qualify them fully and it'll always be 
unambiguous. For instance:

	.std.algorithm.sort(…)

Now, if std.algorithm becomes both a module and a package, you could 
have both a sort function and a sort submodule with no way to 
distinguish between the two, even when fully qualified. I think this is 
why D does not allow modules to have the same name as packages.

I understand that you try to work around this problem by inventing a 
.std.algorithm.package scope. Then you make it's content imported 
automatically inside the .std.algorithm scope for backward 
compatibility (and convenience). The problem is that if 
.std.algorithm.package contains a sort function and there is also a 
module called std.algorithm.sort, the fully-qualified name of that 
'sort' module will become ambiguous. Moreover, whether the 
fully-qualified name .std.algorithm.sort is ambiguous or not depends on 
what modules were imported, which is not a very desirable behaviour.

So to make sure there is no unresolvable fully-qualified names, when 
importing std.algorithm.sort the compiler should make sure that no 
symbol called 'sort' already exist in the .std.algorithm scope (which 
includes the symbols in .std.algorithm.package and all other packages 
inside std.algorithm). This is clearly untenable.

 - - -

I recognize the need. If I may, I'll propose something simpler:

1. allow both std/algorithm.d and std/algorithm/sort.d to exist
2. importing std.algorithm.sort will also implicitly import 
std.algorithm (if it exists) and std (if it exists)
3. if any symbol of std.algorithm clash with the std.algorithm.sort 
module name, you get an error when importing std.algorithm.sort.

Effectively, importing std.algorithm.sort becomes synonymous to 
importing std, std.algorithm, and std.algorithm.sort. This is what's 
needed to detect clashes in fully-qualified names.

The only issue now (beside a few more imports) is that if std.algorithm 
imports any of its submodule it becomes a circular import. That's 
usually fine in D, but not when you have module constructors.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Mar 30 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin
<michel.fortin michelf.com> wrote:


 On 2012-03-30 14:46:19 +0000, Andrei Alexandrescu  =



 <SeeWebsiteForEmail erdani.org> said:


 Destroy!


 Since you're asking=E2=80=A6

 One thing the current system avoids is unresolvable symbols. If two  =



 symbol name clashes, you just qualify them fully and it'll always be  =



 unambiguous. For instance:

 	.std.algorithm.sort(=E2=80=A6)

 Now, if std.algorithm becomes both a module and a package, you could  =



 have both a sort function and a sort submodule with no way to  =



 distinguish between the two, even when fully qualified. I think this i=


s  =


 why D does not allow modules to have the same name as packages.

 I understand that you try to work around this problem by inventing a  =



 .std.algorithm.package scope. Then you make it's content imported  =



 automatically inside the .std.algorithm scope for backward compatibili=


ty  =


 (and convenience). The problem is that if .std.algorithm.package  =



 contains a sort function and there is also a module called  =



 std.algorithm.sort, the fully-qualified name of that 'sort' module wil=


l  =


 become ambiguous. Moreover, whether the fully-qualified name  =



 .std.algorithm.sort is ambiguous or not depends on what modules were  =



 imported, which is not a very desirable behaviour.


So this becomes an error.  I don't see this as a major problem.  Just
don't name a module sort inside std/algorithm.

This is no different than ambiguous templates, which are allowed until y=
ou
want to instantiate one.

-Steve
Apr 02 2012
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-02 13:04:31 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin
 

 The problem is that if .std.algorithm.package
 contains a sort function and there is also a module called
 std.algorithm.sort, the fully-qualified name of that 'sort' module wil l
 become ambiguous. Moreover, whether the fully-qualified name
 .std.algorithm.sort is ambiguous or not depends on what modules were
 imported, which is not a very desirable behaviour.


 
 So this becomes an error.  I don't see this as a major problem.  Just
 don't name a module sort inside std/algorithm.
 
 This is no different than ambiguous templates, which are allowed until y ou
 want to instantiate one.


If you have ambiguous templates in the same module, it'll always be 
ambiguous irrespective of what you import (and you can blame the 
module's designer for it). If you have ambiguous templates residing in 
different modules the symbol will be unambiguous until you've imported 
the second module (same as overloaded functions). At that point you can 
disambiguate using the fully-qualified name of the template (or 
function).

Whereas if the fully-qualified name of a module becomes ambiguous 
because of a symbol in another module, there is no way to disambiguate. 
All you can do is avoid importing the two conflicting modules together, 
just like when you encounter two headers trying to define the same 
symbol in C/C++.

With the way D modules were designed, this cannot happen because you 
can't have a module with the same name as a package. I always thought 
this was done on purpose, but I might be wrong. Whatever we do, I think 
it'd be a nice property to preserve that fully-qualified names should 
always work.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 02 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 02 Apr 2012 20:44:09 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2012-04-02 13:04:31 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


 On Fri, 30 Mar 2012 17:45:36 -0400, Michel Fortin


 The problem is that if .std.algorithm.package
 contains a sort function and there is also a module called
 std.algorithm.sort, the fully-qualified name of that 'sort' module wil  
 l
 become ambiguous. Moreover, whether the fully-qualified name
 .std.algorithm.sort is ambiguous or not depends on what modules were
 imported, which is not a very desirable behaviour.


  So this becomes an error.  I don't see this as a major problem.  Just
 don't name a module sort inside std/algorithm.
  This is no different than ambiguous templates, which are allowed until  
 y ou
 want to instantiate one.


 If you have ambiguous templates in the same module, it'll always be  
 ambiguous irrespective of what you import (and you can blame the  
 module's designer for it). If you have ambiguous templates residing in  
 different modules the symbol will be unambiguous until you've imported  
 the second module (same as overloaded functions). At that point you can  
 disambiguate using the fully-qualified name of the template (or  
 function).

 Whereas if the fully-qualified name of a module becomes ambiguous  
 because of a symbol in another module, there is no way to disambiguate.  
 All you can do is avoid importing the two conflicting modules together,  
 just like when you encounter two headers trying to define the same  
 symbol in C/C++.


How does this happen?  The FQN cannot be ambiguous.

For example, if we change std/algorithm.d into:

std/algorithm/sorting.d which defines sort
std/algorithm/finding.d which defines find
std/algorithm/package.d which imports sorting.d and finding.d

Then how can importing some other module make the FQN ambiguous?  The FQN  
for sort is std.algorithm.sorting.sort.  Now, if you define the symbol  
sort inside std/algorithm/package.d, you have an ambiguity, but so what?   
If you are responsible for the std.algorithm module, aren't you  
responsible for all the files in that pseudo-package?  I don't see how it  
changes things from today.  Basically, my point is the only ambiguity FQN  
in the package.d file can create is with other symbols within that same  
directory, therefore within the same module.

All we have to do is adopt the practice that phobos' package.d will only  
publicly import other modules.  It won't define any symbols.  Then there  
will always be an unambiguous FQN.

Hm... maybe you mean if you can have std/algorithm/package.d do something  
like import std.file, which contains another sort function, is that now  
packaged under std.algorithm.sort?  I don't think that's possible in the  
given DIP.

-Steve
Apr 04 2012
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Mon, 02 Apr 2012 20:44:09 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 

 Whereas if the fully-qualified name of a module becomes ambiguous  
 because of a symbol in another module, there is no way to disambiguate. 
  All you can do is avoid importing the two conflicting modules 
 together,  just like when you encounter two headers trying to define 
 the same  symbol in C/C++.


 
 How does this happen?  The FQN cannot be ambiguous.


Sure it can if I follow DIP16, because module names can become ambiguous.

Let's try this with an example. First, let's define a pretty standard module:

std/algorithm/sort.d:

	module std.algorithm.sort;

	void sort(T)(T[] array);

Here the fully-qualified name of the sort function is 
.std.algorithm.sort.sort. But according to DIP16's lookup rules, the 
sort function is also available (if not ambiguous) at:

	std.sort
	std.algorithm.sort

Question 1: since there is already a module at .std.algorithm.sort, 
doesn't the module name become ambiguous with the sort function it 
itself contains?

Let's assume the module's name take priority and does not conflict so 
we can continue. Now we create the package.d file:

std/algorithm/package.d:

	import std.algorithm.sort;

And now I write this somewhere in my code:

	std.algorithm.sort

Question 2: does std.algorithm.sort refer to the std.algorithm.sort 
*module* or to std.algorithm.package.sort *function* publicly imported 
from the std.algorithm.sort module?

Again, we could decide that the module takes priority. But having 
symbols take priority over one another is not how D has resolved 
ambiguities up to now; what D does usually is make it a hard error. If 
we make it an error the fully-qualified name of anything in the 
std.algorithm.sort module becomes inaccessible. If we do not make it an 
error, the module name shadows the function imported in the package. 
And the problem with shadowing is that it can silently change what code 
you're calling depending on what you've imported (if you need an 
example, just ask).

You might think I'm trying to split hair in four to find flaws, that no 
one is going to do things that dumb, but I unfortunately think the 
problematic pattern is already quite common. How many times have we 
seen modules containing a class, variable, or function having the same 
name as the module's name? What should happen when you publicly import 
those modules in the package.d file?

The practice might not be too prevalent in Phobos because modules tend 
to do a lot of things and are therefore named more generically, but it 
still happens. For instance:

	std.array.array
	std.getopt.getopt
	std.regex.regex

Say you wanted to create a package.d file directly for the whole 
package std, what should be done for those?

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 04 2012
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


  The FQN cannot be ambiguous.


 Sure it can if I follow DIP16, because module names can become ambiguous.

 Let's try this with an example. First, let's define a pretty standard  
 module:

 std/algorithm/sort.d:

 	module std.algorithm.sort;

 	void sort(T)(T[] array);

 Here the fully-qualified name of the sort function is  
 .std.algorithm.sort.sort. But according to DIP16's lookup rules, the  
 sort function is also available (if not ambiguous) at:

 	std.sort
 	std.algorithm.sort

 Question 1: since there is already a module at .std.algorithm.sort,  
 doesn't the module name become ambiguous with the sort function it  
 itself contains?


OK, but when is it ever valid to refer to a module when the semantic  
expectations are for something other than a module?  I can only think of  
two places where module names are used, inside an import statement and  
inside a module statement (three if you count the prefix of a FQN).  Maybe  
I'm missing some case...


 Let's assume the module's name take priority and does not conflict so we  
 can continue. Now we create the package.d file:

 std/algorithm/package.d:

 	import std.algorithm.sort;

 And now I write this somewhere in my code:

 	std.algorithm.sort

 Question 2: does std.algorithm.sort refer to the std.algorithm.sort  
 *module* or to std.algorithm.package.sort *function* publicly imported  
 from the std.algorithm.sort module?


The function.  A symbol that is not specified as the module name in import  
statement or in a module statement is always *not* a module.  I think our  
one saving grace here is that when you want to import a specific symbol  
 from a module, this is not the syntax:

import std.stdio.writefln;

So there is never any ambiguity as to whether you mean a module identifier  
or other symbol.


 You might think I'm trying to split hair in four to find flaws, that no  
 one is going to do things that dumb, but I unfortunately think the  
 problematic pattern is already quite common. How many times have we seen  
 modules containing a class, variable, or function having the same name  
 as the module's name?


Tango anyone? :)  But yes, I think the issue really becomes, we need to  
look at context when deciding the semantic meaning of a symbol.  I don't  
think this violates the context-free grammar, because wouldn't this only  
come into play at the semantic level?  Not a compiler writer/hacker, so I  
don't know.


 Say you wanted to create a package.d file directly for the whole package  
 std, what should be done for those?


No, let's not do that.  Ever. :)

-Steve
Apr 04 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04/04/2012 07:53 PM, Steven Schveighoffer wrote:

 On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
 <michel.fortin michelf.com> wrote:


 On 2012-04-04 14:08:34 +0000, "Steven Schveighoffer"
 <schveiguy yahoo.com> said:


 The FQN cannot be ambiguous.


 Sure it can if I follow DIP16, because module names can become ambiguous.

 Let's try this with an example. First, let's define a pretty standard
 module:

 std/algorithm/sort.d:

 module std.algorithm.sort;

 void sort(T)(T[] array);

 Here the fully-qualified name of the sort function is
 .std.algorithm.sort.sort. But according to DIP16's lookup rules, the
 sort function is also available (if not ambiguous) at:

 std.sort
 std.algorithm.sort

 Question 1: since there is already a module at .std.algorithm.sort,
 doesn't the module name become ambiguous with the sort function it
 itself contains?


 OK, but when is it ever valid to refer to a module when the semantic
 expectations are for something other than a module? I can only think of
 two places where module names are used, inside an import statement and
 inside a module statement (three if you count the prefix of a FQN).
 Maybe I'm missing some case...


__traits(allMembers, pack.age.mod.ule);


...

 You might think I'm trying to split hair in four to find flaws, that
 no one is going to do things that dumb, but I unfortunately think the
 problematic pattern is already quite common. How many times have we
 seen modules containing a class, variable, or function having the same
 name as the module's name?


 Tango anyone? :) But yes, I think the issue really becomes, we need to
 look at context when deciding the semantic meaning of a symbol. I don't
 think this violates the context-free grammar, because wouldn't this only
 come into play at the semantic level? Not a compiler writer/hacker, so I
 don't know.


No symbol is resolved until semantic, but I don't think hiding the 
module/package symbol if any clashing symbol in the module/any 
subpackage exists is a satisfactory solution.
Apr 04 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:


 On 04/04/2012 07:53 PM, Steven Schveighoffer wrote:

 OK, but when is it ever valid to refer to a module when the semantic
 expectations are for something other than a module? I can only think of
 two places where module names are used, inside an import statement and
 inside a module statement (three if you count the prefix of a FQN).
 Maybe I'm missing some case...


 __traits(allMembers, pack.age.mod.ule);


hm... maybe we'd have to have new __traits that would disambiguate, like  
__traits(allModuleMembers...)

This doesn't seem like a huge barrier.


 Tango anyone? :) But yes, I think the issue really becomes, we need to
 look at context when deciding the semantic meaning of a symbol. I don't
 think this violates the context-free grammar, because wouldn't this only
 come into play at the semantic level? Not a compiler writer/hacker, so I
 don't know.


 No symbol is resolved until semantic, but I don't think hiding the  
 module/package symbol if any clashing symbol in the module/any  
 subpackage exists is a satisfactory solution.


Then we must come up with a way to hide the submodules of a virtual module.

I tried this, which fails on the current compiler:

import x;

int x;

So it seems we have two choices here:

1. DIP16 needs to get more complex to make package submodules not  
accesible as individual modules.
2. Start contextually interpreting identifiers at least in the case of  
modules vs. non-modules.

I'd suggest option 2 allows for better backwards compatibility and more  
flexibility.

-Steve
Apr 04 2012
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 19:48:32 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:
 

 No symbol is resolved until semantic, but I don't think hiding the  
 module/package symbol if any clashing symbol in the module/any  
 subpackage exists is a satisfactory solution.


 
 Then we must come up with a way to hide the submodules of a virtual module.
 
 I tried this, which fails on the current compiler:
 
 import x;
 
 int x;
 
 So it seems we have two choices here:
 
 1. DIP16 needs to get more complex to make package submodules not  
 accesible as individual modules.
 2. Start contextually interpreting identifiers at least in the case of  
 modules vs. non-modules.
 
 I'd suggest option 2 allows for better backwards compatibility and more 
  flexibility.


I don't think option 2 is realistic (see my other post).

I don't think option 1 is an improvement over what we have. I mean, if 
you're going to hide the submodules, what is the benefit compared to 
just using a different package name for the implementation modules? You 
can already refactor std.algorithm this way with no change in the 
compiler:

	module std.algorithm;

	public import std.algorithm_impl.sort;
	public import std.algorithm_impl.map;
	public import std.algorithm_impl.blah_blah_blah;
	…

If we add a language feature, it should be an noticeable improvement 
over this situation.

I think we need a third option.

Here's an idea: we could allow modules having a single symbol with the 
same name as the module to behave as if they were the symbol itself, 
just like templates behaves. For instance:

	module std.algorithm.sort;

	void sort(int[] t);

Now you can import std.algorithm.sort and then use the 
std.algorithm.sort fully qualified name as if it was a function, even 
though it's the module name (std.algorithm.sort.sort would be the 
function's name).

Or maybe we could just allow "alias sort this" at module level? Or is 
it allowed already?


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 04 2012
next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 00:50:49 +0000, Michel Fortin <michel.fortin michelf.com> said:


 I think we need a third option.
 
 Here's an idea: we could allow modules having a single symbol with the 
 same name as the module to behave as if they were the symbol itself, 
 just like templates behaves. For instance:
 
 	module std.algorithm.sort;
 
 	void sort(int[] t);
 
 Now you can import std.algorithm.sort and then use the 
 std.algorithm.sort fully qualified name as if it was a function, even 
 though it's the module name (std.algorithm.sort.sort would be the 
 function's name).
 
 Or maybe we could just allow "alias sort this" at module level? Or is 
 it allowed already?


Forgot to follow through with what "import std.algorithm" would do.

If std.algorithm is a package, then it should only contain modules 
(which can each pose for a function/class/struct/variable thanks to the 
feature described above). Importing the package std.algorithm would 
open the std/algorithm.d file, read a list of package from the file, 
and proceed to import each of them (just as if you'd have imported each 
of them separately).

Here is an idea for declaring the list of modules to import when you 
import a package in std/algorithm.d:

	package std.algorithm;

	module std.algorithm.sort;
	module std.algorithm.map;
	...

(I'm _not_ using a package.d file because that'd incur two lookups: one 
for std/algorithm.d in case it's a module, and one for 
std/algorithm/package.d in case it's a package. It also removes the 
possibility of both files existing at the same time which would be 
confusing.)

This way, modules and packages stay exactly the same in the language as 
they are now, except that you can now import a package directly instead 
of importing each module separately. And modules with a single symbol 
with the same name as the module are treated as if they were that 
symbol. This allows migrating a module to a package without breaking 
the existing code base, but the drawback is that each symbol in the 
module becoming a package need to become its own submodule.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 04 2012
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 04 Apr 2012 20:50:49 -0400, Michel Fortin  =

<michel.fortin michelf.com> wrote:


 On 2012-04-04 19:48:32 +0000, "Steven Schveighoffer"  =



 <schveiguy yahoo.com> said:


 On Wed, 04 Apr 2012 14:03:07 -0400, Timon Gehr <timon.gehr gmx.ch>  =





 wrote:


 No symbol is resolved until semantic, but I don't think hiding the  =






 =


 module/package symbol if any clashing symbol in the module/any   =







 subpackage exists is a satisfactory solution.


  Then we must come up with a way to hide the submodules of a virtual =




 =


 module.
  I tried this, which fails on the current compiler:
  import x;
  int x;
  So it seems we have two choices here:
  1. DIP16 needs to get more complex to make package submodules not   =





 accesible as individual modules.
 2. Start contextually interpreting identifiers at least in the case o=




f   =


 modules vs. non-modules.
  I'd suggest option 2 allows for better backwards compatibility and  =





 more  flexibility.


 I don't think option 2 is realistic (see my other post).


I saw your other post.  I see that a major issue with the way DIP16  =

intends to shortcut fully qualified names is demonstrated by this simple=
  =

example:

a/b/c.d:

module a.b.c;

void foo() {}
struct c
{
    static void foo() {}
}

main.d:
import a.b.c;

void main()
{
    a.b.c.foo(); // cannot determine what this is.  Is it the global  =

function, or a shortcut FQN to a.b.c.c.foo?
}

This should be counter-case enough to disqualify that idea -- it will  =

break existing code without ever adding any package.d files.  Thanks for=
  =

explaining.


 I don't think option 1 is an improvement over what we have. I mean, if=


  =


 you're going to hide the submodules, what is the benefit compared to  =



 just using a different package name for the implementation modules?


The advantage is you can split your logical module into submodules for  =

maintenance purposes.


 You can already refactor std.algorithm this way with no change in the =


 =


 compiler:

 	module std.algorithm;

 	public import std.algorithm_impl.sort;
 	public import std.algorithm_impl.map;
 	public import std.algorithm_impl.blah_blah_blah;
 	=E2=80=A6

 If we add a language feature, it should be an noticeable improvement  =



 over this situation.


I agree, option 1 is not very convincing.


 I think we need a third option.

 Here's an idea: we could allow modules having a single symbol with the=


  =


 same name as the module to behave as if they were the symbol itself,  =



 just like templates behaves. For instance:

 	module std.algorithm.sort;

 	void sort(int[] t);

 Now you can import std.algorithm.sort and then use the  =



 std.algorithm.sort fully qualified name as if it was a function, even =


 =


 though it's the module name (std.algorithm.sort.sort would be the  =



 function's name).

 Or maybe we could just allow "alias sort this" at module level? Or is =


it  =


 allowed already?


I don't like this proposal, simply because this means one function/symbo=
l  =

per submodule.  It should be more flexible than that.  But I like the li=
ne  =

of thinking.

Let's re-examine the issue.  We need to allow splitting of a module X.d =
 =

into pieces for maintenance (or possibly accessibility -- disallowing  =

friends).  But we don't want to break code which currently uses FQN to  =

access X's symbols.

I think the following might work:

algorithm.d:

import this =3D std.algorithm_impl.sort;

Which then imports std.algorithm_impl.sort, and effectively aliases all =
 =

its symbols into algorithm.d.  If std.algorithm_impl.sort defines a  =

function called sort, then it's also aliased to std.algorithm.sort.  In =
 =

essence, sort has *two* FQN, but there are no FQN that refer to more tha=
n  =

one symbol.

I purposely left out the package.d idea because it's orthogonal to this.=


-Steve
Apr 05 2012
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 13:46, Steven Schveighoffer a écrit :

 I don't like this proposal, simply because this means one
 function/symbol per submodule. It should be more flexible than that. But
 I like the line of thinking.

 Let's re-examine the issue. We need to allow splitting of a module X.d
 into pieces for maintenance (or possibly accessibility -- disallowing
 friends). But we don't want to break code which currently uses FQN to
 access X's symbols.

 I think the following might work:

 algorithm.d:

 import this = std.algorithm_impl.sort;

 Which then imports std.algorithm_impl.sort, and effectively aliases all
 its symbols into algorithm.d. If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort. In
 essence, sort has *two* FQN, but there are no FQN that refer to more
 than one symbol.

 I purposely left out the package.d idea because it's orthogonal to this.

 -Steve


The behavior you described has been proposed for public import, and have 
been discussed. This is interesting.

You propose an alternative syntax with is fine and have the advantage to 
not disturb already existing public imports, but have the drawback to 
create a new syntax, again.

If such a syntax is adopted, what would be the point of public imports ? 
If it is still useful, then your syntax is a better choice, otherwise, 
we'd better modify public import.
Apr 05 2012
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrot=
e:


 Le 05/04/2012 13:46, Steven Schveighoffer a =C3=A9crit :

 I don't like this proposal, simply because this means one
 function/symbol per submodule. It should be more flexible than that. =




But

 I like the line of thinking.

 Let's re-examine the issue. We need to allow splitting of a module X.=




d

 into pieces for maintenance (or possibly accessibility -- disallowing=





 friends). But we don't want to break code which currently uses FQN to=





 access X's symbols.

 I think the following might work:

 algorithm.d:

 import this =3D std.algorithm_impl.sort;

 Which then imports std.algorithm_impl.sort, and effectively aliases a=




ll

 its symbols into algorithm.d. If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort. I=




n

 essence, sort has *two* FQN, but there are no FQN that refer to more
 than one symbol.

 I purposely left out the package.d idea because it's orthogonal to th=




is.

 -Steve


 The behavior you described has been proposed for public import, and ha=


ve  =


 been discussed. This is interesting.

 You propose an alternative syntax with is fine and have the advantage =


to  =


 not disturb already existing public imports, but have the drawback to =


 =


 create a new syntax, again.

 If such a syntax is adopted, what would be the point of public imports=


 ?  =


 If it is still useful, then your syntax is a better choice, otherwise,=


  =


 we'd better modify public import.


No, public imports simply mean that you can view the publicly imported  =

module.  It does *not* add aliases to the importing module.

for example:

foo.d:

module foo;

int x;

bar.d:
module bar;
public import foo;

int y;

main.d:
import bar; // publicly imports foo.

void main()
{
    foo.x =3D 5; // ok, publicly imported via bar (non public, this woul=
d be  =

an error)
    bar.y =3D 5; // ok
    bar.x =3D 5; // error, public import doesn't create new fully qualif=
ied  =

name for foo.x
}

With the system I propose, using the specific technique would make it so=
  =

bar.x would also be valid, and would refer to foo.x

I think we need new syntax, or a new language feature, because you don't=
  =

want to alter the operation of existing code.  Simply changing public  =

imports would cause lots of existing code to be reinterpreted, possibly =
in  =

a way not intended, or that would be ambiguous.

-Steve
Apr 05 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:

 No, public imports simply mean that you can view the publicly imported
 module. It does *not* add aliases to the importing module.


Have you tried it?
Apr 05 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:


 On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:

 No, public imports simply mean that you can view the publicly imported
 module. It does *not* add aliases to the importing module.


 Have you tried it?


I just did.  OK, what the hell are we arguing about then?!

DIP16 is worksforme :)

See this part of the spec:

http://dlang.org/module.html

Read the part on public modules.  You may understand why I didn't know  
about that "feature" (which seems to work on all the installed compilers I  
have, back to 2.033).  I just read the public import part of TDPL, and  
there it is, all spelled out quite nicely.  I'm going to file a bug  
against the spec...  grrr...

I'm now firmly in the "we don't need to change anything, just refactor the  
modules and use public import" camp.  We don't even need to change  
ANYTHING in the compiler!  Forget everything I said before in this thread  
;)

I suppose the only thing we don't get is being able to have a module and a  
package with the same FQN.  I don't see that being a major issue.

-Steve
Apr 05 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> wrote:

 On 04/05/2012 02:58 PM, Steven Schveighoffer wrote:

 No, public imports simply mean that you can view the publicly imported
 module. It does *not* add aliases to the importing module.


 
 Have you tried it?


 
 I just did. OK, what the hell are we arguing about then?!
 
 DIP16 is worksforme :)
 
 See this part of the spec:
 
 http://dlang.org/module.html
 
 Read the part on public modules. You may understand why I didn't know
 about that "feature" (which seems to work on all the installed compilers I
 have, back to 2.033). I just read the public import part of TDPL, and
 there it is, all spelled out quite nicely. I'm going to file a bug
 against the spec... grrr...
 
 I'm now firmly in the "we don't need to change anything, just refactor the
 modules and use public import" camp. We don't even need to change
 ANYTHING in the compiler! Forget everything I said before in this thread
 ;)
 
 I suppose the only thing we don't get is being able to have a module and a
 package with the same FQN. I don't see that being a major issue.


What doesn't work is being able to turn a module into a package with the same 
name. Right now, we could create a std.alg package with sub-modules containing 
all of std.algorithm's functionality and change std.algorithm to pubicly 
import them all, but you can't turn std.algorithm itself into a package 
without breaking code. The package.d portion of the proposal makes it so that 
you can.

Now, public import's current behavior is _almost_ enough to make the second 
part completely unnecessary. The only change that would be required would be 
to make it so that std.algorithm.x looks in std/algorithm/package.d if there's 
no std.algorithm.x module, because otherwise the public imports in package.d 
would be putting all of the symbols in std.algorithm.package, not 
std.algorithm (that, and package.d isn't legal at present).

So, the whole point of this proposal - to seemlessly allow the transition of a 
module to a package in place - _does_ require a language/compiler change. But 
moving stuff from a module to a new package does work already.

- Jonathan M Davis
Apr 05 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> I  
 suppose the only thing we don't get is being able to have a module and a
 package with the same FQN. I don't see that being a major issue.


 What doesn't work is being able to turn a module into a package with the  
 same
 name. Right now, we could create a std.alg package with sub-modules  
 containing
 all of std.algorithm's functionality and change std.algorithm to pubicly
 import them all, but you can't turn std.algorithm itself into a package
 without breaking code.


But so what?  nobody has any code like:

import std.algorithm.sort;

So who cares where that module goes?  I agree it would be ideal to put it  
there, but I don't think it's strictly necessary.  And there is no need  
for the shortcut for fully qualified names.


 So, the whole point of this proposal - to seemlessly allow the  
 transition of a
 module to a package in place - _does_ require a language/compiler change.


I don't see how.  Just move the code into another module and publicly  
import that module from std/algorithm.d.  Problem pretty much solved.

BTW, importing a directory was already proposed in DIP15.  
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

-Steve
Apr 05 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 14:24:10 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:

 On Thursday, April 05, 2012 09:49:59 Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 09:23:25 -0400, Timon Gehr <timon.gehr gmx.ch> I
 suppose the only thing we don't get is being able to have a module and a
 package with the same FQN. I don't see that being a major issue.


 
 What doesn't work is being able to turn a module into a package with the
 same
 name. Right now, we could create a std.alg package with sub-modules
 containing
 all of std.algorithm's functionality and change std.algorithm to pubicly
 import them all, but you can't turn std.algorithm itself into a package
 without breaking code.


 
 But so what? nobody has any code like:
 
 import std.algorithm.sort;
 
 So who cares where that module goes? I agree it would be ideal to put it
 there, but I don't think it's strictly necessary. And there is no need
 for the shortcut for fully qualified names.


 So, the whole point of this proposal - to seemlessly allow the
 transition of a
 module to a package in place - _does_ require a language/compiler change.


 
 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.


The issue is code organization. If you want to split up std.algorithm (or 
std.datetime or whatever) into multiple modules, you have to create a new 
package with a completely different name with no connection to the original 
save for the fact that the original publicly imports it. For instance, with 
the work that I've done thus far on splitting std.datetime, I've had to create 
a std.dtime package to hold the modules and have std.datetime pubicly import 
them. This automatically creates the issue of what the difference between them 
is (for anyone new to Phobos) and does not indicate their relation at all in 
the hierarchy. It would be much cleaner to be able to turn std/datetime.d into 
std/datetime/ with a package.d in it along with the new modules.

No, we don't _have_ to do something to make it so that std.algorithm can be 
turned into std/algorithm/ while still not breaking code. But it would be very 
nice. Certainly, I don't understand why this DIP would ever have been proposed 
if Andrei didn't find it valuable.


 BTW, importing a directory was already proposed in DIP15.
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15


Yes, but having a package.d with the public imports gives you much finer-
grained control over what gets imported, and DIP15 doesn't solve the problem 
of fully qualified uses of std.alorgithm.sort not breaking when 
std.algorithm.sort gets moved to something like std.algorithm.sorting.d. So, 
DIP15 doesn't work as a means of seemlessly breaking up a module. It just 
_mostly_ works as one (since people usually don't fully qualify symbols). 
Also, package.d would give us a means for documenting a package, which I would 
very much like to be able to do. Having std.datetime give an overview of the 
std.dtime package is definitely worse than having a means of having the package 
document itself - which the package.d file should be able to give us.

The package.d portion of DIP16 allows a means of controlling what importing a 
package would mean, it provides a means of turning a module into package in 
place, and it potentially provides a way of documenting a package - all of 
which are valuable. Whether they're valuable enough to merit a language change 
is obviously up for debate, but certainly, as the designer and primary 
maintainer of one of the main targets for being split up, I very much like the 
idea of being able to split up a module in place rather than having to create 
a new package with a new name with no obvious relation to the original.

- Jonathan M Davis
Apr 05 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:



 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.


 The issue is code organization. If you want to split up std.algorithm (or
 std.datetime or whatever) into multiple modules, you have to create a new
 package with a completely different name with no connection to the  
 original
 save for the fact that the original publicly imports it.


My view is that people will not import the smaller modules, they will only  
ever import std.algorithm.

Look at std.bigint, which imports modules from std.internal.math.  Nobody  
ever imports std.internal.math.??? because they just import std.bigint.


 BTW, importing a directory was already proposed in DIP15.
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15


 Yes, but having a package.d with the public imports gives you much finer-
 grained control over what gets imported, and DIP15 doesn't solve the  
 problem
 of fully qualified uses of std.alorgithm.sort not breaking when
 std.algorithm.sort gets moved to something like std.algorithm.sorting.d.


I think you are forgetting that all current code imports std.algorithm,  
which will register the symbol std.algorithm.sort via public import.  No  
code will break, even if it uses FQN.

The only way to "break" code is to write new import statements.  I don't  
think anyone will do that unknowingly.

-Steve
Apr 05 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/12 4:26 PM, Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
 <jmdavisProg gmx.com> wrote:


 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:



 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.


 The issue is code organization. If you want to split up std.algorithm (or
 std.datetime or whatever) into multiple modules, you have to create a new
 package with a completely different name with no connection to the
 original
 save for the fact that the original publicly imports it.


 My view is that people will not import the smaller modules, they will
 only ever import std.algorithm.


I think we should be looking for a solution that not only allows 
replacing module -> package transparently, but also allows people to 
import the newly introduced fine-grained modules.

Andrei
Apr 05 2012
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 16:43:24 Andrei Alexandrescu wrote:

 On 4/5/12 4:26 PM, Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
 
 <jmdavisProg gmx.com> wrote:

 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:

 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.


 
 The issue is code organization. If you want to split up std.algorithm (or
 std.datetime or whatever) into multiple modules, you have to create a new
 package with a completely different name with no connection to the
 original
 save for the fact that the original publicly imports it.


 
 My view is that people will not import the smaller modules, they will
 only ever import std.algorithm.


 
 I think we should be looking for a solution that not only allows
 replacing module -> package transparently, but also allows people to
 import the newly introduced fine-grained modules.


Yeah. If all we want to do is continue to always import std.algorithm, then 
the DIP is more or less pointless. It's the splitting of the API among 
multiple packages while allowing the programmer to either call/import it like 
he has been or to call/import it from the new module explicitly that the DIP 
is trying to enable.

If we make it possible to split std.algorithm into multiple modules in place, 
then we avoid breaking code, keep the code organized in the same hierarchy - 
only more detailed - and allow the programmer to import on either a roughly or 
finely grained level, depending on which they prefer. And I really like how 
this could enable us to have package-specific documentation, whereas all 
documentation is currently module-specific and doesn't enable us to provide a 
page which gaves an overview of a package. That's not always necessary, but 
there are times when it would be quite nice (e.g. std.datetime).

- Jonathan M Davis
Apr 05 2012
prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 I think we should be looking for a solution that not only allows 
 replacing module -> package transparently, but also allows people to 
 import the newly introduced fine-grained modules.


I think it'd be valuable too. But how do you do that without creating 
ambiguous fully qualified names?

One way would be that by importing a module inside a package, the 
compiler would check first that the package file contains no symbol 
with that module name. In other words, if you import 
std.algorithm.sorting, it'd check that "algorithm" is not a symbol 
defined in the "std" package file (if it exists for std) and that no 
"sorting" symbol exists in the "std.algorithm" package file. That would 
work. But the problem is that if the package file publicly imports all 
of its submodules, then you'll need to parse all the imported files to 
determine if the module you're trying to import conflicts with a 
package-level symbol.

A more practical option would be limit the package files to only two things:

1. a list of modules to import when you import the package (importing 
the package would be akin importing all modules in this list)

2. a list of symbols aliased in the package's namespace (aliased 
symbols can be accessed using the package name as a prefix and through 
selective imports)

Since the list of aliases is stored directly in the package file, 
reading the package file to get the names of each alias is enough to 
tell there is no conflict with the module name when you're trying to 
import it. No need to open the other modules the package refers to, or 
to resolve the symbols the aliases refer to: you only need the name of 
each alias to verify there is no conflict.

For instance, a package file could look like this:

	package std.algorithm;

	// modules to import when a file is importing the package
	// (those are not imported inside this package's namespace)
	module std.algorithm.sorting;
	module std.algorithm.mapping;
	...

	// symbols to alias to this package namespace
	// (importing a module with one of these names is an error)
	alias std.algorithm.sorting.completeSort  completeSort;
	alias std.algorithm.sorting.isSorted      isSorted;
	alias std.algorithm.sorting.partialSort   partialSort;
	alias std.algorithm.sorting.schwartzSort  schwartzSort;
	alias std.algorithm.sorting.sort          sort;
	...

The drawback is that it's hard to maintain the list of fully-qualified 
names up to date if you change it in the various modules. But if the 
goal is only to preserve backward compatibility, creating that list of 
aliases could be a one-time thing. New symbols would continue to be 
imported when you import std.algorithm, but they'd not be available 
directly under std.algorithm: you'd need to use 
std.algorithm.sorting.newSort if you find yourself in a situation that 
requires a fully-qualified name.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 05 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 06/04/2012 01:32, Michel Fortin a écrit :

 On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> said:


 I think we should be looking for a solution that not only allows
 replacing module -> package transparently, but also allows people to
 import the newly introduced fine-grained modules.


 I think it'd be valuable too. But how do you do that without creating
 ambiguous fully qualified names?


It isn't possible. But as already mentioned, all name doesn't make sense 
in all situation, so most of the time, disambiguation can be done.

Plus, we want to be able to split module when they grow, and in such a 
situation, collisions will never happen, because all symbols comes from 
the same module in a first place.
Apr 06 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/6/12 4:43 AM, deadalnix wrote:

 Le 06/04/2012 01:32, Michel Fortin a écrit :

 On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> said:


 I think we should be looking for a solution that not only allows
 replacing module -> package transparently, but also allows people to
 import the newly introduced fine-grained modules.


 I think it'd be valuable too. But how do you do that without creating
 ambiguous fully qualified names?


 It isn't possible. But as already mentioned, all name doesn't make sense
 in all situation, so most of the time, disambiguation can be done.

 Plus, we want to be able to split module when they grow, and in such a
 situation, collisions will never happen, because all symbols comes from
 the same module in a first place.


One other desirable feature is library distribution. Library writers 
should be able to offer library xyz as a sheer directory. Then library 
users should be able to import path.to.xyz and simply get going, or even 
rename xyz to xyz1 and import path.to.xyz1 without a problem.

Andrei
Apr 06 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 06/04/2012 16:49, Andrei Alexandrescu a écrit :

 On 4/6/12 4:43 AM, deadalnix wrote:

 Le 06/04/2012 01:32, Michel Fortin a écrit :

 On 2012-04-05 21:43:24 +0000, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> said:


 I think we should be looking for a solution that not only allows
 replacing module -> package transparently, but also allows people to
 import the newly introduced fine-grained modules.


 I think it'd be valuable too. But how do you do that without creating
 ambiguous fully qualified names?


 It isn't possible. But as already mentioned, all name doesn't make sense
 in all situation, so most of the time, disambiguation can be done.

 Plus, we want to be able to split module when they grow, and in such a
 situation, collisions will never happen, because all symbols comes from
 the same module in a first place.


 One other desirable feature is library distribution. Library writers
 should be able to offer library xyz as a sheer directory. Then library
 users should be able to import path.to.xyz and simply get going, or even
 rename xyz to xyz1 and import path.to.xyz1 without a problem.

 Andrei


Good point. That is an argument for package.d, all.d or _.d .
Apr 06 2012
parent reply David Gileadi <gileadis NSPMgmail.com> writes:
On 4/6/12 10:52 AM, deadalnix wrote:

 Good point. That is an argument for package.d, all.d or _.d .


Or [packagename].d, where [packagename] is the name of its sibling 
folder (std/algorithm.d in the case of std/algorithm/sort.d).
Apr 06 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 06/04/2012 20:03, David Gileadi a écrit :

 On 4/6/12 10:52 AM, deadalnix wrote:

 Good point. That is an argument for package.d, all.d or _.d .


 Or [packagename].d, where [packagename] is the name of its sibling
 folder (std/algorithm.d in the case of std/algorithm/sort.d).


No, because in this case, this isn't in the same folder. Still I prefer 
that opetion because :

1/ You mostly distribute lib as archives, or in a managed way (package 
manager, build system, etc . . .)
2/ It is simpler as import a.b.c always does the same thing, and other 
solution introduce complexity in that process, and possible cases of error.
Apr 06 2012
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 23:43, Andrei Alexandrescu a écrit :

 On 4/5/12 4:26 PM, Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
 <jmdavisProg gmx.com> wrote:


 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:



 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.


 The issue is code organization. If you want to split up std.algorithm
 (or
 std.datetime or whatever) into multiple modules, you have to create a
 new
 package with a completely different name with no connection to the
 original
 save for the fact that the original publicly imports it.


 My view is that people will not import the smaller modules, they will
 only ever import std.algorithm.


 I think we should be looking for a solution that not only allows
 replacing module -> package transparently, but also allows people to
 import the newly introduced fine-grained modules.

 Andrei


Why not limit name collision to name which make sense ?

For instance, import std.a.b.c is a module. if it refers also to a 
function, this import doesn't make any sense, so, even if we have a name 
collision, this isn't a big deal (except maybe for reflection ?).

Same goes for std.a.b.c(); which is a function call, and obviously not 
the module. Here what I propose to resolve names :

1/ import does always find the .d corresponding file. No exception.
2/ Module a.b.c is in package a, a.b and a.b.c . Any package declaration 
in a.b.c match package a.b (one level is removed).
3/ When a name is used in the code and have to be resolved, the 
following process occurs :
  - The compiler find all stuff that have this name.
  - The compiler discard all stuffs that have this name and doesn't make 
sense.
  - If all remaining items are overload of the same item, then standard 
best match rule apply.
  - If all remaining items aren't in the same module, or overload or 
different items, an error occurs. This is never a problem in the case of 
big modules splitted in submodules.

Some examples :

a.d

public import a.b;  // import a/b.d

class b {
     static void foo() {}
}

****************

a/b.d

public import a.b.c;  // import a/b/c.d

void foo(int i) {}

****************

a/b/c.d

void foo() {}

****************

main.d

import a;

foo();  // foo from a/b/c.d
foo(2);  // foo from a/b.d
a.foo();  // foo from a/b/c.d
a.b.foo();  // Error, match both a/b.d and a/b/c.d
a.b.foo(2);  // foo from a/b.d
a.b.c.foo();  // foo from a/b/c.d
Apr 06 2012
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-06 09:41:27 +0000, deadalnix <deadalnix gmail.com> said:


 Why not limit name collision to name which make sense ?


You're proposing that fully qualified names sometime work and sometime 
do not work based on what you've imported and the context of usage. 
This makes any FQN conflict hard to detect, because the error will be 
dependent on the context in which you use the name.

You say it'll never happen for modules spitted in submodules, but I 
disagree. Someday, someone will add a symbol to a submodule without 
checking every package/module file underneath. Unit tests in the module 
won't catch the problem because for them to catch the problem the 
parent module/package has to be imported. Only on certain usages, with 
the right imports and in the right context will the error occur.

In fact, if you add disambiguation based on what "makes sense" you're 
making the error more obscure and harder to detect in the first place 
because sometime it'll occur and sometime not depending on hard to 
predict conditions.

If we allow arbitrary symbols to live inside a package, any conflict in 
FQN should be caught as soon as possible, ideally as soon as you 
compile after having added the symbol, and even if you don't use the 
symbol anywhere. Just like how you get an error today when you try to 
import a module matching a known package name, you shouldn't be allowed 
to import a module matching a known symbol name, or create a symbol 
matching a module fully qualified name.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 06 2012
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:43:24 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 4/5/12 4:26 PM, Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 17:00:56 -0400, Jonathan M Davis
 <jmdavisProg gmx.com> wrote:


 On Thursday, April 05, 2012 15:30:17 Steven Schveighoffer wrote:



 I don't see how. Just move the code into another module and publicly
 import that module from std/algorithm.d. Problem pretty much solved.


 The issue is code organization. If you want to split up std.algorithm  
 (or
 std.datetime or whatever) into multiple modules, you have to create a  
 new
 package with a completely different name with no connection to the
 original
 save for the fact that the original publicly imports it.


 My view is that people will not import the smaller modules, they will
 only ever import std.algorithm.


 I think we should be looking for a solution that not only allows  
 replacing module -> package transparently, but also allows people to  
 import the newly introduced fine-grained modules.


Of course you *can* do this.  I think you mean "and allows one to refer to  
the fine grained module as if it were imported from the top-level module".

I don't think that benefit is very great.  Why shouldn't we expect people  
to use the module name they actually imported to refer to a module?  If  
you want selective import, we have:

import std.algorithm : sort;

I think the real benefit to splitting up the module comes from being able  
to draw clear lines between which pieces of a large module are related.

I feel like most people will still import the main package module, and not  
the submodules.  I don't think I ever wrote a piece of java code that  
didn't have:

import java.io.*;

I keep coming back to std.container.  Already it's large, and full of  
unrelated types.  It's only going to get worse.

Now, I fully agree that having some way to import a package by itself  
without having to import all its modules would be ideal (as well as  
splitting a large module into submodules that live in the same  
namespace).  I think DIP15 has that covered.

-Steve
Apr 06 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:

 I feel like most people will still import the main package module, and not
 the submodules.  I don't think I ever wrote a piece of java code that
 didn't have:
 
 import java.io.*;


Which is actually considered bad practice in Java, though a lot of people do 
like to do it. What's generally considered good practice is to explicitly 
import every module/class, and Eclipse likes to underline imports in red if 
you don't.


 I keep coming back to std.container.  Already it's large, and full of
 unrelated types.  It's only going to get worse.
 
 Now, I fully agree that having some way to import a package by itself
 without having to import all its modules would be ideal (as well as
 splitting a large module into submodules that live in the same
 namespace).  I think DIP15 has that covered.


DIP15 doesn't fix the explicit path problem though. You can't change 
std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.) without 
breaking code. You could make std/algorithm.d publicly import std/alg/* and 
then DIP15 would allow you to import std.alg to get all of its sub-modules, 
but you're still forced to use a module to publicly import symbols as part of 
a migration path, and you can't split a module in place.

DIP15 also less powerful than having a package.d, since it'll automatically 
import ever public symbol when it may be desirable to have it only import a 
subset of them or to publicly import something from another module (e.g. as 
part of a migration path - like moving the benchmarking code in std.datetime 
to std.benchmark rather than lumping it with the std.datetime stuff when it's 
split into separate modules). That may or may not be a big deal, but it does 
mean that DIP15 isn't as powerful as DIP16's package.d solution in that 
regard.

DIP15 also doesn't give a means of documentating a package. By having a 
package.d file, we have a really good place to provide an overarching 
description for the package in ddoc.

So, while I don't like the second part of DIP16 at all (i.e. the std.sort 
stuff), I think that the package.d part is a much better way to go.

- Jonathan M Davis
Apr 06 2012
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-04-07 02:25, Jonathan M Davis wrote:

 On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:

 I feel like most people will still import the main package module, and not
 the submodules.  I don't think I ever wrote a piece of java code that
 didn't have:

 import java.io.*;


 Which is actually considered bad practice in Java, though a lot of people do
 like to do it. What's generally considered good practice is to explicitly
 import every module/class, and Eclipse likes to underline imports in red if
 you don't.


import foo.bar.*

Is used all over the place in SWT.

-- 
/Jacob Carlborg
Apr 07 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, April 07, 2012 18:45:15 Jacob Carlborg wrote:

 On 2012-04-07 02:25, Jonathan M Davis wrote:

 On Friday, April 06, 2012 08:09:28 Steven Schveighoffer wrote:

 I feel like most people will still import the main package module, and
 not
 the submodules.  I don't think I ever wrote a piece of java code that
 didn't have:
 
 import java.io.*;


 
 Which is actually considered bad practice in Java, though a lot of people
 do like to do it. What's generally considered good practice is to
 explicitly import every module/class, and Eclipse likes to underline
 imports in red if you don't.


 
 import foo.bar.*
 
 Is used all over the place in SWT.


Like I said, some people do like to do it, but Eclipse doesn't like you to, 
and there are quite a few Java folks who argue that it's bad practice. I 
forget what the reasons were (maybe increased buld times due to pulling in 
more symbols or more issues with symbol conflicts - I don't recall), but I 
wasn't particularly convinced when I heard them. Regardless though, there's a 
definite contingent against importing with * in the Java community, and it was 
my understanding that that contigent was the majority of that community, but I 
don't know.

- Jonathan M Davis
Apr 07 2012
parent "Kapps" <opantm2+spam gmail.com> writes:
On Saturday, 7 April 2012 at 17:44:20 UTC, Jonathan M Davis wrote

 Like I said, some people do like to do it, but Eclipse doesn't 
 like you to,
 and there are quite a few Java folks who argue that it's bad 
 practice. I
 forget what the reasons were (maybe increased buld times due to 
 pulling in
 more symbols or more issues with symbol conflicts - I don't 
 recall), but I
 wasn't particularly convinced when I heard them. Regardless 
 though, there's a
 definite contingent against importing with * in the Java 
 community, and it was
 my understanding that that contigent was the majority of that 
 community, but I
 don't know.

 - Jonathan M Davis


http://stackoverflow.com/questions/147454/why-is-using-a-wild-card-with-a-java-import-statement-bad

Their reasoning sounds more due to various packages reinventing
things or being poorly split than actual flaws with package
imports themselves. Besides, D already addresses the issues of
indicating which one you want, without the horribly long package
names that Java has. Seems to me their examples are more like
'import std.*' than 'import std.datetime.*'.
Apr 07 2012
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 06 Apr 2012 20:25:23 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 DIP15 doesn't fix the explicit path problem though. You can't change
 std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.)  
 without
 breaking code. You could make std/algorithm.d publicly import std/alg/*  
 and
 then DIP15 would allow you to import std.alg to get all of its  
 sub-modules,
 but you're still forced to use a module to publicly import symbols as  
 part of
 a migration path, and you can't split a module in place.


I think either you or I am missing something.

In DIP15, if you define std/algorithm/_.d, and then import std.algorithm,  
it imports std/algorithm/_.d, which then 1. publicly imports other  
modules, and 2. aliases symbols to the name std.algorithm.symbol.  At  
least, this is how I understand the intent.  It seems equivalent to me to  
the package.d proposal, it's just using _.d instead of package.d.

If you import std.algorithm.sorting, and try and use std.algorithm.sort,  
yes it will not work.  But this does not break existing code (which does  
not import std.algorithm.sorting), and I find it odd that we want to make  
std.algorithm.sort work if you don't import std.algorithm.

-Steve
Apr 09 2012
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, April 09, 2012 08:55:27 Steven Schveighoffer wrote:

 On Fri, 06 Apr 2012 20:25:23 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:

 DIP15 doesn't fix the explicit path problem though. You can't change
 std/algorithm.d into std/algorithm/ (with sorting.d, search.d, etc.)
 without
 breaking code. You could make std/algorithm.d publicly import std/alg/*
 and
 then DIP15 would allow you to import std.alg to get all of its
 sub-modules,
 but you're still forced to use a module to publicly import symbols as
 part of
 a migration path, and you can't split a module in place.


 
 I think either you or I am missing something.
 
 In DIP15, if you define std/algorithm/_.d, and then import std.algorithm,
 it imports std/algorithm/_.d, which then 1. publicly imports other
 modules, and 2. aliases symbols to the name std.algorithm.symbol. At
 least, this is how I understand the intent. It seems equivalent to me to
 the package.d proposal, it's just using _.d instead of package.d.
 
 If you import std.algorithm.sorting, and try and use std.algorithm.sort,
 yes it will not work. But this does not break existing code (which does
 not import std.algorithm.sorting), and I find it odd that we want to make
 std.algorithm.sort work if you don't import std.algorithm.


Okay. I reread DIP15 again. I guess that I scanned over it too quickly before 
and/or misremembered it. I had understood that it was proposing that 
importing std.algorithm where std.algorithm was a package would be the
equivalent of importing std.algorithm.* in Java and that there were no extra
files involved. So clearly, I've been misunderstanding things here.

So, yeah. DIP15 is basically the same as DIP16 except without the std.sort 
nonsense and the fact that it uses _.d instead of package.d. Using package.d 
has the advantage of package being a keyword, making it so that no one is 
going to accidentally create a module that will be treated specially, but it 
has the downside of likely requiring more special handling by the compiler. I 
don't really care which we pick though.

My main point though, misunderstandings aside, is that it would be _really_
nice to be able to split up  a package in place and that without an
enhancement of some kind, we can't do  that without breaking code. DIP15
appears to fit the bill quite nicely in that regard though. The part of
DIP16 which is really bad is the std.sort stuff which. Public importing
combined with either the first part of DIP16 or with DIP15 seems to take
care of the problem quite nicely.

- Jonathan M Davis
Apr 09 2012
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 14:58, Steven Schveighoffer a écrit :

 On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrote:


 Le 05/04/2012 13:46, Steven Schveighoffer a écrit :

 I don't like this proposal, simply because this means one
 function/symbol per submodule. It should be more flexible than that. But
 I like the line of thinking.

 Let's re-examine the issue. We need to allow splitting of a module X.d
 into pieces for maintenance (or possibly accessibility -- disallowing
 friends). But we don't want to break code which currently uses FQN to
 access X's symbols.

 I think the following might work:

 algorithm.d:

 import this = std.algorithm_impl.sort;

 Which then imports std.algorithm_impl.sort, and effectively aliases all
 its symbols into algorithm.d. If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort. In
 essence, sort has *two* FQN, but there are no FQN that refer to more
 than one symbol.

 I purposely left out the package.d idea because it's orthogonal to this.

 -Steve


 The behavior you described has been proposed for public import, and
 have been discussed. This is interesting.

 You propose an alternative syntax with is fine and have the advantage
 to not disturb already existing public imports, but have the drawback
 to create a new syntax, again.

 If such a syntax is adopted, what would be the point of public imports
 ? If it is still useful, then your syntax is a better choice,
 otherwise, we'd better modify public import.


 No, public imports simply mean that you can view the publicly imported
 module. It does *not* add aliases to the importing module.

 for example:

 foo.d:

 module foo;

 int x;

 bar.d:
 module bar;
 public import foo;

 int y;

 main.d:
 import bar; // publicly imports foo.

 void main()
 {
 foo.x = 5; // ok, publicly imported via bar (non public, this would be
 an error)
 bar.y = 5; // ok
 bar.x = 5; // error, public import doesn't create new fully qualified
 name for foo.x
 }

 With the system I propose, using the specific technique would make it so
 bar.x would also be valid, and would refer to foo.x

 I think we need new syntax, or a new language feature, because you don't
 want to alter the operation of existing code. Simply changing public
 imports would cause lots of existing code to be reinterpreted, possibly
 in a way not intended, or that would be ambiguous.

 -Steve


I know. I think you answered too fast.

I wasn't stating that public import are the same as what you propose. I 
was discussing the fact that both probably answer the same need and that 
2 different syntax is something to avoid.

Code wouldn't be broken in unexpected way, because thing will cause 
either a compile error (collision of names, but the problem exists with 
other solutions too) or is illegal in the current shape of the language.
Apr 05 2012
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:26:16 -0400, deadalnix <deadalnix gmail.com> wrot=
e:


 Le 05/04/2012 14:58, Steven Schveighoffer a =C3=A9crit :

 On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com>  =





 wrote:


 Le 05/04/2012 13:46, Steven Schveighoffer a =C3=A9crit :


 No, public imports simply mean that you can view the publicly importe=




d

 module. It does *not* add aliases to the importing module.




[snip]


 I know. I think you answered too fast.

 I wasn't stating that public import are the same as what you propose. =


I  =


 was discussing the fact that both probably answer the same need and th=


at  =


 2 different syntax is something to avoid.

 Code wouldn't be broken in unexpected way, because thing will cause  =



 either a compile error (collision of names, but the problem exists wit=


h  =


 other solutions too) or is illegal in the current shape of the languag=


e.

My assertion above is 100% *WRONG*.  See this bug I just filed.   =

http://d.puremagic.com/issues/show_bug.cgi?id=3D7830

In fact public imports today are *exactly* the same as what I proposed. =
I  =

had no idea.

I think we are (well, at least I am) on a wild goose chase here.  All th=
at  =

is left to discuss is if/how we want to allow importing of packages.

But I think DIP15 already covers that  =

http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15

I think DIP16 is worksforme/invalid, and should be closed.

-Steve
Apr 05 2012
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 08:49:24 -0400, deadalnix <deadalnix gmail.com> wrote:


 If such a syntax is adopted, what would be the point of public imports ?  
 If it is still useful, then your syntax is a better choice, otherwise,  
 we'd better modify public import.


Another way to look at it:

A public import saves you from having to import dependent modules that you  
should be importing anyway.
A 'this' import treats the other modules as if they were actually part of  
the imported module in terms of namespace.

To give an example, std.range imports std.array.  std.array is a module  
all on its own, and has a specific set of functionality.  std.range has a  
different set of functionality, but you would never want to have std.range  
imported without also importing std.array.  Another example would be a  
derived class module publicly importing the base class module.

On the other hand, something like std.container could have 15 different  
container types in it.  Each of these container types should really live  
in their own module, in terms of maintenance and separation of private  
data.  For example, std.container.RedBlackTree has no business accessing  
the private members of std.container.Array.  But because they live in the  
same module, it can.  However, we don't want to define  
std.container.RedBlackTree.RedBlackTree in terms of namespace.  So we have  
one of these combining modules, and everything still lives in the  
std.container namespace, but we get all the benefits of separating the  
code into individual modules.

-Steve
Apr 05 2012
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-05 11:46:38 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 I don't like this proposal, simply because this means one function/symbo l
 per submodule.  It should be more flexible than that.  But I like the li ne
 of thinking.


I have the same reserve about it's lack of flexibility too. It would be 
a nice thing to have for all those libraries having one module per 
class as it'd reduce the length of the fully qualified name you have to 
use when you need to disambiguate. But for moving a module like 
std.algorithm to a package, it's cumbersome.



 Let's re-examine the issue.  We need to allow splitting of a module X.d
 into pieces for maintenance (or possibly accessibility -- disallowing
 friends).  But we don't want to break code which currently uses FQN to
 access X's symbols.
 
 I think the following might work:
 
 algorithm.d:
 
 import this = std.algorithm_impl.sort;
 
 Which then imports std.algorithm_impl.sort, and effectively aliases all
 its symbols into algorithm.d.  If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort.  In
 essence, sort has *two* FQN, but there are no FQN that refer to more tha n
 one symbol.


This is what a public import should do.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 05 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:33:14 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2012-04-05 11:46:38 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


 I don't like this proposal, simply because this means one  
 function/symbo l
 per submodule.  It should be more flexible than that.  But I like the  
 li ne
 of thinking.


 I have the same reserve about it's lack of flexibility too. It would be  
 a nice thing to have for all those libraries having one module per class  
 as it'd reduce the length of the fully qualified name you have to use  
 when you need to disambiguate.


I see the point you are going for.  I don't really like the  
one-class-per-module style as much as how phobos has one module per  
category.  But that's not an excuse not to examine this possibility.

I think it's a different issue than what DIP16/DIP15 is trying to solve.


 But for moving a module like std.algorithm to a package, it's cumbersome.


Not really.  Just move the files to another directory (i.e.  
std/internal/algorithm) and publicly import them.


 Let's re-examine the issue.  We need to allow splitting of a module X.d
 into pieces for maintenance (or possibly accessibility -- disallowing
 friends).  But we don't want to break code which currently uses FQN to
 access X's symbols.
  I think the following might work:
  algorithm.d:
  import this = std.algorithm_impl.sort;
  Which then imports std.algorithm_impl.sort, and effectively aliases all
 its symbols into algorithm.d.  If std.algorithm_impl.sort defines a
 function called sort, then it's also aliased to std.algorithm.sort.  In
 essence, sort has *two* FQN, but there are no FQN that refer to more  
 tha n
 one symbol.


 This is what a public import should do.


I wasn't aware of this.  Mostly because the spec omits that feature.  I  
have to retract almost everything I've argued, because we already have the  
means to fix what I think DIP16 is trying to solve without any compiler  
changes.

-Steve
Apr 05 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 16:47, Steven Schveighoffer a écrit :

 But for moving a module like std.algorithm to a package, it's cumbersome.


 Not really. Just move the files to another directory (i.e.
 std/internal/algorithm) and publicly import them.


std/internal isn't good. If I just want to import sort, I would have to 
do std.internal.algorithm.sort, which isn't good.

Plus, the package accessibility modifier would be broken with such a 
pattern.

The all.d, package.d, _.d, or, as I propose foldername.d (and submodules 
into foldername) do not break package accessibility modifier, which is 
better.
Apr 05 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com> wrot=
e:


 Le 05/04/2012 16:47, Steven Schveighoffer a =C3=A9crit :

 But for moving a module like std.algorithm to a package, it's  =







 cumbersome.


 Not really. Just move the files to another directory (i.e.
 std/internal/algorithm) and publicly import them.


 std/internal isn't good. If I just want to import sort, I would have t=


o  =


 do std.internal.algorithm.sort, which isn't good.


import std.algorithm : sort;

BTW, this doesn't work today, I'll file a bug.


 Plus, the package accessibility modifier would be broken with such a  =



 pattern.

 The all.d, package.d, _.d, or, as I propose foldername.d (and submodul=


es  =


 into foldername) do not break package accessibility modifier, which is=


  =


 better.


I don't really know what you mean, I didn't read that post.  Would you  =

mind posting a link or repeating that argument?

-Steve
Apr 05 2012
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:59:31 -0400, Steven Schveighoffer  =

<schveiguy yahoo.com> wrote:


 On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com>  =



 wrote:


 Le 05/04/2012 16:47, Steven Schveighoffer a =C3=A9crit :

 Not really. Just move the files to another directory (i.e.
 std/internal/algorithm) and publicly import them.


 std/internal isn't good. If I just want to import sort, I would have =




to  =


 do std.internal.algorithm.sort, which isn't good.


 import std.algorithm : sort;

 BTW, this doesn't work today, I'll file a bug.


Nevermind, it works the same as if sort is defined in a std.algorithm  =

module.  I'm kind of surprised the FQN doesn't work though...

-Steve
Apr 05 2012
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:59:31 -0400, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 On Thu, 05 Apr 2012 10:53:34 -0400, deadalnix <deadalnix gmail.com>  
 wrote:

 Plus, the package accessibility modifier would be broken with such a  
 pattern.

 The all.d, package.d, _.d, or, as I propose foldername.d (and  
 submodules into foldername) do not break package accessibility  
 modifier, which is better.


 I don't really know what you mean, I didn't read that post.  Would you  
 mind posting a link or repeating that argument?


Also, nevermind :)  I realize what you are talking about now.

This is a good point.  Of course, there are other mechanisms to do this,  
and DIP15 already proposes an improvement for this.

However, at least in the case of phobos, I don't think there's a lot of  
instances of package accessibility modifier.

-Steve
Apr 05 2012
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2012-04-04 17:53:24 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 But yes, I think the issue really becomes, we need to  look at context 
 when deciding the semantic meaning of a symbol.  I don't  think this 
 violates the context-free grammar, because wouldn't this only  come 
 into play at the semantic level?  Not a compiler writer/hacker, so I  
 don't know.


You'd need a whole lot of context.

Let's say I write:

	.a.b.c.d = .e.f.g.h;

Can you tell me which letters are the name of a package/module, and 
which are the name of something else? It could be this:

	module a.b;
	struct c { static int d; }

	module e.f.g;
	int h;

or it could be this:

	module a;
	struct b { struct c { struct d { static void opAssign(int); } } }

	module e;
	struct f { enum g { h = 1 } }

or a multitude of other variants. If you can't know by looking at the 
assignment above, how will the parser know?

Fully qualified names really need to refer to a single thing.



 Say you wanted to create a package.d file directly for the whole 
 package  std, what should be done for those?


 
 No, let's not do that.  Ever. :)


Phobos is just an example. People will want to use "package.d" files 
for their own libraries too, and will complain if it doesn't work.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Apr 04 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/4/12 10:53 AM, Steven Schveighoffer wrote:

 On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
 <michel.fortin michelf.com> wrote:

 Question 2: does std.algorithm.sort refer to the std.algorithm.sort
 *module* or to std.algorithm.package.sort *function* publicly imported
 from the std.algorithm.sort module?


 The function. A symbol that is not specified as the module name in
 import statement or in a module statement is always *not* a module. I
 think our one saving grace here is that when you want to import a
 specific symbol from a module, this is not the syntax:

 import std.stdio.writefln;

 So there is never any ambiguity as to whether you mean a module
 identifier or other symbol.


Interesting. But isn't there an ambiguity when the symbol is not the 
last one in the chain? Consider:

a.b.c.x

Could be static member d of class c in module a.b, or module member d in 
module a.b.c.


Andrei
Apr 05 2012
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 10:35:52 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 4/4/12 10:53 AM, Steven Schveighoffer wrote:

 On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
 <michel.fortin michelf.com> wrote:

 Question 2: does std.algorithm.sort refer to the std.algorithm.sort
 *module* or to std.algorithm.package.sort *function* publicly imported
 from the std.algorithm.sort module?


 The function. A symbol that is not specified as the module name in
 import statement or in a module statement is always *not* a module. I
 think our one saving grace here is that when you want to import a
 specific symbol from a module, this is not the syntax:

 import std.stdio.writefln;

 So there is never any ambiguity as to whether you mean a module
 identifier or other symbol.


 Interesting. But isn't there an ambiguity when the symbol is not the  
 last one in the chain? Consider:

 a.b.c.x

 Could be static member d of class c in module a.b, or module member d in  
 module a.b.c.


Stop reading all my posts except for the ones I just posted.  I was  
completely wrong in what I thought the compiler couldn't do, because the  
spec is lacking.

I currently think DIP16 is invalid/worksforme (public imports allows  
splitting a module into a package).  All that is left is how we could  
specifically import a package with one import statement.

DIP15 covers that.  But I don't even think we need that, we can fix Phobos  
without any compiler changes.

-Steve
Apr 05 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/5/12 7:43 AM, Steven Schveighoffer wrote:

 I currently think DIP16 is invalid/worksforme (public imports allows
 splitting a module into a package). All that is left is how we could
 specifically import a package with one import statement.


Not entirely (I was aware of the way public import works). An issue does 
exist - there are "too many names", i.e. the alias pulled in the 
importing module and also the name being imported. This makes for odd 
synonyms such as std.algorithm_package.sort.sort being the same as 
std.algorithm.sort. The other issue is that obviously algorithm_package 
and algorithm must have distinct names, which makes the scheme a bit 
awkward at least until we define a compelling convention. I guess we can 
live with all that.

Andrei
Apr 05 2012
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 11:23:12 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 4/5/12 7:43 AM, Steven Schveighoffer wrote:

 I currently think DIP16 is invalid/worksforme (public imports allows
 splitting a module into a package). All that is left is how we could
 specifically import a package with one import statement.


 Not entirely (I was aware of the way public import works). An issue does  
 exist - there are "too many names", i.e. the alias pulled in the  
 importing module and also the name being imported. This makes for odd  
 synonyms such as std.algorithm_package.sort.sort being the same as  
 std.algorithm.sort.


Right, but if one only ever imports std.algorithm, who cares what the  
submodule FQNs are?

AIUI, DIP16 also doesn't really reduce the number of names either.


 The other issue is that obviously algorithm_package and algorithm must  
 have distinct names, which makes the scheme a bit awkward at least until  
 we define a compelling convention. I guess we can live with all that.


Agreed.  I think Don mentioned std.math already does something, we may  
want to look at that model.

A couple issues that still need consideration:

1. If std.algorithm the module becomes std.algorithm the package, what  
happens with ddoc?  We probably *do* need a compiler solution to this.
2. deadalnix pointed out that if we come up with a scheme where the  
package module and its submodules are in the same directory, the package  
accessibility qualifier can be used (hey look, a use for the package  
keyword!).

-Steve
Apr 05 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:

 A couple issues that still need consideration:
 
 1. If std.algorithm the module becomes std.algorithm the package, what
 happens with ddoc? We probably *do* need a compiler solution to this.


That's assuming that you insist on keeping all of the documentation in one 
file. That arguably defeats the purpose of splitting up the modules. If there 
isn't enough in the module to split the documentation, then why do you need to 
split the module?

What _would_ be valuable and the package.d could provide is an overview of the 
package. The ddoc comment for the package.d module can become the 
documentation for the package as a whole.


 2. deadalnix pointed out that if we come up with a scheme where the
 package module and its submodules are in the same directory, the package
 accessibility qualifier can be used (hey look, a use for the package
 keyword!).


Yes. std.datetime will need that once it's split. Without that, much of it 
can't be split and/or code would have to be needlessly duplicated.

- Jonathan M Davis
Apr 05 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:

 A couple issues that still need consideration:

 1. If std.algorithm the module becomes std.algorithm the package, what
 happens with ddoc? We probably *do* need a compiler solution to this.


 That's assuming that you insist on keeping all of the documentation in  
 one
 file. That arguably defeats the purpose of splitting up the modules. If  
 there
 isn't enough in the module to split the documentation, then why do you  
 need to
 split the module?


I thought the whole point was code maintenance?  Not documentation  
splitting... I would have expected people to continue to treat  
std.algorithm like it was one module, even though it imports several  
sub-modules for its implementation.


 What _would_ be valuable and the package.d could provide is an overview  
 of the
 package. The ddoc comment for the package.d module can become the
 documentation for the package as a whole.


 2. deadalnix pointed out that if we come up with a scheme where the
 package module and its submodules are in the same directory, the package
 accessibility qualifier can be used (hey look, a use for the package
 keyword!).


 Yes. std.datetime will need that once it's split. Without that, much of  
 it
 can't be split and/or code would have to be needlessly duplicated.


Hm.. I just thought of something, as long as the main "package" module  
imports everything from the same directory, and doesn't define anything,  
this isn't an issue.

-Steve
Apr 05 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:

 On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:

 A couple issues that still need consideration:
 
 1. If std.algorithm the module becomes std.algorithm the package, what
 happens with ddoc? We probably *do* need a compiler solution to this.


 
 That's assuming that you insist on keeping all of the documentation in
 one
 file. That arguably defeats the purpose of splitting up the modules. If
 there
 isn't enough in the module to split the documentation, then why do you
 need to
 split the module?


 
 I thought the whole point was code maintenance? Not documentation
 splitting... I would have expected people to continue to treat
 std.algorithm like it was one module, even though it imports several
 sub-modules for its implementation.


If the module isn't large enough to be split for documentation, I find it hard 
to believe that it needs to be split for maintenance. And if all you care 
about is sub-modules for implementation and want all of the functions in the 
same module still, then this DIP is pointless. All you have to do is declare 
undocumented sub-modules which hold the various implementations and have the 
actual module call them. We already do this sort of thing in Phobos to get 
around static destructors screaming about circular dependencies.

I only see the need to split of a module as the DIP suggests when it's too 
big, and if the documentation isn't large enough to be an issue, then I don't 
see how the module itself is going to be too large unless it has a ton of 
helper stuff, and that stuff can be seemlessy put in another, undocumented 
module without needing to create a package.

- Jonathan M Davis
Apr 05 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 17:02:13 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Thursday, April 05, 2012 15:26:14 Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 14:33:22 -0400, Jonathan M Davis  
 <jmdavisProg gmx.com>

 wrote:

 On Thursday, April 05, 2012 11:30:26 Steven Schveighoffer wrote:

 A couple issues that still need consideration:

 1. If std.algorithm the module becomes std.algorithm the package,  




 what

 happens with ddoc? We probably *do* need a compiler solution to this.


 That's assuming that you insist on keeping all of the documentation in
 one
 file. That arguably defeats the purpose of splitting up the modules.  


 If

 there
 isn't enough in the module to split the documentation, then why do you
 need to
 split the module?


 I thought the whole point was code maintenance? Not documentation
 splitting... I would have expected people to continue to treat
 std.algorithm like it was one module, even though it imports several
 sub-modules for its implementation.


 If the module isn't large enough to be split for documentation, I find  
 it hard
 to believe that it needs to be split for maintenance.


Why do we ever need to split modules for documentation?  Just fix the doc  
generator so it's not as monolithic.  For instance, have one page per  
class or struct.


 And if all you care
 about is sub-modules for implementation and want all of the functions in  
 the
 same module still, then this DIP is pointless. All you have to do is  
 declare
 undocumented sub-modules which hold the various implementations and have  
 the
 actual module call them. We already do this sort of thing in Phobos to  
 get
 around static destructors screaming about circular dependencies.


You are starting to see my point :)  But I think the issue is not so much  
that you are splitting the implementation, but splitting up the API into  
related modules.

For example, std.container.  Imagine we have a robust set of 15  
containers.  Why should those all be in one file?  They have nothing to do  
with eachother except they are in the same namespace.  Why does  
RedBlackTree need to be able to access the internals of Array?

As the writer of RedBlackTree, I want to be able to test and develop my  
container without having to worry about the rest of std.container.  But I  
also would like to have the FQN of it to be std.container.RedBlackTree.   
Public imports allow this *today* without any changes.

Note that size isn't so much an issue for me as being able to  
compartmentalize and develop individual pieces of a module.

-Steve
Apr 05 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 17:02:13 -0400, Jonathan M Davis <jmdavisProg gmx.com>

 If the module isn't large enough to be split for documentation, I find
 it hard
 to believe that it needs to be split for maintenance.


 
 Why do we ever need to split modules for documentation? Just fix the doc
 generator so it's not as monolithic. For instance, have one page per
 class or struct.


That may or may not be desirable (certainly in the case of smaller types, I'd 
argue that it isn't). By doing it on a module basis, you have far more 
control. But regardless, that would be a major change to ddoc.

In either case, the size of the documentation page for a module is currently 
closely tied to the number of public symbols in the module, so if you have a 
large API, it can become desirable to split it up simply because the 
documentation page for it is too large. And by splitting up the API, you fix 
that problem. Not to mention, if the module is mostly free functions, putting 
the documentation for each class or struct on its own page doesn't help 
anyway. So, while that may be a good change to ddoc in at least some cases, it 
doesn't solve the problem in general. For instance, it would help 
std.datetime, but it wouldn't help std.algorithm at all.


 And if all you care
 about is sub-modules for implementation and want all of the functions in
 the
 same module still, then this DIP is pointless. All you have to do is
 declare
 undocumented sub-modules which hold the various implementations and have
 the
 actual module call them. We already do this sort of thing in Phobos to
 get
 around static destructors screaming about circular dependencies.


 
 You are starting to see my point :) But I think the issue is not so much
 that you are splitting the implementation, but splitting up the API into
 related modules.


As I understand it, the entire point of this DIP is to enable splitting up the 
API cleanly without breaking code. The implementation can already be split up 
seemlessly.

- Jonathan M Davis
Apr 05 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 19:14:42 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:

 Why do we ever need to split modules for documentation? Just fix the doc
 generator so it's not as monolithic. For instance, have one page per
 class or struct.


 That may or may not be desirable (certainly in the case of smaller  
 types, I'd
 argue that it isn't). By doing it on a module basis, you have far more
 control. But regardless, that would be a major change to ddoc.


ddoc's output leaves a lot to be desired.  The unorganized links at the  
top suck.  Using module order to show symbols instead of category of  
symbols.

How is a doc page ever "too big"?  Even std.datetime loads in a second.   
It's more that it's "too disorganized".


 And if all you care
 about is sub-modules for implementation and want all of the functions  


 in

 the
 same module still, then this DIP is pointless. All you have to do is
 declare
 undocumented sub-modules which hold the various implementations and  


 have

 the
 actual module call them. We already do this sort of thing in Phobos to
 get
 around static destructors screaming about circular dependencies.


 You are starting to see my point :) But I think the issue is not so much
 that you are splitting the implementation, but splitting up the API into
 related modules.


 As I understand it, the entire point of this DIP is to enable splitting  
 up the
 API cleanly without breaking code. The implementation can already be  
 split up
 seemlessly.


As can the API via public imports.

-Steve
Apr 06 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, April 06, 2012 07:56:41 Steven Schveighoffer wrote:

 On Thu, 05 Apr 2012 19:14:42 -0400, Jonathan M Davis <jmdavisProg gmx.com>
 
 wrote:

 On Thursday, April 05, 2012 17:33:50 Steven Schveighoffer wrote:

 Why do we ever need to split modules for documentation? Just fix the doc
 generator so it's not as monolithic. For instance, have one page per
 class or struct.


 
 That may or may not be desirable (certainly in the case of smaller
 types, I'd
 argue that it isn't). By doing it on a module basis, you have far more
 control. But regardless, that would be a major change to ddoc.


 
 ddoc's output leaves a lot to be desired.  The unorganized links at the
 top suck.  Using module order to show symbols instead of category of
 symbols.
 
 How is a doc page ever "too big"?  Even std.datetime loads in a second.
 It's more that it's "too disorganized".


There's no question that the links at the top suck, and std.datetime is the 
poster child for why they suck. The page is too large for how its organized, 
and it _does_ take significantly longer to load than other Phobos pages, even 
if it's still usable. There's no question that better links would help 
significantly though.

Regardless, there's enough content there that it would arguably be better 
served if it were multiple pages.


 And if all you care
 about is sub-modules for implementation and want all of the functions


 
 in
 

 the
 same module still, then this DIP is pointless. All you have to do is
 declare
 undocumented sub-modules which hold the various implementations and


 
 have
 

 the
 actual module call them. We already do this sort of thing in Phobos to
 get
 around static destructors screaming about circular dependencies.


 
 You are starting to see my point :) But I think the issue is not so much
 that you are splitting the implementation, but splitting up the API into
 related modules.


 
 As I understand it, the entire point of this DIP is to enable splitting
 up the
 API cleanly without breaking code. The implementation can already be
 split up
 seemlessly.


 
 As can the API via public imports.


Not and keep stuff in the same place in the hierarchy. You have to create a 
completely separate package. The point of the DIP was to make it so that you 
could do it _in place_ by turning a module like std/algorithm.d into the 
package std/algorithm/ with modules inside it.

Right now, I can go and split up std.datetime into a new package (e.g. 
std.dtime) and then publicly import it all from std.datetime, but then the new 
package is completely separate from the module which imports everything. It 
also makes it look like std.datetime should go away in favor of just keeping 
std.dtime (the same with std.algorithm if it gets split), which may or may not 
actually be desirable. Organizationally, it would be much nicer to be able to 
turn std/datetime.d into std/datetime/, and _that_ is what the DIP is trying 
to enable. The rest can be done now but not that. You can't split up the API 
in place.

- Jonathan M Davis
Apr 06 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 17:23, Andrei Alexandrescu a écrit :

 On 4/5/12 7:43 AM, Steven Schveighoffer wrote:

 I currently think DIP16 is invalid/worksforme (public imports allows
 splitting a module into a package). All that is left is how we could
 specifically import a package with one import statement.


 Not entirely (I was aware of the way public import works). An issue does
 exist - there are "too many names", i.e. the alias pulled in the
 importing module and also the name being imported. This makes for odd
 synonyms such as std.algorithm_package.sort.sort being the same as
 std.algorithm.sort. The other issue is that obviously algorithm_package
 and algorithm must have distinct names, which makes the scheme a bit
 awkward at least until we define a compelling convention. I guess we can
 live with all that.

 Andrei


The first one isn't a problem. It isn't too many names, it is 2 names, 
and it happen when explicitly told to do so.

The second is.
Apr 05 2012
prev sibling parent deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 16:35, Andrei Alexandrescu a écrit :

 On 4/4/12 10:53 AM, Steven Schveighoffer wrote:

 On Wed, 04 Apr 2012 12:33:26 -0400, Michel Fortin
 <michel.fortin michelf.com> wrote:

 Question 2: does std.algorithm.sort refer to the std.algorithm.sort
 *module* or to std.algorithm.package.sort *function* publicly imported
 from the std.algorithm.sort module?


 The function. A symbol that is not specified as the module name in
 import statement or in a module statement is always *not* a module. I
 think our one saving grace here is that when you want to import a
 specific symbol from a module, this is not the syntax:

 import std.stdio.writefln;

 So there is never any ambiguity as to whether you mean a module
 identifier or other symbol.


 Interesting. But isn't there an ambiguity when the symbol is not the
 last one in the chain? Consider:

 a.b.c.x

 Could be static member d of class c in module a.b, or module member d in
 module a.b.c.


 Andrei


The whole point of this thread is to import symbols from submodules into 
a module. Obviously, this tends to create collisions. This one is hard 
to solve and should probably be an error.

However, such a collision will also appear with DIP16, and DIP16 cause 
also other type of collisions. So that one is superior - even if not perfect
Apr 05 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-04-04 18:33, Michel Fortin wrote:


 You might think I'm trying to split hair in four to find flaws, that no
 one is going to do things that dumb, but I unfortunately think the
 problematic pattern is already quite common. How many times have we seen
 modules containing a class, variable, or function having the same name
 as the module's name? What should happen when you publicly import those
 modules in the package.d file?


I do that all the time in my libraries.

-- 
/Jacob Carlborg
Apr 04 2012
prev sibling next sibling parent Ary Manzana <ary esperanto.org.ar> writes:
On 3/30/12 10:46 PM, Andrei Alexandrescu wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


Great. Large modules are my main complaint about D. :-)

If I correctly understand the second part (because I couldn't understand 
the text in the proposal until I read some comments here), then it makes 
sense. Is it like this?

sort(...) -> search sort in every module out there
std.sort(...) -> search sort in every module that's in the std package

If both std.algorithm.sort and std.path.sort exist, or something like 
that, then you would anyway get a clash so you'd have to fully qualify it.

But if std.algorithm.sort and foo.bar.sort and you'd import both:

import std.algorithm.package;
import foo.bar.package;

and you'd wanted to use both, then it could be convenient:

std.sort(...)
foo.sort(...)

Though I wonder if this indeed happens a lot. That's why I would wait 
until there's a real need for it. The main complaint people have is not 
having a way to import all files in a directory, which is the first 
point, but I never heard a complaint about the second point.

Also, I think it would make sense to change the first part to this:

* If the compiler sees a request for importing "foo.bar" and "foo/bar" 
is a directory, then automatically look for the file 
"foo/bar/package.d". *If it doesn't exist, automatically expand the 
import to import all files under that directory.* If both "foo/bar.d" 
and "foo/bar/" exists, compilation halts with an error.

That way you have convenience and safety. Most of the time people just 
put in package.d a list of all the files in that directory. Maybe 
sometimes (not sure) people restrict that list to some modules. And in 
those cases you can just restrict the list in package.d

Please, it's the year 2012. Compilers need to be smarter. Save people 
some typing time. You save them typing all the imports. But then you 
make them typing them in that pacakge.d file. Hmm...
Mar 30 2012
prev sibling next sibling parent reply "Chris NS" <ibisbasenji gmail.com> writes:
I'm pretty impressed with the idea, and look forward to its 
implementation, but I do have one question.  How does this affect 
(if at all) the implicit "friend" relationship of declarations?  
Since package "foo.bar" is treated as a single module, do the 
classes in "foo/bar/alpha.d" and "foo/bar/beta.d" have access to 
each other's private members?

I'm not sure whether I favor them losing or keeping their 
"friend" status.

----------
Chris NS
Mar 30 2012
next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Chris NS" <ibisbasenji gmail.com> wrote in message 
news:ugopmohijjcnnrchuwbe forum.dlang.org...

 I'm pretty impressed with the idea, and look forward to its 
 implementation, but I do have one question.  How does this affect (if at 
 all) the implicit "friend" relationship of declarations?  Since package 
 "foo.bar" is treated as a single module, do the classes in 
 "foo/bar/alpha.d" and "foo/bar/beta.d" have access to each other's private 
 members?

 I'm not sure whether I favor them losing or keeping their "friend" status.


There's always the "package" access specifier.
Mar 30 2012
parent "Chris NS" <ibisbasenji gmail.com> writes:
On Saturday, 31 March 2012 at 06:39:07 UTC, Nick Sabalausky wrote:

 "Chris NS" <ibisbasenji gmail.com> wrote in message
 news:ugopmohijjcnnrchuwbe forum.dlang.org...

 I'm pretty impressed with the idea, and look forward to its 
 implementation, but I do have one question.  How does this 
 affect (if at all) the implicit "friend" relationship of 
 declarations?  Since package "foo.bar" is treated as a single 
 module, do the classes in "foo/bar/alpha.d" and 
 "foo/bar/beta.d" have access to each other's private members?

 I'm not sure whether I favor them losing or keeping their 
 "friend" status.


 There's always the "package" access specifier.


True, though it bears revisiting in its own right.  I've never 
been completely satisfied with the horizontal visibility, and 
would have preferred 'package' be defined as "visible within 
current package, *and subpackages* of the current."  But now I'm 
getting a bit off-topic.
Mar 30 2012
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 31 Mar 2012 00:23:32 -0400, Chris NS <ibisbasenji gmail.com> wrote:


 I'm pretty impressed with the idea, and look forward to its  
 implementation, but I do have one question.  How does this affect (if at  
 all) the implicit "friend" relationship of declarations?  Since package  
 "foo.bar" is treated as a single module, do the classes in  
 "foo/bar/alpha.d" and "foo/bar/beta.d" have access to each other's  
 private members?

 I'm not sure whether I favor them losing or keeping their "friend"  
 status.


They would lose their friend status.  Classes that you need to be  
"friends" would have to go into the same submodule, which makes perfect  
sense to me.

-Steve
Apr 02 2012
prev sibling next sibling parent Derek <ddparnell bigpond.com> writes:
On Sat, 31 Mar 2012 01:46:19 +1100, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment ...


We solved this issue in the Euphoria Programming language by introducing  
the concept of 'public include'.

The 'include' directive is similar to DPL's 'import' directive. Normally  
when a file (module) is included, its public identifiers are *only*  
visible to the including module and not further module up the include  
tree. However, if a module is 'public include'd then its public  
identifiers are set up *as if* they were actually declared in the  
including module rather than the included module.

Thus if Euphoria's module called datetime.e gets too large, we split it  
into, say two or three new modules and replace the code in datetime.e with  
corresponding 'public include' statements, thus making it a type of  
package definition. This means that existing application code does not  
have to be modified and future code can choose to include either the  
entire package called datetime.e or individual modules that go into making  
that package.

 From the POV of a developer, creating a package out of what was a module  
is transparent. They do not even know or have to care in order to use it.  
There are no complicated access rules that can one day trip people up.

We made one tweak to the public identifier declaration though to cater for  
situations in which an identifier which was currently local to a module,  
but when moved to a new module when the original was transformed to a  
package, still had to be visible to the code in the package file but  
invisible to code including the package. We created the 'export' qualifier  
on an identifier declaration to signal such an item.

Original module code ...

   function to_seconds(datetime x)  /* A local function to the module */

   public function local_to_gmt(datetime x) /* A publicly exposed function  
*/

New module code ...

   export function to_seconds(datetime x)  /* A function visible to the  
package file only */

   public function local_to_gmt(datetime x) /* A publicly exposed function  
*/


This means that application that include the package datetime.e will not  
see to_seconds() but code in the datetime.e will see it, and  
local_to_gmt() will be seen by the application code and the package code.


-- 
Derek Parnell
Melbourne, Australia
Mar 31 2012
prev sibling next sibling parent "foobar" <foo bar.com> writes:
On Friday, 30 March 2012 at 14:46:16 UTC, Andrei Alexandrescu 
wrote:

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's 
 approval means that he would approve a pull request 
 implementing DIP16 (subject to regular correctness checks).


 Destroy!

 Andrei


Hooray! I was loudly complaining about this issue for years. I'm 
glad this is finally going to be taken care of.

1. Regarding naming - as others mentioned it's inconvenient for 
sorting. How about changing the file extension instead of the 
name? e.g have a "algorithm.package" instead of "algorithm.d"? 
The compiler could even ignore the name of the file as long as it 
has the proper extension - check for existence of "*.package".
The compiler should enforce that there's at most one such package 
file to prevent ambiguities - regardless of the chosen naming 
scheme.

2. Regarding documentation - I'd encourage (via a D style 
guideline) putting the overview of the package in this file. A 
tool such as Ruby's codnar would nicely compliment the reference 
generated by DDoc. see https://www.ruby-toolbox.com/gems/codnar 
for details.

3. std.algorithm should be deprecated and eventually removed. All 
code *is* essentially algorithms and as such importing this 
module makes as much sense as doing:
import code; // generic much?

4. regarding the look-up rules - I'm unsure about this part of 
the proposal. Others already voiced concerns about issues this 
might bring.
Mar 31 2012
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 30/03/2012 16:46, Andrei Alexandrescu a écrit :

 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means
 that he would approve a pull request implementing DIP16 (subject to
 regular correctness checks).


 Destroy!

 Andrei


OK, here is an alternative proposal, based on what has been said here, 
and in other thread. I think it is a better alternative because it is 
almost already working.

1/ you can defined both a.d (d source code) and a (folder) in the same 
folder.
2/ import a; will always look for a.d . a folder. a existing isn't an error.
3/ import a.b will look for b.d in a folder. If a.d exists, it isn't an 
error.
4/ The package a include what in a.d and everything defined in d files 
in the folder a and its subfolders.
5/ What is publicly imported get automatically aliased in the importing 
module. See example below if it is not clear :

a.d :

public import a.b;

b.d :

void foo() {}

automatic aliasing in a.d would look like :

public import a.b;

// Automaticaly added aliases :
alias b.foo foo;

And usage from third party code :

import a;

void main() {
     foo();       // OK
     a.foo();     // OK
     a.b.foo();   // OK
}

And with static import :

static import a;

void main() {
     foo();       // Error
     a.foo();     // OK
     a.b.foo();   // OK
}
Mar 31 2012
prev sibling next sibling parent reply "Martin Nowak" <dawg dawgfoto.de> writes:
On Fri, 30 Mar 2012 16:46:19 +0200, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means  
 that he would approve a pull request implementing DIP16 (subject to  
 regular correctness checks).


 Destroy!

 Andrei


What about supporting package initalization?
I basically proposed that if a submodule of a package
was imported, a static import of the package is implicitly
added.
http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15
Apr 03 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 03/04/2012 19:44, Martin Nowak a écrit :

 On Fri, 30 Mar 2012 16:46:19 +0200, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval
 means that he would approve a pull request implementing DIP16 (subject
 to regular correctness checks).


 Destroy!

 Andrei


 What about supporting package initalization?
 I basically proposed that if a submodule of a package
 was imported, a static import of the package is implicitly
 added.
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP15


That is pretty much what was proposed by people in this thread.

Basically, it boils down to adding automatically aliases with public 
imports.

I would prefer use pkg.d file instead of pkg/_.d (I though of it a lot 
recently, and this is what make more sense, even if not my initial 
proposal).

With pkg/package.d or pkg/_.d you can ends up with unnecessary 
complexity in choosing the file wich is imported, and create error 
cases. For instance :

What happen if both pkg.d and pkg/_.d exists ? If it is not in the same 
path (think -I compiler option). In one case, this is an issue, in the 
other this isn't.

This file convention solution is superior to the _.d one or package.d 
one. DIP15 is superior to D16 IMO.
Apr 04 2012
parent reply "Martin Nowak" <dawg dawgfoto.de> writes:
 What happen if both pkg.d and pkg/_.d exists ? If it is not in the same  
 path (think -I compiler option). In one case, this is an issue, in the  
 other this isn't.


pkg.d would always be a module, hence result in a module/package conflict.
We'd need to directly search for a subdirectory to decide whether it's a  
package.

Maybe someone with experience of big Python projects has some valuable
insights, but I think one point of adding the file inside a folder is to
prevent from unintended pickup of folders.
Apr 04 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 05/04/2012 02:55, Martin Nowak a écrit :

 What happen if both pkg.d and pkg/_.d exists ? If it is not in the
 same path (think -I compiler option). In one case, this is an issue,
 in the other this isn't.


 pkg.d would always be a module, hence result in a module/package conflict.
 We'd need to directly search for a subdirectory to decide whether it's a
 package.


This is true, but why would it be a conflict ?

Package already is a tree structure, so it shouldn't cause much trouble.


 Maybe someone with experience of big Python projects has some valuable
 insights, but I think one point of adding the file inside a folder is to
 prevent from unintended pickup of folders.


Using eponymous trick, you'd always pick up a file, never a folder. 
Hence, lookup rules get easier. Plus it is D-ish.

I have no doubt that this approach is working in python, and could work 
in D. This is exactly why this was my first proposal. But this doesn't 
means that we can't do better. I have tested quite a lot of way to 
achieve that, in several languages, and this is usually a messy topic, 
with no single solution.
Apr 05 2012
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval means  
 that he would approve a pull request implementing DIP16 (subject to  
 regular correctness checks).


 Destroy!


BTW, this case makes the part of DIP16 which wants to shortcut fully  
qualified names invalid, or at least costly (I posted this code in another  
part of the thread, but I thought I'd bring it up higher).

The following is valid code today:

a/b.d:

module a.b;

void foo() {}
struct b
{
    static void foo() {}
}

main.d:
import a.b;

void main()
{
    a.b.foo();
}

If DIP16 were to be implemented, this becomes ambiguous.  Is a.b.foo() the  
module function foo() from a.b, or is it a shortcut for a.b.b.foo()?

The main issue is, because you can shortcut the FQN, and a chain of  
identifiers can have repeated identifiers in them, ambiguity is possible.

Michel Fortin pointed out at least three cases of this in phobos (although  
I can't see how they could be ambiguous, he has a good point).

-Steve
Apr 05 2012
parent reply Ary Manzana <ary esperanto.org.ar> writes:
On 4/5/12 10:55 PM, Steven Schveighoffer wrote:

 On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval
 means that he would approve a pull request implementing DIP16 (subject
 to regular correctness checks).


 Destroy!


 BTW, this case makes the part of DIP16 which wants to shortcut fully
 qualified names invalid, or at least costly (I posted this code in
 another part of the thread, but I thought I'd bring it up higher).

 The following is valid code today:

 a/b.d:

 module a.b;

 void foo() {}
 struct b
 {
 static void foo() {}
 }

 main.d:
 import a.b;

 void main()
 {
 a.b.foo();
 }

 If DIP16 were to be implemented, this becomes ambiguous. Is a.b.foo()
 the module function foo() from a.b, or is it a shortcut for a.b.b.foo()?


It's a shortcut to the module function foo().

If I replace main.d with:

void main() {
   foo();
}

Is foo() the module function foo() from a.b, or is it a shortcut for 
a.b.b.foo()? Here you have no doubts. What your mind does it: find a top 
level symbol "foo" in all imported modules. If you find more than one, 
error.

You must apply the same logic for "a.b.foo()". First you search "foo" in 
the "a.b" symbol. Here you find it: it's the top level function "foo" in 
"a.b". Then you stop searching.

However, if you can't find it in the module "a.b", you search a top 
level symbol "foo" in all modules that are in package "a.b". That's it. 
You don't search "foo" in every possible nesting: just module nesting.


 The main issue is, because you can shortcut the FQN, and a chain of
 identifiers can have repeated identifiers in them, ambiguity is possible.


As I said before, it's not a shortcut of the FQN: it's just a shortcut 
for the module name.
Apr 05 2012
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 05 Apr 2012 23:18:07 -0400, Ary Manzana <ary esperanto.org.ar>  
wrote:


 On 4/5/12 10:55 PM, Steven Schveighoffer wrote:

 On Fri, 30 Mar 2012 10:46:19 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 Starting a new thread from one in announce:

 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP16

 Please comment, after which Walter will approve. Walter's approval
 means that he would approve a pull request implementing DIP16 (subject
 to regular correctness checks).


 Destroy!


 BTW, this case makes the part of DIP16 which wants to shortcut fully
 qualified names invalid, or at least costly (I posted this code in
 another part of the thread, but I thought I'd bring it up higher).

 The following is valid code today:

 a/b.d:

 module a.b;

 void foo() {}
 struct b
 {
 static void foo() {}
 }

 main.d:
 import a.b;

 void main()
 {
 a.b.foo();
 }

 If DIP16 were to be implemented, this becomes ambiguous. Is a.b.foo()
 the module function foo() from a.b, or is it a shortcut for a.b.b.foo()?


 It's a shortcut to the module function foo().

 If I replace main.d with:

 void main() {
    foo();
 }

 Is foo() the module function foo() from a.b, or is it a shortcut for  
 a.b.b.foo()? Here you have no doubts. What your mind does it: find a top  
 level symbol "foo" in all imported modules. If you find more than one,  
 error.


That's slightly different, because you must *always* qualify struct b's  
foo with a preceeding b.


 You must apply the same logic for "a.b.foo()". First you search "foo" in  
 the "a.b" symbol. Here you find it: it's the top level function "foo" in  
 "a.b". Then you stop searching.

 However, if you can't find it in the module "a.b", you search a top  
 level symbol "foo" in all modules that are in package "a.b". That's it.  
 You don't search "foo" in every possible nesting: just module nesting.


What if a.b is a struct, and it's the only possible match?  We don't  
search for it?  At some point the struct b has to come into play.  Or are  
you saying we cannot shortcut struct FQN's?

I suppose if we prefer to match modules before types, then name lookup for  
fully qualified names only becomes ambiguous with packages allowed to have  
their own modules, so it shouldn't affect existing code.


 The main issue is, because you can shortcut the FQN, and a chain of
 identifiers can have repeated identifiers in them, ambiguity is  
 possible.


 As I said before, it's not a shortcut of the FQN: it's just a shortcut  
 for the module name.


My example *was* a shortcut for the module name.  I did not imply that you  
could shortcut the other parts of the FQN.

What about hijacking though?  For example:

module a.b

struct c
{
    static void foo() {}
}

people now use a.c.foo() to avoid having to type the whole thing

But along comes someone who creates:

module a.c;
void foo() {}

Now, doesn't this usurp a.c.foo() without warning?

-Steve
Apr 06 2012