www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Need review: explicit package protection

reply "Dicebot" <public dicebot.lv> writes:
Finally got to cleanup and submit this PR:
https://github.com/D-Programming-Language/dmd/pull/3651

While proposed change is very small (and backwards-compatible)
and not worth separate DIP, it is still a language change and
needs community approval.

Copy of description:

========================================

Currently there is no way to use package protection attribute
with deeply nested package hierarchy, forcing to either use flat
one or public protection. This is one of blocking issues for
further usage of package.d in Phobos and one of reasons why
namespace hacks are so popular.

For example, if helpers in std.internal will be marked as
package, only std.internal will be able to access those, but not
rest of std. This PR fixes it by allowing package(<pkgname>)
syntax to explicitly define owning package.

This new syntax will work:

---
module std.internal.mod1;
package(std) void foo() {}
module std.mod2;
---
import std.internal.mod2;
void bar() { foo(); }
----

Exact semantics can are described by added "protection" tests to
test/compilable (last commit in this PR).

Plain package behavior is unchanged and thus no breaking changes
introduced.
Jun 08 2014
next sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 08/06/14 17:37, Dicebot via Digitalmars-d wrote:
While proposed change is very small (and backwards-compatible)

 and not worth separate DIP, it is still a language change and
 needs community approval.


Looks cool to me. :-)


 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.


Is it possible to permit multiple packages access in this way?  For example,

     package(std.math, std.random) void foo() { ... }
Jun 08 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Sunday, 8 June 2014 at 16:27:12 UTC, Joseph Rushton Wakeling 
via Digitalmars-d wrote:

 Is it possible to permit multiple packages access in this way?  
 For example,

     package(std.math, std.random) void foo() { ... }


Of course it is possible but I don't think it is a good project 
structure to encourage. If these modules are so closely related 
they should be moved to single math-n-stuff package.
Jun 08 2014
parent Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 08/06/14 23:47, Dicebot via Digitalmars-d wrote:

 Of course it is possible but I don't think it is a good project structure to
 encourage. If these modules are so closely related they should be moved to
 single math-n-stuff package.


Yea, I was concerned about the opposite, that you'd wind up with lots of 
package(std) access when in fact it's just a couple of modules that actually 
need access to the functionality in question.
Jun 08 2014
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-06-08 17:37, Dicebot wrote:

 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.


Is the idea that anything nested in the specified package has access to 
the symbol?

-- 
/Jacob Carlborg
Jun 08 2014
parent "Dicebot" <public dicebot.lv> writes:
On Sunday, 8 June 2014 at 19:02:58 UTC, Jacob Carlborg wrote:

 Is the idea that anything nested in the specified package has 
 access to the symbol?


Yes, this is how it is implemented right now.
Jun 08 2014
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
08-Jun-2014 19:37, Dicebot пишет:

 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.


[snip]
Even during my first experiments with turning parts of Phobos into 
packages I've been bitten by this. So basically +1 from me.


-- 
Dmitry Olshansky
Jun 08 2014
prev sibling next sibling parent Mike Parker <aldacron gmail.com> writes:
On 6/9/2014 12:37 AM, Dicebot wrote:


 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.


Big +1 from me. I can see myself making good use of this.
Jun 09 2014
prev sibling next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 08 Jun 2014 11:37:04 -0400, Dicebot <public dicebot.lv> wrote:


 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.


Yes, this becomes more crucial with the idea of splitting up a file  
seamlessly with the package.d idiom. A file that already has  
package-accessible functions CANNOT be split up without an improvement  
like this. Given that nothing can utilize unauthorized functions, you can  
only give more access to your own functions, I think this is a worthwhile  
improvement.

-Steve
Jun 09 2014
prev sibling next sibling parent reply "Dejan Lekic" <dejan.lekic gmail.com> writes:
On Sunday, 8 June 2014 at 15:37:06 UTC, Dicebot wrote:

 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.


+1
Definitely worth merging!

However, package module still have few issues.



Few times I asked myself "what am i importing, package or a
module?" when I used package module, so whenever I import a
package, I add a short comment, something like:

// assuming I have
import foo.bar.baz; // package import


Package module is not possible in projects with flat structure
(projects whose authors did not reserve directories for packages.

Example:
Imagine developer has *all* his D sources in
/home/dejan/src/d/myawesomeproject because he does not like big
directory structures. Say his project has two packages foo.bar
and foo.baz . And has following files in his project directory:

tools.d // module foo.bar.tools;
control.d // module foo.bar.control;
screen.d // module foo.baz.screen;
window.d // module foo.baz.window;
package.d // can be only one 'package.d' within a single
directory!
main.d

So, we have to rename the file into something else. Most likely
developer would try with foo_bar_package.d and foo_baz_package.d,
but it is not possible to do module foo.bar; // conflicts with
previous modules
or
module foo.bar.package; // package is a reserved word

So in this case package module is not possible, and developer has
to use classic approach if he/she wants to import all modules
within foo.bar or foo.baz packages.
Jun 09 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 9 June 2014 at 13:51:30 UTC, Dejan Lekic wrote:

 However, package module still have few issues.



 Few times I asked myself "what am i importing, package or a
 module?" when I used package module, so whenever I import a
 package, I add a short comment, something like:

 // assuming I have
 import foo.bar.baz; // package import


Why is this important? `package.d` is intentionally designed so 
that package imports are indistinguishable from module imports. 
That way library author can adjust module structure without 
breaking user code.



 Package module is not possible in projects with flat structure
 (projects whose authors did not reserve directories for 
 packages.

 Example:
 Imagine developer has *all* his D sources in
 /home/dejan/src/d/myawesomeproject because he does not like big
 directory structures. Say his project has two packages foo.bar
 and foo.baz . And has following files in his project directory:

 tools.d // module foo.bar.tools;
 control.d // module foo.bar.control;
 screen.d // module foo.baz.screen;
 window.d // module foo.baz.window;
 package.d // can be only one 'package.d' within a single
 directory!
 main.d

 ...


I'd call this "INVALID WON"T FIX" :) D module system is defined 
to have strong 1-to-1 matching with file system. Any attempt to 
circumvent that in favor of personal preferences is asking for 
trouble and is not worth supporting.
Jun 10 2014
next sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 10 Jun 2014 11:48:17 +0000
Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 On Monday, 9 June 2014 at 13:51:30 UTC, Dejan Lekic wrote:


 Package module is not possible in projects with flat structure
 (projects whose authors did not reserve directories for
 packages.

 Example:
 Imagine developer has *all* his D sources in
 /home/dejan/src/d/myawesomeproject because he does not like big
 directory structures. Say his project has two packages foo.bar
 and foo.baz . And has following files in his project directory:

 tools.d // module foo.bar.tools;
 control.d // module foo.bar.control;
 screen.d // module foo.baz.screen;
 window.d // module foo.baz.window;
 package.d // can be only one 'package.d' within a single
 directory!
 main.d

 ...


 I'd call this "INVALID WON"T FIX" :) D module system is defined
 to have strong 1-to-1 matching with file system. Any attempt to
 circumvent that in favor of personal preferences is asking for
 trouble and is not worth supporting.


+1

- Jonathan M Davis
Jun 10 2014
prev sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
 I'd call this "INVALID WON"T FIX" :) D module system is defined 
 to have strong 1-to-1 matching with file system. Any attempt to 
 circumvent that in favor of personal preferences is asking for 
 trouble and is not worth supporting.


I can only agree with this, but fact is that D allows flattened 
project structure (all modules in all packages in a single 
directory)... It should be noted that people who prefer such 
structure are on their own. I myself prefer to have a directory 
per-package, as it is IMHO more clear and organised.
Jun 11 2014
prev sibling next sibling parent "Boyd" <gaboonviper gmx.net> writes:
On Sunday, 8 June 2014 at 15:37:06 UTC, Dicebot wrote:

 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.


-----------
Perfect! With this, the package protection actually becomes
useful:)

Cheers,
Boyd
Jun 11 2014
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 08 Jun 2014 15:37:04 +0000
Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.


I would be inclined to argue that this should be done in a way that restricts
its use to subpackages of the package that it's being added to, otherwise this
kind of breaks the whole idea of package and becomes something closer to C++'s
friend. So, std.internal.foo could have symbols marked with package(std), but
mystuff.bar.foo couldn't, because it's not a subpackage of std.

- Jonathan M Davis
Jun 11 2014
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 11 Jun 2014 10:09:18 -0400, Jonathan M Davis via Digitalmars-d  
<digitalmars-d puremagic.com> wrote:


 On Sun, 08 Jun 2014 15:37:04 +0000
 Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Finally got to cleanup and submit this PR:
 https://github.com/D-Programming-Language/dmd/pull/3651

 While proposed change is very small (and backwards-compatible)
 and not worth separate DIP, it is still a language change and
 needs community approval.

 Copy of description:

 ========================================

 Currently there is no way to use package protection attribute
 with deeply nested package hierarchy, forcing to either use flat
 one or public protection. This is one of blocking issues for
 further usage of package.d in Phobos and one of reasons why
 namespace hacks are so popular.

 For example, if helpers in std.internal will be marked as
 package, only std.internal will be able to access those, but not
 rest of std. This PR fixes it by allowing package(<pkgname>)
 syntax to explicitly define owning package.

 This new syntax will work:

 ---
 module std.internal.mod1;
 package(std) void foo() {}
 module std.mod2;
 ---
 import std.internal.mod2;
 void bar() { foo(); }
 ----

 Exact semantics can are described by added "protection" tests to
 test/compilable (last commit in this PR).

 Plain package behavior is unchanged and thus no breaking changes
 introduced.


 I would be inclined to argue that this should be done in a way that  
 restricts
 its use to subpackages of the package that it's being added to,  
 otherwise this
 kind of breaks the whole idea of package and becomes something closer to  
 C++'s
 friend. So, std.internal.foo could have symbols marked with  
 package(std), but
 mystuff.bar.foo couldn't, because it's not a subpackage of std.


I think this is a given. But note that you can write code that "lives" in  
the same package via a module statement today.

i.e., inside mystuff/bar/foo.d:

module std.internal.foo;

-Steve
Jun 11 2014
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 11 Jun 2014 11:27:35 -0400
Steven Schveighoffer via Digitalmars-d <digitalmars-d puremagic.com>
wrote:


 On Wed, 11 Jun 2014 10:09:18 -0400, Jonathan M Davis via
 Digitalmars-d <digitalmars-d puremagic.com> wrote:


 I would be inclined to argue that this should be done in a way
 that restricts
 its use to subpackages of the package that it's being added to,
 otherwise this
 kind of breaks the whole idea of package and becomes something
 closer to C++'s
 friend. So, std.internal.foo could have symbols marked with
 package(std), but
 mystuff.bar.foo couldn't, because it's not a subpackage of std.


 I think this is a given. But note that you can write code that
 "lives" in the same package via a module statement today.

 i.e., inside mystuff/bar/foo.d:

 module std.internal.foo;


True, but then it's treated as part of that package rather than in a package
of its own, so all the code that uses it is affected and not just its
internals. It's still unfortunate that that can be done, but I don't think
that we realistically prevent it. However, allowing for _any_ package to be
added to with package(foo) would not be restrictive in the same way and would
likely be more prone to abuse, and we _can_ prevent that easily enough by
restricting its used to subpackages.

- Jonathan M Davis
Jun 11 2014