www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Should templates have the instantiating scope's protection access

reply Tofu Ninja <joeyemmons yahoo.com> writes:
Sorry if this has been posted before.

One thing that constantly bugs me about D is how the protection 
system works when using templates. Currently the a template 
instantiation has the protection rights from the declaration 
scope, but this is not really congruent with how a lot of 
templates are used.

For example in phobos it is common to see something like:
      module bar;
      void foo(alias pred, R)(R range) { /* somehow uses pred on 
the range elements */ }


Its pretty common throughout std.algorithm. However, because of 
the way the protection system works, the pred that is passed in 
MUST be public. For example the following code won't work:
      private void mypred(RangeElementType x){ /* do something 
with x */ }
      ...
      import bar : foo;
      foo!mypred(someRange);

Because foo is declared in another module, mypred MUST be public. 
I am passing mypred explicitly, there is no way I didn't intend 
for foo to have access to it. This causes lots of unnecessary 
public functions.

An even more annoying consequence of this is that emplace can 
never be used with private constructors, this fundamentally puts 
it in a weaker position than new. This is actually a serious 
problem for std.experimental.allocator. You can't use an 
allocator in the same way you use new.

One possible solution could be that templates have the access 
rights of the instantiating scope. This is honestly what I expect 
most of the time but there are probably cases where this would 
muck things up. Maybe a more case by case solution would be 
better.

Thoughts?
Jul 05 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/5/16 1:46 PM, Tofu Ninja wrote:

 Sorry if this has been posted before.


Pretty recently actually: 
https://forum.dlang.org/post/rduvpfuzfzmzhoknsrkr forum.dlang.org


 One possible solution could be that templates have the access rights of
 the instantiating scope. This is honestly what I expect most of the time
 but there are probably cases where this would muck things up. Maybe a
 more case by case solution would be better.


The clear problem with this solution is that this means you must use the 
instantiating module as part of the template definition. A template 
instantiation with exactly the same parameters must behave exactly the 
same, no matter where it was instantiated from.

What this means, is that each instantiation is now tied to the module 
instantiating it, with no code reuse. I think at the very least, this 
should be opt-in if it even makes sense to do.

-Steve
Jul 05 2016
parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Tuesday, 5 July 2016 at 18:04:31 UTC, Steven Schveighoffer 
wrote:

 The clear problem with this solution is that this means you 
 must use the instantiating module as part of the template 
 definition. A template instantiation with exactly the same 
 parameters must behave exactly the same, no matter where it was 
 instantiated from.

 What this means, is that each instantiation is now tied to the 
 module instantiating it, with no code reuse. I think at the 
 very least, this should be opt-in if it even makes sense to do.


Ah that is a really good point, I knew it would muck something 
up. But I suppose in a lot of cases this doesn't really matter, 
if the template is accessing private things then it already 
couldn't be reused outside of the module. I think it would make a 
difference in cases where the template does some kind of 
reflection or conditional compilation based on if it has access 
or not. Still opt-in I think.

Also there could be code re-use wherever the access rights match. 
So if an argument is marked as opt-in, only the instantiation 
scope's access to that argument would need to be tied to the 
template instantiation. I suppose that means there would be 4 
possible instantiations, one for private, package, protected, and 
public access.
Jul 05 2016
parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Tuesday, 5 July 2016 at 18:48:05 UTC, Tofu Ninja wrote:

 Also there could be code re-use wherever the access rights 
 match. So if an argument is marked as opt-in, only the 
 instantiation scope's access to that argument would need to be 
 tied to the template instantiation. I suppose that means there 
 would be 4 possible instantiations, one for private, package, 
 protected, and public access.


Actually scratch that, it would have to match on access rights to 
the argument and any symbol accessible through the argument. It 
would get even crazier on recursive instantiations, so I think 
you are right, the template would have to be tied to the original 
instantiation module. If the template instantiated any other 
templates that had this behavior it would also have to be tied to 
the original module. Sounds complicated.
Jul 05 2016
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Tuesday, 5 July 2016 at 19:00:14 UTC, Tofu Ninja wrote:

 On Tuesday, 5 July 2016 at 18:48:05 UTC, Tofu Ninja wrote:

 Also there could be code re-use wherever the access rights 
 match. So if an argument is marked as opt-in, only the 
 instantiation scope's access to that argument would need to be 
 tied to the template instantiation. I suppose that means there 
 would be 4 possible instantiations, one for private, package, 
 protected, and public access.


 Actually scratch that, it would have to match on access rights 
 to the argument and any symbol accessible through the argument. 
 It would get even crazier on recursive instantiations, so I 
 think you are right, the template would have to be tied to the 
 original instantiation module. If the template instantiated any 
 other templates that had this behavior it would also have to be 
 tied to the original module. Sounds complicated.


Using mixin templates you can get the behavior you want to some 
extent, with exception that you need to type "mixin" in front 
when you're instantiating them.

I wonder if reusing the mixin statement for normal templates 
would be a good idea to reuse existing code while allowing access 
to private members.
Jul 05 2016
parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Wednesday, 6 July 2016 at 06:43:13 UTC, ZombineDev wrote:

 Using mixin templates you can get the behavior you want to some 
 extent, with exception that you need to type "mixin" in front 
 when you're instantiating them.

 I wonder if reusing the mixin statement for normal templates 
 would be a good idea to reuse existing code while allowing 
 access to private members.


I thought about that, but the problem affects things like emplace 
as well and I somehow doubt that we are going to turn emplace 
into a mixin template.

I think there needs to be some kind of opt-in attribute to 
indicate that the template should use the instantiating module's 
access rights. There are three things it would need to do...

1) It would need to cause the template instantiation to have the 
same access rights as the instantiating module.

2) It would need to tie the instantiation to the instantiating 
module, different module will have different access rights so the 
templates won't match.

3) It would need to transmit these access rights through other 
templates if they also have the opt-in attribute. All the 
templates would have the instantiating scopes access rights. It 
would be needed in cases like allocator.make which ends up 
calling emplace eventually. The access rights would need to be 
transmitted all the way to emplace.

The attribute could be argument specific or applied to the whole 
template, I am not sure which would be better but they could both 
work.
Jul 06 2016
parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Wednesday, 6 July 2016 at 07:47:18 UTC, Tofu Ninja wrote:

 I think there needs to be some kind of opt-in attribute to 
 indicate that the template should use the instantiating 
 module's access rights. There are three things it would need to 
 do...

 1) It would need to cause the template instantiation to have 
 the same access rights as the instantiating module.

 2) It would need to tie the instantiation to the instantiating 
 module, different module will have different access rights so 
 the templates won't match.

 3) It would need to transmit these access rights through other 
 templates if they also have the opt-in attribute. All the 
 templates would have the instantiating scopes access rights. It 
 would be needed in cases like allocator.make which ends up 
 calling emplace eventually. The access rights would need to be 
 transmitted all the way to emplace.

 The attribute could be argument specific or applied to the 
 whole template, I am not sure which would be better but they 
 could both work.


Thinking about this, maybe the choice about attributes should be 
made by the user, so that if I want to give emplace access to 
private ctors of my structs, I have to explicitly declare inside 
my module that I'm going to give emplace those rights (which in 
turn are propagated to whatever templates emplace uses 
internally).
This way the template writer doesn't have to bother whether his 
template needs those rights, it is clear which templates have 
them without looking at their declarations, and users can decide 
not to grant these rights (bonus point: this way nothing changes 
behaviour unless the user wants so).
Jul 06 2016
parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Wednesday, 6 July 2016 at 08:05:38 UTC, Lodovico Giaretta 
wrote:

 Thinking about this, maybe the choice about attributes should 
 be made by the user, so that if I want to give emplace access 
 to private ctors of my structs, I have to explicitly declare 
 inside my module that I'm going to give emplace those rights 
 (which in turn are propagated to whatever templates emplace 
 uses internally).
 This way the template writer doesn't have to bother whether his 
 template needs those rights, it is clear which templates have 
 them without looking at their declarations, and users can 
 decide not to grant these rights (bonus point: this way nothing 
 changes behaviour unless the user wants so).


That is an option too. However there would still need to be some 
way for the rights to get carried through multiple levels of 
templates. I suppose retransmission could be done in two ways, 
explicitly or implicitly. I am not sure which way would be better.

If it was explicit, things like allocator.make would need to 
explicitly retransmit the access rights down to emplace. 
Something like :


auto make(T, A, ARGS...)(A alloc, ARGS args) {
    // ...
    emplace!( ShareAccess T)(/* ... */);
    //...
}

// Some other module
class myClass{
     private this(int x){}
}
// ...
auto x = Mallocator.make!( ShareAccess myClass)(5);
// Access rights get transmitted into make and then make 
retransmits them into emplace


If it was implicit the  ShareAccess in make would not be 
necessary, the access rights would carry through to emplace 
implicitly. I am not sure it would be a good option though 
because access might get accidentally granted somewhere it 
shouldn't.

Having it be opt-in by the user also resolves the range function 
example I had earlier where it needed access to mypred. In this 
case I think the implicit retransmission is probably better, as 
it would allow any range function to not care about 
retransmitting the rights if they pass the argument to other 
templates.
Jul 06 2016
parent ZombineDev <petar.p.kirov gmail.com> writes:
On Wednesday, 6 July 2016 at 08:45:25 UTC, Tofu Ninja wrote:

 On Wednesday, 6 July 2016 at 08:05:38 UTC, Lodovico Giaretta 
 wrote:

 Thinking about this, maybe the choice about attributes should 
 be made by the user, so that if I want to give emplace access 
 to private ctors of my structs, I have to explicitly declare 
 inside my module that I'm going to give emplace those rights 
 (which in turn are propagated to whatever templates emplace 
 uses internally).
 This way the template writer doesn't have to bother whether 
 his template needs those rights, it is clear which templates 
 have them without looking at their declarations, and users can 
 decide not to grant these rights (bonus point: this way 
 nothing changes behaviour unless the user wants so).


 That is an option too. However there would still need to be 
 some way for the rights to get carried through multiple levels 
 of templates. I suppose retransmission could be done in two 
 ways, explicitly or implicitly. I am not sure which way would 
 be better.

 If it was explicit, things like allocator.make would need to 
 explicitly retransmit the access rights down to emplace. 
 Something like :


 auto make(T, A, ARGS...)(A alloc, ARGS args) {
    // ...
    emplace!( ShareAccess T)(/* ... */);
    //...
 }

 // Some other module
 class myClass{
     private this(int x){}
 }
 // ...
 auto x = Mallocator.make!( ShareAccess myClass)(5);
 // Access rights get transmitted into make and then make 
 retransmits them into emplace


 If it was implicit the  ShareAccess in make would not be 
 necessary, the access rights would carry through to emplace 
 implicitly. I am not sure it would be a good option though 
 because access might get accidentally granted somewhere it 
 shouldn't.

 Having it be opt-in by the user also resolves the range 
 function example I had earlier where it needed access to 
 mypred. In this case I think the implicit retransmission is 
 probably better, as it would allow any range function to not 
 care about retransmitting the rights if they pass the argument 
 to other templates.


If mixin could work with normal templates, as well as mixin 
templates, the following would work:

auto make(T, A, ARGS...)(A alloc, ARGS args) {
    // ...
    mixin emplace!T(/* ... */);
    //...
}

// Some other module
class myClass{
     private this(int x){}
}
// ...
auto x = mixin Mallocator.make!myClass(5);
Jul 06 2016