www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Logical location of template instantiations

reply Lodovico Giaretta <lodovico giaretart.net> writes:
import std.conv, core.memory;

struct S
{
	int x;
	private this(int val)
	{
		x = val;
	}
}

void main()
{
	auto ptr = cast(S*)GC.malloc(S.sizeof);
	auto s = ptr.emplace(3);
}

This code does not work, as  the call `ptr.emplace(3)` creates a 
new concrete implementation of emplace with parameters `S` and 
`int`, which logically belongs to module std.conv, and so has no 
access to the private constructor.

But, logically speaking, as I'm able to construct objects of S, I 
should also be able to emplace them (which is the same thing, 
logically) while inside my module. What I mean is that in this 
situation it would be better if the call `ptr.emplace(3)` created 
a new concrete implementation of emplace inside the module that 
called it, to have the correct access permissions.

This is not the first time I run into this limitation (not only 
with functions, but also with structs), so I wonder: wouldn't it 
be worth a way to get this behaviour?

Thank you for your time.

Lodovico Giaretta
Jun 27 2016
next sibling parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
Ping...

Maybe I'm just saying bullshit, but...
Am I really the only one who faced this need?
Jul 01 2016
next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
IMO, this is one of these places where theory meets practice. Do 
what works, write a comment explaining the problem, and move on 
;-)
Jul 01 2016
parent reply Lodovico Giaretta <lodovico giaretart.net> writes:
On Friday, 1 July 2016 at 11:45:12 UTC, Robert burner Schadek 
wrote:

 IMO, this is one of these places where theory meets practice. 
 Do what works, write a comment explaining the problem, and move 
 on ;-)


Yes, well, I successfully bypassed my issues with this thing, but 
I wanted to share my thoughts about the need to express this kind 
of thing (i.e. to give a template instantiation the privileges of 
the instantiating module), and to know if someone else has some 
opinion on this matter.
Jul 01 2016
parent Enamex <enamex+d outlook.com> writes:
On Friday, 1 July 2016 at 12:08:49 UTC, Lodovico Giaretta wrote:

 On Friday, 1 July 2016 at 11:45:12 UTC, Robert burner Schadek 
 wrote:

 IMO, this is one of these places where theory meets practice. 
 Do what works, write a comment explaining the problem, and 
 move on ;-)


 Yes, well, I successfully bypassed my issues with this thing, 
 but I wanted to share my thoughts about the need to express 
 this kind of thing (i.e. to give a template instantiation the 
 privileges of the instantiating module), and to know if someone 
 else has some opinion on this matter.


Yeah. I don't know if it's _needed_ by enough people or by the 
language, but a way to force templates to be hijacked while being 
instantiated (not at point of definition but at instantiation; 
though I imagine this might prove troublesome for reflection and 
template identity, somehow) if we want to. The default hygienic 
behavior is great, but bypassing it when needed is not.
Jul 01 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 01/07/16 11:57, Lodovico Giaretta wrote:

 Ping...

 Maybe I'm just saying bullshit, but...
 Am I really the only one who faced this need?


You're not the only one, it's been brought up before. A 
solution/workaround is that "emplace" invokes the constructor using a 
function pointer, what will bypass the protection.

-- 
/Jacob Carlborg
Jul 01 2016
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/27/16 12:25 PM, Lodovico Giaretta wrote:

 import std.conv, core.memory;

 struct S
 {
     int x;
     private this(int val)
     {
         x = val;
     }
 }

 void main()
 {
     auto ptr = cast(S*)GC.malloc(S.sizeof);
     auto s = ptr.emplace(3);
 }

 This code does not work, as  the call `ptr.emplace(3)` creates a new
 concrete implementation of emplace with parameters `S` and `int`, which
 logically belongs to module std.conv, and so has no access to the
 private constructor.

 But, logically speaking, as I'm able to construct objects of S, I should
 also be able to emplace them (which is the same thing, logically) while
 inside my module. What I mean is that in this situation it would be
 better if the call `ptr.emplace(3)` created a new concrete
 implementation of emplace inside the module that called it, to have the
 correct access permissions.


I wonder what the plans are for std.allocator on this, as I would think 
it would run into the same issues. Andrei?

-Steve
Jul 01 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/01/2016 09:08 AM, Steven Schveighoffer wrote:

 On 6/27/16 12:25 PM, Lodovico Giaretta wrote:

 import std.conv, core.memory;

 struct S
 {
     int x;
     private this(int val)
     {
         x = val;
     }
 }

 void main()
 {
     auto ptr = cast(S*)GC.malloc(S.sizeof);
     auto s = ptr.emplace(3);
 }

 This code does not work, as  the call `ptr.emplace(3)` creates a new
 concrete implementation of emplace with parameters `S` and `int`, which
 logically belongs to module std.conv, and so has no access to the
 private constructor.

 But, logically speaking, as I'm able to construct objects of S, I should
 also be able to emplace them (which is the same thing, logically) while
 inside my module. What I mean is that in this situation it would be
 better if the call `ptr.emplace(3)` created a new concrete
 implementation of emplace inside the module that called it, to have the
 correct access permissions.


 I wonder what the plans are for std.allocator on this, as I would think
 it would run into the same issues. Andrei?


emplace only works with accessible constructors. I understand sometimes 
it's reasonable to ask for more flexibility, but there are limitations. 
-- Andrei
Jul 01 2016
next sibling parent Jacob Carlborg <doob me.com> writes:
On 01/07/16 15:46, Andrei Alexandrescu wrote:


 emplace only works with accessible constructors. I understand sometimes
 it's reasonable to ask for more flexibility, but there are limitations.


It's possible to bypass the protection using a pointer. An alternative 
would be to not invoke the constructor and let the users to that themselves.

-- 
/Jacob Carlborg
Jul 01 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/1/16 9:46 AM, Andrei Alexandrescu wrote:

 On 07/01/2016 09:08 AM, Steven Schveighoffer wrote:



 I wonder what the plans are for std.allocator on this, as I would think
 it would run into the same issues. Andrei?


 emplace only works with accessible constructors. I understand sometimes
 it's reasonable to ask for more flexibility, but there are limitations.


Right, but this puts allocators at a lower footing than the GC which has 
no problem with private ctors. I would have expected to be able to build 
using allocators and private ctors.

-Steve
Jul 01 2016
next sibling parent Basile B. <b2.temp gmx.com> writes:
On Friday, 1 July 2016 at 14:14:00 UTC, Steven Schveighoffer 
wrote:

 On 7/1/16 9:46 AM, Andrei Alexandrescu wrote:

 On 07/01/2016 09:08 AM, Steven Schveighoffer wrote:



 I wonder what the plans are for std.allocator on this, as I 
 would think
 it would run into the same issues. Andrei?


 emplace only works with accessible constructors. I understand 
 sometimes
 it's reasonable to ask for more flexibility, but there are 
 limitations.


 Right, but this puts allocators at a lower footing than the GC 
 which has no problem with private ctors. I would have expected 
 to be able to build using allocators and private ctors.

 -Steve


+1
Jul 01 2016
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 01/07/16 16:14, Steven Schveighoffer wrote:


 Right, but this puts allocators at a lower footing than the GC which has
 no problem with private ctors. I would have expected to be able to build
 using allocators and private ctors.


It's possible to bypass protection using pointers.

-- 
/Jacob Carlborg
Jul 01 2016
prev sibling next sibling parent Basile B. <b2.temp gmx.com> writes:
On Monday, 27 June 2016 at 16:25:27 UTC, Lodovico Giaretta wrote:

 [...]

 This is not the first time I run into this limitation (not only 
 with functions, but also with structs), so I wonder: wouldn't 
 it be worth a way to get this behaviour?


Yes it would be worth. Several ppl have already hit this wall, 
including me. You can vote for this enhancement, it proposes to 
give a "super visual acuity" to traits such as getMember or 
getOverloads:

https://issues.dlang.org/show_bug.cgi?id=15371

You might also look at `hasUDA` from std.traits because it used 
to be affected by this issue.
Jul 01 2016
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 27.06.2016 18:25, Lodovico Giaretta wrote:

 import std.conv, core.memory;

 struct S
 {
      int x;
      private this(int val)
      {
          x = val;
      }
 }

 void main()
 {
      auto ptr = cast(S*)GC.malloc(S.sizeof);
      auto s = ptr.emplace(3);
 }

 This code does not work, as  the call `ptr.emplace(3)` creates a new
 concrete implementation of emplace with parameters `S` and `int`, which
 logically belongs to module std.conv, and so has no access to the
 private constructor.

 But, logically speaking, as I'm able to construct objects of S, I should
 also be able to emplace them (which is the same thing, logically) while
 inside my module. What I mean is that in this situation it would be
 better if the call `ptr.emplace(3)` created a new concrete
 implementation of emplace inside the module that called it, to have the
 correct access permissions.

 This is not the first time I run into this limitation (not only with
 functions, but also with structs), so I wonder: wouldn't it be worth a
 way to get this behaviour?

 Thank you for your time.

 Lodovico Giaretta


The current module (that declares 'S') might not be the only module that 
uses emplace to construct 'S' instances. We want to hide the constructor 
of 'S' from other modules, but not from the current module.
But both modules get identical template instances, so either both see 
the private constructor (through emplace), or none does. To fix this 
properly, there should hence be a way for the two modules to receive 
distinct template instances.
Jul 01 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/1/16 3:02 PM, Timon Gehr wrote:

 The current module (that declares 'S') might not be the only module that
 uses emplace to construct 'S' instances. We want to hide the constructor
 of 'S' from other modules, but not from the current module.
 But both modules get identical template instances, so either both see
 the private constructor (through emplace), or none does. To fix this
 properly, there should hence be a way for the two modules to receive
 distinct template instances.


Emplace needs a constructor alias parameter.

-Steve
Jul 01 2016
next sibling parent Lodovico Giaretta <lodovico giaretart.net> writes:
On Friday, 1 July 2016 at 19:13:45 UTC, Steven Schveighoffer 
wrote:

 On 7/1/16 3:02 PM, Timon Gehr wrote:

 The current module (that declares 'S') might not be the only 
 module that
 uses emplace to construct 'S' instances. We want to hide the 
 constructor
 of 'S' from other modules, but not from the current module.
 But both modules get identical template instances, so either 
 both see
 the private constructor (through emplace), or none does. To 
 fix this
 properly, there should hence be a way for the two modules to 
 receive
 distinct template instances.


 Emplace needs a constructor alias parameter.

 -Steve


Yes, this looks like a sensible solution.
Jul 01 2016
prev sibling parent reply Tofu Ninja <joeyemmons yahoo.com> writes:
On Friday, 1 July 2016 at 19:13:45 UTC, Steven Schveighoffer 
wrote:

 Emplace needs a constructor alias parameter.

 -Steve


That wouldn't work as emplace wouldn't be able to use the alias 
if it was private...

void main(){
      import other;
      test!foo();
}

private void foo(){}

module other;
void test(alias pred)(){
	pred();
}



other.d(5): Error: function main.foo is not accessible from 
module other
main.d(9): Error: template instance other.test!(foo) error 
instantiating
Jul 07 2016
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 7/7/16 7:49 AM, Tofu Ninja wrote:

 On Friday, 1 July 2016 at 19:13:45 UTC, Steven Schveighoffer wrote:

 Emplace needs a constructor alias parameter.

 -Steve


 That wouldn't work as emplace wouldn't be able to use the alias if it
 was private...

 void main(){
      import other;
      test!foo();
 }

 private void foo(){}

 module other;
 void test(alias pred)(){
     pred();
 }



 other.d(5): Error: function main.foo is not accessible from module other
 main.d(9): Error: template instance other.test!(foo) error instantiating


Yes, it is a problem. I still don't understand how the *calling* of a 
private function is the problem, vs. the aliasing of it. Why aren't we 
preventing the aliasing of the private function in the first place (if 
not allowed)? If you can get an alias, you should be able to call it.

I understand that aliases are strange in this way. What we really need 
is an alias to the protection level. An explicit permission given to an 
external template that says "for this one instantiation, you can pretend 
you have access to this".

A workaround, of course, is to use a delegate.

I came across this when learning vibe.d, which gets around the 
requirement in an interesting way: 
https://github.com/rejectedsoftware/vibe.d/issues/1516

-Steve
Jul 07 2016
next sibling parent Tofu Ninja <joeyemmons yahoo.com> writes:
On Thursday, 7 July 2016 at 12:39:51 UTC, Steven Schveighoffer 
wrote:

 Yes, it is a problem. I still don't understand how the 
 *calling* of a private function is the problem, vs. the 
 aliasing of it. Why aren't we preventing the aliasing of the 
 private function in the first place (if not allowed)? If you 
 can get an alias, you should be able to call it.

 I understand that aliases are strange in this way. What we 
 really need is an alias to the protection level. An explicit 
 permission given to an external template that says "for this 
 one instantiation, you can pretend you have access to this".

 A workaround, of course, is to use a delegate.

 I came across this when learning vibe.d, which gets around the 
 requirement in an interesting way: 
 https://github.com/rejectedsoftware/vibe.d/issues/1516

 -Steve


It makes sense that the aliasing a private member works, when it 
is made at the instantiation site it is perfectly visible. It 
only gets mucked up when the alias travels out of that context 
into the template where it is no longer visible.
Jul 07 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thursday, July 07, 2016 08:39:51 Steven Schveighoffer via Digitalmars-d 
wrote:

 Yes, it is a problem. I still don't understand how the *calling* of a
 private function is the problem, vs. the aliasing of it. Why aren't we
 preventing the aliasing of the private function in the first place (if
 not allowed)? If you can get an alias, you should be able to call it.

 I understand that aliases are strange in this way. What we really need
 is an alias to the protection level. An explicit permission given to an
 external template that says "for this one instantiation, you can pretend
 you have access to this".


What would make sense is if the template instantiation ignored the access
level of the aliased symbol and were the same regardless of the access level
of that symbol such that the template compiled regardless, but the check for
accessibility would be done at the call site. Then the template could
compile with a private function just fine, but it wouldn't be callable
except where the private function would be callable.

I suppose that that  could get a bit hairy though when you consider multiple
levels, since if you had something like

void foo(alias pred, Arg)(Arg arg)
{
    bar(pred)(arg);
}

void bar(alias pred, Arg)(Arg arg)
{
    pred(arg);
}

to work even with foo and bar in different modules so long as foo were
called in a place where the function being passed to it were callable. So,
the buck would have to be passed at every point that the symbol were passed
as an alias argument such that the check for accessibility was only done
when the symbol first became an alias.

Regardless, while that idea makes theoretical sense, I'm not sure that it
makes sense from the standpoint of how alias parameters are currently
implemented. I'm inclined to think though that something along those lines
would be the best solution (at least from a usability perspective, not
necessarily from the compiler's perspective).

- Jonathan M Davis
Jul 07 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07.07.2016 17:11, Jonathan M Davis via Digitalmars-d wrote:

   So,
 the buck would have to be passed at every point that the symbol were passed
 as an alias argument such that the check for accessibility was only done
 when the symbol first became an alias.
 ...


No, it is way simpler than that. Just don't ever check for visibility of 
the symbol accessed through an alias.


 Regardless, while that idea makes theoretical sense, I'm not sure that it
 makes sense from the standpoint of how alias parameters are currently
 implemented.


That ought to be completely irrelevant.
Jul 07 2016
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thursday, July 07, 2016 23:20:28 Timon Gehr via Digitalmars-d wrote:

 On 07.07.2016 17:11, Jonathan M Davis via Digitalmars-d wrote:

   So,

 the buck would have to be passed at every point that the symbol were
 passed
 as an alias argument such that the check for accessibility was only done
 when the symbol first became an alias.
 ...


 No, it is way simpler than that. Just don't ever check for visibility of
 the symbol accessed through an alias.


It has to be checked at some point, otherwise using a template alias
parameter would allow you to completely bypass the accessibility modifiers.
It's just that it seems like it should be checked at the point where the
symbol is passed to the template and not within the template itself, whereas
right now, it's checked within the template.

- Jonathan M Davis
Jul 08 2016
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08.07.2016 09:45, Jonathan M Davis via Digitalmars-d wrote:

 On Thursday, July 07, 2016 23:20:28 Timon Gehr via Digitalmars-d wrote:

 On 07.07.2016 17:11, Jonathan M Davis via Digitalmars-d wrote:

    So,

 the buck would have to be passed at every point that the symbol were
 passed
 as an alias argument such that the check for accessibility was only done
 when the symbol first became an alias.
 ...


 No, it is way simpler than that. Just don't ever check for visibility of
 the symbol accessed through an alias.


 It has to be checked at some point, otherwise using a template alias
 parameter would allow you to completely bypass the accessibility modifiers.
 It's just that it seems like it should be checked at the point where the
 symbol is passed to the template and not within the template itself, whereas
 right now, it's checked within the template.

 - Jonathan M Davis


(That's what I said. In any case, it is simple to implement.)
Jul 08 2016
parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, July 08, 2016 15:08:35 Timon Gehr via Digitalmars-d wrote:

 On 08.07.2016 09:45, Jonathan M Davis via Digitalmars-d wrote:

 On Thursday, July 07, 2016 23:20:28 Timon Gehr via Digitalmars-d wrote:

 On 07.07.2016 17:11, Jonathan M Davis via Digitalmars-d wrote:

    So,

 the buck would have to be passed at every point that the symbol were
 passed
 as an alias argument such that the check for accessibility was only done
 when the symbol first became an alias.
 ...


 No, it is way simpler than that. Just don't ever check for visibility of
 the symbol accessed through an alias.


 It has to be checked at some point, otherwise using a template alias
 parameter would allow you to completely bypass the accessibility
 modifiers.
 It's just that it seems like it should be checked at the point where the
 symbol is passed to the template and not within the template itself,
 whereas right now, it's checked within the template.

 - Jonathan M Davis


 (That's what I said. In any case, it is simple to implement.)


Well, then I misunderstood what you meant. But if it's simple to implement,
then it seems to me that it's almost certainly the way we should go. It
would solve the accessibility problem in a clean way without having to
instantiate the template extra times.

- Jonathan M Davis
Jul 08 2016