www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - traits getProtection

reply "Adam D. Ruppe" <destructionator gmail.com> writes:
I've prepared a dmd pull request to add a new __trait:
getProtection.

It is meant to be used along with getMember() to add to
the reflection capabilities, letting us use the existing
protection qualifiers as strings.

 From the test:
==
class Test {
     public int a;
     private int b;
     export string c;
     protected int d;
     package void e() {}
}

void main() {
     Test t;
     static assert(__traits(getProtection, __traits(getMember, t, 
"a")) == "public");
     static assert(__traits(getProtection, __traits(getMember, t, 
"b")) == "private");
     static assert(__traits(getProtection, __traits(getMember, t, 
"c")) == "export");
     static assert(__traits(getProtection, __traits(getMember, t, 
"d")) == "protected");
     static assert(__traits(getProtection, __traits(getMember, t, 
"e")) == "package");
}
==



This will help D automatically generate things like
external interfaces that use the protections.

For instance, I plan to use it in my web.d to only
make functions marked "export" available via the
web interface. Currently, you have to use a naming
convention to hide functions - a leading underscore -
even on private members. This is ok, but not great.

But with the protection trait, we can mark it with
a much more natural "private", or any of the other
specifiers D has.


I'm sure other uses will come up too.
Apr 01 2012
next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
Code: https://github.com/D-Programming-Language/dmd/pull/856
Apr 01 2012
prev sibling next sibling parent James Miller <james aatch.net> writes:
On 2 April 2012 06:27, Adam D. Ruppe <destructionator gmail.com> wrote:

 I've prepared a dmd pull request to add a new __trait:
 getProtection.

 It is meant to be used along with getMember() to add to
 the reflection capabilities, letting us use the existing
 protection qualifiers as strings.

 From the test:
 =3D=3D
 class Test {
 =C2=A0 =C2=A0public int a;
 =C2=A0 =C2=A0private int b;
 =C2=A0 =C2=A0export string c;
 =C2=A0 =C2=A0protected int d;
 =C2=A0 =C2=A0package void e() {}
 }

 void main() {
 =C2=A0 =C2=A0Test t;
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=


 "a")) =3D=3D

 "public");
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=


 "b")) =3D=3D

 "private");
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=


 "c")) =3D=3D

 "export");
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=


 "d")) =3D=3D

 "protected");
 =C2=A0 =C2=A0static assert(__traits(getProtection, __traits(getMember, t,=


 "e")) =3D=3D

 "package");
 }
 =3D=3D



 This will help D automatically generate things like
 external interfaces that use the protections.

 For instance, I plan to use it in my web.d to only
 make functions marked "export" available via the
 web interface. Currently, you have to use a naming
 convention to hide functions - a leading underscore -
 even on private members. This is ok, but not great.

 But with the protection trait, we can mark it with
 a much more natural "private", or any of the other
 specifiers D has.


 I'm sure other uses will come up too.


Looks good, adds a lot to the compile-time reflection capabilities. My
thoughts for it are similar to yours, generating bindings from
"export"ed members, so you don't need to maintain separate lists or
use a naming convention.

--
James Miller
Apr 01 2012
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 01.04.2012 22:27, Adam D. Ruppe wrote:

 I've prepared a dmd pull request to add a new __trait:
 getProtection.

 It is meant to be used along with getMember() to add to
 the reflection capabilities, letting us use the existing
 protection qualifiers as strings.

  From the test:
 ==
 class Test {
 public int a;
 private int b;
 export string c;
 protected int d;
 package void e() {}
 }

 void main() {
 Test t;
 static assert(__traits(getProtection, __traits(getMember, t, "a")) ==
 "public");
 static assert(__traits(getProtection, __traits(getMember, t, "b")) ==
 "private");
 static assert(__traits(getProtection, __traits(getMember, t, "c")) ==
 "export");
 static assert(__traits(getProtection, __traits(getMember, t, "d")) ==
 "protected");
 static assert(__traits(getProtection, __traits(getMember, t, "e")) ==
 "package");
 }
 ==



 This will help D automatically generate things like
 external interfaces that use the protections.

 For instance, I plan to use it in my web.d to only
 make functions marked "export" available via the
 web interface. Currently, you have to use a naming
 convention to hide functions - a leading underscore -
 even on private members. This is ok, but not great.

 But with the protection trait, we can mark it with
 a much more natural "private", or any of the other
 specifiers D has.


 I'm sure other uses will come up too.


It's all nice and well, but I believe part of the reason of say private 
protection is that user is never ever able to see(!) it. Thus it user 
can't depend on private members being there, which is a good thing.
If I read it right, the technique you present allows user code to depend 
on private functions being there.
I argue that we shouldn't even provide a _possibility_ for external 
stuff to depend on private members.
Same argument in limited scope goes for protected.

As for export, I thinks it looks OK.

-- 
Dmitry Olshansky
Apr 02 2012
next sibling parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Monday, 2 April 2012 at 09:26:07 UTC, Dmitry Olshansky wrote:

 It's all nice and well, but I believe part of the reason of say 
 private protection is that user is never ever able to see(!) 
 it. Thus it user can't depend on private members being there, 
 which is a good thing.


That's not the way it is right now in D: if you use the
reflection traits, you can see and access private members.

Worse yet, the way it is now, there's no way to tell
they are marked private when using reflection!

That's one of the reasons why I want to add this - it
lets us check.


I remember asking if private members should be hidden from
reflection before, and the answer was no - I think it had
to do with them being useful for serialization and stuff
like that.
Apr 02 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 02.04.2012 16:04, Adam D. Ruppe wrote:

 On Monday, 2 April 2012 at 09:26:07 UTC, Dmitry Olshansky wrote:

 It's all nice and well, but I believe part of the reason of say
 private protection is that user is never ever able to see(!) it. Thus
 it user can't depend on private members being there, which is a good
 thing.


 That's not the way it is right now in D: if you use the
 reflection traits, you can see and access private members.

 Worse yet, the way it is now, there's no way to tell
 they are marked private when using reflection!


Awful.


 That's one of the reasons why I want to add this - it
 lets us check.


 I remember asking if private members should be hidden from
 reflection before, and the answer was no - I think it had
 to do with them being useful for serialization and stuff
 like that.


Now I get it, traits hasIndirections would need it too, and it's too big 
a feat to lose it.

-- 
Dmitry Olshansky
Apr 02 2012
parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Mon, 02 Apr 2012 08:05:09 -0500, Dmitry Olshansky <dmitry.olsh gmail.com>
wrote:


 On 02.04.2012 16:04, Adam D. Ruppe wrote:

 On Monday, 2 April 2012 at 09:26:07 UTC, Dmitry Olshansky wrote:

 It's all nice and well, but I believe part of the reason of say
 private protection is that user is never ever able to see(!) it. Thus
 it user can't depend on private members being there, which is a good
 thing.


 That's not the way it is right now in D: if you use the
 reflection traits, you can see and access private members.

 Worse yet, the way it is now, there's no way to tell
 they are marked private when using reflection!


 Awful.


As someone who has implemented a runtime reflection library in D, it is
entirely possible to detect whether a function is marked private/protected or
not using __traits today. In fact, IIRC, there is no way to break protection
using __traits; what the op is referring to is .tupleof, which can be used to
break protections of fields if you have the full source code.
Apr 03 2012
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Tuesday, 3 April 2012 at 13:14:00 UTC, Robert Jacques wrote:

 As someone who has implemented a runtime reflection library in 
 D, it is entirely possible to detect whether a function is 
 marked private/protected or not using __traits today.


How did you do it? __traits(compiles) is the best I could
find, and that breaks down if you are friends (easy to
happen with mixins, even if the code is in a separate file)
and is kinda fragile in general since it will swallow unrelated
errors too.
Apr 03 2012
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 4/3/12, Adam D. Ruppe <destructionator gmail.com> wrote:

 is kinda fragile in general since it will swallow unrelated
 errors too.


That's an issue I've noticed recently. msgpack-d allows me to define a
custom serialization routine in my structs/classes, but it has to be a
template. Then msgpack tries to instantiate it via __traits(compiles),
and if it doesn't work it just skips calling the routine alltogether.
So if I have invalid code in my template I'll never know because the
external library just swallows the errors.

It took me a long time to figure out why my routine wasn't called. I
really wish DMD would semantically check templates at least partially
even if they're never instantiated.
Apr 03 2012
prev sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 03 Apr 2012 08:30:25 -0500, Adam D. Ruppe <destructionator gmail.com>
wrote:


 On Tuesday, 3 April 2012 at 13:14:00 UTC, Robert Jacques wrote:

 As someone who has implemented a runtime reflection library in
 D, it is entirely possible to detect whether a function is
 marked private/protected or not using __traits today.


 How did you do it? __traits(compiles) is the best I could
 find, and that breaks down if you are friends (easy to
 happen with mixins, even if the code is in a separate file)
 and is kinda fragile in general since it will swallow unrelated
 errors too.


I did it by having all the inspection routines in a private template inside a
struct in another module.
Apr 03 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 03:04, Robert Jacques a écrit :

 On Tue, 03 Apr 2012 08:30:25 -0500, Adam D. Ruppe
 <destructionator gmail.com> wrote:


 On Tuesday, 3 April 2012 at 13:14:00 UTC, Robert Jacques wrote:

 As someone who has implemented a runtime reflection library in
 D, it is entirely possible to detect whether a function is
 marked private/protected or not using __traits today.


 How did you do it? __traits(compiles) is the best I could
 find, and that breaks down if you are friends (easy to
 happen with mixins, even if the code is in a separate file)
 and is kinda fragile in general since it will swallow unrelated
 errors too.


 I did it by having all the inspection routines in a private template
 inside a struct in another module.


That is neat.

It should be provided as lib by std.traits . And no need for language 
changes.
Apr 04 2012
parent reply "Adam D. Ruppe" <destructionator gmail.com> writes:
On Wednesday, 4 April 2012 at 07:05:59 UTC, deadalnix wrote:

 It should be provided as lib by std.traits . And no need for 
 language changes.


It wouldn't account for the case I actually want: checking
for export.
Apr 04 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 15:52, Adam D. Ruppe a écrit :

 On Wednesday, 4 April 2012 at 07:05:59 UTC, deadalnix wrote:

 It should be provided as lib by std.traits . And no need for language
 changes.


 It wouldn't account for the case I actually want: checking
 for export.


Good point. You are right, it is needed in reflection.
Apr 06 2012
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Monday, April 02, 2012 13:26:05 Dmitry Olshansky wrote:

 It's all nice and well, but I believe part of the reason of say private
 protection is that user is never ever able to see(!) it. Thus it user
 can't depend on private members being there, which is a good thing.
 If I read it right, the technique you present allows user code to depend
 on private functions being there.
 I argue that we shouldn't even provide a _possibility_ for external
 stuff to depend on private members.
 Same argument in limited scope goes for protected.


As it stands, private has _no_ effect on symbol visibility. All it affects is 
symbol accessibility. For instance, if you create a private alias in a module, 
it affects every module that imports your module, or if you create a function 
which causes an overload conflict, it creates an overload conflict regardless
of 
whether it's private or not. C++ is the same way. Access modifiers are just 
that, _access_ modifieres. They _only_ affect accessibility, not visibility.

Now, there are a number of people very unhappy about this state of affairs and 
want private to hide symbols as well (personally, I think that the fact that 
it makes private aliases effectively useless is reason enough to seriously 
reconsider the current behavior), but I don't know if there's any real chance 
of convincing Walter or not.

- Jonathan M Davis
Apr 02 2012
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 02/04/2012 19:10, Jonathan M Davis a écrit :

 On Monday, April 02, 2012 13:26:05 Dmitry Olshansky wrote:

 It's all nice and well, but I believe part of the reason of say private
 protection is that user is never ever able to see(!) it. Thus it user
 can't depend on private members being there, which is a good thing.
 If I read it right, the technique you present allows user code to depend
 on private functions being there.
 I argue that we shouldn't even provide a _possibility_ for external
 stuff to depend on private members.
 Same argument in limited scope goes for protected.


 As it stands, private has _no_ effect on symbol visibility. All it affects is
 symbol accessibility. For instance, if you create a private alias in a module,
 it affects every module that imports your module, or if you create a function
 which causes an overload conflict, it creates an overload conflict regardless
of
 whether it's private or not. C++ is the same way. Access modifiers are just
 that, _access_ modifieres. They _only_ affect accessibility, not visibility.


Reflection should reflect what the reflectee is. Private stuff are part 
of the reflectee.


 Now, there are a number of people very unhappy about this state of affairs and
 want private to hide symbols as well (personally, I think that the fact that
 it makes private aliases effectively useless is reason enough to seriously
 reconsider the current behavior), but I don't know if there's any real chance
 of convincing Walter or not.


This would be a huge mistake. For instance, private method are sometime 
meant to be overridden in subclasses, which is impossible if symbol is 
inaccessible.

NVI for instance would be impossible in such a situation.
Apr 02 2012
parent reply =?utf-8?Q?Simen_Kj=C3=A6r=C3=A5s?= <simen.kjaras gmail.com> writes:
On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix <deadalnix gmail.com> wrote:


 Now, there are a number of people very unhappy about this state of  
 affairs and
 want private to hide symbols as well (personally, I think that the fact  
 that
 it makes private aliases effectively useless is reason enough to  
 seriously
 reconsider the current behavior), but I don't know if there's any real  
 chance
 of convincing Walter or not.


 This would be a huge mistake. For instance, private method are sometime  
 meant to be overridden in subclasses, which is impossible if symbol is  
 inaccessible.

 NVI for instance would be impossible in such a situation.


NVI is perfectly possible with protected.
Apr 02 2012
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04/02/2012 10:59 PM, Simen Kjærås wrote:

 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix <deadalnix gmail.com> wrote:


 Now, there are a number of people very unhappy about this state of
 affairs and
 want private to hide symbols as well (personally, I think that the
 fact that
 it makes private aliases effectively useless is reason enough to
 seriously
 reconsider the current behavior), but I don't know if there's any
 real chance
 of convincing Walter or not.


 This would be a huge mistake. For instance, private method are
 sometime meant to be overridden in subclasses, which is impossible if
 symbol is inaccessible.

 NVI for instance would be impossible in such a situation.




Making private symbols invisible to other modules or at least excluding 
them from symbol clashes is necessary. The current behaviour is not 
useful or desirable in any way.


 NVI is perfectly possible with protected.


Exactly. Furthermore, private implies final anyway.
Apr 02 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 03/04/2012 00:00, Timon Gehr a écrit :

 Making private symbols invisible to other modules or at least excluding
 them from symbol clashes is necessary. The current behaviour is not
 useful or desirable in any way.


This smells like religious coding to me.


 NVI is perfectly possible with protected.


 Exactly. Furthermore, private implies final anyway.


Which is complete bullshit. Even in a module you may want to override a 
method.
Apr 02 2012
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 04/03/2012 08:27 AM, deadalnix wrote:

 Le 03/04/2012 00:00, Timon Gehr a écrit :

 Making private symbols invisible to other modules or at least excluding
 them from symbol clashes is necessary. The current behaviour is not
 useful or desirable in any way.


 This smells like religious coding to me.


Nope.


 NVI is perfectly possible with protected.


 Exactly. Furthermore, private implies final anyway.


 Which is complete bullshit.


It is a questionable design decision. It might be reasonable to allow 
overriding private members, but that would mess with the vtable layout.


 Even in a module you may want to override a
 method.


... which is excluded from the public interface. There are better ways 
to hide a method than to mark it private in an implementation class.
Apr 03 2012
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 02/04/2012 22:59, Simen Kjærås a écrit :

 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix <deadalnix gmail.com> wrote:


 Now, there are a number of people very unhappy about this state of
 affairs and
 want private to hide symbols as well (personally, I think that the
 fact that
 it makes private aliases effectively useless is reason enough to
 seriously
 reconsider the current behavior), but I don't know if there's any
 real chance
 of convincing Walter or not.


 This would be a huge mistake. For instance, private method are
 sometime meant to be overridden in subclasses, which is impossible if
 symbol is inaccessible.

 NVI for instance would be impossible in such a situation.


 NVI is perfectly possible with protected.


You'll loose the ability to define a function, without being able to 
call it.

It is something you want if function have to be called by pair for 
example (using protected here would introduce temporal coupling). It 
make even more sense if you consider combining it with contract programing.
Apr 02 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, April 03, 2012 08:23:49 deadalnix wrote:

 Le 02/04/2012 22:59, Simen Kj=C3=A6r=C3=A5s a =C3=A9crit :

 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix <deadalnix gmail.com>=




 wrote:

 Now, there are a number of people very unhappy about this state o=








f

 affairs and
 want private to hide symbols as well (personally, I think that th=








e

 fact that
 it makes private aliases effectively useless is reason enough to
 seriously
 reconsider the current behavior), but I don't know if there's any=









 real chance
 of convincing Walter or not.


=20
 This would be a huge mistake. For instance, private method are
 sometime meant to be overridden in subclasses, which is impossible=






 if

 symbol is inaccessible.
=20
 NVI for instance would be impossible in such a situation.


=20
 NVI is perfectly possible with protected.


=20
 You'll loose the ability to define a function, without being able to
 call it.


Except that that doesn't even actually work as discussed in this thread=
:

http://www.digitalmars.com/d/archives/digitalmars/D/Re_Module-
level_accessibility_118209.html#N118329

The derived class overrode the function, so it can call it. It may not =
be able=20
to call the base class version, but it can call its own.

Sure, NVI is great, but it works just as well with protected. private d=
oesn't=20
actually prevent you from calling the function in the derived class, an=
d even=20
if it did, it's _not_ worth the cost of making private virtual by defau=
lt. And=20
as D lets the compiler control virtuality, unlike C++, it doesn't make =
any=20
sense to make it so that the programmer can specifically make a private=
=20
function virtual to work with NVI. So, it just makes more sense to use=20=

protected to do NVI.

Now, it may be different with interfaces. TDPL specifically talks about=
 using=20
private for NVI with _interfaces_, not classes. Doing that sort of thin=
g with=20
interfaces requires special treatment already, and it doesn't affect ef=
ficiency=20
like making private always virtual would, so that should be okay. In th=
e=20
general case though, it's just far better to use protected to do NVI wi=
th=20
classes and let private be non-virtual and therefore efficient by defau=
lt rather=20
than inefficient by default.

- Jonathan M Davis
Apr 03 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 04:48, Jonathan M Davis a écrit :

 On Tuesday, April 03, 2012 08:23:49 deadalnix wrote:

 Le 02/04/2012 22:59, Simen Kjærås a écrit :

 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix<deadalnix gmail.com>  wrote:

 Now, there are a number of people very unhappy about this state of
 affairs and
 want private to hide symbols as well (personally, I think that the
 fact that
 it makes private aliases effectively useless is reason enough to
 seriously
 reconsider the current behavior), but I don't know if there's any
 real chance
 of convincing Walter or not.


 This would be a huge mistake. For instance, private method are
 sometime meant to be overridden in subclasses, which is impossible if
 symbol is inaccessible.

 NVI for instance would be impossible in such a situation.


 NVI is perfectly possible with protected.


 You'll loose the ability to define a function, without being able to
 call it.


 Except that that doesn't even actually work as discussed in this thread:

 http://www.digitalmars.com/d/archives/digitalmars/D/Re_Module-
 level_accessibility_118209.html#N118329

 The derived class overrode the function, so it can call it. It may not be able
 to call the base class version, but it can call its own.


It shouldn't be able to do so. This should be reserved for protected 
methods.


 Sure, NVI is great, but it works just as well with protected. private doesn't
 actually prevent you from calling the function in the derived class, and even
 if it did, it's _not_ worth the cost of making private virtual by default. And
 as D lets the compiler control virtuality, unlike C++, it doesn't make any
 sense to make it so that the programmer can specifically make a private
 function virtual to work with NVI. So, it just makes more sense to use
 protected to do NVI.


Visibility and virtuality are 2 completely orthogonal concerns. Mixing 
both is a bad design decision, what ever is the rational behind it. 
Separation of concerns is more important.

Plus, this isn't a real issue, because the final keyword exists.

At the end, for performance concerns, what we want is that the compiler 
or the linker were able to finalize methods that have no override, not 
some dirty trick that break larger more important conception principles.


 Now, it may be different with interfaces. TDPL specifically talks about using
 private for NVI with _interfaces_, not classes. Doing that sort of thing with
 interfaces requires special treatment already, and it doesn't affect efficiency
 like making private always virtual would, so that should be okay. In the
 general case though, it's just far better to use protected to do NVI with
 classes and let private be non-virtual and therefore efficient by default
rather
 than inefficient by default.

 - Jonathan M Davis


It is not that simple. First, it introduce an inconsistency between 
interfaces and classes for no real reasons. The only difference between 
classes and interfaces should be that interface cannot have member data, 
and you can inherit from multiple interfaces. Interface have been 
created to solve the problems that exists for multiple inheritance, and 
that is enough to solve that problem. Everything else is, again, lack of 
separation of concerns.

Second, the same method can be declared in the base class and in an 
interface, and in this case, we cause compile error for nothing.
Apr 04 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, April 04, 2012 09:04:41 deadalnix wrote:

 Le 04/04/2012 04:48, Jonathan M Davis a =C3=A9crit :

 On Tuesday, April 03, 2012 08:23:49 deadalnix wrote:

 Le 02/04/2012 22:59, Simen Kj=C3=A6r=C3=A5s a =C3=A9crit :

 On Mon, 02 Apr 2012 20:02:20 +0200, deadalnix<deadalnix gmail.com=






 =20


wrote:

 Now, there are a number of people very unhappy about this state=












 of

 affairs and
 want private to hide symbols as well (personally, I think that =












the

 fact that
 it makes private aliases effectively useless is reason enough t=












o

 seriously
 reconsider the current behavior), but I don't know if there's a=












ny

 real chance
 of convincing Walter or not.


=20
 This would be a huge mistake. For instance, private method are
 sometime meant to be overridden in subclasses, which is impossib=










le if

 symbol is inaccessible.
=20
 NVI for instance would be impossible in such a situation.


=20
 NVI is perfectly possible with protected.


=20
 You'll loose the ability to define a function, without being able =






to

 call it.


=20
 Except that that doesn't even actually work as discussed in this th=




read:

=20
 http://www.digitalmars.com/d/archives/digitalmars/D/Re_Module-
 level_accessibility_118209.html#N118329
=20
 The derived class overrode the function, so it can call it. It may =




not be

 able to call the base class version, but it can call its own.


=20
 It shouldn't be able to do so. This should be reserved for protected
 methods.


Well, that's the way that it works in C++. And it makes sense if you th=
ink=20
about it. The derived class' method is private to _it_, not the base cl=
ass, so=20
it can still call it, even if it can't call the base class version.


 Sure, NVI is great, but it works just as well with protected. priva=




te

 doesn't actually prevent you from calling the function in the deriv=




ed

 class, and even if it did, it's _not_ worth the cost of making priv=




ate

 virtual by default. And as D lets the compiler control virtuality, =




unlike

 C++, it doesn't make any sense to make it so that the programmer ca=




n

 specifically make a private function virtual to work with NVI. So, =




it

 just makes more sense to use protected to do NVI.


=20
 Visibility and virtuality are 2 completely orthogonal concerns. Mixin=


g

 both is a bad design decision, what ever is the rational behind it.
 Separation of concerns is more important.


They are and they aren't. There are major issues in allowing the overri=
ding of=20
non-virtual functions. It's far less error-prone to simply make it so t=
hat a=20
class' member functions are always virtual unless the compiler can dete=
rmine=20
that it can make them non-virtual (which can only happen when they're f=
inal and=20
don't override anything). D took the same route as Java in this regard.=
 And=20
because it took that route, the programmer has _zero_ control over the=20=

virtuality of a function, and it's very much tied to the access level o=
f a=20
function. That design decision is _not_ going to be changed, so the vir=
tuality=20
of a function is defined by its access level.


 Plus, this isn't a real issue, because the final keyword exists.


It's a _huge_ issue, because it means that nearly every single private=20=

function in a class in D will need to be marked with final in order for=
 it to=20
be non-virtual and inlinable. The default becomes inefficient, and all =
just for=20
NVI.


 At the end, for performance concerns, what we want is that the compil=


er

 or the linker were able to finalize methods that have no override, no=


t

 some dirty trick that break larger more important conception principl=


es.

Because of how the compilation model works, that's impossible. No class=
 can=20
know all of its derived classes. Not only are classes compiled complete=
ly=20
independently of their derived classes, but derived classes could be li=
nked in=20
dynamically at runtime, and those classes could have been written long =
after=20
the base class was written and compiled. So, unless the programmer expl=
icitly=20
marks a function as final, there's no way that the compiler can know th=
at that=20
function won't be overridden.


 Now, it may be different with interfaces. TDPL specifically talks a=




bout

 using private for NVI with _interfaces_, not classes. Doing that so=




rt of

 thing with interfaces requires special treatment already, and it do=




esn't

 affect efficiency like making private always virtual would, so that=





 should be okay. In the general case though, it's just far better to=




 use

 protected to do NVI with classes and let private be non-virtual and=





 therefore efficient by default rather than inefficient by default.
=20
 - Jonathan M Davis


=20
 It is not that simple. First, it introduce an inconsistency between
 interfaces and classes for no real reasons. The only difference betwe=


en

 classes and interfaces should be that interface cannot have member da=


ta,

 and you can inherit from multiple interfaces. Interface have been
 created to solve the problems that exists for multiple inheritance, a=


nd

 that is enough to solve that problem. Everything else is, again, lack=


 of

 separation of concerns.


private is _not_ going to become virtual in classes. I believe that Wal=
ter has=20
stated that it will remain non-virtual (though I'd have to dig through =
the=20
newsgroup for the exact quote). The only question is how to handle it i=
n=20
interfaces, and as far as I know, the intention is to make interfaces f=
ollow=20
TDPL with regards to private.


 Second, the same method can be declared in the base class and in an
 interface, and in this case, we cause compile error for nothing.


Why would there be a compilation error? The base class cannot call the=20=

interface's private function, so any reference to that function would b=
e the=20
base class' function.

- Jonathan M Davis
Apr 04 2012
parent reply deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 10:41, Jonathan M Davis a écrit :

 Plus, this isn't a real issue, because the final keyword exists.


 It's a _huge_ issue, because it means that nearly every single private
 function in a class in D will need to be marked with final in order for it to
 be non-virtual and inlinable. The default becomes inefficient, and all just for
 NVI.


OK, let me restate this. This isn't a language design issue, this is an 



 At the end, for performance concerns, what we want is that the compiler
 or the linker were able to finalize methods that have no override, not
 some dirty trick that break larger more important conception principles.


 Because of how the compilation model works, that's impossible. No class can
 know all of its derived classes. Not only are classes compiled completely
 independently of their derived classes, but derived classes could be linked in
 dynamically at runtime, and those classes could have been written long after
 the base class was written and compiled. So, unless the programmer explicitly
 marks a function as final, there's no way that the compiler can know that that
 function won't be overridden.


Unless something is marked as export, no code can be linked to it at 
runtime. So the linker could finalize any function that isn't export - a 
vast majority. It doesn't, I admit. But this is definitively a 
implementation issue, and fixing it by language design decision is a 
mistake.


 It is not that simple. First, it introduce an inconsistency between
 interfaces and classes for no real reasons. The only difference between
 classes and interfaces should be that interface cannot have member data,
 and you can inherit from multiple interfaces. Interface have been
 created to solve the problems that exists for multiple inheritance, and
 that is enough to solve that problem. Everything else is, again, lack of
 separation of concerns.


 private is _not_ going to become virtual in classes. I believe that Walter has
 stated that it will remain non-virtual (though I'd have to dig through the
 newsgroup for the exact quote). The only question is how to handle it in
 interfaces, and as far as I know, the intention is to make interfaces follow
 TDPL with regards to private.


I know, but Andrei seems to disagree. I also have to dig, to find it, 
but I guess we can trust each other on that.

In general, I'm not convinced by the authoritative argument. Both Andrei 
and Walter are skilled persons, but they can be wrong, especially when 
they don't agree.

If I restate the question, what is the benefit of having classes and 
interfaces behaving differently on this point ? TDPL make it pretty 
clear how it should be handled in interfaces. Classes should work the 
same way. Fixing an implementation issue isn't a valid reason.


 Second, the same method can be declared in the base class and in an
 interface, and in this case, we cause compile error for nothing.


 Why would there be a compilation error? The base class cannot call the
 interface's private function, so any reference to that function would be the
 base class' function.


You have a class C and an interface I, both define the private method foo.

In MyClass : C, I, if I define foo, I get an error because foo isn't 
virtual in C, and if I don't, I don't implement I.
Apr 04 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, April 04, 2012 15:49:59 deadalnix wrote:

 Le 04/04/2012 10:41, Jonathan M Davis a écrit :

 Plus, this isn't a real issue, because the final keyword exists.


 
 It's a _huge_ issue, because it means that nearly every single private
 function in a class in D will need to be marked with final in order for it
 to be non-virtual and inlinable. The default becomes inefficient, and all
 just for NVI.


 
 OK, let me restate this. This isn't a language design issue, this is an

 

 At the end, for performance concerns, what we want is that the compiler
 or the linker were able to finalize methods that have no override, not
 some dirty trick that break larger more important conception principles.


 
 Because of how the compilation model works, that's impossible. No class
 can
 know all of its derived classes. Not only are classes compiled completely
 independently of their derived classes, but derived classes could be
 linked in dynamically at runtime, and those classes could have been
 written long after the base class was written and compiled. So, unless
 the programmer explicitly marks a function as final, there's no way that
 the compiler can know that that function won't be overridden.


 
 Unless something is marked as export, no code can be linked to it at
 runtime. So the linker could finalize any function that isn't export - a
 vast majority. It doesn't, I admit. But this is definitively a
 implementation issue, and fixing it by language design decision is a
 mistake.


In Posix-land, _everything_ is exported. AFAK, this whole idea of marking 
functions as exported or not is a Windows-only thing. And honestly, it's one 
of my biggest complaints about programming in Windows. It's seriously annoying
to have to deal with constantly exporting stuff on Windows. On Linux, it
_just works_. IMHO, shared libraries is one area where Windows seriously
dropped the ball in comparison to *nix. In my experience, there are _far_
fewer issues with them on Linux than Windows with its export and its nonsense
of having to link against a specific version of a static library to be able
to link against a dynamic one.

Regardless, export is a Windows-only thing and is completely unusable for any
kind of optimizations in the general case.


 It is not that simple. First, it introduce an inconsistency between
 interfaces and classes for no real reasons. The only difference between
 classes and interfaces should be that interface cannot have member data,
 and you can inherit from multiple interfaces. Interface have been
 created to solve the problems that exists for multiple inheritance, and
 that is enough to solve that problem. Everything else is, again, lack of
 separation of concerns.


 
 private is _not_ going to become virtual in classes. I believe that Walter
 has stated that it will remain non-virtual (though I'd have to dig
 through the newsgroup for the exact quote). The only question is how to
 handle it in interfaces, and as far as I know, the intention is to make
 interfaces follow TDPL with regards to private.


 
 I know, but Andrei seems to disagree. I also have to dig, to find it,
 but I guess we can trust each other on that.


Actually, I believe that Andrei agreed with Walter in the same thread that 
Walter agreed not to make private virtual, but I'd have to go dig it up, and 
unfortunately, I always have a horrible time tracking down posts. I think that 
it was within the last few months though.


 If I restate the question, what is the benefit of having classes and
 interfaces behaving differently on this point ? TDPL make it pretty
 clear how it should be handled in interfaces. Classes should work the
 same way. Fixing an implementation issue isn't a valid reason.


Allowing private in interfaces is already getting weird, because of what 
interfaces are for and how they work, and doing so is already arguably special 
casing things. Certainly, the reasons for having private in interfaces is 
_completely_ different from having it in classes. private in interfaces is 
_only_ intended for NVI, whereas NVI is _not_ the reason for having private in 
classes, and in fact is fairly rare thing to do with private. I don't think 
that it's _possible_ to overload private in any language other than C++, and I 
believe that a large number of C++ programmers are completely unaware that 
it's possible in C++. Making private in classes work the same way as it does 
in interfaces would be detrimental to classes just for an idiom which many 
programmers _never_ use (though it _is_ a useful one) - and one which works 
just fine with protected.

I don't believe that it's currently the plan, but I don't see any problem with 
just making it so that protected is used in interfaces for NVI instead of 
private. As I already pointed out, in C++, the derived classes can already 
call their versions of the NVI functions, since they're private to _them_, not 
the base class, even if they can't call the base class implementation (if 
there even is one), but interfaces don't _have_ a base class implementation, 
so there's nothing to protect from being called anyway. So, using protected 
wouldn't really change anything, and it would be completely consistent with 
how protected is used normally.


 Second, the same method can be declared in the base class and in an
 interface, and in this case, we cause compile error for nothing.


 
 Why would there be a compilation error? The base class cannot call the
 interface's private function, so any reference to that function would be
 the base class' function.


 
 You have a class C and an interface I, both define the private method foo.
 
 In MyClass : C, I, if I define foo, I get an error because foo isn't
 virtual in C, and if I don't, I don't implement I.


All it has to do is ignore C's private function in MyClass, because it's non-
virtual and _can't_ be overloaded.

- Jonathan M Davis
Apr 04 2012
parent deadalnix <deadalnix gmail.com> writes:
Le 04/04/2012 19:43, Jonathan M Davis a écrit :

 In Posix-land, _everything_ is exported. AFAK, this whole idea of marking
 functions as exported or not is a Windows-only thing. And honestly, it's one
 of my biggest complaints about programming in Windows. It's seriously annoying
 to have to deal with constantly exporting stuff on Windows. On Linux, it
 _just works_. IMHO, shared libraries is one area where Windows seriously
 dropped the ball in comparison to *nix. In my experience, there are _far_
 fewer issues with them on Linux than Windows with its export and its nonsense
 of having to link against a specific version of a static library to be able
 to link against a dynamic one.


New version of gcc implemented extensions to not export everything in 
posix world. IIRC, clang provide this as well.

Again, this is an implementation problem and shouldn't be fixed with 
language design decisions.


 Regardless, export is a Windows-only thing and is completely unusable for any
 kind of optimizations in the general case.


This is why extensions are made for C/C++ compilers on posix systems.


 Allowing private in interfaces is already getting weird, because of what
 interfaces are for and how they work, and doing so is already arguably special
 casing things. Certainly, the reasons for having private in interfaces is
 _completely_ different from having it in classes. private in interfaces is
 _only_ intended for NVI, whereas NVI is _not_ the reason for having private in
 classes, and in fact is fairly rare thing to do with private. I don't think
 that it's _possible_ to overload private in any language other than C++, and I
 believe that a large number of C++ programmers are completely unaware that
 it's possible in C++. Making private in classes work the same way as it does
 in interfaces would be detrimental to classes just for an idiom which many
 programmers _never_ use (though it _is_ a useful one) - and one which works
 just fine with protected.


I understand your point about this being weird in interfaces. Now, 
granted that it is likely to be like this in interfaces, let's make 
things consistent.


 I don't believe that it's currently the plan, but I don't see any problem with
 just making it so that protected is used in interfaces for NVI instead of
 private. As I already pointed out, in C++, the derived classes can already
 call their versions of the NVI functions, since they're private to _them_, not
 the base class, even if they can't call the base class implementation (if
 there even is one), but interfaces don't _have_ a base class implementation,
 so there's nothing to protect from being called anyway. So, using protected
 wouldn't really change anything, and it would be completely consistent with
 how protected is used normally.


I have to agree. The difference, as stated in TDPL, is that you cannot 
call the private one, only override it.


 Second, the same method can be declared in the base class and in an
 interface, and in this case, we cause compile error for nothing.


 Why would there be a compilation error? The base class cannot call the
 interface's private function, so any reference to that function would be
 the base class' function.


 You have a class C and an interface I, both define the private method foo.

 In MyClass : C, I, if I define foo, I get an error because foo isn't
 virtual in C, and if I don't, I don't implement I.


 All it has to do is ignore C's private function in MyClass, because it's non-
 virtual and _can't_ be overloaded.


This is possible, but this have quite a lot of implication, on 
reflection for instance. What about both being defined in the same 
module, where private stuff are accessible.
Apr 05 2012
prev sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Mon, 02 Apr 2012 19:10:01 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Monday, April 02, 2012 13:26:05 Dmitry Olshansky wrote:

 It's all nice and well, but I believe part of the reason of say private
 protection is that user is never ever able to see(!) it. Thus it user
 can't depend on private members being there, which is a good thing.
 If I read it right, the technique you present allows user code to depend
 on private functions being there.
 I argue that we shouldn't even provide a _possibility_ for external
 stuff to depend on private members.
 Same argument in limited scope goes for protected.


 As it stands, private has _no_ effect on symbol visibility. All it  
 affects is
 symbol accessibility. For instance, if you create a private alias in a  
 module,
 it affects every module that imports your module, or if you create a  
 function
 which causes an overload conflict, it creates an overload conflict  
 regardless of
 whether it's private or not. C++ is the same way. Access modifiers are  
 just
 that, _access_ modifieres. They _only_ affect accessibility, not  
 visibility.


Access modification really makes sense for nested scopes.
After all you can have access to a value of a protected type.
In this case changing the visibility could lead to accidental
mismatches. This behavior is taken from C++ and it makes sense.

Due to the .h/.cpp separation you'll also have most of the
implementation symbols invisible. That's where the C++ model
doesn't really map to D if 'import everything' remains the default.
Apr 04 2012