www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - How should overruling of overloaded functions work?

reply Kristian <kjkilpi gmail.com> writes:
Ok, this issue is controversaly. The following explains the situation fi=
ne:

http://groups.google.com/group/alt.comp.lang.learn.c-c++/msg/2bdda463aed=
5d153?hl=3Den&lr=3Dlang_en&ie=3DUTF-8&oe=3DUTF-8&safe=3Doff

In short, should the following work (without using the alias hack):

class Base {
     void f() {...}
     void f(int) {...}
}

class Derived : Base {
     void f() {...}
}

void func() {
     Derived obj =3D new Derived;

     obj.f(10);  //doesn't work
}

There were/are good reason(s) why it should not work (see the link).
I personally think it should work, though. There are a lot of people tha=
t  =

think likewise. Of course, the other half think that it shouldn't work.


This is why I think it should work:

1)
If you like to change the behavior of the function altogether, then you =
 =

should change all the overload functions. If a new overload is later add=
ed  =

to the Base class, then you have to add the corresponding function to th=
e  =

Derived class. That is a problem, but hiding of overloads of Base doesn'=
t  =

work either. Why? See the point 2.


2)
Using the Derived class via the Base class will access the overloads  =

defined in Base. For example:

class Base {
     void f(int v) {printf("%d\n", v);}
     void f(float v) {printf("%f\n", v);}
}

class Derived : Base {
     void f(int v) {printf("%d\n", -v);}
}

void func(Base obj) {
     obj.f(1);     //prints -1
     obj.f(1.0);   //prints 1.0, not -1.0
}

void main() {
     Derived obj =3D new Derived;

     func(obj);
}

We want that Derived changes the behavior of 'f()' so that it'll print  =

positive values as negative ones and vice versa. If 'f()' is used in  =

'main', the hiding of Base's 'f(float)' will prevent it being called.  =

However, this restriction does not apply to inside 'func()'. 'f()' works=
  =

incorrectly here: it prints a positive value for a float, which is not  =

wanted. The programmer of course wants that 'f()' works consistent  =

_throughout_ the code.

When 'f(float)' is added to Base, the corresponding function must also b=
e  =

added to Derived.


3)
If the overloading does not work for derived classes, then the following=
  =

common case does not work without the alias hack:

class String {...}

class Base {
     void f(String s) {...}
     //these functions are provided for convenience...
     final void f(int v) {
         f(new String(v));
         }
     final void f(float v) {
         f(new String(v));
         }
}

class Derived : Base {
     void f(String s) {...}  //overrule the main function that does all =
the  =

work
}

'f(int)' and 'f(float)' are hidden from Derived. You have to use aliasin=
g,  =

or the following coding style:

class Base {
     final void f(String s) {
         do_f(s);
         }
     final void f(int v) {
         do_f(new String(v));
         }
     final void f(float v) {
         do_f(new String(v));
         }

     void do_f(String s) {...}
}

class Derived : Base {
     void do_f(String s) {...}
}


4)
Aliasing nullifies the 'safety net' provided by the language. (Hence the=
  =

safety net does not work.)


5)
The hiding of overloads in Derived works only partially, which is not a =
 =

good thing, I think. It should work completely or not at all. In additio=
n,  =

cases where the hiding is not desirable are far more common (IMHO).
Aug 19 2006
next sibling parent reply "Søren J. Løvborg" <web kwi.dk> writes:
This is the old problem of modifying the base class, while having the 
subclass continue to work as it should. It manifests itself in many ways.

The only way one can be sure to avoid this problem, is to never extend a 
class that one does not control -- i.e. classes defined in libraries etc.

If this rule is followed, there's indeed no reason for the current (C++ 
like) hiding of overloaded functions.

See for instance, "Why extends is evil" at JavaWorld, discussing the 
"fragile base-class" problem.
http://www.javaworld.com/javaworld/jw-08-2003/jw-0801-toolbox.html

Quote:

 I once attended a Java user group meeting where James Gosling
 (Java's inventor) was the featured speaker. During the memorable
 Q&A session, someone asked him: "If you could do Java over again,
 what would you change?" "I'd leave out classes," he replied.


It's a controversial topic. :-)

Søren J. Løvborg
web kwi.dk
Aug 20 2006
next sibling parent Kristian <kjkilpi gmail.com> writes:
On Sun, 20 Aug 2006 14:54:29 +0300, Søren J. Løvborg <web kwi.dk> wrote:

 This is the old problem of modifying the base class, while having the
 subclass continue to work as it should. It manifests itself in many ways.

 The only way one can be sure to avoid this problem, is to never extend a
 class that one does not control -- i.e. classes defined in libraries etc.

 If this rule is followed, there's indeed no reason for the current (C++
 like) hiding of overloaded functions.

 See for instance, "Why extends is evil" at JavaWorld, discussing the
 "fragile base-class" problem.
 http://www.javaworld.com/javaworld/jw-08-2003/jw-0801-toolbox.html

 Quote:

 I once attended a Java user group meeting where James Gosling
 (Java's inventor) was the featured speaker. During the memorable
 Q&A session, someone asked him: "If you could do Java over again,
 what would you change?" "I'd leave out classes," he replied.


 It's a controversial topic. :-)

 Søren J. Løvborg
 web kwi.dk


Thanks for the link to the article, it was interesting. There were good  
points, although inheritance was designed so that you can re-use the old  
code, of course. Inheritance is a good thing, but it can also be a bad  
thing. I guess there are two sides in every programming style, rules, etc.
Aug 20 2006
prev sibling next sibling parent reply nobody <nobody mailinator.com> writes:
Søren J. Løvborg wrote:

 See for instance, "Why extends is evil" at JavaWorld, discussing the 
 "fragile base-class" problem.
 http://www.javaworld.com/javaworld/jw-08-2003/jw-0801-toolbox.html


That article was horrible. It sounds plausible that safe changes in a base
class 
might have undesired effects on a derived class but I would need an example.
All 
his examples ignored the stubs of the base class. Big surprise ending -- he 
fixes the problem of ignoring the stubs of base classes with an interface.

He says "whenever I violate a central OO principle like *implementation
hiding*, 
I end up rewriting that code." Yet the article is really about subtle
variations 
of failing to hide implementation. So subtle in fact that the author did not 
even realize the problem:

   class ObviousProblem
   {
     public DataStore data;
   }

   class SubtleProblem extends DataStore
   {
     // does not override /any/ DataStore methods
   }

Now we have:

   ObviousProblem o = new ObviousProblem();
   o.data.invalidate();

   SubtleProblem  s =  new SubtleProblem();
   s.invalidate();

In short extending a class just because you are using it as a backing store is
a 
bad idea. A big clue that is-a does not hold is when not a single one of your 
base class' methods make sense for the derived class. Yet despite adding and 
removing random elements makes no sense for a Stack his example is:

   class ArrayList
   {
     public void add( int i, Object o ) { }
     public Object remove( int i ) { }
     public void clear() { }
   }

   class Stack extends ArrayList
   {
     public void push( Object article )
     public Object pop()
     public void push_many( Object[] articles )
   }

Now translating that to a more obvious example an implementation hiding failure:

   class StackEquivalent
   {
     public ArrayList data;
     public void push( Object article )
     public Object pop()
     public void push_many( Object[] articles )
   }

Using /any/ of ArrayList's functions on a Stack object will invalidate it. Not 
just clear(). The /exact/ same situation applies to StackEquivalent.



Now the second implementation hiding failutre is not about exposing more than 
one needs but rather less:

   class Stack
   {
     public void push( Object article )
     public Object pop()
     public void push_many( Object[] articles )
   }

   class Monitorable_stack extends Stack
   {
       private int high_water_mark = 0;
       private int current_size;

       public void push( Object article )
       {   if( ++current_size > high_water_mark )
               high_water_mark = current_size;
           super.push(article);
       }

       public Object pop()
       {   --current_size;
           return super.pop();
       }

       public int maximum_size_so_far()
       {   return high_water_mark;
       }
   }

What jumps out when you just look at the function stubs for base and derived is 
that Monitorable_stack chose not to override push_many. It certainly is one of 
the functions you ought to be interested in intercepting.

The only plausible conclusion is that an inference has been made about Stack's 
hidden implementation. It turns out that was exactly the reasoning behind the 
article's author. Yet again has to violate "a central OO principle like 
implementation hiding" in order to make his point.

Given the pattern that all his examples ignored the stubs of the base class one 
can only conclude that he needs to use interfaces to be able to think clearly 
about what certain classes expose. If it works for him then I am happy he 
figured it out. However his stub blindness has nothing fundamental to do with 
inheritance. In short I suspect he has misinterpreted "The fragile base-class 
problem".
Aug 20 2006
parent reply "Søren J. Løvborg" <web kwi.dk> writes:
 What jumps out when you just look at the function stubs for base and 
 derived is that Monitorable_stack chose not to override push_many. It 
 certainly is one of the functions you ought to be interested in 
 intercepting.

 The only plausible conclusion is that an inference has been made about 
 Stack's hidden implementation. It turns out that was exactly the reasoning 
 behind the article's author. Yet again has to violate "a central OO 
 principle like implementation hiding" in order to make his point.


Imagine that push_many was added to the base class in a later version, after 
Stack was created. Then you have the exact same problem, without making any 
assumptions about the implementation as such. And the same problem is used 
to justify the hiding of overloaded methods in C++ (and hence, D), as 
discussed in the original post.

The difference between extending a base class and implementing an interface 
is that if you add a new method to the class, it can introduce subtle bugs, 
while if you add it to an interface, it's a compile-time error unless you 
also add it to every implementation of that interface.

The reason for extending a class is to override "hook" methods to change the 
functionality of the class, e.g. the paint() method of a GUI control, or the 
add() method of a list. However, when you do this, you no longer only have 
to worry about the "public interface" for the class (all the public methods, 
which users of the class may call), but also about keeping a consistent 
"protected interface".

In the public interface, the implementor simply marks the public methods 
with visibility "public", and he's done.

    /* Call this to add an object onto the stack. */
    public void push(Object o)

The contract for using the public interface is simple: Provide valid 
argument values, and the method will do what it's supposed to.

With the "protected interface", correct visibility is the least of the 
problems. The implementor needs to define exactly which methods may be 
overriden by subclasses (not neccessarily all of the public methods), and 
the exact semantics of each method, for instance:

    /* This is guaranteed to be called exactly once for every object added. 
*/
    public void push(Object o)

The contract for using the "protected interface" isn't simple. When 
subclassing, you need to read the documentation, or you'll introduce hard to 
find bugs. The contract cannot be enforced by the compiler, nor can it be 
enforced at runtime.

So yes, you can produce code that works perfectly, in which you extend 
classes (obviously -- everybody does it).
You can also produce code that works perfectly, in which the member 
variables of all classes are public.
By eliminating or minimizing both, you might save yourself some trouble. Or 
you might not.

I, personally, am not letting go of subclasses just yet...

Søren J. Løvborg
web kwi.dk
Aug 20 2006
parent reply nobody <nobody mailinator.com> writes:
Søren J. Løvborg wrote:

 What jumps out when you just look at the function stubs for base and 
 derived is that Monitorable_stack chose not to override push_many. It 
 certainly is one of the functions you ought to be interested in 
 intercepting.

 The only plausible conclusion is that an inference has been made about 
 Stack's hidden implementation. It turns out that was exactly the reasoning 
 behind the article's author. Yet again has to violate "a central OO 
 principle like implementation hiding" in order to make his point.


 
 Imagine that push_many was added to the base class in a later version, after 
 Stack was created. Then you have the exact same problem, without making any 
 assumptions about the implementation as such. And the same problem is used 
 to justify the hiding of overloaded methods in C++ (and hence, D), as 
 discussed in the original post.
 
 The difference between extending a base class and implementing an interface 
 is that if you add a new method to the class, it can introduce subtle bugs, 
 while if you add it to an interface, it's a compile-time error unless you 
 also add it to every implementation of that interface.
 
 The reason for extending a class is to override "hook" methods to change the 
 functionality of the class, e.g. the paint() method of a GUI control, or the 
 add() method of a list. However, when you do this, you no longer only have 
 to worry about the "public interface" for the class (all the public methods, 
 which users of the class may call), but also about keeping a consistent 
 "protected interface".
 
 In the public interface, the implementor simply marks the public methods 
 with visibility "public", and he's done.
 
     /* Call this to add an object onto the stack. */
     public void push(Object o)
 
 The contract for using the public interface is simple: Provide valid 
 argument values, and the method will do what it's supposed to.
 
 With the "protected interface", correct visibility is the least of the 
 problems. The implementor needs to define exactly which methods may be 
 overriden by subclasses (not neccessarily all of the public methods), and 
 the exact semantics of each method, for instance:
 
     /* This is guaranteed to be called exactly once for every object added. 
 */
     public void push(Object o)
 
 The contract for using the "protected interface" isn't simple. When 
 subclassing, you need to read the documentation, or you'll introduce hard to 
 find bugs. The contract cannot be enforced by the compiler, nor can it be 
 enforced at runtime.
 
 So yes, you can produce code that works perfectly, in which you extend 
 classes (obviously -- everybody does it).
 You can also produce code that works perfectly, in which the member 
 variables of all classes are public.
 By eliminating or minimizing both, you might save yourself some trouble. Or 
 you might not.
 
 I, personally, am not letting go of subclasses just yet...
 
 Søren J. Løvborg
 web kwi.dk 
 
 


Thanks for the eloquent and well thought out reply. I realized I was out of
line 
with my criticism of his second example. From this article a damning quote:


 Bill Venners:

 In Effective Java, Bloch offered a good example in which the addAll method
 in class HashSet calls add on itself. A subclass attempts to track the total
 number of objects added to the set by counting the objects passed to both
 addAll and add, but it doesn't work as expected because of addAll's self-use.
 If someone passes three objects to addAll, the total number of objects
 increments by six, not three, because addAll calls add three times.

 http://www.artima.com/intv/issues2.html


In the previous example he only counts in the single add case but in this 
example counting in both cases is incorrect. The correct behavior seems to be 
only possible either by violating the implementation hiding and peeking or 
somehow getting Stack to also grow a length() function magically!

I would certainly quantify this combination of factors as one nasty example of 
safe changes in a base class having undesired effects on a derived class. 
Certainly as you suggested in your first post adding a length function
magically 
is actually pretty easy if you only subclass your own classes.

I am also left wondering how often this concern actually manifests itself. Is
it 
purely theory with no practice? Would Sun silently slip in something like an 
addAll function to HashSet later? How much has the C library changed over time? 
How relevent is C's past to the future of more recent languages?
Aug 20 2006
parent "Søren J. Løvborg" <web kwi.dk> writes:
 I am also left wondering how often this concern actually manifests itself. 
 Is it purely theory with no practice? Would Sun silently slip in something 
 like an addAll function to HashSet later? How much has the C library 
 changed over time? How relevent is C's past to the future of more recent 
 languages?


Well, as of Java 1.2, with the introduction of the unified Collection 
framework, the addElement() method of Vector was superseded by a new method, 
add().

addElement() stays, however, for backwards compatibility, and isn't even 
deprecated (for reasons only known to Sun...)

Unless add() simply redirects to addElement(), old code that only overrides 
addElement() will be broken.

A quick test (with Sun's Java 1.6 beta) shows that in fact, neither method 
calls the other one. Old code could indeed be broken by this.

So, yes, such library changes happens. Which I guess is sort of an argument 
for retaining the C++ overload hiding rules. (Yet, I still think they're 
unneccessary.)

Søren J. Løvborg
web kwi.dk
Aug 21 2006
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Søren J. Løvborg wrote:

 This is the old problem of modifying the base class, while having the 
 subclass continue to work as it should. It manifests itself in many ways.
 
 The only way one can be sure to avoid this problem, is to never extend a 
 class that one does not control -- i.e. classes defined in libraries etc.
 
 If this rule is followed, there's indeed no reason for the current (C++ 
 like) hiding of overloaded functions.
 
 See for instance, "Why extends is evil" at JavaWorld, discussing the 
 "fragile base-class" problem.
 http://www.javaworld.com/javaworld/jw-08-2003/jw-0801-toolbox.html
 
 Quote:

 I once attended a Java user group meeting where James Gosling
 (Java's inventor) was the featured speaker. During the memorable
 Q&A session, someone asked him: "If you could do Java over again,
 what would you change?" "I'd leave out classes," he replied.


 
 It's a controversial topic. :-)
 
 Søren J. Løvborg
 web kwi.dk 
 
 


The thing about classes and direct inheritance is that there is 
intrinsically a strong coupling between the derived and the parent 
class. And, since strong coupling should be minimized, he argues that 
direct inheritance is "evil" and should not be used, but such a blanket, 
generalizing statement can only be considered idiotic. Sure, there are 
disadvantages in using direct inheritance, so one must me aware of such 
disadvantages, and use "extends" only when appropriate, but that doesn't 
mean never use it at all. (just as many other things that should be used 
with parsimony)

The second problem mentioned, which the author failed to conceptualize 
correctly, is that a derived class should respect the "is-a" behavior in 
order to be safe. (Behavior in other words is the class's contracts, 
whether explicitly specified in the code or not.)
If one doesn't do that (which he didn't in his examples), then 
polymorphic behavior becomes unsafe. But again that's not the fault of 
direct inheritance, it's the fault of not using it appropriately. For 
more info on this "is-a" problem, check this page about such subtyping 
problems: http://okmij.org/ftp/Computation/Subtyping
[Design by Contract in a methodology to take this formalization of 
class's contracts and how they should work with inheritance, and allow 
to specify and verify such contracts in the code, which otherwise are 
often left specified only in documentation, people's mind, or not at all]

But in any case, this does not justify the current behavior of the 
overriding of overloaded functions. The coupling between derived and 
parent classes is *intrinsically* very strong, and *any* kind of 
modification to the base class (or for that matter, in the derived class 
as well) is passible of having many repercussions, some of them possibly 
surprising and undesired. But there is no way around it. Why should the 
language try to "fix" just one particular and specific case of changing 
the base class without having to check the repercussions on derived 
classes, when:
a) such "fix" may not be valid at all: if a base class is changed, 
especially the public or protected API of it, it's likely the derived 
classes should be checked anyway, and trying to prevented that may not 
be correct.
b) the "fix" introduces surprising and/or annoying behavior in other 
situations, namely regular overriding of overloaded functions.

Also (like point 2 Kristian mentioned) the "fix" breaks IMO a basic 
"rule" of polymorphic behavior of classes: for an the object whose 
runtime type is the same, it does different behavior for a virtual 
function call depending on the static type of the object. Like 
"foo.func()" will do different things whether foo's static type is Foo 
or FooBar, despite the instance runtime type being FooBar in any case. 
This IMO should not happen.


unless there is some obscure reason for it, can only be considered 
idiotic: http://blogs.msdn.com/nealho/archive/2006/01/19/515173.aspx )

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Aug 25 2006
prev sibling parent reply Kristian <kjkilpi gmail.com> writes:
On Sat, 19 Aug 2006 18:09:25 +0300, Kristian <kjkilpi gmail.com> wrote:

 3)
 If the overloading does not work for derived classes, then the following  
 common case does not work without the alias hack:

 class String {...}

 class Base {
      void f(String s) {...}
      //these functions are provided for convenience...
      final void f(int v) {
          f(new String(v));
          }
      final void f(float v) {
          f(new String(v));
          }
 }

 class Derived : Base {
      void f(String s) {...}  //overrule the main function that does all  
 the work
 }



Okey, okey, why doesn't someone tell me that this actually works!? :)

The functions 'f(int)' and 'f(float)' are not hidden from Derived because  
they're final. Nice!

This was the main reason I protested against the current overload function  
hiding. But because it do work, I will fell silent (and look a bit  
embarrassed). ;)

This enables me to write these convenience functions without the need of  
rewriting (or alias hacking) them in subclasses. Non-final functions  
should be reimplemented anyway (if someone wants to use alias hacking for  
those, be my guest).
Aug 20 2006
parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
Kristian wrote:

 On Sat, 19 Aug 2006 18:09:25 +0300, Kristian <kjkilpi gmail.com> wrote:

 3)
 If the overloading does not work for derived classes, then the 
 following common case does not work without the alias hack:

 class String {...}

 class Base {
      void f(String s) {...}
      //these functions are provided for convenience...
      final void f(int v) {
          f(new String(v));
          }
      final void f(float v) {
          f(new String(v));
          }
 }

 class Derived : Base {
      void f(String s) {...}  //overrule the main function that does 
 all the work
 }


 
 
 Okey, okey, why doesn't someone tell me that this actually works!? :)
 
 The functions 'f(int)' and 'f(float)' are not hidden from Derived 
 because they're final. Nice!
 
 This was the main reason I protested against the current overload 
 function hiding. But because it do work, I will fell silent (and look a 
 bit embarrassed). ;)
 
 This enables me to write these convenience functions without the need of 
 rewriting (or alias hacking) them in subclasses. Non-final functions 
 should be reimplemented anyway (if someone wants to use alias hacking 
 for those, be my guest).


Are you sure it works? It's not working for me, I tried the following code:

-----
import std.stdio;

class String {}

class Base {
     void f(String s) {
     	writefln("Base.f(String)");
     }
     //these functions are provided for convenience...
     final void f(int v) {
         writefln("Base.f(int)");
     }
     final void f(float v) {
         writefln("Base.f(float)");
     }
}

class Derived : Base {
     void f(String s) { writefln("Derived.f"); }
}

void test() {
     Derived obj = new Derived;
     obj.f(10);  // ERROR, doesn't work
}
Aug 22 2006
parent Kristian <kjkilpi gmail.com> writes:
OOOKAY! You're right, that does not work! :( :( :(

There was a stupid error in my test case... and I was too quick to get o=
ut  =

of my house running and yelling "it works!" to people! (When police  =

appeared, I was forced to stop.)

One word: aaargh!
It was too good to be true anyway... :(

I just don't get it.
This is clearly a case where overload functions should not be hidden! I =
 =

mean, if a function is final, you have to be absolutely sure that there =
is  =

no need to change (i.e. override) it in a subclass.

If you cannot change it in a subclass, then there is no harm in changing=
  =

its implementation in the base class. Subclasses will work as normal aft=
er  =

the change. If not, then the function must not be final!


Thank you Walter, and others, for this great language.
... But *please* don't left these kind of glitches in it! Little details=
  =

are important also, programmers have to deal with them all the time!


On Tue, 22 Aug 2006 17:15:56 +0300, Bruno Medeiros  =

<brunodomedeiros+spam com.gmail> wrote:

 Kristian wrote:

 On Sat, 19 Aug 2006 18:09:25 +0300, Kristian <kjkilpi gmail.com> wrot=




e:

 3)
 If the overloading does not work for derived classes, then the  =







 following common case does not work without the alias hack:

 class String {...}

 class Base {
      void f(String s) {...}
      //these functions are provided for convenience...
      final void f(int v) {
          f(new String(v));
          }
      final void f(float v) {
          f(new String(v));
          }
 }

 class Derived : Base {
      void f(String s) {...}  //overrule the main function that does =






 =


 all the work
 }


   Okey, okey, why doesn't someone tell me that this actually works!? =




:)

  The functions 'f(int)' and 'f(float)' are not hidden from Derived  =





 because they're final. Nice!
  This was the main reason I protested against the current overload  =





 function hiding. But because it do work, I will fell silent (and look=




 a  =


 bit embarrassed). ;)
  This enables me to write these convenience functions without the nee=




d  =


 of rewriting (or alias hacking) them in subclasses. Non-final functio=




ns  =


 should be reimplemented anyway (if someone wants to use alias hacking=




  =


 for those, be my guest).


 Are you sure it works? It's not working for me, I tried the following =


 =


 code:

 -----
 import std.stdio;

 class String {}

 class Base {
      void f(String s) {
      	writefln("Base.f(String)");
      }
      //these functions are provided for convenience...
      final void f(int v) {
          writefln("Base.f(int)");
      }
      final void f(float v) {
          writefln("Base.f(float)");
      }
 }

 class Derived : Base {
      void f(String s) { writefln("Derived.f"); }
 }

 void test() {
      Derived obj =3D new Derived;
      obj.f(10);  // ERROR, doesn't work
 }
Aug 22 2006