digitalmars.D - Proposition for change in D regarding Inheriting overloaded methods
- Steven Schveighoffer (121/121) Aug 07 2007 OK,
- Walter Bright (33/77) Aug 07 2007 The next update of the compiler will throw a runtime exception for this
- Regan Heath (73/128) Aug 07 2007 So, in this case:
- Walter Bright (16/128) Aug 07 2007 There is no runtime check or cost for it. The compiler just inserts a
- Steven Schveighoffer (29/80) Aug 07 2007 Hm... I'm slightly ignorant on this issue, not being a compiler develope...
- Walter Bright (13/29) Aug 07 2007 When author A wants to extend the API of A without silently breaking
- Steven Schveighoffer (36/51) Aug 08 2007 I meant a piece of code that will not cause compiler error. I believe i...
- Christopher Wright (13/17) Aug 09 2007 And if you don't want to provide access to those overloads? If, for
- Steven Schveighoffer (26/43) Aug 09 2007 You can't "hide" it, it can be called by casting to the base class. I s...
- Manfred Nowak (31/50) Aug 08 2007 ]
- Bruno Medeiros (7/17) Aug 09 2007 Huh? I don't understand this, how come it's not possible to detect it at...
- Regan Heath (28/43) Aug 09 2007 The feature(1) would be implemented at the call site. In this example:
- Bruno Medeiros (8/26) Aug 10 2007 Ah ok. Indeed, detecting if there is a call to a missing overload can
- Sean Kelly (10/37) Aug 07 2007 I assume the compiler would not throw an exception for the following cas...
- Walter Bright (2/11) Aug 07 2007 Nope, no exception thrown.
- Tiago Carvalho (4/93) Aug 07 2007 For the first example I think the exception solution will work fine. Sin...
- Manfred Nowak (48/49) Aug 07 2007 If accessing data declared private in another not imported module can
- Walter Bright (3/66) Aug 07 2007 mod does not access hidden, neither does def. I don't understand the
- Manfred Nowak (9/11) Aug 10 2007 Not seeing the issue is the issue: D defies code inspection.
- Steven Schveighoffer (40/96) Aug 07 2007 How is this better than the current implementation? In the current
- Regan Heath (51/65) Aug 07 2007 I like it. In fact it solves the existing problem exhibited by the
OK, So I originally posted the Overloading/Inheritance question as I was confused by the behavior of the current D compiler. After reading all of the responses arguments, I believe there are two main camps in the debate. One camp, which I'll name the C++ camp, believes the C++ behavior is the best approach. This camp includes Walter, and believes the current implementation of the D compiler (which mimics C++) is best. The second camp includes myself, and several other users of D who were either aware of this issue, or unaware and surprised by the current implementation. I'll call this the Java camp, as the behavior I desire is most closely imitating Java (although not exactly). There may be other ways of solving this problem, and I welcome those to voice their opinions. I will try to respond to everyone. Now, I am by no means an expert in anything to do with writing languages, so this proposal may come across as a bit stumbly or informal, but I think it's important that I start a new thread in which to sort of draw attention to the fact that I am no longer asking questions about the current implementation, nor am I submitting a bug. What I believe is that the specification itself should be changed. So I'll make my proposal, and let everyone attempt to shoot holes in it/bolster it, until hopefully there is a decision by those important enough to make the changes on whether a change should be made. For those of you who were confused by the original question, I'll describe the behavior of D as it stands, and why I have issues with it. When a class inherits from another class, and redefines one of the base class' methods, using the same name and parameter types, the derived class overrides that method. However, if the method is overloaded, the base class' methods are not considered when calling that method. This is the case even when the derived class has no suitable overrides for the call in question. For example (augmented example from spec): class A { int foo(int x) { ... } int foo(long y) { ... } int foo(char[] s) { ... } } class B : A { override int foo(long x) { ... } } void test() { B b = new B(); A a = b; b.foo(1); // calls B.foo(long), since A.foo(int) not considered a.foo(1); // calls A.foo(int) because it is not overrided by B b.foo("hello"); // generates a compiler error because A is not considered for overrides a.foo("hello"); // calls A.foo(char[]) } To have the compiler consider the base class' overloads before considering the derived class' overloads, an alias can be added: class B : A { alias A.foo foo; override int foo(long x) { ... } } void test() { B b = new B(); A a = b; b.foo(1); // calls A.foo(int) a.foo(1); // calls A.foo(int) b.foo("hello"); // calls A.foo(char[]) a.foo("hello"); // calls A.foo(char[]) } Thus ends the definition of the issue. Here is where my opinion comes in. There are two issues in this scenario, and both of them have to do with the intentions of the author of class B. I think everyone agrees that the author of B intended to handle the case where foo(long) is called. However, does the author intend to handle the case where foo(int) is called? Let's assume he does (which is what the current compiler assumes). The author has not forbidden the user of the class from calling A.foo, because the user can simply cast to an A object, and call A.foo(int) directly. Therefore, if the author meant to override foo(int) by defining just a foo(long), he has failed. From these points, I believe that the above code is an example of an incorrectly implemented override, and I believe that the correct response the compiler should have is to error out, not while compiling test(), but while compiling B, indicating to the user that he should either declare the alias, or override foo(int). This is the point in which my solution differs from Java. Java would allow this to proceed, and call the base class' foo(int) in all cases, which is not what the author intended. The second issue is how to handle the foo(char[]) case. In the current implementation, because A is not searched for overrides, the compiler produces an error indicating that the user tried to call the foo(long) method, but there is no implicit conversion from char[] to long. There are two possibilities. One is that the author did not notice that A.foo(char[]) existed, and intended to override all instances of foo() with his foo(long) override. However, the author is NOT notified of this issue, only the user of the class is notified of this issue. So the potential for unambiguous code to have escaped exists. The second possibility is that the author fully intended to allow the base class to define foo(char[]), but forgot to define the alias. Again, since the compiler gives no error, he is unaware that he is releasing buggy code to the world. I believe the correct assumption of the compiler should be that the user wanted the alias for the base class' foo(char[]), and should alias it implicitly if and only if no suitable match exists on the derived class. In the case where the author did not notice foo(char[]) existed, he problably doesn't mind that foo(char[]) is defined by the base class. If a suitable match exists that is not a direct override of the base class, then the issue reduces to the previous case, where an implicit conversion is required, and the compiler should error out. There is one other possibile solution that I would be willing to concede to, and that is that the compiler errors out if the base class does not override all overloads of a particular method name. This forces the user to either override all overloads of the method, or define the alias. This would be the safest solution, as the author of B must make his intentions perfectly clear. So my proposal is to change the specification so that: If there is a class A, which is a base class of class B, where A defines a method foo(args), and B defines a method foo(args2), such that the types of args2 cannot be implicitly converted to args, and B does not define foo(args), then the definition of B.foo(args) shall be implicitly aliased to A.foo(args). If, using the same assumptions, args2 can be implicitly converted to args, then the compiler should fail to compile B indicating that the user must define B.foo(args) by override or by alias. I believe this will give us the best of both camps, and allow much less code to be released with silent bugs than the current implementation. Let the hole shooting begin... -Steve
Aug 07 2007
Steven Schveighoffer wrote:However, does the author intend to handle the case where foo(int) is called? Let's assume he does (which is what the current compiler assumes). The author has not forbidden the user of the class from calling A.foo, because the user can simply cast to an A object, and call A.foo(int) directly. Therefore, if the author meant to override foo(int) by defining just a foo(long), he has failed. From these points, I believe that the above code is an example of an incorrectly implemented override, and I believe that the correct response the compiler should have is to error out, not while compiling test(), but while compiling B, indicating to the user that he should either declare the alias, or override foo(int). This is the point in which my solution differs from Java. Java would allow this to proceed, and call the base class' foo(int) in all cases, which is not what the author intended.The next update of the compiler will throw a runtime exception for this case.The second issue is how to handle the foo(char[]) case. In the current implementation, because A is not searched for overrides, the compiler produces an error indicating that the user tried to call the foo(long) method, but there is no implicit conversion from char[] to long. There are two possibilities. One is that the author did not notice that A.foo(char[]) existed, and intended to override all instances of foo() with his foo(long) override. However, the author is NOT notified of this issue, only the user of the class is notified of this issue. So the potential for unambiguous code to have escaped exists.But a compile time error is still generated, so I don't regard this as a big problem. The big problems are silent hijacking of code.The second possibility is that the author fully intended to allow the base class to define foo(char[]), but forgot to define the alias. Again, since the compiler gives no error, he is unaware that he is releasing buggy code to the world. I believe the correct assumption of the compiler should be that the user wanted the alias for the base class' foo(char[]), and should alias it implicitly if and only if no suitable match exists on the derived class. In the case where the author did not notice foo(char[]) existed, he problably doesn't mind that foo(char[]) is defined by the base class.The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error. Here's a simple example: void foo() { int i; ... { int i; ... use i for something; } } To a code auditor, that shadowing declaration of i looks like a mistake, because possibly the "use i for something" code was meant to refer to the outer i, not the inner one. (This can happen when code gets updated by multiple people.) To determine if it was an actual mistake, the code auditor is in for some serious spelunking. This is why, in D, shadowing declarations are illegal. It makes life easier for the auditor, because code that looks like a mistake is not allowed.If a suitable match exists that is not a direct override of the base class, then the issue reduces to the previous case, where an implicit conversion is required, and the compiler should error out. There is one other possibile solution that I would be willing to concede to, and that is that the compiler errors out if the base class does not override all overloads of a particular method name. This forces the user to either override all overloads of the method, or define the alias. This would be the safest solution, as the author of B must make his intentions perfectly clear.I am not comfortable with this method, as it will force the derived class programmer to implement overloads that may not be at all meant to exist in the API he defines for that class. I think he should be in full control of the API for the class, and not forced to provide implementations of functions that may be irrelevant clutter. I prefer the solution where attempts to call unoverridden base class overloads will result in a runtime exception.I believe this will give us the best of both camps, and allow much less code to be released with silent bugs than the current implementation.I believe that the enforcing of the override attribute, and the runtime exception case as described, closes all the known hijacking issues.
Aug 07 2007
Walter Bright wrote:The next update of the compiler will throw a runtime exception for this case.So, in this case: class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } when the call to b.set(3) is made you insert a runtime check which looks for methods called 'set' in <actual type of object>, if none of them take a <insert types of parameters> you throw an exception. Is this done at runtime instead of compile time because the parameters cannot always be determined at compile time?It took me a while (because the example seems to be about something totally different) but I think the argument you're making is that you would prefer an error, requiring the author to specify what they want explicitly, rather than for the compiler to make a potentially incorrect assumption, silently. Is that correct? In the original example (trimmed slightly): class A { int foo(int x) { ... } int foo(long y) { ... } int foo(char[] s) { ... } } class B : A { override int foo(long x) { ... } } void test() { B b = new B(); A a = b; b.foo("hello"); // generates a compiler error a.foo("hello"); // calls A.foo(char[]) } you're already making an assumption, you're assuming the author of B does not want to expose foo(char[]) and it's the fact that this assumption is wrong that has caused this entire debate. As others have mentioned, this assumption destroys the "is-a" relationship of inheritance because "foo(char[])" is a method of A but not a method of B. Meaning B "isn't-a" A any more... unless you've referring to a B with a reference to an A, when suddenly, it is. Crazy idea, could the compiler (when it fails to match this overload) cast the object to it's base class and try again, repeat until you hit Object. I guess this would essentially be a modification of the method lookup rules ;) Making the opposite assumption (implicitly aliasing the "foo(char[])") doesn't introduce any silent bugs (that I am aware of) and restores the "is-a" relationship. If the author really didn't want to expose "foo(char[])" then why were they deriving their class from A? It goes against the whole idea of inheritance, doesn't it? In special cases perhaps this is valid, in which case the author should explicitly define an overload and throw and exception or assert or both. Note that I said "special cases" above, I think the most common case is that the "foo(char[])" should be implicitly aliased into the derived class.The second possibility is that the author fully intended to allow the base class to define foo(char[]), but forgot to define the alias. Again, since the compiler gives no error, he is unaware that he is releasing buggy code to the world. I believe the correct assumption of the compiler should be that the user wanted the alias for the base class' foo(char[]), and should alias it implicitly if and only if no suitable match exists on the derived class. In the case where the author did not notice foo(char[]) existed, he problably doesn't mind that foo(char[]) is defined by the base class.The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error. Here's a simple example: void foo() { int i; ... { int i; ... use i for something; } } To a code auditor, that shadowing declaration of i looks like a mistake, because possibly the "use i for something" code was meant to refer to the outer i, not the inner one. (This can happen when code gets updated by multiple people.) To determine if it was an actual mistake, the code auditor is in for some serious spelunking. This is why, in D, shadowing declarations are illegal. It makes life easier for the auditor, because code that looks like a mistake is not allowed.I agree, but I get the impression this was the least favoured suggestion, probably for the very reason you mention here.If a suitable match exists that is not a direct override of the base class, then the issue reduces to the previous case, where an implicit conversion is required, and the compiler should error out. There is one other possibile solution that I would be willing to concede to, and that is that the compiler errors out if the base class does not override all overloads of a particular method name. This forces the user to either override all overloads of the method, or define the alias. This would be the safest solution, as the author of B must make his intentions perfectly clear.I am not comfortable with this method, as it will force the derived class programmer to implement overloads that may not be at all meant to exist in the API he defines for that class. I think he should be in full control of the API for the class, and not forced to provide implementations of functions that may be irrelevant clutter.You're probably correct, but it doesn't solve the "compiler makes the wrong assumption" problem or the "irritating behaviour" problem ;) ReganI believe this will give us the best of both camps, and allow much less code to be released with silent bugs than the current implementation.I believe that the enforcing of the override attribute, and the runtime exception case as described, closes all the known hijacking issues.
Aug 07 2007
Regan Heath wrote:Walter Bright wrote:There is no runtime check or cost for it. The compiler just inserts a call to a library support routine in D's vtbl[] entry for B.set(int).The next update of the compiler will throw a runtime exception for this case.So, in this case: class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } when the call to b.set(3) is made you insert a runtime check which looks for methods called 'set' in <actual type of object>, if none of them take a <insert types of parameters> you throw an exception.Is this done at runtime instead of compile time because the parameters cannot always be determined at compile time?Yes.Yes.It took me a while (because the example seems to be about something totally different) but I think the argument you're making is that you would prefer an error, requiring the author to specify what they want explicitly, rather than for the compiler to make a potentially incorrect assumption, silently. Is that correct?The second possibility is that the author fully intended to allow the base class to define foo(char[]), but forgot to define the alias. Again, since the compiler gives no error, he is unaware that he is releasing buggy code to the world. I believe the correct assumption of the compiler should be that the user wanted the alias for the base class' foo(char[]), and should alias it implicitly if and only if no suitable match exists on the derived class. In the case where the author did not notice foo(char[]) existed, he problably doesn't mind that foo(char[]) is defined by the base class.The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error. Here's a simple example: void foo() { int i; ... { int i; ... use i for something; } } To a code auditor, that shadowing declaration of i looks like a mistake, because possibly the "use i for something" code was meant to refer to the outer i, not the inner one. (This can happen when code gets updated by multiple people.) To determine if it was an actual mistake, the code auditor is in for some serious spelunking. This is why, in D, shadowing declarations are illegal. It makes life easier for the auditor, because code that looks like a mistake is not allowed.In the original example (trimmed slightly): class A { int foo(int x) { ... } int foo(long y) { ... } int foo(char[] s) { ... } } class B : A { override int foo(long x) { ... } } void test() { B b = new B(); A a = b; b.foo("hello"); // generates a compiler error a.foo("hello"); // calls A.foo(char[]) } you're already making an assumption, you're assuming the author of B does not want to expose foo(char[]) and it's the fact that this assumption is wrong that has caused this entire debate.The language is assuming things on the conservative side, not the expansive side, based on the theory that it is better to generate an error for questionable (and easily correctable) constructs than to make a silent (and erroneous) assumption.As others have mentioned, this assumption destroys the "is-a" relationship of inheritance because "foo(char[])" is a method of A but not a method of B.We should not take rules as absolutes when they don't give us desirable behavior.Meaning B "isn't-a" A any more... unless you've referring to a B with a reference to an A, when suddenly, it is.That will generate a runtime error.Crazy idea, could the compiler (when it fails to match this overload) cast the object to it's base class and try again, repeat until you hit Object. I guess this would essentially be a modification of the method lookup rules ;) Making the opposite assumption (implicitly aliasing the "foo(char[])") doesn't introduce any silent bugs (that I am aware of) and restores the "is-a" relationship. If the author really didn't want to expose "foo(char[])" then why were they deriving their class from A? It goes against the whole idea of inheritance, doesn't it?The problem is when the base class implementor wants to add some functionality (or specialization) with a new overload. A's implementor may be a third party, and has no idea about or control over B. His hands shouldn't be tied.
Aug 07 2007
"Walter Bright" wroteRegan Heath wrote:Hm... I'm slightly ignorant on this issue, not being a compiler developer. After reading this, I'm thinking I need to change my proposition to my alternate solution, which is that the compiler should produce an error whenever the derived class does not override the base class's overloads (my alternate solution). From your answer here, it appears that my assumption that the compiler can tell whether a given type of argument could be converted to a base class' argument type might be impossible to tell at compile time. Or is it? In any case, now that I think about it, it produces an O(n^2) run time as the compiler needs to check every argument type in the derived class to see if it can be implicitly converted to every argument type in the base class. This might produce a very slow compiler. I still believe generating a runtime error is no better than the current behavior of the compiler, actually I think it's worse.when the call to b.set(3) is made you insert a runtime check which looks for methods called 'set' in <actual type of object>, if none of them take a <insert types of parameters> you throw an exception.There is no runtime check or cost for it. The compiler just inserts a call to a library support routine in D's vtbl[] entry for B.set(int).Is this done at runtime instead of compile time because the parameters cannot always be determined at compile time?Yes.The conservative side would be to say that it is an error to generate the class in the first place seeing as how it hasn't defined all the behavior it should. The more I look at it, I think the best solution is not to assume anything, and force the author of the class to define it more clearly.In the original example (trimmed slightly): class A { int foo(int x) { ... } int foo(long y) { ... } int foo(char[] s) { ... } } class B : A { override int foo(long x) { ... } } void test() { B b = new B(); A a = b; b.foo("hello"); // generates a compiler error a.foo("hello"); // calls A.foo(char[]) } you're already making an assumption, you're assuming the author of B does not want to expose foo(char[]) and it's the fact that this assumption is wrong that has caused this entire debate.The language is assuming things on the conservative side, not the expansive side, based on the theory that it is better to generate an error for questionable (and easily correctable) constructs than to make a silent (and erroneous) assumption.I still would like a real example of how this is desirable.As others have mentioned, this assumption destroys the "is-a" relationship of inheritance because "foo(char[])" is a method of A but not a method of B.We should not take rules as absolutes when they don't give us desirable behavior.This argument is absurd. How is the author of A's hands tied? If author A changes his API, author B had better take notice. What if author A decides to change the way he stores protected data? How can you prevent author B from having to understand that? Bottom line, if you derive a class, and the base class changes, all bets are off. You may have to change your class. I see no way to prevent this possibility. Even with your exception solution, A can break code which uses instances of B by adding an overload. -SteveIf the author really didn't want to expose "foo(char[])" then why were they deriving their class from A? It goes against the whole idea of inheritance, doesn't it?The problem is when the base class implementor wants to add some functionality (or specialization) with a new overload. A's implementor may be a third party, and has no idea about or control over B. His hands shouldn't be tied.
Aug 07 2007
Steven Schveighoffer wrote:I still would like a real example of how this is desirable.When author A wants to extend the API of A without silently breaking derived B of which author A has no knowledge of.The problem is, author A adds to (not changes) his API, and B starts silently failing. No compile error, no runtime error.This argument is absurd. How is the author of A's hands tied? If author A changes his API, author B had better take notice.If the author really didn't want to expose "foo(char[])" then why were they deriving their class from A? It goes against the whole idea of inheritance, doesn't it?The problem is when the base class implementor wants to add some functionality (or specialization) with a new overload. A's implementor may be a third party, and has no idea about or control over B. His hands shouldn't be tied.What if author A decides to change the way he stores protected data? How can you prevent author B from having to understand that?Author A should be able to *add* methods and fields to his API without possibly causing silent breakage of derived classes. It is ok to cause them to break with a compile or runtime error, but not ok to silently break them. If author A *changes* his API, that's a different thing entirely.Even with your exception solution, A can break code which uses instances of B by adding an overload.It won't be silent breakage, though. The evil is in the *silent* breakage. The author A shouldn't have his hands tied preventing him from adding to his API out of fear of silently breaking his customers' code.
Aug 07 2007
"Walter Bright" <newshound1 digitalmars.com> wrote in message news:f9bej0$21vn$1 digitalmars.com...Steven Schveighoffer wrote:I meant a piece of code that will not cause compiler error. I believe in the case that A adds an overload that B does not override, the compiler will error when author B tries to compile his code, will it not? Is this not a loud error? I'm assuming the author of B must recompile his code when A is updated. I realize now that we may be arguing the same point. I believe my original solution may not be feasible to imlpement, so I have fallen back on my alternate solution. I'll formally state it here: If there is a class A, which is a base class of class B, where A defines a method foo(args), and B does not define foo(args), then the compiler should fail to compile B indicating that the user must define B.foo(args) by override or by alias.I still would like a real example of how this is desirable.When author A wants to extend the API of A without silently breaking derived B of which author A has no knowledge of.I belive that my new solution would produce a compile error upon compiling B. Even in my original solution, you should see no error, but if the arguments of the new method are not implicitly convertable to any of B's methods, then code which uses B would not accidentally call the new method anyways. If they were implicitly convertable, the compiler would error.This argument is absurd. How is the author of A's hands tied? If author A changes his API, author B had better take notice.The problem is, author A adds to (not changes) his API, and B starts silently failing. No compile error, no runtime error.Your definition of silent error is not the same as mine. If author B does not care about new APIs, he could release his code without testing the new method, and without finding out it throws an exception. This means B can compile without the author knowing that this bomb is sitting in the new method. He then can release his code unknowing that it will fail when someone tries to call the new method on his class. To me, it is a silent error to the author of B, maybe not as "evil" as code hijacking, but still silent.What if author A decides to change the way he stores protected data? How can you prevent author B from having to understand that?Author A should be able to *add* methods and fields to his API without possibly causing silent breakage of derived classes. It is ok to cause them to break with a compile or runtime error, but not ok to silently break them.If author A *changes* his API, that's a different thing entirely.If author A adds any methods to his class, it changes the API. Imagine if A added a method with a new name that B did not have an overload for. This is augmenting the class, but will not force an exception because B does not define an override of that method. What if this method "breaks" the class as you have alluded to (but still have not given an example)? B is now forced to take into account this new method. By your argument A's hands are still tied even with your solution. -Steve
Aug 08 2007
Steven Schveighoffer wrote:If there is a class A, which is a base class of class B, where A defines a method foo(args), and B does not define foo(args), then the compiler should fail to compile B indicating that the user must define B.foo(args) by override or by alias.And if you don't want to provide access to those overloads? If, for instance, the new overload computes something that was previously passed in as an argument, but due to your changes, you cannot compute that argument. With your solution, I'd have to override that method and have it throw an exception. And put it in the docs that the method shouldn't be used. How about having a strong and weak override? A strong override hides all overloads of the function it overrides, while a weak one hides only the overload that matches it. Then it's unambiguous, and only a matter of defining syntax. After all, Walter doesn't implicitly know whether each individual programmer wants to allow inherited overloads to work, and neither does dmd, unless they're explicitly told.
Aug 09 2007
"Christopher Wright" <dhasenan gmail.com> wrote in message news:f9evna$284k$1 digitalmars.com...Steven Schveighoffer wrote:You can't "hide" it, it can be called by casting to the base class. I see no issue with you having to override the method and throw an exception, why is this solution not good enough? It is the default solution that Walter is trying to promote. However, in my opinion, I find it unlikely that an author would want this behavior, so it should not be the default. I would have to see an example to understand it more, but I think I'd probably suggest an alternative to overriding and throwing an exception (such as not deriving but defining a new class instead). Can you give me an actual example? I still have never seen one.If there is a class A, which is a base class of class B, where A defines a method foo(args), and B does not define foo(args), then the compiler should fail to compile B indicating that the user must define B.foo(args) by override or by alias.And if you don't want to provide access to those overloads? If, for instance, the new overload computes something that was previously passed in as an argument, but due to your changes, you cannot compute that argument. With your solution, I'd have to override that method and have it throw an exception. And put it in the docs that the method shouldn't be used.How about having a strong and weak override? A strong override hides all overloads of the function it overrides, while a weak one hides only the overload that matches it. Then it's unambiguous, and only a matter of defining syntax. After all, Walter doesn't implicitly know whether each individual programmer wants to allow inherited overloads to work, and neither does dmd, unless they're explicitly told.As I said, you can't hide the override, but as a compromise, what if there were a way you could alias any undefined overrides to throw exceptions instead of call the base class? Something like: alias not_implemented foo; This saves 1) having to code a silly override for each one you wish to "hide", and 2) having multiple copies of identical code which simply throws an exception compiled into your class. I tried doing this with the current compiler by defining a function: void not_implemented(...) But the problem is that it allows one to compile code which calls overrides that didn't exist in a base class :) I think the compiler would have to handle this case in a special way. You would probably want the exception to say something like "class.foo(arg type) not defined," not sure how one would do this without adding a feature to the compiler. -Steve
Aug 09 2007
Walter Bright wrote in reply to Regan Heath: [ reciting omitted quote!]you're already making an assumption, you're assuming the author of B does not want to expose foo(char[]) and it's the fact that this assumption is wrong that has caused this entire debate.[...]As others have mentioned, this assumption destroys the "is-a" relationship of inheritance because "foo(char[])" is a method of A but not a method of B.We should not take rules as absolutes when they don't give us desirable behavior.and in reply to Steven Schveighoffer:If the author really didn't want to expose "foo(char[])" then why were they deriving their class from A? It goes against the whole idea of inheritance, doesn't it?The problem is when the base class implementor wants to add some functionality (or specialization) with a new overload. A's implementor may be a third party, and has no idea about or control over B. His hands shouldn't be tied.The problem is, author A adds to (not changes) his API, and B starts silently failing.[...]If author A *changes* his API, that's a different thing entirely.Please observe that your arguments are showing kind of ambivalence. You want 1) that inheritance gives authors "desirable behaviour". 2) that authors needn't have "ideas about or control over" derived : classes 3) that authors are allowed to add to their API 4) that changing an API is entirely different, where changing is : anything but adding Now assume, that I am author of class C deriving from class B. According to wish 2) I need not have "ideas about or control over" classes D, that derive from my class C. According to wishes 1) and 3) the author of B as well as me can carelessly "add" to the respective APIs, but according to wish 4) are not allowed to "change" those APIs. But there is no other way than to change the API of B, if there is any kind of conflict between the APIs of A and B. This forces me to change the API of my class C. This forces the author of class D ... resulting in a virtual infinite recurrence. Conclusions: 1) I do not believe that in real scenarios virtual infinite recurrences : are considered as "desirable behaviour" according to wish 1). 2) For escaping from this vicious circle use _composition_. 3) This reestablishes the "is-a" relationship of inheritance. -manfred
Aug 08 2007
Walter Bright wrote:Regan Heath wrote: There is no runtime check or cost for it. The compiler just inserts a call to a library support routine in D's vtbl[] entry for B.set(int).Huh? I don't understand this, how come it's not possible to detect it at compile time? Doesn't the full signature of all methods from both parent and child class have to be know at compile time? -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#DIs this done at runtime instead of compile time because the parameters cannot always be determined at compile time?Yes.
Aug 09 2007
Bruno Medeiros wrote:Walter Bright wrote:The feature(1) would be implemented at the call site. In this example: class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } When compiling b.set(3) the compiler determines the type of '3' and performs it's checking based on that type. IANACW (I am not a compiler writer) so I can't think of a case but Walters reply seems to indicate that there are cases where the type of the parameter cannot be determined at compile time. Regan (1) The feature being the one described by Steven where the compiler searches all base classes of D for any overload of 'set' not implemented in D accepting 'int' or 'int' by implicit conversion and gives an error.Regan Heath wrote: There is no runtime check or cost for it. The compiler just inserts a call to a library support routine in D's vtbl[] entry for B.set(int).Huh? I don't understand this, how come it's not possible to detect it at compile time? Doesn't the full signature of all methods from both parent and child class have to be know at compile time?Is this done at runtime instead of compile time because the parameters cannot always be determined at compile time?Yes.
Aug 09 2007
Regan Heath wrote:Bruno Medeiros wrote:Ah ok. Indeed, detecting if there is a call to a missing overload can only be done at runtime. But detecting if there are missing overloads can be done at compile (in the other thread Walter acknowledged it, but stated he preferred not to issue a compile error right there). -- Bruno Medeiros - MSc in CS/E student http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#DWalter Bright wrote:The feature(1) would be implemented at the call site. In this example:Regan Heath wrote: There is no runtime check or cost for it. The compiler just inserts a call to a library support routine in D's vtbl[] entry for B.set(int).Huh? I don't understand this, how come it's not possible to detect it at compile time? Doesn't the full signature of all methods from both parent and child class have to be know at compile time?Is this done at runtime instead of compile time because the parameters cannot always be determined at compile time?Yes.
Aug 10 2007
Regan Heath wrote:Walter Bright wrote:I assume the compiler would not throw an exception for the following case? class D : B { long square; alias B.set set; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } SeanThe next update of the compiler will throw a runtime exception for this case.So, in this case: class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } when the call to b.set(3) is made you insert a runtime check which looks for methods called 'set' in <actual type of object>, if none of them take a <insert types of parameters> you throw an exception.
Aug 07 2007
Sean Kelly wrote:I assume the compiler would not throw an exception for the following case? class D : B { long square; alias B.set set; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } }Nope, no exception thrown.
Aug 07 2007
Walter Bright Wrote:Steven Schveighoffer wrote:For the first example I think the exception solution will work fine. Since any predefined choice could hurt the work of the programmer. For the second, where the argument can't be implicitily converted, I think that if the object in question doesn't have the required method, that method should be looked in the base classes, until it reaches the Object class. And it should do this without needing to declare an alias. I think this is similar to how java works. And it's also similar to Regan sugestion.However, does the author intend to handle the case where foo(int) is called? Let's assume he does (which is what the current compiler assumes). The author has not forbidden the user of the class from calling A.foo, because the user can simply cast to an A object, and call A.foo(int) directly. Therefore, if the author meant to override foo(int) by defining just a foo(long), he has failed. From these points, I believe that the above code is an example of an incorrectly implemented override, and I believe that the correct response the compiler should have is to error out, not while compiling test(), but while compiling B, indicating to the user that he should either declare the alias, or override foo(int). This is the point in which my solution differs from Java. Java would allow this to proceed, and call the base class' foo(int) in all cases, which is not what the author intended.The next update of the compiler will throw a runtime exception for this case.The second issue is how to handle the foo(char[]) case. In the current implementation, because A is not searched for overrides, the compiler produces an error indicating that the user tried to call the foo(long) method, but there is no implicit conversion from char[] to long. There are two possibilities. One is that the author did not notice that A.foo(char[]) existed, and intended to override all instances of foo() with his foo(long) override. However, the author is NOT notified of this issue, only the user of the class is notified of this issue. So the potential for unambiguous code to have escaped exists.But a compile time error is still generated, so I don't regard this as a big problem. The big problems are silent hijacking of code.The second possibility is that the author fully intended to allow the base class to define foo(char[]), but forgot to define the alias. Again, since the compiler gives no error, he is unaware that he is releasing buggy code to the world. I believe the correct assumption of the compiler should be that the user wanted the alias for the base class' foo(char[]), and should alias it implicitly if and only if no suitable match exists on the derived class. In the case where the author did not notice foo(char[]) existed, he problably doesn't mind that foo(char[]) is defined by the base class.The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error. Here's a simple example: void foo() { int i; ... { int i; ... use i for something; } } To a code auditor, that shadowing declaration of i looks like a mistake, because possibly the "use i for something" code was meant to refer to the outer i, not the inner one. (This can happen when code gets updated by multiple people.) To determine if it was an actual mistake, the code auditor is in for some serious spelunking. This is why, in D, shadowing declarations are illegal. It makes life easier for the auditor, because code that looks like a mistake is not allowed.If a suitable match exists that is not a direct override of the base class, then the issue reduces to the previous case, where an implicit conversion is required, and the compiler should error out. There is one other possibile solution that I would be willing to concede to, and that is that the compiler errors out if the base class does not override all overloads of a particular method name. This forces the user to either override all overloads of the method, or define the alias. This would be the safest solution, as the author of B must make his intentions perfectly clear.I am not comfortable with this method, as it will force the derived class programmer to implement overloads that may not be at all meant to exist in the API he defines for that class. I think he should be in full control of the API for the class, and not forced to provide implementations of functions that may be irrelevant clutter. I prefer the solution where attempts to call unoverridden base class overloads will result in a runtime exception.I believe this will give us the best of both camps, and allow much less code to be released with silent bugs than the current implementation.I believe that the enforcing of the override attribute, and the runtime exception case as described, closes all the known hijacking issues.
Aug 07 2007
Walter Bright wrotecloses all the known hijacking issues.If accessing data declared private in another not imported module can be called hijacking there is at least one more issue: Defining a Base class: module def; class Base{int i;} and defining a Derived class with some operations and some private data `hidden': module def2; import std.stdio; import def; class Derived:Base{ private: int hidden=41; // foreign access possible public: void process( inout Base p){ scope d= new Derived; d.hidden= 666; d.i+=1; p= d; // no upcast needed } void read( Base p){ scope d= cast(Derived)p; //downcast needed assert( d !is null); writefln( "def2:", d.hidden); } } and having a module that does some work: module mod; private import std.stdio, def, def2; public: void process( inout Base p){ scope d= new Derived; d.process= p; // stores Base, drops Derived // do something with d? } void read( Base p){ scope d= new Derived; d.read= p; // drops Derived // do something with d? } the following claim is currently (dmd.1.016,win32) true:: ! One can access the `private' data `hidden' defined in `module def2' ! by having access only to sources of `module def' and `module mod'. This are about 25 LOC and if D is indeed designed to support auditing well, it should be easy to spot that leak. -manfred
Aug 07 2007
Manfred Nowak wrote:Walter Bright wrotemod does not access hidden, neither does def. I don't understand the issue here.closes all the known hijacking issues.If accessing data declared private in another not imported module can be called hijacking there is at least one more issue: Defining a Base class: module def; class Base{int i;} and defining a Derived class with some operations and some private data `hidden': module def2; import std.stdio; import def; class Derived:Base{ private: int hidden=41; // foreign access possible public: void process( inout Base p){ scope d= new Derived; d.hidden= 666; d.i+=1; p= d; // no upcast needed } void read( Base p){ scope d= cast(Derived)p; //downcast needed assert( d !is null); writefln( "def2:", d.hidden); } } and having a module that does some work: module mod; private import std.stdio, def, def2; public: void process( inout Base p){ scope d= new Derived; d.process= p; // stores Base, drops Derived // do something with d? } void read( Base p){ scope d= new Derived; d.read= p; // drops Derived // do something with d? } the following claim is currently (dmd.1.016,win32) true:: ! One can access the `private' data `hidden' defined in `module def2' ! by having access only to sources of `module def' and `module mod'. This are about 25 LOC and if D is indeed designed to support auditing well, it should be easy to spot that leak.
Aug 07 2007
Walter Bright wrotemod does not access hidden, neither does def. I don't understand the issue here.Not seeing the issue is the issue: D defies code inspection. Although even you are not seeing the leak, one can at least read the value of `hidden', by adding 5 LOC into some `module hijack;'---which _is_ a leak according to my definition of black hat acting. If it is okay for your intentions with D that black hats are able to read private data, then of course there is no issue zhat needs to be understood. -manfred
Aug 10 2007
"Walter Bright" wrote in message news:f9aalt$b2p$1 digitalmars.com...Steven Schveighoffer wrote:How is this better than the current implementation? In the current implementation, the code compiles and creates an obscure bug because the behavior isn't what the user expects. In this new implementation, the obscure bug is only less obscure because an exception is thrown. The code still compiles properly. Why allow compilation at all?However, does the author intend to handle the case where foo(int) is called? Let's assume he does (which is what the current compiler assumes). The author has not forbidden the user of the class from calling A.foo, because the user can simply cast to an A object, and call A.foo(int) directly. Therefore, if the author meant to override foo(int) by defining just a foo(long), he has failed. From these points, I believe that the above code is an example of an incorrectly implemented override, and I believe that the correct response the compiler should have is to error out, not while compiling test(), but while compiling B, indicating to the user that he should either declare the alias, or override foo(int). This is the point in which my solution differs from Java. Java would allow this to proceed, and call the base class' foo(int) in all cases, which is not what the author intended.The next update of the compiler will throw a runtime exception for this case.The problem is WHEN the compile time error is generated. I do not mind if the author of the class cannot generate object code for his class because of a compile time error (see my alternate solution). However, because the compile error is generated when someone attempts to USE the class, the error is delivered to the incorrect person. That person may not have the ability to fix the error. Silent hijacking of code is not possible with the solution I propose, so that becomes a moot point.The second issue is how to handle the foo(char[]) case. In the current implementation, because A is not searched for overrides, the compiler produces an error indicating that the user tried to call the foo(long) method, but there is no implicit conversion from char[] to long. There are two possibilities. One is that the author did not notice that A.foo(char[]) existed, and intended to override all instances of foo() with his foo(long) override. However, the author is NOT notified of this issue, only the user of the class is notified of this issue. So the potential for unambiguous code to have escaped exists.But a compile time error is still generated, so I don't regard this as a big problem. The big problems are silent hijacking of code.I understand your point of view, and that is why I said that I would concede to a solution where a compiler error is thrown if all overloads are not handled. However, I think it is still incorrect to generate the compiler error on the use of the class rather than the compilation of the class. My preference as I said is to avoid having to specify the alias in the simple case where the arguments are not implicitly convertable, but if there must be a compilation error, I am willing to live with that.The second possibility is that the author fully intended to allow the base class to define foo(char[]), but forgot to define the alias. Again, since the compiler gives no error, he is unaware that he is releasing buggy code to the world. I believe the correct assumption of the compiler should be that the user wanted the alias for the base class' foo(char[]), and should alias it implicitly if and only if no suitable match exists on the derived class. In the case where the author did not notice foo(char[]) existed, he problably doesn't mind that foo(char[]) is defined by the base class.The problem with code that looks like a mistake, but the compiler makes some assumption about it and compiles it anyway, is that the code auditor cannot tell if it was intended behavior or a coding error.The problem is they ARE implemented, and now they will cause runtime exceptions! Also, now this breaks the contract that the base class provides. If you give me an instance of a certain class, and the documentation for that class says that it defines a given method, then that method should be implemented in all derivatives. If you want to derive a class and force the implementation of a base class' method to throw an exception, I think that should be the (for better lack of a word) exception, not the rule. Why derive from a class where you want to shoehorn its functionality into something different? I would recommend to someone trying to do that to define a new class, rather than derive. I challenge anyone to give a real-world example of why this should be possible. As you point out, it is possible to force the implementation that you are specifying by overriding the overloaded method and throwing the exception yourself, and maybe the requirement of extra cluttering functions will discourage people from implementing their code this way. Maybe there could be a specific keyword or something to specify that you want to have the overload throw an exception, so there is less code clutter, but I do not think the default should be throwing an exception. -SteveIf a suitable match exists that is not a direct override of the base class, then the issue reduces to the previous case, where an implicit conversion is required, and the compiler should error out. There is one other possibile solution that I would be willing to concede to, and that is that the compiler errors out if the base class does not override all overloads of a particular method name. This forces the user to either override all overloads of the method, or define the alias. This would be the safest solution, as the author of B must make his intentions perfectly clear.I am not comfortable with this method, as it will force the derived class programmer to implement overloads that may not be at all meant to exist in the API he defines for that class. I think he should be in full control of the API for the class, and not forced to provide implementations of functions that may be irrelevant clutter. I prefer the solution where attempts to call unoverridden base class overloads will result in a runtime exception.
Aug 07 2007
Steven Schveighoffer wrote:So my proposal is to change the specification so that: If there is a class A, which is a base class of class B, where A defines a method foo(args), and B defines a method foo(args2), such that the types of args2 cannot be implicitly converted to args, and B does not define foo(args), then the definition of B.foo(args) shall be implicitly aliased to A.foo(args). If, using the same assumptions, args2 can be implicitly converted to args, then the compiler should fail to compile B indicating that the user must define B.foo(args) by override or by alias. I believe this will give us the best of both camps, and allow much less code to be released with silent bugs than the current implementation. Let the hole shooting begin...I like it. In fact it solves the existing problem exhibited by the original example 2! Here are our original examples given by Walter and used to support the current C++ like behaviour. ----------------------------------- class X1 { void f(int); } // chain of derivations X(n) : X(n-1) class X9: X8 { void f(double); } void g(X9 p) { p.f(1); // X1.f or X9.f ? } ----------------------------------- class B { long x; void set(long i) { x = i; } void set(int i) { x = i; } long squareIt() { return x * x; } } class D : B { long square; void set(long i) { B.set(i); square = x * x; } long squareIt() { return square; } } long foo(B b) { b.set(3); return b.squareIt(); } ----------------------------------- Under your proposal these would both be classify as "incorrectly implemented override"'s and fail to compile with an error. In example 1 the error would occur when compiling X9. In example 2 the error would occur when compiling D. In both cases the addition of 'alias' or an exact override for the 'int' method from the base will resolve the error. In example 1, if other overloads existed in X2 thru X8 i.e. "void f(short);" then both "void f(int);" and "void f(short);" would need to be aliased or defined in X9 to resolve the error. This seems to indicate that all base classes all the way back to the root need to be examined, that could be complex... As I mentioned earlier your proposal will solve the existing problem caused by example 2, which under both C++ and Java like implementations calls B.set(int) then D.squareit() resulting in 0 instead of 9. That, combined with the implicit alias of overloads where no implicit conversion is possible and I think it will be a feature which is both safe and intuitive. I'm interested to see what other people think. Regan
Aug 07 2007