www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Re: What's wrong with D's templates?

reply Kevin Bealer <kevinbealer gmail.com> writes:
dsimcha Wrote:

 == Quote from Yigal Chripun (yigal100 gmail.com)'s article
 but even more frustrating is the fact that
 template compilation bugs will also happen at the client.
 There's a whole range of designs for this and related issues and IMO the
 C++ design is by far the worst of them all. not to mention the fact that
 it isn't an orthogonal design (like many other "features" in c++). I'd
 much prefer a true generics design to be separated from compile-time
 execution of code with e.g. CTFE or AST macros, or other designs.

Since generics work by basically casting stuff to Object (possibly boxing it) and casting back, I wonder if it would be easy to implement generics on top of templates through a minimal wrapper. The main uses for this would be executable bloat (for those that insist that this matters in practice) and allowing virtual functions where templates can't be virtual.

In C++ you could define a MyObject and MyRef (smart pointer to Object) types that implement the methods you need (like comparisons) as virtual functions that are either pure or throw exceptions. Then just define a non-template class that inherits from (or just aggregates and wraps) a vector<MyRef> and map<MyReft, MyRef>. Now you can use this map and vector code to build whatever solution you need, using dynamic_cast<T> to insure the types are what you want. If you want you can throw a type safe wrapper that uses dynamic_cast<T> around this, as long as all the methods of that class are inlined, it should have no extra bloat. Now your back at the Java level of expressiveness. I wonder, though, if you actually save anything. Since vector::operator[](size_t) is just an array index that will get inlined into your code, I think it should have much *less* code bloat in your executable than a function call (to virtual MyRef vector<MyRef>::operator[](size_t)) plus dynamic casts to figure out if all the types are castable. Movie Poster: Dr. Stroustrouplove: or How I Stopped Worrying and Learned to Love the Bloat. As for mixing template code and client code, is it that big of a deal in practice? If you are making something big enough to be worth patenting, won't most of the "heavy lifting" classes probably not be templates? Unless you are marketing template/container libraries specifically I guess... Personally I'm reluctant to buy that sort of thing from someone if I *can't* see the source. If I need a class like "vector", in practice I need to be able to dig into its methods to see what they do, e.g. is it really guaranteed that storage is contiguous, etc. On the other hand, if I was buying code to print a Word document to a pdf file, I could accept that I don't need to look into the code. But something like vector<> or map<> ends up getting so "intimate" with the rest of my design that I want to know how it works in detail just as an end-user. (Yeah, yeah, I know encapsulation says I shouldn't have to.) I think performance outweighs the needs of any particular business model, especially when you can do your own Object based version if you need to. I agree there should be ways to hide the implementation from the client, but if templates aren't one of them, then just factor that into where and how you use templates and when you use OO and virtual instead. It's enough that it's possible and practical to hide your impl, you don't need *every* language feature to make that its #1 priority. The performance / impl-hiding conflict is a fundamental problem -- if the user's compiler can't see the template method definitions, then it can't optimize them very well. If it can, then the user can too. Any method of compiling them that preserves enough info for the compiler to work with will probably be pretty easily and cleanly byte-code-decompilable. Kevin
Dec 21 2009
next sibling parent reply Walter Bright <newshound1 digitalmars.com> writes:
Kevin Bealer wrote:
 The performance / impl-hiding conflict is a fundamental problem -- if
 the user's compiler can't see the template method definitions, then
 it can't optimize them very well.  If it can, then the user can too.
 Any method of compiling them that preserves enough info for the
 compiler to work with will probably be pretty easily and cleanly
 byte-code-decompilable.

Absolutely right. One of the features that C++ exported templates was supposed to provide was obfuscation of the template bodies so that users couldn't see it. My contention was that there was essentially no reasonable method to ensure that. 1. any obfuscation method only has to be cracked by one individual, then everyone can see through it. 2. if you ship the library for multiple compilers, you only have to crack the weakest one 3. if you provide the decryption key to the customer, and you must, and an open source compiler is used, you lose
Dec 21 2009
parent reply Yigal Chripun <yigal100 gmail.com> writes:
On 22/12/2009 05:22, Walter Bright wrote:
 Kevin Bealer wrote:
 The performance / impl-hiding conflict is a fundamental problem -- if
 the user's compiler can't see the template method definitions, then
 it can't optimize them very well. If it can, then the user can too.
 Any method of compiling them that preserves enough info for the
 compiler to work with will probably be pretty easily and cleanly
 byte-code-decompilable.

Absolutely right. One of the features that C++ exported templates was supposed to provide was obfuscation of the template bodies so that users couldn't see it. My contention was that there was essentially no reasonable method to ensure that. 1. any obfuscation method only has to be cracked by one individual, then everyone can see through it. 2. if you ship the library for multiple compilers, you only have to crack the weakest one 3. if you provide the decryption key to the customer, and you must, and an open source compiler is used, you lose

You can also dis-assemble binary libs. That's not the point of this discussion. The point is having proper encapsulation.
Dec 21 2009
parent Walter Bright <newshound1 digitalmars.com> writes:
Yigal Chripun wrote:
 On 22/12/2009 05:22, Walter Bright wrote:
 Kevin Bealer wrote:
 The performance / impl-hiding conflict is a fundamental problem -- if
 the user's compiler can't see the template method definitions, then
 it can't optimize them very well. If it can, then the user can too.
 Any method of compiling them that preserves enough info for the
 compiler to work with will probably be pretty easily and cleanly
 byte-code-decompilable.

Absolutely right. One of the features that C++ exported templates was supposed to provide was obfuscation of the template bodies so that users couldn't see it. My contention was that there was essentially no reasonable method to ensure that. 1. any obfuscation method only has to be cracked by one individual, then everyone can see through it. 2. if you ship the library for multiple compilers, you only have to crack the weakest one 3. if you provide the decryption key to the customer, and you must, and an open source compiler is used, you lose

You can also dis-assemble binary libs. That's not the point of this discussion. The point is having proper encapsulation.

Disassembling is a very different deal, because much information is lost in the emission of assembler. But with template bodies, 100% of the semantic information must be present. Proper encapsulation is a convention thing, because if you have the source (or can extract it somehow) you can defeat any encapsulation if you've a mind to.
Dec 21 2009
prev sibling parent Yigal Chripun <yigal100 gmail.com> writes:
On 21/12/2009 22:41, Kevin Bealer wrote:
 dsimcha Wrote:

 == Quote from Yigal Chripun (yigal100 gmail.com)'s article
 but even more frustrating is the fact that template compilation
 bugs will also happen at the client. There's a whole range of
 designs for this and related issues and IMO the C++ design is by
 far the worst of them all. not to mention the fact that it isn't
 an orthogonal design (like many other "features" in c++). I'd
 much prefer a true generics design to be separated from
 compile-time execution of code with e.g. CTFE or AST macros, or
 other designs.

Since generics work by basically casting stuff to Object (possibly boxing it) and casting back, I wonder if it would be easy to implement generics on top of templates through a minimal wrapper. The main uses for this would be executable bloat (for those that insist that this matters in practice) and allowing virtual functions where templates can't be virtual.

In C++ you could define a MyObject and MyRef (smart pointer to Object) types that implement the methods you need (like comparisons) as virtual functions that are either pure or throw exceptions. Then just define a non-template class that inherits from (or just aggregates and wraps) a vector<MyRef> and map<MyReft, MyRef>. Now you can use this map and vector code to build whatever solution you need, using dynamic_cast<T> to insure the types are what you want. If you want you can throw a type safe wrapper that uses dynamic_cast<T> around this, as long as all the methods of that class are inlined, it should have no extra bloat. Now your back at the Java level of expressiveness. I wonder, though, if you actually save anything. Since vector::operator[](size_t) is just an array index that will get inlined into your code, I think it should have much *less* code bloat in your executable than a function call (to virtual MyRef vector<MyRef>::operator[](size_t)) plus dynamic casts to figure out if all the types are castable. Movie Poster: Dr. Stroustrouplove: or How I Stopped Worrying and Learned to Love the Bloat. As for mixing template code and client code, is it that big of a deal in practice? If you are making something big enough to be worth patenting, won't most of the "heavy lifting" classes probably not be templates? Unless you are marketing template/container libraries specifically I guess... Personally I'm reluctant to buy that sort of thing from someone if I *can't* see the source. If I need a class like "vector", in practice I need to be able to dig into its methods to see what they do, e.g. is it really guaranteed that storage is contiguous, etc. On the other hand, if I was buying code to print a Word document to a pdf file, I could accept that I don't need to look into the code. But something like vector<> or map<> ends up getting so "intimate" with the rest of my design that I want to know how it works in detail just as an end-user. (Yeah, yeah, I know encapsulation says I shouldn't have to.) I think performance outweighs the needs of any particular business model, especially when you can do your own Object based version if you need to. I agree there should be ways to hide the implementation from the client, but if templates aren't one of them, then just factor that into where and how you use templates and when you use OO and virtual instead. It's enough that it's possible and practical to hide your impl, you don't need *every* language feature to make that its #1 priority. The performance / impl-hiding conflict is a fundamental problem -- if the user's compiler can't see the template method definitions, then it can't optimize them very well. If it can, then the user can too. Any method of compiling them that preserves enough info for the compiler to work with will probably be pretty easily and cleanly byte-code-decompilable. Kevin

a few points: Java generics are poorly designed and create holes in the type system. Some of the designers of the language admit this openly. This implementation doesn't represent the general concept. your performance / impl-hiding conflict doesn't exist. I already mentioned in a previous post how the C# compiler handles generics. Also you assume the same (broken) compilation model as in C++. don't.
Dec 21 2009