www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Design deficiency of C++ and D templates

reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Hi there,

after having worked with C++ templates quite intensively (writing a small
expression template linear algebra package that really took the language to
its limits) I found the one fundamental deficiency of templates not their
syntax (which has been nicely solved in D) but the lack of a clean and
tight interface. Now, with D, this problem has not been addressed at all,
and with the newest draft of mixins, I feel that this problem will become
even more pressing:

If you look at a library based on functions, it has a clean interface. Every
function has typed parameters. If you try to call a function with
parameters of the wrong type, the compiler will immediately complain, and
you will easily spot the error. A class-based interface of a library does
not change this at all.

Now, imagine a library based on templates: Every template parameter has some
special meaning. Anyhow, there is no way to specify any restrictions on the
template parameters. If you use the template library, you have to be very
careful to know how to use the individual templates. If you use a template
with the wrong kind of parameters, the compiler will probably complain.
Anyhow, since there was no interface defined, the compiler will not tell
you that the parameter type was inadequate (like it would, if you call a
function with the wrong type of arguments) but it will point somewhere deep
into the template library to some place where that parameter is used.

Actually, without a cleanly specified interface, you may even have problems
to decide whether the bug is in your program or in the library.

Trying to see the big picture: D (and already C++) is actually a combination
of two programming languages: the first one interpreted at compile time,
the second one compiled and later executed at runtime. The second one is
strongly typed and there are powerful tools for debugging. The first one is
not typed at all and nearly impossible to debug if you have anything
moderately complex.

Comparing mixins with C-preprocessor-macros: It is generally percived that
mixins are safer than C-macros. I believe it is the other way around: the
C-preprocessor is mostly trivial. It has no loops, nesting, recursion or
anything like that. After all, it is not a complete programming language. I
have never seen code where the preprocessor commands would have been to
complex to understand. Debugging preprocessor code is mostly trivial. Now,
imagine debugging a program that does not compile in D and uses templates
from a library.

I believe: before D is made even more powerful, it should be considered how
to make it managable. In C++, it was only recently discovered at all that
templates offered a turing complete language which is interpreted at
compile-time. An absolutely ugly language, but still a language which is
even used in recent libraries.

In D, this compile-time language is now being harnessed, and it is
forseeable that people will use it far more than in C++. Anyhow, it still
is an untyped language worse than Perl...

Ciao,
Nobbi
May 17 2004
next sibling parent reply "Walter" <newshound digitalmars.com> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:c8al55$od$1 digitaldaemon.com...

 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it still
 is an untyped language worse than Perl...


I understand your point. There are various proposals floating around for
putting user defined type constraints onto template parameters. I think this
will address the problem.
May 17 2004
parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Walter wrote:


 
 "Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
 news:c8al55$od$1 digitaldaemon.com...

 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it still
 is an untyped language worse than Perl...


 
 I understand your point. There are various proposals floating around for
 putting user defined type constraints onto template parameters. I think
 this will address the problem.


I guess this will be quite difficult to get right. It is probably most
important to become aware that this compile-time language really is a full,
interpreted language and then look at other well-designed interpreted
languages like Python to see what they do to get the code working.

Instead of looking at D from the C++ perspective where the
template-mechanism "accidentally" evolved into a turing complete language,
we should really look at it from the perspective of a good interpreted
language to learn what is needed to handle complex projects.
May 17 2004
parent reply "Walter" <newshound digitalmars.com> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:c8au3i$ep1$1 digitaldaemon.com...

 Instead of looking at D from the C++ perspective where the
 template-mechanism "accidentally" evolved into a turing complete language,
 we should really look at it from the perspective of a good interpreted
 language to learn what is needed to handle complex projects.


This is a good idea, and has been proposed before. Unfortunately, the
proposed syntax for them turned out to be fundamentally unworkable for D.
I'm open to some new ideas here.
May 17 2004
parent Norbert Nemec <Norbert.Nemec gmx.de> writes:
Walter wrote:


 
 "Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
 news:c8au3i$ep1$1 digitaldaemon.com...

 Instead of looking at D from the C++ perspective where the
 template-mechanism "accidentally" evolved into a turing complete
 language, we should really look at it from the perspective of a good
 interpreted language to learn what is needed to handle complex projects.


 
 This is a good idea, and has been proposed before. Unfortunately, the
 proposed syntax for them turned out to be fundamentally unworkable for D.
 I'm open to some new ideas here.


Some ideas are already beginning to form in my head. Seems to become just
beautiful, even without much of a change to the existing language. Anyhow:
at the moment, arrays have higher priority...
May 18 2004
prev sibling next sibling parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Actually, thinking about it all a little more, I realized, that we should
not really look at Python or some other scripting language, but at
*functional* languages!

Took me some time to realize, that the compile-time language of D (just like
the C++-metaprogramming-language) is a *pure functional language*! Constant
symbols can only be assigned once, loops are expressed as recursions, etc.
I'm really looking forward to examine that language closer...



Norbert Nemec wrote:


 Hi there,
 
 after having worked with C++ templates quite intensively (writing a small
 expression template linear algebra package that really took the language
 to its limits) I found the one fundamental deficiency of templates not
 their syntax (which has been nicely solved in D) but the lack of a clean
 and tight interface. Now, with D, this problem has not been addressed at
 all, and with the newest draft of mixins, I feel that this problem will
 become even more pressing:
 
 If you look at a library based on functions, it has a clean interface.
 Every function has typed parameters. If you try to call a function with
 parameters of the wrong type, the compiler will immediately complain, and
 you will easily spot the error. A class-based interface of a library does
 not change this at all.
 
 Now, imagine a library based on templates: Every template parameter has
 some special meaning. Anyhow, there is no way to specify any restrictions
 on the template parameters. If you use the template library, you have to
 be very careful to know how to use the individual templates. If you use a
 template with the wrong kind of parameters, the compiler will probably
 complain. Anyhow, since there was no interface defined, the compiler will
 not tell you that the parameter type was inadequate (like it would, if you
 call a function with the wrong type of arguments) but it will point
 somewhere deep into the template library to some place where that
 parameter is used.
 
 Actually, without a cleanly specified interface, you may even have
 problems to decide whether the bug is in your program or in the library.
 
 Trying to see the big picture: D (and already C++) is actually a
 combination of two programming languages: the first one interpreted at
 compile time, the second one compiled and later executed at runtime. The
 second one is strongly typed and there are powerful tools for debugging.
 The first one is not typed at all and nearly impossible to debug if you
 have anything moderately complex.
 
 Comparing mixins with C-preprocessor-macros: It is generally percived that
 mixins are safer than C-macros. I believe it is the other way around: the
 C-preprocessor is mostly trivial. It has no loops, nesting, recursion or
 anything like that. After all, it is not a complete programming language.
 I have never seen code where the preprocessor commands would have been to
 complex to understand. Debugging preprocessor code is mostly trivial. Now,
 imagine debugging a program that does not compile in D and uses templates
 from a library.
 
 I believe: before D is made even more powerful, it should be considered
 how to make it managable. In C++, it was only recently discovered at all
 that templates offered a turing complete language which is interpreted at
 compile-time. An absolutely ugly language, but still a language which is
 even used in recent libraries.
 
 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it still
 is an untyped language worse than Perl...
 
 Ciao,
 Nobbi
May 18 2004
next sibling parent "Alex A. B." <freshmind fromru.com> writes:
 Actually, thinking about it all a little more, I realized, that we should
 not really look at Python or some other scripting language, but at
 *functional* languages!

 Took me some time to realize, that the compile-time language of D (just


like

 the C++-metaprogramming-language) is a *pure functional language*!


Constant

 symbols can only be assigned once, loops are expressed as recursions, etc.
 I'm really looking forward to examine that language closer...


You're quite right. I came to same conclusion too. I think the thing, which
could be
implemented without much hassle is support for real compile-time lists(as
template parameters)
and some form of list pattern matching. Another good idea would be support
for strings.

As for type-checking some form of template predicates could be implemented.

It could probably look like this:

// this declaration will fail if C is POD-type or doesn't contain
'someMember'
class Some(C) : C
? !(Traits::IsPOD(C)) && C::someMember
{
// ...
};
May 18 2004
prev sibling parent reply "Bent Rasmussen" <exo bent-rasmussen.info> writes:
 Actually, thinking about it all a little more, I realized, that we should
 not really look at Python or some other scripting language, but at
 *functional* languages!


That is definitely an interesting topic. My first instinct to translate my
simple surface functions from ML was to use templates

fun elliptictorus (a,b,c) =
    let
        fun f 1 (u,v) = (a+b*cos(v))*cos(u)
            | f 2 (u,v) = (a+b*cos(v))*sin(u)
            | f 3 (u,v) = c*sin(v)
            | f _ (u,v) = 0.0
    in
        f
    end

So here's one possible* translation into D

template elliptictorus(float a, float b, float c)
{
    float x(float u, float v) {...}
    float y(float u, float v) {...}
    float z(float u, float v) {...}
}

* Except templates only accept integral-typed value-parameters.

Probably not the best of examples though. :-)

Its nice have have a multiparadigm language like D that makes several
programming styles possible and doesn't force you into using classes for
everything. It has a very clean feel.

Also I note that a major goal of D is:


a.. Support multi-paradigm programming, i.e. at a minimum support
imperative, structured, object oriented, and generic programming paradigms.


 Took me some time to realize, that the compile-time language of D (just


like

 the C++-metaprogramming-language) is a *pure functional language*!


Constant

 symbols can only be assigned once, loops are expressed as recursions, etc.
 I'm really looking forward to examine that language closer...



 Norbert Nemec wrote:


 Hi there,

 after having worked with C++ templates quite intensively (writing a




small

 expression template linear algebra package that really took the language
 to its limits) I found the one fundamental deficiency of templates not
 their syntax (which has been nicely solved in D) but the lack of a clean
 and tight interface. Now, with D, this problem has not been addressed at
 all, and with the newest draft of mixins, I feel that this problem will
 become even more pressing:

 If you look at a library based on functions, it has a clean interface.
 Every function has typed parameters. If you try to call a function with
 parameters of the wrong type, the compiler will immediately complain,




and

 you will easily spot the error. A class-based interface of a library




does

 not change this at all.

 Now, imagine a library based on templates: Every template parameter has
 some special meaning. Anyhow, there is no way to specify any




restrictions

 on the template parameters. If you use the template library, you have to
 be very careful to know how to use the individual templates. If you use




a

 template with the wrong kind of parameters, the compiler will probably
 complain. Anyhow, since there was no interface defined, the compiler




will

 not tell you that the parameter type was inadequate (like it would, if




you

 call a function with the wrong type of arguments) but it will point
 somewhere deep into the template library to some place where that
 parameter is used.

 Actually, without a cleanly specified interface, you may even have
 problems to decide whether the bug is in your program or in the library.

 Trying to see the big picture: D (and already C++) is actually a
 combination of two programming languages: the first one interpreted at
 compile time, the second one compiled and later executed at runtime. The
 second one is strongly typed and there are powerful tools for debugging.
 The first one is not typed at all and nearly impossible to debug if you
 have anything moderately complex.

 Comparing mixins with C-preprocessor-macros: It is generally percived




that

 mixins are safer than C-macros. I believe it is the other way around:




the

 C-preprocessor is mostly trivial. It has no loops, nesting, recursion or
 anything like that. After all, it is not a complete programming




language.

 I have never seen code where the preprocessor commands would have been




to

 complex to understand. Debugging preprocessor code is mostly trivial.




Now,

 imagine debugging a program that does not compile in D and uses




templates

 from a library.

 I believe: before D is made even more powerful, it should be considered
 how to make it managable. In C++, it was only recently discovered at all
 that templates offered a turing complete language which is interpreted




at

 compile-time. An absolutely ugly language, but still a language which is
 even used in recent libraries.

 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it




still

 is an untyped language worse than Perl...

 Ciao,
 Nobbi
May 18 2004
parent Norbert Nemec <Norbert.Nemec gmx.de> writes:
Bent Rasmussen wrote:


 Actually, thinking about it all a little more, I realized, that we should
 not really look at Python or some other scripting language, but at
 *functional* languages!


 
 Its nice have have a multiparadigm language like D that makes several
 programming styles possible and doesn't force you into using classes for
 everything. It has a very clean feel.


I think "multiparadigm language" is a bit misleading in this context. What I
am talking about is, that D contains two independant languages: one that is
interpreted at compile time and one, that is compiled to be executed
lateron. The first one is pure functional and not object oriented, the
latter one is mostly imperative (and of course object oriented) with a few
features to support other paradigms to some extent.

The execution of the first language at compile time could probably be
compared to a preprocessor run. Of course, it is too closely knit into the
language to really split this preprocessor out from the real compilation,
but as a concept this really helps. Consider templates, instantiations and
mixins as parts of a powerful functional macro language that outputs some
code that is similar to D with all templates resolved, which is then
compiled.
May 18 2004
prev sibling parent reply "Achilleas Margaritis" <axilmar b-online.gr> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:c8al55$od$1 digitaldaemon.com...

 Hi there,

 after having worked with C++ templates quite intensively (writing a small
 expression template linear algebra package that really took the language


to

 its limits) I found the one fundamental deficiency of templates not their
 syntax (which has been nicely solved in D) but the lack of a clean and
 tight interface. Now, with D, this problem has not been addressed at all,
 and with the newest draft of mixins, I feel that this problem will become
 even more pressing:

 If you look at a library based on functions, it has a clean interface.


Every

 function has typed parameters. If you try to call a function with
 parameters of the wrong type, the compiler will immediately complain, and
 you will easily spot the error. A class-based interface of a library does
 not change this at all.

 Now, imagine a library based on templates: Every template parameter has


some

 special meaning. Anyhow, there is no way to specify any restrictions on


the

 template parameters. If you use the template library, you have to be very
 careful to know how to use the individual templates. If you use a template
 with the wrong kind of parameters, the compiler will probably complain.
 Anyhow, since there was no interface defined, the compiler will not tell
 you that the parameter type was inadequate (like it would, if you call a
 function with the wrong type of arguments) but it will point somewhere


deep

 into the template library to some place where that parameter is used.

 Actually, without a cleanly specified interface, you may even have


problems

 to decide whether the bug is in your program or in the library.

 Trying to see the big picture: D (and already C++) is actually a


combination

 of two programming languages: the first one interpreted at compile time,
 the second one compiled and later executed at runtime. The second one is
 strongly typed and there are powerful tools for debugging. The first one


is

 not typed at all and nearly impossible to debug if you have anything
 moderately complex.

 Comparing mixins with C-preprocessor-macros: It is generally percived that
 mixins are safer than C-macros. I believe it is the other way around: the
 C-preprocessor is mostly trivial. It has no loops, nesting, recursion or
 anything like that. After all, it is not a complete programming language.


I

 have never seen code where the preprocessor commands would have been to
 complex to understand. Debugging preprocessor code is mostly trivial. Now,
 imagine debugging a program that does not compile in D and uses templates
 from a library.

 I believe: before D is made even more powerful, it should be considered


how

 to make it managable. In C++, it was only recently discovered at all that
 templates offered a turing complete language which is interpreted at
 compile-time. An absolutely ugly language, but still a language which is
 even used in recent libraries.

 In D, this compile-time language is now being harnessed, and it is
 forseeable that people will use it far more than in C++. Anyhow, it still
 is an untyped language worse than Perl...

 Ciao,
 Nobbi


Doesn't template specialization cover this topic ? I thought that D was
already capable of defining constraints for template types.
May 18 2004
parent reply Norbert Nemec <Norbert.Nemec gmx.de> writes:
Achilleas Margaritis wrote:

 Doesn't template specialization cover this topic ? I thought that D was
 already capable of defining constraints for template types.


That is just some very basic pattern matching that does not have the
necessary power. What I was talking about are common cases like:

------------------
template X(T) {
        void x(T a) {
                a.something();
        }
}

void myfunc(mytype b) {
        X!(mytype).x(b);
}
------------------

Now, if mytype does not offer a .something() member function, the compiler
will, of course, complain. But is it an error in the template code or is it
an error of the instantiation with an illegal type parameter? The problem
is, that there is no way to specify in the definition of a template what
capabilities a parameter should have.

If you have code that nests over several layers of templates, this really
gives messy compiler errors.

On the other hand, I realize now: looking at the compile-time-language as
full functional programming language, template instantiations would
actually correspond to function calls. If an error occurs several levels
deep, the compiler error would of course have to include a "stack
trace" (which is exactly, what the "instantiated at ..., instantiated
at ..." lists in C++ are!)
May 18 2004
next sibling parent Kevin Bealer <Kevin_member pathlink.com> writes:
In article <c8dukv$2als$1 digitaldaemon.com>, Norbert Nemec says...
.

Now, if mytype does not offer a .something() member function, the compiler
will, of course, complain. But is it an error in the template code or is it
an error of the instantiation with an illegal type parameter? The problem
is, that there is no way to specify in the definition of a template what
capabilities a parameter should have.

If you have code that nests over several layers of templates, this really
gives messy compiler errors.

On the other hand, I realize now: looking at the compile-time-language as
full functional programming language, template instantiations would
actually correspond to function calls. If an error occurs several levels
deep, the compiler error would of course have to include a "stack
trace" (which is exactly, what the "instantiated at ..., instantiated
at ..." lists in C++ are!)


I was thinking about this today (i.e. about templates as functions).

Imagine variables that can hold:
1. types.
2. fragments of code.
3. lists of names, values or statements.

And the ability to:

Loop over these (arbitrary statement bodies or variable names),

Pass them into and return them from "functions", i.e. templates or other code
generation mechanisms.


The functions of this language would produce lists or arrrays of language
elements.

Such a list could be:
a. Passed into a (code gen) function
b. Filtered or processed in some way (not sure...)
c. "Promoted" into actual code (i.e. inserted into the compile stream).

Given a list of names, I would like to (for example) generate 10 classes using
those names or some combination thereof as:

1. variable names.
2. string literals.
3. concatenated with a prefix or postfix string and then 1 or 2.
4. Used to construct member variables or methods, using the name
as a typename for a template parameter.
5. or even as a string substitution.

The last sounds a little like "macro", so maybe a more type safe way (3) should
be available first.

With C, its not so much that the language is bad as it now stands (though it has
some shortcomings that are a doozy), as that the bad techniques were available
first; maybe it always happens in that order though.

Kevin
May 18 2004
prev sibling parent reply "Achilleas Margaritis" <axilmar b-online.gr> writes:
"Norbert Nemec" <Norbert.Nemec gmx.de> wrote in message
news:c8dukv$2als$1 digitaldaemon.com...

 Achilleas Margaritis wrote:

 Doesn't template specialization cover this topic ? I thought that D was
 already capable of defining constraints for template types.


 That is just some very basic pattern matching that does not have the
 necessary power. What I was talking about are common cases like:

 ------------------
 template X(T) {
         void x(T a) {
                 a.something();
         }
 }

 void myfunc(mytype b) {
         X!(mytype).x(b);
 }
 ------------------

 Now, if mytype does not offer a .something() member function, the compiler
 will, of course, complain. But is it an error in the template code or is


it

 an error of the instantiation with an illegal type parameter? The problem
 is, that there is no way to specify in the definition of a template what
 capabilities a parameter should have.

 If you have code that nests over several layers of templates, this really
 gives messy compiler errors.

 On the other hand, I realize now: looking at the compile-time-language as
 full functional programming language, template instantiations would
 actually correspond to function calls. If an error occurs several levels
 deep, the compiler error would of course have to include a "stack
 trace" (which is exactly, what the "instantiated at ..., instantiated
 at ..." lists in C++ are!)


The problem is the messy compiler reporting, not the language. The compiler
could easily report the error without the template parameters, which is
where the problem actually is (the template parameters make reading the
error report difficult).
May 22 2004
parent Norbert Nemec <Norbert.Nemec gmx.de> writes:
Achilleas Margaritis wrote:


 On the other hand, I realize now: looking at the compile-time-language as
 full functional programming language, template instantiations would
 actually correspond to function calls. If an error occurs several levels
 deep, the compiler error would of course have to include a "stack
 trace" (which is exactly, what the "instantiated at ..., instantiated
 at ..." lists in C++ are!)


 
 The problem is the messy compiler reporting, not the language. The
 compiler could easily report the error without the template parameters,
 which is where the problem actually is (the template parameters make
 reading the error report difficult).


There should probably be options to tell the compiler how to give error
reports. Nested templates are really similar to nested function calls in a
program. If an error happens, you might need a detailed stack-trace to find
the problem, but such a stack trace really has to be formatted in a
readable way to be helpful. Today's C++ compilers are just not very helpful
there...
May 22 2004