digitalmars.D.learn - How Different Are Templates from Generics
- Just Dave (72/72) Oct 11 2019 I come from both a C++ and C# background. Those have been the
- Tobias Pankrath (3/5) Oct 11 2019 Probably the D templates relate to C# generics the same way that
- Jonathan M Davis (192/264) Oct 11 2019 Generics and templates are syntactically similiar but are really doing v...
- Just Dave (17/17) Oct 11 2019 Thanks for the thorough explanation. Most of that is how I was
- Jonathan M Davis (21/38) Oct 11 2019 You get a different template instantiation for each set of template
- Ahmet Sait (7/42) Oct 11 2019 Unlike Java, C# actually does generate different code pieces for
- jmh530 (5/6) Oct 12 2019 A very thorough explanation!
- Jonathan M Davis (28/34) Oct 12 2019 Yes, but it's unlikely that it would make any sense to do so. You'd
- jmh530 (13/14) Oct 12 2019 Thanks for the reply.
- Jonathan M Davis (16/30) Oct 12 2019 I wouldn't think that there would be enough different allocator types to
this:
public interface ISomeInterface<T>
{
T Value { get; }
}
public class SomeClass<T> : ISomeInterface<T>
{
T Value { get; set; }
}
public class SomeOtherClass<T> : ISomeInterface<T>
{
T Value { get; set; }
}
public static class Example
{
public static void Foo()
{
var instance1 = new SomeClass<int>(){ Value = 4; };
var instance2 = new SomeClass<int>(){ Value = 2; };
if (instance1 is ISomeInterface<int>)
{
Console.WriteLine("Instance1 is interface!");
}
if (instance2 is ISomeInterface<int>)
{
Console.WriteLine("Instance2 is interface!");
}
}
}
Expected output is both WriteLines get hit:
Instance1 is interface!
Instance2 is interface!
So now the 'D' version:
interface ISomeInterface(T)
{
T getValue();
}
class SomeClass(T) : ISomeInterface!T
{
private:
T t;
public:
this(T t)
{
this.t = t;
}
T getValue()
{
return t;
}
}
class SomeOtherClass(T) : ISomeInterface!T
{
private:
T t;
public:
this(T t)
{
this.t = t;
}
T getValue()
{
return t;
}
}
...which seems to work the same way with preliminary testing. I
guess my question is...templates are different than generics, but
can I feel confident continuing forward with such a design in D
Or are there lots of caveats I should be aware of?
Oct 11 2019
Tobias Pankrath <tobias pankrath.net> On Friday, 11 October 2019 at 14:43:49 UTC, Just Dave wrote:primary languages I have used.C++ templates do.
Oct 11 2019
On Friday, October 11, 2019 8:43:49 AM MDT Just Dave via Digitalmars-d-learn
wrote:
this:
public interface ISomeInterface<T>
{
T Value { get; }
}
public class SomeClass<T> : ISomeInterface<T>
{
T Value { get; set; }
}
public class SomeOtherClass<T> : ISomeInterface<T>
{
T Value { get; set; }
}
public static class Example
{
public static void Foo()
{
var instance1 = new SomeClass<int>(){ Value = 4; };
var instance2 = new SomeClass<int>(){ Value = 2; };
if (instance1 is ISomeInterface<int>)
{
Console.WriteLine("Instance1 is interface!");
}
if (instance2 is ISomeInterface<int>)
{
Console.WriteLine("Instance2 is interface!");
}
}
}
Expected output is both WriteLines get hit:
Instance1 is interface!
Instance2 is interface!
So now the 'D' version:
interface ISomeInterface(T)
{
T getValue();
}
class SomeClass(T) : ISomeInterface!T
{
private:
T t;
public:
this(T t)
{
this.t = t;
}
T getValue()
{
return t;
}
}
class SomeOtherClass(T) : ISomeInterface!T
{
private:
T t;
public:
this(T t)
{
this.t = t;
}
T getValue()
{
return t;
}
}
...which seems to work the same way with preliminary testing. I
guess my question is...templates are different than generics, but
can I feel confident continuing forward with such a design in D
Or are there lots of caveats I should be aware of?
Generics and templates are syntactically similiar but are really doing very
different things.
Generic functions and types operate on Object underneath the hood. If you
have Container<Foo> and Container<Bar>, you really just have
Container<Object> with some syntactic niceties to avoid explicit casts. You
get type checks to ensure that Container<Foo> isn't given a Bar unless Bar
is derived from Foo, and the casts to and from Object when giving
Container<Foo> a Foo are taken care of for you, but it's still always
Container<Object> underneath the hood.
In the case of Java, the type of T in Container<T> or foo<T>() is truly only
a compile time thing, so the bytecode only has Container<Object> and no clue
what type is actually supposed to be used (the casts are there where the
container or function is used, but the container or function has no clue
what the type is; it just sees Object). That makes it possible to cheat with
reflection and put something not derived from Foo in Container<Foo> but
will then usually result in runtime failures when the casts the compiler
information that Container<Foo> contains Foo rather than Object is
maintained at runtime, but you still have a Container<Object>. It's just a
Container<Object> with some metadata which keeps track of the fact that for
this particular object of Container<Object>, Object is always supposed to be
familiar with what the practical benefits that gives are, though I'd expect
that it would mean that reflection code would catch when you're trying to
put a Bar into Container<Foo> and wouldn't let you.
Note that for generics to work, they have to a common base type, and you
only ever get one version of a generic class or function even if it gets
used with many different types derived from Object. For a primitive type
be put into a type derived from Object in order to be used with generics (as
act like this at all.
Templates are literally templates for generating code. A template is nothing
by itself. Something like
struct Container(T)
{
T[] data;
}
or
T foo(T)(T t)
{
return t;
}
doesn't result in any code being in the binary until unless template is
instantiated with a specific type, and when that template is instantiated,
code is generated based on the type that it's instantiated with. So,
Container!int and Container!Foo result in two different versions of
Container being generated and put in the binary - one which operates on int,
and one which operates on Foo. There is no conversion to Object going on
here. The code literally uses int and Foo directly and is generated
specifically for those types. Not only does that mean that the generated
code can be optimized for the specific type rather than being for any
Object, but it also means that the code itself could do something completely
different for each type. e.g. with the template
T foo(T)(T t)
{
static if(is(T == int))
return t + 42;
else static if(is T == float)
return t * 7;
else
return t;
}
foo!int would be equivalent to
int foo(int t)
{
return t + 42;
}
foo!float would be equivalent to
float foo(float t)
{
return t * 7;
}
and foo!(int[]) would be equivalent to
int[] foo(int[] t)
{
return t;
}
and you would literally get functions like that generated in the binary.
Every separate instantiation of foo would result in a separate function in
the binary, and which branches of the static if got compiled in would depend
on which condition in the static if was true (just like with a normal if).
In the case of D (unlike C++, which doesn't have function attributes the way
that D does), because templated functions have attribute inference, the
generated functions can actually have completely different attributes as
well. e.g. with
T addOne(T)(T t)
{
return t + 1;
}
addOne!int would result in something like
int addOne(int t) safe pure nothrow
{
return t + 1;
}
whereas because pointer arithemitic is system, addOne!(int*) would result
in something like
int* addOne(int* t) system pure nothrow
{
return t + 1;
}
And since not all types have +, something like addOne!Object or
addOne!(int[]) wouldn't even compile. D uses template constraints to make it
so that that can be caught before the internals of the template are even
instantiated. e.g.
D addOne(T)(T t)
if(is(typeof(t + 1)))
{
return t + 1;
}
or
D addOne(T)(T t)
if(__traits(compiles, t + 1))
{
return t + 1;
}
would give an error for addOne!Object telling you that the template
constraint failed rather than telling you that the line
return t + 1;
failed to compile. Template constraints can also be used to overload
templates similar to how static if can be used inside them to generate
different code based on the template argument. e.g.
T foo(T)(T t)
if(is(T == int))
{
return t + 42;
}
T foo(T)(T t)
if(is(T == float))
{
return t * 7;
}
T foo(T)(T t)
if(!is(T == int) && !is(T == float))
{
return t;
}
though it's considered better practice to only overload templates when their
API is different and to use static if to change the internals when the API
is the same. So, in that example, the static if version would be better,
whereas something like
auto find(alias pred, T, U)(T[] haystack, U needle)
if(is(pred(T.init, U.init) : bool))
{
...
}
and
auto find(alias pred, T, U, V)(T[] haystack, U needle1, V needle2)
if(is(pred(T.init, U.init) : bool) &&
is(pred(T.init, V.init) : bool))
{
...
}
would use overloads, because the number of parameters is different.
I'm sure that there are other issues to discuss here, but the core
difference between generics and templates is that generics generate a single
piece of code using Object that gets reused every time that the generic is
used, no matter the type(s) that's used with the generic, whereas templates
generate a different piece of code for every set of template arguments. In
fact, in D, something like
auto foo(string file = __FILE__, size_t line = __LINE)(int blah)
{
...
}
would generate a different function for every single line that it's called
on (which is why file and line number are usually used as function arguments
rather than template arguments). C++ fills in __FILE__ and __LINE__ based on
the site of the declaration rather than the call site, so it wouldn't have
quite the same problem, but for both languages, foo!int or foo<int> would
generate a different piece of code than foo!MyClass or foo<MyClass>
generates.
So, you can get what gets called "template bloat" with templates when you
instantiate them with a bunch of different template arguments, because
you're getting a different piece of code generated for each instantiation,
whereas with generics you only get the one version of the generic, which
means that you don't get the bloat, but you also don't have as much
flexibility.
If all you're doing in D is creating templates that operate on types derived
from Object, then you probably won't notice much difference between
templates and generics, but you could notice some subtle differences when
using types not derived from Object (e.g. at least with Java, because it
use primitive types with generics can fail in surprising ways if you're used
to using a language with templates), and because templates outright generate
code, you can do a lot more with them than you could ever do with generics
(e.g. making code differ based on the template arguments by using template
constraints and/or static if). D's compile-time capabilities actually make
it extremely powerful for generating code, and templates are a key part of
that.
- Jonathan M Davis
Oct 11 2019
Thanks for the thorough explanation. Most of that is how I was
thinking it worked. However, that leaves me perplexed. If
templates just generate code then how come:
Wouldnt..
class SomeClass(T) : ISomeInterface!T
and..
class SomeOtherClass(T) : ISomeInterface!T
...generate two different interfaces? Two interfaces that do the
same thing, but two interfaces nonetheless? I assume each type in
D has some form of type id underlying everything, which wouldn't
that make the follow:
if (instance1 is ISomeInterface<int>)
{
Console.WriteLine("Instance1 is interface!");
}
fail? Or is there some extra magic that is making it work with my
experiments?
Oct 11 2019
On Friday, October 11, 2019 12:09:20 PM MDT Just Dave via Digitalmars-d-
learn wrote:
Thanks for the thorough explanation. Most of that is how I was
thinking it worked. However, that leaves me perplexed. If
templates just generate code then how come:
Wouldnt..
class SomeClass(T) : ISomeInterface!T
and..
class SomeOtherClass(T) : ISomeInterface!T
...generate two different interfaces? Two interfaces that do the
same thing, but two interfaces nonetheless? I assume each type in
D has some form of type id underlying everything, which wouldn't
that make the follow:
if (instance1 is ISomeInterface<int>)
{
Console.WriteLine("Instance1 is interface!");
}
fail? Or is there some extra magic that is making it work with my
experiments?
You get a different template instantiation for each set of template
arguments. So, if you have ISomeInterface!int, and you use
ISomeinterface!int somewhere else, because they're both instantiating
ISomeInterface with the same set of template arguments, you only get one
instantiation. So,
class SomeClass : ISomeInterface!int
and
class SomeOtherClass : ISomeInterface!int
would both be implementing the exact same interface. And if you then have
class SomeClass(T) : ISomeInterface!T
and
class SomeOtherClass(T) : ISomeInterface!T
then SomeClass!int and SomeOtherClass!int would both be implementing the
same interface, because in both cases, it would be ISomeInterface!int.
SomeClass!int and SomeOtherClass!float would not be implementing the same
interface, because it would be ISomeInterface!int and ISomeInterface!float,
but ISomeInterface!int doesn't result in multiple instantiations even if
it's used in different parts of the code.
- Jonathan M Davis
Oct 11 2019
On Friday, 11 October 2019 at 17:50:42 UTC, Jonathan M Davis wrote:Generic functions and types operate on Object underneath the hood. If you have Container<Foo> and Container<Bar>, you really just have Container<Object> with some syntactic niceties to avoid explicit casts. You get type checks to ensure that Container<Foo> isn't given a Bar unless Bar is derived from Foo, and the casts to and from Object when giving Container<Foo> a Foo are taken care of for you, but it's still always Container<Object> underneath the hood. In the case of Java, the type of T in Container<T> or foo<T>() is truly only a compile time thing, so the bytecode only has Container<Object> and no clue what type is actually supposed to be used (the casts are there where the container or function is used, but the container or function has no clue what the type is; it just sees Object). That makes it possible to cheat with reflection and put something not derived from Foo in Container<Foo> but will then usually result in runtime failures that kind of type erasure in that the information that Container<Foo> contains Foo rather than Object is maintained at runtime, but you still have a Container<Object>. It's just a Container<Object> with some metadata which keeps track of the fact that for this particular object of Container<Object>, Object is always supposed to be a Foo. As I'm a lot less the practical benefits that gives are, though I'd expect that it would mean that reflection code would catch when you're trying to put a Bar into Container<Foo> and wouldn't let you. Note that for generics to work, they have to a common base type, and you only ever get one version of a generic class or function even if it gets used with many different types derived from Object. For a primitive type like int or float (as well as derived from Object in order to be used with generics (as I Templates don't act like this at all.different value types [1] and reuses the same generated code for reference types. [1] https://alexandrnikitin.github.io/blog/dotnet-generics-under-the-hood/
Oct 11 2019
On Friday, 11 October 2019 at 17:50:42 UTC, Jonathan M Davis wrote:[snip]A very thorough explanation! One follow-up question: would it be possible to mimic the behavior of Java generics in D?
Oct 12 2019
On Saturday, October 12, 2019 2:11:28 PM MDT jmh530 via Digitalmars-d-learn wrote:On Friday, 11 October 2019 at 17:50:42 UTC, Jonathan M Davis wrote:Yes, but it's unlikely that it would make any sense to do so. You'd basically have to do something like auto foo(T)(T obj) if(is(T : Object) && !is(T == Object)) { return foo(cast(Object)obj); } auto foo(Object obj) { ... } And for containers, you'd basically end up with a templated container that was just a wrapper around a non-templated container that operated on Object. If you went with such an approach, you'd get less code in the binary, but you'd also end up with a deeper call stack because of all of the wrappers needed to add the casts for you. However, since Object can't do much of anything, having code that operates on Object isn't usually very useful. You could have the code use a different base class that had whatever operations you wanted, but you're still adding a fair bit of extra machinery just to avoid a few template instantiations. And since idiomatic D doesn't use classes much (rather, best practice is to use a struct unless you need polymorphism or you need something to always be a reference type), and it uses templates quite heavily (that's especially true with range-based code), it would be pretty bizarre to try and use Java's approach in D. - Jonathan M Davis[snip]A very thorough explanation! One follow-up question: would it be possible to mimic the behavior of Java generics in D?
Oct 12 2019
On Saturday, 12 October 2019 at 21:44:57 UTC, Jonathan M Davis wrote:[snip]Thanks for the reply. As with most people, I don't write a lot of D code that uses classes that much. The use case I'm thinking of is with allocators, which - to be honest - is not something I deal with much in my own code. Basically, some of the examples have stuff like ScopedAllocator!Mallocator, which would imply that there is a different ScopedAllocator for each allocator. However, if you apply Java's generics, then you would just have one. Not sure if it would make any kind of difference in real-life code, but still interesting to think about.
Oct 12 2019
On Saturday, October 12, 2019 9:48:02 PM MDT jmh530 via Digitalmars-d-learn wrote:On Saturday, 12 October 2019 at 21:44:57 UTC, Jonathan M Davis wrote:I wouldn't think that there would be enough different allocator types to matter much. Certainly, the amount of code that gets generated by templates for dealing with stuff like ranges would dwarf it. If program size really becomes a problem, then examining how code uses templates and trying to reduce how much they're used could certainly have an impact, but I'd expect it to be fairly rare that attempting to emulate Java's generics would help much - especially since it would only work when classes were involved. The main place that such an approach would have much chance of having an impact would be with regards to container implementations when the code puts a lot of different types of class objects inside of containers, and even that would easily be dwarfed by all of the other template usage in your typical D program. For Java's approach to make much sense, you'd probably have to be writing very Java-like code. - Jonathan M Davis[snip]Thanks for the reply. As with most people, I don't write a lot of D code that uses classes that much. The use case I'm thinking of is with allocators, which - to be honest - is not something I deal with much in my own code. Basically, some of the examples have stuff like ScopedAllocator!Mallocator, which would imply that there is a different ScopedAllocator for each allocator. However, if you apply Java's generics, then you would just have one. Not sure if it would make any kind of difference in real-life code, but still interesting to think about.
Oct 12 2019