digitalmars.D - Can D do some sort of automatic template to "dynamic template"
- Adam (59/59) Sep 18 2015 Fact: Templates are static!
- bitwise (45/105) Sep 18 2015 I think that what you're asking can only be accomplished in an
- Laeeth Isharc (30/90) Sep 19 2015 I think you could also do this using structs with a type enum
- Adam (146/255) Sep 19 2015 I'm thinking I've probably been miunderstood here. Let me clarify
- Adam D. Ruppe (6/7) Sep 19 2015 What you're asking is actually pretty easy. You can just write a
- Adam (45/52) Sep 19 2015 It might be, but I don't see how it can be done easily. I don't
- Kagamin (3/5) Sep 21 2015 Why not just use
- bitwise (11/12) Sep 19 2015 No.
Fact: Templates are static!
But templates are still normal functioning code and can actually
work dynamically. We can see this by including a D compiler
inside an app and have the app re-compile the templates for
specific dynamic types. It doesn't need templates to do of
course, but it shows that "Templates are static" is not quite as
simple as it sounds.
But could D actually create some sort of internal expression of D
templates as dynamic objects that can be used in run-time?
If have a template T that only uses 3 types, then one can simple
do something like(pseudo):
t(object o)
{
// use dynamic reflection or whatever to get the type of the
object o
if (o.type == 'int')
return T!(int)(o.value, o.unit);
if (o.type == 'float')
return T!(float)(o.value, o.unit);
return T!(unknown)(o.value, o.unit);
}
etc...
t then is a "dynamic representation" of the static functionality
of T.
But my guess is that the compiler could essentially do this for
us and allow one to use templates in a dynamic way. Most of the
time it may not be of much use but, say, if one is interacting
with console and T converts a string to a number. T could convert
a number with unit such as "3mm" in to some base representation
such as 0.003(m). float's are handled differently than
ints(rounding depending on some criteria).
Maybe we use the T internally and its lightning fast. But for the
console case, we have write a dynamic version of the template or
create something similar to the code above.
But the compiler already knows, at least, some of the times that
will be used and it should be able to instantiate the template
for each of those types.
e.g., if you use T!(float) in the code, it will know to create
the "dynamic wrapper" to include that type in the list. If you
don't every use T!(float) explicitly then you can simply create a
useless statement like {auto x = T!(float)("3.04mm");} which will
be optimized but the compiler will then include float as a type
used in the dynamic version of T. I may not be making much sense
here, but it's the best I'll do.
The compiler then can do something like assemble the dynamic
version of the template into a new function which can be used at
compile time on any object.
Using a simple special character could inform the compiler to use
the dynamic template version.
object u = userInput();
&T(u); // (instead of using t(u)).
Here the compiler does all the work for us. If u has type float
then this gets effectively dispatched to T!(float)(cast(float)u).
If it's an int, then T!(int)(cast(int)u).
What it mainly does for us is simplify having to do the cast our
self along with the string of if checking for all the types or
having to code that works dynamically
Or is it impossible for the D compiler to accomplish such a task
automatically for us?
Sep 18 2015
bitwise <bitwise.pvt gmail.com>
On Friday, 18 September 2015 at 20:55:06 UTC, Adam wrote:
Fact: Templates are static!
But templates are still normal functioning code and can
actually work dynamically. We can see this by including a D
compiler inside an app and have the app re-compile the
templates for specific dynamic types. It doesn't need templates
to do of course, but it shows that "Templates are static" is
not quite as simple as it sounds.
But could D actually create some sort of internal expression of
D templates as dynamic objects that can be used in run-time?
If have a template T that only uses 3 types, then one can
simple do something like(pseudo):
t(object o)
{
// use dynamic reflection or whatever to get the type of the
object o
if (o.type == 'int')
return T!(int)(o.value, o.unit);
if (o.type == 'float')
return T!(float)(o.value, o.unit);
return T!(unknown)(o.value, o.unit);
}
etc...
t then is a "dynamic representation" of the static
functionality of T.
But my guess is that the compiler could essentially do this for
us and allow one to use templates in a dynamic way. Most of the
time it may not be of much use but, say, if one is interacting
with console and T converts a string to a number. T could
convert a number with unit such as "3mm" in to some base
representation such as 0.003(m). float's are handled
differently than ints(rounding depending on some criteria).
Maybe we use the T internally and its lightning fast. But for
the console case, we have write a dynamic version of the
template or create something similar to the code above.
But the compiler already knows, at least, some of the times
that will be used and it should be able to instantiate the
template for each of those types.
e.g., if you use T!(float) in the code, it will know to create
the "dynamic wrapper" to include that type in the list. If you
don't every use T!(float) explicitly then you can simply create
a useless statement like {auto x = T!(float)("3.04mm");} which
will be optimized but the compiler will then include float as a
type used in the dynamic version of T. I may not be making much
sense here, but it's the best I'll do.
The compiler then can do something like assemble the dynamic
version of the template into a new function which can be used
at compile time on any object.
Using a simple special character could inform the compiler to
use the dynamic template version.
object u = userInput();
&T(u); // (instead of using t(u)).
Here the compiler does all the work for us. If u has type float
then this gets effectively dispatched to
T!(float)(cast(float)u). If it's an int, then
T!(int)(cast(int)u).
What it mainly does for us is simplify having to do the cast
our self along with the string of if checking for all the types
or having to code that works dynamically
Or is it impossible for the D compiler to accomplish such a
task automatically for us?
I think that what you're asking can only be accomplished in an
interpreted or JIT compiled language. With a compiled language,
you have to know all the possible types you will support up front.
If you're getting the input from a console though, you must be
parsing it right? The parser cannot complete it's task without
knowing what it's parsing, and the parser should pass along or
store any relevant information for future use.
For compiled languages, there is the idea of "Type Erasure".
Polymorphism is one way to get this done. You will have to create
your own base object which specifies all possible functionality.
Example:
import std.stdio;
import std.conv;
import std.random;
abstract class BaseObj {
abstract void print();
}
class FloatObj : BaseObj {
float value;
this(float val) { value = val; }
override void print() { writeln("float: " ~ value.to!string);
}
}
class IntObj : BaseObj {
int value;
this(int val) { value = val; }
override void print() { writeln("int: " ~ value.to!string); }
}
BaseObj userInput()
{
if(uniform(0, 2) % 2 == 0)
return new FloatObj(uniform(0.0f, 100.0f));
else
return new IntObj(uniform(0, 100));
}
void main(string[] args)
{
BaseObj[] objs;
foreach(i; 0..10)
{
BaseObj obj = userInput();
obj.print();
}
}
Sep 18 2015
On Friday, 18 September 2015 at 20:55:06 UTC, Adam wrote:
Fact: Templates are static!
But templates are still normal functioning code and can
actually work dynamically. We can see this by including a D
compiler inside an app and have the app re-compile the
templates for specific dynamic types. It doesn't need templates
to do of course, but it shows that "Templates are static" is
not quite as simple as it sounds.
But could D actually create some sort of internal expression of
D templates as dynamic objects that can be used in run-time?
If have a template T that only uses 3 types, then one can
simple do something like(pseudo):
t(object o)
{
// use dynamic reflection or whatever to get the type of the
object o
if (o.type == 'int')
return T!(int)(o.value, o.unit);
if (o.type == 'float')
return T!(float)(o.value, o.unit);
return T!(unknown)(o.value, o.unit);
}
etc...
t then is a "dynamic representation" of the static
functionality of T.
But my guess is that the compiler could essentially do this for
us and allow one to use templates in a dynamic way. Most of the
time it may not be of much use but, say, if one is interacting
with console and T converts a string to a number. T could
convert a number with unit such as "3mm" in to some base
representation such as 0.003(m). float's are handled
differently than ints(rounding depending on some criteria).
Maybe we use the T internally and its lightning fast. But for
the console case, we have write a dynamic version of the
template or create something similar to the code above.
But the compiler already knows, at least, some of the times
that will be used and it should be able to instantiate the
template for each of those types.
e.g., if you use T!(float) in the code, it will know to create
the "dynamic wrapper" to include that type in the list. If you
don't every use T!(float) explicitly then you can simply create
a useless statement like {auto x = T!(float)("3.04mm");} which
will be optimized but the compiler will then include float as a
type used in the dynamic version of T. I may not be making much
sense here, but it's the best I'll do.
The compiler then can do something like assemble the dynamic
version of the template into a new function which can be used
at compile time on any object.
Using a simple special character could inform the compiler to
use the dynamic template version.
object u = userInput();
&T(u); // (instead of using t(u)).
Here the compiler does all the work for us. If u has type float
then this gets effectively dispatched to
T!(float)(cast(float)u). If it's an int, then
T!(int)(cast(int)u).
What it mainly does for us is simplify having to do the cast
our self along with the string of if checking for all the types
or having to code that works dynamically
Or is it impossible for the D compiler to accomplish such a
task automatically for us?
I think you could also do this using structs with a type enum
inside and a union for the data contents. Then you can use
string mixins using traits to create generic runtime functions
that dispatch to the appropriate method/free standing function on
the underlying type. Take a look at Algebraic and Adam Ruppe's
jsvar.
Some of the genericity can be handled at compile time. For
example, I have a templated time series entry struct. A time
series is a slice of such structs. An entry definitively has a
date, and it may have either a bid and ask or (a close and
optionally open, high, low, adjusted close). It has a volume,
and may have an open interest. The date may be a year/month/day
or it might have a time too. I want to store things efficiently
in memory and on disk. So I use
hasMember!(typeof(pricebar.date),"hour") to find out if it's an
intraday date or daily for example. See Andrei's allocator talk,
his code, and maybe that for cerealed.
I have a bunch of templated functions that can do useful things
with such a time series.
But then I want to pass them around wrapped in an a tagged union
and generate versions of these functions that can operate on this
untemplated struct. (I will wrap the slices, not the entries).
That will make it easier to access my work from a dynamic
language like Lua. I haven't written this bit yet, but I have
done something similar in another context. A degree of misery in
trying to figure out std.traits, but that's a one off cost and it
goes quicker if you read others code for inspiration. UDAs help
in specifying which functions should have wrapped versions
generated etc.
Sep 19 2015
On Saturday, 19 September 2015 at 15:56:23 UTC, Laeeth Isharc wrote:On Friday, 18 September 2015 at 20:55:06 UTC, Adam wrote:I'm thinking I've probably been miunderstood here. Let me clarify what I'm talking about and see if it still more sense. Templated functions's types are resolved at compile time, so void foo(T)(T x){} is a templated function that operates on a type T and is a sort of compile time construct(since it depends on the unknown T). I do not mean it only can be used at compile time, but T must be known then. So, foo!T only exists at compile time, right? Yet, for any actual type such as an int, foo!int is a runtime function(it exists in the binary). The compiler figures out all the types used at compile time and creates all the various foo!N's. If K is the set of all the types used at compile type with foo, we could effectively write foo!K as a sort of collection of all the different foo's(but it's not really any different than foo!T except T is any type while K is the actual types used by the programmer). Ok so far? So, if the programmer called foo!int, foo!float in the code, the compiler then constructs foo!K = {foo!int, foo!float} and puts them in the binary. Then these are actual run-time functions and foo!K represents them. foo!K is as real as foo!T is, but restricted to only contains the actual used types and put in the binary. If the compiler could hypothetically put all of foo!T then the binary would be infinite, or, at least very large. So, instead it just keeps foo!K. If you want to expand K at run-time, you are out of luck. Ok so far? we can actually create foo!K easily(assuming foo!T(int)) fooK(string K, object i) { if (K == "int") foo!int(i); else if (K == "float") foo!float(i); else foo!unknown(i); // unknown is special, not needed but it would only add one element to foo!K, so the binary would not be much bigger and we could specialize foo for "unknown" to handle those cases such as throwing an exception. } Ok? now fooK is a function that exists that works at run-time on all types. It is an approximation to foo!T(int). If, say, in your code, you now use foo!string, then on recompile we would have fooK(string K, object i) { if (K == "int") foo!int(i); else if (K == "float") foo!float(i); else if (K == "string") foo!string(i); else foo!unknown(i); } Again, we can do this all by hand. And we can call fooK on objects at run-time rather than known types at compile time. The only reason we can't use foo!T is because the compiler doesn't know all the types we will be using(at run-time), if it did, we could use foo!T at run-time. It simply does the best job it can and gets the compile time uses. It can't do anything about run-time, I guess, unless it dynamically built the functions at run-time, Which is a possibility I suppose but it would require a D compiler inside the code to create the new instance of the template for the new time. fooK though works like any other non-templated function, so it can be used at run time for us. fooK can be generated for us by the compiler if it was "smart enough". It could add type checking and such to remove the string K parameter if it can find out what the type of i is. e.g., use reflection for something like fooK(object i) { string K = typeof(i).stringof; if (K == "int") foo!int(i); else if (K == "float") foo!float(i); else if (K == "string") foo!string(i); else foo!unknown(i); } and, in fact, one could get fancy and use a hash table instead of the if's. Then this would allow one to extend fooK at run-time sort of like oop. (just add your new foo!X to the table and it will be called) The problem for the programmer is that when the number of parameters, the number of if statements grow by #K^n. For 1 parameter it's #K and for 2 it's #K^2 since you'll have to check for all pairs of types. It also makes it hard to maintain the above for new added types. You'll have to do it for every templated function and update all of them every time a new specialization is added. Ok so far? What's cool, if all this makes sense. If fooK is used on "compile time types", it can be optimized out to the static call. So fooK could always be used(we would want to use foo!K for notation, of course) and would function as both a compile time type of templated function and also a run-time function. If foo was implemented with a look up table, then templated functions could be added at run time to extend the behavior. E.g., a plugin could could add a "type handler" to foo!K to handle some new type it's using. Ultimately, what would happen is that templated functions would be "first class citizens" and work not only in compile type but at run-time too. It sort of combines both compile-time reflection(what templates do now) and run-time reflection(what that don't) and the compiler figures it out and keeps track of it all. For the programmer, there would be no difference except for the unknown case. Performance wise, there should be no difference. At compile time, the compiler would do pretty much exactly what it does now, just use the explicitly known type. It would have the same performance as before. But when it can't do this(the types are unknown), it falls back onto the runtime lookup version. Does any of that make sense? ;) The reason one can't implement this as a library function, as far as I know, is because one would have to know all the types the programmer calls foo!T with at "compile-time"(e.g., we would have to have a K, the set of those types). If one could get that information then fooK could be automatically constructed using string mixins. Not sure if optimization would take place, I suppose some static checking could be used to get the compile-time types then a normal lookup table for the others. If such a fooK was known, then programmer would always use that instead and be confident that the compiler would always choose the best situation(if it can use a direct call to the foo!N for some N, then it uses that, if not, it goes through the lookup process). I am making one big assumption here, and that is one can get the type of an object at run-time. I'm not sure if this is possible in D and there maybe other issues with inheritance and such which might cause problems with more complicated cases than I've assumed here. What's more, is with fooK, templated functions can take place as virtual functions! Since fooK encomposes both compile time and dynamic behavior in one) Of course, when used as virtual functions, unless in certain cases when optimization can take place, the run-time behavior will always be used. I doubt anyone would read all so I'll stop now ;)Fact: Templates are static! But templates are still normal functioning code and can actually work dynamically. We can see this by including a D compiler inside an app and have the app re-compile the templates for specific dynamic types. It doesn't need templates to do of course, but it shows that "Templates are static" is not quite as simple as it sounds. But could D actually create some sort of internal expression of D templates as dynamic objects that can be used in run-time? If have a template T that only uses 3 types, then one can simple do something like(pseudo): t(object o) { // use dynamic reflection or whatever to get the type of the object o if (o.type == 'int') return T!(int)(o.value, o.unit); if (o.type == 'float') return T!(float)(o.value, o.unit); return T!(unknown)(o.value, o.unit); } etc... t then is a "dynamic representation" of the static functionality of T. But my guess is that the compiler could essentially do this for us and allow one to use templates in a dynamic way. Most of the time it may not be of much use but, say, if one is interacting with console and T converts a string to a number. T could convert a number with unit such as "3mm" in to some base representation such as 0.003(m). float's are handled differently than ints(rounding depending on some criteria). Maybe we use the T internally and its lightning fast. But for the console case, we have write a dynamic version of the template or create something similar to the code above. But the compiler already knows, at least, some of the times that will be used and it should be able to instantiate the template for each of those types. e.g., if you use T!(float) in the code, it will know to create the "dynamic wrapper" to include that type in the list. If you don't every use T!(float) explicitly then you can simply create a useless statement like {auto x = T!(float)("3.04mm");} which will be optimized but the compiler will then include float as a type used in the dynamic version of T. I may not be making much sense here, but it's the best I'll do. The compiler then can do something like assemble the dynamic version of the template into a new function which can be used at compile time on any object. Using a simple special character could inform the compiler to use the dynamic template version. object u = userInput(); &T(u); // (instead of using t(u)). Here the compiler does all the work for us. If u has type float then this gets effectively dispatched to T!(float)(cast(float)u). If it's an int, then T!(int)(cast(int)u). What it mainly does for us is simplify having to do the cast our self along with the string of if checking for all the types or having to code that works dynamically Or is it impossible for the D compiler to accomplish such a task automatically for us?I think you could also do this using structs with a type enum inside and a union for the data contents. Then you can use string mixins using traits to create generic runtime functions that dispatch to the appropriate method/free standing function on the underlying type. Take a look at Algebraic and Adam Ruppe's jsvar. Some of the genericity can be handled at compile time. For example, I have a templated time series entry struct. A time series is a slice of such structs. An entry definitively has a date, and it may have either a bid and ask or (a close and optionally open, high, low, adjusted close). It has a volume, and may have an open interest. The date may be a year/month/day or it might have a time too. I want to store things efficiently in memory and on disk. So I use hasMember!(typeof(pricebar.date),"hour") to find out if it's an intraday date or daily for example. See Andrei's allocator talk, his code, and maybe that for cerealed. I have a bunch of templated functions that can do useful things with such a time series. But then I want to pass them around wrapped in an a tagged union and generate versions of these functions that can operate on this untemplated struct. (I will wrap the slices, not the entries). That will make it easier to access my work from a dynamic language like Lua. I haven't written this bit yet, but I have done something similar in another context. A degree of misery in trying to figure out std.traits, but that's a one off cost and it goes quicker if you read others code for inspiration. UDAs help in specifying which functions should have wrapped versions generated etc.
Sep 19 2015
On Saturday, 19 September 2015 at 17:46:27 UTC, Adam wrote:I doubt anyone would read all so I'll stop now ;)What you're asking is actually pretty easy. You can just write a function that loops to form the object list yourself or you could make the template add a static module constructor that adds itself to the call list. Then you tag your object with its typeinfo and use that as index into the list.
Sep 19 2015
On Saturday, 19 September 2015 at 18:36:49 UTC, Adam D. Ruppe wrote:On Saturday, 19 September 2015 at 17:46:27 UTC, Adam wrote:It might be, but I don't see how it can be done easily. I don't know enough about D to know if it is implementable as a library solution. Suppose our templated function is auto foo(T)(T x) { return x; } and suppose we specialize it with auto foo(T : object)(T x) { // The static part of the dispatch would look something like static if (is(T == int ) { return foo!int(cast(int)x); .... // This stuff should all be optimized out at compile time if/when the if passes // In fact, it would never be called in this example since int is more special than object. This part is more for when we don't specialize on object and just create a wrapper for the templated function. // If no compile time dispatch, then we need to deal with the unknown type using run-time reflection, we still want to dispatch known types to the foo but no optimization can occur here if (cast(int)x) return foo!int(cast(int)x); // Anymore? else return foo!object(x); // or foo!unknown(x) } foo!object should, in theory, work for all types, run-time or not. But you say constructing foo!object is easy?!?!?! Remember, it has to work in general for all types and number of parameters and should always optimize to a direct template call when the compile time checks are satisfied. Also, the above doesn't allow for dynamic specialization, but that would be easy to add with a lookup table. The main problem I see, which you say is easy, is how to know the list of types to check against? int is the only one used above, but in reality, it depends on context. If the programmer used foo!float, we would have to add that to the dispatch list. We don't want to add it if he doesn't use it because then we are including extra code in the binary that might never be used. It may be used at run time but that's up to the programmer to make sure it is include(by using it at compile time at least ones or through some other method).I doubt anyone would read all so I'll stop now ;)What you're asking is actually pretty easy. You can just write a function that loops to form the object list yourself or you could make the template add a static module constructor that adds itself to the call list. Then you tag your object with its typeinfo and use that as index into the list.
Sep 19 2015
On Saturday, 19 September 2015 at 21:12:52 UTC, Adam wrote:
Suppose our templated function is
auto foo(T)(T x) { return x; }
Why not just use
Sep 21 2015
On Saturday, 19 September 2015 at 17:46:27 UTC, Adam wrote:Does any of that make sense? ;)No. Where exactly do these magical dynamic objects come from? How exactly do you expect user input to generate an object which would require an implementation that was not foreseeable at compile time? This sound like an extreme complication for a problem which is trivial to solve using conventional means, like polymorphism, or a struct with an enum in it. How exactly do you plan to improve on these two approaches? Bit
Sep 19 2015