digitalmars.D.dtl - A 'nother way to do this
- Georg Wrede (78/78) Nov 29 2005 Currently, the DMD template system is making amazing progress!
- Don Clugston (35/83) Nov 30 2005 I wonder how much this is possible without close integration into the
- Ivan Senji (26/30) Nov 30 2005 Why would anyone wish to shoot this idea down? It is IMO a good idea, I
- Larry Evans (26/45) Dec 02 2005 [snip]
- Ivan Senji (9/39) Dec 02 2005 Didn't really look ad that code because I don't like
- Don Clugston (21/57) Dec 05 2005 This sort of thing can already be done with D templates.
- Bruno Medeiros (7/26) Dec 08 2005 Hum.. I'm somewhat confused. How exactly did you do that? Could you show...
- Don Clugston (40/63) Dec 08 2005 It's not pretty, but it's trivial, really. Currently done with a hack
- Larry Evans (7/29) Dec 09 2005 Don, since I've already done my own hack
- Don Clugston (13/19) Dec 12 2005 I agree entirely. I've done enough of it too. That was just a temporary
- Larry Evans (22/36) Dec 12 2005 Thanks very much Don. I've been thinking about moving to D, and
Currently, the DMD template system is making amazing progress! Being an old warrior, I still haven't entirely put my original ideas 6' under. What I'm currently brewing, is a proof-of-concept precompiler. My idea is that (or actually was, before these huge leaps Walter and Don have made), with a suitable precompiler, one should be able to test different ideas about "I want this/that in D", and see for himself whether they crumble once put to actual use. Heh, forcing this thing to everybody who claims to have "this just amazing and _essential_ idea to incorporate in D", would reduce some garbage bandwidth usage from the newsgroups -- without anybody having to shoot 'em down. There are several different ways one could use this for the advancement of D development: - a "greyhound race electric rabbit", to "show what the D template syntax _could_ be capable of" - have the folks with Unbelievable ideas test their ideas themselves (before wasting bandwidth here) - a general tool to see just _how_ far one can go with a decent meta language - specifically, to see if we can develop a _clear_ and _obvious_ syntax to do hopefully _everything_ that's ever been done in BOOST - if folks find ways to do things with this that can't be done with the current (or next month's) D template syntax, this might give more rigorous food for thought (both for and against) such ideas - a theoretical discussion on _what_ should, could, would, be doable with a meta language -- the tool is here Having said all that, this being a preprocessor, makes it possible for _any_ language to use _precisely this_ preprocessor to the "up to now, in that language, intractable problems". Unfortunately. Now, the aim is to incorporate any lessons from this (if there ever will be any), into the DMD compiler itself! (Of course, filtered by The Crowd, and whatever Walter consideres appropriate.) That is, the existing DMD compiler front end. My dream is that such would give D even more power than anything achievable with preprocessor only. --- The current state is, it's half-way between planning and early alpha. Any ideas are welcome. Even if somebody wants to shoot this down, no hard feelings! What I'm currently looking for with this post: - hard problems or examples, to challenge if such can be done with this approach - everyday examples of "ordinary preprocessor things" you've missed from the times you used the C(++) preprocessor - implicit template instantiation - entire classes, and even whole families of classes, wrapped in a single template - templates parameterized on things other than the function parameter types - examples of "what can currently be done with C++ and not (yet) with D" - what would really be cool, but not even dreamable (until today, be it reasonable or not) - corporate internal Best Practices examples - any (I mean _any at all_) dreams or ideas, or even off-hand thoughts, on how the D syntax might be bettered (or worsened :-) ), now that we (RSN) have a tool to check those out with. Such examples might include, ideas from languages currently not thought at all implementable in a strongly typed imperative language, weak SQL things, a robot language (a bit like Logo), logic programming, parser design incorporated right into the language (a la Spirit), etc. --- The ultimate test would be to - reprogram as much as possible from www.boost.org/libs/libraries.htm - getting _all_ of that to work would kill the bystanders - getting it to work so that an average programmer would understand some the metaprogramming involved, would kill the Boost Gurus - getting publicity for such, would just kill <your suggestion here> --- I know, I know, this sounds like the writings of the deranged white haired mad scientist. (I'm not gonna argue, heck, even I am not sure of this thing. But it does look promising.) Whatever we do, we should get things done so fast that the C++ Design-Committee-Bureaucracy won't be able to follow us until we've taken over the world. :-) Even if they'll use our precompiler as-is. (Which currently would be no problem, unfortunately!) --- So, let's hear some crap, please!
Nov 29 2005
Don Clugston <dac nospam.com.au> Georg Wrede wrote:Currently, the DMD template system is making amazing progress! Being an old warrior, I still haven't entirely put my original ideas 6' under. What I'm currently brewing, is a proof-of-concept precompiler. My idea is that (or actually was, before these huge leaps Walter and Don have made), with a suitable precompiler, one should be able to test different ideas about "I want this/that in D", and see for himself whether they crumble once put to actual use. - specifically, to see if we can develop a _clear_ and _obvious_ syntax to do hopefully _everything_ that's ever been done in BOOST - if folks find ways to do things with this that can't be done with the current (or next month's) D template syntax, this might give more rigorous food for thought (both for and against) such ideas - a theoretical discussion on _what_ should, could, would, be doable with a meta language -- the tool is here Having said all that, this being a preprocessor, makes it possible for _any_ language to use _precisely this_ preprocessor to the "up to now, in that language, intractable problems". Unfortunately. Now, the aim is to incorporate any lessons from this (if there ever will be any), into the DMD compiler itself! (Of course, filtered by The Crowd, and whatever Walter consideres appropriate.) That is, the existing DMD compiler front end. My dream is that such would give D even more power than anything achievable with preprocessor only.I wonder how much this is possible without close integration into the language (proper language parsing). I think that the D template system is getting very close to implementing all of the functionality of the C preprocessor. So, it would need to be quite a bit more sophisticated than the C preprocessor.What I'm currently looking for with this post: - hard problems or examples, to challenge if such can be done with this approach - everyday examples of "ordinary preprocessor things" you've missed from the times you used the C(++) preprocessor(1) Token pasting. Suppose there was a keyword '__identifier', with syntax: __identifier(StringLiteral) legal anywhere that an identifier is legal. int __identifier("abc"); would be equivalent to: int abc; BUT: the string literal would need to be evaluated before the token pasting occurs, otherwise there's no point. (2) Stringizing. The inverse of the above. int abc; char [] name = __nameof(abc); is equivalent to: char [] name = "abc"; If 'abc' is an alias, it should use the ultimate name.(Ie, it should use the name which is printed out in an error message).- implicit template instantiation - examples of "what can currently be done with C++ and not (yet) with D"Using my hack to put delegates into C++ (www.codeproject.com/cpp/fastdelegate.asp). you can retrofit polymorphism, along the lines of Chris Diggin's interface idea. See my comment in http://www.codeproject.com/cpp/retrofitpolymorphism2.asp My method requires implicit template instantiation. (Would be *much* nicer in D). But even better would be to set up proper interfaces, the way he does it with C macros. I think this is something you could do with a preprocessor such as you describe.- entire classes, and even whole families of classes, wrapped in a single template - templates parameterized on things other than the function parameter typesThe ultimate test would be to - reprogram as much as possible from www.boost.org/libs/libraries.htm - getting _all_ of that to work would kill the bystanders - getting it to work so that an average programmer would understand some the metaprogramming involved, would kill the Boost GurusThe Boost Metaprogramming Library looks like an easy target, already. It's horribly hacky, and you can't actually do very much with it. Other than implicit template instantiation, I don't think the C++ template system has many advantages left, compared to D.
Nov 30 2005
Georg Wrede wrote:The current state is, it's half-way between planning and early alpha. Any ideas are welcome. Even if somebody wants to shoot this down, no hard feelings!Why would anyone wish to shoot this idea down? It is IMO a good idea, I tryed to make something like this happen too. But as Don Clugston said in his reply to your post a simple preprocesor is hardly going to be enough. If you wish to do anything more complex with syntax you should have a real parser. How is your knowledge on language grammars? I have a working grammar-based parser as a project on dsource and a rather imperfect D-grammar (it doesn't really parse the intire D language). With a fixed grammar and a good lexer hooked up it could easily extend todays D. I have played with it more in the past (too little time now because of the exams), and there are nice things that can be done with it. Just as an example I implemented, a new type of statement in D that looked like this: int[10] numbers; (each[int index] numbers) = index; and it translated to foreach(int index, typeof(numbers[index]) __item; numbers) { __item = index; } And another example was aplply-to-all-like syntax numbers.{writefln(it); it += 2; }; //print each number and add 2 to it. So if you feel this could help you in any way feel free to use it. It is rather hard to start using but it gets easier later. Anyway, good luck with your project!
Nov 30 2005
On 11/30/2005 02:47 AM, Ivan Senji wrote:Georg Wrede wrote:[snip]The current state is, it's half-way between planning and early alpha.How is your knowledge on language grammars? I have a working grammar-based parser as a project on dsource and a rather imperfect D-grammar (it doesn't really parse the intire D language). With a fixed grammar and a good lexer hooked up it could easily extend todays D.[snip]Just as an example I implemented, a new type of statement in D that looked like this: int[10] numbers; (each[int index] numbers) = index; and it translated to foreach(int index, typeof(numbers[index]) __item; numbers) { __item = index; }What I like to see is: (each[int index] int const[10] numbers) = index; IOW, use it to intialize the numbers. This would enable the compiler to know what the values are at each element of numbers; hence, enable better speed optimization. For a concrete example, take a look at little.funvec_.zip in http://tinyurl.com/8bgy5 . In file: little.funvec_constcvec.cpp there's a vector initialized with function pointers. The function pointers have to be entered by the programmer. OTOH, in file: little.funvec_foreach.cpp the function pointers are calculated in a templated CTOR which uses a boost::mpl::for_each to fill the vector. Obviously, funvec_constcvec is faster than funvec_foreach even though they both do *essentially* the same thing, fill a constant vector containing functions pointers (just like a virtual function table ). Could this this be done with your parser? Speed was of prime importance to the original poster of the thread which prompted the little.funvec_* code: http://groups.google.com/group/comp.lang.c++.moderated/msg/23c238cdda9cd878
Dec 02 2005
Larry Evans wrote:What I like to see is: (each[int index] int const[10] numbers) = index; IOW, use it to intialize the numbers. This would enable the compiler to know what the values are at each element of numbers; hence, enable better speed optimization.Interesting.For a concrete example, take a look at little.funvec_.zip in http://tinyurl.com/8bgy5 . In file: little.funvec_constcvec.cpp there's a vector initialized with function pointers. The function pointers have to be entered by the programmer. OTOH, in file: little.funvec_foreach.cpp the function pointers are calculated in a templated CTOR which uses a boost::mpl::for_each to fill the vector. Obviously, funvec_constcvec is faster than funvec_foreach even though they both do *essentially* the same thing, fill a constant vector containing functions pointers (just like a virtual function table ).Didn't really look ad that code because I don't like the way it is structured (unreadable, sorry) :), but...Could this this be done with your parser?The question is "is there a way to write this in D (using templates)?" If the answer is yes, then it can be done, the only problem is you have to write code in the grammar definition that describes that translation to normal D. But it takes free time, and that I do not have :(
Dec 02 2005
Larry Evans wrote:On 11/30/2005 02:47 AM, Ivan Senji wrote:This sort of thing can already be done with D templates. ------- template square(int n) { const int square = n*n; } const [] numbers = generateArray!(10, square); // const int [] numbers = [0, 1, 4, 9, 16, 25, ...] const [] smallfactorials = generateArray!(20, factorial); // const ulong [] smallfactorials = [0, 1, 2, 6, 20, ... ] ------- const T [] generateArray!(int limits, const T metafunction!(int) f ) is a template I've made, it can accept metafunctions returning any constant type. (For example, f could return generateArray!(5, g) where g returns a string literal). Note that it's also possible for the array length itself to be calculated at compile time. BTW, anyone have a better name for generateArray ? I've also tried calling it "generateLookupTable" and "staticforeach", neither of which seem appropriate.Georg Wrede wrote:[snip]The current state is, it's half-way between planning and early alpha.How is your knowledge on language grammars? I have a working grammar-based parser as a project on dsource and a rather imperfect D-grammar (it doesn't really parse the intire D language). With a fixed grammar and a good lexer hooked up it could easily extend todays D.[snip]Just as an example I implemented, a new type of statement in D that looked like this: int[10] numbers; (each[int index] numbers) = index; and it translated to foreach(int index, typeof(numbers[index]) __item; numbers) { __item = index; }What I like to see is: (each[int index] int const[10] numbers) = index; IOW, use it to intialize the numbers. This would enable the compiler to know what the values are at each element of numbers; hence, enable better speed optimization.
Dec 05 2005
Don Clugston wrote:
This sort of thing can already be done with D templates.
-------
template square(int n)
{
const int square = n*n;
}
const [] numbers = generateArray!(10, square);
// const int [] numbers = [0, 1, 4, 9, 16, 25, ...]
const [] smallfactorials = generateArray!(20, factorial);
// const ulong [] smallfactorials = [0, 1, 2, 6, 20, ... ]
-------
const T [] generateArray!(int limits, const T metafunction!(int) f )
is a template I've made, it can accept metafunctions
returning any constant type. (For example, f could return
generateArray!(5, g) where g returns a string literal).
Hum.. I'm somewhat confused. How exactly did you do that? Could you show
the code(for generateArray)?
--
Bruno Medeiros - CS/E student
"Certain aspects of D are a pathway to many abilities some consider to
be... unnatural."
Dec 08 2005
Bruno Medeiros wrote:Don Clugston wrote:It's not pretty, but it's trivial, really. Currently done with a hack (like a C++ preprocessor). --------------- private import hack.hackgenerate; template generateArray(alias generator, int n) { const typeof(generator!(0)) [] generateArray = hackgenerate!(n, generator); } ---------------- I autogenerate a file with the first few entries looking like this: (hackgenerate.d). Has entries for num=1 to 128. I have to use this hack, because D doesn't have proper array literals yet, so arrays can't be concatentated or pulled apart at compile time. As you can imagine, the D code for making the hackgenerate file is very short. -------------- // Autogenerated file - do not edit template hackgenerate(int num, alias gen) { pragma(msg, "Table is too large!"); static assert(0); } template hackgenerate(int num :0, alias gen) { const typeof(gen!(0)) [] hackgenerate = [ gen!(0) ]; } template hackgenerate(int num :1, alias gen) { const typeof(gen!(0)) [] hackgenerate = [ gen!(0), gen!(1) ]; } template hackgenerate(int num :2, alias gen) { const typeof(gen!(0)) [] hackgenerate = [ gen!(0), gen!(1), gen!(2) ]; } --------------This sort of thing can already be done with D templates. ------- template square(int n) { const int square = n*n; } const [] numbers = generateArray!(10, square); // const int [] numbers = [0, 1, 4, 9, 16, 25, ...] const [] smallfactorials = generateArray!(20, factorial); // const ulong [] smallfactorials = [0, 1, 2, 6, 20, ... ] ------- const T [] generateArray!(int limits, const T metafunction!(int) f ) is a template I've made, it can accept metafunctions returning any constant type. (For example, f could return generateArray!(5, g) where g returns a string literal).Hum.. I'm somewhat confused. How exactly did you do that? Could you show the code(for generateArray)?
Dec 08 2005
On 12/09/2005 01:36 AM, Don Clugston wrote:
[snip]
It's not pretty, but it's trivial, really. Currently done with a hack
(like a C++ preprocessor).
---------------
private import hack.hackgenerate;
template generateArray(alias generator, int n)
{
const typeof(generator!(0)) [] generateArray = hackgenerate!(n,
generator);
}
----------------
I autogenerate a file with the first few entries looking like this:
(hackgenerate.d). Has entries for num=1 to 128.
I have to use this hack, because D doesn't have proper array literals
yet, so arrays can't be concatentated or pulled apart at compile time.
As you can imagine, the D code for making the hackgenerate file is very
short.
--------------
// Autogenerated file - do not edit
Don, since I've already done my own hack
(in PP_RECFLD.zip of
http://boost-consulting.com/vault/index.php?&directory=Template%20Metaprogramming
) I was hoping to avoid this. I'd really like something
that can just be done in the language itself without any preprocessing.
Dec 09 2005
Don, since I've already done my own hack (in PP_RECFLD.zip of http://boost-consulting.com/vault/index.php?&directory=Templa e%20Metaprogramming ) I was hoping to avoid this. I'd really like something that can just be done in the language itself without any preprocessing.I agree entirely. I've done enough of it too. That was just a temporary workaround. The main need for the the hack is the absence of array literals, a feature which Walter has said is coming (but not for a while, because it's tricky to implement). Right now, it is possible to avoid the hack for the special case of strings. But unfortunately, it needs a second hack, because const char [] ~= const char is not evaluated at compile time. You need to convert char into char[]. I'm hoping that this limitation will be removed very soon, ie this year, so that we can at least do one special case perfectly. Once we have that, array literals will become a more fun problem for Walter to work on, so it will probably gain increased priority.
Dec 12 2005
On 12/12/2005 02:12 AM, Don Clugston wrote:Thanks very much Don. I've been thinking about moving to D, and this is one more reason to do so (when the new feature is added). However, I'm still wondering if something like mpl::fold is possible, in particular, with placeholders (i.e. mpl::arg). I've needed this for proto_static_disp.zip in: http://boost-consulting.com/vault/ in the 'Template Metaprogramming' directory. I'd also like to see if the policy_ptr can't be somehow incorporated into D as a replacement for the current GC. AFAICT, it would involve adding to each stack frame something like the selected_fields_description_of<FieldsVisitor, RECORD >, defined in: http://cvs.sourceforge.net/viewcvs.py/boost-sandbox/boost-sandbox/boost/fields_visitor/fields_visitor.hpp where RECORD corresponds to the stack frame and FieldsVisitor corresponds to the type of garbage collection needed. But, of course, this is all based on smart_ptr's, which, as pointed out in another post to this newsgroup, D doesn't handle to well because it has no override for operator= :( Oh well. -best regards, LarryDon, since I've already done my own hack (in PP_RECFLD.zip of http://boost-consulting.com/vault/index.php?&directory=Templa e%20Metaprogramming ) I was hoping to avoid this. I'd really like something that can just be done in the language itself without any preprocessing.I agree entirely. I've done enough of it too. That was just a temporary workaround. The main need for the the hack is the absence of array literals, a feature which Walter has said is coming (but not for a while, because it's tricky to implement).
Dec 12 2005