digitalmars.D - Proposal: __traits(code, ...) and/or .codeof
- F i L (73/73) Mar 22 2012 So the discussions about Attributes and Aspect Oriented
- F i L (6/6) Mar 22 2012 ps. Mono-C#'s NRefactory, and Microsoft .Net's forthcoming Roslyn
- CTFE-4-the-win (3/9) Mar 22 2012 Even if I'm closer to the opposite of a MS-evangelist... color me
- Paulo Pinto (6/16) Mar 22 2012 Same here.
- Felix Hufnagel (2/2) Mar 22 2012 in addition to .codeof, let's think about .astof returning an abstract
- F i L (3/5) Mar 22 2012 I agree, as stated on the IRC, .astof (and therefor
- Timon Gehr (2/7) Mar 22 2012 I think that would necessitate the addition of AST macros.
- F i L (3/4) Mar 22 2012 You mean passing DMD's actual AST tree object? Only a stripped
- Timon Gehr (47/51) Mar 22 2012 We have the macro keyword. I envision something like:
- F i L (7/37) Mar 22 2012 This looks substantially more complicated than what I had in
- Timon Gehr (9/49) Mar 23 2012 The idea is that if you have something of the form:
- F i L (3/5) Mar 22 2012 I haven't really looked at DMD's AST structure, but I assume we
- Paulo Pinto (5/7) Mar 22 2012 Would this not require something similar to the expression trees in .NET...
- deadalnix (2/4) Mar 24 2012 WAY better !
- deadalnix (19/89) Mar 24 2012 As of discussion about properties/attributes, I wanted to do a proposal
- luka8088 (74/148) Oct 09 2012 Is this at least similar to what you had in mind ?
So the discussions about Attributes and Aspect Oriented
Programming (AOP) got me thinking... Basically AOP requires
injecting code fragments together in a comprehensible way.
Similarly, Attributes that go beyond note (such as GC.NoScan)
need similar ability.
D already has the ability to mixin arbitrary code fragments at
compile time, and to process those in useful ways through CTFE.
Which rocks. What it lacks is the ability to reflect upon the
actual source code due to IO limitations of CTFE. So creating a
mixin templates which pieces together a unique object is, to my
knowledge, currently next to impossible (and slow since you'd
have to parse and isolate code in .d file multiple times in a
separate process, then compile again to put it all together).
So, to quote Walter, what compelling features would it bring?
Here's an example of a simple AOP program from the AOP wiki page
(probably not the best implementation, but the concept is there):
struct BankType
{
void transfer() { ... }
void getMoneyBack() { ... }
}
struct Logger
{
void transfer() {
log("transferring money...");
}
void getMoneyBack() {
log("User requested money back");
}
}
and now some magic...
string bankCode(T...)(T aspects) {
auto code = "struct Bank {";
auto members = [__traits(allMembers, Bank)];
foreach (m; members) {
code ~= "void "~m~"() {";
code ~= __traits(getMember, Bank, m).codeof;
foreach (a; aspects) {
if (__traits(hasMember, a, m) {
code ~= __traits(getMember, a, m).codeof;
}
}
code ~= "}"
}
return code ~ "}";
}
mixin template Bank(T...)
{
mixin(bankCode(T));
}
mixin Bank!Logger;
void main() {
auto b = Bank();
b.transfer(); // logs
b.getMoneyBack(); // ditto
}
So this would allow us to make "Compilers" within the Compiler
(Codeception), since we could parse/strip/append any existing
code fragments together in endless combination. Generic
"Builders" could probably be built and put into a std.builder lib
for general use.
One particular use I have in mind is for Behavior Objects (Game
Scripts). Each behavior would hold Property(T) objects which
define per-property, per-state "binding" dependencies (eg.
position.x.bind(other.x, State.Idle)) and execution code. On
release, the Property(T) object would be stripped away (leaving
just T) and it's behavior code "compressed" with others into
optimized functions.
I don't know much about the internals of DMD, so I'm not sure
this is a realistic request, but I think the idea is compelling.
Also, for Attributes I'm not sure this technique is really
applicable. But it's possible that the compiler could exploit
this internally for certain Attributes like GC.whatever
Mar 22 2012
Project are the only comparable infrastructures I can think of with this level of reflection, and they're the foundation to some pretty innovative new development tools. NRefactory: http://wiki.sharpdevelop.net/NRefactory.ashx Roslyn: http://msdn.microsoft.com/en-us/vstudio/hh500769
Mar 22 2012
On Thursday, 22 March 2012 at 16:55:34 UTC, F i L wrote:Roslyn Project are the only comparable infrastructures I can think of with this level of reflection, and they're the foundation to some pretty innovative new development tools. NRefactory: http://wiki.sharpdevelop.net/NRefactory.ashx Roslyn: http://msdn.microsoft.com/en-us/vstudio/hh500769Even if I'm closer to the opposite of a MS-evangelist... color me impressed, I like your idea. +1.
Mar 22 2012
Paulo Pinto <pjmlp progtools.org> Am 22.03.2012 18:34, schrieb CTFE-4-the-win:On Thursday, 22 March 2012 at 16:55:34 UTC, F i L wrote:Same here. Microsoft R&D does lots of cool stuff actually, pity that sometimes they get it so wrong, due to the way the management/marketing works. -- PauloProject are the only comparable infrastructures I can think of with this level of reflection, and they're the foundation to some pretty innovative new development tools. NRefactory: http://wiki.sharpdevelop.net/NRefactory.ashx Roslyn: http://msdn.microsoft.com/en-us/vstudio/hh500769Even if I'm closer to the opposite of a MS-evangelist... color me impressed, I like your idea. +1.
Mar 22 2012
in addition to .codeof, let's think about .astof returning an abstract syntax tree.
Mar 22 2012
On Thursday, 22 March 2012 at 18:06:24 UTC, Felix Hufnagel wrote:in addition to .codeof, let's think about .astof returning an abstract syntax tree.I agree, as stated on the IRC, .astof (and therefor .astof.toString()) is a much better concept.
Mar 22 2012
On 03/22/2012 07:08 PM, F i L wrote:On Thursday, 22 March 2012 at 18:06:24 UTC, Felix Hufnagel wrote:I think that would necessitate the addition of AST macros.in addition to .codeof, let's think about .astof returning an abstract syntax tree.I agree, as stated on the IRC, .astof (and therefor .astof.toString()) is a much better concept.
Mar 22 2012
Timon Gehr wrote:I think that would necessitate the addition of AST macros.You mean passing DMD's actual AST tree object? Only a stripped down version?
Mar 22 2012
On 03/22/2012 08:12 PM, F i L wrote:Timon Gehr wrote:We have the macro keyword. I envision something like: macro replaceAggregate(macro newAggregate, macro loop : foreach(x; aggr){statements}, macro x, macro aggr, macro statements) { foreach(x; newAggregate){statements} } void main(){ int[] a = [1,2,3]; int[] b = [2,3,4]; replaceAggregate(b, foreach(x;a){writeln(x);}); } (The syntax looks horrible, but you get the idea: AST walking by pattern matching) Ideally, such macros would allow introducing local helper variables inside the macro that are not accessible from outside and introducing variables/enums in the context the macro is used. (prefixing identifiers that should interfere with the enclosing context with $ would be an option). Another idea: macro someFancyDSL(string dsl){ mixin(someFancyDSLtoD(dsl); } void main(){ int n = 5; someFancyDSL { import n; let x = 1 : x let y = product (take n) x export y; } writeln(y); } // this would also enable less fancy but useful stuff: macro until(macro condition, string str){ while(!condition){ mixin(str); // string and mixin not strictly necessary, // maybe enable the feature on macro params too } } void main(){ bool done = false; int x; until(done){ done = foo(x++); } } This is just a very rough sketch though, we would need a much more refined design. I think getting the symbol scoping right is most important.I think that would necessitate the addition of AST macros.You mean passing DMD's actual AST tree object? Only a stripped down version?
Mar 22 2012
Timon Gehr wrote:We have the macro keyword. I envision something like: macro replaceAggregate(macro newAggregate, macro loop : foreach(x; aggr){statements}, macro x, macro aggr, macro statements) { foreach(x; newAggregate){statements} } void main(){ int[] a = [1,2,3]; int[] b = [2,3,4]; replaceAggregate(b, foreach(x;a){writeln(x);}); } (The syntax looks horrible, but you get the idea: AST walking by pattern matching)This looks substantially more complicated than what I had in mind. I think it's a great idea, but something that could be added after the initial functionality was there.macro until(macro condition, string str){ while(!condition){ mixin(str); // string and mixin not strictly necessary, // maybe enable the feature on macro params too } } void main(){ bool done = false; int x; until(done){ done = foo(x++); } } This is just a very rough sketch though, we would need a much more refined design. I think getting the symbol scoping right is most important.I'm a bit confused about what's actually going on here, but it certainly looks interesting. What exactly is being passed to "string str" in the macro?
Mar 22 2012
On 03/23/2012 12:33 AM, F i L wrote:Timon Gehr wrote:The idea is that if you have something of the form: identifier(arguments) { body } It would get transformed into: identifier(arguments, q{ body });We have the macro keyword. I envision something like: macro replaceAggregate(macro newAggregate, macro loop : foreach(x; aggr){statements}, macro x, macro aggr, macro statements) { foreach(x; newAggregate){statements} } void main(){ int[] a = [1,2,3]; int[] b = [2,3,4]; replaceAggregate(b, foreach(x;a){writeln(x);}); } (The syntax looks horrible, but you get the idea: AST walking by pattern matching)This looks substantially more complicated than what I had in mind. I think it's a great idea, but something that could be added after the initial functionality was there.macro until(macro condition, string str){ while(!condition){ mixin(str); // string and mixin not strictly necessary, // maybe enable the feature on macro params too } } void main(){ bool done = false; int x; until(done){ done = foo(x++); } } This is just a very rough sketch though, we would need a much more refined design. I think getting the symbol scoping right is most important.I'm a bit confused about what's actually going on here, but it certainly looks interesting. What exactly is being passed to "string str" in the macro?
Mar 23 2012
On Thursday, 22 March 2012 at 18:06:24 UTC, Felix Hufnagel wrote:in addition to .codeof, let's think about .astof returning an abstract syntax tree.I haven't really looked at DMD's AST structure, but I assume we could just duplicate that minus any CTFE stuff or codegen stuff.
Mar 22 2012
Am 22.03.2012 19:06, schrieb Felix Hufnagel:in addition to .codeof, let's think about .astof returning an abstract syntax tree.Would this not require something similar to the expression trees in .NET? http://msdn.microsoft.com/en-us/library/bb397951.aspx -- Paulo
Mar 22 2012
Le 22/03/2012 19:06, Felix Hufnagel a écrit :in addition to .codeof, let's think about .astof returning an abstract syntax tree.WAY better !
Mar 24 2012
Le 22/03/2012 17:00, F i L a écrit :
So the discussions about Attributes and Aspect Oriented Programming
(AOP) got me thinking... Basically AOP requires injecting code fragments
together in a comprehensible way. Similarly, Attributes that go beyond
note (such as GC.NoScan) need similar ability.
D already has the ability to mixin arbitrary code fragments at compile
time, and to process those in useful ways through CTFE. Which rocks.
What it lacks is the ability to reflect upon the actual source code due
to IO limitations of CTFE. So creating a mixin templates which pieces
together a unique object is, to my knowledge, currently next to
impossible (and slow since you'd have to parse and isolate code in .d
file multiple times in a separate process, then compile again to put it
all together).
So, to quote Walter, what compelling features would it bring? Here's an
example of a simple AOP program from the AOP wiki page (probably not the
best implementation, but the concept is there):
struct BankType
{
void transfer() { ... }
void getMoneyBack() { ... }
}
struct Logger
{
void transfer() {
log("transferring money...");
}
void getMoneyBack() {
log("User requested money back");
}
}
and now some magic...
string bankCode(T...)(T aspects) {
auto code = "struct Bank {";
auto members = [__traits(allMembers, Bank)];
foreach (m; members) {
code ~= "void "~m~"() {";
code ~= __traits(getMember, Bank, m).codeof;
foreach (a; aspects) {
if (__traits(hasMember, a, m) {
code ~= __traits(getMember, a, m).codeof;
}
}
code ~= "}"
}
return code ~ "}";
}
mixin template Bank(T...)
{
mixin(bankCode(T));
}
mixin Bank!Logger;
void main() {
auto b = Bank();
b.transfer(); // logs
b.getMoneyBack(); // ditto
}
So this would allow us to make "Compilers" within the Compiler
(Codeception), since we could parse/strip/append any existing code
fragments together in endless combination. Generic "Builders" could
probably be built and put into a std.builder lib for general use.
One particular use I have in mind is for Behavior Objects (Game
Scripts). Each behavior would hold Property(T) objects which define
per-property, per-state "binding" dependencies (eg.
position.x.bind(other.x, State.Idle)) and execution code. On release,
the Property(T) object would be stripped away (leaving just T) and it's
behavior code "compressed" with others into optimized functions.
I don't know much about the internals of DMD, so I'm not sure this is a
realistic request, but I think the idea is compelling. Also, for
Attributes I'm not sure this technique is really applicable. But it's
possible that the compiler could exploit this internally for certain
Attributes like GC.whatever
As of discussion about properties/attributes, I wanted to do a proposal
along that line :
ttribute attributeName(AST ast, other template parameters . . .) {
// Eponymous trick
immutable AST attributeName = process(ast);
auto process(AST ast) {
// Do some CTFEable ast magic here.
return newAst;
}
}
And that would be used that way :
attributeName(other template parameters if it make sense)
void fun() {}
And allow us to do inception within fun.
Additionally, I wanted to propose mixin that return the AST of a string.
This isn't as easy as you think, it have a lot lot of implication about
what you get as string. This is definitively interesting, but require
more work for a realistic proposal.
Mar 24 2012
Is this at least similar to what you had in mind ?
http://dpaste.dzfl.pl/a5dc2875
module program;
import std.stdio;
mixin template BankAccount () {
public int amount;
void deposit (int value) { this.amount += value; }
void withdraw (int value) { this.amount -= value; }
auto currentAmount () { return this.amount; }
}
mixin template Logger () {
void deposit (int value) { writeln("User deposited money"); }
void withdraw (int value) { writeln("User requested money
back"); }
}
void main () {
mixin aspect!("bank", BankAccount, Logger);
bank b1;
bank b2;
b1.deposit(10);
b1.deposit(20);
b1.withdraw(5);
writeln("b1.currentAmount: ", b1.currentAmount);
b2.deposit(50);
b2.withdraw(40);
b2.deposit(100);
writeln("b2.currentAmount: ", b2.currentAmount);
}
// generic code
mixin template aspect (string name, T...) {
template aspectDispatch (string name, uint n) {
import convert = std.conv;
static if (n >= 1)
enum aspectDispatch = ""
~ "import std.traits;\n"
~ "static if (__traits(hasMember, data.aspect_" ~
convert.to!string(n) ~ ", `" ~ name ~ "`))\n"
~ " static if (!is(ReturnType!(data.aspect_" ~
convert.to!string(n) ~ "." ~ name ~ ") == void))\n"
~ " return data.aspect_" ~ convert.to!string(n) ~ "."
~ name ~ "(arguments);\n"
~ " else\n"
~ " data.aspect_" ~ convert.to!string(n) ~ "." ~ name
~ "(arguments);\n"
~ aspectDispatch!(name, n - 1)
;
else
enum aspectDispatch = "";
}
auto code () {
import convert = std.conv;
string ret = ""
~ "struct " ~ name ~ " {\n"
~ " struct aspectData {\n"
;
uint i = 0;
foreach (a; T)
ret ~= " mixin " ~ __traits(identifier, a) ~ " aspect_"
~ convert.to!string(++i) ~ ";\n";
ret ~= ""
~ " }\n"
~ " aspectData data;\n"
~ " auto opDispatch (string fn, args...) (args arguments)
{\n"
~ " mixin(aspectDispatch!(fn, " ~ convert.to!string(i) ~
"));\n"
~ " }\n"
~ "}\n"
;
return ret;
}
mixin(code);
}
On Thursday, 22 March 2012 at 16:00:29 UTC, F i L wrote:
So the discussions about Attributes and Aspect Oriented
Programming (AOP) got me thinking... Basically AOP requires
injecting code fragments together in a comprehensible way.
Similarly, Attributes that go beyond note (such as GC.NoScan)
need similar ability.
D already has the ability to mixin arbitrary code fragments at
compile time, and to process those in useful ways through CTFE.
Which rocks. What it lacks is the ability to reflect upon the
actual source code due to IO limitations of CTFE. So creating a
mixin templates which pieces together a unique object is, to my
knowledge, currently next to impossible (and slow since you'd
have to parse and isolate code in .d file multiple times in a
separate process, then compile again to put it all together).
So, to quote Walter, what compelling features would it bring?
Here's an example of a simple AOP program from the AOP wiki
page (probably not the best implementation, but the concept is
there):
struct BankType
{
void transfer() { ... }
void getMoneyBack() { ... }
}
struct Logger
{
void transfer() {
log("transferring money...");
}
void getMoneyBack() {
log("User requested money back");
}
}
and now some magic...
string bankCode(T...)(T aspects) {
auto code = "struct Bank {";
auto members = [__traits(allMembers, Bank)];
foreach (m; members) {
code ~= "void "~m~"() {";
code ~= __traits(getMember, Bank, m).codeof;
foreach (a; aspects) {
if (__traits(hasMember, a, m) {
code ~= __traits(getMember, a, m).codeof;
}
}
code ~= "}"
}
return code ~ "}";
}
mixin template Bank(T...)
{
mixin(bankCode(T));
}
mixin Bank!Logger;
void main() {
auto b = Bank();
b.transfer(); // logs
b.getMoneyBack(); // ditto
}
So this would allow us to make "Compilers" within the Compiler
(Codeception), since we could parse/strip/append any existing
code fragments together in endless combination. Generic
"Builders" could probably be built and put into a std.builder
lib for general use.
One particular use I have in mind is for Behavior Objects (Game
Scripts). Each behavior would hold Property(T) objects which
define per-property, per-state "binding" dependencies (eg.
position.x.bind(other.x, State.Idle)) and execution code. On
release, the Property(T) object would be stripped away (leaving
just T) and it's behavior code "compressed" with others into
optimized functions.
I don't know much about the internals of DMD, so I'm not sure
this is a realistic request, but I think the idea is
compelling. Also, for Attributes I'm not sure this technique is
really applicable. But it's possible that the compiler could
exploit this internally for certain Attributes like GC.whatever
Oct 09 2012
On Tuesday, 9 October 2012 at 13:28:55 UTC, luka8088 wrote:Is this at least similar to what you had in mind ? [ ..code.. ]Yes, I realized, a bit after I originally posted that, that my suggestion was already possible if BankType & Logger where mixin-templates instead of struct/classes. Thanks for the code example though. I still think an built-in .codeof/.astof would be nice, but what D really needs to achieve this in a syntactically pleasing and powerful way, is 'macro' templates (like Nimrod has) which work on the AST directly. I doubt this is a major concern ATM however.
Oct 09 2012
On Tuesday, 9 October 2012 at 19:29:34 UTC, F i L wrote:On Tuesday, 9 October 2012 at 13:28:55 UTC, luka8088 wrote:My point of making this example was to show that nothing is missing in D itself. You just need to be more creative. If you want to write in a manner more similar to your original example (by that I mean without mixin templates) you can use classes, class methods can be turned to delegates with their context pointer changed before execution, and then you would get the same effect. Also having such syntax could be very confusing because someone could introduce some syntax which is very similar to D but behaves differently and it is embedded in a way that looks just like a D code. Also the idea is to have "// generic code" part in some library and not visible to the *user* so the rest of the code would be syntactically pleasing. If you check current phobos code, you will see that there are some examples of using mixins this way. Please also check the comments on https://github.com/D-Programming-Language/dmd/pull/953 (if you haven't done that already).Is this at least similar to what you had in mind ? [ ..code.. ]Yes, I realized, a bit after I originally posted that, that my suggestion was already possible if BankType & Logger where mixin-templates instead of struct/classes. Thanks for the code example though. I still think an built-in .codeof/.astof would be nice, but what D really needs to achieve this in a syntactically pleasing and powerful way, is 'macro' templates (like Nimrod has) which work on the AST directly. I doubt this is a major concern ATM however.
Oct 11 2012