www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Metaprogramming with D

reply Jan =?UTF-8?B?SMO2bmln?= <hrominium gmail.com> writes:
We have two (little) student projects, which should use D for 
meta-programming/code generation.

More specifically string mixins and templates.

I understand (at least I think so :)) string mixins. The task is 
to create a small internal DSL, which is capable of printing out 
D code, which we then will use as a mixin. So basically using 
CTFE to create code at compile time. (any additional 
sources/ideas are welcome).

The second project considers templates.
My idea was to recreate some template expressions, like we all 
know and love (hate) in C++.
However I am not so sure if that is a really good idea.
As far as I understood it, that is basically what ranges in D do.
So this would just mean "code linear algebra with ranges", which 
I am sure plenty of D's libraries do already (mir?). (that 
doesn't mean, it is not a good learning experience for the 
student).


I would love to hear some opinions on that matter.
Jun 06 2020
next sibling parent mw <mingwu gmail.com> writes:
On Saturday, 6 June 2020 at 09:57:36 UTC, Jan Hönig wrote:

 We have two (little) student projects, which should use D for 
 meta-programming/code generation.

 More specifically string mixins and templates.


I found this doc:

https://github.com/PhilippeSigaud/D-templates-tutorial/blob/master/D-templates-tutorial.md


Really, we should put all these docs in a central place.
Jun 06 2020
prev sibling parent reply FunkyD <MrFunky YourDaddy.com> writes:
On Saturday, 6 June 2020 at 09:57:36 UTC, Jan Hönig wrote:

 We have two (little) student projects, which should use D for 
 meta-programming/code generation.

 More specifically string mixins and templates.

 I understand (at least I think so :)) string mixins. The task 
 is to create a small internal DSL, which is capable of printing 
 out D code, which we then will use as a mixin. So basically 
 using CTFE to create code at compile time. (any additional 
 sources/ideas are welcome).

 The second project considers templates.
 My idea was to recreate some template expressions, like we all 
 know and love (hate) in C++.
 However I am not so sure if that is a really good idea.
 As far as I understood it, that is basically what ranges in D 
 do.
 So this would just mean "code linear algebra with ranges", 
 which I am sure plenty of D's libraries do already (mir?). 
 (that doesn't mean, it is not a good learning experience for 
 the student).


 I would love to hear some opinions on that matter.



D is pretty good for meta-programming. For certain other things 
it is terrible. One of the key features of D is it's meta 
programming.

String mixins simply mix in D code. It lets you build generic D 
code. The big problem here is debugging. There is ZERO ability to 
properly debug string mixins. Well, what it boils down to is 
writing out the string mixin and then debugging that. Visual D 
does have the ability to open up the output and one can try to 
compile such things but it can be a pain for complex cases.

It would be nice if D had a special D code string that the IDE 
could interpret properly.

Templates are just templates, just generic code. D is far better 
than C++ and most other languages. Template are more than ranges. 
Ranges use templates.

Templated code simply means generic code. E.g., rather than 
working with specific integers you work with integer variables.

Generally the type of the variable is undefined it constrained 
but otherwise unknown.

R foo(T)(T a)

R and T are types. We can call foo with any type, it no longer 
requires overloading and all that. But if the code in foo depends 
on specific types then it must be dealt with using various means 
to constrain the type.

D basically tries to resolve things after the fact so

R add(T)(T a, T b)
{
     return a + b;
}

this will attempt to + on a and b after the types are known. If 
they can't be added then an error will occur, which is usually 
cryptic for templates. When using templates one generally also 
cannot debug them as easy. One might get a long error message for 
simple issues. R above is deduced from type of a + b, which 
probably would be a T.

So such a simple templated function, or generic function can be 
used to add doubles, ints, floats, etc... anything with + 
operator.

Further more there are more tools available such as passing 
compile time parameters and such.


One can do quite a lot with D's string mixins and templates but 
the biggest issue is error messages and debugging. Sometimes 
things are not so easy to do with templates or require strange 
workarounds, but otherwise probably 98% of all things can be done.

When one is working with generic types one has to learn more 
machinery such as `is`, traits, etc.

The issue in using D should not be made based on it's 
metaprogramming. It will be a mistake! Now, if the only goal is 
to use D for it's metaprogramming then it is an excellent 
choice... although there are other languages starting to compete 
with D and functional languages probably offer better overall 
cohesiveness if you want to go down the functional 
route(ultimately there is little difference but the syntax and 
some of the semantics seem vastly different).

D, IMO, is not capable of writing sophisticated programs... this 
is why you do not see any. No one writes large commercial apps in 
D. There is not one! The D ecosystems is poorly structured 
compared to the top contenders. D is good for small apps, 
utilities, etc. D can be integrated with other apps though but 
then one loses some of the meta capabilities(since they won't 
translate).
Jun 06 2020
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 6/6/20 5:03 PM, FunkyD wrote:> On Saturday, 6 June 2020 at 09:57:36=20
UTC, Jan H=C3=B6nig wrote:


 D is pretty good for meta-programming. For certain other things it is
 terrible.


I am glad I don't know enough about other technologies to feel that way.


 String mixins simply mix in D code. It lets you build generic D code.
 The big problem here is debugging. There is ZERO ability to properly
 debug string mixins.


"Zero" is too strong an assertion and because of the following, false:

   dmd -mixin=3D<filename> ...


 Well, what it boils down to is writing out the
 string mixin and then debugging that.


When I compile my program and there is an issue with a string mixin,=20
Emacs opens the mixin file and shows me the compilation there. This is=20
because dmd's error include the exact line in the <filename> where my=20
mixin was broken. Not a shiny IDE but still pretty awesome support.


 It would be nice if D had a special D code string that the IDE could
 interpret properly.


I must be misunderstanding you but it must be an IDE limitation because=20
of the following:

   writeln(q{
       void foo() {
       }
     });

My IDE gives me syntax highlighting in that string, so it works somehow.


 D basically tries to resolve things after the fact so

 R add(T)(T a, T b)
 {
      return a + b;
 }

 this will attempt to + on a and b after the types are known. If they
 can't be added then an error will occur, which is usually cryptic for
 templates.


That's why we use template constraints and in some cases 'static assert' =

for custom error messages.


 The issue in using D should not be made based on it's metaprogramming.=



I agree. What feature to use usually emerges by itself. For example,=20
when an algorithm is repeated for many types, it is an opportunity for=20
templates.


 D, IMO, is not capable of writing sophisticated programs...


"Sophisticated" is relative but such strong assertions can be falsified=20
by a single counterexample. For example, Weka's product is very=20
sophisticated and is written in D. And there is nothing extra or missing =

in D that makes it incapable in that regard.


 this is why
 you do not see any.


I think the fact that many smart programmers are hostage to other=20
languages is a stronger reason.


 No one writes large commercial apps in D.


False.


 There is
 not one!


False. And again, even if so, that's not because of D, but because of=20
humans. Can you imagine a CTO, say, in Silicon Valley to have guts to=20
bring D instead of C++? With C++, the CTO will never be blamed; but D,=20
he or she can easily be blamed upon failure. Not because of the=20
technologies but because of politics.


 The D ecosystems is poorly structured compared to the top
 contenders.


Agreed but that is not because D is inferior. Again, that's because=20
people happen to be busy with other technologies.


 D is good for small apps, utilities, etc. D can be
 integrated with other apps though but then one loses some of the meta
 capabilities(since they won't translate).


What would one technology that is good for small code not work for a=20
larger system? Threads, communication e.g. memory mapped files, etc. are =

all there. What magical thing would happen and suddenly D won't work=20
beyond a certain limit? If so, is that any different from any other=20
language?

Ali
Jun 06 2020
next sibling parent Don <d o.n> writes:
On Sunday, 7 June 2020 at 00:45:37 UTC, Ali Çehreli wrote:

 False. And again, even if so, that's not because of D, but 
 because of humans. Can you imagine a CTO, say, in Silicon 
 Valley to have guts to bring D instead of C++? With C++, the 
 CTO will never be blamed; but D, he or she can easily be blamed 
 upon failure. Not because of the technologies but because of 
 politics.


In other words, "Nobody ever got fired for choosing [C++]."
Jun 06 2020
prev sibling next sibling parent FunkyD <MrFunky YourDaddy.com> writes:
On Sunday, 7 June 2020 at 00:45:37 UTC, Ali Çehreli wrote:

 On 6/6/20 5:03 PM, FunkyD wrote:> On Saturday, 6 June 2020 at 
 09:57:36 UTC, Jan Hönig wrote:


 D is pretty good for meta-programming. For certain other


 things it is

 terrible.


 I am glad I don't know enough about other technologies to feel 
 that way.


Ok, that isn't my fault.


 String mixins simply mix in D code. It lets you build generic


 D code.

 The big problem here is debugging. There is ZERO ability to


 properly

 debug string mixins.


 "Zero" is too strong an assertion and because of the following, 
 false:


You know the difference between logic and statistics? Ever heard 
of measure theory?

It's a pretty poor argument to claim that "Oh, logically it is 
not 100% true so it can't be true at all". I'm fine with stating 
something as All if it's true 98%+ of the time when it doesn't 
matter if it's 90% or 100% or anything in between.

If, for example, you had cancer almost all over your body I don't 
think someone you fault you if you said all instead of almost 
all. Pedanticism usually isn't a feature but a fault.





   dmd -mixin=<filename> ...


 Well, what it boils down to is writing out the
 string mixin and then debugging that.


 When I compile my program and there is an issue with a string 
 mixin, Emacs opens the mixin file and shows me the compilation 
 there. This is because dmd's error include the exact line in 
 the <filename> where my mixin was broken. Not a shiny IDE but 
 still pretty awesome support.


 It would be nice if D had a special D code string that the


 IDE could

 interpret properly.


 I must be misunderstanding you but it must be an IDE limitation 
 because of the following:

   writeln(q{
       void foo() {
       }
     });

 My IDE gives me syntax highlighting in that string, so it works 
 somehow.


Syntax highlighting is entirely different from semantic analysis. 
Try getting a line number of an error a multi-line string...


 D basically tries to resolve things after the fact so

 R add(T)(T a, T b)
 {
      return a + b;
 }

 this will attempt to + on a and b after the types are known.


 If they

 can't be added then an error will occur, which is usually


 cryptic for

 templates.


 That's why we use template constraints and in some cases 
 'static assert' for custom error messages.


Yeah, that is definitely not the way to go... I could just throw 
rocks at my computer until it works too. Just because one can do 
it doesn't mean one should. In this case one is nearly forced.

Claiming that because we have the most primitive form of a 
solution makes up for having a good solution is pretty primitive.




 The issue in using D should not be made based on it's


 metaprogramming.

 I agree. What feature to use usually emerges by itself. For 
 example, when an algorithm is repeated for many types, it is an 
 opportunity for templates.


 D, IMO, is not capable of writing sophisticated programs...


 "Sophisticated" is relative but such strong assertions can be 
 falsified by a single counterexample. For example, Weka's 
 product is very sophisticated and is written in D. And there is 
 nothing extra or missing in D that makes it incapable in that 
 regard.


As you said, sophisticated is quite relative. It seems you seem 
to think this is a math pissing contest. Anyone that makes a 
decision based on logical facts rather than statistical facts for 
statistical problems is quite illogical.


 this is why
 you do not see any.


 I think the fact that many smart programmers are hostage to 
 other languages is a stronger reason.


Maybe, maybe not... sounds like an excuse. Maybe they are 
actually rational beings who understanding that moving to an 
unpredictable and rather incompletely language is not worth the 
trouble?


 No one writes large commercial apps in D.


 False.


Prove it. Saying something is false is not the same as it being 
false. Since you want to play hardball:

Name one commercial D app that has over 1 million in revenue and 
is used by 10k people. Just one. I'll let you play your games. 
Here I'm talking about an actual app that is entirely written in 
D.  You cannot claim an app is written in D when it just imports 
some D library. Nor are we talking about ad revenue generating 
software. I'm talking about something normal people use as an 
application... like a word processor, painting app, web browser, 
etc. Something external users  use, not a internal to a company.


Just one, come on, you said it was easy!



 There is
 not one!


 False. And again, even if so, that's not because of D, but 
 because of humans. Can you imagine a CTO, say, in Silicon 
 Valley to have guts to bring D instead of C++? With C++, the 
 CTO will never be blamed; but D, he or she can easily be blamed 
 upon failure. Not because of the technologies but because of 
 politics.


Now you are claiming that they don't exist. Which is it? Do they 
exist or do they not exist and they don't exist for some ad hoc 
theories you have?


 The D ecosystems is poorly structured compared to the top
 contenders.


 Agreed but that is not because D is inferior. Again, that's 
 because people happen to be busy with other technologies.


No, that is PRECISELY because D is inferior. You like to move the 
goal posts so you can get the 10 yard 3 pointer when ever you 
like.


 D is good for small apps, utilities, etc. D can be
 integrated with other apps though but then one loses some of


 the meta

 capabilities(since they won't translate).


 What would one technology that is good for small code not work 
 for a larger system? Threads, communication e.g. memory mapped 
 files, etc. are all there. What magical thing would happen and 
 suddenly D won't work beyond a certain limit? If so, is that 
 any different from any other language?


It's called dependencies and cohesion. It's the same reason why 
free energy doesn't scale up. Do you believe in free energy? I 
saw a guy run a toaster off the heat of his balls after rough sex 
with an expensive hooker... maybe we can use this technology to 
provide all the energy to the planet for 10M years with just one 
case? After all, it's just a scale problem. Right? Not just a 
scale problem but surely it must be linear... you know, y = mx + 
b?
Jun 08 2020
prev sibling parent reply Jan =?UTF-8?B?SMO2bmln?= <hrominium gmail.com> writes:
On Sunday, 7 June 2020 at 00:45:37 UTC, Ali Çehreli wrote:

   dmd -mixin=<filename> ...


thanks for the tip!



   writeln(q{
       void foo() {
       }
     });


What is the name of this `q` thing?
How do i find it? Are there any recent tutorials on it?
Jun 08 2020
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On 08.06.20 16:41, Jan Hönig wrote:

 On Sunday, 7 June 2020 at 00:45:37 UTC, Ali Çehreli wrote:


[...]

   writeln(q{
       void foo() {
       }
     });


 
 What is the name of this `q` thing?
 How do i find it? Are there any recent tutorials on it?


https://dlang.org/spec/lex.html#token_strings
https://ddili.org/ders/d.en/literals.html#ix_literals.q{}
Jun 08 2020
parent reply ag0aep6g <anonymous example.com> writes:
On 08.06.20 16:45, ag0aep6g wrote:

 On 08.06.20 16:41, Jan Hönig wrote:

 On Sunday, 7 June 2020 at 00:45:37 UTC, Ali Çehreli wrote:


 [...]

   writeln(q{
       void foo() {
       }
     });


 What is the name of this `q` thing?
 How do i find it? Are there any recent tutorials on it?


 
 https://dlang.org/spec/lex.html#token_strings
 https://ddili.org/ders/d.en/literals.html#ix_literals.q{}


Hm. That second link is somewhat malformed. Better one:
https://ddili.org/ders/d.en/literals.html#ix_literals.q%7B%7D
Jun 08 2020
parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 6/8/20 7:50 AM, ag0aep6g wrote:


 https://ddili.org/ders/d.en/literals.html#ix_literals.q%7B%7D


Thank you.

I am biased but I like my :) index of the book, where all such syntax 
items appear:

   https://ddili.org/ders/d.en/ix.html

Ali
Jun 09 2020
parent ag0aep6g <anonymous example.com> writes:
On 09.06.20 20:16, Ali Çehreli wrote:

 I am biased but I like my :) index of the book, where all such syntax 
 items appear:
 
    https://ddili.org/ders/d.en/ix.html


Oh yeah. It's how I got the link. You might want to fix some of the 
URLs, though. Characters like '{' or '[' need to be percent-encoded.
Jun 09 2020
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 8 June 2020 at 14:41:55 UTC, Jan Hönig wrote:

 What is the name of this `q` thing?


It is just a string that looks like code.
Jun 08 2020
prev sibling parent tastyminerals <tastyminerals gmail.com> writes:
On Monday, 8 June 2020 at 14:41:55 UTC, Jan Hönig wrote:

 On Sunday, 7 June 2020 at 00:45:37 UTC, Ali Çehreli wrote:

   dmd -mixin=<filename> ...


 thanks for the tip!



   writeln(q{
       void foo() {
       }
     });


 What is the name of this `q` thing?
 How do i find it? Are there any recent tutorials on it?


Ali's online book consolidates a lot of D language knowledge like 
this. I forgot about token string literals myself but then 
remembered it was in his book.
Jun 09 2020