www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What are AST Macros?

reply Jonathan M Davis <jmdavisprog gmail.com> writes:
Periodically, the term AST macros come up in the discussions here as a possible 
addition to the language - they even get discussed briefly in TDPL as a
possible 
replacement for template mxins. However, I don't think that I've ever seen an 
actual explanation for what exactly is meant by the term AST macro. This raises 
two questions

1. What are AST macros and how do they differ from C-style macros? We obviously 
aren't going to be adding macros like that to D, since that would be dangerous. 
But what would a macro be then if it's not a textual replacement? The best I
can 
think of would be that you'd have to indicate the macros to be replaced by 
surrounding them with macro() or something rather than letting any and all text 
that matches  be replaced. So, I really don't know what AST macros are supposed 
to be other than they're macros of some kind and better than C-style macros.

2. What does AST stand for? The best that I can come up with for what it could 
stand for would be Abstract Syntax Tree, which is a nice, Computer Sciency, 
compiler-related term, but I haven't a clue how that would relate to macros.
So, 
maybe an answer to the first question would answer this one as well, but an 
explanation would be nice.

- Jonathan M Davis
Jul 08 2010
next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Jonathan M Davis" <jmdavisprog gmail.com> wrote in message 
news:mailman.301.1278636124.24349.digitalmars-d puremagic.com...

 Periodically, the term AST macros come up in the discussions here as a 
 possible
 addition to the language - they even get discussed briefly in TDPL as a 
 possible
 replacement for template mxins. However, I don't think that I've ever seen 
 an
 actual explanation for what exactly is meant by the term AST macro. This 
 raises
 two questions

 1. What are AST macros and how do they differ from C-style macros? We 
 obviously
 aren't going to be adding macros like that to D, since that would be 
 dangerous.
 But what would a macro be then if it's not a textual replacement? The best 
 I can
 think of would be that you'd have to indicate the macros to be replaced by
 surrounding them with macro() or something rather than letting any and all 
 text
 that matches  be replaced. So, I really don't know what AST macros are 
 supposed
 to be other than they're macros of some kind and better than C-style 
 macros.

 2. What does AST stand for? The best that I can come up with for what it 
 could
 stand for would be Abstract Syntax Tree, which is a nice, Computer 
 Sciency,
 compiler-related term, but I haven't a clue how that would relate to 
 macros. So,
 maybe an answer to the first question would answer this one as well, but 
 an
 explanation would be nice.

 - Jonathan M Davis


Short answer: Reflection/introspection on steroids.

Slightly longer answer:

You're right, AST is "Abstract Syntax Tree". As for what that actually 
means: You know the HTML DOM? If you have the DOM for a specific HTML page, 
that DOM is a (mutable) representation of the HTML page's AST. If you 
manipulate the DOM (ie, DHTML), then that's basically AST macros.

Except for one thing: HTML is merely markup and D is fully turing-complete. 
So imagine that the browser *also* has a DOM for the JavaScript itself, so 
the JavaScript can manipulate itself through a DOM...that's what we're 
talking about (although it'd be compile-time in our case). So yea...clearly 
powerful stuff.

Purely hypothetical example:

Suppose you have this code:

module foo;
void main(string[] args)
{
    int i;
    i = 7 + 4 * 3;
}

The AST might look something roughly like this (I'm representing it here in 
XML, so I don't have to try to draw an ASCII-art graph):

<module name="foo">
    <function name="main" type="void">
        <params>
            <param name="args" type="string[]" />
        </params>
        <body>
            <declaration name="i" type="int" />
            <assignment>
                <lvalue>
                    <variable>i</variable>
                </lvalue>
                <rvalue>
                    <add>
                        <literal>7</literal>
                        <multiply>
                            <literal>4</literal>
                            <literal>3</literal>
                        </multiply>
                    </add>
                </rvalue>
            </assignment>
        </body>
    </function>
</module>

(Or it might look totally different then that, depending on how the AST 
system is defined, but you get the idea.)

Now imagine an API that exposes a mutable version of that tree (or something 
similar to it) at compile-time.

Also, if you know what a parse tree is, an AST is similar to that, but 
generally a bit more human-understandable and semantics-oriented, whereas a 
parse tree is purely a literal representation of the formal syntax - no 
semantics all, and rather awkward to deal with.

Another interesting thing: I've never really used LISP, but my understanding 
is that this sort of thing basically forms the heart of LISP and is the 
reason LISP is considered so powerful. (AIUI, LISP was invented purely as a 
hypothetical example of this exact sort of thing, and then somebody went and 
actually implemented a real interpreter for it.)
Jul 08 2010
parent "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:i15uqd$1917$1 digitalmars.com...

 You're right, AST is "Abstract Syntax Tree". As for what that actually


Also, I'm very close to a new release of my Goldie language-processing 
project ( http://www.dsource.org/projects/goldie ). There are some tools in 
it ("Parse" and a customized version of "JsonViewer") that make it *very* 
easy to inspect the parse tree of a source file (although not D yet, sadly) 
and get a feel for how parse trees work. But it also has a fairly simple 
documentation-generating tool (really just an HTML pre-processor) that uses 
its own language, and when you use it, it can output the AST of the 
documentation source. Then you can use that customized "JsonViewer" to get a 
good feel for the AST, and how that works, and how it compares to the parse 
tree. You you might want to keep an eye out on D.announce for that. I'm 
hoping it'll be within the next few weeks, although that depends on 
real-world work...
Jul 08 2010
prev sibling next sibling parent reply "Robert Jacques" <sandford jhu.edu> writes:
On Thu, 08 Jul 2010 20:43:39 -0400, Jonathan M Davis  
<jmdavisprog gmail.com> wrote:


 Periodically, the term AST macros come up in the discussions here as a  
 possible
 addition to the language - they even get discussed briefly in TDPL as a  
 possible
 replacement for template mxins. However, I don't think that I've ever  
 seen an
 actual explanation for what exactly is meant by the term AST macro. This  
 raises
 two questions

 1. What are AST macros and how do they differ from C-style macros? We  
 obviously
 aren't going to be adding macros like that to D, since that would be  
 dangerous.
 But what would a macro be then if it's not a textual replacement? The  
 best I can
 think of would be that you'd have to indicate the macros to be replaced  
 by
 surrounding them with macro() or something rather than letting any and  
 all text
 that matches  be replaced. So, I really don't know what AST macros are  
 supposed
 to be other than they're macros of some kind and better than C-style  
 macros.

 2. What does AST stand for? The best that I can come up with for what it  
 could
 stand for would be Abstract Syntax Tree, which is a nice, Computer  
 Sciency,
 compiler-related term, but I haven't a clue how that would relate to  
 macros. So,
 maybe an answer to the first question would answer this one as well, but  
 an
 explanation would be nice.

 - Jonathan M Davis


Check out Walter's slides and/or talk from the D conference.  
(http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D.announce&article_id=12555)
AST does stand for abstract syntax tree and they are much more like Lisp  
macros as opposed to the C preprocessor.
Jul 08 2010
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Robert Jacques" <sandford jhu.edu> wrote in message 
news:op.vfjy3jlo26stm6 sandford...

 Check out Walter's slides and/or talk from the D conference. 
 (http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D.announce&article_id=12555)
 AST does stand for abstract syntax tree and they are much more like Lisp 
 macros as opposed to the C preprocessor.


Hmm, looks like the D version is much more template-mixin/C-macro-like than 
what I made it sound like in a previous branch of the thread. Basically like 
template mixins, but where the arguments are snippets of code, and overloads 
are resolved by pattern-matching the syntax of their arguments.
Jul 08 2010
prev sibling parent reply retard <re tard.com.invalid> writes:
Thu, 08 Jul 2010 21:43:57 -0400, Robert Jacques wrote:


 Check out Walter's slides and/or talk from the D conference.
 (http://www.digitalmars.com/webnews/newsgroups.php?


art_group=digitalmars.D.announce&article_id=12555)

 AST does stand for abstract syntax tree and they are much more like Lisp
 macros as opposed to the C preprocessor.


Too bad nothing happened. He promised AST macros, but it will probably 
still take several years before we have a preliminary implementation and 
maybe 10-20 years before a written spec & almsot bug-free implementation.
Jul 10 2010
parent reply Don <nospam nospam.com> writes:
retard wrote:

 Thu, 08 Jul 2010 21:43:57 -0400, Robert Jacques wrote:
 

 Check out Walter's slides and/or talk from the D conference.
 (http://www.digitalmars.com/webnews/newsgroups.php?


 art_group=digitalmars.D.announce&article_id=12555)

 AST does stand for abstract syntax tree and they are much more like Lisp
 macros as opposed to the C preprocessor.


 
 Too bad nothing happened. He promised AST macros, but it will probably 
 still take several years before we have a preliminary implementation and 
 maybe 10-20 years before a written spec & almsot bug-free implementation.


Part of what happened was that at the conference, I showed that the 
functionality of string mixins was a superset of what was planned for 
AST macros. Since that time, there has not been any kind of proposal, or 
even a concept. So it's a bit meaningless to talk about "AST macros" 
right now as if they were a planned-but-not-yet-implemented feature.
Jul 11 2010
next sibling parent reply Philippe Sigaud <philippe.sigaud gmail.com> writes:
Don wrote:


 Part of what happened was that at the conference, I showed that the
 functionality of string mixins was a superset of what was planned for AST
 macros. Since that time, there has not been any kind of proposal, or even a
 concept. So it's a bit meaningless to talk about "AST macros" right now as
 if they were a planned-but-not-yet-implemented feature.


That's interesting. Do you have a link or any text I could read on that?
String mixins sure are powerful, but I can't get ird of a feeling of
'cheating' when using them. Maybe with some kind of string interpolation
they could be made more palatable to some?


Philippe
Jul 11 2010
next sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Sun, 11 Jul 2010 14:26:51 +0200, Philippe Sigaud wrote:


 Don wrote:
 
 

 Part of what happened was that at the conference, I showed that the
 functionality of string mixins was a superset of what was planned for
 AST macros. Since that time, there has not been any kind of proposal,
 or even a concept. So it's a bit meaningless to talk about "AST macros"
 right now as if they were a planned-but-not-yet-implemented feature.


 That's interesting. Do you have a link or any text I could read on that?
 String mixins sure are powerful, but I can't get ird of a feeling of
 'cheating' when using them. Maybe with some kind of string interpolation
 they could be made more palatable to some?


I find that using token strings, q{ ... }, rather than ordinary "..." 
strings, sometimes makes string mixins feel less like a hack.  Especially 
considering that my editor (vim) highlights token strings like ordinary 
code -- or, more likely, it doesn't recognise them as strings. ;)

-Lars
Jul 11 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-07-11 08:47:26 -0400, "Lars T. Kyllingstad" 
<public kyllingen.NOSPAMnet> said:


 On Sun, 11 Jul 2010 14:26:51 +0200, Philippe Sigaud wrote:
 

 That's interesting. Do you have a link or any text I could read on that?
 String mixins sure are powerful, but I can't get ird of a feeling of
 'cheating' when using them. Maybe with some kind of string interpolation
 they could be made more palatable to some?


 
 I find that using token strings, q{ ... }, rather than ordinary "..."
 strings, sometimes makes string mixins feel less like a hack.  Especially
 considering that my editor (vim) highlights token strings like ordinary
 code -- or, more likely, it doesn't recognise them as strings. ;)


Personally, I find it *more* like a hack. q{...} is just a way to 
disguise a string as not being one, it's like using a second hack to 
better hide the first hack. But it's too hacks instead of one, and thus 
it's more obscure.

That said, I don't feel like I'm cheating when using string mixins. I 
find them a quite good substitute to AST macros. And perhaps string 
mixins are simpler too: you don't have to learn a new macro syntax, you 
just manipulate strings. Though I'm still waiting for the day we can 
use string mixins in expressions to do things like this:

	int num = 1;
	string result = substitute!"Number: $num";
	assert(result == "Number: 1");

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Jul 11 2010
next sibling parent reply "Rory McGuire" <rmcguire neonova.co.za> writes:
On Sun, 11 Jul 2010 15:29:36 +0200, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2010-07-11 08:47:26 -0400, "Lars T. Kyllingstad"  
 <public kyllingen.NOSPAMnet> said:


 On Sun, 11 Jul 2010 14:26:51 +0200, Philippe Sigaud wrote:


 That's interesting. Do you have a link or any text I could read on  
 that?
 String mixins sure are powerful, but I can't get ird of a feeling of
 'cheating' when using them. Maybe with some kind of string  
 interpolation
 they could be made more palatable to some?


  I find that using token strings, q{ ... }, rather than ordinary "..."
 strings, sometimes makes string mixins feel less like a hack.   
 Especially
 considering that my editor (vim) highlights token strings like ordinary
 code -- or, more likely, it doesn't recognise them as strings. ;)


 Personally, I find it *more* like a hack. q{...} is just a way to  
 disguise a string as not being one, it's like using a second hack to  
 better hide the first hack. But it's too hacks instead of one, and thus  
 it's more obscure.

 That said, I don't feel like I'm cheating when using string mixins. I  
 find them a quite good substitute to AST macros. And perhaps string  
 mixins are simpler too: you don't have to learn a new macro syntax, you  
 just manipulate strings. Though I'm still waiting for the day we can use  
 string mixins in expressions to do things like this:

 	int num = 1;
 	string result = substitute!"Number: $num";
 	assert(result == "Number: 1");


someone already made something like that, I forget where it was. Its old  
now.
Jul 11 2010
parent reply Chad J <chadjoan __spam.is.bad__gmail.com> writes:
On 07/11/2010 12:58 PM, Rory McGuire wrote:

 On Sun, 11 Jul 2010 15:29:36 +0200, Michel Fortin
 <michel.fortin michelf.com> wrote:
 

 On 2010-07-11 08:47:26 -0400, "Lars T. Kyllingstad"
 <public kyllingen.NOSPAMnet> said:


 On Sun, 11 Jul 2010 14:26:51 +0200, Philippe Sigaud wrote:


 That's interesting. Do you have a link or any text I could read on
 that?
 String mixins sure are powerful, but I can't get ird of a feeling of
 'cheating' when using them. Maybe with some kind of string
 interpolation
 they could be made more palatable to some?


  I find that using token strings, q{ ... }, rather than ordinary "..."
 strings, sometimes makes string mixins feel less like a hack. 
 Especially
 considering that my editor (vim) highlights token strings like ordinary
 code -- or, more likely, it doesn't recognise them as strings. ;)


 Personally, I find it *more* like a hack. q{...} is just a way to
 disguise a string as not being one, it's like using a second hack to
 better hide the first hack. But it's too hacks instead of one, and
 thus it's more obscure.

 That said, I don't feel like I'm cheating when using string mixins. I
 find them a quite good substitute to AST macros. And perhaps string
 mixins are simpler too: you don't have to learn a new macro syntax,
 you just manipulate strings. Though I'm still waiting for the day we
 can use string mixins in expressions to do things like this:

     int num = 1;
     string result = substitute!"Number: $num";
     assert(result == "Number: 1");


 
 someone already made something like that, I forget where it was. Its old
 now.


I doubt it.  Not unless they did something more like this:

int num = 1;
string result = mixin(substitute!"Number: $num");
assert(result == "Number: 1");

It'd be really nice to have some sugar for string mixin usage.  It could
be tied into templates to do some really awesome things.
Some prior discussion:
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=105781
Jul 11 2010
parent "Nick Sabalausky" <a a.a> writes:
"Chad J" <chadjoan __spam.is.bad__gmail.com> wrote in message 
news:i1d8j4$1347$1 digitalmars.com...

 On 07/11/2010 12:58 PM, Rory McGuire wrote:

 On Sun, 11 Jul 2010 15:29:36 +0200, Michel Fortin
 <michel.fortin michelf.com> wrote:

     int num = 1;
     string result = substitute!"Number: $num";
     assert(result == "Number: 1");


 someone already made something like that, I forget where it was. Its old
 now.


 I doubt it.  Not unless they did something more like this:

 int num = 1;
 string result = mixin(substitute!"Number: $num");
 assert(result == "Number: 1");

 It'd be really nice to have some sugar for string mixin usage.  It could
 be tied into templates to do some really awesome things.
 Some prior discussion:
 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=105781


Yea, I was really disappointed the only thing that came out of that was an 
improvement in instantiating template mixins, but not string mixins. I use 
string mixins so much more than template ones.
Jul 11 2010
prev sibling next sibling parent Philippe Sigaud <philippe.sigaud gmail.com> writes:
On Sun, Jul 11, 2010 at 18:58, Rory McGuire <rmcguire neonova.co.za> wrote:


 On Sun, 11 Jul 2010 15:29:36 +0200, Michel Fortin <
 michel.fortin michelf.com> wrote:


        int num = 1;
        string result = substitute!"Number: $num";
        assert(result == "Number: 1");


 someone already made something like that, I forget where it was. Its old
 now.


Doesn't Format (in std.metastrings) cover part of this?

string result = Format!("Numbers: %s", to!string(num));

I remember finding it cumbersome to use. For one, it could easily take any
type for argument, and call to!string on them internally. Why bothering me
with this?
But maybe I wasn't using it the right way.

Hmm, I think what I'd like is some numbered substituting scheme:

string result = Substitute!("static if (is (typeof(%1) t == %2!U, U)) {
                                                         alias U Result;
                                                    else
                                                        alias typeof(%1)
Result;",
                                                  a,b);


I agree with Rory than someone already did something similar. But I guess
now it can be done with CTFE on strings. A CT replace.


Philippe
Jul 11 2010
prev sibling parent "Rory McGuire" <rmcguire neonova.co.za> writes:
On Sun, 11 Jul 2010 21:55:30 +0200, Philippe Sigaud  
<philippe.sigaud gmail.com> wrote:


 On Sun, Jul 11, 2010 at 18:58, Rory McGuire <rmcguire neonova.co.za>  
 wrote:

 On Sun, 11 Jul 2010 15:29:36 +0200, Michel Fortin  
 <michel.fortin michelf.com> wrote:

        int num = 1;
        string result = substitute!"Number: $num";
        assert(result == "Number: 1");


 someone already made something like that, I forget where it was. Its  
 old now.


 Doesn't Format (in std.metastrings) cover part of this?

 string result = Format!("Numbers: %s", to!string(num));

 I remember finding it cumbersome to use. For one, it could easily take  
 any type for argument, and call to!string on them internally. Why  

bothering me with this?


 But maybe I wasn't using it the right way.

 Hmm, I think what I'd like is some numbered substituting scheme:

 string result = Substitute!("static if (is (typeof(%1) t == %2!U, U)) {
                                                         alias U Result;
                                                     else
                                                        alias typeof(%1)  
 Result;",                                                  a,b);


 I agree with Rory than someone already did something similar. But I  
 guess now it can be done with CTFE on strings. A CT replace.


 Philippe



I think I found what I was thinking of (not sure its the same one):
http://www.digitalmars.com/d/archives/digitalmars/D/PHP-style_embedded_variables_print_statements_54053.html

-Rory
Jul 11 2010
prev sibling parent reply Don <nospam nospam.com> writes:
Philippe Sigaud wrote:

 Don wrote:
 
 
     Part of what happened was that at the conference, I showed that the
     functionality of string mixins was a superset of what was planned
     for AST macros. Since that time, there has not been any kind of
     proposal, or even a concept. So it's a bit meaningless to talk about
     "AST macros" right now as if they were a
     planned-but-not-yet-implemented feature.
 
 
 That's interesting. Do you have a link or any text I could read on that? 


Not really, it was just casual discussion on a whiteboard after the 
conference. I did post some example code on the newsgroup (announce, I 
think) which did lexing + parsing + basic semantic analysis in 300 lines 
of code. It gives an idea of what AST macros would need to be able to do.


 String mixins sure are powerful, but I can't get ird of a feeling of 
 'cheating' when using them. Maybe with some kind of string interpolation 
 they could be made more palatable to some?


There's little doubt that it's the part of the language which needs the 
most attention in the long term. But the way forward is not at all 
obvious. So it's been deferred.
Jul 11 2010
parent reply pillsy <pillsbury gmail.com> writes:
== Quote from Don (nospam nospam.com)'s article


 Philippe Sigaud wrote:


[...]

 String mixins sure are powerful, but I can't get ird of a feeling of
 'cheating' when using them. Maybe with some kind of string interpolation
 they could be made more palatable to some?





 There's little doubt that it's the part of the language which needs the
 most attention in the long term. But the way forward is not at all
 obvious. So it's been deferred.


I'm not clear on how one is supposed to go about making the equivalent of
hygienic macros using just
string mixins and CTFE.

Cheers,
Pillsy
Jul 11 2010
parent reply retard <re tard.com.invalid> writes:
Sun, 11 Jul 2010 22:03:56 +0000, pillsy wrote:


 == Quote from Don (nospam nospam.com)'s article
 

 Philippe Sigaud wrote:


 [...]

 String mixins sure are powerful, but I can't get ird of a feeling of
 'cheating' when using them. Maybe with some kind of string
 interpolation they could be made more palatable to some?




 

 There's little doubt that it's the part of the language which needs the
 most attention in the long term. But the way forward is not at all
 obvious. So it's been deferred.


 
 I'm not clear on how one is supposed to go about making the equivalent
 of hygienic macros using just string mixins and CTFE.


You can create a trial-and-error gensym function with compile time 
introspection and CTFE/templates. :o) It's very elegant. CTFE solves all 
your AST macro use cases.
Jul 11 2010
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 07/11/2010 06:57 PM, retard wrote:

 Sun, 11 Jul 2010 22:03:56 +0000, pillsy wrote:


 == Quote from Don (nospam nospam.com)'s article


 Philippe Sigaud wrote:


 [...]

 String mixins sure are powerful, but I can't get ird of a feeling of
 'cheating' when using them. Maybe with some kind of string
 interpolation they could be made more palatable to some?





 There's little doubt that it's the part of the language which needs the
 most attention in the long term. But the way forward is not at all
 obvious. So it's been deferred.


 I'm not clear on how one is supposed to go about making the equivalent
 of hygienic macros using just string mixins and CTFE.


 You can create a trial-and-error gensym function with compile time
 introspection and CTFE/templates. :o) It's very elegant. CTFE solves all
 your AST macro use cases.


Wrong. The correct answer is magic.
Jul 11 2010
parent Don <nospam nospam.com> writes:
Ellery Newcomer wrote:

 On 07/11/2010 06:57 PM, retard wrote:

 Sun, 11 Jul 2010 22:03:56 +0000, pillsy wrote:


 == Quote from Don (nospam nospam.com)'s article


 Philippe Sigaud wrote:


 [...]

 String mixins sure are powerful, but I can't get ird of a feeling of
 'cheating' when using them. Maybe with some kind of string
 interpolation they could be made more palatable to some?





 There's little doubt that it's the part of the language which needs the
 most attention in the long term. But the way forward is not at all
 obvious. So it's been deferred.


 I'm not clear on how one is supposed to go about making the equivalent
 of hygienic macros using just string mixins and CTFE.


 You can create a trial-and-error gensym function with compile time
 introspection and CTFE/templates. :o) It's very elegant. CTFE solves all
 your AST macro use cases.


 
 Wrong. The correct answer is magic.


Yes, and it has to be that way. The AST macros which people have been 
talking about are magical.
Jul 12 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 11 Jul 2010 07:01:49 -0400, Don <nospam nospam.com> wrote:


 retard wrote:

 Thu, 08 Jul 2010 21:43:57 -0400, Robert Jacques wrote:


 Check out Walter's slides and/or talk from the D conference.
 (http://www.digitalmars.com/webnews/newsgroups.php?


 art_group=digitalmars.D.announce&article_id=12555)

 AST does stand for abstract syntax tree and they are much more like  
 Lisp
 macros as opposed to the C preprocessor.


  Too bad nothing happened. He promised AST macros, but it will probably  
 still take several years before we have a preliminary implementation  
 and maybe 10-20 years before a written spec & almsot bug-free  
 implementation.


 Part of what happened was that at the conference, I showed that the  
 functionality of string mixins was a superset of what was planned for  
 AST macros. Since that time, there has not been any kind of proposal, or  
 even a concept. So it's a bit meaningless to talk about "AST macros"  
 right now as if they were a planned-but-not-yet-implemented feature.


The problem I see with mixins is they are ugly.  The difference between  
mixins and most other parts of the language is the syntax is still  
super-basic.  It would be the equivalent of having to call functions with  
callfunction(funcname, params).  What we need is a way to make mixins look  
like normal expressions.

I proposed at one time to use macro to define easily callable mixins.  It  
was something along the lines of:

macro log(level, x) mixin("if(" ~ level ~ " == log.level){log.output(" ~ x  
~ ")}");

which makes it possible to have lazy logging without the lazy keyword and  
without the ugliness of mixins at the call site.

If people agree that AST macros are superseded by mixins, then why not  
take over the macro keyword (which if restricted to AST macros is  
effectively a dead keyword), and make programmer's lives and generic  
programming easier?

-Steve
Jul 12 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 If people agree that AST macros are superseded by mixins,


String mixins are a hack, just a bit better than C preprocessor (because they
are scoped), they are not a replacement of clean macros.

Bye,
bearophile
Jul 12 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 12 Jul 2010 08:12:31 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:

 If people agree that AST macros are superseded by mixins,


 String mixins are a hack, just a bit better than C preprocessor (because  
 they are scoped), they are not a replacement of clean macros.


Interesting statement, can you back it up? :)  What can you do with a  
macro that you can't do with a mixin?

-Steve
Jul 12 2010
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 07/12/2010 07:30 AM, Steven Schveighoffer wrote:

 On Mon, 12 Jul 2010 08:12:31 -0400, bearophile
 <bearophileHUGS lycos.com> wrote:


 Steven Schveighoffer:

 If people agree that AST macros are superseded by mixins,


 String mixins are a hack, just a bit better than C preprocessor
 (because they are scoped), they are not a replacement of clean macros.


 Interesting statement, can you back it up? :) What can you do with a
 macro that you can't do with a mixin?

 -Steve


I think the big thing about macros is you don't have to worry about 
lexing and parsing.

if <A> is of the form (Assignable, Assignable, ... ),
and <B> of the form (AssignExp, AssignExp, ... )

how do you readily rewrite

mixin("<A> = <B>;");

to

(<A>[0] = <B>[0], <A>[1] = <B>[1], ... )

without a metric shit ton of library support?

With the most bare bones of AST macros (maybe a ctfe function that gets 
passed a AST* and returns a AST* or something equivalent), it's pretty 
trivial. And you don't need to wrap it in a string literal.
Jul 12 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 12 Jul 2010 09:33:34 -0400, Ellery Newcomer  
<ellery-newcomer utulsa.edu> wrote:


 On 07/12/2010 07:30 AM, Steven Schveighoffer wrote:

 On Mon, 12 Jul 2010 08:12:31 -0400, bearophile
 <bearophileHUGS lycos.com> wrote:


 Steven Schveighoffer:

 If people agree that AST macros are superseded by mixins,


 String mixins are a hack, just a bit better than C preprocessor
 (because they are scoped), they are not a replacement of clean macros.


 Interesting statement, can you back it up? :) What can you do with a
 macro that you can't do with a mixin?

 -Steve


 I think the big thing about macros is you don't have to worry about  
 lexing and parsing.

 if <A> is of the form (Assignable, Assignable, ... ),
 and <B> of the form (AssignExp, AssignExp, ... )

 how do you readily rewrite

 mixin("<A> = <B>;");

 to

 (<A>[0] = <B>[0], <A>[1] = <B>[1], ... )

 without a metric shit ton of library support?

 With the most bare bones of AST macros (maybe a ctfe function that gets  
 passed a AST* and returns a AST* or something equivalent), it's pretty  
 trivial. And you don't need to wrap it in a string literal.


But is that common?  Is there a common need to parse mixin strings and  
replace with other strings?  It seems like the means that AST macros  
operate, but is that the ultimate goal?  Even if it is, can't a string  
mixin system do exactly that?

You say a ton of library support is bad, but why is that?  If the end  
result is the same, what does it matter if you implement it via a library  
vs. the compiler?  Also, to me, it's trivial to understand string  
manipulation where the end result is the language I already know, vs.  
having to learn how the compiler represents syntax.

I haven't really used AST macros.  Does anyone have a good practical  
example of how it's used in other languages to compare with string  
mixins?  Your example seems trivially solved via simple library functions  
such as split or using regex.

-Steve
Jul 12 2010
parent reply pillsy <pillsbury gmail.com> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article

 On Mon, 12 Jul 2010 09:33:34 -0400, Ellery Newcomer
 <ellery-newcomer utulsa.edu> wrote:


[...]

 I think the big thing about macros is you don't have to
 worry about lexing and parsing.





 if <A> is of the form (Assignable, Assignable, ... ),
 and <B> of the form (AssignExp, AssignExp, ... )





 how do you readily rewrite





 mixin("<A> = <B>;");





 to





 (<A>[0] = <B>[0], <A>[1] = <B>[1], ... )





 without a metric shit ton of library support?





 With the most bare bones of AST macros (maybe a ctfe
 function that gets passed a AST* and returns a AST*
 or something equivalent), it's pretty trivial. And
 you don't need to wrap it in a string literal.





 But is that common? Is there a common need to parse mixin
 strings and replace with other strings?


In my experience with Lisp macros (which are fundamentally AST
macros) doing that sort of thing is fairly common when it's easy.


 Even if it is, can't a string
 mixin system do exactly that?


I think it can, actually. Having library functions pick up the
slack may not be a terrible idea at all. Parsing a string into an
AST is just a function, transforming that AST into another AST is a
function, and transforming the resulting AST back into a string
that gets mixed in is a third function.

I see two problems with this plan, both which may be addressable
without too much trouble.

One problem is that string mixins look terrible, what with them
being string literals. Having some sort of syntactic support for
using macros would be a major improvement. I liked your "macro"
keyword suggestion from elsethread.

It requires a lot of straight-up hackery. Consider retard's
suggested way for creating a unique name. Being able to do this is
crucial to writing macros that simultaneously do non-trivial things
and are robust, and the inability to create unique names is one
reason C preprocessor macros are so lame. It would be far nicer if
you could use some sort of __traits form to get a unique name (and
then that could be wrapped in a string-mixin macro)!


 Also, to me, it's trivial to understand string manipulation
 where the end result is the language I already know, vs.
 having to learn how the compiler represents syntax.


String manipulation scales very poorly, it's error prone, and it's
tedious.

Cheers,
Pillsy
Jul 12 2010
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/12/2010 09:57 AM, pillsy wrote:

 == Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article

 On Mon, 12 Jul 2010 09:33:34 -0400, Ellery Newcomer
 <ellery-newcomer utulsa.edu>  wrote:


 [...]

 I think the big thing about macros is you don't have to
 worry about lexing and parsing.





 if<A>  is of the form (Assignable, Assignable, ... ),
 and<B>  of the form (AssignExp, AssignExp, ... )





 how do you readily rewrite





 mixin("<A>  =<B>;");





 to





 (<A>[0] =<B>[0],<A>[1] =<B>[1], ... )





 without a metric shit ton of library support?





 With the most bare bones of AST macros (maybe a ctfe
 function that gets passed a AST* and returns a AST*
 or something equivalent), it's pretty trivial. And
 you don't need to wrap it in a string literal.





 But is that common? Is there a common need to parse mixin
 strings and replace with other strings?


 In my experience with Lisp macros (which are fundamentally AST
 macros) doing that sort of thing is fairly common when it's easy.


 Even if it is, can't a string
 mixin system do exactly that?


 I think it can, actually. Having library functions pick up the
 slack may not be a terrible idea at all. Parsing a string into an
 AST is just a function, transforming that AST into another AST is a
 function, and transforming the resulting AST back into a string
 that gets mixed in is a third function.

 I see two problems with this plan, both which may be addressable
 without too much trouble.

 One problem is that string mixins look terrible, what with them
 being string literals. Having some sort of syntactic support for
 using macros would be a major improvement. I liked your "macro"
 keyword suggestion from elsethread.

 It requires a lot of straight-up hackery. Consider retard's
 suggested way for creating a unique name. Being able to do this is
 crucial to writing macros that simultaneously do non-trivial things
 and are robust, and the inability to create unique names is one
 reason C preprocessor macros are so lame. It would be far nicer if
 you could use some sort of __traits form to get a unique name (and
 then that could be wrapped in a string-mixin macro)!


 Also, to me, it's trivial to understand string manipulation
 where the end result is the language I already know, vs.
 having to learn how the compiler represents syntax.


 String manipulation scales very poorly, it's error prone, and it's
 tedious.

 Cheers,
 Pillsy


Though I sympathise with all of the above, I should add that I have 
looked at languages that feature AST macros (Lisp, Scheme, Dylan) and 
such macros are difficult to create and read as well. We're not looking 
at day and night improvement there.

Andrei
Jul 12 2010
prev sibling parent reply retard <re tard.com.invalid> writes:
Mon, 12 Jul 2010 14:57:35 +0000, pillsy wrote:


 == Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article

 On Mon, 12 Jul 2010 09:33:34 -0400, Ellery Newcomer
 <ellery-newcomer utulsa.edu> wrote:


 [...]

 I think the big thing about macros is you don't have to worry about
 lexing and parsing.




 

 if <A> is of the form (Assignable, Assignable, ... ), and <B> of the
 form (AssignExp, AssignExp, ... )




 

 how do you readily rewrite




 

 mixin("<A> = <B>;");




 

 to




 

 (<A>[0] = <B>[0], <A>[1] = <B>[1], ... )




 

 without a metric shit ton of library support?




 

 With the most bare bones of AST macros (maybe a ctfe function that
 gets passed a AST* and returns a AST* or something equivalent), it's
 pretty trivial. And you don't need to wrap it in a string literal.




 

 But is that common? Is there a common need to parse mixin strings and
 replace with other strings?


 
 In my experience with Lisp macros (which are fundamentally AST macros)
 doing that sort of thing is fairly common when it's easy.
 

 Even if it is, can't a string
 mixin system do exactly that?


 
 I think it can, actually. Having library functions pick up the slack may
 not be a terrible idea at all. Parsing a string into an AST is just a
 function, transforming that AST into another AST is a function, and
 transforming the resulting AST back into a string that gets mixed in is
 a third function.


Yes, this would be possible if the dmd compiler didn't crash, leak memory 
etc. The quality is so shitty that I'd say that implementing a compile 
time AST parser for D 2 is almost impossible. And what's worse is that 
the language is changing constantly so the AST parser meta-library would 
need to be updated once in a while. This is great news for all the fans 
of NIH syndrome.


 
 I see two problems with this plan, both which may be addressable without
 too much trouble.
 
 One problem is that string mixins look terrible, what with them being
 string literals. Having some sort of syntactic support for using macros
 would be a major improvement. I liked your "macro" keyword suggestion
 from elsethread.
 
 It requires a lot of straight-up hackery. Consider retard's suggested
 way for creating a unique name. Being able to do this is crucial to
 writing macros that simultaneously do non-trivial things and are robust,
 and the inability to create unique names is one reason C preprocessor
 macros are so lame. It would be far nicer if you could use some sort of
 __traits form to get a unique name (and then that could be wrapped in a
 string-mixin macro)!
 

 Also, to me, it's trivial to understand string manipulation where the
 end result is the language I already know, vs. having to learn how the
 compiler represents syntax.


 
 String manipulation scales very poorly, it's error prone, and it's
 tedious.


Like Andrei said, real AST macros can also be hairy, but they _are_ an 
improvement over string based macros in many cases. Lisp users have 50+ 
years of experience with meta-programming. You're mostly talking with 
amateurs here. Of course they don't see the value of true macros. One 
fundamental prerequisite when discussing macros is that you've actually 
used them in at least ONE other language.
Jul 13 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/13/2010 10:43 AM, retard wrote:

 Yes, this would be possible if the dmd compiler didn't crash, leak memory
 etc. The quality is so shitty that I'd say that implementing a compile
 time AST parser for D 2 is almost impossible.


Well it should be said that the quality of the compiler has dramatically 
improved over the past months. I think it would be worth writing a 
tokenizer for starters.


 And what's worse is that
 the language is changing constantly so the AST parser meta-library would
 need to be updated once in a while. This is great news for all the fans
 of NIH syndrome.


The rate of language changes has slowed down very visibly since TDPL has 
come out.


 String manipulation scales very poorly, it's error prone, and it's
 tedious.


 Like Andrei said, real AST macros can also be hairy, but they _are_ an
 improvement over string based macros in many cases. Lisp users have 50+
 years of experience with meta-programming. You're mostly talking with
 amateurs here. Of course they don't see the value of true macros. One
 fundamental prerequisite when discussing macros is that you've actually
 used them in at least ONE other language.


Agreed. Reminds me of the Go programmers who don't need generics :o). 
The Sapir-Whorf theory may or may not apply to natural language, but has 
a visible presence in programming languages.


Andrei
Jul 13 2010
parent reply Chris Katko <ckatko gmail.com> writes:
Sorry if this is "re-opening" an old thread, but did anything 
come from this and DIP50? It seems like a really interesting 
concept and this thread was one of the first results for a Google 
search.

Thanks.
Apr 06 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, April 06, 2018 20:33:10 Chris Katko via Digitalmars-d wrote:

 Sorry if this is "re-opening" an old thread, but did anything
 come from this and DIP50? It seems like a really interesting
 concept and this thread was one of the first results for a Google
 search.


D does not have any kind of AST macros and likely never will. Walter is
completely against the idea - though I'd have to go digging through
newsgroup's history to find posts where he talked about it to give the exact
reasons. It's been a while since they were discussed last, and I don't
recall them at the moment. Either way, DIP 50 was part of the old DIP
process where DIPs were not always really reviewed like they should have
been, so it was never officially accepted or rejected. I don't know if
Walter or Andrei have even ever looked at it. And for any DIP to be accepted
now, it must go through the new DIP process: https://github.com/dlang/DIPs

So, regardless of what Walter thinks of AST macros, for them to make it into
the language, someone will have to submit a DIP through the new DIP process,
and no one has done that yet. However, based on what Walter has said in the
past, I would fully expect such a DIP to be rejected.

- Jonathan M Davis
Apr 06 2018
parent reply Zach Tollen <reachzach gggmail.com> writes:
On Friday, 6 April 2018 at 21:23:00 UTC, Jonathan M Davis wrote:

 D does not have any kind of AST macros and likely never will. 
 Walter is completely against the idea - though I'd have to go 
 digging through newsgroup's history to find posts where he 
 talked about it to give the exact reasons. It's been a while 
 since they were discussed last, and I don't recall them at the 
 moment.


I think Walter's reason was that such macros would hide too many 
idiosyncrasies in how they were programmed, such that a lot of 
code which seems simple on the surface will actually obfuscate 
complicated and arbitrary macro-programming patterns. Thus, code 
that uses them will become much less maintainable, because it is 
liable to do so many different and hidden things. Also, the tasks 
for which AST-macros would typically be used are already largely 
accommodated by templates and other features. Thus, the real need 
for them isn't that high.
Apr 06 2018
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 6 April 2018 at 21:45:45 UTC, Zach Tollen wrote:

 I think Walter's reason was that such macros would hide too 
 many idiosyncrasies in how they were programmed, such that a 
 lot of code which seems simple on the surface will actually 
 obfuscate complicated and arbitrary macro-programming patterns. 
 Thus, code that uses them will become much less maintainable, 
 because it is liable to do so many different and hidden things. 
 Also, the tasks for which AST-macros would typically be used 
 are already largely accommodated by templates and other 
 features. Thus, the real need for them isn't that high.


I think it's time to revisit this.

The reason being that templates are only well suited to very 
specific types of meta-programs which have a low degree of 
parameterization and a low degree of abstraction.

Using templates to introspect and manipulate types is like using 
a hammmer's flat back to remove a nail.
It _can_ be done but with an absurd amount of work.
You just have to remove all of the wall around the nail by 
pounding it until the wall has around the nail is disintegrated :)

This is not an exaggeration.

Templates used for introspection (or anything else really that's 
modestly complex) are equally hard to reason about for compilers 
and for programmers. I guess programmers have an advantage when 
it comes to _efficient_ pattern recognition.
Apr 09 2018
next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Monday, 9 April 2018 at 15:30:33 UTC, Stefan Koch wrote:

 On Friday, 6 April 2018 at 21:45:45 UTC, Zach Tollen wrote:

 I think Walter's reason was that such macros would hide too 
 many idiosyncrasies in how they were programmed, such that a 
 lot of code which seems simple on the surface will actually 
 obfuscate complicated and arbitrary macro-programming 
 patterns. Thus, code that uses them will become much less 
 maintainable, because it is liable to do so many different and 
 hidden things. Also, the tasks for which AST-macros would 
 typically be used are already largely accommodated by 
 templates and other features. Thus, the real need for them 
 isn't that high.


 I think it's time to revisit this.

 The reason being that templates are only well suited to very 
 specific types of meta-programs which have a low degree of 
 parameterization and a low degree of abstraction.

 Using templates to introspect and manipulate types is like 
 using a hammmer's flat back to remove a nail.
 It _can_ be done but with an absurd amount of work.
 You just have to remove all of the wall around the nail by 
 pounding it until the wall has around the nail is disintegrated 
 :)

 This is not an exaggeration.

 Templates used for introspection (or anything else really 
 that's modestly complex) are equally hard to reason about for 
 compilers and for programmers. I guess programmers have an 
 advantage when it comes to _efficient_ pattern recognition.


Wouldn't AST macros require either to standardize (and freeze) 
AST representation within the compiler or to maintain an equally 
frozen alternate representation to be exposed? I can't see how 
that wouldn't make the compiler's development slower since all of 
a sudden changing the internal representation would impact user 
code, especially given the low number of breaking changes that 
are accepted today.

I have little experience with AST macros outside Lisp where its 
homoiconicity avoids the issue almost completely so pardon the 
naive question.
Apr 09 2018
parent Jacob Carlborg <doob me.com> writes:
On 2018-04-09 18:39, Cym13 wrote:

 On Monday, 9 April 2018 at 15:30:33 UTC, Stefan Koch wrote:

 On Friday, 6 April 2018 at 21:45:45 UTC, Zach Tollen wrote:

 I think Walter's reason was that such macros would hide too many 
 idiosyncrasies in how they were programmed, such that a lot of code 
 which seems simple on the surface will actually obfuscate complicated 
 and arbitrary macro-programming patterns. Thus, code that uses them 
 will become much less maintainable, because it is liable to do so 
 many different and hidden things. Also, the tasks for which 
 AST-macros would typically be used are already largely accommodated 
 by templates and other features. Thus, the real need for them isn't 
 that high.


 I think it's time to revisit this.

 The reason being that templates are only well suited to very specific 
 types of meta-programs which have a low degree of parameterization and 
 a low degree of abstraction.

 Using templates to introspect and manipulate types is like using a 
 hammmer's flat back to remove a nail.
 It _can_ be done but with an absurd amount of work.
 You just have to remove all of the wall around the nail by pounding it 
 until the wall has around the nail is disintegrated :)

 This is not an exaggeration.

 Templates used for introspection (or anything else really that's 
 modestly complex) are equally hard to reason about for compilers and 
 for programmers. I guess programmers have an advantage when it comes 
 to _efficient_ pattern recognition.


 
 Wouldn't AST macros require either to standardize (and freeze) AST 
 representation within the compiler or to maintain an equally frozen 
 alternate representation to be exposed?


Yes. It can have two separate ASTs, one internal for the compiler and 
one external that are used together with the AST macros. They would 
probably quite similar but they don't need to be exactly the same.


 I can't see how that wouldn't 
 make the compiler's development slower since all of a sudden changing 
 the internal representation would impact user code, especially given the 
 low number of breaking changes that are accepted today.


We already have three clients using the AST: LDC, GDC and the DMD Dub 
package.

-- 
/Jacob Carlborg
Apr 10 2018
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Monday, 9 April 2018 at 15:30:33 UTC, Stefan Koch wrote:

 [snip]

 Using templates to introspect and manipulate types is like 
 using a hammmer's flat back to remove a nail.
 It _can_ be done but with an absurd amount of work.
 You just have to remove all of the wall around the nail by 
 pounding it until the wall has around the nail is disintegrated 
 :)

 This is not an exaggeration.

 Templates used for introspection (or anything else really 
 that's modestly complex) are equally hard to reason about for 
 compilers and for programmers. I guess programmers have an 
 advantage when it comes to _efficient_ pattern recognition.


Could AST macros replace things like  safe/ nogc and enable the 
user to create their own (like a  supersafe that disallows 
 trusted)?
Apr 12 2018
parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 13 April 2018 at 00:37:39 UTC, jmh530 wrote:

 Could AST macros replace things like  safe/ nogc and enable the 
 user to create their own (like a  supersafe that disallows 
  trusted)?


Short Answer: Yes. If the AST-Macro facilities are built with 
that use in mind.
Apr 12 2018
prev sibling parent reply David Bennett <davidbennett bravevision.com> writes:
On Friday, 6 April 2018 at 20:33:10 UTC, Chris Katko wrote:

 Sorry if this is "re-opening" an old thread, but did anything 
 come from this and DIP50? It seems like a really interesting 
 concept and this thread was one of the first results for a 
 Google search.

 Thanks.


Thanks for reminding me about this thread, I thought I would see 
how close i could get to having this work now that I know D's 
edges better:

On Sunday, 11 July 2010 at 13:29:36 UTC, Michel Fortin wrote:

 That said, I don't feel like I'm cheating when using string 
 mixins. I find them a quite good substitute to AST macros. And 
 perhaps string mixins are simpler too: you don't have to learn 
 a new macro syntax, you just manipulate strings. Though I'm 
 still waiting for the day we can use string mixins in 
 expressions to do things like this:

 	int num = 1;
 	string result = substitute!"Number: $num";
 	assert(result == "Number: 1");


So what i came up with was:

--- substituter.d
module substituter;

string substituteForMixin(string input){

     input ~= ` `;
     string output = ``;
     size_t l= input.length-1;

     for(size_t i=0; i<l; i++){

             output ~= (i==0) ? `to!string(` : `"~to!string(`;
             i+=2;
             while(input[i] != '}'){
                 output ~= input[i++];
             }
             output ~= (i==l-1) ? `)` : `)~"`;
         }else{
             if(i==0) output ~= `"`;
             output ~= input[i];
             if(i==l-1) output ~= `"`;
         }
     }
     return output;
}

mixin template Substituter(){
     string substitute(alias string ident)(){
         import std.conv : to;
         mixin(`return `~substituteForMixin(ident)~`;`);
     };
}

 safe unittest{

     mixin Substituter;

     int number = 42;

     assert(

         == "number is 42"
     );
     assert(

         == "43 is the result of number+1"
     );
     assert(

number")
         == "you get 1764 when you square number"
     );
}
---

I think thats as clean as I can make it currently. Also I'm using 

everything.

Not sure if this syntax is documented but you can use enum like:

enum wrapmixin(alias string s) = {mixin(s)};
enum wrapmixin(alias string s){mixin(s)};

And the mixin() is run at the time the enum is called. The only 
problem with this though is the lambda/function is defined in the 
scope enum was defined not where the enum was called. So I think 
its just sugar for:

template wrapmixin(alias string s){alias wrapmixin = {mixin(s);}}
template wrapmixin(alias string s){auto wrapmixin(){mixin(s);}}

To work around this I used a mixin template to work around the 
scope issue.

Now to bring us back to 2018, I think that this enum version of 
the code should inject the function/lambda into the scope of it's 
caller. That would allow us to do a lot of cool things.

Also the other idea I had was to have mixin functions that only 
take compiletime args (they are UFCS-able though, unlike 
templates) and mixin themselves when called like:

---
mixin add1Xtimes(alias int a, alias int t){
     uint i=t;
     do{a++;}while(--i);
     return a;
}
 safe unittest{
     uint n=0;
     n.add1Xtimes!(4).add1Xtimes!(6);
     assert(n==10);
}
---

And that becomes:

---
 safe unittest{
     uint n=0;
     uint __m0_i=4;
     do{n++;}while(--__m0_i);
     uint __m1_i=6;
     do{n++;}while(--__m1_i);
     assert(n==10);
}
---

Notice that if the return is never used it's stripped out.

I haven't really given much thought to the implementation of this 
though.
Apr 13 2018
parent David Bennett <davidbennett bravevision.com> writes:
On Friday, 13 April 2018 at 11:54:12 UTC, David Bennett wrote:

 Also the other idea I had was to have mixin functions that only 
 take compiletime args (they are UFCS-able though, unlike 
 templates) and mixin themselves when called like:

 ---
 mixin add1Xtimes(alias int a, alias int t){
     uint i=t;
     do{a++;}while(--i);
     return a;
 }
  safe unittest{
     uint n=0;
     n.add1Xtimes!(4).add1Xtimes!(6);
     assert(n==10);
 }
 ---

 [snip]

 I haven't really given much thought to the implementation of 
 this though.


With more thought thats probably not a good solution as its 
different from the rest of the language and would be harder then 
it needs to be in order to integrate with the front-end.

What could work though is having a new AutoMixinDecloration that 
was inited with an expression. This expression would contain a 
call chain to regular functions, templates, other 
AutoMixinDecloration etc. Then when it's called it dups the 
contents of these functions into the callers scope.

---
auto ref incXTimesAndCount(string cident, alias uint csym, uint 
times)(auto ref uint a=0){
     uint t = times;
     do{
         mixin(cident~"++;");
         csym++;
         a++;
     }while(--t);
     return a;
}

auto mixin incXAndXTimesAndCount(string cident, alias uint csym, 
uint t1, uint t2)
                 = incXTimesAndCount!(cident, csym, 
t1).incXTimesAndCount!(cident, csym, t2);

 safe unittest{
     uint n=10;
     uint c1=0;
     uint c2=0;
     uint r = n.incXAndXTimesAndCount!("c1", c2, 4, 6);
     assert(n==20);
     assert(c1==10);
     assert(c2==10);
     assert(r==20);
}
---

And it lowers to this in the front end:

---
 safe unittest{
     uint n=10;
     uint c1=0;
     uint c2=0;
     uint __m0_t=4;
     do{n++; c1++; c2++;}while(--__m0_t);
     uint __m1_t=6;
     do{n++; c1++; c2++;}while(--__m1_t);
     uint r = n;
     assert(n==20);
     assert(c1==10);
     assert(c2==10);
     assert(r==20);
}
---

Or it might be even possible to make the current 
TemplateMixinDeclaration to act like this if you add the auto 
keyword to the declaration with a symbol with the same name in 
the scope and the above was just sugar to this:

---
auto mixin template incXAndXTimesAndCount(string cident, alias 
uint csym, uint t1, uint t2)
{
     alias incXAndXTimesAndCount
             = incXTimesAndCount!(cident, csym, 
t1).incXTimesAndCount!(cident, csym, t2);
}
---
Apr 13 2018
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 bearophile:

 String mixins are a hack, just a bit better than C preprocessor (because  
 they are scoped), they are not a replacement of clean macros.


 
 Interesting statement, can you back it up? :)  What can you do with a  
 macro that you can't do with a mixin?


In the meantime others have already given you some answers, I can add one
example. The Brainfuck language allows you to do every kind of thing, I have
seen even compilers written in it, but its usage is very unhandy, it's
bug-prone, requires lot of code to do even simple things, and it's very hard to
modify and debug programs written in it. What I meant is that string mixins in
theory allow you to do many things, but in practice you can't use them for
complex tasks, and you can't modify and fix the resulting code if you try to
use them for more complex tasks. They are not a long-term solution for a
language that seriously wants to improve over C and its preprocessor macros,
they are a hack.

In my opinion the std.bitmanip.bitfields souce code is already past the decency
limit for string mixins, and I hope they will be replaced by something better.

I don't agree with Andrei when he says that CLisp macros (and even more Scheme
macros that are hygienic too) are as hairy and unmantenable as code that uses
string mixins heavily.

In past I have asked for a gensym intrinsic in D, but it was not a serious
request, because I don't want to see string mixing used even more in D :-)

Bye,
bearophile
Jul 12 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 12 Jul 2010 16:22:28 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:

 bearophile:

 String mixins are a hack, just a bit better than C preprocessor  


 (because

 they are scoped), they are not a replacement of clean macros.


 Interesting statement, can you back it up? :)  What can you do with a
 macro that you can't do with a mixin?


 In the meantime others have already given you some answers, I can add  
 one example. The Brainfuck language allows you to do every kind of  
 thing, I have seen even compilers written in it, but its usage is very  
 unhandy, it's bug-prone, requires lot of code to do even simple things,  
 and it's very hard to modify and debug programs written in it. What I  
 meant is that string mixins in theory allow you to do many things, but  
 in practice you can't use them for complex tasks, and you can't modify  
 and fix the resulting code if you try to use them for more complex  
 tasks. They are not a long-term solution for a language that seriously  
 wants to improve over C and its preprocessor macros, they are a hack.


Brainfuck is basically a toy example of a language.  Nobody uses it for  
serious work.  Mixins are much better than a hack, the syntax of using  
them is just not polished.  They are easy to use/understand because a)  
people understand the language and b) people understand string  
manipulation.


 In my opinion the std.bitmanip.bitfields souce code is already past the  
 decency limit for string mixins, and I hope they will be replaced by  
 something better.


How would AST macros make std.bitmanip.bitfields easier to understand?  I  
have never seen it before, and I understood it after a couple minutes  
reading.  I bet the AST version would be actually harder to understand.

Most of the "ugliness" is the part that builds the accessors because it  
contains so much concatenation, but that is what I was proposing.  Having  
used php for the last year in my job, there is one thing I appreciate:  
with a language that is focused on string manipulation, it is so much  
better to be able to simply output variables inside strings rather than  
having to exit a quotation, and use a concatenation operator.  With the  
unused keyword macro, we can make string mixins much easier to  
write/understand without the need to add AST macros.

I'll give you another example -- javascript and HTML editing.  Most people  
would prefer to just use the innerHTML component of an element than have  
to use the DOM methods to create individual elements and add them as  
children, etc.


 I don't agree with Andrei when he says that CLisp macros (and even more  
 Scheme macros that are hygienic too) are as hairy and unmantenable as  
 code that uses string mixins heavily.


I don't have much experience with these languages except that in college  
while taking a scheme course, I wanted to create a bonfire out of all  
parentheses keys.

-Steve
Jul 12 2010
next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:op.vfq0aydoeav7ka localhost.localdomain...

 On Mon, 12 Jul 2010 16:22:28 -0400, bearophile <bearophileHUGS lycos.com> 
 wrote:


 Steven Schveighoffer:

 bearophile:

 String mixins are a hack, just a bit better than C preprocessor


 (because

 they are scoped), they are not a replacement of clean macros.


 Interesting statement, can you back it up? :)  What can you do with a
 macro that you can't do with a mixin?


 In the meantime others have already given you some answers, I can add 
 one example. The Brainfuck language allows you to do every kind of 
 thing, I have seen even compilers written in it, but its usage is very 
 unhandy, it's bug-prone, requires lot of code to do even simple things, 
 and it's very hard to modify and debug programs written in it. What I 
 meant is that string mixins in theory allow you to do many things, but 
 in practice you can't use them for complex tasks, and you can't modify 
 and fix the resulting code if you try to use them for more complex 
 tasks. They are not a long-term solution for a language that seriously 
 wants to improve over C and its preprocessor macros, they are a hack.


 Brainfuck is basically a toy example of a language.  Nobody uses it for 
 serious work.  Mixins are much better than a hack, the syntax of using 
 them is just not polished.  They are easy to use/understand because a) 
 people understand the language and b) people understand string 
 manipulation.


But when you're talking about the string being actual code, you're not 
always talking about typical string manipulation, sometimes you're talking 
about parsing which is only "string manipulation" superficially.


 I'll give you another example -- javascript and HTML editing.  Most people 
 would prefer to just use the innerHTML component of an element than have 
 to use the DOM methods to create individual elements and add them as 
 children, etc.


For writing, yes, but there's also reading: How many people do you know who 
would rather find the elements they want by parsing innerHTML instead of 
just dealing with the readily-available tree? None, I would hope, but the 
latter is essentially what we have to do for many of the more advanced 
things that string mixins *technically* replace AST macros for.
Jul 12 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/12/2010 06:21 PM, Nick Sabalausky wrote:

 "Steven Schveighoffer"<schveiguy yahoo.com>  wrote in message
 news:op.vfq0aydoeav7ka localhost.localdomain...

 On Mon, 12 Jul 2010 16:22:28 -0400, bearophile<bearophileHUGS lycos.com>
 wrote:


 Steven Schveighoffer:

 bearophile:

 String mixins are a hack, just a bit better than C preprocessor


 (because

 they are scoped), they are not a replacement of clean macros.


 Interesting statement, can you back it up? :)  What can you do with a
 macro that you can't do with a mixin?


 In the meantime others have already given you some answers, I can add
 one example. The Brainfuck language allows you to do every kind of
 thing, I have seen even compilers written in it, but its usage is very
 unhandy, it's bug-prone, requires lot of code to do even simple things,
 and it's very hard to modify and debug programs written in it. What I
 meant is that string mixins in theory allow you to do many things, but
 in practice you can't use them for complex tasks, and you can't modify
 and fix the resulting code if you try to use them for more complex
 tasks. They are not a long-term solution for a language that seriously
 wants to improve over C and its preprocessor macros, they are a hack.


 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.  Mixins are much better than a hack, the syntax of using
 them is just not polished.  They are easy to use/understand because a)
 people understand the language and b) people understand string
 manipulation.


 But when you're talking about the string being actual code, you're not
 always talking about typical string manipulation, sometimes you're talking
 about parsing which is only "string manipulation" superficially.


 I'll give you another example -- javascript and HTML editing.  Most people
 would prefer to just use the innerHTML component of an element than have
 to use the DOM methods to create individual elements and add them as
 children, etc.


 For writing, yes, but there's also reading: How many people do you know who
 would rather find the elements they want by parsing innerHTML instead of
 just dealing with the readily-available tree? None, I would hope, but the
 latter is essentially what we have to do for many of the more advanced
 things that string mixins *technically* replace AST macros for.


But bitfields and other similar code generating samples don't parse - 
they generate.

Andrei
Jul 12 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:i1ge6c$c9s$1 digitalmars.com...

 On 07/12/2010 06:21 PM, Nick Sabalausky wrote:

 "Steven Schveighoffer"<schveiguy yahoo.com>  wrote in message

 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.  Mixins are much better than a hack, the syntax of using
 them is just not polished.  They are easy to use/understand because a)
 people understand the language and b) people understand string
 manipulation.


 But when you're talking about the string being actual code, you're not
 always talking about typical string manipulation, sometimes you're 
 talking
 about parsing which is only "string manipulation" superficially.


 I'll give you another example -- javascript and HTML editing.  Most 
 people
 would prefer to just use the innerHTML component of an element than have
 to use the DOM methods to create individual elements and add them as
 children, etc.


 For writing, yes, but there's also reading: How many people do you know 
 who
 would rather find the elements they want by parsing innerHTML instead of
 just dealing with the readily-available tree? None, I would hope, but the
 latter is essentially what we have to do for many of the more advanced
 things that string mixins *technically* replace AST macros for.


 But bitfields and other similar code generating samples don't parse - they 
 generate.


I already agreed to that part ("For writing, yes..."). But there are other 
uses that *do* parse, and others that do both. The point is NOT that string 
mixins are *always* unsatisfactory as a replacement for AST macros. The 
point is that *there are perfectly legitimate use-cases* where string mixins 
are unsatisfactory as a replacement for AST macros. I think you've already 
agreed to this in other posts.
Jul 12 2010
parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 7/12/2010 19:41, Nick Sabalausky wrote:

 I already agreed to that part ("For writing, yes..."). But there are other 
 uses that *do* parse, and others that do both. The point is NOT that string 
 mixins are *always* unsatisfactory as a replacement for AST macros. The 
 point is that *there are perfectly legitimate use-cases* where string mixins 
 are unsatisfactory as a replacement for AST macros. I think you've already 
 agreed to this in other posts.


The great strength of string mixins is that you can use them to add the
AST macros to D.  The great weakness of string mixins is that doing so
requires a full (and extendable) CTFE D parser, and that no such parser
is provided by Phobos.

It would be interesting to try the "extendable CTFE D parser" route
instead of the "AST macros as built-in primitive" route.  For example,
you could use it to add new lexical tokens to D, something that would be
impossible with AST macros.


-- 
Rainer Deyke - rainerd eldwood.com
Jul 12 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Rainer Deyke" <rainerd eldwood.com> wrote in message 
news:i1gs16$1oj3$1 digitalmars.com...

 On 7/12/2010 19:41, Nick Sabalausky wrote:

 I already agreed to that part ("For writing, yes..."). But there are 
 other
 uses that *do* parse, and others that do both. The point is NOT that 
 string
 mixins are *always* unsatisfactory as a replacement for AST macros. The
 point is that *there are perfectly legitimate use-cases* where string 
 mixins
 are unsatisfactory as a replacement for AST macros. I think you've 
 already
 agreed to this in other posts.


 The great strength of string mixins is that you can use them to add the
 AST macros to D.  The great weakness of string mixins is that doing so
 requires a full (and extendable) CTFE D parser, and that no such parser
 is provided by Phobos.


Seems to me that would lead to unnecessary decreases in compilation 
performance. Such as superfluous re-parsing. And depending how exactly DMD 
does CTFE, a CTFE D parser could be slower than just simply having DMD do 
the parsing directly.
Jul 13 2010
parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 7/13/2010 01:03, Nick Sabalausky wrote:

 "Rainer Deyke" <rainerd eldwood.com> wrote in message 
 news:i1gs16$1oj3$1 digitalmars.com...

 The great strength of string mixins is that you can use them to add the
 AST macros to D.  The great weakness of string mixins is that doing so
 requires a full (and extendable) CTFE D parser, and that no such parser
 is provided by Phobos.


 
 Seems to me that would lead to unnecessary decreases in compilation 
 performance. Such as superfluous re-parsing. And depending how exactly DMD 
 does CTFE, a CTFE D parser could be slower than just simply having DMD do 
 the parsing directly.


True. I tend to ignore compile-time costs.  The performance of computers
is increasing exponentially.  The length of the average computer program
is more or less stable.  Therefore this particular problem will
eventually solve itself.

Of course, this is just my perspective as a developer who uses a
compiler maybe twenty times a day.  If I was writing my own compiler
which was going to be used thousands of times a day by thousands of
different developers, I'd have a different attitude.


-- 
Rainer Deyke - rainerd eldwood.com
Jul 13 2010
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Rainer Deyke" <rainerd eldwood.com> wrote in message 
news:i1h52o$2kb7$1 digitalmars.com...

 On 7/13/2010 01:03, Nick Sabalausky wrote:

 "Rainer Deyke" <rainerd eldwood.com> wrote in message
 news:i1gs16$1oj3$1 digitalmars.com...

 The great strength of string mixins is that you can use them to add the
 AST macros to D.  The great weakness of string mixins is that doing so
 requires a full (and extendable) CTFE D parser, and that no such parser
 is provided by Phobos.


 Seems to me that would lead to unnecessary decreases in compilation
 performance. Such as superfluous re-parsing. And depending how exactly 
 DMD
 does CTFE, a CTFE D parser could be slower than just simply having DMD do
 the parsing directly.


 True. I tend to ignore compile-time costs.  The performance of computers
 is increasing exponentially.  The length of the average computer program
 is more or less stable.  Therefore this particular problem will
 eventually solve itself.


The people behind C++ probably thought the same thing ;)


 Of course, this is just my perspective as a developer who uses a
 compiler maybe twenty times a day.  If I was writing my own compiler
 which was going to be used thousands of times a day by thousands of
 different developers, I'd have a different attitude.
Jul 13 2010
prev sibling parent reply Don <nospam nospam.com> writes:
Rainer Deyke wrote:

 On 7/13/2010 01:03, Nick Sabalausky wrote:

 "Rainer Deyke" <rainerd eldwood.com> wrote in message 
 news:i1gs16$1oj3$1 digitalmars.com...

 The great strength of string mixins is that you can use them to add the
 AST macros to D.  The great weakness of string mixins is that doing so
 requires a full (and extendable) CTFE D parser, and that no such parser
 is provided by Phobos.


 Seems to me that would lead to unnecessary decreases in compilation 
 performance. Such as superfluous re-parsing. And depending how exactly DMD 
 does CTFE, a CTFE D parser could be slower than just simply having DMD do 
 the parsing directly.


 
 True. I tend to ignore compile-time costs.  The performance of computers
 is increasing exponentially.  The length of the average computer program
 is more or less stable.  Therefore this particular problem will
 eventually solve itself.
 
 Of course, this is just my perspective as a developer who uses a
 compiler maybe twenty times a day.  If I was writing my own compiler
 which was going to be used thousands of times a day by thousands of
 different developers, I'd have a different attitude.
 
 


CTFE isn't intrinsically slow. The reason it's so slow in DMD right now 
is that the treatment of CTFE variables is done in an absurdly 
inefficient way.
Jul 13 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Don:

 CTFE isn't intrinsically slow. The reason it's so slow in DMD right now 
 is that the treatment of CTFE variables is done in an absurdly 
 inefficient way.


Are those arrays managed like immutable arrays in a functional language? :-)
To implement mutable arrays with an immutable data structure you need a GC
designed for it (like the Haskell one) and a different underlying
implementation, like a finger tree, that avoids most of the copying and
allocations :-)

Bye,
bearophile
Jul 13 2010
next sibling parent Don <nospam nospam.com> writes:
bearophile wrote:

 Don:

 CTFE isn't intrinsically slow. The reason it's so slow in DMD right now 
 is that the treatment of CTFE variables is done in an absurdly 
 inefficient way.


 
 Are those arrays managed like immutable arrays in a functional language? :-)


*Everything* is done with copy-on-write. It's not an appropriate way to 
implement mutable variables in an interpreter.


 To implement mutable arrays with an immutable data structure you need a GC
designed for it (like the Haskell one) and a different underlying
implementation, like a finger tree, that avoids most of the copying and
allocations :-)


Yes, and even that would never work very well for a D interpreter.
Jul 13 2010
prev sibling parent retard <re tard.com.invalid> writes:
Tue, 13 Jul 2010 04:58:04 -0400, bearophile wrote:


 Don:

 CTFE isn't intrinsically slow. The reason it's so slow in DMD right now
 is that the treatment of CTFE variables is done in an absurdly
 inefficient way.


 
 Are those arrays managed like immutable arrays in a functional language?
 :-) To implement mutable arrays with an immutable data structure you
 need a GC designed for it (like the Haskell one) and a different
 underlying implementation, like a finger tree, that avoids most of the
 copying and allocations :-)


Why so many smileys?
Jul 13 2010
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Don" <nospam nospam.com> wrote in message 
news:i1h6br$2ngl$1 digitalmars.com...

 Rainer Deyke wrote:

 On 7/13/2010 01:03, Nick Sabalausky wrote:

 "Rainer Deyke" <rainerd eldwood.com> wrote in message 
 news:i1gs16$1oj3$1 digitalmars.com...

 The great strength of string mixins is that you can use them to add the
 AST macros to D.  The great weakness of string mixins is that doing so
 requires a full (and extendable) CTFE D parser, and that no such parser
 is provided by Phobos.


 Seems to me that would lead to unnecessary decreases in compilation 
 performance. Such as superfluous re-parsing. And depending how exactly 
 DMD does CTFE, a CTFE D parser could be slower than just simply having 
 DMD do the parsing directly.


 True. I tend to ignore compile-time costs.  The performance of computers
 is increasing exponentially.  The length of the average computer program
 is more or less stable.  Therefore this particular problem will
 eventually solve itself.

 Of course, this is just my perspective as a developer who uses a
 compiler maybe twenty times a day.  If I was writing my own compiler
 which was going to be used thousands of times a day by thousands of
 different developers, I'd have a different attitude.


 CTFE isn't intrinsically slow. The reason it's so slow in DMD right now is 
 that the treatment of CTFE variables is done in an absurdly inefficient 
 way.


Doesn't it at least have some intrinsic overhead? The best way I can think 
of to do it (in fact, the way Nemerle does it, and I really wish D did too) 
is to actually fully compile the CTFE then dynamiclly link it in and run it 
natively (erm, well, Nemerle is .NET, not native, but you get the idea). 
Even that would seem to have a little bit of overhead compared to just 
running code that's already built-in to the compiler (it has to compile a 
chunk of code, and then link it, both of which take more time than just 
calling an internal function). And if you're not doing it that way then 
you're interpreting, which inherently adds some fetch/dispatch processing.

Yea, certainly it could be made to be fairly fast, and the overhead doesn't 
*have* to be big enough to be noticable. But why not just do it the *really* 
fast way right from the start? Especially in a language like D that's 
supposed to have an emphasis on short compile times.
Jul 13 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Nick Sabalausky" <a a.a> wrote in message 
news:i1i8o8$1kb1$1 digitalmars.com...

 "Don" <nospam nospam.com> wrote in message 
 news:i1h6br$2ngl$1 digitalmars.com...

 Rainer Deyke wrote:

 On 7/13/2010 01:03, Nick Sabalausky wrote:

 "Rainer Deyke" <rainerd eldwood.com> wrote in message 
 news:i1gs16$1oj3$1 digitalmars.com...

 The great strength of string mixins is that you can use them to add 
 the
 AST macros to D.  The great weakness of string mixins is that doing so
 requires a full (and extendable) CTFE D parser, and that no such 
 parser
 is provided by Phobos.


 Seems to me that would lead to unnecessary decreases in compilation 
 performance. Such as superfluous re-parsing. And depending how exactly 
 DMD does CTFE, a CTFE D parser could be slower than just simply having 
 DMD do the parsing directly.


 True. I tend to ignore compile-time costs.  The performance of computers
 is increasing exponentially.  The length of the average computer program
 is more or less stable.  Therefore this particular problem will
 eventually solve itself.

 Of course, this is just my perspective as a developer who uses a
 compiler maybe twenty times a day.  If I was writing my own compiler
 which was going to be used thousands of times a day by thousands of
 different developers, I'd have a different attitude.


 CTFE isn't intrinsically slow. The reason it's so slow in DMD right now 
 is that the treatment of CTFE variables is done in an absurdly 
 inefficient way.


 Doesn't it at least have some intrinsic overhead? The best way I can think 
 of to do it (in fact, the way Nemerle does it, and I really wish D did 
 too) is to actually fully compile the CTFE then dynamiclly link it in and 
 run it natively (erm, well, Nemerle is .NET, not native, but you get the 
 idea). Even that would seem to have a little bit of overhead compared to 
 just running code that's already built-in to the compiler (it has to 
 compile a chunk of code, and then link it, both of which take more time 
 than just calling an internal function). And if you're not doing it that 
 way then you're interpreting, which inherently adds some fetch/dispatch 
 processing.

 Yea, certainly it could be made to be fairly fast, and the overhead 
 doesn't *have* to be big enough to be noticable. But why not just do it 
 the *really* fast way right from the start? Especially in a language like 
 D that's supposed to have an emphasis on short compile times.


Plus, there's the whole duplication of code: D parser in C++, plus a D 
parser in D CTFE.
Jul 13 2010
parent reply Johan Granberg <lijat.meREM OVEgmail.com> writes:
Nick Sabalausky wrote:

 Plus, there's the whole duplication of code: D parser in C++, plus a D
 parser in D CTFE.


On the other hand this would give us an D parser in D that might help in
making D self hosting.
Jul 14 2010
next sibling parent Clemens <eriatarka84 gmail.com> writes:
Johan Granberg Wrote:


 Nick Sabalausky wrote:

 Plus, there's the whole duplication of code: D parser in C++, plus a D
 parser in D CTFE.


 
 On the other hand this would give us an D parser in D that might help in
 making D self hosting.
 


http://code.google.com/p/dil/

"Dil is a hand-crafted compiler implementation for the D programming language
written in D 1.0 using the Tango standard library. The lexer and the parser are
fully implemented."

http://www.dsource.org/projects/ddmd

"DDMD is a direct port of official Digital Mars D compiler to the D Programming
Language."

Both of these are not fully functional compilers, but lexing and parsing seems
to be reasonably complete in both of them.
Jul 14 2010
prev sibling parent reply Justin Johansson <no spam.com> writes:
Johan Granberg wrote:

 Nick Sabalausky wrote:

 Plus, there's the whole duplication of code: D parser in C++, plus a D
 parser in D CTFE.


 
 On the other hand this would give us an D parser in D that might help in
 making D self hosting.


Not a chance -- pigs will fly before that and before that that, some
pigs will wear a show of lipstick.
Jul 14 2010
parent Johan Granberg <lijat.meREM OVEgmail.com> writes:
Justin Johansson wrote:


 Johan Granberg wrote:

 Nick Sabalausky wrote:

 Plus, there's the whole duplication of code: D parser in C++, plus a D
 parser in D CTFE.


 
 On the other hand this would give us an D parser in D that might help in
 making D self hosting.


 
 Not a chance -- pigs will fly before that and before that that, some
 pigs will wear a show of lipstick.


I can still wish :)
Jul 14 2010
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 12 Jul 2010 19:21:08 -0400, Nick Sabalausky <a a.a> wrote:


 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in message
 news:op.vfq0aydoeav7ka localhost.localdomain...

 On Mon, 12 Jul 2010 16:22:28 -0400, bearophile  
 <bearophileHUGS lycos.com>
 wrote:


 Steven Schveighoffer:

 bearophile:

 String mixins are a hack, just a bit better than C preprocessor


 (because

 they are scoped), they are not a replacement of clean macros.


 Interesting statement, can you back it up? :)  What can you do with a
 macro that you can't do with a mixin?


 In the meantime others have already given you some answers, I can add
 one example. The Brainfuck language allows you to do every kind of
 thing, I have seen even compilers written in it, but its usage is very
 unhandy, it's bug-prone, requires lot of code to do even simple things,
 and it's very hard to modify and debug programs written in it. What I
 meant is that string mixins in theory allow you to do many things, but
 in practice you can't use them for complex tasks, and you can't modify
 and fix the resulting code if you try to use them for more complex
 tasks. They are not a long-term solution for a language that seriously
 wants to improve over C and its preprocessor macros, they are a hack.


 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.  Mixins are much better than a hack, the syntax of using
 them is just not polished.  They are easy to use/understand because a)
 people understand the language and b) people understand string
 manipulation.


 But when you're talking about the string being actual code, you're not
 always talking about typical string manipulation, sometimes you're  
 talking
 about parsing which is only "string manipulation" superficially.


Not in most cases.  For extremes, I think perhaps there could be a niche  
for that, but it can be had via string mixins and CTFE (as long as CTFE is  
robust enough).  Having a compiler written in D wouldn't hurt either ;)


 I'll give you another example -- javascript and HTML editing.  Most  
 people
 would prefer to just use the innerHTML component of an element than have
 to use the DOM methods to create individual elements and add them as
 children, etc.


 For writing, yes, but there's also reading: How many people do you know  
 who
 would rather find the elements they want by parsing innerHTML instead of
 just dealing with the readily-available tree? None, I would hope, but the
 latter is essentially what we have to do for many of the more advanced
 things that string mixins *technically* replace AST macros for.


But you can't manipulate D code "on the fly" as you can HTML.   
Essentially, you are *always* writing code when using string manipulation  
to do mixins, because the compiler hasn't parsed it yet.

Essentially, there is no requirement that the input that you are about to  
manipulate must be valid D code, only the output should be.  That isn't to  
say it can't be valid D code, but in most cases, wouldn't you rather have  
simpler input?

-Steve
Jul 13 2010
prev sibling next sibling parent reply pillsy <pillsbury gmail.com> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article:


 On Mon, 12 Jul 2010 16:22:28 -0400, bearophile <bearophileHUGS lycos.com>
 wrote:


[...]

 In my opinion the std.bitmanip.bitfields souce code is already past the
 decency limit for string mixins, and I hope they will be replaced by
 something better.


 How would AST macros make std.bitmanip.bitfields easier to understand?  I
 have never seen it before, and I understood it after a couple minutes
 reading.  I bet the AST version would be actually harder to understand.


That's not such a great example. The problems will start piling up when you
have more complicated arguments.
Say you want your macro to look like it has an argument list with two entries,
called like

     foo(x, y)

which acts like a string mixin, and receives "x, y" as the argument string.
Then you split at the comma, and
you're fine. You go on with your day, and the next thing you know someone is
complaining that your foo macro
doesn't work because it chokes on

    foo(bar(a, b), c)

and

    foo("d, e", f)

By the time you deal with all the possibilities, you'll have written most of a
parser. That parser has gotta parse
the text into *something*, and it'll probably be a tree.

There's no reason you can't use library functions to parse strings containing D
code into that tree; indeed,
having access to such functions in the standard library would be useful in all
sorts of ways. RIght now D is only
a couple steps away from where it would need to be in order to allow people to
build a really useful macro
system in libraries.

Cheers,
Pillsy
[...]
Jul 12 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 12 Jul 2010 20:00:12 -0400, pillsy <pillsbury gmail.com> wrote:


 == Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article:


 On Mon, 12 Jul 2010 16:22:28 -0400, bearophile  
 <bearophileHUGS lycos.com>
 wrote:


 [...]

 In my opinion the std.bitmanip.bitfields souce code is already past  


 the

 decency limit for string mixins, and I hope they will be replaced by
 something better.


 How would AST macros make std.bitmanip.bitfields easier to understand?   
 I
 have never seen it before, and I understood it after a couple minutes
 reading.  I bet the AST version would be actually harder to understand.


 That's not such a great example. The problems will start piling up when  
 you have more complicated arguments.
 Say you want your macro to look like it has an argument list with two  
 entries, called like

      foo(x, y)

 which acts like a string mixin, and receives "x, y" as the argument  
 string. Then you split at the comma, and
 you're fine.


This isn't exactly what I meant to do with macros.  What I wanted was for  
the commas to separate the expressions, and pass those expressions via  
stringification.  It isn't a fully fleshed out idea, but here is what I  
was thinking:

macro foo(arg1, arg2) mixin("$arg1 + $arg2;") // $ inside string  
references parameter, just a possibility

foo(x, y); // translates to mixin("x + y;");

These are just ideas, perhaps there are issues with them, but I think we  
can come up with reasonable rules that allow a lot of flexibility, and at  
the same time, allow mixins to be masked as normal expressions, resulting  
in greater usability in generic code, and much more beautiful code.


 You go on with your day, and the next thing you know someone is  
 complaining that your foo macro
 doesn't work because it chokes on

     foo(bar(a, b), c)

 and

     foo("d, e", f)

 By the time you deal with all the possibilities, you'll have written  
 most of a parser. That parser has gotta parse
 the text into *something*, and it'll probably be a tree.


Or, you say "foo isn't meant to deal with that input" :)

I don't see people complaining that bitfields doesn't accept such various  
strings as arguments...


 There's no reason you can't use library functions to parse strings  
 containing D code into that tree; indeed,
 having access to such functions in the standard library would be useful  
 in all sorts of ways. RIght now D is only
 a couple steps away from where it would need to be in order to allow  
 people to build a really useful macro
 system in libraries.


I think a full parser would not hurt, but most of the time, you just want  
to use the expressions in generating some code.  Pulling apart an  
expression and reassmbling it can solve some problems, but the majority of  
the reason to need macros is to generate code, not to read it and rewrite  
it.

-Steve
Jul 13 2010
parent "Nick Sabalausky" <a a.a> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:op.vfr7dti0eav7ka localhost.localdomain...

 Pulling apart an  expression and reassmbling it can solve some problems, 
 but the majority of  the reason to need macros is to generate code, not to 
 read it and rewrite  it.


I can't help wondering if that is largely just because we're accustomed to 
not being able to do the "pull apart" without significant effort. I think we 
need to look at some of the use-cases in languages like Nemerle that have 
that sort of thing.
Jul 13 2010
prev sibling parent reply retard <re tard.com.invalid> writes:
Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.  



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation, it
 is so much better to be able to simply output variables inside strings
 rather than having to exit a quotation, and use a concatenation
 operator.


Php is basically a toy example of a language.  Nobody uses it for serious 
work. You just think your work is serious, but it isn't. Php shows what 
the world would look like if it was created by an "intelligent designer".


 I don't have much experience with these languages except that in college
 while taking a scheme course, I wanted to create a bonfire out of all
 parentheses keys.


Sounds really mature. "I have no experience with D, it's just some shitty 
language with lots of semicolons - yea, lots of semicolons is all I 
remember and that after finding Python I've never looked back - Python is 
the most native high-performance language I know". You've probably missed 
99% of the power of Lisp by only focusing on parenthesis.
Jul 13 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/13/2010 10:48 AM, retard wrote:

 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation, it
 is so much better to be able to simply output variables inside strings
 rather than having to exit a quotation, and use a concatenation
 operator.


 Php is basically a toy example of a language.  Nobody uses it for serious
 work.


Facebook.

Andrei
Jul 13 2010
next sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 07/13/2010 11:09 AM, Andrei Alexandrescu wrote:

 On 07/13/2010 10:48 AM, retard wrote:

 Php is basically a toy example of a language. Nobody uses it for serious
 work.


 Facebook.

 Andrei


Wait, which side are you arguing for?
Jul 13 2010
prev sibling next sibling parent retard <re tard.com.invalid> writes:
Tue, 13 Jul 2010 11:09:07 -0500, Andrei Alexandrescu wrote:


 On 07/13/2010 10:48 AM, retard wrote:

 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it
 for serious work.



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation, it
 is so much better to be able to simply output variables inside strings
 rather than having to exit a quotation, and use a concatenation
 operator.


 Php is basically a toy example of a language.  Nobody uses it for
 serious work.


 
 Facebook.


If they paid you as much in the academia, you'd probably enjoy your life 
more writing D or C++. I guess this is also an ethical issue, I despise 
guys like Zuckerberg who don't respect people's privacy and spy on their 
personal data. He probably pays well if you're a famous world class C++ 
book author, but I'd find my position a bit uneasy.
Jul 13 2010
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:i1i332$19jl$2 digitalmars.com...

 On 07/13/2010 10:48 AM, retard wrote:

 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation, it
 is so much better to be able to simply output variables inside strings
 rather than having to exit a quotation, and use a concatenation
 operator.


 Php is basically a toy example of a language.  Nobody uses it for serious
 work.


 Facebook.


Nothing personal, but I wouldn't consider any of those social networking 
sites to be remotely "serious". They're like the web-app equivilent of PHP - 
useless garbage. And just because they're popular doesn't mean they're not 
useless garbage - just look at any fad. They're the modern pet rock, and I 
hope they die out just as quick. (Again, nothing personal.)
Jul 13 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/13/2010 01:00 PM, Nick Sabalausky wrote:

 "Andrei Alexandrescu"<SeeWebsiteForEmail erdani.org>  wrote in message
 news:i1i332$19jl$2 digitalmars.com...

 On 07/13/2010 10:48 AM, retard wrote:

 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation, it
 is so much better to be able to simply output variables inside strings
 rather than having to exit a quotation, and use a concatenation
 operator.


 Php is basically a toy example of a language.  Nobody uses it for serious
 work.


 Facebook.


 Nothing personal, but I wouldn't consider any of those social networking
 sites to be remotely "serious".


Me neither until I started working for them. Facebook is one of the most 
difficult companies to get hired at as a programmer, and as a direct 
consequence has amassed a lot of talented folks working on hard, 
interesting problems.

Andrei
Jul 13 2010
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:i1ibci$1omk$1 digitalmars.com...

 On 07/13/2010 01:00 PM, Nick Sabalausky wrote:

 Nothing personal, but I wouldn't consider any of those social networking
 sites to be remotely "serious".


 Me neither until I started working for them. Facebook is one of the most 
 difficult companies to get hired at as a programmer,


Just out of pure curiosity, what sorts of criterea do they use?


 and as a direct consequence has amassed a lot of talented folks working on 
 hard, interesting problems.
Jul 13 2010
prev sibling parent reply retard <re tard.com.invalid> writes:
Tue, 13 Jul 2010 13:30:43 -0500, Andrei Alexandrescu wrote:


 On 07/13/2010 01:00 PM, Nick Sabalausky wrote:

 "Andrei Alexandrescu"<SeeWebsiteForEmail erdani.org>  wrote in message
 news:i1i332$19jl$2 digitalmars.com...

 On 07/13/2010 10:48 AM, retard wrote:

 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it
 for serious work.



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation,
 it is so much better to be able to simply output variables inside
 strings rather than having to exit a quotation, and use a
 concatenation operator.


 Php is basically a toy example of a language.  Nobody uses it for
 serious work.


 Facebook.


 Nothing personal, but I wouldn't consider any of those social
 networking sites to be remotely "serious".


 
 Me neither until I started working for them. Facebook is one of the most
 difficult companies to get hired at as a programmer, and as a direct
 consequence has amassed a lot of talented folks working on hard,
 interesting problems.


Such as email scams.

http://a.imageshack.us/img823/3307/serioususe.jpg

I see a pattern here. Why should I trust You or Your Employer?
Jul 14 2010
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/14/2010 12:07 PM, retard wrote:

 Tue, 13 Jul 2010 13:30:43 -0500, Andrei Alexandrescu wrote:


 On 07/13/2010 01:00 PM, Nick Sabalausky wrote:

 "Andrei Alexandrescu"<SeeWebsiteForEmail erdani.org>   wrote in message
 news:i1i332$19jl$2 digitalmars.com...

 On 07/13/2010 10:48 AM, retard wrote:

 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it
 for serious work.



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation,
 it is so much better to be able to simply output variables inside
 strings rather than having to exit a quotation, and use a
 concatenation operator.


 Php is basically a toy example of a language.  Nobody uses it for
 serious work.


 Facebook.


 Nothing personal, but I wouldn't consider any of those social
 networking sites to be remotely "serious".


 Me neither until I started working for them. Facebook is one of the most
 difficult companies to get hired at as a programmer, and as a direct
 consequence has amassed a lot of talented folks working on hard,
 interesting problems.


 Such as email scams.

 http://a.imageshack.us/img823/3307/serioususe.jpg

 I see a pattern here. Why should I trust You or Your Employer?


Not sure I understand. The monthly bitter quip?

Andrei
Jul 14 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 13/07/2010 19:00, Nick Sabalausky wrote:

 "Andrei Alexandrescu"<SeeWebsiteForEmail erdani.org>  wrote in message
 news:i1i332$19jl$2 digitalmars.com...

 On 07/13/2010 10:48 AM, retard wrote:

 Facebook.


 Nothing personal, but I wouldn't consider any of those social networking
 sites to be remotely "serious". They're like the web-app equivilent of PHP -
 useless garbage. And just because they're popular doesn't mean they're not
 useless garbage - just look at any fad. They're the modern pet rock, and I
 hope they die out just as quick. (Again, nothing personal.)


lol...

-- 
Bruno Medeiros - Software Engineer
Jul 19 2010
prev sibling next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 13 Jul 2010 11:48:30 -0400, retard <re tard.com.invalid> wrote:


 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation, it
 is so much better to be able to simply output variables inside strings
 rather than having to exit a quotation, and use a concatenation
 operator.


 Php is basically a toy example of a language.  Nobody uses it for serious
 work. You just think your work is serious, but it isn't. Php shows what
 the world would look like if it was created by an "intelligent designer".


No, it's you who are not serious ;)

php sucks as a language, but its what I have to work with.  There are some  
good parts to it.


 I don't have much experience with these languages except that in college
 while taking a scheme course, I wanted to create a bonfire out of all
 parentheses keys.


 Sounds really mature. "I have no experience with D, it's just some shitty
 language with lots of semicolons - yea, lots of semicolons is all I
 remember and that after finding Python I've never looked back - Python is
 the most native high-performance language I know". You've probably missed
 99% of the power of Lisp by only focusing on parenthesis.


So go join the lisp newsgroup and give out online high fives.  I'll just  
stay right here, thanks :)

-Steve
Jul 13 2010
parent reply retard <re tard.com.invalid> writes:
Tue, 13 Jul 2010 13:09:08 -0400, Steven Schveighoffer wrote:


 On Tue, 13 Jul 2010 11:48:30 -0400, retard <re tard.com.invalid> wrote:
 

 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:


 Brainfuck is basically a toy example of a language.  Nobody uses it
 for serious work.



 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation, it
 is so much better to be able to simply output variables inside strings
 rather than having to exit a quotation, and use a concatenation
 operator.


 Php is basically a toy example of a language.  Nobody uses it for
 serious work. You just think your work is serious, but it isn't. Php
 shows what the world would look like if it was created by an
 "intelligent designer".


 
 No, it's you who are not serious ;)
 
 php sucks as a language, but its what I have to work with.  There are
 some good parts to it.


There are no good parts in PHP, but you're right that it is used in the 
real world.


 

 I don't have much experience with these languages except that in
 college while taking a scheme course, I wanted to create a bonfire out
 of all parentheses keys.


 Sounds really mature. "I have no experience with D, it's just some
 shitty language with lots of semicolons - yea, lots of semicolons is
 all I remember and that after finding Python I've never looked back -
 Python is the most native high-performance language I know". You've
 probably missed 99% of the power of Lisp by only focusing on
 parenthesis.


 
 So go join the lisp newsgroup and give out online high fives.  I'll just
 stay right here, thanks :)


It doesn't matter what language you use. The concepts are the same 
regardless of the syntax. If I liked the parenthesis hell, I'd probably 
use Lisp. But it's still not a good argument to bring up every time Lisp 
is being discussed. You're dismissing all intelligent discussion by 
ranting about the syntax. Almost everyone here agrees that Lisp has too 
little syntax, so that point doesn't bring any new value.
Jul 13 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 13 Jul 2010 13:20:03 -0400, retard <re tard.com.invalid> wrote:


 It doesn't matter what language you use. The concepts are the same
 regardless of the syntax. If I liked the parenthesis hell, I'd probably
 use Lisp. But it's still not a good argument to bring up every time Lisp
 is being discussed. You're dismissing all intelligent discussion by
 ranting about the syntax. Almost everyone here agrees that Lisp has too
 little syntax, so that point doesn't bring any new value.


Maybe you misunderstood my original statement.  Scheme may introduce some  
nice paradigms, and be well designed, but I just couldn't get past the  
parentheses.  Therefore, I have little experience in scheme, even though I  
used it.  That's all I was saying.

My professor would write some scheme example on the board, and then write  
a bunch of closing parentheses as he was saying "cdr, cons, etc." so I was  
absolutely thoroughly lost most of the time :)  I'm surprised I even  
passed that class.  We had to write a scheme interpreter in scheme, it was  
probably way too advanced for my brain at the time.

-Steve
Jul 13 2010
parent retard <re tard.com.invalid> writes:
Tue, 13 Jul 2010 13:30:19 -0400, Steven Schveighoffer wrote:


 On Tue, 13 Jul 2010 13:20:03 -0400, retard <re tard.com.invalid> wrote:
 

 It doesn't matter what language you use. The concepts are the same
 regardless of the syntax. If I liked the parenthesis hell, I'd probably
 use Lisp. But it's still not a good argument to bring up every time
 Lisp is being discussed. You're dismissing all intelligent discussion
 by ranting about the syntax. Almost everyone here agrees that Lisp has
 too little syntax, so that point doesn't bring any new value.


 
 Maybe you misunderstood my original statement.  Scheme may introduce
 some nice paradigms, and be well designed, but I just couldn't get past
 the parentheses.  Therefore, I have little experience in scheme, even
 though I used it.  That's all I was saying.
 
 My professor would write some scheme example on the board, and then
 write a bunch of closing parentheses as he was saying "cdr, cons, etc."
 so I was absolutely thoroughly lost most of the time :)  I'm surprised I
 even passed that class.  We had to write a scheme interpreter in scheme,
 it was probably way too advanced for my brain at the time.


Ok. There are some code visualization tools for Lisp/Scheme with color 
encoded blocks and indentation instead of parentheses. You could also 
rewrite 'car' and 'cdr' as 'head' and 'tail'.
Jul 13 2010
prev sibling next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"retard" <re tard.com.invalid> wrote in message 
news:i1i1se$ord$1 digitalmars.com...

 Php is basically a toy example of a language.  Nobody uses it for serious
 work. You just think your work is serious, but it isn't. Php shows what
 the world would look like if it was created by an "intelligent designer".


I want to have that enlarged, framed and mounted on my wall.

("That's what she said!" Ha h..umm...no...sorry...)

When I first read "Nobody uses it for serious work." I was about to hit 
Reply and object, but the next sentence hit the nail right on the head. 
*I've* used PHP plenty of times simply because I had to, but when you get 
down to it, none of that (and really no web app in general IMO) *really* 
qualifies as serious work. We may get paid to make it, but it's all just 
pointless dinky shit (and most of it's made by amateurs anyway).
Jul 13 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 13 Jul 2010 14:08:15 -0400, Nick Sabalausky <a a.a> wrote:


 "retard" <re tard.com.invalid> wrote in message
 news:i1i1se$ord$1 digitalmars.com...

 Php is basically a toy example of a language.  Nobody uses it for  
 serious
 work. You just think your work is serious, but it isn't. Php shows what
 the world would look like if it was created by an "intelligent  
 designer".


 I want to have that enlarged, framed and mounted on my wall.

 ("That's what she said!" Ha h..umm...no...sorry...)

 When I first read "Nobody uses it for serious work." I was about to hit
 Reply and object, but the next sentence hit the nail right on the head.
 *I've* used PHP plenty of times simply because I had to, but when you get
 down to it, none of that (and really no web app in general IMO) *really*
 qualifies as serious work. We may get paid to make it, but it's all just
 pointless dinky shit (and most of it's made by amateurs anyway).


Just for fun, how would you define a serious language?

1. Major major websites use it (facebook is one example, there are many  
more).
2. There are lots of books about it.
3. people make serious money doing it, not just play money.
4. major IDEs support it (I use netbeans, and probably would be lost  
without it).
5. The documentation is complete and well written.
6. It supports interfaces to many major technologies (many databases,  
apache, pdf, etc.)

I'd agree that php has some serious drawbacks, I can't tell you how many  
times I miss static typing when working on a daily basis (try forgetting  
to use $this when writing an object).  But it's just another language, one  
that works, and is well defined.  I'd also agree that there are lots of  
non-serious developers using it, which means the quality of any one piece  
of code has a better chance of being crappy than a piece of C++ code, but  
I've seen a lot of crappy C++ code too.

-Steve
Jul 13 2010
parent reply "Nick Sabalausky" <a a.a> writes:
"Steven Schveighoffer" <schveiguy yahoo.com> wrote in message 
news:op.vfsn2hvweav7ka localhost.localdomain...

 Just for fun, how would you define a serious language?

 1. Major major websites use it (facebook is one example, there are many 
 more).
 2. There are lots of books about it.
 3. people make serious money doing it, not just play money.
 4. major IDEs support it (I use netbeans, and probably would be lost 
 without it).
 5. The documentation is complete and well written.
 6. It supports interfaces to many major technologies (many databases, 
 apache, pdf, etc.)


For me, there'd be two criteria (in no order):

1. Useful
2. Doesn't suck

:)
Jul 13 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/13/2010 02:00 PM, Nick Sabalausky wrote:

 "Steven Schveighoffer"<schveiguy yahoo.com>  wrote in message
 news:op.vfsn2hvweav7ka localhost.localdomain...

 Just for fun, how would you define a serious language?

 1. Major major websites use it (facebook is one example, there are many
 more).
 2. There are lots of books about it.
 3. people make serious money doing it, not just play money.
 4. major IDEs support it (I use netbeans, and probably would be lost
 without it).
 5. The documentation is complete and well written.
 6. It supports interfaces to many major technologies (many databases,
 apache, pdf, etc.)


 For me, there'd be two criteria (in no order):

 1. Useful
 2. Doesn't suck

 :)


That leaves only D on the table :o).

Andrei
Jul 13 2010
parent "Nick Sabalausky" <a a.a> writes:
"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:i1ihke$253j$6 digitalmars.com...

 On 07/13/2010 02:00 PM, Nick Sabalausky wrote:

 "Steven Schveighoffer"<schveiguy yahoo.com>  wrote in message
 news:op.vfsn2hvweav7ka localhost.localdomain...

 Just for fun, how would you define a serious language?

 1. Major major websites use it (facebook is one example, there are many
 more).
 2. There are lots of books about it.
 3. people make serious money doing it, not just play money.
 4. major IDEs support it (I use netbeans, and probably would be lost
 without it).
 5. The documentation is complete and well written.
 6. It supports interfaces to many major technologies (many databases,
 apache, pdf, etc.)


 For me, there'd be two criteria (in no order):

 1. Useful
 2. Doesn't suck

 :)


 That leaves only D on the table :o).


Yup :)

Well, and Nemerle if you're doing something that doesn't need systems 
capabilities.
Jul 13 2010
prev sibling parent Jonathan M Davis <jmdavisprog gmail.com> writes:
On Tuesday, July 13, 2010 12:00:45 Nick Sabalausky wrote:

 "Steven Schveighoffer" <schveiguy yahoo.com> wrote in message
 news:op.vfsn2hvweav7ka localhost.localdomain...
 

 Just for fun, how would you define a serious language?
 
 1. Major major websites use it (facebook is one example, there are many
 more).
 2. There are lots of books about it.
 3. people make serious money doing it, not just play money.
 4. major IDEs support it (I use netbeans, and probably would be lost
 without it).
 5. The documentation is complete and well written.
 6. It supports interfaces to many major technologies (many databases,
 apache, pdf, etc.)


 
 For me, there'd be two criteria (in no order):
 
 1. Useful
 2. Doesn't suck
 
 :)


Arguably, _every_ computer language sucks - if nothing else because the 
designers of each and every language had to make decisions with pros and cons, 
and they aren't omniscient, so they're bound to have made mistakes of some 
variety in the language design. The real questions are how well the language 
does what you're trying to do and how close it operates to how you think.

Depending on what you're doing, any particular language could be fantastic or
it 
could be horrible. And there are always drawbacks of some kind no matter _what_ 
you do. So, really, they all suck. It's more a question of which sucks the
least 
than which doesn't suck.

Personally, I find that D is the closest to what I'm looking for in a language, 
and I think that it's the best language out there. But it's by no means
perfect, 
and it never will be no matter how much Walter, Andrei, et al work on it. Every 
language has problems and that's not going to change. If nothing else, there
are 
just too many places in computing where there are tradeoffs.

- Jonathan M Davis
Jul 13 2010
prev sibling parent reply Justin Johansson <no spam.com> writes:
Nick Sabalausky wrote:

 "retard" <re tard.com.invalid> wrote in message 
 news:i1i1se$ord$1 digitalmars.com...

 Php is basically a toy example of a language.  Nobody uses it for serious
 work. You just think your work is serious, but it isn't. Php shows what
 the world would look like if it was created by an "intelligent designer".


 
 I want to have that enlarged, framed and mounted on my wall.
 
 ("That's what she said!" Ha h..umm...no...sorry...)
 
 When I first read "Nobody uses it for serious work." I was about to hit 
 Reply and object, but the next sentence hit the nail right on the head. 
 *I've* used PHP plenty of times simply because I had to, but when you get 
 down to it, none of that (and really no web app in general IMO) *really* 
 qualifies as serious work. We may get paid to make it, but it's all just 
 pointless dinky shit (and most of it's made by amateurs anyway).


I know what you mean.  While I did reply and sort of object by citing
the Magento ecommerce platform as a Php thing that is widely used,
I noted that it (Php) is a sad reality.  As for retard's paragraph
above, taken in toto with all it's subtlety, this is absolutely
brilliant writing. :-)

If retard were to write a book, I would be sure to buy a copy.
I think there must be quite a bit of mutual respect between retard
and Andrei.  Andrei is a brilliant writer also.

Cheers
Justin Johansson
Jul 13 2010
parent retard <re tard.com.invalid> writes:
Wed, 14 Jul 2010 07:45:18 +0930, Justin Johansson wrote:


 Nick Sabalausky wrote:

 "retard" <re tard.com.invalid> wrote in message
 news:i1i1se$ord$1 digitalmars.com...

 Php is basically a toy example of a language.  Nobody uses it for
 serious work. You just think your work is serious, but it isn't. Php
 shows what the world would look like if it was created by an
 "intelligent designer".


 I want to have that enlarged, framed and mounted on my wall.
 
 ("That's what she said!" Ha h..umm...no...sorry...)
 
 When I first read "Nobody uses it for serious work." I was about to hit
 Reply and object, but the next sentence hit the nail right on the head.
 *I've* used PHP plenty of times simply because I had to, but when you
 get down to it, none of that (and really no web app in general IMO)
 *really* qualifies as serious work. We may get paid to make it, but
 it's all just pointless dinky shit (and most of it's made by amateurs
 anyway).


 
 I know what you mean.  While I did reply and sort of object by citing
 the Magento ecommerce platform as a Php thing that is widely used, I
 noted that it (Php) is a sad reality.  As for retard's paragraph above,
 taken in toto with all it's subtlety, this is absolutely brilliant
 writing. :-)
 
 If retard were to write a book, I would be sure to buy a copy. I think
 there must be quite a bit of mutual respect between retard and Andrei. 
 Andrei is a brilliant writer also.


Of course I admire him as person, but you're ranking my pseudonym a lot 
higher than necessary.

Lately I've completely lossed interest in programming and focused on 
managing techies. I kept learning new and new languages and concepts, but 
eventually was left empty handed. The problem is, many language suck and 
you not only need good technology but also people with adeguate skills. 
The situation looks much brighter (at least to me) when you can focus on 
higher level concepts and forget programming language constructs. Of 
course it's still interesting to read new papers about PLs. LtU is a 
great site.
Jul 13 2010
prev sibling parent Justin Johansson <no spam.com> writes:
retard wrote:

 Mon, 12 Jul 2010 16:53:12 -0400, Steven Schveighoffer wrote:
 

 Brainfuck is basically a toy example of a language.  Nobody uses it for
 serious work.  


 

 Having used php for the last year in my job, there is one thing I
 appreciate: with a language that is focused on string manipulation, it
 is so much better to be able to simply output variables inside strings
 rather than having to exit a quotation, and use a concatenation
 operator.


 
 Php is basically a toy example of a language.  Nobody uses it for serious 
 work. You just think your work is serious, but it isn't. Php shows what 
 the world would look like if it was created by an "intelligent designer".


Yep, Php is totally LIC (lacking intellectual content).

However, as sad as the state of affairs is, to say that nobody
uses it for serious work is plain wrong.

Magento ECommerce Platform
http://www.magentocommerce.com/

The Magento platform gets mega downloads and has tonnes and
tonnes of plugins developed by heaps and stacks of developers.


 I don't have much experience with these languages except that in college
 while taking a scheme course, I wanted to create a bonfire out of all
 parentheses keys.


 
 Sounds really mature. "I have no experience with D, it's just some shitty 
 language with lots of semicolons - yea, lots of semicolons is all I 
 remember and that after finding Python I've never looked back - Python is 
 the most native high-performance language I know". You've probably missed 
 99% of the power of Lisp by only focusing on parenthesis.


Many have heard the sad story of Viaweb, the online store application
developed by Paul Graham and Robert Morris and written in LISP

(Read all about the LISP secret weapon here)
http://www.paulgraham.com/avg.html

My understanding is that upon Yahoo acquiring Viaweb from Graham and
Morris for $megabucks, they canned LISP and rewrote the entire codebase
in some infidel PL.
Jul 13 2010
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/12/2010 03:22 PM, bearophile wrote:

 Steven Schveighoffer:

 bearophile:

 String mixins are a hack, just a bit better than C preprocessor
 (because they are scoped), they are not a replacement of clean
 macros.


 Interesting statement, can you back it up? :)  What can you do with
 a macro that you can't do with a mixin?


 In the meantime others have already given you some answers, I can add
 one example. The Brainfuck language allows you to do every kind of
 thing, I have seen even compilers written in it, but its usage is
 very unhandy, it's bug-prone, requires lot of code to do even simple
 things, and it's very hard to modify and debug programs written in
 it. What I meant is that string mixins in theory allow you to do many
 things, but in practice you can't use them for complex tasks, and you
 can't modify and fix the resulting code if you try to use them for
 more complex tasks. They are not a long-term solution for a language
 that seriously wants to improve over C and its preprocessor macros,
 they are a hack.

 In my opinion the std.bitmanip.bitfields souce code is already past
 the decency limit for string mixins, and I hope they will be replaced
 by something better.


Clearly a more structured approach would be better. On the other hand 
bitfields is literally the first code-generating tool I ever wrote, so I 
expect better idiomatic code will come about after the community's 
experience increases.


 I don't agree with Andrei when he says that CLisp macros (and even
 more Scheme macros that are hygienic too) are as hairy and
 unmantenable as code that uses string mixins heavily.


I said:

=======
Though I sympathise with all of the above, I should add that I have 
looked at languages that feature AST macros (Lisp, Scheme, Dylan) and 
such macros are difficult to create and read as well. We're not looking 
at day and night improvement there.
=======

and you claim, quite unequivocally, that I said:

=======
CLisp macros (and even more Scheme macros that are hygienic too) are as 
hairy and unmaintainable as code that uses string mixins heavily.
=======

I mean WTF?

Back on topic: http://www.apl.jhu.edu/~hall/Lisp-Notes/Macros.html. 
Compare the simple macro in the beginning and the correct macro at the end.


Andrei
Jul 12 2010
next sibling parent "Nick Sabalausky" <a a.a> writes:
"Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> wrote in message 
news:i1g5aq$2vkj$1 digitalmars.com...

 Back on topic: http://www.apl.jhu.edu/~hall/Lisp-Notes/Macros.html. 
 Compare the simple macro in the beginning and the correct macro at the 
 end.


I can't read ordinary lisp, but Nemerle's macros seem pretty damn easy to 
me:

http://nemerle.org/Macros_tutorial

And note that deconstructing the tree typically makes use of Nemerle's 
excellent pattern-matching:

http://nemerle.org/Grok_Variants_and_matching#Matching

I've long been of the opinion that Nemerle's CTFE, macros and pattern 
matching make D's counterparts look terrible in comparison (too bad Nemerle 
doesn't do low-level).
Jul 12 2010
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 Back on topic: http://www.apl.jhu.edu/~hall/Lisp-Notes/Macros.html. 
 Compare the simple macro in the beginning and the correct macro at the end.


CLisp macros are messy, they can be not easy to read and understand, and worse
of all they can fragment the community because every large program that defines
many complex macros essentially defines a new variety of CLisp. This makes it
hard to read, modify and reuse for other purposes CLisp code written by other
people (but allows the creation of domain-specific code, that can reduce
program size and make it simpler). Python has refused macros mostly because of
this risk of fragmentation. A significant part of the success of Python comes
from the ecosystem of tons of little modules that you can download and use in
your code with usually only little effort. A large usage of macros can kill
this.

On the other hand I believe that D string mixins, if used to replace some of
the purposes of CLisp macros, are worse than CLisp macros.

Scheme (and maybe Clojure too) macros are more hygienic so they remove some of
the problems shown in that Macros.html page, this means that a good Square
macro written in Scheme looks better and is simpler than that CLips one (on the
other hand Scheme macros can be a little less powerful, I am not expert enough
to give you examples).

Bye,
bearophile
Jul 12 2010