www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP78 - macros without syntax extensions

reply "Kagamin" <spam here.lot> writes:
http://wiki.dlang.org/DIP78 - Proposal for a macro system without 
syntactical extensions to the language. Hence it doesn't allow 
arbitrary syntax.
May 26 2015
next sibling parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Tuesday, 26 May 2015 at 20:23:11 UTC, Kagamin wrote:

 http://wiki.dlang.org/DIP78 - Proposal for a macro system 
 without syntactical extensions to the language. Hence it 
 doesn't allow arbitrary syntax.


If this proposal is considered, it is required to propose to look
at the implementation of macros in Nemerle. Many believe that it
is in Nemerle macros implemented the most successful compared to
other modern languages. Of course, the most successful macros are
implemented in Lisp, but the syntax of the language is poor :)

Some information here:
http://www.nemerle.org/About

Great article about macros Nemerle, but in Russian:
http://rsdn.ru/article/nemerle/NemerleStingFormating.xml

Of course, it is necessary to look at Rust and Nim, but macros
Nemerle implemented better.
I also think that the D needed macros, but I sure would not want
to see the lite version of the macros. If implemented, then at
least to the end and complete! Or not to implement macros in
general...
Although if you create something in between lite and full version
of the macro, then I think it would be a big plus for D, so
entirely without macros somehow uncomfortable. Some mixins little.
Sometimes I think that D do not need these macros, and sometimes
it seems to me that they are severely lacking.Although the
competition is also necessary to consider the other languages
that have at least some macros, such as, Nim/Rust.
In general, D needs some macros.
May 26 2015
parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 26 May 2015 at 23:47:41 UTC, Dennis Ritchie wrote:

 If this proposal is considered, it is required to propose to 
 look
 at the implementation of macros in Nemerle. Many believe that it
 is in Nemerle macros implemented the most successful compared to
 other modern languages. Of course, the most successful macros 
 are
 implemented in Lisp, but the syntax of the language is poor :)


The problem with declarative macro system is that you would need 
to learn yet another language. Possibly turing-complete. And a 
declarative turing-complete language is an overkill both for 
usage and implementation. Imperative macros get it done in an 
intuitive way in the existing language.
May 27 2015
next sibling parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 08:14:36 UTC, Kagamin wrote:

 The problem with declarative macro system is that you would 
 need to learn yet another language. Possibly turing-complete. 
 And a declarative turing-complete language is an overkill both 
 for usage and implementation. Imperative macros get it done in 
 an intuitive way in the existing language.


Yes, declarative macros can destroy an imperative language, 
because they can change the syntax of the language beyond 
recognition. Ie macrosystem will be more powerful than the 
language itself. Unfortunately, Nemerle, Scala and turned :)
But, in my opinion, there should be options to combine 
declarative macros with imperative languages. You just need to 
properly limit these macros to a certain moment.
Is not it possible to combine in the desired degree of 
declarative macros with imperative languages?
May 27 2015
parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 09:31:33 UTC, Dennis Ritchie wrote:

 Is not it possible to combine in the desired degree of 
 declarative macros with imperative languages?


The point is to streamline ast creation. I think, helper methods 
can be provided by phobos in, say, std.macros to simplify ast 
creation for common constructs. Well, in a way libraries are a 
form of DSL...
May 27 2015
parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 09:40:35 UTC, Kagamin wrote:

 The point is to streamline ast creation. I think, helper 
 methods can be provided by phobos in, say, std.macros to 
 simplify ast creation for common constructs. Well, in a way 
 libraries are a form of DSL...


std.macros it may, this is the right way! At least, it would suit 
me :)
May 27 2015
prev sibling parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Wednesday, 27 May 2015 at 08:14:36 UTC, Kagamin wrote:

 On Tuesday, 26 May 2015 at 23:47:41 UTC, Dennis Ritchie wrote:

 If this proposal is considered, it is required to propose to 
 look
 at the implementation of macros in Nemerle. Many believe that 
 it
 is in Nemerle macros implemented the most successful compared 
 to
 other modern languages. Of course, the most successful macros 
 are
 implemented in Lisp, but the syntax of the language is poor :)


 The problem with declarative macro system is that you would 
 need to learn yet another language. Possibly turing-complete. 
 And a declarative turing-complete language is an overkill both 
 for usage and implementation. Imperative macros get it done in 
 an intuitive way in the existing language.


But D already has such a declarative language - the one used in 
template metaprogramming. I think a macro system that plays well 
with that template metaprogramming sub-language will be really 
nice. For example, CTFE that works like a macro and returns 
types/aliases:

     Auto deduceType(Auto args) {
         // some complex imperative code to deduce the type from 
the args
         return DeducedType;
     }

     struct Foo(T...) {
         deduceType(T) value;
     }
May 27 2015
parent reply Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 05/27/15 17:47, Idan Arye via Digitalmars-d wrote:

 On Wednesday, 27 May 2015 at 08:14:36 UTC, Kagamin wrote:

 On Tuesday, 26 May 2015 at 23:47:41 UTC, Dennis Ritchie wrote:

 If this proposal is considered, it is required to propose to look
 at the implementation of macros in Nemerle. Many believe that it
 is in Nemerle macros implemented the most successful compared to
 other modern languages. Of course, the most successful macros are
 implemented in Lisp, but the syntax of the language is poor :)


 The problem with declarative macro system is that you would need to learn yet
another language. Possibly turing-complete. And a declarative turing-complete
language is an overkill both for usage and implementation. Imperative macros
get it done in an intuitive way in the existing language.


 
 But D already has such a declarative language - the one used in template
metaprogramming. I think a macro system that plays well with that template
metaprogramming sub-language will be really nice. For example, CTFE that works
like a macro and returns types/aliases:
 
     Auto deduceType(Auto args) {
         // some complex imperative code to deduce the type from the args
         return DeducedType;
     }
 
     struct Foo(T...) {
         deduceType(T) value;
     }


That already works. Eg:

   alias deduceType(Args...) = typeof({
      // some complex imperative code to deduce the type from the args
      import std.range;
      return mixin(iota(Args.length).map!q{`Args[`~text(a)~']'}().join("+"));
   }());

   struct Foo(T...) {
      deduceType!(T) value;
   }

   static assert(is(typeof(Foo!(short, ubyte, bool).value)==int));


What all these proposals seem to be about is:

a) better introspection (ie exposing a (preferably simplified and std) AST)
b) AST injection
c) "better" syntax
d) better "optimizations", meaning skipping the emission of code and data
     that is never used at runtime.

artur
May 27 2015
parent "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 17:03:30 UTC, Artur Skawina wrote:

 That already works. Eg:

    alias deduceType(Args...) = typeof({
       // some complex imperative code to deduce the type from 
 the args
       import std.range;
       return 
 mixin(iota(Args.length).map!q{`Args[`~text(a)~']'}().join("+"));
    }());

    struct Foo(T...) {
       deduceType!(T) value;
    }

    static assert(is(typeof(Foo!(short, ubyte, 
 bool).value)==int));


 What all these proposals seem to be about is:

 a) better introspection (ie exposing a (preferably simplified 
 and std) AST)
 b) AST injection
 c) "better" syntax
 d) better "optimizations", meaning skipping the emission of 
 code and data
      that is never used at runtime.

 artur


Maybe extending capabilities of templates would be a better 
direction. They already accept aliases which are essentially 
simple hygienic identifier expressions.
May 28 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-05-26 22:23, Kagamin wrote:

 http://wiki.dlang.org/DIP78 - Proposal for a macro system without
 syntactical extensions to the language. Hence it doesn't allow arbitrary
 syntax.


I'm not sure how this is different for DIP50. I can see the obvious 
differences in the syntax how to declare a macro and so on. But 
otherwise they look very similar.

-- 
/Jacob Carlborg
May 26 2015
parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 06:54:56 UTC, Jacob Carlborg wrote:

 I'm not sure how this is different for DIP50. I can see the 
 obvious differences in the syntax how to declare a macro and so 
 on. But otherwise they look very similar.


I believe 78 allows a simpler implementation and requires less 
changes to the language, though I'm not sure if frontend can get 
it done on a purely semantical level: similar techniques like 
mixins and static if rely on syntax. Maybe a macro should be 
called with a special syntax, e.g. myAssert!(a==b);
May 27 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-05-27 10:06, Kagamin wrote:


 I believe 78 allows a simpler implementation and requires less changes
 to the language, though I'm not sure if frontend can get it done on a
 purely semantical level: similar techniques like mixins and static if
 rely on syntax.


DIP50 would require a minimal amount of syntax change, the only thing is 
prefixing a function with the "macro" keyword. DIP78 on the other hand, 
it's not so easy to see that a function declaration is actual a macro 
declaration.

In that case I would prefer the "macro" keyword. It's already a 
reserved, for exactly this purpose, so it will be backwards compatible.

I would say that neither of the DIP's contain a lot of detail. So it's 
hard to know which one is more complex. In DIP50 there's quite a lot of 
optional/bonus features. To me, implementing only the minimal 
requirements look very similar to DIP78.

One thing that would be more complex in DIP50 would be the "Context" 
class. Although I'm not sure if that's needed in DIP78 as well.


 Maybe a macro should be called with a special syntax,
 e.g. myAssert!(a==b);


I have thought of that too. But I haven't been able to come up with a 
syntax that looks good and doesn't conflict with any existing 
syntax/symbol. The above syntax is already used for template instantiation.

-- 
/Jacob Carlborg
May 27 2015
next sibling parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg wrote:

 I have thought of that too. But I haven't been able to come up 
 with a syntax that looks good and doesn't conflict with any 
 existing syntax/symbol. The above syntax is already used for 
 template instantiation.
May 27 2015
parent reply Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d writes:
On Wed, 27 May 2015 12:07:09 +0000
Dennis Ritchie via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg wrote:

 I have thought of that too. But I haven't been able to come up 
 with a syntax that looks good and doesn't conflict with any 
 existing syntax/symbol. The above syntax is already used for 
 template instantiation.


 



Yep, I like this symbol for macro too.
May 27 2015
next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:




 Yep, I like this symbol for macro too.


https://issues.dlang.org/show_bug.cgi?id=2660
May 27 2015
parent reply "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 12:37:51 UTC, Kagamin wrote:

 On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:




 Yep, I like this symbol for macro too.


 https://issues.dlang.org/show_bug.cgi?id=2660


OK. This symbol is already used in a wrong place :)

http://dlang.org/lex.html#special-token-sequence

You can still use `:`. The colon is used even to refer to some 
Lisp macros, for example:

(loop :for key :in keys
     :collect (cons key 0))

Still, as an option, you can use the `'` . This symbol is also 
used in Lisp, in some places, for example:

(key-weight 'sweet)

Or apostrophes, too, somewhere involved in the D?
May 27 2015
next sibling parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 12:55:05 UTC, Dennis Ritchie wrote:

 Or apostrophes, too, somewhere involved in the D?


Similarly, this option is no longer, as is used as:
char c = 'c';

Though...
May 27 2015
prev sibling parent reply "Meta" <jared771 gmail.com> writes:
On Wednesday, 27 May 2015 at 12:55:05 UTC, Dennis Ritchie wrote:

 On Wednesday, 27 May 2015 at 12:37:51 UTC, Kagamin wrote:

 On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:




 Yep, I like this symbol for macro too.


 https://issues.dlang.org/show_bug.cgi?id=2660


 OK. This symbol is already used in a wrong place :)

 http://dlang.org/lex.html#special-token-sequence

 You can still use `:`. The colon is used even to refer to some 
 Lisp macros, for example:

 (loop :for key :in keys
     :collect (cons key 0))

 Still, as an option, you can use the `'` . This symbol is also 
 used in Lisp, in some places, for example:

 (key-weight 'sweet)

 Or apostrophes, too, somewhere involved in the D?


As far as I know, the : and ' symbols in Lisp don't have anything 
to do with macros. : is for keyword arguments and ' is for 
creating AST literals. It makes sense that these would be heavily 
used with macros, of course, but they are not part of Lisp's 
macro system.
May 27 2015
parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 13:12:05 UTC, Meta wrote:

 As far as I know, the : and ' symbols in Lisp don't have 
 anything to do with macros. : is for keyword arguments and ' is 
 for creating AST literals. It makes sense that these would be 
 heavily used with macros, of course, but they are not part of 
 Lisp's macro system.


Yes, but IMO, these symbols can be activated in the future 
macrosystem D.
May 27 2015
prev sibling parent reply "ixid" <adamsibson hotmail.com> writes:
On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:

 On Wed, 27 May 2015 12:07:09 +0000
 Dennis Ritchie via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:


 On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg 
 wrote:

 I have thought of that too. But I haven't been able to come 
 up with a syntax that looks good and doesn't conflict with 
 any existing syntax/symbol. The above syntax is already used 
 for template instantiation.


 



 Yep, I like this symbol for macro too.


What's wrong with using the word macro? We don't want symbol soup.
May 27 2015
parent reply "weaselcat" <weaselcat gmail.com> writes:
On Wednesday, 27 May 2015 at 12:39:52 UTC, ixid wrote:

 On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:

 On Wed, 27 May 2015 12:07:09 +0000
 Dennis Ritchie via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:


 On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg 
 wrote:

 I have thought of that too. But I haven't been able to come 
 up with a syntax that looks good and doesn't conflict with 
 any existing syntax/symbol. The above syntax is already 
 used for template instantiation.


 



 Yep, I like this symbol for macro too.


 What's wrong with using the word macro? We don't want symbol 
 soup.


this, there's a reason people don't use lisp.
May 27 2015
parent "Dennis Ritchie" <dennis.ritchie mail.ru> writes:
On Wednesday, 27 May 2015 at 15:20:38 UTC, weaselcat wrote:

 On Wednesday, 27 May 2015 at 12:39:52 UTC, ixid wrote:

 On Wednesday, 27 May 2015 at 12:34:49 UTC, Daniel Kozák wrote:

 On Wed, 27 May 2015 12:07:09 +0000
 Dennis Ritchie via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:


 On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg 
 wrote:

 I have thought of that too. But I haven't been able to 
 come up with a syntax that looks good and doesn't conflict 
 with any existing syntax/symbol. The above syntax is 
 already used for template instantiation.


 



 Yep, I like this symbol for macro too.


 What's wrong with using the word macro? We don't want symbol 
 soup.


 this, there's a reason people don't use lisp.


Lisp is not used due to other reasons, symbol soup typical for 
Perl :)
$? ? s:;s:s;;$?: : s;;=]=>%-{<-|}<&|`{; ;
y; -/:- [-`{-};`-{/" -; ;
s;;$_;see
May 27 2015
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 27 May 2015 at 11:53:00 UTC, Jacob Carlborg wrote:

 DIP50 would require a minimal amount of syntax change, the only 
 thing is prefixing a function with the "macro" keyword. DIP78 
 on the other hand, it's not so easy to see that a function 
 declaration is actual a macro declaration.


Well, that's the point: the function is a normal function, only 
some of its parameters require specially prepared arguments, this 
can't be missed as soon as arguments are passed to the respective 
parameters.


 In that case I would prefer the "macro" keyword. It's already a 
 reserved, for exactly this purpose, so it will be backwards 
 compatible.


Well, maybe, I just didn't need the keyword.


 One thing that would be more complex in DIP50 would be the 
 "Context" class. Although I'm not sure if that's needed in 
 DIP78 as well.


No, passing of Context is not proposed.


 Maybe a macro should be called with a special syntax,
 e.g. myAssert!(a==b);


 I have thought of that too. But I haven't been able to come up 
 with a syntax that looks good and doesn't conflict with any 
 existing syntax/symbol. The above syntax is already used for 
 template instantiation.


I mean, the template instantiation syntax can inform the compiler 
that the expression is evaluated at compile time with possible 
code generation, so that the compiler is prepared to what macro 
will do. This resembles similarity between macros and templates. 
If macros can use existing syntax of a function call, I see no 
problem if they use another existing syntax.
May 27 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-05-27 14:17, Kagamin wrote:


 Well, that's the point: the function is a normal function, only some of
 its parameters require specially prepared arguments, this can't be
 missed as soon as arguments are passed to the respective parameters.


I prefer to be more explicit in this case, especially since the keyword 
is already available.


 In that case I would prefer the "macro" keyword. It's already a
 reserved, for exactly this purpose, so it will be backwards compatible.


 Well, maybe, I just didn't need the keyword.


I don't need it either, it's just what I preferred.


 No, passing of Context is not proposed.


The feeling I have it that it's hard to know if it's needed or not 
without implementing/using the macro system.


 I mean, the template instantiation syntax can inform the compiler that
 the expression is evaluated at compile time with possible code
 generation, so that the compiler is prepared to what macro will do. This
 resembles similarity between macros and templates. If macros can use
 existing syntax of a function call, I see no problem if they use another
 existing syntax.


True, but how would that the syntax look like for template macro, if 
possible?

-- 
/Jacob Carlborg
May 27 2015
parent reply "Idan Arye" <GenericNPC gmail.com> writes:
On Wednesday, 27 May 2015 at 19:15:34 UTC, Jacob Carlborg wrote:

 On 2015-05-27 14:17, Kagamin wrote:


 Well, that's the point: the function is a normal function, 
 only some of
 its parameters require specially prepared arguments, this 
 can't be
 missed as soon as arguments are passed to the respective 
 parameters.


 I prefer to be more explicit in this case, especially since the 
 keyword is already available.


I think it's more important to be explicit in the macro 
invocation than in the macro declaration. You can tell from the 
macro declaration that it's a macro you are looking at, even 
without the `macro` keyword, but the proposed syntaxes offer no 
no way to tell between a regular function call and a macro 
invocation by looking at the callsite alone.
May 27 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-05-27 21:41, Idan Arye wrote:


 I think it's more important to be explicit in the macro invocation than
 in the macro declaration. You can tell from the macro declaration that
 it's a macro you are looking at, even without the `macro` keyword,


It depends. As far as I can see, just looking at the macro declaration 
it's just a regular function declaration. You need to look at the types.

First, you need to know that "Auto" is a special type. Second, you need 
to know that it's only a special type if the fully qualified name is 
"core.macros.Auto".

It's not enough to look at the imports in the current module, since D 
supports public imports:

module foo;

public import core.macros;

module bar;

import foo;

Auto myAssert(Auto condition, Auto message)
{
     return message;
}

Not so easy to tell that "myAssert" is a macro declaration just by 
looking in the module "bar".

-- 
/Jacob Carlborg
May 27 2015
parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 28 May 2015 at 06:39:11 UTC, Jacob Carlborg wrote:

 Auto myAssert(Auto condition, Auto message)
 {
     return message;
 }

 Not so easy to tell that "myAssert" is a macro declaration just 
 by looking in the module "bar".


Well, that's the point: it's a normal function.
May 28 2015
parent Jacob Carlborg <doob me.com> writes:
On 2015-05-28 10:24, Kagamin wrote:


 Well, that's the point: it's a normal function.


But it's not.

-- 
/Jacob Carlborg
May 28 2015
prev sibling next sibling parent "ZombineDev" <valid_email he.re> writes:
On Tuesday, 26 May 2015 at 20:23:11 UTC, Kagamin wrote:

 http://wiki.dlang.org/DIP78 - Proposal for a macro system 
 without syntactical extensions to the language. Hence it 
 doesn't allow arbitrary syntax.


I really like how powerful and easy to use are AST macros in nim. 
IMO, if D gets such, it would become the language with the best 
meta-programming abilities D -> :D
Here's a nice presentation about them:
http://www.infoq.com/presentations/nimrod
May 27 2015
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 5/26/15 2:23 PM, Kagamin wrote:

 http://wiki.dlang.org/DIP78 - Proposal for a macro system without
 syntactical extensions to the language. Hence it doesn't allow arbitrary
 syntax.


Quote from dconf W&A ask me anything:

Q: "Will we get a macro system"

Both Walter and Andrei: "no"

Doesn't seem like much of a point for this proposal then. There didn't 
seem to be any wiggle room.

-Steve
May 28 2015
parent "Kagamin" <spam here.lot> writes:
On Thursday, 28 May 2015 at 13:34:24 UTC, Steven Schveighoffer 
wrote:

 Quote from dconf W&A ask me anything:

 Q: "Will we get a macro system"

 Both Walter and Andrei: "no"


It's safe to close the corresponding DIPs?
May 29 2015
prev sibling parent "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 26 May 2015 at 20:23:11 UTC, Kagamin wrote:

 http://wiki.dlang.org/DIP78 - Proposal for a macro system 
 without syntactical extensions to the language. Hence it 
 doesn't allow arbitrary syntax.


This is not even remotely enough to make a DIP. You seem to be 
using some API to build AST nodes, but this API is completely 
undefined.

Generally, you don't want an API to create AST. Both the 
programmer and the compiler already know one: the language syntax 
itself.
May 28 2015