www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Anyone interested in a working grammar for D?

reply Kelly Wilson <Kelly_member pathlink.com> writes:
Hey everyone,

Great language Walter et al. I saw this language and the active group here when
I was seeking a project to work on in my spare time. I decided to see if I could
make a usable grammar for D and start building a new compiler (should only take
a few weeks for a new optimizing compiler, I figured.......just kidding ;).

Anyways, I have a mostly working GLR grammar that was originally based on Ivan
Senji's Dgrammar(I hope that using your grammar was ok Ivan...if you don't want
me to release this then let me know, as I have to release this under the license
for Elkhound...I don't know if your license is compatible?? Or should I just
include your license? ...I don't know how those things work). 

There are 4 constructs that I have found which won't parse....I'm sure there are
more. I can, however, parse Mango and phobos (minus the four cases ;) with this
grammar, so I think I have found most problems. The grammar contains a ton of
conflicts (about 800) and about 10 ambiguities but these are resolved using a
great parser generator called Elkhound. I can get rid of many of the conflicts
with ease (about 400), but then a few rarely used constructs wouldn't parse.

I have a lexer based on the one supplied with the Elkhound/Elsa (the C++ parser
and therefore not a totally correct lexer for D...I'll work on it later) and a
sort of mini project set up to build the lexer and parser. The parser takes one
file and parses it, reporting syntax errors using the automated Elkhound
reporting mechanisms. The parse tree (along with the ambiguities) can be viewed
using a commmand line switch.

The Abstract Sytax Tree is also being generated (though it is woefully lacking
information, at this point) by Elkhound. My next step will be to populate the
AST with meaningful info and start on semantic checking. Of course, since there
is no semantic checking at this point, my parser accepts too many programs. Most
of the time when I try to parse something really strange and it works, I can
just check the program using dmd and see a semantic error (ie. I rarely produce
any program which I can parse but dmd will fail with a grammatical
error....though I haven't written a ton of strange programs to check at this
point ;)

Anyways, I just thought I would let everyone know that I have this small project
underway. If anyone is interested in it please email me (you can find my address
on my University of Calgary website for now. Just search for "Kelly Wilson
computer science" on google. I am at the top of the resulting page :) I should
be able to send you a copy to twiddle around with if you want one.

Hope everyone is doing well,
K.Wilson

P.S. I think MSTA/COCOM could also use this grammar (changed syntax of course)
with success, as I had started out using MSTA.
Feb 23 2006
parent reply Brad Roberts <braddr puremagic.com> writes:
On Thu, 23 Feb 2006, Kelly Wilson wrote:


 Hey everyone,
 
 Great language Walter et al. I saw this language and the active group here when
 I was seeking a project to work on in my spare time. I decided to see if I
could
 make a usable grammar for D and start building a new compiler (should only take
 a few weeks for a new optimizing compiler, I figured.......just kidding ;).
 
 Anyways, I have a mostly working GLR grammar that was originally based on Ivan
 Senji's Dgrammar(I hope that using your grammar was ok Ivan...if you don't want
 me to release this then let me know, as I have to release this under the
license
 for Elkhound...I don't know if your license is compatible?? Or should I just
 include your license? ...I don't know how those things work). 


Outside academic interest, have you considered using dmd's front end 
directly?  It is freely available (by several definitions).  Since it's 
the 'one true source' that defines the language in practice, re-use of it 
would make for a much more accurate tool to build on top of.

I don't mean to downplay your effort.  Rewriting something can often be 
both fun and educational.  Sometimes it even results in something truely 
better.  But it is, as you've found, a lot of effort to get to be perfect.

Walter releases the source to the front end in large part to help 
eliminate the duplication of effort.  It's what gdc uses to build the d 
front end for gcc.  The bulk of the now code in gdc is the glue layer 
between dmd's frontend and gcc's IR.  The rest is the runtime support to 
work with gcc and linux's runtime (exceptions being a good example).

Later,
Brad
Feb 23 2006
parent reply kellywilson nowhere.com writes:
In article <Pine.LNX.4.64.0602231405380.30259 bellevue.puremagic.com>, Brad
Roberts says...

Outside academic interest, have you considered using dmd's front end 
directly?  It is freely available (by several definitions).  Since it's 
the 'one true source' that defines the language in practice, re-use of it 
would make for a much more accurate tool to build on top of.


Mostly academic interest here Brad. For future work I need a complete AST in a
managable (and familiar) format.


I don't mean to downplay your effort.  Rewriting something can often be 
both fun and educational.  Sometimes it even results in something truely 
better.  But it is, as you've found, a lot of effort to get to be perfect.


Extremely fun and not perfect, as you say. But that is beside the point, for me.


Walter releases the source to the front end in large part to help 
eliminate the duplication of effort.  It's what gdc uses to build the d 
front end for gcc.  The bulk of the now code in gdc is the glue layer 
between dmd's frontend and gcc's IR.  The rest is the runtime support to 
work with gcc and linux's runtime (exceptions being a good example).


Understood. The problem here is that I have looked at the "glue" and it is not
what I would call "easily readable" for someone unfamiliar with gcc internals. I
also have seen several attempts at producing a grammar in the past (Charlie,
Ivan and someone with an ANTLR grammar but I can't remember his name off hand).
Just wanted to make one available, if some people wanted it (I also wished to
see Elkhound a little more in depth). 

Ivan emailed me within minutes stating that he would like to see the grammar, as
he was about to revisit the problem....thus eliminating some duplication of
effort ;)

By the way, if there is a free dmd compiler then why did someone go through the
effort of gluing the gcc backend on? I think "there is always more than one way
to skin a cat" ;)

Thanks,
K.Wilson
Feb 23 2006
parent reply "Marcus R. Brown" <mrbrown budcat.com> writes:
kellywilson nowhere.com wrote:

 By the way, if there is a free dmd compiler then why did someone go
 through the effort of gluing the gcc backend on? I think "there is always
 more than one way to skin a cat" ;)


DMD only targets Win32 and Linux (x86).  GDC can potentially target any
platform supported by GCC, with the proper platform support added to phobos.
Feb 23 2006
next sibling parent reply kellywilson nowhere.com writes:
In article <dtlecg$1o84$1 digitaldaemon.com>, Marcus R. Brown says...

kellywilson nowhere.com wrote:

 By the way, if there is a free dmd compiler then why did someone go
 through the effort of gluing the gcc backend on? I think "there is always
 more than one way to skin a cat" ;)


DMD only targets Win32 and Linux (x86).  GDC can potentially target any
platform supported by GCC, with the proper platform support added to phobos.


Didn't realise dmd was only targeted at those platforms. I assume that phobos
support is lagging for any others, though. To be remedied by the active
community I see here, I hope ;)

Thanks for the heads up Marcus,
K.Wilson

By the way, I was just suggesting that someone else had fun with gdc so there is
no harm in me trying some new directions as well.
Feb 23 2006
parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Kelly Wilson wrote:


 Didn't realise dmd was only targeted at those platforms. I assume that phobos
 support is lagging for any others, though. To be remedied by the active
 community I see here, I hope ;)


gphobos,
the GNU "version" of Phobos that comes with GDC, works on others too.

Currently GDC works on:
* Linux x86 (PPC coming)
* Mac OS X 10.2 - 10.4
* FreeBSD
* Cygwin / MinGW
* Solaris
* AIX
* SkyOS (unstable)

The main item for the GDC release we're waiting on, is 64-bit support.

--anders


PS. There is a separate D.gnu newsgroup, for things specific to GNU D.
Feb 23 2006
prev sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Marcus R. Brown wrote:


 By the way, if there is a free dmd compiler then why did someone go

 through the effort of gluing the gcc backend on? I think "there is always
 more than one way to skin a cat"  ;) 





 DMD only targets Win32 and Linux (x86).  GDC can potentially target any
 platform supported by GCC, with the proper platform support added to phobos.


Also,
GDC is under the GNU General Public License - while DMD is proprietary.
(i.e.: the back-end is non-distributable, the front-end is GPL/Artistic)

Mac OS X is a popular target for GDC, we have both PowerPC and Intel...
http://sourceforge.net/project/showfiles.php?group_id=135857&package_id=149174

--anders
Feb 23 2006