www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Compiling DMD parser?

reply Markus Koskimies <markus reaaliaika.net> writes:
Hi,

I thought that I could play around a little bit with DMD parser. I have 
DMD 1.030 parser sources, and I made a simple makefile for just compiling 
those sources (with gcc 4.3.1).

---
Some questions:

- These are C++ sources, not C sources, although the suffix is .c and .h, 
right? A little bit sad, I thought that I could wrap the parser to D :( I 
have not much interest playing with C++ at freetime...

- Are these files meant to be compiled alone without making modifications 
to files? For example:

- In "attrib.c", if linux is defined, it tries to include mem.h from "../
root"; why not the one in the src/ directory?

- There is no such file that "id.h", which probably contains class/struct/
enum "Id"

- init.h does not include "hdrgen.h", while uses HdrGenState

---
How these parser codes are meant to be used? Am I missing something?
Jul 12 2008
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Markus Koskimies wrote:

 Hi,
 
 I thought that I could play around a little bit with DMD parser. I have 
 DMD 1.030 parser sources, and I made a simple makefile for just compiling 
 those sources (with gcc 4.3.1).
 
 ---
 Some questions:
 
 - These are C++ sources, not C sources, although the suffix is .c and .h, 
 right? A little bit sad, I thought that I could wrap the parser to D :( I 
 have not much interest playing with C++ at freetime...
 
 - Are these files meant to be compiled alone without making modifications 
 to files? For example:
 
 - In "attrib.c", if linux is defined, it tries to include mem.h from "../
 root"; why not the one in the src/ directory?
 
 - There is no such file that "id.h", which probably contains class/struct/
 enum "Id"
 
 - init.h does not include "hdrgen.h", while uses HdrGenState
 
 ---
 How these parser codes are meant to be used? Am I missing something?


They aren't.  They're just meant to be for anyone who is interested in 
taking them and trying to build a compiler or other tool out of them. 
They don't compile into anything useful as-is.

I think there's a project, DMDFE, based on those sources that is 
compileable though.  http://www.dsource.org/projects/dmdfe


--bb
Jul 12 2008
next sibling parent Markus Koskimies <markus reaaliaika.net> writes:
On Sun, 13 Jul 2008 13:58:27 +0900, Bill Baxter wrote:


 They aren't.  They're just meant to be for anyone who is interested in
 taking them and trying to build a compiler or other tool out of them.
 They don't compile into anything useful as-is.


Well, I thought that they could be compiled to object files, and you 
would be able to wrap & link them to your own project... But if they are 
meant just to be rewritten, that's of course different story.
 

 I think there's a project, DMDFE, based on those sources that is
 compileable though.  http://www.dsource.org/projects/dmdfe


I'll check it out.
Jul 12 2008
prev sibling parent reply Markus Koskimies <markus reaaliaika.net> writes:
On Sun, 13 Jul 2008 13:58:27 +0900, Bill Baxter wrote:


 I think there's a project, DMDFE, based on those sources that is
 compileable though.  http://www.dsource.org/projects/dmdfe


Yes, that worked, thanks!

Next I'll try to figure out, how to easily import the parsed results to 
D... Hmmh, I might have an idea; if I modify that DMDFE to output the 
parse tree in some kind of ASCII stream, I might be able to construct it 
in D; hmmh, that sounds somewhat silly... Well, I try to figure out 
something. Again, thanks for that link! :D
Jul 12 2008
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Markus Koskimies wrote:

 On Sun, 13 Jul 2008 13:58:27 +0900, Bill Baxter wrote:
 

 I think there's a project, DMDFE, based on those sources that is
 compileable though.  http://www.dsource.org/projects/dmdfe


 
 Yes, that worked, thanks!
 
 Next I'll try to figure out, how to easily import the parsed results to 
 D... Hmmh, I might have an idea; if I modify that DMDFE to output the 
 parse tree in some kind of ASCII stream, I might be able to construct it 
 in D; hmmh, that sounds somewhat silly... Well, I try to figure out 
 something. Again, thanks for that link! :D


No need to go ascii.  Just wrap up the data in some structs and add some 
extern(C) functions that you can call from D which return those structs.

Or if you're doing D2.0, then there's that extern(C++) thing that may work.

--bb
Jul 12 2008
parent Markus Koskimies <markus reaaliaika.net> writes:
On Sun, 13 Jul 2008 14:31:29 +0900, Bill Baxter wrote:


 Markus Koskimies wrote:

 On Sun, 13 Jul 2008 13:58:27 +0900, Bill Baxter wrote:
 

 I think there's a project, DMDFE, based on those sources that is
 compileable though.  http://www.dsource.org/projects/dmdfe


 
 Yes, that worked, thanks!
 
 Next I'll try to figure out, how to easily import the parsed results to
 D... Hmmh, I might have an idea; if I modify that DMDFE to output the
 parse tree in some kind of ASCII stream, I might be able to construct
 it in D; hmmh, that sounds somewhat silly... Well, I try to figure out
 something. Again, thanks for that link! :D


 
 No need to go ascii.  Just wrap up the data in some structs and add some
 extern(C) functions that you can call from D which return those structs.


Yes, I thought that, too, but I thought that it could be quite an effort. 
Maybe later :) I thought that if I put the dmdfe dumping info from 
parsing, the same can be used for importing the tree to D. But I'll play 
around for awhile to find a suitable way of doing it.


 Or if you're doing D2.0, then there's that extern(C++) thing that may
 work.


Does it work with g++? I haven't yet used D2.0, but maybe it is time to 
try it out, too :)
Jul 12 2008
prev sibling next sibling parent reply Lars Ivar Igesund <larsivar igesund.net> writes:
Markus Koskimies wrote:


 Hi,
 
 I thought that I could play around a little bit with DMD parser. I have
 DMD 1.030 parser sources, and I made a simple makefile for just compiling
 those sources (with gcc 4.3.1).
 
 ---
 Some questions:
 
 - These are C++ sources, not C sources, although the suffix is .c and .h,
 right? A little bit sad, I thought that I could wrap the parser to D :( I
 have not much interest playing with C++ at freetime...
 
 - Are these files meant to be compiled alone without making modifications
 to files? For example:
 
 - In "attrib.c", if linux is defined, it tries to include mem.h from "../
 root"; why not the one in the src/ directory?
 
 - There is no such file that "id.h", which probably contains class/struct/
 enum "Id"
 
 - init.h does not include "hdrgen.h", while uses HdrGenState
 
 ---
 How these parser codes are meant to be used? Am I missing something?


The DParser project on dsource may be of interest too, it is a D port of the
parsing parts of the DMD frontend.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource, #d.tango & #D: larsivi
Dancing the Tango
Jul 12 2008
parent Markus Koskimies <markus reaaliaika.net> writes:
On Sun, 13 Jul 2008 08:23:32 +0200, Lars Ivar Igesund wrote:


 The DParser project on dsource may be of interest too, it is a D port of
 the parsing parts of the DMD frontend.


Aargh, I have constantly mixed it with this one (this is found if you 
google it):

http://dparser.sourceforge.net/

I have wondered that what it really has to do with D... But now I figured 
out, that using:

http://www.dsource.org/projects/dparser

It's exactly what I expected it to be :D
Jul 13 2008
prev sibling next sibling parent reply "Koroskin Denis" <2korden+dmd gmail.com> writes:
On Sun, 13 Jul 2008 08:33:54 +0400, Markus Koskimies  
<markus reaaliaika.net> wrote:


 Hi,

 I thought that I could play around a little bit with DMD parser. I have
 DMD 1.030 parser sources, and I made a simple makefile for just compiling
 those sources (with gcc 4.3.1).

 ---
 Some questions:

 - These are C++ sources, not C sources, although the suffix is .c and .h,
 right? A little bit sad, I thought that I could wrap the parser to D :( I
 have not much interest playing with C++ at freetime...

 - Are these files meant to be compiled alone without making modifications
 to files? For example:

 - In "attrib.c", if linux is defined, it tries to include mem.h from "../
 root"; why not the one in the src/ directory?

 - There is no such file that "id.h", which probably contains  
 class/struct/
 enum "Id"

 - init.h does not include "hdrgen.h", while uses HdrGenState

 ---
 How these parser codes are meant to be used? Am I missing something?


I'm working on something that could be of interest to you.
I have a working DMD to D bridge via C abi. It's not complete but most  
important data structures are available from D, i.e. you can pass some .d  
files to it and get an AST back. I've written a small source-code analyzer  
as a proof-of-concept with it.

Think of it as a DParser (http://www.dsource.org/projects/dparser) but  
with official implementation. A huge plus is that almost no maintaining  
effort needed.

Bridge is written from scratch with a mix of C++ and D1 and is not based  
on DMDFE (http://www.dsource.org/projects/dmdfe). I haven't released the  
sources yet, so contact me if you would like to look at the them.
Jul 13 2008
parent reply Markus Koskimies <markus reaaliaika.net> writes:
On Mon, 14 Jul 2008 02:41:34 +0400, Koroskin Denis wrote:


 Think of it as a DParser (http://www.dsource.org/projects/dparser) but
 with official implementation. A huge plus is that almost no maintaining
 effort needed.


Yes, I was already wondering, how DParser will keep up-to-date. It would 
be a huge effort to go through the new C++ files and check, what need to 
be changed in DParser...


 Bridge is written from scratch with a mix of C++ and D1 and is not based
 on DMDFE (http://www.dsource.org/projects/dmdfe). I haven't released the
 sources yet, so contact me if you would like to look at the them.


I put some email :)
Jul 13 2008
next sibling parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Markus Koskimies wrote:

 On Mon, 14 Jul 2008 02:41:34 +0400, Koroskin Denis wrote:
 

 Think of it as a DParser (http://www.dsource.org/projects/dparser) but
 with official implementation. A huge plus is that almost no maintaining
 effort needed.


 
 Yes, I was already wondering, how DParser will keep up-to-date. It would 
 be a huge effort to go through the new C++ files and check, what need to 
 be changed in DParser...


Given that DParser is apparently still back at version 1.014, I think 
the answer to how it will keep up-to-date is simply "it won't".

--bb
Jul 13 2008
prev sibling parent Leandro Lucarella <llucax gmail.com> writes:
Markus Koskimies, el 13 de julio a las 23:11 me escribiste:

 On Mon, 14 Jul 2008 02:41:34 +0400, Koroskin Denis wrote:
 

 Think of it as a DParser (http://www.dsource.org/projects/dparser) but
 with official implementation. A huge plus is that almost no maintaining
 effort needed.


 
 Yes, I was already wondering, how DParser will keep up-to-date. It would 
 be a huge effort to go through the new C++ files and check, what need to 
 be changed in DParser...


It would be much more easy if the DMD official frontend were in a public
repository like Phobos, and one could track it's development in a daily
basis, not dealing with a huge contextless diff made from 2 released
versions of DMD.

Please, please, please walter, put the frontend in a public repository =)

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
Ya ni el cielo me quiere, ya ni la muerte me visita
Ya ni el sol me calienta, ya ni el viento me acaricia
Jul 14 2008
prev sibling parent reply davidl <davidl 126.com> writes:
在 Sun, 13 Jul 2008 12:33:54 +0800，Markus Koskimies  
<markus reaaliaika.net> 写道:


 Hi,

 I thought that I could play around a little bit with DMD parser. I have
 DMD 1.030 parser sources, and I made a simple makefile for just compiling
 those sources (with gcc 4.3.1).

 ---
 Some questions:

 - These are C++ sources, not C sources, although the suffix is .c and .h,
 right? A little bit sad, I thought that I could wrap the parser to D :( I
 have not much interest playing with C++ at freetime...

 - Are these files meant to be compiled alone without making modifications
 to files? For example:

 - In "attrib.c", if linux is defined, it tries to include mem.h from "../
 root"; why not the one in the src/ directory?

 - There is no such file that "id.h", which probably contains  
 class/struct/
 enum "Id"

 - init.h does not include "hdrgen.h", while uses HdrGenState

 ---
 How these parser codes are meant to be used? Am I missing something?


actually dparser is not 1.014 anymore. it's somewhat 1.024ish
the parsing ability has been improved a lot, it can parse itself, and some  
ctfes.
700 or so cases from dstress fail.


-- 
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/
Jul 13 2008
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
davidl wrote:

 在 Sun, 13 Jul 2008 12:33:54 +0800，Markus Koskimies 
 <markus reaaliaika.net> 写道:
 

 Hi,

 I thought that I could play around a little bit with DMD parser. I have
 DMD 1.030 parser sources, and I made a simple makefile for just compiling
 those sources (with gcc 4.3.1).

 ---
 Some questions:

 - These are C++ sources, not C sources, although the suffix is .c and .h,
 right? A little bit sad, I thought that I could wrap the parser to D :( I
 have not much interest playing with C++ at freetime...

 - Are these files meant to be compiled alone without making modifications
 to files? For example:

 - In "attrib.c", if linux is defined, it tries to include mem.h from "../
 root"; why not the one in the src/ directory?

 - There is no such file that "id.h", which probably contains 
 class/struct/
 enum "Id"

 - init.h does not include "hdrgen.h", while uses HdrGenState

 ---
 How these parser codes are meant to be used? Am I missing something?


 
 actually dparser is not 1.014 anymore. it's somewhat 1.024ish
 the parsing ability has been improved a lot, it can parse itself, and 
 some ctfes.
 700 or so cases from dstress fail.


I hope you don't mind that I too the liberty of updating the dparser 
wiki page to reflect that information.

--bb
Jul 13 2008
parent davidl <davidl 126.com> writes:
在 Mon, 14 Jul 2008 11:32:57 +0800，Bill Baxter  
<dnewsgroup billbaxter.com> 写道:


 davidl wrote:

 在 Sun, 13 Jul 2008 12:33:54 +0800，Markus Koskimies  
 <markus reaaliaika.net> 写道:


 Hi,

 I thought that I could play around a little bit with DMD parser. I have
 DMD 1.030 parser sources, and I made a simple makefile for just  
 compiling
 those sources (with gcc 4.3.1).

 ---
 Some questions:

 - These are C++ sources, not C sources, although the suffix is .c and  
 .h,
 right? A little bit sad, I thought that I could wrap the parser to D  
 :( I
 have not much interest playing with C++ at freetime...

 - Are these files meant to be compiled alone without making  
 modifications
 to files? For example:

 - In "attrib.c", if linux is defined, it tries to include mem.h from  
 "../
 root"; why not the one in the src/ directory?

 - There is no such file that "id.h", which probably contains  
 class/struct/
 enum "Id"

 - init.h does not include "hdrgen.h", while uses HdrGenState

 ---
 How these parser codes are meant to be used? Am I missing something?


  actually dparser is not 1.014 anymore. it's somewhat 1.024ish
 the parsing ability has been improved a lot, it can parse itself, and  
 some ctfes.
 700 or so cases from dstress fail.


 I hope you don't mind that I too the liberty of updating the dparser  
 wiki page to reflect that information.

 --bb


Appreciated!

Wish you get envolved! :D

-- 
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/
Jul 13 2008
prev sibling parent reply Markus Koskimies <markus reaaliaika.net> writes:
On Mon, 14 Jul 2008 11:26:13 +0800, davidl wrote:


 actually dparser is not 1.014 anymore. it's somewhat 1.024ish the
 parsing ability has been improved a lot, it can parse itself, and some
 ctfes.
 700 or so cases from dstress fail.


:o

I fetched the sources (I tried to select the latest ones), and attached 
that DParser to my own code. I copied the set-ups from trunk/d1.0/dparser/
dmd/main.d, and do parsing and semantic passes 1 to 3 to build the parsed 
tree.

Problems:

- Parser segfaults, if it parses std.stdio

- I'm trying to write a tree walker; I have walk(X) functions for 
different types of elements (Module, FuncDeclaration, 
DeclStatement, ...). But I have hard times to determine the types of 
statements, since Statement class contains only a few "isXXXX()" 
functions. I tried to use typeid(typeof(s)), but it returns - as you may 
guess - always the type that was declared in the function arguments.

---
I use DMD1.030, and I have those 1.030 phobos sources.

I'll try to figure out how to extract that statement type information 
from parse tree, and could wait for newer versions of DParser.

At the same time I'll try to get dil compiled; although dil does not 
generate code, it might still be a suitable platform to write a software 
to generate some common warnings (just for my personal use, I'm not 
intended to start to write "dlint"). Unfortunately it means that I need 
to figure out how those DSSS & Tango things work :)
Jul 13 2008
next sibling parent reply davidl <davidl 126.com> writes:
在 Mon, 14 Jul 2008 12:07:00 +0800，Markus Koskimies  
<markus reaaliaika.net> 写道:


 On Mon, 14 Jul 2008 11:26:13 +0800, davidl wrote:


 actually dparser is not 1.014 anymore. it's somewhat 1.024ish the
 parsing ability has been improved a lot, it can parse itself, and some
 ctfes.
 700 or so cases from dstress fail.


 :o

 I fetched the sources (I tried to select the latest ones), and attached
 that DParser to my own code. I copied the set-ups from  
 trunk/d1.0/dparser/
 dmd/main.d, and do parsing and semantic passes 1 to 3 to build the parsed
 tree.

 Problems:

 - Parser segfaults, if it parses std.stdio

 - I'm trying to write a tree walker; I have walk(X) functions for
 different types of elements (Module, FuncDeclaration,
 DeclStatement, ...). But I have hard times to determine the types of
 statements, since Statement class contains only a few "isXXXX()"
 functions. I tried to use typeid(typeof(s)), but it returns - as you may
 guess - always the type that was declared in the function arguments.

 ---
 I use DMD1.030, and I have those 1.030 phobos sources.

 I'll try to figure out how to extract that statement type information
 from parse tree, and could wait for newer versions of DParser.

 At the same time I'll try to get dil compiled; although dil does not
 generate code, it might still be a suitable platform to write a software
 to generate some common warnings (just for my personal use, I'm not
 intended to start to write "dlint"). Unfortunately it means that I need
 to figure out how those DSSS & Tango things work :)


err, i didn't maintain the trunk/d1.0/dparser/dmd/main.d
and surely the d2.0 branch also not maintained at the moment.

use d1.0/test_dparser

It shouldn't segfault. std.stdio has been parsed a lot of times here :)
It's a good start from there I think

-- 
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/
Jul 13 2008
parent reply Markus Koskimies <markus reaaliaika.net> writes:
On Mon, 14 Jul 2008 12:31:39 +0800, davidl wrote:


 err, i didn't maintain the trunk/d1.0/dparser/dmd/main.d and surely the
 d2.0 branch also not maintained at the moment.

 use d1.0/test_dparser

 It shouldn't segfault. std.stdio has been parsed a lot of times here :)
 It's a good start from there I think


Yes, that worked :)

Sorry to use unmaintained things, but there is such a little information 
written in that package :) I just took first file I saw containing main, 
after I had written a somewhat functional Makefile :)

But that does not solve my problem to determine, which kind of statement 
type there is in the tree; I can currently extract only those, which have 
isXXXX functions in the Statement class.
Jul 13 2008
parent reply Markus Koskimies <markus reaaliaika.net> writes:
On Mon, 14 Jul 2008 05:45:59 +0000, Markus Koskimies wrote:


 But that does not solve my problem to determine, which kind of statement
 type there is in the tree; I can currently extract only those, which
 have isXXXX functions in the Statement class.


Here is an example:

test.d:
---
import std.stdio;

void main()
{
    int x;
    
    writefln("%d", x);
    
    return;
}
---

When I walk through the module (test), I get following result:

---
test: module
    object: import
    std: import
    Func: void main()
    Function body:
        Compound statement:
            Compound statement:
                Decl statement: int x = x (in a declar Exp)=  0;

==>             Unknown statement: (writefln)("%d",(x));

                Compound statement:
==>                 Unknown statement: 0;

                    Return statement: return ;
---

I'm wondering how to extract the type of those statements... At least 
they are not any of the Statement.isXXXX -types.
Jul 13 2008
parent reply davidl <davidl 126.com> writes:
在 Mon, 14 Jul 2008 14:01:48 +0800，Markus Koskimies  
<markus reaaliaika.net> 写道:


 On Mon, 14 Jul 2008 05:45:59 +0000, Markus Koskimies wrote:


 But that does not solve my problem to determine, which kind of statement
 type there is in the tree; I can currently extract only those, which
 have isXXXX functions in the Statement class.


 Here is an example:

 test.d:
 ---
 import std.stdio;

 void main()
 {
     int x;
    writefln("%d", x);
    return;
 }
 ---

 When I walk through the module (test), I get following result:

 ---
 test: module
     object: import
     std: import
     Func: void main()
     Function body:
         Compound statement:
             Compound statement:
                 Decl statement: int x = x (in a declar Exp)=  0;

 ==>             Unknown statement: (writefln)("%d",(x));

                 Compound statement:
 ==>                 Unknown statement: 0;

                     Return statement: return ;
 ---

 I'm wondering how to extract the type of those statements... At least
 they are not any of the Statement.isXXXX -types.


Oh, it's pretty awesome you get that far in such a short time.
I bet you can print instance.classinfo.name to get the type name of that  
instance.
And I want to have your code be integrated into the dparser trunk for  
illustrating
how to use dparser.


-- 
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/
Jul 14 2008
parent Markus Koskimies <markus reaaliaika.net> writes:
On Mon, 14 Jul 2008 18:59:16 +0800, davidl wrote:


 I'm wondering how to extract the type of those statements... At least
 they are not any of the Statement.isXXXX -types.


 
 Oh, it's pretty awesome you get that far in such a short time. I bet you
 can print instance.classinfo.name to get the type name of that instance.


Hmmh, I'll try that... I'm still playing around to get used to these new 
D things (dsss, tango, tango-phobos -coexistence etc)...


 And I want to have your code be integrated into the dparser trunk for
 illustrating how to use dparser.


I needed to make some modifications to dparser sources to get them 
compiled to Linux (if I remeber correctly). I'll try to do that walking 
thing, and then I get fresh, unmodified dparser and see, if I get it 
compiled without any hacking.
Jul 14 2008
prev sibling next sibling parent Markus Koskimies <markus reaaliaika.net> writes:
On Mon, 14 Jul 2008 04:07:00 +0000, Markus Koskimies wrote:


 At the same time I'll try to get dil compiled; although dil does not
 generate code, it might still be a suitable platform to write a software
 to generate some common warnings (just for my personal use, I'm not
 intended to start to write "dlint"). Unfortunately it means that I need
 to figure out how those DSSS & Tango things work :)


I finally made it, but that was extremely hard :D Now dil compiles \o/

I'm not sure where to tell these findings, so I write them here; 
hopefully someone finds them with google, if he/she enters same 
problems...

DSSS, GDC, Tango and Linux-AMD64:

---

First thing to know is that currently GDC (nor DMD) does not work 
completely with 64-bit Linuxes. Everything else seems to work, but not 
vararg's (that is, writef & writefln).

So, install a chroot'ed 32-bit environment, instructions:

http://saftsack.fs.uni-bayreuth.de/~dun3/archives/using-a-x86-chroot-to-
run-32-bit-apps-on-debian-amd64/94.html

---

You may install DMD, it does not cause any harm :)

---

Second thing; if you followed those instructions, Debian sid comes with 
newer GCC than is supported by GDC. So don't install it (it may work, but 
better to use correct version)! Install first GDC (currently GDC-4.1), 
and install the same version of GCC and G++ (4.1).

Try to compile a simple hello.d with GDC to check, that it really works. 
With correct versions of all necessary things (gcc, g++, gdc and their 
libraries & utils), gdc should work OK.

---

Next thing to install is DSSS. It should work also, quite easily. Make it 
use GDC (to be able to build tango with GDC). I was not able to install 
curl to chrooted sid, so I'm not able to use dsss net installs.

---

When you have GDC and DSSS working, you can try to get tango working. I 
fought with tango for several hours. I don't really know what phase 
didn't work, but here are something:

* I tried both dsss build/install, and build/install scripts in tango 
package (currently 0.99.6). Both seemed to work (no errors), but GDC 
didn't build my simple tango-hello.d; there were continuously error 
messages like:

hello.o: In function `_Dmain':
hello.d:(.text+0x20): undefined reference to 
`_D5tango2io7Console4CoutC5tango2io7Console7Console6Output'
hello.d:(.text+0x2f): undefined reference to 
`_D5tango2io7Console7Console6Output6appendMFAaZC5tango2io7Console7Console6Output'
hello.d:(.text+0x3b): undefined reference to 
`_D5tango2io7Console7Console6Output7newlineMFZC5tango2io7Console7Console6Output'
hello.o:(.data+0x38): undefined reference to 
`_D5tango2io7Console12__ModuleInfoZ'

There were probably two reasons for this:

1) libgphobos.a in /usr/lib/gcc/i486-linux-gnu/4.1.3/ was probably 
incorrectly build, and I didn't notice to update it.

2) It seems, that libDG-tango-core.a does not include all the necessary 
files. I ran "ar r libDG-tango-core.a *.o" in tango/ directory (where the 
libDG* files are), since there was object files like 
tango.core.Exception.o), which were not in that core library.

---

* In tango directory, use script:

./build-tango.sh gdc

...To build libgtango.a, which you copy to correct location (good guess: /
usr/lib/gcc/i486-linux-gnu/4.1.3/)

* It could be easier first to try to get plain gdc builds to work:

$ gdmd -version=Posix tango-hello.d -ofhello -W-lgtango

---

Now when I got those things working, I'm not going to uninstall them and 
try again, so I can't tell step-by-step instructions for getting them 
working ;D
Jul 14 2008
prev sibling parent reply Markus Koskimies <markus reaaliaika.net> writes:
On Mon, 14 Jul 2008 04:07:00 +0000, Markus Koskimies wrote:


 actually dparser is not 1.014 anymore. it's somewhat 1.024ish the
 parsing ability has been improved a lot, it can parse itself, and some
 ctfes.
 700 or so cases from dstress fail.


 
 :o
 
 I fetched the sources (I tried to select the latest ones), and attached
 that DParser to my own code.


Jihuu! Now it works!

I wrote a somewhat general purpose AST walker. It can be inherited and 
modified for different purposes. As an example, I made a class for 
scanning unused local variables:

class UnusedScan : ASTWalk
{
    /* To track down variable usage from nested functions, the
     * variable reference table is global.
     */
    bool[VarDeclaration] referenced;
    
    /* Frame holds list of local variable definitions */
    class Frame { VarDeclaration[] vars; }

    Stack!(Frame) frames;

    //-------------------------------------------------------------------------
    // Whenever we meet a new variable frame, push it. When popping it 
out,
    // we check, if the variable was referenced.
    //-------------------------------------------------------------------------
    
    void push()
    {
        frames.push( new Frame() );
    }
    void pop()
    {
        Frame frame = frames.pop();
        
        foreach(VarDeclaration var; frame.vars)
        {
            if(referenced[var]) continue;
            writefln("%s: Unused variable: %s",
                var.loc.toString(),
                var.toString()
            );
        }
    }
    
    //-------------------------------------------------------------------------
    // Constructor
    //-------------------------------------------------------------------------
    
    this()
    {
        frames = new Stack!(Frame)();
    }

    //-------------------------------------------------------------------------
    // Whenever we meet a function, class or similar, make a new frame.
    // After scanning, check if there were unreferenced variables.
    //-------------------------------------------------------------------------

    override void walk(Module m)
    {
        push(); super.walk(m); pop();
    }
    
    override void walk(ClassDeclaration d)
    {
        push(); super.walk(d); pop();
    }
    
    override void walk(FuncDeclaration f)
    {
        push(); super.walk(f); pop();
    }
    
    //-------------------------------------------------------------------------
    // For every var declaration, put it to frame.
    //-------------------------------------------------------------------------
    override void walk(VarDeclaration var)
    {
        referenced[var] = false;
        frames.top().vars ~= var;
    }

    //-------------------------------------------------------------------------
    // When var is used, mark it to frame.
    //-------------------------------------------------------------------------
    override void walk(VarExp var)
    {
        referenced[cast(VarDeclaration)var.var] = true;
    }
}

---

Then I tested it against a small D program:

import std.stdio;

void main()
{
    int a, b;
    int unused;

    // a is used inside expression
    writefln("%d", 23+a);

    // b is used from local function
    int f1() { int f2() { return b; } return f2(); }

    // unused is shadowed; still, the main unused should remain
    // not referred.
    void f3() { int unused; unused = 1; }
    
    return;
}

---

And it reports:

PASS 1: Building parse tree...
semantic1 done
semantic2 done
semantic3 done 0
PASS 2: Walking parse tree...
UnusedLocal1.d(8): Unused variable: unused

...Just as it should do :D

I'll put the full sources available somewhere (to my home pages, at 
least), and continue developing walking & warning thing.

Davidl, do you want the sources for your repository?
Jul 15 2008
next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Markus Koskimies, el 15 de julio a las 08:44 me escribiste:
[snip]

 And it reports:
 
 PASS 1: Building parse tree...
 semantic1 done
 semantic2 done
 semantic3 done 0
 PASS 2: Walking parse tree...
 UnusedLocal1.d(8): Unused variable: unused
 
 ...Just as it should do :D


Very nice!


 I'll put the full sources available somewhere (to my home pages, at 
 least), and continue developing walking & warning thing.


That would be very useful.

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
careful to all animals (never washing spiders down the plughole),
keep in contact with old friends (enjoy a drink now and then),
will frequently check credit at (moral) bank (hole in the wall),
Jul 15 2008
parent reply Markus Koskimies <markus reaaliaika.net> writes:
On Tue, 15 Jul 2008 10:29:01 -0300, Leandro Lucarella wrote:


 I'll put the full sources available somewhere (to my home pages, at
 least), and continue developing walking & warning thing.


 
 That would be very useful.


I'm currently hacking more scanners:

- Scanning/warning, if function return value is discarded, i.e.

	int f() { ... }

	f(); // Warn

	cast(void)f();	// Don't warn

- Warning for suspicious looking infinite loops, i.e.

	for(;3.1415926535894;) { ... }	// Warn

	for(;;) { }	// Don't warn


There are some problems to make them correct, since DParser does not 
always store all necessary information, e.g. "for(;true;)" = "for(;;)"

I probably include scanning of constant if/switch values with the loop 
scanning... That general purpose walking class makes implementation of 
these kinds of checkings relatively easy.

There will surely missing / wrongly working pieces in the code, but those 
can be corrected later :)
Jul 15 2008
parent davidl <davidl 126.com> writes:
在 Tue, 15 Jul 2008 22:37:56 +0800，Markus Koskimies  
<markus reaaliaika.net> 写道:


 On Tue, 15 Jul 2008 10:29:01 -0300, Leandro Lucarella wrote:


 I'll put the full sources available somewhere (to my home pages, at
 least), and continue developing walking & warning thing.


 That would be very useful.


 I'm currently hacking more scanners:

 - Scanning/warning, if function return value is discarded, i.e.

 	int f() { ... }

 	f(); // Warn

 	cast(void)f();	// Don't warn

 - Warning for suspicious looking infinite loops, i.e.

 	for(;3.1415926535894;) { ... }	// Warn

 	for(;;) { }	// Don't warn


 There are some problems to make them correct, since DParser does not
 always store all necessary information, e.g. "for(;true;)" = "for(;;)"

 I probably include scanning of constant if/switch values with the loop
 scanning... That general purpose walking class makes implementation of
 these kinds of checkings relatively easy.

 There will surely missing / wrongly working pieces in the code, but those
 can be corrected later :)


The unused variable bug has been appeared in the bugzilla, and yes, i did  
think about it a bit.
The way I think is modify the VarDeclaration to have a member flag to show  
this var is used or not.
But this require a more complicated way of coding, your walker might be  
the better way to code. If
we can balance the performance issue, I think we open something really  
interesting in balancing of
design pattern and performance.

-- 
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/
Jul 15 2008
prev sibling next sibling parent reply davidl <davidl 126.com> writes:
在 Tue, 15 Jul 2008 16:44:25 +0800，Markus Koskimies  
<markus reaaliaika.net> 写道:


 On Mon, 14 Jul 2008 04:07:00 +0000, Markus Koskimies wrote:


 actually dparser is not 1.014 anymore. it's somewhat 1.024ish the
 parsing ability has been improved a lot, it can parse itself, and some
 ctfes.
 700 or so cases from dstress fail.


 :o

 I fetched the sources (I tried to select the latest ones), and attached
 that DParser to my own code.


 Jihuu! Now it works!

 I wrote a somewhat general purpose AST walker. It can be inherited and
 modified for different purposes. As an example, I made a class for
 scanning unused local variables:

 class UnusedScan : ASTWalk
 {
     /* To track down variable usage from nested functions, the
      * variable reference table is global.
      */
     bool[VarDeclaration] referenced;
    /* Frame holds list of local variable definitions */
     class Frame { VarDeclaration[] vars; }

     Stack!(Frame) frames;

     //-------------------------------------------------------------------------
     // Whenever we meet a new variable frame, push it. When popping it
 out,
     // we check, if the variable was referenced.
     //-------------------------------------------------------------------------
    void push()
     {
         frames.push( new Frame() );
     }
     void pop()
     {
         Frame frame = frames.pop();
        foreach(VarDeclaration var; frame.vars)
         {
             if(referenced[var]) continue;
             writefln("%s: Unused variable: %s",
                 var.loc.toString(),
                 var.toString()
             );
         }
     }
    //-------------------------------------------------------------------------
     // Constructor
     //-------------------------------------------------------------------------
    this()
     {
         frames = new Stack!(Frame)();
     }

     //-------------------------------------------------------------------------
     // Whenever we meet a function, class or similar, make a new frame.
     // After scanning, check if there were unreferenced variables.
     //-------------------------------------------------------------------------

     override void walk(Module m)
     {
         push(); super.walk(m); pop();
     }
    override void walk(ClassDeclaration d)
     {
         push(); super.walk(d); pop();
     }
    override void walk(FuncDeclaration f)
     {
         push(); super.walk(f); pop();
     }
    //-------------------------------------------------------------------------
     // For every var declaration, put it to frame.
     //-------------------------------------------------------------------------
     override void walk(VarDeclaration var)
     {
         referenced[var] = false;
         frames.top().vars ~= var;
     }

     //-------------------------------------------------------------------------
     // When var is used, mark it to frame.
     //-------------------------------------------------------------------------
     override void walk(VarExp var)
     {
         referenced[cast(VarDeclaration)var.var] = true;
     }
 }

 ---

 Then I tested it against a small D program:

 import std.stdio;

 void main()
 {
     int a, b;
     int unused;

     // a is used inside expression
     writefln("%d", 23+a);

     // b is used from local function
     int f1() { int f2() { return b; } return f2(); }

     // unused is shadowed; still, the main unused should remain
     // not referred.
     void f3() { int unused; unused = 1; }
    return;
 }

 ---

 And it reports:

 PASS 1: Building parse tree...
 semantic1 done
 semantic2 done
 semantic3 done 0
 PASS 2: Walking parse tree...
 UnusedLocal1.d(8): Unused variable: unused

 ...Just as it should do :D

 I'll put the full sources available somewhere (to my home pages, at
 least), and continue developing walking & warning thing.

 Davidl, do you want the sources for your repository?


Of course!
Send me the patch? If you get a dsource account, maybe you're the right  
guy to cooperate on this interesting project, I may grant you write access  
;)

-- 
使用 Opera 革命性的电子邮件客户程序: http://www.opera.com/mail/
Jul 15 2008
parent Markus Koskimies <markus reaaliaika.net> writes:
On Tue, 15 Jul 2008 23:00:07 +0800, davidl wrote:


 Of course!
 Send me the patch? If you get a dsource account, maybe you're the right
 guy to cooperate on this interesting project, I may grant you write
 access ;)


I'll put you some email, just wait-a-moment :)
Jul 15 2008
prev sibling parent reply Markus Koskimies <markus reaaliaika.net> writes:
On Tue, 15 Jul 2008 08:44:25 +0000, Markus Koskimies wrote:


 I'll put the full sources available somewhere (to my home pages, at
 least), and continue developing walking & warning thing.


Sources are available:

http://reaaliaika.net/showarea.php/download/dwalk

dwalk.tar.ge opens to directory named dwalk. dwalk/dparser should contain 
link to DParser sources (you may also upload them to dwalk/ directory 
with svn etc).

Package contains:

- Sources: dwalk/src/ 
- Test files: dwalk/testfiles

About sources: You probably first take a look to main.d. Then you may 
want to look at the scanners (ScanXXXModule.d). Finally, you might want 
to take a look at the ASTWalk (ASTWalking.d).

If you get the thing compiled, you should have executable named dwalk, 
which can:

- Detect unused things; there may still be some false reporting, as well 
as it can surely undetect some unused things. imports are not checked at 
all. Nor templates (or they may be, if they go with the defaults).

- Detect expression result discarding, i.e. unused return values. There 
are some pecularities with "void main(...)" functions, which cause false 
alarms.

- Detect constant conditions, in for/while/do and if/switch; take a look 
at the testfiles/TestInfLoop1.d for seeing, which kind of infinite or 
dead loops are scanned.

Any reports, comments & suggestions are welcome :D
Jul 15 2008
parent Markus Koskimies <markus reaaliaika.net> writes:
On Tue, 15 Jul 2008 20:40:21 +0000, Markus Koskimies wrote:


 On Tue, 15 Jul 2008 08:44:25 +0000, Markus Koskimies wrote:
 

 I'll put the full sources available somewhere (to my home pages, at
 least), and continue developing walking & warning thing.


 
 Sources are available:
 
 http://reaaliaika.net/showarea.php/download/dwalk
 


Oh, some additions... Packet includes binary for i386-linux. Packet 
compiles with DMD1.0 and phobos. If compiling to linux, you may need to 
hack (current) DParser sources a little; if I remember correctly, change 
memicmp -> memcmp somewhere (changes things case sensitive, bu so what), 
and somewhere else there was function like DebugOutputA, change it to 
writefln.
Jul 15 2008