www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Parse d source file by using compiler

reply tcak <1ltkrs+3wyh1ow7kzn1k sharklasers.com> writes:
I checked for a flag in this page http://dlang.org/dmd-linux.html 
, but couldn't have found any for this purpose.

Is there a way to parse a d source file so it generates a tree in 
JSON, XML, or something-that-can-be-processed-easily file format?

---

My real purpose:

I need to generate hash code (e.g. MD5) for a part of source code 
(let's say a class, struct, or a function). So whether the codes 
are changed or not can be detected. As you will guess, comments, 
text formatting etc. shouldn't affect the hash result.


Use-Case:

I am writing a code generator/back up system. It will check the 
last available code file. If important changes are done in a 
specific part of code, it will increase version number by 1.
Nov 08 2015
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 09/11/15 6:49 PM, tcak wrote:

 I checked for a flag in this page http://dlang.org/dmd-linux.html , but
 couldn't have found any for this purpose.

 Is there a way to parse a d source file so it generates a tree in JSON,
 XML, or something-that-can-be-processed-easily file format?

 ---

 My real purpose:

 I need to generate hash code (e.g. MD5) for a part of source code (let's
 say a class, struct, or a function). So whether the codes are changed or
 not can be detected. As you will guess, comments, text formatting etc.
 shouldn't affect the hash result.


 Use-Case:

 I am writing a code generator/back up system. It will check the last
 available code file. If important changes are done in a specific part of
 code, it will increase version number by 1.


There is a json output, but it hasn't been all that maintained.
Use something like DScanner, instead.
Nov 08 2015
prev sibling next sibling parent BBasile <b2.temp gmx.com> writes:
On Monday, 9 November 2015 at 05:49:25 UTC, tcak wrote:

 I checked for a flag in this page 
 http://dlang.org/dmd-linux.html , but couldn't have found any 
 for this purpose.

 Is there a way to parse a d source file so it generates a tree 
 in JSON, XML, or something-that-can-be-processed-easily file 
 format?

 ---

 My real purpose:

 I need to generate hash code (e.g. MD5) for a part of source 
 code (let's say a class, struct, or a function). So whether the 
 codes are changed or not can be detected. As you will guess, 
 comments, text formatting etc. shouldn't affect the hash result.


 Use-Case:

 I am writing a code generator/back up system. It will check the 
 last available code file. If important changes are done in a 
 specific part of code, it will increase version number by 1.


You could write your own tool using libdparse[1]: parse, visit 
the AST, create a signature for the declarations that are 
interesting.

---
https://github.com/Hackerpilot/libdparse
Nov 08 2015
prev sibling next sibling parent ZombineDev <valid_email he.re> writes:
On Monday, 9 November 2015 at 05:49:25 UTC, tcak wrote:

 I checked for a flag in this page 
 http://dlang.org/dmd-linux.html , but couldn't have found any 
 for this purpose.

 Is there a way to parse a d source file so it generates a tree 
 in JSON, XML, or something-that-can-be-processed-easily file 
 format?

 ---

 My real purpose:

 I need to generate hash code (e.g. MD5) for a part of source 
 code (let's say a class, struct, or a function). So whether the 
 codes are changed or not can be detected. As you will guess, 
 comments, text formatting etc. shouldn't affect the hash result.


 Use-Case:

 I am writing a code generator/back up system. It will check the 
 last available code file. If important changes are done in a 
 specific part of code, it will increase version number by 1.


You're use case is really interesting! AFAIK, currently, the 
lexer part of the DMD frontend is the only part that can be 
easily used standalone. Daniel Murphy (who drove a lot of the 
work towards DDMD) published the lexer on DUB: 
http://code.dlang.org/packages/ddmd. However the package is 
outdated, because it is based on DMD v2.067.
As the whole frontend is now in D, you should be able to import 
any of the D modules and work with them, but it may not be as 
easy, as their API is geared only towards the DMD driver.
I personally want to help refactor DDMD to be usable as a 
library, so you can use for all sorts of cool things (like 
runtime JIT, IDE support, REPL, and so on), but I'm quite busy at 
the moment :(
Nov 09 2015
prev sibling parent Brian Schott <briancschott gmail.com> writes:
On Monday, 9 November 2015 at 05:49:25 UTC, tcak wrote:

 I checked for a flag in this page 
 http://dlang.org/dmd-linux.html , but couldn't have found any 
 for this purpose.

 Is there a way to parse a d source file so it generates a tree 
 in JSON, XML, or something-that-can-be-processed-easily file 
 format?

 ---

 My real purpose:

 I need to generate hash code (e.g. MD5) for a part of source 
 code (let's say a class, struct, or a function). So whether the 
 codes are changed or not can be detected. As you will guess, 
 comments, text formatting etc. shouldn't affect the hash result.


 Use-Case:

 I am writing a code generator/back up system. It will check the 
 last available code file. If important changes are done in a 
 specific part of code, it will increase version number by 1.


dscanner --ast path/to/file.d \
     | xmllint --xpath 
"//classDeclaration[name='ClassYouCareAbout']" - \
     | md5sum

https://github.com/Hackerpilot/Dscanner

The only problem here is that D-Scanner's XML output includes 
<ddoc> tags. You should be able to strip those out with sed or 
something.
Nov 09 2015