www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - A Simple D Lexer (written in D)

reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
http://aljudy.org/dstuff/simple.d.lexer.zip

This is something I'm working on -lazily-, as a part of a bigger project.
It's not heavliy tested or anything .. read the doc.txt that comes with 
it ..
The code uses several Objects, but it's not entirely OO (unless you 
think of modules as singleton classes!)

I tried to make the code modular and cohesive. Hopefully, most bugs 
would be easily fixable.

I'll quote myself here (from doc.txt) to give an example of what this 
lexer can do:
The following code segment
<code>	
void main()
{
    //comment
    /+
       /* */
     /+
       +/
    +/
    printf("hi");
}
</code>
produces the folloing token list:
type                             text
----------------------------------------
TOK.TOKvoid                     void
TOK.TOKidentifier               main
TOK.TOKopenparn                 (
TOK.TOKcloseparen               )
TOK.TOKopencurly                {
TOK.TOKidentifier               printf
TOK.TOKopenparen                (
TOK.TOKstring                   "hi"
TOK.TOKcloseparen               )
TOK.TOKsemicolon                ;
TOK.TOKclosecurly               }


The code ships with this example, to compile it, cd to the directory 
where you unzipped it, and invoke build against it:

build main.d -clean


and run it

main



For details, see doc.txt in the zip file. As you can see, the lexer 
handles nesting comments pretty well.

I am also working on a parser, but do not expect anything.
I'm a kind of guy who works on alot of projects but doesn't finish most 
of them, which is why I'm releasing this lexer now. Basically I don't 
know whether or not I will continue working on this baby, and I don't 
want it to be wasted for nothing.

I hope people will find it useful.
Enjoy.
Sep 03 2005
next sibling parent reply Derek Parnell <derek psych.ward> writes:
On Sun, 04 Sep 2005 00:36:52 -0600, Hasan Aljudy wrote:


 http://aljudy.org/dstuff/simple.d.lexer.zip
 


[snip]

 Basically I don't 
 know whether or not I will continue working on this baby, and I don't 
 want it to be wasted for nothing.


It compiled and worked well. 


 I hope people will find it useful.


I have a few uses for such a beastie. Thanks.

-- 
Derek Parnell
Melbourne, Australia
4/09/2005 6:43:44 PM
Sep 04 2005
parent Hasan Aljudy <hasan.aljudy gmail.com> writes:
Derek Parnell wrote:

 On Sun, 04 Sep 2005 00:36:52 -0600, Hasan Aljudy wrote:
 
 

http://aljudy.org/dstuff/simple.d.lexer.zip


 
 [snip]
 

Basically I don't 
know whether or not I will continue working on this baby, and I don't 
want it to be wasted for nothing.


 
 
 It compiled and worked well. 
 
 

I hope people will find it useful.


 
 
 I have a few uses for such a beastie. Thanks.
 


Thank you. It's good to hear that.
Sep 04 2005
prev sibling next sibling parent Hasan Aljudy <hasan.aljudy gmail.com> writes:
I just noticed a slight problem in lex/string_scanner.d, so I fixed it 
and uploaded a new `version`.
also, the
"string"\escape"string2"
issue is fixed too.

Hasan Aljudy wrote:

 http://aljudy.org/dstuff/simple.d.lexer.zip
 
 This is something I'm working on -lazily-, as a part of a bigger project.
 It's not heavliy tested or anything .. read the doc.txt that comes with 
 it ..
 The code uses several Objects, but it's not entirely OO (unless you 
 think of modules as singleton classes!)
 
 I tried to make the code modular and cohesive. Hopefully, most bugs 
 would be easily fixable.
 
 I'll quote myself here (from doc.txt) to give an example of what this 
 lexer can do:
 The following code segment
 <code>   
 void main()
 {
    //comment
    /+
       /* */
     /+
       +/
    +/
    printf("hi");
 }
 </code>
 produces the folloing token list:
 type                             text
 ----------------------------------------
 TOK.TOKvoid                     void
 TOK.TOKidentifier               main
 TOK.TOKopenparn                 (
 TOK.TOKcloseparen               )
 TOK.TOKopencurly                {
 TOK.TOKidentifier               printf
 TOK.TOKopenparen                (
 TOK.TOKstring                   "hi"
 TOK.TOKcloseparen               )
 TOK.TOKsemicolon                ;
 TOK.TOKclosecurly               }
 
 
 The code ships with this example, to compile it, cd to the directory 
 where you unzipped it, and invoke build against it:
  >build main.d -clean
 
 and run it
  >main
 
 
 For details, see doc.txt in the zip file. As you can see, the lexer 
 handles nesting comments pretty well.
 
 I am also working on a parser, but do not expect anything.
 I'm a kind of guy who works on alot of projects but doesn't finish most 
 of them, which is why I'm releasing this lexer now. Basically I don't 
 know whether or not I will continue working on this baby, and I don't 
 want it to be wasted for nothing.
 
 I hope people will find it useful.
 Enjoy.
Sep 04 2005
prev sibling parent Dannerbeck Dieter <no.spam.dd gmx.de> writes:
Hasan Aljudy schrieb:

 http://aljudy.org/dstuff/simple.d.lexer.zip


Link doesn't work.
(
Not Found
The requested URL http:// was not found on this server.
)
Jan 20 2006