www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - how to use dmd.lexer.Lexer

reply jonatjano <jonatjano gmail.com> writes:
Hello,
I'm trying to write a script which parse a c header and then 
ouput basic bindings for multiple languages.

I read from the forum some times ago that the D compiler could be 
used as a library and found [the lexer page in the 
documentation](https://dlang.org/library/dmd/lexer/lexer.html)

but `import dmd.lexer: Lexer` fails with the following error :
```
modlibBinder.d(3): Error: unable to read module `lexer`
modlibBinder.d(3):        Expected 'dmd/lexer.d' or 
'dmd/lexer/package.d' in one of the following import paths:
import path[0] = .
import path[1] = /usr/include/dmd/phobos
import path[2] = /usr/include/dmd/druntime/import
Failed: ["/usr/bin/dmd", "-v", "-o-", "modlibBinder.d", "-I."]
```

What are the steps to take to be able to call the lexer (and 
maybe also the parser) over a C header file ?
Jan 06 2023
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
Dmd is a separate code base and is not available in a regular D build.

You have to provide it and then link it in.

The easiest way to do it is by using dmd-fe via dub.

Examples: https://github.com/dlang/dmd/tree/master/compiler/test/dub_package

So something like this:

```json
{
	"name": "lexer_test",
	"dependencies": {
		"dmd:frontend": "~>2.101.0"
	}
}
```

source/lexer_test.d:

```d
module lexer_test.d;

void main()
{
     import dmd.globals;
     import dmd.lexer;
     import dmd.tokens;

     immutable expected = [
         TOK.void_,
         TOK.identifier,
         TOK.leftParenthesis,
         TOK.rightParenthesis,
         TOK.leftCurly,
         TOK.rightCurly
     ];

     immutable sourceCode = "void test() {} // foobar";
     scope lexer = new Lexer("test", sourceCode.ptr, 0, 
sourceCode.length, 0, 0);
     lexer.nextToken;

     TOK[] result;

     do
     {
         result ~= lexer.token.value;
     } while (lexer.nextToken != TOK.endOfFile);

     assert(result == expected);
}
```

And then just ``$ dub run``.
Jan 06 2023