digitalmars.D - ES6 Parser in Dlang
- James Lu (6/6) Feb 27 2020 I want to implement an ES6 (JavaScript) parser in Dlang so I can
- Dejan Lekic (2/8) Feb 27 2020 Have a look at similar work: https://github.com/higgsjs/Higgs
- JN (3/9) Feb 27 2020 Check this out: http://code.dlang.org/packages/es6-grammar looks
- =?UTF-8?Q?S=c3=b6nke_Ludwig?= (10/17) Feb 27 2020 There is also DMDScript, which is more or less ECMAScript v3. It should
- Meta (3/9) Feb 27 2020 There's also Pegged:
- Petar Kirov [ZombineDev] (3/13) Feb 27 2020 http://code.dlang.org/packages/es6-grammar
- Stefan Koch (3/9) Feb 27 2020 I would write my own. Maybe with some tooling to take care of the
- Sebastiaan Koppe (14/20) Feb 27 2020 I dunno. I wrote the es6-grammar library that uses pegged. Then I
- aberba (3/18) Feb 27 2020 Nice!!
- drug (2/29) Feb 27 2020
- Sebastiaan Koppe (9/22) Feb 27 2020 No, it was mainly speed and memory usage.
- drug (4/15) Feb 27 2020 Yes, I've seen your PR so I asked this question) Nice work!
- Sebastiaan Koppe (5/8) Feb 28 2020 It is already used in industry. Rather successfully I would say.
- James Lu (4/17) Mar 02 2020 Thanks. I will use this to create wrappers that ensure JS
- Walter Bright (2/3) Feb 27 2020 https://github.com/DigitalMars/DMDScript
I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?
Feb 27 2020
Dejan Lekic <dejan.lekic gmail.com> On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?Have a look at similar work: https://github.com/higgsjs/Higgs
Feb 27 2020
On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?Check this out: http://code.dlang.org/packages/es6-grammar looks like a good starting point.
Feb 27 2020
Am 27.02.2020 um 15:06 schrieb James Lu:I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?There is also DMDScript, which is more or less ECMAScript v3. It should pretty straight forward to get that updated to v5, as there haven't many parser related changes (property getters/setters is an exception). I was already thinking of whether it makes sense to invest that work, because that would enable running the TypeScript compiler through it. v6 has a few more parser related changes, but still nothing really bad. The DMDScript parser is in https://github.com/DigitalMars/DMDScript/blob/master/engine/source/dmdscript/parse.d Higgs is much closer to ES6, though.
Feb 27 2020
On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?There's also Pegged: https://code.dlang.org/packages/pegged
Feb 27 2020
On Thursday, 27 February 2020 at 16:49:19 UTC, Meta wrote:On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:http://code.dlang.org/packages/es6-grammar is based on Pegged, btw.I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?There's also Pegged: https://code.dlang.org/packages/pegged
Feb 27 2020
On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?I would write my own. Maybe with some tooling to take care of the boring stuff.
Feb 27 2020
On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?I dunno. I wrote the es6-grammar library that uses pegged. Then I decided to hand write mine own. https://github.com/skoppe/jazr It is a pretty complete and fast JS minifier. It has lots of passes and minifies code pretty well: https://github.com/skoppe/jazr/blob/master/source/es6/minifier.d#L286 In general it is about 30 times faster than uglify-js (and uses 3-4 times less memory). It parses JS code at about 60 Mb/s. The minifier was a bit slower at 20 Mb/s if I remember correctly. I fuzzed the lexer and parser, so there are little to no spurious crashes on weird input. Use it as you please, or just as an indication of the effort involved.
Feb 27 2020
On Thursday, 27 February 2020 at 22:22:59 UTC, Sebastiaan Koppe wrote:On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:Nice!![...]I dunno. I wrote the es6-grammar library that uses pegged. Then I decided to hand write mine own. https://github.com/skoppe/jazr It is a pretty complete and fast JS minifier. It has lots of passes and minifies code pretty well: https://github.com/skoppe/jazr/blob/master/source/es6/minifier.d#L286 In general it is about 30 times faster than uglify-js (and uses 3-4 times less memory). It parses JS code at about 60 Mb/s. The minifier was a bit slower at 20 Mb/s if I remember correctly. I fuzzed the lexer and parser, so there are little to no spurious crashes on weird input. Use it as you please, or just as an indication of the effort involved.
Feb 27 2020
28.02.2020 01:22, Sebastiaan Koppe пишет:On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:What was the reason? Pegged parsing errors?I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?I dunno. I wrote the es6-grammar library that uses pegged. Then I decided to hand write mine own.https://github.com/skoppe/jazr It is a pretty complete and fast JS minifier. It has lots of passes and minifies code pretty well: https://github.com/skoppe/jazr/blob/master/source/es6/minifier.d#L286 In general it is about 30 times faster than uglify-js (and uses 3-4 times less memory). It parses JS code at about 60 Mb/s. The minifier was a bit slower at 20 Mb/s if I remember correctly. I fuzzed the lexer and parser, so there are little to no spurious crashes on weird input. Use it as you please, or just as an indication of the effort involved.
Feb 27 2020
On Friday, 28 February 2020 at 05:24:16 UTC, drug wrote:28.02.2020 01:22, Sebastiaan Koppe пишет:No, it was mainly speed and memory usage. Pegged is pretty convenient for small grammar or small input files, but es6-grammar contains about 500 grammar rules and I was running it on Mb's of javascript. Incidentally I improved pegged's parsing errors in https://github.com/PhilippeSigaud/Pegged/pull/269 you now get a much more complete parsetree on errors. Bastiaan, however, found some memory usage problems but I hope to tackle them soonish.On Thursday, 27 February 2020 at 14:06:19 UTC, James Lu wrote:What was the reason? Pegged parsing errors?I want to implement an ES6 (JavaScript) parser in Dlang so I can implement a transpiler. Because it's a transpiler, I need to be able to visit and rewrite AST nodes, then generate code from the AST. Ideally, the parser generator supports streaming the input into the AST visitor out to the output. What parser generator library should I use for this purpose?I dunno. I wrote the es6-grammar library that uses pegged. Then I decided to hand write mine own.
Feb 27 2020
On 2/28/20 10:30 AM, Sebastiaan Koppe wrote:No, it was mainly speed and memory usage. Pegged is pretty convenient for small grammar or small input files, but es6-grammar contains about 500 grammar rules and I was running it on Mb's of javascript. Incidentally I improved pegged's parsing errors in https://github.com/PhilippeSigaud/Pegged/pull/269 you now get a much more complete parsetree on errors. Bastiaan, however, found some memory usage problems but I hope to tackle them soonish.Yes, I've seen your PR so I asked this question) Nice work! I'm curious can Pegged be improved to be usable in industry and become standard parser generator in D land de-facto.
Feb 27 2020
On Friday, 28 February 2020 at 07:53:22 UTC, drug wrote:Yes, I've seen your PR so I asked this question) Nice work!thxI'm curious can Pegged be improved to be usable in industry and become standard parser generator in D land de-facto.It is already used in industry. Rather successfully I would say. Making and keeping something a de-facto standard requires a lot of work spread over years, I don't see that happening.
Feb 28 2020
On Thursday, 27 February 2020 at 22:22:59 UTC, Sebastiaan Koppe wrote:I dunno. I wrote the es6-grammar library that uses pegged. Then I decided to hand write mine own. https://github.com/skoppe/jazr It is a pretty complete and fast JS minifier. It has lots of passes and minifies code pretty well: https://github.com/skoppe/jazr/blob/master/source/es6/minifier.d#L286 In general it is about 30 times faster than uglify-js (and uses 3-4 times less memory). It parses JS code at about 60 Mb/s. The minifier was a bit slower at 20 Mb/s if I remember correctly. I fuzzed the lexer and parser, so there are little to no spurious crashes on weird input. Use it as you please, or just as an indication of the effort involved.Thanks. I will use this to create wrappers that ensure JS closures can be serialized.
Mar 02 2020
On 2/27/2020 6:06 AM, James Lu wrote:What parser generator library should I use for this purpose?https://github.com/DigitalMars/DMDScript
Feb 27 2020