www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - BNF grammar for D?

reply "landaire" <landergriffith+d gmail.com> writes:
There have been similar questions asked on this forum before 
(most recent one in June with no result: 
http://forum.dlang.org/thread/CAFMGiz8Fyv2A+ScQPQtEYEsFSybZNfY--nsxL5rqNoWWNA4agg mai
.gmail.com?page=1), 
but I was curious if anyone happens to have a BNF grammar for D 
laying around. I've searched all over and can't seem to find 
anything.

I'm trying to make a plugin for IntelliJ IDEA to support D 
(https://github.com/landaire/intelliD is what I have now 
utilizing DDT's lexer) but a lot of what's required to add 
advanced features like code folding, code completion, etc. is 
generally autogenerated by JetBrains's Grammar Kit plugin which 
only generates that stuff from a BNF grammar.

I know nothing about writing language grammars but based off what 
I've seen it doesn't look like it'd be *too* difficult to pick 
up, so if anyone also has suggestions for good readings I'd 
appreciate that as well.
Oct 27 2014
next sibling parent reply "Brian Schott" <briancschott gmail.com> writes:
On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote:

 I've searched all over and can't seem to find anything.


There are links to my projects in that thread. 
https://github.com/Hackerpilot/libdparse should help. The doc 
comments for the parser tell you the grammar that they implement. 
(The DGrammar project is based on these ddoc comments. This 
reminds me that I need to update it.) The official language spec 
on http://dlang.org/grammar.html has been getting better 
recently. You may want to look at that as well.


 I'm trying to make a plugin for IntelliJ IDEA to support D 
 (https://github.com/landaire/intelliD is what I have now 
 utilizing DDT's lexer) but a lot of what's required to add 
 advanced features like code folding, code completion, etc. is 
 generally autogenerated by JetBrains's Grammar Kit plugin which 
 only generates that stuff from a BNF grammar.


Beware that D's grammar is ambiguous in several places.
Oct 27 2014
next sibling parent "landaire" <landergriffith+d gmail.com> writes:
On Monday, 27 October 2014 at 22:30:15 UTC, Brian Schott wrote:

 On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote:

 I've searched all over and can't seem to find anything.


 There are links to my projects in that thread.


Sorry, what I meant was I couldn't find a BNF grammar :)

I must have overlooked libdparse but I did find  
https://github.com/Hackerpilot/DGrammar which helped quite a bit! 
But cool, thanks for mentioning the grammar comments.

I was more or less hoping I wouldn't have to redo a bunch of the 
work that someone's already done for me by writing the grammar. 
Even if I do have to do it from scratch it should be a good 
learning experience.


 Beware that D's grammar is ambiguous in several places.


Do you have any specific examples off the top of your head?
Oct 27 2014
prev sibling parent reply Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
 There are links to my projects in that thread. https://github.com/
 Hackerpilot/libdparse should help. The doc comments for the parser tell
 you the grammar that they implement. (The DGrammar project is based on
 these ddoc comments. This reminds me that I need to update it.) The
 official language spec on http://dlang.org/grammar.html has been getting
 better recently. You may want to look at that as well.


Not only was I ninja'd by the actual expert, just realized your project is
explicitly based on mine... go look at Brian's stuff, it's much better than
mine.


From my mucking about before, I think you'll get more mileage from using


libdparse directly than trying to (re)implement a parser for the plugin.
For one, you save yourself all the pain of fixing weird edge cases and
keeping up to date.
Oct 27 2014
parent reply "landaire" <landergriffith+d gmail.com> writes:
On Monday, 27 October 2014 at 22:43:55 UTC, Jeremy Powers via 
Digitalmars-d wrote:

From my mucking about before, I think you'll get more mileage 
from using
 libdparse directly than trying to (re)implement a parser for 
 the plugin.
 For one, you save yourself all the pain of fixing weird edge 
 cases and
 keeping up to date.


I agree, but integrating something external with IntelliJ's 
platform seems like kind of a PITA. It looks like a lot of stuff 
makes heavy use of PSI trees (see: 
https://confluence.jetbrains.com/display/IDEADEV/Developing+Custom+Language+Plugins+for+IntelliJ+IDEA#DevelopingCustomLanguagePluginsforIntelliJIDEA-ImplementingaParserandPSI)

I'm no expert in developing IntelliJ plugins either though...
Oct 27 2014
next sibling parent Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
 Glad to see you turned up here.


I'm usually lurking around...



 I agree, but integrating something external with IntelliJ's platform seems
 like kind of a PITA. It looks like a lot of stuff makes heavy use of PSI
 trees (see: https://confluence.jetbrains.com/display/IDEADEV/

 DevelopingCustomLanguagePluginsforIntelliJIDEA-ImplementingaParserandPSI)

 I'm no expert in developing IntelliJ plugins either though...


The totally-not-thought-out idea I had was to have the IntelliJ tree map to
the results of libdparse.  So each AST (or PSI) call is just a front for
the info provided by libdparse (with caching etc).  Then all the work
shifts away from writing a parser to writing the hairy integration bits.
Oct 27 2014
prev sibling parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 27/10/2014 22:53, landaire wrote:

 On Monday, 27 October 2014 at 22:43:55 UTC, Jeremy Powers via
 Digitalmars-d wrote:

 From my mucking about before, I think you'll get more mileage from using
 libdparse directly than trying to (re)implement a parser for the plugin.
 For one, you save yourself all the pain of fixing weird edge cases and
 keeping up to date.


 I agree, but integrating something external with IntelliJ's platform
 seems like kind of a PITA. It looks like a lot of stuff makes heavy use
 of PSI trees (see:
 https://confluence.jetbrains.com/display/IDEADEV/Developing+Custom+Language+Plugins+for+IntelliJ+IDEA#DevelopingCustomLanguagePluginsforIntelliJIDEA-ImplementingaParserandPSI)


 I'm no expert in developing IntelliJ plugins either though...


Have you considered the option of creating a Psi parser than creates 
PsiElements from the AST parsed by DDT's DTool ? That would should be 
fairly straightforward.

With of a fork of DDT/Dtool, you can even easily modify the DDT/Dtool 
parser to generate those PsiElements directly.


-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Oct 31 2014
prev sibling next sibling parent reply Jeremy Powers via Digitalmars-d <digitalmars-d puremagic.com> writes:
This looks eerily familiar...

I took a stab at an Intellij plugin a long time ago, and happened to pick
the same name:

https://github.com/elendel-/intelliD

Been meaning to pick back up again, but haven't had the chance... was
waiting for the Eclipse plugin code to mature, then got distracted.  Feel
free to take anything that looks useful.


Brian Schott (Hackerpilot) has done a lot of useful work grammar wise.  See:

https://github.com/Hackerpilot/DGrammar

An interesting path to take for an intellij plugin would be to use his
DCD/libdparse for all the heavy lifting, with the plugin just farming out
the work to external daemon process(es).  That would avoid having yet
another lexer/parser implementation to keep up to date.




On Mon, Oct 27, 2014 at 3:06 PM, landaire via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


 There have been similar questions asked on this forum before (most recent
 one in June with no result: http://forum.dlang.org/thread/CAFMGiz8Fyv2A+
 ScQPQtEYEsFSybZNfY--nsxL5rqNoWWNA4agg mail.gmail.com?page=1), but I was
 curious if anyone happens to have a BNF grammar for D laying around. I've
 searched all over and can't seem to find anything.

 I'm trying to make a plugin for IntelliJ IDEA to support D (
 https://github.com/landaire/intelliD is what I have now utilizing DDT's
 lexer) but a lot of what's required to add advanced features like code
 folding, code completion, etc. is generally autogenerated by JetBrains's
 Grammar Kit plugin which only generates that stuff from a BNF grammar.

 I know nothing about writing language grammars but based off what I've
 seen it doesn't look like it'd be *too* difficult to pick up, so if anyone
 also has suggestions for good readings I'd appreciate that as well.
Oct 27 2014
next sibling parent reply "landaire" <landergriffith+d gmail.com> writes:
On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via 
Digitalmars-d wrote:

 This looks eerily familiar...

 I took a stab at an Intellij plugin a long time ago, and 
 happened to pick
 the same name:

 https://github.com/elendel-/intelliD

 Been meaning to pick back up again, but haven't had the 
 chance... was
 waiting for the Eclipse plugin code to mature, then got 
 distracted.  Feel
 free to take anything that looks useful.


 Brian Schott (Hackerpilot) has done a lot of useful work 
 grammar wise.  See:

 https://github.com/Hackerpilot/DGrammar

 An interesting path to take for an intellij plugin would be to 
 use his
 DCD/libdparse for all the heavy lifting, with the plugin just 
 farming out
 the work to external daemon process(es).  That would avoid 
 having yet
 another lexer/parser implementation to keep up to date.


I actually forked yours originally to get it running in IDEA 14, 
then decided to mess with the lexer since it was causing some 
problems and went off from there. I saw it was somewhat inactive 
and tried contacting you, but couldn't find any of your contact 
info anywhere! Glad to see you turned up here.
Oct 27 2014
parent reply "Kingsley" <kingsley.hendrickse gmail.com> writes:
On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote:

 On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via 
 Digitalmars-d wrote:

 This looks eerily familiar...

 I took a stab at an Intellij plugin a long time ago, and 
 happened to pick
 the same name:

 https://github.com/elendel-/intelliD

 Been meaning to pick back up again, but haven't had the 
 chance... was
 waiting for the Eclipse plugin code to mature, then got 
 distracted.  Feel
 free to take anything that looks useful.


 Brian Schott (Hackerpilot) has done a lot of useful work 
 grammar wise.  See:

 https://github.com/Hackerpilot/DGrammar

 An interesting path to take for an intellij plugin would be to 
 use his
 DCD/libdparse for all the heavy lifting, with the plugin just 
 farming out
 the work to external daemon process(es).  That would avoid 
 having yet
 another lexer/parser implementation to keep up to date.


 I actually forked yours originally to get it running in IDEA 
 14, then decided to mess with the lexer since it was causing 
 some problems and went off from there. I saw it was somewhat 
 inactive and tried contacting you, but couldn't find any of 
 your contact info anywhere! Glad to see you turned up here.


Hi guys,

I have been working on an intellij plugin which is here: 
https://github.com/kingsleyh/DLanguage

I only started learning D a couple of weeks ago and I haven't 
written an intellij plugin before - and also I'm not very 
familiar with parsing/lexing.

On my first pass - I got all the infrastructure working - e.g. 
run configurations, project creation, file creation etc. And I'm 
working on a BNF and JFlex by hand - but I realise it will take 
some time - so I have a branch which implements the DDT 
parser/lexer so I could get something working while I work on the 
bnf - however it breaks the contextual run configurations - which 
is pretty much essential for the plugin to be useful.

I'd like to get the DDT parser/lexer working but I'm not really 
sure where to go from where I am now. My DDT branch is called 
with_ddt.

If anyone has any pointers to what I need to do next that would 
be very helpful - even in terms of reading or reference material. 
I guess it all comes down to the DParserDefinition class - as my 
master branch uses a FlexAdapter which gives access to the 
context. But in the with_ddt branch its just using the DParser - 
which I think needs to have the hooks into the AST tree 
implemented or something like that.

Anyway let me know if you have any pointers or offers of help :)

--K
Dec 16 2014
parent reply "Kingsley" <kingsley.hendrickse gmail.com> writes:
On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote:

 On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote:

 On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via 
 Digitalmars-d wrote:

 This looks eerily familiar...

 I took a stab at an Intellij plugin a long time ago, and 
 happened to pick
 the same name:

 https://github.com/elendel-/intelliD

 Been meaning to pick back up again, but haven't had the 
 chance... was
 waiting for the Eclipse plugin code to mature, then got 
 distracted.  Feel
 free to take anything that looks useful.


 Brian Schott (Hackerpilot) has done a lot of useful work 
 grammar wise.  See:

 https://github.com/Hackerpilot/DGrammar

 An interesting path to take for an intellij plugin would be 
 to use his
 DCD/libdparse for all the heavy lifting, with the plugin just 
 farming out
 the work to external daemon process(es).  That would avoid 
 having yet
 another lexer/parser implementation to keep up to date.


 I actually forked yours originally to get it running in IDEA 
 14, then decided to mess with the lexer since it was causing 
 some problems and went off from there. I saw it was somewhat 
 inactive and tried contacting you, but couldn't find any of 
 your contact info anywhere! Glad to see you turned up here.


 Hi guys,

 I have been working on an intellij plugin which is here: 
 https://github.com/kingsleyh/DLanguage

 I only started learning D a couple of weeks ago and I haven't 
 written an intellij plugin before - and also I'm not very 
 familiar with parsing/lexing.

 On my first pass - I got all the infrastructure working - e.g. 
 run configurations, project creation, file creation etc. And 
 I'm working on a BNF and JFlex by hand - but I realise it will 
 take some time - so I have a branch which implements the DDT 
 parser/lexer so I could get something working while I work on 
 the bnf - however it breaks the contextual run configurations - 
 which is pretty much essential for the plugin to be useful.

 I'd like to get the DDT parser/lexer working but I'm not really 
 sure where to go from where I am now. My DDT branch is called 
 with_ddt.

 If anyone has any pointers to what I need to do next that would 
 be very helpful - even in terms of reading or reference 
 material. I guess it all comes down to the DParserDefinition 
 class - as my master branch uses a FlexAdapter which gives 
 access to the context. But in the with_ddt branch its just 
 using the DParser - which I think needs to have the hooks into 
 the AST tree implemented or something like that.

 Anyway let me know if you have any pointers or offers of help :)

 --K


Actually I guess what I need is a way to make the PSI structure. 
I'm not sure how I would do that with the DDT code I have 
imported.
Dec 16 2014
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 17/12/2014 1:35 p.m., Kingsley wrote:

 On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote:

 On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote:

 On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via
 Digitalmars-d wrote:

 This looks eerily familiar...

 I took a stab at an Intellij plugin a long time ago, and happened to
 pick
 the same name:

 https://github.com/elendel-/intelliD

 Been meaning to pick back up again, but haven't had the chance... was
 waiting for the Eclipse plugin code to mature, then got distracted.
 Feel
 free to take anything that looks useful.


 Brian Schott (Hackerpilot) has done a lot of useful work grammar
 wise.  See:

 https://github.com/Hackerpilot/DGrammar

 An interesting path to take for an intellij plugin would be to use his
 DCD/libdparse for all the heavy lifting, with the plugin just
 farming out
 the work to external daemon process(es).  That would avoid having yet
 another lexer/parser implementation to keep up to date.


 I actually forked yours originally to get it running in IDEA 14, then
 decided to mess with the lexer since it was causing some problems and
 went off from there. I saw it was somewhat inactive and tried
 contacting you, but couldn't find any of your contact info anywhere!
 Glad to see you turned up here.


 Hi guys,

 I have been working on an intellij plugin which is here:
 https://github.com/kingsleyh/DLanguage

 I only started learning D a couple of weeks ago and I haven't written
 an intellij plugin before - and also I'm not very familiar with
 parsing/lexing.

 On my first pass - I got all the infrastructure working - e.g. run
 configurations, project creation, file creation etc. And I'm working
 on a BNF and JFlex by hand - but I realise it will take some time - so
 I have a branch which implements the DDT parser/lexer so I could get
 something working while I work on the bnf - however it breaks the
 contextual run configurations - which is pretty much essential for the
 plugin to be useful.

 I'd like to get the DDT parser/lexer working but I'm not really sure
 where to go from where I am now. My DDT branch is called with_ddt.

 If anyone has any pointers to what I need to do next that would be
 very helpful - even in terms of reading or reference material. I guess
 it all comes down to the DParserDefinition class - as my master branch
 uses a FlexAdapter which gives access to the context. But in the
 with_ddt branch its just using the DParser - which I think needs to
 have the hooks into the AST tree implemented or something like that.

 Anyway let me know if you have any pointers or offers of help :)

 --K


 Actually I guess what I need is a way to make the PSI structure. I'm not
 sure how I would do that with the DDT code I have imported.


I've sent you an email reguarding my own work on getting a plugin 
together using Dscanner. As well as the source code.
Feel free to use it.
Dec 16 2014
parent reply "Kingsley" <kingsley.hendrickse gmail.com> writes:
On Wednesday, 17 December 2014 at 03:03:59 UTC, Rikki Cattermole 
wrote:

 On 17/12/2014 1:35 p.m., Kingsley wrote:

 On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote:

 On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote:

 On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via
 Digitalmars-d wrote:

 This looks eerily familiar...

 I took a stab at an Intellij plugin a long time ago, and 
 happened to
 pick
 the same name:

 https://github.com/elendel-/intelliD

 Been meaning to pick back up again, but haven't had the 
 chance... was
 waiting for the Eclipse plugin code to mature, then got 
 distracted.
 Feel
 free to take anything that looks useful.


 Brian Schott (Hackerpilot) has done a lot of useful work 
 grammar
 wise.  See:

 https://github.com/Hackerpilot/DGrammar

 An interesting path to take for an intellij plugin would be 
 to use his
 DCD/libdparse for all the heavy lifting, with the plugin 
 just
 farming out
 the work to external daemon process(es).  That would avoid 
 having yet
 another lexer/parser implementation to keep up to date.


 I actually forked yours originally to get it running in IDEA 
 14, then
 decided to mess with the lexer since it was causing some 
 problems and
 went off from there. I saw it was somewhat inactive and tried
 contacting you, but couldn't find any of your contact info 
 anywhere!
 Glad to see you turned up here.


 Hi guys,

 I have been working on an intellij plugin which is here:
 https://github.com/kingsleyh/DLanguage

 I only started learning D a couple of weeks ago and I haven't 
 written
 an intellij plugin before - and also I'm not very familiar 
 with
 parsing/lexing.

 On my first pass - I got all the infrastructure working - 
 e.g. run
 configurations, project creation, file creation etc. And I'm 
 working
 on a BNF and JFlex by hand - but I realise it will take some 
 time - so
 I have a branch which implements the DDT parser/lexer so I 
 could get
 something working while I work on the bnf - however it breaks 
 the
 contextual run configurations - which is pretty much 
 essential for the
 plugin to be useful.

 I'd like to get the DDT parser/lexer working but I'm not 
 really sure
 where to go from where I am now. My DDT branch is called 
 with_ddt.

 If anyone has any pointers to what I need to do next that 
 would be
 very helpful - even in terms of reading or reference 
 material. I guess
 it all comes down to the DParserDefinition class - as my 
 master branch
 uses a FlexAdapter which gives access to the context. But in 
 the
 with_ddt branch its just using the DParser - which I think 
 needs to
 have the hooks into the AST tree implemented or something 
 like that.

 Anyway let me know if you have any pointers or offers of help 
 :)

 --K


 Actually I guess what I need is a way to make the PSI 
 structure. I'm not
 sure how I would do that with the DDT code I have imported.


 I've sent you an email reguarding my own work on getting a 
 plugin together using Dscanner. As well as the source code.
 Feel free to use it.


I've successfully integrated the Psi generation stuff with the 
DDT parser/lexer now with a huge thanks to Rikki. So now I think 
I can add all the other ps related stuff - code insight, 
refactoring, completion etc etc

Of course its the first cut - so I'll have to do some code 
cleanup once I understand what I'm doing!

Thanks for your help and support :)
Dec 17 2014
parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 17/12/2014 09:13, Kingsley wrote:

 On Wednesday, 17 December 2014 at 03:03:59 UTC, Rikki Cattermole wrote:

 On 17/12/2014 1:35 p.m., Kingsley wrote:

 On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote:

 On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote:

 On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers via
 Digitalmars-d wrote:

 This looks eerily familiar...

 I took a stab at an Intellij plugin a long time ago, and happened to
 pick
 the same name:

 https://github.com/elendel-/intelliD

 Been meaning to pick back up again, but haven't had the chance... was
 waiting for the Eclipse plugin code to mature, then got distracted.
 Feel
 free to take anything that looks useful.


 Brian Schott (Hackerpilot) has done a lot of useful work grammar
 wise.  See:

 https://github.com/Hackerpilot/DGrammar

 An interesting path to take for an intellij plugin would be to use
 his
 DCD/libdparse for all the heavy lifting, with the plugin just
 farming out
 the work to external daemon process(es).  That would avoid having yet
 another lexer/parser implementation to keep up to date.


 I actually forked yours originally to get it running in IDEA 14, then
 decided to mess with the lexer since it was causing some problems and
 went off from there. I saw it was somewhat inactive and tried
 contacting you, but couldn't find any of your contact info anywhere!
 Glad to see you turned up here.


 Hi guys,

 I have been working on an intellij plugin which is here:
 https://github.com/kingsleyh/DLanguage

 I only started learning D a couple of weeks ago and I haven't written
 an intellij plugin before - and also I'm not very familiar with
 parsing/lexing.

 On my first pass - I got all the infrastructure working - e.g. run
 configurations, project creation, file creation etc. And I'm working
 on a BNF and JFlex by hand - but I realise it will take some time - so
 I have a branch which implements the DDT parser/lexer so I could get
 something working while I work on the bnf - however it breaks the
 contextual run configurations - which is pretty much essential for the
 plugin to be useful.

 I'd like to get the DDT parser/lexer working but I'm not really sure
 where to go from where I am now. My DDT branch is called with_ddt.

 If anyone has any pointers to what I need to do next that would be
 very helpful - even in terms of reading or reference material. I guess
 it all comes down to the DParserDefinition class - as my master branch
 uses a FlexAdapter which gives access to the context. But in the
 with_ddt branch its just using the DParser - which I think needs to
 have the hooks into the AST tree implemented or something like that.

 Anyway let me know if you have any pointers or offers of help :)

 --K


 Actually I guess what I need is a way to make the PSI structure. I'm not
 sure how I would do that with the DDT code I have imported.


 I've sent you an email reguarding my own work on getting a plugin
 together using Dscanner. As well as the source code.
 Feel free to use it.


 I've successfully integrated the Psi generation stuff with the DDT
 parser/lexer now with a huge thanks to Rikki. So now I think I can add
 all the other ps related stuff - code insight, refactoring, completion
 etc etc

 Of course its the first cut - so I'll have to do some code cleanup once
 I understand what I'm doing!

 Thanks for your help and support :)


Also, if you have questions about the DDT parser/lexer, or if there are 
changes that might help your project, let me know, I'll be glad to add 
such changes (assuming they also fit well with the way DDT uses the 
parser/lexer).


-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Dec 17 2014
parent reply "Kingsley" <kingsley.hendrickse gmail.com> writes:
On Wednesday, 17 December 2014 at 12:31:32 UTC, Bruno Medeiros 
wrote:

 On 17/12/2014 09:13, Kingsley wrote:

 On Wednesday, 17 December 2014 at 03:03:59 UTC, Rikki 
 Cattermole wrote:

 On 17/12/2014 1:35 p.m., Kingsley wrote:

 On Tuesday, 16 December 2014 at 23:25:02 UTC, Kingsley wrote:

 On Monday, 27 October 2014 at 22:40:20 UTC, landaire wrote:

 On Monday, 27 October 2014 at 22:35:16 UTC, Jeremy Powers 
 via
 Digitalmars-d wrote:

 This looks eerily familiar...

 I took a stab at an Intellij plugin a long time ago, and 
 happened to
 pick
 the same name:

 https://github.com/elendel-/intelliD

 Been meaning to pick back up again, but haven't had the 
 chance... was
 waiting for the Eclipse plugin code to mature, then got 
 distracted.
 Feel
 free to take anything that looks useful.


 Brian Schott (Hackerpilot) has done a lot of useful work 
 grammar
 wise.  See:

 https://github.com/Hackerpilot/DGrammar

 An interesting path to take for an intellij plugin would 
 be to use
 his
 DCD/libdparse for all the heavy lifting, with the plugin 
 just
 farming out
 the work to external daemon process(es).  That would 
 avoid having yet
 another lexer/parser implementation to keep up to date.


 I actually forked yours originally to get it running in 
 IDEA 14, then
 decided to mess with the lexer since it was causing some 
 problems and
 went off from there. I saw it was somewhat inactive and 
 tried
 contacting you, but couldn't find any of your contact info 
 anywhere!
 Glad to see you turned up here.


 Hi guys,

 I have been working on an intellij plugin which is here:
 https://github.com/kingsleyh/DLanguage

 I only started learning D a couple of weeks ago and I 
 haven't written
 an intellij plugin before - and also I'm not very familiar 
 with
 parsing/lexing.

 On my first pass - I got all the infrastructure working - 
 e.g. run
 configurations, project creation, file creation etc. And 
 I'm working
 on a BNF and JFlex by hand - but I realise it will take 
 some time - so
 I have a branch which implements the DDT parser/lexer so I 
 could get
 something working while I work on the bnf - however it 
 breaks the
 contextual run configurations - which is pretty much 
 essential for the
 plugin to be useful.

 I'd like to get the DDT parser/lexer working but I'm not 
 really sure
 where to go from where I am now. My DDT branch is called 
 with_ddt.

 If anyone has any pointers to what I need to do next that 
 would be
 very helpful - even in terms of reading or reference 
 material. I guess
 it all comes down to the DParserDefinition class - as my 
 master branch
 uses a FlexAdapter which gives access to the context. But 
 in the
 with_ddt branch its just using the DParser - which I think 
 needs to
 have the hooks into the AST tree implemented or something 
 like that.

 Anyway let me know if you have any pointers or offers of 
 help :)

 --K


 Actually I guess what I need is a way to make the PSI 
 structure. I'm not
 sure how I would do that with the DDT code I have imported.


 I've sent you an email reguarding my own work on getting a 
 plugin
 together using Dscanner. As well as the source code.
 Feel free to use it.


 I've successfully integrated the Psi generation stuff with the 
 DDT
 parser/lexer now with a huge thanks to Rikki. So now I think I 
 can add
 all the other ps related stuff - code insight, refactoring, 
 completion
 etc etc

 Of course its the first cut - so I'll have to do some code 
 cleanup once
 I understand what I'm doing!

 Thanks for your help and support :)


 Also, if you have questions about the DDT parser/lexer, or if 
 there are changes that might help your project, let me know, 
 I'll be glad to add such changes (assuming they also fit well 
 with the way DDT uses the parser/lexer).


Hi Bruno,

Thanks very much. I do have a couple of questions about DDT in 
relation to my plugin.

Firstly - I'm not too familiar with parsing/lexing but at the 
moment the Psi Structure I have implemented that comes from the 
DDT parser/lexer is not in any kind of hierarchy. All the 
PsiElements are available but all at the same level. Is this how 
the DDT parser works? Or is it down to my implementation of the 
Parser/Lexer that wraps it to create some hierarchy.

For intellij it's going to be vastly easier to have a hierarchy 
with nested elements in order to get hold of a structure 
representing a class or a function for example - in order to do 
things like get the start and end lines of a class definition in 
order to apply code folding and to use for searching for classes 
and stuff.

Secondly - how active it the development of DDT - does it keep up 
with the D2 releases.

--Kingsley
Dec 17 2014
next sibling parent reply "Kingsley" <kingsley.hendrickse gmail.com> writes:
 Hi Bruno,

 Thanks very much. I do have a couple of questions about DDT in 
 relation to my plugin.

 Firstly - I'm not too familiar with parsing/lexing but at the 
 moment the Psi Structure I have implemented that comes from the 
 DDT parser/lexer is not in any kind of hierarchy. All the 
 PsiElements are available but all at the same level. Is this 
 how the DDT parser works? Or is it down to my implementation of 
 the Parser/Lexer that wraps it to create some hierarchy.

 For intellij it's going to be vastly easier to have a hierarchy 
 with nested elements in order to get hold of a structure 
 representing a class or a function for example - in order to do 
 things like get the start and end lines of a class definition 
 in order to apply code folding and to use for searching for 
 classes and stuff.

 Secondly - how active it the development of DDT - does it keep 
 up with the D2 releases.

 --Kingsley


After doing a bit more research it looks like I have to create 
the psi hierarchy myself - my current psi structure is flat 
because I'm just converting the DeeTokens into PsiElements 
directly. I've still got some experimentation to do. On the plus 
side I implemented commenting, code folding but everything else 
needs a psi hierarchy
Dec 17 2014
parent reply "Kingsley" <kingsley.hendrickse gmail.com> writes:
On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote:

 Hi Bruno,

 Thanks very much. I do have a couple of questions about DDT in 
 relation to my plugin.

 Firstly - I'm not too familiar with parsing/lexing but at the 
 moment the Psi Structure I have implemented that comes from 
 the DDT parser/lexer is not in any kind of hierarchy. All the 
 PsiElements are available but all at the same level. Is this 
 how the DDT parser works? Or is it down to my implementation 
 of the Parser/Lexer that wraps it to create some hierarchy.

 For intellij it's going to be vastly easier to have a 
 hierarchy with nested elements in order to get hold of a 
 structure representing a class or a function for example - in 
 order to do things like get the start and end lines of a class 
 definition in order to apply code folding and to use for 
 searching for classes and stuff.

 Secondly - how active it the development of DDT - does it keep 
 up with the D2 releases.

 --Kingsley


 After doing a bit more research it looks like I have to create 
 the psi hierarchy myself - my current psi structure is flat 
 because I'm just converting the DeeTokens into PsiElements 
 directly. I've still got some experimentation to do. On the 
 plus side I implemented commenting, code folding but everything 
 else needs a psi hierarchy


I've done some more investigation and I do need to build the 
parser myself in order to create the various constructs. I've 
made a start but I haven't gotten very far yet because I don't 
fully understand the correct way to proceed.

I also had a look at using the DeeParser - because it already 
does most of what I want. However the intellij plugin wants a 
PsiParser which returns an intellij ASTNode in the primary parse 
method. I can't see an easy way to hook this up with DeeParser 
because the ParsedResult although had a node method on it - gives 
back the wrong type of ASTNode.

Any pointers on how I might get the DeeParser to interface to an 
intellij ASTNode would be appreciated.
Dec 18 2014
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 19/12/2014 10:19 a.m., Kingsley wrote:

 On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote:

 Hi Bruno,

 Thanks very much. I do have a couple of questions about DDT in
 relation to my plugin.

 Firstly - I'm not too familiar with parsing/lexing but at the moment
 the Psi Structure I have implemented that comes from the DDT
 parser/lexer is not in any kind of hierarchy. All the PsiElements are
 available but all at the same level. Is this how the DDT parser
 works? Or is it down to my implementation of the Parser/Lexer that
 wraps it to create some hierarchy.

 For intellij it's going to be vastly easier to have a hierarchy with
 nested elements in order to get hold of a structure representing a
 class or a function for example - in order to do things like get the
 start and end lines of a class definition in order to apply code
 folding and to use for searching for classes and stuff.

 Secondly - how active it the development of DDT - does it keep up
 with the D2 releases.

 --Kingsley


 After doing a bit more research it looks like I have to create the psi
 hierarchy myself - my current psi structure is flat because I'm just
 converting the DeeTokens into PsiElements directly. I've still got
 some experimentation to do. On the plus side I implemented commenting,
 code folding but everything else needs a psi hierarchy


 I've done some more investigation and I do need to build the parser
 myself in order to create the various constructs. I've made a start but
 I haven't gotten very far yet because I don't fully understand the
 correct way to proceed.

 I also had a look at using the DeeParser - because it already does most
 of what I want. However the intellij plugin wants a PsiParser which
 returns an intellij ASTNode in the primary parse method. I can't see an
 easy way to hook this up with DeeParser because the ParsedResult
 although had a node method on it - gives back the wrong type of ASTNode.

 Any pointers on how I might get the DeeParser to interface to an
 intellij ASTNode would be appreciated.


Read my codebase again, it'll answer a lot of questions. Your parser is 
different, but what it produces shouldn't be. and yes it supports 
hierarchies.
Dec 18 2014
parent reply "Kingsley" <kingsley.hendrickse gmail.com> writes:
On Friday, 19 December 2014 at 02:53:02 UTC, Rikki Cattermole 
wrote:

 On 19/12/2014 10:19 a.m., Kingsley wrote:

 On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote:

 Hi Bruno,

 Thanks very much. I do have a couple of questions about DDT 
 in
 relation to my plugin.

 Firstly - I'm not too familiar with parsing/lexing but at 
 the moment
 the Psi Structure I have implemented that comes from the DDT
 parser/lexer is not in any kind of hierarchy. All the 
 PsiElements are
 available but all at the same level. Is this how the DDT 
 parser
 works? Or is it down to my implementation of the 
 Parser/Lexer that
 wraps it to create some hierarchy.

 For intellij it's going to be vastly easier to have a 
 hierarchy with
 nested elements in order to get hold of a structure 
 representing a
 class or a function for example - in order to do things like 
 get the
 start and end lines of a class definition in order to apply 
 code
 folding and to use for searching for classes and stuff.

 Secondly - how active it the development of DDT - does it 
 keep up
 with the D2 releases.

 --Kingsley


 After doing a bit more research it looks like I have to 
 create the psi
 hierarchy myself - my current psi structure is flat because 
 I'm just
 converting the DeeTokens into PsiElements directly. I've 
 still got
 some experimentation to do. On the plus side I implemented 
 commenting,
 code folding but everything else needs a psi hierarchy


 I've done some more investigation and I do need to build the 
 parser
 myself in order to create the various constructs. I've made a 
 start but
 I haven't gotten very far yet because I don't fully understand 
 the
 correct way to proceed.

 I also had a look at using the DeeParser - because it already 
 does most
 of what I want. However the intellij plugin wants a PsiParser 
 which
 returns an intellij ASTNode in the primary parse method. I 
 can't see an
 easy way to hook this up with DeeParser because the 
 ParsedResult
 although had a node method on it - gives back the wrong type 
 of ASTNode.

 Any pointers on how I might get the DeeParser to interface to 
 an
 intellij ASTNode would be appreciated.


 Read my codebase again, it'll answer a lot of questions. Your 
 parser is different, but what it produces shouldn't be. and yes 
 it supports hierarchies.


Hi

So finally after a lot of wrestling with the internals of 
intellij I finally managed to get a working parser implementation 
that produces a psi hierarchy based on the DeeParser from the ddt 
code.

The main issue was that Intellij only wants you to create a 
parser using their toolset - which is either with a BNF grammar 
that you can then generate the parser - or with a hand written 
parser. Since I'm already using the DDT lexer and there is a 
perfectly good DDT parser as well - I just wanted to re-use the 
DDT parser.

However Intellij does not provide any way to create a custom 
AST/PSI structure or use an external parser. So I basically had 
to wrap the DeeParse inside the Intellij parser and sync them up 
programmatically. It's not the most efficient way in the world 
but it at least works.

In the long term I will write a BNF grammar for Intellij (using 
their toolkit) but I can see that will take me several months so 
this is a quick way to get the plugin up and running with all the 
power of intellij extras without spending several months stuck 
learning all about the complexities of grammar parsing and lexing.

Thanks very much for you help. Once I get a bit more of the cool 
stuff done I will release the plugin.
Dec 20 2014
next sibling parent reply "Kingsley" <kingsley.hendrickse gmail.com> writes:
On Sunday, 21 December 2014 at 00:34:06 UTC, Kingsley wrote:

 On Friday, 19 December 2014 at 02:53:02 UTC, Rikki Cattermole 
 wrote:

 On 19/12/2014 10:19 a.m., Kingsley wrote:

 On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley 
 wrote:

 Hi Bruno,

 Thanks very much. I do have a couple of questions about DDT 
 in
 relation to my plugin.

 Firstly - I'm not too familiar with parsing/lexing but at 
 the moment
 the Psi Structure I have implemented that comes from the DDT
 parser/lexer is not in any kind of hierarchy. All the 
 PsiElements are
 available but all at the same level. Is this how the DDT 
 parser
 works? Or is it down to my implementation of the 
 Parser/Lexer that
 wraps it to create some hierarchy.

 For intellij it's going to be vastly easier to have a 
 hierarchy with
 nested elements in order to get hold of a structure 
 representing a
 class or a function for example - in order to do things 
 like get the
 start and end lines of a class definition in order to apply 
 code
 folding and to use for searching for classes and stuff.

 Secondly - how active it the development of DDT - does it 
 keep up
 with the D2 releases.

 --Kingsley


 After doing a bit more research it looks like I have to 
 create the psi
 hierarchy myself - my current psi structure is flat because 
 I'm just
 converting the DeeTokens into PsiElements directly. I've 
 still got
 some experimentation to do. On the plus side I implemented 
 commenting,
 code folding but everything else needs a psi hierarchy


 I've done some more investigation and I do need to build the 
 parser
 myself in order to create the various constructs. I've made a 
 start but
 I haven't gotten very far yet because I don't fully 
 understand the
 correct way to proceed.

 I also had a look at using the DeeParser - because it already 
 does most
 of what I want. However the intellij plugin wants a PsiParser 
 which
 returns an intellij ASTNode in the primary parse method. I 
 can't see an
 easy way to hook this up with DeeParser because the 
 ParsedResult
 although had a node method on it - gives back the wrong type 
 of ASTNode.

 Any pointers on how I might get the DeeParser to interface to 
 an
 intellij ASTNode would be appreciated.


 Read my codebase again, it'll answer a lot of questions. Your 
 parser is different, but what it produces shouldn't be. and 
 yes it supports hierarchies.


 Hi

 So finally after a lot of wrestling with the internals of 
 intellij I finally managed to get a working parser 
 implementation that produces a psi hierarchy based on the 
 DeeParser from the ddt code.

 The main issue was that Intellij only wants you to create a 
 parser using their toolset - which is either with a BNF grammar 
 that you can then generate the parser - or with a hand written 
 parser. Since I'm already using the DDT lexer and there is a 
 perfectly good DDT parser as well - I just wanted to re-use the 
 DDT parser.


Hi Bruno - would be easy to return the list of tokens included 
for each node in the DeeParser?

 However Intellij does not provide any way to create a custom 
 AST/PSI structure or use an external parser. So I basically had 
 to wrap the DeeParse inside the Intellij parser and sync them 
 up programmatically. It's not the most efficient way in the 
 world but it at least works.

 In the long term I will write a BNF grammar for Intellij (using 
 their toolkit) but I can see that will take me several months 
 so this is a quick way to get the plugin up and running with 
 all the power of intellij extras without spending several 
 months stuck learning all about the complexities of grammar 
 parsing and lexing.

 Thanks very much for you help. Once I get a bit more of the 
 cool stuff done I will release the plugin.
Dec 22 2014
parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 22/12/2014 11:44, Kingsley wrote:

 Hi Bruno - would be easy to return the list of tokens included for each
 node in the DeeParser?


You can create an utility method that does that, if you have a 
DeeParseResult.

The DeeParseResult has a 'tokenList' member, ordered by the source 
range. With a node's source range, you can do a binary search in that 
list to find the corresponding tokens.

For more convenience, I guess DeeParser could be modified so that this 
information - in the form of a sublist of 'tokenList' - would be stored 
directly in each ASTNode, in the 'data' field. This way one would not 
need to provide the DeeParseResult directly.


-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Jan 08 2015
prev sibling parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 21/12/2014 00:34, Kingsley wrote:

 On Friday, 19 December 2014 at 02:53:02 UTC, Rikki Cattermole wrote:

 On 19/12/2014 10:19 a.m., Kingsley wrote:

 On Wednesday, 17 December 2014 at 21:05:05 UTC, Kingsley wrote:

 Hi Bruno,

 Thanks very much. I do have a couple of questions about DDT in
 relation to my plugin.

 Firstly - I'm not too familiar with parsing/lexing but at the moment
 the Psi Structure I have implemented that comes from the DDT
 parser/lexer is not in any kind of hierarchy. All the PsiElements are
 available but all at the same level. Is this how the DDT parser
 works? Or is it down to my implementation of the Parser/Lexer that
 wraps it to create some hierarchy.

 For intellij it's going to be vastly easier to have a hierarchy with
 nested elements in order to get hold of a structure representing a
 class or a function for example - in order to do things like get the
 start and end lines of a class definition in order to apply code
 folding and to use for searching for classes and stuff.

 Secondly - how active it the development of DDT - does it keep up
 with the D2 releases.

 --Kingsley


 After doing a bit more research it looks like I have to create the psi
 hierarchy myself - my current psi structure is flat because I'm just
 converting the DeeTokens into PsiElements directly. I've still got
 some experimentation to do. On the plus side I implemented commenting,
 code folding but everything else needs a psi hierarchy


 I've done some more investigation and I do need to build the parser
 myself in order to create the various constructs. I've made a start but
 I haven't gotten very far yet because I don't fully understand the
 correct way to proceed.

 I also had a look at using the DeeParser - because it already does most
 of what I want. However the intellij plugin wants a PsiParser which
 returns an intellij ASTNode in the primary parse method. I can't see an
 easy way to hook this up with DeeParser because the ParsedResult
 although had a node method on it - gives back the wrong type of ASTNode.

 Any pointers on how I might get the DeeParser to interface to an
 intellij ASTNode would be appreciated.


 Read my codebase again, it'll answer a lot of questions. Your parser
 is different, but what it produces shouldn't be. and yes it supports
 hierarchies.


 Hi

 So finally after a lot of wrestling with the internals of intellij I
 finally managed to get a working parser implementation that produces a
 psi hierarchy based on the DeeParser from the ddt code.

 The main issue was that Intellij only wants you to create a parser using
 their toolset - which is either with a BNF grammar that you can then
 generate the parser - or with a hand written parser. Since I'm already
 using the DDT lexer and there is a perfectly good DDT parser as well - I
 just wanted to re-use the DDT parser.

 However Intellij does not provide any way to create a custom AST/PSI
 structure or use an external parser. So I basically had to wrap the
 DeeParse inside the Intellij parser and sync them up programmatically.
 It's not the most efficient way in the world but it at least works.

 In the long term I will write a BNF grammar for Intellij (using their
 toolkit) but I can see that will take me several months so this is a
 quick way to get the plugin up and running with all the power of
 intellij extras without spending several months stuck learning all about
 the complexities of grammar parsing and lexing.

 Thanks very much for you help. Once I get a bit more of the cool stuff
 done I will release the plugin.


Again, I'm not familiar with Intellij internals or the PSI structure, so 
I don't know how the DDT parser can be adapted to that.

However, I do suspect there should be a way to create a PSI structure 
from the DDT ASTNode structure.

PS: Sorry for the long delay in replying, I don't often check the 
digitalmars.D newsgroup/forum, and can sometimes be behind on the posts 
there. If you want to grab my attention, better to post on 
digitalmars.D.ide as I keep a closer eye on that newsgroup.

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Jan 08 2015
prev sibling parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 17/12/2014 17:19, Kingsley wrote:

 Secondly - how active it the development of DDT - does it keep up with
 the D2 releases.


Yes, it keeps up, and I plan to keep that up for the foreseeable future. 
(Since language grammar changes are fairly uncommon, and easy to 
implement when they happen).

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Jan 08 2015
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 27/10/2014 22:35, Jeremy Powers via Digitalmars-d wrote:

 An interesting path to take for an intellij plugin would be to use his
 DCD/libdparse for all the heavy lifting, with the plugin just farming
 out the work to external daemon process(es).  That would avoid having
 yet another lexer/parser implementation to keep up to date.


Integrating DCD might work well, because autocompletion is used 
sporadically (and is invoked explicitly by the user). But parsing is 
probably invoked quite often by IntelliJ, and as such it could be tricky 
to integrate with libdparse (because you'd have to invoke an external 
process all the time)

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Oct 31 2014
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 1/11/2014 12:35 a.m., Bruno Medeiros wrote:

 On 27/10/2014 22:35, Jeremy Powers via Digitalmars-d wrote:

 An interesting path to take for an intellij plugin would be to use his
 DCD/libdparse for all the heavy lifting, with the plugin just farming
 out the work to external daemon process(es).  That would avoid having
 yet another lexer/parser implementation to keep up to date.


 Integrating DCD might work well, because autocompletion is used
 sporadically (and is invoked explicitly by the user). But parsing is
 probably invoked quite often by IntelliJ, and as such it could be tricky
 to integrate with libdparse (because you'd have to invoke an external
 process all the time)


 From my small test file with my attempt at this, its fine to call e.g. 
DScanner to grab the ast. Its actually pretty fast which I was a little 
shocked about.
Oct 31 2014
parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 31/10/2014 12:16, Rikki Cattermole wrote:

 On 1/11/2014 12:35 a.m., Bruno Medeiros wrote:

 On 27/10/2014 22:35, Jeremy Powers via Digitalmars-d wrote:

 An interesting path to take for an intellij plugin would be to use his
 DCD/libdparse for all the heavy lifting, with the plugin just farming
 out the work to external daemon process(es).  That would avoid having
 yet another lexer/parser implementation to keep up to date.


 Integrating DCD might work well, because autocompletion is used
 sporadically (and is invoked explicitly by the user). But parsing is
 probably invoked quite often by IntelliJ, and as such it could be tricky
 to integrate with libdparse (because you'd have to invoke an external
 process all the time)


  From my small test file with my attempt at this, its fine to call e.g.
 DScanner to grab the ast. Its actually pretty fast which I was a little
 shocked about.


How fast is "pretty fast"? Milliseconds please.

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Nov 07 2014
parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 8/11/2014 4:16 a.m., Bruno Medeiros wrote:

 On 31/10/2014 12:16, Rikki Cattermole wrote:

 On 1/11/2014 12:35 a.m., Bruno Medeiros wrote:

 On 27/10/2014 22:35, Jeremy Powers via Digitalmars-d wrote:

 An interesting path to take for an intellij plugin would be to use his
 DCD/libdparse for all the heavy lifting, with the plugin just farming
 out the work to external daemon process(es).  That would avoid having
 yet another lexer/parser implementation to keep up to date.


 Integrating DCD might work well, because autocompletion is used
 sporadically (and is invoked explicitly by the user). But parsing is
 probably invoked quite often by IntelliJ, and as such it could be tricky
 to integrate with libdparse (because you'd have to invoke an external
 process all the time)


  From my small test file with my attempt at this, its fine to call e.g.
 DScanner to grab the ast. Its actually pretty fast which I was a little
 shocked about.


 How fast is "pretty fast"? Milliseconds please.


I didn't benchmark it. So no idea.
Nov 07 2014
prev sibling next sibling parent reply "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote:

 There have been similar questions asked on this forum before 
 (most recent one in June with no result: 
 http://forum.dlang.org/thread/CAFMGiz8Fyv2A+ScQPQtEYEsFSybZNfY--nsxL5rqNoWWNA4agg mai
.gmail.com?page=1), 
 but I was curious if anyone happens to have a BNF grammar for D 
 laying around. I've searched all over and can't seem to find 
 anything.

 I'm trying to make a plugin for IntelliJ IDEA to support D 
 (https://github.com/landaire/intelliD is what I have now 
 utilizing DDT's lexer) but a lot of what's required to add 
 advanced features like code folding, code completion, etc. is 
 generally autogenerated by JetBrains's Grammar Kit plugin which 
 only generates that stuff from a BNF grammar.

 I know nothing about writing language grammars but based off 
 what I've seen it doesn't look like it'd be *too* difficult to 
 pick up, so if anyone also has suggestions for good readings 
 I'd appreciate that as well.


I've also been playing with getting a Intellij IDEA plugin up but 
it utilises DScanner. The only issue I'm having right now is for 
some reason its adding two elements to the Psi's tree instead of 
just one. Where the second has correct data but first is the 
correct type.
Not to mention getting dscanner to output the correct line 
number/index ext. Fun times.
Oct 27 2014
next sibling parent reply "landaire" <landergriffith+d gmail.com> writes:
On Monday, 27 October 2014 at 23:17:14 UTC, Rikki Cattermole 
wrote:

 On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote:
 I've also been playing with getting a Intellij IDEA plugin up 
 but it utilises DScanner. The only issue I'm having right now 
 is for some reason its adding two elements to the Psi's tree 
 instead of just one. Where the second has correct data but 
 first is the correct type.
 Not to mention getting dscanner to output the correct line 
 number/index ext. Fun times.


Is your work published on GitHub or elsewhere? I'd like to take a 
look at your approach.
Oct 27 2014
parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Monday, 27 October 2014 at 23:27:20 UTC, landaire wrote:

 On Monday, 27 October 2014 at 23:17:14 UTC, Rikki Cattermole 
 wrote:

 On Monday, 27 October 2014 at 22:06:28 UTC, landaire wrote:
 I've also been playing with getting a Intellij IDEA plugin up 
 but it utilises DScanner. The only issue I'm having right now 
 is for some reason its adding two elements to the Psi's tree 
 instead of just one. Where the second has correct data but 
 first is the correct type.
 Not to mention getting dscanner to output the correct line 
 number/index ext. Fun times.


 Is your work published on GitHub or elsewhere? I'd like to take 
 a look at your approach.


Not yet. But send me an email first lastname.co.nz I'll either 
reply by that one or a gmail account (ssl certs). We can have a 
chat/send it to you about it.
Oct 27 2014
prev sibling parent reply "Brian Schott" <briancschott gmail.com> writes:
On Monday, 27 October 2014 at 23:17:14 UTC, Rikki Cattermole 
wrote:

 Not to mention getting dscanner to output the correct line 
 number/index ext. Fun times.


If the lexer is giving the wrong line numbers for tokens, I want 
to know about it. Do you have a file that reproduces this issue?
Oct 27 2014
parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Monday, 27 October 2014 at 23:50:22 UTC, Brian Schott wrote:

 On Monday, 27 October 2014 at 23:17:14 UTC, Rikki Cattermole 
 wrote:

 Not to mention getting dscanner to output the correct line 
 number/index ext. Fun times.


 If the lexer is giving the wrong line numbers for tokens, I 
 want to know about it. Do you have a file that reproduces this 
 issue?


Not wrong numbers, just not outputting them with --ast (forked 
and working on it as I need it).
Although if it could also lex semi colons and whitespace it would 
be a huge help! (not needed).
Oct 27 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 27/10/2014 22:06, landaire wrote:

 I'm trying to make a plugin for IntelliJ IDEA to support D
 (https://github.com/landaire/intelliD is what I have now utilizing DDT's
 lexer) but a lot of what's required to add advanced features like code
 folding, code completion, etc. is generally autogenerated by JetBrains's
 Grammar Kit plugin which only generates that stuff from a BNF grammar.


Nice to hear someone re-using DDT's DTool code for another tool. (it's 
called DTool BTW, not plugin_tooling - that's just the directory :P )

 From https://github.com/landaire/intelliD:
"Backported to JDK 6 as the module made heavy use of JDK 7 and JDK 6 or 
lower is required for IntelliJ plugin development"

WUT?? That is so backwards... I even have a hard time believing that. 
(I'm no IntelliJ expert so I don't know for sure what is the case). But 
JDK 6 is getting quite old, 7 has been out for some time now, and is 
already superseded by 8 (and 9 is being concocted as we speak - the pace 
of Java changes has picked up now that Oracle took charge)

Are you really *sure* you can't write IntelliJ plugins with Java code 
using JDK 7 and above? IntelliJ is supposed to be Java IDE that is all 
fresh and up-to-date! (As opposed to Eclipse, whose code base has 
stagnated, I readily admit that, ;'( )

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Oct 31 2014
parent reply Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d writes:
V Fri, 31 Oct 2014 11:41:58 +0000
Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> napsáno:


 On 27/10/2014 22:06, landaire wrote:

 I'm trying to make a plugin for IntelliJ IDEA to support D
 (https://github.com/landaire/intelliD is what I have now utilizing
 DDT's lexer) but a lot of what's required to add advanced features
 like code folding, code completion, etc. is generally autogenerated
 by JetBrains's Grammar Kit plugin which only generates that stuff
 from a BNF grammar.


 
 Nice to hear someone re-using DDT's DTool code for another tool.
 (it's called DTool BTW, not plugin_tooling - that's just the
 directory :P )
 
  From https://github.com/landaire/intelliD:
 "Backported to JDK 6 as the module made heavy use of JDK 7 and JDK 6
 or lower is required for IntelliJ plugin development"
 
 WUT?? That is so backwards... I even have a hard time believing that. 
 (I'm no IntelliJ expert so I don't know for sure what is the case).
 But JDK 6 is getting quite old, 7 has been out for some time now, and
 is already superseded by 8 (and 9 is being concocted as we speak -
 the pace of Java changes has picked up now that Oracle took charge)
 
 Are you really *sure* you can't write IntelliJ plugins with Java code 
 using JDK 7 and above? IntelliJ is supposed to be Java IDE that is
 all fresh and up-to-date! (As opposed to Eclipse, whose code base has 
 stagnated, I readily admit that, ;'( )
 


You can write intellij idea plugins with JDK 7 or even JDK 8, but that
plugins will only work with IDEA running on same version of JDK. You
must use same version for IDEA SDK and for IDEA itself. Currently
IDEA and all plugins are compatible with JDK 6, so this is why JDK 6 is
recommended.

At the moment many LTS linux distros has JDK 6 (ubuntu 12.04 LTS,
RedHat/CentOS 6). But ubuntu 14.04 and readhat/centos 7 are here now, so
maybe jdk7 will became recommended version soon.
Oct 31 2014
parent Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 31/10/2014 12:02, Daniel Kozák via Digitalmars-d wrote:

 You can write intellij idea plugins with JDK 7 or even JDK 8, but that
 plugins will only work with IDEA running on same version of JDK. You
 must use same version for IDEA SDK and for IDEA itself. Currently
 IDEA and all plugins are compatible with JDK 6, so this is why JDK 6 is
 recommended.


Ah, but that is totally fine then. It's actually the same scenario with 
Eclipse: The base Eclipse platform (and most official Eclipse plugins  - 
that is, the plugins released by eclipse.org) are made to run in JVM 6. 
But if you write a plugin that requires JVM 7, then you must run Eclipse 
with a JVM 7, naturally. (BTW, that's the case with DDT)

If you guys are writing an Intellij plugin for D, requiring the user to 
run it in a JVM 7 is perfectly reasonable. Even JVM 8 is trivial to 
install. There is not point in making your (the plugin developer's) life 
harder when installing a JVM 7 is trivial, in any OS.

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Nov 07 2014