digitalmars.D - GSOC 2011
- Luca Boasso (18/18) Mar 22 2011 Hello,
- Bernard Helyer (8/18) Mar 22 2011 Thank you for your interest!
- Bruno Medeiros (36/54) Mar 23 2011 The student is free to present their own ideas, or to flesh out in more
- Jesse Phillips (2/5) Mar 23 2011 If a student is really interested in learning more about language parsin...
- spir (15/18) Mar 23 2011 It is instead more specific: "The aim is to boost the production of fund...
- Bruno Medeiros (15/32) Mar 23 2011 Yeah, I agree that D tools are (generally speaking) much more useful
- Luca Boasso (21/57) Mar 23 2011 Sorry for the late reply,
- Andrei Alexandrescu (16/24) Mar 23 2011 There are two aspects: writing an ANTLR grammar that parses D (which you...
- Jacob Carlborg (12/40) Mar 24 2011 Isn't it obvious, to be able to create tools that needs to parse/lex D
- spir (8/51) Mar 24 2011 Along with your motivations, Jacob, see also:
- spir (7/28) Mar 24 2011 http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD
- Bruno Medeiros (24/52) Mar 24 2011 There is a clear business motivation:
- Luca Boasso (16/73) Mar 24 2011 Hello,
- spir (8/40) Mar 23 2011 That's not the point i meant, neither: a parser for D (& in d) is, imo, ...
Hello, I am Luca and I am finishing my master in computer engineering. I am currently an intern at Panasonic in Cupertino. My great passion has been always programming language design and implementation and I am studying the D programming language. I like to play with different languages and in my master thesis I created a small DSL. I am familiar with parsing theory and I wrote simple parsers by hand and with the help of ANTLR. I am really interested in the following ideas (sorted by interest): - http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#LexingandParsing - http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#ANTLRandJavabasedDparserforIDEusage - http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD I think that I can give my contribution and I would be more than happy to work with you. This would be for me a great opportunity to contribute to an open source project and a way to learn more about D and parsing. Could you give me more information about the help needed? Thank you Luca Boasso
Mar 22 2011
On Tue, 22 Mar 2011 10:47:59 -0700, Luca Boasso wrote:I am really interested in the following ideas (sorted by interest): - http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#LexingandParsing - http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#ANTLRandJavabasedDparserforIDEusage- http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD I think that I can give my contribution and I would be more than happy to work with you. This would be for me a great opportunity to contribute to an open source project and a way to learn more about D and parsing. Could you give me more information about the help needed?Thank you for your interest! Ilya Pupatenko has posted to this NG about their interest to work on the Lexing and Parsing, so you may want to contact them and discuss collaboration. I think if you are familiar with ANTLR, an ANTLR grammar and a Java parser would be very useful. I'm just a Joe Schmo, so that's just one man's opinion -- not an official recommendation, mind.
Mar 22 2011
On 22/03/2011 17:47, Luca Boasso wrote:Hello, I am Luca and I am finishing my master in computer engineering. I am currently an intern at Panasonic in Cupertino. My great passion has been always programming language design and implementation and I am studying the D programming language. I like to play with different languages and in my master thesis I created a small DSL. I am familiar with parsing theory and I wrote simple parsers by hand and with the help of ANTLR. I am really interested in the following ideas (sorted by interest): - http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#LexingandParsing - http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#ANTLRandJavabasedDparserforIDEusage - http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD I think that I can give my contribution and I would be more than happy to work with you. This would be for me a great opportunity to contribute to an open source project and a way to learn more about D and parsing. Could you give me more information about the help needed? Thank you Luca BoassoThe student is free to present their own ideas, or to flesh out in more detail a proposal for existing ideas. The first two items (LexingandParsing, ANTLRandJavabasedDparserforIDEusage) are fairly concrete ideas. The last one, D Tools in D is far more general, and can involve a lot of different things. But there are two things that I would like to say regarding that one. First, I think that for GSoC we should focus only on proposals that, if completed successfully, would be usable and useful on their own, or add something of value to another project/tool/library that would be useful. In other words, I don't find it desirable to work on a project that would need more work done on it afterwards to only then make a usable tool or library (for example a D-based D parser). The second, partially related to the above, is that I find that writing certain kinds of D tools in D is a huge waste of time. The prime example is IDE stuff. It is crazy to be reinventing the wheel here, to not use the frameworks and functionality that existing IDE platforms (Eclipse, Netbeans, Visual Studio, etc.) offer. (More like rebuilding a castle than reinventing the wheel) In this IDE case, building a tool in D is only worthwhile if you are able to integrate it with the IDE platform. For some kinds of tools, like build tools, debuggers, this is fairly easy. But for parsing and semantic functionality (for example, code complete), it is much harder. But it is not unheard of. For example the Goclipse plugin (Eclipse-based Go IDE) uses Gocode - an external daemon process - for its core semantic functionality. (I haven't tried it yet to see how well it works). But it's all very like the ideas presented here: http://vimeo.com/16069687 by Steve Yegge regarding new toolchains. My overall point is that people should think of delivering something of value to D developers, and not just an incomplete project. And also make it as sustainable as possible (easier to keep up to date with changes in D or other tools). Of course, all of this is my personal opinion, and doesn't necessarily reflect the opinion of the rest of the community, or the D GSoC umbrella organization. -- Bruno Medeiros - Software Engineer
Mar 23 2011
Bruno Medeiros Wrote:Of course, all of this is my personal opinion, and doesn't necessarily reflect the opinion of the rest of the community, or the D GSoC umbrella organization.If a student is really interested in learning more about language parsing and apply skills they have already learned, D provides a great opportunity. While it would be great to have a complete solution, D does need the basics, though I wouldn't know what would make a good summer long task.
Mar 23 2011
On 03/23/2011 02:40 PM, Bruno Medeiros wrote:The first two items (LexingandParsing, ANTLRandJavabasedDparserforIDEusage) are fairly concrete ideas. The last one, D Tools in D is far more general, and can involve a lot of different thingsIt is instead more specific: "The aim is to boost the production of fundamental tools for analysis of D source code in D: lexical, syntactic, at best first steps of semantic analysis; producing code representations as D data structures." Its usefulness /in the long term/ can be said more general in that it would enable *further* creation of D tools in D (*), potentially useful to all D programmers. Unlike eg a general-purpose parsing lib in D which is specific to the parsing domain; on the other hand, a general-purpose parsing lib does not parse only D code ;-). (*) See (again): http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD Denis -- _________________ vita es estrany spir.wikidot.com
Mar 23 2011
On 23/03/2011 14:59, spir wrote:On 03/23/2011 02:40 PM, Bruno Medeiros wrote:Yeah, I agree that D tools are (generally speaking) much more useful than a D parsing library. But that's not the point, the point is that with the limited manpower we have available (not just in terms of GSoC, but generally), it's very risky and unwise to embark on projects that then remain incomplete and never see the light of day as part of something useful. It's like comparing a bicycle to a car engine. Yeah, the car engine can allow to build a nice car but it's worthless on it's own and at least the bicycle can get somewhere faster. Even a skate or roller-blades are more useful. How much development has it even been in any D tools in D lately (in the last year or so)? -- Bruno Medeiros - Software EngineerThe first two items (LexingandParsing, ANTLRandJavabasedDparserforIDEusage) are fairly concrete ideas. The last one, D Tools in D is far more general, and can involve a lot of different thingsIt is instead more specific: "The aim is to boost the production of fundamental tools for analysis of D source code in D: lexical, syntactic, at best first steps of semantic analysis; producing code representations as D data structures." Its usefulness /in the long term/ can be said more general in that it would enable *further* creation of D tools in D (*), potentially useful to all D programmers. Unlike eg a general-purpose parsing lib in D which is specific to the parsing domain; on the other hand, a general-purpose parsing lib does not parse only D code ;-). (*) See (again): http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD Denis
Mar 23 2011
Sorry for the late reply, even tough I'm not an ANTLR expert, given my previous experience with the tool and having read most of the official book, I could help more on this GSOC idea. I have looked at http://www.antlr.org/grammar/list and on google and I could not find any recent ANTLR grammar for D. In my opinion this project could be a good contribution for the D community while being practical and useful in the short term. The current ANTLR v3 is much more powerful than the previous version since it can handle LL(*) grammars instead of the classic LL(k) one [More on LL(*) at: http://www.antlr.org/papers/LL-star-PLDI11.pdf]. I had a quick look at the parse.c of dmd compiler. It's a recursive descent parser but honestly I don't know the grammar enough to understand if it is LL(k) but definitively it could be parsed efficiently with the LL(*) parsing strategy. I think that the essential aspects listed in the idea "ANTLR and Java based D parser for IDE usage" could be pursued in the given 3 months time-line. Could somebody tell me something more about the AST that should be generated as output of the parser? Luca Boasso On 3/23/11, Bruno Medeiros <brunodomedeiros+spam com.gmail> wrote:On 23/03/2011 14:59, spir wrote:On 03/23/2011 02:40 PM, Bruno Medeiros wrote:Yeah, I agree that D tools are (generally speaking) much more useful than a D parsing library. But that's not the point, the point is that with the limited manpower we have available (not just in terms of GSoC, but generally), it's very risky and unwise to embark on projects that then remain incomplete and never see the light of day as part of something useful. It's like comparing a bicycle to a car engine. Yeah, the car engine can allow to build a nice car but it's worthless on it's own and at least the bicycle can get somewhere faster. Even a skate or roller-blades are more useful. How much development has it even been in any D tools in D lately (in the last year or so)? -- Bruno Medeiros - Software EngineerThe first two items (LexingandParsing, ANTLRandJavabasedDparserforIDEusage) are fairly concrete ideas. The last one, D Tools in D is far more general, and can involve a lot of different thingsIt is instead more specific: "The aim is to boost the production of fundamental tools for analysis of D source code in D: lexical, syntactic, at best first steps of semantic analysis; producing code representations as D data structures." Its usefulness /in the long term/ can be said more general in that it would enable *further* creation of D tools in D (*), potentially useful to all D programmers. Unlike eg a general-purpose parsing lib in D which is specific to the parsing domain; on the other hand, a general-purpose parsing lib does not parse only D code ;-). (*) See (again): http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD Denis
Mar 23 2011
On 3/23/11 11:42 AM, Luca Boasso wrote:Sorry for the late reply, even tough I'm not an ANTLR expert, given my previous experience with the tool and having read most of the official book, I could help more on this GSOC idea. I have looked at http://www.antlr.org/grammar/list and on google and I could not find any recent ANTLR grammar for D. In my opinion this project could be a good contribution for the D community while being practical and useful in the short term.There are two aspects: writing an ANTLR grammar that parses D (which you are referring to) and also adding to ANTLR the capability of generating D. The latter would allow D users to create and use parsers for various languages. It would also require close cooperation with the ANTLR people for acceptance etc. (Sticking with the former project for now.) This is a technically interesting project. I'd need to hear more about the high-level motivation. What is the "business" motivation? There is already a parser for D written in C++ (the open-source front-end of dmd), so the project would need to build a good case for the added value. Also there are other D parser projects that people work on (ddmd comes to mind), so I'd need to hear a lot more in the way of justifying the utility of such a project. Thanks, Andrei
Mar 23 2011
On 2011-03-24 05:30, Andrei Alexandrescu wrote:On 3/23/11 11:42 AM, Luca Boasso wrote:Isn't it obvious, to be able to create tools that needs to parse/lex D code. If parser/lexer is included in Phobos that is basically a port of the C++ one in DMD and DMD starts to use the D version it will be a huge win. The parser/lexer in Phobos will always automatically be up to date with the one the compiler uses because they are actually the same. As far as I know DDMD cannot be easily used to build other tools that needs to parse/lex D code in it's current state. That is a long term goal but not even the short term goal, of being up to date with DMD and produce the same code as DMD, has been reached yet. -- /Jacob CarlborgSorry for the late reply, even tough I'm not an ANTLR expert, given my previous experience with the tool and having read most of the official book, I could help more on this GSOC idea. I have looked at http://www.antlr.org/grammar/list and on google and I could not find any recent ANTLR grammar for D. In my opinion this project could be a good contribution for the D community while being practical and useful in the short term.There are two aspects: writing an ANTLR grammar that parses D (which you are referring to) and also adding to ANTLR the capability of generating D. The latter would allow D users to create and use parsers for various languages. It would also require close cooperation with the ANTLR people for acceptance etc. (Sticking with the former project for now.) This is a technically interesting project. I'd need to hear more about the high-level motivation. What is the "business" motivation? There is already a parser for D written in C++ (the open-source front-end of dmd), so the project would need to build a good case for the added value. Also there are other D parser projects that people work on (ddmd comes to mind), so I'd need to hear a lot more in the way of justifying the utility of such a project. Thanks, Andrei
Mar 24 2011
On 03/24/2011 10:48 AM, Jacob Carlborg wrote:On 2011-03-24 05:30, Andrei Alexandrescu wrote:Along with your motivations, Jacob, see also: http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD Denis -- _________________ vita es estrany spir.wikidot.comOn 3/23/11 11:42 AM, Luca Boasso wrote:Isn't it obvious, to be able to create tools that needs to parse/lex D code. If parser/lexer is included in Phobos that is basically a port of the C++ one in DMD and DMD starts to use the D version it will be a huge win. The parser/lexer in Phobos will always automatically be up to date with the one the compiler uses because they are actually the same. As far as I know DDMD cannot be easily used to build other tools that needs to parse/lex D code in it's current state. That is a long term goal but not even the short term goal, of being up to date with DMD and produce the same code as DMD, has been reached yet.Sorry for the late reply, even tough I'm not an ANTLR expert, given my previous experience with the tool and having read most of the official book, I could help more on this GSOC idea. I have looked at http://www.antlr.org/grammar/list and on google and I could not find any recent ANTLR grammar for D. In my opinion this project could be a good contribution for the D community while being practical and useful in the short term.There are two aspects: writing an ANTLR grammar that parses D (which you are referring to) and also adding to ANTLR the capability of generating D. The latter would allow D users to create and use parsers for various languages. It would also require close cooperation with the ANTLR people for acceptance etc. (Sticking with the former project for now.) This is a technically interesting project. I'd need to hear more about the high-level motivation. What is the "business" motivation? There is already a parser for D written in C++ (the open-source front-end of dmd), so the project would need to build a good case for the added value. Also there are other D parser projects that people work on (ddmd comes to mind), so I'd need to hear a lot more in the way of justifying the utility of such a project. Thanks, Andrei
Mar 24 2011
On 03/24/2011 05:30 AM, Andrei Alexandrescu wrote:On 3/23/11 11:42 AM, Luca Boasso wrote:http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD Denis -- _________________ vita es estrany spir.wikidot.comSorry for the late reply, even tough I'm not an ANTLR expert, given my previous experience with the tool and having read most of the official book, I could help more on this GSOC idea. I have looked at http://www.antlr.org/grammar/list and on google and I could not find any recent ANTLR grammar for D. In my opinion this project could be a good contribution for the D community while being practical and useful in the short term.There are two aspects: writing an ANTLR grammar that parses D (which you are referring to) and also adding to ANTLR the capability of generating D. The latter would allow D users to create and use parsers for various languages. It would also require close cooperation with the ANTLR people for acceptance etc. (Sticking with the former project for now.) This is a technically interesting project. I'd need to hear more about the high-level motivation. What is the "business" motivation? There is already a parser for D written in C++ (the open-source front-end of dmd), so the project would need to build a good case for the added value. Also there are other D parser projects that people work on (ddmd comes to mind), so I'd need to hear a lot more in the way of justifying the utility of such a project.
Mar 24 2011
On 24/03/2011 04:30, Andrei Alexandrescu wrote:On 3/23/11 11:42 AM, Luca Boasso wrote:There is a clear business motivation: If the ANTLR grammar that parses D is then used to generate a Java-based parser, this can be applied to the DDT IDE project and used instead of the current parser. This issue has been discussed in more detail here: http://www.digitalmars.com/d/archives/digitalmars/D/ide/Future_of_Descent_and_D_Eclipse_IDE_635.html The summary of the situation is: DDT currently uses the same parser as Descent, which is the parser of the Java port of the DMD frontend. Descent is no longer maintained, and neither is the port, so it has fallen behind D latest syntax, and it means certain syntaxes are not recognized properly by either IDE: http://code.google.com/a/eclipselabs.org/p/ddt/issues/detail?id=6 It may also be the case that the parser port has some performance issues (namely with memory usage, a sensitive point with the JVM). This has not been confirmed to be the case with the parser, but according to Ary B. it happens at least with the DMD semantic engine than Descent uses. Whereas DMD is a throwaway process and may not care much about certain memory management, the parser in Eclipse keeps running repeatedly... The alternative solution to this "business" issue is to keep doing things with the DMD port, but I don't think that's the best idea (see that discussion with Ary for more details). But it may as well be on the table for consideration to potential students. -- Bruno Medeiros - Software EngineerSorry for the late reply, even tough I'm not an ANTLR expert, given my previous experience with the tool and having read most of the official book, I could help more on this GSOC idea. I have looked at http://www.antlr.org/grammar/list and on google and I could not find any recent ANTLR grammar for D. In my opinion this project could be a good contribution for the D community while being practical and useful in the short term.There are two aspects: writing an ANTLR grammar that parses D (which you are referring to) and also adding to ANTLR the capability of generating D. The latter would allow D users to create and use parsers for various languages. It would also require close cooperation with the ANTLR people for acceptance etc. (Sticking with the former project for now.) This is a technically interesting project. I'd need to hear more about the high-level motivation. What is the "business" motivation? There is already a parser for D written in C++ (the open-source front-end of dmd), so the project would need to build a good case for the added value. Also there are other D parser projects that people work on (ddmd comes to mind), so I'd need to hear a lot more in the way of justifying the utility of such a project. Thanks, Andrei
Mar 24 2011
Hello, I'm new in the D community so I don't have a fully understanding of the short term needs of the community itself. After reading the posts at [http://www.digitalmars.com/d/archives/digitalmars/D/ide/Future_of_Descent_and_D_Eclipse_IDE_635.html] I agree with Bruno that an ANTLR parser could be useful. With the help of a parser generator it's also possible to experiment with the syntax of the language since potential future changes of D grammar are (almost) handled by a tool instead of a manual modification of the hand written parser. On the other hand the D language v2, as far as I know, is quite stable, so there is no a strong need to experiment. I think that adding to ANTLR the capability of generating D would be a complex task that cannot be accomplished in the time frame of GSOC, considering that I will not be able to work full time. Thanks, Luca Boasso On 3/24/11, Bruno Medeiros <brunodomedeiros+spam com.gmail> wrote:On 24/03/2011 04:30, Andrei Alexandrescu wrote:On 3/23/11 11:42 AM, Luca Boasso wrote:There is a clear business motivation: If the ANTLR grammar that parses D is then used to generate a Java-based parser, this can be applied to the DDT IDE project and used instead of the current parser. This issue has been discussed in more detail here: http://www.digitalmars.com/d/archives/digitalmars/D/ide/Future_of_Descent_and_D_Eclipse_IDE_635.html The summary of the situation is: DDT currently uses the same parser as Descent, which is the parser of the Java port of the DMD frontend. Descent is no longer maintained, and neither is the port, so it has fallen behind D latest syntax, and it means certain syntaxes are not recognized properly by either IDE: http://code.google.com/a/eclipselabs.org/p/ddt/issues/detail?id=6 It may also be the case that the parser port has some performance issues (namely with memory usage, a sensitive point with the JVM). This has not been confirmed to be the case with the parser, but according to Ary B. it happens at least with the DMD semantic engine than Descent uses. Whereas DMD is a throwaway process and may not care much about certain memory management, the parser in Eclipse keeps running repeatedly... The alternative solution to this "business" issue is to keep doing things with the DMD port, but I don't think that's the best idea (see that discussion with Ary for more details). But it may as well be on the table for consideration to potential students. -- Bruno Medeiros - Software EngineerSorry for the late reply, even tough I'm not an ANTLR expert, given my previous experience with the tool and having read most of the official book, I could help more on this GSOC idea. I have looked at http://www.antlr.org/grammar/list and on google and I could not find any recent ANTLR grammar for D. In my opinion this project could be a good contribution for the D community while being practical and useful in the short term.There are two aspects: writing an ANTLR grammar that parses D (which you are referring to) and also adding to ANTLR the capability of generating D. The latter would allow D users to create and use parsers for various languages. It would also require close cooperation with the ANTLR people for acceptance etc. (Sticking with the former project for now.) This is a technically interesting project. I'd need to hear more about the high-level motivation. What is the "business" motivation? There is already a parser for D written in C++ (the open-source front-end of dmd), so the project would need to build a good case for the added value. Also there are other D parser projects that people work on (ddmd comes to mind), so I'd need to hear a lot more in the way of justifying the utility of such a project. Thanks, Andrei
Mar 24 2011
On 03/23/2011 05:44 PM, Bruno Medeiros wrote:On 23/03/2011 14:59, spir wrote:That's not the point i meant, neither: a parser for D (& in d) is, imo, a far smaller project than a general-purpose parsing lib. Why do you think the opposite? Denis -- _________________ vita es estrany spir.wikidot.comOn 03/23/2011 02:40 PM, Bruno Medeiros wrote:Yeah, I agree that D tools are (generally speaking) much more useful than a D parsing library. But that's not the point, the point is that with the limited manpower we have available (not just in terms of GSoC, but generally), it's very risky and unwise to embark on projects that then remain incomplete and never see the light of day as part of something useful. It's like comparing a bicycle to a car engine. Yeah, the car engine can allow to build a nice car but it's worthless on it's own and at least the bicycle can get somewhere faster. Even a skate or roller-blades are more useful. How much development has it even been in any D tools in D lately (in the last year or so)?The first two items (LexingandParsing, ANTLRandJavabasedDparserforIDEusage) are fairly concrete ideas. The last one, D Tools in D is far more general, and can involve a lot of different thingsIt is instead more specific: "The aim is to boost the production of fundamental tools for analysis of D source code in D: lexical, syntactic, at best first steps of semantic analysis; producing code representations as D data structures." Its usefulness /in the long term/ can be said more general in that it would enable *further* creation of D tools in D (*), potentially useful to all D programmers. Unlike eg a general-purpose parsing lib in D which is specific to the parsing domain; on the other hand, a general-purpose parsing lib does not parse only D code ;-). (*) See (again): http://prowiki.org/wiki4d/wiki.cgi?GSOC_2011_Ideas#DtoolsinD Denis
Mar 23 2011