digitalmars.D - Intresa in Static Analysis for D ?
- Stefan Koch (4/4) Nov 15 2016 Hi Guys,
- Stefan Koch (20/24) Nov 15 2016 Whoops I accidentally pressed enter too soon.
- ketmar (1/1) Nov 15 2016 if it will be independent utility -- yes. dmdfe part -- no.
- Stefan Koch (6/7) Nov 15 2016 It would relay on the parser of the dmdfe, and parts of dmd's
- ketmar (6/13) Nov 15 2016 the more features dmdfe will get, the slower (and more
- Dicebot (23/29) Nov 15 2016 protected-headers="v1"
- Stefan Koch (9/12) Nov 15 2016 It seems I did not clearly state this.
- Dicebot (16/26) Nov 15 2016 protected-headers="v1"
- Stefan Koch (6/24) Nov 15 2016 It's a compiler frontend.
- Dicebot (26/35) Nov 16 2016 protected-headers="v1"
- Stefan Koch (6/24) Nov 16 2016 Point taken.
- Basile B. (15/19) Nov 15 2016 cyclic complexity
- Stefan Koch (4/25) Nov 15 2016 I think you mixed up Pow and Xor.
- Basile B. (4/36) Nov 15 2016 pffff you dont get the point.
- Stefan Koch (3/6) Nov 15 2016 I really don't get it.
- Basile B. (9/13) Nov 16 2016 By the way there are several problems with static analysis of D
- Stefan Koch (4/18) Nov 16 2016 Certainly!
Hi Guys, I was wondering how much interest in static analysis exists in this community . DMD already has rudimentary support for these kinds of things.
Nov 15 2016
On Tuesday, 15 November 2016 at 14:45:29 UTC, Stefan Koch wrote:Hi Guys, I was wondering how much interest in static analysis exists in this community . DMD already has rudimentary support for these kinds of things.Whoops I accidentally pressed enter too soon. I am currently working on extending the existing support a little because the new CTFE engine really profits form having more information up-front. At the moment it is rather cumbersome to write analysis passes. And passing information from pass to pass is not exactly trivial either. Therefore I am planning to spent some time next year to implement and polish a facility that allows matching patterns on dmd's AST in an event driven style. However this will only bear fruit if there is actual interest in static analysis for D. One possible application would be to automatically produce a precise complexity analysis for algorithms based on the compilers knowledge of the range interface. Another one is to automatically reduce template-bloat and other duplicated code-blocks early on. (This one is my intended target.) Cheers, Stefan
Nov 15 2016
if it will be independent utility -- yes. dmdfe part -- no.
Nov 15 2016
On Tuesday, 15 November 2016 at 15:28:14 UTC, ketmar wrote:if it will be independent utility -- yes. dmdfe part -- no.It would relay on the parser of the dmdfe, and parts of dmd's semantic analysis. DMD will be much nicer to work with the facility envision. As I see it there is little reason for writing yet another D parser.
Nov 15 2016
ketmar <ketmar ketmar.no-ip.org> On Tuesday, 15 November 2016 at 15:33:30 UTC, Stefan Koch wrote:On Tuesday, 15 November 2016 at 15:28:14 UTC, ketmar wrote:the more features dmdfe will get, the slower (and more memory-hungry) it will become. so i don't want any fancy (and even useful ;-) features in dmd itself (that's what i meant when i wrote "dmdfe"). if it will be separate app based on dmdfe -- sure, it would be great!if it will be independent utility -- yes. dmdfe part -- no.It would relay on the parser of the dmdfe, and parts of dmd's semantic analysis. DMD will be much nicer to work with the facility envision. As I see it there is little reason for writing yet another D parser.
Nov 15 2016
protected-headers="v1" From: Dicebot <public dicebot.lv> Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D Subject: Re: interest in Static Analysis for D ? References: <ehleqkikdocyelpnfnyd forum.dlang.org> <usrfcbxcovbelzllipso forum.dlang.org> In-Reply-To: <usrfcbxcovbelzllipso forum.dlang.org> --oG0XUFoli2dcMEEm0ESSBLCLuAi3l7HeU Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/15/2016 04:57 PM, Stefan Koch wrote:On Tuesday, 15 November 2016 at 14:45:29 UTC, Stefan Koch wrote:I'd put it in a different perspective - we desperately need DMD FE (with all semantic phases) available as fully-stand alone independent library. For example, there is already DScanner (https://github.com/Hackerpilot/Dscanner) for static analysis but amount of things one can check with only lexer/parser is very limited in D. Once we have such library, it opens up world of many useful tools - separating all warnings into dedicated static analysis tool, providing automatic project upgrade scripts which work reliably, visualize mixin expansions and so on. But I'd sincerely advise against any ad-hoc solution that is built into DMD itself. --oG0XUFoli2dcMEEm0ESSBLCLuAi3l7HeU--Hi Guys, I was wondering how much interest in static analysis exists in this community . DMD already has rudimentary support for these kinds of things.
Nov 15 2016
On Tuesday, 15 November 2016 at 16:39:22 UTC, Dicebot wrote:[ .... ] But I'd sincerely advise against any ad-hoc solution that is built into DMD itself.It seems I did not clearly state this. I mean to provide the general plumbing that is needed for the dmd-fe, to utilize it for static analysis. I do not intend to cram all sorts of checking functionality inside the compiler. Though what I am planning will also make it easier to extend the optimizer inside dmd, and improve it's efficiency.
Nov 15 2016
protected-headers="v1" From: Dicebot <public dicebot.lv> Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D Subject: Re: interest in Static Analysis for D ? References: <ehleqkikdocyelpnfnyd forum.dlang.org> <usrfcbxcovbelzllipso forum.dlang.org> <o0fdnq$g3d$1 digitalmars.com> <pyunclenspvtxpvjddfv forum.dlang.org> In-Reply-To: <pyunclenspvtxpvjddfv forum.dlang.org> --WiDlsJjOUXOamnwlQ4L08WRDLTwBlltqd Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/15/2016 06:46 PM, Stefan Koch wrote:On Tuesday, 15 November 2016 at 16:39:22 UTC, Dicebot wrote:When doing so, try to forget that static analysis is the intended goal. It has to be naturally usable for any other purpose too, focusing on one specific application case is likely only harm design decisions. --WiDlsJjOUXOamnwlQ4L08WRDLTwBlltqd--[ .... ] But I'd sincerely advise against any ad-hoc solution that is built into DMD itself.=20 It seems I did not clearly state this. I mean to provide the general plumbing that is needed for the dmd-fe, to utilize it for static analysis. I do not intend to cram all sorts of checking functionality inside the compiler.
Nov 15 2016
On Tuesday, 15 November 2016 at 16:59:49 UTC, Dicebot wrote:On 11/15/2016 06:46 PM, Stefan Koch wrote:It's a compiler frontend. I cannot see any other purposes then code-analysis and code-transformation. Regarding Transformations I wanted to write a good D transformation tool for years now.On Tuesday, 15 November 2016 at 16:39:22 UTC, Dicebot wrote:When doing so, try to forget that static analysis is the intended goal. It has to be naturally usable for any other purpose too, focusing on one specific application case is likely only harm design decisions.[ .... ] But I'd sincerely advise against any ad-hoc solution that is built into DMD itself.It seems I did not clearly state this. I mean to provide the general plumbing that is needed for the dmd-fe, to utilize it for static analysis. I do not intend to cram all sorts of checking functionality inside the compiler.
Nov 15 2016
protected-headers="v1" From: Dicebot <public dicebot.lv> Newsgroups: d,i,g,i,t,a,l,m,a,r,s,.,D Subject: Re: interest in Static Analysis for D ? References: <ehleqkikdocyelpnfnyd forum.dlang.org> <usrfcbxcovbelzllipso forum.dlang.org> <o0fdnq$g3d$1 digitalmars.com> <pyunclenspvtxpvjddfv forum.dlang.org> <o0feu6$ibj$1 digitalmars.com> <ptennjzgkejamtjajqti forum.dlang.org> In-Reply-To: <ptennjzgkejamtjajqti forum.dlang.org> --5TxRSBr36LiGEbq4cUxMQRHSJr9XuPBLo Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 11/15/2016 07:05 PM, Stefan Koch wrote:Very good observation :PWhen doing so, try to forget that static analysis is the intended goal. It has to be naturally usable for any other purpose too, focusing on one specific application case is likely only harm design decisions.=20 It's a compiler frontend.I cannot see any other purposes then code-analysis and code-transformat=ion.Regarding Transformations I wanted to write a good D transformation too=lfor years now.With that approach you can also say that code analysis is subset of code of code transformation (it transforms code to list of warnings!) and thus that is the only purpose. I doubt it is very useful definition :) Language frontend as a library must allow any sort of reasoning about source code that compiler can normally do. What library user will do with that reasoning is not of ones concern - it can be analysis or transpilation to another language or even dumb AST pretty-printing. No matter, in the core one has to be able to do `import dlang.frontend` and write own visiting code. --5TxRSBr36LiGEbq4cUxMQRHSJr9XuPBLo--
Nov 16 2016
On Wednesday, 16 November 2016 at 10:45:54 UTC, Dicebot wrote:On 11/15/2016 07:05 PM, Stefan Koch wrote:Point taken. My plan does not include to change the modular structure of the frontend. It's more focused on improving perforance and ease-of-use of the AST and AST-Visitors.Very good observation :P[...]It's a compiler frontend.I cannot see any other purposes then code-analysis and code-transformation. Regarding Transformations I wanted to write a good D transformation tool for years now.With that approach you can also say that code analysis is subset of code of code transformation (it transforms code to list of warnings!) and thus that is the only purpose. I doubt it is very useful definition :) Language frontend as a library must allow any sort of reasoning about source code that compiler can normally do. What library user will do with that reasoning is not of ones concern - it can be analysis or transpilation to another language or even dumb AST pretty-printing. No matter, in the core one has to be able to do `import dlang.frontend` and write own visiting code.
Nov 16 2016
On Tuesday, 15 November 2016 at 14:45:29 UTC, Stefan Koch wrote:Hi Guys, I was wondering how much interest in static analysis exists in this community . DMD already has rudimentary support for these kinds of things.cyclic complexity NPath complexity Halstead complexity 3 nice fields of static analysis. I'm currently working on Halstead (https://github.com/BBasile/Coedit/blob/master/dastworx/src/halstead.d) The Halstead metric (1977) is often considered as ratio of the number of line code, while it's actually not at all. The Halstead is based on the operations and the operations arguments. You can have small functions using a lot of operands that will be bug prone. You can have huge functions with very few operands used and that are not bug prone. The Halstead metric can detect them. It can tell you: "take care of this function". ;]
Nov 15 2016
On Tuesday, 15 November 2016 at 16:28:30 UTC, Basile B. wrote:On Tuesday, 15 November 2016 at 14:45:29 UTC, Stefan Koch wrote:I think you mixed up Pow and Xor. You will probably want to change from a AA to a normal array and just assign a numeric index to each expression you care about :)Hi Guys, I was wondering how much interest in static analysis exists in this community . DMD already has rudimentary support for these kinds of things.cyclic complexity NPath complexity Halstead complexity 3 nice fields of static analysis. I'm currently working on Halstead (https://github.com/BBasile/Coedit/blob/master/dastworx/src/halstead.d) The Halstead metric (1977) is often considered as ratio of the number of line code, while it's actually not at all. The Halstead is based on the operations and the operations arguments. You can have small functions using a lot of operands that will be bug prone. You can have huge functions with very few operands used and that are not bug prone. The Halstead metric can detect them. It can tell you: "take care of this function". ;]
Nov 15 2016
On Tuesday, 15 November 2016 at 16:38:39 UTC, Stefan Koch wrote:On Tuesday, 15 November 2016 at 16:28:30 UTC, Basile B. wrote:pffff you dont get the point. The point is when an IDE message leads you to this: http://imgur.com/a/6zLHUOn Tuesday, 15 November 2016 at 14:45:29 UTC, Stefan Koch wrote:I think you mixed up Pow and Xor. You will probably want to change from a AA to a normal array and just assign a numeric index to each expression you care about :)Hi Guys, I was wondering how much interest in static analysis exists in this community . DMD already has rudimentary support for these kinds of things.cyclic complexity NPath complexity Halstead complexity 3 nice fields of static analysis. I'm currently working on Halstead (https://github.com/BBasile/Coedit/blob/master/dastworx/src/halstead.d) The Halstead metric (1977) is often considered as ratio of the number of line code, while it's actually not at all. The Halstead is based on the operations and the operations arguments. You can have small functions using a lot of operands that will be bug prone. You can have huge functions with very few operands used and that are not bug prone. The Halstead metric can detect them. It can tell you: "take care of this function". ;]
Nov 15 2016
On Tuesday, 15 November 2016 at 16:50:47 UTC, Basile B. wrote:pffff you dont get the point. The point is when an IDE message leads you to this: http://imgur.com/a/6zLHUI really don't get it. What is actionable about that metric ?
Nov 15 2016
On Tuesday, 15 November 2016 at 16:53:42 UTC, Stefan Koch wrote:On Tuesday, 15 November 2016 at 16:50:47 UTC, Basile B. wrote:The Halstead complexity is able to detect the functions that need more unit test and more reviews.pffff you dont get the point. The point is when an IDE message leads you to this: http://imgur.com/a/6zLHUI really don't get it. What is actionable about that metric ?
Nov 15 2016
On Tuesday, 15 November 2016 at 16:57:44 UTC, Basile B. wrote:On Tuesday, 15 November 2016 at 16:53:42 UTC, Stefan Koch wrote:Like shown here: https://www.youtube.com/watch?v=_asYu0f38ks&nohtml5=False#t=6.543481 ;)On Tuesday, 15 November 2016 at 16:50:47 UTC, Basile B. wrote:The Halstead complexity is able to detect the functions that need more unit test and more reviews.pffff you dont get the point. The point is when an IDE message leads you to this: http://imgur.com/a/6zLHUI really don't get it. What is actionable about that metric ?
Nov 15 2016
On Tuesday, 15 November 2016 at 14:45:29 UTC, Stefan Koch wrote:Hi Guys, I was wondering how much interest in static analysis exists in this community . DMD already has rudimentary support for these kinds of things.By the way there are several problems with static analysis of D code. Mostly related to template metaprogramming and conditional compilation. - UFCS: at least in libdparse, not visited with a FunctionCall - static if - version - mixin() and mixin Does your DDMD-based AST handle these cases ?
Nov 16 2016
On Wednesday, 16 November 2016 at 14:06:16 UTC, Basile B. wrote:On Tuesday, 15 November 2016 at 14:45:29 UTC, Stefan Koch wrote:Certainly! However only on mixin that are actually used. Only the conditions that are taken are visited.Hi Guys, I was wondering how much interest in static analysis exists in this community . DMD already has rudimentary support for these kinds of things.By the way there are several problems with static analysis of D code. Mostly related to template metaprogramming and conditional compilation. - UFCS: at least in libdparse, not visited with a FunctionCall - static if - version - mixin() and mixin Does your DDMD-based AST handle these cases ?
Nov 16 2016