www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Advanced code coverage analysis

reply thinwybk <thinwybk mailbox.org> writes:
Hi everyone,

as far as I know there is a statement coverage analyzer built 
into DMD https://dlang.org/code_coverage.html . Are there code 
coverage analyzers for D which support branch coverage and other 
more advanced coverage metrics 
https://en.wikipedia.org/wiki/Code_coverage ?

Cheers,
thinwybk
Dec 04 2017
next sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Monday, 4 December 2017 at 20:33:56 UTC, thinwybk wrote:

 Hi everyone,

 as far as I know there is a statement coverage analyzer built 
 into DMD https://dlang.org/code_coverage.html . Are there code 
 coverage analyzers for D which support branch coverage and 
 other more advanced coverage metrics 
 https://en.wikipedia.org/wiki/Code_coverage ?

 Cheers,
 thinwybk


No, the compiler reports only statement coverage. Higher levels 
reports could be made by a tool that would use the LST file and 
the AST produced by libdparse.
Dec 05 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/5/2017 2:55 AM, Basile B. wrote:

 On Monday, 4 December 2017 at 20:33:56 UTC, thinwybk wrote:

 Hi everyone,

 as far as I know there is a statement coverage analyzer built into DMD 
 https://dlang.org/code_coverage.html . Are there code coverage analyzers for D 
 which support branch coverage and other more advanced coverage metrics 
 https://en.wikipedia.org/wiki/Code_coverage ?

 Cheers,
 thinwybk


 
 No, the compiler reports only statement coverage. Higher levels reports could
be 
 made by a tool that would use the LST file and the AST produced by libdparse.


The wiki lists function coverage, statement coverage, branch coverage, and 
condition coverage. The builtin coverage analyzer does all of them.

However, if you put multiple statements, conditions, etc., on one line, you'll 
get the sum of the usage counts.
Dec 05 2017
parent reply Basile B. <b2.temp gmx.com> writes:
On Wednesday, 6 December 2017 at 03:15:38 UTC, Walter Bright 
wrote:

 On 12/5/2017 2:55 AM, Basile B. wrote:

 On Monday, 4 December 2017 at 20:33:56 UTC, thinwybk wrote:

 Hi everyone,

 as far as I know there is a statement coverage analyzer built 
 into DMD https://dlang.org/code_coverage.html . Are there 
 code coverage analyzers for D which support branch coverage 
 and other more advanced coverage metrics 
 https://en.wikipedia.org/wiki/Code_coverage ?

 Cheers,
 thinwybk


 
 No, the compiler reports only statement coverage. Higher 
 levels reports could be made by a tool that would use the LST 
 file and the AST produced by libdparse.


 The wiki lists function coverage, statement coverage, branch 
 coverage, and condition coverage. The builtin coverage analyzer 
 does all of them.

 However, if you put multiple statements, conditions, etc., on 
 one line, you'll get the sum of the usage counts.


Indeed, but there's still some room for a tool. I'd add this to 
my IDE since for now coverage is not indicated in the code editor 
but in the messages (a message for each line that's not covered).

Anyway it's a two days project (at most), i've proposed it to 
D-Scanner:

https://github.com/dlang-community/D-Scanner/issues/540
Dec 05 2017
parent thinwybk <thinwybk mailbox.org> writes:
On Wednesday, 6 December 2017 at 06:05:32 UTC, Basile B. wrote:

 Indeed, but there's still some room for a tool. I'd add this to 
 my IDE since for now coverage is not indicated in the code 
 editor but in the messages (a message for each line that's not 
 covered).

 Anyway it's a two days project (at most), i've proposed it to 
 D-Scanner:

 https://github.com/dlang-community/D-Scanner/issues/540


Having tooling support for advanced coverage analysis would make 
D more interesting to people in the industrial domain. Some other 
aspects would need attention as well (garbage collection). 
However I am looking forward to the DConf 2018 and I am curious 
about the future development of D and its ecosystem...
Dec 07 2017
prev sibling parent reply Johan Engelen <j j.nl> writes:
On Monday, 4 December 2017 at 20:33:56 UTC, thinwybk wrote:

 Hi everyone,

 as far as I know there is a statement coverage analyzer built 
 into DMD https://dlang.org/code_coverage.html . Are there code 
 coverage analyzers for D which support branch coverage and 
 other more advanced coverage metrics 
 https://en.wikipedia.org/wiki/Code_coverage ?


LDC's PGO output can be used for that. See [1]. Except that LDC 
does not yet output full source mappings for the counters 
(`-fcoverage-mapping`).
Because the DMD frontend rewrites code besides semantic analysis, 
it is problematic to create correct source mappings, and I have 
thusfar refrained from implementing that.
(For folks interested in contributing to LDC: it'd be a nice 
project to work on, with nice incremental steps. It's quite 
possible that there is already enough info in the IR for IR-based 
(instead of current PGO instrumentation) instrumentation/mapping 
to work well)

-Johan

[1] https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
Dec 05 2017
parent thinwybk <thinwybk mailbox.org> writes:
On Tuesday, 5 December 2017 at 12:39:53 UTC, Johan Engelen wrote:

 LDC's PGO output can be used for that. See [1]. Except that LDC 
 does not yet output full source mappings for the counters 
 (`-fcoverage-mapping`).
 Because the DMD frontend rewrites code besides semantic 
 analysis, it is problematic to create correct source mappings, 
 and I have thusfar refrained from implementing that.
 (For folks interested in contributing to LDC: it'd be a nice 
 project to work on, with nice incremental steps. It's quite 
 possible that there is already enough info in the IR for 
 IR-based (instead of current PGO instrumentation) 
 instrumentation/mapping to work well)


Advanced code coverage would have been a nice to have. Especially 
in the technical and/or embedded domain where DMD is in most 
cases no option anyway (LDC or GDC are the two choices here).

Unfortunately other projects have higher priority for me right 
now.
Dec 07 2017