digitalmars.D - [core.reflect] looking at module dependencies
- Stefan Koch (66/66) Oct 01 2021 Good day everyone,
- Stefan Koch (5/72) Oct 01 2021 Note. Those are only the _direct_ dependencies as I don't crawl
- Basile B. (7/9) Oct 02 2021 To which extent self-linting would be work with core.reflect ?
- Stefan Koch (5/14) Oct 02 2021 If you can inject code as a trigger it would work just fine.
- Stefan Koch (17/22) Oct 02 2021 You don't even need to have an explicit list of symbols.
Good day everyone, I know multiple programs have been written in order to make sense of phobos dependencies. I am going to show how you can get the set of modules a module is 'statically' dependent upon. (this means the set won't include non-instantiated templates. or failing `static if` statements) ```D import core.reflect.reflect; import core.reflect.transitiveVisitor; import std_traits = std.traits; import std_stdio = std.stdio; import core.stdc.stdio; static immutable node = nodeFromName("std_traits", ReflectFlags.Everything); static immutable node2 = nodeFromName("std_stdio", ReflectFlags.Everything); import std; // for basename join and writeln void main() { auto std_traits_deps = referencedFiles(node); auto std_stdio_deps = referencedFiles(node2); writeln("std.traits deps: ", std_traits_deps.map!(e => baseName(e)).join("\n")); writeln("std.stdio deps: ", std_stdio_deps.map!(e => baseName(e)).join(", ")); } string[] referencedFiles(const Node n) { class FileLocationVisitor : TransitiveVisitor { alias visit = TransitiveVisitor.visit; override void visit(Location loc) { if (loc.filename !in fileSet) { fileSet[loc.filename] = 1; } } int[string] fileSet; } scope fileLocationCollector = new FileLocationVisitor(); (cast()n).accept(fileLocationCollector); return fileLocationCollector.fileSet.keys; } ``` and the result of these few lines is the following output. ``` std.traits deps: traits.d std.stdio deps: , time.d, string.d, memory.d-mixin-32, errno.d, checkedint.d, uio.d, time.d, stdio.d, capacity.d, functional.d, array.d, atomic.d, comparison.d, utf.d, memory.d, inet.d, string.d, atomic.d-mixin-324, lifetime.d, time.d, unistd.d, atomic.d-mixin-271, destruction.d, equality.d, mutation.d, traits.d, conv.d, atomic.d, comparison.d, demangle.d, cstring.d, functional.d-mixin-446, bitop.d, in_.d, sysv_x64.d, hash.d, stdlib.d, fcntl.d, memory.d, lifetime.d, exception.d, exception.d, stdio.d, wchar_.d, netdb.d, ascii.d, stdio.d, socket.d, typecons.d, object.d, convert.d, stdio.d-mixin-5219, traits.d-mixin-127, signal.d ``` Which could surely be prettified. with the appropriate Phobos range functions. So now you know :) why hello world takes ages. importing std.stdio pulls in the world :) Please let me know if this needs more explanation and if you found the usage of core.reflect intuitive.
Oct 01 2021
On Friday, 1 October 2021 at 14:44:24 UTC, Stefan Koch wrote:Good day everyone, I know multiple programs have been written in order to make sense of phobos dependencies. I am going to show how you can get the set of modules a module is 'statically' dependent upon. (this means the set won't include non-instantiated templates. or failing `static if` statements) ```D import core.reflect.reflect; import core.reflect.transitiveVisitor; import std_traits = std.traits; import std_stdio = std.stdio; import core.stdc.stdio; static immutable node = nodeFromName("std_traits", ReflectFlags.Everything); static immutable node2 = nodeFromName("std_stdio", ReflectFlags.Everything); import std; // for basename join and writeln void main() { auto std_traits_deps = referencedFiles(node); auto std_stdio_deps = referencedFiles(node2); writeln("std.traits deps: ", std_traits_deps.map!(e => baseName(e)).join("\n")); writeln("std.stdio deps: ", std_stdio_deps.map!(e => baseName(e)).join(", ")); } string[] referencedFiles(const Node n) { class FileLocationVisitor : TransitiveVisitor { alias visit = TransitiveVisitor.visit; override void visit(Location loc) { if (loc.filename !in fileSet) { fileSet[loc.filename] = 1; } } int[string] fileSet; } scope fileLocationCollector = new FileLocationVisitor(); (cast()n).accept(fileLocationCollector); return fileLocationCollector.fileSet.keys; } ``` and the result of these few lines is the following output. ``` std.traits deps: traits.d std.stdio deps: , time.d, string.d, memory.d-mixin-32, errno.d, checkedint.d, uio.d, time.d, stdio.d, capacity.d, functional.d, array.d, atomic.d, comparison.d, utf.d, memory.d, inet.d, string.d, atomic.d-mixin-324, lifetime.d, time.d, unistd.d, atomic.d-mixin-271, destruction.d, equality.d, mutation.d, traits.d, conv.d, atomic.d, comparison.d, demangle.d, cstring.d, functional.d-mixin-446, bitop.d, in_.d, sysv_x64.d, hash.d, stdlib.d, fcntl.d, memory.d, lifetime.d, exception.d, exception.d, stdio.d, wchar_.d, netdb.d, ascii.d, stdio.d, socket.d, typecons.d, object.d, convert.d, stdio.d-mixin-5219, traits.d-mixin-127, signal.d ``` Which could surely be prettified. with the appropriate Phobos range functions. So now you know :) why hello world takes ages. importing std.stdio pulls in the world :) Please let me know if this needs more explanation and if you found the usage of core.reflect intuitive.Note. Those are only the _direct_ dependencies as I don't crawl the import-graph right now. Reflecting over import graphs is currently being implemented. However the trees get so huge that I have trouble debugging it.
Oct 01 2021
On Friday, 1 October 2021 at 14:44:24 UTC, Stefan Koch wrote:Please let me know if this needs more explanation and if you found the usage of core.reflect intuitive.To which extent self-linting would be work with core.reflect ? I'm thinking to something like add `import self_lint;`. That module would implement several visitors, like the D-scanner checks. Then during compilation you can also lint, for example using `enum _ = lintSymbols!(SequenceOfSymbolToLint).` to trigger the checks.
Oct 02 2021
On Saturday, 2 October 2021 at 10:16:56 UTC, Basile B. wrote:On Friday, 1 October 2021 at 14:44:24 UTC, Stefan Koch wrote:If you can inject code as a trigger it would work just fine. However it cannot just be the import. Adding an import must never change the module that is the importer.Please let me know if this needs more explanation and if you found the usage of core.reflect intuitive.To which extent self-linting would be work with core.reflect ? I'm thinking to something like add `import self_lint;`. That module would implement several visitors, like the D-scanner checks. Then during compilation you can also lint, for example using `enum _ = lintSymbols!(SequenceOfSymbolToLint).` to trigger the checks.
Oct 02 2021
On Saturday, 2 October 2021 at 10:45:29 UTC, Stefan Koch wrote:If you can inject code as a trigger it would work just fine. However it cannot just be the import. Adding an import must never change the module that is the importer.You don't even need to have an explicit list of symbols. adding an ```D static immutable complaintString = lintModule.complaintString; static assert(!complaintString, complaintString); ``` would be all that's required, and it could optionally match UDAs which you would do using ```D static immutable complaintString = lintModuleWithUDA(nodeFromName("myLintUDA")).complaintString; static assert(!complaintString, complaintString); ``` note that this use would store a complaintString in the binary but since it'll abort compilation if there are complaints that would never actually bloat anything ;)using `enum _ = lintSymbols!(SequenceOfSymbolToLint).`
Oct 02 2021