digitalmars.D - ModuleInfo, Object.localClasses(), and Object.find() - any users?
- Walter Bright (10/10) Dec 09 2022 Currently, the ModuleInfo struct generated for each module also includes...
- Adam D Ruppe (2/3) Dec 09 2022 This is what we should do!
- H. S. Teoh (15/27) Dec 09 2022 I don't. Didn't even know they existed.
- rikki cattermole (4/9) Dec 09 2022 Its a little too indirect I think.
- H. S. Teoh (9/21) Dec 09 2022 One idea is to turn the array into a template function that returns an
- Steven Schveighoffer (8/24) Dec 09 2022 Please remove. You have to deprecate the functionality first (biggest
- Walter Bright (2/5) Dec 09 2022 Thanks for the link.
- Walter Bright (3/5) Dec 09 2022 https://github.com/dlang/dmd/pull/14681
- JN (5/10) Dec 10 2022 There are some matches for these names when searching on Github.
- Walter Bright (2/4) Dec 10 2022 Thank you, good to know.
- Guillaume Piolat (2/3) Dec 10 2022 Yes, please.
- ryuukk_ (2/15) Dec 11 2022 I vote for 4
- cc (7/20) Dec 12 2022 This would affect Object.factory, yes? I've used that previously
- Steven Schveighoffer (6/12) Dec 12 2022 You need to register those classes with a system, to replace the
- Walter Bright (2/4) Dec 12 2022 Also, ModuleInfo can't know what to *exclude*.
- Steven Schveighoffer (6/11) Dec 12 2022 If Object.factory is to be a supported feature (I don't think it should
- Walter Bright (3/7) Feb 12 2023 It doesn't need to support all classes. The user is going to know which ...
- Walter Bright (6/12) Dec 12 2022 I'm curious as to the reason it was being used.
- cc (23/24) Dec 12 2022 This was an older project ported from Objective-C that read
- Adam D Ruppe (18/21) Dec 12 2022 What I like to do is have classes register themselves. You can do
- Steven Schveighoffer (9/30) Dec 12 2022 The downside there is if you run into cycles. Clearly, this isn't going
- Adam D Ruppe (10/11) Dec 12 2022 Right. I think we'd need to redesign ModuleInfo a little here...
- H. S. Teoh (14/17) Dec 12 2022 [...]
- Walter Bright (2/2) Dec 12 2022 Some good ideas in this thread!
- cc (3/4) Dec 12 2022 In replacement, is something like `__traits(allModules)` not
- rikki cattermole (10/15) Dec 12 2022 No unfortunately.
- cc (3/8) Dec 13 2022 Sounds good to me.
- Dukc (2/5) Dec 13 2022 I don't.
- Dadoum (27/28) Dec 13 2022 I use those in this way:
- psyscout (9/12) Jan 17 2023 I was planning to use "localClasses" to build a factory aware of
- Puneet Goel (11/18) Feb 12 2023 I have trodden the path of mixin templates. I believe the only
- Puneet Goel (3/3) Feb 12 2023 I meant "until a mechanism is developed to break circular
- Richard (Rikki) Andrew Cattermole (5/7) Feb 12 2023 There is a mechanism to do this. Rearchitect your code, you need more
- Walter Bright (3/13) Feb 12 2023 Your wish is my command:
- Adam D Ruppe (2/4) Feb 12 2023 How is that supposed to work?
- Walter Bright (2/9) Feb 12 2023 https://github.com/dlang/dlang.org/pull/3524
- Adam D Ruppe (2/4) Feb 13 2023 So it doesn't work and you don't even understand why.
- Walter Bright (12/14) Feb 15 2023 Some methods given A imports B and B imports A:
- FeepingCreature (6/12) Feb 16 2023 You can always break import cycles by combining A and B into a
- deadalnix (2/3) Feb 12 2023 Yes.
Currently, the ModuleInfo struct generated for each module also includes a pointer to the ClassInfo for every class in that module. The only reference to this data is in the functions Object.localClasses() and Object.find(). The idea is to be able to instantiate a class via a text string rather than a link to the name. This adds a lot of extra size. 1. Does anybody use Object.localClasses() or Object.find() ? 2. Does anybody need them? 3. Could it be changed to only include the classes marked `export` ? 4. Could we just remove it entirely?
Dec 09 2022
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:4. Could we just remove it entirely?This is what we should do!
Dec 09 2022
On Fri, Dec 09, 2022 at 02:20:28PM -0800, Walter Bright via Digitalmars-d wrote:Currently, the ModuleInfo struct generated for each module also includes a pointer to the ClassInfo for every class in that module.Wow. I had no idea such a thing even exists!The only reference to this data is in the functions Object.localClasses() and Object.find(). The idea is to be able to instantiate a class via a text string rather than a link to the name. This adds a lot of extra size. 1. Does anybody use Object.localClasses() or Object.find() ?I don't. Didn't even know they existed.2. Does anybody need them?Not me.3. Could it be changed to only include the classes marked `export` ?Sounds like a reasonably safe approach.4. Could we just remove it entirely?No idea if some D project somewhere out there might use it, so the spectre of code breakage is there... But maybe it's possible to make this pay-as-you-go? I.e., if nobody actually calls Object.localClasses or Object.find, then it won't be included. Only if somebody actually references it, it will be included. Sorta like a template. But I've no idea how feasible it is to implement such a thing. T -- I am not young enough to know everything. -- Oscar Wilde
Dec 09 2022
On 10/12/2022 11:35 AM, H. S. Teoh wrote:But maybe it's possible to make this pay-as-you-go? I.e., if nobody actually calls Object.localClasses or Object.find, then it won't be included. Only if somebody actually references it, it will be included. Sorta like a template. But I've no idea how feasible it is to implement such a thing.Its a little too indirect I think. I just can't see anything pulling in the symbol that have to be generated for the localClasses array and the linker will just strip it out.
Dec 09 2022
On Sat, Dec 10, 2022 at 01:19:11PM +1300, rikki cattermole via Digitalmars-d wrote:On 10/12/2022 11:35 AM, H. S. Teoh wrote:One idea is to turn the array into a template function that returns an array, the instantiation of which somehow triggers the generation of the extra info. But the question is, is this feature even being used at all in the first place... if not, pulling stunts of this sort is just wasted effort. T -- Making non-nullable pointers is just plugging one hole in a cheese grater. -- Walter BrightBut maybe it's possible to make this pay-as-you-go? I.e., if nobody actually calls Object.localClasses or Object.find, then it won't be included. Only if somebody actually references it, it will be included. Sorta like a template. But I've no idea how feasible it is to implement such a thing.Its a little too indirect I think. I just can't see anything pulling in the symbol that have to be generated for the localClasses array and the linker will just strip it out.
Dec 09 2022
On 12/9/22 5:20 PM, Walter Bright wrote:Currently, the ModuleInfo struct generated for each module also includes a pointer to the ClassInfo for every class in that module. The only reference to this data is in the functions Object.localClasses() and Object.find(). The idea is to be able to instantiate a class via a text string rather than a link to the name. This adds a lot of extra size. 1. Does anybody use Object.localClasses() or Object.find() ? 2. Does anybody need them? 3. Could it be changed to only include the classes marked `export` ? 4. Could we just remove it entirely?Please remove. You have to deprecate the functionality first (biggest problem is Object.factory). But it is broken anyway. I have pointed this out for years. ModuleInfo *does* include this information. But it's not guaranteed to generate ModuleInfo just if a class is present. See for instance: https://issues.dlang.org/show_bug.cgi?id=16423 -Steve
Dec 09 2022
On 12/9/2022 7:03 PM, Steven Schveighoffer wrote:I have pointed this out for years. ModuleInfo *does* include this information. But it's not guaranteed to generate ModuleInfo just if a class is present. See for instance: https://issues.dlang.org/show_bug.cgi?id=16423Thanks for the link.
Dec 09 2022
On 12/9/2022 7:03 PM, Steven Schveighoffer wrote:Please remove. You have to deprecate the functionality first (biggest problem is Object.factory). But it is broken anyway.https://github.com/dlang/dmd/pull/14681 One way to find out who's using it.
Dec 09 2022
On Saturday, 10 December 2022 at 04:05:14 UTC, Walter Bright wrote:On 12/9/2022 7:03 PM, Steven Schveighoffer wrote:There are some matches for these names when searching on Github. Most are Phobos forks, but there are some actual projects which seem to use those functionalities.Please remove. You have to deprecate the functionality first (biggest problem is Object.factory). But it is broken anyway.https://github.com/dlang/dmd/pull/14681 One way to find out who's using it.
Dec 10 2022
On 12/10/2022 12:52 PM, JN wrote:There are some matches for these names when searching on Github. Most are Phobos forks, but there are some actual projects which seem to use those functionalities.Thank you, good to know.
Dec 10 2022
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:4. Could we just remove it entirely?Yes, please.
Dec 10 2022
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:Currently, the ModuleInfo struct generated for each module also includes a pointer to the ClassInfo for every class in that module. The only reference to this data is in the functions Object.localClasses() and Object.find(). The idea is to be able to instantiate a class via a text string rather than a link to the name. This adds a lot of extra size. 1. Does anybody use Object.localClasses() or Object.find() ? 2. Does anybody need them? 3. Could it be changed to only include the classes marked `export` ? 4. Could we just remove it entirely?I vote for 4
Dec 11 2022
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:Currently, the ModuleInfo struct generated for each module also includes a pointer to the ClassInfo for every class in that module. The only reference to this data is in the functions Object.localClasses() and Object.find(). The idea is to be able to instantiate a class via a text string rather than a link to the name. This adds a lot of extra size. 1. Does anybody use Object.localClasses() or Object.find() ? 2. Does anybody need them? 3. Could it be changed to only include the classes marked `export` ? 4. Could we just remove it entirely?This would affect Object.factory, yes? I've used that previously in commercial software. Can partially work around it with `std.traits.moduleName` and `__traits(allMembers, somemodule)`, but what other options are there for finding classes outside of the current module other than those explicitly referenced in the calling module?
Dec 12 2022
On 12/12/22 3:40 PM, cc wrote:This would affect Object.factory, yes? I've used that previously in commercial software. Can partially work around it with `std.traits.moduleName` and `__traits(allMembers, somemodule)`, but what other options are there for finding classes outside of the current module other than those explicitly referenced in the calling module?You need to register those classes with a system, to replace the functionality that `ModuleInfo` and `Object.factory` provide. But it would be better to do that anyway, since the compiler isn't completely consistent with which classes it inserts into `ModuleInfo`. -Steve
Dec 12 2022
On 12/12/2022 12:57 PM, Steven Schveighoffer wrote:But it would be better to do that anyway, since the compiler isn't completely consistent with which classes it inserts into `ModuleInfo`.Also, ModuleInfo can't know what to *exclude*.
Dec 12 2022
On 12/12/22 11:34 PM, Walter Bright wrote:On 12/12/2022 12:57 PM, Steven Schveighoffer wrote:If Object.factory is to be a supported feature (I don't think it should be), then it has to include *all* classes. Otherwise, you have odd unrelated conditions that break the feature, and something that brittle really shouldn't be used. -SteveBut it would be better to do that anyway, since the compiler isn't completely consistent with which classes it inserts into `ModuleInfo`.Also, ModuleInfo can't know what to *exclude*.
Dec 12 2022
On 12/12/2022 9:29 PM, Steven Schveighoffer wrote:If Object.factory is to be a supported feature (I don't think it should be), then it has to include *all* classes. Otherwise, you have odd unrelated conditions that break the feature, and something that brittle really shouldn't be used.It doesn't need to support all classes. The user is going to know which modules he wants to find the classes in.
Feb 12 2023
On 12/12/2022 12:40 PM, cc wrote:This would affect Object.factory, yes?Yes.I've used that previously in commercial software.I'm curious as to the reason it was being used.Can partially work around it with `std.traits.moduleName` and `__traits(allMembers, somemodule)`, but what other options are there for finding classes outside of the current module other than those explicitly referenced in the calling module?One option is for the code that is "publishing" a class to be used by Object.factory is to simply make a function in that code that can be called that returns an array of classinfo's for the published classes.
Dec 12 2022
On Monday, 12 December 2022 at 20:58:40 UTC, Walter Bright wrote:I'm curious as to the reason it was being used.This was an older project ported from Objective-C that read external definition data, looking at it now it can (and probably should) be replaced. The allowed class list still needed to be verified so it's the same issue of explicitly registering classes, so disregard that example. In newer stuff I have RPC and serialization/remote object duplication libraries where I explicitly register modules to scan for classes by UDA (or have them indirectly register their own modules when one class calls in to the library). It would be nice to be able to preemptively scan/register classes in arbitrary modules without needing to list them in a second area of code. One (minor) example I have is a toString(OutputRange)() promoter that calls the correct templated function when the variable type doesn't match the typeid. This *doesn't* use ModuleInfo, but it could. If derived classes exist across multiple files, it can miss registering some of them if they call out of order. Otherwise I need to explicitly register each class I want handled here, which is the same "make one change in multiple places" problem that avoiding is one of my favorite strengths of D. tl;dr I can live without it, but now that I'm reminded it exists, it seems like a nice thing to have. 😕
Dec 12 2022
On Monday, 12 December 2022 at 21:49:41 UTC, cc wrote:It would be nice to be able to preemptively scan/register classes in arbitrary modules without needing to list them in a second area of code.What I like to do is have classes register themselves. You can do a `mixin RegisterThis;` thing that you import from the lib. The downside is if you forget to register a child, it won't remind you until you try to use it. But the upside is you can do all kinds of custom work in there, it avoids scanning modules, and can be done without a central list. The RegisterThis thing adds a static constructor that appends the factory to the runtime list. Would be nice if druntime offered some thing like this to replace the existing Object.factory - you can migrate by like `mixin imported!"core.factory".Register;` to make it possible. Then an extra would-be-even-nicer is if there was some way so all objects that inherit from a parent or implement a particular interface either 1) reminded you to register it or 2) just self-registered automatically. (You can kinda do this with the curiously-recurring template pattern but still subclasses can fall through the cracks.)
Dec 12 2022
On 12/12/22 5:01 PM, Adam D Ruppe wrote:On Monday, 12 December 2022 at 21:49:41 UTC, cc wrote:The downside there is if you run into cycles. Clearly, this isn't going to contribute to a cycle, but there's no way to specify that it won't. A way to address cycle problems would be nice here. On one hand, you have to depend on a module that initializes the registry, so you depend on the static ctor sorting to work, but on the other hand, there aren't going to be any cycles for this particular thing. I'm not sure of the right answer, and I've thought about it a lot. -SteveIt would be nice to be able to preemptively scan/register classes in arbitrary modules without needing to list them in a second area of code.What I like to do is have classes register themselves. You can do a `mixin RegisterThis;` thing that you import from the lib. The downside is if you forget to register a child, it won't remind you until you try to use it. But the upside is you can do all kinds of custom work in there, it avoids scanning modules, and can be done without a central list. The RegisterThis thing adds a static constructor that appends the factory to the runtime list. Would be nice if druntime offered some thing like this to replace the existing Object.factory - you can migrate by like `mixin imported!"core.factory".Register;` to make it possible. Then an extra would-be-even-nicer is if there was some way so all objects that inherit from a parent or implement a particular interface either 1) reminded you to register it or 2) just self-registered automatically. (You can kinda do this with the curiously-recurring template pattern but still subclasses can fall through the cracks.)
Dec 12 2022
On Tuesday, 13 December 2022 at 01:39:43 UTC, Steven Schveighoffer wrote:A way to address cycle problems would be nice here.Right. I think we'd need to redesign ModuleInfo a little here... and I actually think the dependency tree can be figured out at compile time. Yes, separate compilation is a thing, but you still have to `import` your dependencies. but idk i gotta get to bed, maybe i'll see about writing more tomorrow. You're right it isn't especially easy, but if we are changing ModuleInfo maybe this sis the opportunity to redefine things to make it at least doable.
Dec 12 2022
On 13/12/22 4:17, Adam D Ruppe wrote:Yes, separate compilation is a thing, but you still have to `import` your dependencies.There's dynamic loading of shared libraries, where you don't even know at compilation time what classes there might be coming. For instance, if you implement a plugin system, and want the classes to add themselves automagically to some central registry. In my view, the best solution for this would be to fix / extend the `this` template parameter [1] to every part of the class, in this case including static constructors: ```d struct MyClassInfo { /* ... */ } MyClassInfo[string] myClassRegistry; class MyRoot { static this(this C) { import std.traits : fullyQualifiedName; // Gather C's metadata and store it myClassRegistry[fullyQualifiedName] = MyClassInfo(/*...*/); } } ``` This way, you could create a hierarchy where everything inheriting from `MyRoot` would register automatically. In an ideal world the current ClassInfo (or even an improved version of it) would be part of Object, but not of ProtoObject (what's the status of DIP1042, btw?). [1]: https://issues.dlang.org/show_bug.cgi?id=10488
Dec 13 2022
On 13/12/22 9:52, Arafel wrote:static this(this C)This should obviously be `static this(this C)()`
Dec 13 2022
On Mon, Dec 12, 2022 at 09:49:41PM +0000, cc via Digitalmars-d wrote: [...]It would be nice to be able to preemptively scan/register classes in arbitrary modules without needing to list them in a second area of code.[...] The way I did it in one project was to tag each struct (I was using structs instead of classes, but the same principle applies) with a UDA, and then import all modules that contain the struct definitions, and use getSymbolsByUDA to retrieve all the corresponding symbols. Using this mechanism, I didn't have to manually maintain a list of structs to include; the code automatically picked up the complete list. Of course, modulo importing all the modules that contain the definitions. Of course, having classes self-register also works too. T -- MASM = Mana Ada Sistem, Man!
Dec 12 2022
Some good ideas in this thread! P.S. ModuleInfo doesn't list structs, only classes.
Dec 12 2022
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:4. Could we just remove it entirely?In replacement, is something like `__traits(allModules)` not possible at that phase of compilation?
Dec 12 2022
On 13/12/2022 6:32 PM, cc wrote:On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:No unfortunately. But there is something we can do, which has been on my todo list for a while now for dub: When building a binary (executable/shared library) produce a module which contains all modules that has or is compiled into the binary. We already kinda have this, but its only for unittesting. Just gotta make it more generic. I intended this to go with injectSourceFiles for registering stuff like web routes.4. Could we just remove it entirely?In replacement, is something like `__traits(allModules)` not possible at that phase of compilation?
Dec 12 2022
On Tuesday, 13 December 2022 at 05:49:49 UTC, rikki cattermole wrote:But there is something we can do, which has been on my todo list for a while now for dub: When building a binary (executable/shared library) produce a module which contains all modules that has or is compiled into the binary.Sounds good to me.
Dec 13 2022
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:This adds a lot of extra size. 1. Does anybody use Object.localClasses() or Object.find() ? 2. Does anybody need them?I don't.
Dec 13 2022
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:2. Does anybody need them?I use those in this way: ```d foreach(mod; ModuleInfo) { foreach(cla; mod.localClasses) { if(cla.base && cla.isBackendBuilder) { BackendBuilder backendBuilder = cast(BackendBuilder) cla.create(); backendBuilder.__ctor(); ushort score = backendBuilder.evaluateEnvironment(); if (score > maxScore) { maxScore = score; bestBuilder = backendBuilder; } else { destroy(backendBuilder); } } } } ``` My use case here is to enumerate all the classes that can initialize a backend that are currently loaded in the runtime. But I only need to iterate through all types. If there was a way to make templates that are automatically instatiated into all child classes, it would probably allow the right classes to automatically register to a list on Runtime initialization, and thus optimizing this code.
Dec 13 2022
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:Currently, the ModuleInfo struct generated for each module also includes a pointer to the ClassInfo for every class in that module.I was planning to use "localClasses" to build a factory aware of classes to instantiate. The approach with mixin template doesn't look that handy, it is easy to forget it. It would be great to have another way of registering classes in factory, but as I can see there is only localClasses is straight forward option as for me. So let me please know, if it is going to deprecated soon. Thank you
Jan 17 2023
On Tuesday, 17 January 2023 at 08:43:50 UTC, psyscout wrote:On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:I have trodden the path of mixin templates. I believe the only way to register classes for a factory mechanism is to involve "static this" constructors. For any non-trivial code it gets me to unresolvable circular dependencies in the modules. I request the developers not to rush with deprecating Object.factory until a mechanism is developed to creak circular dependencies with modules. I see some discussion on the following thread which may be related to what I am trying to say. https://forum.dlang.org/post/mailman.1221.1513724189.9493.digitalmars-d puremagic.comCurrently, the ModuleInfo struct generated for each module also includes a pointer to the ClassInfo for every class in that module.I was planning to use "localClasses" to build a factory aware of classes to instantiate. The approach with mixin template doesn't look that handy, it is easy to forget it.
Feb 12 2023
I meant "until a mechanism is developed to break circular dependencies between the modules" .
Feb 12 2023
On 13/02/2023 3:46 AM, Puneet Goel wrote:I meant "until a mechanism is developed to break circular dependencies between the modules".There is a mechanism to do this. Rearchitect your code, you need more leafs in your codebase. The druntime check is a sort, it exists for a reason, it also fails because of your code. There is nothing to do on the druntime/dmd side.
Feb 12 2023
On 2/12/2023 6:44 AM, Puneet Goel wrote:I have trodden the path of mixin templates. I believe the only way to register classes for a factory mechanism is to involve "static this" constructors. For any non-trivial code it gets me to unresolvable circular dependencies in the modules. I request the developers not to rush with deprecating Object.factory until a mechanism is developed to creak circular dependencies with modules. I see some discussion on the following thread which may be related to what I am trying to say. https://forum.dlang.org/post/mailman.1221.1513724189.9493.digitalmars-d puremagic.comYour wish is my command: https://github.com/dlang/dmd/pull/14699
Feb 12 2023
On Monday, 13 February 2023 at 00:36:30 UTC, Walter Bright wrote:Your wish is my command: https://github.com/dlang/dmd/pull/14699How is that supposed to work?
Feb 12 2023
On 2/12/2023 5:29 PM, Adam D Ruppe wrote:On Monday, 13 February 2023 at 00:36:30 UTC, Walter Bright wrote:https://github.com/dlang/dlang.org/pull/3524Your wish is my command: https://github.com/dlang/dmd/pull/14699How is that supposed to work?
Feb 12 2023
On Monday, 13 February 2023 at 04:04:48 UTC, Walter Bright wrote:So it doesn't work and you don't even understand why.How is that supposed to work?https://github.com/dlang/dlang.org/pull/3524
Feb 13 2023
On 2/12/2023 6:44 AM, Puneet Goel wrote:I request the developers not to rush with deprecating Object.factory until a mechanism is developed to creak circular dependencies with modules.Some methods given A imports B and B imports A: 1. Break up A and B so that the common part goes in C. The static constructor goes in C. I don't have the math to prove it, but I'm pretty sure that any A<=>B can be replaced with A<=C and B<=C. In fact, I think Go requires it (does not allow cyclical imports). 2. A bit kludgier way (but no refactoring needed) is for the C static constructor to call extern(C) void Actor(); and extern(C) void Bctor(); and have A provide a definition for Actor and B provide a definition for Bctor. 3. declare the constructors with pragma(crt_constructor). These are not order dependent, but one must take into account that they'll be run by the C startup code before druntime is initialized, meaning no GC will be available.
Feb 15 2023
On Thursday, 16 February 2023 at 04:11:03 UTC, Walter Bright wrote:Some methods given A imports B and B imports A: 1. Break up A and B so that the common part goes in C. The static constructor goes in C. I don't have the math to prove it, but I'm pretty sure that any A<=>B can be replaced with A<=C and B<=C. In fact, I think Go requires it (does not allow cyclical imports).You can always break import cycles by combining A and B into a new module C. Then, as an extra step, sometimes some parts of C can be split out into separate modules A and B again. :)
Feb 16 2023
On Friday, 9 December 2022 at 22:20:28 UTC, Walter Bright wrote:4. Could we just remove it entirely?Yes.
Feb 12 2023