www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Is there a way to use Object.factory with templated classes? Or some

reply Chad Joan <chadjoan gmail.com> writes:
Hi all,

I'm implementing a deep-copy method for a tree of templated class 
instances.  As part of this, I need some way to copy each node.  
I want to avoid code that does things like casting objects into 
byte arrays and then copying raw bytes; I want all operations to 
be memory safe things that I can use at compile-time.  So I 
planned to make all of these have default constructors and use 
Object.factory to at least create the correct instance type at 
the destination.  The classes can implement auxiliary copy 
methods if they need to copy anything that isn't already handled 
by their deepCopy method.

But I ran into a problem: Object.factory doesn't seem to be 
compatible with templated classes.

Here is an example:

import std.stdio;

class Root(T)
{
	T x;
}

class Extended(T) : Root!T
{
	T y;
}

void main()
{
	Root!int foo = new Extended!int();

	auto name = foo.classinfo.name;
	writefln("foo's name is '%s'", name);
	// foo's name is 'main.Extended!int.Extended'

	Object   obj = Object.factory(name);
	writefln("Is obj null? %s", obj is null);

	Root!int bar = cast(Root!int)obj; // Still going to be null.
	writefln("Is bar null? %s", obj is null);

	//bar.x = 3; // crash!
}


I had a look at Object.factory.  It seems very simple.  Perhaps 
too simple.  I think this might be a dead end.  Have I missed 
something?  Can it actually handle templates somehow, but I just 
don't know how to calculate the correct string to hand it?

If Object.factory is incapable of this, is there some other 
CTFE-friendly way to copy templated class instances?

If I have to, I can probably make these things register 
themselves in some list of delegates that can be used to 
instantiate the correct class.  Or something like that.  But I am 
hoping that there is a better way that involves less boilerplate.

Thanks!
- Chad
Sep 26 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 26 September 2018 at 20:41:38 UTC, Chad Joan wrote:
 I'm implementing a deep-copy method for a tree of templated 
 class instances.  As part of this, I need some way to copy each 
 node.
 [...]
 that isn't already handled by their deepCopy method.
I would strongly suggest just using that virtual method and having the child classes override it, then you call it from any of them and get the right result. Object.factory kinda sux and I'd actually like to remove it (among other people). There's no plan to actually do that, but still, just on principle I want to turn people away. But even as you can see, the implementation is lacking and it isn't great design anyway - the interface with virtual methods does better work. It also wouldn't work in ctfe anyway, object.factory relies on runtime stuff.
 If Object.factory is incapable of this, is there some other 
 CTFE-friendly way to copy templated class instances?
I think you can copy typeinfo().init and then call typeinfo().defaultConstructor - this is iirc what Object.factory does, but once you already have the typeinfo you can use it directly and bypass the string lookup. But you'd really be better off with a virtual copy method. I say those string factory things should only be used if you are starting with a string, like deserialization. interface Copyable { Copyable copy(); } class Whatever(T) : Copyable { Whatever!T copy() { auto c = new Whatever!T(); c.tupleof = this.tupleof; return c; } } that kind of method. the template implements the interface so little boilerplate and it works and can be customized etc and fits in well. If you call it from the interface, you get an instance of the interface (tho note since it is virtual, the underlying type is still what you need). If you call from the child static type, you get it right back. Yay, fits liskov and works!
 If I have to, I can probably make these things register 
 themselves in some list of delegates that can be used to 
 instantiate the correct class.  Or something like that.  But I 
 am hoping that there is a better way that involves less 
 boilerplate.
that's not a terrible idea if you need delegates keyed to strings...
Sep 26 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, September 26, 2018 3:24:07 PM MDT Adam D. Ruppe via 
Digitalmars-d-learn wrote:
 Object.factory kinda sux and I'd actually like to remove it
 (among other people). There's no plan to actually do that, but
 still, just on principle I want to turn people away.
While there may not currently be plans to be remove it, as there _are_ plans to add ProtoObject as the new root class underneath Object, at some point here, it's likely that a large percentage of classes won't have anything to do with Object, so relying on Object.factory to be able to construct class Objects in general isn't likely to be a viable path in the long term - though presumably it would work for a code base that's written specifically with it in mind. Personally, I'm hoping that we eventually get to the point where Walter and Andrei are willing to outright deprecate Object itself, but I expect that ProtoObject will have to have been in use for a while before we have any chance of that happening. Either way, I think that it's clear that most code bases should go with a solution other than Object.factory if at all reasonably possible. - Jonathan M Davis
Sep 26 2018
parent reply Chad Joan <chadjoan gmail.com> writes:
On Wednesday, 26 September 2018 at 23:32:36 UTC, Jonathan M Davis 
wrote:
 On Wednesday, September 26, 2018 3:24:07 PM MDT Adam D. Ruppe 
 via Digitalmars-d-learn wrote:
 Object.factory kinda sux and I'd actually like to remove it 
 (among other people). There's no plan to actually do that, but 
 still, just on principle I want to turn people away.
While there may not currently be plans to be remove it, as there _are_ plans to add ProtoObject as the new root class underneath Object, at some point here, it's likely that a large percentage of classes won't have anything to do with Object, so relying on Object.factory to be able to construct class Objects in general isn't likely to be a viable path in the long term - though presumably it would work for a code base that's written specifically with it in mind. Personally, I'm hoping that we eventually get to the point where Walter and Andrei are willing to outright deprecate Object itself, but I expect that ProtoObject will have to have been in use for a while before we have any chance of that happening. Either way, I think that it's clear that most code bases should go with a solution other than Object.factory if at all reasonably possible. - Jonathan M Davis
That's interesting! Thanks for mentioning. If you don't mind, what are the complaints regarding Object? Or can you link me to discussions/issues/documents that point out the shortcomings/pitfalls? I've probably run into a bunch of them, but I realize D has come a long way since that original design and I wouldn't be surprised if there's a lot more for me to learn here.
Sep 26 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, September 26, 2018 10:20:58 PM MDT Chad Joan via Digitalmars-
d-learn wrote:
 On Wednesday, 26 September 2018 at 23:32:36 UTC, Jonathan M Davis

 wrote:
 On Wednesday, September 26, 2018 3:24:07 PM MDT Adam D. Ruppe

 via Digitalmars-d-learn wrote:
 Object.factory kinda sux and I'd actually like to remove it
 (among other people). There's no plan to actually do that, but
 still, just on principle I want to turn people away.
While there may not currently be plans to be remove it, as there _are_ plans to add ProtoObject as the new root class underneath Object, at some point here, it's likely that a large percentage of classes won't have anything to do with Object, so relying on Object.factory to be able to construct class Objects in general isn't likely to be a viable path in the long term - though presumably it would work for a code base that's written specifically with it in mind. Personally, I'm hoping that we eventually get to the point where Walter and Andrei are willing to outright deprecate Object itself, but I expect that ProtoObject will have to have been in use for a while before we have any chance of that happening. Either way, I think that it's clear that most code bases should go with a solution other than Object.factory if at all reasonably possible. - Jonathan M Davis
That's interesting! Thanks for mentioning. If you don't mind, what are the complaints regarding Object? Or can you link me to discussions/issues/documents that point out the shortcomings/pitfalls? I've probably run into a bunch of them, but I realize D has come a long way since that original design and I wouldn't be surprised if there's a lot more for me to learn here.
I can point you to the related DIP, though it's a WIP in progress https://github.com/andralex/DIPs/blob/ProtoObject/DIPs/DIPxxxx.md There are also these enhancement requests for removing the various member functions from Object (though they're likely to be superceded by the DIP): https://issues.dlang.org/show_bug.cgi?id=9769 https://issues.dlang.org/show_bug.cgi?id=9770 https://issues.dlang.org/show_bug.cgi?id=9771 https://issues.dlang.org/show_bug.cgi?id=9772 Basically, the problems tend to come in two areas: 1. Because of how inheritance works, once you have a function on a class, you're forcing a certain set of attributes on that function - be it type qualifiers like const or shared or scope classes like pure or safe. In some cases, derived classes can be more restricted when they override the function (e.g. an overide can be safe when the original is system), but that only goes so far, and when you use the base class API, you're stuck with whatever attributes it has. Regardless, derived classes can't be _less_ restrictive. In fact, the only reason that it's currently possible to use == with const class references in D right now is because of a hack. The free function opEquals that gets called when you use == on two class references actually casts away const so that it can then call the member function opEquals (which doesn't work with const). So, if the member function opEquals mutates the object, you actuall get undefined behavior. And because Object.opEquals defines both the parameter and invisible this parameter as mutable, derived classes have to do the same when they override it; otherwise, they'd be overloading it rather than overriding it. Object and its member functions really come from D1 and predate all of the various attributes in D2 - including const. But even if we could just add all of the attributes that we thought should be there without worrying about breaking existing code, there would be no right answer. For instance, while in the vast majority of cases, opEquals really should be const, having it be const does not work with types that lazily initialize some members (since unlike in C++, D does not have backdoors for const - when something is const, it really means const, and it's undefined behavior to cast away const and mutate the object). So, having Object.opEquals be const might work in 99% of cases, but it wouldn't work in all. The same could be said for other attributes such as pure or nothrow. Forcing a particular set of attributes on these functions on everyone is detrimental. And honestly, it really isn't necessary. Having them on Object comes from a Java-esque design where you don't have templates. With proper templates like D2 has, there normally isn't a reason to operate on an Object. You templatize the code rather than relying on a common base class. So, there's no need to have Object.toString in order have toString for all classes or Object.opEquals to have opEquals for all classes. Each class can define it however it sees fit. Now, once a particular class in a hierarchy has defined a function like opEquals or toString, that affects any classes derived from it, but then only the classes derived from it are restricted by those choices, not every single class in the entire language as has been the case with Object. 2. The other big issue has been that built-in monitor. It allows us to have synchronized classes, but in most cases, it's unnecessary overhead. _Most_ classes don't do anything with synchronized, so why have the monitor? It really should just be in those classes that need it. With Object as the base class for all D class, every class gets it whether it needs it or not. With the ProtoObject DIP, only those classes which specifically ask for it (or which don't bother to specify a base class and thus continue to use Object as their base class) will continue to have a monitor object. A related issue that Andrei likes to bring up occasionally (though I don't think that much of anyone else has complained about) is that synchronized is one of those things that the language can do that we can't duplicate without the languages help. With synchronized, you can have a const or immutable object with a mutex inside it which works perfectly fine, but without synchronized, that's not possible because of the transitivity of const and immutable. synchronized and the monitor object give us a backdoor that we can't emulate, and Andrei doesn't like language features where the language has a superpower that you can't emulate (another, unrelated example that he likes to bring up sometimes would be how when you pass a dynamic array to a templated function, it's instantiated with the tail-const version of the type, which doesn't work with user-defined types and actually would pose some interesting problems to implement for user-defined types). So, in any case, because of D's powerful template system, there's no need to have any member functions on Object. There arguably isn't even any need to have any root class type. But having a root class type with member functions has proven to be a _big_ problem when attributes come into play and a minor one with regards to unnecessary overhead because of synchronized classes. - Jonathan M Davis
Sep 26 2018
parent reply Chad Joan <chadjoan gmail.com> writes:
On Thursday, 27 September 2018 at 05:12:06 UTC, Jonathan M Davis 
wrote:
 On Wednesday, September 26, 2018 10:20:58 PM MDT Chad Joan via 
 Digitalmars- d-learn wrote:
 ...

 That's interesting!  Thanks for mentioning.

 If you don't mind, what are the complaints regarding Object?  
 Or can you link me to discussions/issues/documents that point 
 out the shortcomings/pitfalls?

 I've probably run into a bunch of them, but I realize D has 
 come a long way since that original design and I wouldn't be 
 surprised if there's a lot more for me to learn here.
I can point you to the related DIP, though it's a WIP in progress https://github.com/andralex/DIPs/blob/ProtoObject/DIPs/DIPxxxx.md There are also these enhancement requests for removing the various member functions from Object (though they're likely to be superceded by the DIP): https://issues.dlang.org/show_bug.cgi?id=9769 https://issues.dlang.org/show_bug.cgi?id=9770 https://issues.dlang.org/show_bug.cgi?id=9771 https://issues.dlang.org/show_bug.cgi?id=9772 Basically, the problems tend to come in two areas: 1. Because of how inheritance works, once you have a function on a class, you're forcing a certain set of attributes on that function - be it type qualifiers like const or shared or scope classes like pure or safe. In some cases, derived classes can be more restricted when they override the function (e.g. an overide can be safe when the original is system), but that only goes so far, and when you use the base class API, you're stuck with whatever attributes it has. Regardless, derived classes can't be _less_ restrictive. In fact, the only reason that it's currently possible to use == with const class references in D right now is because of a hack. The free function opEquals that gets called when you use == on two class references actually casts away const so that it can then call the member function opEquals (which doesn't work with const). So, if the member function opEquals mutates the object, you actuall get undefined behavior. And because Object.opEquals defines both the parameter and invisible this parameter as mutable, derived classes have to do the same when they override it; otherwise, they'd be overloading it rather than overriding it.
You're right, I wouldn't be caught dead wearing that. :) But yeah, thanks for pointing that out. Now I know not to mutate things in an opEquals, even if it makes sense from the class's point of view, just in case. At least until this all gets sorted out and code gets updated to not inherit from Object.
 Object and its member functions really come from D1 and predate 
 all of the various attributes in D2 - including const. But even 
 if we could just add all of the attributes that we thought 
 should be there without worrying about breaking existing code, 
 there would be no right answer. For instance, while in the vast 
 majority of cases, opEquals really should be const, having it 
 be const does not work with types that lazily initialize some 
 members (since unlike in C++, D does not have backdoors for 
 const - when something is const, it really means const, and 
 it's undefined behavior to cast away const and mutate the 
 object). So, having Object.opEquals be const might work in 99% 
 of cases, but it wouldn't work in all. The same could be said 
 for other attributes such as pure or nothrow. Forcing a 
 particular set of attributes on these functions on everyone is 
 detrimental. And honestly, it really isn't necessary.

 Having them on Object comes from a Java-esque design where you 
 don't have templates. With proper templates like D2 has, there 
 normally isn't a reason to operate on an Object. You templatize 
 the code rather than relying on a common base class. So, 
 there's no need to have Object.toString in order have toString 
 for all classes or Object.opEquals to have opEquals for all 
 classes. Each class can define it however it sees fit. Now, 
 once a particular class in a hierarchy has defined a function 
 like opEquals or toString, that affects any classes derived 
 from it, but then only the classes derived from it are 
 restricted by those choices, not every single class in the 
 entire language as has been the case with Object.
That makes sense. Also, compile-time inheritance/duck-typing FTW, again. This is also reminding me of how it's always bugged me that there isn't a way to operator overload opEquals with a static method (or even a free function?), given that it would allow the class/struct implementer to guard against (or even interact intelligently with) null values: import std.stdio; class A { int payload; bool opEquals(int rhs) { if ( rhs == int.max ) return false; else return this.payload == rhs; } } class B { int payload; static bool opEquals(B lhs, int rhs) { if ( lhs is null && rhs == int.max ) return true; else { if ( rhs == int.max ) return false; else return lhs.payload == rhs; } } } void main() { A a1 = new A(); assert(a1 != int.max); /+A a2 = null; if ( a2 == int.max ) writeln("Even though it'd be nice to compare these things, "~ "we should crash before this writeln."); +/ B b2 = null; //if ( b2 == int.max ) if ( B.opEquals(b2, int.max) ) writeln("Correct!"); else assert(0); }
 2. The other big issue has been that built-in monitor. It 
 allows us to have synchronized classes, but in most cases, it's 
 unnecessary overhead. _Most_ classes don't do anything with 
 synchronized, so why have the monitor? It really should just be 
 in those classes that need it. With Object as the base class 
 for all D class, every class gets it whether it needs it or 
 not. With the ProtoObject DIP, only those classes which 
 specifically ask for it (or which don't bother to specify a 
 base class and thus continue to use Object as their base class) 
 will continue to have a monitor object.
Makes sense (to fix).
 A related issue that Andrei likes to bring up occasionally 
 (though I don't think that much of anyone else has complained 
 about) is that synchronized is one of those things that the 
 language can do that we can't duplicate without the languages 
 help. With synchronized, you can have a const or immutable 
 object with a mutex inside it which works perfectly fine, but 
 without synchronized, that's not possible because of the 
 transitivity of const and immutable. synchronized and the 
 monitor object give us a backdoor that we can't emulate, and 
 Andrei doesn't like language features where the language has a 
 superpower that you can't emulate (another, unrelated example 
 that he likes to bring up sometimes would be how when you pass 
 a dynamic array to a templated function, it's instantiated with 
 the tail-const version of the type, which doesn't work with 
 user-defined types and actually would pose some interesting 
 problems to implement for user-defined types).
I can sympathize. I really get this sour feeling every time I want to write a really smooth type that behaves like it came with the language and integrates with everything really well, only to realize that there are various unsolvable corner-cases that poke holes in it. D is still much better at this operator overloading thing than any of the other languages I've used, but it'd be so much better if it were just completely *perfect* at it ;) I'm uh, all too used to languages just lopping off all of my arms and legs (just don't implement operator overloading, because that solves the problem!) and then saying that everyone is equal (because everyone is a basket case now). Merely a flesh wound etc etc...
 So, in any case, because of D's powerful template system, 
 there's no need to have any member functions on Object. There 
 arguably isn't even any need to have any root class type. But 
 having a root class type with member functions has proven to be 
 a _big_ problem when attributes come into play and a minor one 
 with regards to unnecessary overhead because of synchronized 
 classes.

 - Jonathan M Davis
Wouldn't it be helpful to have a root class type just to have a "Top" type at runtime, even if it had no members? Ex: so you could do things like make an array ProtoObject[] foo; that can contain any runtime polymorphic variables. Thank you for the enlightening post and thorough explanation. It's been a while since I've been able to do any D programming, so it's nice to come back and see all of the thoughtful considerations behind it and the thoughtfulness generally found in this community. I appreciate it.
Sep 27 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, September 27, 2018 1:41:23 AM MDT Chad Joan via Digitalmars-d-
learn wrote:
 On Thursday, 27 September 2018 at 05:12:06 UTC, Jonathan M Davis
 This is also reminding me of how it's always bugged me that there
 isn't a way to operator overload opEquals with a static method
 (or even a free function?), given that it would allow the
 class/struct implementer to guard against (or even interact
 intelligently with) null values:
That's not really an issue with D. With classes, when you have a == b, it doesn't lower to a.opEquals(b). Rather, it lowers to opEquals(a, b), and the free function opEquals is defined as bool opEquals(Object lhs, Object rhs) { // If aliased to the same object or both null => equal if (lhs is rhs) return true; // If either is null => non-equal if (lhs is null || rhs is null) return false; // If same exact type => one call to method opEquals if (typeid(lhs) is typeid(rhs) || !__ctfe && typeid(lhs).opEquals(typeid(rhs))) /* CTFE doesn't like typeid much. 'is' works, but opEquals doesn't (issue 7147). But CTFE also guarantees that equal TypeInfos are always identical. So, no opEquals needed during CTFE. */ { return lhs.opEquals(rhs); } // General case => symmetric calls to method opEquals return lhs.opEquals(rhs) && rhs.opEquals(lhs); } /************************ * Returns true if lhs and rhs are equal. */ bool opEquals(const Object lhs, const Object rhs) { // A hack for the moment. return opEquals(cast()lhs, cast()rhs); } So, it already takes care of checking for null or even if the two references point to the same object. For structs, a == b, does lower to a.opEquals(b), but for better or worse, structs are designed so that their init values need to be valid, or you're going to have problems in general. Trying to work around that is fighting a losing battle.
 Wouldn't it be helpful to have a root class type just to have a
 "Top" type at runtime, even if it had no members?  Ex: so you
 could do things like make an array ProtoObject[] foo; that can
 contain any runtime polymorphic variables.
Maybe? It's not something that I've personally found to be particularly useful. Once you can templatize code, the need to have a common base class gets pretty hard to argue for, but I don't know that it's non-existent. Also, for better or worse, you can already get it with void* - and cover more types no less (albeit less safely). But from what I understand of what Andrei is intending, ProtoObject will end up being the new root class for all D classos, so having ProtoObject[] would work for all extern(D) classes. Of course, that still potentially leaves exern(C++) classes and interfaces (which could be extern(C++) or COM even right now and aren't derived from Object). So, things are already a bit weird with classes when you start interacting with other languages through D. is(T : Object) and is(T == class) do _not_ mean quite the same thing even though you'd think that they would. And I haven't done enough with extern(C++) or COM in D to claim to understand all of the subtleties. If you're not messing with them directly or writing generic code that's going to mess with them, it doesn't really matter though. -Jonathan M Davis
Sep 27 2018
parent reply Chad Joan <chadjoan gmail.com> writes:
On Thursday, 27 September 2018 at 08:19:41 UTC, Jonathan M Davis 
wrote:
 On Thursday, September 27, 2018 1:41:23 AM MDT Chad Joan via 
 Digitalmars-d- learn wrote:
 On Thursday, 27 September 2018 at 05:12:06 UTC, Jonathan M 
 Davis
 This is also reminding me of how it's always bugged me that 
 there isn't a way to operator overload opEquals with a static 
 method (or even a free function?), given that it would allow 
 the class/struct implementer to guard against (or even 
 interact intelligently with) null values:
That's not really an issue with D. With classes, when you have a == b, it doesn't lower to a.opEquals(b). Rather, it lowers to opEquals(a, b), and the free function opEquals is defined as bool opEquals(Object lhs, Object rhs) { // If aliased to the same object or both null => equal if (lhs is rhs) return true; // If either is null => non-equal if (lhs is null || rhs is null) return false; // If same exact type => one call to method opEquals if (typeid(lhs) is typeid(rhs) || !__ctfe && typeid(lhs).opEquals(typeid(rhs))) /* CTFE doesn't like typeid much. 'is' works, but opEquals doesn't (issue 7147). But CTFE also guarantees that equal TypeInfos are always identical. So, no opEquals needed during CTFE. */ { return lhs.opEquals(rhs); } // General case => symmetric calls to method opEquals return lhs.opEquals(rhs) && rhs.opEquals(lhs); } /************************ * Returns true if lhs and rhs are equal. */ bool opEquals(const Object lhs, const Object rhs) { // A hack for the moment. return opEquals(cast()lhs, cast()rhs); } So, it already takes care of checking for null or even if the two references point to the same object. For structs, a == b, does lower to a.opEquals(b), but for better or worse, structs are designed so that their init values need to be valid, or you're going to have problems in general. Trying to work around that is fighting a losing battle.
The spec seems to have the homogeneous cases covered: classes with classes or structs with structs. What I'm more worried about is stuff like when you have a class compared to a struct or builtin type, or maybe a struct compared to a builtin type (especially more complicated builtin types like arrays!). The homogeneous cases are important for making a type consistent with itself, but the other cases are important for integrating a type with everything else in the ecosystem. Notably, "alias this" is awesome and has more or less solved that for me in the pedestrian cases I tend to encounter. I can write a struct and alias this to some reference variable that will be representative of my struct's "nullness" or other states of existence. But I wouldn't be surprised if there are corner-cases I haven't encountered yet (actually I think I just remembered that this bit me a little bit once or twice) where having a single alias-this isn't sufficient to cover all of the possible things my struct/class could be compared to (ex: if the type's null-state corresponded to int.max for ints and float.nan for floats, and you can't just use opEquals, such as when the type is a class and could be precisely null).
 Wouldn't it be helpful to have a root class type just to have 
 a "Top" type at runtime, even if it had no members?  Ex: so 
 you could do things like make an array ProtoObject[] foo; that 
 can contain any runtime polymorphic variables.
Maybe? It's not something that I've personally found to be particularly useful. Once you can templatize code, the need to have a common base class gets pretty hard to argue for, but I don't know that it's non-existent. Also, for better or worse, you can already get it with void* - and cover more types no less (albeit less safely). But from what I understand of what Andrei is intending, ProtoObject will end up being the new root class for all D classos, so having ProtoObject[] would work for all extern(D) classes. Of course, that still potentially leaves exern(C++) classes and interfaces (which could be extern(C++) or COM even right now and aren't derived from Object). So, things are already a bit weird with classes when you start interacting with other languages through D. is(T : Object) and is(T == class) do _not_ mean quite the same thing even though you'd think that they would. And I haven't done enough with extern(C++) or COM in D to claim to understand all of the subtleties. If you're not messing with them directly or writing generic code that's going to mess with them, it doesn't really matter though. -Jonathan M Davis
Gotcha. Quite a rabbit hole :)
Sep 27 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, September 27, 2018 2:44:25 AM MDT Chad Joan via Digitalmars-d-
learn wrote:
 The spec seems to have the homogeneous cases covered: classes
 with classes or structs with structs.  What I'm more worried
 about is stuff like when you have a class compared to a struct or
 builtin type, or maybe a struct compared to a builtin type
 (especially more complicated builtin types like arrays!).  The
 homogeneous cases are important for making a type consistent with
 itself, but the other cases are important for integrating a type
 with everything else in the ecosystem.

 Notably, "alias this" is awesome and has more or less solved that
 for me in the pedestrian cases I tend to encounter.  I can write
 a struct and alias this to some reference variable that will be
 representative of my struct's "nullness" or other states of
 existence.

 But I wouldn't be surprised if there are corner-cases I haven't
 encountered yet (actually I think I just remembered that this bit
 me a little bit once or twice) where having a single alias-this
 isn't sufficient to cover all of the possible things my
 struct/class could be compared to (ex: if the type's null-state
 corresponded to int.max for ints and float.nan for floats, and
 you can't just use opEquals, such as when the type is a class and
 could be precisely null).
For two types to be compared, they must be the the same type - or they must be implicitly convertible to the same type, in which case, they're converted to that type and then compared. So, as far as D's design of comparison goes, there is no need worry about comparing differing types. At most, you need to worry about what implicit type conversions exist, and D isn't big on implicit type conversions, because they tend to cause subtle bugs. So, while they definitely affect comparison, they don't affect anywhere near as much as they would in a language like C++. In general, you're not going to get very far if you're trying to make it possible to compare a user-defined type against other types in D without explicitly converting it first. - Jonathan M Davis
Sep 27 2018
parent Chad Joan <chadjoan gmail.com> writes:
On Thursday, 27 September 2018 at 09:58:25 UTC, Jonathan M Davis 
wrote:
 For two types to be compared, they must be the the same type - 
 or they must be implicitly convertible to the same type, in 
 which case, they're converted to that type and then compared. 
 So, as far as D's design of comparison goes, there is no need 
 worry about comparing differing types. At most, you need to 
 worry about what implicit type conversions exist, and D isn't 
 big on implicit type conversions, because they tend to cause 
 subtle bugs. So, while they definitely affect comparison, they 
 don't affect anywhere near as much as they would in a language 
 like C++. In general, you're not going to get very far if 
 you're trying to make it possible to compare a user-defined 
 type against other types in D without explicitly converting it 
 first.

 - Jonathan M Davis
That makes sense, but requiring types to be explicitly converted before comparisons kinda throws sand on the cake when I'm ostensibly trying to make things that interact seamlessly with existing types. "alias this" is still awesome, so it's usually fine regardless :) Thanks for the explanation.
Sep 27 2018
prev sibling parent reply Chad Joan <chadjoan gmail.com> writes:
On Wednesday, 26 September 2018 at 21:24:07 UTC, Adam D. Ruppe 
wrote:
 On Wednesday, 26 September 2018 at 20:41:38 UTC, Chad Joan 
 wrote:
 I'm implementing a deep-copy method for a tree of templated 
 class instances.  As part of this, I need some way to copy 
 each node.
 [...]
 that isn't already handled by their deepCopy method.
I would strongly suggest just using that virtual method and having the child classes override it, then you call it from any of them and get the right result.
The tree nodes are potentially very diverse, but the tree structure itself will be very homogeneous. I'm writing a parser generator backend and the tree is expressions of various operators (Sequence, OrderedChoice, UnorderedChoice, Repetition, etc). I'm trying to keep it simple: everything is an expression, and expressions can contain other expressions (though some are always leaves in the tree). At some level, if I let things implement their own deepCopy, then it means there are potentially other classes and state out there to iterate that the rest of the code doesn't know about. That could be bad, and expressions shouldn't contain anything besides expressions! This probably contrasts a lot with other use-cases, like serialization. And I wouldn't be surprised if things change later on and I end up with some kind of auxiliary virtual copy function that does what you suggest, but is specifically for handling special out-of-band mutable reference data that the expressions might need to carry someday. I suppose I've never considered just how hard/impossible it is to have a generic way to copy things. Well, maybe a little bit at various points, but not this bad ;) There are so many dimensions to the problem and it seems like the context and requirements will always be really important. So it can be made simple under the right constraints (ex: everything is immutable!), but the constraints are always different depending on the requirements (ex: ... but in this hypothetical, we need mutability, so then it's gotta happen a different way).
 Object.factory kinda sux and I'd actually like to remove it 
 (among other people). There's no plan to actually do that, but 
 still, just on principle I want to turn people away.

 But even as you can see, the implementation is lacking and it 
 isn't great design anyway - the interface with virtual methods 
 does better work. It also wouldn't work in ctfe anyway, 
 object.factory relies on runtime stuff.
Good to know! I don't think I've even used it much, if at all. I suppose I won't miss it if it goes ;)
 If Object.factory is incapable of this, is there some other 
 CTFE-friendly way to copy templated class instances?
I think you can copy typeinfo().init and then call typeinfo().defaultConstructor - this is iirc what Object.factory does, but once you already have the typeinfo you can use it directly and bypass the string lookup.
I'm having trouble looking this up. Could you link me to the docs for this?
 But you'd really be better off with a virtual copy method. I 
 say those string factory things should only be used if you are 
 starting with a string, like deserialization.


 interface Copyable {
    Copyable copy();
 }

 class Whatever(T) : Copyable {
    Whatever!T copy() {
        auto c = new Whatever!T();
        c.tupleof = this.tupleof;
        return c;
    }
 }


 that kind of method. the template implements the interface so 
 little boilerplate and it works and can be customized etc and 
 fits in well. If you call it from the interface, you get an 
 instance of the interface (tho note since it is virtual, the 
 underlying type is still what you need). If you call from the 
 child static type, you get it right back. Yay, fits liskov and 
 works!
As above, I think this might be a very clean and effective solution for a different class of use-cases :) I'll keep it in mind though.
 If I have to, I can probably make these things register 
 themselves in some list of delegates that can be used to 
 instantiate the correct class.  Or something like that.  But I 
 am hoping that there is a better way that involves less 
 boilerplate.
that's not a terrible idea if you need delegates keyed to strings...
Right. At some level I just need a function that I can call like this: auto newThing = makeAnother(originalThing); and perhaps makeAnother(...) can just lookup originalThing's classname in an associative array of delegates. Or maybe I can just hash some part of originalThing's type information. I can put all of the ugly registration boilerplate into a mixin template and somehow force that to always be mixed-into any descendant classes. OK, it's probably getting too far into the weeds now, but it seems doable and I'll reach for that if I need to. ... Things at least seem much more clear already. Thanks a bunch!
Sep 26 2018
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 27 September 2018 at 05:04:09 UTC, Chad Joan wrote:
 The tree nodes are potentially very diverse, but the tree 
 structure itself will be very homogeneous.
The virtual method can still handle that case! But, if your child array of expressions is already accessible through the base interface, I think now your code is going to look something more like: Expression copy(Expression e) { // I kinda hate the cast, but create is too generic lol // and we already know so it ok here Expression n = cast(Expression) typeid(e).create(); foreach(child; e.children) n.children ~= copy(child); return n; } that's not too bad :)
 I'm having trouble looking this up.  Could you link me to the 
 docs for this?
Well, the create method is actually better anyway (and actually documented! I just forgot about it in that first post) but you can see more of the guts here http://dpldocs.info/experimental-docs/object.TypeInfo_Class.html
 As above, I think this might be a very clean and effective 
 solution for a different class of use-cases :)  I'll keep it in 
 mind though.
Yeah. And I did make one mistake: the tupleof assignment trick wouldn't work well for references, so lol it isn't much of a deep copy. You'd want to deep copy any arrays too probably. But sounds like you are already doing that, yay.
Sep 27 2018
parent Chad Joan <chadjoan gmail.com> writes:
On Thursday, 27 September 2018 at 13:23:15 UTC, Adam D. Ruppe 
wrote:
 On Thursday, 27 September 2018 at 05:04:09 UTC, Chad Joan wrote:
 As above, I think this might be a very clean and effective 
 solution for a different class of use-cases :)  I'll keep it 
 in mind though.
Yeah. And I did make one mistake: the tupleof assignment trick wouldn't work well for references, so lol it isn't much of a deep copy. You'd want to deep copy any arrays too probably. But sounds like you are already doing that, yay.
You're right though, if I end up adding boilerplate anyways, I may as well have a good shallow copy to begin with.
Sep 27 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/26/18 4:41 PM, Chad Joan wrote:
 Hi all,
 
 I'm implementing a deep-copy method for a tree of templated class 
 instances.  As part of this, I need some way to copy each node. I want 
 to avoid code that does things like casting objects into byte arrays and 
 then copying raw bytes; I want all operations to be memory safe things 
 that I can use at compile-time.  So I planned to make all of these have 
 default constructors and use Object.factory to at least create the 
 correct instance type at the destination.  The classes can implement 
 auxiliary copy methods if they need to copy anything that isn't already 
 handled by their deepCopy method.
 
 But I ran into a problem: Object.factory doesn't seem to be compatible 
 with templated classes.
Object.factory is a really old poorly supported type of reflection. I would not depend on it for anything. You are better off using your own registration system. As far as choosing the design for your problem, you can use: auto obj = typeid(obj).create(); which is going to work better, and doesn't require a linear search through all modules/classes like Object.factory. If it were me, I'd probably instead implement a clone virtual method. This way if any custom things need to occur when copying the data, it can be handled. -Steve
Sep 26 2018
parent reply Chad Joan <chadjoan gmail.com> writes:
On Wednesday, 26 September 2018 at 21:25:07 UTC, Steven 
Schveighoffer wrote:
 ...
 Object.factory is a really old poorly supported type of 
 reflection. I would not depend on it for anything.
Roger that. Will avoid :)
 You are better off using your own registration system.

 As far as choosing the design for your problem, you can use:

 auto obj = typeid(obj).create();

 which is going to work better, and doesn't require a linear 
 search through all modules/classes like Object.factory.
How does this work? The language reference states that typeid(Type) returns "an instance of class TypeInfo corresponding to Type". (https://dlang.org/spec/expression.html#typeid_expressions) But then the TypeInfo class doesn't seem to have a .create() method, or at least not one in the documentation: https://dlang.org/phobos/object.html#.TypeInfo It looks like what I want, so it might be very helpful.
 If it were me, I'd probably instead implement a clone virtual 
 method. This way if any custom things need to occur when 
 copying the data, it can be handled.

 -Steve
I'm thinking I want to avoid the virtual method in this case, for reasons I wrote about in my response to Adam (https://forum.dlang.org/post/zovficijurwhuurrrfjf forum.dlang.org). But I think it's probably a good suggestion in most cases; I suspect that most of the time wanting to write a "deepCopy" method is going to be in response to some problem that will respond well to just virtualizing the method. Thanks for the advice!
Sep 26 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 27 September 2018 at 05:18:14 UTC, Chad Joan wrote:
 How does this work?

 The language reference states that typeid(Type) returns "an 
 instance of class TypeInfo corresponding to Type".
 (https://dlang.org/spec/expression.html#typeid_expressions)

 But then the TypeInfo class doesn't seem to have a .create() 
 method, or at least not one in the documentation:
 https://dlang.org/phobos/object.html#.TypeInfo
I forgot about the create method, lol, that is what you want. But typeid(obj) - pass it an existing object of the type btw - when obj is a class will return TypeInfo_Class - a subclass of TypeInfo. It is *that* which has the create method. https://dlang.org/phobos/object.html#.TypeInfo_Class.create (or less horrible docs http://dpldocs.info/experimental-docs/object.TypeInfo_Class.html )
Sep 27 2018
parent Chad Joan <chadjoan gmail.com> writes:
On Thursday, 27 September 2018 at 13:16:58 UTC, Adam D. Ruppe 
wrote:
 On Thursday, 27 September 2018 at 05:18:14 UTC, Chad Joan wrote:
 How does this work?

 The language reference states that typeid(Type) returns "an 
 instance of class TypeInfo corresponding to Type".
 (https://dlang.org/spec/expression.html#typeid_expressions)

 But then the TypeInfo class doesn't seem to have a .create() 
 method, or at least not one in the documentation:
 https://dlang.org/phobos/object.html#.TypeInfo
I forgot about the create method, lol, that is what you want. But typeid(obj) - pass it an existing object of the type btw - when obj is a class will return TypeInfo_Class - a subclass of TypeInfo. It is *that* which has the create method. https://dlang.org/phobos/object.html#.TypeInfo_Class.create (or less horrible docs http://dpldocs.info/experimental-docs/object.TypeInfo_Class.html )
Yep, that solves the mystery! Thanks!
Sep 27 2018
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/27/18 1:18 AM, Chad Joan wrote:
 On Wednesday, 26 September 2018 at 21:25:07 UTC, Steven Schveighoffer 
 wrote:
 ...
 Object.factory is a really old poorly supported type of reflection. I 
 would not depend on it for anything.
Roger that.  Will avoid :)
 You are better off using your own registration system.

 As far as choosing the design for your problem, you can use:

 auto obj = typeid(obj).create();

 which is going to work better, and doesn't require a linear search 
 through all modules/classes like Object.factory.
How does this work? The language reference states that typeid(Type) returns "an instance of class TypeInfo corresponding to Type". (https://dlang.org/spec/expression.html#typeid_expressions) But then the TypeInfo class doesn't seem to have a .create() method, or at least not one in the documentation: https://dlang.org/phobos/object.html#.TypeInfo
typeid sometimes gives you a more derived type than TypeInfo. Including for classes and structs. In the past, .classinfo gave you a different thing than typeid(obj), but now it is the same thing: auto obj = new Object; // classinfo and typeid are the same object assert(obj.classinfo is typeid(obj)); // and the same type static assert(is(typeof(obj.classinfo) == typeof(typeid(obj)))); I wouldn't use classinfo any more, I generally use typeid. -Steve
Sep 27 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 9/27/18 10:20 AM, Steven Schveighoffer wrote:
 typeid sometimes gives you a more derived type than TypeInfo. Including 
 for classes and structs.
 
 In the past, .classinfo gave you a different thing than typeid(obj), but 
 now it is the same thing:
 
 
      auto obj = new Object;
      // classinfo and typeid are the same object
      assert(obj.classinfo is typeid(obj));
      // and the same type
      static assert(is(typeof(obj.classinfo) == typeof(typeid(obj))));
 
 I wouldn't use classinfo any more, I generally use typeid.
I should add that typeid does give you the derived TypeInfo_Class, not the concrete one. that is: Object obj; // = null //typeid(obj); // segfault, can't dereference null pointer class C {} obj = new C; static assert(is(typeof(obj) == Object)); writeln(typeid(obj)); // C, not Object So it really is a drop-in replacement for classinfo. I think classinfo is still there to avoid breaking existing code that uses it. -Steve
Sep 27 2018
parent Chad Joan <chadjoan gmail.com> writes:
On Thursday, 27 September 2018 at 14:23:48 UTC, Steven 
Schveighoffer wrote:
 On 9/27/18 10:20 AM, Steven Schveighoffer wrote:
 typeid sometimes gives you a more derived type than TypeInfo. 
 Including for classes and structs.
 
 In the past, .classinfo gave you a different thing than 
 typeid(obj), but now it is the same thing:
 
 
      auto obj = new Object;
      // classinfo and typeid are the same object
      assert(obj.classinfo is typeid(obj));
      // and the same type
      static assert(is(typeof(obj.classinfo) == 
 typeof(typeid(obj))));
 
 I wouldn't use classinfo any more, I generally use typeid.
I should add that typeid does give you the derived TypeInfo_Class, not the concrete one. that is: Object obj; // = null //typeid(obj); // segfault, can't dereference null pointer class C {} obj = new C; static assert(is(typeof(obj) == Object)); writeln(typeid(obj)); // C, not Object So it really is a drop-in replacement for classinfo. I think classinfo is still there to avoid breaking existing code that uses it. -Steve
Interesting! That's yet another thing I hadn't realized had changed. Good to know.
Sep 27 2018
prev sibling parent reply Alex <sascha.orlov gmail.com> writes:
On Wednesday, 26 September 2018 at 20:41:38 UTC, Chad Joan wrote:

 class Root(T)
 {
 	T x;
 }

 class Extended(T) : Root!T
 {
 	T y;
 }
Sorry for a technical aside, but would this be something for you? https://forum.dlang.org/post/vtaxcxpufrovwfrkbyye forum.dlang.org I mean... In either case, there is something curious in the Extended/Root usage, as they both are bound to the same type. And if so, you could get rid of the templatization in the Extended class, either by templating Root on the Extended or use the T of Root in Extended.
Sep 27 2018
parent reply Chad Joan <chadjoan gmail.com> writes:
On Thursday, 27 September 2018 at 08:56:22 UTC, Alex wrote:
 On Wednesday, 26 September 2018 at 20:41:38 UTC, Chad Joan 
 wrote:

 class Root(T)
 {
 	T x;
 }

 class Extended(T) : Root!T
 {
 	T y;
 }
Sorry for a technical aside, but would this be something for you? https://forum.dlang.org/post/vtaxcxpufrovwfrkbyye forum.dlang.org I mean... In either case, there is something curious in the Extended/Root usage, as they both are bound to the same type. And if so, you could get rid of the templatization in the Extended class, either by templating Root on the Extended or use the T of Root in Extended.
Perhaps it's half some form of unintended neural network fabrication that happened as I wrote the example and half that the original code isn't that well thought out yet. The original code looks more like this: template Nodes(T) { class Root { T x; } class Extended : Root { T y; } } I will probably end up going with the latter suggestion and have Extended use the Root's T. That would probably make sense for what I'm doing. In my case, the T allows the caller to configure what kind of output the thing provides... IIRC (it's been a while since I touched this code o.O). Thanks for pointing that out.
Sep 27 2018
parent Chad Joan <chadjoan gmail.com> writes:
On Thursday, 27 September 2018 at 18:37:27 UTC, Chad Joan wrote:
 I will probably end up going with the latter suggestion and 
 have Extended use the Root's T.  That would probably make sense 
 for what I'm doing.  In my case, the T allows the caller to 
 configure what kind of output the thing provides... IIRC (it's 
 been a while since I touched this code o.O).

 Thanks for pointing that out.
Wait, actually the T is the element type of the input range, like "char" if the parser is to parse UTF8 strings. I'd already figured that out too. (*゚ー゚)ゞ
Sep 27 2018