www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Defining some stuff for each class in turn

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I am becoming increasingly aware that we need to provide some means to 
define certain members (data and functions) for each class as if they 
were pasted there.

Right now that right is reserved to the compiler, which generates e.g. 
one static classinfo object for each class. But users would want to also 
define members for each class automatically. This is often the case with 
contravariant-argument functions (that we discussed recently) and other 
important cases such as factories and cloning.

For starters, assume I want to define one static int for each class in a 
hierarchy.

class Counted {
     static uint counter;
     ...
}

Then subclasses of Counted would all share counter, something I don't 
want. I want each subclass to have its own counter, so I need to ask 
derived classes to *also* define counter:

class A : Counted {
     static uint counter;
     ...
}

With the suggested feature, there would be the ability to define counter 
  for each class in turn.

class Counted {
     mixin(Derived)
     {
         // Insert here stuff that must be "pasted" for each subclass
         // of Counted (including Counted itself).
         // Use "Derived" as the name of the current subtype of Counter
         static uint counter;
         static if (is(Counted == Derived))
             ref uint getCounter() { return counter; }
         else
             override ref uint getCounter() { return counter; }
     }
     ...
}

The code above does something quite neat - it defines an overridable 
function Counted.getCounter in the base class, and then overrides it in 
*every* class inheriting that base class, to return that class' own 
counter.

The same should go in interface definitions - the feature should allow 
you to

There are quite a few immediate applications of this: opEquals, opCmp, 
clone, various factories and object pools, and most importantly 
reflection. To enable custom reflection with today's D, we'd have to 
require each class to insert some code inside the class body. With the 
mechanism described above, we allow the base class or an interface (e.g. 
Reflectable) to inject the code into the derived class' body.

What do you think?


Andrei
Oct 01 2009
next sibling parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Thu, Oct 1, 2009 at 12:25 PM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 I am becoming increasingly aware that we need to provide some means to
 define certain members (data and functions) for each class as if they wer=


e

 pasted there.

 Right now that right is reserved to the compiler, which generates e.g. on=


e

 static classinfo object for each class. But users would want to also defi=


ne

 members for each class automatically. This is often the case with
 contravariant-argument functions (that we discussed recently) and other
 important cases such as factories and cloning.

 For starters, assume I want to define one static int for each class in a
 hierarchy.

 class Counted {
 =A0 =A0static uint counter;
 =A0 =A0...
 }

 Then subclasses of Counted would all share counter, something I don't wan=


t.

 I want each subclass to have its own counter, so I need to ask derived
 classes to *also* define counter:

 class A : Counted {
 =A0 =A0static uint counter;
 =A0 =A0...
 }

 With the suggested feature, there would be the ability to define counter
 =A0for each class in turn.

 class Counted {
 =A0 =A0mixin(Derived)
 =A0 =A0{
 =A0 =A0 =A0 =A0// Insert here stuff that must be "pasted" for each subcla=


ss

 =A0 =A0 =A0 =A0// of Counted (including Counted itself).
 =A0 =A0 =A0 =A0// Use "Derived" as the name of the current subtype of Cou=


nter

 =A0 =A0 =A0 =A0static uint counter;
 =A0 =A0 =A0 =A0static if (is(Counted =3D=3D Derived))
 =A0 =A0 =A0 =A0 =A0 =A0ref uint getCounter() { return counter; }
 =A0 =A0 =A0 =A0else
 =A0 =A0 =A0 =A0 =A0 =A0override ref uint getCounter() { return counter; }
 =A0 =A0}
 =A0 =A0...
 }

 The code above does something quite neat - it defines an overridable
 function Counted.getCounter in the base class, and then overrides it in
 *every* class inheriting that base class, to return that class' own count=


er.

 The same should go in interface definitions - the feature should allow yo=


u

 to

 There are quite a few immediate applications of this: opEquals, opCmp,
 clone, various factories and object pools, and most importantly reflectio=


n.

 To enable custom reflection with today's D, we'd have to require each cla=


ss

 to insert some code inside the class body. With the mechanism described
 above, we allow the base class or an interface (e.g. Reflectable) to inje=


ct

 the code into the derived class' body.

 What do you think?


I think it sounds interesting enough, but I can't help but wonder if
this is a feature that you've really thought through (especially wrt.
how it interacts with mechanisms such as template mixins and normal
symbol inheritance), or if you just want it to support some pattern
you want to use in Phobos 2.
Oct 01 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jarrett Billingsley wrote:

 On Thu, Oct 1, 2009 at 12:25 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


[code injection]

 What do you think?


 
 I think it sounds interesting enough, but I can't help but wonder if
 this is a feature that you've really thought through (especially wrt.
 how it interacts with mechanisms such as template mixins and normal
 symbol inheritance), or if you just want it to support some pattern
 you want to use in Phobos 2.


I've known for a long time this was in store if we want to define decent 
reflection. It's also been a perennial source of trouble with a lot of 
code that needs to inject members.

Let me give another example. When we discussed opCmp around here, people 
said that's the old way of doing things and that the right way is to use 
an interface Comparator!T, akin to Java's Comparator<T>:

http://java.sun.com/j2se/1.5.0/docs/api/java/util/Comparator.html

That only solves exactly one level of inheritance. It's a more elaborate 
solution that doesn't quite help that much. Consider:

interface Comparator(T)
{
     int opCmp(T rhs); // yum, exact type
}

class Widget : Comparator!Widget
{
     override opCmp(Widget rhs) { ... } // yum, exact type
}

class Gadget : Widget
{
     override opCmp(Gadget rhs) { ... } // ERROR!!!!!!!!
}

Of course that didn't work. Gadget.opCmp doesn't override anything. 
Gadget needs to remember to inherit Comparator!Gadget:

class Gadget : Widget, Comparator!Gadget
{
     override opCmp(Gadget rhs) { ... } // oh ok
}

So any class X in Widget's hierarchy that forgets to inherit 
Comparator!X undergoes the risk of having the wrong opCmp called for it.

With injection, Comparator can be made to work for any interface:

interface Comparator(T)
{
     int opCmp(Comparator!T rhs);
     mixin(Impl) // for each implementation Impl
     {
         int opCmp(Impl rhs);
         override int opCmp(Comparator!T rhs)
         {
             return opCmp(cast(Impl) rhs);
         }
     }
}

class Widget : Comparator!Widget { ... }

Now every derived class of Widget, including Widget itself, commits to 
define a opCmp with the proper signature (failure to do so results in a 
linker error), and also defines the interface function in terms of it.


Andrei
Oct 01 2009
next sibling parent grauzone <none example.net> writes:
Andrei Alexandrescu wrote:

 Jarrett Billingsley wrote:

 I think it sounds interesting enough, but I can't help but wonder if
 this is a feature that you've really thought through (especially wrt.
 how it interacts with mechanisms such as template mixins and normal
 symbol inheritance), or if you just want it to support some pattern
 you want to use in Phobos 2.


 
 I've known for a long time this was in store if we want to define decent 
 reflection. It's also been a perennial source of trouble with a lot of 
 code that needs to inject members.


I don't see how that would be needed for reflection, and I've written 
some custom reflection/serialization myself. Yay for tupleof being able 
to read even private members.
Oct 01 2009
prev sibling next sibling parent reply "Phil Deets" <pjdeets2 gmail.com> writes:
On Thu, 01 Oct 2009 12:53:46 -0500, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 interface Comparator(T)
 {
      int opCmp(Comparator!T rhs);
      mixin(Impl) // for each implementation Impl
      {
          int opCmp(Impl rhs);
          override int opCmp(Comparator!T rhs)
          {
              return opCmp(cast(Impl) rhs);
          }
      }
 }


I like that you can use this in interfaces too. It seems to allow for  
something similar to multiple inheritance, but presumably without the  
implementation difficulty.

Phil Deets
Oct 01 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Phil Deets wrote:

 On Thu, 01 Oct 2009 12:53:46 -0500, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> wrote:

 interface Comparator(T)
 {
      int opCmp(Comparator!T rhs);
      mixin(Impl) // for each implementation Impl
      {
          int opCmp(Impl rhs);
          override int opCmp(Comparator!T rhs)
          {
              return opCmp(cast(Impl) rhs);
          }
      }
 }


 
 I like that you can use this in interfaces too. It seems to allow for 
 something similar to multiple inheritance, but presumably without the 
 implementation difficulty.
 
 Phil Deets


That's a great point. To compare this feature with Scala's mixins (which 
I studied and now feel are a good feature), my understanding is that 
Scala mixins are an interface plus a default implementation. The feature 
proposed above is an interface with a prescribed implementation. That 
way, Scala mixins seem to be somewhat more flexible.

Andrei
Oct 01 2009
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 01 Oct 2009 13:53:46 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 Jarrett Billingsley wrote:

 On Thu, Oct 1, 2009 at 12:25 PM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 [code injection]

 What do you think?


  I think it sounds interesting enough, but I can't help but wonder if
 this is a feature that you've really thought through (especially wrt.
 how it interacts with mechanisms such as template mixins and normal
 symbol inheritance), or if you just want it to support some pattern
 you want to use in Phobos 2.


 I've known for a long time this was in store if we want to define decent  
 reflection. It's also been a perennial source of trouble with a lot of  
 code that needs to inject members.

 Let me give another example. When we discussed opCmp around here, people  
 said that's the old way of doing things and that the right way is to use  
 an interface Comparator!T, akin to Java's Comparator<T>:

 http://java.sun.com/j2se/1.5.0/docs/api/java/util/Comparator.html

 That only solves exactly one level of inheritance. It's a more elaborate  
 solution that doesn't quite help that much. Consider:

 interface Comparator(T)
 {
      int opCmp(T rhs); // yum, exact type
 }

 class Widget : Comparator!Widget
 {
      override opCmp(Widget rhs) { ... } // yum, exact type
 }

 class Gadget : Widget
 {
      override opCmp(Gadget rhs) { ... } // ERROR!!!!!!!!
 }

 Of course that didn't work. Gadget.opCmp doesn't override anything.  
 Gadget needs to remember to inherit Comparator!Gadget:

 class Gadget : Widget, Comparator!Gadget
 {
      override opCmp(Gadget rhs) { ... } // oh ok
 }

 So any class X in Widget's hierarchy that forgets to inherit  
 Comparator!X undergoes the risk of having the wrong opCmp called for it.

 With injection, Comparator can be made to work for any interface:

 interface Comparator(T)
 {
      int opCmp(Comparator!T rhs);
      mixin(Impl) // for each implementation Impl
      {
          int opCmp(Impl rhs);
          override int opCmp(Comparator!T rhs)
          {
              return opCmp(cast(Impl) rhs);
          }
      }
 }

 class Widget : Comparator!Widget { ... }

 Now every derived class of Widget, including Widget itself, commits to  
 define a opCmp with the proper signature (failure to do so results in a  
 linker error), and also defines the interface function in terms of it.


This still isn't optimal.  For example, two different derivatives of the  
same class could try to compare to eachother and end up passing null into  
your opCmp.  It would be nice if the compiler could reject unrelated  
comparisons:

class X: Comparator!X
{
   int opCmp(X) {}
}

class Y: X
{
   int opCmp(Y) {}
}

class Z: X
{
   int opCmp(Z) {}
}

void main()
{
   auto y = new Y;
   auto z = new Z;
   z < y; // error, rejected because they can't possibly be related
}

If there was a way to do that it would be cool.  I also think the  
auto-defined opCmp should look like this:

override int opCmp(Comparator!T rhs)
{
   if(auto imp = cast(Impl) rhs)
      return opCmp(imp);
   return false;
}

That would cut down on calls into your actual opCmp function with a null  
argument.

-Steve
Oct 02 2009
prev sibling next sibling parent reply Max Samukha <spambox d-coding.com> writes:
On Thu, 01 Oct 2009 11:25:03 -0500, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:


I am becoming increasingly aware that we need to provide some means to 
define certain members (data and functions) for each class as if they 
were pasted there.


I am glad you have finally bumped into the problem. It has been
waiting for a solution for quite some time.

For example, the upcoming reflection mechanism in QtD (which uses its
own metaobjects instead of the limited classinfo) requires manual
insertion of a mixin into derived classes unless the mixin is inserted
implicitly when a signal/property/etc is declared.

A couple of votes for the feature.
Oct 01 2009
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2009-10-01 14:05:10 -0400, Max Samukha <spambox d-coding.com> said:


 For example, the upcoming reflection mechanism in QtD (which uses its
 own metaobjects instead of the limited classinfo) requires manual
 insertion of a mixin into derived classes unless the mixin is inserted
 implicitly when a signal/property/etc is declared.


I'm using the exact same solution here for the D/Objective-C bridge and 
it's working surprisingly well.

I certainly wouldn't turn down the possibility of inheriting an 
implementation in subclasses, it'd make things a little less hackish.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Oct 01 2009
prev sibling next sibling parent Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  1 de octubre a las 11:25 me escribiste:

 What do you think?


I think this is close to Python metaclasses[1], but much more specific.
It's amazing how many cool things one can do with Python metaclasses
(there are several articles discussing them[2]) and decorators.  I would
love to see something similar in D but operating at compile-time.  I think
it would be nice to point to a more general solution (like AST macros)
than implementing all sort of small and specific tricks for each problem
instead.

[1] http://docs.python.org/reference/datamodel.html#customizing-class-creation
[2] http://en.wikipedia.org/wiki/Metaclass
    http://www.ibm.com/developerworks/linux/library/l-pymeta.html

-- 
Leandro Lucarella (AKA luca)                      http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Wake from your sleep,
the drying of your tears,
Today we escape, we escape.
Oct 01 2009
prev sibling next sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Thu, 01 Oct 2009 20:25:03 +0400, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:


 I am becoming increasingly aware that we need to provide some means to  
 define certain members (data and functions) for each class as if they  
 were pasted there.

 Right now that right is reserved to the compiler, which generates e.g.  
 one static classinfo object for each class. But users would want to also  
 define members for each class automatically. This is often the case with  
 contravariant-argument functions (that we discussed recently) and other  
 important cases such as factories and cloning.

 For starters, assume I want to define one static int for each class in a  
 hierarchy.

 class Counted {
      static uint counter;
      ...
 }

 Then subclasses of Counted would all share counter, something I don't  
 want. I want each subclass to have its own counter, so I need to ask  
 derived classes to *also* define counter:

 class A : Counted {
      static uint counter;
      ...
 }

 With the suggested feature, there would be the ability to define counter  
   for each class in turn.

 class Counted {
      mixin(Derived)
      {
          // Insert here stuff that must be "pasted" for each subclass
          // of Counted (including Counted itself).
          // Use "Derived" as the name of the current subtype of Counter
          static uint counter;
          static if (is(Counted == Derived))
              ref uint getCounter() { return counter; }
          else
              override ref uint getCounter() { return counter; }
      }
      ...
 }

 The code above does something quite neat - it defines an overridable  
 function Counted.getCounter in the base class, and then overrides it in  
 *every* class inheriting that base class, to return that class' own  
 counter.

 The same should go in interface definitions - the feature should allow  
 you to

 There are quite a few immediate applications of this: opEquals, opCmp,  
 clone, various factories and object pools, and most importantly  
 reflection. To enable custom reflection with today's D, we'd have to  
 require each class to insert some code inside the class body. With the  
 mechanism described above, we allow the base class or an interface (e.g.  
 Reflectable) to inject the code into the derived class' body.

 What do you think?


 Andrei


This is cool. I'd also add Serialization task to the application list  
(currently I manually insert "mixin Serializable!();" into every class.

There is one thing that I dislike about it, though. I believe mixin(Foo)  
{} syntax is a bit far-fetched, it is not obvious that Foo represents  
derived type inside a mixin scope.

I'd like to remind you that we could just use typeof(this) instead of  
Foo/Derived/etc:

static if (is (typeof(this) == Counted) {
     ref uint getCounter() { return counter; }
} else {
     override ref uint getCounter() { return counter; }
}

If you remember, we had a similar discussion about a half year ago. It was  
about ICloneable interface that forces all derived class to implement  
proper clone() method. We could define it as:

// forces implementor class and all the derivatives to implement proper  
clone() method
interface ICloneable
{
     scope mixin {	// alternative syntax
         typeof(this) clone();
     }
}

and also be able to declare a method non-recursively (so that derived  
class don't have to implement that method):

interface ICloneableNonRecursive
{
     typeof(this) clone();
}

Other possible solution could be to use a special "derived" keyword. We  
already have a class-level compile-time constant - "super". We could also  
introduce its counter-part ("derived"), which will expand into a concrete  
derived class type. This will simplify the syntax a bit and will allow to  
get rid of mixin and an addition level of indentation:

interface ICloneable
{
     derived clone();
}

interface IComparable
{
     int opCmp(derived other);
}

As a downside, I don't know how to define a field in every class this way  
("int derived.counter;" maybe?).
Oct 01 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Denis Koroskin wrote:

 On Thu, 01 Oct 2009 20:25:03 +0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


[...]

 What do you think?


 Andrei


 
 This is cool. I'd also add Serialization task to the application list 
 (currently I manually insert "mixin Serializable!();" into every class.


Yah, serialization is close to reflection so it's definitely part of the 
target applications of the features.


 There is one thing that I dislike about it, though. I believe mixin(Foo) 
 {} syntax is a bit far-fetched, it is not obvious that Foo represents 
 derived type inside a mixin scope.
 
 I'd like to remind you that we could just use typeof(this) instead of 
 Foo/Derived/etc:
 
 static if (is (typeof(this) == Counted) {
     ref uint getCounter() { return counter; }
 } else {
     override ref uint getCounter() { return counter; }
 }


Yah, I thought a lot along the likes of typeof(this). We need some 
symbolic alias for "the struct or class currently being defined" and 
typeof(this) is an obvious choice. I just fear that it might be 
confusing that typeof(this) works even outside any method, i.e. when 
there is no "this".


 If you remember, we had a similar discussion about a half year ago. It 
 was about ICloneable interface that forces all derived class to 
 implement proper clone() method. We could define it as:
 
 // forces implementor class and all the derivatives to implement proper 
 clone() method
 interface ICloneable
 {
     scope mixin {    // alternative syntax
         typeof(this) clone();
     }
 }


Yah, I remember.


 and also be able to declare a method non-recursively (so that derived 
 class don't have to implement that method):
 
 interface ICloneableNonRecursive
 {
     typeof(this) clone();
 }
 
 Other possible solution could be to use a special "derived" keyword. We 
 already have a class-level compile-time constant - "super". We could 
 also introduce its counter-part ("derived"), which will expand into a 
 concrete derived class type. This will simplify the syntax a bit and 
 will allow to get rid of mixin and an addition level of indentation:
 
 interface ICloneable
 {
     derived clone();
 }
 
 interface IComparable
 {
     int opCmp(derived other);
 }
 
 As a downside, I don't know how to define a field in every class this 
 way ("int derived.counter;" maybe?).


I dislike adding yet another keyword. In fact, I want to propose Walter 
to eliminate "super" as a keyword and make it just an alias for the base 
class as if you defined it by hand.


Andrei
Oct 01 2009
prev sibling parent reply Christopher Wright <dhasenan gmail.com> writes:
Andrei Alexandrescu wrote:

 I am becoming increasingly aware that we need to provide some means to 
 define certain members (data and functions) for each class as if they 
 were pasted there.


Most of the examples given would be well served by decent builtin 
reflection. Walter thinks reflection is too expensive to be active by 
default. Find a cheaper way to provide runtime reflection. It's not as 
sexy as using templates, but it's DRY and easier to use. Slower, but you 
don't pay the cost of reflection multiple times if you have multiple 
libraries requiring reflection.
Oct 01 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Christopher Wright wrote:

 Andrei Alexandrescu wrote:

 I am becoming increasingly aware that we need to provide some means to 
 define certain members (data and functions) for each class as if they 
 were pasted there.


 
 Most of the examples given would be well served by decent builtin 
 reflection. Walter thinks reflection is too expensive to be active by 
 default. Find a cheaper way to provide runtime reflection. It's not as 
 sexy as using templates, but it's DRY and easier to use. Slower, but you 
 don't pay the cost of reflection multiple times if you have multiple 
 libraries requiring reflection.


What cheaper way would be than allowing a base class to prescribe 
reflection for its hierarchy? Where do templates even enter the mix? 
What's slower and why? Why do reflection as a language feature 
(increases base language size, buggy, rigid) instead of allowing it as a 
library if we so can? I'm totally against that.

Andrei
Oct 01 2009
parent Christopher Wright <dhasenan gmail.com> writes:
Andrei Alexandrescu wrote:

 Christopher Wright wrote:

 Andrei Alexandrescu wrote:

 I am becoming increasingly aware that we need to provide some means 
 to define certain members (data and functions) for each class as if 
 they were pasted there.


 Most of the examples given would be well served by decent builtin 
 reflection. Walter thinks reflection is too expensive to be active by 
 default. Find a cheaper way to provide runtime reflection. It's not as 
 sexy as using templates, but it's DRY and easier to use. Slower, but 
 you don't pay the cost of reflection multiple times if you have 
 multiple libraries requiring reflection.


 
 What cheaper way would be than allowing a base class to prescribe 
 reflection for its hierarchy? Where do templates even enter the mix? 
 What's slower and why? Why do reflection as a language feature 
 (increases base language size, buggy, rigid) instead of allowing it as a 
 library if we so can? I'm totally against that.
 
 Andrei


Once you get two or three libraries using reflection, it's cheaper to 
have the language provide it than to have separate, incompatible 
reflection systems.

It doesn't matter whether the language provides reflection or the 
standard library. (The only way for me to tell is whether I'm importing 
from core or from std.) Either way, it should be standardized and easy 
to opt in or out (depending on the default), and there should be no way 
or no cost for opting in twice.

You have to use templates to get reflection info, currently. Unless CTFE 
has gotten a lot better, in which case it's effectively the same as 
using templates, but with a bit less executable bloat.

Template-based solutions can be faster than reflection-based solutions 
because you can access fields and methods directly. But runtime 
reflection doesn't disallow templates.
Oct 02 2009