www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why can't static functions be virtual

reply Grinny <Grinny_member pathlink.com> writes:
Why can't static functions be virtual? 
Wouldn't it be nice to be able to create
functions like Java's 
java.awt.ToolKit.getDefaultToolkit which can be
overridden by a subclass to return a 
single instance of a certain class  ?
Oct 08 2004
next sibling parent Sjoerd van Leent <svanleent wanadoo.nl> writes:
Grinny wrote:

 Why can't static functions be virtual? 
 Wouldn't it be nice to be able to create
 functions like Java's 
 java.awt.ToolKit.getDefaultToolkit which can be
 overridden by a subclass to return a 
 single instance of a certain class  ? 
 
 


In Java, static functions are *NOT* virtual. Simply because it is not 
needed. If you want to use java.awt.ToolKit.getDefaultToolkit() you are 
refering to the ToolKit class.

If you decide to inherit the ToolKit class, and make a new 
getDefaultToolkit() function, you need to perform another upcall like 
com.company.YourToolKit.getDefaultToolkit(). This function does not 
override the original function.

Regards,
Sjoerd
Oct 09 2004
prev sibling parent reply John McAuley <john_mcauley bigfoot.com> writes:
I was going to ask why they weren't virtual.

I wrote this:

  class CBase
  {
    static void hello()
    {
      printf("hello from CBase\n");
    }
  }

  class CDerived : CBase
  {
    static void hello()
    {
      printf("hello from CDerived\n");
    }
  }

  void main()
  {
    CDerived derived = new CDerived;
    CBase base  = derived;
    
    base.hello(); // prints Hello from CBase
    
    void function() fn;
    fn = &CDerived.hello;
    fn =&base.hello; // compiler error
  }

Are statics non-virtual because that's the way c++ does it?

Surely the compiler can see base.hello() and then:

* look up the CBase class definition to find hello
* get its index in the vtable and note that it is static
* dereference base to get the CDerived vtable
* get the address that corresponds to hello
* call hello without passing base as a secret this pointer

Its the same sequence of steps as for a non-static virtual. The only difference
is that you don't pass base as a hidden parameter. The difference is how
parameters are passed on the stack.

Then when the compiler sees:

   fn = &base.hello;

It does the same vtable lookup and assigns the found address to fn because
statics don't need the 'secret this pointer'.

I expected that &base.hello would be equivalent to &CDerived.hello

and that &derived.CBase.hello would be equivalent to &CBase.hello just as
derived.CBase.hello() prints "Hello from CBase"

Anyway, the question is: why doesn't D do this? Why the distinction between
static members and non-static members?
Jan 16 2007
next sibling parent reply Daniel Keep <daniel.keep+lists gmail.com> writes:
I suppose it does it this way because every other statically typed 
language I know of does it this way.  And they do it because, AFAIK, it 
makes sense to.  The method you are calling isn't attached to the 
instance at all--it's attached to the class itself.

When you say "base.hello()", the compiler's actually cutting you some 
slack and saying "well, I know what you *really* mean, so I'll let you 
off" and calls the method.

class Foo { static void bar() { ... } }

is pretty much the same as

void Foo_bar() { with(Foo) { ... } }

Which means that "base" has absolutely nothing to do with looking up the 
method whatsoever (well, apart from what the container type is).

Look at it like this: static methods don't HAVE a vtable; they're just 
methods that sit somewhere in memory that just happen to be related to a 
certain class.  There's simply no mechanism for looking up derived 
versions at runtime.

So yeah; this same thing caught me once or twice when I first started 
with C++, but I've honestly never been able to come up with a compelling 
reason to do it any other way.

	-- Daniel
Jan 16 2007
parent reply Xinok <xnknet gmail.com> writes:
Simply put, static functions don't have a 'this' pointer, so it's impossible to
make a static function a virtual function.
Objects with virtual functions have a 'hidden ID' which sets a unique value for
each class type (AFAIK anyways...). If you don't have the 'this' pointer, you
can't read the hidden ID in the object, so you don't know what type the object
is, making it impossible to know which virtual function to call.
Jan 16 2007
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Xinok wrote:

 Simply put, static functions don't have a 'this' pointer, so it's impossible
to make a static function a virtual function.
 Objects with virtual functions have a 'hidden ID' which sets a unique value
for each class type (AFAIK anyways...). If you don't have the 'this' pointer,
you can't read the hidden ID in the object, so you don't know what type the
object is, making it impossible to know which virtual function to call.


I think there is a usage case for a function that acts virtual when 
called with an object instance, but static when called with just the 
class name.

It almost seems like you should be able to achieve it by just defining 
two versions of the method:

     foo() { ... }
     static foo() { ... }

If you think of the non-static version as having a hidden 'this' 
parameter, those two are not ambiguous according to standard overloading 
rules.

On the other hand, you can pretty easily just make two versions of the 
function to achieve much the same thing, and just have the virtual 
version call the static version.

     static Foo() {}
     foo() { Foo(); }

Then if you have an object instance you can call inst.foo() and 
otherwise you can call Klass.Foo().

--bb
Jan 16 2007
parent reply John McAuley <john_mcauley bigfoot.com> writes:
Bill Baxter Wrote:



 I think there is a usage case for a function that acts virtual when 
 called with an object instance, but static when called with just the 
 class name.
 


Definitely.


 It almost seems like you should be able to achieve it by just defining 
 two versions of the method:
 
      foo() { ... }
      static foo() { ... }
 
 If you think of the non-static version as having a hidden 'this' 
 parameter, those two are not ambiguous according to standard overloading 
 rules.
 
 On the other hand, you can pretty easily just make two versions of the 
 function to achieve much the same thing, and just have the virtual 
 version call the static version.
 
      static Foo() {}
      foo() { Foo(); }
 
 Then if you have an object instance you can call inst.foo() and 
 otherwise you can call Klass.Foo().
 
 --bb


I've done something similar in c++

class CFoo : public CBase
{
   public:
   static const wchar_t *StaticTag()
   {
      static const wchar_t tag[] = L"foo";
      return tag;
   }
   static CBase *StaticCreate()
   {
      return new CFoo;
   }

   virtual const wchar_t *VirtualTag() { return StaticTag(); }
   virtual CBase VirtualCreate() { return StaticCreate(); }
}

I was reading xml and creating a tree of objects with the aid of a factory that
had an STL map that mapped string to object creator.

You would register with the factory using:
   pFactory->Register(CFoo::StaticTag(), CFoo::StaticCreator);

I had a GUI that made new nodes like this:

   CFoo *pFoo = CFoo::StaticCreator();

So far we are using statics, but when I want to write the tree back to xml I
need virtuals.

void WriteXml(CBase *pRoot)
{
   printf("<%s>\n", pRoot->VirtualTag());

   // foreach child of pRoot
   {
      WriteXml(child);
   }

   printf("<\\%s>\n", pRoot->VirtualTag());
}

It worked but I had about 30 node types in a single schema and I had several
schemas. I don't like the extra baggage of declaring separate static and
virtual versions just because the language doesn't have virtual statics. (Evil
C style macros to the rescue :-) )
Jan 17 2007
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
John McAuley wrote:

 Bill Baxter Wrote:
 
 

 I think there is a usage case for a function that acts virtual when 
 called with an object instance, but static when called with just the 
 class name.


 
 Definitely.
 

 It almost seems like you should be able to achieve it by just defining 
 two versions of the method:

      foo() { ... }
      static foo() { ... }

 If you think of the non-static version as having a hidden 'this' 
 parameter, those two are not ambiguous according to standard overloading 
 rules.

 On the other hand, you can pretty easily just make two versions of the 
 function to achieve much the same thing, and just have the virtual 
 version call the static version.

      static Foo() {}
      foo() { Foo(); }

 Then if you have an object instance you can call inst.foo() and 
 otherwise you can call Klass.Foo().

 --bb


 
 I've done something similar in c++
 
 class CFoo : public CBase
 {
    public:
    static const wchar_t *StaticTag()
    {
       static const wchar_t tag[] = L"foo";
       return tag;
    }
    static CBase *StaticCreate()
    {
       return new CFoo;
    }
 
    virtual const wchar_t *VirtualTag() { return StaticTag(); }
    virtual CBase VirtualCreate() { return StaticCreate(); }
 }
 
 I was reading xml and creating a tree of objects with the aid of a factory
that had an STL map that mapped string to object creator.
 
 You would register with the factory using:
    pFactory->Register(CFoo::StaticTag(), CFoo::StaticCreator);
 
 I had a GUI that made new nodes like this:
 
    CFoo *pFoo = CFoo::StaticCreator();
 
 So far we are using statics, but when I want to write the tree back to xml I
need virtuals.
 
 void WriteXml(CBase *pRoot)
 {
    printf("<%s>\n", pRoot->VirtualTag());
 
    // foreach child of pRoot
    {
       WriteXml(child);
    }
 
    printf("<\\%s>\n", pRoot->VirtualTag());
 }
 
 It worked but I had about 30 node types in a single schema and I had several
schemas. I don't like the extra baggage of declaring separate static and
virtual versions just because the language doesn't have virtual statics. (Evil
C style macros to the rescue :-) )


Seems like it should be possible for the compiler to realize when you're 
not using 'this' in a method and create the vtable entry for the 
function sans hidden 'this' parameter.  Then it could make such a 
function callable via either Class.method or instance.method syntax.

--bb
Jan 17 2007
parent John McAuley <john_mcauley bigfoot.com> writes:
Bill Baxter Wrote:


 
 Seems like it should be possible for the compiler to realize when you're 
 not using 'this' in a method and create the vtable entry for the 
 function sans hidden 'this' parameter.  Then it could make such a 
 function callable via either Class.method or instance.method syntax.
 
 --bb


Pretty much.

Though I don't think that this pointers appear anywhere in the vtable. I
suspect that the vtable is a plain old fashioned array of addresses living
somewhere in read only memory. One address per func, just 1s and zeros.

The compiler would have written references, resolved by the linker, fixed up to
absolute addresses by the OS's app/module loader.

The this pointer will probably be passed to instance funcs via the stack frame.

You write:

   class CFoo
   {
      int m_i;
      void func() { m_i = 2; }
   }

the compiler emits:

   void name_mangled_func(CFoo sneakyThis)
   {
      sneakyThis.m_i = 2;
   }

and:

   foo.func() is emited as:

   func_address = find_address_of_func(foo, funcIndex)
   func_address(foo)

I don't know exactly what the d compiler is doing but I suspect that its
something like the above. find_address_of_func is probably inline pointer
dereferencing.

Soooo... for statics

   class CFoo
  {
      static int s_i;
      int m_i;

      static void func()
      {
         s_i = 4;
      }
   }

the compiler emits:

   void name_mangled_func()
   {
      // s_i is a single var somewhere in memory
      // don't need this pointers to find it
      //
      s_i = 4; 
   }

and:

   foo.func() is emited as:

   func_address = find_address_of_func(foo, funcIndex)
   func_address()

So, as you can see, its just the 'line' that calls the function that changes.
func_address() instead of func_address(foo)

-Disclaimer

  I don't actually KNOW what the d compiler is doing. This is all quesswork. It
would be great if a d compiler expert could chip in to say if its possible or
not from an implementation standpoint.
Jan 17 2007
prev sibling parent reply John McAuley <john_mcauley bigfoot.com> writes:
Xinok Wrote:


 Simply put, static functions don't have a 'this' pointer, so it's impossible
to make a static function a virtual function.
 Objects with virtual functions have a 'hidden ID' which sets a unique value
for each class type (AFAIK anyways...). If you don't have the 'this' pointer,
you can't read the hidden ID in the object, so you don't know what type the
object is, making it impossible to know which virtual function to call.


That doesn't sound right.

For example:

   CBase *baseRef = new CDerived;

   baseRef.hello(); // hello is a virtual static

I HAVE an object reference/pointer, its baseRef.

The compiler knows the definition of the CBase class. So it knows the vtable
offset and the fact that hello is static.

The compiler can emit code that dereferences baseRef to get the CDerived
vtable, gets the function address and calls it without passing baseRef as the
'hidden this pointer'. The code calls the func as a static member and not as an
instance member.

Voila! virtual static call.

Have I missed something? Should I slink away all embarrassed? :-)
Jan 17 2007
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
John McAuley wrote:

 Xinok Wrote:
 

 Simply put, static functions don't have a 'this' pointer, so it's impossible
to make a static function a virtual function.
 Objects with virtual functions have a 'hidden ID' which sets a unique value
for each class type (AFAIK anyways...). If you don't have the 'this' pointer,
you can't read the hidden ID in the object, so you don't know what type the
object is, making it impossible to know which virtual function to call.


 
 That doesn't sound right.
 
 For example:
 
    CBase *baseRef = new CDerived;
 
    baseRef.hello(); // hello is a virtual static
 
 I HAVE an object reference/pointer, its baseRef.
 
 The compiler knows the definition of the CBase class. So it knows the vtable
offset and the fact that hello is static.
 
 The compiler can emit code that dereferences baseRef to get the CDerived
vtable, gets the function address and calls it without passing baseRef as the
'hidden this pointer'. The code calls the func as a static member and not as an
instance member.
 
 Voila! virtual static call.
 
 Have I missed something? Should I slink away all embarrassed? :-)
 


The problem is that static methods generally don't have offsets in the 
vtable at all, so getting the CDerived vtable doesn't help.

--bb
Jan 17 2007
parent reply John McAuley <john_mcauley bigfoot.com> writes:
Bill Baxter Wrote:


 The problem is that static methods generally don't have offsets in the 
 vtable at all, so getting the CDerived vtable doesn't help.
 
 --bb


Change the compiler to write the offsets.
Jan 17 2007
parent reply James Dennett <jdennett acm.org> writes:
John McAuley wrote:

 Bill Baxter Wrote:
 

 The problem is that static methods generally don't have offsets in the 
 vtable at all, so getting the CDerived vtable doesn't help.

 --bb


 
 Change the compiler to write the offsets.


And then what happens when one virtual static calls
another?  There's no "this" pointer in the first,
so it can't use virtual lookup to find the second,
and so it suddenly uses non-virtual lookup as soon
as we insert a trivial forwarding function.  Not
good.  The idea of virtual dispatch is very much
tied to there being an object; the idea of static
methods is very much about there *not* being an
object.  Attempts to make them meet in the middle
cause unpleasant side-effects, and in my opinion
aren't worthwhile.  This has been discussed in the
context of C++ many times, and the issues are the
same as with D in this case.  Even overloading
static with non-static methods is somewhat
problematic, and not because of implementation
issues.

-- James
Jan 17 2007
parent John McAuley <john_mcauley bigfoot.com> writes:
James Dennett Wrote:


 John McAuley wrote:

 Bill Baxter Wrote:
 

 The problem is that static methods generally don't have offsets in the 
 vtable at all, so getting the CDerived vtable doesn't help.

 --bb


 
 Change the compiler to write the offsets.


 
 And then what happens when one virtual static calls
 another?  There's no "this" pointer in the first,
 so it can't use virtual lookup to find the second,
 and so it suddenly uses non-virtual lookup as soon
 as we insert a trivial forwarding function.  Not
 good.  The idea of virtual dispatch is very much
 tied to there being an object; the idea of static
 methods is very much about there *not* being an
 object.  Attempts to make them meet in the middle
 cause unpleasant side-effects, and in my opinion
 aren't worthwhile.  This has been discussed in the
 context of C++ many times, and the issues are the
 same as with D in this case.  Even overloading
 static with non-static methods is somewhat
 problematic, and not because of implementation
 issues.
 
 -- James


Is this what you mean?

   class Cbase
   {
      static bool isLessThanILike(int x)
      {
         return test(x);
      }
      static bool test(int x)
      {
         return x < 55;
      }
   }

   class CFoo : CBase
   {
      static bool test(int x)
      {
         return x < 2;
      }
   }

   CBase baseRef = new CFoo;

   if (baseRef.isLessThanILike(20))
   {
      printf("Well I never expected that!\n");
   }
   else
   {
      printf("Am I a god of programming or what?\n");
   }

I think I've cleverly made CFoo less discriminating by overriding test(). D
finds CFoo's vtable, gets the address of CBase.isLessThanILike which contains a
hard coded call to CBase.test.

Ouch!

OTOH

The call to test was visibly lacking in an object ref so 'of course' it wasn't
virtual. 

But maybe we are so used to having 'invisible this pointers' around that we'd
make that mistake quite often.
Jan 17 2007
prev sibling parent reply Boris Kolar <boris.kolar globera.com> writes:
== Quote from John McAuley (john_mcauley bigfoot.com)'s article

 Xinok Wrote:

 Simply put, static functions don't have a 'this' pointer, so it's impossible




to make a static function a virtual function.

 Objects with virtual functions have a 'hidden ID' which sets a unique value




for each class type (AFAIK anyways...). If you don't have the 'this' pointer,
you
can't read the hidden ID in the object, so you don't know what type the object
is,
making it impossible to know which virtual function to call.

 That doesn't sound right.
 For example:
    CBase *baseRef = new CDerived;
    baseRef.hello(); // hello is a virtual static
 I HAVE an object reference/pointer, its baseRef.
 The compiler knows the definition of the CBase class. So it knows the vtable


offset and the fact that hello is static.

 The compiler can emit code that dereferences baseRef to get the CDerived
vtable,


gets the function address and calls it without passing baseRef as the 'hidden
this
pointer'. The code calls the func as a static member and not as an instance
member.

 Voila! virtual static call.
 Have I missed something? Should I slink away all embarrassed? :-)


If it worked the way you describe, it would be a little bit slower. The way it
works now,
compiler doesn't need to use object reference/pointer at all. It works just as
if
you would
type "CBase.hello()" (and note that "." does not mean hello is invoked on an
instance, it's
just as if "CBase.hello" was a name of a global function).
Jan 17 2007
next sibling parent reply John McAuley <john_mcauley bigfoot.com> writes:
Boris Kolar Wrote:


 If it worked the way you describe, it would be a little bit slower. The way it
 works now,
 compiler doesn't need to use object reference/pointer at all. It works just as
if
 you would
 type "CBase.hello()" (and note that "." does not mean hello is invoked on an
 instance, it's
 just as if "CBase.hello" was a name of a global function).


I'm not saying deprecate calling via classname.funcname. Keep it. 

There is absolutley no reason to get rid of it, and plenty of reasons to keep
it.

Its just that if the programmer chooses to call via an object reference then
the call should be virtual.

Also:

   class CBase
   {
      static void hello() { printf("hello from CBase"); }
   }

   class CFoo : Cbase
   {
      static void hello() { printf("hello from CFoo"); }
   }

   void function() fn;

   fn = &CBase.hello; // legal, will print hello from CBase if called

   fn = &Cfoo.hello // legal, hello from CFoo

   CBase objectRef = new CFoo;

   // legal, because compiler emits code to chase down address at
   // run time.
   // address has same type as fn
   //
   fn = &objectRef.hello; // calling fn would print hello from CFoo

I'm not worried about any speed issues. If I want to call as fast as possible
or I want to be SURE that I call the CBase version then I'd use CBase.hello();. 

If I want virtual goodness then I use objectRef.hello();

The programmer has the choice of how to call.
Jan 17 2007
parent Leandro Lucarella <llucarella integratech.com.ar> writes:
John McAuley escribió:

 Boris Kolar Wrote:
 

 If it worked the way you describe, it would be a little bit slower. The way it
 works now,
 compiler doesn't need to use object reference/pointer at all. It works just as
if
 you would
 type "CBase.hello()" (and note that "." does not mean hello is invoked on an
 instance, it's
 just as if "CBase.hello" was a name of a global function).


 
 I'm not saying deprecate calling via classname.funcname. Keep it. 


[snip]


 The programmer has the choice of how to call.


BTW, instance methods are all virtual (except if the compiler thinks it 
can be optimized) and nobody is complaining about that. I don't see why 
this couldn't be true for static methods too.

Even so, you can use the final to indicate you don't want it to be 
virtual explicitly.

-- 
Leandro Lucarella
Integratech S.A.
4571-5252
Jan 17 2007
prev sibling parent reply John McAuley <john_mcauley bigfoot.com> writes:
Sorry guys for not actually *listening* to your replies.

I inherited someone else's thread by replying to an old post.

The gist of his question was: why is it not *currently possible* to have
virtual statics?

I got a lot of informative and factual responses. Thanks guys.

Of course, I was going off on one, imagining how easy it would be to implement
virtual statics. So I didn't listen properly to the responses.

But moving on, I'd like to change the question to: 

 Do you think that it would be possible to implement virtual statics in D?
Jan 17 2007
parent reply James Dennett <jdennett acm.org> writes:
John McAuley wrote:

 Sorry guys for not actually *listening* to your replies.
 
 I inherited someone else's thread by replying to an old post.
 
 The gist of his question was: why is it not *currently possible* to have
virtual statics?
 
 I got a lot of informative and factual responses. Thanks guys.
 
 Of course, I was going off on one, imagining how easy it would be to implement
virtual statics. So I didn't listen properly to the responses.
 
 But moving on, I'd like to change the question to: 
 
  Do you think that it would be possible to implement virtual statics in D?


Quite possible, given one of many possible specifications;
the difficulty is in making such a specification useful
rather than brittle and confusing.  The difficulties with
the concept of virtual statics *are* conceptual, not
implementation issues.

If D has one advantage over C++, it is in its relative
simplicity.  Bogging it down with this kind of conceptual
murkiness won't help it as a language, IMO.

-- James
Jan 17 2007
parent Benji Smith <dlanguage benjismith.net> writes:
James Dennett wrote:

 Quite possible, given one of many possible specifications;
 the difficulty is in making such a specification useful
 rather than brittle and confusing.  The difficulties with
 the concept of virtual statics *are* conceptual, not
 implementation issues.
 
 If D has one advantage over C++, it is in its relative
 simplicity.  Bogging it down with this kind of conceptual
 murkiness won't help it as a language, IMO.


Good points.

However, one significant benefit that would come from virtual statics 
would be the ability to add static methods to interfaces (I can't tell 
you how often I've wanted to define a static method in an interface!)

And I think the implementation could be pretty straightforward:

1) Instead of a this pointer, static methods always get a ClassInfo pointer.

2) Each ClassInfo would have to put static methods into a StaticVTable, 
rather than just calling those methods by offset.

Personally, I'd be in favor of it. I think it'd be pretty useful.

--benji
Jan 17 2007
prev sibling next sibling parent Marcio <mqmnews123 sglebs.com> writes:
John McAuley wrote:

 I was going to ask why they weren't virtual.


	In Smalltalk they are called class methods and they are virtual.

	| p |
	p := 2 3. "an instance of Point"
	p printString. "how a point instance prints itself"
	Point printString. "how the Point class prints itself"


	#printString is sent to class Point on the last line. But it is not 
defined there, so the method lookup/dispatch is done by the VM as usual. 
You get "virtual static".

	This is possible because classes are 1st class citizens at runtime, and 
therefore they are objects too. "Eat your own dog food" - the system is 
implemented in itself. This means that classes have their own classes 
(metaclasses) which hold their MethodDictionary (vtable if you will - 
which, by the way, is an object as well!) and the system is totally 
uniform. In other words,

	p class. "will return Point, the class of all Points"
	Point class. "will return Point's class, a metaclass"


	The system is not infinite (meta meta meta meta ...) because at some 
point the system has a beautiful strange loop. Many such strange loops 
exist in other areas, and if you like a mind bender that covers this 
obscure art of strange loops, see "Godel, Escher, Bach: An Eternal 
Golden Braid" by Douglas R. Hofstadter (ISBN-10: 0465026567).

	It's very useful and powerful to have class methods be virtual as well.
	
marcio
Jan 17 2007
prev sibling parent Sebastian Biallas <groups.5.sepp spamgourmet.com> writes:
John McAuley wrote:

 Are statics non-virtual because that's the way c++ does it?
 
 Surely the compiler can see base.hello() and then:
 
 * look up the CBase class definition to find hello
 * get its index in the vtable and note that it is static
 * dereference base to get the CDerived vtable
 * get the address that corresponds to hello
 * call hello without passing base as a secret this pointer


What if hello want to call another "static virtual" function? This
wouldn't work without a this pointer.
Jan 17 2007