www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - __typeid

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Progress is moving along nicely with __typeid. Currently all types aside 
from `struct`s, `class`es and `interface`s are supported:

https://github.com/dlang/druntime/pull/3174

I don't see major issues ahead, so __typeid!(T) should supplant the 
functionality provided by typeid(T) with no or little magic needed on 
the compiler side. It would allow deleting a gnarly part of the compiler 
- so convoluted, in fact, that Walter himself gave up on modularizing it 
and decided to wait for __typeid to be finished so he throws all that 
away in one fell swoop. That's going to be fun.

Reading a bit more about TypeInfo and related stuff, I figure (and 
correct me if I'm wrong) that the entire machinery is currently used for 
the following purposes (only):

* Built-in associative arrays (which ought to be phased out)

* Inform the GC of what it needs to scan (provided in an awkward manner 
though, this is subject to a later discussion)

* Dynamic cast information for class objects

* Object creation for Object.factory() (which also should be phased out)

The API provided by TypeInfo has evolved with these uses in mind, so the 
bulk of it is concerned with primitives for associative arrays such as 
comparison, hashing, and swapping.

The more exciting part is not just replacing the existing functionality, 
but taking it forward. In the future:

* Built-in associative arrays should be templated and make no use of 
typeid at all

* The GC interface should be done through functions, not data 
structures. __typeid!(T) would provide a function scan(T*) that scans 
the given object and all pointers it embeds, transitively. That is easy 
to implement in an efficient manner because the static information about 
T (and therefore its .tupleof) is available when the function is 
generated. Templates, wheee!

* Dynamic cast information for class objects should be kept, perhaps 
made more efficiently than a linear search.

* Object creation primitives should be only provided in an opt-in 
manner. If a class hierarchy wants to expose a factory function, all it 
needs to do is inherit a tag interface:

interface DynamicallyConstructible {}

During the generation of __typeid, the inheritance of this tag is 
detected and the appropriate construction code is generated.

* An important use of __typeid will be to implement Variant "the right 
way". Using the __typeid instead of the unsightly pointer to handler 
function in Variant would go a long way toward simplifying it. the 
__typeid API would be therefore geared toward the needs of that type. 
I've come to the realization that that type is essential for the use of 
D in dynamic contexts. I'd name that type Box, put it in druntime, and 
make it available to Das Besser C.

* Once Box is present, much more is possible. It's not difficult to do 
introspection during compilation on a class object and expose its 
methods at runtime. We'd do that only if another tag interface is 
inherited, again in an opt-in manner:

interface DynamicallyInvokable {}

So it becomes possible to expose objects at runtime, making uses like 
this possible:

// Inside the library
class Widget : DynamicallyInvokable
{
     int frobnicate(double a, int b) { ... }
}

// Application that does NOT import the widget module
// Load the typeid(Widget) from a dynamic library
TypeInfo ti = loadType("./mylibrary.so", "widget.Widget");
// Make a widget. Note that we do NOT have the Widget class definition!
Object w = ti.make();
// Call Widget.frobnicate an get its result
Box result = ti.invoke(w, "frobnicate", 3.14, 42);
// The Box contains an int
int x = result.get!int;

All of this is well within reach, but it's a fair amount of work.
Aug 01 2020
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
Oh good so we are finally tackling runtime reflection-esque capabilities!

Obvious concern of mine: how it handles shared library support including 
when unloading.
Aug 01 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/1/20 12:23 PM, rikki cattermole wrote:

 Oh good so we are finally tackling runtime reflection-esque capabilities!
 
 Obvious concern of mine: how it handles shared library support including 
 when unloading.


In an ideal world the GC would automatically close libraries of which 
code is no longer used. I don't know if that's possible.
Aug 01 2020
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 02/08/2020 6:23 AM, Andrei Alexandrescu wrote:

 On 8/1/20 12:23 PM, rikki cattermole wrote:

 Oh good so we are finally tackling runtime reflection-esque capabilities!

 Obvious concern of mine: how it handles shared library support 
 including when unloading.


 
 In an ideal world the GC would automatically close libraries of which 
 code is no longer used. I don't know if that's possible.


globalVar = new SharedLibThingy;

unload(sharedLib);

// ugh oh globalVar still exists!

Something I was thinking about this, would be a kind of 'audit pass' 
that the GC could offer. Maybe even nullifying references that it finds.
Aug 01 2020
prev sibling next sibling parent Bruce Carneal <bcarneal gmail.com> writes:
On Saturday, 1 August 2020 at 16:12:58 UTC, Andrei Alexandrescu 
wrote:

 Progress is moving along nicely with __typeid. Currently all 
 types aside from `struct`s, `class`es and `interface`s are 
 supported:

 [...]


I'm also interested in the runtime implications but I'm even 
happier with the compile-time ramifications.  Knocking back 
complexity while boosting capability?  You betcha!

Judging from the issues forum Walter and the other front-enders 
have been fighting a rising tide of ICEs.  Sounds like this will 
help a bunch.  Thanks.
Aug 01 2020
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2020-08-01 18:12, Andrei Alexandrescu wrote:

 Progress is moving along nicely with __typeid.


I recommend __typeid to be private and have the compiler lowering to 
__traits(getMember, object, "__typeid") to bypass private. This is to 
make sure this symbol is not accessed directly.

-- 
/Jacob Carlborg
Aug 01 2020
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/1/20 1:29 PM, Jacob Carlborg wrote:

 On 2020-08-01 18:12, Andrei Alexandrescu wrote:

 Progress is moving along nicely with __typeid.


 
 I recommend __typeid to be private and have the compiler lowering to 
 __traits(getMember, object, "__typeid") to bypass private. This is to 
 make sure this symbol is not accessed directly.


That's related to the matter of immutability. Currently all __typeid 
objects are immutable and live in static read-only storage. That's nice 
and would make it impossible to mess with them. However, that makes them 
very unpleasant to use, e.g. you can't use them as out parameters, can't 
assign them etc. All that old story with tail const.
Aug 01 2020
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 1 August 2020 at 18:26:54 UTC, Andrei Alexandrescu 
wrote:

 However, that makes them very unpleasant to use, e.g. you can't 
 use them as out parameters, can't assign them etc.


It seems to me that immutable out params SHOULD work - it would 
follow the same rules as initializing an immutable class member 
in a constructor.

An out param is only actually written once, so the implementation 
just needs to realize it is construction, not assignment, and 
then perhaps it can be made to work.

I suppose the rule would be an out param may be declared before 
use (indeed, it must be), but then it could not actually be 
initialized or assigned outside the function.


void foo(out immutable T t) {
    t = new immutable T; // OK, initial construction
}

immutable T a;

foo(a); // if a was already initialized, this would be an error.
// but if not it should allow it



We do something similar for classes so it seems doable in theory.


  * * *

Of course we should just have const(Object) ref.
Aug 01 2020
prev sibling parent Bruce Carneal <bcarneal gmail.com> writes:
On Saturday, 1 August 2020 at 18:26:54 UTC, Andrei Alexandrescu 
wrote:

 On 8/1/20 1:29 PM, Jacob Carlborg wrote:

 On 2020-08-01 18:12, Andrei Alexandrescu wrote:

 Progress is moving along nicely with __typeid.


 
 I recommend __typeid to be private and have the compiler 
 lowering to __traits(getMember, object, "__typeid") to bypass 
 private. This is to make sure this symbol is not accessed 
 directly.


 That's related to the matter of immutability. Currently all 
 __typeid objects are immutable and live in static read-only 
 storage. That's nice and would make it impossible to mess with 
 them. However, that makes them very unpleasant to use, e.g. you 
 can't use them as out parameters, can't assign them etc. All 
 that old story with tail const.


That unpleasantness, and the attendant complexities of the 
current workarounds, motivates Stefan's type function work.  I 
agree with beerconf Atila, IIUC, that a good solution to this 
will probably involve a generalization to some form of restricted 
type variables.  Perhaps mutable forms that are canonized to 
unique? immutables.
Aug 01 2020
prev sibling next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Saturday, 1 August 2020 at 16:12:58 UTC, Andrei Alexandrescu 
wrote:

 * An important use of __typeid will be to implement Variant 
 "the right way". Using the __typeid instead of the unsightly 
 pointer to handler function in Variant would go a long way 
 toward simplifying it. the __typeid API would be therefore 
 geared toward the needs of that type. I've come to the 
 realization that that type is essential for the use of D in 
 dynamic contexts. I'd name that type Box, put it in druntime, 
 and make it available to Das Besser C.


In other languages, a Box is a wrapper that turns a value type 
into a reference type. [1][2][3] It is not a container for a 
single dynamically-typed value--in fact, its value is typically 
statically-typed.

Most languages do not have a type like the one D currently calls 
Variant, but in those that do, the most widely-accepted name 
seems to be Any. [4][5]

[1] 
https://docs.oracle.com/javase/tutorial/java/data/autoboxing.html
[2] https://doc.rust-lang.org/std/boxed/struct.Box.html
[3] 
https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/types/boxing-and-unboxing
[4] https://en.cppreference.com/w/cpp/utility/any
[5] https://www.scala-lang.org/api/current/scala/Any.html
Aug 01 2020
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/1/20 10:47 PM, Paul Backus wrote:

 On Saturday, 1 August 2020 at 16:12:58 UTC, Andrei Alexandrescu wrote:

 * An important use of __typeid will be to implement Variant "the right 
 way". Using the __typeid instead of the unsightly pointer to handler 
 function in Variant would go a long way toward simplifying it. the 
 __typeid API would be therefore geared toward the needs of that type. 
 I've come to the realization that that type is essential for the use 
 of D in dynamic contexts. I'd name that type Box, put it in druntime, 
 and make it available to Das Besser C.


 
 In other languages, a Box is a wrapper that turns a value type into a 
 reference type. [1][2][3] It is not a container for a single 
 dynamically-typed value--in fact, its value is typically statically-typed.
 
 Most languages do not have a type like the one D currently calls 
 Variant, but in those that do, the most widely-accepted name seems to be 
 Any. [4][5]
 
 [1] https://docs.oracle.com/javase/tutorial/java/data/autoboxing.html
 [2] https://doc.rust-lang.org/std/boxed/struct.Box.html
 [3] 
 https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/types
boxing-and-unboxing 
 
 [4] https://en.cppreference.com/w/cpp/utility/any
 [5] https://www.scala-lang.org/api/current/scala/Any.html


Any is great.
Aug 02 2020
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01.08.20 18:12, Andrei Alexandrescu wrote:

 
 Reading a bit more about TypeInfo and related stuff, I figure (and 
 correct me if I'm wrong) that the entire machinery is currently used for 
 the following purposes (only):
 
 * Built-in associative arrays (which ought to be phased out)
 
 * Inform the GC of what it needs to scan (provided in an awkward manner 
 though, this is subject to a later discussion)
 
 * Dynamic cast information for class objects
 
 * Object creation for Object.factory() (which also should be phased out)


There's also its toString. I have on occasion used 
writeln(typeid(classInstance));
Aug 02 2020
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/2/20 11:34 AM, Timon Gehr wrote:

 On 01.08.20 18:12, Andrei Alexandrescu wrote:

 Reading a bit more about TypeInfo and related stuff, I figure (and 
 correct me if I'm wrong) that the entire machinery is currently used 
 for the following purposes (only):

 * Built-in associative arrays (which ought to be phased out)

 * Inform the GC of what it needs to scan (provided in an awkward 
 manner though, this is subject to a later discussion)

 * Dynamic cast information for class objects

 * Object creation for Object.factory() (which also should be phased out)


 
 There's also its toString. I have on occasion used 
 writeln(typeid(classInstance));


Noted, thanks.
Aug 09 2020
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I implemented as far as I could from the classinfo API:

https://github.com/dlang/druntime/pull/3174

I couldn't get my hands on number of things, such as:

* the vtable (only vptr is accessible, not the size)
* the vtables of implemented interfaces
* the info whether a class is a COM class (I was surprised to learn that 
IUnknown is only defined in Windows-specific modules... it should be 
available to Posix too.
* hasOffTi
Aug 09 2020