• Marco Leise (25/25) Nov 10 2014 I know the runtime is supposed to work it all out, call
• Steven Schveighoffer (30/53) Nov 10 2014 I don't know about your specific issue. But I do know how the runtime
• Marco Leise (22/60) Nov 11 2014 Thank you for the explanation. I guess I'm dealing with case
• Marco Leise (2/2) Nov 11 2014 Filed as:

Marco Leise <Marco.Leise gmx.de> writes:

I know the runtime is supposed to work it all out, call
thread module dtors, then shared dtors and then terminate.

But now I have a case where it is broken and I don't know if
we expect the runtime to figure it out, especially when it
comes to separate compilation and such. Basically on one end
I have a singleton template that uses a `shared static ~this()`
to destroy the object. On the other end I have a global list
of reference counted resources, also heavily templated and
relying on a `shared static ~this()` to free the memory
associated with the ref counts etc.
When I build something new from these parts, like a singleton
object that contains on of these ref counted resources, the
runtime is unable to figure out that it has to destroy the
singleton (and its resource with it), before it can call the
destructor of the resource list. For now I just wrote
`import Lib.Sys.Resource;` into the singleton destructor
(i.e. every singleton imports it, whether it needs it or not),
but that doesn't scale obviously.

Should the runtime be able to reliably figure out even such
tough cases? The alternative, disallowing static dtors in
templates isn't appealing.

Out of curiosity: What module do templated static dtors belong
to anyways. And how does that effect when they are called?

-- 
Marco

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 11/10/14 9:57 AM, Marco Leise wrote:
 I know the runtime is supposed to work it all out, call
 thread module dtors, then shared dtors and then terminate.

 But now I have a case where it is broken and I don't know if
 we expect the runtime to figure it out, especially when it
 comes to separate compilation and such. Basically on one end
 I have a singleton template that uses a `shared static ~this()`
 to destroy the object. On the other end I have a global list
 of reference counted resources, also heavily templated and
 relying on a `shared static ~this()` to free the memory
 associated with the ref counts etc.
 When I build something new from these parts, like a singleton
 object that contains on of these ref counted resources, the
 runtime is unable to figure out that it has to destroy the
 singleton (and its resource with it), before it can call the
 destructor of the resource list. For now I just wrote
 `import Lib.Sys.Resource;` into the singleton destructor
 (i.e. every singleton imports it, whether it needs it or not),
 but that doesn't scale obviously.

 Should the runtime be able to reliably figure out even such
 tough cases? The alternative, disallowing static dtors in
 templates isn't appealing.

 Out of curiosity: What module do templated static dtors belong
 to anyways. And how does that effect when they are called?

I don't know about your specific issue. But I do know how the runtime 
calls static ctors/dtors because I rewrote that part a few years ago.

First, any ctors/dtors in a specific module are called in the order they 
appear in the file.

Second, the compiler records 3 things about a module:
1. Is it a standalone module? That is, does it not import any other modules.
2. If not, what modules does it import.
3. Does it have any dtors or ctors (it has a flag for each kind)

When run, the runtime uses this information to build a graph of the 
ordering to call the static ctors. If it detects any cycles in that 
graph, where module A imports directly or indirectly module B, and 
module B imports directly or indirectly module A, and both of them have 
the *same kind* of static ctor or dtor, then it errors and refuses to 
run the program (I'm sure all of you have seen this error).

Now, if that doesn't happen, it has a list of modules, of what imports 
whatever else. And it can call the static ctors in the order of that 
graph making sure dependent modules are constructed first. It does this 
separately for shared and unshared ctors/dtors. Standalone modules are 
called first, and are not included in the search for cycles.

On program/thread termination, it calls the dtors in the opposite order.

Two things that it CANNOT detect:

1. If you circumvent the module system to call functions in static 
ctor/dtors using extern(C).
2. If you establish any dependencies during runtime, such as pushing 
into module A's global array a reference to a module B global object, 
without module A or B importing each other. It won't know to call module 
A's dtor first.

I'm guessing the latter is what you are having issues with.

-Steve

Marco Leise <Marco.Leise gmx.de> writes:

Am Mon, 10 Nov 2014 10:06:01 -0500
schrieb Steven Schveighoffer <schveiguy yahoo.com>:

 I don't know about your specific issue. But I do know how the runtime 
 calls static ctors/dtors because I rewrote that part a few years ago.
 
 First, any ctors/dtors in a specific module are called in the order they 
 appear in the file.
 
 Second, the compiler records 3 things about a module:
 1. Is it a standalone module? That is, does it not import any other modules.
 2. If not, what modules does it import.
 3. Does it have any dtors or ctors (it has a flag for each kind)
 
 When run, the runtime uses this information to build a graph of the 
 ordering to call the static ctors. If it detects any cycles in that 
 graph, where module A imports directly or indirectly module B, and 
 module B imports directly or indirectly module A, and both of them have 
 the *same kind* of static ctor or dtor, then it errors and refuses to 
 run the program (I'm sure all of you have seen this error).
 
 Now, if that doesn't happen, it has a list of modules, of what imports 
 whatever else. And it can call the static ctors in the order of that 
 graph making sure dependent modules are constructed first. It does this 
 separately for shared and unshared ctors/dtors. Standalone modules are 
 called first, and are not included in the search for cycles.
 
 On program/thread termination, it calls the dtors in the opposite order.
 
 Two things that it CANNOT detect:
 
 1. If you circumvent the module system to call functions in static 
 ctor/dtors using extern(C).
 2. If you establish any dependencies during runtime, such as pushing 
 into module A's global array a reference to a module B global object, 
 without module A or B importing each other. It won't know to call module 
 A's dtor first.
 
 I'm guessing the latter is what you are having issues with.
 
 -Steve

Thank you for the explanation. I guess I'm dealing with case
2. But then, how is D supposed to deal with this situation?
Naturally the concepts of "external resources" and "singleton"
are orthogonal and the two modules _should_ not know about
each other.
Granted, a D singleton class is never going to be part of an
external system resource ;), but building on the idea,
whenever we have static dtors in templates, we risk that the
template arguments introduce a "hidden" dependency. The type
is `T` and Singleton.d does not import the module that `T`
came from, which would transitively import Resource.d and make
sure static dtors in there are deferred until after
Singleton.d's dtors were called.

Effectively templated module dtors are crashes waiting to be
triggered!

The compiler must add any modules as dependencies that are
passed in through template arguments as if the concrete type
was directly used with any required import.

-- 
Marco

Marco Leise <Marco.Leise gmx.de> writes:

Filed as:
https://issues.dlang.org/show_bug.cgi?id=13712