www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Destructor called while object is still alive

reply frame <frame86 live.com> writes:
Not sure if this is expected behaviour, but I find this weird:
The destructor is called on the object just because it gets out 
of scope?

I expect that either GC will not touch the object or only if no 
reference is on the object anymore.

If the object is used in the loop, this will not happen but the 
GC or compiler should not decide to kill the object if a valid 
pointer is still in current scope? Even this is a "intelligent" 
feature by the compiler for non reachable code like that loop, 
it's still confusing. There are maybe situations while the object 
should stay in background, eg. socket related/event stuff. If 
there is a valid pointer, there is no excuse to reap it.


class Foo {
     ~this() {
         writefln("Warning: destructor called (%s) on %s", 
typeid(this), this.__vptr);
     }
}

class Bar {
     static Foo create() {
         return new Foo();
     }
}

void main() {
     auto foo = Bar.create();
     writefln("address: %s", foo.__vptr);

     while(true) {
         Thread.sleep(1.seconds);
         GC.collect();
     }
}

// DMD32 v2.094.1 -m64 on windows
Oct 22 2020
next sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
On Fri, Oct 23, 2020 at 8:20 AM frame via Digitalmars-d <
digitalmars-d puremagic.com> wrote:


  Even this is a "intelligent"
 feature by the compiler for non reachable code like that loop,
 it's still confusing. There are maybe situations while the object
 should stay in background, eg. socket related/event stuff. If
 there is a valid pointer, there is no excuse to reap it.


There is no use of foo anywhere so there is no reason to not collect it.
You even does not need to have while in your code:

void main() {
     auto foo = Bar.create();
     writefln("address: %s", foo.__vptr);
     GC.collect();
     Thread.sleep(1.seconds);
     //foo.sayHello(); // if uncommented GC would not call destructor until
end of main
}
Oct 22 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 06:37:43 UTC, Daniel Kozak wrote:

 On Fri, Oct 23, 2020 at 8:20 AM frame via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:


  Even this is a "intelligent"
 feature by the compiler for non reachable code like that loop,
 it's still confusing. There are maybe situations while the 
 object
 should stay in background, eg. socket related/event stuff. If
 there is a valid pointer, there is no excuse to reap it.


 There is no use of foo anywhere so there is no reason to not 
 collect it.


It is in use until the end of main. Think locking. Seems like the 
pointer is optimized away and then the GC collects. Makes RAII 
useless with GC.

Basically a consequence of the optimizer assuming that there is 
no GC. A rather serious correctness issue.
Oct 22 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/20 2:57 AM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 06:37:43 UTC, Daniel Kozak wrote:

 On Fri, Oct 23, 2020 at 8:20 AM frame via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:


  Even this is a "intelligent"
 feature by the compiler for non reachable code like that loop,
 it's still confusing. There are maybe situations while the object
 should stay in background, eg. socket related/event stuff. If
 there is a valid pointer, there is no excuse to reap it.


 There is no use of foo anywhere so there is no reason to not collect it.


 
 It is in use until the end of main. Think locking. Seems like the 
 pointer is optimized away and then the GC collects. Makes RAII useless 
 with GC.


But you don't get RAII with classes or pointers. Only with structs.


 
 Basically a consequence of the optimizer assuming that there is no GC. A 
 rather serious correctness issue.
 
 


I don't think so. A class reference itself doesn't have a destructor. 
The compiler is free to elide storage just like it was a pointer. One 
cannot expect any specific behavior from the GC in this case, as the 
correctness of the program doesn't depend on the object staying in place 
for the duration of the function. Here there is just a disconnect in how 
the OP expects the compiler to implement the machine code.

Use a type that *does* support RAII (i.e. struct with dtor), and it will 
be allocated and properly destroyed.

-Steve
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 16:00:58 UTC, Steven Schveighoffer 
wrote:

 But you don't get RAII with classes or pointers. Only with 
 structs.


But that is only a consequence of the optimizer not being written 
with a GC in mind. That is the point. Isn't D structs the same as 
C++ PODs? I thought C++ classes with virtuals/RTTI is supposed to 
map to D classes? Then RAII has to work.


 I don't think so. A class reference itself doesn't have a 
 destructor. The compiler is free to elide storage just like it 
 was a pointer. One cannot expect any specific behavior from the 
 GC in this case, as the correctness of the program doesn't 
 depend on the object staying in place for the duration of the 
 function. Here there is just a disconnect in how the OP expects 
 the compiler to implement the machine code.


That would be very surprising semantics as you expect a pointer 
to be LIVE throughout the scope in which it was instantiated. If 
it is LIVE then it cannot be collected by the GC.

This all boils down to the optimizer being GC ignorant. I 
understand that it is too much work to make the optimizer take 
that into consideration, but it most certainly goes against the 
expected norm for the semantics of BLOCKS that LIVE pointers are 
being collected.
Oct 23 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/20 12:14 PM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 16:00:58 UTC, Steven Schveighoffer wrote:

 But you don't get RAII with classes or pointers. Only with structs.


 
 But that is only a consequence of the optimizer not being written with a 
 GC in mind. That is the point. Isn't D structs the same as C++ PODs? I 
 thought C++ classes with virtuals/RTTI is supposed to map to D classes? 
 Then RAII has to work.


No, D classes map to C++ pointers to classes.

How does C++ RAII work for class pointers that aren't used?


 

 I don't think so. A class reference itself doesn't have a destructor. 
 The compiler is free to elide storage just like it was a pointer. One 
 cannot expect any specific behavior from the GC in this case, as the 
 correctness of the program doesn't depend on the object staying in 
 place for the duration of the function. Here there is just a 
 disconnect in how the OP expects the compiler to implement the machine 
 code.


 
 That would be very surprising semantics as you expect a pointer to be 
 LIVE throughout the scope in which it was instantiated. If it is LIVE 
 then it cannot be collected by the GC.


Expectation for what purpose?

I think of code being executed in the way I write it. We all know that 
optimizers don't do that. As long as it's not changing the semantic 
meaning of the program, I'm fine with it.

If you look at this code:

void main()
{
    int i;
    foo();
}

Does i even exist? Does it exist after the first line? Is it important 
if the compiler allocates space for it on the stack? Does it get 
initialized to 0?

All of these things are items that have no bearing on the execution of 
the program, so it's fine for the optimizer to just completely get rid 
of the variable storage. This is no different.


 
 This all boils down to the optimizer being GC ignorant. I understand 
 that it is too much work to make the optimizer take that into 
 consideration, but it most certainly goes against the expected norm for 
 the semantics of BLOCKS that LIVE pointers are being collected.
 


I think the optimizer is correct. There are no live pointers, because 
the object is not used, ever.

How many times has someone posted code touting a benchmark of something 
and the optimizer gets rid of the entire program? How is this any different?

-Steve
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 16:30:52 UTC, Steven Schveighoffer 
wrote:

 No, D classes map to C++ pointers to classes.


Huh? D has to be able to call virtual destructors for C++ objects.


 How does C++ RAII work for class pointers that aren't used?


What do you mean? The pointer is valid until the end of the 
scope. Then the object is destructed if it is a sole owning 
pointer.

_all_ GC pointers are conceptually owning pointers. They have to 
stay live throughout the whole scope.



 Expectation for what purpose?


Basic CS conventions?



 I think of code being executed in the way I write it. We all 
 know that optimizers don't do that. As long as it's not 
 changing the semantic meaning of the program, I'm fine with it.


But it did! That was the point!



 If you look at this code:

 void main()
 {
    int i;
    foo();
 }

 Does i even exist? Does it exist after the first line? Is it 
 important if the compiler allocates space for it on the stack? 
 Does it get initialized to 0?


What do you mean? There are no side effects? Do D allow side 
effects in destructors? If yes, then you cannot cap the lifetime 
of the object at your own will.


 just completely get rid of the variable storage. This is no 
 different.


It clearly is. The order of the writeln() execution was wrong.


 I think the optimizer is correct. There are no live pointers, 
 because the object is not used, ever.


Of course it is used. It has a destructor with a side effect.
Oct 23 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/20 12:48 PM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 16:30:52 UTC, Steven Schveighoffer wrote:

 No, D classes map to C++ pointers to classes.


 
 Huh? D has to be able to call virtual destructors for C++ objects.


And when are those called? When the lifetime of the *object* is over, 
not the lifetime of the pointer to the object.

My point was that C++ doesn't have class references, they have class 
pointers, which is akin to D class references.


 

 How does C++ RAII work for class pointers that aren't used?


 
 What do you mean? The pointer is valid until the end of the scope. Then 
 the object is destructed if it is a sole owning pointer.


I mean, if I compile this code for C++, does it store the pointer on the 
stack?

void foo()
{
    SomeClass* ptr = new SomeClass();
    ... // bunch of other code that never uses ptr
}

C++ doesn't have automatic management using pointers. So the answer is, 
the optimizer might just not store `ptr`. Just like it doesn't happen in D.


 
 _all_ GC pointers are conceptually owning pointers. They have to stay 
 live throughout the whole scope.


It doesn't have to even have a pointer if it's never used.

However, even though this case isn't "bad", there are bad cases that can 
be a problem:

auto c = storeWithC(new C);

If the function is something that squirrels away the parameter into a 
C-malloc'd block or C global (basically anything that is not scanned by 
the GC), and then returns the parameter, c is still not allocated on the 
stack, and the GC might collect it.

If DMD isn't smart enough to see that the reference may have escaped, 
then it shouldn't elide storage.

Under this view, I think actually the OP's case is a problem. Because 
Bar.create() could potentially be saving the result into a C heap block, 
in which case eliding the pointer storage is potentially going to cause 
memory corruption.

The optimizer is wrong, and should be changed.

Note that LDC does not do this optimization.

-Steve
Oct 23 2020
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Oct 23, 2020 at 01:47:34PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:
[...]

 If DMD isn't smart enough to see that the reference may have escaped,
 then it shouldn't elide storage.
 
 Under this view, I think actually the OP's case is a problem. Because
 Bar.create() could potentially be saving the result into a C heap
 block, in which case eliding the pointer storage is potentially going
 to cause memory corruption.
 
 The optimizer is wrong, and should be changed.
 
 Note that LDC does not do this optimization.


[...]

Yet another reason for me to avoid DMD and use LDC instead...


T

-- 
This sentence is false.
Oct 23 2020
prev sibling next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 17:47:34 UTC, Steven Schveighoffer 
wrote:

 And when are those called? When the lifetime of the *object* is 
 over, not the lifetime of the pointer to the object.


But the owning pointer is keeping the object alive.

A gc pointer is keeping the object alive, and is keeping it alive 
throughout its own lifetime. So yes, the lifetime of the object 
is supposed be at least as long as the owning pointer. And the 
lifetime lasts till the end of the scope. If there is a 
sideeffect then it should be considered to be _live_ until the 
end of the scope.

With the GC semantics you don't get a guarantee for when it will 
be destructed, but you should have strong guarantees for how long 
the lifetime at least can be expected to be.

Ideally you would also have guarantees for eventual destruction 
(i.e. that all objects are destroyed before the program 
terminates). With precise GC sans union, you should be able to 
get that.



 My point was that C++ doesn't have class references, they have 
 class pointers, which is akin to D class references.


Well, "reference" and "pointer" are two words for the same thing, 
although D has some syntactical and semantic restrictions on 
class references.

That is pretty irrelevant though, as D class references with GC 
are owning pointers

C++ have owning pointers named unique_ptr and shared_ptr. Naked 
pointers are borrowing pointers, or for manual management. Forget 
about the mechanisms, conceptually that is the case.



 I mean, if I compile this code for C++, does it store the 
 pointer on the stack?

 void foo()
 {
    SomeClass* ptr = new SomeClass();
    ... // bunch of other code that never uses ptr
 }

 C++ doesn't have automatic management using pointers. So the 
 answer is, the optimizer might just not store `ptr`. Just like 
 it doesn't happen in D.


But C++ compilers don't provide a GC, and naked pointers are not 
owning pointers.



 _all_ GC pointers are conceptually owning pointers. They have 
 to stay live throughout the whole scope.


 It doesn't have to even have a pointer if it's never used.


But a destructor with a side effect IS usage, and it should never 
be executed when there is a conceptually live pointer pointing to 
the object.

It is very simple, really.
Oct 23 2020
prev sibling parent reply frame <frame86 live.com> writes:
On Friday, 23 October 2020 at 17:47:34 UTC, Steven Schveighoffer 
wrote:

 On 10/23/20 12:48 PM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 16:30:52 UTC, Steven 
 Schveighoffer wrote:

 No, D classes map to C++ pointers to classes.







 It doesn't have to even have a pointer if it's never used.
 ...
 The optimizer is wrong, and should be changed.

 Note that LDC does not do this optimization.

 -Steve


So how can I avoid this issue with DMD? Disabling any 
optimization? In fact it does not occur with debug mode enabled.

scope doesn't change a thing.

Actually I do use the pointer in real code, (just one method call 
before a loop but still used).
Oct 23 2020
next sibling parent frame <frame86 live.com> writes:
On Friday, 23 October 2020 at 19:23:03 UTC, frame wrote:

 In fact it does not occur with debug mode enabled.


Correction: Issue disappears with -g flag, still exists with 
-debug flag
Oct 23 2020
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Oct 23, 2020 at 07:23:03PM +0000, frame via Digitalmars-d wrote:

 On Friday, 23 October 2020 at 17:47:34 UTC, Steven Schveighoffer wrote:


[...]

 The optimizer is wrong, and should be changed.
 
 Note that LDC does not do this optimization.




[...]

 So how can I avoid this issue with DMD? Disabling any optimization? In
 fact it does not occur with debug mode enabled.


[...]

IMNSHO, drop dmd and use ldc2 instead.  Life is too short to have to
deal with dmd backend bugs when I'm trying to get things *done*.  This
isn't the first time the dmd backend has problems when optimization /
inlining are enabled, and I fear it won't be the last.


T

-- 
MS Windows: 64-bit rehash of 32-bit extensions and a graphical shell for a
16-bit patch to an 8-bit operating system originally coded for a 4-bit
microprocessor, written by a 2-bit company that can't stand 1-bit of
competition.
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 19:36:38 UTC, H. S. Teoh wrote:

 On Fri, Oct 23, 2020 at 07:23:03PM +0000, frame via 
 Digitalmars-d wrote:

 On Friday, 23 October 2020 at 17:47:34 UTC, Steven 
 Schveighoffer wrote:


 [...]

 The optimizer is wrong, and should be changed.
 
 Note that LDC does not do this optimization.




 [...]

 So how can I avoid this issue with DMD? Disabling any 
 optimization? In fact it does not occur with debug mode 
 enabled.


 [...]

 IMNSHO, drop dmd and use ldc2 instead.  Life is too short to 
 have to deal with dmd backend bugs when I'm trying to get 
 things *done*.  This isn't the first time the dmd backend has 
 problems when optimization / inlining are enabled, and I fear 
 it won't be the last.


LDC is better at detecting unreachable code so it actually does 
worse than DMD.

https://run.dlang.io/is/2U5OgM

LDC -O3 fails
DMD -release succeeds
Oct 23 2020
parent reply Johan Engelen <j j.nl> writes:
On Friday, 23 October 2020 at 21:55:36 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 23 October 2020 at 19:36:38 UTC, H. S. Teoh wrote:

 On Fri, Oct 23, 2020 at 07:23:03PM +0000, frame via 
 Digitalmars-d wrote:

 On Friday, 23 October 2020 at 17:47:34 UTC, Steven 
 Schveighoffer wrote:


 [...]

 The optimizer is wrong, and should be changed.
 
 Note that LDC does not do this optimization.




 [...]

 So how can I avoid this issue with DMD? Disabling any 
 optimization? In fact it does not occur with debug mode 
 enabled.


 [...]

 IMNSHO, drop dmd and use ldc2 instead.  Life is too short to 
 have to deal with dmd backend bugs when I'm trying to get 
 things *done*.  This isn't the first time the dmd backend has 
 problems when optimization / inlining are enabled, and I fear 
 it won't be the last.


 LDC is better at detecting unreachable code so it actually does 
 worse than DMD.


I would certainly expect so. Same for GDC.

We can fix this in LDC (quite easily it looks like. We can force 
LLVM to put&keep certain pointers on the stack), but I first 
would like to see consensus about this issue from the language 
designers and see it in text in the language spec.

-Johan
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 23:33:59 UTC, Johan Engelen wrote:

 We can fix this in LDC (quite easily it looks like. We can 
 force LLVM to put&keep certain pointers on the stack), but I 
 first would like to see consensus about this issue from the 
 language designers and see it in text in the language spec.


Maybe you only need to do it in functions that are marked as not 
returning normally (bottom). It is already implicit in the 
current spec that variables are valid throughout their scope, so 
it is the current behaviour that should be documented as it 
violates static binding principles.
Oct 23 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 24 October 2020 at 06:50:13 UTC, Ola Fosheim Grøstad 
wrote:

 Maybe you only need to do it in functions that are marked as 
 not returning normally (bottom).


That would require that functions are marked as "sometimes does 
not return normally"...

I guess another option is to go over the ssa after optimization 
and do a rerun with spilled pointers to the stack... Spilling all 
potential gc pointers returned from functions to the stack is 
exessive. It would include void pointers etc.
Oct 24 2020
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/20 3:23 PM, frame wrote:

 On Friday, 23 October 2020 at 17:47:34 UTC, Steven Schveighoffer wrote:

 On 10/23/20 12:48 PM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 16:30:52 UTC, Steven Schveighoffer wrote:

 No, D classes map to C++ pointers to classes.






 

 It doesn't have to even have a pointer if it's never used.
 ...
 The optimizer is wrong, and should be changed.

 Note that LDC does not do this optimization.


 
 So how can I avoid this issue with DMD? Disabling any optimization? In 
 fact it does not occur with debug mode enabled.


I don't know if you need to do that. As much discussion as this post has 
generated, and even though the optimization is wrong, the specific 
circumstances that could cause a piece of memory to be collected before 
it's used is really unlikely. To spell it out again:

1. You generate a GC block (i.e. new class)
2. You store it as part of an initializer for a stack local
3. You never use it in the stack frame
4. But you also in that same initialization put it into a location that 
is not scanned by the GC (e.g. C-malloc'd data)

If you aren't doing all 4 of these things, then you don't have to worry.


 Actually I do use the pointer in real code, (just one method call before 
 a loop but still used).
 


So it shouldn't be collected then, even with optimizations.

-Steve
Oct 23 2020
parent reply frame <frame86 live.com> writes:
On Friday, 23 October 2020 at 19:39:52 UTC, Steven Schveighoffer 
wrote:

 On 10/23/20 3:23 PM, frame wrote:

 On Friday, 23 October 2020 at 17:47:34 UTC, Steven 
 Schveighoffer wrote:

 On 10/23/20 12:48 PM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 16:30:52 UTC, Steven 
 Schveighoffer wrote:







 1. You generate a GC block (i.e. new class)
 2. You store it as part of an initializer for a stack local
 3. You never use it in the stack frame
 4. But you also in that same initialization put it into a 
 location that is not scanned by the GC (e.g. C-malloc'd data)

 If you aren't doing all 4 of these things, then you don't have 
 to worry.


 Actually I do use the pointer in real code, (just one method 
 call before a loop but still used).
 


 So it shouldn't be collected then, even with optimizations.

 -Steve


But I doesn't do 3. and it's collected :(
There is no special release optimization, I just call it via rdmd 
defaults.

Even if I do not run into memory problems, calling the destructor 
on a live object is an unepxected side effect which makes the 
dtor concept useless for me. It closes the remote connection the 
object hold (please don't ask about the meaning, it just the way 
it has to work) and this must not happen while the program is 
still running.
Oct 23 2020
next sibling parent reply Johan Engelen <j j.nl> writes:
On Friday, 23 October 2020 at 20:27:37 UTC, frame wrote:

 Even if I do not run into memory problems, calling the 
 destructor on a live object is an unepxected side effect which 
 makes the dtor concept useless for me. It closes the remote 
 connection the object hold (please don't ask about the meaning, 
 it just the way it has to work) and this must not happen while 
 the program is still running.


Hi frame,
   Regardless of the topic being discussed, you should not do 
resource management using ctor/dtor of a GC-managed object. The 
dtor is not guaranteed to be called (not even on program exit). 
Indeed, the dtor as you know from stack allocated objects works 
very differently for GC-allocated objects; this feels like the 
dtor concept is useless (I agree, I've also grown used to 
RAII-thinking about dtors).

cheers,
   Johan
Oct 23 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/20 7:30 PM, Johan Engelen wrote:

 On Friday, 23 October 2020 at 20:27:37 UTC, frame wrote:

 Even if I do not run into memory problems, calling the destructor on a 
 live object is an unepxected side effect which makes the dtor concept 
 useless for me. It closes the remote connection the object hold 
 (please don't ask about the meaning, it just the way it has to work) 
 and this must not happen while the program is still running.


 
 Hi frame,
    Regardless of the topic being discussed, you should not do resource 
 management using ctor/dtor of a GC-managed object. The dtor is not 
 guaranteed to be called (not even on program exit). Indeed, the dtor as 
 you know from stack allocated objects works very differently for 
 GC-allocated objects; this feels like the dtor concept is useless (I 
 agree, I've also grown used to RAII-thinking about dtors).
 


A destructor is literally for resource management. For example, if a 
class represents an OS handle, and that handle is encapsulated, the 
destructor is the right place to clean that up.

If you want it cleaned up synchronously, you explicitly destroy the 
object, or provide a function to clean it up.

But if you don't clean it up, and the GC is cleaning up the object, 
there's no reason to leak the resource.

I wrote a program long ago that used classes to manage open files. I 
never worried about cleaning up the file handles, and it worked fine.

-Steve
Oct 24 2020
next sibling parent Johan Engelen <j j.nl> writes:
On Saturday, 24 October 2020 at 11:56:54 UTC, Steven 
Schveighoffer wrote:

 On 10/23/20 7:30 PM, Johan Engelen wrote:

 On Friday, 23 October 2020 at 20:27:37 UTC, frame wrote:

 Even if I do not run into memory problems, calling the 
 destructor on a live object is an unepxected side effect 
 which makes the dtor concept useless for me. It closes the 
 remote connection the object hold (please don't ask about the 
 meaning, it just the way it has to work) and this must not 
 happen while the program is still running.


 
 Hi frame,
    Regardless of the topic being discussed, you should not do 
 resource management using ctor/dtor of a GC-managed object. 
 The dtor is not guaranteed to be called (not even on program 
 exit). Indeed, the dtor as you know from stack allocated 
 objects works very differently for GC-allocated objects; this 
 feels like the dtor concept is useless (I agree, I've also 
 grown used to RAII-thinking about dtors).
 


 A destructor is literally for resource management. For example, 
 if a class represents an OS handle, and that handle is 
 encapsulated, the destructor is the right place to clean that 
 up.


My point is that the destructor of a GC-managed object may not be 
called at all, hence in general (!) plain GC+destructor is not 
suitable for resource management. If you are trying to manage 
resources that are released automatically (and correctly) by the 
OS upon program termination, you are lucky and can get away with 
it.

-Johan
Oct 24 2020
prev sibling parent reply Kagamin <spam here.lot> writes:
On Saturday, 24 October 2020 at 11:56:54 UTC, Steven 
Schveighoffer wrote:

 I wrote a program long ago that used classes to manage open 
 files. I never worried about cleaning up the file handles, and 
 it worked fine.


Programs that don't care to close file handles are the reason why 
people believe that windows can't run for more than a week 
without restarts, a file handle is small for the program, but 
kernel allocates a fairly big structure about 4kb; when the 
program opens a million handles, it consumes 4gb of kernel 
memory, when a non-technical user sees this system wide memory 
shortage, he concludes time has come to restart the system. 
Particularly Nvidia user service was guilty of this and given 
that it's installed on most machines, you get the corresponding 
result.
Oct 25 2020
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/25/20 5:17 AM, Kagamin wrote:

 On Saturday, 24 October 2020 at 11:56:54 UTC, Steven Schveighoffer wrote:

 I wrote a program long ago that used classes to manage open files. I 
 never worried about cleaning up the file handles, and it worked fine.


 
 Programs that don't care to close file handles are the reason why people 
 believe that windows can't run for more than a week without restarts, a 
 file handle is small for the program, but kernel allocates a fairly big 
 structure about 4kb; when the program opens a million handles, it 
 consumes 4gb of kernel memory, when a non-technical user sees this 
 system wide memory shortage, he concludes time has come to restart the 
 system. Particularly Nvidia user service was guilty of this and given 
 that it's installed on most machines, you get the corresponding result.


This was not on Windows, and the GC cleaned up the file handles as it 
ran. They weren't left open constantly.

But even if you have synchronous management of files, having a 
destructor clean up a file that obviously isn't used any more isn't a 
bad thing.

-Steve
Oct 25 2020
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 25 October 2020 at 15:50:36 UTC, Steven Schveighoffer 
wrote:

 But even if you have synchronous management of files, having a 
 destructor clean up a file that obviously isn't used any more 
 isn't a bad thing.


Do you mean by the GC?

Well, it can be a bad thing if the GC most of the time run 
destructors if you do transactions that complete in destructors 
or have buffers that are flushed and closed in destructors. For 
correctness it is actually better that the GC never run 
destructors. Or... that the language is adjusted so that you get 
precise guarantees.
Oct 25 2020
prev sibling next sibling parent Guillaume Piolat <first.name guess.com> writes:
On Sunday, 25 October 2020 at 15:50:36 UTC, Steven Schveighoffer 
wrote:

 But even if you have synchronous management of files, having a 
 destructor clean up a file that obviously isn't used any more 
 isn't a bad thing.

 -Steve


The real problem with coincidental correctness comes when your 
resource dependency graph gets more complex.

The typical example is:

   1. You've loaded a library upon which all library object 
depend, you get a handle H to be released with releaseH(H). You 
put it in a class.

   2. This handle H allows you to create a library object A, to be 
released with releaseA(H, A). A cannot be released if the library 
has been released. So In A you keep a reference to H so that it's 
not cleared by the GC ; else you can't call releaseA(H, A) in A's 
destructor. The problem is that A is polymorphic, so it can't be 
a struct.

   3. Your A goes out of scope at the end of the program, the GC 
"chooses" to call H destructor first, and then A's destructor, 
where the H member is dead. The user end up with the conclusion: 
"the GC has collected a live object".
      However at first you don't see the bug because up to know 
A's destructor was called first.
Oct 25 2020
prev sibling parent Kagamin <spam here.lot> writes:
On Sunday, 25 October 2020 at 15:50:36 UTC, Steven Schveighoffer 
wrote:

 But even if you have synchronous management of files, having a 
 destructor clean up a file that obviously isn't used any more 
 isn't a bad thing.


Yes, GC environments differentiate between resource management 
and defensive cleanup, the latter is done by 
destructor/finalizer, the former is done synchronously, they also 
run in different environments, so the code is often different too.
Oct 26 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/20 4:27 PM, frame wrote:

 On Friday, 23 October 2020 at 19:39:52 UTC, Steven Schveighoffer wrote:

 On 10/23/20 3:23 PM, frame wrote:

 On Friday, 23 October 2020 at 17:47:34 UTC, Steven Schveighoffer wrote:

 On 10/23/20 12:48 PM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 16:30:52 UTC, Steven Schveighoffer 
 wrote:







 1. You generate a GC block (i.e. new class)
 2. You store it as part of an initializer for a stack local
 3. You never use it in the stack frame
 4. But you also in that same initialization put it into a location 
 that is not scanned by the GC (e.g. C-malloc'd data)

 If you aren't doing all 4 of these things, then you don't have to worry.


 Actually I do use the pointer in real code, (just one method call 
 before a loop but still used).


 So it shouldn't be collected then, even with optimizations.


 But I doesn't do 3. and it's collected :(
 There is no special release optimization, I just call it via rdmd defaults.


Wait, so you use it inside the function after declaring it? And the 
pointer isn't stored in the stack?

This is different from your original code. Can you post an example of 
that happening?


 
 Even if I do not run into memory problems, calling the destructor on a 
 live object is an unepxected side effect which makes the dtor concept 
 useless for me. It closes the remote connection the object hold (please 
 don't ask about the meaning, it just the way it has to work) and this 
 must not happen while the program is still running.


Sure, but if you use it during your function, it should be stored on the 
stack, and therefore not collected.

-Steve
Oct 24 2020
parent frame <frame86 live.com> writes:
On Saturday, 24 October 2020 at 11:52:27 UTC, Steven 
Schveighoffer wrote:

 On 10/23/20 4:27 PM, frame wrote:

 On Friday, 23 October 2020 at 19:39:52 UTC, Steven 
 Schveighoffer wrote:

 On 10/23/20 3:23 PM, frame wrote:

 On Friday, 23 October 2020 at 17:47:34 UTC, Steven 
 Schveighoffer wrote:

 On 10/23/20 12:48 PM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 16:30:52 UTC, Steven 
 Schveighoffer wrote:













 Wait, so you use it inside the function after declaring it? And 
 the pointer isn't stored in the stack?

 This is different from your original code. Can you post an 
 example of that happening?


Sorry, did not notice it.

This is basically the code I posted at begin. A class object 
holds a connection and the whole object can be fetched via a 
static method, returning as class variable. There is no problem 
if the compiler does not detect unreachable code. But for testing 
purposes I just attached a loop below the instantion after 
calling connect() on my new object. After a while, the destructor 
was called while the loop was still running and the object idle 
but alive.

I'm not sure how this have an impact on other scenarios - if the 
object would have started a thread and communicate via socket, 
for example. But I think it would be the same behaviour if the 
compiler want to be smart.

It smells wrong since the object pointer was still valid and 
resides in the current scope and GC should be only allowed to 
purge the object after leaving the scope.

Resource management is no point here. It was just a side effect 
that the connection was closed because the connection was created 
by that class and the complete object was just killed by GC. 
addRoot() also solves the problem for the testcase.

So again, the object was alive and valid - not total out of scope 
or garbage - just unused after that line.
Oct 30 2020
prev sibling next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 19:23:03 UTC, frame wrote:

 So how can I avoid this issue with DMD? Disabling any 
 optimization? In fact it does not occur with debug mode enabled.


A dirty fix is to manually add the pointer as a GC root.
Oct 23 2020
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 19:39:52 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 23 October 2020 at 19:23:03 UTC, frame wrote:

 So how can I avoid this issue with DMD? Disabling any 
 optimization? In fact it does not occur with debug mode 
 enabled.


 A dirty fix is to manually add the pointer as a GC root.


Another fix is to make the compiler think the end is reachable by 
warpping the infinitie loop in an if statements that tests 
something that the compiler cannot prove to be true. E.g.

if (alwaystrue()) inifinteloop
Oct 23 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
Not the prettiest or fastest trick, but the following code will 
prevent the end of the scope being viewed as unreachable. You 
could do a lot better with just loading a 1 into a register using 
machine language. Probably something else you can test in the 
standard library that the compiler does not have access to.


     auto nevernull = malloc(1);
     if (nevernull != null){
         free(nevernull);

         while(1){…};

     }
     // this is no longer viewed as unreachable
Oct 23 2020
prev sibling parent reply frame <frame86 live.com> writes:
On Friday, 23 October 2020 at 19:39:52 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 23 October 2020 at 19:23:03 UTC, frame wrote:

 So how can I avoid this issue with DMD? Disabling any 
 optimization? In fact it does not occur with debug mode 
 enabled.


 A dirty fix is to manually add the pointer as a GC root.


Well that works thanks. But shouldn't the runtime do this 
automatically?
Oct 23 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 20:46:13 UTC, frame wrote:

 On Friday, 23 October 2020 at 19:39:52 UTC, Ola Fosheim Grøstad 
 wrote:

 On Friday, 23 October 2020 at 19:23:03 UTC, frame wrote:

 So how can I avoid this issue with DMD? Disabling any 
 optimization? In fact it does not occur with debug mode 
 enabled.


 A dirty fix is to manually add the pointer as a GC root.


 Well that works thanks. But shouldn't the runtime do this 
 automatically?


No, because it means that the GC will not try to release the 
memory...
Maybe cleaner to use a global variable instead?
Oct 23 2020
prev sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 23 October 2020 at 19:23:03 UTC, frame wrote:

 So how can I avoid this issue with DMD? Disabling any 
 optimization?


You avoid it by correctly designing your code. Resource 
management with class destructors is incorrect design.
Oct 23 2020
parent reply frame <frame86 live.com> writes:
On Saturday, 24 October 2020 at 06:35:00 UTC, Kagamin wrote:

 On Friday, 23 October 2020 at 19:23:03 UTC, frame wrote:

 So how can I avoid this issue with DMD? Disabling any 
 optimization?


 You avoid it by correctly designing your code. Resource 
 management with class destructors is incorrect design.


Yeah, I understand but this doesn't help if resources are wiped 
when the object gets destroyed too. I have no problem with that 
GC may not call the destructor - I have a problem if the GC call 
the destructor out of the wild. It's also ok that the 
optimizations are done in my point of view but as a newbie, for 
me this should be well documented somewhere.
Oct 24 2020
parent Kagamin <spam here.lot> writes:
On Saturday, 24 October 2020 at 09:02:41 UTC, frame wrote:

 Yeah, I understand but this doesn't help if resources are wiped 
 when the object gets destroyed too.


Correct design solves your problem. If you rely exclusively on GC 
for resource management, this scenario is not supported, because 
it's unworkable.
Oct 24 2020
prev sibling parent reply Kagamin <spam here.lot> writes:
On Friday, 23 October 2020 at 06:57:05 UTC, Ola Fosheim Grøstad 
wrote:

 It is in use until the end of main. Think locking. Seems like 
 the pointer is optimized away and then the GC collects. Makes 
 RAII useless with GC.

 Basically a consequence of the optimizer assuming that there is 
 no GC. A rather serious correctness issue.


Your C++ glasses are too thick, the snippet uses reference type, 
there's no RAII there.
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 16:27:53 UTC, Kagamin wrote:

 Your C++ glasses are too thick, the snippet uses reference 
 type, there's no RAII there.


"Resource acquisition is initialization" is typically done with 
reference types. Doesn't matter whether you name them "struct" or 
"class". So what? It is a completely bogus argument. Since D now 
has a precise GC you should get reliable destruction if you avoid 
unions.
Oct 23 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/23/20 12:33 PM, Ola Fosheim Grøstad wrote:

 On Friday, 23 October 2020 at 16:27:53 UTC, Kagamin wrote:

 Your C++ glasses are too thick, the snippet uses reference type, 
 there's no RAII there.


 
 "Resource acquisition is initialization" is typically done with 
 reference types. Doesn't matter whether you name them "struct" or 
 "class". So what? It is a completely bogus argument. Since D now has a 
 precise GC you should get reliable destruction if you avoid unions.
 


There's no RAII for classes or structs allocated on the heap in C++. You 
have to manually manage the lifetime (either by manually freeing it or 
encapsulating it into an RAII structure like shared_ptr). You only get 
RAII for items allocated on the stack.

But the *resource* is likely a reference. The *owner* of the resource is 
the struct that lives on the stack. A class reference is simply a 
pointer, not an RAII-enabled structure.

-Steve
Oct 23 2020
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 06:15:32 UTC, frame wrote:

 Not sure if this is expected behaviour, but I find this weird:
 The destructor is called on the object just because it gets out 
 of scope?


The destructor may not be called at all by the GC, or it might be 
called whenever there is no pointer found in memory/registers. 
Since D is essentially using C-style code-gen/optimizers that has 
little to no knowledge of a GC it may be called prematurely or 
never.

So basically, destructors do not play well with GC. I've in the 
past argued that one should not allow destructors on GC objects 
since there are no proper guarantees anyway. That could also 
speed up collection. But people prefer convenience over 
correctness...

IMO, rule of thumb: do not rely on destructors being called at 
the appropriate time for objects on the GC heap. Do explicit 
finalisation instead.
Oct 23 2020
next sibling parent reply Daniel N <no public.email> writes:
On Friday, 23 October 2020 at 07:32:02 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 23 October 2020 at 06:15:32 UTC, frame wrote:

 Not sure if this is expected behaviour, but I find this weird:
 The destructor is called on the object just because it gets 
 out of scope?


 The destructor may not be called at all by the GC, or it might 
 be called whenever there is no pointer found in 
 memory/registers. Since D is essentially using C-style 
 code-gen/optimizers that has little to no knowledge of a GC it 
 may be called prematurely or never.


Simple fix:
auto foo = Bar.create();
=>
scope foo = Bar.create();
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 07:39:37 UTC, Daniel N wrote:

 On Friday, 23 October 2020 at 07:32:02 UTC, Ola Fosheim Grøstad 
 wrote:

 On Friday, 23 October 2020 at 06:15:32 UTC, frame wrote:

 Not sure if this is expected behaviour, but I find this weird:
 The destructor is called on the object just because it gets 
 out of scope?


 The destructor may not be called at all by the GC, or it might 
 be called whenever there is no pointer found in 
 memory/registers. Since D is essentially using C-style 
 code-gen/optimizers that has little to no knowledge of a GC it 
 may be called prematurely or never.


 Simple fix:
 auto foo = Bar.create();
 =>
 scope foo = Bar.create();


Does not work with LDC -O3:

void main() {
     scope foo = Bar.create();
     GC.collect();
     writefln("Collection done");
     exit(0);
}

Output:

Warning: destructor called (onlineapp.Foo) on 55A899C1C570
Collection done
Oct 23 2020
parent reply Boris Carvajal <boris2.9 gmail.com> writes:
On Friday, 23 October 2020 at 08:32:45 UTC, Ola Fosheim Grøstad 
wrote:

 Does not work with LDC -O3:

 void main() {
     scope foo = Bar.create();
     GC.collect();
     writefln("Collection done");
     exit(0);
 }

 Output:

 Warning: destructor called (onlineapp.Foo) on 55A899C1C570
 Collection done


Wow I couldn't believe 'scope' didn't work because it's used a 
lot for this same purpose.

But after checking DMD sources I realized that 'scope' works only 
if it precedes a new expression initializer (scope var = new 
ClassName...) which makes sense but it's really confusing.

That means in OP example 'scope' does nothing, it's just like 
'auto' and the class is heap allocated.
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 12:53:52 UTC, Boris Carvajal wrote:

 That means in OP example 'scope' does nothing, it's just like 
 'auto' and the class is heap allocated.


Hm. I think what happens is that exit() makes the destructor call 
unreachable therefore the pointer to the object is not saved on 
the stack and not retained in any registers. Thus GC.collect 
finds no references to it.

It works if you remove exit().
Oct 23 2020
parent reply Boris Carvajal <boris2.9 gmail.com> writes:
On Friday, 23 October 2020 at 13:11:49 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 23 October 2020 at 12:53:52 UTC, Boris Carvajal 
 wrote:

 That means in OP example 'scope' does nothing, it's just like 
 'auto' and the class is heap allocated.


 Hm. I think what happens is that exit() makes the destructor 
 call unreachable therefore the pointer to the object is not 
 saved on the stack and not retained in any registers. Thus 
 GC.collect finds no references to it.

 It works if you remove exit().


exit() is irrelevant, I checked the asm output.
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 13:23:18 UTC, Boris Carvajal wrote:

 exit() is irrelevant, I checked the asm output.


Ok, I guess GC.collect() is called after main() by the runtime 
then.
Oct 23 2020
parent reply Boris Carvajal <boris2.9 gmail.com> writes:
On Friday, 23 October 2020 at 13:34:56 UTC, Ola Fosheim Grøstad 
wrote:

 On Friday, 23 October 2020 at 13:23:18 UTC, Boris Carvajal 
 wrote:

 exit() is irrelevant, I checked the asm output.


 Ok, I guess GC.collect() is called after main() by the runtime 
 then.


Just to clarify my point.
I was talking about 'scope foo = Bar.create();' that does 
nothing, neither the class is stack allocated nor the pointer is 
kept until the end of the function.

I think you are totally right about the optimizer vs D GC.
Oct 23 2020
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 13:48:52 UTC, Boris Carvajal wrote:

 On Friday, 23 October 2020 at 13:34:56 UTC, Ola Fosheim Grøstad 
 wrote:

 On Friday, 23 October 2020 at 13:23:18 UTC, Boris Carvajal 
 wrote:

 exit() is irrelevant, I checked the asm output.


 Ok, I guess GC.collect() is called after main() by the runtime 
 then.


 Just to clarify my point.
 I was talking about 'scope foo = Bar.create();' that does 
 nothing, neither the class is stack allocated nor the pointer 
 is kept until the end of the function.


I understand.

I guess the goal of  safe makes the expected semantics for scope 
(explicit destruction) impossible.

The compiler does not know if there are more references to the 
scope-pointed object. D would need an "isolated" pointer type for 
that.

I think an ARC implementation written for C++ shared_ptr would 
have been interesting for class-objects, but it is not on the 
table...
Oct 23 2020
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Friday, 23 October 2020 at 13:58:04 UTC, Ola Fosheim Grøstad 
wrote:

 I guess the goal of  safe makes the expected semantics for 
 scope (explicit destruction) impossible.


(I meant "deterministic destruction")
Oct 23 2020
prev sibling parent reply Guillaume Piolat <first.name guess.com> writes:
On Friday, 23 October 2020 at 07:32:02 UTC, Ola Fosheim Grøstad 
wrote:

 So basically, destructors do not play well with GC. I've in the 
 past argued that one should not allow destructors on GC objects 
 since there are no proper guarantees anyway. That could also 
 speed up collection. But people prefer convenience over 
 correctness...


100% agree.

I think Andrei also argued once that class destructors should not 
be called by the GC.
But what happened instead later on is that GC-allocated struct 
also did get their destructor called by the GC.

Not that the very-much-unused "GC Proof Resource Class"[1] idiom 
turns the GC into a detector for... overreliance on the GC for 
freeing resources.

[1] http://p0nce.github.io/d-idioms/#GC-proof-resource-class



 IMO, rule of thumb: do not rely on destructors being called at 
 the appropriate time for objects on the GC heap. Do explicit 
 finalisation instead.


For the GC objects that transitively own something that isn't 
memory.

If A owns B owns (a GC-allocated int[]),
then A and B destructors don't need to be called manually.
Oct 24 2020
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 10/24/20 8:01 AM, Guillaume Piolat wrote:

 Not that the very-much-unused "GC Proof Resource Class"[1] idiom turns 
 the GC into a detector for... overreliance on the GC for freeing resources.
 
 [1] http://p0nce.github.io/d-idioms/#GC-proof-resource-class


Please update this:

https://dlang.org/phobos/core_memory.html#.GC.inFinalizer

-Steve
Oct 24 2020