www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - delete and references?

reply Aaron Watters <arw1961 yahoo.com> writes:
Hi guys.  D looks cool.  A couple of things confuse me.
The first is: what happens if I do a

delete cl;

after storing a reference to cl somewhere?  Can I use the stored
reference?  Is it invalid?  Please elucidate, thanks.

   -- Aaron Watters

ps: sorry if this is a repeat, but if it is it might be a good addition
  to the FAQ...
Aug 07 2009
parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Fri, Aug 7, 2009 at 1:20 PM, Aaron Watters<arw1961 yahoo.com> wrote:

 Hi guys. =A0D looks cool. =A0A couple of things confuse me.
 The first is: what happens if I do a

 delete cl;

 after storing a reference to cl somewhere? =A0Can I use the stored
 reference? =A0Is it invalid? =A0Please elucidate, thanks.


If you delete something and attempt to use another reference to the
deleted object, the behavior's undefined.  Basically the same thing as
accessing a dangling pointer in C or C++.  'delete' is really just
there for when you *know* that there are no other references and you
want to ensure that it gets cleaned up immediately.
Aug 07 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Jarrett Billingsley Wrote:


 On Fri, Aug 7, 2009 at 1:20 PM, Aaron Watters<arw1961 yahoo.com> wrote:

 Hi guys.  D looks cool.  A couple of things confuse me.
 The first is: what happens if I do a

 delete cl;

 after storing a reference to cl somewhere?  Can I use the stored
 reference?  Is it invalid?  Please elucidate, thanks.


 
 If you delete something and attempt to use another reference to the
 deleted object, the behavior's undefined.  Basically the same thing as
 accessing a dangling pointer in C or C++.  'delete' is really just
 there for when you *know* that there are no other references and you
 want to ensure that it gets cleaned up immediately.


If you're interested in how this behaves in the runtime, "delete cl;" tells the
memory manager to reclaim the memory at 'cl'. It does so by putting its block
of data back onto its internal freelist. At that point the first 4 bytes are
used in a linked list. If you still have references to that memory and then
modify it, you break the memory manager internal state, which is gonna be hard
to track back. After the memory has been reallocated, your old reference and
the new one are now fighting for the same memory block. In either cases, your
old reference will cause trouble that will be very hard to isolate.

A good rule of thumb in D is that if you're unsure whether references are still
alive, don't delete the object, the GC will automatically reclaim the memory
once it detects there are no more references to it. It is also faster to go
that way, lots of delete calls are more expensive than one GC sweep, and the
sweep is going to happen sooner or later anyways.

However, delete is most useful when you need large blocks of data for short
periods of time.
Aug 07 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jeremie Pelletier wrote:

 Jarrett Billingsley Wrote:
 

 On Fri, Aug 7, 2009 at 1:20 PM, Aaron Watters<arw1961 yahoo.com> wrote:

 Hi guys.  D looks cool.  A couple of things confuse me.
 The first is: what happens if I do a

 delete cl;

 after storing a reference to cl somewhere?  Can I use the stored
 reference?  Is it invalid?  Please elucidate, thanks.


 If you delete something and attempt to use another reference to the
 deleted object, the behavior's undefined.  Basically the same thing as
 accessing a dangling pointer in C or C++.  'delete' is really just
 there for when you *know* that there are no other references and you
 want to ensure that it gets cleaned up immediately.


 
 If you're interested in how this behaves in the runtime, "delete cl;" tells
the memory manager to reclaim the memory at 'cl'. It does so by putting its
block of data back onto its internal freelist. At that point the first 4 bytes
are used in a linked list. If you still have references to that memory and then
modify it, you break the memory manager internal state, which is gonna be hard
to track back. After the memory has been reallocated, your old reference and
the new one are now fighting for the same memory block. In either cases, your
old reference will cause trouble that will be very hard to isolate.
 
 A good rule of thumb in D is that if you're unsure whether references are
still alive, don't delete the object, the GC will automatically reclaim the
memory once it detects there are no more references to it. It is also faster to
go that way, lots of delete calls are more expensive than one GC sweep, and the
sweep is going to happen sooner or later anyways.
 
 However, delete is most useful when you need large blocks of data for short
periods of time.


Great description.

FWIW, I am trying to convince Walter to not reclaim memory in delete, 
but instead only call destructors. D continues C++'s mistake of 
conflating lifetime termination with memory reclamation.

As there will always be a need for reclaiming memory, malloc can always 
supplant that need.


Andrei
Aug 08 2009
next sibling parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Sat, Aug 8, 2009 at 9:17 AM, Andrei
Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

 FWIW, I am trying to convince Walter to not reclaim memory in delete, but
 instead only call destructors. D continues C++'s mistake of conflating
 lifetime termination with memory reclamation.


Why?  Instead of dangling pointers, you'd end up with pointers to
finalized objects, which would probably lead to harder-to-catch bugs
(since then you wouldn't even get a segfault).

And what if you used it to try to delete a temporary array?  Yes, you
could use malloc/free for the temp array, but.. it just seems silly to
have to dip into C functions to do that.
Aug 08 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jarrett Billingsley wrote:

 On Sat, Aug 8, 2009 at 9:17 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

 FWIW, I am trying to convince Walter to not reclaim memory in delete, but
 instead only call destructors. D continues C++'s mistake of conflating
 lifetime termination with memory reclamation.


 
 Why?  Instead of dangling pointers, you'd end up with pointers to
 finalized objects, which would probably lead to harder-to-catch bugs
 (since then you wouldn't even get a segfault).


Getting a segfault == lucky


 And what if you used it to try to delete a temporary array?  Yes, you
 could use malloc/free for the temp array, but.. it just seems silly to
 have to dip into C functions to do that.


Why? It's not a frequent need.


Andrei
Aug 08 2009
next sibling parent reply Leandro Lucarella <llucax gmail.com> writes:
Andrei Alexandrescu, el  8 de agosto a las 08:42 me escribiste:

 Jarrett Billingsley wrote:

On Sat, Aug 8, 2009 at 9:17 AM, Andrei
Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

FWIW, I am trying to convince Walter to not reclaim memory in delete, but
instead only call destructors. D continues C++'s mistake of conflating
lifetime termination with memory reclamation.


Why?  Instead of dangling pointers, you'd end up with pointers to
finalized objects, which would probably lead to harder-to-catch bugs
(since then you wouldn't even get a segfault).


 
 Getting a segfault == lucky


You could easily get a segfault *if* the memory is big enough (> 1 page)
*and* if there is some support from the OS. Remove the RW permissions from
the page and when somebody wants to access it or write it... BOOM! You'll
get your segfault.

So it's not only lucky, so can choose your own destiny ;)

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
CAMPAÑA POR LA PAZ: APLASTARON JUGUETES BÉLICOS
	-- Crónica TV
Aug 08 2009
parent Leandro Lucarella <llucax gmail.com> writes:
Leandro Lucarella, el  8 de agosto a las 11:22 me escribiste:

 Andrei Alexandrescu, el  8 de agosto a las 08:42 me escribiste:

 Jarrett Billingsley wrote:

On Sat, Aug 8, 2009 at 9:17 AM, Andrei
Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

FWIW, I am trying to convince Walter to not reclaim memory in delete, but
instead only call destructors. D continues C++'s mistake of conflating
lifetime termination with memory reclamation.


Why?  Instead of dangling pointers, you'd end up with pointers to
finalized objects, which would probably lead to harder-to-catch bugs
(since then you wouldn't even get a segfault).


 
 Getting a segfault == lucky


 
 You could easily get a segfault *if* the memory is big enough (> 1 page)
 *and* if there is some support from the OS. Remove the RW permissions from
 the page and when somebody wants to access it or write it... BOOM! You'll
 get your segfault.
 
 So it's not only lucky, so can choose your own destiny ;)


But, well, that has nothing to do with putting that memory in the free list
or not, it can be implemented for both approaches...

-- 
Leandro Lucarella (luca) | Blog colectivo: http://www.mazziblog.com.ar/blog/
----------------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------------
JUGAR COMPULSIVAMENTE ES PERJUDICIAL PARA LA SALUD.
	-- Casino de Mar del Plata
Aug 08 2009
prev sibling next sibling parent reply Jarrett Billingsley <jarrett.billingsley gmail.com> writes:
On Sat, Aug 8, 2009 at 9:42 AM, Andrei
Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

 Jarrett Billingsley wrote:

 On Sat, Aug 8, 2009 at 9:17 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

 FWIW, I am trying to convince Walter to not reclaim memory in delete, b=






ut

 instead only call destructors. D continues C++'s mistake of conflating
 lifetime termination with memory reclamation.


 Why? =A0Instead of dangling pointers, you'd end up with pointers to
 finalized objects, which would probably lead to harder-to-catch bugs
 (since then you wouldn't even get a segfault).


 Getting a segfault =3D=3D lucky


And the alternative - using a "dead" object - is any better?


 And what if you used it to try to delete a temporary array? =A0Yes, you
 could use malloc/free for the temp array, but.. it just seems silly to
 have to dip into C functions to do that.


 Why? It's not a frequent need.


That's enough of a justification to change delete's behavior?

I just don't know why it should be changed.  It's a blunt tool for
blunt uses.  When you delete something, you're telling the GC "I don't
need this anymore, *trust me*", so *not* freeing memory associated
with it seems silly.
Aug 08 2009
next sibling parent Jeremie Pelletier <jeremiep gmail.com> writes:
Jarrett Billingsley Wrote:


 On Sat, Aug 8, 2009 at 9:42 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

 Jarrett Billingsley wrote:

 On Sat, Aug 8, 2009 at 9:17 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

 FWIW, I am trying to convince Walter to not reclaim memory in delete, but
 instead only call destructors. D continues C++'s mistake of conflating
 lifetime termination with memory reclamation.


 Why?  Instead of dangling pointers, you'd end up with pointers to
 finalized objects, which would probably lead to harder-to-catch bugs
 (since then you wouldn't even get a segfault).


 Getting a segfault == lucky


 
 And the alternative - using a "dead" object - is any better?
 

 And what if you used it to try to delete a temporary array?  Yes, you
 could use malloc/free for the temp array, but.. it just seems silly to
 have to dip into C functions to do that.


 Why? It's not a frequent need.


 
 That's enough of a justification to change delete's behavior?
 
 I just don't know why it should be changed.  It's a blunt tool for
 blunt uses.  When you delete something, you're telling the GC "I don't
 need this anymore, *trust me*", so *not* freeing memory associated
 with it seems silly.


I just had an idea for new semantics for delete. With proper use it could speed
up the GC a lot, and with misuse it could make it worse.

When deleting memory, that block goes on a special linked list with an internal
entry wrapper as to not alter the memory. It is not finalized, nor is its GC
state changed (has pointers, no scan, etc). Once the GC is asked to do a sweep,
that linked list is checked first for unused memory, and free blocks are put
back on the freelist. If enough can be reclaimed there for the allocation
request which triggered the sweep, the entire set of allocations doesn't have
to be scanned.

Well thats the idea in a nutshell, it popped in my mind as I read Andrei's
comment about delete doing only a finalize. Since we don't have a moving GC
with multiple heaps (short lived heap moving up to long lived heap, and only
the short lived one is scanned often), we could optimize how the sweep is being
done.

That would however require new ABI routines to use with do-it-now deletes, such
as scoped allocations as their usage may rely on the fact that the finalizer is
executed when the scope exits.

Please give me some input on that idea.
Aug 08 2009
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Jarrett Billingsley wrote:

 On Sat, Aug 8, 2009 at 9:42 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

 Jarrett Billingsley wrote:

 On Sat, Aug 8, 2009 at 9:17 AM, Andrei
 Alexandrescu<SeeWebsiteForEmail erdani.org> wrote:

 FWIW, I am trying to convince Walter to not reclaim memory in delete, but
 instead only call destructors. D continues C++'s mistake of conflating
 lifetime termination with memory reclamation.


 Why?  Instead of dangling pointers, you'd end up with pointers to
 finalized objects, which would probably lead to harder-to-catch bugs
 (since then you wouldn't even get a segfault).


 Getting a segfault == lucky


 
 And the alternative - using a "dead" object - is any better?


Yes. I've internalized this so deeply, I won't even debate it.


 And what if you used it to try to delete a temporary array?  Yes, you
 could use malloc/free for the temp array, but.. it just seems silly to
 have to dip into C functions to do that.


 Why? It's not a frequent need.


 
 That's enough of a justification to change delete's behavior?


I believe it is.


 I just don't know why it should be changed.  It's a blunt tool for
 blunt uses.  When you delete something, you're telling the GC "I don't
 need this anymore, *trust me*", so *not* freeing memory associated
 with it seems silly.


We do have blunt tools for blunt uses: malloc and free. We don't need 
more of those.


Andrei
Aug 08 2009
prev sibling parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Andrei Alexandrescu wrote:

 ...
 

 And what if you used it to try to delete a temporary array?  Yes, you
 could use malloc/free for the temp array, but.. it just seems silly to
 have to dip into C functions to do that.


 
 Why? It's not a frequent need.


I use it quite a bit, mostly because scope doesn't work for arrays.

Also, how would you use malloc on class instances?

Also also, I think D's current memory leakage problems should be fixed
BEFORE you start taking away our ability to compensate for the GC.
Aug 08 2009
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-08-08 09:17:28 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 Great description.
 
 FWIW, I am trying to convince Walter to not reclaim memory in delete, 
 but instead only call destructors. D continues C++'s mistake of 
 conflating lifetime termination with memory reclamation.


I don't see how this changes anything. Instead of accessing a 
deallocated object, you'll access a finaized but not yet deallocated 
object. In both cases, it's a bug.

Wouldn't it be better to have a system to track unique pointers? If you 
knew that no other pointer points to a given object or memory block, 
you can finalize and deallocate it safely. In fact, the current 
semantics of a scope object assume that the programmer will not leave 
any dangling pointers at the end of the scope, so it's already assuming 
uniqueness, just not enforcing it.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Aug 08 2009
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Michel Fortin wrote:

 On 2009-08-08 09:17:28 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 

 Great description.

 FWIW, I am trying to convince Walter to not reclaim memory in delete, 
 but instead only call destructors. D continues C++'s mistake of 
 conflating lifetime termination with memory reclamation.


 
 I don't see how this changes anything. Instead of accessing a 
 deallocated object, you'll access a finaized but not yet deallocated 
 object. In both cases, it's a bug.


The former case has you using memory that was possibly allocated to an 
object of different type. This makes for intractable errors.


 Wouldn't it be better to have a system to track unique pointers? If you 
 knew that no other pointer points to a given object or memory block, you 
 can finalize and deallocate it safely. In fact, the current semantics of 
 a scope object assume that the programmer will not leave any dangling 
 pointers at the end of the scope, so it's already assuming uniqueness, 
 just not enforcing it.


Unique pops in an out as a topic for future directions. It is quite 
difficult to make it at the same time correct, inconspicuous, and 
useful. We have firmly decided to do away with unique for D2.


Andrei
Aug 08 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
Andrei Alexandrescu Wrote:


 Michel Fortin wrote:

 On 2009-08-08 09:17:28 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 

 Great description.

 FWIW, I am trying to convince Walter to not reclaim memory in delete, 
 but instead only call destructors. D continues C++'s mistake of 
 conflating lifetime termination with memory reclamation.


 
 I don't see how this changes anything. Instead of accessing a 
 deallocated object, you'll access a finaized but not yet deallocated 
 object. In both cases, it's a bug.


 
 The former case has you using memory that was possibly allocated to an 
 object of different type. This makes for intractable errors.
 

 Wouldn't it be better to have a system to track unique pointers? If you 
 knew that no other pointer points to a given object or memory block, you 
 can finalize and deallocate it safely. In fact, the current semantics of 
 a scope object assume that the programmer will not leave any dangling 
 pointers at the end of the scope, so it's already assuming uniqueness, 
 just not enforcing it.


 
 Unique pops in an out as a topic for future directions. It is quite 
 difficult to make it at the same time correct, inconspicuous, and 
 useful. We have firmly decided to do away with unique for D2.
 
 
 Andrei


I agree, scope arguments can easily be casted away and live beyond the scope.
It's impossible to know if a pointer is unique without scanning the entire heap
and pausing all threads to scan their stacks. And adding reference counts to
allocations would break the need to have a GC in the first place.

The best thing to do is to not use delete when uncertain about the uniqueness
of a pointer.
Aug 08 2009
parent reply bearophile <bearophileHUGS lycos.com> writes:
Jeremie Pelletier:

It's impossible to know if a pointer is unique without scanning the entire heap
and pausing all threads to scan their stacks.<


Are you saying this after reading Bartoz blog posts?

Bye,
bearophile
Aug 08 2009
parent reply Jeremie Pelletier <jeremiep gmail.com> writes:
bearophile Wrote:


 Jeremie Pelletier:

It's impossible to know if a pointer is unique without scanning the entire heap
and pausing all threads to scan their stacks.<


 
 Are you saying this after reading Bartoz blog posts?
 
 Bye,
 bearophile


No I haven't, do you have a link?
Aug 08 2009
parent bearophile <bearophileHUGS lycos.com> writes:
Jeremie Pelletier:

No I haven't, do you have a link?<


With the power of Google this is the blog of Bartosz Milewski, you may read
about the last ten posts or so:
http://bartoszmilewski.wordpress.com/

Bye,
bearophile
Aug 08 2009