www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - allocate array with new

reply "Namespace" <rswhite4 googlemail.com> writes:
Is there any other difference if I allocate an array with "new", 
except that it is initialized?
And must i delete it by myself, or take the GC care about that?
The documentary leaves me hanging there, unfortunately, and gives 
me no 100% answer.

Once before, thanks for the reply.
May 15 2012
next sibling parent reply "Kagamin" <spam here.lot> writes:
Difference with what?
new is a safe feature: it allocates in the GC heap
May 15 2012
parent reply "Namespace" <rswhite4 googlemail.com> writes:
On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:

 Difference with what?
 new is a safe feature: it allocates in the GC heap


That's what i mean. So i have to delete it yourself with "delete 
arr;", or not?
May 15 2012
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, May 15, 2012 11:26:49 Namespace wrote:

 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:

 Difference with what?
 new is a safe feature: it allocates in the GC heap


 
 That's what i mean. So i have to delete it yourself with "delete
 arr;", or not?


No. _Never_ use delete. It's going to be deprecated. The GC worries about 
freeing memory allocated on the GC heap, and new always allocates on the GC 
heap. If you don't want to allocate on the GC heap, then use malloc and free, 
in which case you _do_ need worry about freeing the memory.

If you need to force destruction before the GC collects an object, you can 
call clear on that object to have its destructor called and its vtbl zeroed 
out, but it's memory still isn't freed. That's the GC's job.

If you really have to, you can use core.memory to manipulate the GC heap 
(including calling GC.free), but you really shouldn't be messing with any of 
that unless you really need to and you know what you're doing.

- Jonathan M Davis
May 15 2012
next sibling parent "Namespace" <rswhite4 googlemail.com> writes:
On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis wrote:

 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:

 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:

 Difference with what?
 new is a safe feature: it allocates in the GC heap


 
 That's what i mean. So i have to delete it yourself with 
 "delete
 arr;", or not?


 No. _Never_ use delete. It's going to be deprecated. The GC 
 worries about
 freeing memory allocated on the GC heap, and new always 
 allocates on the GC
 heap. If you don't want to allocate on the GC heap, then use 
 malloc and free,
 in which case you _do_ need worry about freeing the memory.

 If you need to force destruction before the GC collects an 
 object, you can
 call clear on that object to have its destructor called and its 
 vtbl zeroed
 out, but it's memory still isn't freed. That's the GC's job.

 If you really have to, you can use core.memory to manipulate 
 the GC heap
 (including calling GC.free), but you really shouldn't be 
 messing with any of
 that unless you really need to and you know what you're doing.

 - Jonathan M Davis


Understood. Never use delete. Fine. :)
May 15 2012
prev sibling next sibling parent reply "Ondrej Pokorny" <pokorny.ondrej gmail.com> writes:
On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis wrote:

 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:

 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:

 Difference with what?
 new is a safe feature: it allocates in the GC heap


 
 That's what i mean. So i have to delete it yourself with 
 "delete
 arr;", or not?


 No. _Never_ use delete. It's going to be deprecated. The GC 
 worries about
 freeing memory allocated on the GC heap, and new always 
 allocates on the GC
 heap. If you don't want to allocate on the GC heap, then use 
 malloc and free,
 in which case you _do_ need worry about freeing the memory.

 If you need to force destruction before the GC collects an 
 object, you can
 call clear on that object to have its destructor called and its 
 vtbl zeroed
 out, but it's memory still isn't freed. That's the GC's job.

 If you really have to, you can use core.memory to manipulate 
 the GC heap
 (including calling GC.free), but you really shouldn't be 
 messing with any of
 that unless you really need to and you know what you're doing.

 - Jonathan M Davis


Hi,
does this hold for structs too?

struct H
{
this(int a){writeln("ctor");}
~this(){writeln("dtor");}
};

...

H* h = new H(5);
clear(h);
...

output:

ctor

seems like destructor is not called.

if I change declaration of H to class H. output is following:

ctor
dtor

I tried to create object according to RAII idiom and in case of 
struct H my file handle remained open  and I was still able to 
write to file...

Ondrej
May 15 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 15.05.2012 16:27, Ondrej Pokorny wrote:

 On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis wrote:

 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:

 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:

 Difference with what?
 new is a safe feature: it allocates in the GC heap


 That's what i mean. So i have to delete it yourself with "delete
 arr;", or not?


 No. _Never_ use delete. It's going to be deprecated. The GC worries about
 freeing memory allocated on the GC heap, and new always allocates on
 the GC
 heap. If you don't want to allocate on the GC heap, then use malloc
 and free,
 in which case you _do_ need worry about freeing the memory.

 If you need to force destruction before the GC collects an object, you
 can
 call clear on that object to have its destructor called and its vtbl
 zeroed
 out, but it's memory still isn't freed. That's the GC's job.

 If you really have to, you can use core.memory to manipulate the GC heap
 (including calling GC.free), but you really shouldn't be messing with
 any of
 that unless you really need to and you know what you're doing.

 - Jonathan M Davis


 Hi,
 does this hold for structs too?

 struct H
 {
 this(int a){writeln("ctor");}
 ~this(){writeln("dtor");}
 };

 ...

 H* h = new H(5);
 clear(h);
 ...

 output:

 ctor

 seems like destructor is not called.

 if I change declaration of H to class H. output is following:

 ctor
 dtor

 I tried to create object according to RAII idiom and in case of struct H
 my file handle remained open and I was still able to write to file...

 Ondrej


I thought in C++ RAII is about (i.e. even in C++ no heap allocation):
H h = H(5);

Same works in D. A call to clear  in you code above doesn't call 
destructor, it only zeros out pointer.
If you absolutely need pointers & heap and yet manual memory managment use:
clear(*h);


Explanation:
clear(x) calls x.__dtor if x is struct or class, then assigns
x = T.init;

A better way might be to just check if x.__dtor is callbale (thus 
pointer to sstruct will also work).

-- 
Dmitry Olshansky
May 15 2012
next sibling parent reply "Ondrej Pokorny" <pokorny.ondrej gmail.com> writes:
On Tuesday, 15 May 2012 at 12:36:30 UTC, Dmitry Olshansky wrote:

 On 15.05.2012 16:27, Ondrej Pokorny wrote:

 On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis 
 wrote:

 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:

 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:

 Difference with what?
 new is a safe feature: it allocates in the GC heap


 That's what i mean. So i have to delete it yourself with 
 "delete
 arr;", or not?


 No. _Never_ use delete. It's going to be deprecated. The GC 
 worries about
 freeing memory allocated on the GC heap, and new always 
 allocates on
 the GC
 heap. If you don't want to allocate on the GC heap, then use 
 malloc
 and free,
 in which case you _do_ need worry about freeing the memory.

 If you need to force destruction before the GC collects an 
 object, you
 can
 call clear on that object to have its destructor called and 
 its vtbl
 zeroed
 out, but it's memory still isn't freed. That's the GC's job.

 If you really have to, you can use core.memory to manipulate 
 the GC heap
 (including calling GC.free), but you really shouldn't be 
 messing with
 any of
 that unless you really need to and you know what you're doing.

 - Jonathan M Davis


 Hi,
 does this hold for structs too?

 struct H
 {
 this(int a){writeln("ctor");}
 ~this(){writeln("dtor");}
 };

 ...

 H* h = new H(5);
 clear(h);
 ...

 output:

 ctor

 seems like destructor is not called.

 if I change declaration of H to class H. output is following:

 ctor
 dtor

 I tried to create object according to RAII idiom and in case 
 of struct H
 my file handle remained open and I was still able to write to 
 file...

 Ondrej


 I thought in C++ RAII is about (i.e. even in C++ no heap 
 allocation):
 H h = H(5);

 Same works in D. A call to clear  in you code above doesn't 
 call destructor, it only zeros out pointer.
 If you absolutely need pointers & heap and yet manual memory 
 managment use:
 clear(*h);


 Explanation:
 clear(x) calls x.__dtor if x is struct or class, then assigns
 x = T.init;

 A better way might be to just check if x.__dtor is callbale 
 (thus pointer to sstruct will also work).


I don't need this stuff I am playing with language and wonder
why. Originally I had there auto h = H(5); and after changing H
to struct I did not realized why that happened.

Thanks a lot for explanation, now it makes perfect sense, but I
have still little bit unpleasant felling about it because of
behavior of ref to classes vs pointers to structs with clear.
Both are dynamically allocated data, so I was expecting same
response from clear(x).

Ondrej
May 15 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 15.05.2012 17:12, Ondrej Pokorny wrote:

 On Tuesday, 15 May 2012 at 12:36:30 UTC, Dmitry Olshansky wrote:

 On 15.05.2012 16:27, Ondrej Pokorny wrote:

 On Tuesday, 15 May 2012 at 09:44:08 UTC, Jonathan M Davis wrote:

 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:

 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:

 Difference with what?
 new is a safe feature: it allocates in the GC heap


 That's what i mean. So i have to delete it yourself with "delete
 arr;", or not?


 No. _Never_ use delete. It's going to be deprecated. The GC worries
 about
 freeing memory allocated on the GC heap, and new always allocates on
 the GC
 heap. If you don't want to allocate on the GC heap, then use malloc
 and free,
 in which case you _do_ need worry about freeing the memory.

 If you need to force destruction before the GC collects an object, you
 can
 call clear on that object to have its destructor called and its vtbl
 zeroed
 out, but it's memory still isn't freed. That's the GC's job.

 If you really have to, you can use core.memory to manipulate the GC
 heap
 (including calling GC.free), but you really shouldn't be messing with
 any of
 that unless you really need to and you know what you're doing.

 - Jonathan M Davis


 Hi,
 does this hold for structs too?

 struct H
 {
 this(int a){writeln("ctor");}
 ~this(){writeln("dtor");}
 };

 ...

 H* h = new H(5);
 clear(h);
 ...

 output:

 ctor

 seems like destructor is not called.

 if I change declaration of H to class H. output is following:

 ctor
 dtor

 I tried to create object according to RAII idiom and in case of struct H
 my file handle remained open and I was still able to write to file...

 Ondrej


 I thought in C++ RAII is about (i.e. even in C++ no heap allocation):
 H h = H(5);

 Same works in D. A call to clear in you code above doesn't call
 destructor, it only zeros out pointer.
 If you absolutely need pointers & heap and yet manual memory managment
 use:
 clear(*h);


 Explanation:
 clear(x) calls x.__dtor if x is struct or class, then assigns
 x = T.init;

 A better way might be to just check if x.__dtor is callbale (thus
 pointer to sstruct will also work).


 I don't need this stuff I am playing with language and wonder
 why. Originally I had there auto h = H(5); and after changing H
 to struct I did not realized why that happened.

 Thanks a lot for explanation, now it makes perfect sense, but I
 have still little bit unpleasant felling about it because of
 behavior of ref to classes vs pointers to structs with clear.
 Both are dynamically allocated data, so I was expecting same
 response from clear(x).


Like I said it's easily fixable at a cost of breaking some code that 
relies on this clear(ptr) zeroing out pointer. I think that it's wrong 
code anyway. So I I'll wrap up a pull request to do just that.


-- 
Dmitry Olshansky
May 15 2012
prev sibling parent reply "Namespace" <rswhite4 googlemail.com> writes:
And what if i use an array of structs? Also clear instead of
delete?
May 15 2012
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 5/15/12, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 If you're going to use anything, use clear.


He might as well just use this if he wants the GC to take care of things:
arr = null;

clear(arr) is a little bit of a misnomer. If you have another slice
pointing to the same array clear() won't really release memory at all,
it will just make the first slice null. so using "arr = null" is good
enough.

Also good to note is that clear/null assign is very different from
what delete does. delete won't care that you have other slices
pointing to the same array and you'll end up with slices that point to
garbage after a call to delete.
May 15 2012
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 15 May 2012 05:43:45 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Tuesday, May 15, 2012 11:26:49 Namespace wrote:

 On Tuesday, 15 May 2012 at 09:23:51 UTC, Kagamin wrote:

 Difference with what?
 new is a safe feature: it allocates in the GC heap


 That's what i mean. So i have to delete it yourself with "delete
 arr;", or not?


 No. _Never_ use delete. It's going to be deprecated. The GC worries about
 freeing memory allocated on the GC heap, and new always allocates on the  
 GC
 heap. If you don't want to allocate on the GC heap, then use malloc and  
 free,
 in which case you _do_ need worry about freeing the memory.

 If you need to force destruction before the GC collects an object, you  
 can
 call clear on that object to have its destructor called and its vtbl  
 zeroed
 out, but it's memory still isn't freed. That's the GC's job.

 If you really have to, you can use core.memory to manipulate the GC heap
 (including calling GC.free), but you really shouldn't be messing with  
 any of
 that unless you really need to and you know what you're doing.


Well, the D GC is conservative, and will always be at least partly  
conservative.  So freeing memory manually is not really a crime.

I agree you should avoid delete, due to its possible deprecation.

-Steve
May 16 2012
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 16, 2012 11:51:59 Steven Schveighoffer wrote:

 Well, the D GC is conservative, and will always be at least partly
 conservative. So freeing memory manually is not really a crime.


The idea with a GC is that it's supposed to manage the memory allocated 
through it. So, if you're worrying about freeing GC memory, you're fighting the 
GC and how it works. And unless you're keeping very close track of the number 
of reference to a particular chunk of memory, freeing it manually is 
dangerous. So, anyone looking to use delete is generally going about things 
the wrong way.

However, it _is_ true that sometimes you need to intervene in order to get 
sections of code as performant as they need to be (particularly since D's 
current GC is not the most performant), which is part of the reason that 
core.memory provides what it does. But it should generally be something used 
when it's clear that you need that extra performance, not as a matter of 
course, because using it is inherently unsafe. If there's something that you 
know ahead of time needs to have more deterministic deallocation, then it's 
probably better to be use malloc and free with ref-counting than trying to 
manually manage GC memory.

- Jonathan M Davis
May 16 2012
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 16 May 2012 12:52:33 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Wednesday, May 16, 2012 11:51:59 Steven Schveighoffer wrote:

 Well, the D GC is conservative, and will always be at least partly
 conservative. So freeing memory manually is not really a crime.


 The idea with a GC is that it's supposed to manage the memory allocated
 through it. So, if you're worrying about freeing GC memory, you're  
 fighting the
 GC and how it works. And unless you're keeping very close track of the  
 number
 of reference to a particular chunk of memory, freeing it manually is
 dangerous. So, anyone looking to use delete is generally going about  
 things
 the wrong way.

 However, it _is_ true that sometimes you need to intervene in order to  
 get
 sections of code as performant as they need to be (particularly since D's
 current GC is not the most performant), which is part of the reason that
 core.memory provides what it does. But it should generally be something  
 used
 when it's clear that you need that extra performance, not as a matter of
 course, because using it is inherently unsafe. If there's something that  
 you
 know ahead of time needs to have more deterministic deallocation, then  
 it's
 probably better to be use malloc and free with ref-counting than trying  
 to
 manually manage GC memory.


Where the GC gets into trouble is with allocation of large chunks of data.

If the data 'contains pointers', it creates a very good chance that the  
data keeps some unknown number of blocks from being deallocated.

And just by the sheer fact of how big the data is, it's bound to be  
pointed at by some piece of stack data, global data, or TLS data.

So you can go a long way by manually managing larger chunks of data.  I  
think it's worth worrying about as you get into large chunks (on the order  
of 10MB or more).

What really really sucks about this is, as you allocate larger chunks of  
data, the chances that the GC incorrectly keeps garbage memory get  
larger.  In other words, the more memory you allocate, the less likely it  
is that the GC will give you some of it back to you!

Precise GC will help this *tremendously*, but I'm unsure if we'll ever get  
to the point of having a fully-precise GC.  There will always be a need to  
do manual freeing of memory.

-Steve
May 16 2012
next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:

 Where the GC gets into trouble is with allocation of large chunks of data.
 
 If the data 'contains pointers', it creates a very good chance that the
 data keeps some unknown number of blocks from being deallocated.
 
 And just by the sheer fact of how big the data is, it's bound to be
 pointed at by some piece of stack data, global data, or TLS data.
 
 So you can go a long way by manually managing larger chunks of data. I
 think it's worth worrying about as you get into large chunks (on the order
 of 10MB or more).
 
 What really really sucks about this is, as you allocate larger chunks of
 data, the chances that the GC incorrectly keeps garbage memory get
 larger. In other words, the more memory you allocate, the less likely it
 is that the GC will give you some of it back to you!


Using 64-bit helps with this.


 Precise GC will help this *tremendously*, but I'm unsure if we'll ever get
 to the point of having a fully-precise GC.


Any improvement to the GC is a welcome improvement.


 There will always be a need to
 do manual freeing of memory.


Yes, but my point is that it's something you do when the need arises due to 
performance concerns in a particular section of code. It's not something that 
you do as a matter of course in your code in general. If you're always trying 
to free memory as soon as you don't need it anymore rather than letting the GC 
do its job (or try to anyway), then you might as well be using malloc and 
free.

- Jonathan M Davis
May 16 2012
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, May 16, 2012 at 02:20:47PM -0400, Jonathan M Davis wrote:

 On Wednesday, May 16, 2012 13:52:12 Steven Schveighoffer wrote:

 Where the GC gets into trouble is with allocation of large chunks of
 data.
 
 If the data 'contains pointers', it creates a very good chance that
 the data keeps some unknown number of blocks from being deallocated.
 
 And just by the sheer fact of how big the data is, it's bound to be
 pointed at by some piece of stack data, global data, or TLS data.
 
 So you can go a long way by manually managing larger chunks of data.
 I think it's worth worrying about as you get into large chunks (on
 the order of 10MB or more).
 
 What really really sucks about this is, as you allocate larger
 chunks of data, the chances that the GC incorrectly keeps garbage
 memory get larger. In other words, the more memory you allocate, the
 less likely it is that the GC will give you some of it back to you!


 
 Using 64-bit helps with this.


Yeah, IIRC somebody did some tests comparing false pointer incidence in
32-bit vs. 64-bit environments, and found that false pointer incidence
is much lower in 64-bit.

But of course, a large-enough chunk of memory will still have a non-zero
chance of a false pointer into it, which is bad because it will just
keep sticking around.



 Precise GC will help this *tremendously*, but I'm unsure if we'll
 ever get to the point of having a fully-precise GC.


 
 Any improvement to the GC is a welcome improvement.


Didn't Walter say that we're pushing toward precise scanning?

Of course, the fact that we interface directly with C (and have union
types) means that there will always be _some_ place that can't be fully
precise.

But having more precise info, such as which references are slices and
which are free pointers, would help a lot. For example, if the 10MB
chunk is referenced by a slice of 10 bytes, the GC could in theory
release everything except those 10 bytes. You're still out of luck with
free pointers, but in regular D code those should be quite rare.



 There will always be a need to
 do manual freeing of memory.


 
 Yes, but my point is that it's something you do when the need arises
 due to performance concerns in a particular section of code. It's not
 something that you do as a matter of course in your code in general.
 If you're always trying to free memory as soon as you don't need it
 anymore rather than letting the GC do its job (or try to anyway), then
 you might as well be using malloc and free.


[...]

It would be nice if there was an easy way of switching allocators, so
that you can, for example, indicate that all allocations done inside a
particular function should use malloc/free (including, say, AA's,
arrays, etc.), and have the runtime automatically switch back to the GC
on scope exit.


T

-- 
Laissez-faire is a French term commonly interpreted by Conservatives to mean
'lazy fairy,' which is the belief that if governments are lazy enough, the Good
Fairy will come down from heaven and do all their work for them.
May 16 2012
prev sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On 15/05/12 11:43, Jonathan M Davis wrote:

 No. _Never_ use delete. It's going to be deprecated.


What if you turn the GC off?
May 15 2012
prev sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, May 16, 2012 12:05:02 H. S. Teoh wrote:

 It would be nice if there was an easy way of switching allocators, so
 that you can, for example, indicate that all allocations done inside a
 particular function should use malloc/free (including, say, AA's,
 arrays, etc.), and have the runtime automatically switch back to the GC
 on scope exit.


Once we have custom allocators, that should be easy enough as long as you 
passing the allocator around. But new will always use the GC, and malloc will 
always do what it does in C (it _is_ the C function after all), so anything 
you want in terms of custom allocation is going to have to go through custom 
allocator objects. However, since the custom allocators are going to be 
runtime entities (rather than having classes templated on their allocators 
like the STL does), it would be easy enough to choose the appropriate 
allocator at runtime and have each function use a different allocator if it so 
chooses.

- Jonathan M Davis
May 16 2012