www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Arrays of structs

reply "John Burton" <john.burton jbmail.com> writes:
I'm a c++ programmer trying to understand how memory allocation 
works in D.

I created a struct and added a destructor to it. My understanding 
is that structs have deterministic destructors - they are called 
when the struct goes out of scope (unless it is allocated with 
new).

Now if I put instances of the struct in a fixed size array

data[6] d;
d[3] = data(1, 2, 3);

then the destructor on all the contents is called when the array 
goes out of scope.

However if I add them to a dynamic array...

data[] d;
d ~= data(1, 2, 3)

Then the destructor appears to be called at some random time 
later. So it looks like it's the garbage collection that is doing 
this. That seems to go against the specification of how struct 
works... I'm not creating the item with "new" and as far as I can 
tell the array is storing instances of objects, not pointers to 
objects?

Is my understanding correct?
Is it documented anywhere how memory allocation works for this?

Is a dynamic array in fact storing an array of GC'd pointers to 
the structs? Or something else...
Aug 27 2015
next sibling parent reply "SimonN" <eiderdaus gmail.com> writes:
Hi,

On Thursday, 27 August 2015 at 10:05:31 UTC, John Burton wrote:

 understanding is that structs have deterministic destructors - 
 they are called when the struct goes out of scope


Yes, when they are declared right at the scope, and not contained 
in something that might live past the current scope.


 However if I add them to a dynamic array...
 Then the destructor appears to be called at some random time 
 later. So it looks like it's the garbage collection that is 
 doing this.


Yeah.


 That seems to go against the specification of how struct 
 works... I'm not creating the item with "new" and as far as I 
 can tell the array is storing instances of objects, not 
 pointers to objects?


The array is storing the full structs, yes. The array is GC-ably 
allocated and will therefore not go out of scope at end of scope. 
Because the array doesn't vanish here, the structs inside will 
not go out of scope either; they treat the array as their scope.

(I'm sure others could explain this more formally.)

There is no reference left to the array, so the GC may at a 
random later time deallocate the array, and thereby call ~this() 
on each struct.


 Is my understanding correct?


Explicit "new" is not the only way to put objects on the GC'ed 
heap. Putting them in a GC-ed array like this is another way. Or 
having them as a component in a class, then instantiating that 
class.


 Is it documented anywhere how memory allocation works for this?


I'll leave this for others, too.


 Is a dynamic array in fact storing an array of GC'd pointers to 
 the structs?


No, it's the full struct.

-- Simon
Aug 27 2015
parent reply "John Burton" <john.burton jbmail.com> writes:
Ok that's great thank you.

It's quite hard trying to get a proper understanding of memory 
allocation in D after years of C++ / RAII / unique_ptr / 
shared_ptr . I understand the details of course but it's going to 
take a while to really know it.

Is there any way to explicitly free a dynamic array when I am 
done with it?
(I'm more interested in the contained items destructors being 
called than in the memory / gc aspects)

I fear I'm trying to write C++ in D and need to get my mind 
around different ways to do things...
Aug 27 2015
parent reply "John Burton" <john.burton jbmail.com> writes:
Thanks again for the updates. I've experimented some more and 
believe I understand.

To be honest I'm finding it very hard to find the right idioms in 
D for safe and efficient programming when I'm so used to C++ / 
RAII everywhere. I'll adapt though :P
Aug 27 2015
next sibling parent "Gary Willoughby" <dev nomad.so> writes:
On Thursday, 27 August 2015 at 10:49:02 UTC, John Burton wrote:

 Thanks again for the updates. I've experimented some more and 
 believe I understand.

 To be honest I'm finding it very hard to find the right idioms 
 in D for safe and efficient programming when I'm so used to C++ 
 / RAII everywhere. I'll adapt though :P


There's also std.container.array with deterministic memory usage 
not reliant on the GC, as an alternative to the built-in arrays.

http://dlang.org/phobos/std_container_array.html
Aug 27 2015
prev sibling next sibling parent "Olivier Pisano" <olivier.pisano laposte.net> writes:
On Thursday, 27 August 2015 at 10:49:02 UTC, John Burton wrote:

 To be honest I'm finding it very hard to find the right idioms 
 in D for safe and efficient programming when I'm so used to C++ 
 / RAII everywhere. I'll adapt though :P


This is true for every new language you learn. You first stick to 
the idioms you already know before getting the right style.
My first Python code looked like C and my first D code looked 
like Java.

BTW, if you need a growable array with deterministic destruction, 
you should have a look at std.container.array.Array(T), which is 
equivalent to a shared_ptr<vector<T>>.
Aug 27 2015
prev sibling parent reply "BBasile" <bb.temp gmx.com> writes:
On Thursday, 27 August 2015 at 10:49:02 UTC, John Burton wrote:

 Thanks again for the updates. I've experimented some more and 
 believe I understand.

 To be honest I'm finding it very hard to find the right idioms 
 in D for safe and efficient programming when I'm so used to C++ 
 / RAII everywhere. I'll adapt though :P


You can also do explicit
* memory allocation/deallocation
* class construction/destruction
* struct/union construction/destruction

Personally my background is Object-Pascal/Delphi which use 
similar memory managment to C++ (though more RAII with ownership 
than RAII with refcounting). For example I have those routines in 
my home-cooked general library:

https://github.com/BBasile/iz/blob/master/import/iz/types.d#L125
https://github.com/BBasile/iz/blob/master/import/iz/types.d#L150
https://github.com/BBasile/iz/blob/master/import/iz/types.d#L191

they allow to do manual memory managment and you'll find similar 
routines in several other user libraries, for example
* https://github.com/etcimon/memutils
* https://github.com/Dgame/m3
* etc...
Aug 27 2015
parent reply anonymous <anonymous example.com> writes:
On Thursday 27 August 2015 13:15, BBasile wrote:


 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L125
 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L150
 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L191


Your use of  trusted is wrong and dangerous.  trusted functions are supposed 
to be memory-safe, but you're marking unsafe functions with it.

Things like a  trusted `free` [1] are just plain wrong. `free` isn't memory-
safe.

The problems with  trusted templates can be more subtle. Even if the 
template body itself doesn't do anything unsafe, the template arguments are 
being trusted, too. So if the template ever calls any code from the 
arguments (including constructors, destructors, postblits, ...), then it 
cannot be marked  trusted.


[1] https://github.com/BBasile/iz/blob/master/import/iz/types.d#L112
Aug 27 2015
parent reply "BBasile" <bb.temp gmx.com> writes:
On Thursday, 27 August 2015 at 11:45:14 UTC, anonymous wrote:

 On Thursday 27 August 2015 13:15, BBasile wrote:


 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L125
https://github.com/BBasile/iz/blob/master/import/iz/types.d#L150
https://github.com/BBasile/iz/blob/master/import/iz/types.d#L191


 Your use of  trusted is wrong and dangerous.  trusted functions 
 are supposed to be memory-safe, but you're marking unsafe 
 functions with it.

 Things like a  trusted `free` [1] are just plain wrong. `free` 
 isn't memory- safe.

 The problems with  trusted templates can be more subtle. Even 
 if the template body itself doesn't do anything unsafe, the 
 template arguments are being trusted, too. So if the template 
 ever calls any code from the arguments (including constructors, 
 destructors, postblits, ...), then it cannot be marked  trusted.


 [1] 
 https://github.com/BBasile/iz/blob/master/import/iz/types.d#L112


the pointer is checked before the call. Yes it can be dangling 
but free goes in pair with the newPtr funct. I plan to do better 
when Andrei's allocators will be released:
https://github.com/BBasile/phobos/blob/showcase-construct/std/experimental/allocator/showcase.d#L105

Anyway. I cheat a bit with attributes but as long as it's only 
for me...I know this kinds of functions are not phobos-level.
Aug 27 2015
parent reply anonymous <anonymous example.com> writes:
On Thursday 27 August 2015 14:35, BBasile wrote:


 Anyway. I cheat a bit with attributes but as long as it's only
 for me...I know this kinds of functions are not phobos-level.


Sure, as long as you're cautious and regard those functions as unsafe, you 
may be fine. You still risk accidentally writing unsafe code that's marked 
 safe, but that's up to you.

But when you show such code in the learn group, I think it's important to 
point out that this usage of  trusted is wrong. If only to warn newbies 
about it.
Aug 27 2015
parent "BBasile" <bb.temp gmx.com> writes:
On Thursday, 27 August 2015 at 12:56:26 UTC, anonymous wrote:

 On Thursday 27 August 2015 14:35, BBasile wrote:


 Anyway. I cheat a bit with attributes but as long as it's only 
 for me...I know this kinds of functions are not phobos-level.


 Sure, as long as you're cautious and regard those functions as 
 unsafe, you may be fine. You still risk accidentally writing 
 unsafe code that's marked  safe, but that's up to you.

 But when you show such code in the learn group, I think it's 
 important to point out that this usage of  trusted is wrong. If 
 only to warn newbies about it.


:handshake:
Aug 27 2015
prev sibling parent "BBasile" <bb.temp gmx.com> writes:
On Thursday, 27 August 2015 at 10:05:31 UTC, John Burton wrote:

 I'm a c++ programmer trying to understand how memory allocation 
 works in D.

 I created a struct and added a destructor to it. My 
 understanding is that structs have deterministic destructors - 
 they are called when the struct goes out of scope (unless it is 
 allocated with new).

 Now if I put instances of the struct in a fixed size array

 data[6] d;
 d[3] = data(1, 2, 3);

 then the destructor on all the contents is called when the 
 array goes out of scope.

 However if I add them to a dynamic array...

 data[] d;
 d ~= data(1, 2, 3)

 Then the destructor appears to be called at some random time 
 later. So it looks like it's the garbage collection that is 
 doing this. That seems to go against the specification of how 
 struct works... I'm not creating the item with "new" and as far 
 as I can tell the array is storing instances of objects, not 
 pointers to objects?

 Is my understanding correct?
 Is it documented anywhere how memory allocation works for this?

 Is a dynamic array in fact storing an array of GC'd pointers to 
 the structs? Or something else...


With a local scope,
- a static array  of data will have the destructors called on 
exit because memory for the memebers is not allocated on the 
GC-hep but on the stack frame.
- a dynamic array  of data will have the destructors called on 
next GC collection because the memory for the memebers is 
allocated on the GC-heap.
- a dynamic array of pointer to data will have the destructors 
called on next GC collection because the memory for the memebers 
is allocated on the GC-heap.

you can see this in this small program. deactivate to commented 
GC.collect to see the difference:

---
struct Foo {
     long v0, v1;
     ~this(){writeln(typeof(this).stringof);}
}

void localteststatic(){
     Foo[1] fl;
}

void localtestdynamic1(){
     Foo[] fl;
     fl.length = 1;
     fl.length = 0;
}

void localtestdynamic2(){
     Foo* [] fl;
     fl ~= new Foo(1);
}

void localtestdynamic3(){
     Foo[] fl;
     fl.length = 1;
     fl.length = 0;
}

void main(string[] args)
{
     import core.memory;
     localteststatic; writeln("done local test static");
     localtestdynamic3; writeln("done local test dynamic 3");
     //GC.collect;
     localtestdynamic1; writeln("done local test dynamic 1");
     //GC.collect;
     localtestdynamic2;  writeln("done local test dynamic 2");
     //GC.collect;
}
---

Also for the second question:
* fl[]: each element has a .sizeof 16 (long .size_of * 2)
* fl* []: each element has a .sizeof size_t.sizeof (this a 
pointer so 4 or 8).
Aug 27 2015