www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - C Memory

reply "Namespace" <rswhite4 googlemail.com> writes:
Quick question: I have a SDL_Surface in one of my classes and the 
SDL_Surface contains (obviously) memory to the pixel data. Since 
I cannot free this memory with the DTor: what will happen? AFAIK 
this cannot be freed by the GC because it was not allocated by 
it. So AFAIK this creates a memory leak. Am I right?
May 04 2013
parent reply "Diggory" <diggsey googlemail.com> writes:
On Sunday, 5 May 2013 at 06:35:38 UTC, Namespace wrote:

 Quick question: I have a SDL_Surface in one of my classes and 
 the SDL_Surface contains (obviously) memory to the pixel data. 
 Since I cannot free this memory with the DTor: what will 
 happen? AFAIK this cannot be freed by the GC because it was not 
 allocated by it. So AFAIK this creates a memory leak. Am I 
 right?


Why can't you free the memory from the destructor?
May 04 2013
parent reply "Namespace" <rswhite4 googlemail.com> writes:
On Sunday, 5 May 2013 at 06:43:17 UTC, Diggory wrote:

 On Sunday, 5 May 2013 at 06:35:38 UTC, Namespace wrote:

 Quick question: I have a SDL_Surface in one of my classes and 
 the SDL_Surface contains (obviously) memory to the pixel data. 
 Since I cannot free this memory with the DTor: what will 
 happen? AFAIK this cannot be freed by the GC because it was 
 not allocated by it. So AFAIK this creates a memory leak. Am I 
 right?


 Why can't you free the memory from the destructor?


Every time I tried this, I got this: 
core.exception.InvalidMemoryOperationError
I do not know if I get that, because the GC has already cleaned 
this store or because of something else.
May 05 2013
parent reply "Diggory" <diggsey googlemail.com> writes:
On Sunday, 5 May 2013 at 07:23:25 UTC, Namespace wrote:

 On Sunday, 5 May 2013 at 06:43:17 UTC, Diggory wrote:

 On Sunday, 5 May 2013 at 06:35:38 UTC, Namespace wrote:

 Quick question: I have a SDL_Surface in one of my classes and 
 the SDL_Surface contains (obviously) memory to the pixel 
 data. Since I cannot free this memory with the DTor: what 
 will happen? AFAIK this cannot be freed by the GC because it 
 was not allocated by it. So AFAIK this creates a memory leak. 
 Am I right?


 Why can't you free the memory from the destructor?


 Every time I tried this, I got this: 
 core.exception.InvalidMemoryOperationError
 I do not know if I get that, because the GC has already cleaned 
 this store or because of something else.


Sounds like a bug - if the pointer is not into GC memory then the 
GC shouldn't touch it.
May 05 2013
parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 05 May 2013 09:28:05 +0200
schrieb "Diggory" <diggsey googlemail.com>:


 On Sunday, 5 May 2013 at 07:23:25 UTC, Namespace wrote:

 On Sunday, 5 May 2013 at 06:43:17 UTC, Diggory wrote:

 On Sunday, 5 May 2013 at 06:35:38 UTC, Namespace wrote:

 Quick question: I have a SDL_Surface in one of my classes and 
 the SDL_Surface contains (obviously) memory to the pixel 
 data. Since I cannot free this memory with the DTor: what 
 will happen? AFAIK this cannot be freed by the GC because it 
 was not allocated by it. So AFAIK this creates a memory leak. 
 Am I right?


 Why can't you free the memory from the destructor?


 Every time I tried this, I got this: 
 core.exception.InvalidMemoryOperationError
 I do not know if I get that, because the GC has already cleaned 
 this store or because of something else.


 
 Sounds like a bug - if the pointer is not into GC memory then the 
 GC shouldn't touch it.


You have to provide some more details here:

* How was the SDL_Surface allocated?
* What function do you use to free it?
* When does the InvalidMemoryOperationError occur? Exactly when you
  call the free funtion or later (likely a garbage collection run)?

I guess you got the SDL_Surface from SetVideoMode or some similar
SDL function. Then it was allocated by SDL and likely not with the GC
(unless SDL allows to provide custom allocators and you did that). You
probably tried to free it with GC.free or 'delete' which only works for
GC allocated memory?

Anyway, if the Surface was allocated by SDL you should use
http://sdl.beuc.net/sdl.wiki/SDL_FreeSurface to free it. You can also
call this function in a destructor, you just shouldn't free GC memory
in a destructor.
May 05 2013
parent reply "Namespace" <rswhite4 googlemail.com> writes:
Here a test example:
http://dpaste.1azy.net/2cfc8ead

The memory is allocated through the SDL as you can see.
May 05 2013
next sibling parent Johannes Pfau <nospam example.com> writes:
Am Sun, 05 May 2013 11:52:52 +0200
schrieb "Namespace" <rswhite4 googlemail.com>:


 Here a test example:
 http://dpaste.1azy.net/2cfc8ead
 
 The memory is allocated through the SDL as you can see.


Sorry for this late reply, I actually hoped someone else would coma up
with a solution. I don't have SDL2 here so I can't test that program
but it looks correct to me.

Does this also happen if the GC collection is not at program exit? So
if you place the surface creation to a separate function, make sure
it's no longer reachable and then force a GC collection, will it still
cause the Exception?

Are you sure you can still access the surface (i.e. call FreeSurface)
after you already called SDL_DestroyWindow on the last window? I don't
know SDL enough, but that sounds a little dangerous to me.
May 11 2013
prev sibling parent reply "Mike Parker" <aldacron gmail.com> writes:
On Sunday, 5 May 2013 at 09:52:53 UTC, Namespace wrote:

 Here a test example:
 http://dpaste.1azy.net/2cfc8ead

 The memory is allocated through the SDL as you can see.


1) You don't need to call unload on any Derelict loaders. That is 
handled automatically by Derelict using static module destructors 
just as you have here. And that leads to

2) Module destructors are run before the gc runs its final 
collection at shutdown. So if in a destructor you try to call 
into one of the libraries Derelict binds, you will be calling 
into a library that is already unloaded, hence the error.

Never rely on destructors to release memory. Aside from this 
Derelict-specific issue, you have no control over when, if, or in 
what order the destructors are called. This can lead to all sorts 
of subtle bugs.

The IMO ideal way to handle this is to give your app a 
terminate() function somewhere that is always called before the 
app exits and which initiates a clean release of system 
resources. Assuming a game:

****
void main() {
     initialize();
     run();
     scope(exit) terminate();
}

void terminate() {
     Game.terminate();
     Audio.terminate();
     Network.terminate();
     Graphics.terminate();
     ...
}
****

I adopted this style long ago in C and it applies well to D. I 
never rely on destructors except for logging, statistics, or 
other non-critical things. I release all C-side memory through a 
terminate chain.
May 12 2013
next sibling parent "Mike Parker" <aldacron gmail.com> writes:
I should expand a bit. The heart of the issue in this case is 
that you need precise control over the order of deallocation: 
resource allocated through the shared libraries need to be 
deallocated before the libraries are unloaded. Even without the 
static module destructors, you can't rely on normal destructors 
to do that.

I could just entirely skip unloading the libraries. The OS should 
handle that automatically when the process exits, meaning the GC 
should have done its final shutdown by then. I've thought about 
implementing such a change before, as this issue keeps cropping 
up around Derelict because so many people start out relying on 
D's destructors to release memory for them. Then again, the up 
side is that it highlights one of the issues of relying on 
destructors for this sort of thing. Another being that your 
SDL_Surface might never be deallocated at all until the app exits 
if you only rely on destructors, which could have a negative 
impact in a long running game. And yet another being that the GC 
doesn't know about the memory your SDL_Surface points to and 
could happily destruct your object, thereby freeing the surcface, 
while another pointer to it exists somewhere else in the program. 
Better just to use a terminate chain.
May 12 2013
prev sibling parent reply "Namespace" <rswhite4 googlemail.com> writes:
 1) You don't need to call unload on any Derelict loaders. That 
 is handled automatically by Derelict using static module 
 destructors just as you have here. And that leads to


That good to know.


 2) Module destructors are run before the gc runs its final 
 collection at shutdown. So if in a destructor you try to call 
 into one of the libraries Derelict binds, you will be calling 
 into a library that is already unloaded, hence the error.


As I thought. But I have currently no real solution for that.


 Never rely on destructors to release memory. Aside from this 
 Derelict-specific issue, you have no control over when, if, or 
 in what order the destructors are called. This can lead to all 
 sorts of subtle bugs.


Never heard of RAII? ;) D structs should be able to do this 
technique.
That's why I do this that way. It's more comfortable and you 
don't forget to free memory.


 The IMO ideal way to handle this is to give your app a 
 terminate() function somewhere that is always called before the 
 app exits and which initiates a clean release of system 
 resources. Assuming a game:

 ****
 void main() {
     initialize();
     run();
     scope(exit) terminate();
 }

 void terminate() {
     Game.terminate();
     Audio.terminate();
     Network.terminate();
     Graphics.terminate();
     ...
 }
 ****

 I adopted this style long ago in C and it applies well to D. I 
 never rely on destructors except for logging, statistics, or 
 other non-critical things. I release all C-side memory through 
 a terminate chain.


Well, maybe I could adapt it to my problem. It's a bit annoying 
that you have to call it by yourself.  :)
May 12 2013
parent reply "Namespace" <rswhite4 googlemail.com> writes:
I get the error, although I don't call any unload method (or quit 
any SDL component) and although I recompiled derelict3, where I 
comment out all static (shared) DTor's.
Maybe the only solution would be (as you said) to transfer all 
deallocations into a terminate method, but that is no opinion for 
me (and would cause a growing memory, if the game runs a long 
time). And I want to use RAII. So I must live with this invalid 
memory error.
But thanks to you and all others for your help. :)
May 12 2013
next sibling parent reply "Mike Parker" <aldacron gmail.com> writes:
On Sunday, 12 May 2013 at 10:11:57 UTC, Namespace wrote:

 I get the error, although I don't call any unload method (or 
 quit any SDL component) and although I recompiled derelict3, 
 where I comment out all static (shared) DTor's.
 Maybe the only solution would be (as you said) to transfer all 
 deallocations into a terminate method, but that is no opinion 
 for me (and would cause a growing memory, if the game runs a 
 long time). And I want to use RAII. So I must live with this 
 invalid memory error.


I think you need to reconsider your use-case for RAII here. 
Wanting to use it is fine, but letting these errors persist just 
so you can do so is not something I would recommend.

What you are missing here is that there's no such thing as that 
this "(and would cause a growing memory, if the game runs a long 
time)" is an issue you are likely to run into by using D's class 
destructors to handle resource deallocation (as I mentioned in a 
previous post), but not with the system I describe here.

You're misunderstanding the purpose of the terminate call. It's 
there not to release all the memory your app ever uses, but to 
clean up any resources /still hanging around/ at shutdown. You 
need to pay special attention to memory management when 
developing a game. RAII isn't going to help you with D's classes, 
but you can still get a comfortable system in place that keeps 
deallocations localized in specific places, allowing you to free 
as few or as many resources as you want. You're likely already 
using some sort of manager structure for most of your game 
systems anyway. Even in a simple Asteroids game you'd have a list 
of Sprites that are active. That list becomes your record of 
resources that need to be deallocated. So just add one function 
that does that, make sure it's called at shutdown, and now you've 
avoided this error you're having. By going with this approach, 
you can have exact control over when resources are released and 
in what order. That should always be an overriding goal in game 
development.

I promise you, this will save you a lot of headaches down the 
road by eliminating a whole class of potential memory-related 
bugs (and they will pop up eventually) and code maintenance 
costs. One of the side-effects of mixing memory managed by the GC 
with that allocated outside of the GC is that you have to be 
responsible for the latter.
May 12 2013
next sibling parent "Mike Parker" <aldacron gmail.com> writes:
On Sunday, 12 May 2013 at 14:06:49 UTC, Mike Parker wrote:



 What you are missing here is that there's no such thing as that 
 this "(and would cause a growing memory, if the game runs a 
 long time)" is an issue you are likely to run into by using D's


This was the result of a botched edit. You can ignore everything 
from "what" to "that" and just start with "this".
May 12 2013
prev sibling next sibling parent "Mike Parker" <aldacron gmail.com> writes:
On Sunday, 12 May 2013 at 14:06:49 UTC, Mike Parker wrote:



 What you are missing here is that there's no such thing as that 
 this "(and would cause a growing memory, if the game runs a 
 long time)" is an issue you are likely to run into by using D's


This was the result of a botched edit. You can ignore everything
from "what" to "that" and just start with "this".
May 12 2013
prev sibling parent "Namespace" <rswhite4 googlemail.com> writes:
I use RAII of course only with D structs, not with classes.
Tonight I will take a closer look what the source of evil is.
May 12 2013
prev sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 12.05.2013 12:11, schrieb Namespace:

 I get the error, although I don't call any unload method (or quit any
 SDL component) and although I recompiled derelict3, where I comment out
 all static (shared) DTor's.
 Maybe the only solution would be (as you said) to transfer all
 deallocations into a terminate method, but that is no opinion for me
 (and would cause a growing memory, if the game runs a long time). And I
 want to use RAII. So I must live with this invalid memory error.
 But thanks to you and all others for your help. :)


Do you have multiple threads in your application? If you do so it is 
possible that you (or the derelict library) does a API call in a 
different then the main thread which might lead to the error you 
describe. As the GC runs the destructors in any arbitrary thread you can 
not free any SDL resources inside a destructor.

-- 
Kind Regards
Benjamin Thaut
May 12 2013
parent "Namespace" <rswhite4 googlemail.com> writes:
 Do you have multiple threads in your application? If you do so 
 it is possible that you (or the derelict library) does a API 
 call in a different then the main thread which might lead to 
 the error you describe. As the GC runs the destructors in any 
 arbitrary thread you can not free any SDL resources inside a 
 destructor.


No, I have no other threads. But currently I'm working on my VBO, 
so it could take some time until I could search for the problem 
and a possible solution.
May 12 2013