www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - C Memory

reply "Namespace" <rswhite4 googlemail.com> writes:
Quick question: I have a SDL_Surface in one of my classes and the 
SDL_Surface contains (obviously) memory to the pixel data. Since 
I cannot free this memory with the DTor: what will happen? AFAIK 
this cannot be freed by the GC because it was not allocated by 
it. So AFAIK this creates a memory leak. Am I right?
May 04 2013
parent reply "Diggory" <diggsey googlemail.com> writes:
On Sunday, 5 May 2013 at 06:35:38 UTC, Namespace wrote:
 Quick question: I have a SDL_Surface in one of my classes and 
 the SDL_Surface contains (obviously) memory to the pixel data. 
 Since I cannot free this memory with the DTor: what will 
 happen? AFAIK this cannot be freed by the GC because it was not 
 allocated by it. So AFAIK this creates a memory leak. Am I 
 right?
Why can't you free the memory from the destructor?
May 04 2013
parent reply "Namespace" <rswhite4 googlemail.com> writes:
On Sunday, 5 May 2013 at 06:43:17 UTC, Diggory wrote:
 On Sunday, 5 May 2013 at 06:35:38 UTC, Namespace wrote:
 Quick question: I have a SDL_Surface in one of my classes and 
 the SDL_Surface contains (obviously) memory to the pixel data. 
 Since I cannot free this memory with the DTor: what will 
 happen? AFAIK this cannot be freed by the GC because it was 
 not allocated by it. So AFAIK this creates a memory leak. Am I 
 right?
Why can't you free the memory from the destructor?
Every time I tried this, I got this: core.exception.InvalidMemoryOperationError I do not know if I get that, because the GC has already cleaned this store or because of something else.
May 05 2013
parent reply "Diggory" <diggsey googlemail.com> writes:
On Sunday, 5 May 2013 at 07:23:25 UTC, Namespace wrote:
 On Sunday, 5 May 2013 at 06:43:17 UTC, Diggory wrote:
 On Sunday, 5 May 2013 at 06:35:38 UTC, Namespace wrote:
 Quick question: I have a SDL_Surface in one of my classes and 
 the SDL_Surface contains (obviously) memory to the pixel 
 data. Since I cannot free this memory with the DTor: what 
 will happen? AFAIK this cannot be freed by the GC because it 
 was not allocated by it. So AFAIK this creates a memory leak. 
 Am I right?
Why can't you free the memory from the destructor?
Every time I tried this, I got this: core.exception.InvalidMemoryOperationError I do not know if I get that, because the GC has already cleaned this store or because of something else.
Sounds like a bug - if the pointer is not into GC memory then the GC shouldn't touch it.
May 05 2013
parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 05 May 2013 09:28:05 +0200
schrieb "Diggory" <diggsey googlemail.com>:

 On Sunday, 5 May 2013 at 07:23:25 UTC, Namespace wrote:
 On Sunday, 5 May 2013 at 06:43:17 UTC, Diggory wrote:
 On Sunday, 5 May 2013 at 06:35:38 UTC, Namespace wrote:
 Quick question: I have a SDL_Surface in one of my classes and 
 the SDL_Surface contains (obviously) memory to the pixel 
 data. Since I cannot free this memory with the DTor: what 
 will happen? AFAIK this cannot be freed by the GC because it 
 was not allocated by it. So AFAIK this creates a memory leak. 
 Am I right?
Why can't you free the memory from the destructor?
Every time I tried this, I got this: core.exception.InvalidMemoryOperationError I do not know if I get that, because the GC has already cleaned this store or because of something else.
Sounds like a bug - if the pointer is not into GC memory then the GC shouldn't touch it.
You have to provide some more details here: * How was the SDL_Surface allocated? * What function do you use to free it? * When does the InvalidMemoryOperationError occur? Exactly when you call the free funtion or later (likely a garbage collection run)? I guess you got the SDL_Surface from SetVideoMode or some similar SDL function. Then it was allocated by SDL and likely not with the GC (unless SDL allows to provide custom allocators and you did that). You probably tried to free it with GC.free or 'delete' which only works for GC allocated memory? Anyway, if the Surface was allocated by SDL you should use http://sdl.beuc.net/sdl.wiki/SDL_FreeSurface to free it. You can also call this function in a destructor, you just shouldn't free GC memory in a destructor.
May 05 2013
parent reply "Namespace" <rswhite4 googlemail.com> writes:
Here a test example:
http://dpaste.1azy.net/2cfc8ead

The memory is allocated through the SDL as you can see.
May 05 2013
next sibling parent Johannes Pfau <nospam example.com> writes:
Am Sun, 05 May 2013 11:52:52 +0200
schrieb "Namespace" <rswhite4 googlemail.com>:

 Here a test example:
 http://dpaste.1azy.net/2cfc8ead
 
 The memory is allocated through the SDL as you can see.
Sorry for this late reply, I actually hoped someone else would coma up with a solution. I don't have SDL2 here so I can't test that program but it looks correct to me. Does this also happen if the GC collection is not at program exit? So if you place the surface creation to a separate function, make sure it's no longer reachable and then force a GC collection, will it still cause the Exception? Are you sure you can still access the surface (i.e. call FreeSurface) after you already called SDL_DestroyWindow on the last window? I don't know SDL enough, but that sounds a little dangerous to me.
May 11 2013
prev sibling parent reply "Mike Parker" <aldacron gmail.com> writes:
On Sunday, 5 May 2013 at 09:52:53 UTC, Namespace wrote:
 Here a test example:
 http://dpaste.1azy.net/2cfc8ead

 The memory is allocated through the SDL as you can see.
1) You don't need to call unload on any Derelict loaders. That is handled automatically by Derelict using static module destructors just as you have here. And that leads to 2) Module destructors are run before the gc runs its final collection at shutdown. So if in a destructor you try to call into one of the libraries Derelict binds, you will be calling into a library that is already unloaded, hence the error. Never rely on destructors to release memory. Aside from this Derelict-specific issue, you have no control over when, if, or in what order the destructors are called. This can lead to all sorts of subtle bugs. The IMO ideal way to handle this is to give your app a terminate() function somewhere that is always called before the app exits and which initiates a clean release of system resources. Assuming a game: **** void main() { initialize(); run(); scope(exit) terminate(); } void terminate() { Game.terminate(); Audio.terminate(); Network.terminate(); Graphics.terminate(); ... } **** I adopted this style long ago in C and it applies well to D. I never rely on destructors except for logging, statistics, or other non-critical things. I release all C-side memory through a terminate chain.
May 12 2013
next sibling parent "Mike Parker" <aldacron gmail.com> writes:
I should expand a bit. The heart of the issue in this case is 
that you need precise control over the order of deallocation: 
resource allocated through the shared libraries need to be 
deallocated before the libraries are unloaded. Even without the 
static module destructors, you can't rely on normal destructors 
to do that.

I could just entirely skip unloading the libraries. The OS should 
handle that automatically when the process exits, meaning the GC 
should have done its final shutdown by then. I've thought about 
implementing such a change before, as this issue keeps cropping 
up around Derelict because so many people start out relying on 
D's destructors to release memory for them. Then again, the up 
side is that it highlights one of the issues of relying on 
destructors for this sort of thing. Another being that your 
SDL_Surface might never be deallocated at all until the app exits 
if you only rely on destructors, which could have a negative 
impact in a long running game. And yet another being that the GC 
doesn't know about the memory your SDL_Surface points to and 
could happily destruct your object, thereby freeing the surcface, 
while another pointer to it exists somewhere else in the program. 
Better just to use a terminate chain.
May 12 2013
prev sibling parent reply "Namespace" <rswhite4 googlemail.com> writes:
 1) You don't need to call unload on any Derelict loaders. That 
 is handled automatically by Derelict using static module 
 destructors just as you have here. And that leads to
That good to know.
 2) Module destructors are run before the gc runs its final 
 collection at shutdown. So if in a destructor you try to call 
 into one of the libraries Derelict binds, you will be calling 
 into a library that is already unloaded, hence the error.
As I thought. But I have currently no real solution for that.
 Never rely on destructors to release memory. Aside from this 
 Derelict-specific issue, you have no control over when, if, or 
 in what order the destructors are called. This can lead to all 
 sorts of subtle bugs.
Never heard of RAII? ;) D structs should be able to do this technique. That's why I do this that way. It's more comfortable and you don't forget to free memory.
 The IMO ideal way to handle this is to give your app a 
 terminate() function somewhere that is always called before the 
 app exits and which initiates a clean release of system 
 resources. Assuming a game:

 ****
 void main() {
     initialize();
     run();
     scope(exit) terminate();
 }

 void terminate() {
     Game.terminate();
     Audio.terminate();
     Network.terminate();
     Graphics.terminate();
     ...
 }
 ****

 I adopted this style long ago in C and it applies well to D. I 
 never rely on destructors except for logging, statistics, or 
 other non-critical things. I release all C-side memory through 
 a terminate chain.
Well, maybe I could adapt it to my problem. It's a bit annoying that you have to call it by yourself. :)
May 12 2013
parent reply "Namespace" <rswhite4 googlemail.com> writes:
I get the error, although I don't call any unload method (or quit 
any SDL component) and although I recompiled derelict3, where I 
comment out all static (shared) DTor's.
Maybe the only solution would be (as you said) to transfer all 
deallocations into a terminate method, but that is no opinion for 
me (and would cause a growing memory, if the game runs a long 
time). And I want to use RAII. So I must live with this invalid 
memory error.
But thanks to you and all others for your help. :)
May 12 2013
next sibling parent reply "Mike Parker" <aldacron gmail.com> writes:
On Sunday, 12 May 2013 at 10:11:57 UTC, Namespace wrote:
 I get the error, although I don't call any unload method (or 
 quit any SDL component) and although I recompiled derelict3, 
 where I comment out all static (shared) DTor's.
 Maybe the only solution would be (as you said) to transfer all 
 deallocations into a terminate method, but that is no opinion 
 for me (and would cause a growing memory, if the game runs a 
 long time). And I want to use RAII. So I must live with this 
 invalid memory error.
I think you need to reconsider your use-case for RAII here. Wanting to use it is fine, but letting these errors persist just so you can do so is not something I would recommend. What you are missing here is that there's no such thing as that this "(and would cause a growing memory, if the game runs a long time)" is an issue you are likely to run into by using D's class destructors to handle resource deallocation (as I mentioned in a previous post), but not with the system I describe here. You're misunderstanding the purpose of the terminate call. It's there not to release all the memory your app ever uses, but to clean up any resources /still hanging around/ at shutdown. You need to pay special attention to memory management when developing a game. RAII isn't going to help you with D's classes, but you can still get a comfortable system in place that keeps deallocations localized in specific places, allowing you to free as few or as many resources as you want. You're likely already using some sort of manager structure for most of your game systems anyway. Even in a simple Asteroids game you'd have a list of Sprites that are active. That list becomes your record of resources that need to be deallocated. So just add one function that does that, make sure it's called at shutdown, and now you've avoided this error you're having. By going with this approach, you can have exact control over when resources are released and in what order. That should always be an overriding goal in game development. I promise you, this will save you a lot of headaches down the road by eliminating a whole class of potential memory-related bugs (and they will pop up eventually) and code maintenance costs. One of the side-effects of mixing memory managed by the GC with that allocated outside of the GC is that you have to be responsible for the latter.
May 12 2013
next sibling parent "Mike Parker" <aldacron gmail.com> writes:
On Sunday, 12 May 2013 at 14:06:49 UTC, Mike Parker wrote:


 What you are missing here is that there's no such thing as that 
 this "(and would cause a growing memory, if the game runs a 
 long time)" is an issue you are likely to run into by using D's
This was the result of a botched edit. You can ignore everything from "what" to "that" and just start with "this".
May 12 2013
prev sibling next sibling parent "Mike Parker" <aldacron gmail.com> writes:
On Sunday, 12 May 2013 at 14:06:49 UTC, Mike Parker wrote:


 What you are missing here is that there's no such thing as that 
 this "(and would cause a growing memory, if the game runs a 
 long time)" is an issue you are likely to run into by using D's
This was the result of a botched edit. You can ignore everything from "what" to "that" and just start with "this".
May 12 2013
prev sibling parent "Namespace" <rswhite4 googlemail.com> writes:
I use RAII of course only with D structs, not with classes.
Tonight I will take a closer look what the source of evil is.
May 12 2013
prev sibling parent reply Benjamin Thaut <code benjamin-thaut.de> writes:
Am 12.05.2013 12:11, schrieb Namespace:
 I get the error, although I don't call any unload method (or quit any
 SDL component) and although I recompiled derelict3, where I comment out
 all static (shared) DTor's.
 Maybe the only solution would be (as you said) to transfer all
 deallocations into a terminate method, but that is no opinion for me
 (and would cause a growing memory, if the game runs a long time). And I
 want to use RAII. So I must live with this invalid memory error.
 But thanks to you and all others for your help. :)
Do you have multiple threads in your application? If you do so it is possible that you (or the derelict library) does a API call in a different then the main thread which might lead to the error you describe. As the GC runs the destructors in any arbitrary thread you can not free any SDL resources inside a destructor. -- Kind Regards Benjamin Thaut
May 12 2013
parent "Namespace" <rswhite4 googlemail.com> writes:
 Do you have multiple threads in your application? If you do so 
 it is possible that you (or the derelict library) does a API 
 call in a different then the main thread which might lead to 
 the error you describe. As the GC runs the destructors in any 
 arbitrary thread you can not free any SDL resources inside a 
 destructor.
No, I have no other threads. But currently I'm working on my VBO, so it could take some time until I could search for the problem and a possible solution.
May 12 2013