digitalmars.D.learn - Memory leaks
- develop32 (7/7) Oct 13 2013 Windows task manager shows constantly increasing memory usage of
- Benjamin Thaut (14/20) Oct 13 2013 If your programm is a 32-bit application the GC most likely leaks memory...
- develop32 (9/31) Oct 13 2013 Thanks, will try 64-bit when possible.
- Adam D. Ruppe (6/9) Oct 13 2013 I set breakpoints on gc_malloc and gc_qalloc too. I'm pretty sure
- develop32 (4/13) Oct 13 2013 Thank you, that helped to find it.
- develop32 (4/26) Oct 13 2013 I've added GC.collect and GC.minimize() at the end of each frame,
- Benjamin Thaut (10/32) Oct 13 2013 Well the GC doesn't kick in always. It only collects memory when its
- develop32 (6/51) Oct 13 2013 Indeed, I tried to work around it and use manual memory
- Benjamin Thaut (6/45) Oct 13 2013 Did you profile how much time the GC takes every frame?
- develop32 (4/71) Oct 13 2013 Currently it is 3 ms, would love to drop that to one.
Windows task manager shows constantly increasing memory usage of my D application, yet there is ZERO allocation done by it at the time. I have made a hook inside the rt/lifetime.d for _d_allocmemory, and I do get notified when an array or a class object in constructed. What could be the source of rapidly climbing (hundred kilobytes per second) memory usage?
Oct 13 2013
Am 13.10.2013 14:08, schrieb develop32:Windows task manager shows constantly increasing memory usage of my D application, yet there is ZERO allocation done by it at the time. I have made a hook inside the rt/lifetime.d for _d_allocmemory, and I do get notified when an array or a class object in constructed. What could be the source of rapidly climbing (hundred kilobytes per second) memory usage?If your programm is a 32-bit application the GC most likely leaks memory because it thinks its still referenced. Try compilling your application for 64-bit. If the problem goes away it's not your fault. A possible workaround for this situation is to allocate large blocks of memory which you know will not contain any pointers with GC.malloc manually and specifying the "don't scan" flag. If the problem still exists in a 64-bit application you most likely continue to allocate memory that remains referenced. Make sure you don't have any infinitly growing arrays or other containers that still reference the memory you allocated. -- Kind Regards Benjamin Thaut
Oct 13 2013
On Sunday, 13 October 2013 at 13:18:47 UTC, Benjamin Thaut wrote:Am 13.10.2013 14:08, schrieb develop32:Thanks, will try 64-bit when possible. No, there are no allocations in my code, I have set breakpoints wherever I can throughout it and there are no allocations being made every frame (its a video game). I don't use built-in dynamic array directly, have my own wrapper for it. My question is, are there any more places in druntime that I can change so that I will get a notification when allocation occurs, besides _d_allocmemory?Windows task manager shows constantly increasing memory usage of my D application, yet there is ZERO allocation done by it at the time. I have made a hook inside the rt/lifetime.d for _d_allocmemory, and I do get notified when an array or a class object in constructed. What could be the source of rapidly climbing (hundred kilobytes per second) memory usage?If your programm is a 32-bit application the GC most likely leaks memory because it thinks its still referenced. Try compilling your application for 64-bit. If the problem goes away it's not your fault. A possible workaround for this situation is to allocate large blocks of memory which you know will not contain any pointers with GC.malloc manually and specifying the "don't scan" flag. If the problem still exists in a 64-bit application you most likely continue to allocate memory that remains referenced. Make sure you don't have any infinitly growing arrays or other containers that still reference the memory you allocated.
Oct 13 2013
On Sunday, 13 October 2013 at 13:46:13 UTC, develop32 wrote:My question is, are there any more places in druntime that I can change so that I will get a notification when allocation occurs, besides _d_allocmemory?I set breakpoints on gc_malloc and gc_qalloc too. I'm pretty sure they catch more than d_allocmemory. qalloc in particular catches accidental usage of array literals. Yes, they always allocate at runtime unless specifically marked static... i'd love for that to change!
Oct 13 2013
"develop32" <develop32 gmail.com> On Sunday, 13 October 2013 at 14:01:09 UTC, Adam D. Ruppe wrote:On Sunday, 13 October 2013 at 13:46:13 UTC, develop32 wrote:Thank you, that helped to find it. Its as shame that usage of static arrays doesn't prevent heap allocation.My question is, are there any more places in druntime that I can change so that I will get a notification when allocation occurs, besides _d_allocmemory?I set breakpoints on gc_malloc and gc_qalloc too. I'm pretty sure they catch more than d_allocmemory. qalloc in particular catches accidental usage of array literals. Yes, they always allocate at runtime unless specifically marked static... i'd love for that to change!
Oct 13 2013
On Sunday, 13 October 2013 at 13:18:47 UTC, Benjamin Thaut wrote:Am 13.10.2013 14:08, schrieb develop32:I've added GC.collect and GC.minimize() at the end of each frame, looks like the growth has stopped, so its not a 32-bit problem yet.Windows task manager shows constantly increasing memory usage of my D application, yet there is ZERO allocation done by it at the time. I have made a hook inside the rt/lifetime.d for _d_allocmemory, and I do get notified when an array or a class object in constructed. What could be the source of rapidly climbing (hundred kilobytes per second) memory usage?If your programm is a 32-bit application the GC most likely leaks memory because it thinks its still referenced. Try compilling your application for 64-bit. If the problem goes away it's not your fault. A possible workaround for this situation is to allocate large blocks of memory which you know will not contain any pointers with GC.malloc manually and specifying the "don't scan" flag. If the problem still exists in a 64-bit application you most likely continue to allocate memory that remains referenced. Make sure you don't have any infinitly growing arrays or other containers that still reference the memory you allocated.
Oct 13 2013
Am 13.10.2013 15:52, schrieb develop32:On Sunday, 13 October 2013 at 13:18:47 UTC, Benjamin Thaut wrote:Well the GC doesn't kick in always. It only collects memory when its neccessary (a certain limit is reached). For a game it is usually a good idea to run the GC every frame to avoid the ocasional long pause times that otherwise occur. Or you could work around the GC entierly. Kind Regards Benjamin Thaut -- Kind Regards Benjamin ThautAm 13.10.2013 14:08, schrieb develop32:I've added GC.collect and GC.minimize() at the end of each frame, looks like the growth has stopped, so its not a 32-bit problem yet.Windows task manager shows constantly increasing memory usage of my D application, yet there is ZERO allocation done by it at the time. I have made a hook inside the rt/lifetime.d for _d_allocmemory, and I do get notified when an array or a class object in constructed. What could be the source of rapidly climbing (hundred kilobytes per second) memory usage?If your programm is a 32-bit application the GC most likely leaks memory because it thinks its still referenced. Try compilling your application for 64-bit. If the problem goes away it's not your fault. A possible workaround for this situation is to allocate large blocks of memory which you know will not contain any pointers with GC.malloc manually and specifying the "don't scan" flag. If the problem still exists in a 64-bit application you most likely continue to allocate memory that remains referenced. Make sure you don't have any infinitly growing arrays or other containers that still reference the memory you allocated.
Oct 13 2013
On Sunday, 13 October 2013 at 14:13:39 UTC, Benjamin Thaut wrote:Am 13.10.2013 15:52, schrieb develop32:Indeed, I tried to work around it and use manual memory management a year ago, but everything felt so ugly that I decided to throw that and go the idiomatic way. I'm bit surprised that even with GC running every frame I haven't noticed a drop in performance, even when using more than 1 gb RAM.On Sunday, 13 October 2013 at 13:18:47 UTC, Benjamin Thaut wrote:Well the GC doesn't kick in always. It only collects memory when its neccessary (a certain limit is reached). For a game it is usually a good idea to run the GC every frame to avoid the ocasional long pause times that otherwise occur. Or you could work around the GC entierly. Kind Regards Benjamin ThautAm 13.10.2013 14:08, schrieb develop32:I've added GC.collect and GC.minimize() at the end of each frame, looks like the growth has stopped, so its not a 32-bit problem yet.Windows task manager shows constantly increasing memory usage of my D application, yet there is ZERO allocation done by it at the time. I have made a hook inside the rt/lifetime.d for _d_allocmemory, and I do get notified when an array or a class object in constructed. What could be the source of rapidly climbing (hundred kilobytes per second) memory usage?If your programm is a 32-bit application the GC most likely leaks memory because it thinks its still referenced. Try compilling your application for 64-bit. If the problem goes away it's not your fault. A possible workaround for this situation is to allocate large blocks of memory which you know will not contain any pointers with GC.malloc manually and specifying the "don't scan" flag. If the problem still exists in a 64-bit application you most likely continue to allocate memory that remains referenced. Make sure you don't have any infinitly growing arrays or other containers that still reference the memory you allocated.
Oct 13 2013
Am 13.10.2013 16:23, schrieb develop32:On Sunday, 13 October 2013 at 14:13:39 UTC, Benjamin Thaut wrote:Did you profile how much time the GC takes every frame? For me that was 8-9 ms every frame. That was not acceptable for my case. -- Kind Regards Benjamin ThautAm 13.10.2013 15:52, schrieb develop32:Indeed, I tried to work around it and use manual memory management a year ago, but everything felt so ugly that I decided to throw that and go the idiomatic way. I'm bit surprised that even with GC running every frame I haven't noticed a drop in performance, even when using more than 1 gb RAM.On Sunday, 13 October 2013 at 13:18:47 UTC, Benjamin Thaut wrote:Well the GC doesn't kick in always. It only collects memory when its neccessary (a certain limit is reached). For a game it is usually a good idea to run the GC every frame to avoid the ocasional long pause times that otherwise occur. Or you could work around the GC entierly. Kind Regards Benjamin ThautAm 13.10.2013 14:08, schrieb develop32:I've added GC.collect and GC.minimize() at the end of each frame, looks like the growth has stopped, so its not a 32-bit problem yet.Windows task manager shows constantly increasing memory usage of my D application, yet there is ZERO allocation done by it at the time. I have made a hook inside the rt/lifetime.d for _d_allocmemory, and I do get notified when an array or a class object in constructed. What could be the source of rapidly climbing (hundred kilobytes per second) memory usage?If your programm is a 32-bit application the GC most likely leaks memory because it thinks its still referenced. Try compilling your application for 64-bit. If the problem goes away it's not your fault. A possible workaround for this situation is to allocate large blocks of memory which you know will not contain any pointers with GC.malloc manually and specifying the "don't scan" flag. If the problem still exists in a 64-bit application you most likely continue to allocate memory that remains referenced. Make sure you don't have any infinitly growing arrays or other containers that still reference the memory you allocated.
Oct 13 2013
On Sunday, 13 October 2013 at 14:31:13 UTC, Benjamin Thaut wrote:Am 13.10.2013 16:23, schrieb develop32:Currently it is 3 ms, would love to drop that to one. Though I'm using a very slow laptop, can't play any AAA game released in last 5 years.On Sunday, 13 October 2013 at 14:13:39 UTC, Benjamin Thaut wrote:Did you profile how much time the GC takes every frame? For me that was 8-9 ms every frame. That was not acceptable for my case.Am 13.10.2013 15:52, schrieb develop32:Indeed, I tried to work around it and use manual memory management a year ago, but everything felt so ugly that I decided to throw that and go the idiomatic way. I'm bit surprised that even with GC running every frame I haven't noticed a drop in performance, even when using more than 1 gb RAM.On Sunday, 13 October 2013 at 13:18:47 UTC, Benjamin Thaut wrote:Well the GC doesn't kick in always. It only collects memory when its neccessary (a certain limit is reached). For a game it is usually a good idea to run the GC every frame to avoid the ocasional long pause times that otherwise occur. Or you could work around the GC entierly. Kind Regards Benjamin ThautAm 13.10.2013 14:08, schrieb develop32:I've added GC.collect and GC.minimize() at the end of each frame, looks like the growth has stopped, so its not a 32-bit problem yet.Windows task manager shows constantly increasing memory usage of my D application, yet there is ZERO allocation done by it at the time. I have made a hook inside the rt/lifetime.d for _d_allocmemory, and I do get notified when an array or a class object in constructed. What could be the source of rapidly climbing (hundred kilobytes per second) memory usage?If your programm is a 32-bit application the GC most likely leaks memory because it thinks its still referenced. Try compilling your application for 64-bit. If the problem goes away it's not your fault. A possible workaround for this situation is to allocate large blocks of memory which you know will not contain any pointers with GC.malloc manually and specifying the "don't scan" flag. If the problem still exists in a 64-bit application you most likely continue to allocate memory that remains referenced. Make sure you don't have any infinitly growing arrays or other containers that still reference the memory you allocated.
Oct 13 2013