digitalmars.D.learn - I left my program open for 9 hours and it used up 700mb of ram, could
- Gan (6/6) Jan 27 2015 I feel like the only way I can get better right now is if someone
- bearophile (14/15) Jan 27 2015 Most tiny classes probably should be structs. More generally, use
- Gan (5/20) Jan 27 2015 Is there some special stuff I gotta do extra with structs? Do
- bearophile (7/11) Jan 27 2015 Most of your usages of tiny structs should be by value. So just
- Gan (5/16) Jan 27 2015 Thanks. I'll give structs a try.
- Gan (3/25) Jan 27 2015 Curiously, my CPU usage went from 10% to 5% after I changed to
- Rikki Cattermole (2/29) Jan 27 2015 Force a GC.collect() now and again. Disable it at the beginning too.
- Gan (7/45) Jan 27 2015 I did a test and ran GC.collect() every loop but my memory usage
- Rikki Cattermole (8/45) Jan 27 2015 I had a quick look at your code.
- Gan (48/95) Jan 27 2015 I commented out some stuff and it appears my massive memory
- Rikki Cattermole (5/92) Jan 27 2015 Try with only drawStart stripped.
- Gan (5/119) Jan 27 2015 Without drawStar, my program starts at 24mb and slowly increases.
- FG (3/6) Jan 27 2015 Looks OK, so probably it is not the cause by itself.
- Vladimir Panteleev (22/23) Jan 27 2015 Hi,
I feel like the only way I can get better right now is if someone with more knowledge can give me some advice on the code I have written. Here's the link: http://cl.ly/0s0Q1L1S3v0E How can I make it use less CPU/RAM? (Most code is in the Misc/SpaceBackground.d)
Jan 27 2015
Gan:How can I make it use less CPU/RAM?Most tiny classes probably should be structs. More generally, use a struct every time you don't need a class. You can start with those two: struct SBRange { double left = 0.0, right = 0.0, top = 0.0, bottom = 0.0; } struct Point(T) { T x, y; } This probably isn't enough to solve your problems, but it's a start. Bye, bearophile
Jan 27 2015
On Tuesday, 27 January 2015 at 15:45:47 UTC, bearophile wrote:Gan:Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released? On a second question, do I ever need to manually release objects I create with new?How can I make it use less CPU/RAM?Most tiny classes probably should be structs. More generally, use a struct every time you don't need a class. You can start with those two: struct SBRange { double left = 0.0, right = 0.0, top = 0.0, bottom = 0.0; } struct Point(T) { T x, y; } This probably isn't enough to solve your problems, but it's a start. Bye, bearophile
Jan 27 2015
Gan:Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:Gan:Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.Gan:Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On 28/01/2015 9:59 a.m., Gan wrote:On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:Force a GC.collect() now and again. Disable it at the beginning too.On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.Gan:Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole wrote:On 28/01/2015 9:59 a.m., Gan wrote:I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:Force a GC.collect() now and again. Disable it at the beginning too.On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.Gan:Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On 28/01/2015 11:30 a.m., Gan wrote:On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole wrote:I had a quick look at your code. I think its safe to assume the SFML isn't the problem. Whats happening is on every iteration of the event loop, you are allocating, holding a reference somewhere and continuing on. The GC even if it does run cannot release that memory as it is still held. At this point maybe strip out what happens on each loop iteration to find out what is holding references. Divide and conquer.On 28/01/2015 9:59 a.m., Gan wrote:I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:Force a GC.collect() now and again. Disable it at the beginning too.On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.Gan:Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On Tuesday, 27 January 2015 at 22:30:13 UTC, Gan wrote:On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole wrote:I commented out some stuff and it appears my massive memory consumption comes from my tile.redraw function: void redraw() { undrawn = false; canvas.clear(); //Add stars for (int i = 0; i < controller.starsPerTile; i++) { Random gen; gen.seed(unpredictableSeed); int x = uniform(0, controller.tileSize, gen); int y = uniform(0, controller.tileSize, gen); double s = uniform(0, 10000, gen) / 10000.0; double size = (s * (controller.starSizeMax - controller.starSizeMin)) + controller.starSizeMin; if (x - size / 2 < 0) { x += size / 2; } if (y - size / 2 < 0) { y += size / 2; } if (x + size / 2 > controller.tileSize) { x -= size / 2; } if (y + size / 2 > controller.tileSize) { y -= size / 2; } drawStar(canvas, x, y, size, size); } canvas.display(); } void drawStar(RenderTarget target, int centerX, int centerY, double width, double height) { CircleShape star; if (controller.multiColor == false) { star = controller.stars[0]; } else { Random gen; gen.seed(unpredictableSeed); star = controller.stars[uniform(0, controller.stars.length - 1, gen)]; } star.position = Vector2f(centerX, centerY); star.origin = Vector2f(0.5, 0.5); star.scale = Vector2f(width / 100.0, height / 100.0); target.draw(star); } Would you know why this is using hundreds of mb of rams?On 28/01/2015 9:59 a.m., Gan wrote:I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:Force a GC.collect() now and again. Disable it at the beginning too.On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.Gan:Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On 28/01/2015 11:39 a.m., Gan wrote:On Tuesday, 27 January 2015 at 22:30:13 UTC, Gan wrote:Try with only drawStart stripped. If it still does it, then its redraw. Also try with just a hard coded call to drawStart. Something smells wrong here.On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole wrote:I commented out some stuff and it appears my massive memory consumption comes from my tile.redraw function: void redraw() { undrawn = false; canvas.clear(); //Add stars for (int i = 0; i < controller.starsPerTile; i++) { Random gen; gen.seed(unpredictableSeed); int x = uniform(0, controller.tileSize, gen); int y = uniform(0, controller.tileSize, gen); double s = uniform(0, 10000, gen) / 10000.0; double size = (s * (controller.starSizeMax - controller.starSizeMin)) + controller.starSizeMin; if (x - size / 2 < 0) { x += size / 2; } if (y - size / 2 < 0) { y += size / 2; } if (x + size / 2 > controller.tileSize) { x -= size / 2; } if (y + size / 2 > controller.tileSize) { y -= size / 2; } drawStar(canvas, x, y, size, size); } canvas.display(); } void drawStar(RenderTarget target, int centerX, int centerY, double width, double height) { CircleShape star; if (controller.multiColor == false) { star = controller.stars[0]; } else { Random gen; gen.seed(unpredictableSeed); star = controller.stars[uniform(0, controller.stars.length - 1, gen)]; } star.position = Vector2f(centerX, centerY); star.origin = Vector2f(0.5, 0.5); star.scale = Vector2f(width / 100.0, height / 100.0); target.draw(star); } Would you know why this is using hundreds of mb of rams?On 28/01/2015 9:59 a.m., Gan wrote:I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:Force a GC.collect() now and again. Disable it at the beginning too.On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.Gan:Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On Tuesday, 27 January 2015 at 22:42:25 UTC, Rikki Cattermole wrote:On 28/01/2015 11:39 a.m., Gan wrote:Without drawStar, my program starts at 24mb and slowly increases. With drawStar my program starts at 70mb and increases at twice the rate.On Tuesday, 27 January 2015 at 22:30:13 UTC, Gan wrote:Try with only drawStart stripped. If it still does it, then its redraw. Also try with just a hard coded call to drawStart. Something smells wrong here.On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole wrote:I commented out some stuff and it appears my massive memory consumption comes from my tile.redraw function: void redraw() { undrawn = false; canvas.clear(); //Add stars for (int i = 0; i < controller.starsPerTile; i++) { Random gen; gen.seed(unpredictableSeed); int x = uniform(0, controller.tileSize, gen); int y = uniform(0, controller.tileSize, gen); double s = uniform(0, 10000, gen) / 10000.0; double size = (s * (controller.starSizeMax - controller.starSizeMin)) + controller.starSizeMin; if (x - size / 2 < 0) { x += size / 2; } if (y - size / 2 < 0) { y += size / 2; } if (x + size / 2 > controller.tileSize) { x -= size / 2; } if (y + size / 2 > controller.tileSize) { y -= size / 2; } drawStar(canvas, x, y, size, size); } canvas.display(); } void drawStar(RenderTarget target, int centerX, int centerY, double width, double height) { CircleShape star; if (controller.multiColor == false) { star = controller.stars[0]; } else { Random gen; gen.seed(unpredictableSeed); star = controller.stars[uniform(0, controller.stars.length - 1, gen)]; } star.position = Vector2f(centerX, centerY); star.origin = Vector2f(0.5, 0.5); star.scale = Vector2f(width / 100.0, height / 100.0); target.draw(star); } Would you know why this is using hundreds of mb of rams?On 28/01/2015 9:59 a.m., Gan wrote:I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:Force a GC.collect() now and again. Disable it at the beginning too.On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.Gan:Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released?Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)On a second question, do I ever need to manually release objects I create with new?Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
On 2015-01-27 at 23:39, Gan wrote:I commented out some stuff and it appears my massive memory consumption comes from my tile.redraw function: ... Would you know why this is using hundreds of mb of rams?Looks OK, so probably it is not the cause by itself. I would add a piece of code to SpaceBackground.draw that for every few calls to this function would print out tiles.length and stack.length to the console. Just to make sure that those arrays don't grow out of control, before accusing anything else.
Jan 27 2015
On Tuesday, 27 January 2015 at 22:39:31 UTC, Gan wrote:Would you know why this is using hundreds of mb of rams?Hi, What type is CircleShape? If it is a class, or otherwise contains pointers, then this is probably the source of your problem. You are storing high-entropy data (floating-point / random numbers) within the same type as one containing pointers. This is problematic with a non-precise GC, because the GC will consider the random numbers as possibly pointers, thus pinning random objects within the memory address space. You should be able to work around this problem by: - Making CircleShape a static struct without any pointers - If you need to have pointers associated with CircleShape: splitting pointers and non-pointers into separate arrays of structs - Manually managing memory allocations, and storing the CircleShape instances outside the managed D heap - Building your program for x86_64 - 64 bits of address space will make fake pointer pinning very unlikely - Using a precise GC - none are currently available for D, but one is under development: https://github.com/D-Programming-Language/druntime/pull/1057
Jan 27 2015
On 2015-01-28 at 03:04, Vladimir Panteleev wrote:What type is CircleShape? If it is a class, or otherwise contains pointers, then this is probably the source of your problem.class CircleShape : Shape is defined in dsfml.graphics.circleshape, so there's no going around this...- Building your program for x86_64 - 64 bits of address space will make fake pointer pinning very unlikelyThe binary included in the zip was 64-bit, so fake pointers shouldn't be that much of a problem. Oh, and I take back what I said about suspecting that SpaceBackground.stack grows infinitely. It probably doesn't. I don't have DSFML installed, and therefore couldn't recreate the behaviour.
Jan 27 2015
On Wednesday, 28 January 2015 at 02:50:11 UTC, FG wrote:On 2015-01-28 at 03:04, Vladimir Panteleev wrote:Is there a way to set a variable to be cleared in the new GC collection or to forcibly release a variable? I have an inkling that it might be the Sprite class or something with the RenderTextures.What type is CircleShape? If it is a class, or otherwise contains pointers, then this is probably the source of your problem.class CircleShape : Shape is defined in dsfml.graphics.circleshape, so there's no going around this...- Building your program for x86_64 - 64 bits of address space will make fake pointer pinning very unlikelyThe binary included in the zip was 64-bit, so fake pointers shouldn't be that much of a problem. Oh, and I take back what I said about suspecting that SpaceBackground.stack grows infinitely. It probably doesn't. I don't have DSFML installed, and therefore couldn't recreate the behaviour.
Jan 27 2015