www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - I left my program open for 9 hours and it used up 700mb of ram, could

reply "Gan" <avisaria me.com> writes:
I feel like the only way I can get better right now is if someone 
with more knowledge can give me some advice on the code I have 
written.

Here's the link: http://cl.ly/0s0Q1L1S3v0E

How can I make it use less CPU/RAM?
(Most code is in the Misc/SpaceBackground.d)
Jan 27 2015
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Gan:

 How can I make it use less CPU/RAM?
Most tiny classes probably should be structs. More generally, use a struct every time you don't need a class. You can start with those two: struct SBRange { double left = 0.0, right = 0.0, top = 0.0, bottom = 0.0; } struct Point(T) { T x, y; } This probably isn't enough to solve your problems, but it's a start. Bye, bearophile
Jan 27 2015
parent reply "Gan" <avisaria me.com> writes:
On Tuesday, 27 January 2015 at 15:45:47 UTC, bearophile wrote:
 Gan:

 How can I make it use less CPU/RAM?
Most tiny classes probably should be structs. More generally, use a struct every time you don't need a class. You can start with those two: struct SBRange { double left = 0.0, right = 0.0, top = 0.0, bottom = 0.0; } struct Point(T) { T x, y; } This probably isn't enough to solve your problems, but it's a start. Bye, bearophile
Is there some special stuff I gotta do extra with structs? Do they need manually allocated and released? On a second question, do I ever need to manually release objects I create with new?
Jan 27 2015
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Gan:

 Is there some special stuff I gotta do extra with structs? Do 
 they need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release 
 objects I create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Jan 27 2015
parent reply "Gan" <avisaria me.com> writes:
On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:
 Gan:

 Is there some special stuff I gotta do extra with structs? Do 
 they need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release 
 objects I create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?
Jan 27 2015
parent reply "Gan" <avisaria me.com> writes:
On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:
 Gan:

 Is there some special stuff I gotta do extra with structs? Do 
 they need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release 
 objects I create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?
Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.
Jan 27 2015
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 28/01/2015 9:59 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:
 Gan:

 Is there some special stuff I gotta do extra with structs? Do they
 need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release objects I
 create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?
Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.
Force a GC.collect() now and again. Disable it at the beginning too.
Jan 27 2015
parent reply "Gan" <avisaria me.com> writes:
On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole 
wrote:
 On 28/01/2015 9:59 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:
 Gan:

 Is there some special stuff I gotta do extra with structs? 
 Do they
 need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release 
 objects I
 create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?
Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.
Force a GC.collect() now and again. Disable it at the beginning too.
I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?
Jan 27 2015
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 28/01/2015 11:30 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole wrote:
 On 28/01/2015 9:59 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:
 Gan:

 Is there some special stuff I gotta do extra with structs? Do they
 need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release objects I
 create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?
Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.
Force a GC.collect() now and again. Disable it at the beginning too.
I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?
I had a quick look at your code. I think its safe to assume the SFML isn't the problem. Whats happening is on every iteration of the event loop, you are allocating, holding a reference somewhere and continuing on. The GC even if it does run cannot release that memory as it is still held. At this point maybe strip out what happens on each loop iteration to find out what is holding references. Divide and conquer.
Jan 27 2015
prev sibling parent reply "Gan" <avisaria me.com> writes:
On Tuesday, 27 January 2015 at 22:30:13 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole 
 wrote:
 On 28/01/2015 9:59 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile 
 wrote:
 Gan:

 Is there some special stuff I gotta do extra with structs? 
 Do they
 need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release 
 objects I
 create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?
Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.
Force a GC.collect() now and again. Disable it at the beginning too.
I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?
I commented out some stuff and it appears my massive memory consumption comes from my tile.redraw function: void redraw() { undrawn = false; canvas.clear(); //Add stars for (int i = 0; i < controller.starsPerTile; i++) { Random gen; gen.seed(unpredictableSeed); int x = uniform(0, controller.tileSize, gen); int y = uniform(0, controller.tileSize, gen); double s = uniform(0, 10000, gen) / 10000.0; double size = (s * (controller.starSizeMax - controller.starSizeMin)) + controller.starSizeMin; if (x - size / 2 < 0) { x += size / 2; } if (y - size / 2 < 0) { y += size / 2; } if (x + size / 2 > controller.tileSize) { x -= size / 2; } if (y + size / 2 > controller.tileSize) { y -= size / 2; } drawStar(canvas, x, y, size, size); } canvas.display(); } void drawStar(RenderTarget target, int centerX, int centerY, double width, double height) { CircleShape star; if (controller.multiColor == false) { star = controller.stars[0]; } else { Random gen; gen.seed(unpredictableSeed); star = controller.stars[uniform(0, controller.stars.length - 1, gen)]; } star.position = Vector2f(centerX, centerY); star.origin = Vector2f(0.5, 0.5); star.scale = Vector2f(width / 100.0, height / 100.0); target.draw(star); } Would you know why this is using hundreds of mb of rams?
Jan 27 2015
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 28/01/2015 11:39 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 22:30:13 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole wrote:
 On 28/01/2015 9:59 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile wrote:
 Gan:

 Is there some special stuff I gotta do extra with structs? Do they
 need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release objects I
 create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?
Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.
Force a GC.collect() now and again. Disable it at the beginning too.
I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?
I commented out some stuff and it appears my massive memory consumption comes from my tile.redraw function: void redraw() { undrawn = false; canvas.clear(); //Add stars for (int i = 0; i < controller.starsPerTile; i++) { Random gen; gen.seed(unpredictableSeed); int x = uniform(0, controller.tileSize, gen); int y = uniform(0, controller.tileSize, gen); double s = uniform(0, 10000, gen) / 10000.0; double size = (s * (controller.starSizeMax - controller.starSizeMin)) + controller.starSizeMin; if (x - size / 2 < 0) { x += size / 2; } if (y - size / 2 < 0) { y += size / 2; } if (x + size / 2 > controller.tileSize) { x -= size / 2; } if (y + size / 2 > controller.tileSize) { y -= size / 2; } drawStar(canvas, x, y, size, size); } canvas.display(); } void drawStar(RenderTarget target, int centerX, int centerY, double width, double height) { CircleShape star; if (controller.multiColor == false) { star = controller.stars[0]; } else { Random gen; gen.seed(unpredictableSeed); star = controller.stars[uniform(0, controller.stars.length - 1, gen)]; } star.position = Vector2f(centerX, centerY); star.origin = Vector2f(0.5, 0.5); star.scale = Vector2f(width / 100.0, height / 100.0); target.draw(star); } Would you know why this is using hundreds of mb of rams?
Try with only drawStart stripped. If it still does it, then its redraw. Also try with just a hard coded call to drawStart. Something smells wrong here.
Jan 27 2015
parent "Gan" <avisaria me.com> writes:
On Tuesday, 27 January 2015 at 22:42:25 UTC, Rikki Cattermole 
wrote:
 On 28/01/2015 11:39 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 22:30:13 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 21:36:51 UTC, Rikki Cattermole 
 wrote:
 On 28/01/2015 9:59 a.m., Gan wrote:
 On Tuesday, 27 January 2015 at 19:59:08 UTC, Gan wrote:
 On Tuesday, 27 January 2015 at 19:26:12 UTC, bearophile 
 wrote:
 Gan:

 Is there some special stuff I gotta do extra with 
 structs? Do they
 need manually allocated and released?
Most of your usages of tiny structs should be by value. So just keep in mind they are values. Even when you iterate with a foreach on a mutable array of them :-)
 On a second question, do I ever need to manually release 
 objects I
 create with new?
Usually not. How much advanced do you want to be? :-) Bye, bearophile
Thanks. I'll give structs a try. When I start the program, it runs fine at 35mb of ram. It only keeps 15 objects stored in the arrays at a time so why do you think my ram usage increases to 700+ after many hours?
Curiously, my CPU usage went from 10% to 5% after I changed to structs on Point and Range. Though my memory still climbs high.
Force a GC.collect() now and again. Disable it at the beginning too.
I did a test and ran GC.collect() every loop but my memory usage continues to rise. I can see how you'd be able to lower CPU usage by running GC.collect() every now and then but right now I'm stuck on the memory issue. Perhaps my problem lies in the C++ library SFML?
I commented out some stuff and it appears my massive memory consumption comes from my tile.redraw function: void redraw() { undrawn = false; canvas.clear(); //Add stars for (int i = 0; i < controller.starsPerTile; i++) { Random gen; gen.seed(unpredictableSeed); int x = uniform(0, controller.tileSize, gen); int y = uniform(0, controller.tileSize, gen); double s = uniform(0, 10000, gen) / 10000.0; double size = (s * (controller.starSizeMax - controller.starSizeMin)) + controller.starSizeMin; if (x - size / 2 < 0) { x += size / 2; } if (y - size / 2 < 0) { y += size / 2; } if (x + size / 2 > controller.tileSize) { x -= size / 2; } if (y + size / 2 > controller.tileSize) { y -= size / 2; } drawStar(canvas, x, y, size, size); } canvas.display(); } void drawStar(RenderTarget target, int centerX, int centerY, double width, double height) { CircleShape star; if (controller.multiColor == false) { star = controller.stars[0]; } else { Random gen; gen.seed(unpredictableSeed); star = controller.stars[uniform(0, controller.stars.length - 1, gen)]; } star.position = Vector2f(centerX, centerY); star.origin = Vector2f(0.5, 0.5); star.scale = Vector2f(width / 100.0, height / 100.0); target.draw(star); } Would you know why this is using hundreds of mb of rams?
Try with only drawStart stripped. If it still does it, then its redraw. Also try with just a hard coded call to drawStart. Something smells wrong here.
Without drawStar, my program starts at 24mb and slowly increases. With drawStar my program starts at 70mb and increases at twice the rate.
Jan 27 2015
prev sibling next sibling parent FG <home fgda.pl> writes:
On 2015-01-27 at 23:39, Gan wrote:
 I commented out some stuff and it appears my massive memory consumption comes
from my tile.redraw function:
 ...
 Would you know why this is using hundreds of mb of rams?
Looks OK, so probably it is not the cause by itself. I would add a piece of code to SpaceBackground.draw that for every few calls to this function would print out tiles.length and stack.length to the console. Just to make sure that those arrays don't grow out of control, before accusing anything else.
Jan 27 2015
prev sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 27 January 2015 at 22:39:31 UTC, Gan wrote:
 Would you know why this is using hundreds of mb of rams?
Hi, What type is CircleShape? If it is a class, or otherwise contains pointers, then this is probably the source of your problem. You are storing high-entropy data (floating-point / random numbers) within the same type as one containing pointers. This is problematic with a non-precise GC, because the GC will consider the random numbers as possibly pointers, thus pinning random objects within the memory address space. You should be able to work around this problem by: - Making CircleShape a static struct without any pointers - If you need to have pointers associated with CircleShape: splitting pointers and non-pointers into separate arrays of structs - Manually managing memory allocations, and storing the CircleShape instances outside the managed D heap - Building your program for x86_64 - 64 bits of address space will make fake pointer pinning very unlikely - Using a precise GC - none are currently available for D, but one is under development: https://github.com/D-Programming-Language/druntime/pull/1057
Jan 27 2015
parent reply FG <home fgda.pl> writes:
On 2015-01-28 at 03:04, Vladimir Panteleev wrote:
 What type is CircleShape?
 If it is a class, or otherwise contains pointers, then this is probably the
source of your problem.
class CircleShape : Shape is defined in dsfml.graphics.circleshape, so there's no going around this...
 - Building your program for x86_64 - 64 bits of address space will make fake
pointer pinning very unlikely
The binary included in the zip was 64-bit, so fake pointers shouldn't be that much of a problem. Oh, and I take back what I said about suspecting that SpaceBackground.stack grows infinitely. It probably doesn't. I don't have DSFML installed, and therefore couldn't recreate the behaviour.
Jan 27 2015
parent "Gan" <avisaria me.com> writes:
On Wednesday, 28 January 2015 at 02:50:11 UTC, FG wrote:
 On 2015-01-28 at 03:04, Vladimir Panteleev wrote:
 What type is CircleShape?
 If it is a class, or otherwise contains pointers, then this is 
 probably the source of your problem.
class CircleShape : Shape is defined in dsfml.graphics.circleshape, so there's no going around this...
 - Building your program for x86_64 - 64 bits of address space 
 will make fake pointer pinning very unlikely
The binary included in the zip was 64-bit, so fake pointers shouldn't be that much of a problem. Oh, and I take back what I said about suspecting that SpaceBackground.stack grows infinitely. It probably doesn't. I don't have DSFML installed, and therefore couldn't recreate the behaviour.
Is there a way to set a variable to be cleared in the new GC collection or to forcibly release a variable? I have an inkling that it might be the Sprite class or something with the RenderTextures.
Jan 27 2015