www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Game and GC

reply Leonardo <leotada523 gmail.com> writes:
Hi, I'm new to language and games.
Many people say that GC is bad and can slow down your project in 
some moments.
What can happen if I create a game using D without worrying with 
memory management?
(using full GC)
Feb 22
next sibling parent reply StickYourLeftFootIn <The hole.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
What do you think will happen? Anytime you delegate power to something else what can go wrong? Nothing is perfect. The GC exists to automate a job. The job it does is not the problem... It does it well. The issue is when it does it. It's like the noisy garbage man coming in as 3AM to get your trash... are you ok with that? Some people are.
Feb 22
parent Leonardo <leotada523 gmail.com> writes:
On Friday, 23 February 2018 at 02:02:12 UTC, StickYourLeftFootIn 
wrote:
 On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
What do you think will happen? Anytime you delegate power to something else what can go wrong? Nothing is perfect. The GC exists to automate a job. The job it does is not the problem... It does it well. The issue is when it does it. It's like the noisy garbage man coming in as 3AM to get your trash... are you ok with that? Some people are.
I understand, I'm not saying GC is bad, but I want to know if it would be slow to the point of being noticeable in a game. If so, what is the best way to do this? Placing nogc everywhere? Thanks.
Feb 22
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, February 23, 2018 01:54:07 Leonardo via Digitalmars-d-learn 
wrote:
 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project in
 some moments.
 What can happen if I create a game using D without worrying with
 memory management?
 (using full GC)
The GC won't slow down your code in general (in fact, it will probably speed it up in comparison to reference counting), but whenever the GC does a collection, that means that it stops all threads that it manages. So, you could suddenly have everything stop for 100ms (the actual length of a collection is going to depend on how much memory the GC has to scan, and I don't know what the typical length of a collection is; that will depend on the program). For programs that can afford to occasionally stop like that, that's not a problem. For a game that's trying to maintain 60fps, that's likely a really big deal. There are a number of ways to handle it, though the biggest is to simply minimize how much you allocate on the GC heap and how much memory has to be scanned for pointers that refer to GC-allocated memory. Other stuff includes disabling the GC while critical pieces of code are running and having critical threads not be managed by the GC or use GC-allocated memory. I would suggest that you read this series of articles on the official D blog: https://dlang.org/blog/the-gc-series/ - Jonathan M Davis
Feb 22
parent Leonardo <leotada523 gmail.com> writes:
On Friday, 23 February 2018 at 02:16:38 UTC, Jonathan M Davis 
wrote:
 The GC won't slow down your code in general (in fact, it will 
 probably speed it up in comparison to reference counting), but 
 whenever the GC does a collection, that means that it stops all 
 threads that it manages. So, you could suddenly have everything 
 stop for 100ms (the actual length of a collection is going to 
 depend on how much memory the GC has to scan, and I don't know 
 what the typical length of a collection is; that will depend on 
 the program). For programs that can afford to occasionally stop 
 like that, that's not a problem. For a game that's trying to 
 maintain 60fps, that's likely a really big deal.
 - Jonathan M Davis
That's what I thought for a game, but probably no one tested yet to see the impact. Thanks, I'll read on.
Feb 22
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
It depends what kind of game it is and how sloppy your code is in general. Every game I have written in D i don't think about it, and it is fine. But I also prefer to do simpler games (think 80's or 90's style more than the modern stuff) anyway.
Feb 22
prev sibling next sibling parent Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
Don't let the GC prevent you from writing a game in D. D is the most flexible language I have ever used; you just need to learn how to deal with the GC in your game. Jonathan M. Davis gave you some good advice and explained the fundamental problem with the GC in real time games (the potential for pauses during reclamation). You can avoid that in a number of ways like temporarily disabling the GC during the real-time part of your game, just to name one. More can be found in the resources below. https://wiki.dlang.org/Memory_Management http://p0nce.github.io/d-idioms/#The-impossible-real-time-thread automem (https://github.com/atilaneves/automem) also gives you reference counting in D (C++ style), if that will work better for your use case. Mike
Feb 22
prev sibling next sibling parent reply Norm <norm.rowtree gmail.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
Have a look at https://github.com/gecko0307/atrium and see how memory is handled there. TBH though every game I've written I have not worried about the GC and just code it up. This works fine for 2d games, platformers etc. If it ever does bite you can always schedule the pauses (they are deterministic in the sense a collect will occur on allocation) or do pretty much what every game does in C++/C and allocate in pools. Cheers, Norm
Feb 22
parent Leonardo <leotada523 gmail.com> writes:
On Friday, 23 February 2018 at 03:25:33 UTC, Norm wrote:
 On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
Have a look at https://github.com/gecko0307/atrium and see how memory is handled there. TBH though every game I've written I have not worried about the GC and just code it up. This works fine for 2d games, platformers etc. If it ever does bite you can always schedule the pauses (they are deterministic in the sense a collect will occur on allocation) or do pretty much what every game does in C++/C and allocate in pools. Cheers, Norm
Atrium game use Dlib, more specific this module to manually manage memory. Appears to be very easy to use. Thanks. https://github.com/gecko0307/dlib/wiki/dlib.core.memory
Apr 05
prev sibling next sibling parent JN <666total wp.pl> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
Most people who say GC isn't suitable for games are overreacting. D gives you some ways to avoid GC in many cases. People make games in languages like Java, where you can't avoid GC for even the smallest allocations. Especially for 2D games, which just don't have much going on on the screen, you won't be bothered by GC. Even for 3D, as long as you're not making the next Call of Duty, GC shouldn't be a blocker for you. However, you might want to avoid too many allocations, just as you'd do in any other GC language (and non-GC ones too actually). For example, when doing particle emitters, allocating thousands of particles per frame is a bad idea, you'd rather want to use an object pool pattern ( http://www.gameprogrammingpatterns.com/object-pool.html ) - for example preallocate an array of 1000 particles, and when a particle dies, instead of allocating a new one, just reset the values of the one that died with the new one.
Feb 23
prev sibling next sibling parent Dukc <ajieskola gmail.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
If you do not worry about memory management at all, it will probably lead to a need to redesign your game. And that's regardless whether you allocate manually, via GC or using reference counting. You should laways make sure you do not have to continuously allocate in a tight loop. By tight I mean somthing thats executed hundreds or thousands of times per second. I do not mean that you should not allocate there, but make sure you can easily move the allocation out such a loop if necessary. GC is most likely a good option, as others have said. It does use more memory than RC or manual management, and leads to short pauses, but is almost as fast as manual management on average. You can: 1: Time the garbage collecions manually so that they happen when responsiveness isn't important. It's likely something like 100ms so even a short such moment will do. For example, when a racing car comes to stop or gets airborne, so that input wouldn't matter anyway. 2: If you have long intervals without such pauses, you can recycle the all the memory you have freed to make sure the program does not accumulate so much that it needs to collect. This is hard, so I recommend it only if 1. isn't feasible or you want to challege yourself. If neither of these are possible, or you think your game will be at limits of the RAM capacity no matter the optimizations (shouldn't happen for an indie game), then you should consider avoiding garbage collection from get-go.
Feb 23
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:
 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)
From my experience a combination of the following is necessary: - not having the audio thread registered - using pools aggressively for game entities
Feb 23
next sibling parent Leonardo <leotada523 gmail.com> writes:
On Saturday, 24 February 2018 at 07:12:21 UTC, Guillaume Piolat 
wrote:
 From my experience a combination of the following is necessary:
 - not having the audio thread registered
 - using pools aggressively for game entities
I'll read the resources you gave. Thanks for the all answers. Great community here.
Feb 25
prev sibling parent reply solidstate1991 <laszloszeremi outlook.com> writes:
On Saturday, 24 February 2018 at 07:12:21 UTC, Guillaume Piolat 
wrote:
 From my experience a combination of the following is necessary:
 - not having the audio thread registered
 - using pools aggressively for game entities
Also you can save a lot of clockcycles if you put nogc everywhere you don't allocate on the heap, the stack will be automatically cleaned up. I'm currently thinking on restructuring the way my engine handles display lists on sprites (dynamic array contains the priority, multiple associative arrays for Coordinates, sprites, attributes), however if enabling exception handling in nogc parts will enable associative array indexing, I'll just skip the whole procedure, otherwise probably moving the whole thing to rbtree.
Apr 08
parent reply Chris Katko <ckatko gmail.com> writes:
On Monday, 9 April 2018 at 00:25:21 UTC, solidstate1991 wrote:
 On Saturday, 24 February 2018 at 07:12:21 UTC, Guillaume Piolat 
 wrote:
 From my experience a combination of the following is necessary:
 - not having the audio thread registered
 - using pools aggressively for game entities
Also you can save a lot of clockcycles if you put nogc everywhere you don't allocate on the heap, the stack will be automatically cleaned up. I'm currently thinking on restructuring the way my engine handles display lists on sprites (dynamic array contains the priority, multiple associative arrays for Coordinates, sprites, attributes), however if enabling exception handling in nogc parts will enable associative array indexing, I'll just skip the whole procedure, otherwise probably moving the whole thing to rbtree.
Why... associative arrays? Wouldn't that become expensive when you hit 1,000s, or 10,000's of objects, for something as tiny as a coordinate (two or three floats) lookup?
Apr 08
parent solidstate1991 <laszloszeremi outlook.com> writes:
On Monday, 9 April 2018 at 01:01:18 UTC, Chris Katko wrote:
 Why... associative arrays? Wouldn't that become expensive when 
 you hit 1,000s, or 10,000's of objects, for something as tiny 
 as a coordinate (two or three floats) lookup?
Well, that's the other reason why I was looking for a different solution. Currently it's quite fast, maybe because it uses integers for the most part (only transformation effects use floats due to easier workarounds with vectorization), might replace the multiple associative arrays with a single auto-sorting one.
Apr 08
prev sibling parent reply Chris Katko <ckatko gmail.com> writes:
I'm in the same boat. I do games. But I love D's syntax and 
template power. So I'm doing a full experiment.

Honestly, if D is that big a liability, you'll encounter it long 
before it's "too late" to port it to C++.

Last night I had stuttering issues, but I realized there was a 
single, C-function, being called too many times (and never 
deallocating!).

But previously, I've also had stutter issues. Now granted, I test 
on a "crap" laptop 2 GB RAM / Celeron processor. But it'll be 60 
FPS ... then spike down. If this happens again with my current 
project, what I'm going to do, is hack the open source garbage 
collector to fire off an event/console message EVERY TIME it 
actually pauses to collect. Because it's possible the GC isn't 
actually the problem, or, some simple change to a line of code 
may prevent the GC from being a problem.

That said, there's also  nogc (but that's also a bit of a lie 
because they never tell you that ANY THREAD running GC code can 
pause ALL THREADS for a collection.)

But if you're making games, you should really be using static 
pools anyway. What's the MAXIMUM number of objects/trees/maps 
your game will have at a time? It's simple (regardless of D, C, 
Python, or Lua). Static. Pools. Basically, you just allocate at 
startup a simple fixed-length array for all your objects. That 
way, you're never asking the OS for memory = Never needing the 
garbage collector. If you don't use all that memory? Who cares. 
RAM is cheap. And if your program CAN swell in size, that means 
your low-end PCs will fail without knowing why.

So you just put all your objects in fixed length arrays of size 
MAX_OBJECTS, MAX_ENEMIES, MAX_ITEMS, etc. And deleting an object 
is as simple as erasing it, or marking it as "bool is_deleted = 
true;" and adding a new object is simply finding the first 
"is_deleted" and re-running the constructor / re-using the 
carcass of the dead object.

99% of AAA studios use static pools. Now technically, static 
pools are "chunks" of fixed length arrays. So you could have one 
pool for a "map", and start loading another pool for the next map 
you're going to enter, and then when you finally transfer to the 
next map, you then free the static pool by marking it as deleted. 
And repeat as necessary. So it's a very macro-level amount of 
allocations. We're talking like, less than a dozen actual 
entities. (Depends on gametype, of course. But the 
order-of-magnitude helps convey it.)
Apr 06
parent Danni Coy <danni.coy gmail.com> writes:
gc causes unpredictabilities in performance*. With games it tends to be
worst case performance that matters.

I would reccomend using std.experimental.allocator (even if you still use
the default GC backed allocator). This will allow you to swap out your
allocator for a more specialised one as your requirements become more
concrete.

auto foo = new CustomStruct();

becomes

auto foo = allocator.make!CustomStruct();

The next thing you probably want is  nogc - Last time I checked getting
IAllocator objects are a bit tricky to use in  nogc code. Currently I am
using https://github.com/radcapricorn/alloctraits to get around this
limitation (You will still need an allocator that doesn't use the GC, I use
Mallocator for test purposes).

* The GC itself is deterministic, but it is really easy to write code that
triggers GC pauses at times that is difficult track down.







On Sat, Apr 7, 2018 at 7:55 AM, Chris Katko via Digitalmars-d-learn <
digitalmars-d-learn puremagic.com> wrote:

 I'm in the same boat. I do games. But I love D's syntax and template
 power. So I'm doing a full experiment.

 Honestly, if D is that big a liability, you'll encounter it long before
 it's "too late" to port it to C++.

 Last night I had stuttering issues, but I realized there was a single,
 C-function, being called too many times (and never deallocating!).

 But previously, I've also had stutter issues. Now granted, I test on a
 "crap" laptop 2 GB RAM / Celeron processor. But it'll be 60 FPS ... then
 spike down. If this happens again with my current project, what I'm going
 to do, is hack the open source garbage collector to fire off an
 event/console message EVERY TIME it actually pauses to collect. Because
 it's possible the GC isn't actually the problem, or, some simple change to
 a line of code may prevent the GC from being a problem.

 That said, there's also  nogc (but that's also a bit of a lie because they
 never tell you that ANY THREAD running GC code can pause ALL THREADS for a
 collection.)

 But if you're making games, you should really be using static pools
 anyway. What's the MAXIMUM number of objects/trees/maps your game will have
 at a time? It's simple (regardless of D, C, Python, or Lua). Static. Pools.
 Basically, you just allocate at startup a simple fixed-length array for all
 your objects. That way, you're never asking the OS for memory = Never
 needing the garbage collector. If you don't use all that memory? Who cares.
 RAM is cheap. And if your program CAN swell in size, that means your
 low-end PCs will fail without knowing why.

 So you just put all your objects in fixed length arrays of size
 MAX_OBJECTS, MAX_ENEMIES, MAX_ITEMS, etc. And deleting an object is as
 simple as erasing it, or marking it as "bool is_deleted = true;" and adding
 a new object is simply finding the first "is_deleted" and re-running the
 constructor / re-using the carcass of the dead object.

 99% of AAA studios use static pools. Now technically, static pools are
 "chunks" of fixed length arrays. So you could have one pool for a "map",
 and start loading another pool for the next map you're going to enter, and
 then when you finally transfer to the next map, you then free the static
 pool by marking it as deleted. And repeat as necessary. So it's a very
 macro-level amount of allocations. We're talking like, less than a dozen
 actual entities. (Depends on gametype, of course. But the
 order-of-magnitude helps convey it.)
Apr 07