www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Game and GC

reply Leonardo <leotada523 gmail.com> writes:
Hi, I'm new to language and games.
Many people say that GC is bad and can slow down your project in 
some moments.
What can happen if I create a game using D without worrying with 
memory management?
(using full GC)
Feb 22 2018
next sibling parent reply StickYourLeftFootIn <The hole.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


What do you think will happen? Anytime you delegate power to 
something else what can go wrong? Nothing is perfect.

The GC exists to automate a job. The job it does is not the 
problem... It does it well. The issue is when it does it. It's 
like the noisy garbage man coming in as 3AM to get your trash... 
are you ok with that? Some people are.
Feb 22 2018
parent Leonardo <leotada523 gmail.com> writes:
On Friday, 23 February 2018 at 02:02:12 UTC, StickYourLeftFootIn 
wrote:

 On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


 What do you think will happen? Anytime you delegate power to 
 something else what can go wrong? Nothing is perfect.

 The GC exists to automate a job. The job it does is not the 
 problem... It does it well. The issue is when it does it. It's 
 like the noisy garbage man coming in as 3AM to get your 
 trash... are you ok with that? Some people are.


I understand, I'm not saying GC is bad, but I want to know if it 
would be slow to the point of being noticeable in a game. If so, 
what is the best way to do this? Placing  nogc everywhere? Thanks.
Feb 22 2018
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, February 23, 2018 01:54:07 Leonardo via Digitalmars-d-learn 
wrote:

 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project in
 some moments.
 What can happen if I create a game using D without worrying with
 memory management?
 (using full GC)


The GC won't slow down your code in general (in fact, it will probably speed
it up in comparison to reference counting), but whenever the GC does a
collection, that means that it stops all threads that it manages. So, you
could suddenly have everything stop for 100ms (the actual length of a
collection is going to depend on how much memory the GC has to scan, and I
don't know what the typical length of a collection is; that will depend on
the program). For programs that can afford to occasionally stop like that,
that's not a problem. For a game that's trying to maintain 60fps, that's
likely a really big deal.

There are a number of ways to handle it, though the biggest is to simply
minimize how much you allocate on the GC heap and how much memory has to be
scanned for pointers that refer to GC-allocated memory. Other stuff includes
disabling the GC while critical pieces of code are running and having
critical threads not be managed by the GC or use GC-allocated memory.

I would suggest that you read this series of articles on the official D
blog:

https://dlang.org/blog/the-gc-series/

- Jonathan M Davis
Feb 22 2018
parent Leonardo <leotada523 gmail.com> writes:
On Friday, 23 February 2018 at 02:16:38 UTC, Jonathan M Davis 
wrote:

 The GC won't slow down your code in general (in fact, it will 
 probably speed it up in comparison to reference counting), but 
 whenever the GC does a collection, that means that it stops all 
 threads that it manages. So, you could suddenly have everything 
 stop for 100ms (the actual length of a collection is going to 
 depend on how much memory the GC has to scan, and I don't know 
 what the typical length of a collection is; that will depend on 
 the program). For programs that can afford to occasionally stop 
 like that, that's not a problem. For a game that's trying to 
 maintain 60fps, that's likely a really big deal.



 - Jonathan M Davis


That's what I thought for a game, but probably no one tested yet 
to see the impact. Thanks, I'll read on.
Feb 22 2018
prev sibling next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


It depends what kind of game it is and how sloppy your code is in 
general.

Every game I have written in D i don't think about it, and it is 
fine. But I also prefer to do simpler games (think 80's or 90's 
style more than the modern stuff) anyway.
Feb 22 2018
prev sibling next sibling parent Mike Franklin <slavo5150 yahoo.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


Don't let the GC prevent you from writing a game in D.  D is the 
most flexible language I have ever used; you just need to learn 
how to deal with the GC in your game.

Jonathan M. Davis gave you some good advice and explained the 
fundamental problem with the GC in real time games (the potential 
for pauses during reclamation).  You can avoid that in a number 
of ways like temporarily disabling the GC during the real-time 
part of your game, just to name one.  More can be found in the 
resources below.

https://wiki.dlang.org/Memory_Management
http://p0nce.github.io/d-idioms/#The-impossible-real-time-thread

automem (https://github.com/atilaneves/automem) also gives you 
reference counting in D (C++ style), if that will work better for 
your use case.

Mike
Feb 22 2018
prev sibling next sibling parent reply Norm <norm.rowtree gmail.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


Have a look at https://github.com/gecko0307/atrium and see how 
memory is handled there.

TBH though every game I've written I have not worried about the 
GC and just code it up. This works fine for 2d games, platformers 
etc. If it ever does bite you can always schedule the pauses 
(they are deterministic in the sense a collect will occur on 
allocation) or do pretty much what every game does in C++/C and 
allocate in pools.

Cheers,
Norm
Feb 22 2018
parent Leonardo <leotada523 gmail.com> writes:
On Friday, 23 February 2018 at 03:25:33 UTC, Norm wrote:

 On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


 Have a look at https://github.com/gecko0307/atrium and see how 
 memory is handled there.

 TBH though every game I've written I have not worried about the 
 GC and just code it up. This works fine for 2d games, 
 platformers etc. If it ever does bite you can always schedule 
 the pauses (they are deterministic in the sense a collect will 
 occur on allocation) or do pretty much what every game does in 
 C++/C and allocate in pools.

 Cheers,
 Norm


Atrium game use Dlib, more specific this module to manually 
manage memory.
Appears to be very easy to use. Thanks.
https://github.com/gecko0307/dlib/wiki/dlib.core.memory
Apr 05 2018
prev sibling next sibling parent JN <666total wp.pl> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


Most people who say GC isn't suitable for games are overreacting. 
D gives you some ways to avoid GC in many cases. People make 
games in languages like Java, where you can't avoid GC for even 
the smallest allocations. Especially for 2D games, which just 
don't have much going on on the screen, you won't be bothered by 
GC.

Even for 3D, as long as you're not making the next Call of Duty, 
GC shouldn't be a blocker for you. However, you might want to 
avoid too many allocations, just as you'd do in any other GC 
language (and non-GC ones too actually). For example, when doing 
particle emitters, allocating thousands of particles per frame is 
a bad idea, you'd rather want to use an object pool pattern ( 
http://www.gameprogrammingpatterns.com/object-pool.html ) - for 
example preallocate an array of 1000 particles, and when a 
particle dies, instead of allocating a new one, just reset the 
values of the one that died with the new one.
Feb 23 2018
prev sibling next sibling parent Dukc <ajieskola gmail.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


If you do not worry about memory management at all, it will 
probably lead to a need to redesign your game. And that's 
regardless whether you allocate manually, via GC or using 
reference counting.

You should laways make sure you do not have to continuously 
allocate in a tight loop. By tight I mean somthing thats executed 
hundreds or thousands of times per second. I do not mean that you 
should not allocate there, but make sure you can easily move the 
allocation out such a loop if necessary.

GC is most likely a good option, as others have said. It does use 
more memory than RC or manual management, and leads to short 
pauses, but is almost as fast as manual management on average. 
You can:

1: Time the garbage collecions manually so that they happen when 
responsiveness isn't important.  It's likely something like 100ms 
so even a short such moment will do. For example, when a racing 
car comes to stop or gets airborne, so that input wouldn't matter 
anyway.

2: If you have long intervals without such pauses, you can 
recycle the all the memory you have freed to make sure the 
program does not accumulate so much that it needs to collect. 
This is hard, so I recommend it only if 1. isn't feasible or you 
want to challege yourself.

If neither of these are possible, or you think your game will be 
at limits of the RAM capacity no matter the optimizations 
(shouldn't happen for an indie game), then you should consider 
avoiding garbage collection from get-go.
Feb 23 2018
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Friday, 23 February 2018 at 01:54:07 UTC, Leonardo wrote:

 Hi, I'm new to language and games.
 Many people say that GC is bad and can slow down your project 
 in some moments.
 What can happen if I create a game using D without worrying 
 with memory management?
 (using full GC)


 From my experience a combination of the following is necessary:
- not having the audio thread registered
- using pools aggressively for game entities
Feb 23 2018
next sibling parent Leonardo <leotada523 gmail.com> writes:
On Saturday, 24 February 2018 at 07:12:21 UTC, Guillaume Piolat 
wrote:

 From my experience a combination of the following is necessary:
 - not having the audio thread registered
 - using pools aggressively for game entities


I'll read the resources you gave.
Thanks for the all answers. Great community here.
Feb 25 2018
prev sibling parent reply solidstate1991 <laszloszeremi outlook.com> writes:
On Saturday, 24 February 2018 at 07:12:21 UTC, Guillaume Piolat 
wrote:

 From my experience a combination of the following is necessary:
 - not having the audio thread registered
 - using pools aggressively for game entities


Also you can save a lot of clockcycles if you put  nogc 
everywhere you don't allocate on the heap, the stack will be 
automatically cleaned up.

I'm currently thinking on restructuring the way my engine handles 
display lists on sprites (dynamic array contains the priority, 
multiple associative arrays for Coordinates, sprites, 
attributes), however if enabling exception handling in  nogc 
parts will enable associative array indexing, I'll just skip the 
whole procedure, otherwise probably moving the whole thing to 
rbtree.
Apr 08 2018
parent reply Chris Katko <ckatko gmail.com> writes:
On Monday, 9 April 2018 at 00:25:21 UTC, solidstate1991 wrote:

 On Saturday, 24 February 2018 at 07:12:21 UTC, Guillaume Piolat 
 wrote:

 From my experience a combination of the following is necessary:
 - not having the audio thread registered
 - using pools aggressively for game entities


 Also you can save a lot of clockcycles if you put  nogc 
 everywhere you don't allocate on the heap, the stack will be 
 automatically cleaned up.

 I'm currently thinking on restructuring the way my engine 
 handles display lists on sprites (dynamic array contains the 
 priority, multiple associative arrays for Coordinates, sprites, 
 attributes), however if enabling exception handling in  nogc 
 parts will enable associative array indexing, I'll just skip 
 the whole procedure, otherwise probably moving the whole thing 
 to rbtree.


Why... associative arrays? Wouldn't that become expensive when 
you hit 1,000s, or 10,000's of objects, for something as tiny as 
a coordinate (two or three floats) lookup?
Apr 08 2018
parent solidstate1991 <laszloszeremi outlook.com> writes:
On Monday, 9 April 2018 at 01:01:18 UTC, Chris Katko wrote:

 Why... associative arrays? Wouldn't that become expensive when 
 you hit 1,000s, or 10,000's of objects, for something as tiny 
 as a coordinate (two or three floats) lookup?


Well, that's the other reason why I was looking for a different 
solution. Currently it's quite fast, maybe because it uses 
integers for the most part (only transformation effects use 
floats due to easier workarounds with vectorization), might 
replace the multiple associative arrays with a single 
auto-sorting one.
Apr 08 2018
prev sibling parent reply Chris Katko <ckatko gmail.com> writes:
I'm in the same boat. I do games. But I love D's syntax and 
template power. So I'm doing a full experiment.

Honestly, if D is that big a liability, you'll encounter it long 
before it's "too late" to port it to C++.

Last night I had stuttering issues, but I realized there was a 
single, C-function, being called too many times (and never 
deallocating!).

But previously, I've also had stutter issues. Now granted, I test 
on a "crap" laptop 2 GB RAM / Celeron processor. But it'll be 60 
FPS ... then spike down. If this happens again with my current 
project, what I'm going to do, is hack the open source garbage 
collector to fire off an event/console message EVERY TIME it 
actually pauses to collect. Because it's possible the GC isn't 
actually the problem, or, some simple change to a line of code 
may prevent the GC from being a problem.

That said, there's also  nogc (but that's also a bit of a lie 
because they never tell you that ANY THREAD running GC code can 
pause ALL THREADS for a collection.)

But if you're making games, you should really be using static 
pools anyway. What's the MAXIMUM number of objects/trees/maps 
your game will have at a time? It's simple (regardless of D, C, 
Python, or Lua). Static. Pools. Basically, you just allocate at 
startup a simple fixed-length array for all your objects. That 
way, you're never asking the OS for memory = Never needing the 
garbage collector. If you don't use all that memory? Who cares. 
RAM is cheap. And if your program CAN swell in size, that means 
your low-end PCs will fail without knowing why.

So you just put all your objects in fixed length arrays of size 
MAX_OBJECTS, MAX_ENEMIES, MAX_ITEMS, etc. And deleting an object 
is as simple as erasing it, or marking it as "bool is_deleted = 
true;" and adding a new object is simply finding the first 
"is_deleted" and re-running the constructor / re-using the 
carcass of the dead object.

99% of AAA studios use static pools. Now technically, static 
pools are "chunks" of fixed length arrays. So you could have one 
pool for a "map", and start loading another pool for the next map 
you're going to enter, and then when you finally transfer to the 
next map, you then free the static pool by marking it as deleted. 
And repeat as necessary. So it's a very macro-level amount of 
allocations. We're talking like, less than a dozen actual 
entities. (Depends on gametype, of course. But the 
order-of-magnitude helps convey it.)
Apr 06 2018
parent Danni Coy <danni.coy gmail.com> writes:
gc causes unpredictabilities in performance*. With games it tends to be
worst case performance that matters.

I would reccomend using std.experimental.allocator (even if you still use
the default GC backed allocator). This will allow you to swap out your
allocator for a more specialised one as your requirements become more
concrete.

auto foo = new CustomStruct();

becomes

auto foo = allocator.make!CustomStruct();

The next thing you probably want is  nogc - Last time I checked getting
IAllocator objects are a bit tricky to use in  nogc code. Currently I am
using https://github.com/radcapricorn/alloctraits to get around this
limitation (You will still need an allocator that doesn't use the GC, I use
Mallocator for test purposes).

* The GC itself is deterministic, but it is really easy to write code that
triggers GC pauses at times that is difficult track down.







On Sat, Apr 7, 2018 at 7:55 AM, Chris Katko via Digitalmars-d-learn <
digitalmars-d-learn puremagic.com> wrote:


 I'm in the same boat. I do games. But I love D's syntax and template
 power. So I'm doing a full experiment.

 Honestly, if D is that big a liability, you'll encounter it long before
 it's "too late" to port it to C++.

 Last night I had stuttering issues, but I realized there was a single,
 C-function, being called too many times (and never deallocating!).

 But previously, I've also had stutter issues. Now granted, I test on a
 "crap" laptop 2 GB RAM / Celeron processor. But it'll be 60 FPS ... then
 spike down. If this happens again with my current project, what I'm going
 to do, is hack the open source garbage collector to fire off an
 event/console message EVERY TIME it actually pauses to collect. Because
 it's possible the GC isn't actually the problem, or, some simple change to
 a line of code may prevent the GC from being a problem.

 That said, there's also  nogc (but that's also a bit of a lie because they
 never tell you that ANY THREAD running GC code can pause ALL THREADS for a
 collection.)

 But if you're making games, you should really be using static pools
 anyway. What's the MAXIMUM number of objects/trees/maps your game will have
 at a time? It's simple (regardless of D, C, Python, or Lua). Static. Pools.
 Basically, you just allocate at startup a simple fixed-length array for all
 your objects. That way, you're never asking the OS for memory = Never
 needing the garbage collector. If you don't use all that memory? Who cares.
 RAM is cheap. And if your program CAN swell in size, that means your
 low-end PCs will fail without knowing why.

 So you just put all your objects in fixed length arrays of size
 MAX_OBJECTS, MAX_ENEMIES, MAX_ITEMS, etc. And deleting an object is as
 simple as erasing it, or marking it as "bool is_deleted = true;" and adding
 a new object is simply finding the first "is_deleted" and re-running the
 constructor / re-using the carcass of the dead object.

 99% of AAA studios use static pools. Now technically, static pools are
 "chunks" of fixed length arrays. So you could have one pool for a "map",
 and start loading another pool for the next map you're going to enter, and
 then when you finally transfer to the next map, you then free the static
 pool by marking it as deleted. And repeat as necessary. So it's a very
 macro-level amount of allocations. We're talking like, less than a dozen
 actual entities. (Depends on gametype, of course. But the
 order-of-magnitude helps convey it.)
Apr 07 2018