digitalmars.D.bugs - [Issue 5326] New: GC -- 99% CPU in gc gcx Gcx mark()
- d-bugmail puremagic.com (34/34) Dec 05 2010 http://d.puremagic.com/issues/show_bug.cgi?id=5326
- d-bugmail puremagic.com (21/21) Dec 06 2010 http://d.puremagic.com/issues/show_bug.cgi?id=5326
- d-bugmail puremagic.com (7/7) Dec 06 2010 http://d.puremagic.com/issues/show_bug.cgi?id=5326
- d-bugmail puremagic.com (18/21) Dec 10 2010 memory.
- d-bugmail puremagic.com (18/22) Dec 10 2010 http://d.puremagic.com/issues/show_bug.cgi?id=5326
- d-bugmail puremagic.com (6/6) Dec 11 2010 http://d.puremagic.com/issues/show_bug.cgi?id=5326
- d-bugmail puremagic.com (11/11) Feb 06 2011 http://d.puremagic.com/issues/show_bug.cgi?id=5326
http://d.puremagic.com/issues/show_bug.cgi?id=5326 Summary: GC -- 99% CPU in gc gcx Gcx mark() Product: D Version: D2 Platform: x86_64 OS/Version: Windows Status: NEW Severity: normal Priority: P2 Component: druntime AssignedTo: sean invisibleduck.org ReportedBy: wfunction hotmail.com I was running a piece of code that allocated a huge block of memory at once, then allocated small chunks after that inside a block, deleting them at the end of the scope. I managed to get the code down to this: auto temp = new void[0x8080 * 4096]; for (int i = 0; i < 0x8080 * 4; i++) { scope auto temp2 = new void[0x10100]; } What happens is, after the first few ten thousands of collections (for me, it was around after 62,000 collections), the time spent in the mark() method shoots up to around 99% of the application's time, and the program effectively hangs at the allocation. The numbers above were specific to my system; they're rather arbitrary. All you need to do is: (1) Run the code, and break into the process before around 1 second. Notice that, when you do this, it's often inside memset() or some other function, and not the GC. (2) Run the code, and break into the process after several seconds have passed. At this time, it's practically guaranteed that you're in the mark() method -- and in fact, for me, it froze my application. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Dec 05 2010
http://d.puremagic.com/issues/show_bug.cgi?id=5326 nfxjfg gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |nfxjfg gmail.com That's somewhat excpected. The mark() function is the core of the garbage collector, where it basically follows all references in the program's heap. void[] arrays can potentially contain references, so the GC has to mark() these as well. (Why are you using void[]? 10 dollars on you have no real reason.) What makes things worse: each time there's a pointer into a large memory block, the GC has to do a linear scan to find the beginning of the memory block. And you are alocating LARGE blocks (32k pages!), so that might become significant. I think what's happening is that your heap is exploding due to false pointers. The memory allocations are so large that there's always something that looks like a pointer pointing into it, so it never gets free'd. A larger heap means higher collection times. So, this is probably not a bug. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Dec 06 2010
http://d.puremagic.com/issues/show_bug.cgi?id=5326 PS: scope doesn't work on arrays. Your example code never explicitly free's the memory. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Dec 06 2010
http://d.puremagic.com/issues/show_bug.cgi?id=5326PS: scope doesn't work on arrays. Your example code never explicitly free's thememory. That explains everything... but then, how do we delete arrays?void[] arrays can potentially contain references, so the GC has to mark() theseas well. I had NO IDEA that was the case!! (see below)(Why are you using void[]? 10 dollars on you have no real reason.)I actually did have a reason, but I was misunderstanding the situation. I used void[]'s in places where I did not want ubyte[]s to be interpreted as numbers, so that they would not be dereferenceable. (To me, a void[] could contain binary *data*, whereas a ubyte[] logically means that it contains numbers -- and while it's OK to reference a number in a list, it's *not* OK to me to just randomly reference an arbitrary block of data, which might contain code inside a buffer.) Of course, I wasn't understanding their meaning correctly, so I won't do that anymore; thank you very much for the explanation!! -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Dec 10 2010
http://d.puremagic.com/issues/show_bug.cgi?id=5326You can free heap allocated arrays by "delete somearray;" (deprecated in D2). I'm not sure what's the proper way of deleting an array is. As you can see in the implementation for "delete <array>;", it's not that trivial: http://www.dsource.org/projects/druntime/browser/trunk/src/rt/lifetime.d#L937 PS: That code may contain a bug. It may pass a pointer to gc_free() that is not the start of a memory block, but gc_free() could have trouble with that and have undefined effects on the GC heap integrity, but I digress. When you allocate a class using scope on classes, it is normally allocated on the stack. Doing this with arrays is hard; you need to use alloca() and turn the resulting void* into an array... Also, the idea with void[] was probably that it could contain *anything* that can exist in memory. Thus the GC has to assume that it may contain pointers. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------PS: scope doesn't work on arrays. Your example code never explicitly free's thememory. That explains everything... but then, how do we delete arrays?
Dec 10 2010
http://d.puremagic.com/issues/show_bug.cgi?id=5326 Ah, all right, thank you for the information! -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Dec 11 2010
http://d.puremagic.com/issues/show_bug.cgi?id=5326 Brad Roberts <braddr puremagic.com> changed: What |Removed |Added ---------------------------------------------------------------------------- Platform|x86_64 |x86 --- Mass migration of bugs marked as x86-64 to just x86. The platform run on isn't what's relevant, it's if the app is a 32 or 64 bit app. -- Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email ------- You are receiving this mail because: -------
Feb 06 2011