• Andres Clari (14/14) Jan 21 2018 Hi, is there any way to get from the GC all allocated objects, so
• Dmitry Olshansky (4/18) Jan 21 2018 Are you by chance using vibe.d?
• Andres Clari (17/39) Jan 21 2018 Yeah.
• thedeemon (4/8) Jan 22 2018 All threads withing a process share the same heap, so whatever
• Andres Clari (4/12) Jan 22 2018 Well the destructor of some Json objects and strings should be
• thedeemon (7/12) Jan 22 2018 If some objects were on the stack of the thread proc, their

Andres Clari <andres steelcode.net> writes:

Hi, is there any way to get from the GC all allocated objects, so 
I can see their size and find where I'm leaking memory? Or 
perhaps a good tool to help with this issue...

I tried building my program with "profile-gc" but I got an 
invalid MemoryOperationError with no stack trace... so no luck 
using that. And the leak is only obvious when running in 
production, where after 6 hours I've seen 1.5Gb of memory usage.

Right now, I'm able to extend the life of my program by injecting 
calls to:
GC.collect
GCminimize

Within an event that happens every 5 seconds, which I estimate 
will give me about a week before the program's memory is again at 
+1.5Gb.

Dmitry Olshansky <dmitry.olsh gmail.com> writes:

On Sunday, 21 January 2018 at 17:28:13 UTC, Andres Clari wrote:
 Hi, is there any way to get from the GC all allocated objects, 
 so I can see their size and find where I'm leaking memory? Or 
 perhaps a good tool to help with this issue...

 I tried building my program with "profile-gc" but I got an 
 invalid MemoryOperationError with no stack trace... so no luck 
 using that. And the leak is only obvious when running in 
 production, where after 6 hours I've seen 1.5Gb of memory usage.


Are you by chance using vibe.d?
In versions 0.8.x I observe a huge memory leak. Might be your 
problem.

 Right now, I'm able to extend the life of my program by 
 injecting calls to:
 GC.collect
 GCminimize

 Within an event that happens every 5 seconds, which I estimate 
 will give me about a week before the program's memory is again 
 at +1.5Gb.

Andres Clari <andres steelcode.net> writes:

On Monday, 22 January 2018 at 06:15:24 UTC, Dmitry Olshansky 
wrote:
 On Sunday, 21 January 2018 at 17:28:13 UTC, Andres Clari wrote:
 Hi, is there any way to get from the GC all allocated objects, 
 so I can see their size and find where I'm leaking memory? Or 
 perhaps a good tool to help with this issue...

 I tried building my program with "profile-gc" but I got an 
 invalid MemoryOperationError with no stack trace... so no luck 
 using that. And the leak is only obvious when running in 
 production, where after 6 hours I've seen 1.5Gb of memory 
 usage.


 Are you by chance using vibe.d?
 In versions 0.8.x I observe a huge memory leak. Might be your 
 problem.

 Right now, I'm able to extend the life of my program by 
 injecting calls to:
 GC.collect
 GCminimize

 Within an event that happens every 5 seconds, which I estimate 
 will give me about a week before the program's memory is again 
 at +1.5Gb.


Yeah.

Although after doing some manual tracking and isolation of the 
suspect routines, I found that using "std.concurrency.spawn" to 
run a http post request with the "requests" library was the 
leaking suspect.

I'm triggering some 15 requests every 5 seconds in production, so 
it's obvious there, but querying "GC.usedSize" I was able to 
observe the same on my dev environment.

After removing the "spawn" call, and wrapping up the post code 
inside a "runTask", it seems to work properly now, and so far I 
have a consistently small memory size in production.

Not sure why "spawn" would leak like that tho. I would assume 
that once the thread exits, it would get destroyed and it's 
resources reclaimed, specially when I have calls to "GC.collect 
and GC.minimize".

thedeemon <dlang thedeemon.com> writes:

On Monday, 22 January 2018 at 06:48:00 UTC, Andres Clari wrote:
 Not sure why "spawn" would leak like that tho. I would assume 
 that once the thread exits, it would get destroyed and it's 
 resources reclaimed, specially when I have calls to "GC.collect 
 and GC.minimize".

All threads withing a process share the same heap, so whatever 
one thread allocated in that heap is not freed or reclaimed 
automatically when thread dies, it stays in the heap.

Andres Clari <andres steelcode.net> writes:

On Monday, 22 January 2018 at 15:56:53 UTC, thedeemon wrote:
 On Monday, 22 January 2018 at 06:48:00 UTC, Andres Clari wrote:
 Not sure why "spawn" would leak like that tho. I would assume 
 that once the thread exits, it would get destroyed and it's 
 resources reclaimed, specially when I have calls to 
 "GC.collect and GC.minimize".

 All threads withing a process share the same heap, so whatever 
 one thread allocated in that heap is not freed or reclaimed 
 automatically when thread dies, it stays in the heap.

Well the destructor of some Json objects and strings should be 
called when the thread exits tho. Why wouldn’t the GC take care 
of this?

thedeemon <dlang thedeemon.com> writes:

On Monday, 22 January 2018 at 16:37:19 UTC, Andres Clari wrote:
 All threads withing a process share the same heap, so whatever 
 one thread allocated in that heap is not freed or reclaimed 
 automatically when thread dies, it stays in the heap.

 Well the destructor of some Json objects and strings should be 
 called when the thread exits tho.

If some objects were on the stack of the thread proc, their 
destructors run, yes, as usual for leaving a scope. Strings don't 
have destructors, afaik.
Thread-local storage gets cleaned up too. But everything that 
lives in heap, stays there, until GC finds them. Objects in heap 
don't have any information on them  which thread they belong to.