www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Extreme memory usage when `synchronized( this )` is used

reply "tcak" <tcak gmail.com> writes:
[code]
import std.stdio;

class Connection{
	private void other() shared{}

	public void close() shared{
		synchronized( this ){
			other();
		}
	}

	public void hasData() shared{ writeln("Has Data"); }
}

void main() {
	for(long i=0; i < 250_000_000; ++i){
		auto conn = new shared Connection();

		conn.hasData();

		conn.close();
	}
}
[/code]

With this code, memory usage of program is increasing very fast. 
In about 10 seconds, it reached 100MB for me.

If I comment out `synchronized( this )` line with its 
parentheses, OR remove `(this)` from it, it suddenly turns 
normal. Very little memory usage.

What's happening? Is object instance being stored somewhere at 
each iteration?

--

I tried the same thing by creating synchronisation object instead 
of object itself as blow, still usage lots of memory.

[code]
import std.stdio;

class Connection{
	private Object syncObject;

	public this() shared{
		syncObject = new shared Object();
	}

	private void other() shared{}

	public void close() shared{
		synchronized( syncObject ){
			other();
		}
	}

	public void hasData() shared{ writeln("Has Data"); }
}

void main() {
	for(long i=0; i < 250_000_000; ++i){
		auto conn = new shared Connection();

		conn.hasData();

		conn.close();
	}
}
[/code]
May 11 2015
next sibling parent "tcak" <tcak gmail.com> writes:
On Monday, 11 May 2015 at 09:09:09 UTC, tcak wrote:

 [code]
 import std.stdio;

 class Connection{
 	private void other() shared{}

 	public void close() shared{
 		synchronized( this ){
 			other();
 		}
 	}

 	public void hasData() shared{ writeln("Has Data"); }
 }

 void main() {
 	for(long i=0; i < 250_000_000; ++i){
 		auto conn = new shared Connection();

 		conn.hasData();

 		conn.close();
 	}
 }
 [/code]

 With this code, memory usage of program is increasing very 
 fast. In about 10 seconds, it reached 100MB for me.

 If I comment out `synchronized( this )` line with its 
 parentheses, OR remove `(this)` from it, it suddenly turns 
 normal. Very little memory usage.

 What's happening? Is object instance being stored somewhere at 
 each iteration?

 --

 I tried the same thing by creating synchronisation object 
 instead of object itself as blow, still usage lots of memory.

 [code]
 import std.stdio;

 class Connection{
 	private Object syncObject;

 	public this() shared{
 		syncObject = new shared Object();
 	}

 	private void other() shared{}

 	public void close() shared{
 		synchronized( syncObject ){
 			other();
 		}
 	}

 	public void hasData() shared{ writeln("Has Data"); }
 }

 void main() {
 	for(long i=0; i < 250_000_000; ++i){
 		auto conn = new shared Connection();

 		conn.hasData();

 		conn.close();
 	}
 }
 [/code]


I use DMD 2.067.1 on Ubuntu 14.04 64 bit
May 11 2015
prev sibling parent reply Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d-learn writes:
On Mon, 11 May 2015 09:09:07 +0000
tcak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:


 [code]
 import std.stdio;
 
 class Connection{
 	private void other() shared{}
 
 	public void close() shared{
 		synchronized( this ){
 			other();
 		}
 	}
 
 	public void hasData() shared{ writeln("Has Data"); }
 }
 
 void main() {
 	for(long i=0; i < 250_000_000; ++i){
 		auto conn = new shared Connection();
 
 		conn.hasData();
 
 		conn.close();
 	}
 }
 [/code]
 
 With this code, memory usage of program is increasing very fast. 
 In about 10 seconds, it reached 100MB for me.
 
 If I comment out `synchronized( this )` line with its 
 parentheses, OR remove `(this)` from it, it suddenly turns 
 normal. Very little memory usage.
 
 What's happening? Is object instance being stored somewhere at 
 each iteration?
 
 --
 
 I tried the same thing by creating synchronisation object instead 
 of object itself as blow, still usage lots of memory.
 
 [code]
 import std.stdio;
 
 class Connection{
 	private Object syncObject;
 
 	public this() shared{
 		syncObject = new shared Object();
 	}
 
 	private void other() shared{}
 
 	public void close() shared{
 		synchronized( syncObject ){
 			other();
 		}
 	}
 
 	public void hasData() shared{ writeln("Has Data"); }
 }
 
 void main() {
 	for(long i=0; i < 250_000_000; ++i){
 		auto conn = new shared Connection();
 
 		conn.hasData();
 
 		conn.close();
 	}
 }
 [/code]


I think synchronize(this) prevents GC from collect memory
May 11 2015
parent reply "tcak" <tcak gmail.com> writes:
On Monday, 11 May 2015 at 09:20:50 UTC, Daniel Kozák wrote:

 On Mon, 11 May 2015 09:09:07 +0000
 tcak via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:


 I think synchronize(this) prevents GC from collect memory


I am not sure whether this is expected behaviour from 
`synchronization` keyword.

Similar code in Java. It has settled on about 62M and didn't 
increase at all.

[code]
public class Main{
	private void other(){}

	public void close(){
		synchronized( this ){
			other();
		}
	}

	public void hasData(){ System.out.println("Has Data"); }

	public static void main( String[] args ){
		for(long i=0; i < 250000000; ++i){
			Main conn = new Main();

			conn.hasData();

			conn.close();

			conn = null;
		}
	}
}
[/code]
May 11 2015
next sibling parent Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d-learn writes:
On Mon, 11 May 2015 09:40:28 +0000
tcak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:


 On Monday, 11 May 2015 at 09:20:50 UTC, Daniel Kozák wrote:

 On Mon, 11 May 2015 09:09:07 +0000
 tcak via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:


 I think synchronize(this) prevents GC from collect memory


 
 I am not sure whether this is expected behaviour from 
 `synchronization` keyword.
 
 Similar code in Java. It has settled on about 62M and didn't 
 increase at all.
 
 [code]
 public class Main{
 	private void other(){}
 
 	public void close(){
 		synchronized( this ){
 			other();
 		}
 	}
 
 	public void hasData(){ System.out.println("Has Data"); }
 
 	public static void main( String[] args ){
 		for(long i=0; i < 250000000; ++i){
 			Main conn = new Main();
 
 			conn.hasData();
 
 			conn.close();
 
 			conn = null;
 		}
 	}
 }
 [/code]


try destroy

void main() {
    for(long i=0; i < 250_000_000; ++i){
        auto conn = new shared Connection();
        
        conn.hasData();
        
        conn.close();
        destroy(conn);
    }
}
May 11 2015
prev sibling parent reply Daniel =?UTF-8?B?S296w6Fr?= via Digitalmars-d-learn writes:
On Mon, 11 May 2015 09:40:28 +0000
tcak via Digitalmars-d-learn <digitalmars-d-learn puremagic.com> wrote:


 On Monday, 11 May 2015 at 09:20:50 UTC, Daniel Kozák wrote:

 On Mon, 11 May 2015 09:09:07 +0000
 tcak via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:


 I think synchronize(this) prevents GC from collect memory


 
 I am not sure whether this is expected behaviour from 
 `synchronization` keyword.
 
 Similar code in Java. It has settled on about 62M and didn't 
 increase at all.
 
 [code]
 public class Main{
 	private void other(){}
 
 	public void close(){
 		synchronized( this ){
 			other();
 		}
 	}
 
 	public void hasData(){ System.out.println("Has Data"); }
 
 	public static void main( String[] args ){
 		for(long i=0; i < 250000000; ++i){
 			Main conn = new Main();
 
 			conn.hasData();
 
 			conn.close();
 
 			conn = null;
 		}
 	}
 }
 [/code]


But yes I would say, it is not intentional behaviour. It should use
weak reference, so It would be possible for GC to collect it. Please
fill issue
May 11 2015
parent reply "Daniel Kozak" <kozzi11 gmail.com> writes:
On Monday, 11 May 2015 at 10:24:57 UTC, Daniel Kozák wrote:

 On Mon, 11 May 2015 09:40:28 +0000
 tcak via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:


 On Monday, 11 May 2015 at 09:20:50 UTC, Daniel Kozák wrote:

 On Mon, 11 May 2015 09:09:07 +0000
 tcak via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:


 I think synchronize(this) prevents GC from collect memory


 
 I am not sure whether this is expected behaviour from 
 `synchronization` keyword.
 
 Similar code in Java. It has settled on about 62M and didn't 
 increase at all.
 
 [code]
 public class Main{
 	private void other(){}
 
 	public void close(){
 		synchronized( this ){
 			other();
 		}
 	}
 
 	public void hasData(){ System.out.println("Has Data"); }
 
 	public static void main( String[] args ){
 		for(long i=0; i < 250000000; ++i){
 			Main conn = new Main();
 
 			conn.hasData();
 
 			conn.close();
 
 			conn = null;
 		}
 	}
 }
 [/code]


 But yes I would say, it is not intentional behaviour. It should 
 use
 weak reference, so It would be possible for GC to collect it. 
 Please
 fill issue


And on 2.066.1 it works correctly, so it is a regression
May 11 2015
parent reply "tcak" <tcak gmail.com> writes:
On Monday, 11 May 2015 at 10:30:11 UTC, Daniel Kozak wrote:

 On Monday, 11 May 2015 at 10:24:57 UTC, Daniel Kozák wrote:

 On Mon, 11 May 2015 09:40:28 +0000
 tcak via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:


 But yes I would say, it is not intentional behaviour. It 
 should use
 weak reference, so It would be possible for GC to collect it. 
 Please
 fill issue


 And on 2.066.1 it works correctly, so it is a regression


I really didn't know that `destroy` would make any difference. 
But I made changes in my libraries. Before null'g objects, I used 
`destroy` as well, and memory consumption is not increasing at 
all. Cheers for that ;)

I think `destroy`'s behaviour should be documented much better.

If I can reach to my bugzilla account (Everytime I forget my 
password), I will report it now.
May 11 2015
next sibling parent "tcak" <tcak gmail.com> writes:
On Monday, 11 May 2015 at 10:48:22 UTC, tcak wrote:

 On Monday, 11 May 2015 at 10:30:11 UTC, Daniel Kozak wrote:

 On Monday, 11 May 2015 at 10:24:57 UTC, Daniel Kozák wrote:

 On Mon, 11 May 2015 09:40:28 +0000
 tcak via Digitalmars-d-learn 
 <digitalmars-d-learn puremagic.com> wrote:


 But yes I would say, it is not intentional behaviour. It 
 should use
 weak reference, so It would be possible for GC to collect it. 
 Please
 fill issue


 And on 2.066.1 it works correctly, so it is a regression


 I really didn't know that `destroy` would make any difference. 
 But I made changes in my libraries. Before null'g objects, I 
 used `destroy` as well, and memory consumption is not 
 increasing at all. Cheers for that ;)

 I think `destroy`'s behaviour should be documented much better.

 If I can reach to my bugzilla account (Everytime I forget my 
 password), I will report it now.


Reported this issue

https://issues.dlang.org/show_bug.cgi?id=14573
May 11 2015
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 05/11/2015 03:48 AM, tcak wrote:


 I think `destroy`'s behaviour should be documented much better.


Could others tell us the missing pieces please.

Perhaps related to this discussion, the following document says "does 
not initiate a GC cycle or free any GC memory":

   http://dlang.org/library/object/destroy.html

Thank you,
Ali
May 11 2015
parent reply "tcak" <tcak gmail.com> writes:
On Monday, 11 May 2015 at 15:38:55 UTC, Ali Çehreli wrote:

 On 05/11/2015 03:48 AM, tcak wrote:


 I think `destroy`'s behaviour should be documented much


 better.

 Could others tell us the missing pieces please.

 Perhaps related to this discussion, the following document says 
 "does not initiate a GC cycle or free any GC memory":

   http://dlang.org/library/object/destroy.html

 Thank you,
 Ali


Setting object's reference to null, and destroy are very 
different based on my experience. Because if same things were 
happening at the end, that memory increase shouldn't have been 
happening. What I meant with documenting it better was this.
May 11 2015
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 05/11/2015 09:50 AM, tcak wrote:


 On Monday, 11 May 2015 at 15:38:55 UTC, Ali Çehreli wrote:

 On 05/11/2015 03:48 AM, tcak wrote:


 I think `destroy`'s behaviour should be documented much


 better.

 Could others tell us the missing pieces please.

 Perhaps related to this discussion, the following document says "does
 not initiate a GC cycle or free any GC memory":

   http://dlang.org/library/object/destroy.html

 Thank you,
 Ali


 Setting object's reference to null, and destroy are very different based
 on my experience.


I see. There are the following differences that I know:

Setting the object to null should not have any immediate effect. It 
merely erases a single reference to the object. If there is no other 
reference to it, the object will be collected in the future, including 
running its destructor.

Regarding your original code, setting it to null should have no effect, 
because the test code had a single reference anyway.

destroy() executes the destructor of the object right away. So, just the 
memory that was used for the object remains but provided that the 
destructor is written correctly, the resources that the object owned 
should be returned.


 Because if same things were happening at the end, that memory
 increase shouldn't have been happening.


Yeah, luckily it is just a regression that should be fixed soon.


 What I meant with documenting it better was this.


There is also the new behavior of running the destructors of a struct 
object even if were constructed by new. In the past, the destructors of 
dynamic struct objects would not be executed by the GC.

Ali
May 11 2015