www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - About GC

reply Alexandr Druzhinin <drug2004 bk.ru> writes:
In the following code http://dpaste.dzfl.pl/f4ae16aa
memory consumption depends on DATA_SIZE value. Bigger value bigger 
consumption - it's natural. Consumption starts from some value and 
continue to some threshold. After reachin this threshold comsumption 
doesn't enlarge. This threshold depends on DATA_SIZE value and some 
others. My question is how can I find this threshold to know is this 
value suitable for real using or I should change something in my code. 
Is there some guarantee that this threshold will be less than real 
memory on the box? How can I control this threshold?

P.S. or may be I do something wrong in the snippet?
Feb 24 2013
next sibling parent Alexandr Druzhinin <drug2004 bk.ru> writes:
24.02.2013 21:26, Alexandr Druzhinin пишет:

 In the following code http://dpaste.dzfl.pl/f4ae16aa
 memory consumption depends on DATA_SIZE value. Bigger value bigger
 consumption - it's natural. Consumption starts from some value and
 continue to some threshold. After reachin this threshold comsumption
 doesn't enlarge. This threshold depends on DATA_SIZE value and some
 others. My question is how can I find this threshold to know is this
 value suitable for real using or I should change something in my code.
 Is there some guarantee that this threshold will be less than real
 memory on the box? How can I control this threshold?

 P.S. or may be I do something wrong in the snippet?


For example if DATA_SIZE = 300 000 memory consumption is 60-70 M, 200 
000 - ~40M, 100 000 - ~ 18M (win7 64bits dmd 2.062)
Feb 24 2013
prev sibling next sibling parent Alexandr Druzhinin <drug2004 bk.ru> writes:
24.02.2013 21:26, Alexandr Druzhinin пишет:

 In the following code http://dpaste.dzfl.pl/f4ae16aa
 memory consumption depends on DATA_SIZE value. Bigger value bigger
 consumption - it's natural. Consumption starts from some value and
 continue to some threshold. After reachin this threshold comsumption
 doesn't enlarge. This threshold depends on DATA_SIZE value and some
 others. My question is how can I find this threshold to know is this
 value suitable for real using or I should change something in my code.
 Is there some guarantee that this threshold will be less than real
 memory on the box? How can I control this threshold?

 P.S. or may be I do something wrong in the snippet?


as I understand the threshold is reached when gc starts freeing the same 
amount of memory as it allocates? and because of it consumption 
stabilizes? So adjusting or restriction of this threshold in more 
complex applications is something very hard. And it's possible that if 
application suddenly gets data more than before application may crashes 
because this threshold oversize than available memory?
Feb 24 2013
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 02/24/2013 06:26 AM, Alexandr Druzhinin wrote:

 In the following code http://dpaste.dzfl.pl/f4ae16aa
 memory consumption depends on DATA_SIZE value. Bigger value bigger
 consumption - it's natural. Consumption starts from some value and
 continue to some threshold. After reachin this threshold comsumption
 doesn't enlarge. This threshold depends on DATA_SIZE value and some
 others.


What have you observed that threshold value to be? Your program has a 
constant 40 for the 'threshold' parameter. Is that the special value?


 My question is how can I find this threshold to know is this
 value suitable for real using or I should change something in my code.
 Is there some guarantee that this threshold will be less than real
 memory on the box? How can I control this threshold?


I don't understand the question yet but in general such optimizations 
depend on the actual program. You must make measurements to find that 
out for your program.


 For example if DATA_SIZE = 300 000 memory consumption is 60-70
 M, 200 000 - ~40M, 100 000 - ~ 18M (win7 64bits dmd 2.062)


I think that behavior is natural too but again, I don't understand the 
threshold from those figures.


 And it's possible that if application suddenly gets data
 more than before application may crashes because this threshold
 oversize than available memory?


Possible. Sometimes, there may be a lot more memory that requested but 
the memory may be fragmented so that there is not any memory chunk as 
large as the requested amount.

Ali
Feb 24 2013
parent Alexandr Druzhinin <drug2004 bk.ru> writes:
24.02.2013 23:28, Ali Çehreli пишет:

 On 02/24/2013 06:26 AM, Alexandr Druzhinin wrote:
  > In the following code http://dpaste.dzfl.pl/f4ae16aa
  > memory consumption depends on DATA_SIZE value. Bigger value bigger
  > consumption - it's natural. Consumption starts from some value and
  > continue to some threshold. After reachin this threshold comsumption
  > doesn't enlarge. This threshold depends on DATA_SIZE value and some
  > others.

 What have you observed that threshold value to be? Your program has a
 constant 40 for the 'threshold' parameter. Is that the special value?

  > My question is how can I find this threshold to know is this
  > value suitable for real using or I should change something in my code.
  > Is there some guarantee that this threshold will be less than real
  > memory on the box? How can I control this threshold?

 I don't understand the question yet but in general such optimizations
 depend on the actual program. You must make measurements to find that
 out for your program.

  > For example if DATA_SIZE = 300 000 memory consumption is 60-70
  > M, 200 000 - ~40M, 100 000 - ~ 18M (win7 64bits dmd 2.062)

 I think that behavior is natural too but again, I don't understand the
 threshold from those figures.

  > And it's possible that if application suddenly gets data
  > more than before application may crashes because this threshold
  > oversize than available memory?

 Possible. Sometimes, there may be a lot more memory that requested but
 the memory may be fragmented so that there is not any memory chunk as
 large as the requested amount.

 Ali


Sorry, this was reduced code and it became non-informative. I didn't 
mean threshold argument of DataStorage.getSnapshot. Saying `threshold` I 
meant the value at which size of used memory stabilizes.
I mean that when if I use bigger value of DATA_SIZE I get more memory 
used and it's normal. But I have no idea how much memory will be used if 
DATA_SIZE changes. And in complex code (i.e. real application) this 
dependence of size of used memory will be unknown for developer?
Feb 24 2013