• Nikolay (25/25) Apr 18 2015 I wrote simple proof of concept library. The main aim is to
• Gary Willoughby (4/32) Apr 18 2015 This reminds me of another useful library here:
• Nikolay (3/6) Apr 18 2015 Thanks for link.
• =?UTF-8?B?Ik3DoXJjaW8=?= Martins" (2/30) Apr 18 2015 I would make the inplace buffer's size a template parameter :)

"Nikolay" <sibnick gmail.com> writes:

I wrote simple proof of concept library. The main aim is to
reduce GC usage and improve data locality by replacing dynamic
array for small immutable arrays.


You can find more info here:
    * wiki - https://bitbucket.org/sibnick/inplacearray/wiki/Home
    * source code - https://bitbucket.org/sibnick/inplacearray/src

I include small demo - word counter. This program builds AA array
(word => counter) for given text file in UTF-8. It is possible
switch between builtin array/string and inplace array by changing
one line code. I used:

    -  unpacked List of all page titles -
https://dumps.wikimedia.org/enwiki/20150304/enwiki-20150304-all-titles.gz
as test sample.
    - Ubuntu 14.04 x64
    - dmd 2.67.0
    - dub build -b release
    - ./inplace_array --DRT-gcopt=profile:1 test_big.txt

Here results.

Buildin string:
GC summary: 1355 MB, 12 GC 5659 ms, Pauses 3116 ms < 1436 ms

Inplace array:
GC summary: 715 MB, 22 GC 5281 ms, Pauses 4207 ms < 938 ms

As you can see it is possible noticable reduce memory allocation
(1355Mb -> 715Mb) and max pause time (1436ms -> 938ms).

So it is possible improve situation with GC from new side.

"Gary Willoughby" <dev nomad.so> writes:

On Saturday, 18 April 2015 at 10:49:01 UTC, Nikolay wrote:
 I wrote simple proof of concept library. The main aim is to
 reduce GC usage and improve data locality by replacing dynamic
 array for small immutable arrays.


 You can find more info here:
    * wiki - https://bitbucket.org/sibnick/inplacearray/wiki/Home
    * source code - 
 https://bitbucket.org/sibnick/inplacearray/src

 I include small demo - word counter. This program builds AA 
 array
 (word => counter) for given text file in UTF-8. It is possible
 switch between builtin array/string and inplace array by 
 changing
 one line code. I used:

    -  unpacked List of all page titles -
 https://dumps.wikimedia.org/enwiki/20150304/enwiki-20150304-all-titles.gz
 as test sample.
    - Ubuntu 14.04 x64
    - dmd 2.67.0
    - dub build -b release
    - ./inplace_array --DRT-gcopt=profile:1 test_big.txt

 Here results.

 Buildin string:
 GC summary: 1355 MB, 12 GC 5659 ms, Pauses 3116 ms < 1436 ms

 Inplace array:
 GC summary: 715 MB, 22 GC 5281 ms, Pauses 4207 ms < 938 ms

 As you can see it is possible noticable reduce memory allocation
 (1355Mb -> 715Mb) and max pause time (1436ms -> 938ms).

 So it is possible improve situation with GC from new side.

This reminds me of another useful library here:

https://bitbucket.org/infognition/dstuff/src/

See gcarena.d

"Nikolay" <sibnick gmail.com> writes:

 This reminds me of another useful library here:

 https://bitbucket.org/infognition/dstuff/src/

 See gcarena.d

Thanks for link.

The main difference is that I want elimanate pair pointer+data at 
all. It is more effective to store small array as value type.

=?UTF-8?B?Ik3DoXJjaW8=?= Martins" <marcioapm gmail.com> writes:

On Saturday, 18 April 2015 at 10:49:01 UTC, Nikolay wrote:
 I wrote simple proof of concept library. The main aim is to
 reduce GC usage and improve data locality by replacing dynamic
 array for small immutable arrays.


 You can find more info here:
    * wiki - https://bitbucket.org/sibnick/inplacearray/wiki/Home
    * source code - 
 https://bitbucket.org/sibnick/inplacearray/src

 I include small demo - word counter. This program builds AA 
 array
 (word => counter) for given text file in UTF-8. It is possible
 switch between builtin array/string and inplace array by 
 changing
 one line code. I used:

    -  unpacked List of all page titles -
 https://dumps.wikimedia.org/enwiki/20150304/enwiki-20150304-all-titles.gz
 as test sample.
    - Ubuntu 14.04 x64
    - dmd 2.67.0
    - dub build -b release
    - ./inplace_array --DRT-gcopt=profile:1 test_big.txt

 Here results.

 Buildin string:
 GC summary: 1355 MB, 12 GC 5659 ms, Pauses 3116 ms < 1436 ms

 Inplace array:
 GC summary: 715 MB, 22 GC 5281 ms, Pauses 4207 ms < 938 ms

 As you can see it is possible noticable reduce memory allocation
 (1355Mb -> 715Mb) and max pause time (1436ms -> 938ms).

 So it is possible improve situation with GC from new side.

I would make the inplace buffer's size a template parameter :)