www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - associative arrays with manual memory management

reply "Ilya Yaroshenko" <ilyayaroshenko gmail.com> writes:
http://code.dlang.org/packages/aammm/~master


Associative arrays with manual memory management

All enries and buckets would be dealocated and disposed by 
internal implementation's destructor.
The destructor is called by garbage collector (by default).


```D
     //std.experimental.allocator is included into `aammm`
     import std.experimental.allocator.mallocator;
     import aammm;

     auto a = AA!(string, int, shared 
Mallocator)(Mallocator.instance);
     a["foo"] = 0;
     a.remove("foo"); //dealocates and disposes the entry
     assert(a == null); // should not crash
```
AAMMM is based on Andrei's allocators and Martin's associative 
arrays.

References:
http://erdani.com/d/phobos-prerelease/std_experimental_allocator.html
https://github.com/D-Programming-Language/druntime/pull/1282

Best Regards,
Ilya
Aug 24 2015
next sibling parent "extrawurst" <stephan extrawurst.org> writes:
On Monday, 24 August 2015 at 12:01:52 UTC, Ilya Yaroshenko wrote:

 http://code.dlang.org/packages/aammm/~master


 Associative arrays with manual memory management

 [...]


Awesome, I was waiting for something like that. Thank you!
Aug 24 2015
prev sibling next sibling parent reply "Per =?UTF-8?B?Tm9yZGzDtnci?= <per.nordlow gmail.com> writes:
On Monday, 24 August 2015 at 12:01:52 UTC, Ilya Yaroshenko wrote:

Nice! I'll try this!


     auto a = AA!(string, int, shared Mallocator)


When does the third parameter need to be qualified as `shared`?
Aug 25 2015
parent reply "Ilya Yaroshenko" <ilyayaroshenko gmail.com> writes:
On Wednesday, 26 August 2015 at 06:41:41 UTC, Per Nordlöw wrote:

 On Monday, 24 August 2015 at 12:01:52 UTC, Ilya Yaroshenko 
 wrote:

 Nice! I'll try this!


     auto a = AA!(string, int, shared Mallocator)


 When does the third parameter need to be qualified as `shared`?


Only if you would use a shared allocator like Mallocator or 
GCAllocator.

I will add factory template like Dmitry suggested.

auto a  = aa!(string, int)(Mallocator.instance); // 3rd CT param 
is deduced

Ilya
Aug 26 2015
parent reply "Per =?UTF-8?B?Tm9yZGzDtnci?= <per.nordlow gmail.com> writes:
On Wednesday, 26 August 2015 at 09:20:33 UTC, Ilya Yaroshenko 
wrote:

 Only if you would use a shared allocator like Mallocator or 
 GCAllocator.


Are there cases where a non-shared version of Mallocator or 
GCAllocator is motivated?

If not could, maybe the shared-ness could be inferred?
Aug 26 2015
parent "Ilya Yaroshenko" <ilyayaroshenko gmail.com> writes:
On Wednesday, 26 August 2015 at 13:12:38 UTC, Per Nordlöw wrote:

 On Wednesday, 26 August 2015 at 09:20:33 UTC, Ilya Yaroshenko 
 wrote:

 Only if you would use a shared allocator like Mallocator or 
 GCAllocator.


 Are there cases where a non-shared version of Mallocator or 
 GCAllocator is motivated?


Mallocator and GCAllocator are always shared.
Possibly D will have ThreadLocalGCAllocator in the future.


 If not could, maybe the shared-ness could be inferred?


`aa` function just uses shared type for shared argument and vise 
versa.
GCAllocator.instance and Mallocator.instance. are always shared.
Aug 26 2015
prev sibling next sibling parent reply "Per =?UTF-8?B?Tm9yZGzDtnci?= <per.nordlow gmail.com> writes:
On Monday, 24 August 2015 at 12:01:52 UTC, Ilya Yaroshenko wrote:

 http://code.dlang.org/packages/aammm/~master


It would be nice to have a test example for other allocators. I'm 
especially interested in how much speed we can gain with using a 
non-shared BlockAllocator in combination with aammm.AA.

Can the whole AA be made ` safe pure nothrow` if the allocator is 
stored inside the AA itself (non-shared)?

I'm especially interested in using this for keys and values only 
with no indirections.
Aug 25 2015
next sibling parent "Ilya Yaroshenko" <ilyayaroshenko gmail.com> writes:
On Wednesday, 26 August 2015 at 06:50:26 UTC, Per Nordlöw wrote:

 On Monday, 24 August 2015 at 12:01:52 UTC, Ilya Yaroshenko 
 wrote:

 http://code.dlang.org/packages/aammm/~master


 It would be nice to have a test example for other allocators. 
 I'm especially interested in how much speed we can gain with 
 using a non-shared BlockAllocator in combination with aammm.AA.

 Can the whole AA be made ` safe pure nothrow` if the allocator 
 is stored inside the AA itself (non-shared)?



Yes, except constructor. To do not use constructor you can use 
`makeAA` and `disposeAA`.


 I'm especially interested in using this for keys and values 
 only with no indirections.
Aug 26 2015
prev sibling parent "Ilya Yaroshenko" <ilyayaroshenko gmail.com> writes:
On Wednesday, 26 August 2015 at 06:50:26 UTC, Per Nordlöw wrote:

 On Monday, 24 August 2015 at 12:01:52 UTC, Ilya Yaroshenko 
 wrote:

 http://code.dlang.org/packages/aammm/~master


 It would be nice to have a test example for other allocators. 
 I'm especially interested in how much speed we can gain with 
 using a non-shared BlockAllocator in combination with aammm.AA.

 Can the whole AA be made ` safe pure nothrow` if the allocator 
 is stored inside the AA itself (non-shared)?


EDIT: Plus I think to RefCountingAA.


 I'm especially interested in using this for keys and values 
 only with no indirections.
Aug 26 2015
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 24-Aug-2015 15:01, Ilya Yaroshenko wrote:

 http://code.dlang.org/packages/aammm/~master


 Associative arrays with manual memory management

 All enries and buckets would be dealocated and disposed by internal
 implementation's destructor.
 The destructor is called by garbage collector (by default).


Rox!



 ```D
      //std.experimental.allocator is included into `aammm`
      import std.experimental.allocator.mallocator;
      import aammm;

      auto a = AA!(string, int, shared Mallocator)(Mallocator.instance);


Sure hope a factory to do IFTI is available? So that the following works:

auto a  = aa!(string, int)(Mallocator.instance); // 3rd CT param is deduced


      a["foo"] = 0;
      a.remove("foo"); //dealocates and disposes the entry
      assert(a == null); // should not crash
 ```




-- 
Dmitry Olshansky
Aug 25 2015
parent reply "Ilya Yaroshenko" <ilyayaroshenko gmail.com> writes:
On Wednesday, 26 August 2015 at 06:52:01 UTC, Dmitry Olshansky 
wrote:

      auto a = AA!(string, int, shared 
 Mallocator)(Mallocator.instance);


 Sure hope a factory to do IFTI is available? So that the 
 following works:

 auto a  = aa!(string, int)(Mallocator.instance); // 3rd CT 
 param is deduced


Just added to master branch) Thanks!
Aug 26 2015
parent reply "Per =?UTF-8?B?Tm9yZGzDtnci?= <per.nordlow gmail.com> writes:
On Wednesday, 26 August 2015 at 10:48:11 UTC, Ilya Yaroshenko >> 
auto a  = aa!(string, int)(Mallocator.instance); // 3rd CT

highlights

It would be nice to also see an example at
https://github.com/arexeu/aammm

that shows AA-usage in conjunction with some other allocator such 
as FreeList and add a note about the performance improvement this 
gives.

In that case, would it be possible to have factory functions for 
AA that automatically derives allocator parameters from key and 
value type for specific allocators?

What do you say?
Aug 26 2015
next sibling parent reply "Ilya Yaroshenko" <ilyayaroshenko gmail.com> writes:
On Wednesday, 26 August 2015 at 12:10:17 UTC, Per Nordlöw wrote:

 highlights

 It would be nice to also see an example at
 https://github.com/arexeu/aammm

 that shows AA-usage in conjunction with some other allocator 
 such as FreeList and add a note about the performance 
 improvement this gives.

 In that case, would it be possible to have factory functions 
 for AA that automatically derives allocator parameters from key 
 and value type for specific allocators?

 What do you say?


It is possible but not so useful, thought. 
`AA!(...).Entry.sizeof` and `AA!(...).Entry.alignof` should be 
accessible from user code since v0.0.3 . They can be used to 
construct allocators. I will add example with FreeList.
Aug 26 2015
parent reply "Per =?UTF-8?B?Tm9yZGzDtnci?= <per.nordlow gmail.com> writes:
On Wednesday, 26 August 2015 at 14:21:41 UTC, Ilya Yaroshenko 
wrote:

 It is possible but not so useful, thought.


Why is it not so useful?


 `AA!(...).Entry.sizeof` and `AA!(...).Entry.alignof` should be 
 accessible from user code since v0.0.3 . They can be used to 
 construct allocators. I will add example with FreeList.


Great! Thanks!
Aug 26 2015
parent "Ilya Yaroshenko" <ilyayaroshenko gmail.com> writes:
On Wednesday, 26 August 2015 at 14:24:54 UTC, Per Nordlöw wrote:

 On Wednesday, 26 August 2015 at 14:21:41 UTC, Ilya Yaroshenko 
 wrote:

 It is possible but not so useful, thought.


 Why is it not so useful?


Because looks like allocators have different additional params, 
that dose not related to AA.
Aug 26 2015
prev sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Wednesday, 26 August 2015 at 12:10:17 UTC, Per Nordlöw wrote:

 On Wednesday, 26 August 2015 at 10:48:11 UTC, Ilya Yaroshenko

 auto a  = aa!(string, int)(Mallocator.instance); // 3rd CT




 highlights

 It would be nice to also see an example at
 https://github.com/arexeu/aammm

 that shows AA-usage in conjunction with some other allocator 
 such as FreeList and add a note about the performance 
 improvement this gives.


I have not any significant performance improvement with 
FreeList/SharedFreeList comparing with Mallocator. However, the 
difference can be significant for server with 20-30 core CPUs 
because malloc function use internal lock.

Simple benchmark:
https://github.com/arexeu/aammm/commit/260ef4f94618b85463dec5c90e0b894b08750d07#diff-24efeb785f1d7039ab1c9bc29ba99c99R1071
Sep 13 2015