www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Memory Allocation

reply Enigma <Enigma dot.com> writes:
I have a memory buffer allocated using different methods. It is 
simply a pointer and a size.

I would like to be able to manage this buffer by treating it as a 
memory pool or heap. I think I can use allocators to do this but 
not sure how.

Effectively I want something like new or malloc but it pulls from 
the memory buffer rather than the program heap.

// Allocated once at program start
void* FancyBuffer = FancyAlloc(1000);


Then when I want to use the buffer I'll do stuff like



auto myptr = FancyMalloc(10);

....

FancyFree(myptr);


or whatever.

The main thing is, I don't want to have to write my own allocator 
to manage this buffer as it seems that D's allocators would do a 
better job.

I imagine that most of the time the buffer will not have more 
than one piece of code using it(no overlapping uses) but since I 
won't be 100% sure, I need allow for the cases where there might 
be overlapping usage. (else I wouldn't ever have to worry about 
"allocating or releasing" from it.

Thanks.
Mar 29 2017
next sibling parent reply Faux Amis <faux amis.com> writes:
On 2017-03-29 21:19, Enigma wrote:

 I have a memory buffer allocated using different methods. It is simply a
 pointer and a size.


Can you maybe just tread it like an array and slice it for allocation?
Mar 29 2017
parent Faux Amis <faux amis.com> writes:
On 2017-03-29 23:30, Faux Amis wrote:

 On 2017-03-29 21:19, Enigma wrote:

 I have a memory buffer allocated using different methods. It is simply a
 pointer and a size.


 Can you maybe just tread it like an array and slice it for allocation?


*treat*
Mar 29 2017
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:

 I have a memory buffer allocated using different methods. It is 
 simply a pointer and a size.

 I would like to be able to manage this buffer by treating it as 
 a memory pool or heap. I think I can use allocators to do this 
 but not sure how.

 Effectively I want something like new or malloc but it pulls 
 from the memory buffer rather than the program heap.

 // Allocated once at program start
 void* FancyBuffer = FancyAlloc(1000);


 Then when I want to use the buffer I'll do stuff like



 auto myptr = FancyMalloc(10);

 ....

 FancyFree(myptr);


 or whatever.

 The main thing is, I don't want to have to write my own 
 allocator to manage this buffer as it seems that D's allocators 
 would do a better job.

 I imagine that most of the time the buffer will not have more 
 than one piece of code using it(no overlapping uses) but since 
 I won't be 100% sure, I need allow for the cases where there 
 might be overlapping usage. (else I wouldn't ever have to worry 
 about "allocating or releasing" from it.

 Thanks.


It looks like you are looking for this: 
http://dlang.org/phobos-prerelease/std_experimental_allocator_building_blocks_region.html.
Mar 29 2017
parent reply Enigma <Enigma dot.com> writes:
On Wednesday, 29 March 2017 at 21:36:14 UTC, Petar Kirov 
[ZombineDev] wrote:

 On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:

 [...]


 It looks like you are looking for this: 
 http://dlang.org/phobos-prerelease/std_experimental_allocator_building_blocks_region.html.


But these seem to require passing a mallocator. I simply want to 
pass an already allocated region/block/buffer and have the 
allocators use it. I will allocate and free on my own.
Mar 29 2017
parent reply "H. S. Teoh via Digitalmars-d-learn" <digitalmars-d-learn puremagic.com> writes:
On Wed, Mar 29, 2017 at 11:01:12PM +0000, Enigma via Digitalmars-d-learn wrote:

 On Wednesday, 29 March 2017 at 21:36:14 UTC, Petar Kirov [ZombineDev] wrote:

 On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:

 [...]


 
 It looks like you are looking for this:
 http://dlang.org/phobos-prerelease/std_experimental_allocator_building_blocks_region.html.


 
 But these seem to require passing a mallocator. I simply want to pass
 an already allocated region/block/buffer and have the allocators use
 it. I will allocate and free on my own.


Huh?  Where does it say that a mallocator is required?

As far as I can tell, you could simply do this:

	void[] myBuffer = ...;
	auto allocator = Region!()(myBuffer);
	auto p = allocator.allocate(...);

The default parent allocator is NullAllocator, which does nothing, so
that leaves the management of myBuffer entirely up to you.


T

-- 
Life would be easier if I had the source code. -- YHL
Mar 29 2017
parent Enigma <Enigma dot.com> writes:
On Wednesday, 29 March 2017 at 23:26:04 UTC, H. S. Teoh wrote:

 On Wed, Mar 29, 2017 at 11:01:12PM +0000, Enigma via 
 Digitalmars-d-learn wrote:

 On Wednesday, 29 March 2017 at 21:36:14 UTC, Petar Kirov 
 [ZombineDev] wrote:

 [...]


 
 But these seem to require passing a mallocator. I simply want 
 to pass an already allocated region/block/buffer and have the 
 allocators use it. I will allocate and free on my own.


 Huh?  Where does it say that a mallocator is required?

 As far as I can tell, you could simply do this:

 	void[] myBuffer = ...;
 	auto allocator = Region!()(myBuffer);
 	auto p = allocator.allocate(...);

 The default parent allocator is NullAllocator, which does 
 nothing, so that leaves the management of myBuffer entirely up 
 to you.


 T


Thanks. All the examples show using a size rather than an array 
so I assumed that was the case.
Mar 30 2017
prev sibling parent reply Gary Willoughby <dev nomad.so> writes:
On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:

 I have a memory buffer allocated using different methods. It is 
 simply a pointer and a size.

 I would like to be able to manage this buffer by treating it as 
 a memory pool or heap. I think I can use allocators to do this 
 but not sure how.


You can re-task the GC to use an arena if that's suitable:

https://bitbucket.org/infognition/dstuff/src/1dca752af1021ed5998bb041004465674695113f/gcarena.d?fileviewer=file-view-default
Mar 30 2017
parent Enigma <Enigma dot.com> writes:
On Thursday, 30 March 2017 at 11:22:05 UTC, Gary Willoughby wrote:

 On Wednesday, 29 March 2017 at 19:19:48 UTC, Enigma wrote:

 I have a memory buffer allocated using different methods. It 
 is simply a pointer and a size.

 I would like to be able to manage this buffer by treating it 
 as a memory pool or heap. I think I can use allocators to do 
 this but not sure how.


 You can re-task the GC to use an arena if that's suitable:

 https://bitbucket.org/infognition/dstuff/src/1dca752af1021ed5998bb041004465674695113f/gcarena.d?fileviewer=file-view-default


cool. thanks.
Mar 30 2017