www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - safe containers with std.experimental.allocator

reply bitwise <bitwise.pvt gmail.com> writes:
When std.experimental.allocator was created, I was under the 
impression it was meant to be used to make containers. But since 
allocate() returns void[], casts are required before using that 
memory, which is unsafe.

Is my only option to wrap the casts in an  trusted helper 
function? or am I missing something?

  Thanks
Jan 20 2017
next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:

 [...]


I can see that 'TypedAllocator' exists, but with a different 
interface than the rest of the allocators. Why wouldn't all 
allocators just have a common interface? Why wouldn't Mallocator 
take a template parameter allowing usage with 'ubyte' or 'void' 
for uninitialized memory via 'allocate' or 
'allocateUninitialized' functions?

For my own purposes, I would need my containers to have a default 
allocator(like Mallocator), and support an aligned 
allocator(doesn't seem to be present in std.experimental).
Jan 20 2017
parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 21 Jan 2017 04:01:06 +0000, bitwise wrote:


 On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:

 [...]


 
 I can see that 'TypedAllocator' exists, but with a different interface
 than the rest of the allocators. Why wouldn't all allocators just have a
 common interface?


TypedAllocator combines several allocators, using a series of Policies to 
determine which one to use for a given type.

The interface for allocators is mostly given by IAllocator and mostly 
implemented with UFCS functions defined in 
std.experimental.allocator.package. The one aspect that isn't given with 
IAllocator (because OOP doesn't support it well) is that not all 
allocators have to support aligned allocations.

TypedAllocator overrides some of those default implementations in order 
to use its Policies. It doesn't expose the untyped variants so that you 
can't get around its Policies.

Unfortunately, there's no way to require that a TypedAllocator return an 
aligned allocation aside from using an aligned allocator for the relevant 
policy.
Jan 21 2017
prev sibling next sibling parent reply Eugene Wissner <belka caraus.de> writes:
On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:

 When std.experimental.allocator was created, I was under the 
 impression it was meant to be used to make containers. But 
 since allocate() returns void[], casts are required before 
 using that memory, which is unsafe.

 Is my only option to wrap the casts in an  trusted helper 
 function? or am I missing something?

  Thanks


System functions that do memory allocation, give you a memory 
block as a void*. So a cast is anyway required to cast to the 
required data type.

Allocators don't cast from void* because you would need to 
implement the same logic in each allocator. Therefore there are 
functions like make/makeArray/dispose that take an allocator, 
cast and initialize data.

As for safity part. Allocators can't provide the same safity as 
the garbage collector anyway, because if you use manual memory 
management you get all the problems like you can pass to 
deallocate() a wrong pointer or to free a memory block twice. But 
with the right abstractions and enough testing, you can reduce 
such problems and make the code more trusted.

Alligned allocator isn't present in the 
std.experimental.allocator because IAllocator has a method 
alignedAllocate() (or similar) - so any allocator can be designed 
to provide aligned allocations.
Jan 20 2017
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 05:06:07 UTC, Eugene Wissner 
wrote:

 Alligned allocator isn't present in the 
 std.experimental.allocator because IAllocator has a method 
 alignedAllocate()


I hadn't noticed this, but I immediately see two problems:

1) there is no alignedDeallocate() which is needed because 
aligned allocators usually prepend metadata containing a pointer 
to the actual start of the memory to be passed to free(). So 
deallocate() can't know if the memory came from allocate() or 
alignedAllocate(), and hence does not now how to properly free 
the memory.

2) The whole point of using an allocator is that is allocates 
memory in a different way, but provides a standard interface. It 
shouldn't be up to a container to know whether to call allocate() 
or alignedAllocate(). A container should simply call allocate() 
of whatever allocator it was given.
Jan 20 2017
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sat, 21 Jan 2017 05:52:49 +0000, bitwise wrote:


 On Saturday, 21 January 2017 at 05:06:07 UTC, Eugene Wissner wrote:

 Alligned allocator isn't present in the std.experimental.allocator
 because IAllocator has a method alignedAllocate()


 
 I hadn't noticed this, but I immediately see two problems:
 
 1) there is no alignedDeallocate() which is needed because aligned
 allocators usually prepend metadata containing a pointer to the actual
 start of the memory to be passed to free(). So deallocate() can't know
 if the memory came from allocate() or alignedAllocate(), and hence does
 not now how to properly free the memory.


Check the code. CAllocatorImpl's implementation of alignedAllocate 
returns null if the implementation doesn't support alignedAllocate. 
(CAllocatorImpl is the provided implementation of IAllocator.)


 2) The whole point of using an allocator is that is allocates memory in
 a different way, but provides a standard interface. It shouldn't be up
 to a container to know whether to call allocate()
 or alignedAllocate(). A container should simply call allocate() of
 whatever allocator it was given.


If you need aligned memory, you need to use alignedAllocate. You could 
unconditionally wrap the given allocator using BitmappedBlock, but this 
adds unnecessary overhead if the user passed in an allocator that 
supports aligned allocation.
Jan 20 2017
prev sibling parent reply Eugene Wissner <belka caraus.de> writes:
On Saturday, 21 January 2017 at 05:52:49 UTC, bitwise wrote:

 I hadn't noticed this, but I immediately see two problems:

 1) there is no alignedDeallocate() which is needed because 
 aligned allocators usually prepend metadata containing a 
 pointer to the actual start of the memory to be passed to 
 free(). So deallocate() can't know if the memory came from 
 allocate() or alignedAllocate(), and hence does not now how to 
 properly free the memory.

 2) The whole point of using an allocator is that is allocates 
 memory in a different way, but provides a standard interface. 
 It shouldn't be up to a container to know whether to call 
 allocate() or alignedAllocate(). A container should simply call 
 allocate() of whatever allocator it was given.


yes, agreed. alignedAllocate() seems then to make the interface 
complexer without reason.
Jan 20 2017
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/21/2017 01:06 AM, Eugene Wissner wrote:

 On Saturday, 21 January 2017 at 05:52:49 UTC, bitwise wrote:

 I hadn't noticed this, but I immediately see two problems:

 1) there is no alignedDeallocate() which is needed because aligned
 allocators usually prepend metadata containing a pointer to the actual
 start of the memory to be passed to free(). So deallocate() can't know
 if the memory came from allocate() or alignedAllocate(), and hence
 does not now how to properly free the memory.

 2) The whole point of using an allocator is that is allocates memory
 in a different way, but provides a standard interface. It shouldn't be
 up to a container to know whether to call allocate() or
 alignedAllocate(). A container should simply call allocate() of
 whatever allocator it was given.


 yes, agreed. alignedAllocate() seems then to make the interface
 complexer without reason.


alignedAllocate provides access to OS/clib-provided primitives for 
aligned allocation. Those don't require a special deallocation function, 
see e.g. http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei
Jan 21 2017
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 16:28:16 UTC, Andrei Alexandrescu 
wrote:

 alignedAllocate provides access to OS/clib-provided primitives 
 for aligned allocation. Those don't require a special 
 deallocation function, see e.g. 
 http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei


This makes sense, but then shouldn't alignedAllocate() be a free 
function that could be used to make an aligned allocator?

Again, the point of an allocator is to provide a standard 
interface for memory allocation. The user of which shouldn't have 
to know how that memory was allocated, and I can't think of a 
case where this would be desired either.
Jan 21 2017
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 01/21/2017 12:14 PM, bitwise wrote:

 On Saturday, 21 January 2017 at 16:28:16 UTC, Andrei Alexandrescu wrote:

 alignedAllocate provides access to OS/clib-provided primitives for
 aligned allocation. Those don't require a special deallocation
 function, see e.g.
 http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei


 This makes sense, but then shouldn't alignedAllocate() be a free
 function that could be used to make an aligned allocator?


You mean an aligned allocation? But then the actual method, and whether 
it is supported at all or not, depends on the allocator.


 Again, the point of an allocator is to provide a standard interface for
 memory allocation. The user of which shouldn't have to know how that
 memory was allocated, and I can't think of a case where this would be
 desired either.


I don't understand this. It's possible there's a confusion at a 
different level, and we're discussing the consequences.

To reboot:

* Some applications need memory aligned at special powers of two, most 
are okay with more generic alignments such as word-level. See the 
"alignment" enum that each allocator must define.

* Some allocators do offer allocations at unusual powers of two by means 
of alignedAllocate. Don't call it naively! Only applications that need 
such things (e.g. page-aligned, cache-line-aligned, etc) should need those.

* Those allocators must be able to deallocate memory allocated with 
allocate() or alignedAllocate() with a single primitive deallocate().


Andrei
Jan 21 2017
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 17:26:35 UTC, Andrei Alexandrescu 
wrote:

 On 01/21/2017 12:14 PM, bitwise wrote:

 On Saturday, 21 January 2017 at 16:28:16 UTC, Andrei 
 Alexandrescu wrote:

 alignedAllocate provides access to OS/clib-provided 
 primitives for
 aligned allocation. Those don't require a special deallocation
 function, see e.g.
 http://en.cppreference.com/w/c/memory/aligned_alloc. -- Andrei


 This makes sense, but then shouldn't alignedAllocate() be a 
 free
 function that could be used to make an aligned allocator?


 You mean an aligned allocation? But then the actual method, and 
 whether it is supported at all or not, depends on the allocator.


 Again, the point of an allocator is to provide a standard 
 interface for
 memory allocation. The user of which shouldn't have to know 
 how that
 memory was allocated, and I can't think of a case where this 
 would be
 desired either.


 I don't understand this. It's possible there's a confusion at a 
 different level, and we're discussing the consequences.

 To reboot:

 * Some applications need memory aligned at special powers of 
 two, most are okay with more generic alignments such as 
 word-level. See the "alignment" enum that each allocator must 
 define.

 * Some allocators do offer allocations at unusual powers of two 
 by means of alignedAllocate. Don't call it naively! Only 
 applications that need such things (e.g. page-aligned, 
 cache-line-aligned, etc) should need those.

 * Those allocators must be able to deallocate memory allocated 
 with allocate() or alignedAllocate() with a single primitive 
 deallocate().


 Andrei


About alignedMalloc:

In C++ for example, I may want to use a vector full some SIMD 
type:

class alignas(16) Vec4 {
     union {
         struct { float x, y, z, w; };
         __m128 m;
     };
};

std::vector<Vec4> points = { ... };

In C++ however, 'new' does not respect over-alignment:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0035r2.html

Even if new respected alignment, there is no gauruntee all 
containers, STL or otherwise, would use 'new' as opposed to 
malloc by default(maybe one day?)

So I use a custom aligned allocator:

template <class T, int ALIGN>
class AlignedAllocator {
     T* allocate(size_type n) {
         return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
     }
};

SIMD operations(aligned load and store) can now safely be used on 
the contents of the std::vector<Vec4>.

std::vector knows nothing about the alignment of the memory it 
uses. It only knows to call allocate() of whatever allocator it's 
given. If I had an allocator with a function 'alignedAllocate' it 
wouldn't do any good. I believe this is the _correct_ design, and 
that a container _shouldn't_ have to know about where from, or 
what kind of memory it's getting.

Considering the above use case, alignedAllocate() is redundant, 
and possibly confusing.

About missing alignedDeallocate:

while aligned_alloc(), which works in combination with regular 
'free()', is supposed to be standard as of C++11, it's still not 
supported in visual studio 2015. Instead, one must use 
_aligned_malloc, and _aligned_free. Passing memory from 
_aligned_malloc to the regular version of free() causes a crash. 
Thus, different deallocation methods are needed for both. Also, 
there's homegrown aligned_allocate functions like the following, 
which require special deallocation functions because of the exta 
metadata prepended to the memory:
https://github.com/dlang/phobos/blob/366f6e4e66abe96bca9fd69d03042e08f787d040/std/experimental/allocator/mallocator.d#L134-L134

I suppose you could use aligned allocation for _all_ allocations, 
even allocations with default alignment, but that would add extra 
metadata(at least 8 bytes) to _all_ allocations even when its 
unnecessary.

So a solution could be to include the alignment as a template 
parameter of Mallocator, or provide an second 
AlignedMallocator(uint). The allocate() function of either option 
would return aligned memory if the 'alignment' template parameter 
was non-default. Then, the idea of memory alignment would be 
abstracted away from the containers themselves.

struct Mallocator(uint alignment = platformAlignment){}){}
or
struct AlignedMallocator(uint alignment = platformAlignment){}){}
Jan 21 2017
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/21/17 5:44 PM, bitwise wrote:

 About alignedMalloc:

 In C++ for example, I may want to use a vector full some SIMD type:

 class alignas(16) Vec4 {
     union {
         struct { float x, y, z, w; };
         __m128 m;
     };
 };

 std::vector<Vec4> points = { ... };

 In C++ however, 'new' does not respect over-alignment:
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0035r2.html

 Even if new respected alignment, there is no gauruntee all containers,
 STL or otherwise, would use 'new' as opposed to malloc by default(maybe
 one day?)

 So I use a custom aligned allocator:

 template <class T, int ALIGN>
 class AlignedAllocator {
     T* allocate(size_type n) {
         return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
     }
 };

 SIMD operations(aligned load and store) can now safely be used on the
 contents of the std::vector<Vec4>.

 std::vector knows nothing about the alignment of the memory it uses. It
 only knows to call allocate() of whatever allocator it's given. If I had
 an allocator with a function 'alignedAllocate' it wouldn't do any good.
 I believe this is the _correct_ design, and that a container _shouldn't_
 have to know about where from, or what kind of memory it's getting.


I understand. That's a questionable design. It only works by virtue of a 
long-distance convention between the rigged allocator and the element 
type of the vector.


 Considering the above use case, alignedAllocate() is redundant, and
 possibly confusing.


Well, you just made use of it in the rigged allocator.


 About missing alignedDeallocate:

 while aligned_alloc(), which works in combination with regular 'free()',
 is supposed to be standard as of C++11, it's still not supported in
 visual studio 2015. Instead, one must use _aligned_malloc, and
 _aligned_free. Passing memory from _aligned_malloc to the regular
 version of free() causes a crash. Thus, different deallocation methods
 are needed for both. Also, there's homegrown aligned_allocate functions
 like the following, which require special deallocation functions because
 of the exta metadata prepended to the memory:
 https://github.com/dlang/phobos/blob/366f6e4e66abe96bca9fd69d03042e08f787d040/std/experimental/allocator/mallocator.d#L134-L134


 I suppose you could use aligned allocation for _all_ allocations, even
 allocations with default alignment, but that would add extra metadata(at
 least 8 bytes) to _all_ allocations even when its unnecessary.

 So a solution could be to include the alignment as a template parameter
 of Mallocator, or provide an second AlignedMallocator(uint). The
 allocate() function of either option would return aligned memory if the
 'alignment' template parameter was non-default. Then, the idea of memory
 alignment would be abstracted away from the containers themselves.

 struct Mallocator(uint alignment = platformAlignment){}){}
 or
 struct AlignedMallocator(uint alignment = platformAlignment){}){}


It seems a matter of time until aligned_alloc gets implemented on Windows.


Andrei
Jan 21 2017
next sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 23:24:52 UTC, Andrei Alexandrescu 
wrote:

 On 1/21/17 5:44 PM, bitwise wrote:

 About alignedMalloc:

 In C++ for example, I may want to use a vector full some SIMD 
 type:

 class alignas(16) Vec4 {
     union {
         struct { float x, y, z, w; };
         __m128 m;
     };
 };

 std::vector<Vec4> points = { ... };

 In C++ however, 'new' does not respect over-alignment:
 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0035r2.html

 Even if new respected alignment, there is no gauruntee all 
 containers,
 STL or otherwise, would use 'new' as opposed to malloc by 
 default(maybe
 one day?)

 So I use a custom aligned allocator:

 template <class T, int ALIGN>
 class AlignedAllocator {
     T* allocate(size_type n) {
         return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
     }
 };

 SIMD operations(aligned load and store) can now safely be used 
 on the
 contents of the std::vector<Vec4>.

 std::vector knows nothing about the alignment of the memory it 
 uses. It
 only knows to call allocate() of whatever allocator it's 
 given. If I had
 an allocator with a function 'alignedAllocate' it wouldn't do 
 any good.
 I believe this is the _correct_ design, and that a container 
 _shouldn't_
 have to know about where from, or what kind of memory it's 
 getting.


 I understand. That's a questionable design. It only works by 
 virtue of a long-distance convention between the rigged 
 allocator and the element type of the vector.


I don't understand what's questionable about it. I don't see how 
abstracting the alignment away from the consumer of an allocator 
is a bad thing.


 Considering the above use case, alignedAllocate() is 
 redundant, and
 possibly confusing.


 Well, you just made use of it in the rigged allocator.


I made use of what I would expect to be a non-member helper 
function. I'm saying that I don't believe alignedAllocate() 
should be a part of the standard interface of an allocator, and 
that allocators should be specialized such that allocate() 
returns memory with whatever alignment is needed.



 About missing alignedDeallocate:

 while aligned_alloc(), which works in combination with regular 
 'free()',
 is supposed to be standard as of C++11, it's still not 
 supported in
 visual studio 2015. Instead, one must use _aligned_malloc, and
 _aligned_free. Passing memory from _aligned_malloc to the 
 regular
 version of free() causes a crash. Thus, different deallocation 
 methods
 are needed for both. Also, there's homegrown aligned_allocate 
 functions
 like the following, which require special deallocation 
 functions because
 of the exta metadata prepended to the memory:
 https://github.com/dlang/phobos/blob/366f6e4e66abe96bca9fd69d03042e08f787d040/std/experimental/allocator/mallocator.d#L134-L134


 I suppose you could use aligned allocation for _all_ 
 allocations, even
 allocations with default alignment, but that would add extra 
 metadata(at
 least 8 bytes) to _all_ allocations even when its unnecessary.

 So a solution could be to include the alignment as a template 
 parameter
 of Mallocator, or provide an second AlignedMallocator(uint). 
 The
 allocate() function of either option would return aligned 
 memory if the
 'alignment' template parameter was non-default. Then, the idea 
 of memory
 alignment would be abstracted away from the containers 
 themselves.

 struct Mallocator(uint alignment = platformAlignment){}){}
 or
 struct AlignedMallocator(uint alignment = 
 platformAlignment){}){}


 It seems a matter of time until aligned_alloc gets implemented 
 on Windows.


But how much time? Visual studio always lags behind in standards 
conformance.

Also, there is still the fact that some may need to use 
home-grown aligned allocation functions like the ones I linked 
above that prepend metadata to the memory returned, in which case 
they will need specialized deallocation functions.
Jan 21 2017
prev sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 23:24:52 UTC, Andrei Alexandrescu 
wrote:

 Andrei


Anyways, design opinions aside, I would be satisfied if an 
aligned allocator were included in std.allocators that provided 
aligned heap memory using allocate()/deallocate(). The rest is 
just details with no practical relevance for my immediate plans.

   Thanks
Jan 21 2017
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/21/17 10:07 PM, bitwise wrote:

 On Saturday, 21 January 2017 at 23:24:52 UTC, Andrei Alexandrescu wrote:

 Andrei


 Anyways, design opinions aside, I would be satisfied if an aligned
 allocator were included in std.allocators that provided aligned heap
 memory using allocate()/deallocate(). The rest is just details with no
 practical relevance for my immediate plans.

   Thanks


That should be easy and useful as an adapter over another allocator that 
supports alignedAllocate(). Please create an issue for that. Thanks! -- 
Andrei
Jan 22 2017
next sibling parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 22 January 2017 at 13:21:41 UTC, Andrei Alexandrescu 
wrote:

 On 1/21/17 10:07 PM, bitwise wrote:

 On Saturday, 21 January 2017 at 23:24:52 UTC, Andrei 
 Alexandrescu wrote:

 Andrei


 Anyways, design opinions aside, I would be satisfied if an 
 aligned
 allocator were included in std.allocators that provided 
 aligned heap
 memory using allocate()/deallocate(). The rest is just details 
 with no
 practical relevance for my immediate plans.

   Thanks


 That should be easy and useful as an adapter over another 
 allocator that supports alignedAllocate(). Please create an 
 issue for that. Thanks! -- Andrei


I would, but I'm not sure I can use these allocators.

Ran into this:
http://forum.dlang.org/post/akohcwiotlcrodemhexx forum.dlang.org

As stated in that thread, I can mark the member instance of the 
allocator in the container as 'shared', but then, if I use an 
allocator _without_ 'shared' methods, I get errors as well.

I'm not sure if there is a solution for this, but at this point, 
I try to just avoid 'shared' at all costs. The fact that 
Mallocator is thread safe should not stop me from using it in a 
non-thread safe class, but it does.
Jan 24 2017
parent bitwise <bitwise.pvt gmail.com> writes:
On Tuesday, 24 January 2017 at 19:38:31 UTC, bitwise wrote:

 I would, but I'm not sure I can use these allocators.

 Ran into this:
 http://forum.dlang.org/post/akohcwiotlcrodemhexx forum.dlang.org

 As stated in that thread, I can mark the member instance of the 
 allocator in the container as 'shared', but then, if I use an 
 allocator _without_ 'shared' methods, I get errors as well.

 I'm not sure if there is a solution for this, but at this 
 point, I try to just avoid 'shared' at all costs. The fact that 
 Mallocator is thread safe should not stop me from using it in a 
 non-thread safe class, but it does.


So I guess there is no solution for this?

I've written my own allocators at this point, which is not 
something I did with great joy.
Jan 25 2017
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 22 January 2017 at 13:21:41 UTC, Andrei Alexandrescu 
wrote:

 -- Andrei


After exploring this idea( safe containers), then looking at 
std.Array again, it's actually pretty smart. I had concerns about 
efficiency, but RefCounted seems to force allocation like 
make_shared<>, eliminating the access overhead I thought was 
present. It doesn't look like RefCounted will work once 
allocators are introduced, but you seem to already be working on 
generalized ref-counted buffers. Well played, sir.

I decided to make my own containers classes, in the hopes that 
one day, I can just slap  rc in front of them and be done with 
the GC.
Jan 26 2017
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Sat, 21 Jan 2017 22:44:49 +0000, bitwise wrote:

 So I use a custom aligned allocator:
 
 template <class T, int ALIGN>
 class AlignedAllocator {
      T* allocate(size_type n) {
          return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
      }
 };
 
 SIMD operations(aligned load and store) can now safely be used on the
 contents of the std::vector<Vec4>.


This is fine when you have fine-grained control of which allocators are 
used for which purpose and you know more about what the code needs than 
whoever is consuming the allocator.

alignedAllocate() is for when the code consuming the allocator knows more 
about alignment needs than whoever is supplying the allocator, which is 
more likely to crop up with coarse-grained allocator control.
Jan 21 2017
parent reply bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 22 January 2017 at 00:13:10 UTC, Chris Wright wrote:

 On Sat, 21 Jan 2017 22:44:49 +0000, bitwise wrote:

 So I use a custom aligned allocator:
 
 template <class T, int ALIGN>
 class AlignedAllocator {
      T* allocate(size_type n) {
          return (T*)_aligned_malloc(ALIGN, n * sizeof(T));
      }
 };
 
 SIMD operations(aligned load and store) can now safely be used 
 on the contents of the std::vector<Vec4>.



 alignedAllocate() is for when the code consuming the allocator 
 knows more about alignment needs than whoever is supplying the 
 allocator, which is more likely to crop up with coarse-grained 
 allocator control.


I understand this, but I've given a solid example which supports 
the former, and haven't seen a counter example which supports the 
latter.
Jan 21 2017
parent reply Chris Wright <dhasenan gmail.com> writes:
On Sun, 22 Jan 2017 00:53:45 +0000, bitwise wrote:

 I understand this, but I've given a solid example which supports the
 former, and haven't seen a counter example which supports the latter.


Because examples that are easy to generate are small and small examples 
don't have coarse-grained anything.
Jan 21 2017
parent bitwise <bitwise.pvt gmail.com> writes:
On Sunday, 22 January 2017 at 01:02:09 UTC, Chris Wright wrote:

 On Sun, 22 Jan 2017 00:53:45 +0000, bitwise wrote:

 I understand this, but I've given a solid example which 
 supports the former, and haven't seen a counter example which 
 supports the latter.


 Because examples that are easy to generate are small and small 
 examples don't have coarse-grained anything.


Please explain
Jan 21 2017
prev sibling parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:

 When std.experimental.allocator was created, I was under the 
 impression it was meant to be used to make containers. But 
 since allocate() returns void[], casts are required before 
 using that memory, which is unsafe.

 Is my only option to wrap the casts in an  trusted helper 
 function? or am I missing something?

  Thanks


There's no need for casts. You can just use the high-level 
wrappers:
Jan 21 2017
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Saturday, 21 January 2017 at 09:56:29 UTC, ZombineDev wrote:

 On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:

 When std.experimental.allocator was created, I was under the 
 impression it was meant to be used to make containers. But 
 since allocate() returns void[], casts are required before 
 using that memory, which is unsafe.

 Is my only option to wrap the casts in an  trusted helper 
 function? or am I missing something?

  Thanks


 There's no need for casts. You can just use the high-level 
 wrappers:











Though, since you are implementing a container that manually 
manages memory, you won't be able to get away with no  trusted 
parts in your code. 

help, but you'll still need to call dispose/deallocate somewhere yourself and
you're the only that knows when is safe to do this.
If you look at how Rust's standard library is implemented, you'll 
see the same pattern. The users have a safe API, but underneath 
you'll that the library uses many 'unsafe' blocks where the 
compiler doesn't see the full picture which only the author of 
the code knows.
Jan 21 2017
parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 21 January 2017 at 10:15:46 UTC, ZombineDev wrote:

 On Saturday, 21 January 2017 at 09:56:29 UTC, ZombineDev wrote:

 On Saturday, 21 January 2017 at 02:47:49 UTC, bitwise wrote:

 When std.experimental.allocator was created, I was under the 
 impression it was meant to be used to make containers. But 
 since allocate() returns void[], casts are required before 
 using that memory, which is unsafe.

 Is my only option to wrap the casts in an  trusted helper 
 function? or am I missing something?

  Thanks


 There's no need for casts. You can just use the high-level 
 wrappers:











 Though, since you are implementing a container that manually 
 manages memory, you won't be able to get away with no  trusted 
 parts in your code. 

help, but you'll still need to call dispose/deallocate somewhere yourself and
you're the only that knows when is safe to do this.
 If you look at how Rust's standard library is implemented, 
 you'll see the same pattern. The users have a safe API, but 
 underneath you'll that the library uses many 'unsafe' blocks 
 where the compiler doesn't see the full picture which only the 
 author of the code knows.


I guess you're right. This is my first time looking at  safe in 
detail. I figured something like malloc() could  trusted, but I 
suppose free cannot.

I pictured being able to use an  safe allocator with an  safe 
container. How else could one reliably create  safe containers, 
or any  safe object for that matter, that allows the substitution 
of arbitrary and possibly unsafe components?

I guess the answer is responsible programming, but that seems 
counter intuitive given the goals of  safe.
Jan 21 2017