www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Shared, ref, arrays, and reserve template instantiation

reply James Blachly <james.blachly gmail.com> writes:
When I add the "shared" attribute to an array, I am no longer 
able to call reserve because the template won't instantiate:

Error: template object.reserve cannot deduce function from 
argument types !()(shared(int[]), int), candidates are:
/dlang/dmd/linux/bin64/../../src/druntime/import/object.d(4091):  
       object.reserve(T)(ref T[] arr, size_t newcapacity)

1. Shared modifies the type, so the template does not match. Even 
casting does not seem to work however. Is there something about 
shared that makes it unable to be taken by reference?
2. Is there a workaround for me to be able to preallocate the 
array?

Kind regards
Sep 12 2018
next sibling parent Neia Neutuladh <neia ikeran.org> writes:
On Wednesday, 12 September 2018 at 23:41:16 UTC, James Blachly 
wrote:

 When I add the "shared" attribute to an array, I am no longer 
 able to call reserve because the template won't instantiate:

 Error: template object.reserve cannot deduce function from 
 argument types !()(shared(int[]), int), candidates are:
 /dlang/dmd/linux/bin64/../../src/druntime/import/object.d(4091):
       object.reserve(T)(ref T[] arr, size_t newcapacity)

 1. Shared modifies the type, so the template does not match. 
 Even casting does not seem to work however. Is there something 
 about shared that makes it unable to be taken by reference?
 2. Is there a workaround for me to be able to preallocate the 
 array?

 Kind regards


I'm guessing you tried something like:

     shared char[] x;
     // Doesn't work; it casts the result of x.reserve
     cast(char[])x.reserve(100);
     // Doesn't work; (cast(char[])x) is not an lvalue, so it 
can't be ref
     (cast(char[])x).reserve(100);

Arrays are passed and stored like pointers, and `reserve` 
modifies the array, which is why the thing needs to be ref.

Anyway, it works like this:

     // Cast and store in a variable so it can be ref
     auto b = cast(char[]) x;
     // Okay, reallocates b (changes b.ptr), doesn't change x
     b.reserve(100);
     // Copy changes back to the shared variable
     x = cast(shared) b;
Sep 12 2018
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, September 12, 2018 5:41:16 PM MDT James Blachly via 
Digitalmars-d-learn wrote:

 When I add the "shared" attribute to an array, I am no longer
 able to call reserve because the template won't instantiate:

 Error: template object.reserve cannot deduce function from
 argument types !()(shared(int[]), int), candidates are:
 /dlang/dmd/linux/bin64/../../src/druntime/import/object.d(4091):
        object.reserve(T)(ref T[] arr, size_t newcapacity)

 1. Shared modifies the type, so the template does not match. Even
 casting does not seem to work however. Is there something about
 shared that makes it unable to be taken by reference?
 2. Is there a workaround for me to be able to preallocate the
 array?


You can't do much of anything with shared while it's shared, which is pretty
much the whole point. The way that shared needs to be used in general is
essentially

synchronized(mutexForSharedObj)
{
    auto local = cast(Type)sharedObj;
    // do stuff with local...

    // ensure that no thread-local references to local / sharedObj exist
    // before releasing the mutex
}

// shared object is now essentially unusable again

Doing pretty much _any_ operation on a shared object while it's shared
(other than atomic operations from core.atomic) is wrong, because it's not
thread-safe. The compiler prevents most operations but not as many as it
should (e.g. copying is currently legal). That will likely be fixed in the
future, but exactly what's going to happen to shared in all of the fine
details hasn't been sorted out yet. The basics work, but not all of the
details are as they should be yet.

So, if you're doing anything like calling reserve or ~= on a shared array,
then you need to protect it with the mutex that you have for it and cast
away shared first. However, if you're just dealing with constructing the
array, then what you should do is create it as thread-local and then cast it
to shared after you're done setting it up and are ready to share it across
threads (after which, all further operations on it should be protected by a
mutex or use atomics, otherwise they're not thread-safe).

- Jonathan M Davis
Sep 12 2018
parent reply James Blachly <james.blachly gmail.com> writes:
Great -- Thank you both.

I previously found Unqual, but it looks like that needs template 
support so wasn't feasible, hence my question.

Neia is right that I tried to cast as in the second case ( but 
without UFCS -- reserve( cast(int[]), N); ).  As an aside, what 
is going on behind the scenes with the compiler when casting away 
a property? I did not think cast operations copied data, so was 
surprised that a cast value is not an lvalue.

Regarding Jonathan's comments, we had definitely protected the ~= 
operations with Mutex, but realized we were doing lots of array 
appends in a hot loop, and since we have an idea of cardinality 
ahead of time just wanted to preallocate.  Since it is all 
initialization before concurrent code enters the picture, we'll 
do what you've suggested and set it up as TL and then cast to 
shared.

James
Sep 12 2018
parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, September 12, 2018 9:42:19 PM MDT James Blachly via 
Digitalmars-d-learn wrote:

 Neia is right that I tried to cast as in the second case ( but
 without UFCS -- reserve( cast(int[]), N); ).  As an aside, what
 is going on behind the scenes with the compiler when casting away
 a property? I did not think cast operations copied data, so was
 surprised that a cast value is not an lvalue.


Well, you basically get a temporary variable when you cast an object, and
those are rvalues. And while casting with regards to type qualifiers such as
const or shared, you're not actually changing the data, plenty of other
casts do - e.g. float and long don't even have the same size, but you can
cast from one to the other. So, even in principle, only some casts could
result in lvalues even if we wanted them to.

- Jonathan M Davis
Sep 12 2018