www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Pointer across threads

reply "Chris" <wendlec tcd.ie> writes:
The following

struct DATA {
  short* data;
  size_t len;
}

// data and len are provided by a C function
// ...
auto data = mymodule.getSpeechData();
// cast to immutable, because of concurrency
immutable short* tmp = cast(immutable)(data.data);
auto proc = spawn(&processData, thisTid);
send(processData, tmp, data.len);

However, processData never receives "tmp". Is this because tmp 
and data.data are still pointing to the same address? If this is 
the case, why doesn't the compiler warn me? Or is it something 
else (pointer not visible across threads?).

Is there a work around? (I wanted to avoid having to convert 
short* to short[] and .idup it.)
Nov 04 2014
next sibling parent "Chris" <wendlec tcd.ie> writes:
On Tuesday, 4 November 2014 at 12:47:11 UTC, Chris wrote:

 The following

 struct DATA {
  short* data;
  size_t len;
 }

 // data and len are provided by a C function
 // ...
 auto data = mymodule.getSpeechData();
 // cast to immutable, because of concurrency
 immutable short* tmp = cast(immutable)(data.data);
 auto proc = spawn(&processData, thisTid);
 send(processData, tmp, data.len);

 However, processData never receives "tmp". Is this because tmp 
 and data.data are still pointing to the same address? If this 
 is the case, why doesn't the compiler warn me? Or is it 
 something else (pointer not visible across threads?).

 Is there a work around? (I wanted to avoid having to convert 
 short* to short[] and .idup it.)


Sorry, it should read

send(proc, tmp, data.len);
Nov 04 2014
prev sibling next sibling parent reply "anonymous" <anonymous example.com> writes:
On Tuesday, 4 November 2014 at 12:47:11 UTC, Chris wrote:

 The following

 struct DATA {
  short* data;
  size_t len;
 }

 // data and len are provided by a C function
 // ...
 auto data = mymodule.getSpeechData();
 // cast to immutable, because of concurrency
 immutable short* tmp = cast(immutable)(data.data);
 auto proc = spawn(&processData, thisTid);
 send(processData, tmp, data.len);

 However, processData never receives "tmp". Is this because tmp 
 and data.data are still pointing to the same address? If this 
 is the case, why doesn't the compiler warn me? Or is it 
 something else (pointer not visible across threads?).

 Is there a work around? (I wanted to avoid having to convert 
 short* to short[] and .idup it.)


Please provide complete test cases. It makes it way easier for
others to help.

One thing I noticed is that receiving immutable(short*) doesn't
work. Instead, it has to be immutable(short)* on the receiving
end. This is because immutable(T*) is automatically converted to
immutable(T)* on function calls. And apparently receive's
matching mechanism is inconveniently literal.
Nov 04 2014
parent reply "Chris" <wendlec tcd.ie> writes:
On Tuesday, 4 November 2014 at 14:01:16 UTC, anonymous wrote:

 On Tuesday, 4 November 2014 at 12:47:11 UTC, Chris wrote:

 The following

 struct DATA {
 short* data;
 size_t len;
 }

 // data and len are provided by a C function
 // ...
 auto data = mymodule.getSpeechData();
 // cast to immutable, because of concurrency
 immutable short* tmp = cast(immutable)(data.data);
 auto proc = spawn(&processData, thisTid);
 send(processData, tmp, data.len);

 However, processData never receives "tmp". Is this because tmp 
 and data.data are still pointing to the same address? If this 
 is the case, why doesn't the compiler warn me? Or is it 
 something else (pointer not visible across threads?).

 Is there a work around? (I wanted to avoid having to convert 
 short* to short[] and .idup it.)


 Please provide complete test cases. It makes it way easier for
 others to help.

 One thing I noticed is that receiving immutable(short*) doesn't
 work. Instead, it has to be immutable(short)* on the receiving
 end. This is because immutable(T*) is automatically converted to
 immutable(T)* on function calls. And apparently receive's
 matching mechanism is inconveniently literal.


Ah, thanks a lot, that was it! It has to be immutable(short)* on 
the receiving end, now it works.

receive (
   (immutable(short)* data, size_t len) {
    //...
});

It is indeed inconveniently literal, but I guess there's a reason 
for it.

I'm still curious, though, how D handles this internally, because 
data.data is still mutable while the other reference to the same 
address (tmp) is not. What if I change data.data while the other 
thread is being executed?


PS In this case, it was hard to provide a complete test case, 
because there are things going on in a C function and other D 
modules.
Nov 04 2014
next sibling parent "Chris" <wendlec tcd.ie> writes:
On Tuesday, 4 November 2014 at 14:36:19 UTC, Chris wrote:

 On Tuesday, 4 November 2014 at 14:01:16 UTC, anonymous wrote:

 On Tuesday, 4 November 2014 at 12:47:11 UTC, Chris wrote:

 The following

 struct DATA {
 short* data;
 size_t len;
 }

 // data and len are provided by a C function
 // ...
 auto data = mymodule.getSpeechData();
 // cast to immutable, because of concurrency
 immutable short* tmp = cast(immutable)(data.data);
 auto proc = spawn(&processData, thisTid);
 send(processData, tmp, data.len);

 However, processData never receives "tmp". Is this because 
 tmp and data.data are still pointing to the same address? If 
 this is the case, why doesn't the compiler warn me? Or is it 
 something else (pointer not visible across threads?).

 Is there a work around? (I wanted to avoid having to convert 
 short* to short[] and .idup it.)


 Please provide complete test cases. It makes it way easier for
 others to help.

 One thing I noticed is that receiving immutable(short*) doesn't
 work. Instead, it has to be immutable(short)* on the receiving
 end. This is because immutable(T*) is automatically converted 
 to
 immutable(T)* on function calls. And apparently receive's
 matching mechanism is inconveniently literal.


 Ah, thanks a lot, that was it! It has to be immutable(short)* 
 on the receiving end, now it works.

 receive (
   (immutable(short)* data, size_t len) {
    //...
 });

 It is indeed inconveniently literal, but I guess there's a 
 reason for it.

 I'm still curious, though, how D handles this internally, 
 because data.data is still mutable while the other reference to 
 the same address (tmp) is not. What if I change data.data while 
 the other thread is being executed?


Just tested it.

writeln(data.data[0]);
data.data[0] = -11;
send(play, tmp, data.len);
writeln(tmp[0]);
//
receive (
    (immutable(short)* data, size_t len) {
     writeln(data[0];
});

prints
0     // original value
-11   // tmp
-11  // data in the other thread

Is this behavior intended?
Nov 04 2014
prev sibling next sibling parent reply "anonymous" <anonymous example.com> writes:
On Tuesday, 4 November 2014 at 14:36:19 UTC, Chris wrote:

 I'm still curious, though, how D handles this internally, 
 because data.data is still mutable while the other reference to 
 the same address (tmp) is not. What if I change data.data while 
 the other thread is being executed?


Changing *data.data would be undefined behaviour. Don't do it.
Once data is typed as immutable, it must not change anymore. By
casting you're side-stepping the type system, so that you have to
make sure of such things yourself.
Nov 04 2014
parent "Chris" <wendlec tcd.ie> writes:
On Tuesday, 4 November 2014 at 14:47:49 UTC, anonymous wrote:

 On Tuesday, 4 November 2014 at 14:36:19 UTC, Chris wrote:

 I'm still curious, though, how D handles this internally, 
 because data.data is still mutable while the other reference 
 to the same address (tmp) is not. What if I change data.data 
 while the other thread is being executed?


 Changing *data.data would be undefined behaviour. Don't do it.
 Once data is typed as immutable, it must not change anymore. By
 casting you're side-stepping the type system, so that you have 
 to
 make sure of such things yourself.


Hm. I'm not planning to change data.data, of course, but I was 
worried (and curious) about potential safety issues. I suppose I 
could convert short* to short[] > idup it > pass a reference to 
the C function (that expects short*). However, I want to avoid 
this, because there shouldn't be any additional operations (it's 
a (near) real time system).
Nov 04 2014
prev sibling parent reply "thedeemon" <dlang thedeemon.com> writes:
On Tuesday, 4 November 2014 at 14:36:19 UTC, Chris wrote:

 I'm still curious, though, how D handles this internally, 
 because data.data is still mutable while the other reference to 
 the same address (tmp) is not. What if I change data.data while 
 the other thread is being executed?


"immutable" is part of the static type system, it's a label that 
only exists and makes sense at compile time, for compiler and the 
programmer. Casting a mutable data pointer to immutable data 
pointer is a no-op, just a copy of pointer. Address stays the 
same, data stays the same. So if you mutate the data it will lead 
to "immutable" data being changed just because it's not really 
immutable, you've just fooled yourself when doing the cast.
Nov 04 2014
parent "Chris" <wendlec tcd.ie> writes:
On Tuesday, 4 November 2014 at 16:07:11 UTC, thedeemon wrote:

 On Tuesday, 4 November 2014 at 14:36:19 UTC, Chris wrote:

 I'm still curious, though, how D handles this internally, 
 because data.data is still mutable while the other reference 
 to the same address (tmp) is not. What if I change data.data 
 while the other thread is being executed?


 "immutable" is part of the static type system, it's a label 
 that only exists and makes sense at compile time, for compiler 
 and the programmer. Casting a mutable data pointer to immutable 
 data pointer is a no-op, just a copy of pointer. Address stays 
 the same, data stays the same. So if you mutate the data it 
 will lead to "immutable" data being changed just because it's 
 not really immutable, you've just fooled yourself when doing 
 the cast.


That's what I was thinking too.
Nov 04 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
There is assumeUnique which effectively does cast to immutable 
but is more clear to the reader in a sense why breaking type 
system was considered legit. I recommend using it over raw cast.
Nov 04 2014