www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - concurrency problem with pointers

reply "evilrat" <evilrat666 gmail.com> writes:
i hate asking for help, but... now i'm pushed in the corner. so 
here is my problem:
i have struct with handle, and i need to send this struct or 
handle to second thread, all fine but feeding this struct or 
handle to send results in multiple errors(mostly complains about 
thread local data). and unfortunatelly shared and __gshared 
doesn't helps(complains about unknown size of handle).

well, i wish i could describe in details without code but... here 
is simplified example
--------

void main() {
   MyData data;
   // load data
   ...

   MyWindow wnd(data);
   wnd.show();
   auto tid = spawn(&newViewThread);
   send(tid, wnd);
}

// struct because i want benefit from RAII, but if necessary 
could switch to class
struct MyWindow {
   this ( MyData data ) { ... }
   // WINDOW defined as struct but has no size
   WINDOW* handle;
}

// 3rd party func to get window
WINDOW* createWindow(...);

----------
shortly speaking, WINDOW is pointer to window in C library so 
it's "shared", and i need it in another thread to make opengl 
context current in that thread, but compiler simply doesn't allow 
me to do anything with that, and i can't change definition since 
it's 3rd party bindings. any suggestions?
Jul 18 2013
next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/18/2013 08:29 AM, evilrat wrote:


 i hate asking for help


Please do so. Otherwise we can't learn from others' issues. :)


 well, i wish i could describe in details without code


Can you show it with a short program without using platform specific 
definitions?

Ali
Jul 18 2013
parent reply "evilrat" <evilrat666 gmail.com> writes:
On Thursday, 18 July 2013 at 17:23:20 UTC, Ali Çehreli wrote:

 On 07/18/2013 08:29 AM, evilrat wrote:


 i hate asking for help


 Please do so. Otherwise we can't learn from others' issues. :)


that's who i am. long story, but in short, people hated when i 
ask something and after few minutes i find solution. i bother 
everyone ;(


 well, i wish i could describe in details without code


 Can you show it with a short program without using platform 
 specific definitions?


here is reduced example without platform-specific stuff,  
requires derelict3 and glfw.
http://pastebin.com/rhB14YNs
Jul 19 2013
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Friday, 19 July 2013 at 15:16:23 UTC, evilrat wrote:

 On Thursday, 18 July 2013 at 17:23:20 UTC, Ali Çehreli wrote:

 On 07/18/2013 08:29 AM, evilrat wrote:


 i hate asking for help


 Please do so. Otherwise we can't learn from others' issues. :)


 that's who i am. long story, but in short, people hated when i 
 ask something and after few minutes i find solution. i bother 
 everyone ;(


 well, i wish i could describe in details without code


 Can you show it with a short program without using platform 
 specific definitions?


 here is reduced example without platform-specific stuff,  
 requires derelict3 and glfw.
 http://pastebin.com/rhB14YNs


I can't get that to compile, I keep getting 
Derelict3/import/derelict/glfw3/types.d(290): Error: struct 
derelict.glfw3.types.GLFWwindow unknown size

I suspect I'm compiling wrong somehow.
Jul 19 2013
parent "evilrat" <evilrat666 gmail.com> writes:
On Friday, 19 July 2013 at 15:35:27 UTC, John Colvin wrote:

 On Friday, 19 July 2013 at 15:16:23 UTC, evilrat wrote:

 here is reduced example without platform-specific stuff,  
 requires derelict3 and glfw.
 http://pastebin.com/rhB14YNs


 I can't get that to compile, I keep getting 
 Derelict3/import/derelict/glfw3/types.d(290): Error: struct 
 derelict.glfw3.types.GLFWwindow unknown size

 I suspect I'm compiling wrong somehow.


no you don't. this is because of the definition of GLFWwindow in 
derelict3 (it's really predefined as struct, not void*)
Jul 19 2013
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/19/2013 08:16 AM, evilrat wrote:


 here is reduced example without platform-specific stuff, requires
 derelict3 and glfw.
 http://pastebin.com/rhB14YNs


Further reduced by removing the need for derelict3 and glfw:

import std.stdio;
import std.concurrency;
import core.thread;

alias GLFWwindow = void;

struct Window {
     void create() {
         auto tid = spawn(&runNewView);
         send(tid, _wnd );        // <-- ERROR

         thread_joinAll();
     }

     GLFWwindow* _wnd;
}

void runNewView() {
     while ( true ) {
         receive (
             (string msg) { writeln(msg); },
             (const(GLFWwindow*) window) { }
                  );
     }
}

void main() {
     auto window = Window();
     // runs main loop
     window.create();
}

Error: static assert  "Aliases to mutable thread-local data not allowed."
        instantiated from here: send!(void*)

Is that the problem? (If so, why don't you say so? ;))

Then there are two solutions:

a) Make _wnd a shared(GLFWwindow):

struct Window {
     // ...

     shared(GLFWwindow)* _wnd;
}

The code now compiles.

b) As Sean Kelly said, cast to shared before sending:

         send(tid, cast(shared(GLFWwindow)*)_wnd );

With the b option you may need to change the definition of the receiving 
delegate to shared as well:

             (const(shared(GLFWwindow)*) window) { }

But that change is not needed in that simple code.

Ali

P.S. Additionally, the const on the receiving side probably should apply 
only to what is being pointed at (as opposed to the pointer itself):

             (const(GLFWwindow)* window) { }
Jul 19 2013
parent reply "evilrat" <evilrat666 gmail.com> writes:
On Friday, 19 July 2013 at 15:36:33 UTC, Ali Çehreli wrote:

 Error: static assert  "Aliases to mutable thread-local data not 
 allowed."
        instantiated from here: send!(void*)

 Is that the problem? (If so, why don't you say so? ;))

 Then there are two solutions:

 a) Make _wnd a shared(GLFWwindow):

 struct Window {
     // ...

     shared(GLFWwindow)* _wnd;
 }

 The code now compiles.

 b) As Sean Kelly said, cast to shared before sending:

         send(tid, cast(shared(GLFWwindow)*)_wnd );

 With the b option you may need to change the definition of the 
 receiving delegate to shared as well:

             (const(shared(GLFWwindow)*) window) { }

 But that change is not needed in that simple code.


unfortunately both variants fails for me, i think this is because 
GLFWwindow is (pre)defined as struct instead alias void :(

maybe there is something to override type definition without 
modifying 3rd party libraries code?

/Users/evilrat/Documents/prog/Derelict3/import/derelict/glfw3/types.d(290): 
Error: struct derelict.glfw3.types.GLFWwindow unknown size
/Users/evilrat/Documents/prog/Derelict3/import/derelict/glfw3/types.d(290): 
Error: struct derelict.glfw3.types.GLFWwindow no size yet for 
forward reference

anyway thanks to both authors, now i could avoid such problems in 
the future \0/

 P.S. Additionally, the const on the receiving side probably 
 should apply only to what is being pointed at (as opposed to 
 the pointer itself):

             (const(GLFWwindow)* window) { }


ah yes, such a shame. i'm sometimes forgot about this little 
detail :(
Jul 19 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/19/2013 09:00 AM, evilrat wrote:


 i think this is because GLFWwindow is (pre)defined as struct instead 


alias

 void :(


Further reduced:

import std.concurrency;

struct S;

void main()
{
     receive((S * p){});
}

Error: struct deneme.S is forward referenced when looking for 'toHash'
Error: struct deneme.S is forward referenced when looking for 'opCmp'
Error: struct deneme.S is forward referenced when looking for 'toString'
Error: struct deneme.S unknown size
Error: struct deneme.S no size yet for forward reference
Error: struct deneme.S unknown size
Error: struct deneme.S no size yet for forward reference

Ali
Jul 19 2013
parent reply "evilrat" <evilrat666 gmail.com> writes:
On Friday, 19 July 2013 at 17:14:10 UTC, Ali Çehreli wrote:

 On 07/19/2013 09:00 AM, evilrat wrote:


 i think this is because GLFWwindow is (pre)defined as struct


 instead alias

 void :(


 Further reduced:

 import std.concurrency;

 struct S;

 void main()
 {
     receive((S * p){});
 }
Error....
 Ali


i don't think this is compiler bug, it's just how it works. yet 
this is another thing one must remember when writing library - 
never use struct predefenitions(or any other forward refs).
so concluding all of this, it is clear this is a derelict library 
bug, and that we also need something to allow rewrite defenitions 
at least at module level, if nothing like this already exists.
Jul 19 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/19/2013 11:07 AM, evilrat wrote:


 i don't think this is compiler bug, it's just how it works.


But what code needs toHash and others for the struct type even though we 
are just passing a pointer around?


 yet this is
 another thing one must remember when writing library - never use struct
 predefenitions(or any other forward refs).
 so concluding all of this, it is clear this is a derelict library bug,


I am not convinced yet. I think it is something that std.concurrency is 
doing (perhaps because it relies on Variant?). Otherwise there is no 
problem with what derelict does. The following code compiles:

struct S;

void myReceive(void delegate (S *) func)
{
     S * p;
     func(p);
}

void main()
{
     myReceive((S * p){});
}

Ali
Jul 19 2013
parent reply "evilrat" <evilrat666 gmail.com> writes:
On Friday, 19 July 2013 at 18:25:26 UTC, Ali Çehreli wrote:

 On 07/19/2013 11:07 AM, evilrat wrote:


 i don't think this is compiler bug, it's just how it works.


 But what code needs toHash and others for the struct type even 
 though we are just passing a pointer around?


 yet this is
 another thing one must remember when writing library - never


 use struct

 predefenitions(or any other forward refs).
 so concluding all of this, it is clear this is a derelict


 library bug,

 I am not convinced yet. I think it is something that 
 std.concurrency is doing (perhaps because it relies on 
 Variant?). Otherwise there is no problem with what derelict 
 does. The following code compiles:


maybe you are right about Variant, and yet it still not compiler 
bug, but phobos bug. rewriting std.variant and std.concurrency 
just because of forward refs is no go.
i think problem is that with a struct we can't just generate code 
for this methods(toHash,opCmp,etc) by comparing its address, even 
if it's not extern(C/C++) and not found by linker, or do we 
actually could?
Jul 19 2013
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 07/19/2013 11:39 AM, evilrat wrote:


 i think problem is that with a struct we can't just generate code for
 this methods(toHash,opCmp,etc) by comparing its address


That would be wrong. Yes the compiler needs the definition of the struct 
to generate toHash, opCmp, and toString but note that there is no 
expression that needs S in that sample code.

The code uses a pointer to S, which happens to have trivial toHash, 
opCmp, and toString.

I think now I see where the problem is coming from. Note that the three 
"compiles" lines below do exercise toHash, opCmp, and toString on S*.

import std.stdio;

struct S;

void main()
{
     S* p0, p1;

     int[S*] arr;
     arr[p0] = 0;               // compiles
     writefln("%s", p0);        // compiles
     writeln(p0 < p1);          // compiles

     writeln(p0.toString());    // Error: struct S is forward referenced
}

However, calling toString() directly on the pointer naturally forwards 
the call to S and that requires the complete definition of the struct.

This seems to be a problem with any template that has to work with 
pointers. The following template cannot work with a pointer:

import std.stdio;

struct S;

void foo(T)(T val)
{
     writeln("%s", val.toString());    // ERROR
}

void main()
{
     S * p;
     foo(p);
}

So, the solution is to do something like the following:

import std.stdio;

struct S;

void foo(T)(T val)
{
     static if (is (T : U*, U)) {
         writefln("(%s)%s", T.stringof, val);

     } else {
         writefln("%s", val.toString());
     }
}

void main()
{
     S * p;
     foo(p);
}

And I think the following code in std/concurrency.d is the reason:

               ( Variant val )
// ...
                   throw new MessageMismatch(
                       format("Unexpected message type: expected '%s', 
got '%s'",
                           exp, val.type.toString()));
               } );

val.type returns a TypeInfo and it seems like there is no TypeInfo 
special for pointers to incomplete types. Shouldn't there be?

Ali
Jul 19 2013
parent "evilrat" <evilrat666 gmail.com> writes:
On Friday, 19 July 2013 at 19:05:27 UTC, Ali Çehreli wrote:

 This seems to be a problem with any template that has to work 
 with pointers. The following template cannot work with a 
 pointer:
 ...

 val.type returns a TypeInfo and it seems like there is no 
 TypeInfo special for pointers to incomplete types. Shouldn't 
 there be?

 Ali


maybe it should. but it doesn't matters what i think about how it 
should be, i just hope D would be user friendly, and current 
situation with that making it quite not so friendly.
Jul 20 2013
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Jul 18, 2013, at 8:29 AM, evilrat <evilrat666 gmail.com> wrote:

=20
 shortly speaking, WINDOW is pointer to window in C library so it's =


"shared", and i need it in another thread to make opengl context current =
in that thread, but compiler simply doesn't allow me to do anything with =
that, and i can't change definition since it's 3rd party bindings. any =
suggestions?

In short, you're trying to send a pointer via std.concurrency.  The =
easiest way is to cast it to/from shared.  Otherwise, you'd need to make =
it an opaque type like a size_t rather than a type D can tell is a =
reference.=
Jul 18 2013
parent "evilrat" <evilrat666 gmail.com> writes:
On Thursday, 18 July 2013 at 23:36:34 UTC, Sean Kelly wrote:

 In short, you're trying to send a pointer via std.concurrency.  
 The easiest way is to cast it to/from shared.  Otherwise, you'd 
 need to make it an opaque type like a size_t rather than a type 
 D can tell is a reference.


sure cast to size_t and back works, but can shoot in the leg 
later anytime :(
Jul 19 2013