www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Shared an non-shared

reply Begah <mathieu.roux222 gmail.com> writes:
I seem to be missing something.
It seems that if you want to create a shared object of a 
structure ( or class ), then I have to copy every functions and 
add "shared" to it. This seems way more work than it should.
For example why can't this simply work :

   class Image {
	string name;
	int contents;
	
	this(string name, int contents) {
		this.name = name;
		this.contents = contents;
	}
	
	void printContents() {
		writeln(name, " : ", contents);
	}
	
	void changeContents(int newContents) {
		this.contents = newContents;
	}
   }

   void main()
   {
	Image im = new Image("Test", 15);
	im.printContents();
	im.changeContents(45);
	im.printContents();
	
	shared Image imShared = new shared Image("Test2", 80);
	imShared.printContents();
	imShared.changeContents(6);
	imShared.printContents();
   }

How can I make a method that accepts being called by both a 
shared and non-shared object, to prevent having to copy methods 
and adding a "shared"?

I am not looking for an explanation for how to handle 
multi-threading ( synchronization and so on ). I am looking to 
use pre-coded classes and structures ( without using __gshared ) 
with non-shared and shared objects.
Oct 04 2016
next sibling parent ag0aep6g <anonymous example.com> writes:
On 10/04/2016 09:22 PM, Begah wrote:

 I seem to be missing something.
 It seems that if you want to create a shared object of a structure ( or
 class ), then I have to copy every functions and add "shared" to it.
 This seems way more work than it should.
 For example why can't this simply work :

   class Image {
     string name;
     int contents;

     this(string name, int contents) {
         this.name = name;
         this.contents = contents;
     }

     void printContents() {
         writeln(name, " : ", contents);
     }

     void changeContents(int newContents) {
         this.contents = newContents;
     }
   }

   void main()
   {
     Image im = new Image("Test", 15);
     im.printContents();
     im.changeContents(45);
     im.printContents();

     shared Image imShared = new shared Image("Test2", 80);
     imShared.printContents();
     imShared.changeContents(6);
     imShared.printContents();
   }


Non-`shared` methods are not required to be somehow thread-safe at all. 
So they can't be allowed to be called on `shared` objects.

The compiler can't automatically ensure thread-safety, because it can't 
know what can run in parallel and what can't.


 How can I make a method that accepts being called by both a shared and
 non-shared object, to prevent having to copy methods and adding a "shared"?


I would have thought that you can call a shared method on an unshared 
object. Apparently not. There's probably a reason for that, but I don't 
see it at the moment.


 I am not looking for an explanation for how to handle multi-threading (
 synchronization and so on ). I am looking to use pre-coded classes and
 structures ( without using __gshared ) with non-shared and shared objects.


Ensure thread-safety and cast `shared` away.
Oct 04 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, October 04, 2016 19:22:10 Begah via Digitalmars-d-learn wrote:

 How can I make a method that accepts being called by both a
 shared and non-shared object, to prevent having to copy methods
 and adding a "shared"?


You could templatize them, but really, the idea is that you _don't_ call
much of anything on a shared object. It's not thread-safe to do so unless
it's protected by a mutex or sychronized block. shared is as unwieldy as it
is in part because it's easy to use incorrectly, and most code should not be
using shared at all, because the stuff that actually needs to be shared
across threads is normally pretty minimal. The fact that most operations are
illegal on shared objects prevents misuse. But of course, it leaves the
question of how you go about actually doing anything with a shared object,
since obviously, you're going to do need to do more than just create the
thing.

TDPL talks about synchronized classes where the outer layer of shared gets
automatically cast away within the member functions of the class (since the
compiler can guarantee that only one thread at a time is in the member
functions of the class). However, synchronized classes haven't been
implemented as of yet, and we just have synchronized functions, which do
protect those functions with a mutex, but without requiring that all
functions within the class be synchronized and make it illegal to access the
member variables except via the member functions, the compiler can't cast
away the outer layer of shared, because it can't guarantee that the member
is actually protected by a mutex.

So, the way to solve this is that when you want to operate on a shared
object, you first make sure that it's protected by a mutex (like you would
in a language like C or C++), then you cast away shared to operate on the
object, and then when you're done, you make sure that no non-shared
references to the object exist, and release the mutex. e.g. something like

shared MyClass mySharedObj = getSharedObj();

synchronized(myMutex)
{
    auto obj = cast(MyClass)mySharedObj;

    // do stuff with obj...

    // obj should be the only non-shared reference to the object
    // referred to by mySharedObj when the sychronized block exits.
}

So, ultimately, the code is basically the same as what you'd do in C/C++
except that you have to cast away shared to actually do much of anything to
the object. It _is_ unfortunately more error-prone than synchronized classes
would be, because you're forced to do a manual cast, but since we don't
actually have synchronized class, you don't have much choice, and since
synchronized classes would only strip away the outer layer of shared,
there's a halfway decent chance that you'd still need to do something like
this even if we did have synchronized classes.

- Jonathan M Davis
Oct 05 2016
parent reply Begah <mathieu.roux222 gmail.com> writes:
On Wednesday, 5 October 2016 at 07:36:58 UTC, Jonathan M Davis 
wrote:

 On Tuesday, October 04, 2016 19:22:10 Begah via 
 Digitalmars-d-learn wrote:

 How can I make a method that accepts being called by both a 
 shared and non-shared object, to prevent having to copy 
 methods and adding a "shared"?


 You could templatize them, but really, the idea is that you 
 _don't_ call much of anything on a shared object. It's not 
 thread-safe to do so unless it's protected by a mutex or 
 sychronized block. shared is as unwieldy as it is in part 
 because it's easy to use incorrectly, and most code should not 
 be using shared at all, because the stuff that actually needs 
 to be shared across threads is normally pretty minimal.


Thanks for the reply,

One of my problem is that, i need all of my data to be accessible 
by both threads ( I have two ). I am making a 3d application and 
decided to separate the update loop and the render loop ( I just 
created another thread for the update loop meanwhile the render 
loop has to remain on the main thread ).

I just want to ensure that when my render loop ( or update loop ) 
updates/render an object, the other loop cannot ( To avoid 
potential bugs whereas one loop changes the position component 
and the render loop renders the object with only the new x 
position because the y and z variable haven't been changed yet ).

As i will have many of those objects, do i need to create a mutex 
for everyone of them?
Oct 05 2016
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Wednesday, October 05, 2016 11:25:57 Begah via Digitalmars-d-learn wrote:

 On Wednesday, 5 October 2016 at 07:36:58 UTC, Jonathan M Davis

 wrote:

 On Tuesday, October 04, 2016 19:22:10 Begah via

 Digitalmars-d-learn wrote:

 How can I make a method that accepts being called by both a
 shared and non-shared object, to prevent having to copy
 methods and adding a "shared"?


 You could templatize them, but really, the idea is that you
 _don't_ call much of anything on a shared object. It's not
 thread-safe to do so unless it's protected by a mutex or
 sychronized block. shared is as unwieldy as it is in part
 because it's easy to use incorrectly, and most code should not
 be using shared at all, because the stuff that actually needs
 to be shared across threads is normally pretty minimal.


 Thanks for the reply,

 One of my problem is that, i need all of my data to be accessible
 by both threads ( I have two ). I am making a 3d application and
 decided to separate the update loop and the render loop ( I just
 created another thread for the update loop meanwhile the render
 loop has to remain on the main thread ).

 I just want to ensure that when my render loop ( or update loop )
 updates/render an object, the other loop cannot ( To avoid
 potential bugs whereas one loop changes the position component
 and the render loop renders the object with only the new x
 position because the y and z variable haven't been changed yet ).

 As i will have many of those objects, do i need to create a mutex
 for everyone of them?


Unless you're writing lock-free algorithms (which really should only be done
by experts, and even then, they should probably reconsider it, since they're
so insanely hard to get right), _every_ variable/object that's going to be
accessible from multiple threads needs to be protected by a mutex so that
it's guaranteed that only one thread accesses the object at a time. That
would be just as true in C/C++ as it is in D. It's just that D requires that
they be marked as shared. That being said, how many objects should be
protected by a given mutex depends entirely on what you're doing. In some
cases, it makes sense to protect a lot of objects with the same mutex (e.g.
all of the member variables of a class could be protected with a single
mutex, which is what would happen with synchronized functions/classes), and
in other cases, it makes sense to have as many as a mutex per variable.
Having fewer mutexes is easier to handle, but it can also mean that code
gets blocked waiting more. And of course, in some cases, the state in
question is really spread across multiple variables, and they all need to be
protected together. I really can't judge how many mutexes would be needed
without knowing what you're doing.

That being said, if you're dealing with a rendering loop where you have one
thread updating the information, and another thread rendering, you probably
want to be using double or triple buffering.

https://en.wikipedia.org/wiki/Multiple_buffering

Basically, you have one buffer (or object or group of objects or whatever is
holding the state) which is updated by one thread, and then when it's ready,
it's swapped with another buffer. So, only the swap would need to be
protected by a mutex, because each buffer would only be referenced by a
single thread at a time. So, you could probably do something as simple as
having a pointer/reference to the buffer (or whatever object is holding the
state), and you swap that. There are a variety of ways that you could do it,
but one way would be something like

//---- update loop ----
while(cond)
{
    // update data in backBuffer...

    // update done

    synchronized(mutex)
    {
        auto temp = cast(Buffer)sharedBuffer;
        sharedBuffer = cast(shared Buffer)backBuffer;
        backBuffer = temp;
    }
}

//---- render loop ----
while(cond)
{
    // render...

    // wait...

    synchronized(mutex)
    {
        auto temp = cast(Buffer)sharedBuffer;
        if(temp !is frontBuffer)
        {
            sharedBuffer = cast(shared Buffer)frontBuffer;
            frontBuffer = temp;
        }
    }
}

This isn't necessarily the best way to do it, but in this particular scheme,
you have 3 buffers so that the render loop always has something to render
from while the update loop can keep swapping buffers as it updates and isn't
stuck waiting for the render loop to be done with the buffer, whereas the
render loop can always just render from the same buffer as long as there
isn't an update.

This does end up locking in a somewhat different way from the basic example
I gave before, because it's just the swap that's being protected, with each
thread passing ownership of a buffer via the middle/shared buffer. However,
it does rely on making sure that all references to the buffer and everything
that it references are passed to the other thread each time so that
operating on a buffer outside of the area protected by the mutex doesn't
risk operating on anything could be being accessed by the other thread.

Essentially, you can use whatever scheme would make sense in C/C++. You just
have to cast away shared in the area protected by the mutex. Everything
else is basically the same, including making sure that either the
unprotected data is only ever accessed by one thread or that all accesses
which could occur on both threads be protected by a mutex.

- Jonathan M Davis
Oct 05 2016
parent reply Begah <mathieu.roux222 gmail.com> writes:
On Wednesday, 5 October 2016 at 12:48:07 UTC, Jonathan M Davis 
wrote:

 On Wednesday, October 05, 2016 11:25:57 Begah via 
 Digitalmars-d-learn wrote:

 [...]


 Unless you're writing lock-free algorithms (which really should 
 only be done by experts, and even then, they should probably 
 reconsider it, since they're so insanely hard to get right), 
 _every_ variable/object that's going to be accessible from 
 multiple threads needs to be protected by a mutex so that it's 
 guaranteed that only one thread accesses the object at a time. 
 That would be just as true in C/C++ as it is in D. It's just 
 that D requires that they be marked as shared. That being said, 
 how many objects should be protected by a given mutex depends 
 entirely on what you're doing. In some cases, it makes sense to 
 protect a lot of objects with the same mutex (e.g. all of the 
 member variables of a class could be protected with a single 
 mutex, which is what would happen with synchronized 
 functions/classes), and in other cases, it makes sense to have 
 as many as a mutex per variable. Having fewer mutexes is easier 
 to handle, but it can also mean that code gets blocked waiting 
 more. And of course, in some cases, the state in question is 
 really spread across multiple variables, and they all need to 
 be protected together. I really can't judge how many mutexes 
 would be needed without knowing what you're doing.

 [...]


Thanks,

Although the triple buffer seems a good idea, there is one 
problem.
I will need three time as much ram than what i currently need.
Not to mention, every time i switch buffer i will need to copy 
all changes made to the updated buffer to the next buffer to be 
updated (Which i think, doing it a few hundred times a second 
might become a bottleneck ).
Oct 05 2016
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Wednesday, October 05, 2016 21:10:01 Begah via Digitalmars-d-learn wrote:

 Although the triple buffer seems a good idea, there is one
 problem.
 I will need three time as much ram than what i currently need.
 Not to mention, every time i switch buffer i will need to copy
 all changes made to the updated buffer to the next buffer to be
 updated (Which i think, doing it a few hundred times a second
 might become a bottleneck ).


Well, I certainly can't tell what the exact best solution is in your case,
but if you can't do some kind of buffer swapping, I wouldn't think that
there's any point in separating the update and render threads, because if
you only have one buffer to deal with, the update thread can't actually
update anything while the render thread renders it, because otherwise, the
render thread would be accessing data while the update thread mutated it.
And properly protecting that code with a mutex would force each thread to
wait for the other would pretty much negate the benefits of putting that
code in separate threads. Certainly, the way that your typical grahpics
stuff paints things involves some variant of double or triple buffering. But
since I obviously don't know all of the details of your problem, I can't
really give better advice.

In any case, hopefully you understand the shared stuff and mutex stuff well
enough now to figure out what works best for your particular use case.

- Jonathan M Davis
Oct 06 2016