www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Locking data

reply IntegratedDimensions <IntegratedDimensions gmail.com> writes:
an idea to lock data by removing the reference:

class A
{
    Lockable!Data data;
}

The idea is that when the data is going to be used, the user 
locks the data. The trick here is that data is a pointer to the 
data and the pointer is set to null when locked so no other data 
can use it(they see a null reference). To unlock, the data is 
reassigned:

auto d = data.lock(); // A.data is now null deals with sync issues
//use d
d = data.unlock(d); // restores A.data (A.data = d;) and sets d 
to null so any future reference is an error(could produce bugs 
but should mostly be access violations)


Anyone else trying to use data will see that it is null while it 
is locked.

This basically pushes the standard locking mechanisms in to the 
Lockable!data(first come first serve) and code that has not 
captured the data see's it simply as null.

Anyone use know if there exists an idiom like this and what it is 
called? Maybe some idiomatic code that is efficient?


Ideally I'd want to be able to treat the Lockable!Data as Data. 
Right now I have to classes(or pointers to structs) and few weird 
hacks. I think what I'll have to end up doing is having a 
Locked!Data structure that is returned instead or use an UFCS. 
The idea being to attach the lock and unlock methods.
May 22 2018
next sibling parent reply Alex <sascha.orlov gmail.com> writes:
On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions 
wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 The idea is that when the data is going to be used, the user 
 locks the data. The trick here is that data is a pointer to the 
 data and the pointer is set to null when locked so no other 
 data can use it(they see a null reference). To unlock, the data 
 is reassigned:

 auto d = data.lock(); // A.data is now null deals with sync 
 issues
 //use d
 d = data.unlock(d); // restores A.data (A.data = d;) and sets d 
 to null so any future reference is an error(could produce bugs 
 but should mostly be access violations)


 Anyone else trying to use data will see that it is null while 
 it is locked.

 This basically pushes the standard locking mechanisms in to the 
 Lockable!data(first come first serve) and code that has not 
 captured the data see's it simply as null.

 Anyone use know if there exists an idiom like this and what it 
 is called? Maybe some idiomatic code that is efficient?


 Ideally I'd want to be able to treat the Lockable!Data as Data. 
 Right now I have to classes(or pointers to structs) and few 
 weird hacks. I think what I'll have to end up doing is having a 
 Locked!Data structure that is returned instead or use an UFCS. 
 The idea being to attach the lock and unlock methods.


Are you aware of NullableRef?
https://dlang.org/library/std/typecons/nullable_ref.html

Does it fit somehow?
May 22 2018
parent reply IntegratedDimensions <IntegratedDimensions gmail.com> writes:
On Tuesday, 22 May 2018 at 22:10:52 UTC, Alex wrote:

 On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions 
 wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 The idea is that when the data is going to be used, the user 
 locks the data. The trick here is that data is a pointer to 
 the data and the pointer is set to null when locked so no 
 other data can use it(they see a null reference). To unlock, 
 the data is reassigned:

 auto d = data.lock(); // A.data is now null deals with sync 
 issues
 //use d
 d = data.unlock(d); // restores A.data (A.data = d;) and sets 
 d to null so any future reference is an error(could produce 
 bugs but should mostly be access violations)


 Anyone else trying to use data will see that it is null while 
 it is locked.

 This basically pushes the standard locking mechanisms in to 
 the Lockable!data(first come first serve) and code that has 
 not captured the data see's it simply as null.

 Anyone use know if there exists an idiom like this and what it 
 is called? Maybe some idiomatic code that is efficient?


 Ideally I'd want to be able to treat the Lockable!Data as 
 Data. Right now I have to classes(or pointers to structs) and 
 few weird hacks. I think what I'll have to end up doing is 
 having a Locked!Data structure that is returned instead or use 
 an UFCS. The idea being to attach the lock and unlock methods.


 Are you aware of NullableRef?
 https://dlang.org/library/std/typecons/nullable_ref.html


Yes.


 Does it fit somehow?


Not really. It could be used to wrap the data when used as a 
struct but it offers none of the locking features which is the 
ultimate goal.

The idea is simply that one can lock the data to get exclusive 
access. Normally standard locking tricks are used but in this 
case the idea is to do all that behind the scenes. Locking the 
data makes all other accessors see null data(and crash or 
properly test). This is a first come first serve or single access 
type of pattern but sort of removes the data from prying eyes 
while it is being used.
May 22 2018
parent reply Sjoerd Nijboer <sjoerdnijboer gmail.com> writes:
On Tuesday, 22 May 2018 at 22:17:05 UTC, IntegratedDimensions 
wrote:

 On Tuesday, 22 May 2018 at 22:10:52 UTC, Alex wrote:

 On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions 
 wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 The idea is that when the data is going to be used, the user 
 locks the data. The trick here is that data is a pointer to 
 the data and the pointer is set to null when locked so no 
 other data can use it(they see a null reference). To unlock, 
 the data is reassigned:

 auto d = data.lock(); // A.data is now null deals with sync 
 issues
 //use d
 d = data.unlock(d); // restores A.data (A.data = d;) and sets 
 d to null so any future reference is an error(could produce 
 bugs but should mostly be access violations)


 Anyone else trying to use data will see that it is null while 
 it is locked.

 This basically pushes the standard locking mechanisms in to 
 the Lockable!data(first come first serve) and code that has 
 not captured the data see's it simply as null.

 Anyone use know if there exists an idiom like this and what 
 it is called? Maybe some idiomatic code that is efficient?


 Ideally I'd want to be able to treat the Lockable!Data as 
 Data. Right now I have to classes(or pointers to structs) and 
 few weird hacks. I think what I'll have to end up doing is 
 having a Locked!Data structure that is returned instead or 
 use an UFCS. The idea being to attach the lock and unlock 
 methods.


 Are you aware of NullableRef?
 https://dlang.org/library/std/typecons/nullable_ref.html


 Yes.


 Does it fit somehow?


 Not really. It could be used to wrap the data when used as a 
 struct but it offers none of the locking features which is the 
 ultimate goal.

 The idea is simply that one can lock the data to get exclusive 
 access. Normally standard locking tricks are used but in this 
 case the idea is to do all that behind the scenes. Locking the 
 data makes all other accessors see null data(and crash or 
 properly test). This is a first come first serve or single 
 access type of pattern but sort of removes the data from prying 
 eyes while it is being used.


how about something like

  import core.atomic;
  class Lockable!Data
  {
     private __gshared Lockable!Data data;
  }

  struct Locked!Lockable!Data
  {
     private TailShared!Lockable!Data lockedData;
     private Lockable!Data lockableData;

     this(Lockable!Data lockableData)
     {
         this.lockableData = lockableData;
         while( (lockedData= atomicLoad(lockableData)) !is null)
             if(cas(lockedData, lockedData, null))
                 break;
     }

     ~this()
     {
         atomicStore(lockedData, lockableData );
     }

     alias lockedData.data this;
  }

With something like this you should be able to do RAII like 
semantics on your data.
You share a Lockable!Data which you want use, and then you can 
access it if you instantiate a Locked!Lockable!Data struct with 
it.
FYI, my atomics isn't all that great so don't expect this to work.
May 22 2018
parent IntegratedDimensions <IntegratedDimensions gmail.com> writes:
On Tuesday, 22 May 2018 at 23:09:24 UTC, Sjoerd Nijboer wrote:

 On Tuesday, 22 May 2018 at 22:17:05 UTC, IntegratedDimensions 
 wrote:

 On Tuesday, 22 May 2018 at 22:10:52 UTC, Alex wrote:

 On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions 
 wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 The idea is that when the data is going to be used, the user 
 locks the data. The trick here is that data is a pointer to 
 the data and the pointer is set to null when locked so no 
 other data can use it(they see a null reference). To unlock, 
 the data is reassigned:

 auto d = data.lock(); // A.data is now null deals with sync 
 issues
 //use d
 d = data.unlock(d); // restores A.data (A.data = d;) and 
 sets d to null so any future reference is an error(could 
 produce bugs but should mostly be access violations)


 Anyone else trying to use data will see that it is null 
 while it is locked.

 This basically pushes the standard locking mechanisms in to 
 the Lockable!data(first come first serve) and code that has 
 not captured the data see's it simply as null.

 Anyone use know if there exists an idiom like this and what 
 it is called? Maybe some idiomatic code that is efficient?


 Ideally I'd want to be able to treat the Lockable!Data as 
 Data. Right now I have to classes(or pointers to structs) 
 and few weird hacks. I think what I'll have to end up doing 
 is having a Locked!Data structure that is returned instead 
 or use an UFCS. The idea being to attach the lock and unlock 
 methods.


 Are you aware of NullableRef?
 https://dlang.org/library/std/typecons/nullable_ref.html


 Yes.


 Does it fit somehow?


 Not really. It could be used to wrap the data when used as a 
 struct but it offers none of the locking features which is the 
 ultimate goal.

 The idea is simply that one can lock the data to get exclusive 
 access. Normally standard locking tricks are used but in this 
 case the idea is to do all that behind the scenes. Locking the 
 data makes all other accessors see null data(and crash or 
 properly test). This is a first come first serve or single 
 access type of pattern but sort of removes the data from 
 prying eyes while it is being used.


 how about something like

  import core.atomic;
  class Lockable!Data
  {
     private __gshared Lockable!Data data;
  }

  struct Locked!Lockable!Data
  {
     private TailShared!Lockable!Data lockedData;
     private Lockable!Data lockableData;

     this(Lockable!Data lockableData)
     {
         this.lockableData = lockableData;
         while( (lockedData= atomicLoad(lockableData)) !is null)
             if(cas(lockedData, lockedData, null))
                 break;
     }

     ~this()
     {
         atomicStore(lockedData, lockableData );
     }

     alias lockedData.data this;
  }

 With something like this you should be able to do RAII like 
 semantics on your data.
 You share a Lockable!Data which you want use, and then you can 
 access it if you instantiate a Locked!Lockable!Data struct with 
 it.
 FYI, my atomics isn't all that great so don't expect this to 
 work.


This is similar to what I am doing... maybe "identical".
May 22 2018
prev sibling parent reply Malte <no valid.mail> writes:
On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions 
wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 [...]


This sounds like you are looking for is an atomic swap. Afaik it 
doesn't exist in the standard library. You could use asm for the 
XCHG, but that would make your code x86 dependent.
I think the easiest way would be to just use a mutex and tryLock.
May 23 2018
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 24/05/2018 1:20 AM, Malte wrote:

 On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 [...]


 
 This sounds like you are looking for is an atomic swap. Afaik it doesn't 
 exist in the standard library. You could use asm for the XCHG, but that 
 would make your code x86 dependent.
 I think the easiest way would be to just use a mutex and tryLock.


What are you talking about? :p

http://dpldocs.info/experimental-docs/core.atomic.cas.1.html
May 23 2018
parent reply Malte <no valid.mail> writes:
On Wednesday, 23 May 2018 at 13:24:35 UTC, rikki cattermole wrote:

 On 24/05/2018 1:20 AM, Malte wrote:

 On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions 
 wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 [...]


 
 This sounds like you are looking for is an atomic swap. Afaik 
 it doesn't exist in the standard library. You could use asm 
 for the XCHG, but that would make your code x86 dependent.
 I think the easiest way would be to just use a mutex and 
 tryLock.


 What are you talking about? :p

 http://dpldocs.info/experimental-docs/core.atomic.cas.1.html


That is Compare-and-set.
To make an exchange using cas I first have to read the value, 
then write to it expecting to be still the value I read before. 
That are more instructions than just a swap. If a cas fails, I 
have to redo everything. An exchange never fails, I just might 
not get the result I would like to have (null instead of pointer).
May 23 2018
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 24/05/2018 1:29 AM, Malte wrote:

 On Wednesday, 23 May 2018 at 13:24:35 UTC, rikki cattermole wrote:

 On 24/05/2018 1:20 AM, Malte wrote:

 On Tuesday, 22 May 2018 at 21:45:07 UTC, IntegratedDimensions wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 [...]


 This sounds like you are looking for is an atomic swap. Afaik it 
 doesn't exist in the standard library. You could use asm for the 
 XCHG, but that would make your code x86 dependent.
 I think the easiest way would be to just use a mutex and tryLock.


 What are you talking about? :p

 http://dpldocs.info/experimental-docs/core.atomic.cas.1.html


 
 That is Compare-and-set.
 To make an exchange using cas I first have to read the value, then write 
 to it expecting to be still the value I read before. That are more 
 instructions than just a swap. If a cas fails, I have to redo 
 everything. An exchange never fails, I just might not get the result I 
 would like to have (null instead of pointer).


So you want a load + store as swap in a single function (that is optimized).
In that case, please create an issue on bugzilla (issues.dlang.org).
May 23 2018
parent Malte <no valid.mail> writes:
On Wednesday, 23 May 2018 at 13:36:20 UTC, rikki cattermole wrote:

 On 24/05/2018 1:29 AM, Malte wrote:

 On Wednesday, 23 May 2018 at 13:24:35 UTC, rikki cattermole 
 wrote:

 On 24/05/2018 1:20 AM, Malte wrote:

 On Tuesday, 22 May 2018 at 21:45:07 UTC, 
 IntegratedDimensions wrote:

 an idea to lock data by removing the reference:

 class A
 {
    Lockable!Data data;
 }

 [...]


 This sounds like you are looking for is an atomic swap. 
 Afaik it doesn't exist in the standard library. You could 
 use asm for the XCHG, but that would make your code x86 
 dependent.
 I think the easiest way would be to just use a mutex and 
 tryLock.


 What are you talking about? :p

 http://dpldocs.info/experimental-docs/core.atomic.cas.1.html


 
 That is Compare-and-set.
 To make an exchange using cas I first have to read the value, 
 then write to it expecting to be still the value I read 
 before. That are more instructions than just a swap. If a cas 
 fails, I have to redo everything. An exchange never fails, I 
 just might not get the result I would like to have (null 
 instead of pointer).


 So you want a load + store as swap in a single function (that 
 is optimized).
 In that case, please create an issue on bugzilla 
 (issues.dlang.org).


No, as I said, that is already one instruction on X86: 
https://www.felixcloutier.com/x86/XCHG.html
Just being able to use that instruction with the standard library 
would be good.

You could also use it with compiler intrinsics. Something like

import ldc.intrinsics;
T* tryGetPtr(T)(T** a) {
    return 
 cast(T*)llvm_atomic_rmw_xchg!size_t(cast(shared(size_t)*)a, 0);
}
void restorePtr(T)(T** a, T* b) {
    
 llvm_atomic_rmw_xchg!size_t(cast(shared(size_t)*)a,cast(size_t)b);
}


I would just go with mutexes unless your really need to go that 
low level though, much saner.
May 23 2018