www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Movable resource handles

reply Matt Elkins <notreal fake.com> writes:
Hi all -- I am very new to D, coming from a background heavy on 
C++ as well as some other languages. Consequently, I am trying to 
get my head around D idioms and could use some help.

In a pet project of mine I have to deal with a lot (both many 
kinds and many instances) of handles to non-memory resources. 
These handles need deterministic destruction and are generally 
not copyable (at least, I don't want destructors run twice), but 
I do want to be able to move them ala C++'s move 
constructor/assignment. In C++ I could use std::unique_ptr with a 
custom deleter or roll my own variant of the same, but I am not 
sure how to do this in D. std.typecons.Unique doesn't appear to 
fit the bill, and my attempts to roll my own generic wrapper are 
producing results more akin to std::auto_ptr than std::unique_ptr 
(sorry for all the C++ references, it's the paradigm I know).

So to summarize, I want a struct (or something) to achieve these 
semantics:
* Stores a handle to a resource
* On destruction, cleans up the resource (custom deleter)
* Prohibits copying or in some other fashion prevents 
double-destruction
* Allows implicit moving (that is, allows destructive copy when 
the moved-from object is an rvalue)
* Is lightweight (in particular, does not touch the heap)

I have been able to implement a generic struct that addresses all 
of those except for allowing moving. Note that moving needs to be 
implicit -- that is, not require calling something like release() 
if it is an rvalue.

I apologize if this has been asked before; I found a related 
discussion from a few years ago 
(http://forum.dlang.org/post/jtogeq$2ndj$1 digitalmars.com), but 
it was focused on performance and didn't seem to address my 
specific concern.

Thanks in advance! And thanks for such a neat language!
Jan 27 2016
parent reply ZombineDev <valid_email he.re> writes:
On Wednesday, 27 January 2016 at 23:20:27 UTC, Matt Elkins wrote:

 Hi all -- I am very new to D, coming from a background heavy on 
 C++ as well as some other languages. Consequently, I am trying 
 to get my head around D idioms and could use some help.

 In a pet project of mine I have to deal with a lot (both many 
 kinds and many instances) of handles to non-memory resources. 
 These handles need deterministic destruction and are generally 
 not copyable (at least, I don't want destructors run twice), 
 but I do want to be able to move them ala C++'s move 
 constructor/assignment. In C++ I could use std::unique_ptr with 
 a custom deleter or roll my own variant of the same, but I am 
 not sure how to do this in D. std.typecons.Unique doesn't 
 appear to fit the bill, and my attempts to roll my own generic 
 wrapper are producing results more akin to std::auto_ptr than 
 std::unique_ptr (sorry for all the C++ references, it's the 
 paradigm I know).

 So to summarize, I want a struct (or something) to achieve 
 these semantics:
 * Stores a handle to a resource
 * On destruction, cleans up the resource (custom deleter)
 * Prohibits copying or in some other fashion prevents 
 double-destruction
 * Allows implicit moving (that is, allows destructive copy when 
 the moved-from object is an rvalue)
 * Is lightweight (in particular, does not touch the heap)

 I have been able to implement a generic struct that addresses 
 all of those except for allowing moving. Note that moving needs 
 to be implicit -- that is, not require calling something like 
 release() if it is an rvalue.

 I apologize if this has been asked before; I found a related 
 discussion from a few years ago 
 (http://forum.dlang.org/post/jtogeq$2ndj$1 digitalmars.com), 
 but it was focused on performance and didn't seem to address my 
 specific concern.

 Thanks in advance! And thanks for such a neat language!


In my project I have something like this:

```
import std.experimental.allocator :
   make, dispose;
import std.algorithm.mutation :
   move, moveEmplace;

struct UniqueRef(T, alias allocator)
{
   private T* handle;

   // I use a modified
   // version ofstd.typecons.Proxy.
   // Forwards most operations
   // except for opAssign to
   // *handle;
   mixin Proxy!handle;

   this(Args...)(auto ref Args args)
   {
     this.handle =
       allocator.make!T(args);
   }

   // Should work after Phobos PR 3956
   // is merged.
   //  disable this();

   // Necessarry to ensure unique
   // ownership.
    disable this(this);

   // Steels the handle from
   // lvalues & rvalues
   void opAssign()
     (auto ref typeof(this) other)
   {
     destroy(this);
     this.handle =
       moveEmplace(other.handle);
   }

   ~this()
   {
     if (this.handle is null)
       return;

     allocator.dispose(this.handle);
     this.handle = null;
   }
}

// Initialation
auto a = UniqueRed!int(41);

// Transparent forwarding
assert (a.handle !is null);
assert (a == 41);
assert (++a == 42);

// Unique ownership
// auto b = a; doesn't compile

// Move semantics
auto b = move(a);
assert (b == 42);
assert (a.handle is null);

// Consumption
void useHandle()
   (auto ref UniqueRef!int h)
{
   UniqueRef!(int)[] arrayOfHandles =
     getArr();

   arrayOfHandles[0] = move(h);
}

// Transfer ownership
useHandle(b);

// Accepts rvalues
useHandle(UniqueRef!int(7));

void borrow(const ref UniqueRef!int h)
{
   import std.stdio;
   int val = h + 0;
   writeln(val);
}

auto d = UniqueRef!int(12);
borrow(d);
assert (d.handle !is null);

// and so on...
```

I don't know if it's 100% memory safe, but for me it works good 
enough. Sorry if there are typos/mistakes, I'm writting on a 
phone.
Jan 27 2016
parent reply Matt Elkins <notreal fake.com> writes:
On Thursday, 28 January 2016 at 00:14:18 UTC, ZombineDev wrote:

 On Wednesday, 27 January 2016 at 23:20:27 UTC, Matt Elkins 
 wrote:

 [...]


 In my project I have something like this:

 ```
 import std.experimental.allocator :
   make, dispose;
 import std.algorithm.mutation :
   move, moveEmplace;

 struct UniqueRef(T, alias allocator)
 {
   private T* handle;

   // I use a modified
   // version ofstd.typecons.Proxy.
   // Forwards most operations
   // except for opAssign to
   // *handle;
   mixin Proxy!handle;

   this(Args...)(auto ref Args args)
   {
     this.handle =
       allocator.make!T(args);
   }

   // Should work after Phobos PR 3956
   // is merged.
   //  disable this();

   // Necessarry to ensure unique
   // ownership.
    disable this(this);

   // Steels the handle from
   // lvalues & rvalues
   void opAssign()
     (auto ref typeof(this) other)
   {
     destroy(this);
     this.handle =
       moveEmplace(other.handle);
   }

   ~this()
   {
     if (this.handle is null)
       return;

     allocator.dispose(this.handle);
     this.handle = null;
   }
 }

 // Initialation
 auto a = UniqueRed!int(41);

 // Transparent forwarding
 assert (a.handle !is null);
 assert (a == 41);
 assert (++a == 42);

 // Unique ownership
 // auto b = a; doesn't compile

 // Move semantics
 auto b = move(a);
 assert (b == 42);
 assert (a.handle is null);

 // Consumption
 void useHandle()
   (auto ref UniqueRef!int h)
 {
   UniqueRef!(int)[] arrayOfHandles =
     getArr();

   arrayOfHandles[0] = move(h);
 }

 // Transfer ownership
 useHandle(b);

 // Accepts rvalues
 useHandle(UniqueRef!int(7));

 void borrow(const ref UniqueRef!int h)
 {
   import std.stdio;
   int val = h + 0;
   writeln(val);
 }

 auto d = UniqueRef!int(12);
 borrow(d);
 assert (d.handle !is null);

 // and so on...
 ```

 I don't know if it's 100% memory safe, but for me it works good 
 enough. Sorry if there are typos/mistakes, I'm writting on a 
 phone.


Thanks for the response! I learned more about D just from 
studying this example. However, I'm not sure that this works for 
what I need. For example, if I want to move one of these handles 
into an array:

///// Not sure how to tag this as a code block on the forum /////
alias UR = UniqueRef!(int, theAllocator);
auto a = UR(41);
UR[] handles;

// Both of the following lines yield:
// Error: ... UniqueRef is not copyable because it is annotated 
with  disable
// handles ~= move(a);
// handles ~= a;
//// End code block ////

This is essentially the same error I get on my attempts to 
implement these semantics, as well.
Jan 27 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 03:02:34 UTC, Matt Elkins wrote:

 Thanks for the response! I learned more about D just from 
 studying this example.


  Hmmm you might read the free online book, i'm going through it 
and finding it useful so far as a refresher.

http://ddili.org/ders/d.en/index.html


 ///// Not sure how to tag this as a code block on the forum


  Since the forum is text only, i like to use bbcode as [code] 
blocks (if there's a preferred method, I haven't one). So:

[code]
  auto pi = 3.14159;
[/code]



 /////
 alias UR = UniqueRef!(int, theAllocator);
 auto a = UR(41);
 UR[] handles;

 // Both of the following lines yield:
 // Error: ... UniqueRef is not copyable because it is annotated 
 with  disable
 // handles ~= move(a);
 // handles ~= a;
 //// End code block ////

 This is essentially the same error I get on my attempts to 
 implement these semantics, as well.


  hmmm... You'd best be allocating the reference directly on the 
array, you should be able to add to it.

[code]
  handles ~= UR(41);
[/code]
Jan 27 2016
next sibling parent Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 03:38:32 UTC, Era Scarecrow wrote:

  hmmm... You'd best be allocating the reference directly on the 
 array, you should be able to add to it.


  Forgot to mention. Since UniqueRef is likely just to ensure 
nothing else can copy away with the data, you don't HAVE to make 
it unique until the last step. So you could do all your 
calculations, then wrap it as a last step when returning it, say 
outside of the function. Then the assignment happens when the 
function returns and not where you generated it.

[code]
UR func(...) {
   int val;

   // calculates while it's in a safe/pure environment

   return UR(val);
}
[/code]
Jan 27 2016
prev sibling parent reply Matt Elkins <notreal fake.com> writes:
On Thursday, 28 January 2016 at 03:38:32 UTC, Era Scarecrow wrote:

 On Thursday, 28 January 2016 at 03:02:34 UTC, Matt Elkins wrote:

 Thanks for the response! I learned more about D just from 
 studying this example.


  Hmmm you might read the free online book, i'm going through it 
 and finding it useful so far as a refresher.

 http://ddili.org/ders/d.en/index.html



Thanks. I've also got Andrei's book (The D Programming Language) 
which I read before starting this.



 ///// Not sure how to tag this as a code block on the forum


  Since the forum is text only, i like to use bbcode as [code] 
 blocks (if there's a preferred method, I haven't one). So:

 [code]
  auto pi = 3.14159;
 [/code]


Cool, thanks!


 /////
 alias UR = UniqueRef!(int, theAllocator);
 auto a = UR(41);
 UR[] handles;

 // Both of the following lines yield:
 // Error: ... UniqueRef is not copyable because it is 
 annotated with  disable
 // handles ~= move(a);
 // handles ~= a;
 //// End code block ////

 This is essentially the same error I get on my attempts to 
 implement these semantics, as well.


  hmmm... You'd best be allocating the reference directly on the 
 array, you should be able to add to it.

 [code]
  handles ~= UR(41);
 [/code]


Hm. Maybe I should show my original attempt to do this:

[code]
import std.algorithm;

struct ResourceHandle(T, DestroyPolicy, T Default = T.init)
{
     // Constructors/Destructor
     this(in T handle) nothrow {m_handle = handle;}
      disable this(this);
     ~this() {DestroyPolicy.destroy(m_handle);}

     // Operators
      disable void opAssign(ref ResourceHandle lvalue);
     ref ResourceHandle opAssign(ResourceHandle rvalue) 
{swap(m_handle, rvalue.m_handle); return this;}
     ref ResourceHandle opAssign(in T handle) 
{DestroyPolicy.destroy(m_handle); m_handle = handle; return this;}

     // Methods
     T get() const nothrow {return m_handle;}
     T release() nothrow {T result = m_handle; m_handle = Default; 
return result;}

     private:
         T m_handle = Default;
}

unittest
{
     static uint lastDestroyed;
     struct Destroyer {static void destroy(uint resource) 
{lastDestroyed = resource;}}
     alias RH = ResourceHandle!(uint, Destroyer, 0u);

     assert (lastDestroyed == 0);
     {auto handle = RH(7);}
     assert (lastDestroyed == 7);

     {
         auto handle = RH(8);
         assert(handle.release() == 8);
         assert(handle.get() == uint.init);
         assert (lastDestroyed == 7);
     }
     assert (lastDestroyed == uint.init);

     {
         auto handle = RH(8);
         assert(handle.get() == 8);
         assert (lastDestroyed == uint.init);
     }
     assert (lastDestroyed == 8);

     auto handle1 = RH(1);
     auto handle2 = RH(2);
     assert(handle1.get() == 1);
     assert(handle2.get() == 2);

     assert (lastDestroyed == 8);
     handle2 = handle1.release();
     assert (lastDestroyed == 2);
     assert (handle2.get() == 1);
     assert (handle1.get() == uint.init);

     RH[] handles;
     //handles ~= RH(3); // Fails because post-blit constructor is 
 disabled
}
[/code]

By way of comparison, in C++ I could do something like this 
(typing this directly into the post without checking it, so may 
have syntax errors:

[code]
// Compare to the last two lines of the unit test
using RH = std::unique_ptr<unsigned int, Destroyer>;
std::vector<RH> handles;
handles.push_back(RH(3)); // Moves in the "3" resource
[/code]

Since D appears to lack an equivalent for C++'s move semantics, I 
wonder whether this is even possible to achieve? Maybe if there 
were some way to test for lvalue vs rvalue in the copy 
constructor...except that D has a post-blit constructor, not a 
copy constructor. I suspect I am handicapping myself by thinking 
in C++ terms.
Jan 27 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 03:55:22 UTC, Matt Elkins wrote:

     //handles ~= RH(3); // Fails because post-blit constructor 
 is  disabled


  It really comes down to that an array qualifies as as an Lvalue 
operator; But I _think_ this is a bug in the language since you 
should be able to assign a new Rvalue to a new array element just 
being created/added. We'll have to get input from Walter or 
Andrei.
Jan 27 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 05:27:16 UTC, Era Scarecrow wrote:

 On Thursday, 28 January 2016 at 03:55:22 UTC, Matt Elkins wrote:

     //handles ~= RH(3); // Fails because post-blit constructor 
 is  disabled


  It really comes down to that an array qualifies as an Lvalue 
 operator; But I _think_ this is a bug in the language since you 
 should be able to assign a new Rvalue to a new array element 
 just being created/added. We'll have to get input from Walter 
 or Andrei.


Alright here's a minimalistic test of the problem. Probably end 
up adding this to bugzilla.

struct S {
    disable this(this);
}

unittest {
  // S s = S(); //normal assign, not important
  S a[];
  a ~= S();  //doesn't assign, postblit disabled
}
Jan 27 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Thursday, 28 January 2016 at 05:39:55 UTC, Era Scarecrow wrote:

  It really comes down to that an array qualifies as an Lvalue 
 operator; But I _think_ this is a bug in the language since you 
 should be able to assign a new Rvalue to a new array element 
 just being created/added. We'll have to get input from Walter 
 or Andrei.


  Alright, closest bug match i could find is 
https://issues.dlang.org/show_bug.cgi?id=7032 which is sorta the 
same thing, but not quite...
Jan 27 2016
parent Matt Elkins <notreal fake.com> writes:
On Thursday, 28 January 2016 at 06:13:32 UTC, Era Scarecrow wrote:

 On Thursday, 28 January 2016 at 05:39:55 UTC, Era Scarecrow 
 wrote:

  It really comes down to that an array qualifies as an Lvalue 
 operator; But I _think_ this is a bug in the language since 
 you should be able to assign a new Rvalue to a new array 
 element just being created/added. We'll have to get input from 
 Walter or Andrei.


  Alright, closest bug match i could find is 
 https://issues.dlang.org/show_bug.cgi?id=7032 which is sorta 
 the same thing, but not quite...


Ah, ok. A bug is better than a language design limitation -- it 
might get fixed at some point! I'll figure out a workaround for 
my use case.

Thanks for all the time you spent, I appreciate the help.
Jan 29 2016