www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - RC buffer

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I've eliminated all UTF nonsense from 
https://github.com/dlang/phobos/pull/4878 resulting in a bare reference 
counted buffer of (qualified) ubyte.

The goal is to get the buffer  safe and be able to have a reasonable 
explanation for each and every cast. (There are no safety annotations 
currently, but the code passes unittests and supports qualifiers.) No 
undefined behavior casts should be used (e.g. that cast away 
immutability), only simple adjustments for realities that the type 
system is unable to cover.

In order to make opAssign safe, a language change will be necessary.


Andrei
Nov 01 2016
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 2 November 2016 at 05:00:23 UTC, Andrei 
Alexandrescu wrote:

 I've eliminated all UTF nonsense from 
 https://github.com/dlang/phobos/pull/4878 resulting in a bare 
 reference counted buffer of (qualified) ubyte.

 The goal is to get the buffer  safe and be able to have a 
 reasonable explanation for each and every cast. (There are no 
 safety annotations currently, but the code passes unittests and 
 supports qualifiers.) No undefined behavior casts should be 
 used (e.g. that cast away immutability), only simple 
 adjustments for realities that the type system is unable to 
 cover.

 In order to make opAssign safe, a language change will be 
 necessary.


 Andrei


Shouldn't all those` assert(b ~ c == "bc".representation);`s be 
static asserts?
Nov 01 2016
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 2 November 2016 at 05:32:13 UTC, Nicholas Wilson 
wrote:

 On Wednesday, 2 November 2016 at 05:00:23 UTC, Andrei 
 Alexandrescu wrote:

 I've eliminated all UTF nonsense from 
 https://github.com/dlang/phobos/pull/4878 resulting in a bare 
 reference counted buffer of (qualified) ubyte.

 The goal is to get the buffer  safe and be able to have a 
 reasonable explanation for each and every cast. (There are no 
 safety annotations currently, but the code passes unittests 
 and supports qualifiers.) No undefined behavior casts should 
 be used (e.g. that cast away immutability), only simple 
 adjustments for realities that the type system is unable to 
 cover.

 In order to make opAssign safe, a language change will be 
 necessary.


 Andrei


 Shouldn't all those` assert(b ~ c == "bc".representation);`s be 
 static asserts?


Derp. copied wrong line. I meant all the typeof asserts.
Nov 01 2016
prev sibling next sibling parent Daniel9 <andreikovalsa mail.ru> writes:
On Wednesday, 2 November 2016 at 05:00:23 UTC, Andrei 
Alexandrescu wrote:

 I've eliminated all UTF nonsense from 
 https://github.com/dlang/phobos/pull/4878 resulting in a bare 
 reference counted buffer of (qualified) ubyte.

 The goal is to get the buffer  safe and be able to have a 
 reasonable explanation for each and every cast. (There are no 
 safety annotations currently, but the code passes unittests and 
 supports qualifiers.) No undefined behavior casts should be 
 used (e.g. that cast away immutability), only simple 
 adjustments for realities that the type system is unable to 
 cover.

 In order to make opAssign safe, a language change will be 
 necessary.


 Andrei


Great goal, I wish you to do it!
Nov 02 2016
prev sibling next sibling parent reply Nick Treleaven <nick geany.org> writes:
On Wednesday, 2 November 2016 at 05:00:23 UTC, Andrei 
Alexandrescu wrote:

 In order to make opAssign safe, a language change will be 
 necessary.


Technically, it should be possible with runtime checks:

https://forum.dlang.org/post/aeeffshzkfjbrejztksj forum.dlang.org

The checking overheads disappear when -noboundschecks is passed. 
The user has to manually copy the RCSlice when necessary for 
correct code.

The RCSlice in the link is just a basic proof of concept, using 
RCRef for temporary references. It doesn't handle reallocations - 
I think to have early diagnostics of potential errors it would 
need to have a separate RCRef reference count from the main 
count. That would be used to check there are no RCRef references 
alive when the main count is one and a reallocation could 
potentially occur - e.g. when appending. I think with this 
(hypothetical) RCSlice it's bug-prone to allow the user *not* to 
make a temporary copy of the RCSlice in this scenario because 
errors could depend on unknown runtime behaviour, not showing up 
in testing.

I mention this partly because I think this scenario has an impact 
on the design of a DIP to address when to automatically add 
reference counting bumps - perhaps a  rcbump attribute is 
necessary. Otherwise the compiler can't know if an external 
function taking an RCString performs an append or not.
Nov 02 2016
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/02/2016 07:29 AM, Nick Treleaven wrote:

 On Wednesday, 2 November 2016 at 05:00:23 UTC, Andrei Alexandrescu wrote:

 In order to make opAssign safe, a language change will be necessary.


 Technically, it should be possible with runtime checks:

 https://forum.dlang.org/post/aeeffshzkfjbrejztksj forum.dlang.org

 The checking overheads disappear when -noboundschecks is passed. The
 user has to manually copy the RCSlice when necessary for correct code.


It seems opAssign for RCSlice is unsafe, is that right? Consider (with 
your codebase):

 safe void fun(ref RCSlice!int a, ref RCSlice!int b)
{
     a = RCSlice!int();
     ... use b ...
}

 safe void gun(ref RCSlice!int s)
{
     fun(s, s);
}

Assume the reference count of s is 1 upon entering gun. Then the first 
thing opAssign does is to call the destructor of the slice, which 
deallocates memory. Subsequently the other reference (b) is dangling.


Andrei
Nov 02 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/02/2016 11:32 AM, Andrei Alexandrescu wrote:

 On 11/02/2016 07:29 AM, Nick Treleaven wrote:

 On Wednesday, 2 November 2016 at 05:00:23 UTC, Andrei Alexandrescu wrote:

 In order to make opAssign safe, a language change will be necessary.


 Technically, it should be possible with runtime checks:

 https://forum.dlang.org/post/aeeffshzkfjbrejztksj forum.dlang.org

 The checking overheads disappear when -noboundschecks is passed. The
 user has to manually copy the RCSlice when necessary for correct code.


 It seems opAssign for RCSlice is unsafe, is that right? Consider (with
 your codebase):

  safe void fun(ref RCSlice!int a, ref RCSlice!int b)
 {
     a = RCSlice!int();
     ... use b ...
 }

  safe void gun(ref RCSlice!int s)
 {
     fun(s, s);
 }

 Assume the reference count of s is 1 upon entering gun. Then the first
 thing opAssign does is to call the destructor of the slice, which
 deallocates memory. Subsequently the other reference (b) is dangling.


Ah, never mind, the two names refer to the same object. -- Andrei
Nov 02 2016
prev sibling next sibling parent reply anonymous <anon anon.au> writes:
On Wednesday, 2 November 2016 at 05:00:23 UTC, Andrei 
Alexandrescu wrote:

 I've eliminated all UTF nonsense from 
 https://github.com/dlang/phobos/pull/4878 resulting in a bare 
 reference counted buffer of (qualified) ubyte.


 Andrei


BTW about this PR: why RCString an not more generally RCArray ?
Nov 02 2016
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/2/16 7:42 AM, anonymous wrote:

 BTW about this PR: why RCString an not more generally RCArray ?


RCArray comes on top of RCBuffer. That way we isolate the matter of 
reference counting the buffer itself from the matter of managing the 
objects stored in the buffer. It's a simple matter of modularity. 
(Matters of alignment make this a bit more involved than it might seem, 
but nothing crazy.)

What is important (and probably unprecedented) about this PR is that 
it's the first reference counted artifact that works with qualifiers 
without resorting to undefined casts. That's the Big Thing about it. It 
harkens back to Dicebot idea to stash the reference counter in the 
allocator.


Andrei
Nov 02 2016
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/02/2016 01:00 AM, Andrei Alexandrescu wrote:

 In order to make opAssign safe, a language change will be necessary.


Take that back, no need for a language change as long as nobody returns 
references to refcounted memory.

Just pushed a commit to add  safe-ty annotations everywhere. Works as 
advertised! So now we have a buffer of ubyte that is reference counted, 
safe, and supports qualifiers properly (no shared yet).

https://github.com/dlang/phobos/pull/4878


Andrei
Nov 02 2016