www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Is borrow checker the right thing for D?

reply IGotD- <nise nise.com> writes:
This is regarding the blog post

https://dlang.org/blog/2019/07/15/ownership-and-borrowing-in-d/

and the associated DIPs.

What I'm worried about here is how this change would change the 
aesthetics and usability of D. Right now it is Java like, the 
garbage collector takes care of the memory and you can focus on 
being productive. Rust is a fad right now with its borrow checker 
memory model, but I think it is a fad because it is too limiting. 
The irony of the Rust model is that I often see people revert to 
reference counted objects when they cannot satisfy the borrow 
checker. Satisfying the borrow checker often makes you rethink 
the algorithm and sometimes that isn't even possible.

However, if the borrow checker is a step towards transparent and 
solid reference counting, then it would be a good thing. Why 
isn't reference counting considered as the automatic memory 
management in D? It is easy to understand, it fits with nearly 
all existing algorithms, it fits well with multi threading, it 
has no GC stalls. I would rather use the static analysis for the 
reference counting, for example optimizing away unnecessary 
increase/decreases of the RC.

There is a reason I use D and not Rust and that is that D is not 
Rust. I rather have the more pragmatic Java way of doing memory 
management. When I want to think about it, I'll just use manual 
memory management.

What do you think about refocusing the memory management efforts 
towards solid RC instead?
Aug 15
next sibling parent reply bachmeier <no spam.net> writes:
On Thursday, 15 August 2019 at 19:53:59 UTC, IGotD- wrote:
 This is regarding the blog post

 https://dlang.org/blog/2019/07/15/ownership-and-borrowing-in-d/

 and the associated DIPs.

 What I'm worried about here is how this change would change the 
 aesthetics and usability of D. Right now it is Java like, the 
 garbage collector takes care of the memory and you can focus on 
 being productive.
I don't think the GC will ever be removed. My understanding is that the option would be available for those that want the whole OB thing. I'd immediately move to another language if D wasn't a GC language.
 What do you think about refocusing the memory management 
 efforts towards solid RC instead?
There have been many lengthy threads on this topic. To my knowledge, they have never arrived at a satisfactory solution, but I can't say I've followed every detail of the discussions.
Aug 15
parent IGotD- <nise nise.com> writes:
On Thursday, 15 August 2019 at 20:34:03 UTC, bachmeier wrote:
 I don't think the GC will ever be removed. My understanding is 
 that the option would be available for those that want the 
 whole OB thing. I'd immediately move to another language if D 
 wasn't a GC language.
Isn't there a risk that would fragment the language like C++ where there are 11 ways to do the same thing. The preferred way seems change every year or so. Also wouldn't it lead to incompatibility issues when it comes libraries?
Aug 15
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 15 August 2019 at 19:53:59 UTC, IGotD- wrote:
 This is regarding the blog post

 https://dlang.org/blog/2019/07/15/ownership-and-borrowing-in-d/

 and the associated DIPs.

 What I'm worried about here is how this change would change the 
 aesthetics and usability of D. Right now it is Java like, the 
 garbage collector takes care of the memory and you can focus on 
 being productive.
Yes. And no. Java only has primitives as value types, and you can't take their address. Everything else is on the heap (not necessarily in practice, but conceptually, yes), so the GC is enough for safety. D has structs, pointers, global variables: lots of ways to screw up that the GC can't help with. That's where DIP1000 et al come in. I agree that D should be easier to use than Rust (it already is), and that the GC should be the default way of handling memory. We don't have any plans to change that. But, right now, it's possible to corrupt memory in D without the compiler telling you, and that's not a thing we want.
Aug 16
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 16.08.19 11:34, Atila Neves wrote:
 But, right now, it's possible to corrupt memory in D without the 
 compiler telling you,
It's important to note that the reason why this happens is that there is trusted code that doesn't account for all ways safe code can access its interface. I.e., it's bad trusted code, and inexpressive type system, not unsound type system. The only ways to fix this are 1) add more runtime checking to trusted code and don't directly expose mutable references where this is problematic. 2) give more type system tools to trusted code to restrict how safe code may access its interface, so exposing mutable references directly is valid in more circumstances.
 and that's not a thing we want.
None of the borrow/ownership proposals in the pipeline do anything that would allow trusted code to restrict the set of possible access patterns from safe code. They are a form of Rust cargo cult.
Aug 17
next sibling parent reply victoroak <anyone any.com> writes:
On Saturday, 17 August 2019 at 12:09:19 UTC, Timon Gehr wrote:
 
 [...]

 None of the borrow/ownership proposals in the pipeline do 
 anything that would allow  trusted code to restrict the set of 
 possible access patterns from  safe code. They are a form of 
 Rust cargo cult.
As someone who likes Rust, I like the idea of having just one mutable reference to a strict. I'm no language designer but it makes sense to try to borrow a model that already works in another language. I do think that Rust has a lot more than just one mutable reference that contributes to the safety and maybe it does not makes sense for D to do that. The problem is that I can't think any way besides returning a refcounted pointer in opIndex to make a safe Vector that has a reserve method or a insert that reallocate. If you have a better way that does not involve copying Rust I think you could let us know.
Aug 17
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 17.08.19 14:36, victoroak wrote:
 On Saturday, 17 August 2019 at 12:09:19 UTC, Timon Gehr wrote:
 [...]

 None of the borrow/ownership proposals in the pipeline do anything 
 that would allow  trusted code to restrict the set of possible access 
 patterns from  safe code. They are a form of Rust cargo cult.
As someone who likes Rust,
Rust is a fine language.
 I like the idea of having just one mutable 
 reference to a strict. I'm no language designer but it makes sense to 
 try to borrow a model that already works in another language.
Not without understanding what it does and why it works, and what features it may not be (easily) compatible with. https://en.wikipedia.org/wiki/Cargo_cult E.g., this "plane" will not fly: http://erwin.bernhardt.net.nz/oceania/images/vanuatutannacargocult02.jpg It superficially looks like a plane, but it does not actually have the same function.
 I do think 
 that Rust has a lot more than just one mutable reference that 
 contributes to the safety and maybe it does not makes sense for D to do 
 that.
 ...
If any of the proposals would in fact guarantee that at any time there was either one mutable access path or an arbitrary amount of non-mutable access paths to each value, I wouldn't call them a cargo cult. But that still wouldn't necessarily make sense, because safe D code can access raw pointers, while safe Rust code can't. (This is because D has a built-in garbage collector. In Rust, GC is not entirely trivial to make safe, useful and efficient at the same time, and they haven't settled on a solution.)
 The problem is that I can't think any way besides returning a refcounted 
 pointer in opIndex to make a safe Vector that has a reserve method or a 
 insert that reallocate.
 ...
As I said, there is no actual proposal in the pipeline that allows making a safe vector of the kind that Rust has, even though there are proposals that superficially make the D type system look more similar to Rust's to a casual observer. I keep pointing it out but so far this has not been acknowledged as a fatal flaw of those proposals.
 If you have a better way that does not involve copying Rust
(To get a _copy_ of Rust, download rustc. Anything else requires some language design expertise.) I don't understand why you think I think using Rust for inspiration is a bad thing. I'm not opposed to doing what Rust does. Rust _actually does some version of what I suggested in my last post_. The problem is that it seems that very few people actively participating on this forum actually understand what it is that Rust does. (E.g., a very common misconception is that Rust unsafe is like D system. Rust unsafe is actually like D trusted. Another misconception is that Rust ensures that raw mutable pointers don't alias.)
 I think you could let us know.
We first need to agree on what the actual problem is before discussing solutions. And once we do, I don't care whether we (actually!) do what Rust does or whether we instead settle for fractional permissions implemented in a super-general dependently-typed typestate system. The only thing that's important to me is that the solution works and doesn't have obvious design flaws. There are existing proposals being considered that fail those criteria, and that just shouldn't happen. Pointing that out is a meaningful contribution, but it takes away some of my spare time from more useful contributions such as participating in a working group that designs solutions that actually work. However, such more useful endeavors can't be fruitful (and I'm not betting my precious spare time on them) unless decision makers actually acquire _some_ expertise on this subject matter.
Aug 17
next sibling parent Russel Winder <russel winder.org.uk> writes:
On Sat, 2019-08-17 at 22:42 +0200, Timon Gehr via Digitalmars-d wrote:
 On 17.08.19 14:36, victoroak wrote:
 On Saturday, 17 August 2019 at 12:09:19 UTC, Timon Gehr wrote:
 [...]
=20
 None of the borrow/ownership proposals in the pipeline do anything=
=20
 that would allow  trusted code to restrict the set of possible access=
=20
 patterns from  safe code. They are a form of Rust cargo cult.
=20 As someone who likes Rust,
=20 Rust is a fine language.
I spend most of my programming time using Rust. As with any other programmi= ng language it has its pluses and minuses, but overall I like it. But you do h= ave to admit that Rust programming is a Cargo cult:=20 https://doc.rust-lang.org/cargo/ Just as many would have D programming be a Dub cult:=20 https://en.wikipedia.org/wiki/Dub_music oh sorry, of course I meant:=20 https://dub.pm/getting_started :-) --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Aug 18
prev sibling parent reply victoroak <anyone any.com> writes:
On Saturday, 17 August 2019 at 20:42:46 UTC, Timon Gehr wrote:
 I don't understand why you think I think using Rust for 
 inspiration is a bad thing. I'm not opposed to doing what Rust 
 does. Rust _actually does some version of what I suggested in 
 my last post_. The problem is that it seems that very few 
 people actively participating on this forum actually understand 
 what it is that Rust does. (E.g., a very common misconception 
 is that Rust unsafe is like D  system. Rust unsafe is actually 
 like D  trusted. Another misconception is that Rust ensures 
 that raw mutable pointers don't alias.)
Rust do have unsafe functions that can only be called inside unsafe blocks so it's like system functions in D. But I agree with what you are saying, we need a better understanding on the subject before trying to push it into D. Sorry for misunderstanding what you said before.
Aug 18
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 18.08.19 14:51, victoroak wrote:
 On Saturday, 17 August 2019 at 20:42:46 UTC, Timon Gehr wrote:
 I don't understand why you think I think using Rust for inspiration is 
 a bad thing. I'm not opposed to doing what Rust does. Rust _actually 
 does some version of what I suggested in my last post_. The problem is 
 that it seems that very few people actively participating on this 
 forum actually understand what it is that Rust does. (E.g., a very 
 common misconception is that Rust unsafe is like D  system. Rust 
 unsafe is actually like D  trusted. Another misconception is that Rust 
 ensures that raw mutable pointers don't alias.)
Rust do have unsafe functions that can only be called inside unsafe blocks so it's like system functions in D.
This is true of course. What I mean is some people think unsafe _blocks_ are like system in D, even though they are Rust's version of trusted. Thanks!
Aug 18
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Saturday, 17 August 2019 at 12:09:19 UTC, Timon Gehr wrote:
 On 16.08.19 11:34, Atila Neves wrote:
 But, right now, it's possible to corrupt memory in D without 
 the compiler telling you,
It's important to note that the reason why this happens is that there is trusted code that doesn't account for all ways safe code can access its interface. I.e., it's bad trusted code, and inexpressive type system, not unsound type system. The only ways to fix this are 1) add more runtime checking to trusted code and don't directly expose mutable references where this is problematic. 2) give more type system tools to trusted code to restrict how safe code may access its interface, so exposing mutable references directly is valid in more circumstances.
 and that's not a thing we want.
None of the borrow/ownership proposals in the pipeline do anything that would allow trusted code to restrict the set of possible access patterns from safe code. They are a form of Rust cargo cult.
So you've mentioned, but I'm afraid I didn't understand the argument (it didn't help it was over several emails). I would really appreciate an explanation of why you think the current proposals are inadequate, and what we could do instead. Thanks!
Aug 19
prev sibling parent Russel Winder <russel winder.org.uk> writes:
On Thu, 2019-08-15 at 19:53 +0000, IGotD- via Digitalmars-d wrote:
 This is regarding the blog post
=20
 https://dlang.org/blog/2019/07/15/ownership-and-borrowing-in-d/
=20
 and the associated DIPs.
=20
 What I'm worried about here is how this change would change the=20
 aesthetics and usability of D. Right now it is Java like, the=20
 garbage collector takes care of the memory and you can focus on=20
 being productive. Rust is a fad right now with its borrow checker=20
 memory model, but I think it is a fad because it is too limiting.=20
 The irony of the Rust model is that I often see people revert to=20
 reference counted objects when they cannot satisfy the borrow=20
 checker. Satisfying the borrow checker often makes you rethink=20
 the algorithm and sometimes that isn't even possible.
Or maybe they switched to using reference counted objects because trying to use stack based objects was the wrong way of doing what they were trying to do. I had this problem a year or so ago with part of Me TV in Rust. The problem was not the borrow checker, the problem was I should have been usin= g heap objects not stack object for the things the borrow checker was complaining about. For managing stack objects, the borrow checker is a huge boon.
 However, if the borrow checker is a step towards transparent and=20
 solid reference counting, then it would be a good thing. Why=20
 isn't reference counting considered as the automatic memory=20
 management in D? It is easy to understand, it fits with nearly=20
 all existing algorithms, it fits well with multi threading, it=20
 has no GC stalls. I would rather use the static analysis for the=20
 reference counting, for example optimizing away unnecessary=20
 increase/decreases of the RC.
Aren't the borrow checker and reference counting two separate domains of memory management? The Rust borrow checker is about stack management, reference counting is about heap management. Reference counting works well in Python, garbage collection works well in G= o. I find garbage collection works well in D, why is reference counting needed= . Rust of course uses reference counting (or atomic reference counting). Reference counting is a runtime activity, is it even feasible to do it at compile time?
 There is a reason I use D and not Rust and that is that D is not=20
 Rust. I rather have the more pragmatic Java way of doing memory=20
 management. When I want to think about it, I'll just use manual=20
 memory management.
=20
 What do you think about refocusing the memory management efforts=20
 towards solid RC instead?
Rust chose reference counting for a reason. Go chose garbage collection for= a reason. Java chose garbage collection for a reason. D having garbage collection seems fine =E2=80=93 except that the GC of D seems highly unsoph= isticated compared to those of Go and Java, but I am told that this is of necessity. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Aug 16