www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP74 - where is at?

reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
So I have another upcoming opportunity to introduce D in my workplace,
this time as a front-end/plugin language to our C++ infrastructure,
which is promising since I already have considerable experience in
this area (my work at Remedy with Quantum Break), and there is a lot
of recent work to interact better with C++, which we will stress-test
extensively.

You only get so many shots at this; but this is a particularly
promising opportunity, since the C++ code is a nightmare, and the
contrast against D will allow a lot of coders to see the advantage.

There is however one critical missing feature, DIP74... where is it at
currently? How is it going? Is it likely to be accepted in the
near-term? Some sort of approximate timeline?

I think it would be a mistake for me to introduce this without DIP74,
since we will rely on it VERY heavily, and the machinery to
work-around it will start to look just as heavy-weight as the C++ code
I'm trying to deprecate... but then waiting on it starts to look like
missing the window of opportunity.

Thoughts?
Oct 10 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 So I have another upcoming opportunity to introduce D in my 
 workplace, this time as a front-end/plugin language to our C++ 
 infrastructure, which is promising since I already have 
 considerable experience in this area (my work at Remedy with 
 Quantum Break), and there is a lot of recent work to interact 
 better with C++, which we will stress-test extensively.

 You only get so many shots at this; but this is a particularly 
 promising opportunity, since the C++ code is a nightmare, and 
 the contrast against D will allow a lot of coders to see the 
 advantage.

 There is however one critical missing feature, DIP74... where 
 is it at currently? How is it going? Is it likely to be 
 accepted in the near-term? Some sort of approximate timeline?

 I think it would be a mistake for me to introduce this without 
 DIP74, since we will rely on it VERY heavily, and the machinery 
 to work-around it will start to look just as heavy-weight as 
 the C++ code I'm trying to deprecate... but then waiting on it 
 starts to look like missing the window of opportunity.

 Thoughts?


It doesn't looks like it is getting implemented. And, to be 
honest, I'd rather go a principle approach + library support 
rather than a pie of hacks.

The pile of hacks approach is what made C++ C++.
Oct 10 2015
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
I'm rather in favour of DIP74... what's unprincipled about it? What
would you do instead?

On 11 October 2015 at 10:20, deadalnix via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 So I have another upcoming opportunity to introduce D in my workplace,
 this time as a front-end/plugin language to our C++ infrastructure, which is
 promising since I already have considerable experience in this area (my work
 at Remedy with Quantum Break), and there is a lot of recent work to interact
 better with C++, which we will stress-test extensively.

 You only get so many shots at this; but this is a particularly promising
 opportunity, since the C++ code is a nightmare, and the contrast against D
 will allow a lot of coders to see the advantage.

 There is however one critical missing feature, DIP74... where is it at
 currently? How is it going? Is it likely to be accepted in the near-term?
 Some sort of approximate timeline?

 I think it would be a mistake for me to introduce this without DIP74,
 since we will rely on it VERY heavily, and the machinery to work-around it
 will start to look just as heavy-weight as the C++ code I'm trying to
 deprecate... but then waiting on it starts to look like missing the window
 of opportunity.

 Thoughts?



 It doesn't looks like it is getting implemented. And, to be honest, I'd
 rather go a principle approach + library support rather than a pie of hacks.

 The pile of hacks approach is what made C++ C++.
Oct 10 2015
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 01:48:05 UTC, Manu wrote:

 I'm rather in favour of DIP74... what's unprincipled about it? 
 What would you do instead?


Well, DIP25 and DIP74 are ad hoc adding to the language to 
support specific use cases. I think the whole thing is wrong 
headed.

Ideally when designing the language you want to have a set of 
principled tools added to the language that enable the uses cases 
you want. You don't bake the uses cases into the language. 
Inevitably, new use cases crop up, they get poor support and/or 
ends up as new addition into the language, with more and more 
complexity along the road.

The problem at hand is fairly well know at this stage: it is 
ownership. Everything else can be done as library.

The DIP25/DIP74 combo is quite limited. It doesn't solve 
ownership transfer, it forces an ownership model on the user, it 
doesn't reduce friction with the GC, it doesn't allow to delegate 
memory management to the user (the only kind that make sense). 
and worse, as these issue aren't orthogonal to these DIP, they 
actively make these problem harder to solve.

Solution have been proposed to solve these. It is not hardly new. 
I mean C++ obviously got the hint: 
https://github.com/isocpp/CppCoreGuidelines/blob/09aef9bd86d933bc1e1ffe344eb2e73d2de15685/docs/Lifetimes%20I%20and%20II%20-%20v0.9.1.pdf

If we go these DIP road, there is no coming back and this will 
get in the way of a principled approach.
Oct 10 2015
next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote:

 If we go these DIP road, there is no coming back and this will 
 get in the way of a principled approach.


Then come up with an alternative DIP which shows a better way to 
solve this. As it stands, it looks likely that we'll end up with 
some form of DIP 74, and if you have a better proposal, then now 
is probably the time to do it.

Personally, I obviously haven't been following this closely 
enough, because I don't understand why something like RefCounted 
can't be made to do what we need with regards to reference 
counting and classes. It does get a bit nasty when inheritance 
and whatnot get involved, but C++ was able to solve that easily 
enough, and we should be able to do the same.

The one place in-language where I'm sure that something like 
RefCounted doesn't do it is exceptions, since we really should 
have a way to make those reference counted, but you can only 
throw something derived from Throwable - which means a class and 
not a wrapper around one. So, we need a tweak of some kind to the 
language there, but that's pretty specific, whereas it _should_ 
be possible to get something like RefCounted to at least solve 
the normal cases. Clearly though, Walter and Andrei have come to 
the conclusion that it's not.

- Jonathan M Davis
Oct 10 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 04:35:03 UTC, Jonathan M Davis 
wrote:

 On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote:

 If we go these DIP road, there is no coming back and this will 
 get in the way of a principled approach.


 Then come up with an alternative DIP which shows a better way 
 to solve this. As it stands, it looks likely that we'll end up 
 with some form of DIP 74, and if you have a better proposal, 
 then now is probably the time to do it.


I have, other have as well, in fact there was a lengthy 
discussion in private between Walter, Andrei, myself and some 
other where very precise proposal has been made.

I'm sorry but that's bullshit. I'm tired of reexplaining the same 
thing again and again while is it clear that nobody cares about 
facts here. If people would care about facts, the DIP25 fiasco 
would have been enough to put the ideas back on the table.


 The one place in-language where I'm sure that something like 
 RefCounted doesn't do it is exceptions, since we really should 
 have a way to make those reference counted, but you can only 
 throw something derived from Throwable - which means a class 
 and not a wrapper around one. So, we need a tweak of some kind 
 to the language there, but that's pretty specific, whereas it 
 _should_ be possible to get something like RefCounted to at 
 least solve the normal cases. Clearly though, Walter and Andrei 
 have come to the conclusion that it's not.


Proposal included solution for this problem. Exception ownerhip 
is transferred to the runtime when thrown, and the runtime 
transfer it back to the catch block that effectively catches it.
Oct 10 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 04:56:25 UTC, deadalnix wrote:

 On Sunday, 11 October 2015 at 04:35:03 UTC, Jonathan M Davis 
 wrote:

 On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote:

 If we go these DIP road, there is no coming back and this 
 will get in the way of a principled approach.


 Then come up with an alternative DIP which shows a better way 
 to solve this. As it stands, it looks likely that we'll end up 
 with some form of DIP 74, and if you have a better proposal, 
 then now is probably the time to do it.


 I have, other have as well, in fact there was a lengthy 
 discussion in private between Walter, Andrei, myself and some 
 other where very precise proposal has been made.

 I'm sorry but that's bullshit. I'm tired of reexplaining the 
 same thing again and again while is it clear that nobody cares 
 about facts here. If people would care about facts, the DIP25 
 fiasco would have been enough to put the ideas back on the 
 table.


Well, if they won't listen, they won't listen. And if they're 
wrong, we'll be worse off for it. Unfortunately, I wasn't 
involved in those discussions and haven't looked into DIP 25 much 
(I was too busy at the time of the major discussion for it IIRC). 
So, I'm not familiar enough with it to have a properly informed 
opinion. But convincing Walter and Andrei is typically pretty 
difficult. They do come around eventually at least some of the 
time though. So, as frustrating as such discussions can be, they 
do bear fruit at least some of the time (whether it's by them 
convincing you or by you convincing them). And since DIP 25 has 
only been implemented with a flag rather than adding it to the 
language proper yet, there's still time to convince them before 
we're fully committed to it - as difficult as convincing them may 
be.

- Jonathan M Davis
Oct 10 2015
parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 05:18:26 UTC, Jonathan M Davis 
wrote:

 Well, if they won't listen, they won't listen. And if they're 
 wrong, we'll be worse off for it. Unfortunately, I wasn't 
 involved in those discussions and haven't looked into DIP 25 
 much (I was too busy at the time of the major discussion for it 
 IIRC). So, I'm not familiar enough with it to have a properly 
 informed opinion. But convincing Walter and Andrei is typically 
 pretty difficult. They do come around eventually at least some 
 of the time though. So, as frustrating as such discussions can 
 be, they do bear fruit at least some of the time (whether it's 
 by them convincing you or by you convincing them). And since 
 DIP 25 has only been implemented with a flag rather than adding 
 it to the language proper yet, there's still time to convince 
 them before we're fully committed to it - as difficult as 
 convincing them may be.

 - Jonathan M Davis


Truth be told at the time I was doubtful a DIP25 like approach, 
but willing to give it a try. They had me convinced of that much. 
Maybe that wasn't that powerful, but maybe that was powerful 
enough for basics uses cases and would cut it.

Seeing what comes out of it confirmed that no, it won't, and once 
you get DIP25 + DIP74 you are in the same ballpark as other 
proposals in term of language additions, but barley measure in 
terms of capabilities.
Oct 10 2015
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 14:35, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote:

 If we go these DIP road, there is no coming back and this will get in the
 way of a principled approach.



 Then come up with an alternative DIP which shows a better way to solve this.
 As it stands, it looks likely that we'll end up with some form of DIP 74,
 and if you have a better proposal, then now is probably the time to do it.

 Personally, I obviously haven't been following this closely enough, because
 I don't understand why something like RefCounted can't be made to do what we
 need with regards to reference counting and classes. It does get a bit nasty
 when inheritance and whatnot get involved, but C++ was able to solve that
 easily enough, and we should be able to do the same.


C++ didn't solve anything(?). C++ doesn't support ref-counting at all!
shared_ptr is not a part of the language, or a proper ref counting
mechanism. It's just a hack; it's awkward, and really inefficient (the
compiler can't optimise it).
ARC requires language knowledge. I don't know what language primitives
can possibly allow the compiler to do proper ref fiddling optimisation
with a lib?
Oct 10 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 06:55:50 UTC, Manu wrote:

 On 11 October 2015 at 14:35, Jonathan M Davis via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote:

 If we go these DIP road, there is no coming back and this 
 will get in the way of a principled approach.



 Then come up with an alternative DIP which shows a better way 
 to solve this. As it stands, it looks likely that we'll end up 
 with some form of DIP 74, and if you have a better proposal, 
 then now is probably the time to do it.

 Personally, I obviously haven't been following this closely 
 enough, because I don't understand why something like 
 RefCounted can't be made to do what we need with regards to 
 reference counting and classes. It does get a bit nasty when 
 inheritance and whatnot get involved, but C++ was able to 
 solve that easily enough, and we should be able to do the same.


 C++ didn't solve anything(?). C++ doesn't support ref-counting 
 at all!
 shared_ptr is not a part of the language, or a proper ref 
 counting
 mechanism. It's just a hack; it's awkward, and really 
 inefficient (the
 compiler can't optimise it).
 ARC requires language knowledge. I don't know what language 
 primitives
 can possibly allow the compiler to do proper ref fiddling 
 optimisation
 with a lib?


Ref-counting with shared_ptr works just fine. It just doesn't 
optimize out any of the incrementing or decrementing. And while 
having those optimized out would be nice, I don't think that in 
and of itself that makes it worth having ref-counting in the 
language rather than in the library.

Maybe C++ ref-counting hasn't work well for your use cases, but 
I've used it for years, and it's worked great for what I've 
needed.

- Jonathan M Davis
Oct 11 2015
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 17:16, Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 06:55:50 UTC, Manu wrote:

 On 11 October 2015 at 14:35, Jonathan M Davis via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote:

 If we go these DIP road, there is no coming back and this will get in
 the way of a principled approach.




 Then come up with an alternative DIP which shows a better way to solve
 this. As it stands, it looks likely that we'll end up with some form of DIP
 74, and if you have a better proposal, then now is probably the time to do
 it.

 Personally, I obviously haven't been following this closely enough,
 because I don't understand why something like RefCounted can't be made to do
 what we need with regards to reference counting and classes. It does get a
 bit nasty when inheritance and whatnot get involved, but C++ was able to
 solve that easily enough, and we should be able to do the same.



 C++ didn't solve anything(?). C++ doesn't support ref-counting at all!
 shared_ptr is not a part of the language, or a proper ref counting
 mechanism. It's just a hack; it's awkward, and really inefficient (the
 compiler can't optimise it).
 ARC requires language knowledge. I don't know what language primitives
 can possibly allow the compiler to do proper ref fiddling optimisation
 with a lib?



 Ref-counting with shared_ptr works just fine. It just doesn't optimize out
 any of the incrementing or decrementing.


I wouldn't call that 'working', I'd call that 'grossly over-working' at best.
I'd personally diagnose signals firing at improper times to be a bug,
and that's effectively what we have ;)

I think it's absolutely worthwhile that a language should know that
refcounting is a thing, and what the proper handling is. I don't know
any way to model it accurately without language support.


 And while having those optimized
 out would be nice, I don't think that in and of itself that makes it worth
 having ref-counting in the language rather than in the library.


For real? If you're handling ref-counted objects, ref-fiddling often
completely overwhelms your code at the interface level, and god forbid
you have some sort of adaptation layers, shims, small temporaries,
call-throughs, or iterations.
Problem is amplified big time if your ref-counting functions are
foreign library functions, which is pretty common in C api's. These
functions need to be wrapped with a primitive that properly knows when
the functions should be called.

I'm seeing objects with RC==8+ or so across one event handling call-tree.


 Maybe C++ ref-counting hasn't work well for your use cases, but I've used it
 for years, and it's worked great for what I've needed.


Sure. If you don't care, I'm sure it's fine. But I don't feel it's
reasonable to say C++ has ref counting. You might as well say "C++ has
garbage collection" (which is probably actually more true than this)
;) ... Objective-C supports ref counting.
Oct 11 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 07:55:08 UTC, Manu wrote:

 Sure. If you don't care, I'm sure it's fine. But I don't feel 
 it's reasonable to say C++ has ref counting. You might as well 
 say "C++ has garbage collection" (which is probably actually 
 more true than this) ;) ... Objective-C supports ref counting.


Reference counting is now part of C++'s standard library. They 
may not have the kind of reference counting that you want, but 
they support reference counting as part of the C++ standard.

And certainly for my use cases, having wrapper classes like 
smart_ptr has worked fantastically. Any performance hit that 
might be incurred be extraneous increments and decrements is 
dwarfed by everything else that the program is doing. And I think 
that it's pretty clear that the same is true for a _lot_ of 
programs.

I can certainly believe that it's not true for the kind of 
programs that you write, and if we can better support those 
environments, great. But most programmers don't program in a 
world where it normally matters whether a couple of extra math 
operations are done or whether a function call is virtual or not. 
It matters in certain areas of the program but not in general. We 
don't want to be doing unnecessary operations, and efficiency in 
other areas can have a huge impact on our programs, but I think 
that it's quite clear that you live in a world that cares way 
more about every little ounce of efficiency than most programmers 
do.

So, I think that it's a huge stretch to say that C++ doesn't have 
ref-counting or that it doesn't work - or even that a similar 
solution in D wouldn't work. It would work for many people. If we 
can come up with one that better supports your use cases, then 
great. As long as the cost elsewhere isn't too high, why not? But 
that doesn't mean that a solution like smart_ptr isn't a 
fantastic one for many of us.

- Jonathan M Davis
Oct 11 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 12 Oct 2015 7:31 am, "Jonathan M Davis via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 07:55:08 UTC, Manu wrote:

 Sure. If you don't care, I'm sure it's fine. But I don't feel it's




reasonable to say C++ has ref counting. You might as well say "C++ has
garbage collection" (which is probably actually more true than this) ;) ...
Objective-C supports ref counting.

 Reference counting is now part of C++'s standard library. They may not


have the kind of reference counting that you want, but they support
reference counting as part of the C++ standard.

 And certainly for my use cases, having wrapper classes like smart_ptr has


worked fantastically. Any performance hit that might be incurred be
extraneous increments and decrements is dwarfed by everything else that the
program is doing. And I think that it's pretty clear that the same is true
for a _lot_ of programs.

 I can certainly believe that it's not true for the kind of programs that


you write, and if we can better support those environments, great. But most
programmers don't program in a world where it normally matters whether a
couple of extra math operations are done or whether a function call is
virtual or not. It matters in certain areas of the program but not in
general. We don't want to be doing unnecessary operations, and efficiency
in other areas can have a huge impact on our programs, but I think that
it's quite clear that you live in a world that cares way more about every
little ounce of efficiency than most programmers do.

 So, I think that it's a huge stretch to say that C++ doesn't have


ref-counting or that it doesn't work - or even that a similar solution in D
wouldn't work. It would work for many people. If we can come up with one
that better supports your use cases, then great. As long as the cost
elsewhere isn't too high, why not? But that doesn't mean that a solution
like smart_ptr isn't a fantastic one for many of us.

 - Jonathan M Davis


Incidentally, I tried to use shared_ptr initially, but it took about 20
minutes before I realised I had to pass an rc pointer from a method...
Since rc is a wrapper, like you said above, you lose it as soon as you're
within a method. I then had to write an invasive rc implementation and hard
create a new rc instance from 'this', all over the place. Seriously, I
don't understand how anyone can say shared_ptr is a success. It's a massive
kludge, and it alone has me firmly convinced that rc is weak, inconvenient,
and highly complicated without language support.
Oct 11 2015
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 23:08:58 UTC, Manu wrote:

 Incidentally, I tried to use shared_ptr initially, but it took 
 about 20 minutes before I realised I had to pass an rc pointer 
 from a method... Since rc is a wrapper, like you said above, 
 you lose it as soon as you're within a method.


And here you have it. You asked earlier how DIP25 and DIP74 
connect, and this is where. If you want to ensure reference 
counting safety, you must make sure that reference do not escape 
in uncontrolled manner.


 I then had to write an invasive rc implementation and hard 
 create a new rc instance from 'this', all over the place. 
 Seriously, I don't understand how anyone can say shared_ptr is 
 a success. It's a massive kludge, and it alone has me firmly 
 convinced that rc is weak, inconvenient, and highly complicated 
 without language support.


By C++'s standards, this is not that messy.
Oct 11 2015
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 23:08:58 UTC, Manu wrote:

 Incidentally, I tried to use shared_ptr initially, but it took 
 about 20 minutes before I realised I had to pass an rc pointer 
 from a method... Since rc is a wrapper, like you said above, 
 you lose it as soon as you're within a method. I then had to 
 write an invasive rc implementation and hard create a new rc 
 instance from 'this', all over the place. Seriously, I don't 
 understand how anyone can say shared_ptr is a success. It's a 
 massive kludge, and it alone has me firmly convinced that rc is 
 weak, inconvenient, and highly complicated without language 
 support.


If you use shared_ptr, what typically happens is that you use 
shared_ptr everywhere and naked pointers pretty much disappear 
from your code. The main problem is when a type needs to return a 
pointer to itself, but in my own code, I've found that need to be 
rare (if anything, I'm more likely to pass an object out of its 
own member function by &, not by pointer). The primary 
counter-example is when a class that it owns needs to have access 
to it, but then in most cases, that class doesn't need to pass 
that pointer on to anyone else, and since it lives for as long as 
its parent does, it doesn't have to worry about ref-counting. I 
suppose that using & would work just as well in those cases, 
though I don't tend to use & much outside of passing into a 
function and returning out of it (generally when having a copy is 
okay but I want it to be possible to avoid it), because it's too 
easy to make inadvertent copies.

Having ref-counting built into the type does solve some of those 
problems, but it can cause others. I doubt that there was ever 
any chance that the C++ standards committee would have ever gone 
with built-in ref-counting though given how invasive that is in 
the class hierarchy and how it would not work very well with 
existing code. In our case, with classes, built-in ref-counting 
like DIP 74 proposes is probably better.

- Jonathan M Davis
Oct 11 2015
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 11 October 2015 at 23:08:58 UTC, Manu wrote:

 Incidentally, I tried to use shared_ptr initially, but it took 
 about 20 minutes before I realised I had to pass an rc pointer 
 from a method... Since rc is a wrapper, like you said above, 
 you lose it as soon as you're within a method. I then had to 
 write an invasive rc implementation and hard create a new rc 
 instance from 'this', all over the place. Seriously, I don't 
 understand how anyone can say shared_ptr is a success. It's a 
 massive kludge, and it alone has me firmly convinced that rc is 
 weak, inconvenient, and highly complicated without language 
 support.


Shared_ptr and unique_ptr represent ownership, not references. So 
you should not pass them around unless you transfer ownership.

When the owner allows someone to borrow a reference, that 
borrowing point has to ensure that the reference does not outlive 
the lifespan of the object. This is not so hard to get right.

shared_ptr has several advantages:

1. It works with existing classes (from foreign libraries) 
without any notion of RC.

2. It allows weak references and cycles without keeping the whole 
object allocated.

3. It supports concurrency.

5. The reference counting mechanism can be done in a way that 
fits with polymorphism since it does not make any assumptions 
about the structure of object itself.

6. The object itself can remain fully immutable.

Disadvantages:

1. double indirection

2. two allocations
Oct 11 2015
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/11/15 10:16 AM, Jonathan M Davis wrote:

 On Sunday, 11 October 2015 at 06:55:50 UTC, Manu wrote:

 On 11 October 2015 at 14:35, Jonathan M Davis via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote:

 If we go these DIP road, there is no coming back and this will get
 in the way of a principled approach.



 Then come up with an alternative DIP which shows a better way to
 solve this. As it stands, it looks likely that we'll end up with some
 form of DIP 74, and if you have a better proposal, then now is
 probably the time to do it.

 Personally, I obviously haven't been following this closely enough,
 because I don't understand why something like RefCounted can't be
 made to do what we need with regards to reference counting and
 classes. It does get a bit nasty when inheritance and whatnot get
 involved, but C++ was able to solve that easily enough, and we should
 be able to do the same.


 C++ didn't solve anything(?). C++ doesn't support ref-counting at all!
 shared_ptr is not a part of the language, or a proper ref counting
 mechanism. It's just a hack; it's awkward, and really inefficient (the
 compiler can't optimise it).
 ARC requires language knowledge. I don't know what language primitives
 can possibly allow the compiler to do proper ref fiddling optimisation
 with a lib?


 Ref-counting with shared_ptr works just fine. It just doesn't optimize
 out any of the incrementing or decrementing. And while having those
 optimized out would be nice, I don't think that in and of itself that
 makes it worth having ref-counting in the language rather than in the
 library.

 Maybe C++ ref-counting hasn't work well for your use cases, but I've
 used it for years, and it's worked great for what I've needed.


Yah, it's a success story and a good baseline. Two things that are 
suboptimal are the necessity of two allocations unless make_shared is 
used, and compulsive interlocked inc/dec. We ought to address both 
issues, but by and large we're looking at a successful experience that 
we want to have D users benefit from as well. -- Andrei
Oct 11 2015
prev sibling next sibling parent Freddy <Hexagonalstar64 gmail.com> writes:
On Sunday, 11 October 2015 at 04:16:11 UTC, deadalnix wrote:

 The problem at hand is fairly well know at this stage: it is 
 ownership. Everything else can be done as library.


This.
Oct 10 2015
prev sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 14:16, deadalnix via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 01:48:05 UTC, Manu wrote:

 I'm rather in favour of DIP74... what's unprincipled about it? What would
 you do instead?


 Well, DIP25 and DIP74 are ad hoc adding to the language to support specific
 use cases. I think the whole thing is wrong headed.


I don't really see that DIP25 and DIP74 are particularly closely related.
DIP25 is a lame cop-out with respect to scope. I was a major backer of
a proper scope implementation, and given that, I don't/didn't support
DIP25 at all.

DIP74 on the other hand introduces 2 magic functions that the compiler
calls as the appropriate moments to implement ref counting. The
implementation is somewhat detail, theoretically changeable in the
future, but I think language support for ref-counting primitives is
critical for ref counting to be efficient and simple. Look at C++, and
we see a disaster. C++ basically implemented rval-references to
improve (not solve) the RC problem... we don't have rval-ref's in D,
so we're screwed from the library angle.



 Ideally when designing the language you want to have a set of principled
 tools added to the language that enable the uses cases you want. You don't
 bake the uses cases into the language. Inevitably, new use cases crop up,
 they get poor support and/or ends up as new addition into the language, with
 more and more complexity along the road.

 The problem at hand is fairly well know at this stage: it is ownership.
 Everything else can be done as library.


Yup, I was firmly behind scope (ie, borrowing), but that was
destroyed, and DIP25 kinda cements the death of that :(

That said, this has nothing to do with ref-counting. Scheduling (and
optimising) calls to inc/dec is not something I've seen language tools
implement efficiently. The compiler needs to understand that it has
the authority to elide pairs appropriately, even when the optimiser is
unable to see the code that implements the inc/dec pair. I can't think
of any way to do this without a foundation like DIP74...?



 The DIP25/DIP74 combo is quite limited. It doesn't solve ownership transfer,
 it forces an ownership model on the user, it doesn't reduce friction with
 the GC, it doesn't allow to delegate memory management to the user (the only
 kind that make sense). and worse, as these issue aren't orthogonal to these
 DIP, they actively make these problem harder to solve.


How? I don't see how opInc/opDec really changes the underlying type
system or allocations mechanisms much, it just gives the compiler the
opportunity to understand the optimise reference counting, which is a
well understood problem.



 Solution have been proposed to solve these. It is not hardly new. I mean C++
 obviously got the hint:
 https://github.com/isocpp/CppCoreGuidelines/blob/09aef9bd86d933bc1e1ffe344eb2e73d2de15685/docs/Lifetimes%20I%20and%20II%20-%20v0.9.1.pdf

 If we go these DIP road, there is no coming back and this will get in the
 way of a principled approach.


DIP25 is done, I rejected it but it's done... I don't really see how
DIP74 ruins anything?
Oct 10 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/11/15 9:48 AM, Manu via Digitalmars-d wrote:

 C++ basically implemented rval-references to
 improve (not solve) the RC problem...


Interesting, haven't heard of this viewpoint. Could you please give 
detail on that? -- Andrei
Oct 11 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 17:10, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 10/11/15 9:48 AM, Manu via Digitalmars-d wrote:

 C++ basically implemented rval-references to
 improve (not solve) the RC problem...



 Interesting, haven't heard of this viewpoint. Could you please give detail
 on that? -- Andrei


Yeah, that is a terrible claim, but that do offer some advantage (in
this situation as with others).

The language doesn't support ref counting; constructors/destructors
are a pretty crude mechanism to implement them, since they don't model
the ref-counting pattern but a superset, which technically allows it
to work, but it's pretty lame and inefficient to do ref-counting this
way, and it's impossible for code (or the compiler) to distinguish
between actual ref-counting events and must behave conservatively.
rvalue-references allow the language to elide some unnecessary
assignment logic, which can be further used to perform some inc/dec
elision, but it starts to get extremely complex at this point, and it
still doesn't do the trick. It's a sort of band-aid, but certainly not
just for RC.
Oct 11 2015
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 06:48:54 UTC, Manu wrote:

 I don't really see that DIP25 and DIP74 are particularly 
 closely related.


Ho they are. It is all about ownership.


 DIP25 is a lame cop-out with respect to scope. I was a major 
 backer of
 a proper scope implementation, and given that, I don't/didn't 
 support
 DIP25 at all.


DIP25 being a sorry excuse for ownership is the very reason DIP74 
is needed. Rust can do reference counting just fine all in 
library without need for core language support.


 DIP74 on the other hand introduces 2 magic functions that the 
 compiler calls as the appropriate moments to implement ref 
 counting. The implementation is somewhat detail, theoretically 
 changeable in the future, but I think language support for 
 ref-counting primitives is critical for ref counting to be 
 efficient and simple. Look at C++, and we see a disaster. C++ 
 basically implemented rval-references to improve (not solve) 
 the RC problem... we don't have rval-ref's in D, so we're 
 screwed from the library angle.


C++ ref counting is not slow for the reasons you think it is. ARC 
is overly complex and also not as fast as you think it is.

First, they are slow because in C++ (or ObjC), you don't know 
what is shared and what isn't. As a result, you got to go the 
pessimistic road and use thread safe ref counting. This can be a 
solved problem in D purely using the type qualifier.

Second, exceptions. Their unwinding makes pair of inc/dec hard to 
recognize, plus now cleanup is need at pretty much every frames, 
meaning incredibly slow exceptions + various control flow 
optimization that aren't possible anymore (tail call being one of 
them).

One that one, nobody has a solution. Not ARC (that's why swift 
does not have exception), not C++ and not us. IMO the obvious 
solution here is to allow ourselves to leak on unwind and rely on 
the GC. Yes, sometime the GC is best. That being said, it is up 
to the application to decide if leaking is acceptable or not, and 
so this can't be baked in the language (this also means that the 
user must be in charge the of deallocation policy, which puts a 
major dent into DIP74).

The 3rd problem is safety. This is where ownership shines.

The 4th problem is elision of refcount operations when not 
needed. Some thing, ownership can solve this. No need to do 
refcount operations when not necessary.

The compiler is perfectly able to optimize pairs of inc/dec 
granted it has the rights infos. And a proper ownership system 
provide the right infos.


 Yup, I was firmly behind scope (ie, borrowing), but that was
 destroyed, and DIP25 kinda cements the death of that :(


It is implemented with a flag. That allowed us to try it and get 
real life experience. That is a good thing. We aren't committed 
to it at this stage. And now we have actual experimentation that 
shows that it is too underpowered to be really useful. The smart 
move here is to admit defeat and reconsider.
Oct 11 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 17:30, deadalnix via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 06:48:54 UTC, Manu wrote:

 I don't really see that DIP25 and DIP74 are particularly closely related.



 Ho they are. It is all about ownership.


 DIP25 is a lame cop-out with respect to scope. I was a major backer of
 a proper scope implementation, and given that, I don't/didn't support
 DIP25 at all.


 DIP25 being a sorry excuse for ownership is the very reason DIP74 is needed.
 Rust can do reference counting just fine all in library without need for
 core language support.


 DIP74 on the other hand introduces 2 magic functions that the compiler
 calls as the appropriate moments to implement ref counting. The
 implementation is somewhat detail, theoretically changeable in the future,
 but I think language support for ref-counting primitives is critical for ref
 counting to be efficient and simple. Look at C++, and we see a disaster. C++
 basically implemented rval-references to improve (not solve) the RC
 problem... we don't have rval-ref's in D, so we're screwed from the library
 angle.


 C++ ref counting is not slow for the reasons you think it is. ARC is overly
 complex and also not as fast as you think it is.

 First, they are slow because in C++ (or ObjC), you don't know what is shared
 and what isn't. As a result, you got to go the pessimistic road and use
 thread safe ref counting. This can be a solved problem in D purely using the
 type qualifier.

 Second, exceptions. Their unwinding makes pair of inc/dec hard to recognize,
 plus now cleanup is need at pretty much every frames, meaning incredibly
 slow exceptions + various control flow optimization that aren't possible
 anymore (tail call being one of them).

 One that one, nobody has a solution. Not ARC (that's why swift does not have
 exception), not C++ and not us. IMO the obvious solution here is to allow
 ourselves to leak on unwind and rely on the GC. Yes, sometime the GC is
 best. That being said, it is up to the application to decide if leaking is
 acceptable or not, and so this can't be baked in the language (this also
 means that the user must be in charge the of deallocation policy, which puts
 a major dent into DIP74).

 The 3rd problem is safety. This is where ownership shines.

 The 4th problem is elision of refcount operations when not needed. Some
 thing, ownership can solve this. No need to do refcount operations when not
 necessary.

 The compiler is perfectly able to optimize pairs of inc/dec granted it has
 the rights infos. And a proper ownership system provide the right infos.


 Yup, I was firmly behind scope (ie, borrowing), but that was
 destroyed, and DIP25 kinda cements the death of that :(


 It is implemented with a flag. That allowed us to try it and get real life
 experience. That is a good thing. We aren't committed to it at this stage.
 And now we have actual experimentation that shows that it is too
 underpowered to be really useful. The smart move here is to admit defeat and
 reconsider.


Nothing here explains to me how it is that the compiler can do without
RC primitives that the compiler knows it is allowed to
optimise/schedule as it likes?
How does it look in rust that it is able to optimise calls to inc/dec
primitives?

You also make the points that I expect to bank on in D; we are
thread-local by default (big advantage!), nothrow is pervasive (big
advantage!), but the language still doesn't know the primitives that
it can use to optimise.

The ownership proposal solving the problem isn't entirely correct.
Sure, borrowed pointers imply RC elision, and it's particularly useful
for writing API's that are allocation/ownership agnostic, but it
doesn't solve the problem as you suggest. Once a pointer is borrowed,
it can't be assigned to another RC pointer anymore. If you must use
borrowing to implement RC elision across calls, then you lose the
ability to give someone else a reference down the call-tree.
Passing an RC object proper to a function also elides RC fiddling, but
the compiler can't distinguish between RC fiddling and copying logic
unless the RC primitives are explicit. I find that this is a very
common case.
Oct 11 2015
parent =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
Manu via Digitalmars-d <digitalmars-d puremagic.com> wrote: 

 Nothing here explains to me how it is that the compiler can do without
 RC primitives that the compiler knows it is allowed to
 optimise/schedule as it likes?
 How does it look in rust that it is able to optimise calls to inc/dec
 primitives?
 
 You also make the points that I expect to bank on in D; we are
 thread-local by default (big advantage!), nothrow is pervasive (big
 advantage!), but the language still doesn't know the primitives that
 it can use to optimise.
 
 The ownership proposal solving the problem isn't entirely correct.
 Sure, borrowed pointers imply RC elision, and it's particularly useful
 for writing API's that are allocation/ownership agnostic, but it
 doesn't solve the problem as you suggest. Once a pointer is borrowed,
 it can't be assigned to another RC pointer anymore. If you must use
 borrowing to implement RC elision across calls, then you lose the
 ability to give someone else a reference down the call-tree.
 Passing an RC object proper to a function also elides RC fiddling, but
 the compiler can't distinguish between RC fiddling and copying logic
 unless the RC primitives are explicit. I find that this is a very
 common case.


The key is, that RC in Rust is a move-by-default type. That means if you
are passing RCs around, you are actually passing ownership.
Only if you explicitly call 'clone', ownership is split up.
That means in the default case, no inc/dec is necessary.

Even more important, RC isn't used very often.
True shared ownership is actually quite rare. References (borrowed
pointers) and 'Box'es (unique pointers) are much more common.
shared_ptr is much overused in C++ IMO.

Tobi
Oct 11 2015
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 00:20:08 UTC, deadalnix wrote:

 It doesn't looks like it is getting implemented. And, to be 
 honest, I'd rather go a principle approach + library support 
 rather than a pie of hacks.

 The pile of hacks approach is what made C++ C++.


AFAIK, Walter and Andrei are still in favor of something that's 
at least similar to DIP 74. Andrei made a comment in a thread 
just the other day that indicated that he was in favor of having 
a way to build reference counting into classes. So, I don't know 
why you think that it's not going to be implemented other than 
the fact that it hasn't been implemented yet. It wouldn't 
surprise me if the DIP needed some tweaking though.

Regardless, now is not the best time to ask this sort of 
question, because Walter and Andrei are on their trip to Romania, 
and they're going to have a limited presence in the newsgroup at 
the moment.

It wouldn't surprise me though if something like the possibility 
of getting D into another company relied on something like DIP 74 
helped push it along and got it sorted out faster. Clearly, 
Walter and Andrei think that it's an issue and have done some 
work on it at the theoretical level, but I don't know where it 
sits on their priority list. And even if DIP 74 were fully sorted 
out tomorrow, it would still require Walter or someone else 
actually implementing it, and that's probably not a small 
undertaking.

- Jonathan M Davis
Oct 10 2015
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 02:01:09 UTC, Jonathan M Davis 
wrote:

 AFAIK, Walter and Andrei are still in favor of something that's 
 at least similar to DIP 74. Andrei made a comment in a thread 
 just the other day that indicated that he was in favor of 
 having a way to build reference counting into classes. So, I 
 don't know why you think that it's not going to be implemented 
 other than the fact that it hasn't been implemented yet. It 
 wouldn't surprise me if the DIP needed some tweaking though.


Yes, and that's quite ridiculous. I mean this is getting into 
ridiculous ego battle. Remind of that concept vs static if grand 
debate, the peak of ridiculousness (everybody know you don't need 
type system when you have if statement and vice versa, so the 
same must be true at compile time as well).

When a direction obviously showed to be the wrong one, the 
rational thing to do is not to double down in order to not admit 
one is wrong.

DIP25 implementation showed a ton of limitations and pitfalls. It 
isn't even possible to do a slice type with eager deallocation, 
just one with deferred deallocation, with complex strategies to 
make it all safe.

It is a sign of a poorly thought out language addition.


 It wouldn't surprise me though if something like the 
 possibility of getting D into another company relied on 
 something like DIP 74 helped push it along and got it sorted 
 out faster. Clearly, Walter and Andrei think that it's an issue 
 and have done some work on it at the theoretical level, but I 
 don't know where it sits on their priority list. And even if 
 DIP 74 were fully sorted out tomorrow, it would still require 
 Walter or someone else actually implementing it, and that's 
 probably not a small undertaking.

 - Jonathan M Davis


Yeah, we saw what happens with attributes. Don't get me wrong, 
attribute are a very useful addition to the language and all, but 
the current implementation has some serious flaws, none of them 
could be addressed as it was pushed out of the door in an 
inconsequent manner. The fact that dlang.org is littered of 
antipaterns usage of them is quite telling.

I'm all for pushing useful feature, especially if that can drive 
adoption in a company. But using it as an excuse for releasing 
half backed feature is VERY short sighted.
Oct 10 2015
next sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 14:25, deadalnix via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 02:01:09 UTC, Jonathan M Davis wrote:

 AFAIK, Walter and Andrei are still in favor of something that's at least
 similar to DIP 74. Andrei made a comment in a thread just the other day that
 indicated that he was in favor of having a way to build reference counting
 into classes. So, I don't know why you think that it's not going to be
 implemented other than the fact that it hasn't been implemented yet. It
 wouldn't surprise me if the DIP needed some tweaking though.


 Yes, and that's quite ridiculous. I mean this is getting into ridiculous ego
 battle. Remind of that concept vs static if grand debate, the peak of
 ridiculousness (everybody know you don't need type system when you have if
 statement and vice versa, so the same must be true at compile time as well).

 When a direction obviously showed to be the wrong one, the rational thing to
 do is not to double down in order to not admit one is wrong.

 DIP25 implementation showed a ton of limitations and pitfalls. It isn't even
 possible to do a slice type with eager deallocation, just one with deferred
 deallocation, with complex strategies to make it all safe.

 It is a sign of a poorly thought out language addition.


 It wouldn't surprise me though if something like the possibility of
 getting D into another company relied on something like DIP 74 helped push
 it along and got it sorted out faster. Clearly, Walter and Andrei think that
 it's an issue and have done some work on it at the theoretical level, but I
 don't know where it sits on their priority list. And even if DIP 74 were
 fully sorted out tomorrow, it would still require Walter or someone else
 actually implementing it, and that's probably not a small undertaking.

 - Jonathan M Davis



 Yeah, we saw what happens with attributes. Don't get me wrong, attribute are
 a very useful addition to the language and all, but the current
 implementation has some serious flaws, none of them could be addressed as it
 was pushed out of the door in an inconsequent manner. The fact that
 dlang.org is littered of antipaterns usage of them is quite telling.


What's wrong with attributes? I can think of some needed additions to
finish the job, but what would you say is fundamentally wrong with
them as is?



 I'm all for pushing useful feature, especially if that can drive adoption in
 a company. But using it as an excuse for releasing half backed feature is
 VERY short sighted.


What would you do instead? I'm happy with DIP74, and I haven't
followed threads where people have objected and why...?
Oct 10 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/11/15 7:25 AM, deadalnix wrote:

 On Sunday, 11 October 2015 at 02:01:09 UTC, Jonathan M Davis wrote:

 AFAIK, Walter and Andrei are still in favor of something that's at
 least similar to DIP 74. Andrei made a comment in a thread just the
 other day that indicated that he was in favor of having a way to build
 reference counting into classes. So, I don't know why you think that
 it's not going to be implemented other than the fact that it hasn't
 been implemented yet. It wouldn't surprise me if the DIP needed some
 tweaking though.


 Yes, and that's quite ridiculous. I mean this is getting into ridiculous
 ego battle.


It's unlikely that adding inflammation to this would help too much - 
sticking to the technical points is more helpful.

You're saying that as far as you can tell you have made the perfect 
argument in favor of your proposed approach (have you written it up as a 
DIP?) and that essentially there is no reasonable way your idea could 
not be accepted. If you failed to raise the response you think that 
proposal deserves, getting emotional about it can't possibly be more 
productive than reiterating the technical point.

We're all on the same boat. We want to make D better. Let's.


Andrei
Oct 11 2015
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 07:08:26 UTC, Andrei Alexandrescu 
wrote:

 On 10/11/15 7:25 AM, deadalnix wrote:

 On Sunday, 11 October 2015 at 02:01:09 UTC, Jonathan M Davis 
 wrote:

 AFAIK, Walter and Andrei are still in favor of something 
 that's at
 least similar to DIP 74. Andrei made a comment in a thread 
 just the
 other day that indicated that he was in favor of having a way 
 to build
 reference counting into classes. So, I don't know why you 
 think that
 it's not going to be implemented other than the fact that it 
 hasn't
 been implemented yet. It wouldn't surprise me if the DIP 
 needed some
 tweaking though.


 Yes, and that's quite ridiculous. I mean this is getting into 
 ridiculous
 ego battle.


 It's unlikely that adding inflammation to this would help too 
 much - sticking to the technical points is more helpful.

 You're saying that as far as you can tell you have made the 
 perfect argument in favor of your proposed approach (have you 
 written it up as a DIP?) and that essentially there is no 
 reasonable way your idea could not be accepted. If you failed 
 to raise the response you think that proposal deserves, getting 
 emotional about it can't possibly be more productive than 
 reiterating the technical point.

 We're all on the same boat. We want to make D better. Let's.


 Andrei


No I'm not saying I've made the perfect argument.

I'm saying that DIP 25 was implemented and showed that it was too 
limited. The various experiment with it showed that is wasn't 
enough (ref counted objects) or required to jump though a lot of 
hoops for somewhat disappointing result (ref counted arrays).

DIP74 is a direct result of DIP25 limitations. I think we should 
start by acknowledging that the DIP25 experiment was not a 
success before taking it for granted and proceed.

I do think a buffed up version of DIP25 would make DIP74 obsolete 
(or implementable as a library).
Oct 11 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/11/15 10:53 AM, deadalnix wrote:

 I'm saying that DIP 25 was implemented and showed that it was too
 limited. The various experiment with it showed that is wasn't enough
 (ref counted objects) or required to jump though a lot of hoops for
 somewhat disappointing result (ref counted arrays).

 DIP74 is a direct result of DIP25 limitations. I think we should start
 by acknowledging that the DIP25 experiment was not a success before
 taking it for granted and proceed.

 I do think a buffed up version of DIP25 would make DIP74 obsolete (or
 implementable as a library).


Walter and I are happy with DIP25, and the fact of the matter is we 
weren't surprised more complementary work is needed. So no, I won't 
acknowledge what I don't believe.

I'd say the one way to get things looked at seriously is to create a 
DIP. That's no guarantee it will be accepted but there is a guarantee 
that our chat at DConf is not sufficient even as a basis for further study.


Andrei
Oct 11 2015
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 13:51:18 UTC, Andrei Alexandrescu 
wrote:

 Walter and I are happy with DIP25, and the fact of the matter 
 is we weren't surprised more complementary work is needed. So 
 no, I won't acknowledge what I don't believe.


That is an empty statement. What is there to be happy about ?

Also the complementary argument pretty much destroy the best 
argument you and Walter made for DIP25 : it is simple. I mean, 
one need to look at the big picture. DIP25 + complementary 
addition is not simple anymore.


 I'd say the one way to get things looked at seriously is to 
 create a DIP. That's no guarantee it will be accepted but there 
 is a guarantee that our chat at DConf is not sufficient even as 
 a basis for further study.


Yeah there are IRL discussion, there are many posts in the forum, 
there are by mail discussions at DIP25 creation time, there are 
at least one DIP already.

The only rebuttal to all of this is "Walter and I are happy with 
DIP25, and the fact of the matter", while everybody else is 
wondering what there is to be happy about.
Oct 11 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote:

 On Sunday, 11 October 2015 at 13:51:18 UTC, Andrei Alexandrescu 
 wrote:

 Walter and I are happy with DIP25, and the fact of the matter 
 is we weren't surprised more complementary work is needed. So 
 no, I won't acknowledge what I don't believe.


 That is an empty statement. What is there to be happy about ?

 Also the complementary argument pretty much destroy the best 
 argument you and Walter made for DIP25 : it is simple. I mean, 
 one need to look at the big picture. DIP25 + complementary 
 addition is not simple anymore.


 I'd say the one way to get things looked at seriously is to 
 create a DIP. That's no guarantee it will be accepted but 
 there is a guarantee that our chat at DConf is not sufficient 
 even as a basis for further study.


 Yeah there are IRL discussion, there are many posts in the 
 forum, there are by mail discussions at DIP25 creation time, 
 there are at least one DIP already.

 The only rebuttal to all of this is "Walter and I are happy 
 with DIP25, and the fact of the matter", while everybody else 
 is wondering what there is to be happy about.


Also, I'm sorry but there is no me writing once again a document 
about what alternative are possible. Spending hours to write 
documents so that one is answered something along the line of "we 
are happy with the other thing, but we can't give any reason why" 
is something I've engaged in several time in already, and has no 
desire to indulge into this if I have reason to think the same 
will happen. Your answer here are telling me one thing: it won't 
be taken seriously.
Oct 11 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/11/15 9:57 PM, deadalnix wrote:

 On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote:

 On Sunday, 11 October 2015 at 13:51:18 UTC, Andrei Alexandrescu wrote:

 Walter and I are happy with DIP25, and the fact of the matter is we
 weren't surprised more complementary work is needed. So no, I won't
 acknowledge what I don't believe.


 That is an empty statement. What is there to be happy about ?

 Also the complementary argument pretty much destroy the best argument
 you and Walter made for DIP25 : it is simple. I mean, one need to look
 at the big picture. DIP25 + complementary addition is not simple anymore.


 I'd say the one way to get things looked at seriously is to create a
 DIP. That's no guarantee it will be accepted but there is a guarantee
 that our chat at DConf is not sufficient even as a basis for further
 study.


 Yeah there are IRL discussion, there are many posts in the forum,
 there are by mail discussions at DIP25 creation time, there are at
 least one DIP already.

 The only rebuttal to all of this is "Walter and I are happy with
 DIP25, and the fact of the matter", while everybody else is wondering
 what there is to be happy about.


 Also, I'm sorry but there is no me writing once again a document about
 what alternative are possible.


Could you please point to the document you have already written?


 Spending hours to write documents so that
 one is answered something along the line of "we are happy with the other
 thing, but we can't give any reason why" is something I've engaged in
 several time in already, and has no desire to indulge into this if I
 have reason to think the same will happen. Your answer here are telling
 me one thing: it won't be taken seriously.


There's a bit of a stalemate here. So we have:

1. You say that DIP25 is a failure. More so, you demand that is admitted 
without evidence. What I see is a feature that solves one problem, and 
solves it well: annotating a function that returns a reference to its 
argument. The syntactic cost is low, the impact on existing code is 
small, and the impact on safety is positive. Walter and I think it is 
the simplest solution of all considered.

2. You refuse to write a DIP under the assumption it will not be taken 
seriously. Conversely if you do write a DIP there is an expectation it 
will be approved just because you put work in it. I don't think 
rewarding work is the right way to go. We need to reward good work. The 
"work" part (i.e. a DIP) is a prerequisite; you can't demand to 
implement a complex feature based on posts and discussions.

So I'm not sure how we can move forward from here. If you want to 
discuss DIP74, great, it can be discussed because it exists. My personal 
opinion on DIP74 is it has holes and corner cases so it seems it doesn't 
quite hit the spot. One option is to make it work, another is to take a 
different attack on the problem. But we need the appropriate DIP.


Andrei
Oct 11 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 20:35:05 UTC, Andrei Alexandrescu 
wrote:

 Could you please point to the document you have already written?


For instance, we had a discussion with Walter and Mark that 
eventually yielded DIP25. In there, I made the following proposal 
:

http://pastebin.com/LMkuTbgN

I made several other very detailed proposal. Other did. It's not 
about me here. Others simply abandoned as far as I can tell. I'm 
just a stubborn idiot.


 There's a bit of a stalemate here. So we have:

 1. You say that DIP25 is a failure. More so, you demand that is 
 admitted without evidence. What I see is a feature that solves 
 one problem, and solves it well: annotating a function that 
 returns a reference to its argument. The syntactic cost is low, 
 the impact on existing code is small, and the impact on safety 
 is positive. Walter and I think it is the simplest solution of 
 all considered.


It is indeed the simplest. However, experiences that have been 
made and discussed in the forum showed it was often too simple to 
be really useful. I cited example of this, namely the RCArray 
thing and the existence of DIP74.

I don't think the simplicity argument holds water in general as 
long as we don't have the whole thing. DIP25 + DIP74 + ... must 
be measured against the alternative.


 2. You refuse to write a DIP under the assumption it will not 
 be taken seriously. Conversely if you do write a DIP there is 
 an expectation it will be approved just because you put work in 
 it. I don't think rewarding work is the right way to go. We 
 need to reward good work. The "work" part (i.e. a DIP) is a 
 prerequisite; you can't demand to implement a complex feature 
 based on posts and discussions.


No that is inaccurate. I think I have evidence that it won't be 
taken seriously. To start with, there are already several DIP on 
the subject and they are not discussed at ALL. Namely :

http://wiki.dlang.org/DIP35
http://wiki.dlang.org/DIP36
http://wiki.dlang.org/DIP69
http://wiki.dlang.org/DIP71

These do not even register as a blip on the radar. I don't see 
how adding my to the pile would change anything.

There are not considered because DIP25 is "simpler" and you and 
Walter "like it". As long as nothing changes here, there is 
really no point in wasting my time.


 So I'm not sure how we can move forward from here. If you want 
 to discuss DIP74, great, it can be discussed because it exists. 
 My personal opinion on DIP74 is it has holes and corner cases 
 so it seems it doesn't quite hit the spot. One option is to 
 make it work, another is to take a different attack on the 
 problem. But we need the appropriate DIP.


Let's start by the beginning: what good design was enabled by 
DIP25 ? As long as none is presented, we can't consider it a 
success.
Oct 11 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/12/15 1:44 AM, deadalnix wrote:

 On Sunday, 11 October 2015 at 20:35:05 UTC, Andrei Alexandrescu wrote:

 Could you please point to the document you have already written?


 For instance, we had a discussion with Walter and Mark that eventually
 yielded DIP25. In there, I made the following proposal :

 http://pastebin.com/LMkuTbgN


This is an unstructured text. Could you please use it as a basis for a 
formal proposal?


 I made several other very detailed proposal.


Where are they?


 Other did. It's not about
 me here. Others simply abandoned as far as I can tell. I'm just a
 stubborn idiot.


At this point it would be great to just keep it calm and reduce 
inflammation. It won't achieve anything.


 There's a bit of a stalemate here. So we have:

 1. You say that DIP25 is a failure. More so, you demand that is
 admitted without evidence. What I see is a feature that solves one
 problem, and solves it well: annotating a function that returns a
 reference to its argument. The syntactic cost is low, the impact on
 existing code is small, and the impact on safety is positive. Walter
 and I think it is the simplest solution of all considered.


 It is indeed the simplest. However, experiences that have been made and
 discussed in the forum showed it was often too simple to be really
 useful. I cited example of this, namely the RCArray thing and the
 existence of DIP74.

 I don't think the simplicity argument holds water in general as long as
 we don't have the whole thing. DIP25 + DIP74 + ... must be measured
 against the alternative.


What is the alternative? Some handwaving asking to do ownership a la 
Rust cannot be analyzed.


 2. You refuse to write a DIP under the assumption it will not be taken
 seriously. Conversely if you do write a DIP there is an expectation it
 will be approved just because you put work in it. I don't think
 rewarding work is the right way to go. We need to reward good work.
 The "work" part (i.e. a DIP) is a prerequisite; you can't demand to
 implement a complex feature based on posts and discussions.


 No that is inaccurate. I think I have evidence that it won't be taken
 seriously. To start with, there are already several DIP on the subject
 and they are not discussed at ALL. Namely :

 http://wiki.dlang.org/DIP35
 http://wiki.dlang.org/DIP36
 http://wiki.dlang.org/DIP69
 http://wiki.dlang.org/DIP71

 These do not even register as a blip on the radar. I don't see how
 adding my to the pile would change anything.


Creating a DIP is no guarantee it will be approved, or discussed 
immediately. These in particular - I've been over most. I think DIP35 is 
not good. DIP36 I didn't look at yet, but was aware of it and will 
definitely do. DIP69 is obviously known to me because my name is on it. 
DIP71 is very sketchy and is not in reviewable form.


 There are not considered because DIP25 is "simpler" and you and Walter
 "like it". As long as nothing changes here, there is really no point in
 wasting my time.


That is a fair assessment. Basically I believe DIP25 is good language 
design, and I have evidence for it. The evidence you showed failed to 
convince me the design is a hack, and yelling at me is unlikely to help. 
Please decide as you find fit. At some point it is clear that several 
language designers will disagree on estimating the quality of something.


 So I'm not sure how we can move forward from here. If you want to
 discuss DIP74, great, it can be discussed because it exists. My
 personal opinion on DIP74 is it has holes and corner cases so it seems
 it doesn't quite hit the spot. One option is to make it work, another
 is to take a different attack on the problem. But we need the
 appropriate DIP.


 Let's start by the beginning: what good design was enabled by DIP25 ? As
 long as none is presented, we can't consider it a success.


Probably git grep in phobos may be a good starting point.


Andrei
Oct 11 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 12 October 2015 at 06:02:47 UTC, Andrei Alexandrescu 
wrote:

 There are not considered because DIP25 is "simpler" and you 
 and Walter
 "like it". As long as nothing changes here, there is really no 
 point in
 wasting my time.


 That is a fair assessment. Basically I believe DIP25 is good 
 language design, and I have evidence for it. The evidence you 
 showed failed to convince me the design is a hack, and yelling 
 at me is unlikely to help. Please decide as you find fit. At 
 some point it is clear that several language designers will 
 disagree on estimating the quality of something.


If you are wondering why I'm inflammatory, here you go. You are 
pulling me the old prove a negative trick. You have good evidence 
that DIP25 is good design ? Good, because I have none. And that's 
my proof. As long as I have no evidence that DIP25 is good, DIP25 
is bad.


 Probably git grep in phobos may be a good starting point.


I don't think grepping for return will have a good noise to 
signal ratio. You also mentioned several time that you have good 
evidence that DIP25 rox. Yet, every time you post that without 
any evidence, I'm a bit more convinced that none exists.
Oct 12 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/12/15 10:21 AM, deadalnix wrote:

 On Monday, 12 October 2015 at 06:02:47 UTC, Andrei Alexandrescu wrote:

 There are not considered because DIP25 is "simpler" and you and Walter
 "like it". As long as nothing changes here, there is really no point in
 wasting my time.


 That is a fair assessment. Basically I believe DIP25 is good language
 design, and I have evidence for it. The evidence you showed failed to
 convince me the design is a hack, and yelling at me is unlikely to
 help. Please decide as you find fit. At some point it is clear that
 several language designers will disagree on estimating the quality of
 something.


 If you are wondering why I'm inflammatory, here you go. You are pulling
 me the old prove a negative trick. You have good evidence that DIP25 is
 good design ? Good, because I have none. And that's my proof. As long as
 I have no evidence that DIP25 is good, DIP25 is bad.


Instead of assuming my purpose here is to pull tricks on you and 
manipulate the dialog politically, it's more productive to just stick to 
the technical discussion. I only started the dialog to get more informed 
about technical things.


 Probably git grep in phobos may be a good starting point.


 I don't think grepping for return will have a good noise to signal
 ratio.


Sorry, I meant to git grep for "return ref".


 You also mentioned several time that you have good evidence that
 DIP25 rox. Yet, every time you post that without any evidence, I'm a bit
 more convinced that none exists.


The motivation is in the document and follows many discussions derived 
from it. It's all about functions and particularly member functions 
returning up references to data safely.

Amaury, you and Timon are probably the most competent PL theorists in 
this forum. He did great work: found real problems with DIP25 that need 
to be looked at. In that light, spending time protesting and yelling 
figuratively at people is a distinctly unproductive way to spend your 
time as a very talented contributor. Do great work. It will be 
recognized. Don't point me at your past posts. They are not great work 
and you know it. Don't point me at those related DIPs. They are not 
great work and you know it. Don't find reasons to not do great work 
because it'll be wasted on my ego. Do great work and you will prevail.

This tone of discussion has carried its course. I'm done arguing so if 
you want to continue arguing, great - last word is yours. In the recent 
times I've done my best to reduce my participation to unproductive 
discussions in forums, and the added perspective and time for real work 
have been very valuable. I suggest everyone to try it.


Andrei
Oct 12 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 12 October 2015 at 07:44:47 UTC, Andrei Alexandrescu 
wrote:

 Instead of assuming my purpose here is to pull tricks on you 
 and manipulate the dialog politically, it's more productive to 
 just stick to the technical discussion. I only started the 
 dialog to get more informed about technical things.


I'm not assuming this. I'm fairly confident that you have the 
best intention for D in mind rather than winning internet debate 
points. So do I.

Still you are pulling the above mentioned trick. i don't think 
you did it on purpose. I think you did it because you are 
convinced that DIP25 is great, in fact so convinced that you take 
thing for granted that aren't.

It is a natural tendency we all have to lower the standard of 
evidence we require when we are already convinced. You, me, 
everybody.


 I don't think grepping for return will have a good noise to 
 signal
 ratio.


 Sorry, I meant to git grep for "return ref".


So I did. I found 2 uses of return ref in current master that 
aren't in unitests. These are both related to unicode decoding. 
These are legit use of DIP25, and I recognize their value.

Still I don't think this is valuable enough to have its own 
syntax. I also, on the other hand presented cases of things that 
people thought could be done with DIP25, but ended up to be 
either not possible or to have a disappointingly low complexity 
to added value ratio.

I can agree on something: DIP25 is the right direction. But it is 
too little to pay for itself and I haven't seen anything here 
that would suggest otherwize.


 You also mentioned several time that you have good evidence 
 that
 DIP25 rox. Yet, every time you post that without any evidence, 
 I'm a bit
 more convinced that none exists.


 The motivation is in the document and follows many discussions 
 derived from it. It's all about functions and particularly 
 member functions returning up references to data safely.

 Amaury, you and Timon are probably the most competent PL 
 theorists in this forum. He did great work: found real problems 
 with DIP25 that need to be looked at. In that light, spending 
 time protesting and yelling figuratively at people is a 
 distinctly unproductive way to spend your time as a very 
 talented contributor. Do great work. It will be recognized. 
 Don't point me at your past posts. They are not great work and 
 you know it. Don't point me at those related DIPs. They are not 
 great work and you know it. Don't find reasons to not do great 
 work because it'll be wasted on my ego. Do great work and you 
 will prevail.

 This tone of discussion has carried its course. I'm done 
 arguing so if you want to continue arguing, great - last word 
 is yours. In the recent times I've done my best to reduce my 
 participation to unproductive discussions in forums, and the 
 added perspective and time for real work have been very 
 valuable. I suggest everyone to try it.


Ok, I'll write a DIP.
Oct 12 2015
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 12 October 2015 at 08:10:44 UTC, deadalnix wrote:

 Ok, I'll write a DIP.


That would be great to see.  N.B. it's not only about Walter and 
Andrei: with a detailed, written-up proposal in place, the rest 
of us can examine it and, assuming we like it, lend our vocal 
support to the idea.
Oct 13 2015
prev sibling parent Johannes Pfau <nospam example.com> writes:
Am Mon, 12 Oct 2015 10:44:47 +0300
schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:


 Probably git grep in phobos may be a good starting point.
  


 I don't think grepping for return will have a good noise to signal
 ratio.  


 
 Sorry, I meant to git grep for "return ref".


AFAICS this returns only 6 results:

// false positive
std/functional.d:        static ref int func_ref() { return refvar; }

// test functions for return ref
std/functional.d:    ref int foo(return ref int a) { return a; }
std/traits.d:        void test(scope int, ref int, out int, lazy
  int, int, return ref int) { }
std/traits.d:        ref const(Inner[string]) retfunc( return ref Inner
var1 );



// Real examples

// I don't see the advantage here. Simple example, private API anyway,
// not using  safe
std/stdio.d:    private char[] takeFront(return ref char[4] buf)

// A good, but very simple example
std/utf.d:wchar[] toUTF16(return ref wchar[2] buf, dchar c) nothrow
   nogc  safe



So AFAICS it's not used much and the use cases are rather simple
compared to RCSlice / RC!T / Unique and all these complex cases that
have been discussed related to ownership.
Oct 12 2015
prev sibling parent reply Jacob <JJ12343 gmail.com> writes:
On Monday, 12 October 2015 at 07:21:58 UTC, deadalnix wrote:

 On Monday, 12 October 2015 at 06:02:47 UTC, Andrei Alexandrescu 
 wrote:

 There are not considered because DIP25 is "simpler" and you 
 and Walter
 "like it". As long as nothing changes here, there is really 
 no point in
 wasting my time.


 That is a fair assessment. Basically I believe DIP25 is good 
 language design, and I have evidence for it. The evidence you 
 showed failed to convince me the design is a hack, and yelling 
 at me is unlikely to help. Please decide as you find fit. At 
 some point it is clear that several language designers will 
 disagree on estimating the quality of something.


 If you are wondering why I'm inflammatory, here you go. You are 
 pulling me the old prove a negative trick. You have good 
 evidence that DIP25 is good design ? Good, because I have none. 
 And that's my proof. As long as I have no evidence that DIP25 
 is good, DIP25 is bad.


I've noticed that you seem to be quite arrogant. Usually it is a 
result of ignorance. Your statement basically proves that.

Maybe you should take a break from programming for a while and 
work on your attitude?

While you have no proof of this, If you do a little soul 
searching you'll find that the world doesn't revolve around you. 
Put down your toys and get out of the sandbox and you might learn 
something!
Oct 12 2015
parent reply Ice Cream Overload <thegreatone google.com> writes:
On Monday, 12 October 2015 at 16:56:27 UTC, Jacob wrote:

 I've noticed that you seem to be quite arrogant. Usually it is 
 a result of ignorance. Your statement basically proves that.

 Maybe you should take a break from programming for a while and 
 work on your attitude?

 While you have no proof of this, If you do a little soul 
 searching you'll find that the world doesn't revolve around 
 you. Put down your toys and get out of the sandbox and you 
 might learn something!


Dude you are kind of being a jerk. He's just arguing against, 
rather passionately, a design decision he thinks is poor. It 
values no one if individuals remain quiet and conform. Certainly 
not the leadership. What you see as arrogance is really just 
passion.
Oct 12 2015
parent reply Jacob <JJ12343 gmail.com> writes:
On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload 
wrote:

 On Monday, 12 October 2015 at 16:56:27 UTC, Jacob wrote:

 I've noticed that you seem to be quite arrogant. Usually it is 
 a result of ignorance. Your statement basically proves that.

 Maybe you should take a break from programming for a while and 
 work on your attitude?

 While you have no proof of this, If you do a little soul 
 searching you'll find that the world doesn't revolve around 
 you. Put down your toys and get out of the sandbox and you 
 might learn something!


 Dude you are kind of being a jerk. He's just arguing against, 
 rather passionately, a design decision he thinks is poor. It 
 values no one if individuals remain quiet and conform. 
 Certainly not the leadership. What you see as arrogance is 
 really just passion.


Passion or not,

"If you are wondering why I'm inflammatory, here you go. You are 
pulling me the old prove a negative trick. You have good evidence 
that DIP25 is good design ? Good, because I have none. And that's 
my proof. As long as I have no evidence that DIP25 is good, DIP25 
is bad."

That statement shows a lot of arrogance.

It he not really just saying "I have no clue if X is true, but 
since I don't know, I'll just assume it's false and assume you 
are wrong.".

That's not very logical. Why wouldn't he just as well assume X is 
true?

The fact is, he can't and shouldn't make such statements about X 
if he has no "evidence" about it.

Instead, wouldn't the proper approach be to discuss, learn, and 
share what one things in a positive way so everyone can learn 
about X and reach a more intelligent understanding of it?

Hilter was very passionate too, are you saying he was right?
Oct 13 2015
next sibling parent John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote:

 On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload 
 wrote:

 On Monday, 12 October 2015 at 16:56:27 UTC, Jacob wrote:

 I've noticed that you seem to be quite arrogant. Usually it 
 is a result of ignorance. Your statement basically proves 
 that.

 Maybe you should take a break from programming for a while 
 and work on your attitude?

 While you have no proof of this, If you do a little soul 
 searching you'll find that the world doesn't revolve around 
 you. Put down your toys and get out of the sandbox and you 
 might learn something!


 Dude you are kind of being a jerk. He's just arguing against, 
 rather passionately, a design decision he thinks is poor. It 
 values no one if individuals remain quiet and conform. 
 Certainly not the leadership. What you see as arrogance is 
 really just passion.


 Passion or not,

 "If you are wondering why I'm inflammatory, here you go. You 
 are pulling me the old prove a negative trick. You have good 
 evidence that DIP25 is good design ? Good, because I have none. 
 And that's my proof. As long as I have no evidence that DIP25 
 is good, DIP25 is bad."

 That statement shows a lot of arrogance.

 It he not really just saying "I have no clue if X is true, but 
 since I don't know, I'll just assume it's false and assume you 
 are wrong.".

 That's not very logical. Why wouldn't he just as well assume X 
 is true?


The context is that all additions have an intrinsic cost because 
they increase complexity and restrict future design choices, 
therefore the onus is on any change/addition to justify its value 
in order to overcome that cost.

I think you are completely misreading the situation, or trolling. 
This is just
Oct 13 2015
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote:

 On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload 
 wrote:

 On Monday, 12 October 2015 at 16:56:27 UTC, Jacob wrote:

 I've noticed that you seem to be quite arrogant. Usually it 
 is a result of ignorance. Your statement basically proves 
 that.

 Maybe you should take a break from programming for a while 
 and work on your attitude?

 While you have no proof of this, If you do a little soul 
 searching you'll find that the world doesn't revolve around 
 you. Put down your toys and get out of the sandbox and you 
 might learn something!


 Dude you are kind of being a jerk. He's just arguing against, 
 rather passionately, a design decision he thinks is poor. It 
 values no one if individuals remain quiet and conform. 
 Certainly not the leadership. What you see as arrogance is 
 really just passion.


 Passion or not,

 "If you are wondering why I'm inflammatory, here you go. You 
 are pulling me the old prove a negative trick. You have good 
 evidence that DIP25 is good design ? Good, because I have none. 
 And that's my proof. As long as I have no evidence that DIP25 
 is good, DIP25 is bad."

 That statement shows a lot of arrogance.

 It he not really just saying "I have no clue if X is true, but 
 since I don't know, I'll just assume it's false and assume you 
 are wrong.".

 That's not very logical. Why wouldn't he just as well assume X 
 is true?


Because people with half a brain know that's not how it works.

Proves me that unicorn do not exists. I'm waiting. And remember, 
having no evidence that they do exists doesn't mean they do not !


 Hilter was very passionate too, are you saying he was right?


ICH BIN EIN POLYNOMIAL!
Oct 13 2015
next sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
On Tuesday, 13 October 2015 at 17:59:28 UTC, deadalnix wrote:

 Hilter was very passionate too, are you saying he was right?


 ICH BIN EIN POLYNOMIAL!


As this thread has run it course starting with the Hitler 
comparison, and is therefor OT.
You have to explain to me, why your are a polynomial :-)
Oct 13 2015
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 13 October 2015 at 18:18:46 UTC, Robert burner 
Schadek wrote:

 On Tuesday, 13 October 2015 at 17:59:28 UTC, deadalnix wrote:

 Hilter was very passionate too, are you saying he was right?


 ICH BIN EIN POLYNOMIAL!


 As this thread has run it course starting with the Hitler 
 comparison, and is therefor OT.
 You have to explain to me, why your are a polynomial :-)


That's a reference to The Oatmeal : 
http://theoatmeal.com/comics/atheism
Oct 13 2015
parent Robert burner Schadek <rburners gmail.com> writes:
On Tuesday, 13 October 2015 at 18:24:13 UTC, deadalnix wrote:


 That's a reference to The Oatmeal : 
 http://theoatmeal.com/comics/atheism


thanks
Oct 13 2015
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 13 Oct 2015 17:59:26 +0000
schrieb deadalnix <deadalnix gmail.com>:


 It he not really just saying "I have no clue if X is true, but 
 since I don't know, I'll just assume it's false and assume you 
 are wrong.".

 That's not very logical. Why wouldn't he just as well assume X 
 is true?


 
 Because people with half a brain know that's not how it works.
 
 Proves me that unicorn do not exists. I'm waiting. And remember, 
 having no evidence that they do exists doesn't mean they do not !


Ok, so here we arrived in d.religion. Today: "Agnostic vs.
atheist, who is right." And: "Testimony: I tried to change the
world but God didn't give me the source code."

-- 
Marco
Oct 13 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 13 October 2015 at 18:23:05 UTC, Marco Leise wrote:

 Ok, so here we arrived in d.religion. Today: "Agnostic vs. 
 atheist, who is right." And: "Testimony: I tried to change the 
 world but God didn't give me the source code."


Well, I talk about D-ifying code sometimes, but saying that out 
loud is problematic...

- Jonathan M Davis
Oct 13 2015
prev sibling parent reply Ice Cream Overload <thegreatone google.com> writes:
On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote:

 On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload 
 wrote:

 [...]


 Passion or not,

 "If you are wondering why I'm inflammatory, here you go. You 
 are pulling me the old prove a negative trick. You have good 
 evidence that DIP25 is good design ? Good, because I have none. 
 And that's my proof. As long as I have no evidence that DIP25 
 is good, DIP25 is bad."

 That statement shows a lot of arrogance.

 It he not really just saying "I have no clue if X is true, but 
 since I don't know, I'll just assume it's false and assume you 
 are wrong.".

 That's not very logical. Why wouldn't he just as well assume X 
 is true?

 The fact is, he can't and shouldn't make such statements about 
 X if he has no "evidence" about it.

 Instead, wouldn't the proper approach be to discuss, learn, and 
 share what one things in a positive way so everyone can learn 
 about X and reach a more intelligent understanding of it?

 Hilter was very passionate too, are you saying he was right?


Since you brought up the Hilter reference, it's useful to mention 
that he was successful because people blindly followed him and 
his actions...

But really...do we always have to resort to Hitler references 
when discussing disagreements? Seems very cliche.
Oct 13 2015
parent reply Jacob <JJ12343 gmail.com> writes:
On Tuesday, 13 October 2015 at 18:13:33 UTC, Ice Cream Overload 
wrote:

 On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote:

 On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload 
 wrote:

 [...]


 Passion or not,

 "If you are wondering why I'm inflammatory, here you go. You 
 are pulling me the old prove a negative trick. You have good 
 evidence that DIP25 is good design ? Good, because I have 
 none. And that's my proof. As long as I have no evidence that 
 DIP25 is good, DIP25 is bad."

 That statement shows a lot of arrogance.

 It he not really just saying "I have no clue if X is true, but 
 since I don't know, I'll just assume it's false and assume you 
 are wrong.".

 That's not very logical. Why wouldn't he just as well assume X 
 is true?

 The fact is, he can't and shouldn't make such statements about 
 X if he has no "evidence" about it.

 Instead, wouldn't the proper approach be to discuss, learn, 
 and share what one things in a positive way so everyone can 
 learn about X and reach a more intelligent understanding of it?

 Hilter was very passionate too, are you saying he was right?


 Since you brought up the Hilter reference, it's useful to 
 mention that he was successful because people blindly followed 
 him and his actions...

 But really...do we always have to resort to Hitler references 
 when discussing disagreements? Seems very cliche.


It's only cliche if you aren't interested in the truth. It 
doesn't matter if I used Hilter or any other person that was 
"passionate" but wrong.

Hilter is just the greatest example and most obvious example. If 
you actually understand the issue, you would realize it has 
nothing to do with Hilter.
Oct 13 2015
next sibling parent reply I SCREAM for ICECream <thegreatone google.com> writes:
On Tuesday, 13 October 2015 at 21:17:43 UTC, Jacob wrote:

 On Tuesday, 13 October 2015 at 18:13:33 UTC, Ice Cream Overload 
 wrote:

 On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote:

 On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream 
 Overload wrote:

 [...]


 Passion or not,

 "If you are wondering why I'm inflammatory, here you go. You 
 are pulling me the old prove a negative trick. You have good 
 evidence that DIP25 is good design ? Good, because I have 
 none. And that's my proof. As long as I have no evidence that 
 DIP25 is good, DIP25 is bad."

 That statement shows a lot of arrogance.

 It he not really just saying "I have no clue if X is true, 
 but since I don't know, I'll just assume it's false and 
 assume you are wrong.".

 That's not very logical. Why wouldn't he just as well assume 
 X is true?

 The fact is, he can't and shouldn't make such statements 
 about X if he has no "evidence" about it.

 Instead, wouldn't the proper approach be to discuss, learn, 
 and share what one things in a positive way so everyone can 
 learn about X and reach a more intelligent understanding of 
 it?

 Hilter was very passionate too, are you saying he was right?


 Since you brought up the Hilter reference, it's useful to 
 mention that he was successful because people blindly followed 
 him and his actions...

 But really...do we always have to resort to Hitler references 
 when discussing disagreements? Seems very cliche.


 It's only cliche if you aren't interested in the truth. It 
 doesn't matter if I used Hilter or any other person that was 
 "passionate" but wrong.

 Hilter is just the greatest example and most obvious example. 
 If you actually understand the issue, you would realize it has 
 nothing to do with Hilter.


I don't know. Whenever someone runs out of arguments and is 
forced to go on the attack, Hitler seems to be the first insult 
people reach for. Thus once I see Hitler references pop up, I 
throw out that individual's credibility. Solves bandwidth 
problems of who is worth listening to.

Some soul searching for you, is I'd refrain from such references 
as it only hurts your credibility. Not the one you sling it at.
Oct 13 2015
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Oct 13, 2015 at 09:37:27PM +0000, I SCREAM for ICECream via
Digitalmars-d wrote:

 On Tuesday, 13 October 2015 at 21:17:43 UTC, Jacob wrote:

On Tuesday, 13 October 2015 at 18:13:33 UTC, Ice Cream Overload wrote:

On Tuesday, 13 October 2015 at 12:16:26 UTC, Jacob wrote:

On Monday, 12 October 2015 at 19:35:33 UTC, Ice Cream Overload wrote:

[...]


Passion or not,

"If you are wondering why I'm inflammatory, here you go. You are
pulling me the old prove a negative trick. You have good evidence
that DIP25 is good design ? Good, because I have none. And that's
my proof.  As long as I have no evidence that DIP25 is good, DIP25
is bad."

That statement shows a lot of arrogance.

It he not really just saying "I have no clue if X is true, but
since I don't know, I'll just assume it's false and assume you are
wrong.".

That's not very logical. Why wouldn't he just as well assume X is
true?

The fact is, he can't and shouldn't make such statements about X if
he has no "evidence" about it.

Instead, wouldn't the proper approach be to discuss, learn, and
share what one things in a positive way so everyone can learn about
X and reach a more intelligent understanding of it?

Hilter was very passionate too, are you saying he was right?


Since you brought up the Hilter reference, it's useful to mention
that he was successful because people blindly followed him and his
actions...

But really...do we always have to resort to Hitler references when
discussing disagreements? Seems very cliche.


It's only cliche if you aren't interested in the truth. It doesn't
matter if I used Hilter or any other person that was "passionate" but
wrong.

Hilter is just the greatest example and most obvious example. If you
actually understand the issue, you would realize it has nothing to do
with Hilter.


 
 I don't know. Whenever someone runs out of arguments and is forced to
 go on the attack, Hitler seems to be the first insult people reach
 for. Thus once I see Hitler references pop up, I throw out that
 individual's credibility.  Solves bandwidth problems of who is worth
 listening to.
 
 Some soul searching for you, is I'd refrain from such references as it
 only hurts your credibility. Not the one you sling it at.


Godwin's Law[1] has been invoked, boys and girls.  The game is now over.
Thanks for playing.  You may go home now.  Have a nice day.

[1] https://en.wikipedia.org/wiki/Godwin's_law


T

-- 
"I'm not childish; I'm just in touch with the child within!" - RL
Oct 13 2015
next sibling parent Give me the Ice Cream of the World <thegreatone google.com> writes:
On Tuesday, 13 October 2015 at 21:46:22 UTC, H. S. Teoh wrote:

 On Tue, Oct 13, 2015 at 09:37:27PM +0000, I SCREAM for ICECream 
 via Digitalmars-d wrote:

 On Tuesday, 13 October 2015 at 21:17:43 UTC, Jacob wrote:

[...]


 
 I don't know. Whenever someone runs out of arguments and is 
 forced to go on the attack, Hitler seems to be the first 
 insult people reach for. Thus once I see Hitler references pop 
 up, I throw out that individual's credibility.  Solves 
 bandwidth problems of who is worth listening to.
 
 Some soul searching for you, is I'd refrain from such 
 references as it only hurts your credibility. Not the one you 
 sling it at.


 Godwin's Law[1] has been invoked, boys and girls.  The game is 
 now over. Thanks for playing.  You may go home now.  Have a 
 nice day.

 [1] https://en.wikipedia.org/wiki/Godwin's_law


 T


Amazingly true :D
Oct 13 2015
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 13 October 2015 at 21:46:22 UTC, H. S. Teoh wrote:

 Godwin's Law[1] has been invoked, boys and girls.  The game is 
 now over. Thanks for playing.  You may go home now.  Have a 
 nice day.


Neh, Godwin's law only states that as time progresses the 
probability of invoking "Hitler" as an example increases, but it 
says nothing about whether the comparison was useful or not.
Oct 13 2015
parent reply Ice Cream Madness <thegreatone google.com> writes:
On Tuesday, 13 October 2015 at 22:00:28 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 13 October 2015 at 21:46:22 UTC, H. S. Teoh wrote:

 Godwin's Law[1] has been invoked, boys and girls.  The game is 
 now over. Thanks for playing.  You may go home now.  Have a 
 nice day.


 Neh, Godwin's law only states that as time progresses the 
 probability of invoking "Hitler" as an example increases, but 
 it says nothing about whether the comparison was useful or not.


The D language, does have a 'feature' creep problem.
Oct 13 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday, 13 October 2015 at 22:10:30 UTC, Ice Cream Madness 
wrote:

 The D language, does have a 'feature' creep problem.


Maybe, but at this point, I think that C++ is actually getting 
features faster than D is. We talk about adding features or 
tweaking existing features to fix problems, but we're long past 
the point where we're frequently adding features. If anything, 
the typical complaint now is that we're _not_ making changes, 
even when we've been discussing them (e.g. this very thread was 
started to ask what the deal with a proposed change is, because 
it hasn't gone anywhere yet, and it's a change proposed by Walter 
and Andrei no less).

- Jonathan M Davis
Oct 13 2015
next sibling parent reply Ice Cream Desserter <thegreatone google.com> writes:
On Tuesday, 13 October 2015 at 22:20:02 UTC, Jonathan M Davis 
wrote:

 On Tuesday, 13 October 2015 at 22:10:30 UTC, Ice Cream Madness 
 wrote:

 The D language, does have a 'feature' creep problem.


 Maybe, but at this point, I think that C++ is actually getting 
 features faster than D is. We talk about adding features or 
 tweaking existing features to fix problems, but we're long past 
 the point where we're frequently adding features. If anything, 
 the typical complaint now is that we're _not_ making changes, 
 even when we've been discussing them (e.g. this very thread was 
 started to ask what the deal with a proposed change is, because 
 it hasn't gone anywhere yet, and it's a change proposed by 
 Walter and Andrei no less).

 - Jonathan M Davis


C++ also has a 'feature' creep problem.
Oct 13 2015
parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 13 October 2015 at 22:21:24 UTC, Ice Cream Desserter 
wrote:

 On Tuesday, 13 October 2015 at 22:20:02 UTC, Jonathan M Davis 
 wrote:

 On Tuesday, 13 October 2015 at 22:10:30 UTC, Ice Cream Madness 
 wrote:

 The D language, does have a 'feature' creep problem.


 Maybe, but at this point, I think that C++ is actually getting 
 features faster than D is. We talk about adding features or 
 tweaking existing features to fix problems, but we're long 
 past the point where we're frequently adding features. If 
 anything, the typical complaint now is that we're _not_ making 
 changes, even when we've been discussing them (e.g. this very 
 thread was started to ask what the deal with a proposed change 
 is, because it hasn't gone anywhere yet, and it's a change 
 proposed by Walter and Andrei no less).

 - Jonathan M Davis


 C++ also has a 'feature' creep problem.


If I may the problem is not that much how feature rich the 
language is, but how orthogonal these feature are. The less 
orthogonal they are, the more special cases you get when they 
interact with each other.
Oct 13 2015
prev sibling next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 13 October 2015 at 22:20:02 UTC, Jonathan M Davis 
wrote:

 Maybe, but at this point, I think that C++ is actually getting 
 features faster than D is.


And as a result advanced c++ analyzers work on lowlevel IR, not 
at the language level.

That has many consequences, one is time consumption, another is 
accuracy.


 We talk about adding features or tweaking existing features to 
 fix problems, but we're long past the point where we're 
 frequently adding features.


More syntax sugar is not problematic. It is what you have left 
when you factor out everything that is expressible by other 
mechanisms that matters.
Oct 13 2015
prev sibling parent rsw0x <anonymous anonymous.com> writes:
On Tuesday, 13 October 2015 at 22:20:02 UTC, Jonathan M Davis 
wrote:

 On Tuesday, 13 October 2015 at 22:10:30 UTC, Ice Cream Madness 
 wrote:

 The D language, does have a 'feature' creep problem.


 Maybe, but at this point, I think that C++ is actually getting 
 features faster than D is. We talk about adding features or 
 tweaking existing features to fix problems, but we're long past 
 the point where we're frequently adding features. If anything, 
 the typical complaint now is that we're _not_ making changes, 
 even when we've been discussing them (e.g. this very thread was 
 started to ask what the deal with a proposed change is, because 
 it hasn't gone anywhere yet, and it's a change proposed by 
 Walter and Andrei no less).

 - Jonathan M Davis


DIP74 is also just an alternative solution to a problem that has 
a long history, and nothing is being done about it.
Oct 13 2015
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 13 October 2015 at 21:17:43 UTC, Jacob wrote:

 It's only cliche if you aren't interested in the truth. It 
 doesn't matter if I used Hilter or any other person that was 
 "passionate" but wrong.


You are right, but Andrei and Walter often go into "passionate 
but wrong" mode too... It's a curse of D and probably will keep 
it from reaching a mature state. DIP25 is no exception.

In language design it is better to have a small set of features  
in the core language that are easy to reason about as a whole. D 
has many simple features, but the combinatorial explosion is 
quite high.

For instance, how is DIP25 going to work with coroutines that 
yield? So you transfer a refererence by "return ref", then what 
you call yields and the object is destructed. When the coroutine 
is later resumed the object no longer exists, so you have a 
memory unsafe situation. So then you have to add the requirement 
that "return ref" functions cannot call anything that yields... 
After some time you realize that it is possible to pass in a 
lambda that can destroy the object. Then you forbid passing in 
lambdas...

What one should have realized is that if reasoning about 
correctness isn't obvious then you need proofs. There is no 
obvious memory safety in D and there are no proofs.

Meaning, you would be better off using a general static analyzer 
because you get more flexibility and the same level of memory 
safety.
Oct 13 2015
prev sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 12 October 2015 at 16:02, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 DIP69 is obviously known to me because my name is on it.


What is the problem with DIP69? It looks like a great direction to me.
It's almost exactly what I've been begging for for years!
As long as methods can overload on scope, then it can be used to
implement efficient RC. Also many other situations are improved in
general.

The only thing there that's not clear to me from DIP69 is:
  scope ref T func(scope ref T x) { return x; }

This needs to work, or you can't chain... but it doesn't look like
it's addressed?
Oct 13 2015
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/11/2015 10:35 PM, Andrei Alexandrescu wrote:

 ...

 1. You say that DIP25 is a failure. More so, you demand that is admitted
 without evidence.


FWIW, DIP25 is insufficiently formal and/or incorrect.
I have been able to find those holes in the implementation pretty 
quickly (I'll also put them to bugzilla):


---

enum N=100;
ref int[N] foo(return ref int[N] s) safe{ return s; }
int[N]* bar(return ref int[N] s) safe{ return &foo(s); }
ref int[N] baz() safe{ int[N] s; return *bar(s); }

// for illustration:
void bang(ref int[N] x) safe{ x[]=0x25BAD; }
void main() safe{ bang(baz()); }

---

enum N=100;
ref int[N] fun(ref int[N] x) safe{
     ref int[N] bar(){ return x; }
     return bar();
}
ref int[N] hun() safe{
     int[N] k;
     return fun(k);
}

// for illustration:
void bang(ref int[N] x) safe{ x[]=0x25BAD; }
void main() safe{ bang(hun); }

---

ref int foo() safe{
     struct S{
         int x;
         ref int bar()return{ return x; }
     }
     return S().bar();
}

// fun fact: paste it twice into a D file to get an ICE :-)

---

ref int foo() safe{
     int x;
     struct S{
         ref int bar(){ return x; }
     }
     return S().bar();
}
// (the DIP quite explicitly allows this,
// as x lives longer than 'this'.)

---

Basically every technique I have tried to use for escaping references to 
locals and non-return parameters directly after reading the DIP has been 
fruitful. This seems like a pretty obvious failure to achieve "with the 
proposed rules, it is impossible in safe code to have ref refer to a 
destroyed object" to me.
Had the DIP explicitly mentioned all necessary rules in more detail, 
there would probably be fewer cases like this. Its true complexity would 
then also be more apparent. (Be it low or high, but at least it would be 
a leveled playing ground.)


 What I see is a feature that solves one problem, and
 solves it well: annotating a function that returns a reference to its
 argument.


When you say "well", what do you mean?

My personal definition of "well" for such cases is that all the 
contextual information needed to type check an expression can be encoded 
into the type signature of a function. (Unfortunately, D violates this 
rule quite often.) I don't think there is a syntax-free way of fixing 
the holes which satisfies this, as the return attribute inherently 
throws away e.g. the information about which nested scope a reference 
returned after being accessed through a context pointer refers to.



 The syntactic cost is low,


I have noticed you tend to attach a large weight to "syntactic cost". 
How to evaluate syntactic cost of a feature? I think concepts that exist 
but have no syntax have a larger cost.


 and the impact on safety is positive.
 Walter and I think it is the simplest solution of all considered.


I'm not entirely sure. It has the lowest amount of syntax though.
Oct 11 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/12/15 2:39 AM, Timon Gehr wrote:

 On 10/11/2015 10:35 PM, Andrei Alexandrescu wrote:

 ...

 1. You say that DIP25 is a failure. More so, you demand that is admitted
 without evidence.


 FWIW, DIP25 is insufficiently formal and/or incorrect.


That I agree with. We need to get a lot better at making precise DIPs.


 I have been able to find those holes in the implementation pretty
 quickly (I'll also put them to bugzilla):


Thanks. That's great work!


Andrei
Oct 11 2015
prev sibling parent reply Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Sunday, 11 October 2015 at 20:35:05 UTC, Andrei Alexandrescu 
wrote:

 On 10/11/15 9:57 PM, deadalnix wrote:

 On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote:

 On Sunday, 11 October 2015 at 13:51:18 UTC, Andrei 
 Alexandrescu wrote:

 Walter and I are happy with DIP25, and the fact of the 
 matter is we
 weren't surprised more complementary work is needed. So no, 
 I won't
 acknowledge what I don't believe.


 That is an empty statement. What is there to be happy about ?

 Also the complementary argument pretty much destroy the best 
 argument
 you and Walter made for DIP25 : it is simple. I mean, one 
 need to look
 at the big picture. DIP25 + complementary addition is not 
 simple anymore.


 I'd say the one way to get things looked at seriously is to 
 create a
 DIP. That's no guarantee it will be accepted but there is a 
 guarantee
 that our chat at DConf is not sufficient even as a basis for 
 further
 study.


 Yeah there are IRL discussion, there are many posts in the 
 forum,
 there are by mail discussions at DIP25 creation time, there 
 are at
 least one DIP already.

 The only rebuttal to all of this is "Walter and I are happy 
 with
 DIP25, and the fact of the matter", while everybody else is 
 wondering
 what there is to be happy about.


 Also, I'm sorry but there is no me writing once again a 
 document about
 what alternative are possible.


 Could you please point to the document you have already written?


 Spending hours to write documents so that
 one is answered something along the line of "we are happy with 
 the other
 thing, but we can't give any reason why" is something I've 
 engaged in
 several time in already, and has no desire to indulge into 
 this if I
 have reason to think the same will happen. Your answer here 
 are telling
 me one thing: it won't be taken seriously.


 There's a bit of a stalemate here. So we have:

 1. You say that DIP25 is a failure. More so, you demand that is 
 admitted without evidence. What I see is a feature that solves 
 one problem, and solves it well: annotating a function that 
 returns a reference to its argument. The syntactic cost is low, 
 the impact on existing code is small, and the impact on safety 
 is positive. Walter and I think it is the simplest solution of 
 all considered.


Except that it isn't a solution to the problems it was claimed to 
solve. For example, Walter tried to build a safe RCArray 
implementation with it, and it turned out that it's still unsafe 
in the presence of aliasing.

I still see DIP25 as going in the right direction, but it from my 
POV it must at least a) be usable with all kinds of references, 
not just `ref`, and without double indirections, b) solve the 
problems with global variables (probably easy, just make them 
unsafe), and c) keep track of aliases or otherwise handle them 
correctly to avoid the RCArray problems.

(An additional nice-to-have feature would be a method to make a 
variable inaccessible that can be enforced at compile time, 
because that will enable various uniqueness-related things. This 
is closely related, but not a necessity.)


 2. You refuse to write a DIP under the assumption it will not 
 be taken seriously. Conversely if you do write a DIP there is 
 an expectation it will be approved just because you put work in 
 it. I don't think rewarding work is the right way to go. We 
 need to reward good work. The "work" part (i.e. a DIP) is a 
 prerequisite; you can't demand to implement a complex feature 
 based on posts and discussions.


The problem is the signals we get from you and Walter. From 
various posts (or lack of response to certain questions) and the 
way you've treated this entire topic so far, I got the impression 
that you both are opposed to anything similar to Rust's approach. 
Unfortunately, we know that Rust's approach (or other solutions 
involving linear type systems) is the only thing that can provide 
a compiler-verifiable free-at-runtime solution. That's the real 
stalemate as I see it. I hope you see how that's not particularly 
motivating.
Oct 12 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/12/15 4:38 PM, Marc Schütz wrote:

 The problem is the signals we get from you and Walter. From various
 posts (or lack of response to certain questions) and the way you've
 treated this entire topic so far, I got the impression that you both are
 opposed to anything similar to Rust's approach. Unfortunately, we know
 that Rust's approach (or other solutions involving linear type systems)
 is the only thing that can provide a compiler-verifiable free-at-runtime
 solution. That's the real stalemate as I see it. I hope you see how
 that's not particularly motivating.


Yes, I agree. Experience with Rust is still young, but there seems to 
have already been a backlash; programmers try it and it's just too 
arcane to use in constant preoccupation about them ownership rules.

Copying linear types and going whole-hog stealing the ownership system 
from Rust doesn't sound like the best strategy to Walter and myself. At 
least one PL researcher whose opinion I trust believes linear types 
don't have a future.

D has its own context and its own approach to matters. I believe 
creative solutions are possible that achieve much of what we need 
without going the Rust way, which seems not appropriate for us.

I agree that DIP74 has problems; it has had several corner cases that 
needed patching. That's a signal DIP74 doesn't quite cut with the grain. 
What we need here is good creative work that takes us where we want to 
be without breaking the complexity bank.

BTW where we want to be is: expressive reference counting without loss 
of  safe-ty and hopefully good uniqueness control again with  safe-ty 
and possibly a bit of compiler help.


Andrei
Oct 12 2015
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 10/12/2015 06:49 PM, Andrei Alexandrescu wrote:

 Experience with Rust is still young, but there seems to have already
 been a backlash; programmers try it and it's just too arcane to use
 in constant preoccupation about them ownership rules. ...



 Copying linear types and going whole-hog stealing the ownership system
 from Rust doesn't sound like the best strategy to Walter and myself.


D can do ownership without copying the details of the Rust way. I think 
that Marc's point was that basic similarities should be allowed, because 
the design space isn't that vast.


 At least one PL researcher whose opinion I trust believes linear types
 don't have a future.


We already have linear types. ( disable this(this)).


 D has its own context and its own approach to matters. I believe creative
 solutions are possible that achieve much of what we need without going
 the Rust way, which seems not appropriate for us.


What do we need?
Oct 12 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/13/15 3:09 AM, Timon Gehr wrote:

 On 10/12/2015 06:49 PM, Andrei Alexandrescu wrote:

 Experience with Rust is still young, but there seems to have
 already been a backlash; programmers try it and it's just too
 arcane to use in constant preoccupation about them ownership rules.
 ...



 Copying linear types and going whole-hog stealing the ownership
 system from Rust doesn't sound like the best strategy to Walter and
 myself.


 D can do ownership without copying the details of the Rust way. I
 think that Marc's point was that basic similarities should be
 allowed, because the design space isn't that vast.


 At least one PL researcher whose opinion I trust believes linear
 types don't have a future.


 We already have linear types. ( disable this(this)).


That much is not too complicated.


 D has its own context and its own approach to matters. I believe
 creative solutions are possible that achieve much of what we need
 without going the Rust way, which seems not appropriate for us.


 What do we need?


Next paragraph after the one you quoted:


 BTW where we want to be is: expressive reference counting without
 loss of  safe-ty and hopefully good uniqueness control again with
  safe-ty and possibly a bit of compiler help.



Andrei
Oct 12 2015
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote:

 The only rebuttal to all of this is "Walter and I are happy 
 with DIP25, and the fact of the matter", while everybody else 
 is wondering what there is to be happy about.


To be fair, you haven't really said much better. You're claiming 
that it's clear that it's a failure, whereas Andrei is saying 
that he doesn't see a problem with it. No concrete arguments are 
being given. The closest is that you think that the fact that we 
need something like DIP 74 means that DIP 25's advantage of being 
simple is lost and that that somehow shows that DIP 25 is a 
failure.

I don't think that it's at all clear that DIP 25 and DIP 74 are 
even particularly related (and Manu has stated the same). One 
deals with escaping ref parameters (which doesn't involved 
reference counting), whereas the other has to do with reference 
counting. It may very well be that there's a way to cover both 
issues with some sort of ownership model, but that's not clear 
from simply looking at the DIPs or the current discussion. All we 
really have is your assertion that there's a better way.

I'm completely open to there being a better solution than DIP 25 
or DIP 74, but I have yet to see one explained. The only 
alternatives to DIP 74 that I've really seen are suggestions that 
we should be able to do it with library types (like C++ does) and 
not require built-in ref-counting. If there have been any 
in-depth explanations or discussions on how an ownership model 
could solve this, I haven't seen them (though there may very well 
have been one in a previous thread that I missed), so I really 
don't know what that would even entail.

The only place that it's clear to me that an ownership model of 
some kind would be nice would be one related to threads so that 
it we could safely pass mutable data between threads, but 
honestly, I'm okay with it being unsafe to pass mutable data 
across threads. D's threading model makes it pretty easy to get 
that right, and most programs don't need it anyway.

But I have no idea how an ownership model for memory in general 
would work. I'd probably have a much better idea if I had read up 
on Rust (which I really haven't), but every discussion I've seen 
on Rust's model seems to indicate that it gets pretty complicated 
- complicated enough that while it might be more correct, it 
risks being unusable for the average programmer. I don't know how 
true that is, but I do think that if we're going to seriously 
discuss an ownership model in D, we need something concrete 
discuss and debate. I know that writing a DIP takes time (even a 
small one; it surprised me how long it took to write DIP 82 given 
how simple it is, and I've written DIPs before), but if we don't 
have something like that to discuss, then we're just going to be 
going in circles here. If you have a concrete proposal that you 
think would really benefit D, then please write up a DIP for it. 
We could all be way better off for it. Maybe it'll be rejected, 
but without it, we're definitely not going to end up with some 
kind of ownership model like you want. And if you're right, then 
we will be worse off. Concrete proposals are required to have 
concrete discussions and move forward.

- Jonathan M Davis
Oct 11 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 20:56:28 UTC, Jonathan M Davis 
wrote:

 On Sunday, 11 October 2015 at 18:52:44 UTC, deadalnix wrote:

 The only rebuttal to all of this is "Walter and I are happy 
 with DIP25, and the fact of the matter", while everybody else 
 is wondering what there is to be happy about.


 To be fair, you haven't really said much better. You're 
 claiming that it's clear that it's a failure, whereas Andrei is 
 saying that he doesn't see a problem with it. No concrete 
 arguments are being given. The closest is that you think that 
 the fact that we need something like DIP 74 means that DIP 25's 
 advantage of being simple is lost and that that somehow shows 
 that DIP 25 is a failure.


That's not true. I explained in this thread why it is too 
limited, most notably :

1/ It was barely able to provide a non GC managed array type.

More details here: 
http://forum.dlang.org/thread/mcg8qq$1mbr$1 digitalmars.com

Notably, it is impossible to reclaim the memory eagerly, but only 
to clean all memory at once when all reference to one of the 
memory location is gone. This in turn lead to the need to pretty 
much have a memory allocator in the array. And jump through many 
hoops.

2/ It is completely unable to handle reference types (hence 
DIP74).

Note that if you don't have indirection, there is no memory 
allocation problem to begin with as value type will be either on 
stack or involve some reference at some point.

But this whole thing is really a shift of the burden of proof. 
What do DIP25 enabled that is really worth it ?
Oct 11 2015
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 22:07:44 UTC, deadalnix wrote:

 That's not true. I explained in this thread why it is too 
 limited, most notably :

 1/ It was barely able to provide a non GC managed array type.

 More details here: 
 http://forum.dlang.org/thread/mcg8qq$1mbr$1 digitalmars.com

 Notably, it is impossible to reclaim the memory eagerly, but 
 only to clean all memory at once when all reference to one of 
 the memory location is gone. This in turn lead to the need to 
 pretty much have a memory allocator in the array. And jump 
 through many hoops.

 2/ It is completely unable to handle reference types (hence 
 DIP74).

 Note that if you don't have indirection, there is no memory 
 allocation problem to begin with as value type will be either 
 on stack or involve some reference at some point.

 But this whole thing is really a shift of the burden of proof. 
 What do DIP25 enabled that is really worth it ?


DIP 25 fixes a serious flaw in ref. It makes it possible to 
guarantee that ref is  safe. Without it, ref is not  safe. As I 
understand it, that's the whole point of DIP 25, and it has 
nothing to do with reference counting at all. It doesn't even 
really have anything to do with the heap beyond the fact that a 
ref parameter might refer to an object on the heap - the safety 
problem is with the stack, not the heap.

Walter's proposal in that link does take advantage of the fact 
that DIP 25 restricts what can be returned by ref, but that's as 
close as DIP 25 gets to ref-counting, whereas all of the rest of 
the stuff you're talking about here has to do with ref-counting.

DIP 25 is intended to plug the holes in ref to solve the ref 
safety problem, whereas DIP 74 is intended to build ref-counting 
into the language so that it can be done safely. They both relate 
to  safe code, but as far as I can tell, they're pretty much 
orthogonal, and arguing against flaws in DIP 74 doesn't really 
show flaws in DIP 25. When you're arguing about them together 
rather than separately, it seems like you're seeing some other 
part of the puzzle that I'm not.

- Jonathan M Davis
Oct 11 2015
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 20:56:28 UTC, Jonathan M Davis 
wrote:

 To be fair, you haven't really said much better. You're 
 claiming that it's clear that it's a failure, whereas Andrei is 
 saying that he doesn't see a problem with it. No concrete 
 arguments are being given. The closest is that you think that 
 the fact that we need something like DIP 74 means that DIP 25's 
 advantage of being simple is lost and that that somehow shows 
 that DIP 25 is a failure.


OK first thing first, that is not how that works. DIP25 has been 
out for a while, so one should have something to show for it. 
What good thing came out of DIP25 ? That should be what is 
discussed here rather than having to argue why it is not good 
enough.

See my other messages here to see what isn't good with it. It 
simply isn't allowing enough to pay for itself.


 I don't think that it's at all clear that DIP 25 and DIP 74 are 
 even particularly related (and Manu has stated the same).


They are the same thing. If you can guarantee that some reference 
does not escape in uncontrollable manner, you can have ref 
counting as library. The need for DIP74 only arise because DIP25 
cannot ensure that the reference does not escape.


 But I have no idea how an ownership model for memory in general 
 would work. I'd probably have a much better idea if I had read 
 up on Rust (which I really haven't), but every discussion I've 
 seen on Rust's model seems to indicate that it gets pretty 
 complicated - complicated enough that while it might be more 
 correct, it risks being unusable for the average programmer. I 
 don't know how true that is, but I do think that if we're going 
 to seriously discuss an ownership model in D, we need something 
 concrete discuss and debate. I know that writing a DIP takes 
 time (even a small one; it surprised me how long it took to 
 write DIP 82 given how simple it is, and I've written DIPs 
 before), but if we don't have something like that to discuss, 
 then we're just going to be going in circles here. If you have 
 a concrete proposal that you think would really benefit D, then 
 please write up a DIP for it. We could all be way better off 
 for it. Maybe it'll be rejected, but without it, we're 
 definitely not going to end up with some kind of ownership 
 model like you want. And if you're right, then we will be worse 
 off. Concrete proposals are required to have concrete 
 discussions and move forward.

 - Jonathan M Davis


Yes, you should definitively read on Rust, or read this :
https://github.com/isocpp/CppCoreGuidelines/blob/09aef9bd86d933bc1e1ffe344eb2e73d2de15685/docs/Lifetimes%20I%20and%20II%20-%20v0.9.1.pdf

Which is basically C++ adopting the idea, but no, it doesn't come 
from Rut, no, I tell you, C++ invented a long time ago, C++ 
invented everything, stop doubting us.

Things is, Rust has that one mechanism to do everything. As a 
result it is fairly complex, because it has to handle all common 
cases. We can do something fairly similar, yet simpler, if we 
accept to rely on the GC or unsafe constructs for things that do 
not fit well in the model.
Oct 11 2015
prev sibling next sibling parent reply Freddy <Hexagonalstar64 gmail.com> writes:
On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 [...]


Speaking of DIP74 can't we just wrap a class in a struct with use 
reference counting with and use alias this?
Oct 10 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:

 On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 [...]


 Speaking of DIP74 can't we just wrap a class in a struct with 
 use reference counting with and use alias this?


You can. It is not safe, but it will do. Using type qualifier, 
one can get better than C++'s shared_ptr already.
Oct 10 2015
parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 15:57, deadalnix via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:

 On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 [...]



 Speaking of DIP74 can't we just wrap a class in a struct with use
 reference counting with and use alias this?



 You can. It is not safe, but it will do. Using type qualifier, one can get
 better than C++'s shared_ptr already.


How?
Oct 11 2015
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 07:01:35 UTC, Manu wrote:

 On 11 October 2015 at 15:57, deadalnix via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:

 On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 [...]



 Speaking of DIP74 can't we just wrap a class in a struct with 
 use reference counting with and use alias this?



 You can. It is not safe, but it will do. Using type qualifier, 
 one can get better than C++'s shared_ptr already.


 How?


In C++, you need to assume things are shared, and, as such, use 
thread safe inc/dec . That means compiler won't be able to 
optimize them. D can do better as sharedness is part of the type 
system.
Oct 11 2015
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sun, 11 Oct 2015 07:32:26 +0000
schrieb deadalnix <deadalnix gmail.com>:


 In C++, you need to assume things are shared, and, as such, use 
 thread safe inc/dec . That means compiler won't be able to 
 optimize them. D can do better as sharedness is part of the type 
 system.


 
With the little nag that `shared` itself is not fleshed out.

-- 
Marco
Oct 11 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, 12 October 2015 at 03:59:04 UTC, Marco Leise wrote:

 Am Sun, 11 Oct 2015 07:32:26 +0000
 schrieb deadalnix <deadalnix gmail.com>:


 In C++, you need to assume things are shared, and, as such, 
 use thread safe inc/dec . That means compiler won't be able to 
 optimize them. D can do better as sharedness is part of the 
 type system.


 With the little nag that `shared` itself is not fleshed out.


Well, it really should be better fleshed out, but the reality of 
the matter is that it actually works pretty well as it is. The 
problem is primarily that it's a pain to use - and to a certain 
extent, that's actually a good thing, but it does make it harder 
to use correctly. Better support for detecting when shared can be 
safely cast away would be nice. e.g. being able to do something 
like

synchronized(mutex_for_foo)
{
     // foo is now implicitly treated as thread-local
}
// foo is now treated as shared again

would be nice. However, figuring out how to do that safely is 
very tricky.

Ultimately though, I think that the problem with shared generally 
comes down to folks not liking the fact that you pretty much 
can't do anything with a shared variable until you cast away 
shared, but the fact that it won't let you do much is actually 
protecting you.

We do need to take another look at shared and see what we can do 
to improve it, but I'm sure that it's ultimately going to be much 
different from how it is now.

- Jonathan M Davis
Oct 11 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/12/15 7:19 AM, Jonathan M Davis wrote:

 On Monday, 12 October 2015 at 03:59:04 UTC, Marco Leise wrote:

 Am Sun, 11 Oct 2015 07:32:26 +0000
 schrieb deadalnix <deadalnix gmail.com>:


 In C++, you need to assume things are shared, and, as such, use
 thread safe inc/dec . That means compiler won't be able to optimize
 them. D can do better as sharedness is part of the type system.


 With the little nag that `shared` itself is not fleshed out.


 Well, it really should be better fleshed out, but the reality of the
 matter is that it actually works pretty well as it is. The problem is
 primarily that it's a pain to use - and to a certain extent, that's
 actually a good thing, but it does make it harder to use correctly.


Yah, I'd like to make "finalizing the language" a priority going 
forward, and finalizing shared is a big topic. It's hard to present to 
the world a language with fuzzy corners. -- Andrei
Oct 12 2015
parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 12 Oct 2015 10:28:55 +0300
schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:


 On 10/12/15 7:19 AM, Jonathan M Davis wrote:

 On Monday, 12 October 2015 at 03:59:04 UTC, Marco Leise wrote:

 Am Sun, 11 Oct 2015 07:32:26 +0000
 schrieb deadalnix <deadalnix gmail.com>:


 In C++, you need to assume things are shared, and, as such, use
 thread safe inc/dec . That means compiler won't be able to optimize
 them. D can do better as sharedness is part of the type system.


 With the little nag that `shared` itself is not fleshed out.


 Well, it really should be better fleshed out, but the reality of the
 matter is that it actually works pretty well as it is. The problem is
 primarily that it's a pain to use - and to a certain extent, that's
 actually a good thing, but it does make it harder to use correctly.


 
 Yah, I'd like to make "finalizing the language" a priority going 
 forward, and finalizing shared is a big topic. It's hard to present to 
 the world a language with fuzzy corners. -- Andrei


Wouldn't it be great if everyone took notes of the currently
perceived shortcomings of shared so that there is a pile of
use- and corner-cases to look at for a redesign?
Then you would filter by valid use case and undesired usage and
practically had everyone's input already under consideration
when it hits the discussion forums.
My own experience is what Jonathan describes. It is annoying
in a good way. It makes you wait and think twice and more
often than not you really attempted an illegal access on shared
data. This went well up to a point when I had a component that
held references to shared things. There is no such thing as a
shared struct with shared fields in the type system. Once you
share the outer struct, the inner stuff's shared status is
merged with the outer and you can't cast it back to what it
was before. So it works for plain integral types, but
aggregates and what others mentioned about thread
local GC heaps are my big question marks.

-- 
Marco
Oct 12 2015
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Monday, 12 October 2015 at 19:35:34 UTC, Marco Leise wrote:

 Wouldn't it be great if everyone took notes of the currently
 perceived shortcomings of shared so that there is a pile of
 use- and corner-cases to look at for a redesign?


The problem with shared is that shared should not be constant 
over time, it should be related to behavioural typing/type state.

Synchronization is a temporal factor not a static typing factor.
Oct 13 2015
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 13 October 2015 at 12:40:46 UTC, Ola Fosheim Grøstad 
wrote:

 On Monday, 12 October 2015 at 19:35:34 UTC, Marco Leise wrote:

 Wouldn't it be great if everyone took notes of the currently
 perceived shortcomings of shared so that there is a pile of
 use- and corner-cases to look at for a redesign?


 The problem with shared is that shared should not be constant 
 over time, it should be related to behavioural typing/type 
 state.

 Synchronization is a temporal factor not a static typing factor.


That said, if you have:

1. writer-ownership as a feature
2. mark variables as "only writable by one owner"

Then the compiler can drop some read locks for the _owning_ 
thread.

But D does not have ownership as a feature beyond "thread local 
memory"?
Oct 13 2015
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 13 October 2015 at 13:02:43 UTC, Ola Fosheim Grøstad 
wrote:

 That said, if you have:

 1. writer-ownership as a feature
 2. mark variables as "only writable by one owner"

 Then the compiler can drop some read locks for the _owning_ 
 thread.

 But D does not have ownership as a feature beyond "thread local 
 memory"?


I think the D designers should take a long and hard look at Pony:

isolated: deny global read/write, deny local read/write
transition: deny global read/write, deny local write
reference: deny global read/write
value: deny global write, deny local write
box: deny global write aliases
tag: allow all aliases

_loosely_ translated:

isolated: void*
transition: const T*
reference: T*
value: immutable T*
box: globally as shared const T*, locally as shared T*
tag: shared T*

So D lacks some way to express the "box" type?
Oct 13 2015
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Tuesday, 13 October 2015 at 16:04:07 UTC, Ola Fosheim Grøstad 
wrote:

 _loosely_ translated:

 isolated: void*
 transition: const T*
 reference: T*
 value: immutable T*
 box: globally as shared const T*, locally as shared T*
 tag: shared T*


The above turned out rather allegorical (and possibly confusing), 
the following is a lockfree interpretation that is a little bit 
more useful and somewhat more accurate:

ISOLATED: Unique!T
referable as void*, shared void*
transferrable to other threads

TRANSITION: Unique!T
referable as const T*, void*, shared void*
convertible to immutable T*

REFERENCE: T*
referable as T*, const T*, void*, shared void*

VALUE: immutable T*
referable as immutable T*, const T*, shared const T*, void*, 
shared void*
shareable with other threads

BOX: const T*
referable as const T*, void*, shared void*
aliasing with T*, immutable T*

TAG: void T* or shared void*
referable as void T*, shared void*
aliasing with all types as it is a pure identity


Then we can do a new interpretation with locking/synchronization 
and get something like:

LOCKED BOX: shared const T*
referable as shared const T*, void*, shared void*
aliasing with shared T*, immutable T*

LOCKED TAG: shared T*
referable as shared T*, shared const T*, void T*, shared void*

More or less...
Oct 13 2015
prev sibling next sibling parent reply Freddy <Hexagonalstar64 gmail.com> writes:
On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:

 Speaking of DIP74 can't we just wrap a class in a struct with 
 use reference counting with and use alias this?


Also how will DIP74 work with incremental compilation?
---
extern (D) class RcClass;

void func(RcClass a)
{
     //opps
     auto b = a;
     return;
}
---
Oct 10 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 16:06, Freddy via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:

 Speaking of DIP74 can't we just wrap a class in a struct with use
 reference counting with and use alias this?



 Also how will DIP74 work with incremental compilation?
 ---
 extern (D) class RcClass;

 void func(RcClass a)
 {
     //opps
     auto b = a;
     return;
 }
 ---


extern (D) class RcClass
{
  void opInc();
  void opDec(); // or whatever they're called
}

?
Oct 11 2015
prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:

 On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 [...]


 Speaking of DIP74 can't we just wrap a class in a struct with 
 use reference counting with and use alias this?


alias is problematic, because it allows the class reference to 
escape. opDispatch doesn't have that problem, though there may be 
other complications that it introduces (I don't know). It does 
get kind of complicated though when you consider member functions 
which return the a reference to the object and things like that. 
So, while it's generally feasible, it's not that hard for it to 
become unsafe. How much that matters is debatable, but it could 
make it so that reference counting classes is infeasible in  safe 
code.

- Jonathan M Davis
Oct 10 2015
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 11 October 2015 at 06:10:32 UTC, Jonathan M Davis 
wrote:

 On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:

 On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 [...]


 Speaking of DIP74 can't we just wrap a class in a struct with 
 use reference counting with and use alias this?


 alias is problematic, because it allows the class reference to 
 escape. opDispatch doesn't have that problem, though there may 
 be other complications that it introduces (I don't know). It 
 does get kind of complicated though when you consider member 
 functions which return the a reference to the object and things 
 like that. So, while it's generally feasible, it's not that 
 hard for it to become unsafe. How much that matters is 
 debatable, but it could make it so that reference counting 
 classes is infeasible in  safe code.

 - Jonathan M Davis


Ok dispatch has that problem. You can escape the this pointer 
from within member functions. Always. There is no  safe reference 
counting without language support. Either the RC is backed into 
the language (DIP74) or ownership is baked into the language, so 
that RC can be baked into the library.
Oct 10 2015
parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 16:17, deadalnix via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Sunday, 11 October 2015 at 06:10:32 UTC, Jonathan M Davis wrote:

 On Sunday, 11 October 2015 at 05:52:45 UTC, Freddy wrote:

 On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 [...]



 Speaking of DIP74 can't we just wrap a class in a struct with use
 reference counting with and use alias this?



 alias is problematic, because it allows the class reference to escape.
 opDispatch doesn't have that problem, though there may be other
 complications that it introduces (I don't know). It does get kind of
 complicated though when you consider member functions which return the a
 reference to the object and things like that. So, while it's generally
 feasible, it's not that hard for it to become unsafe. How much that matters
 is debatable, but it could make it so that reference counting classes is
 infeasible in  safe code.

 - Jonathan M Davis



 Ok dispatch has that problem. You can escape the this pointer from within
 member functions. Always. There is no  safe reference counting without
 language support. Either the RC is backed into the language (DIP74) or
 ownership is baked into the language, so that RC can be baked into the
 library.


Ownership is only part of the problem (and I support your ideas about
ownership, I think D is completely going the wrong way on ownership).
There is still the problem that the inc/dec primitives need to be
known for the compiler to optimise. Constructors/destructors don't do
it semantically. That seems to be what DIP74 is introducing, and even
if the ownership problems were addressed/improved, DIP74 would just
remap to those concepts. We still need the inc/dec primitives either
way.
Oct 11 2015
prev sibling parent reply Freddy <Hexagonalstar64 gmail.com> writes:
On Sunday, 11 October 2015 at 06:10:32 UTC, Jonathan M Davis 
wrote:

 alias is problematic, because it allows the class reference to 
 escape. opDispatch doesn't have that problem, though there may 
 be other complications that it introduces (I don't know). It 
 does get kind of complicated though when you consider member 
 functions which return the a reference to the object and things 
 like that. So, while it's generally feasible, it's not that 
 hard for it to become unsafe. How much that matters is 
 debatable, but it could make it so that reference counting 
 classes is infeasible in  safe code.

 - Jonathan M Davis


Can't we make opAssign and this(this) work with classes add 
 disable to them? Also rec counted classes shouldn't have members 
of that have their type, they should have to have members of the 
aliased struct containing their type.
Oct 11 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 14:27:36 UTC, Freddy wrote:

 On Sunday, 11 October 2015 at 06:10:32 UTC, Jonathan M Davis 
 wrote:

 alias is problematic, because it allows the class reference to 
 escape. opDispatch doesn't have that problem, though there may 
 be other complications that it introduces (I don't know). It 
 does get kind of complicated though when you consider member 
 functions which return the a reference to the object and 
 things like that. So, while it's generally feasible, it's not 
 that hard for it to become unsafe. How much that matters is 
 debatable, but it could make it so that reference counting 
 classes is infeasible in  safe code.

 - Jonathan M Davis


 Can't we make opAssign and this(this) work with classes add 
  disable to them? Also rec counted classes shouldn't have 
 members of that have their type, they should have to have 
 members of the aliased struct containing their type.


I don't know if opAssign can be overloaded for a class or not. 
That gets a bit wonky IMHO given that you never assign classes - 
only references to them. Similarly, class objects never get 
postblitted. And something as simple as a parent-child 
relationship tends to cause serious problems restricting 
references to smart pointers. e.g. take this C++ code

class Parent
{
     friend class Child;
public
     Parent() _child(make_shared<Child>(this)) {}
private:
     shared_ptr<Child> _child;
}

class Child
{
public:
     Child(Parent* parent) : _parent(parent) {}
private:
     Parent* _parent;
}

auto parent = make_shared<Parent>();

In this case in C++, because the ref-counting is not built-in to 
the type, there is no way for the Child to have access to its 
parent via a shared_ptr. It has to be done via a normal pointer. 
D has exactly this same problem. If the ref-counting isn't 
built-in, then there are cases where you have to let a 
non-ref-counted reference escape.

The problem can be reduced by making it so that RefCounted (or 
whatever the wrapper struct is) doesn't allow accidental, direct 
access to the wrapped class object. But some stuff will simply be 
impossible without providing direct access.

By having the class reference contain the reference count, the 
problem can significantly reduced, because then it's possible to 
create a shared pointer from the class object without ending up 
with a separate reference count. But you still have the problem 
of that class object being passed around and not necessarily 
always being put in a wrapper object like it's supposed to be. 
So, you still have a safety issue.

The only way for this to be completely safe is for the 
ref-counting to be built into the language such that there is no 
way to have a reference being passed around which isn't 
ref-counted. Now, we can certainly take the approach of saying 
that we don't care about that being completely safe (in which 
case, ref-counting classes will be limited if not outright banned 
in  safe code), but if we want  safe ref-counting, we have no 
choice.

- Jonathan M Davis
Oct 11 2015
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 11 October 2015 at 21:15:38 UTC, Jonathan M Davis 
wrote:

 In this case in C++, because the ref-counting is not built-in 
 to the type, there is no way for the Child to have access to 
 its parent via a shared_ptr. It has to be done via a normal 
 pointer. D has exactly this same problem. If the ref-counting 
 isn't built-in, then there are cases where you have to let a 
 non-ref-counted reference escape.


In this case one should use unique_ptr, so the D discussion about 
ref counting is irrelevant. If you have truely shared_ptr 
semantics and back pointers, then one shoud use weak_ptr for this 
since the back pointers don't own the resource. Keep in mind that 
shared_ptr only denote ownership, not resource usage.

C++ is now moving towards Rust and static analysis, but as a step 
seperate from compilation. And that makes sense since it is time 
consuming and doesn't have to be done for codegen.
Oct 11 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 22:12:50 UTC, Ola Fosheim Grøstad 
wrote:

 On Sunday, 11 October 2015 at 21:15:38 UTC, Jonathan M Davis 
 wrote:

 In this case in C++, because the ref-counting is not built-in 
 to the type, there is no way for the Child to have access to 
 its parent via a shared_ptr. It has to be done via a normal 
 pointer. D has exactly this same problem. If the ref-counting 
 isn't built-in, then there are cases where you have to let a 
 non-ref-counted reference escape.


 In this case one should use unique_ptr, so the D discussion 
 about ref counting is irrelevant. If you have truely shared_ptr 
 semantics and back pointers, then one shoud use weak_ptr for 
 this since the back pointers don't own the resource. Keep in 
 mind that shared_ptr only denote ownership, not resource usage.


I don't want get into arguments about smart_ptr and unique_ptr. 
It's completely irrelevant to my point.

weak_ptr doesn't work when the child is constructed by the parent 
in the parent's constructor. The fact that the parent does not 
have access to the smart pointer type that it's about to be put 
into means that it cannot give that to the child. It has to give 
it a normal pointer. And that means that you can't make it so 
that all accesses to the parent object are via a smart pointer. 
And in the case of D, that means that it doesn't work to restrict 
all access to a class object to a wrapper struct like RefCounted, 
and it becomes quite easy for a reference to the class object to 
escape the wrapper struct. And that means that it's not  safe, 
because the wrapper struct might destroy the class object when 
it's destroyed (because its ref-count reached 0) while a 
reference to that class object is still floating around 
somewhere. Obviously, it can work (it works in C++). It just 
means that it can't be  safe and that the programmer has to worry 
about ensuring that memory doesn't get used incorrectly. But part 
of the whole point of DIP 74 is so that we can have  safe 
ref-counting in D.

- Jonathan M Davis
Oct 11 2015
parent reply Ola Fosheim Gr <ola.fosheim.grostad+dlang gmail.com> writes:
On Sunday, 11 October 2015 at 22:33:44 UTC, Jonathan M Davis 
wrote:

 I don't want get into arguments about smart_ptr and unique_ptr. 
 It's completely irrelevant to my point.


Ok, but keep in mind that since unique_ptr is rc which max out at 
1 and therefore don't need extra support beyond having an owning 
pointer. So whatever is done for reasoning about rc also has to 
work for unique. If not thenD is doomed to fail, because nobody 
wants rc when unique is sufficient.


 weak_ptr doesn't work when the child is constructed by the 
 parent in the parent's constructor. The fact that the parent 
 does not have access to the smart pointer type that it's about 
 to be put into means that it cannot give that to the child. It 
 has to give it a normal pointer. And that means that you can't 
 make it so that all accesses to the parent object are via a 
 smart pointer. And in the case of D, that means that it doesn't 
 work to restrict all access to a class object to a wrapper 
 struct like RefCounted, and it becomes quite easy for a 
 reference to the class object to escape the wrapper struct.


Well, the wrapper approach is no good (is it part of dip74?) 
since it messes up alignment etc, so the refcount interface 
should be part of the parent object. The child can be given the 
Refcount interface from dip74, it won't use it for a weak 
reference until it dereference (borrow). A weak reference just 
delays destruction until the borrowing is completed.

So it works(?) if you dont allow dereferencing of the weak 
pointer in the constructor of the child.


 get used incorrectly. But part of the whole point of DIP 74 is 
 so that we can have  safe ref-counting in D.


Well,  safe in D is broken and should just be dropped in favour 
of something sane like real pointer analysis instead of piling up 
bad designs like dip25. You only have to run it for release... 
Who cares if it is slow?
Oct 11 2015
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, 11 October 2015 at 23:07:18 UTC, Ola Fosheim Gr wrote:

 On Sunday, 11 October 2015 at 22:33:44 UTC, Jonathan M Davis 
 wrote:
 Well, the wrapper approach is no good (is it part of dip74?) 
 since it messes up alignment etc, so the refcount interface 
 should be part of the parent object. The child can be given the 
 Refcount interface from dip74, it won't use it for a weak 
 reference until it dereference (borrow). A weak reference just 
 delays destruction until the borrowing is completed.


DIP 74 proposes built-in reference counting. There is no wrapper 
type, and the compiler is able to elide increments and decrements 
of the ref count if it determines that they're unnecessary. If a 
type is ref-counted with DIP 74, it's always ref-counted, and it 
doesn't have anything for weak references (presumably leaving 
that up to the GC to take care of).


 get used incorrectly. But part of the whole point of DIP 74 is 
 so that we can have  safe ref-counting in D.


 Well,  safe in D is broken and should just be dropped in favour 
 of something sane like real pointer analysis instead of piling 
 up bad designs like dip25. You only have to run it for 
 release... Who cares if it is slow?


 safe isn't going anywhere. And it mostly works just fine. It's 
primary flaw is that it's been done via blacklisting operations 
rather than whitelisting them, but that doesn't stop it from 
working. It just makes the implementation more error-prone. The 
only thing preventing the use of  safe in most cases is functions 
that haven't been updated to use it yet (in which case  trusted 
can be used). Walter and Andrei are completely behind  safe and 
are going to be in favor of fixing any holes in it that are 
found, not getting rid of it. And I don't see any reason why that 
can't work or that it's a bad idea.

- Jonathan M Davis
Oct 11 2015
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= writes:
On Sunday, 11 October 2015 at 23:46:42 UTC, Jonathan M Davis 
wrote:

  safe isn't going anywhere. And it mostly works just fine. It's 
 primary flaw is that it's been done via blacklisting operations 
 rather than whitelisting them, but that doesn't stop it from 
 working. It just makes the implementation more error-prone.


Has nothing with whitelisting or blacklisting, that's basically 
the same thing for a finite set of features.

The real problem is that it cannot be assumed to work until 
proven correct, formally. So it does not give you anything more 
than a weak sanitizer would have done until such proofs have been 
verified. The fact that it is difficult to be convince oneself 
that you don't have holes reinforces this viewpoint.

If reasoning is difficult, you need proofs.


 Walter and Andrei are completely behind  safe and are going to 
 be in favor of fixing any holes in it that are found, not 
 getting rid of it. And I don't see any reason why that can't 
 work or that it's a bad idea.


This has been discussed before.  trusted regions can be correct 
in isolation, but it will make assumptions about what goes into 
 safe regions, so when the  safe region changes  trusted regions 
cannot assumed to be correct anymore. Therefore you need a prover 
to prove the  trusted region to remain safe whenever you change 
the surrounding  safe region. That means  trusted has to be truly 
exceptional and rare and impose verfiied restrictions on the 
assumptions it makes, which is not likely to happen.

This comes in conflict with the very nature of system level 
programming.
Oct 11 2015
prev sibling parent Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 11 October 2015 at 15:52, Freddy via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 [...]



 Speaking of DIP74 can't we just wrap a class in a struct with use reference
 counting with and use alias this?


Because the language doesn't have ref counting. And... no.
If the 'solution' D offers is more complicated than C++, why would I
leave C++? I certainly won't win over any new D users when I show them
how awesome D is at making their already horribly complex code even
worse.
Oct 11 2015
prev sibling parent bitwise <bitwise.pvt gmail.com> writes:
On Saturday, 10 October 2015 at 23:25:49 UTC, Manu wrote:

 So I have another upcoming opportunity to introduce D in my 
 workplace, this time as a front-end/plugin language to our C++ 
 infrastructure, which is promising since I already have 
 considerable experience in this area (my work at Remedy with 
 Quantum Break), and there is a lot of recent work to interact 
 better with C++, which we will stress-test extensively.

 You only get so many shots at this; but this is a particularly 
 promising opportunity, since the C++ code is a nightmare, and 
 the contrast against D will allow a lot of coders to see the 
 advantage.

 There is however one critical missing feature, DIP74... where 
 is it at currently? How is it going? Is it likely to be 
 accepted in the near-term? Some sort of approximate timeline?

 I think it would be a mistake for me to introduce this without 
 DIP74, since we will rely on it VERY heavily, and the machinery 
 to work-around it will start to look just as heavy-weight as 
 the C++ code I'm trying to deprecate... but then waiting on it 
 starts to look like missing the window of opportunity.

 Thoughts?


I recently brought this up here:
http://forum.dlang.org/thread/aetzsanbypaimljrguzb forum.dlang.org

We discussed pros and cons, etc..

I restated my main concern here, and Walter seemed to agree:
http://forum.dlang.org/post/mv29l2$1rln$1 digitalmars.com

So it seems there is hope for DIP74(or something similar), but it 
may take time.

I asked Andrei when we may have another "D Vision" or similar 
document, and his response was here:
http://forum.dlang.org/post/mv7is9$m41$1 digitalmars.com

      Bit
Oct 11 2015