www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Proposal 2: Exceptions and nogc

reply Walter Bright <newshound2 digitalmars.com> writes:
My previous version did not survive implementation. Here's the revised version. 
I have submitted it as a DIP, and there's a trial implementation up:

     https://github.com/dlang/dmd/pull/6681

---------------------------------------------
	Exceptions and  nogc version 2

Problem
=======

Exceptions are assumed to be GC collected by the EH design, in that no
attempt is made to control copies or lifetimes. This dependency is not
a performance issue, as exceptions are presumed to be slow.

The issue is it impairs use of  nogc on any code that throws exceptions,
and prevents building D programs that do not link in the GC runtime.

To fix this, the allocation and destruction of the exception objects
must be completely controlled.

Solution
========

Make Throwable optionally ref counted. Add a field `_refcount` which
is !=0 when it is a ref counted instance. The number of parents of
a refcounted Throwable is _refcount+1. This member is private and only
accessible via the  system member function refcount(), to prevent
its use in  safe code.

The only place a refcounted Throwable is ever created is when the following
statement is in the user code:

     throw new E(string);

where E is Throwable or derived from Throwable. Instead of calling the usual
_d_newclass() to allocate E on the GC heap, it will call the new function
_d_newThrowable() which will allocate E and intialize it for refcounting.

When the exception is thrown, either _d_throwc() or _d_throwdwarf() gets called,
which is where druntime takes over. The refcount is incremented in these
functions,
usually from 1 to 2.

The thrown object will then wind up either in a catch statement, or in the
`next`
linked list of thrown exceptions in flight.

A catch variable `e` as in:

     catch (E e)
     {
         ...
     }

becomes an RAII object, meaning that it is destroyed at the closing }. Such
destruction is done by calling:

     _d_delThrowable(e);

which will test e._refcount to see if it is a ref counted object. If not,
it does nothing. If it is, the refcount is decremented, and if it hits
one then e's destructor is called, followed by free'ing the memory used
by `e`.

In catch blocks, `e` is regarded as `scope`, so that it cannot escape the
catch block. As a special case, if `e` is thrown in the catch block as:

     throw e;

then `e` can escape, and this works because as mentioned before `_d_throwc()` or
`_d_throwdwarf()` will increment the reference count.

The destructor for `Throwable` will, if the refcount is 1, call 
_d_delThrowable(e.next),
i.e. on the head of the chained list of exceptions. This means that the chained
list can be a mixed collection of refcounted and not-refcounted Throwables,
although
the refcounted ones can only be free'd when their antecedent gets reaped by
whatever
means.

The chained list must be protected so it cannot be altered or escaped by user
code.

Legacy Code Breakage
--------------------

This will break an unknown amount of existing code.

Breakage will come in the form of:

1. leaking Exception objects from catch clauses (caught by
the compiler)

2. Disallowing Exception objects with postblit fields.

3. Catch objects being 'scope' will cause problems in that
everything done with those objects will also have to be 'scope'.
The most likely problem will be printing the objects which
relies on Object.toString() which is not 'scope'. One possible
solution is to force Throwable.toString() to be 'scope', which
will likely cause minimal disruption. Of course, compiling
with -dip1000 will disable such checking and can work in
the interim. Code that needs to leak the thrown exception object
can clone the object.

Conclusion
----------

The result of this should be no leaking memory, no need to link
in the GC, and memory safety.

GC, stack, and refcounted exception objects can coexist in the same
program.

References
----------

http://www.digitalmars.com/d/archives/digitalmars/D/Exceptions_in_nogc_code_299261.html
Apr 08 2017
next sibling parent Joakim <dlang joakim.fea.st> writes:
On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 My previous version did not survive implementation. Here's the 
 revised version. I have submitted it as a DIP, and there's a 
 trial implementation up:

     https://github.com/dlang/dmd/pull/6681


Two small procedural notes

- I don't see a DIP proposal in this list:

https://github.com/dlang/DIPs/pulls

- I suggest you not use -dip1006 as the flag to enable this, as 
the DIP numbering used to be assigned after the DIP proposal was 
merged and there's currently another proposal in that list that 
makes the same mistake of claiming that number.  Better to use 
some other flag for now, until you're sure of the DIP number, or 
the name clashes will be confusing.
Apr 08 2017
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 My previous version did not survive implementation. Here's the 
 revised version. I have submitted it as a DIP, and there's a 
 trial implementation up:

     https://github.com/dlang/dmd/pull/6681

 [ . . . ]


Pardon my ignorance but it does not seem to be clear how 
preallocated exceptions would work. What about exceptions 
allocated ultimately by the GC but actually via an intermediate 
allocator (e.g. FallBack)?
Apr 08 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/8/2017 9:30 PM, Nicholas Wilson wrote:

 Pardon my ignorance but it does not seem to be clear how preallocated
exceptions
 would work. What about exceptions allocated ultimately by the GC but actually
 via an intermediate allocator (e.g. FallBack)?


You can allocate them any way you like - the runtime won't recognize those as 
its own refcounted objects, and will not try to free them.
Apr 08 2017
prev sibling next sibling parent reply Daniel N <no public.email> writes:
On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 Solution
 ========

 Make Throwable optionally ref counted.


I like this new direction...


 The only place a refcounted Throwable is ever created is when 
 the following
 statement is in the user code:

     throw new E(string);


... but why not go all the way, making it "always" refcounted? 
(for any "new E", not emplace).
Apr 09 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/9/2017 1:16 AM, Daniel N wrote:

 ... but why not go all the way, making it "always" refcounted? (for any "new
E",
 not emplace).


Backwards compatibility, for one. For another, a general mechanism for safe 
refcounting of classes has eluded us.
Apr 09 2017
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 9 April 2017 at 20:14:24 UTC, Walter Bright wrote:

 For another, a general mechanism for safe refcounting of 
 classes has eluded us.


The only thing you need to get backed into the language is to 
make sure things do not escape in uncontrolled manner. Everything 
else is library.

You wouldn't have this problem if you had listened to myself and 
Marc when defining DIP1000, because that's exactly what you've 
been warned about at the time.

Quoting from the timeline ML from Nov 2014:

[...]

Every expression has now has a lifetime associated with it, and 
can be marked as "scope". it is only possible to assign b to a if 
b has a lifetime equal or greater than a's.

An infinite lifetime is a lifetime greater or equal than any 
other lifetime. Expression of infinite lifetime are:
  - literals
  - GC heap allocated objects
  - statics and enums.
  - rvalues of type that do not contain indirections.
  - non scope rvalues.

Dereference share the lifetime of the dereferenced expression (ie 
infinite lifetime unless the expression is scope). Address of 
expression shared the lifetime of the base expression, and in 
addition gain the scope flag.

Comment: Using these rule, we basically define any indirection 
being of infinite lifetime by default, and we propagate the 
lifetime when scope. The addition of the scope flag for address 
of is necessary to disallow taking address->dereference to yield 
an infinite lifetime.

Variables delcarations (including parameters) have the lifetime 
of the block they are declared in (2 pitfalls here, I don't have 


these variables shared the lifetime of the variable, unless they 
qualify for infinite lifetime. Parameter's lifetime are 
unordered, meaning smaller than infinite, greater than the 
function's scope, but not equal to each other nor greater/smaller 
than each others.

[...]
Apr 09 2017
prev sibling next sibling parent reply MysticZach <reachzach ggmail.com> writes:
On Sunday, 9 April 2017 at 20:14:24 UTC, Walter Bright wrote:

 On 4/9/2017 1:16 AM, Daniel N wrote:

 ... but why not go all the way, making it "always" refcounted? 
 (for any "new E",
 not emplace).


 Backwards compatibility, for one. For another, a general 
 mechanism for safe refcounting of classes has eluded us.


Hi guys. Hey Walter. So, about this point. On the lifetime study 
thread, http://forum.dlang.org/post/56301A8C.1060808 erdani.com , 
the following two problems were stated by Andrei, but I don't 
think they were adequately addressed in the subsequent posts:

===
Widget global;
 rc class Widget {
    int x;
    void fun() {
      global = null;
      ++x;
    }
}

void main() {
    global = new Widget;
    global.fun();
}

In this example, if global has a refcount==1 upon entering fun(), 
the assignment "global = null" deletes the Widget object and ++x 
accesses dangling memory.

I should add here another pattern that turned problematic for our 
older attempts in DIP74:

C c = new C();
foo(c);

int foo(scope C d) {
      c = new C();    // c's old instance gets deleted
      return d.i;        // oops! d is invalid
}
===

So here's my analysis of both these problems.

When calling a function, the reference count for an object must 
increase by the total number of aliases created by the call — 
*minus the ones lost*. Globals are special in this regard, 
because access to them is not lost in the called function. Local 
variables, however, are lost to the called function — they cannot 
be accessed except through the new alias they receive as a 
parameter. Thus, only globals must modify the reference count 
when passed.

Thinking about this is made easier if we turn all accessible 
aliases into function parameters. For this, we need to consider 
the set of globals as a hidden parameter to the function. Any 
global is therefore already passed to all functions. If we pass 
it again via parameter, that amounts to two aliases, thus two 
references. The same logic applies, 1. to duplicating any given 
variable in the argument list, e.g. "fun(c, c);", 2. to 
duplicating the hidden 'this' parameter, 3. to recognizing and 
outer function's stack frame as an additional alias.

All these can be verified at compile time. Also, it is optimistic 
regarding the most common case, i.e. passing a local does not 
require increasing the reference count.

--Zach
Apr 11 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/11/2017 10:24 AM, MysticZach wrote:

 Hi guys. Hey Walter. So, about this point. On the lifetime study thread,
 http://forum.dlang.org/post/56301A8C.1060808 erdani.com , the following two
 problems were stated by Andrei, but I don't think they were adequately
addressed
 in the subsequent posts:


The way they will be addressed is to increment the reference count in the call 
to the function that takes a reference to an RC object.
Apr 12 2017
parent MysticZach <reachzach ggmail.com> writes:
On Wednesday, 12 April 2017 at 19:01:25 UTC, Walter Bright wrote:

 On 4/11/2017 10:24 AM, MysticZach wrote:

 Hi guys. Hey Walter. So, about this point. On the lifetime 
 study thread,
 http://forum.dlang.org/post/56301A8C.1060808 erdani.com , the 
 following two
 problems were stated by Andrei, but I don't think they were 
 adequately addressed
 in the subsequent posts:


 The way they will be addressed is to increment the reference 
 count in the call to the function that takes a reference to an 
 RC object.


Makes sense, and I thought the same thing until after I wrote my 
post — but now I realize it's not quite good enough. Merely 
incrementing the refcount (and decrementing it afterwards) has 
the following problem:

 rc class RC;

void fun(ref RC a, RC b) {
    a = new RC; // (1)
    // b...
}

void main() {
    auto r = new RC; // (2)
    --> r.opInc(); // compiler insert
    fun(r, r);
    --> r.opDec();
}

Let's assume the reference counting scheme involves the methods 
opInc and opDec, inserted automatically by the compiler as the 
result of detecting a duplicate parameter. If you read closely, 
you'll realize that at mark 1 above, the program will leak the 
data acquired at mark 2. The assign statement of the refcounted 
object will decrement the data it points to before it is 
reassigned. But since the data at (2)'s refcount was prematurely 
incremented, it will fall to 1 and never get deleted. 
Furthermore, the opDec in main() will decrement and delete the 
data acquired at mark 1, thinking it was the mark 2 data.

The problem is that the calling context at "fun(r,r);" fails to 
keep a real reference to the mark 2 data. If one of the 
parameters is sent by reference, we can't assume it points to the 
same data upon returning as when it was sent. And the mark 2 data 
can't be deleted before fun() returns, or it will invalidate 'b' 
in fun(). This suggests we need to save a real, separate 
reference to 'r' before sending two or more versions of it.

That said, I believe the following compiler inserts in main() 
would do the trick:

void main() {
    auto r = new RC; // (2)
    --> RC __rsave = r; // compiler insert
    --> scope(exit) __rsave = null;
    fun(r, r);
}

This solution uses only the assign method of RC to inc and dec 
the refcount, suggesting that opInc and opDec are ultimately 
unnecessary, except as (very rare) optimizations. But you still 
need for the compiler to distinguish a refcounted class from a 
regular one, so maybe the presence of an opDec() could indicate 
that, opDec being the quasi-destroyer that turns a refcounted 
class variable into an RAII type. Otherwise, you might need 
something like an  rc attribute.

At any rate, your original claim was, "a general mechanism for 
safe refcounting of classes has eluded us." Is this still the 
case, considering the above?

As far as the original post, even if a general mechanism were 
found, it doesn't mean you'd have to use it in the case of 
Exception anyway. To generalize the expression 'throw new ...', 
you'd have to redefine 'new' to be different with refcounted 
classes than with regular ones. Exceptions are probably worth a 
more specialized solution like the one you proposed. Otherwise 
everyone will have to change their 'throw new' code to 
accommodate ' nogc'. Adam D. Ruppe gave us his solution here:

http://arsdnet.net/exception.d

tl;dr Everyone would now have to say things like 'throw 
emplace!...' or 'raise!"my_exception"(...)'.

I guess the proposed hack of 'throw new Exception' is simply the 
shadow of D's original philosophy of wanting the GC to do too 
much.
Apr 13 2017
prev sibling parent MysticZach <reachzach ggmail.com> writes:
On Sunday, 9 April 2017 at 20:14:24 UTC, Walter Bright wrote:

 ... a general mechanism for safe refcounting of classes has 
 eluded us.


Regardless of my other comments, which are maybe a little 
uninformed, DIP74 seems pretty good:

https://wiki.dlang.org/DIP74

Manu started a thread asking about it, but that thread dissolved 
into a bitter dispute:

http://forum.dlang.org/post/mailman.1002.1444519548.22025.digitalmars-d puremagic.com

The only valuable information regarding the question was in this 
post of Andrei's, which did not say very much:

http://forum.dlang.org/post/mvgoa9$1gda$1 digitalmars.com

So what are the corner cases of DIP74? What are the actual 
problems with it?
Apr 15 2017
prev sibling next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 The only place a refcounted Throwable is ever created is when 
 the following
 statement is in the user code:

     throw new E(string);


object aMemoryLeak;

void someFunc()
{   throw (aMemoryLeak = new Exception("hello world!"));
}

Would the compiler warn about this or make the exception normally 
garbage collected?

I'm not sure, but perhaps an entirely different syntax would be 
in place for throwing a refcounted object, so there's no special 
casing. For example, could a throw statement without the new 
keyword mean a refcounted throw?
Apr 09 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/9/2017 1:35 AM, Dukc wrote:

 object aMemoryLeak;

 void someFunc()
 {   throw (aMemoryLeak = new Exception("hello world!"));
 }

 Would the compiler warn about this or make the exception normally garbage
 collected?


That would be a regular gc allocated Exception.



 I'm not sure, but perhaps an entirely different syntax would be in place for
 throwing a refcounted object, so there's no special casing. For example, could
a
 throw statement without the new keyword mean a refcounted throw?


Didn't want to do a new syntax.
Apr 09 2017
parent reply Andrew Godfrey <X y.com> writes:
On Sunday, 9 April 2017 at 20:15:46 UTC, Walter Bright wrote:

 On 4/9/2017 1:35 AM, Dukc wrote:

 object aMemoryLeak;

 void someFunc()
 {   throw (aMemoryLeak = new Exception("hello world!"));
 }

 Would the compiler warn about this or make the exception 
 normally garbage
 collected?


 That would be a regular gc allocated Exception.


Iternally, would we create a temp Exception with _refcount = 2, 
and then on assignment to 'aMemoryLeak', change the refcount to 0?

I understand that the compiler would probably optimize that away 
in this case, but it seems that's a general answer that would 
work.

So in that case:
1. Do you really need the "_refcount=1" state as currently 
defined? I'd think the only code which sees this state, has just 
decremented it from 2 and is about to delete it.

2. Echoing others here: This seems like a general model D could 
use, where you have refcounted objects, but at any time, a gc 
reference could be taken, which is indicated by setting the 
refcount to 0 but leaving the object alive. Is it general? If 
not, what is special about Exceptions that makes it work here?
Apr 09 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/9/2017 5:12 PM, Andrew Godfrey wrote:

 Is it general?


No.


 If not, what is special about Exceptions that makes it work here?


It only works because all ways that such exceptions can leak are controlled. D 
doesn't have copy construction for making copies of class references. You 
couldn't use this scheme, for example, to stuff class references into struct
fields.
Apr 09 2017
parent reply Andrew Godfrey <X y.com> writes:
On Monday, 10 April 2017 at 00:48:39 UTC, Walter Bright wrote:

 On 4/9/2017 5:12 PM, Andrew Godfrey wrote:

 Is it general?


 No.


 If not, what is special about Exceptions that makes it work 
 here?


 It only works because all ways that such exceptions can leak 
 are controlled. D doesn't have copy construction for making 
 copies of class references. You couldn't use this scheme, for 
 example, to stuff class references into struct fields.


Ok. So then if I have created a refcounted Exception, and later 
(in another function) I take a reference to it (by stuffing it 
into a struct field, say), how does that work? What is the value 
of _refcount after that happens? Is it 0? Or do we take "one more 
reference to indicate the GC owns it"?
Apr 09 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/9/2017 6:32 PM, Andrew Godfrey wrote:

 Ok. So then if I have created a refcounted Exception, and later (in another
 function) I take a reference to it (by stuffing it into a struct field, say),
 how does that work?


You can't, because the refcounted Exception will be marked with 'scope' in the 
catch block.
Apr 09 2017
parent reply Andrew Godfrey <X y.com> writes:
On Monday, 10 April 2017 at 01:54:54 UTC, Walter Bright wrote:

 On 4/9/2017 6:32 PM, Andrew Godfrey wrote:

 Ok. So then if I have created a refcounted Exception, and 
 later (in another
 function) I take a reference to it (by stuffing it into a 
 struct field, say),
 how does that work?


 You can't, because the refcounted Exception will be marked with 
 'scope' in the catch block.


In your proposal, you wrote:


 The only place a refcounted Throwable is ever created is when 
 the following statement is in the user code:



    throw new E(string);


Did you mean to use the "scope" keyword somewhere in the line 
above?
Apr 09 2017
parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 10 April 2017 at 02:04:35 UTC, Andrew Godfrey wrote:

 On Monday, 10 April 2017 at 01:54:54 UTC, Walter Bright wrote:

    throw new E(string);


 Did you mean to use the "scope" keyword somewhere in the line 
 above?


No. In the scope the exceptions are instantiated they are not 
scoped. He meant that if, and only if, they are instantiated in 
the way mentioned above they are reference counted. If an 
exception is instantiated otherwise, it is garbage-collected like 
all other classes. Otherwise, someone could store a refcounted 
exception into a global object reference. Then the program would 
have to check for reference count every time when adding or 
deleting an object reference, slowing down the whole program.

In catch blocks, where the caught exception may or may not be 
refcounted, they are marked scope so there won't be refcounted 
classes running wild, only at the catch scope.
Apr 09 2017
parent reply Andrew Godfrey <X y.com> writes:
On Monday, 10 April 2017 at 06:17:43 UTC, Dukc wrote:

 On Monday, 10 April 2017 at 02:04:35 UTC, Andrew Godfrey wrote:

 On Monday, 10 April 2017 at 01:54:54 UTC, Walter Bright wrote:

    throw new E(string);


 Did you mean to use the "scope" keyword somewhere in the line 
 above?


 No. In the scope the exceptions are instantiated they are not 
 scoped. He meant that if, and only if, they are instantiated in 
 the way mentioned above they are reference counted. If an 
 exception is instantiated otherwise, it is garbage-collected 
 like all other classes. Otherwise, someone could store a 
 refcounted exception into a global object reference. Then the 
 program would have to check for reference count every time when 
 adding or deleting an object reference, slowing down the whole 
 program.

 In catch blocks, where the caught exception may or may not be 
 refcounted, they are marked scope so there won't be refcounted 
 classes running wild, only at the catch scope.


Thanks for explaining!
Now I get it.

I'm just curious: The proposal doesn't mention interop with C++ 
exception handlers. I don't know the status of that so I'll just 
ask: Can C++ code catch D exceptions?
Apr 10 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/10/2017 6:41 AM, Andrew Godfrey wrote:

 I'm just curious: The proposal doesn't mention interop with C++ exception
 handlers. I don't know the status of that so I'll just ask: Can C++ code catch
D
 exceptions?


No. D can catch C++ exceptions.
Apr 11 2017
next sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tuesday, April 11, 2017 01:05:10 Walter Bright via Digitalmars-d wrote:

 On 4/10/2017 6:41 AM, Andrew Godfrey wrote:

 I'm just curious: The proposal doesn't mention interop with C++
 exception
 handlers. I don't know the status of that so I'll just ask: Can C++ code
 catch D exceptions?


 No. D can catch C++ exceptions.


What does this look like? I recall you discussing planning adding the
ability for D to catch C++ exceptions, but I was unaware that it had been
implemented yet. Certainly, I couldn't find it in the documentation when I
went looking a month or two back. But I wouldn't think that catching
Exception would work if it's a C++ exception - even if it's a
std::exception, since std::exception doesn't have the same functions as
Exception or Throwable. So, how would you catch a C++ exception in D?

- Jonathan M Davis
Apr 11 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/11/2017 1:54 AM, Jonathan M Davis via Digitalmars-d wrote:

 No. D can catch C++ exceptions.


 What does this look like? I recall you discussing planning adding the
 ability for D to catch C++ exceptions, but I was unaware that it had been
 implemented yet. Certainly, I couldn't find it in the documentation when I
 went looking a month or two back. But I wouldn't think that catching
 Exception would work if it's a C++ exception - even if it's a
 std::exception, since std::exception doesn't have the same functions as
 Exception or Throwable. So, how would you catch a C++ exception in D?


You define a C++ class in D that matches a C++ one, throw a pointer to that 
class in C++, and catch it in D.

It currently works on Elf and MachO, but not on Windows.
Apr 11 2017
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
This idea could be generalized:

-DRuntime would add an interface "ReferenceCountable".
-Throwable would implement it.
-When a new expression of ReferenceCountable type is used to 
assign to a scope variable or argument, it's guaranteed to be 
 nogc.
-Non-scoped objects, ReferenceCountable or no, are assumed to not 
be reference counted.

I wonder it this is what Jack Stouffer was thinking?
Apr 11 2017
parent reply Nick Treleaven <nick geany.org> writes:
On Tuesday, 11 April 2017 at 11:08:34 UTC, Dukc wrote:

 This idea could be generalized:

 -DRuntime would add an interface "ReferenceCountable".
 -Throwable would implement it.
 -When a new expression of ReferenceCountable type is used to 
 assign to a scope variable or argument, it's guaranteed to be 
  nogc.


scope c = new Object;

This should be recognised as a scope class instance. It doesn't 
need reference counting, as it should always be destroyed at the 
end of current scope. The compiler can allocate on the stack (if 
the instance isn't too big):

https://forum.dlang.org/post/mailman.1045.1471418257.3131.digitalmars-d-announce puremagic.com

If it is too big, the compiler could use a  nogc heap allocator.
Apr 12 2017
parent reply Dukc <ajieskola gmail.com> writes:
On Wednesday, 12 April 2017 at 15:08:44 UTC, Nick Treleaven wrote:

 This should be recognised as a scope class instance. It doesn't 
 need reference counting, as it should always be destroyed at 
 the end of current scope.


The reason it needs:

{   scope Object ob = new RefCountableType("foo");
     scope ob2 = ob;
     ob = new RefCountableType("bar");
}

The "foo" instance would leak if the destruction would be done by 
calling at end of scope. You also cannot call the destructor when 
a new object is assigned to it, because then ob2 would become 
dangling.

What could act without GC or RC, is an immutable scoped object. 
But that's a different matter altogether, because throwing 
lifetimed objects did not work according to Walter.
Apr 12 2017
parent Nick Treleaven <nick geany.org> writes:
On Thursday, 13 April 2017 at 05:29:28 UTC, Dukc wrote:

 The reason it needs:

 {   scope Object ob = new RefCountableType("foo");
     scope ob2 = ob;
     ob = new RefCountableType("bar");
 }

 The "foo" instance would leak if the destruction would be done 
 by calling at end of scope.


I assume you mean if the compiler calls `delete ob` just before 
}. What I described would be implemented by the compiler deleting 
the original value of ob just before }.
Apr 15 2017
prev sibling next sibling parent reply Andrew Godfrey <X y.com> writes:
On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 My previous version did not survive implementation. Here's the 
 revised version. I have submitted it as a DIP, and there's a 
 trial implementation up:

 [...]


Just a quick note to reduce confusion for reviewers:


 The number of parents of a refcounted Throwable is _refcount+1.


I believe what you mean here is "_refcount-1".
Later you say that the exception is reclaimed when the refcount 
goes to 1.
Apr 09 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/9/2017 12:05 PM, Andrew Godfrey wrote:

 Just a quick note to reduce confusion for reviewers:


 The number of parents of a refcounted Throwable is _refcount+1.


 I believe what you mean here is "_refcount-1".
 Later you say that the exception is reclaimed when the refcount goes to 1.


Thanks for the correction.
Apr 09 2017
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 My previous version did not survive implementation. Here's the 
 revised version. I have submitted it as a DIP, and there's a 
 trial implementation up:

     https://github.com/dlang/dmd/pull/6681


1. This still adds another special case to the language under the 
guise of not breaking backward compatibility. This is exactly how 
C++ became the mess it is today.

2. You're adding in ref-counting in an area where it's only 
available to one very specific part of the language. This is one 
of the reasons people don't like Go: it offers language features 
only in specific places at the compiler devs discretion. It would 
be far better to ACTUALLY add in ref-counting as a real part of 
the language (like Andrei has been pushing for the past two 
years, RCStr anyone?). Or to go with deadalnix's scheme which 
would allow it to be part of the library and  safe. This would 
allow it to be used in other areas in user code where it makes 
sense.

For the fourth time:

You're missing the forrest for the trees. D needs a general 
solution to the problem of GC code in Phobos. This tackles one 
specific area via special case but leaves every other GC 
allocation in Phobos, with no way to make it  safe  nogc. These 
will either require the holistic approach eventually or more 
special cases.
Apr 09 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/9/2017 4:14 PM, Jack Stouffer wrote:

 1. This still adds another special case to the language under the guise of not
 breaking backward compatibility. This is exactly how C++ became the mess it is
 today.

 2. You're adding in ref-counting in an area where it's only available to one
 very specific part of the language. This is one of the reasons people don't
like
 Go: it offers language features only in specific places at the compiler devs
 discretion. It would be far better to ACTUALLY add in ref-counting as a real
 part of the language (like Andrei has been pushing for the past two years,
RCStr
 anyone?). Or to go with deadalnix's scheme which would allow it to be part of
 the library and  safe. This would allow it to be used in other areas in user
 code where it makes sense.


You're right, it is a special case. On the other hand, it adds no new syntax at 
all, and the amount of code breakage should be very small (we'll see). Because 
of that, it should be upwards compatible if a more general scheme is devised, 
and it solves an immediate need.



 For the fourth time:

 You're missing the forrest for the trees. D needs a general solution to the
 problem of GC code in Phobos. This tackles one specific area via special case
 but leaves every other GC allocation in Phobos, with no way to make it  safe
  nogc. These will either require the holistic approach eventually or more
 special cases.


I'm not missing it. I'm well aware of that, though it is incorrect that GC code 
cannot be made  safe. It is implicitly safe.

Disagreeing with something is not the same as being unaware of it, ignoring it, 
etc. There have been many partial solutions proposed, and many promising ones 
that turned out to have fatal flaws. Rust has a total solution, but it comes at 
a very high cost - applications have to be re-engineered to use it, not just
add 
some annotations.

Bob Vila waving his hand with "Just install track lighting" just isn't good 
enough :-)

   https://www.youtube.com/watch?v=P9FHlhWqbus&feature=youtu.be&t=17

If it was as easy as installing track lighting, why aren't there any languages 
with pointers and memory safe ref counting (other than Rust)?

D will get memory safe ref counting (DIP1000 enables that), but it will be for 
structs, not classes. Exceptions are classes.
Apr 09 2017
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2017-04-09 05:26, Walter Bright wrote:

 My previous version did not survive implementation. Here's the revised
 version. I have submitted it as a DIP, and there's a trial
 implementation up:


What exactly does the user have to do to use throw a RC exception 
instead of a GC exception?

-- 
/Jacob Carlborg
Apr 10 2017
next sibling parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Monday, 10 April 2017 at 13:00:52 UTC, Jacob Carlborg wrote:

 On 2017-04-09 05:26, Walter Bright wrote:

 My previous version did not survive implementation. Here's the 
 revised
 version. I have submitted it as a DIP, and there's a trial
 implementation up:


 What exactly does the user have to do to use throw a RC 
 exception instead of a GC exception?


The compiler makes it refcounted if and only if you are 
"throwing" and "newing" in the same statment, i.e.

     throw new E(); // refcounted

I can see 2 reasons for this:

1) Because you are throwing the new object, your code has no 
opportunity to misuse the "ref-counted" object.  The exception 
handler takes over immediately so you have no opportunity to leak 
the object.  The only place that can reference it will be in the 
catch chain which will have the cleanup code auto generated by 
the compiler.

2) No syntax change! Because this isn't a general solution for 
ref-counted objects, a "no-syntax" feature allows the solution to 
be rolled out with no library changes which means we aren't 
investing a lot of unnecessary turmoil in case a new general 
solution comes out.

After thinking about the proposal the whole thing seems to be 
quite clever.  I think it takes some time to wrap your mind 
around why though.

If you want to force a GC exception (though I'm not sure why you 
would), you could do this:

     auto e = new E(); // GC
     throw e;

I like the discussion about this proposal and I think Walter has 
addressed the big concerns.  Everyone agrees this only solves a 
specific problem, but it's a BIG problem and since it doesn't 
change the syntax it can easily be swapped out for any future 
general solutions.

Also I think requiring a copy to escape an exception is fine, 
anyone who doesn't think so doesn't understand the overhead of an 
exception. But if it's really an issue for people (people who 
want to force GC exceptions for some reason), you could add a new 
method to the Exception object that gives you an escapable 
reference.

Exception getEscapableReference();

A refcounted exception would create a copy, whereas a GC 
exception would just return itself.
Apr 10 2017
parent Jacob Carlborg <doob me.com> writes:
On 2017-04-10 22:10, Jonathan Marler wrote:


 The compiler makes it refcounted if and only if you are "throwing" and
 "newing" in the same statment, i.e.

     throw new E(); // refcounted


Aha, now I understand. I don't think that was very clear from the beginning.

-- 
/Jacob Carlborg
Apr 11 2017
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/10/2017 6:00 AM, Jacob Carlborg wrote:

 What exactly does the user have to do to use throw a RC exception instead of a
 GC exception?


case 1:

    throw new Exception(args...);  // the only way an RC exception
				// is ever created

case 2:

    catch (Exception e) {
        ...
        throw e;  // increments `e`s refcount, if `e` is refcounted
    }
Apr 10 2017
prev sibling next sibling parent reply Lurker <lurker gmail.com> writes:
Everything looks good except this one line:

On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

     throw new E(string);


I don't like it for 2 reasons:

a) E e = new E(string); throw e;

    Should be *exactly* the same as

    throw new E(string).

    I think it's obvious why.


b) Using 'new' in  nogc code feels plain wrong.


I think a library method of explicitly creating a ref-counted 
throwable would be better. One can then do

E e = refCountedThrowable!(E)(string);
throw e;

or, interchangeably,

throw refCountedThrowable!(E)(string);

and

throw new E(string);

would be illegal because new cannot be called from within a  nogc 
method.

(haven't ever used  nogc so I'm probably wrong)
Apr 10 2017
parent reply Jonathan Marler <johnnymarler gmail.com> writes:
On Monday, 10 April 2017 at 20:52:21 UTC, Lurker wrote:

 Everything looks good except this one line:

 On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

     throw new E(string);


 I don't like it for 2 reasons:

 a) E e = new E(string); throw e;

    Should be *exactly* the same as

    throw new E(string).

    I think it's obvious why.


I agree this "seems wrong" at first, but because D does not have 
a general solution for reference counted objects, the first case 
you show can't be implemented safely.  The only reason that 
"throw new E()" can be safe is that control is ALWAYS and 
IMMEDIATELY given up to the exception handler, so no general 
solution is required for reference counted objects because the 
only place you need to implement reference counted management is 
in the exception handler itself.

Another way to think of it is that this proposal makes "throw 
new" into a special operator that is different than composing the 
"throw" and "new" operations independently. Once you realize this 
it's easy to understand and explain to others as well, i.e.

"throw"     operator (throw a Throwable object)
"new"       operator (create a GC object)
"throw new" operator (create and throw a reference-counted 
Throwable object)

Another way this could be done is instead of overloading "throw 
new", you could introduce a new operator like 
"throw_ref_counted", but then all code that wants to use 
ref-counted exceptions needs to change.  I think whether that's 
justified is debatable but one argument against it is that 
if/when D implements a general solution for ref-counted objects 
all the code would need to change AGAIN.


 b) Using 'new' in  nogc code feels plain wrong.


 I think a library method of explicitly creating a ref-counted 
 throwable would be better. One can then do

 E e = refCountedThrowable!(E)(string);
 throw e;

 or, interchangeably,

 throw refCountedThrowable!(E)(string);

 and

 throw new E(string);

 would be illegal because new cannot be called from within a 
  nogc method.

 (haven't ever used  nogc so I'm probably wrong)


Again since D does not support ref-counted objects in general, 
you can't do this. You could discuss how D should implement 
general ref-counted objects but that is a HARD problem that's not 
likely to be solved soon.

The beauty of this proposal is it's a relatively simple solution 
to a BIG problem.
Apr 10 2017
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/10/2017 2:44 PM, Jonathan Marler wrote:

 Another way to think of it is that this proposal makes "throw new" into a
 special operator that is different than composing the "throw" and "new"
 operations independently. Once you realize this it's easy to understand and
 explain to others as well, i.e.

 "throw"     operator (throw a Throwable object)
 "new"       operator (create a GC object)
 "throw new" operator (create and throw a reference-counted Throwable object)


That's right. Thanks for helping out, your explanations are better than mine!

(Once I realized this myself, the implementation got a lot simpler.)
Apr 10 2017
parent reply rjframe <dlang ryanjframe.com> writes:
On Mon, 10 Apr 2017 16:43:40 -0700, Walter Bright wrote:


 On 4/10/2017 2:44 PM, Jonathan Marler wrote:

 Another way to think of it is that this proposal makes "throw new" into
 a special operator that is different than composing the "throw" and
 "new" operations independently. Once you realize this it's easy to
 understand and explain to others as well, i.e.

 "throw"     operator (throw a Throwable object)
 "new"       operator (create a GC object)
 "throw new" operator (create and throw a reference-counted Throwable
 object)


 
 That's right. Thanks for helping out, your explanations are better than
 mine!
 
 (Once I realized this myself, the implementation got a lot simpler.)


I have no problem with this specific change, but this method of solving 
problems is going to turn D into a horrible language with all kinds of 
weird edge cases. I left Python for D mostly because the language was 
becoming one hack built on top of another; it would be nice if D could 
avoid that path. The occasional kludge may be necessary, but it shouldn't 
be normal or the first thought.

--Ryan
Apr 11 2017
next sibling parent HaraldZealot <harald_zealot tut.by> writes:
On Sunday, 09 April 2017 at 23:14:08 UTC, Jack Stoufer wrote:

You're missing the forrest for the trees. D needs a general 
solution to the problem of GC code in Phobos. This >tackles one 
specific area via special case but leaves every other GC 
allocation in Phobos, with no way to make >it  safe  nogc. These 
will either require the holistic approach eventually or more 
special cases.


On Tuesday, 11 April 2017 at 11:39:20 UTC, rjframe wrote:

 I have no problem with this specific change, but this method of 
 solving problems is going to turn D into a horrible language 
 with all kinds of weird edge cases. I left Python for D mostly 
 because the language was becoming one hack built on top of 
 another; it would be nice if D could avoid that path. The 
 occasional kludge may be necessary, but it shouldn't be normal 
 or the first thought.

 --Ryan


I'm personally belongs to the camp of perfectionists and holists. 
But there is bitter experience I want to share, that to be too 
purist with such approach just make unable things to happen. 
"Release early, release often" isn't just a good words, but real 
need, because of lack of early feedback your "perfect" solution 
just "SUDDENLY" may not match real use cases, and you don't have 
use cases before USE cases :)

So if something going to happen and solve bunch of problem it 
should be done (in such way that not prevent more general 
holistic solution to be gone into life at some point). The real 
issue is only bake transitional process in such way that it 
addresses the problems of majority.

So what could be done:
1. Implement this feature; but apply it only if special compiler 
flag is specified (the issue I see for this particular feature, 
that you should have versioned function declaration, IIRC there 
is no possibility to version of an attribute), emit all warnings, 
that may you current code clashes with upcoming feature
2. Introduce new flag that disable this feature and remove 
previous flag for enabling (from now it would be enabled by 
default), make the warnings errors.
3. Remove disabling flag.
(All this describes situation if feature is accepted _as is_ by 
language users).


Furthermore it seems to me that D need something like official 
binary nightmare releases. It introduces new features in the 
manner I have described, but also have a flag like 
`-disable-all-fresh-feature` which affect all features at stage 
2, for those of us who need industrial compilation. This 
"nightmare releases" require however much more frequent major 
release cycle (like each 2-3 months), and that isn't bad _per se_.
Apr 11 2017
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/11/2017 4:39 AM, rjframe wrote:

 I have no problem with this specific change, but this method of solving
 problems is going to turn D into a horrible language with all kinds of
 weird edge cases. I left Python for D mostly because the language was
 becoming one hack built on top of another; it would be nice if D could
 avoid that path. The occasional kludge may be necessary, but it shouldn't
 be normal or the first thought.


On the other hand, overly principled languages tend to not be as successful, 
because what people need to do with programs is often dirty.

Monads, and "functional reactive programming", are obtuse things that come
about 
when a functional programming language requires 100% purity and immutability.

Back in the 80's, like everyone else, I went about creating a GUI user
interface 
library. I discovered something interesting - what is orthogonal and consistent 
to a computer is anything but when dealing with people. What people view as 
orthogonal and consistent is a rat's nest of exceptions in the code to
implement 
it. This is what makes a user interface library fiendishly difficult to pull
off.

Language design is like that, too.

I hear what you're saying, and agree in principle. That is why this feature 
comes with no new syntax, and existing code should "just work" with it to the 
largest extent possible.
Apr 11 2017
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 11 April 2017 at 17:43:20 UTC, Walter Bright wrote:

 On the other hand, overly principled languages tend to not be 
 as successful, because what people need to do with programs is 
 often dirty.

 Monads, and "functional reactive programming", are obtuse 
 things that come about when a functional programming language 
 requires 100% purity and immutability.


Monads and "functional reactive programming" are not a 
manifestation of being principled but being ideological.

Javascript is principled.

Language that historically are very unprincipled such as PHP and 
C++, first didn't succeeded because they were unprincipled, but 
because they were quick and dirty hack solving people's problems 
where no other solution was available, and are moving toward 
being more principled with each version.
Apr 12 2017
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 11.04.2017 19:43, Walter Bright wrote:

 On 4/11/2017 4:39 AM, rjframe wrote:

 I have no problem with this specific change, but this method of solving
 problems is going to turn D into a horrible language with all kinds of
 weird edge cases. I left Python for D mostly because the language was
 becoming one hack built on top of another; it would be nice if D could
 avoid that path. The occasional kludge may be necessary, but it shouldn't
 be normal or the first thought.


 On the other hand, overly principled languages tend to not be as
 successful, because what people need to do with programs is often dirty.

 Monads, and "functional reactive programming", are obtuse things


They are actually obvious things that happen to usually be explained in 
the language of PL researchers, who are not always interested in 
creating a popular system.


 that
 come about when a functional programming language requires 100% purity
 and immutability.
 ...


Haskell does not do that. Why do you think it does?


 Back in the 80's, like everyone else, I went about creating a GUI user
 interface library. I discovered something interesting - what is
 orthogonal and consistent to a computer is anything but when dealing
 with people. What people view as orthogonal and consistent is a rat's
 nest of exceptions in the code to implement it. This is what makes a
 user interface library fiendishly difficult to pull off.
 ...


Would you mind sharing an example?


 Language design is like that, too.


But there is an objective standard of generality, orthogonality and
consistency, set by examples like pure type systems (such as CoC), or
even just the inference rules of first-order logic.


 I hear what you're saying, and agree in principle. That is why this
 feature comes with no new syntax, and existing code should "just work"
 with it to the largest extent possible.


The benefit is that "throw new Exception" becomes  nogc.

In terms of orthogonality and consistency, the proposal does not do so well:

- Orthogonality of 'throw' and 'new' is removed.

- Built-in new being  nogc sometimes but not at other times is inconsistent.

I think this is true for both computers and users.

I don't feel strongly about whether or not the proposal should be 
accepted as it does not really affect me, but it is a trade-off.
Apr 14 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/14/2017 3:41 PM, Timon Gehr wrote:

 Haskell does not do that. Why do you think it does?


Do what? I did not mention Haskell.


 Back in the 80's, like everyone else, I went about creating a GUI user
 interface library. I discovered something interesting - what is
 orthogonal and consistent to a computer is anything but when dealing
 with people. What people view as orthogonal and consistent is a rat's
 nest of exceptions in the code to implement it. This is what makes a
 user interface library fiendishly difficult to pull off.
 ...


 Would you mind sharing an example?


It was 30 years ago. I don't remember specifics, but I can think of one - the 
tab key. It means different things depending on what box you're in. Some more 
thoughts - is a box in a fixed position, or is it movable? Is a box 
user-resizeable or not? Is the size of a box determined by its content do do
you 
throw up scroll bars? or maybe relay out the contents? How do fonts affect
this? 
How about screen resolution? Which parts of a box should be selectable with cut 
operations and which not? It just goes on and on.

There's the famous Windows 'start' button which you click on to turn the 
computer off. Of course, it is consistent from a programming point of view, 
because it starts the shutdown program.

In any case, look at English. It is not remotely consistent, and drives 
programmers to fits trying to parse it. But people find it intuitive.
Apr 14 2017
next sibling parent Meta <jared771 gmail.com> writes:
But people find it intuitive.


This is a terrible example as English is anything but intuitive. 
Just ask any ESL student.

No human language is intuitive, which is why we strive to create 
intuitiveness and regularity in the languages we create, 
programming or otherwise.

If D ever becomes a fraction as messy and inconsistent as 
English, we will know that we have failed. I personally see this 
language change as a step in that direction, but I am not 
strongly invested either way.
Apr 14 2017
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 15.04.2017 01:29, Walter Bright wrote:

 On 4/14/2017 3:41 PM, Timon Gehr wrote:

 Haskell does not do that. Why do you think it does?


 Do what?


Require 100% purity and immutability. (You're underquoting.)


 I did not mention Haskell.
 ...


You said "come about when". Those things "came about" in Haskell.

This is the original monad paper (in an fp context): 
homepages.inf.ed.ac.uk/wadler/papers/marktoberdorf/baastad.pdf

(The paper mentions other programming languages in the intro, but it is 
easy to discover that this is just a courtesy/a way to establish relevance.)

This is the original FRP paper: http://conal.net/papers/icfp97/


 Back in the 80's, like everyone else, I went about creating a GUI user
 interface library. I discovered something interesting - what is
 orthogonal and consistent to a computer is anything but when dealing
 with people. What people view as orthogonal and consistent is a rat's
 nest of exceptions in the code to implement it. This is what makes a
 user interface library fiendishly difficult to pull off.
 ...


 Would you mind sharing an example?


 It was 30 years ago. I don't remember specifics, but I can think of one
 - the tab key. It means different things depending on what box you're
 in.


Inserting a tab character vs. moving to the next box?


 Some more thoughts - is a box in a fixed position, or is it movable?
 Is a box user-resizeable or not? Is the size of a box determined by its
 content do do you throw up scroll bars? or maybe relay out the contents?
 How do fonts affect this? How about screen resolution? Which parts of a
 box should be selectable with cut operations and which not? It just goes
 on and on.
 ...


I see. Maybe that is why I dislike using GUIs so much. :)


 There's the famous Windows 'start' button which you click on to turn the
 computer off. Of course, it is consistent from a programming point of
 view, because it starts the shutdown program.

 In any case, look at English. It is not remotely consistent, and drives
 programmers to fits trying to parse it. But people find it intuitive.


People get used to convoluted behaviours especially when in large 
groups. Synthetic languages are a great deal more orthogonal and 
consistent, because this is actually useful. I don't buy that aiming for 
inconsistencies is a way to make a system more intuitive when the user 
didn't have prior exposure to a similar system. Monads being unpopular 
has no bearing on whether the proposed feature is intuitive. (It isn't. 
There is a lot of precedent for a lack of explicit monads, but none for 
the proposal.)
Apr 14 2017
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/14/2017 10:04 PM, Timon Gehr wrote:

 I don't buy that aiming for inconsistencies is a way to
 make a system more intuitive when the user didn't have prior exposure to a
 similar system.


Please propose a better design.
Apr 15 2017
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 15.04.2017 10:45, Walter Bright wrote:

 On 4/14/2017 10:04 PM, Timon Gehr wrote:

 I don't buy that aiming for inconsistencies is a way to
 make a system more intuitive when the user didn't have prior exposure
 to a
 similar system.


 Please propose a better design.


I'm criticizing the justification, not the design. (Hence OT.)

What I'm saying is that there is no way that lack of 
orthogonality/consistency can be an argument in favour of the design.

The arguments in favour of the design (that I can see) are that it is 
simple, easy to explain, easy to implement, easy to ignore, that it 
works around the problem, and that it is your own.
Apr 15 2017
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 15.04.2017 07:04, Timon Gehr wrote:

 You said "come about when". Those things "came about" in Haskell.

 This is the original monad paper (in an fp context):
 homepages.inf.ed.ac.uk/wadler/papers/marktoberdorf/baastad.pdf


Actually, it is this one: 
https://page.mi.fu-berlin.de/scravy/realworldhaskell/materialien/the-essence-of-functional-programming.pdf
Apr 15 2017
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 10 April 2017 at 21:44:32 UTC, Jonathan Marler wrote:

 "throw"     operator (throw a Throwable object)
 "new"       operator (create a GC object)
 "throw new" operator (create and throw a reference-counted 
 Throwable object)


There is no need for this, the compiler already understands the 
notion of unique for newed objects and object returned from pure 
functions.
Apr 12 2017
prev sibling next sibling parent reply Meta <jared771 gmail.com> writes:
My knee jerk reaction is that it's a very bad thing that "new" 
means the same thing everywhere in the language (allocate and 
initialize some GC-managed memory), except for this one case.
Apr 10 2017
next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Monday, 10 April 2017 at 23:16:48 UTC, Meta wrote:

 My knee jerk reaction is that it's a very bad thing that "new" 
 means the same thing everywhere in the language (allocate and 
 initialize some GC-managed memory), except for this one case.


`new SomeClass` has never implied GC allocation only - see 
https://dlang.org/spec/class.html#allocators, though I agree it 
looks kind of odd to add a special case. But it's hard to miss 
the benefits - all code that was not  nogc only because of 
allocating new exceptions and throwing them, becomes  nogc for 
free. And this is huge - just grep for uses of enforce in Phobos 
and many other prominent projects.
Apr 10 2017
prev sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Monday, 10 April 2017 at 23:16:48 UTC, Meta wrote:

 My knee jerk reaction is that it's a very bad thing that "new" 
 means the same thing everywhere in the language (allocate and 
 initialize some GC-managed memory), except for this one case.


Actually, in addition to user defined overloads (which are 
deprecated, granted), `new` also can mean stack allocation 
already! See the `scope` storage class and scope classes.

The language is free to optimize `new` as it sees fit. (IMO, 
that's the only real justification for it even being a built in 
language feature instead of an ordinary library function.) The 
real changes in this proposal are:

1) if the compiler is aware that new will be implemented in such 
a manner as to not use the gc, it will pass the static  nogc fit.

2) the catch block is restricted a bit in order to enable this 
optimization.

`new` itself isn't really changing since it was already allowed 
to do this in cases the compiler can prove its lifetime.
Apr 10 2017
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/10/2017 5:11 PM, Adam D. Ruppe wrote:

 `new` itself isn't really changing since it was already allowed to do this in
 cases the compiler can prove its lifetime.


Right. It isn't required to actually allocate it on the GC, just that it
behaves 
as if it did.

The real change in semantics is the implicit addition of 'scope' on the catch 
object declaration.
Apr 10 2017
prev sibling parent reply Meta <jared771 gmail.com> writes:
On Tuesday, 11 April 2017 at 00:11:01 UTC, Adam D. Ruppe wrote:

 On Monday, 10 April 2017 at 23:16:48 UTC, Meta wrote:

 My knee jerk reaction is that it's a very bad thing that "new" 
 means the same thing everywhere in the language (allocate and 
 initialize some GC-managed memory), except for this one case.


 Actually, in addition to user defined overloads (which are 
 deprecated, granted), `new` also can mean stack allocation 
 already! See the `scope` storage class and scope classes.


I don't agree as you have to add `scope` for the class to be 
allocated on the stack.


 The language is free to optimize `new` as it sees fit. (IMO, 
 that's the only real justification for it even being a built in 
 language feature instead of an ordinary library function.) The 
 real changes in this proposal are:


Yes, but it must be done in such a way that a user never knows 
the difference. I have my doubts about whether replacing all heap 
allocations in `throw new` expressions will be as transparent. 
Actually it's already not as there are restrictions on catch 
blocks that catch refcounted Exceptions.
Apr 10 2017
parent Jacob Carlborg <doob me.com> writes:
On 2017-04-11 04:11, Meta wrote:


 I don't agree as you have to add `scope` for the class to be allocated
 on the stack.


It's enough to add "scope" when declaring the variable:

scope foo = new Object;

-- 
/Jacob Carlborg
Apr 11 2017
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 My previous version did not survive implementation. Here's the 
 revised version. I have submitted it as a DIP, and there's a 
 trial implementation up:

 [...]


Instead of adding new runtime helper functions like 
_d_newThrowable and _d_delThrowable, can we leverage the existing 
(though deprecated) support for class (de)allocators and 
essentially divide the _d_delThrowable implementation between the 
destructor and the deallocator member function? This will go hand 
in hand with the work that Lucia has been doing on making the 
runtime more generic and reducing the number of special cases in 
the compiler [1] [2] [3]. Obviously you have done the work 
already, but in theory at least, with my proposal users should be 
able to override the memory allocation method in derived 
exception classes, without any changes to druntime.

[1]: https://github.com/dlang/druntime/pull/1781
[2]: https://github.com/dlang/druntime/pull/1792
[3]: http://dconf.org/2017/talks/cojocaru.html
Apr 10 2017
next sibling parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 11 April 2017 at 00:46:51 UTC, Petar Kirov 
[ZombineDev] wrote:

 On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 My previous version did not survive implementation. Here's the 
 revised version. I have submitted it as a DIP, and there's a 
 trial implementation up:

 [...]


 Instead of adding new runtime helper functions like 
 _d_newThrowable and _d_delThrowable, can we leverage the 
 existing (though deprecated) support for class (de)allocators 
 and essentially divide the _d_delThrowable implementation 
 between the destructor and the deallocator member function? 
 This will go hand in hand with the work that Lucia has been 
 doing on making the runtime more generic and reducing the 
 number of special cases in the compiler [1] [2] [3]. Obviously 
 you have done the work already, but in theory at least, with my 
 proposal users should be able to override the memory allocation 
 method in derived exception classes, without any changes to 
 druntime.

 [1]: https://github.com/dlang/druntime/pull/1781
 [2]: https://github.com/dlang/druntime/pull/1792
 [3]: http://dconf.org/2017/talks/cojocaru.html


Apart from that the proposal seems fine to me. I would much 
prefer a more general/principled solution to the problem like the 
one deadalnix proposes, but that shouldn't stop us from 
experimenting with an ad hoc approach in the short-term, while a 
more solid solution is developed.
Apr 10 2017
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/10/2017 5:46 PM, Petar Kirov [ZombineDev] wrote:

 Instead of adding new runtime helper functions like _d_newThrowable and
 _d_delThrowable, can we leverage the existing (though deprecated) support for
 class (de)allocators and essentially divide the _d_delThrowable implementation
 between the destructor and the deallocator member function? This will go hand
in
 hand with the work that Lucia has been doing on making the runtime more generic
 and reducing the number of special cases in the compiler [1] [2] [3]. Obviously
 you have done the work already, but in theory at least, with my proposal users
 should be able to override the memory allocation method in derived exception
 classes, without any changes to druntime.


I considered that, but discarded it because then all Throwables would be 
allocated that way, in a completely uncontrolled manner.
Apr 10 2017
prev sibling parent Chris M. <chrismohrfeld comcast.net> writes:
On Sunday, 9 April 2017 at 03:26:14 UTC, Walter Bright wrote:

 My previous version did not survive implementation. Here's the 
 revised version. I have submitted it as a DIP, and there's a 
 trial implementation up:

 [...]


Maybe someone already explained this or the reason is obvious, 
but why couldn't we use  nogc to make Exceptions refcounted as 
opposed to needing to specifically use

     throw new Exception();
Apr 14 2017