www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - collectNoStack should be axed

reply Steven Schveighoffer <schveiguy gmail.com> writes:
So I am helping to add a new GC to dlang, and one thing I have 
run into is that all correctly-written GC implementations must 
implement the function `collectNoStack`.

This is defined the the GC interface here: 
https://github.com/dlang/dmd/blob/c09adbbc2793aedcc3569681acfc42260d3b0e4b/druntime/src/core/gc/gcinterface.d#L59

When looking further into what this actually means, alarmingly, 
it means exactly what it says -- run a collection cycle without 
examining *any* thread stacks for roots.

What in God's name is the point of this? Won't this just collect 
things that are still *actively being referenced by threads*?! 
The answer is -- yes.

Well, I wanted to know more about how this could be valid, so I 
did more research and it's kind of a fun story. Some of this is 
conjecture as I wasn't around for the beginnings of this, and 
having help filling in the holes is appreciated.

In looking to see which code actually calls this, I can find only 
one use, here: 
https://github.com/dlang/dmd/blob/c09adbbc2793aedcc3569681acfc42260d3b0e4b/druntime/src/core/internal/gc/proxy.d#L119

Quoting the code in that snippet, so you can keep the 
"explanatory comment" in mind:

```d
// NOTE: There may be daemons threads still running when this 
routine is
//       called.  If so, cleaning memory out from under then is a 
good
//       way to make them crash horribly.  This probably doesn't 
matter
//       much since the app is supposed to be shutting down 
anyway, but
//       I'm disabling cleanup for now until I can think about it 
some
//       more.
//
// NOTE: Due to popular demand, this has been re-enabled.  It 
still has
//       the problems mentioned above though, so I guess we'll 
see.

instance.collectNoStack();  // not really a 'collect all' -- 
still scans
                             // static data area, roots, and 
ranges.
```

Which is the "collect" configuration of how to terminate the GC.

In a way, this makes sense, because if you are terminating the 
GC, the GC is going away, and it doesn't really matter if 
anything is referring to the data, those references are all gonna 
die.

OK.... but why skip just the thread stacks? In fact, why scan 
*anything at all*? I'm not the first one to think this, there's a 
second configuration, which does exactly this, which is in 
another case of that switch.

To try and pin down why this is there, and what the "popular 
demand" note means, I started using git blame (I have to say, the 
world is a better place with git and github around, I shudder to 
think how I would have had to find the history of this with 
subversion).

Aaaaand I traced it back to the *beginning of druntime*. Yes, 
this is the repository after the very first commit from Sean 
Kelly for druntime: 
https://github.com/dlang/dmd/blob/6837c0cd426f7e828aec1a2bdc941ac9b722dd14/src/gc/basic/gc.d#L73

So, I thought, maybe I will email Sean? He might know why this 
note is there.

But wait! druntime takes its lineage from Tango! And Tango is 
also on github >:)

And now, we find out when the first note was written: 
https://github.com/SiegeLord/Tango-D2/commit/03ea5067558829b8c99e3cf12bb0e55c43e29269

Hoooold on a second. The line that was commented out was... not 
the full collect. That was *already commented out*, and actually, 
it was just doing what I proposed above -- collecting all blocks 
regardless of roots.

The note was added when *that* was commented out, and apparently, 
the Tango runtime just didn't do any collection at the end of a 
program.

What about the second note? That got added "by popular demand" 
later:

https://github.com/SiegeLord/Tango-D2/commit/5984ec967eaffb1d3c1c7504e9349f18c8b36038

This means, the `_fullCollectNoStack` was added back in (and 
apparently the second call to run the destructors and clean all 
garbage, which must have been separated back then). I can guess 
because people thought it should be done.

The note concerns "deamon threads". What is a daemon thread? It's 
a thread that does not get joined at the end of execution (that 
is still the same, and you can see the explanation here: 
https://dlang.org/phobos/core_thread_threadbase.html#.ThreadBase.isDaemon). I
checked, and literally this is the only place the `isDaemon` flag is used.
Daemon threads still are stopped for GC, and still get scanned. They just
aren't waited for at the end of main.

OK, now the note actually makes sense -- if you clean all the 
garbage at the end of main *without scanning thread stacks*, then 
you clean out memory that the daemon threads may still be using.

But.. does it? When did this *ever work*? Isn't the GC going away?

I wanted to find out the true entomology of this... "thing". So I 
kept going back. And as it turns out, the `collectNoStack` 
function comes from D1! That's right, we still have that to look 
at as well: 
https://github.com/dlang/phobos/blob/1f763bca8d8db14cd4e7af89b1667569c002361c/internal/gc/gc.d#L171

Hm.. OK, so this is what D always did. But why? I wanted to find 
out exactly what happened differently when the fullCollectNoStack 
function was called, and I got my answer:

https://github.com/dlang/phobos/blob/1f763bca8d8db14cd4e7af89b1667569c002361c/internal/gc/gcx.d#L1031

Peruse through that file, and you'll see the `nostack` variable 
is used in one place: 
https://github.com/dlang/phobos/blob/1f763bca8d8db14cd4e7af89b1667569c002361c/internal/gc/gcx.d#L2030

And look there... it's only skipping the stack scanning *if there 
is exactly one thread*.

In other words, with D1, where this poorly named 
`fullCollectNoStack` function existed, it actually would scan 
with stacks as long as you created multiple threads. That is, in 
certain (very common) cases, the `fullCollectNoStack` would scan 
stacks. Should it have been called 
`fullCollectMaybeNoStacksIfSingleThreaded`? I digress...

And in fact, when D1 was compiled in "single threaded mode", 
indeed scanning of thread stack was skipped: 
https://github.com/dlang/phobos/blob/1f763bca8d8db14cd4e7af89b1667569c002361c/internal/gc/gcx.d#L2117

Let's think back to why the heck we have this going on. My theory 
is that people who are new to GC or don't really understand how 
GC works, run a test like the following:

```d
struct S
{
    ~this() {
      printf("Destroying!\n");
    }
}

void main()
{
    S *s = new S;
}
```

If they don't get a printout, they post an angry/confused message 
on the forums saying



If the stack of `main` is scanned, it's possible there's still a 
reference to the `s` there. It could even still be in registers 
for the thread. And that might mean that the GC won't clean it up.

The truth is, there is no guarantee any destructors are run. And 
especially in 32-bit D (which is what D was exclusively for a 
long time), random 32-bit numbers might accidentally "point" at 
the memory block.

So maybe, the solution Walter came up with (and I'm just guessing 
here), is hey, we are shutting down anyways, just avoid scanning 
the main thread stack, and we can satisfy the unwashed masses.

But that brings us back to *WHY THE HELL DO WE STILL HAVE THIS*? 
My guess is that the note keeps people from removing it. If we 
are doing a scan at all, scanning thread stacks as roots should 
be a trivial addition to the scan. Skipping it just adds an extra 
layer of complication to the implementation that is unnecessary. 
But that note where "I'm disabling cleanup for now until I can 
think about it some more" seems to be applying to an actual scan 
(not the blunt destruction of all memory, which is the line 
commented out when the note was added). That is causing people to 
hesitate and leave things be. Someone was behind that "I", and I 
probably should step on that someone's toes, they knew what they 
were doing.

And they did, but what they did isn't what the code says (my 
hypothesis).

So my solution is, let's just get rid of this extra function. 
Let's get rid of any idea of doing a half-ass scan that at best 
collects some extra stuff that might not be referenced and at 
worst pulls the rug out from still-running threads. And if you 
actually called this somehow in the middle of a program, it will 
corrupt all your memory immediately.

I did a PR to just see what happens when we do a full scan 
instead of the "no stack" scan, and the results are pretty 
positive. I'm going to update the PR to really remove all the 
tentacles of the "nostack" variable, but I wanted to bring this 
story to light because it's too long and bizarre to explain in 
the notes of a PR.

https://github.com/dlang/dmd/pull/16401

If there are any good reasons why we should have this, or I got 
something wrong, please let me know!

-Steve
Apr 21
next sibling parent reply Daniel N <no public.email> writes:
On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer 
wrote:

 If they don't get a printout, they post an angry/confused 
 message on the forums saying



 If the stack of `main` is scanned, it's possible there's still 
 a reference to the `s` there. It could even still be in 
 registers for the thread. And that might mean that the GC won't 
 clean it up.

 The truth is, there is no guarantee any destructors are run. 
 And especially in 32-bit D (which is what D was exclusively for 
 a long time), random 32-bit numbers might accidentally "point" 
 at the memory block.


Great digging, I suspect your are correct.

Why do we even support destructors for GC allocated objects? You 
can't depend on it closing file-descriptors or other resources.
Apr 21
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/04/2024 7:57 AM, Daniel N wrote:

 On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer wrote:

 If they don't get a printout, they post an angry/confused message on 
 the forums saying



 If the stack of `main` is scanned, it's possible there's still a 
 reference to the `s` there. It could even still be in registers for 
 the thread. And that might mean that the GC won't clean it up.

 The truth is, there is no guarantee any destructors are run. And 
 especially in 32-bit D (which is what D was exclusively for a long 
 time), random 32-bit numbers might accidentally "point" at the memory 
 block.


 
 Great digging, I suspect your are correct.
 
 Why do we even support destructors for GC allocated objects? You can't 
 depend on it closing file-descriptors or other resources.


I understand why removing it would seem like a great idea.

But it would mean total segmentation of reference counted types from 
going into non-RC memory.

I can't even get Walter to agree to have RC in the language (even if it 
is reluctantly), let alone that...
Apr 21
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/21/2024 1:01 PM, Richard (Rikki) Andrew Cattermole wrote:

 I can't even get Walter to agree to have RC in the language (even if it is 
 reluctantly), let alone that...


A few years back, we really tried to find a way to do it that was memory safe. 
We failed. Timon presented us with a use-after-free case that we couldn't 
resolve. Re-orienting D around a memory-unsafe construct is not the future for
D.

There's also the problem that the decrement has to be in a finally block, 
because of exceptions, which makes things bloated and slow.

P.S. A borrow checker resolves the use-after-free case, but RC isn't needed if 
one is using a borrow checker.
Apr 21
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/04/2024 9:07 AM, Walter Bright wrote:

 On 4/21/2024 1:01 PM, Richard (Rikki) Andrew Cattermole wrote:

 I can't even get Walter to agree to have RC in the language (even if 
 it is reluctantly), let alone that...


 
 A few years back, we really tried to find a way to do it that was memory 
 safe. We failed. Timon presented us with a use-after-free case that we 
 couldn't resolve. Re-orienting D around a memory-unsafe construct is not 
 the future for D.
 
 There's also the problem that the decrement has to be in a finally 
 block, because of exceptions, which makes things bloated and slow.


I suspect that I need to learn more about the exception hooks.

My current understanding is that there is a subset that are entirely 
optional and are only needed to be called if cleanup occurs. Not quite 
the same thing as an exception catch.


 P.S. A borrow checker resolves the use-after-free case, but RC isn't 
 needed if one is using a borrow checker.


You are applying the borrow checker to both the owner and the borrow.

I want it only on the borrow.

The borrow isn't exclusive, it doesn't have the guarantees isolated 
would give. All it guarantees is that the borrow cannot escape the owner.

If you apply the borrow checker to the owner as well, you miss out on 
the ability to use it with data structures, DOM's, that sort of thing. 
All things I want to use it for.
Apr 21
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/04/2024 9:21 AM, Richard (Rikki) Andrew Cattermole wrote:

 On 22/04/2024 9:07 AM, Walter Bright wrote:

 On 4/21/2024 1:01 PM, Richard (Rikki) Andrew Cattermole wrote:

 I can't even get Walter to agree to have RC in the language (even if 
 it is reluctantly), let alone that...


 A few years back, we really tried to find a way to do it that was 
 memory safe. We failed. Timon presented us with a use-after-free case 
 that we couldn't resolve. Re-orienting D around a memory-unsafe 
 construct is not the future for D.

 There's also the problem that the decrement has to be in a finally 
 block, because of exceptions, which makes things bloated and slow.


 
 I suspect that I need to learn more about the exception hooks.
 
 My current understanding is that there is a subset that are entirely 
 optional and are only needed to be called if cleanup occurs. Not quite 
 the same thing as an exception catch.


After reviewing Windows documentation and assembly for linux, it appears 
my previous understanding is correct.

You only have to deal with cleanup which is checked for anyway, if you 
throw an exception.

So in other words, slow path only. I don't understand the issue here.

It is equivalent to a struct stored on the stack having its destructor 
called when an exception is thrown.
Apr 21
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Sunday, 21 April 2024 at 19:57:04 UTC, Daniel N wrote:

 Why do we even support destructors for GC allocated objects? 
 You can't depend on it closing file-descriptors or other 
 resources.


You can't depend on the GC running. But if for some reason it 
does get run, why wouldn't you close the resource instead of 
leaking it?

Note that even if you are cleaning up the descriptor manually, 
using `destroy` is a valid mechanism.

I used destructors to close things like this a long time ago in a 
project and it worked OK.

-Steve
Apr 21
prev sibling parent =?UTF-8?B?Q2hsb8Op?= <chloekek use.startmail.com> writes:
On 4/21/24 21:57, Daniel N wrote:

 On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer wrote:
 
 Why do we even support destructors for GC allocated objects? You can't 
 depend on it closing file-descriptors or other resources.
 
 
 


Closing non-memory resources is not the only use case for destructors on 
GC-allocated objects. Another use case is freeing non-GC-allocated 
memory that the object encapsulates. Non-deterministic destruction is 
perfectly fine for such a use case. An example of this in Phobos is 
std.zlib.Compress.
Apr 22
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer 
wrote:

 So I am helping to add a new GC to dlang, and one thing I have 
 run into is that all correctly-written GC implementations must 
 implement the function `collectNoStack`.


Great to hear that something is coming in the GC land.


 I did a PR to just see what happens when we do a full scan 
 instead of the "no stack" scan, and the results are pretty 
 positive. I'm going to update the PR to really remove all the 
 tentacles of the "nostack" variable, but I wanted to bring this 
 story to light because it's too long and bizarre to explain in 
 the notes of a PR.

 https://github.com/dlang/dmd/pull/16401

 If there are any good reasons why we should have this, or I got 
 something wrong, please let me know!


As you've found it - to run finalizers on everything that might 
be still referenced by the main thread. For the unwashed masses 
that expect finalizers to run at shutdown.

--
Dmitry Olshansky
CEO   Glowlabs
https://olshansky.me
Apr 21
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Quite an awesome bit of detective work! A fine and most entertaining read it
is. 
I recommend you include that text in any PR to fix it.


 So maybe, the solution Walter came up with (and I'm just guessing here), is 


hey, we are shutting down anyways, just avoid scanning the main thread stack, 
and we can satisfy the unwashed masses.

If I did write that function, I don't remember doing so, or why I would. 
Unfortunately, D1 predates our use of github, so it wouldn't be easy trying to 
figure out who wrote that.
Apr 21
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/04/2024 8:58 AM, Walter Bright wrote:

 Quite an awesome bit of detective work! A fine and most entertaining 
 read it is. I recommend you include that text in any PR to fix it.
 
  > So maybe, the solution Walter came up with (and I'm just guessing 
 here), is hey, we are shutting down anyways, just avoid scanning the 
 main thread stack, and we can satisfy the unwashed masses.
 
 If I did write that function, I don't remember doing so, or why I would. 
 Unfortunately, D1 predates our use of github, so it wouldn't be easy 
 trying to figure out who wrote that.


Nothing in your email archive?
Apr 21
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/04/2024 9:03 AM, Richard (Rikki) Andrew Cattermole wrote:

 On 22/04/2024 8:58 AM, Walter Bright wrote:

 Quite an awesome bit of detective work! A fine and most entertaining 
 read it is. I recommend you include that text in any PR to fix it.

  > So maybe, the solution Walter came up with (and I'm just guessing 
 here), is hey, we are shutting down anyways, just avoid scanning the 
 main thread stack, and we can satisfy the unwashed masses.

 If I did write that function, I don't remember doing so, or why I 
 would. Unfortunately, D1 predates our use of github, so it wouldn't be 
 easy trying to figure out who wrote that.


 
 Nothing in your email archive?


dmd 0.5 is the oldest dmd available on release archive has it.

So either its in your email archive or you wrote it.
Apr 21
prev sibling parent reply Brad Roberts <braddr puremagic.com> writes:
Before github we used dsource.org and subversion:
     http://dsource.org/projects/druntime/browser

It looks like the history on github retains that history, which matches 
with my memory of how we transitioned over.  The history seems to start 
at about 2008.

What I'm not seeing is much pre-D2 history.  There's some, but I thought 
I'd built up a full D1 history, or mostly full -- at least a per-release 
snapshot.  Did those early release tarball/zips not contain the druntime 
code maybe?

On 4/21/2024 1:58 PM, Walter Bright via Digitalmars-d wrote:

 Quite an awesome bit of detective work! A fine and most entertaining 
 read it is. I recommend you include that text in any PR to fix it.
 
  > So maybe, the solution Walter came up with (and I'm just guessing 
 here), is hey, we are shutting down anyways, just avoid scanning the 
 main thread stack, and we can satisfy the unwashed masses.
 
 If I did write that function, I don't remember doing so, or why I would. 
 Unfortunately, D1 predates our use of github, so it wouldn't be easy 
 trying to figure out who wrote that.
Apr 21
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/04/2024 9:55 AM, Brad Roberts wrote:

 Before github we used dsource.org and subversion:
      http://dsource.org/projects/druntime/browser
 
 It looks like the history on github retains that history, which matches 
 with my memory of how we transitioned over.  The history seems to start 
 at about 2008.
 
 What I'm not seeing is much pre-D2 history.  There's some, but I thought 
 I'd built up a full D1 history, or mostly full -- at least a per-release 
 snapshot.  Did those early release tarball/zips not contain the druntime 
 code maybe?
 
 On 4/21/2024 1:58 PM, Walter Bright via Digitalmars-d wrote:

 Quite an awesome bit of detective work! A fine and most entertaining 
 read it is. I recommend you include that text in any PR to fix it.

  > So maybe, the solution Walter came up with (and I'm just guessing 
 here), is hey, we are shutting down anyways, just avoid scanning the 
 main thread stack, and we can satisfy the unwashed masses.

 If I did write that function, I don't remember doing so, or why I 
 would. Unfortunately, D1 predates our use of github, so it wouldn't be 
 easy trying to figure out who wrote that.




The oldest version of D that is in the release archive is 0.5, which has it.

That is from 17/11/2002

So wayyyyy before dsource.
Apr 21
next sibling parent Brad Roberts <braddr puremagic.com> writes:
On 4/21/2024 2:58 PM, Richard (Rikki) Andrew Cattermole via 
Digitalmars-d wrote:

 On 22/04/2024 9:55 AM, Brad Roberts wrote:

 Before github we used dsource.org and subversion:
      http://dsource.org/projects/druntime/browser

 It looks like the history on github retains that history, which 
 matches with my memory of how we transitioned over.  The history seems 
 to start at about 2008.

 What I'm not seeing is much pre-D2 history.  There's some, but I 
 thought I'd built up a full D1 history, or mostly full -- at least a 
 per-release snapshot.  Did those early release tarball/zips not 
 contain the druntime code maybe?

 On 4/21/2024 1:58 PM, Walter Bright via Digitalmars-d wrote:

 Quite an awesome bit of detective work! A fine and most entertaining 
 read it is. I recommend you include that text in any PR to fix it.

  > So maybe, the solution Walter came up with (and I'm just guessing 
 here), is hey, we are shutting down anyways, just avoid scanning the 
 main thread stack, and we can satisfy the unwashed masses.

 If I did write that function, I don't remember doing so, or why I 
 would. Unfortunately, D1 predates our use of github, so it wouldn't 
 be easy trying to figure out who wrote that.




 
 The oldest version of D that is in the release archive is 0.5, which has 
 it.
 
 That is from 17/11/2002
 
 So wayyyyy before dsource.


Ok, the older code is in the phobos repository and goes back to phobos 
0.1.  I did that import, and the commit is dated september 9, 2007.  The 
exact same history as on dsource.org for phobos, just to double check. 
So, that's the best history that Walter was able to dig up for me.

So, it's been there that way from the earliest history I ever saw.
Apr 21
prev sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/04/2024 9:58 AM, Richard (Rikki) Andrew Cattermole wrote:

 The oldest version of D that is in the release archive is 0.5, which has 
 it.
 
 That is from 17/11/2002
 
 So wayyyyy before dsource.


Found an even older one!

dmdalpha19.zip from ftp.digitalmars.com

Contains fullcollectNoStack still.

But this time its in C.

Also:

```
//
// Copyright (C) 2001 by Digital Mars
// All Rights Reserved
// Written by Walter Bright
// licensed to Chromium Communications
```

Interesting header.

I've gone through that ftp server. Doesn't appear to be anything older.

At this point somebody is gonna have to get lucky, perhaps booting up 
some old machines or backups.
Apr 21
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/04/2024 10:26 AM, Richard (Rikki) Andrew Cattermole wrote:

 On 22/04/2024 9:58 AM, Richard (Rikki) Andrew Cattermole wrote:

 The oldest version of D that is in the release archive is 0.5, which 
 has it.

 That is from 17/11/2002

 So wayyyyy before dsource.


 
 Found an even older one!
 
 dmdalpha19.zip from ftp.digitalmars.com
 
 Contains fullcollectNoStack still.
 
 But this time its in C.
 
 Also:
 
 ```
 //
 // Copyright (C) 2001 by Digital Mars
 // All Rights Reserved
 // Written by Walter Bright
 // licensed to Chromium Communications
 ```
 
 Interesting header.
 
 I've gone through that ftp server. Doesn't appear to be anything older.
 
 At this point somebody is gonna have to get lucky, perhaps booting up 
 some old machines or backups.


I've been able to find an alternative GC implementation.

August 21 2002, https://web.archive.org/web/20170913015932/http://opend.org/

Unfortunately that GC doesn't look like alpha19.zip's and that is older 
as its from February 8 2002.

So that is a complete dead end (it doesn't have it).

Welp I'm out of ideas, looks like that is it.
Apr 21
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Sunday, 21 April 2024 at 21:55:32 UTC, Brad Roberts wrote:

 Before github we used dsource.org and subversion:
     http://dsource.org/projects/druntime/browser

 It looks like the history on github retains that history, which 
 matches with my memory of how we transitioned over.  The 
 history seems to start at about 2008.

 What I'm not seeing is much pre-D2 history.  There's some, but 
 I thought I'd built up a full D1 history, or mostly full -- at 
 least a per-release snapshot.  Did those early release 
 tarball/zips not contain the druntime code maybe?


druntime was D2 only. D1 never supported druntime if I recall 
correctly. If you wanted what was in druntime, you used Tango 
(because that's where druntime came from). Remember that druntime 
is the attempt to be able to have both D2 phobos and Tango using 
the same runtime (this was the big problem with using both Tango 
and phobos libs -- incompatible runtimes).

For non-Tango D1, the runtime code was all in phobos.

So if you look at Tango history you will find where it comes 
from. I traced it all the way back from the beginning, where it 
was copied from D1 phobos (which had the function as I showed). 
Once I found it always was nostack, I just stopped going back.

I was going to start looking at dsource for tango history, but 
remembered there was a D2 port which also has the D1 history ;)

-Steve
Apr 21
prev sibling next sibling parent John Dougan <jdougan acm.org> writes:
On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer 
wrote:

 So I am helping to add a new GC to dlang, and one thing I have 
 run into is that all correctly-written GC implementations must 
 implement the function `collectNoStack`.


...A couple hundered lines removed...

 https://github.com/dlang/dmd/pull/16401

 If there are any good reasons why we should have this, or I got 
 something wrong, please let me know!

 -Steve


Beautiful! I love to see software archaeology in action. Are 
those the only serious WTFs so far?

  --John
Apr 21
prev sibling next sibling parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer 
wrote:

 To try and pin down why this is there, and what the "popular 
 demand" note means, I started using git blame (I have to say, 
 the world is a better place with git and github around, I 
 shudder to think how I would have had to find the history of 
 this with subversion).


Hey! dsource.org is still alive! ;-)

http://www.dsource.org/projects/phobos/browser/branches/phobos-1.x/phobos

http://www.dsource.org/projects/tango/browser/trunk
Apr 22
prev sibling next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer 
wrote:

 So I am helping to add a new GC to dlang, and one thing I have 
 run into is that all correctly-written GC implementations must 
 implement the function `collectNoStack`.

 [...]


Amazing.

I'm biased towards deleting code, so...
Apr 22
prev sibling parent reply cc <cc nevernet.com> writes:
On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer 
wrote:

 If they don't get a printout, they post an angry/confused 
 message on the forums saying




If a user writes a destructor, and that destructor is never 
called sometime after a `new`, then yes, I think "Why isn't the 
GC working?" is a valid and fair question for that user to ask.  
I don't see any way in which that expectation should be 
considered *unreasonable*. If the answer is "No you don't 
understand, the GC internals such that this- that- and the other 
thing- means you can't have guaranteed destructors", then I'd say 
you've failed to make the sale and the customer is right to walk 
away.

As an aside, I would observe if one wants them to stay, one 
probably shouldn't be in the habit of calling them "the unwashed 
masses", either.
Apr 23
next sibling parent reply Quirin Schroll <qs.il.paperinik gmail.com> writes:
On Tuesday, 23 April 2024 at 07:38:25 UTC, cc wrote:

 On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer 
 wrote:

 If they don't get a printout, they post an angry/confused 
 message on the forums saying




 If a user writes a destructor, and that destructor is never 
 called sometime after a `new`, then yes, I think "Why isn't the 
 GC working?" is a valid and fair question for that user to ask. 
  I don't see any way in which that expectation should be 
 considered *unreasonable*. If the answer is "No you don't 
 understand, the GC internals such that this- that- and the 
 other thing- means you can't have guaranteed destructors", then 
 I'd say you've failed to make the sale and the customer is 
 right to walk away.


I always thought destructors run on GC-allocated objects some 
time between the last reference to the object disappears and the 
application exits. If you want some cleanup to run promptly after 
last reference is gone, a destructor is not the right tool. (You 
need reference counting or something like that, if a unique owner 
cannot be established.) The destructor not running at all is 
really surprising if you allocated the object normally.
Apr 23
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Tuesday, 23 April 2024 at 12:18:34 UTC, Quirin Schroll wrote:

 The destructor not running at all is really surprising if you 
 allocated the object normally.


It normally happens but is not guaranteed, especially with a 
conservative collector (one which does not know whether a word of 
memory is a pointer or not).

-Steve
Apr 23
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Tuesday, 23 April 2024 at 07:38:25 UTC, cc wrote:

 On Sunday, 21 April 2024 at 19:28:11 UTC, Steven Schveighoffer 
 wrote:

 If they don't get a printout, they post an angry/confused 
 message on the forums saying




 If a user writes a destructor, and that destructor is never 
 called sometime after a `new`, then yes, I think "Why isn't the 
 GC working?" is a valid and fair question for that user to ask. 
  I don't see any way in which that expectation should be 
 considered *unreasonable*. If the answer is "No you don't 
 understand, the GC internals such that this- that- and the 
 other thing- means you can't have guaranteed destructors", then 
 I'd say you've failed to make the sale and the customer is 
 right to walk away.


This is a property of ALL GCs, even Java does not guarantee 
finalizers will run.

I agree it's reasonable to ask such a question, and we get them 
all the time. The thing is, even if we do something as drastic as 
not scan live stacks for roots, you still may end up not seeing 
the destructor run. This is a fact of life for GC, and it's 
better to learn that fact than to try and fight it.

Typically, this is not a question of "why isn't my stuff getting 
cleaned up", it's more that they are trying to poke the GC to see 
if it's working (usually a small test program). I've done it 
too...

And ironically, collecting extra "live" blocks actually hides 
from people the true nature of GC and finalization. In truth, you 
should never *rely* on the GC to clean up anything but memory. 
Resources other than memory should be cleaned up synchronously 
when you are done with them, as they are far more rare than 
memory.


 As an aside, I would observe if one wants them to stay, one 
 probably shouldn't be in the habit of calling them "the 
 unwashed masses", either.


Sorry, it's just my dry humor. It was more intended as 
(good-natured) satire on Walter (I imagine him scoffing at the 
insolent nothings that complain about his perfect language) than 
an actual slight on the people asking.

-Steve
Apr 23
parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Tuesday, 23 April 2024 at 14:24:14 UTC, Steven Schveighoffer 
wrote:

 This is a property of ALL GCs, even Java does not guarantee 
 finalizers will run.
 -Steve


That's why it is recommended to use try-with-resorces statement 
to manage non-memory resources such as http connections opened 
files and etc. in java

For D code it would make sense to have such resources, reference 
counted.
Apr 23