www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - GC attribute modifications use case?

reply Steven Schveighoffer <schveiguy gmail.com> writes:
A general request for information here.

The D GC is in the unfortunate state of having internal 
attributes modifiable at any time by user code.

For example, you can set or clear the `NO_SCAN` bit. Or add a 
`FINALIZE` bit. But not properly store the correct data for 
finalization.

In the new GC, we have some restrictions on this. For example, we 
put all scannable memory in a completely separate area from 
non-scannable allocations. This means we can save some time by 
checking only if allocations in the scannable region are 
allocated, instead of having to check all memory to see if it's 
scannable.

But obviously, this means setting the bit is not feasible -- it 
would require reallocating.

I question completely the utility of this mechanism -- setting 
attributes after the fact.

There is in fact one bit which does need to be modified, and 
that's when you add a finalizer to a block (or at least, add the 
bit to say finalize the block). But I think this is better left 
to a dedicated function. Oh, and the new GC already supports this 
mechanism.

So does anyone have any good use cases for setting arbitrary GC 
attributes after allocation?

For reference, here are the list of attributes:

```d
enum BlkAttr : uint
     {
         NONE        = 0b0000_0000, /// No attributes set.
         FINALIZE    = 0b0000_0001, /// Finalize the data in this 
block on collect.
         NO_SCAN     = 0b0000_0010, /// Do not scan through this 
block on collect.
         NO_MOVE     = 0b0000_0100, /// Do not move this memory 
block on collect.
         /**
         This block contains the info to allow appending.

         This can be used to manually allocate arrays. Initial 
slice size is 0.

         Note: The slice's usable size will not match the block 
size. Use
         $(REF1 capacity, object) to retrieve actual usable 
capacity.

         Example:
         ----
         // Allocate the underlying array.
         int*  pToArray = cast(int*)GC.malloc(10 * int.sizeof, 
GC.BlkAttr.NO_SCAN | GC.BlkAttr.APPENDABLE);
         // Bind a slice. Check the slice has capacity information.
         int[] slice = pToArray[0 .. 0];
         assert(capacity(slice) > 0);
         // Appending to the slice will not relocate it.
         slice.length = 5;
         slice ~= 1;
         assert(slice.ptr == p);
         ----
         */
         APPENDABLE  = 0b0000_1000,

         /**
         This block is guaranteed to have a pointer to its base 
while it is
         alive.  Interior pointers can be safely ignored.  This 
attribute is
         useful for eliminating false pointers in very large data 
structures
         and is only implemented for data structures at least a 
page in size.
         */
         NO_INTERIOR = 0b0001_0000,

         STRUCTFINAL = 0b0010_0000, // the block has a finalizer 
for (an array of) structs
     }
```

The reason I'm asking is that I am reorganizing the GC API a bit, 
and one unittest is explicitly testing that setting/clearing the 
`NO_SCAN` bit is working. I want to disable/remove this unittest.

And of course, as a bit of irony, this unittest was added by me 
back in 2014. I am my own worst enemy...

-Steve
Dec 21
next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On Sunday, 22 December 2024 at 04:05:50 UTC, Steven Schveighoffer 
wrote:
 A general request for information here.

 The D GC is in the unfortunate state of having internal 
 attributes modifiable at any time by user code.

 [...]
I’m doing the same in my allocator. Simply assuming that NO_SCAN is unsettable
 But obviously, this means setting the bit is not feasible -- it 
 would require reallocating.

 [...]
Dec 22
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Sunday, 22 December 2024 at 04:05:50 UTC, Steven Schveighoffer 
wrote:

 So does anyone have any good use cases for setting arbitrary GC 
 attributes after allocation?
Rikki pointed out a nice search on github to find all D code that calls `setAttr`. 90% of the use of `setAttr` is either garbage (setting the attribute to something that is already set), or setting `NO_MOVE`, which is currently a no-op (supposedly a "future" GC might be able to move allocations, but this is actually impossible in D). There are some intriguing uses of `setAttr` for `NO_SCAN`, like a weak-reference object, which sets the attr of `this` in the constructor. Really, the correct way to do this is to pass in the right attr on allocation, but D hasn't traditionally made this very easy to do. However, I've decided to support the existing unittest for now, and address this a separate way later. Will expand on this in the future... -Steve
Dec 22