www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - OK to do bit-packing with GC pointers?

reply Ben Jones <fake fake.fake> writes:
I'm looking to store a pointer to one of 2 unrelated (no 
inheritance relationship) classes and use the LSb to track which 
type I have.  Is this going to cause any problems with the GC?  
For one of the classes I'll have a "pointer" to 1 byte past the 
start of the object.  It seems like std.bitmanip.taggedClassRep 
does something similar, so I assume it's OK, but wanted to double 
check.

Here's the type:

```
struct EitherClass(C1, C2) if (is(C1 == class) && is(C2 == 
class)){


     private size_t data;

public:
      safe:
      nogc:
     nothrow:

     this(C1 c1)  trusted {
         data = cast(size_t)(cast(void*)(c1));
     }

     this(C2 c2)  trusted {
         data = cast(size_t)(cast(void*)(c2));
         data |= 1;
     }

     typeof(this) opAssign(C1 c1)  trusted {
         data = cast(size_t)(cast(void*)(c1));
         return this;
     }

     typeof(this) opAssign(C2 c2)  trusted {
         data = cast(size_t)(cast(void*)(c2));
         data |= 1;
         return this;
     }

     bool holds(C)() const if(is(C == C1) || is(C == C2)){

         static if(is(C == C1)){
             return (data & 1) == 0;
         } else {
             return (data & 1) != 0;
         }
     }

     auto get(C)() const  trusted if(is(C == C1) || is(C == C2)) {
         static if(is(C == C1)){
             assert(holds!C1);
             return cast(C1)(cast(void*)(data));
         } else {
             assert(holds!C2);
             return cast(C2)(cast(void*)(data & ~(1UL)));
         }
     }

}

```
Jul 22 2022
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/22/22 12:50 PM, Ben Jones wrote:

 I'm looking to store a pointer to one of 2 unrelated (no inheritance 
 relationship) classes and use the LSb to track which type I have.  Is 
 this going to cause any problems with the GC? For one of the classes 
 I'll have a "pointer" to 1 byte past the start of the object.  It seems 
 like std.bitmanip.taggedClassRep does something similar, so I assume 
 it's OK, but wanted to double check.


It's specifically undefined behavior by the spec, but in practice, I 
think it will work, as long as the block you have isn't marked as not 
allowing interior pointers.

See: https://dlang.org/spec/garbage.html#pointers_and_gc

Specifically "Do not take advantage of alignment of pointers to store 
bit flags in the low order bits"

-Steve
Jul 22 2022
parent reply Ben Jones <fake fake.fake> writes:
On Friday, 22 July 2022 at 16:57:21 UTC, Steven Schveighoffer 
wrote:

 It's specifically undefined behavior by the spec, but in 
 practice, I think it will work, as long as the block you have 
 isn't marked as not allowing interior pointers.

 See: https://dlang.org/spec/garbage.html#pointers_and_gc

 Specifically "Do not take advantage of alignment of pointers to 
 store bit flags in the low order bits"

 -Steve


Can you elaborate on why it's probably OK in practice?

I guess the alternative to is to to make them structs instead of 
classes and manually alloc/free them (or keep them as classes, 
but still avoid the GC)?

Seems like std.bitmanip.taggedClassRef should have a big warning 
on it, right?

Thanks
Jul 22 2022
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 7/22/22 2:34 PM, Ben Jones wrote:


 
 Can you elaborate on why it's probably OK in practice?


Because the GC deals with interior pointers just fine. Blocks with the 
"no interior" bit set are very rare, and for only specialized use, so 
normally this should not be a problem.

I have argued in the past that the spec is effectively impotent on this, 
since you can easily construct an equivalent pointer without bit 
manipulation, and the GC *must* handle this case.

```d
ubyte *ptr = cast(ubyte *)intptr;
ptr += lowbits;
```


 
 I guess the alternative to is to to make them structs instead of classes 
 and manually alloc/free them (or keep them as classes, but still avoid 
 the GC)?


The alternative is to store the bits in a separate piece of memory than 
the poitner.


 Seems like std.bitmanip.taggedClassRef should have a big warning on it, 
 right?


Probably. Though like I said, I doubt it matters. Maybe someone with 
more type theory or GC knowledge knows whether it should be OK or not.

-Steve
Jul 22 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 23 July 2022 at 00:55:14 UTC, Steven Schveighoffer 
wrote:

 Probably. Though like I said, I doubt it matters. Maybe someone 
 with more type theory or GC knowledge knows whether it should 
 be OK or not.


Has nothing to do with type theory, only about GC implementation. 
But his object has no pointer in it so it should be allocated in 
a "no scan" heap, that can't work.

Also `char*` can't work as char cannot contain pointers. I guess 
you would need to use `void*`.

But you need to understand the internals of the GC implementation 
to do stuff like this.
Jul 23 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 23 July 2022 at 08:32:12 UTC, Ola Fosheim Grøstad 
wrote:

 Also `char*` can't work as char cannot contain pointers. I 
 guess you would need to use `void*`.


Well, that is wrong for the standard collection where the typing 
is dynamic (based on allocation not on type system). Then any 
pointer should work as long as it stays within the boundary of 
the allocated object.
Jul 23 2022
prev sibling next sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Jul 22, 2022 at 04:50:44PM +0000, Ben Jones via Digitalmars-d-learn
wrote:

 I'm looking to store a pointer to one of 2 unrelated (no inheritance
 relationship) classes and use the LSb to track which type I have.  Is
 this going to cause any problems with the GC?  For one of the classes
 I'll have a "pointer" to 1 byte past the start of the object.  It
 seems like std.bitmanip.taggedClassRep does something similar, so I
 assume it's OK, but wanted to double check.


https://dlang.org/spec/garbage.html#pointers_and_gc

Section 28.3, paragraph 3 "Undefined Behaviour", 4th item:

	Do not take advantage of alignment of pointers to store bit
	flags in the low order bits:

	p = cast(void*)(cast(int)p | 1);  // error: undefined behavior


T

-- 
Never step over a puddle, always step around it. Chances are that whatever made
it is still dripping.
Jul 22 2022
prev sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 7/22/22 09:50, Ben Jones wrote:

 any problems with the GC?


The slides don't seem to mention the GC but Amaury Séchet had given a 
presentation on bit packing:

   http://dconf.org/2016/talks/sechet.html

Ali
Aug 09 2022