digitalmars.D.learn - Linker issues with struct postblit
- Thomas Gregory (124/127) Oct 27 2021 I am a maintainer of the
- Imperatorn (3/8) Oct 29 2021 Postblit?
- Stanislav Blinov (2/11) Oct 29 2021 https://dlang.org/spec/struct.html#struct-postblit
- James Blachly (9/21) Oct 30 2021 I imagine Imperatorn is quite familiar with postblit and was pointing
- Thomas Gregory (22/45) Nov 01 2021 Yes, one solution would be to add a postblit.
I am a maintainer of the [dhtslib](https://github.com/blachlylab/dhtslib) package and I have been running into issues with a new implementation of reference counting we are using. Below is the implementation (which is basically our replacement for `RefCounted`). ```d /// Template struct that wraps an htslib /// pointer and reference counts it and then /// destroys with destroyFun when it goes /// truly out of scope struct SafeHtslibPtr(T, alias destroyFun) if(!isPointer!T && isSomeFunction!destroyFun) { safe nogc nothrow: /// data pointer T * ptr; /// reference counting shared int* refct; /// initialized? bool initialized; /// ctor that respects scope this(T * rawPtr) trusted return scope { this.ptr = rawPtr; this.refct = cast(shared int *) calloc(int.sizeof,1); (*this.refct) = 1; this.initialized = true; } /// postblit that respects scope this(this) trusted return scope { if(initialized)atomicOp!"+="(*this.refct, 1); } /// allow SafeHtslibPtr to be used as /// underlying ptr type alias getRef this; /// get underlying data pointer property nothrow pure nogc ref inout(T*) getRef() inout return { return ptr; } /// take ownership of underlying data pointer property nothrow pure nogc T* moveRef() { T * ptr; move(this.getRef, ptr); return ptr; } /// dtor that respects scope ~this() trusted return scope { if(!this.initialized) return; if(atomicOp!"-="(*this.refct, 1)) return; if(this.ptr){ free(cast(int*)this.refct); /// if destroy function return is void /// just destroy /// else if return is int /// destroy then check return value /// else don't compile static if(is(ReturnType!destroyFun == void)) destroyFun(this.ptr); else static if(is(ReturnType!destroyFun == int)) { auto err = destroyFun(this.ptr); if(err != 0) hts_log_errorNoGC!__FUNCTION__("Couldn't destroy/close "~T.stringof~" * data using function "~__traits(identifier, destroyFun)); }else{ static assert(0, "HtslibPtr doesn't recognize destroy function return type"); } } } } ``` This can be used as such to reference count a pointer created from the c library [htslib](https://github.com/samtools/htslib) as such: ```d /// bam1_t is a struct from c bindings /// bam_destroy1 is a function to clean up a bam1_t * /// that is created from the c bindings alias Bam1 = SafeHtslibPtr!(bam1_t, bam_destroy1); auto b = Bam1(bam_init1()); ``` The issue presents with `SAMRecord`: ```d struct SAMRecord { /// Backing SAM/BAM row record Bam1 b; /// Corresponding SAM/BAM header data SAMHeader h; ``` dhtslib itself builds fine on both dmd and ldc compilers but when it is used as a dependency it seems to have issues building on any compiler that is not ldc > v1.24.0: ``` _D39TypeInfo_S7dhtslib3sam6record9SAMRecord6__initZ: error: undefined reference to `_D7dhtslib3sam6record9SAMRecord15__fieldPostblitMFNbNiNlNeZv' ``` Though I only experience this when trying to create an array of `SAMRecord`s. One solution I have found is using `std.array.Appender` instead of arrays. Another solution I have found is to define an explicit postblit for `SAMReader`: ``` this(this) { this.h = h; this.b = b; } ``` Looking through ldc changelogs, the closest thing I could attribute this to is this change for ldc-1.25.0:- Struct TypeInfos are emitted into referencing object files only, and special TypeInfo member functions into the owningI suspect this is something to do with the alias'd function in `SafeHtslibPtr`. Is there something I should be doing differently?
Oct 27 2021
On Thursday, 28 October 2021 at 01:39:10 UTC, Thomas Gregory wrote:I am a maintainer of the [dhtslib](https://github.com/blachlylab/dhtslib) package and I have been running into issues with a new implementation of reference counting we are using. [...]Postblit?
Oct 29 2021
On Friday, 29 October 2021 at 11:05:14 UTC, Imperatorn wrote:On Thursday, 28 October 2021 at 01:39:10 UTC, Thomas Gregory wrote:https://dlang.org/spec/struct.html#struct-postblitI am a maintainer of the [dhtslib](https://github.com/blachlylab/dhtslib) package and I have been running into issues with a new implementation of reference counting we are using. [...]Postblit?
Oct 29 2021
On 10/29/21 7:10 AM, Stanislav Blinov wrote:On Friday, 29 October 2021 at 11:05:14 UTC, Imperatorn wrote:I imagine Imperatorn is quite familiar with postblit and was pointing out that it is strange to use postblit constructor, which is deprecated, in a "new implementation". The original post was long and really buried the lede, but OP was pointing out what looks like a compiler bug: a linker error referencing a postblit symbol only shows up when compiling with DMD or with LDC earlier than 1.25. Can someone give some insight?On Thursday, 28 October 2021 at 01:39:10 UTC, Thomas Gregory wrote:https://dlang.org/spec/struct.html#struct-postblitI am a maintainer of the [dhtslib](https://github.com/blachlylab/dhtslib) package and I have been running into issues with a new implementation of reference counting we are using. [...]Postblit?
Oct 30 2021
On Sunday, 31 October 2021 at 03:51:49 UTC, James Blachly wrote:On 10/29/21 7:10 AM, Stanislav Blinov wrote:I apologize for the long-winded question.On Friday, 29 October 2021 at 11:05:14 UTC, Imperatorn wrote:I imagine Imperatorn is quite familiar with postblit and was pointing out that it is strange to use postblit constructor, which is deprecated, in a "new implementation". The original post was long and really buried the lede, but OP was pointing out what looks like a compiler bug: a linker error referencing a postblit symbol only shows up when compiling with DMD or with LDC earlier than 1.25. Can someone give some insight?On Thursday, 28 October 2021 at 01:39:10 UTC, Thomas Gregory wrote:https://dlang.org/spec/struct.html#struct-postblitI am a maintainer of the [dhtslib](https://github.com/blachlylab/dhtslib) package and I have been running into issues with a new implementation of reference counting we are using. [...]Postblit?Yes, one solution would be to add a postblit. However, the compiler should be able to generate default postblit and it does. I am only running into issues linking, which leads me to believe this is a compiler or language bug (fixed as of ldc-1.25). Is there some way to to avoid writing an explicit postblit for every struct that uses `SafeHtslibPtr`? Many of `dhtslib`'s structs wrap c type pointers that are reference counted and owned by `SafeHtslibPtr` like so: ```D // wrappedCPtr is an alias defined in another module as: alias wrappedCPtr = SafeHtslibPtr!(c_type, destroy_c_type_fun); // It is imported here struct WrapperTypeName { /// Backing C type pointer wrappedCPtr b; ... } ```Postblit?
Nov 01 2021