digitalmars.D.learn - Having a strange issue with std.net.curl.HTTP as a struct dependency
- NoBigDeal256 (49/49) Jul 09 2017 I'm currently learning D and started working on one of my first
- NoBigDeal256 (2/2) Jul 10 2017 Still haven't figured out what's wrong. Any help would be
- ketmar (3/7) Jul 10 2017 known bug in curl finalization on program exit. should be fixed in the n...
- NoBigDeal256 (6/15) Jul 10 2017 Ah. It's interesting that it works perfectly fine when used
- ketmar (2/6) Jul 10 2017 nope. it is a kind of "known bug nobody bothered to report". ;-)
- NoBigDeal256 (4/9) Jul 12 2017 Okay, so I decided to ditch curl and switch over to
- Adam D. Ruppe (21/24) Jul 12 2017 This tells me the problem is in the collection order at the end
- crimaniak (3/8) Jul 12 2017 I'd try also calling GC.collect() manually before the exit.
- =?UTF-8?Q?Ali_=c3=87ehreli?= (4/5) Jul 13 2017 For what it's worth, defining it as __gshared seems to be a workaround:
- =?UTF-8?Q?Ali_=c3=87ehreli?= (11/15) Jul 13 2017 I sometimes amuse myself. What use is that? :o)
I'm currently learning D and started working on one of my first
projects which is an API wrapper. I'm currently having an issue
with my program getting a InvalidMemoryOperationError upon
exiting the process on Windows 7. On my Debian VM I get a
segmentation fault.
I have tried to minimize the code as much as I possibly can while
still reproducing the error. Here is the code:
import std.net.curl;
struct ThingA {
HTTP http;
this(HTTP http) {
this.http = http;
arrayOfThingBs();
}
ThingB[] arrayOfThingBs() {
ThingB[] thingBs;
thingBs ~= ThingB(this);
return thingBs;
}
}
struct ThingB {
ThingA thingA;
this(ThingA thingA) {
this.thingA = thingA;
}
}
void main() {
auto http = HTTP();
auto thingA = ThingA(http);
}
If I comment out the HTTP dependency like:
struct ThingA {
//HTTP http;
this(HTTP http) {
//this.http = http;
arrayOfThingBs();
}
ThingB[] arrayOfThingBs() {
ThingB[] thingBs;
thingBs ~= ThingB(this);
return thingBs;
}
}
The error goes away. The error also goes away if
ThingA.arrayOfThingBs returns a single instance of ThingB instead
of an array of ThingB. Removing ThingBs dependency on ThingA also
gets rid of the error. I'm new to low level languages in general
so maybe I'm doing something wrong, but this seems like really
strange behavior.
Jul 09 2017
NoBigDeal256 <nobigdeal256 gmail.com> Still haven't figured out what's wrong. Any help would be appreciated.
Jul 10 2017
NoBigDeal256 wrote:I'm currently learning D and started working on one of my first projects which is an API wrapper. I'm currently having an issue with my program getting a InvalidMemoryOperationError upon exiting the process on Windows 7. On my Debian VM I get a segmentation fault.known bug in curl finalization on program exit. should be fixed in the next release. for now you have to live with it, or build your own phobos.
Jul 10 2017
On Monday, 10 July 2017 at 20:31:12 UTC, ketmar wrote:NoBigDeal256 wrote:Ah. It's interesting that it works perfectly fine when used directly in main, and only becomes an issue when it's a struct member. Do you happen to have a link to the bug report for this specific issue that I could look at? I looked at the known bugs list and couldn't find anything related to this.I'm currently learning D and started working on one of my first projects which is an API wrapper. I'm currently having an issue with my program getting a InvalidMemoryOperationError upon exiting the process on Windows 7. On my Debian VM I get a segmentation fault.known bug in curl finalization on program exit. should be fixed in the next release. for now you have to live with it, or build your own phobos.
Jul 10 2017
NoBigDeal256 wrote:Do you happen to have a link to the bug report for this specific issue that I could look at? I looked at the known bugs list and couldn't find anything related to this.nope. it is a kind of "known bug nobody bothered to report". ;-)
Jul 10 2017
On Monday, 10 July 2017 at 20:55:19 UTC, ketmar wrote:NoBigDeal256 wrote:Okay, so I decided to ditch curl and switch over to dlang-requests, which doesn't depend on curl, and I'm running into this same issue again... I don't think this is a curl issue.Do you happen to have a link to the bug report for this specific issue that I could look at? I looked at the known bugs list and couldn't find anything related to this.nope. it is a kind of "known bug nobody bothered to report". ;-)
Jul 12 2017
On Wednesday, 12 July 2017 at 08:24:12 UTC, NoBigDeal256 wrote:On Monday, 10 July 2017 at 20:55:19 UTC, ketmar wrote:looks like circular dependency between thingA and thingB lead to problems on exit. this code fails on osx too.NoBigDeal256 wrote:Okay, so I decided to ditch curl and switch over to dlang-requests, which doesn't depend on curl, and I'm running into this same issue again... I don't think this is a curl issue.Do you happen to have a link to the bug report for this specific issue that I could look at? I looked at the known bugs list and couldn't find anything related to this.nope. it is a kind of "known bug nobody bothered to report". ;-)
Jul 12 2017
ikod wrote:On Wednesday, 12 July 2017 at 08:24:12 UTC, NoBigDeal256 wrote:yet at least with curl there was definitely a bug with "slightly too early curl shutdown".On Monday, 10 July 2017 at 20:55:19 UTC, ketmar wrote:looks like circular dependency between thingA and thingB lead to problems on exit. this code fails on osx too.NoBigDeal256 wrote:Okay, so I decided to ditch curl and switch over to dlang-requests, which doesn't depend on curl, and I'm running into this same issue again... I don't think this is a curl issue.Do you happen to have a link to the bug report for this specific issue that I could look at? I looked at the known bugs list and couldn't find anything related to this.nope. it is a kind of "known bug nobody bothered to report". ;-)
Jul 12 2017
On Monday, 10 July 2017 at 00:10:36 UTC, NoBigDeal256 wrote:The error goes away. The error also goes away if ThingA.arrayOfThingBs returns a single instance of ThingB instead of an array of ThingB.This tells me the problem is in the collection order at the end of the program. The HTTP inside the struct inside the array gets cleaned up by the garbage collector (aka GC) at program termination... and it might be doing that AFTER unloading curl, leading to the segfault, OR it could be running some GC operation in the destructor, leading to InvalidMemoryOperationError (calling a GC function while the GC is running is not allowed). Which you get is apparently dependent on which system you are on, but both are wrong. If it isn't in the array, the destructor gets called at scope exit, which happens before program termination and outside the GC, so everything happens as it is supposed to. HTTP's innards have a RefCounted implementation which has a destructor that calls GC.removeRange. I *think* that can do the invalid mem operation. And the implementation itself can do curl_shutdown, which is your potential segfault. So I'd say the answer is prolly to keep HTTP away from the GC. Manually free its arrays, or keep them outside arrays in the first place.
Jul 12 2017
On Wednesday, 12 July 2017 at 13:47:06 UTC, Adam D. Ruppe wrote:This tells me the problem is in the collection order at the end of the program....So I'd say the answer is prolly to keep HTTP away from the GC. Manually free its arrays, or keep them outside arrays in the first place.I'd try also calling GC.collect() manually before the exit.
Jul 12 2017
On 07/09/2017 05:10 PM, NoBigDeal256 wrote:
HTTP http;
For what it's worth, defining it as __gshared seems to be a workaround:
__gshared HTTP http;
Ali
Jul 13 2017
On 07/13/2017 12:01 AM, Ali Çehreli wrote:On 07/09/2017 05:10 PM, NoBigDeal256 wrote:I sometimes amuse myself. What use is that? :o) What I must have meant is you can have a __gshared array of those and pass pointers to elements to constructors: __gshared HTTP[] https; struct ThingA { HTTP *http; this(HTTP *http) { // ... Not easy but a workaround... AliHTTP http;For what it's worth, defining it as __gshared seems to be a workaround: __gshared HTTP http;
Jul 13 2017