www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Understanding SIGSEGV issues

reply Russel Winder <russel winder.org.uk> writes:
So I have a D program that used to work. I come back to it, recompile it, a=
nd:

|> dub run -- ~/lib/DigitalTelevision/DVBv5/uk-CrystalPalace__RW=20
Performing "debug" build using /usr/bin/ldc2 for x86_64.
libdvbv5_d 0.1.1: target for configuration "library" is up to date.
dvb-tune ~master: target for configuration "application" is up to date.
To force a rebuild of up-to-date targets, run again with --force.
Running ./bin/dvb-tune /home/users/russel/lib/DigitalTelevision/DVBv5/uk-Cr=
ystalPalace__RW
Device: Silicon Labs Si2168, adapter  0, frontend  0
Program exited with code -11

(gdb) r ~/lib/DigitalTelevision/DVBv5/uk-CrystalPalace__RW
Starting program: /home/users/russel/Repositories/Git/Masters/Public/DVBTun=
e/bin/dvb-tune ~/lib/DigitalTelevision/DVBv5/uk-CrystalPalace__RW
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Device: Silicon Labs Si2168, adapter  0, frontend  0

Program received signal SIGSEGV, Segmentation fault.
__GI___libc_free (mem=3D0xa) at malloc.c:3093
3093	malloc.c: No such file or directory.
(gdb)=20

Can anyone give me any hints as to where to start even getting a glimmer of=
 an
understanding of WTF is going on?

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder      t: +44 20 7585 2200
41 Buckmaster Road    m: +44 7770 465 077
London SW11 1EN, UK   w: www.russel.org.uk
Jan 02
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 3 January 2019 at 06:25:46 UTC, Russel Winder wrote:
 So I have a D program that used to work. I come back to it, 
 recompile it, and:

 [...]
 __GI___libc_free (mem=0xa) at malloc.c:3093
You've tried to free a pointer that, while not null, was derived from a pointer that was, i.e. an offset to a field of a struct. A backtrace would help a lot, otherwise it really is just guessing.
Jan 02
parent reply Russel Winder <russel winder.org.uk> writes:
On Thu, 2019-01-03 at 07:52 +0000, Nicholas Wilson via Digitalmars-d-learn
wrote:
 On Thursday, 3 January 2019 at 06:25:46 UTC, Russel Winder wrote:
 So I have a D program that used to work. I come back to it,=20
 recompile it, and:
=20
 [...]
 __GI___libc_free (mem=3D0xa) at malloc.c:3093
=20 You've tried to free a pointer that, while not null, was derived=20 from a pointer that was, i.e. an offset to a field of a struct. =20 A backtrace would help a lot, otherwise it really is just=20 guessing.
Sorry about that, fairly obvious that the backtrace is needed in hindsight.= :- ) #0 __GI___libc_free (mem=3D0xa) at malloc.c:3093 #1 0x000055555558f174 in dvb_file_free (dvb_file=3D0x5555555a1320) at dvb_= file.d:282 #2 0x000055555558edcc in types.File_Ptr.~this() (this=3D...) at types.d:83 #3 0x0000555555574809 in channels.TransmitterData.__fieldDtor() (this=3D<e= rror reading variable: Cannot access memory at address 0xa>) at channels.d:= 144 #4 0x000055555556aeda in channels.TransmitterData.__aggrDtor() (this=3D...= ) at channels.d:144 #5 0x000055555556ab53 in D main (args=3D...) at main.d:33 Which indicates that the destructor is being called before the instance has been constructed. Which is a real WTF. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 03
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Thursday, 3 January 2019 at 08:35:17 UTC, Russel Winder wrote:
 Sorry about that, fairly obvious that the backtrace is needed 
 in hindsight. :- )

 #0  __GI___libc_free (mem=0xa) at malloc.c:3093
 #1  0x000055555558f174 in dvb_file_free 
 (dvb_file=0x5555555a1320) at dvb_file.d:282
 #2  0x000055555558edcc in types.File_Ptr.~this() (this=...) at 
 types.d:83
 #3  0x0000555555574809 in 
 channels.TransmitterData.__fieldDtor() (this=<error reading 
 variable: Cannot access memory at address 0xa>) at 
 channels.d:144
 #4  0x000055555556aeda in channels.TransmitterData.__aggrDtor() 
 (this=...) at channels.d:144
 #5  0x000055555556ab53 in D main (args=...) at main.d:33

 Which indicates that the destructor is being called before the 
 instance has been constructed. Which is a real WTF.
Not quite, this occurs as a TransmitterData object goes out of scope at the end of main(stick a fflush'ed printf there to see): TransmitterData is a struct that has no destructor defined but has a field of type File_Ptr that does. The compiler generates a destructor, __aggrDtor, which calls the fields that have destructors, __fieldDtor (e.g. the File_Ptr) which in turn calls its destructor, File_Ptr.~this(). As you can see from the stack trace #3, the File_Ptr is null. The solution to this is to either ensure it is initialised in the constructor of TransmitterData, or account for it possibly being null by defining a destructor for TransmitterData.
Jan 03
parent reply Russel Winder <russel winder.org.uk> writes:
On Thu, 2019-01-03 at 11:23 +0000, Nicholas Wilson via Digitalmars-d-learn
wrote:
 On Thursday, 3 January 2019 at 08:35:17 UTC, Russel Winder wrote:
 Sorry about that, fairly obvious that the backtrace is needed=20
 in hindsight. :- )
=20
 #0  __GI___libc_free (mem=3D0xa) at malloc.c:3093
 #1  0x000055555558f174 in dvb_file_free=20
 (dvb_file=3D0x5555555a1320) at dvb_file.d:282
 #2  0x000055555558edcc in types.File_Ptr.~this() (this=3D...) at=20
 types.d:83
 #3  0x0000555555574809 in=20
 channels.TransmitterData.__fieldDtor() (this=3D<error reading=20
 variable: Cannot access memory at address 0xa>) at=20
 channels.d:144
 #4  0x000055555556aeda in channels.TransmitterData.__aggrDtor()=20
 (this=3D...) at channels.d:144
 #5  0x000055555556ab53 in D main (args=3D...) at main.d:33
=20
 Which indicates that the destructor is being called before the=20
 instance has been constructed. Which is a real WTF.
=20 Not quite, this occurs as a TransmitterData object goes out of=20 scope at the end of main(stick a fflush'ed printf there to see):
I am not sure this analysis is correct. The code never reaches the end of main.=20
 TransmitterData is a struct that has no destructor defined but=20
 has a field of type File_Ptr that does. The compiler generates a=20
 destructor, __aggrDtor, which calls the fields that have=20
 destructors, __fieldDtor (e.g. the File_Ptr) which in turn calls=20
 its destructor, File_Ptr.~this().
TransmitterData has a destructor defined but with no code in it. This used = to work fine =E2=80=93 but I cannot be certain which version of LDC that was. The problem does seem to be in the construction of the TransmitterData obje= ct because a destructor is being called on the File_Ptr field as part of the transmitterData constructor.
 As you can see from the stack trace #3, the File_Ptr is null. The=20
 solution to this is to either ensure it is initialised in the=20
 constructor of TransmitterData, or account for it possibly being=20
 null by defining a destructor for TransmitterData.
For some reason it seems File_Ptr.~this() is being called before File_Ptr.this() in the TransmitterData.this(). This is totally weird. Having added some writeln statements: (gdb) bt #0 0x00005555555932e0 in dvb_file_free (dvb_file=3D0x0) at dvb_file.d:276 #1 0x0000555555592fbc in types.File_Ptr.~this() (this=3D...) at types.d:83 #2 0x000055555558cdf6 in _D3std6format__T14formattedWriteTSQBg5stdio4File1= 7LockingTextWriterTaTS5types8File_PtrZQCtFKQChxAaQBcZk (w=3D..., fmt=3D...,= _param_2=3D...) at /usr/lib/ldc/x86_64-linux-gnu/include/d/std/format.d:47= 2 #3 0x000055555558c6a8 in _D3std5stdio4File__T5writeTAyaTS5types8File_PtrTa= ZQBeMFQBcQBbaZv (this=3D..., _param_0=3D..., _param_1=3D..., _param_2=3D10 = '\n') at channels.d:1586 #4 0x00005555555749ce in _D3std5stdio__T7writelnTAyaTS5types8File_PtrZQBeF= QzQxZv (_param_0=3D..., _param_1=3D...) at channels.d:3917 #5 0x000055555556af49 in _D8channels15TransmitterData6__ctorMFNcxAyaxkxE10= libdvbv5_d8dvb_file16dvb_file_formatsZSQDiQDc (this=3D..., path=3D..., dels= ys=3D0, format=3Dlibdvbv5_d.dvb_file.dvb_file_formats.FILE_DVBV5) at channe= ls.d:143 #6 0x000055555556aa9c in D main (args=3D...) at main.d:34 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 04
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 5 January 2019 at 07:34:17 UTC, Russel Winder wrote:
 TransmitterData has a destructor defined but with no code in 
 it. This used to work fine – but I cannot be certain which 
 version of LDC that was.

 The problem does seem to be in the construction of the 
 TransmitterData object because a destructor is being called on 
 the File_Ptr field as part of the transmitterData constructor.

 As you can see from the stack trace #3, the File_Ptr is null. 
 The solution to this is to either ensure it is initialised in 
 the constructor of TransmitterData, or account for it possibly 
 being null by defining a destructor for TransmitterData.
For some reason it seems File_Ptr.~this() is being called before File_Ptr.this() in the TransmitterData.this(). This is totally weird. Having added some writeln statements: (gdb) bt #0 0x00005555555932e0 in dvb_file_free (dvb_file=0x0) at dvb_file.d:276 #1 0x0000555555592fbc in types.File_Ptr.~this() (this=...) at types.d:83 #2 0x000055555558cdf6 in _D3std6format__T14formattedWriteTSQBg5stdio4File17LockingTextWriterTaTS5types8File_ trZQCtFKQChxAaQBcZk (w=..., fmt=..., _param_2=...) at /usr/lib/ldc/x86_64-linux-gnu/include/d/std/format.d:472
Maybe it is a problem with copying a File_Ptr (e.g. missing a increase of the reference count)? Like, `auto a = File_Ptr(); { auto b = a; }` and b calls the destructor on scope exit. That would be consistent with having problems copying to object to pass to writeln.
Jan 05
next sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2019-01-05 at 10:31 +0000, Nicholas Wilson via Digitalmars-d-learn
wrote:
[=E2=80=A6]
=20
 Maybe it is a problem with copying a File_Ptr (e.g. missing a=20
 increase of the reference count)? Like, `auto a =3D File_Ptr(); {=20
 auto b =3D a; }` and b calls the destructor on scope exit.
 That would be consistent with having problems copying to object=20
 to pass to writeln.
I found the problem and then two minutes later read your email and bingo we have found the problem. Previously I had used File_Ptr* and on this occasion I was using File_Ptr a= nd there was no copy constructor because I have disable this(this). Except th= at clearly copying a value is not copying a value in this case. Clearly this situation is what is causing the destructor to be called on an unconstructe= d value. But I have no idea why. The question now, of course, is should I have been using File_Ptr instead o= f File_Ptr* in the first place. I am beginning to think I should have been. M= ore thinking needed. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 05
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 5 January 2019 at 10:52:48 UTC, Russel Winder wrote:
 I found the problem and then two minutes later read your email 
 and bingo we have found the problem.
Well done.
 Previously I had used File_Ptr* and on this occasion I was 
 using File_Ptr and there was no copy constructor because I have 
  disable this(this). Except that clearly copying a value is not 
 copying a value in this case. Clearly this situation is what is 
 causing the destructor to be called on an unconstructed value. 
 But I have no idea why.
Could you post a minimised example? Its a bit hard to guess without one.
 The question now, of course, is should I have been using 
 File_Ptr instead of File_Ptr* in the first place. I am 
 beginning to think I should have been. More thinking needed.
From the name, File_Ptr sounds like it is wrapping a reference to a resource. So compare with C's FILE/ D's File which is a reference counted wrapper of a FILE*. Would you ever use a File* (or a FILE**)? Probably not, I never have.
Jan 05
parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2019-01-05 at 11:30 +0000, Nicholas Wilson via Digitalmars-d-learn
wrote:
=20
[=E2=80=A6]
 Could you post a minimised example? Its a bit hard to guess=20
 without one.
Indeed. I should do that to see if I can reproduce the problem to submit a proper bug report. [=E2=80=A6]
  From the name, File_Ptr sounds like it is wrapping a reference to=20
 a resource. So compare with C's FILE/ D's File which is a=20
 reference counted wrapper of a FILE*. Would you ever use a File*=20
 (or a FILE**)? Probably not, I never have.
File_Ptr is wrapping a dvb_file * from libdvbv5 to try and make things a bi= t for D and to ensure RAII. libdvbv5 is a C API with classic C approach to handling objects and data structures. My DStep/with manual binding is at https://github.com/russel/libdvbv5_d and the application using it (which is causing the problems) is at=20 https://github.com/russel/DVBTune I have a feeling that I am really not doing things in a D idiomatic way. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 05
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 5 January 2019 at 12:14:15 UTC, Russel Winder wrote:
 Indeed. I should do that to see if I can reproduce the problem 
 to submit a proper bug report.

 File_Ptr is wrapping a dvb_file * from libdvbv5 to try and make 
 things a bit for D and to ensure RAII. libdvbv5 is a C API with 
 classic C approach to handling objects and data structures.

 My DStep/with manual binding is at 
 https://github.com/russel/libdvbv5_d and the application using 
 it (which is causing the problems) is at 
 https://github.com/russel/DVBTune
Your problem possibly (probably?) stems from auto channelsData = TransmitterData(args[1]).scan(frontendId); The temporary TransmitterData(args[1]) is, well, temporary and its destructor runs after that expression is done. As the returned object from scan references data from the temporary, you have a stale pointer.
 I have a feeling that I am really not doing things in a D 
 idiomatic way.
Some driveby style comments then:
	bool opEquals()(const FrontendId other) const {
		if (this is other) return true;
		if (other is null) return false;
		return this.adapter_number == other.adapter_number && 
this.frontend_number == other.frontend_number;
	}
The compiler generated default opEquals will do basically the same thing. Ditto for the other types. You usually want to take a const ref for opEquals since there is no point copying it.
 if (other is null)
I'm surprised the compiler doesn't warn or error on that as the only way that could make sense would be if it had an alias this to a pointer type. You should consider reference counting your pointer wrapper types, FrontendParameters_Ptr/File_Ptr/ScanHandler_Ptr You seem to like const, good! You don't need to take `const int`s as parameters, you're getting a copy anyway. You have a bunch of redundant casts as well. I'll have another looks tomorrow when I'm a bit more awake.
Jan 05
parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2019-01-05 at 13:14 +0000, Nicholas Wilson via Digitalmars-d-learn
wrote:
=20
[=E2=80=A6]
 Your problem possibly (probably?) stems from
=20
 auto channelsData =3D TransmitterData(args[1]).scan(frontendId);
=20
 The temporary TransmitterData(args[1]) is, well, temporary and=20
 its destructor runs after that expression is done. As the=20
 returned object from scan references data from the temporary, you=20
 have a stale pointer.
Actually that is not a worry since the TransmitterData instance is only nee= ded to call the scan function which creates a ChannelsData instance that holds = no references to the TransmitterData instance. It turns out that whilst the code used to run, and now doesn't, all the thi= ngs we have been talking of are nothing to do with the core problem. It turns o= ut that the function call to initialise the File_Ptr object is returning a val= id object with invalid data. Thus the unknown change is likely in the libdvbv5 library either due to the C API doing different things or the adapter creat= ed using DStep doing the wrong thing. =20
 I have a feeling that I am really not doing things in a D=20
 idiomatic way.
=20 Some driveby style comments then: =20
 	bool opEquals()(const FrontendId other) const {
 		if (this is other) return true;
 		if (other is null) return false;
 		return this.adapter_number =3D=3D other.adapter_number &&=20
 this.frontend_number =3D=3D other.frontend_number;
 	}
=20 The compiler generated default opEquals will do basically the=20 same thing. Ditto for the other types. You usually want to take a=20 const ref for opEquals since there is no point copying it.
I deleted them, added a test or three (should have done this ages ago) and = the tests pass. So teh generated methods do the needful. Thanks for that "heads up".
 if (other is null)
=20 I'm surprised the compiler doesn't warn or error on that as the=20 only way that could make sense would be if it had an alias this=20 to a pointer type. =20 You should consider reference counting your pointer wrapper=20 types, FrontendParameters_Ptr/File_Ptr/ScanHandler_Ptr
Very true. For now I have forbidden copying, which is wrong for this sort o= f thing. If D had the equivalent of C++ std::shared_ptr or Rust std::rc::Rc o= r std::rc::Arc, that would be the way forward But I guess having explicit reference counting is not too hard.
 You seem to like const, good! You don't need to take `const int`s=20
 as parameters, you're getting a copy anyway. You have a bunch of=20
 redundant casts as well.
I am a person who always makes Java variables final, and loves that Rust variables are immutable by default! --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 08
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Tuesday, 8 January 2019 at 10:23:30 UTC, Russel Winder wrote:
 Actually that is not a worry since the TransmitterData instance 
 is only needed to call the scan function which creates a 
 ChannelsData instance that holds no references to the 
 TransmitterData instance.

 It turns out that whilst the code used to run, and now doesn't, 
 all the things
 we have been talking of are nothing to do with the core 
 problem. It turns out
 that the function call to initialise the File_Ptr object is 
 returning a valid
 object with invalid data. Thus the unknown change is likely in 
 the libdvbv5
 library either due to the C API doing different things or the 
 adapter created
 using DStep doing the wrong thing.
Ahh. Good that you've found that, I can't help you much more with that then.
 The compiler generated default opEquals will do basically the 
 same thing. Ditto for the other types. You usually want to 
 take a const ref for opEquals since there is no point copying 
 it.
I deleted them, added a test or three (should have done this ages ago) and the tests pass. So teh generated methods do the needful. Thanks for that "heads up".
No problems, less code is good code.
 Very true. For now I have forbidden copying, which is wrong for 
 this sort of thing. If D had the equivalent of C++ 
 std::shared_ptr or Rust std::rc::Rc or std::rc::Arc, that would 
 be the way forward But I guess having explicit reference 
 counting is not too hard.
I believe Andrei and Razvan are working on that, part of that being the Copy Constructor DIP. Hopefully it will arrive soon.
 You seem to like const, good! You don't need to take `const 
 int`s as parameters, you're getting a copy anyway. You have a 
 bunch of redundant casts as well.
I am a person who always makes Java variables final, and loves that Rust variables are immutable by default!
Indeed, less to think about is always nice. Good luck figuring out why your data is dud. Nic
Jan 08
parent reply Russel Winder <russel winder.org.uk> writes:
On Tue, 2019-01-08 at 11:51 +0000, Nicholas Wilson via Digitalmars-d-learn
wrote:
=20
[=E2=80=A6]
 Ahh. Good that you've found that, I can't help you much more with=20
 that then.
Indeed. :-) Your hep to get to this point though has been invaluable. Thanks you for putting in the time and effort. [=E2=80=A6]
 Good luck figuring out why your data is dud.
It really is totally weird. My new Rust binding to libdvbv5 and associated version of the same application works fine. So libdvbv5 itself is not the cuprit. This has to mean it is something about the D compilers that has changed the way the D binding to libdvbv5 behaves. If only the D plugin to CLion were much further down the road this would be much easier to fix. I had an issue in the Rust and it was fixed in a couple= of minutes because of the way CLion drives GDB for you. Using GDB manually is such a f###### pain. This alone becomes a mountain that leads to the though= t of giving up on the D version.=20 In an ideal world JetBrains would take over the D plugin, but that isn't go= ng to happen =E2=80=93 unlike what happened for Go and Rust. What the D plugin= needs is some full time workers: the great work by the current volunteers is slow progress by nature of it being volunteer effort by a few people. =20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 09
next sibling parent reply Johannes Loher <johannes.loher fg4f.de> writes:
On Wednesday, 9 January 2019 at 16:48:47 UTC, Russel Winder wrote:
 On Tue, 2019-01-08 at 11:51 +0000, Nicholas Wilson via 
 Digitalmars-d-learn wrote:
 [...]
[…]
 [...]
[...]
If debugger integration is that important to you, you might want to try out visual studio code with the corresponding plugins (you need a separate plugin for debugger support). I found it to work quite decently.
Jan 09
parent Russel Winder <russel winder.org.uk> writes:
On Wed, 2019-01-09 at 20:03 +0000, Johannes Loher via Digitalmars-d-learn
wrote:
=20
[=E2=80=A6]
 If debugger integration is that important to you, you might want=20
 to try out visual studio code with the corresponding plugins (you=20
 need a separate plugin for debugger support). I found it to work=20
 quite decently.
Or I could stop doing D and Rust programming and switch to Kotlin and Java = and get stuck into helping the IntelliJ-DLanguage team as I said I would a coup= le of years ago. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 09
prev sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 9 January 2019 at 16:48:47 UTC, Russel Winder wrote:
 It really is totally weird. My new Rust binding to libdvbv5 and 
 associated version of the same application works fine. So 
 libdvbv5 itself is not the cuprit. This has to mean it is 
 something about the D compilers that has changed the way the D 
 binding to libdvbv5 behaves.
Hmm, if you think the binding could be the problem you could try using app as an alternative, see if it makes any difference.
Jan 09
parent Russel Winder <russel winder.org.uk> writes:
On Thu, 2019-01-10 at 07:36 +0000, Nicholas Wilson via Digitalmars-d-learn
wrote:
[=E2=80=A6]
 Hmm, if you think the binding could be the problem you could try=20
 using app as an alternative, see if it makes any difference.
I did a proper update of the generated files of the binding, and magically everything works again. I do not recollect any change being significant, bu= t clearly something was. So the problem is that I had failed to notice the update of libdvbv5 with a breaking change and update the binding accordingly. Of course I got lots of other good stuff done with the code by having this problem and people such as yourself commenting. Big win all round. :-) Thanks again for helping on this. =20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 10
prev sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2019-01-05 at 10:52 +0000, Russel Winder wrote:
 On Sat, 2019-01-05 at 10:31 +0000, Nicholas Wilson via Digitalmars-d-lear=
n
 wrote:
 [=E2=80=A6]
 Maybe it is a problem with copying a File_Ptr (e.g. missing a=20
 increase of the reference count)? Like, `auto a =3D File_Ptr(); {=20
 auto b =3D a; }` and b calls the destructor on scope exit.
 That would be consistent with having problems copying to object=20
 to pass to writeln.
=20 I found the problem and then two minutes later read your email and bingo =
we
 have found the problem.
=20
 Previously I had used File_Ptr* and on this occasion I was using File_Ptr
 and
 there was no copy constructor because I have  disable this(this). Except
 that
 clearly copying a value is not copying a value in this case. Clearly this
 situation is what is causing the destructor to be called on an unconstruc=
ted
 value. But I have no idea why.
=20
 The question now, of course, is should I have been using File_Ptr instead=
of
 File_Ptr* in the first place. I am beginning to think I should have been.
 More
 thinking needed.
Switching to using File_Ptr* I now get the SIGSEGV at the end of main as yo= u were thinking before. Oh f###. This code used to work. :-( --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 05
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/5/19 6:33 AM, Russel Winder wrote:
 On Sat, 2019-01-05 at 10:52 +0000, Russel Winder wrote:
 On Sat, 2019-01-05 at 10:31 +0000, Nicholas Wilson via Digitalmars-d-learn
 wrote:
 […]
 Maybe it is a problem with copying a File_Ptr (e.g. missing a
 increase of the reference count)? Like, `auto a = File_Ptr(); {
 auto b = a; }` and b calls the destructor on scope exit.
 That would be consistent with having problems copying to object
 to pass to writeln.
I found the problem and then two minutes later read your email and bingo we have found the problem. Previously I had used File_Ptr* and on this occasion I was using File_Ptr and there was no copy constructor because I have disable this(this). Except that clearly copying a value is not copying a value in this case. Clearly this situation is what is causing the destructor to be called on an unconstructed value. But I have no idea why. The question now, of course, is should I have been using File_Ptr instead of File_Ptr* in the first place. I am beginning to think I should have been. More thinking needed.
Switching to using File_Ptr* I now get the SIGSEGV at the end of main as you were thinking before. Oh f###. This code used to work. :-(
Russel, make sure your destructor both checks whether the underlying resource is set, and clears it to invalid when freeing it. Even types that can't be copied can be moved, or temporarily created as rvalues. When they are moved the shell they get moved out of is still destructed! So it has to have a state where it can be destroyed, even though there is no resource. Maybe some inspiration here: https://github.com/MartinNowak/io/blob/master/src/std/io/file.d#L189-L196 -Steve
Jan 08
parent reply Russel Winder <russel winder.org.uk> writes:
On Tue, 2019-01-08 at 09:59 -0500, Steven Schveighoffer via Digitalmars-d-
learn wrote:
=20
[=E2=80=A6]
=20
 Russel, make sure your destructor both checks whether the underlying=20
 resource is set, and clears it to invalid when freeing it.
=20
 Even types that can't be copied can be moved, or temporarily created as=
=20
 rvalues. When they are moved the shell they get moved out of is still=20
 destructed! So it has to have a state where it can be destroyed, even=20
 though there is no resource.
I have added tests in the destructor but given the constructor should throw= an exception on a failure to initialise the internal state correctly, it reall= y ought to be unnecessary. but I guess it cant hurt being there! As I noted to Nicholas it seems the application is getting a valid data structure returned with invalid data and that is where the SIGSEGV is. This= is really weird as I have just finished a Rust version of the same application and it works fine. And this D version used to work fine. It is a real myste= ry why there is a problem now. Sadly the D plugin to CLion doesn't as yet have the same functionality as t= he Rust plugin. Debugging these sorts of thing is just so much better in CLio= n than trying to work GDB manually.
 Maybe some inspiration here:=20
 https://github.com/MartinNowak/io/blob/master/src/std/io/file.d#L189-L196
=20
I will check that out, thanks for the pointer. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 09
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/9/19 11:39 AM, Russel Winder wrote:
 On Tue, 2019-01-08 at 09:59 -0500, Steven Schveighoffer via Digitalmars-d-
 learn wrote:

 […]
 Russel, make sure your destructor both checks whether the underlying
 resource is set, and clears it to invalid when freeing it.

 Even types that can't be copied can be moved, or temporarily created as
 rvalues. When they are moved the shell they get moved out of is still
 destructed! So it has to have a state where it can be destroyed, even
 though there is no resource.
I have added tests in the destructor but given the constructor should throw an exception on a failure to initialise the internal state correctly, it really ought to be unnecessary. but I guess it cant hurt being there!
The point is that some libraries are not robust enough to handle freeing data multiple times. And with the way postblit/dtors work, you have to handle this properly in D.
 As I noted to Nicholas it seems the application is getting a valid data
 structure returned with invalid data and that is where the SIGSEGV is. This is
 really weird as I have just finished a Rust version of the same application
 and it works fine. And this D version used to work fine. It is a real mystery
 why there is a problem now.
Hm... your description of having the problem happen at the end of main seems to suggest it has something to do with destruction. -Steve
Jan 10
parent reply Russel Winder <russel winder.org.uk> writes:
On Thu, 2019-01-10 at 10:00 -0500, Steven Schveighoffer via Digitalmars-d-
learn wrote:
[=E2=80=A6]
=20
 Hm... your description of having the problem happen at the end of main=
=20
 seems to suggest it has something to do with destruction.
=20
It seems that there was a change in one file of libdvbv5 1.14.x =E2=86=92 1= .16.y that introduced a breaking change wrt the D binding. I did a regeneration using DStep, didn't notice anything significant, and yet everything now works aga= in. So it was very significant. The underlying problem here was that I had failed to notice the upgrade of libdvbv5! =20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 10
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/10/19 12:30 PM, Russel Winder wrote:
 On Thu, 2019-01-10 at 10:00 -0500, Steven Schveighoffer via Digitalmars-d-
 learn wrote:
 […]
 Hm... your description of having the problem happen at the end of main
 seems to suggest it has something to do with destruction.
It seems that there was a change in one file of libdvbv5 1.14.x → 1..16.y that introduced a breaking change wrt the D binding. I did a regeneration using DStep, didn't notice anything significant, and yet everything now works again. So it was very significant. The underlying problem here was that I had failed to notice the upgrade of libdvbv5!
That is one problem with linking against C or C++ code -- changes to certain things (e.g. struct layout) don't change the mangling. You may want to consider using dpp instead if possible. -Steve
Jan 10
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Jan 10, 2019 at 01:09:22PM -0500, Steven Schveighoffer via
Digitalmars-d-learn wrote:
 On 1/10/19 12:30 PM, Russel Winder wrote:
 On Thu, 2019-01-10 at 10:00 -0500, Steven Schveighoffer via Digitalmars-d-
 learn wrote:
 […]
 Hm... your description of having the problem happen at the end of
 main seems to suggest it has something to do with destruction.
 
It seems that there was a change in one file of libdvbv5 1.14.x → 1..16.y that introduced a breaking change wrt the D binding. I did a regeneration using DStep, didn't notice anything significant, and yet everything now works again. So it was very significant. The underlying problem here was that I had failed to notice the upgrade of libdvbv5!
That is one problem with linking against C or C++ code -- changes to certain things (e.g. struct layout) don't change the mangling.
Yeah, this is the same problem with shared library soname versioning on Posix. Technically everytime the ABI changes the version must be bumped, but since this is not automated, it's prone to human error, or rather, negligence. It makes one wonder if there should somehow be a way of encapsulating the changes to the ABI in a way that can be automatically checked. (It has to be automatic, otherwise it would be too onerous and nobody would do it in practice.) The most obvious way is to mangle the field types of the struct as part of the struct's mangled name, though this does introduce a lot of symbol bloat (and may need another round of ridiculously-long symbol names that need some manner of compression to keep under control). Barring that, perhaps some kind of low-collision hash of the struct contents where the kind of small changes that tend to happen in code will be highly unlikely to collide, so any such changes will be easily detected. If one were paranoid, one could use cryptographic hashes for pretty much guaranteeing uniqueness, but that'd be total overkill. // OTOH, perhaps the more pertinent issue here is that the bindings were generated *manually as a separate step* outside of the build system. Ideally, you'd automate the generation of bindings as part of your build, so that they will *always* be up-to-date. I'm a big fan of automation, because this is the kind of tedious housekeeping that humans are really, really good at forgetting and/or screwing up. (Side-note: and this is why I insist that my build systems must support generic dependency building. All these sorts of tasks *need* to be part of the build rather than done by hand, precisely to prevent these sorts of time-wasting, head-scratching mishaps.)
 You may want to consider using dpp instead if possible.
[...] Or this. Which is essentially equivalent to automatically generating bindings. T -- Маленькие детки - маленькие бедки.
Jan 10
prev sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Thu, 2019-01-10 at 13:09 -0500, Steven Schveighoffer via Digitalmars-d-
learn wrote:
=20
[=E2=80=A6]
 That is one problem with linking against C or C++ code -- changes to=20
 certain things (e.g. struct layout) don't change the mangling.
I am having nightmares trying to decide what to do with the Rust version ba= sed around generate on demand or on version change. With bindgen in Rust though= , there is no need for manual tweaking so automated is possible. Except that = it puts a massive dependency burden on any project using it. DStep generated bindings tend to need some manual tweaking that cannot be automated, which is surprising given that bindgen can do things without man= ual intervention for Rust.
 You may want to consider using dpp instead if possible.
DPP cannot build "out of the box" on Debian Sid, so I have not actually tri= ed it. There are three audiences here: 1. People building libraries for their own use on their own machines. 2. People building for OS distributions. 3. People building to distribute to others who will not be building for themselves.=20 Categories 1 and 2 could probably cope with automated generation despite th= e huge dependency burden, assuming there are no version conflicts =E2=80=93 t= his seems to be a massive unsolved problem with Rust/Cargo/crates.io :-( . Category 3 needs as light a weight and speed of build as possible, and the ability to dynamically adapt to the APIs and ABIs of execution at run time. On the other hand, suspect I may be the sole user of RUst and D bindings to libdvbv5! --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 10
parent reply Jacob Carlborg <doob me.com> writes:
On 2019-01-11 06:31, Russel Winder wrote:

 DStep generated bindings tend to need some manual tweaking that cannot be
 automated, which is surprising given that bindgen can do things without manual
 intervention for Rust.
It's not surprising at all. Different tools, different approaches, different amount of man power, different output languages and so on. Please file any bugs and/or enhancement requests that you find. -- /Jacob Carlborg
Jan 13
parent reply Russel Winder <russel winder.org.uk> writes:
On Sun, 2019-01-13 at 21:56 +0100, Jacob Carlborg via Digitalmars-d-learn
wrote:
 On 2019-01-11 06:31, Russel Winder wrote:
=20
 DStep generated bindings tend to need some manual tweaking that cannot =
be
 automated, which is surprising given that bindgen can do things without
 manual
 intervention for Rust.
=20 It's not surprising at all. Different tools, different approaches,=20 different amount of man power, different output languages and so on.
Indeed. Apologies for any aggressiveness that I might have appeared to set out, it was not intended. It's really just frustration.=20 To balance things out a bit: DStep does some stuff that Bindgen just doesn'= t even tackle. I am finding I am having to do some manual stuff with Bindgen.
 Please file any bugs and/or enhancement requests that you find.
Wilco. But I'll try and create test cases rather than just point at the ful= l project. --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Jan 14
parent Jacob Carlborg <doob me.com> writes:
On 2019-01-14 15:07, Russel Winder wrote:

 Wilco. But I'll try and create test cases rather than just point at the full
 project.
Reduced test cases are definitely appreciated. Unfortunately I don't use DStep enough to find all bugs and enhancements. Hopefully that will change. -- /Jacob Carlborg
Jan 14