www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - What is going on with the ubuntu/debian debacle ?

reply deadalnix <deadalnix gmail.com> writes:
A lot of users are reporting errors on debian and ubuntu. .o 
generated by dmd do not link and libphobos.a is unusable.

This is very bad and we should consider a hotfix. Is someone on 
it ?
Nov 19 2016
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/19/2016 3:03 PM, deadalnix wrote:

 A lot of users are reporting errors on debian and ubuntu. .o generated by dmd
do
 not link and libphobos.a is unusable.

 This is very bad and we should consider a hotfix. Is someone on it ?


Bugzilla numbers?
Nov 19 2016
parent reply H. S. Teoh <hsteoh quickfur.ath.cx> writes:
On Sunday, 20 November 2016 at 03:28:33 UTC, Walter Bright wrote:

 On 11/19/2016 3:03 PM, deadalnix wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do
 not link and libphobos.a is unusable.

 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 Bugzilla numbers?


https://issues.dlang.org/show_bug.cgi?id=16794  (links to other 
bugs included in report)
Nov 25 2016
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Friday, 25 November 2016 at 23:27:52 UTC, H. S. Teoh wrote:

 On Sunday, 20 November 2016 at 03:28:33 UTC, Walter Bright 
 wrote:

 On 11/19/2016 3:03 PM, deadalnix wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do
 not link and libphobos.a is unusable.

 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 Bugzilla numbers?


 https://issues.dlang.org/show_bug.cgi?id=16794  (links to other 
 bugs included in report)


I don't think compiling everything as PIC is an appropriate way 
to go.

Still, I find myself unable to use D at all on my debian boxes.
Nov 25 2016
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Nov 25, 2016 at 11:59:26PM +0000, deadalnix via Digitalmars-d wrote:

 On Friday, 25 November 2016 at 23:27:52 UTC, H. S. Teoh wrote:

 On Sunday, 20 November 2016 at 03:28:33 UTC, Walter Bright wrote:

 On 11/19/2016 3:03 PM, deadalnix wrote:

 A lot of users are reporting errors on debian and ubuntu. .o
 generated by dmd do not link and libphobos.a is unusable.
 
 This is very bad and we should consider a hotfix. Is someone on
 it ?


 
 Bugzilla numbers?


 
 https://issues.dlang.org/show_bug.cgi?id=16794  (links to other bugs
 included in report)


 
 I don't think compiling everything as PIC is an appropriate way to go.
 
 Still, I find myself unable to use D at all on my debian boxes.


Well, until DMD supports PIE (which if I understand correctly is a
subset of PIC), there seems to be no easy way around this debacle.

The other alternative is to make dmd invoke gcc with -no-pie when
linking.  Is there an easy way to do this?


T

-- 
An elephant: A mouse built to government specifications. -- Robert Heinlein
Nov 25 2016
prev sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Friday, 25 November 2016 at 23:27:52 UTC, H. S. Teoh wrote:

 On Sunday, 20 November 2016 at 03:28:33 UTC, Walter Bright 
 wrote:

 On 11/19/2016 3:03 PM, deadalnix wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do
 not link and libphobos.a is unusable.

 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 Bugzilla numbers?


 https://issues.dlang.org/show_bug.cgi?id=16794  (links to other 
 bugs included in report)


How Ubuntu-based distributions, but that are late on updates (I 
think to Mint for example which will not need PIE compatibility 
before months) will be affected by the need .deb ?
Nov 30 2016
parent Basile B. <b2.temp gmx.com> writes:
On Wednesday, 30 November 2016 at 21:32:37 UTC, Basile B. wrote:

 How Ubuntu-based distributions, but that are late on updates (I 
 think to Mint for example which will not need PIE compatibility 
 before months) will be affected by the need .deb ?


~~need~~ new deb
Nov 30 2016
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 19 November 2016 at 23:03:05 UTC, deadalnix wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do not link and libphobos.a is unusable.

 This is very bad and we should consider a hotfix. Is someone on 
 it ?


This is the PIE by default problem, yes? In which case the hot 
fix should just always enable -fPIC for debian/ubuntu.

  See also 
https://forum.dlang.org/thread/cgifwxqogkvpfrjcioum forum.dlang.org
Nov 19 2016
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Sunday, 20 November 2016 at 05:01:06 UTC, Nicholas Wilson 
wrote:

 On Saturday, 19 November 2016 at 23:03:05 UTC, deadalnix wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do not link and libphobos.a is unusable.

 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 This is the PIE by default problem, yes? In which case the hot 
 fix should just always enable -fPIC for debian/ubuntu.

  See also 
 https://forum.dlang.org/thread/cgifwxqogkvpfrjcioum forum.dlang.org


That's the hotfix, but in general it's overkill. -fpie is a 
subset of -fpic, see 
https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html
Nov 20 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 20 November 2016 at 09:00:08 UTC, John Colvin wrote:

 On Sunday, 20 November 2016 at 05:01:06 UTC, Nicholas Wilson 
 wrote:

 On Saturday, 19 November 2016 at 23:03:05 UTC, deadalnix wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do not link and libphobos.a is unusable.

 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 This is the PIE by default problem, yes? In which case the hot 
 fix should just always enable -fPIC for debian/ubuntu.

  See also 
 https://forum.dlang.org/thread/cgifwxqogkvpfrjcioum forum.dlang.org


 That's the hotfix, but in general it's overkill. -fpie is a 
 subset of -fpic, see 
 https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html


The shipped binaries are not compiled with that flag, so, unless 
I use my homebrew of everything, nothing works.
Nov 20 2016
parent John Colvin <john.loughran.colvin gmail.com> writes:
On Sunday, 20 November 2016 at 10:22:10 UTC, deadalnix wrote:

 On Sunday, 20 November 2016 at 09:00:08 UTC, John Colvin wrote:

 On Sunday, 20 November 2016 at 05:01:06 UTC, Nicholas Wilson 
 wrote:

 On Saturday, 19 November 2016 at 23:03:05 UTC, deadalnix 
 wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do not link and libphobos.a is unusable.

 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 This is the PIE by default problem, yes? In which case the 
 hot fix should just always enable -fPIC for debian/ubuntu.

  See also 
 https://forum.dlang.org/thread/cgifwxqogkvpfrjcioum forum.dlang.org


 That's the hotfix, but in general it's overkill. -fpie is a 
 subset of -fpic, see 
 https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html


 The shipped binaries are not compiled with that flag, so, 
 unless I use my homebrew of everything, nothing works.


Yes, as I said, -fpic* is the hotfix until we support -fpie. We 
should definitely provide alternative downloads compiled with 
-fpic, at least the deb packages if not also the zip.

* side note: fpic and fPIC aren't exactly the same on all 
platforms, but for x86(_64) iirc they are identical.
Nov 20 2016
prev sibling next sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 2016-11-19 at 23:03 +0000, deadalnix via Digitalmars-d wrote:

 A lot of users are reporting errors on debian and ubuntu. .o=C2=A0
 generated by dmd do not link and libphobos.a is unusable.
=20
 This is very bad and we should consider a hotfix. Is someone on=C2=A0
 it ?


Is this related to the gdc cannot be installed on Debian Sid at the
moment problem?

Are they using the DMD Deb file or the Deb from D-APT?

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Nov 20 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 20 November 2016 at 13:19:57 UTC, Russel Winder wrote:

 On Sat, 2016-11-19 at 23:03 +0000, deadalnix via Digitalmars-d 
 wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do not link and libphobos.a is unusable.
 
 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 Is this related to the gdc cannot be installed on Debian Sid at 
 the moment problem?

 Are they using the DMD Deb file or the Deb from D-APT?


This doesn't affect installation. But nothing link anymore.
Nov 20 2016
parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 21 November 2016 at 01:20:06 UTC, deadalnix wrote:

 This doesn't affect installation. But nothing link anymore.


Ho Hi !

The .deb do NOT work. Can we get an hotfix out there ?
Nov 21 2016
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Monday, 21 November 2016 at 19:16:07 UTC, deadalnix wrote:

 On Monday, 21 November 2016 at 01:20:06 UTC, deadalnix wrote:

 This doesn't affect installation. But nothing link anymore.


 Ho Hi !

 The .deb do NOT work. Can we get an hotfix out there ?


Which deb does not work where ?
I will try to reproduce and fix the issue.
Nov 21 2016
parent reply Pjotr Prins <pjotr.public12 thebird.nl> writes:
On Monday, 21 November 2016 at 19:23:23 UTC, Stefan Koch wrote:

 I will try to reproduce and fix the issue.


We are also working on a new GNU Guix package for ldc latest. 
More on that soon.
Nov 22 2016
next sibling parent Johan Engelen <j j.nl> writes:
On Tuesday, 22 November 2016 at 10:30:00 UTC, Pjotr Prins wrote:

 We are also working on a new GNU Guix package for ldc latest. 
 More on that soon.


We are very close to finishing 1.1.0-beta4. Because beta3 has 
serious regressions, please wait until beta4.

Thanks,
   Johan
Nov 22 2016
prev sibling parent Pjotr Prins <pjotr.public12 thebird.nl> writes:
I have created a relocatable GNU Guix 64-bits binary of ldc 1.1.0 
(just released) that will install on any Linux distribution.

   LDC - the LLVM D compiler (1.1.0):
     based on DMD v2.071.2 and LLVM 3.7.1
     built with LDC - the LLVM D compiler (0.17.1)
     Default target: x86_64-unknown-linux-gnu

ldc can be installed and run in a HOME directory without root
privileges by simply unpacking the tarball from

     
http://biogems.info/contrib/genenetwork/pk9rkm4zvdp6pglam7s280x1x8y5rvbz-ldc-1.1.0-x86_64.tar.bz2
     md5sum fe2508135eadc87fcc31027524c11ec5

and running the contained

   ./install.sh TARGETDIR.

 From TARGETDIR ldc2 can compile hello.d and
also a more complex project where I had to tell it where to find
libphobos.a (which is included).

I need a few people to test drive this. Please have a go if you 
have a
few minutes because it is important we can deploy such software 
on HPC
systems. GNU Guix is great because it gives us a fully 
reproducible
build system and relocatable Guix does not require administrator
rights, nor Docker.

In a future version we will update LLVM and add dup (support for 
both already exists in Guix).
Jan 30 2017
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Nov 19, 2016 at 11:03:05PM +0000, deadalnix via Digitalmars-d wrote:

 A lot of users are reporting errors on debian and ubuntu. .o generated
 by dmd do not link and libphobos.a is unusable.
 
 This is very bad and we should consider a hotfix. Is someone on it ?


FWIW, I'm using Debian/unstable and compiling dmd/druntime/phobos
directly from git HEAD, and it seems to work fine.

Does this problem only happen for the shipped .deb packages?


T

-- 
The trouble with TCP jokes is that it's like hearing the same joke over and
over.
Nov 22 2016
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 22 November 2016 at 21:19:13 UTC, H. S. Teoh wrote:

 On Sat, Nov 19, 2016 at 11:03:05PM +0000, deadalnix via 
 Digitalmars-d wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do not link and libphobos.a is unusable.
 
 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 FWIW, I'm using Debian/unstable and compiling 
 dmd/druntime/phobos directly from git HEAD, and it seems to 
 work fine.

 Does this problem only happen for the shipped .deb packages?


 T


It happens with anything compiled without fPIC as far as I can 
tell. unless the relocation emission bug has been fixed in 
master, then that should happen as well.
Nov 22 2016
prev sibling parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Tuesday, 22 November 2016 at 21:19:13 UTC, H. S. Teoh wrote:

 On Sat, Nov 19, 2016 at 11:03:05PM +0000, deadalnix via 
 Digitalmars-d wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do not link and libphobos.a is unusable.
 
 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 FWIW, I'm using Debian/unstable and compiling 
 dmd/druntime/phobos directly from git HEAD, and it seems to 
 work fine.

 Does this problem only happen for the shipped .deb packages?


 T


I think it's not a problem for you, because your system's linker 
probably adds fPIC by default when you're building druntime and 
phobos. In contrast the binaries available on dlang.org/download 
were probably built on machine that does not add -fPIC by default.
Nov 23 2016
parent reply H. S. Teoh <hsteoh quickfur.ath.cx> writes:
On Wednesday, 23 November 2016 at 14:00:15 UTC, ZombineDev wrote:

 On Tuesday, 22 November 2016 at 21:19:13 UTC, H. S. Teoh wrote:

 On Sat, Nov 19, 2016 at 11:03:05PM +0000, deadalnix via 
 Digitalmars-d wrote:

 A lot of users are reporting errors on debian and ubuntu. .o 
 generated by dmd do not link and libphobos.a is unusable.
 
 This is very bad and we should consider a hotfix. Is someone 
 on it ?


 FWIW, I'm using Debian/unstable and compiling 
 dmd/druntime/phobos directly from git HEAD, and it seems to 
 work fine.

 Does this problem only happen for the shipped .deb packages?


 T


 I think it's not a problem for you, because your system's 
 linker probably adds fPIC by default when you're building 
 druntime and phobos. In contrast the binaries available on 
 dlang.org/download were probably built on machine that does not 
 add -fPIC by default.


Haha, I spoke too soon.  It appears that the reason I didn't see 
this problem was because I hadn't upgraded gcc in a while. After 
updating to the latest gcc-6.2 a couple of days ago, dmd now 
fails horribly with linker errors.  I couldn't even rebuild dmd 
because the bootstrapping dmd 2.069 that I'm using also doesn't 
have Phobos compiled with -fPIC, so it dies when trying to 
compile idgen.

Anyway, I needed things working again, so I did some further 
investigation. Here's what I found:

- Under the latest Debian/Ubuntu gcc-6.2, it's *still* possible 
to retain the old (working) behaviour by specifying -no-pie to 
all invocations of gcc.  (By "all invocations" I mean also 
internal calls to gcc from inside dmd.)  Using this workaround in 
a wrapper script specified in HOST_CXX to dmd's posix.mak, it's 
possible to bootstrap dmd successfully from an older dmd that 
doesn't have Phobos compiled with -fPIC.

- Druntime and Phobos' makefiles already support the PIC=1 
option, which will build libphobos2.a with -fPIC.  To get dmd 
working again, make sure you `make clean; make PIC=1` in druntime 
followed by phobos (in that order, since otherwise libphobos2.a 
may pick up old non-PIC objects from druntime and it will break).

- After this is done, libphobos2.a is now built with -fPIC, so it 
is now compatible with Debian/Ubuntu's PIE-by-default 
configuration.  However, there is one last thing: dmd itself will 
emit a dmd-personality-v0 symbol in the generated object file(s), 
which will fail to link with the new libphobos2.a because, by 
default, this symbol is not specified as PIC, so the linker 
considers it incompatible with the new libphobos2.a.  So now we 
need to append "-fPIC" to the settings in the system dmd.conf, 
wherever you installed it (in my case, it was in /etc/dmd.conf), 
otherwise you will have to manually type "-fPIC" every time you 
invoke dmd.

- I did some googling, and found references to compiling with 
`-fPIC -defaultlib=libphobos2.so` instead, i.e., if you don't 
want to recompile druntime/phobos with -fPIC yourself.  This also 
works, *provided* you install the Phobos .so's in your system's 
lib directories.  If you're running dmd off the git repos 
directly, like I am, this requires symlinking libphobos2.*.so 
into /usr/local/lib or /usr/lib, otherwise none of your D 
executables will run.  I consider this too intrusive, especially 
for the bleeding edge phobos off git HEAD, so I settled for 
building libphobos.a with -fPIC instead, as described above.

Hope this info will be helpful to anyone else who's experiencing 
this problem. It cost me almost 2-3 hours' worth of frustration. 
:-/
Nov 25 2016
next sibling parent Johannes Pfau <nospam example.com> writes:
Am Fri, 25 Nov 2016 22:52:18 +0000
schrieb H. S. Teoh <hsteoh quickfur.ath.cx>:



 - I did some googling, and found references to compiling with 
 `-fPIC -defaultlib=libphobos2.so` instead, i.e., if you don't 
 want to recompile druntime/phobos with -fPIC yourself.  This also 
 works, *provided* you install the Phobos .so's in your system's 
 lib directories.  If you're running dmd off the git repos 
 directly, like I am, this requires symlinking libphobos2.*.so 
 into /usr/local/lib or /usr/lib, otherwise none of your D 
 executables will run.  I consider this too intrusive, especially 
 for the bleeding edge phobos off git HEAD, so I settled for 
 building libphobos.a with -fPIC instead, as described above.
 


You can also add the install path of the phobos/druntime.so files to the
LD_LIBRARY_PATH environment variable.
Nov 26 2016
prev sibling parent Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Friday, November 25, 2016 22:52:18 H. S. Teoh via Digitalmars-d wrote:

 Hope this info will be helpful to anyone else who's experiencing
 this problem. It cost me almost 2-3 hours' worth of frustration.


Thanks for the info, though I haven't managed to really get much working on
my machine. What really gets my goat about all this though is that with the
recent update from PC-BSD to TrueOS, my desktop no longer is able to work
with D properly (since while FreeBSD 9, 10, and 11 all work with D,
FreeBSD-current - which is what TrueOS is based on - has serious issues with
druntime), and then when I went to redo my laptop the other day, I set it up
with Kubuntu, because the Cannonical folks were actually willing to build
the ZFS drivers into the kernel, and I was sick of problems caused by the
ZFS drivers not loading properly, which means that _both_ of my machines are
now crippled with regards to D. So, I can't really do much with D right now
outside of a VM. Having one machine crippled would be bad enough, but both?
Definitely rotten luck.

I don't know how much I'll be able to do about the TrueOS/FreeBSD problem,
but I'm clearly going to have to spend time at least narrowing down the
problem, since AFAIK not many of us use FreeBSD around here. So, if I don't
at least jumpstart things on that one, I'm probably out of luck for the
forseeable future. But it looks like it's probably a druntime problem, so I
_might_ be able to fix it. It looks like this Debian/Ubuntu problem is
totally up to the compiler devs though and completely outside of my area of
expertise. It would be really nice though if it at least _one_ of those
problems were fixed.

- Jonathan M Davis
Nov 29 2016