www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Shared library packaging issues

reply "GreatEmerald" <pastas4 gmail.com> writes:
I'm currently attempting to package my D project 
(https://github.com/GreatEmerald/libarcomage/) as a shared 
library for openSUSE and whatnot, but I'm running into some 
policy problems.

First, in order to build a shared library, it has to link against 
libphobos2.so. libphobos2.so is provided by the DMD package. 
Which means... The users of my library have to have DMD installed 
at runtime. Even though they don't intend to compile anything.

Ideally, this problem would be solved by splitting libphobos2.so 
into its own package. After all, even the licenses of DMD and 
Phobos2 are different. This can be done downstream, but for 
instance for openSUSE, there's not much of a downstream to begin 
with. And this is needed for each distribution.

While I'm at it, in the DMD RPM it's also set as Provides 
"libphobos2.so.0.66" instead of the expected 
"libphobos2.so.0.66()(64bit)" on openSUSE. I'm not too sure if 
this is distribution-specific or a standard practice, though.

Second, what's the suggested method of providing includes? Put 
all the source of the library into includes? Make .di files? What 
about templates? Is there a suggested directory structure?
Nov 15 2014
next sibling parent "tcak" <tcak gmail.com> writes:
Well, because of this reason, I completely left D and started 
writing my shared library in pure C. As much as I miss the syntax 
of D, I needed to do this (Where are you betterC? Are you there 
yet?). Since it is not a big library, that wouldn't be a big 
problem.

Anyway, I think instead of installing DMD to target OS just to 
get libphobos, I think you can just copy it from development OS. 
That should solve the problem. But make sure to name the library 
correctly. If my experience is accurate, it should be with the 
name libphobos2.so.0.66. If still doesn't work (tells that 
libphobos is not found), create a link to library with the name 
libphobos2.so as well.

About your problem solving part, the thing is that Phobos 
includes druntime inside it, and druntime is what makes most 
features of D tick like GC, arrays, etc. Also there are things 
like compile time regular expression, etc. Thus, neither leaving 
nor keeping the system in its current state is making everybody 
happy. I am still looking forward for "betterC" though. Which 
would allow me to write in D instead of C when I am not looking 
for D's libraries.

On Saturday, 15 November 2014 at 11:21:49 UTC, GreatEmerald wrote:

 I'm currently attempting to package my D project 
 (https://github.com/GreatEmerald/libarcomage/) as a shared 
 library for openSUSE and whatnot, but I'm running into some 
 policy problems.

 First, in order to build a shared library, it has to link 
 against libphobos2.so. libphobos2.so is provided by the DMD 
 package. Which means... The users of my library have to have 
 DMD installed at runtime. Even though they don't intend to 
 compile anything.

 Ideally, this problem would be solved by splitting 
 libphobos2.so into its own package. After all, even the 
 licenses of DMD and Phobos2 are different. This can be done 
 downstream, but for instance for openSUSE, there's not much of 
 a downstream to begin with. And this is needed for each 
 distribution.

 While I'm at it, in the DMD RPM it's also set as Provides 
 "libphobos2.so.0.66" instead of the expected 
 "libphobos2.so.0.66()(64bit)" on openSUSE. I'm not too sure if 
 this is distribution-specific or a standard practice, though.

 Second, what's the suggested method of providing includes? Put 
 all the source of the library into includes? Make .di files? 
 What about templates? Is there a suggested directory structure?
Nov 15 2014
prev sibling parent reply Jordi Sayol via Digitalmars-d <digitalmars-d puremagic.com> writes:
El 15/11/14 a les 12:21, GreatEmerald via Digitalmars-d ha escrit:


 Ideally, this problem would be solved by splitting libphobos2.so into its own
package.


Since dmd v2.064.0, a shared phobos2 library package already exist for Debian
based systems:
<http://downloads.dlang.org/releases/2014/libphobos2-66_2.066.1-0_i386.deb>
<http://downloads.dlang.org/releases/2014/libphobos2-66_2.066.1-0_amd64.deb>

As every dmd release breaks libphobos2 API, these deb packages can be installed
in diverse versions at same time.
I.e. "libphobos2-65_2.065.0-0_amd64.deb" and
"libphobos2-66_2.066.1-0_amd64.deb" can be installed together. This allow to
have diverse dmd programs linked to diverse libphobos2 shared libraries at the
same system.


 Second, what's the suggested method of providing includes? Put all the source
of the library into includes? Make .di files? What about templates? Is there a
suggested directory structure?


The default path for "d" and "di" sources should be "/usr/include/d/".
DMD deb packages uses "/usr/include/dmd/" to avoid sources conflict with the
ldc sources packaged for Debian.

Regards,
-- 
Jordi Sayol
Nov 15 2014
next sibling parent reply "GreatEmerald" <pastas4 gmail.com> writes:
On Saturday, 15 November 2014 at 14:49:23 UTC, Jordi Sayol via 
Digitalmars-d wrote:

 Since dmd v2.064.0, a shared phobos2 library package already 
 exist for Debian based systems:
 <http://downloads.dlang.org/releases/2014/libphobos2-66_2.066.1-0_i386.deb>
 <http://downloads.dlang.org/releases/2014/libphobos2-66_2.066.1-0_amd64.deb>


Nice. Now if only there were RPMs as well... If only as a 
template.


 The default path for "d" and "di" sources should be 
 "/usr/include/d/".


Ah, thanks.
Nov 15 2014
parent reply Jordi Sayol via Digitalmars-d <digitalmars-d puremagic.com> writes:
El 15/11/14 a les 18:29, GreatEmerald via Digitalmars-d ha escrit:


 Nice. Now if only there were RPMs as well... If only as a template.


Meanwhile there is not an RPM package for phobos 2 shared library, I think you
can spread it together with your library:
"/usr/lib/i386-linux-gnu/libphobos2.so.0.66.1" (for 32-bit)
"/usr/lib/x86_64-linux-gnu/libphobos2.so.0.66.1" (for 64-bit)

It has no sense to install ALL the dmd compiler tools into the final runtime
computer.

Regards,
-- 
Jordi Sayol
Nov 15 2014
parent reply "GreatEmerald" <pastas4 gmail.com> writes:
On Saturday, 15 November 2014 at 18:36:28 UTC, Jordi Sayol via 
Digitalmars-d wrote:

 Meanwhile there is not an RPM package for phobos 2 shared 
 library, I think you can spread it together with your library:
 "/usr/lib/i386-linux-gnu/libphobos2.so.0.66.1" (for 32-bit)
 "/usr/lib/x86_64-linux-gnu/libphobos2.so.0.66.1" (for 64-bit)



That wouldn't really work well if I also want to use LuaD as a 
shared library, as both packages would then conflict with each 
other over providing libphobos2. Probably better to just depend 
on DMD for a while.

Though it does make me wonder, how do GDC and LDC handle this 
issue? Do they also include libphobos2 as part of the compiler?
Nov 15 2014
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 15 Nov 2014 21:05, "GreatEmerald via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 On Saturday, 15 November 2014 at 18:36:28 UTC, Jordi Sayol via


Digitalmars-d wrote:

 Meanwhile there is not an RPM package for phobos 2 shared library, I




think you can spread it together with your library:

 "/usr/lib/i386-linux-gnu/libphobos2.so.0.66.1" (for 32-bit)
 "/usr/lib/x86_64-linux-gnu/libphobos2.so.0.66.1" (for 64-bit)




 That wouldn't really work well if I also want to use LuaD as a shared


library, as both packages would then conflict with each other over
providing libphobos2. Probably better to just depend on DMD for a while.

 Though it does make me wonder, how do GDC and LDC handle this issue? Do


they also include libphobos2 as part of the compiler?

It's a libgphobos-dev package for gdc.

Iain.
Nov 15 2014
parent "GreatEmerald" <pastas4 gmail.com> writes:
On Saturday, 15 November 2014 at 22:19:07 UTC, Iain Buclaw via 
Digitalmars-d wrote:

 It's a libgphobos-dev package for gdc.


Ah, so they repackage it as well. Same with LDC.

Not that it helps much, given that there's also no GDC on 
openSUSE. Oh well.
Nov 15 2014
prev sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Saturday, 15 November 2014 at 21:01:49 UTC, GreatEmerald wrote:

 Though it does make me wonder, how do GDC and LDC handle this 
 issue? Do they also include libphobos2 as part of the compiler?


That's something entirely up to the distro packagers to get right 
in the case of LDC. We don't directly offer .debs/.rpms or what 
have you as part of our binary releases.

That being said, for example on Arch Linux, Phobos and druntime 
are provided as separate packages (thanks, Dicebot!).

David
Nov 15 2014
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Sat, 15 Nov 2014 15:35:48 +0100
schrieb Jordi Sayol via Digitalmars-d
<digitalmars-d puremagic.com>:


 El 15/11/14 a les 12:21, GreatEmerald via Digitalmars-d ha escrit:
 

 Ideally, this problem would be solved by splitting libphobos2.so into its own
package.


 
 Since dmd v2.064.0, a shared phobos2 library package already exist for Debian
based systems:
 <http://downloads.dlang.org/releases/2014/libphobos2-66_2.066.1-0_i386.deb>
 <http://downloads.dlang.org/releases/2014/libphobos2-66_2.066.1-0_amd64.deb>
 
 As every dmd release breaks libphobos2 API, these deb packages can be
installed in diverse versions at same time.
 I.e. "libphobos2-65_2.065.0-0_amd64.deb" and
"libphobos2-66_2.066.1-0_amd64.deb" can be installed together. This allow to
have diverse dmd programs linked to diverse libphobos2 shared libraries at the
same system.
 

 Second, what's the suggested method of providing includes? Put all the source
of the library into includes? Make .di files? What about templates? Is there a
suggested directory structure?


 
 The default path for "d" and "di" sources should be "/usr/include/d/".
 DMD deb packages uses "/usr/include/dmd/" to avoid sources conflict with the
ldc sources packaged for Debian.
 
 Regards,


Exactly a year ago I created a poll[1] about the include path
and 65% voted for "dlang" over just "d". We should stick to
"/usr/include/dlang" now for a consistent experience across
distributions. Compilers' import paths are less of an issue as
they are either built-in or configured in dmd.conf. But for
build tools it would be nice if imports for library X could
always be found in "/usr/include/dlang/X/".

[1]
http://forum.dlang.org/thread/20131112205019.12585bbd marco-leise


-- 
Marco
Nov 16 2014
parent reply "GreatEmerald" <pastas4 gmail.com> writes:
On Monday, 17 November 2014 at 02:29:54 UTC, Marco Leise wrote:

 Exactly a year ago I created a poll[1] about the include path
 and 65% voted for "dlang" over just "d". We should stick to
 "/usr/include/dlang" now for a consistent experience across
 distributions. Compilers' import paths are less of an issue as
 they are either built-in or configured in dmd.conf. But for
 build tools it would be nice if imports for library X could
 always be found in "/usr/include/dlang/X/".


Fair enough, although if the issue with different compilers 
making a different ABI persists, then wouldn't it be a good idea 
to put it into /usr/include/{dmd,gdc,ldc} or something?

But otherwise I pinged some people on openSUSE and it seems there 
is some interest in D, so now we have a separate project for it. 
Maintaining a separate libphobos2 doesn't look very hard after 
all.

Though I'm still a bit unsure about the whole druntime part. The 
official RPM doesn't include any libraries for druntime, only 
includes, albeit building it from source does produce a druntime 
library. Is it worth separating it out into its own package? Or 
should it be part of dmd-devel instead? (druntime-devel doesn't 
really sound right if there is no actual druntime package 
itself...)
Nov 17 2014
next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 17 November 2014 at 19:36:10 UTC, GreatEmerald wrote:

 But otherwise I pinged some people on openSUSE and it seems 
 there is some interest in D, so now we have a separate project 
 for it.


That's great to hear! Duncan Mac-Vicar started work on an LDC 
packages, which I'm keeping up-to-date since then:
https://build.opensuse.org/package/show/home:marc_schuetz:branches:home:dmacvicar/ldc

I'm going to ask him, and then submit this package to the new 
project.

I was a bit afraid of what will become of it when we switch to 
DDMD, because then we'll need to have an already working D 
compiler, but this project will make it easier.

BTW, I'm unsure how to handle DMD's license:
https://github.com/D-Programming-Language/dmd/blob/master/src/backendlicense.txt

I think we need to ask Walter first before DMD can be put on the 
OBS.
Nov 18 2014
parent "GreatEmerald" <pastas4 gmail.com> writes:
On Tuesday, 18 November 2014 at 11:14:06 UTC, Marc Schütz wrote:

 That's great to hear! Duncan Mac-Vicar started work on an LDC 
 packages, which I'm keeping up-to-date since then:
 https://build.opensuse.org/package/show/home:marc_schuetz:branches:home:dmacvicar/ldc

 I'm going to ask him, and then submit this package to the new 
 project.


Yeap, please do :)


 BTW, I'm unsure how to handle DMD's license:
 https://github.com/D-Programming-Language/dmd/blob/master/src/backendlicense.txt

 I think we need to ask Walter first before DMD can be put on 
 the OBS.


Yea, it's a bit of a problem. It would be nice to do that.
Nov 18 2014
prev sibling next sibling parent "GreatEmerald" <pastas4 gmail.com> writes:
On Monday, 17 November 2014 at 19:36:10 UTC, GreatEmerald wrote:

 Though I'm still a bit unsure about the whole druntime part. 
 The official RPM doesn't include any libraries for druntime, 
 only includes, albeit building it from source does produce a 
 druntime library. Is it worth separating it out into its own 
 package? Or should it be part of dmd-devel instead? 
 (druntime-devel doesn't really sound right if there is no 
 actual druntime package itself...)


Answering my own question, looks like libphobos2 statically links 
to druntime, so those includes should probably be part of 
libphobos2-devel.
Nov 18 2014
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 17 Nov 2014 19:36:08 +0000
schrieb "GreatEmerald" <pastas4 gmail.com>:


 On Monday, 17 November 2014 at 02:29:54 UTC, Marco Leise wrote:

 Exactly a year ago I created a poll[1] about the include path
 and 65% voted for "dlang" over just "d". We should stick to
 "/usr/include/dlang" now for a consistent experience across
 distributions. Compilers' import paths are less of an issue as
 they are either built-in or configured in dmd.conf. But for
 build tools it would be nice if imports for library X could
 always be found in "/usr/include/dlang/X/".


 
 Fair enough, although if the issue with different compilers 
 making a different ABI persists, then wouldn't it be a good idea 
 to put it into /usr/include/{dmd,gdc,ldc} or something?


ABI differences in source files only apply to druntime/Phobos
and these imports are generally on the default import path. I
was more concerned with GtkD and similar 3rd party libraries,
for which you have to specify the import path.
Or am I missing a case where 3rd party import files should also
differ for each compiler?

As for the ABI incompatibilities of libraries themselves I'm
using -rpath and install them to /opt/<compiler>-<version>/lib
or GCC's own version specific library path in case of GDC. My
hope is that some day the issue can be settled and this is
only a stop-gap measure. (We'd still have a pretty fast
changing Phobos and possibly ABI changes due to bug fixes or
enhancements, so it is likely we end up with some "dlang-2070"
pre- or postfix, but not one library per compiler vendor and
Dlang version as it is now.)

-- 
Marco
Nov 18 2014
parent reply "GreatEmerald" <pastas4 gmail.com> writes:
On Tuesday, 18 November 2014 at 16:26:58 UTC, Marco Leise wrote:

 ABI differences in source files only apply to druntime/Phobos
 and these imports are generally on the default import path. I
 was more concerned with GtkD and similar 3rd party libraries,
 for which you have to specify the import path.


Fair enough, include files shouldn't be compiler-specific. But 
then why have them in a dlang subdirectory and not simply in each 
package's own subdirectory in includes root? Files in 
/usr/includes/libarcomage are probably easier to find than those 
in /usr/includes/dlang/libarcomage.
Nov 18 2014
parent Marco Leise <Marco.Leise gmx.de> writes:
Am Tue, 18 Nov 2014 17:10:18 +0000
schrieb "GreatEmerald" <pastas4 gmail.com>:


 On Tuesday, 18 November 2014 at 16:26:58 UTC, Marco Leise wrote:

 ABI differences in source files only apply to druntime/Phobos
 and these imports are generally on the default import path. I
 was more concerned with GtkD and similar 3rd party libraries,
 for which you have to specify the import path.


 
 Fair enough, include files shouldn't be compiler-specific. But 
 then why have them in a dlang subdirectory and not simply in each 
 package's own subdirectory in includes root? Files in 
 /usr/includes/libarcomage are probably easier to find than those 
 in /usr/includes/dlang/libarcomage.


I don't know. I guess maybe because /usr/include typically
contains .h files or to avoid directory name conflicts with
existing C libraries. I guess we could drop it if we agreed on
that.

-- 
Marco
Nov 18 2014