www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Snap packages for D compilers and core projects

reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
Hello all,

I thought it might be time to share more generally something I've 
been working on for a little while: snap packages for some of the 
core D projects.

For those who don't know, snap packages are a new format 
developed by Ubuntu to facilitate upstreams being able to provide 
the latest versions of their apps directly to users.  The format 
is also designed to provide effective confinement for apps, so 
that they can only access the parts of the host system that they 
need to.  While developed by Ubuntu, the format is gaining quite 
a bit of of cross-distro traction: see http://snapcraft.io/ for 
more information.

I started by trying to snap LDC, mainly because the cmake build 
system made for a very easy integration with the snapcraft 
package-build system.  The LDC developers have been kind enough 
to accept this as an official contribution, and a first 
submission is currently waiting for review in the Ubuntu snap 
store, based on the following package definition:
https://github.com/ldc-developers/ldc2.snap

However, I have also created two other snap package definitions, 
one for DUB, and one (just this afternoon!) for DMD:
https://github.com/WebDrake/dub.snap/tree/classic-snap
https://github.com/WebDrake/dmd.snap/tree/classic-snap

As you can probably see from the package definitions, one of the 
attractions of the format is how remarkably easy it is to define 
a snap package.  People running Ubuntu 16.04 or later may like to 
try installing snapcraft and building these packages for 
themselves.

However, now that they exist, I'd like to try publishing these to 
the official snap store.  Rather than do this as some random 
developer, I'd quite like to publish them as official D language 
packages.  Note that the snap store supports multiple 'channels' 
with different levels of stability -- 'edge' and 'beta' being the 
testing ones -- so any issues with the packages can be worked 
through before a stable release is made.

The question is, (i) is this a welcome proposal? and (ii) if it 
is welcome, what do people see as the best way to go about this?

I would also welcome feedback on the current package definitions, 
which can be provided in the associated PRs:
https://github.com/WebDrake/dub.snap/pull/3
https://github.com/WebDrake/dmd.snap/pull/1

In creating the DUB snap package I also had to write a snapcraft 
plugin for DUB itself.  This would be good to submit upstream, 
but I'd like to get some feedback on it before doing so.  The 
plugin is in the `dub.py` file included in the PR above: it 
sufficies for the current purpose, but would need more work 
before it covered all potential use-cases.

Lastly, I have to mention that in creating the above packages, I 
received a great deal of very warm and welcoming support from the 
folks on the Snapcraft mailing list.  They were very eager to 
help me solve problems I encountered, and to help me find the 
most straightforward way to get the results I wanted out of these 
packages.

Anyway, hope this is interesting to everyone -- any thoughts, 
questions, feedback ... ?

Thanks & best wishes,

     -- Joe
Jan 29 2017
next sibling parent reply qznc <qznc web.de> writes:
On Sunday, 29 January 2017 at 20:07:50 UTC, Joseph Rushton 
Wakeling wrote:

 The question is, (i) is this a welcome proposal? and (ii) if it 
 is welcome, what do people see as the best way to go about this?


No comments? Well, there seems to be no downside (apart from the 
work).

So far, I considered Snap an Ubuntu-only initiative of 
yet-another-package-format. If it really gains cross-distro 
support, this is a great way for better D support on Fedora, 
Arch, etc. What is the evidence for cross-distro support? Can it 
be measured somehow?

Is it legally possible to distribute DMD this way? Afaik only 
dlang.org is allowed to distribute it publically due to the 
backend licence issue.
Jan 30 2017
next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 30 January 2017 at 14:40:13 UTC, qznc wrote:

 No comments? Well, there seems to be no downside (apart from 
 the work).


Yea, I'm a little sad to see the apparent lack of 
feedback/interest :-\  I had quite a lot of fun creating these 
packages and was hoping for a bit more curiosity.


 So far, I considered Snap an Ubuntu-only initiative of 
 yet-another-package-format. If it really gains cross-distro 
 support, this is a great way for better D support on Fedora, 
 Arch, etc. What is the evidence for cross-distro support? Can 
 it be measured somehow?


Snap packaging started as something Ubuntu were developing for 
their use-case, but started gaining cross-distro interest last 
year (probably because AFAICT its feature-set and simplicity of 
use is quite a bit ahead of alternatives like Flatpak).

Installation instructions for various distros:
http://snapcraft.io/docs/core/install


 Is it legally possible to distribute DMD this way? Afaik only 
 dlang.org is allowed to distribute it publically due to the 
 backend licence issue.


Yes, that's a very good point that I wanted to raise.  There is a 
legal agreement that you're asked to make with respect to usage 
and distribution of packages.  It looks like it ought to be 
compatible with constraints on DMD (re)distribution, but it would 
obviously need to be reviewed and agreed to by Walter.  This is 
one reason why if I do take this forward, I'd like to do so with 
some sort of official backing.

It doesn't stop me moving forward with DUB, of course, but it'd 
be nice to do that officially as well.
Jan 30 2017
next sibling parent reply bachmeier <no spam.net> writes:
On Monday, 30 January 2017 at 16:53:34 UTC, Joseph Rushton 
Wakeling wrote:

 Yea, I'm a little sad to see the apparent lack of 
 feedback/interest :-\


I have interest, but as I've never heard of Snap before, I have 
no comments. Others may be in the same boat. Installation of D 
compilers is probably not an issue for most of us that read these 
messages.
Jan 30 2017
parent Jon Degenhardt <jond noreply.com> writes:
On Monday, 30 January 2017 at 19:28:58 UTC, bachmeier wrote:

 On Monday, 30 January 2017 at 16:53:34 UTC, Joseph Rushton 
 Wakeling wrote:

 Yea, I'm a little sad to see the apparent lack of 
 feedback/interest :-\


 I have interest, but as I've never heard of Snap before, I have 
 no comments. Others may be in the same boat. Installation of D 
 compilers is probably not an issue for most of us that read 
 these messages.


Same for me. It'd be great to have simpler and more widespread 
distribution of D and D built applications. But I don't have 
enough background to comment on Snap per se.

--Jon
Jan 30 2017
prev sibling parent reply qznc <qznc web.de> writes:
On Monday, 30 January 2017 at 16:53:34 UTC, Joseph Rushton 
Wakeling wrote:

 Snap packaging started as something Ubuntu were developing for 
 their use-case, but started gaining cross-distro interest last 
 year (probably because AFAICT its feature-set and simplicity of 
 use is quite a bit ahead of alternatives like Flatpak).


I just tried FlatPak and Snap. Snap is actually useable.

FlatPak might be superior technology with its sandboxing, but I'm 
no expert. FlatPak has no central repository, which makes it 
unuseable for the layman at the moment. It feels like the PPA 
situation in Ubuntu.

Snap has stuff like pulseaudio (audio daemon) or ogre (rendering 
engine) in the repo. That seems weird, because these are not 
applications. The first belongs into the base system, the second 
into a development environment.
Feb 03 2017
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Friday, 3 February 2017 at 08:51:38 UTC, qznc wrote:

 I just tried FlatPak and Snap. Snap is actually useable.


One of the first things that struck me about snap packaging was 
the ease of its syntax and how straightforward it was to get 
things working.  I actually started creating snap packages as an 
easy way to build and install D compilers and other tools on my 
own system.  It's both simpler and cleaner than doing those 
builds by hand or writing custom scripts.

The fact that it's then trivially easy to share the results with 
everyone else is the cherry on the cake :-)


 FlatPak might be superior technology with its sandboxing, but 
 I'm no expert. FlatPak has no central repository, which makes 
 it unuseable for the layman at the moment. It feels like the 
 PPA situation in Ubuntu.


I'm no expert on sandboxing either, but snapcraft certainly also 
provides confinement -- and to be honest my impression is it does 
so in a more effective and flexible way than Flatpak.  Some of 
the non-Ubuntu distros may however be disabling the confinement 
right now as some issues with different confinement systems are 
worked through.


 Snap has stuff like pulseaudio (audio daemon) or ogre 
 (rendering engine) in the repo. That seems weird, because these 
 are not applications. The first belongs into the base system, 
 the second into a development environment.


I think this is by design.  For example, it makes perfect sense 
that a daemon like pulseaudio might live in an isolated, confined 
environment where only applications that specifically need access 
are allowed to talk to it.  Probably its presence in the store is 
in order to allow people to explore working with it in that way.

I can't speak to ogre, but I don't think the use-cases of snap 
packaging are intended to be limited to applications in the long 
run.  It also makes sense that a development library could be 
provided via a read-only filesystem, so that while anyone might 
_read_ it to build or run a program, it couldn't be overwritten 
by another application.

For an example of a library snap, I believe some core KDE 
frameworks are being distributed as a snap in their own right, 
which other KDE application snaps can then depend on.
Feb 03 2017
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 01/30/2017 09:40 AM, qznc wrote:

 Is it legally possible to distribute DMD this way? Afaik only dlang.org
 is allowed to distribute it publically due to the backend licence issue.


I don't understand where people keep getting that idea. It very clearly 
states that all you need is to ask permission. It's always been that 
way, and no reasonable request (or any at all to my knowledge) has ever 
been denied.
Jan 30 2017
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 30 January 2017 at 19:07:03 UTC, Nick Sabalausky wrote:

 I don't understand where people keep getting that idea. It very 
 clearly states that all you need is to ask permission. It's 
 always been that way, and no reasonable request (or any at all 
 to my knowledge) has ever been denied.


Yea, I don't see there being any particular issue here beyond 
that permission needs to be sought and obtained.

In case it wasn't obvious,  Walter: I'm asking ;-)
Jan 30 2017
parent reply Joakim <dlang joakim.fea.st> writes:
On Monday, 30 January 2017 at 23:47:50 UTC, Joseph Rushton 
Wakeling wrote:

 On Monday, 30 January 2017 at 19:07:03 UTC, Nick Sabalausky 
 wrote:

 I don't understand where people keep getting that idea. It 
 very clearly states that all you need is to ask permission. 
 It's always been that way, and no reasonable request (or any 
 at all to my knowledge) has ever been denied.


 Yea, I don't see there being any particular issue here beyond 
 that permission needs to be sought and obtained.

 In case it wasn't obvious,  Walter: I'm asking ;-)


He can't read every forum thread, you should email him.  That's 
what I did when I got his permission to put dmd in FreeBSD ports.
Jan 30 2017
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 31 January 2017 at 05:41:38 UTC, Joakim wrote:

 He can't read every forum thread, you should email him.  That's 
 what I did when I got his permission to put dmd in FreeBSD 
 ports.


Yes, I understand that, and I was going to do so anyway.  But I 
was interested in any case in some more general 
discussion/feedback.
Jan 31 2017
prev sibling parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 29 January 2017 at 20:07:50 UTC, Joseph Rushton 
Wakeling wrote:

 I started by trying to snap LDC, mainly because the cmake build 
 system made for a very easy integration with the snapcraft 
 package-build system.  The LDC developers have been kind enough 
 to accept this as an official contribution, and a first 
 submission is currently waiting for review in the Ubuntu snap 
 store, based on the following package definition:
 https://github.com/ldc-developers/ldc2.snap


This LDC snap is now published in the official snap store.  See:
https://forum.dlang.org/thread/rkxyvsmgwhfkigybjpig forum.dlang.org

... for more details.
Feb 03 2017