www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Proposal: DPM, the D Package Manager

reply "Nathan M. Swan" <nathanmswan gmail.com> writes:
Here's my proposal for a D Package Manager. I want to make sure 
it has a good design before I write any code, so here it is:

http://wiki.dlang.org/User:Nathan_M._Swan/DPM_Proposal

I know there are many gaps in this, I want to see which are the 
most important to fill.

I plan to make a central CPR in addition to the dpm program.

Thanks in advance for your feedback!

NMS
Jan 02 2013
next sibling parent reply "David Nadlinger" <see klickverbot.at> writes:
On Wednesday, 2 January 2013 at 21:02:51 UTC, Nathan M. Swan 
wrote:

 Here's my proposal for a D Package Manager. I want to make sure 
 it has a good design before I write any code, so here it is:

 http://wiki.dlang.org/User:Nathan_M._Swan/DPM_Proposal


Sorry for being a bit blunt, but: What advantages would this 
provide over just using the package management facilities of 
one's favorite distro? Keep in mind: http://xkcd.com/927/ ;)

The biggest problem one faces when trying to package D libraries 
right now is that the different D compilers (DMD, GDC, LDC), but 
also multiple versions of the _same_ compiler, are not binary 
compatible to each other. So, if you don't want to use a build 
tool like rdmd to build *everything* from scratch each time, you 
need to manage multiple binaries around for any given library, 
one for each compiler version.

As it is actually quite common to have multiple D compiler 
versions installed side-by-side – e.g. DMD for fast iteration and 
GDC/LDC for release builds, or transitioning one's projects one 
by one to a new DMD version after the release – I think that any 
design which doesn't address this issue isn't worth the effort 
over just using an existing solution.

It might be a good idea to have an extensive look at the popular 
package management systems of other languages like Ruby and Node 
the issue, particularly the ability to easily "freeze" an 
environment for deployment (see npm, bundler). The way Ruby 
(resp. rvm/rbenv) handle the binary compatibility issue, though, 
always seemed a bit backwards to me, which is probably a result 
of needing to retrofit the feature over an already popular 
package manager.

David
Jan 02 2013
next sibling parent Marco Nembrini <marco.nembrini.co gmail.com> writes:
On 03.01.2013 09:33, David Nadlinger wrote:


 Sorry for being a bit blunt, but: What advantages would this provide
 over just using the package management facilities of one's favorite
 distro? Keep in mind: http://xkcd.com/927/ ;)


If you want to distribute your software or library to multiple distros 
or OSes, you would only need to learn how to work with a future dpm 
instead of learning the packaging system of all your different supported 
distributions. And the competing standards should not be a problem if 
the community decides to endorse on particular way of doing things 
(provided it's compatible with other approaches).


 The biggest problem one faces when trying to package D libraries right
 now is that the different D compilers (DMD, GDC, LDC), but also multiple
 versions of the _same_ compiler, are not binary compatible to each
 other. So, if you don't want to use a build tool like rdmd to build
 *everything* from scratch each time, you need to manage multiple
 binaries around for any given library, one for each compiler version.

 As it is actually quite common to have multiple D compiler versions
 installed side-by-side – e.g. DMD for fast iteration and GDC/LDC for
 release builds, or transitioning one's projects one by one to a new DMD
 version after the release – I think that any design which doesn't
 address this issue isn't worth the effort over just using an existing
 solution.


Doesn't this mean a D package manager is needed to deal with this problem?


-- 

Marco Nembrini
Jan 02 2013
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-01-02 23:33, David Nadlinger wrote:


 Sorry for being a bit blunt, but: What advantages would this provide
 over just using the package management facilities of one's favorite
 distro? Keep in mind: http://xkcd.com/927/ ;)


There is no package manager for Windows or Mac OS X (I'm not counting 
the App Store) out of the box.

They are usually to cumbersome to create packages for and most of them 
rely on make, which is just horrible.

-- 
/Jacob Carlborg
Jan 03 2013
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2013-01-02 22:02, Nathan M. Swan wrote:

 Here's my proposal for a D Package Manager. I want to make sure it has a
 good design before I write any code, so here it is:

 http://wiki.dlang.org/User:Nathan_M._Swan/DPM_Proposal

 I know there are many gaps in this, I want to see which are the most
 important to fill.

 I plan to make a central CPR in addition to the dpm program.


This is what I'm working on:

https://github.com/jacob-carlborg/orbit/wiki/Orbit-Package-Manager-for-D

Yes, I will change the package scripts to be written in D.

-- 
/Jacob Carlborg
Jan 02 2013
prev sibling next sibling parent Marco Nembrini <marco.nembrini.co gmail.com> writes:
On 03.01.2013 08:02, Nathan M. Swan wrote:

 Here's my proposal for a D Package Manager. I want to make sure it has a
 good design before I write any code, so here it is:

 http://wiki.dlang.org/User:Nathan_M._Swan/DPM_Proposal

 I know there are many gaps in this, I want to see which are the most
 important to fill.

 I plan to make a central CPR in addition to the dpm program.

 Thanks in advance for your feedback!

 NMS


First off, thanks for the effort. I think a package manager is a great 
idea. However it's also a huge undertaking, are you sure you want to do 
that from scratch?
Maybe it's possible to adapt an existing system to D's needs?

I'm not convinced that the advantage of having all your "support" tools 
rewritten in D is enough to always justify writing them from zero, 
unless they also improve on existing software.


I really like how pip/virtualenv works in python, maybe they can provide 
some inspiration?

Also I think it would be a good idea to have a list of user cases to 
guide the design, so that first it's decided how dpm should work, and 
then an API and implementation is done. Should i just post a few here or 
add them to the talk page or discussion page on the wiki?


-- 

Marco Nembrini
Jan 02 2013
prev sibling next sibling parent "Michael" <pr m1xa.com> writes:
Windows have NuGet for Visual Studio or console client.

It is difficult to create package for each OS. In this case we 
need a script that make a package for each OS or a one maintainer.

Or we need a one tool(script written in D) for the D (I mean D - 
the D Platform) and website with list of packages. Source code on 
bitbucket or github, full package available on repository for D 
Platform. Distribution via torrents or cdn.

I prefer second one.
Jan 03 2013
prev sibling parent reply "Kevin Kane" <kevin.kane gmail.com> writes:
I don't know if this thread is dead but...

I have some comments/suggestions because I am considering 
implementing something as well and if you want some help I'd be 
willing.  My thought would be to model it on Perl's CPAN.  A 
package should be a bare bones gzip of the source, a makefile, 
some unit tests, a way to identify depends, a readme and some 
kind of license identifier.  A client side application 
(hopefully) (eventually) would be  packaged with the various D 
compilers would provide a shell/command line system for saying:

dpm install <package>, it fetches the latest gzip, and proceeds 
to compile for the architecture that you are on.  Installing it 
someplace local for you to import at a later time.

It would probably be nice if we could secure a 'contrib' 
namespace for this.  So individuals would go and submit their 
work to some repository of packages, next time people run dpm it 
would let them know that a new package list is available and 
using these packages in your D programs becomes as simple as 
doing this:

dpm install some.package
<bunch of downloading/compiling/installing happens>

then in your project where you want to use this thing you add:

import contrib.some.package; and voila its done.

What I like about this approach:
   1) its simple/easy/there is already a working pattern that this 
can be modeled after
   2) it actually helps different OSes maintainers create packages 
because you've already done a fair bit of work for them.
   3) it could provide a community supported repository of D code, 
that encourages contribution and puts useful things in a single 
place making D development that much better of an experience.
   4) its cross platform without having to really worry to much 
about it

Just some thoughts from a D newb who is really loving the 
language but wish I could just install user contributed modules 
from a repository and contribute back some of the things that I 
am working on.
Jul 08 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 8 July 2013 at 20:46:21 UTC, Kevin Kane wrote:

 I don't know if this thread is dead but...


Have you considered dub? It seems like current main bet as D 
package manager
http://registry.vibed.org/
Jul 08 2013
parent "w0rp" <devw0rp gmail.com> writes:
On Monday, 8 July 2013 at 20:51:09 UTC, Dicebot wrote:

 On Monday, 8 July 2013 at 20:46:21 UTC, Kevin Kane wrote:

 I don't know if this thread is dead but...


 Have you considered dub? It seems like current main bet as D 
 package manager
 http://registry.vibed.org/


With more adoption, dub can be become D's answer to Pythons pip, 
if it isn't already.
Jul 09 2013