www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Make dub part of the standard dmd distribution

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Let's make this part of 2.068:

https://issues.dlang.org/show_bug.cgi?id=14636

It's preapproved. Who would want to work on it?


Thanks,

Andrei
May 31 2015
next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


Please declare a standard unix location for D 'includes'. Nobody
agrees where in the filesystem D files should be.
I use /usr/include/d2/ for my stuff (I saw it precedented a few times
before, but it doesn't seem that great), but I want a standard place
that stuff bundled by linux package managers can agree on.

As for dub, I'd use it if it worked like a package manager; dub get
libcurl-d libqt-d zlib-d libsdl2-d etc
I have no use for it as a build system, and therefore it's expression
of dependencies is no use to me. I just want something that works the
same way as '-dev' packages already work perfectly well in linux, that
is, they fetch headers and libs, and put them in a standard location
that all the tooling can find.
May 31 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.


I thought it does that.

If dub doesn't allow me to type one command to download and install all 
I need about a package, we need to add that pronto. I consider it a 
dealbreaker.


Andrei
May 31 2015
parent reply "Brad Anderson" <eco gnuk.net> writes:
On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:

 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; 
 dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's 
 expression
 of dependencies is no use to me. I just want something that 
 works the
 same way as '-dev' packages already work perfectly well in 
 linux, that
 is, they fetch headers and libs, and put them in a standard 
 location
 that all the tooling can find.


 I thought it does that.

 If dub doesn't allow me to type one command to download and 
 install all I need about a package, we need to add that pronto. 
 I consider it a dealbreaker.


 Andrei


dub fetch does this already (though probably not quite what you 
are thinking of). You'd need to specify the paths manually 
because if it installed them to the global compiler paths we'd 
have dependency hell (what if 5 projects I have need 3 different 
versions of a library?). Also, you'd need root permissions.

That's not really how you use dub though. dub simply isn't a good 
fit for people who want it to be a system package manager. Its 
goals are different. If people want that they should work on 
getting libraries added to their preferred system's package 
registries.

With dub you specify the dependencies in the dub config file, not 
in some obscure section of an INSTALL file as a command the users 
need to run. You can checkout a project using dub and with a 
single command have dub download and build all the dependencies 
(and their dependencies) and then build your project against them.

dub is about making it easy for 99% of users. If you need your 
own build system then using dub just to download packages is 
overkill. Use git submodules or add something to do a download of 
your dependencies from github as part of your custom build system.
May 31 2015
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/31/15 10:05 PM, Brad Anderson wrote:

 dub fetch does this already (though probably not quite what you are
 thinking of). You'd need to specify the paths manually because if it
 installed them to the global compiler paths we'd have dependency hell
 (what if 5 projects I have need 3 different versions of a library?).
 Also, you'd need root permissions.


Usability is key here. Installing stuff should really be as simple as 
pasting a command. That's how Flash won over Java applets etc. etc.

I installed a Python program the other day. I think it's rather obscure 
- it's called "lcinvestor". I got from knowing nothing about how to 
install Python stuff to having the application running in minutes.

The installation page at https://pypi.python.org/pypi/lcinvestor 
mentions at a point:

====
The easiest way to install is with pip:

sudo pip install lcinvestor
====

So I ran that. It didn't find pip. So I googled for it (first hit if you 
google for "pip"). Essentially I downloaded the script get-pip.py and 
then ran:

python get-pip.py

That worked nicely. Then I tried running pip again, it worked, and boom, 
I was up and running.

But wait, there's more. I then wanted to install lcinvestor on a machine 
I didn't have sudo access to. Sure enough, searching for things like 
``pip without sudo'' took me to 
http://kazhack.org/?post/2014/12/12/pip-gem-install-without-sudo where I 
figured all I had to do:

pip install --local lcinvestor

Boom, I was set - the installation target was ~/.local/bin/. Sure enough 
that was in PATH and all so the program just worked.

That's the benchmark, folks. We need to get as good or better.


 That's not really how you use dub though. dub simply isn't a good fit
 for people who want it to be a system package manager. Its goals are
 different. If people want that they should work on getting libraries
 added to their preferred system's package registries.


Then we need a different program than dub. "dip" :o).


 With dub you specify the dependencies in the dub config file, not in
 some obscure section of an INSTALL file as a command the users need to
 run. You can checkout a project using dub and with a single command have
 dub download and build all the dependencies (and their dependencies) and
 then build your project against them.


That seems reasonable.


 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.


That seems reasonable, too.

So are we there yet?


Andrei
May 31 2015
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 01:21 AM, Andrei Alexandrescu wrote:

 I installed a Python program the other day. I think it's rather obscure
 - it's called "lcinvestor". I got from knowing nothing about how to
 install Python stuff to having the application running in minutes.


That doesn't even remotely match my experience trying to use/install 
python-based software/libs:

https://semitwist.com/articles/article/view/why-i-hate-python-or-any-dynamic-language-really
May 31 2015
prev sibling next sibling parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:

 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.



 I thought it does that.

 If dub doesn't allow me to type one command to download and install all I
 need about a package, we need to add that pronto. I consider it a
 dealbreaker.


 Andrei



 dub fetch does this already (though probably not quite what you are thinking
 of). You'd need to specify the paths manually because if it installed them
 to the global compiler paths we'd have dependency hell (what if 5 projects I
 have need 3 different versions of a library?). Also, you'd need root
 permissions.


Yeah, but regardless, that's what I want.
I don't have version hell with C libs distributed this way...? Is this
a problem that people are specifically trying to avoid?



 That's not really how you use dub though. dub simply isn't a good fit for
 people who want it to be a system package manager. Its goals are different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.


Right, so, someone decide a path, we'll write it on dlang.org, and
then everyone will agree and fall in line :)



 With dub you specify the dependencies in the dub config file, not in some
 obscure section of an INSTALL file as a command the users need to run. You
 can checkout a project using dub and with a single command have dub download
 and build all the dependencies (and their dependencies) and then build your
 project against them.


I get it, it sounds great... if your app suits the model.
I have no D-only projects, all my programs combine many languages and
ecosystems.
There are also existing build systems that are well established that I
prefer to use, integrate with IDE's, etc.

I don't mind if people use dub, but I just want a way to fetch libs
that the compilers will then find automatically.



 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.


Point is, I don't have to do this with C. I just install the dev
package, once, and I'm done. Package manager distributes updates
automatically, everything it exactly how I want it.
It's just not a wheel I have any interest in reinventing.

I'm not trying to heckle. I just want someone to declare the word on
where in the filesystem we put .d files, parallel to c includes.
I think the single most important thing is for bindings against C libs
that are installed by common -dev packages, it would be easiest if the
bindings were fetched and installed the same way as the libs they
refer to.
May 31 2015
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 1/06/2015 5:57 p.m., Manu via Digitalmars-d wrote:

 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:

 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.



 I thought it does that.

 If dub doesn't allow me to type one command to download and install all I
 need about a package, we need to add that pronto. I consider it a
 dealbreaker.


 Andrei



 dub fetch does this already (though probably not quite what you are thinking
 of). You'd need to specify the paths manually because if it installed them
 to the global compiler paths we'd have dependency hell (what if 5 projects I
 have need 3 different versions of a library?). Also, you'd need root
 permissions.


 Yeah, but regardless, that's what I want.
 I don't have version hell with C libs distributed this way...? Is this
 a problem that people are specifically trying to avoid?



 That's not really how you use dub though. dub simply isn't a good fit for
 people who want it to be a system package manager. Its goals are different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.


 Right, so, someone decide a path, we'll write it on dlang.org, and
 then everyone will agree and fall in line :)



 With dub you specify the dependencies in the dub config file, not in some
 obscure section of an INSTALL file as a command the users need to run. You
 can checkout a project using dub and with a single command have dub download
 and build all the dependencies (and their dependencies) and then build your
 project against them.


 I get it, it sounds great... if your app suits the model.
 I have no D-only projects, all my programs combine many languages and
 ecosystems.
 There are also existing build systems that are well established that I
 prefer to use, integrate with IDE's, etc.

 I don't mind if people use dub, but I just want a way to fetch libs
 that the compilers will then find automatically.



 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.


 Point is, I don't have to do this with C. I just install the dev
 package, once, and I'm done. Package manager distributes updates
 automatically, everything it exactly how I want it.
 It's just not a wheel I have any interest in reinventing.

 I'm not trying to heckle. I just want someone to declare the word on
 where in the filesystem we put .d files, parallel to c includes.
 I think the single most important thing is for bindings against C libs
 that are installed by common -dev packages, it would be easiest if the
 bindings were fetched and installed the same way as the libs they
 refer to.


Perhaps a new dub command (--locate)?

$ dub build <package> --version=<version> --arch=<arch> 
--compiler=<compiler> --locate
Base: ~/.dub/<package>-<version>/
Binary: ~/.dub/<package>-<version>/output.lib
ImportPath: ~/.dub/<package>-<version>/source
Pass

Well you get the general gist. Fetches and builds if necessary for 
specific arch ext.
Really it just minimizes output and only output location info.
May 31 2015
prev sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:

 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d

 That's not really how you use dub though. dub simply isn't a good fit for
 people who want it to be a system package manager. Its goals are different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.


 Right, so, someone decide a path, we'll write it on dlang.org, and
 then everyone will agree and fall in line :)


Not sure how serious/joking you are about that, but when has declaring a 
standard whatever like that ever worked for anything ever? ;) Linux 
can't even agree with Linux on where things should go, apparently that's 
why C on linux has pkg-config.
May 31 2015
next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 1 Jun 2015 08:45, "Nick Sabalausky via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:

 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d

 That's not really how you use dub though. dub simply isn't a good fit






for

 people who want it to be a system package manager. Its goals are






different.

 If people want that they should work on getting libraries added to their
 preferred system's package registries.



 Right, so, someone decide a path, we'll write it on dlang.org, and
 then everyone will agree and fall in line :)


 Not sure how serious/joking you are about that, but when has declaring a


standard whatever like that ever worked for anything ever? ;) Linux can't
even agree with Linux on where things should go, apparently that's why C on
linux has pkg-config.


Leave it to the distribution is the safe option in my experience.  To have
something along the lines of what Manu wants, I guess we need something
like virtualenv to allow building in a clean/standard environment.



 May 31 2015



  parent reply "Adam D. Ruppe" <destructionator gmail.com>  writes:


Writing to something that requires root is just doing it wrong 
IMO, unless the user specifically opts in to it.

/usr/anything is thus wrong, it should all be done locally. My 
preference is actually right in the current directory, do it on a 
per-project level. But if you must go higher, the user's home 
directory is also acceptable.



 Jun 01 2015



  parent  "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>  writes:


On Monday, 1 June 2015 at 13:03:28 UTC, Adam D. Ruppe wrote:

 Writing to something that requires root is just doing it wrong 
 IMO, unless the user specifically opts in to it.

 /usr/anything is thus wrong, it should all be done locally. My 
 preference is actually right in the current directory, do it on 
 a per-project level. But if you must go higher, the user's home 
 directory is also acceptable.


I believe Manu was referring to (system) package managers. Those 
that are run by users (like rubygems/bundle, pip, dub) must of 
course never write outside the users' home directory.



 Jun 01 2015



prev sibling next sibling parent  Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 1 Jun 2015 08:45, "Nick Sabalausky via Digitalmars-d"
 <digitalmars-d puremagic.com> wrote:

 On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:

 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d

 That's not really how you use dub though. dub simply isn't a good fit
 for

 people who want it to be a system package manager. Its goals are
 different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.



 Right, so, someone decide a path, we'll write it on dlang.org, and
 then everyone will agree and fall in line :)


 Not sure how serious/joking you are about that, but when has declaring a
 standard whatever like that ever worked for anything ever? ;) Linux can't
 even agree with Linux on where things should go, apparently that's why C on
 linux has pkg-config.


 Leave it to the distribution is the safe option in my experience.  To have
 something along the lines of what Manu wants, I guess we need something like
 virtualenv to allow building in a clean/standard environment.


Yeah, I think I can see 2 parallel problems here.
1. There is a lib installed from a -dev package managed by the
distribution... I just want the complementary .d files. (this is what
I actually care about)
2. There is some open-source D code which isn't distributed as a
binary, it's just a git repo and you fetch it and build it locally. (I
find that I rarely need this, so I don't have much opinion on this
case)

For case 1, my preference would be a distro managed package alongside
the lib itself, and installed into a standard location. If dub could
pull the bindings I want and put them in some common location, fine.
For case 2... I dunno. What if you offer a lib that falls into case 2;
source is available, user can fetch and build against it locally, but
it contains C code too? dub isn't exactly a build system which can
express a complex build environment.
I can't create a dub package for my engine, which might be of interest
to D gamedevs.



 Jun 01 2015



prev sibling  parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 1 Jun 2015 09:09, "Manu via Digitalmars-d" <digitalmars-d puremagic.com>
wrote:

 On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 1 Jun 2015 08:45, "Nick Sabalausky via Digitalmars-d"
 <digitalmars-d puremagic.com> wrote:

 On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:

 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d

 That's not really how you use dub though. dub simply isn't a good fit
 for

 people who want it to be a system package manager. Its goals are
 different.
 If people want that they should work on getting libraries added to










their

 preferred system's package registries.



 Right, so, someone decide a path, we'll write it on dlang.org, and
 then everyone will agree and fall in line :)


 Not sure how serious/joking you are about that, but when has declaring






a

 standard whatever like that ever worked for anything ever? ;) Linux






can't

 even agree with Linux on where things should go, apparently that's why






C on

 linux has pkg-config.


 Leave it to the distribution is the safe option in my experience.  To




have

 something along the lines of what Manu wants, I guess we need something




like

 virtualenv to allow building in a clean/standard environment.


 Yeah, I think I can see 2 parallel problems here.
 1. There is a lib installed from a -dev package managed by the
 distribution... I just want the complementary .d files. (this is what
 I actually care about)
 2. There is some open-source D code which isn't distributed as a
 binary, it's just a git repo and you fetch it and build it locally. (I
 find that I rarely need this, so I don't have much opinion on this
 case)

 For case 1, my preference would be a distro managed package alongside
 the lib itself, and installed into a standard location. If dub could
 pull the bindings I want and put them in some common location, fine.
 For case 2... I dunno. What if you offer a lib that falls into case 2;
 source is available, user can fetch and build against it locally, but
 it contains C code too? dub isn't exactly a build system which can
 express a complex build environment.
 I can't create a dub package for my engine, which might be of interest
 to D gamedevs.


In one sense this can be solved at the distribution level.  If dub was
provided through your package manager, the package maintainers can ensure
that dub was configured to understand where all system sources are located
(or will be located).

Can all of dubs default settings be dumped to json and be overriden?



 Jun 01 2015



 next sibling parent reply "Dicebot" <public dicebot.lv>  writes:


On Monday, 1 June 2015 at 07:19:28 UTC, Iain Buclaw wrote:

 In one sense this can be solved at the distribution level.  If 
 dub was
 provided through your package manager, the package maintainers 
 can ensure
 that dub was configured to understand where all system sources 
 are located
 (or will be located).

 Can all of dubs default settings be dumped to json and be 
 overriden?


The way I do it right now is simply adding -I/usr/include/dlang 
-I/usr/include/dlang/<compiler> to dmd.conf / ldc.conf / patch 
GDC in a similar way.

And surely if dub would ever attempt to write anything in /usr on 
its own I will simply throw it away from Arch packages.



 Jun 01 2015



  parent reply "Wyatt" <wyatt.epp gmail.com>  writes:


On Monday, 1 June 2015 at 11:14:43 UTC, Dicebot wrote:

 The way I do it right now is simply adding -I/usr/include/dlang 
 -I/usr/include/dlang/<compiler> to dmd.conf / ldc.conf / patch 
 GDC in a similar way.

 And surely if dub would ever attempt to write anything in /usr 
 on its own I will simply throw it away from Arch packages.


This whole conversation reminds me a lot of the perl ecosystem.  
How does Arch deal with CPAN?

-Wyatt



 Jun 01 2015



  parent reply "Dicebot" <public dicebot.lv>  writes:


On Monday, 1 June 2015 at 15:25:13 UTC, Wyatt wrote:

 On Monday, 1 June 2015 at 11:14:43 UTC, Dicebot wrote:

 The way I do it right now is simply adding 
 -I/usr/include/dlang -I/usr/include/dlang/<compiler> to 
 dmd.conf / ldc.conf / patch GDC in a similar way.

 And surely if dub would ever attempt to write anything in /usr 
 on its own I will simply throw it away from Arch packages.


 This whole conversation reminds me a lot of the perl ecosystem.
  How does Arch deal with CPAN?

 -Wyatt


Same as pip and gems and all similar kind of crap. Bad. That is 
why I don't use them.

Though it is hard to invent a package management for interpreted 
language that is not insane. It is much better with native ones 
because runtime dependencies are separated from development 
dependencies.



 Jun 01 2015



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2015-06-01 18:56, Dicebot wrote:


 Though it is hard to invent a package management for interpreted
 language that is not insane. It is much better with native ones because
 runtime dependencies are separated from development dependencies.


Rubygems supports development dependencies.

-- 
/Jacob Carlborg



 Jun 02 2015



  parent reply "Dicebot" <public dicebot.lv>  writes:


On Tuesday, 2 June 2015 at 07:02:50 UTC, Jacob Carlborg wrote:

 On 2015-06-01 18:56, Dicebot wrote:


 Though it is hard to invent a package management for 
 interpreted
 language that is not insane. It is much better with native 
 ones because
 runtime dependencies are separated from development 
 dependencies.


 Rubygems supports development dependencies.


You must have though of different "development dependencies". I 
don't mean tools and stuff like that but all sorts of small 
static libraries (in native compilation world). With scripting 
language this unavoidably propagates on the end user unless you 
merge all deps into your sources inline for distribution (which 
no one does).



 Jun 02 2015



  parent  Jacob Carlborg <doob me.com>  writes:


On 2015-06-02 20:54, Dicebot wrote:


 You must have though of different "development dependencies". I don't
 mean tools and stuff like that but all sorts of small static libraries
 (in native compilation world). With scripting language this unavoidably
 propagates on the end user unless you merge all deps into your sources
 inline for distribution (which no one does).


Oh, you mean like that.

-- 
/Jacob Carlborg



 Jun 02 2015



prev sibling  parent  =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 01.06.2015 um 09:19 schrieb Iain Buclaw via Digitalmars-d:

 On 1 Jun 2015 09:09, "Manu via Digitalmars-d"
 <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
  >
  > On 1 June 2015 at 16:54, Iain Buclaw via Digitalmars-d
  > <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
  > >
  > > On 1 Jun 2015 08:45, "Nick Sabalausky via Digitalmars-d"
  > > <digitalmars-d puremagic.com <mailto:digitalmars-d puremagic.com>>
 wrote:
  > >>
  > >> On 06/01/2015 01:57 AM, Manu via Digitalmars-d wrote:
  > >>>
  > >>> On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
  > >>>>
  > >>>> That's not really how you use dub though. dub simply isn't a
 good fit
  > >>>> for
  > >>>>
  > >>>> people who want it to be a system package manager. Its goals are
  > >>>> different.
  > >>>> If people want that they should work on getting libraries added
 to their
  > >>>> preferred system's package registries.
  > >>>
  > >>>
  > >>> Right, so, someone decide a path, we'll write it on dlang.org
 <http://dlang.org>, and
  > >>> then everyone will agree and fall in line :)
  > >>>
  > >>
  > >> Not sure how serious/joking you are about that, but when has
 declaring a
  > >> standard whatever like that ever worked for anything ever? ;)
 Linux can't
  > >> even agree with Linux on where things should go, apparently that's
 why C on
  > >> linux has pkg-config.
  > >>
  > >
  > > Leave it to the distribution is the safe option in my experience.
 To have
  > > something along the lines of what Manu wants, I guess we need
 something like
  > > virtualenv to allow building in a clean/standard environment.
  >
  > Yeah, I think I can see 2 parallel problems here.
  > 1. There is a lib installed from a -dev package managed by the
  > distribution... I just want the complementary .d files. (this is what
  > I actually care about)
  > 2. There is some open-source D code which isn't distributed as a
  > binary, it's just a git repo and you fetch it and build it locally. (I
  > find that I rarely need this, so I don't have much opinion on this
  > case)
  >
  > For case 1, my preference would be a distro managed package alongside
  > the lib itself, and installed into a standard location. If dub could
  > pull the bindings I want and put them in some common location, fine.
  > For case 2... I dunno. What if you offer a lib that falls into case 2;
  > source is available, user can fetch and build against it locally, but
  > it contains C code too? dub isn't exactly a build system which can
  > express a complex build environment.
  > I can't create a dub package for my engine, which might be of interest
  > to D gamedevs.

 In one sense this can be solved at the distribution level.  If dub was
 provided through your package manager, the package maintainers can
 ensure that dub was configured to understand where all system sources
 are located (or will be located).

 Can all of dubs default settings be dumped to json and be overriden?


It would be possible to put a ".dub/settings.json" with custom settings 
into the distribution package (hmm, /etc/dub/settings.json or similar 
should probably supported in addition to that...).



 Jun 04 2015



prev sibling next sibling parent  Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 1 Jun 2015 07:57, "Manu via Digitalmars-d" <digitalmars-d puremagic.com>
wrote:

 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:

 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.



 I thought it does that.

 If dub doesn't allow me to type one command to download and install






all I

 need about a package, we need to add that pronto. I consider it a
 dealbreaker.


 Andrei



 dub fetch does this already (though probably not quite what you are




thinking

 of). You'd need to specify the paths manually because if it installed




them

 to the global compiler paths we'd have dependency hell (what if 5




projects I

 have need 3 different versions of a library?). Also, you'd need root
 permissions.


 Yeah, but regardless, that's what I want.
 I don't have version hell with C libs distributed this way...? Is this
 a problem that people are specifically trying to avoid?



 That's not really how you use dub though. dub simply isn't a good fit




for

 people who want it to be a system package manager. Its goals are




different.

 If people want that they should work on getting libraries added to their
 preferred system's package registries.


 Right, so, someone decide a path, we'll write it on dlang.org, and
 then everyone will agree and fall in line :)



 With dub you specify the dependencies in the dub config file, not in




some

 obscure section of an INSTALL file as a command the users need to run.




You

 can checkout a project using dub and with a single command have dub




download

 and build all the dependencies (and their dependencies) and then build




your

 project against them.


 I get it, it sounds great... if your app suits the model.
 I have no D-only projects, all my programs combine many languages and
 ecosystems.
 There are also existing build systems that are well established that I
 prefer to use, integrate with IDE's, etc.

 I don't mind if people use dub, but I just want a way to fetch libs
 that the compilers will then find automatically.


Just to be clear, libs are source libraries, right?


 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.


 Point is, I don't have to do this with C. I just install the dev
 package, once, and I'm done. Package manager distributes updates
 automatically, everything it exactly how I want it.
 It's just not a wheel I have any interest in reinventing.


This is a feature of your distribution, and not the language itself.  I'm
having talks with the Debian toolchain maintainer, we want to start
shipping D programs and libraries with Debian/Ubuntu.

Binary libraries are going to be the most interesting problem here because
dmd and ldc will be shut out from using them.

This is a semi call-out to the ldc devs, we should really align our ABIs
together.



 Jun 01 2015



prev sibling next sibling parent  Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 1 June 2015 at 17:09, Iain Buclaw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On 1 Jun 2015 07:57, "Manu via Digitalmars-d" <digitalmars-d puremagic.com>
 wrote:

 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:

 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.



 I thought it does that.

 If dub doesn't allow me to type one command to download and install all
 I
 need about a package, we need to add that pronto. I consider it a
 dealbreaker.


 Andrei



 dub fetch does this already (though probably not quite what you are
 thinking
 of). You'd need to specify the paths manually because if it installed
 them
 to the global compiler paths we'd have dependency hell (what if 5
 projects I
 have need 3 different versions of a library?). Also, you'd need root
 permissions.


 Yeah, but regardless, that's what I want.
 I don't have version hell with C libs distributed this way...? Is this
 a problem that people are specifically trying to avoid?



 That's not really how you use dub though. dub simply isn't a good fit
 for
 people who want it to be a system package manager. Its goals are
 different.
 If people want that they should work on getting libraries added to their
 preferred system's package registries.


 Right, so, someone decide a path, we'll write it on dlang.org, and
 then everyone will agree and fall in line :)



 With dub you specify the dependencies in the dub config file, not in
 some
 obscure section of an INSTALL file as a command the users need to run.
 You
 can checkout a project using dub and with a single command have dub
 download
 and build all the dependencies (and their dependencies) and then build
 your
 project against them.


 I get it, it sounds great... if your app suits the model.
 I have no D-only projects, all my programs combine many languages and
 ecosystems.
 There are also existing build systems that are well established that I
 prefer to use, integrate with IDE's, etc.

 I don't mind if people use dub, but I just want a way to fetch libs
 that the compilers will then find automatically.


 Just to be clear, libs are source libraries, right?


Well, it should support libs that aren't JUST source. Binary libs
needs some .d files to declare the API and all that, so at least some
source would always present. A lib on its own us no use :)
But any solution needs to support closed-source libraries too. Not
everything can be conveniently distributed as full-source, and many
libs are bindings against binary C libs which would probably be
sourced externally.



 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.


 Point is, I don't have to do this with C. I just install the dev
 package, once, and I'm done. Package manager distributes updates
 automatically, everything it exactly how I want it.
 It's just not a wheel I have any interest in reinventing.


 This is a feature of your distribution, and not the language itself.  I'm
 having talks with the Debian toolchain maintainer, we want to start shipping
 D programs and libraries with Debian/Ubuntu.


Do want! :)


 Binary libraries are going to be the most interesting problem here because
 dmd and ldc will be shut out from using them.


?
Why is?


 This is a semi call-out to the ldc devs, we should really align our ABIs
 together.


Oh, are the LDC/GDC ABI's not consistent on linux?
Surely many/most libs are just references to C '-dev' packages as dependencies?



 Jun 01 2015



prev sibling next sibling parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 1 June 2015 at 09:37, Manu via Digitalmars-d <digitalmars-d puremagic.com

 wrote:



 On 1 June 2015 at 17:09, Iain Buclaw via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 1 Jun 2015 07:57, "Manu via Digitalmars-d" <


 digitalmars-d puremagic.com>

 wrote:

 On 1 June 2015 at 15:05, Brad Anderson via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu wrote:

 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's










 expression

 of dependencies is no use to me. I just want something that works










 the

 same way as '-dev' packages already work perfectly well in linux,










 that

 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.



 I thought it does that.

 If dub doesn't allow me to type one command to download and install








 all

 I
 need about a package, we need to add that pronto. I consider it a
 dealbreaker.


 Andrei



 dub fetch does this already (though probably not quite what you are
 thinking
 of). You'd need to specify the paths manually because if it installed
 them
 to the global compiler paths we'd have dependency hell (what if 5
 projects I
 have need 3 different versions of a library?). Also, you'd need root
 permissions.


 Yeah, but regardless, that's what I want.
 I don't have version hell with C libs distributed this way...? Is this
 a problem that people are specifically trying to avoid?



 That's not really how you use dub though. dub simply isn't a good fit
 for
 people who want it to be a system package manager. Its goals are
 different.
 If people want that they should work on getting libraries added to






 their

 preferred system's package registries.


 Right, so, someone decide a path, we'll write it on dlang.org, and
 then everyone will agree and fall in line :)



 With dub you specify the dependencies in the dub config file, not in
 some
 obscure section of an INSTALL file as a command the users need to run.
 You
 can checkout a project using dub and with a single command have dub
 download
 and build all the dependencies (and their dependencies) and then build
 your
 project against them.


 I get it, it sounds great... if your app suits the model.
 I have no D-only projects, all my programs combine many languages and
 ecosystems.
 There are also existing build systems that are well established that I
 prefer to use, integrate with IDE's, etc.

 I don't mind if people use dub, but I just want a way to fetch libs
 that the compilers will then find automatically.


 Just to be clear, libs are source libraries, right?


 Well, it should support libs that aren't JUST source. Binary libs
 needs some .d files to declare the API and all that, so at least some
 source would always present. A lib on its own us no use :)
 But any solution needs to support closed-source libraries too. Not
 everything can be conveniently distributed as full-source, and many
 libs are bindings against binary C libs which would probably be
 sourced externally.



 dub is about making it easy for 99% of users. If you need your own






 build

 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.


 Point is, I don't have to do this with C. I just install the dev
 package, once, and I'm done. Package manager distributes updates
 automatically, everything it exactly how I want it.
 It's just not a wheel I have any interest in reinventing.


 This is a feature of your distribution, and not the language itself.  I'm
 having talks with the Debian toolchain maintainer, we want to start


 shipping

 D programs and libraries with Debian/Ubuntu.


 Do want! :)


 Binary libraries are going to be the most interesting problem here


 because

 dmd and ldc will be shut out from using them.


 ?
 Why is?


 This is a semi call-out to the ldc devs, we should really align our ABIs
 together.


 Oh, are the LDC/GDC ABI's not consistent on linux?
 Surely many/most libs are just references to C '-dev' packages as
 dependencies?


Our druntimes need to be aligned up.  The first thing you'd notice is that
each may give undefined references to things the other compiler does not
include in it's runtime.



 Jun 01 2015



  parent  "Dicebot" <public dicebot.lv>  writes:


On Monday, 1 June 2015 at 07:42:02 UTC, Iain Buclaw wrote:

 Our druntimes need to be aligned up.  The first thing you'd 
 notice is that
 each may give undefined references to things the other compiler 
 does not
 include in it's runtime.


Unifying GDC/LDC ABI would simplify packager life a lot.



 Jun 01 2015



prev sibling next sibling parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Mon, 01 Jun 2015 05:05:20 +0000, Brad Anderson wrote:


 dub is about making it easy for 99% of users. If you need your own build
 system then using dub just to download packages is overkill. Use git
 submodules or add something to do a download of your dependencies from
 github as part of your custom build system.


now you killed dub for "normal" GNU/Linux programmer.

dub as a build tool sux. no need to discuss that, it simply sux.

and now you killed the only sensible way to use it: as package manager.

that's why, for example, i never considered adding anything to dub=20
repository: that repository is completely useless for me. and other=20
people who aren't use dub to build their projects will ignore dub=20
repository too.=



 Jun 01 2015



 next sibling parent reply "extrawurst" <stephan extrawurst.org>  writes:


On Monday, 1 June 2015 at 10:24:46 UTC, ketmar wrote:

 On Mon, 01 Jun 2015 05:05:20 +0000, Brad Anderson wrote:


 dub is about making it easy for 99% of users. If you need your 
 own build
 system then using dub just to download packages is overkill. 
 Use git
 submodules or add something to do a download of your 
 dependencies from
 github as part of your custom build system.


 now you killed dub for "normal" GNU/Linux programmer.

 dub as a build tool sux. no need to discuss that, it simply sux.


why do you think it sucks ?


 and now you killed the only sensible way to use it: as package 
 manager.

 that's why, for example, i never considered adding anything to 
 dub
 repository: that repository is completely useless for me. and 
 other
 people who aren't use dub to build their projects will ignore 
 dub
 repository too.





 Jun 01 2015



 next sibling parent reply "Atila Neves" <atila.neves gmail.com>  writes:


 dub as a build tool sux. no need to discuss that, it simply 
 sux.


 why do you think it sucks ?


dub is a good package manager, but it's not a good build system. 
For small projects that just want to list dependencies and have 
it build, it's sufficient. Any larger project will have needs 
that won't be easily solved by using dub. Which is why I wrote my 
own.

Atila



 Jun 01 2015



  parent reply "extrawurst" <stephan extrawurst.org>  writes:


On Monday, 1 June 2015 at 12:24:18 UTC, Atila Neves wrote:

 dub as a build tool sux. no need to discuss that, it simply 
 sux.


 why do you think it sucks ?


 dub is a good package manager, but it's not a good build 
 system. For small projects that just want to list dependencies 
 and have it build, it's sufficient. Any larger project will 
 have needs that won't be easily solved by using dub. Which is 
 why I wrote my own.

 Atila


Hi Atila,

you are talking about cross language builds ?
Other than that limitation, what do you miss ?
I have a couple of rather sizable projects running with dub 
without problems and one could argue vibe.d is a pretty big 
project too which works fine.

The D community already suffered enough from not uniting 
workforces behind at least SOME standards (hint at tango), a 
plethora of options for newcomers to choose from for basics like 
building is also daunting IMHO.

Stephan



 Jun 01 2015



 next sibling parent  "Atila Neves" <atila.neves gmail.com>  writes:


On Monday, 1 June 2015 at 12:34:47 UTC, extrawurst wrote:

 dub is a good package manager, but it's not a good build 
 system. For small projects that just want to list dependencies 
 and have it build, it's sufficient. Any larger project will 
 have needs that won't be easily solved by using dub. Which is 
 why I wrote my own.

 Atila


 Hi Atila,

 you are talking about cross language builds ?


That too.


 Other than that limitation, what do you miss ?


Everything I could do (and have) in cmake. Here are two really 
simple examples:

1) Automatically run a script to generate the file containing 
"main" before doing the build, with proper dependency tracking
2) Concurrently building release and unittest builds

I've worked on complicated build systems before, I know the kind 
of thing I'd need.


 I have a couple of rather sizable projects running with dub 
 without problems and one could argue vibe.d is a pretty big 
 project too which works fine.


I wouldn't consider vibe.d to be a large project.


 The D community already suffered enough from not uniting 
 workforces behind at least SOME standards (hint at tango), a 
 plethora of options for newcomers to choose from for basics 
 like building is also daunting IMHO.


I fully support dub being the default package manager and for 
most people/projects it is a good enough build system. That's why 
my build system builds on dub instead of wanting to replace it. 
It leaves the package dependency part to dub and focusses on 
build descriptions that can leverage dub's existing knowledge of 
the build.

Atila



 Jun 01 2015



prev sibling  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 08:34 AM, extrawurst wrote:

 On Monday, 1 June 2015 at 12:24:18 UTC, Atila Neves wrote:

 dub as a build tool sux. no need to discuss that, it simply sux.


 why do you think it sucks ?


 dub is a good package manager, but it's not a good build system. For
 small projects that just want to list dependencies and have it build,
 it's sufficient. Any larger project will have needs that won't be
 easily solved by using dub. Which is why I wrote my own.

 Atila


 Hi Atila,

 you are talking about cross language builds ?
 Other than that limitation, what do you miss ?


Unlike rdmd, dub always tries to compile all sources regardless of 
whether they're actually imported. This has been a constant source of 
problems for me, including breakage of conditional importing under 
various circumstances.



 Jun 01 2015



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2015-06-01 20:39, Nick Sabalausky wrote:


 Unlike rdmd, dub always tries to compile all sources regardless of
 whether they're actually imported. This has been a constant source of
 problems for me, including breakage of conditional importing under
 various circumstances.


Isn't that the only way to compile it if the project is a library? 
without having to use workarounds like create a new module which imports 
all other modules.

-- 
/Jacob Carlborg



 Jun 01 2015



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/02/2015 02:53 AM, Jacob Carlborg wrote:

 On 2015-06-01 20:39, Nick Sabalausky wrote:


 Unlike rdmd, dub always tries to compile all sources regardless of
 whether they're actually imported. This has been a constant source of
 problems for me, including breakage of conditional importing under
 various circumstances.


 Isn't that the only way to compile it if the project is a library?
 without having to use workarounds like create a new module which imports
 all other modules.


My libs always have a "package.d" or equivalent anyway, so I don't find 
it to be any sort of "workaround".

Even without that, rdmd-style still works if the lib is being used as a 
source lib rather than a pre-compiled static (or dynamic) lib.

But yea, if you're doing a precompiled lib and you don't want any sort 
of "root" module (don't see it as all that big of a deal though), then 
yea I guess you'd have to collect the list of sources dub-style instead 
of rdmd-style.



 Jun 02 2015



prev sibling  parent  ketmar <ketmar ketmar.no-ip.org>  writes:


On Mon, 01 Jun 2015 11:59:12 +0000, extrawurst wrote:


 dub as a build tool sux. no need to discuss that, it simply sux.


=20
 why do you think it sucks ?


'cause it's not more than a simple script in it's core. it can't do good=20
conditional builds, it can't track dependencies for generated files or=20
non-D code, and so on. it's "rdmd with some bells and some features=20
stripped".

this is fine for simple pure D projects, but for complex projects one=20
have to use supplement build tool. and if we have *two* build tools in=20
use, it's logical to remove one that is more tied to the language. and=20
it's dub.

so while it's ok to use dub to build simple things, it's not a good build=20
system. my k8jam, while can't do "consolidated" (i.e. non-separate ;-)=20
builds, can build D code with C/C++ libraries (which are built from=20
source too), generate files, track dependencies, do 'configure' work,=20
allows "dlang.require" operators with dependency tracking between=20
libraries and so on. it can't download libraries, though -- i choose to=20
not implement that, hoping that dub can fulfill that role.=



 Jun 02 2015



prev sibling  parent reply "Adam D. Ruppe" <destructionator gmail.com>  writes:


On Monday, 1 June 2015 at 10:24:46 UTC, ketmar wrote:

 and now you killed the only sensible way to use it: as package 
 manager.


I don't care for it as a package manager either because it is too 
complex. As you know, I like to do my things as individual files. 
I don't go through the pain of making a new github repo with a 
gigantic folder layout for every little thing.

One of the things I really like about D is how modules are 
encapsulated into individual files. The code is there, the 
documentation is there, the tests are there. No separate headers 
or anything else.


The only package management scheme I'd even consider using is one 
that has a per-file option... and even then, I probably wouldn't 
bother because individual files are so easy to manage anyway that 
downloading a new program to use them doesn't exceed the 
"existing option is good enough for me" threshold.


But even if I don't use it myself, it would be kinda nice if the 
repository could handle my files just so I can toss it up there 
and not have to answer questions about that anymore. I'd be 
willing to add them all individually to the repo, even listing 
the dependencies manually if I had to (tip: make this a web form 
that I can just submit to add stuff to it, no json upload).... 
but I'm not willing to separate my little repo into 25 other 
repos and separate my folder into 125 other folders to be added 
to it.



 Jun 01 2015



 next sibling parent reply Rikki Cattermole <alphaglosined gmail.com>  writes:


On 2/06/2015 1:11 a.m., Adam D. Ruppe wrote:

 On Monday, 1 June 2015 at 10:24:46 UTC, ketmar wrote:

 and now you killed the only sensible way to use it: as package manager.


 I don't care for it as a package manager either because it is too
 complex. As you know, I like to do my things as individual files. I
 don't go through the pain of making a new github repo with a gigantic
 folder layout for every little thing.

 One of the things I really like about D is how modules are encapsulated
 into individual files. The code is there, the documentation is there,
 the tests are there. No separate headers or anything else.


 The only package management scheme I'd even consider using is one that
 has a per-file option... and even then, I probably wouldn't bother
 because individual files are so easy to manage anyway that downloading a
 new program to use them doesn't exceed the "existing option is good
 enough for me" threshold.


 But even if I don't use it myself, it would be kinda nice if the
 repository could handle my files just so I can toss it up there and not
 have to answer questions about that anymore. I'd be willing to add them
 all individually to the repo, even listing the dependencies manually if
 I had to (tip: make this a web form that I can just submit to add stuff
 to it, no json upload).... but I'm not willing to separate my little
 repo into 25 other repos and separate my folder into 125 other folders
 to be added to it.


Something I've thought about but haven't suggested up to now is making 
dub-repo to support personal projects.
Basically one repo per person preconfigured if e.g. Github/Bitbucket is 
specified.

github.com/rikkimax/dubcode

Just as an idea for what I'm meaning url wise.

So then when specifying a dependency in dub, its just a matter of saying 
<person>:<subproject>

It would also help with people having there own util libraries all listed.

Of course then you'd only have to dump your code in there and have a dub 
file with each file being its own sub package.

Seems simple to implement, but idk if it is the answer.



 Jun 01 2015



 next sibling parent  "Adam D. Ruppe" <destructionator gmail.com>  writes:


I think the way I'd do it is you still make projects in the repo 
that are referenced like normally, just it refers to a file in a 
specific repo instead of a whole repo.

Maybe using the subpackage feature it can get close enough. I'll 
try using an empty source folder then do the others as a single 
additional file.

But actually, the multiple listing on the repo is the most 
important part to me, these are really individual things, they 
shouldn't be listed as "miscellaneous from me" on the listing...



 Jun 01 2015



prev sibling  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 09:22 AM, Rikki Cattermole wrote:

 Something I've thought about but haven't suggested up to now is making
 dub-repo to support personal projects.
 Basically one repo per person preconfigured if e.g. Github/Bitbucket is
 specified.

 github.com/rikkimax/dubcode

 Just as an idea for what I'm meaning url wise.

 So then when specifying a dependency in dub, its just a matter of saying
 <person>:<subproject>


Already works.

$ git clone https://github.com/rikkimax/dubcode.git path-to-dubcode
$ dub add-local path-to-dubcode

Done.



 Jun 01 2015



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 02:51 PM, Nick Sabalausky wrote:

 On 06/01/2015 09:22 AM, Rikki Cattermole wrote:

 Something I've thought about but haven't suggested up to now is making
 dub-repo to support personal projects.
 Basically one repo per person preconfigured if e.g. Github/Bitbucket is
 specified.

 github.com/rikkimax/dubcode

 Just as an idea for what I'm meaning url wise.

 So then when specifying a dependency in dub, its just a matter of saying
 <person>:<subproject>


 Already works.

 $ git clone https://github.com/rikkimax/dubcode.git path-to-dubcode
 $ dub add-local path-to-dubcode

 Done.


Ehh, to clarify a few things:

$ cd my-project
$ cat dub.json
{
     "name": "my-project",
     "dependencies": {
         "not-in-dub-repo": "~>1.1.0"
     }
}
$ git clone https://github.com/user/not-in-dub-repo.git 
path/to/not-in-dub-repo
$ cat path/to/not-in-dub-repo/dub.json
{
     "name": "not-in-dub-repo"
}
$ dub add-local path/to/not-in-dub-repo



 Jun 01 2015



prev sibling next sibling parent reply "ponce" <contact gam3sfrommars.fr>  writes:


On Monday, 1 June 2015 at 13:11:43 UTC, Adam D. Ruppe wrote:

 The only package management scheme I'd even consider using is 
 one that has a per-file option... and even then, I probably 
 wouldn't bother because individual files are so easy to manage 
 anyway that downloading a new program to use them doesn't 
 exceed the "existing option is good enough for me" threshold.


What happens when you find a bug in your library? Then every user 
has new work to do to merge the changes. With DUB you don't do a 
thing for patch versions in user code. _This provide a way to 
warn your users and make them upgrade_. This is the main interest 
of DUB.

You could use git submodule for that, but not everyone use git, 
and not everyone want to search commit logs for the right version.



 Jun 01 2015



  parent  "Adam D. Ruppe" <destructionator gmail.com>  writes:


On Monday, 1 June 2015 at 14:09:32 UTC, ponce wrote:

 What happens when you find a bug in your library?


Users still have to test bug fixes because what I consider a bug 
they might consider a feature. I dread software updates not 
because downloading a file or applying a patch is hard, but 
because I fear unrelated and/or silent breakage.



 Jun 01 2015



prev sibling  parent  ketmar <ketmar ketmar.no-ip.org>  writes:


On Mon, 01 Jun 2015 13:11:42 +0000, Adam D. Ruppe wrote:


 One of the things I really like about D is how modules are encapsulated
 into individual files. The code is there, the documentation is there,
 the tests are there. No separate headers or anything else.


i like this too.


 But even if I don't use it myself, it would be kinda nice if the
 repository could handle my files just so I can toss it up there and not
 have to answer questions about that anymore. I'd be willing to add them
 all individually to the repo, even listing the dependencies manually if
 I had to (tip: make this a web form that I can just submit to add stuff
 to it, no json upload).... but I'm not willing to separate my little
 repo into 25 other repos and separate my folder into 125 other folders
 to be added to it.


yep. i don't feel like splitting my IV repo to alot of individual=20
repositories. quite the contrary, i'm moving modules from separate=20
repositories to IV when they are ready. some of IV things are perfectly=20
usable with vanilla, so they can go to code.dlang.org, but... but there=20
is no way to do that now.=



 Jun 02 2015



prev sibling  parent  "Morbid.Obesity" <Morbid.Obesity mail.com>  writes:


On Monday, 1 June 2015 at 05:05:21 UTC, Brad Anderson wrote:

 On Monday, 1 June 2015 at 04:36:06 UTC, Andrei Alexandrescu 
 wrote:

 On 5/31/15 8:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; 
 dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's 
 expression
 of dependencies is no use to me. I just want something that 
 works the
 same way as '-dev' packages already work perfectly well in 
 linux, that
 is, they fetch headers and libs, and put them in a standard 
 location
 that all the tooling can find.


 I thought it does that.

 If dub doesn't allow me to type one command to download and 
 install all I need about a package, we need to add that 
 pronto. I consider it a dealbreaker.


 Andrei


 dub fetch does this already (though probably not quite what you 
 are thinking of). You'd need to specify the paths manually 
 because if it installed them to the global compiler paths we'd 
 have dependency hell (what if 5 projects I have need 3 
 different versions of a library?). Also, you'd need root 
 permissions.


I wonder if essentially hashing a library(ultimately each file) 
could be used to avoid dll hell. Essentially hash the .d files 
and then different versions will have different hashes. When you 
compile your project and it is successful, the compiler/linker 
can write a configuration file that includes the hashes of the 
various files used. This then could be used to avoid dll, if one 
has a proper way to "check" for versioning(online database) then 
one can find the proper versions.

Obviously small changes to any d file, even if it is non-breaking 
will change the hash. This is not a bad thing but since there 
will be no correlation between hashes and versions one must have 
a database of versions and their hashes.

The good thing is that if, say, a project doesn't compile after 
updating a "library" to the "wrong" versions, the configuration 
file won't be modified since the compiler won't be able to 
compile the code. One can then easily use the (old) config file, 
if it exists from a previous (working) build, to deal with the 
versioning issue. (e.g., hash is different for mylib.d, means 
wrong version. Get correct version)

In fact, maybe the way to go is for the config file essentially 
to keep a history of several last working builds, e.g.,

05/31/15 - mylib.d[3FAB9999ABCCC34C3C] - Compiled OK
          - myotherlib.d[4C934DFFFFFFF34C3C] - Compiled OK
          - myproject.d [FFABC33CCA4367C4EE] - Compiled OK
          - Project [3493493428439294292] - Compiled OK

06/01/15 - mylib.d[3FAB9999ABCCC34C3C] - Compiled OK
          - myotherlib.d[4C954D5F3F2F434EEC] - Compiled OK
          - myproject.d [FFABC33CCA4367C4EE] - Failed
          - Project [3493493428439294292] - Failed

In the above case, the myotherlib.d's hash changed. the project 
did not compile. Hence we have a versioning issue. It is easy to 
see what the problem is. One then simply needs an online datable 
to match the hashes to the versions or simply search all 
"versions" and hash them.

Also, one benefits from simply hashing any file to get it's 
"version". If you had 10 myotherlib.d files from the past but 
don't know which is which(different copies scattered across 
different backups, etc), simply hashing them and finding the 
correct one(in this case, 4C934DFFFFFFF34C3C) can be used to get 
the project working again.

Since most of this stuff can be automated, it seems like it would 
be a better way to deal with versioning than the standard methods 
which relies on users correctly defining the versions and 
maintain them.



 Jun 01 2015



prev sibling next sibling parent reply "weaselcat" <weaselcat gmail.com>  writes:


On Monday, 1 June 2015 at 03:48:31 UTC, Manu wrote:

 On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff (I saw it precedented a few 
 times
 before, but it doesn't seem that great), but I want a standard 
 place
 that stuff bundled by linux package managers can agree on.

 As for dub, I'd use it if it worked like a package manager; dub 
 get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's 
 expression
 of dependencies is no use to me. I just want something that 
 works the
 same way as '-dev' packages already work perfectly well in 
 linux, that
 is, they fetch headers and libs, and put them in a standard 
 location
 that all the tooling can find.


run dub fetch --help



 May 31 2015



  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 1 June 2015 at 14:57, weaselcat via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 On Monday, 1 June 2015 at 03:48:31 UTC, Manu wrote:

 On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?



 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff (I saw it precedented a few times
 before, but it doesn't seem that great), but I want a standard place
 that stuff bundled by linux package managers can agree on.

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.



 run dub fetch --help


Interesting. I'm amazed this never came up before in discussion...
I've talked about this so many times.
So, DMD/LDC/GDC know where to look to find these packages? What
happens if the package includes a binary lib?

That that, I still want someone to declare an official path for D
'includes' in the *nix filesystem, so D lib packages have somewhere to
install...



 May 31 2015



 next sibling parent  "weaselcat" <weaselcat gmail.com>  writes:


On Monday, 1 June 2015 at 05:27:00 UTC, Manu wrote:

 On 1 June 2015 at 14:57, weaselcat via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 1 June 2015 at 03:48:31 UTC, Manu wrote:

 On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?



 Please declare a standard unix location for D 'includes'. 
 Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff (I saw it precedented a 
 few times
 before, but it doesn't seem that great), but I want a 
 standard place
 that stuff bundled by linux package managers can agree on.

 As for dub, I'd use it if it worked like a package manager; 
 dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's 
 expression
 of dependencies is no use to me. I just want something that 
 works the
 same way as '-dev' packages already work perfectly well in 
 linux, that
 is, they fetch headers and libs, and put them in a standard 
 location
 that all the tooling can find.



 run dub fetch --help


 Interesting. I'm amazed this never came up before in 
 discussion...
 I've talked about this so many times.
 So, DMD/LDC/GDC know where to look to find these packages? What
 happens if the package includes a binary lib?


dub uses git to manage packages, it keeps the list of D packages 
on http://code.dlang.org/

the help is a bit unintuitive, it just gives a brief overview 
with dub --help, you have to issue a subcommand to get the help 
about it.


 That that, I still want someone to declare an official path for 
 D
 'includes' in the *nix filesystem, so D lib packages have 
 somewhere to
 install...


the problem is that XDG didn't really define a standard for 
user-level libraries and binaries, so it's a huge mess.

Dub is actually violating part of the XDG standard as is because 
it defaults to ~/.dub
http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html



 May 31 2015



prev sibling next sibling parent  "Atila Neves" <atila.neves gmail.com>  writes:


On Monday, 1 June 2015 at 05:27:00 UTC, Manu wrote:

 On 1 June 2015 at 14:57, weaselcat via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On Monday, 1 June 2015 at 03:48:31 UTC, Manu wrote:

 On 1 June 2015 at 09:01, Andrei Alexandrescu via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?



 Please declare a standard unix location for D 'includes'. 
 Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff (I saw it precedented a 
 few times
 before, but it doesn't seem that great), but I want a 
 standard place
 that stuff bundled by linux package managers can agree on.

 As for dub, I'd use it if it worked like a package manager; 
 dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's 
 expression
 of dependencies is no use to me. I just want something that 
 works the
 same way as '-dev' packages already work perfectly well in 
 linux, that
 is, they fetch headers and libs, and put them in a standard 
 location
 that all the tooling can find.



 run dub fetch --help


 Interesting. I'm amazed this never came up before in 
 discussion...
 I've talked about this so many times.
 So, DMD/LDC/GDC know where to look to find these packages? What
 happens if the package includes a binary lib?


No. But dub knows where the packages are, and the usual way to 
use it is to use dub to compile. If not, there's always `dub 
describe`, which will tell you exactly where the dependencies are.

Atila


 That that, I still want someone to declare an official path for 
 D
 'includes' in the *nix filesystem, so D lib packages have 
 somewhere to
 install...





 May 31 2015



prev sibling  parent reply Johannes Pfau <nospam example.com>  writes:


Am Mon, 1 Jun 2015 15:26:51 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:


 So, DMD/LDC/GDC know where to look to find these packages?


IIRC no.


 What
 happens if the package includes a binary lib?
 
 That that, I still want someone to declare an official path for D
 'includes' in the *nix filesystem, so D lib packages have somewhere to
 install...


GDC looks in /usr/include/d2. Dicebot changed that
to /usr/include/d/gdc for archlinux and there are some reasons why this
is (for now) necessary:

Sharing D sources/headers between compilers is possible, but there are
some headers (druntime,phobos) which can be compiler specific. As long
as these are not in a common import path it's not a problem though.

The main problem is we don't have ABI compatibility. This means we can't
share the libraries between compilers. So we could make 'import foo'
work but linking with -lfoo without manually adjusting linker paths is
not possible.

There are some solutions, all have drawbacks:
* Have compiler specific directories for libraries
  (/usr/lib/gdc/libvibed.a, /usr/lib/dmd/libvibed.a). Only works for
  static libraries. Can only have one version of a library installed
* have per-library directories. I guess dub uses this. Can have
  multiple versions of the same library. Drawback: compiler can't know
  the correct library path.[1]
* Shared libraries should be installed in a common location (/usr/lib)
  anyway. This is only possible if we have ABI compatibility[2].
  Library versioning is limited to the standard C/C++ like versioning.

I guess the main reason why we can't have a C/C++ like out of the box
experience is ABI compatibility. There'll always be some quirks as long
as we don't fix this.


[1] you could actually build the directory structure logic into the
compilers. Do something like dmd --library=vibe.d:0.7.23 where this
argument sets up correct import paths and linking. I think .NET uses
something similar (GAC). With ABI compatible compilers we might even
make this work for shared libraries. Could be implemented as a small
tool calling the compiler. You could also do something like this:

Create a package format: .dlib (simply a renamed .tar.xz)
Contents:
   - PACKAGEINFO //meta information (version)
   - include/*   //include files
   - doc/*       //documentation in standard format for IDEs
   - lib/dmd/libfoo.a
   - lib/gdc/libfoo.a

Write a small tool xdmd. Implement:
   //simply move to /usr/share/dlib/libfoo/version/
   //or ~/.share/dlib/libfoo/version/
   xdmd --install libfoo.dlib
   Note: when installing using a systems package manager, packages
   could be installed in the same way: pacman -S libfoo could call xdmd
   --install.

   //extract /usr/share/dlib/libfoo/0.9.3/libfoo.dlib
   //to temporary directory. Pass link and include flags and
   //rest of arguments to dmd
   xdmd --library=foo:0.9.3 test.d
   //directly use local package
   xdmd --library=libfoo-0.9.3.dlib test.d

This is essentially are very slimmed down dub. Would be a nice weekend
exercise to write such a tool.

[2] or you use one compiler exclusively.



 Jun 01 2015



 next sibling parent  Jacob Carlborg <doob me.com>  writes:


On 2015-06-01 20:03, Johannes Pfau wrote:


 Create a package format: .dlib (simply a renamed .tar.xz)
 Contents:
     - PACKAGEINFO //meta information (version)
     - include/*   //include files
     - doc/*       //documentation in standard format for IDEs
     - lib/dmd/libfoo.a
     - lib/gdc/libfoo.a

 Write a small tool xdmd. Implement:
     //simply move to /usr/share/dlib/libfoo/version/
     //or ~/.share/dlib/libfoo/version/
     xdmd --install libfoo.dlib
     Note: when installing using a systems package manager, packages
     could be installed in the same way: pacman -S libfoo could call xdmd
     --install.

     //extract /usr/share/dlib/libfoo/0.9.3/libfoo.dlib
     //to temporary directory. Pass link and include flags and
     //rest of arguments to dmd
     xdmd --library=foo:0.9.3 test.d
     //directly use local package
     xdmd --library=libfoo-0.9.3.dlib test.d

 This is essentially are very slimmed down dub. Would be a nice weekend
 exercise to write such a tool.


So, a package manager ;). I tried that, but people were complaining that 
I used Ruby for the meta information and that binary packages would 
cause problems. Then Dub came and I gave up on that project.

-- 
/Jacob Carlborg



 Jun 01 2015



prev sibling next sibling parent  Mathias Lang via Digitalmars-d <digitalmars-d puremagic.com>  writes:


2015-06-01 20:03 GMT+02:00 Johannes Pfau via Digitalmars-d <
digitalmars-d puremagic.com>:


 The main problem is we don't have ABI compatibility. This means we can't
 share the libraries between compilers. So we could make 'import foo'
 work but linking with -lfoo without manually adjusting linker paths is
 not possible.

 There are some solutions, all have drawbacks:
 * Have compiler specific directories for libraries
   (/usr/lib/gdc/libvibed.a, /usr/lib/dmd/libvibed.a). Only works for
   static libraries. Can only have one version of a library installed
 * have per-library directories. I guess dub uses this. Can have
   multiple versions of the same library. Drawback: compiler can't know
   the correct library path.[1]
 * Shared libraries should be installed in a common location (/usr/lib)
   anyway. This is only possible if we have ABI compatibility[2].
   Library versioning is limited to the standard C/C++ like versioning.

 I guess the main reason why we can't have a C/C++ like out of the box
 experience is ABI compatibility. There'll always be some quirks as long
 as we don't fix this.


Glad to see that post !

But in addition of the ABI compatibility between compilers, we would need
to have ABI compatibility between releases of the same major version if we
were to walk the C[++] route, and we do not want that, for various reason,
the most obvious one being that you won't be able to add an attribute like
 nogc / nothrow to a free function without doing a major release.

There is good thing about C way of doing thing, but there is also some bad.
For example, you can't have multiple versions of the same library installed
- you don't need it -, but D definitely needs that. That also leads to
major releases ending up in a different package - think qt4 / qt5 -. So,
while C use SemVer for ABI compatibility, we should use SemVer for source
compatibility, or we'll end up with an unmanageable mess.



 Jun 02 2015



prev sibling  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 2 June 2015 at 04:03, Johannes Pfau via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Am Mon, 1 Jun 2015 15:26:51 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:


 So, DMD/LDC/GDC know where to look to find these packages?


 IIRC no.


 What
 happens if the package includes a binary lib?

 That that, I still want someone to declare an official path for D
 'includes' in the *nix filesystem, so D lib packages have somewhere to
 install...


 GDC looks in /usr/include/d2. Dicebot changed that
 to /usr/include/d/gdc for archlinux and there are some reasons why this
 is (for now) necessary:

 Sharing D sources/headers between compilers is possible, but there are
 some headers (druntime,phobos) which can be compiler specific. As long
 as these are not in a common import path it's not a problem though.

 The main problem is we don't have ABI compatibility. This means we can't
 share the libraries between compilers. So we could make 'import foo'
 work but linking with -lfoo without manually adjusting linker paths is
 not possible.

 There are some solutions, all have drawbacks:
 * Have compiler specific directories for libraries
   (/usr/lib/gdc/libvibed.a, /usr/lib/dmd/libvibed.a). Only works for
   static libraries. Can only have one version of a library installed
 * have per-library directories. I guess dub uses this. Can have
   multiple versions of the same library. Drawback: compiler can't know
   the correct library path.[1]
 * Shared libraries should be installed in a common location (/usr/lib)
   anyway. This is only possible if we have ABI compatibility[2].
   Library versioning is limited to the standard C/C++ like versioning.

 I guess the main reason why we can't have a C/C++ like out of the box
 experience is ABI compatibility. There'll always be some quirks as long
 as we don't fix this.


I'm inclined to suggest, this would be an excellent focus for 2.068...
The compilers need to become aligned, and ABI issues seems like a
critical issue to focus on.



 Jun 02 2015



  parent  "David Nadlinger" <code klickverbot.at>  writes:


On Wednesday, 3 June 2015 at 01:23:37 UTC, Manu wrote:

 I'm inclined to suggest, this would be an excellent focus for 
 2.068...
 The compilers need to become aligned, and ABI issues seems like 
 a
 critical issue to focus on.


There are much bigger fish to fry at this point. Phobos breaks 
ABI compatibility with every single release anyway.

  - David



 Jun 03 2015



prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 05/31/2015 11:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.


Ditto. Dub's great if you let it be your buildsystem, but as soon as you 
want to use it as a package-manager-only it becomes an uphill battle 
with dub fighting back every step of the way (I speak from experience). 
That problem is worse if your project is a library that you want 
fetchable through dub.

That's why I've been working on some PRs to fix all that. The first step 
is here (building/expanding on a flag another contributer recently added):

https://github.com/D-Programming-Language/dub/pull/572

(Although I think I may need to change --data=libs and separate system 
libs from dependency libs since linkers expect those to be passed in 
differently.)

The next step will be to add a flag to have that data get output in a 
compiler-ready argument list (your choice of compiler, of course). 
That'll let you do something comparable to "pkg-config --cflags --libs".

Then finally, a shortcut flag so you can just do something kinda like:

rdmd --build-only `dub describe --cmdargs=rdmd` src/main.d

And all your dependencies will be properly referenced (-I..., linker 
flags, etc).



 May 31 2015



 next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com>  writes:


On Monday, 1 June 2015 at 06:20:18 UTC, Nick Sabalausky wrote:

 Ditto. Dub's great if you let it be your buildsystem, but as 
 soon as you want to use it as a package-manager-only it becomes 
 an uphill battle with dub fighting back every step of the way 
 (I speak from experience). That problem is worse if your 
 project is a library that you want fetchable through dub.


And this is when Jacob Carlborg chimes in and says "I told you 
so." ;) His favorite complaint about dub has always been that it 
combined package management and the build tool into one.

dub works fantastically if you want to do everything in the 
standard way without any funny stuff, but it clearly doesn't have 
the power of a tool like make, cmake, etc. As soon as you need to 
do anything funny - like include anything related to other 
languages in your build, or put things in a specific layout 
because of some company-specific thing, or anything that wasn't 
explicitly planned for by the folks writing dub, dub just won't 
work.

Now, to be fair, I don't know how you can be pulling in all kinds 
of stray libraries with who-knows-what for their build systems 
and expect it to work very well via a single tool (at minimum, 
you have the problem that the machine that you're trying to build 
them on will likely be missing some of those tools), but that's 
pretty much what we need to handle the general case. Without 
that, dub will work great for the normal case (which _will_ be 
plenty for many, many projects), but it won't work in the general 
case - and corporate folks in particular are probably going to 
have to skip out on using it because of abnormal requirements on 
their part.

Even simply splitting dub out so that it can pull in packages 
without necessarily being able to build them would be nice (maybe 
adding a flag to the dub.json file indicating whether dub can be 
used as a build tool for the project or whether it's supposed to 
just act as a package manager for it), though that does add the 
problem of code.dlang.org becoming more confusing, since you 
wouldn't be able to rely on all projects on it being built for 
you by dub as part of the process of grabbing packages.

It's a complicate problem, and dub went a route which works in 
the 90% case, but doesn't work for more complicated cases, which 
will make dub unusable for some projects - especially corporate 
ones. I don't know how fixable it is without a major redesign of 
dub.

- Jonathan M Davis



 Jun 01 2015



 next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 10:49 AM, Jonathan M Davis wrote:

 Even simply splitting dub out so that it can pull in packages without
 necessarily being able to build them would be nice (maybe adding a flag
 to the dub.json file indicating whether dub can be used as a build tool
 for the project or whether it's supposed to just act as a package
 manager for it), though that does add the problem of code.dlang.org
 becoming more confusing, since you wouldn't be able to rely on all
 projects on it being built for you by dub as part of the process of
 grabbing packages.


I've brought up something like that, but it was met with very strong 
opposition. :(


 It's a complicate problem, and dub went a route which works in the 90%
 case, but doesn't work for more complicated cases, which will make dub
 unusable for some projects - especially corporate ones. I don't know how
 fixable it is without a major redesign of dub.


Personally, I think "90%" is generous here, but I admit that's splitting 
hairs.

Unfortunately, "less than 100%" is a big problem in this case because 
those libs/projects that don't fit the "90%" effectively cannot even 
participate in the dub repository for others to use (at the very LEAST, 
they get pounded with "fails to build via dub"). So NOT using dub to 
build is freaking viral.

And *using* certain dub-based projects inside one which doesn't build 
via dub can be a major pain, as anyone who's tried to use vibe.d from a 
non-dub-based project should be able to attest (just *try* to figure out 
all the -I... and linker flags you need *and* double-check/maintain them 
every time you upgrade vibe.d).

So anything less than 100% effectively splits the whole damn ecosystem. 
NOT acceptable.

Dub's project generation feature is NOT a solution here, because you 
still have to manage builds dub's way, *exactly* as if you were using 
dub's build system, because...really...you still ARE. So all the 
dealbreakers for those 10+% projects still apply equally.

On the plus side though, I *do* think this is basically solvable. The 
most immediate way (and what I'm working towards) is to create a 
dub.json that feeds dub a 'dummy-src/dummy.d' to keep dub's build system 
happy and safely away from the real source files, and then use:

"(pre|post)-build-command-posix": "./my-real-buildscript"
"(pre|post)-build-command-win": "my-real-buildscript"

To trick dub into using a different build tool.

It'd be easily to design/publish a skeleton dub.json to do exactly that. 
(Kinda sucks to NEED that, but at least it'll work, and once it's in 
place them maybe it can be made easier. The non-buildsytem parts of 
dub.json really ARE very trivial, and if someone had a problem with even 
that trivial little bit, then I don't see how they could develop any 
useful software in the first place.)

I'm fairly certain the ONLY real thing still preventing all that above 
from actually working is that dub currently provides no (practical) way 
to retrieve the necessary compiler/linker args in order to actually use 
your dependencies (again, try using vibe.d in, say, an rdmd-based 
project). I say no "practical" way here because "dub describe -> parse 
the JSON -> extract the needed info -> convert the needed info to 
compiler flags" is NOT exactly practical or reasonable to expect people 
to do *just* to use RDMD or otherwise opt-out of dub's build-system.)

Again though, I'm working on that:

https://github.com/D-Programming-Language/dub/pull/572

That lets you directly grab any build info you need from dub. My next PR 
will then let you ask dub to automatically format the output for your 
compiler's CLI. And then one last one to provide convenience switches 
for the most likely configurations (like 
"--just-gimme-whatever-flags-I-need-to-pass-to-rdmd-so-I-can-actually-u
e-my-dependencies", 
but obviously less verbose :))

With all that, any project should be able to participate in the dub 
ecosystem AND opt-out of dub's build system WITHOUT segregating the 
ecosystem in any way.



 Jun 01 2015



  parent reply "Atila Neves" <atila.neves gmail.com>  writes:


 I'm fairly certain the ONLY real thing still preventing all 
 that above from actually working is that dub currently provides 
 no (practical) way to retrieve the necessary compiler/linker 
 args in order to actually use your dependencies (again, try 
 using vibe.d in, say, an rdmd-based project). I say no 
 "practical" way here because "dub describe -> parse the JSON -> 
 extract the needed info -> convert the needed info to compiler 
 flags" is NOT exactly practical or reasonable to expect people 
 to do *just* to use RDMD or otherwise opt-out of dub's 
 build-system.)


My build system already uses the output from `dub describe` to 
produce, well, a build system from it. I designed it with 
multiple backends in mind, right now the ones that I implemented 
are ninja and make.

If not for `dub describe` bugs (I'm working on that), you'd be 
able to point it at any dub project and get make/ninja files for 
it. Or integrate whatever you want with the existing files to 
compile that dub already knows about.

In fact, for any dub project that doesn't use configurations 
(that's the bug), you can do that _right now_.

Atila



 Jun 01 2015



  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 01:43 PM, Atila Neves wrote:

 I'm fairly certain the ONLY real thing still preventing all that above
 from actually working is that dub currently provides no (practical)
 way to retrieve the necessary compiler/linker args in order to
 actually use your dependencies (again, try using vibe.d in, say, an
 rdmd-based project). I say no "practical" way here because "dub
 describe -> parse the JSON -> extract the needed info -> convert the
 needed info to compiler flags" is NOT exactly practical or reasonable
 to expect people to do *just* to use RDMD or otherwise opt-out of
 dub's build-system.)


 My build system already uses the output from `dub describe` to produce,
 well, a build system from it. I designed it with multiple backends in
 mind, right now the ones that I implemented are ninja and make.

 If not for `dub describe` bugs (I'm working on that), you'd be able to
 point it at any dub project and get make/ninja files for it. Or
 integrate whatever you want with the existing files to compile that dub
 already knows about.

 In fact, for any dub project that doesn't use configurations (that's the
 bug), you can do that _right now_.

 Atila


That's certainly cool. I *do* think "dub describe"'s json is an awesome 
thing to have, exactly for those sorts of purposes.

But I do maintain that an ability to easily opt-out of dub's buildsystem 
*needs* to be built-in and should not *require* external tools (though 
external tools should certainly still be possible). And this is even 
more crucial if dub's to be bundled with dmd and rdmd (the latter of 
which dub currently does not play nicely with).



 Jun 01 2015



  parent  "Atila Neves" <atila.neves gmail.com>  writes:


 But I do maintain that an ability to easily opt-out of dub's 
 buildsystem *needs* to be built-in and should not *require* 
 external tools (though external tools should certainly still be 
 possible). And this is even more crucial if dub's to be bundled 
 with dmd and rdmd (the latter of which dub currently does not 
 play nicely with).


Maybe I can manage to get my code into dub, I don't know. I think 
it's extremely useful to separate concerns, which is why I 
started with something new instead of trying to add code to dub. 
Like I said before, for most people its build system is enough. 
When it's not, there's reggae (my tool). I hope, anyway.

The other reason is that I don't want to depend on dub. Maybe I 
have my priorities wrong, who knows.

Atila



 Jun 01 2015



prev sibling next sibling parent  Jacob Carlborg <doob me.com>  writes:


On 2015-06-01 16:49, Jonathan M Davis wrote:


 And this is when Jacob Carlborg chimes in and says "I told you so." ;)
 His favorite complaint about dub has always been that it combined
 package management and the build tool into one.


:)

-- 
/Jacob Carlborg



 Jun 01 2015



prev sibling  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 01.06.2015 um 16:49 schrieb Jonathan M Davis:

 On Monday, 1 June 2015 at 06:20:18 UTC, Nick Sabalausky wrote:

 Ditto. Dub's great if you let it be your buildsystem, but as soon as
 you want to use it as a package-manager-only it becomes an uphill
 battle with dub fighting back every step of the way (I speak from
 experience). That problem is worse if your project is a library that
 you want fetchable through dub.


 And this is when Jacob Carlborg chimes in and says "I told you so." ;)
 His favorite complaint about dub has always been that it combined
 package management and the build tool into one.


This is actually completely irrelevant to the issue. Making the build 
part separate would not at all solve this. Any work on such a separate 
build tool (e.g. to support other languages) could instead as well go 
into DUB itself. Of course you could make the system completely build 
tool agnostic, but then you'd basically lose interoperability between 
packages, as each package might choose its own build tool.


 dub works fantastically if you want to do everything in the standard way
 without any funny stuff, but it clearly doesn't have the power of a tool
 like make, cmake, etc. As soon as you need to do anything funny - like
 include anything related to other languages in your build, or put things
 in a specific layout because of some company-specific thing, or anything
 that wasn't explicitly planned for by the folks writing dub, dub just
 won't work.


This is really not true. There is currently just one limitation w.r.t. 
the directory structure and that is the naming scheme of the folder to 
which packages are downloaded ("somepackage-1.0.0"). There are some 
packages that put their sources into the root of the repository and 
expect to be put into a folder with the name of the repository, which 
will then be added as an import folder. We are going to solve that.

Everything else can be done with the sourcePaths, sourceFiles, 
importPaths, ... settings. You basically have the same possibilities as 
you have when directly invoking the compiler.

Separate language support is still missing, but you always have the 
possibility to add generic "preBuildCommands" or "preGenerateCommands" 
to invoke separate tools or scripts. Of course it's desirable to get 
everything done with the same tool instead of relying on additional 
build time dependencies, but it's still *possible* to do these things.


 Now, to be fair, I don't know how you can be pulling in all kinds of
 stray libraries with who-knows-what for their build systems and expect
 it to work very well via a single tool (at minimum, you have the problem
 that the machine that you're trying to build them on will likely be
 missing some of those tools), but that's pretty much what we need to
 handle the general case. Without that, dub will work great for the
 normal case (which _will_ be plenty for many, many projects), but it
 won't work in the general case - and corporate folks in particular are
 probably going to have to skip out on using it because of abnormal
 requirements on their part.

 Even simply splitting dub out so that it can pull in packages without
 necessarily being able to build them would be nice (maybe adding a flag
 to the dub.json file indicating whether dub can be used as a build tool
 for the project or whether it's supposed to just act as a package
 manager for it), though that does add the problem of code.dlang.org
 becoming more confusing, since you wouldn't be able to rely on all
 projects on it being built for you by dub as part of the process of
 grabbing packages.


I guess using the proper targetType and preBuildCommand settings it 
should already be possible to invoke a separate build tool and still 
integrate the generated object/library files into the rest of the DUB 
build process. But I'd have to try that out - it would definitely be 
good to have something like this added to the cookbook [1].

Of course things like these always potentially degrade interoperability. 
DUB's build description was chosen explicitly to avoid unnecessary 
dependencies on a particular compiler/platform, which is where the most 
general build tools (especially "make") utterly fail.


 It's a complicate problem, and dub went a route which works in the 90%
 case, but doesn't work for more complicated cases, which will make dub
 unusable for some projects - especially corporate ones. I don't know how
 fixable it is without a major redesign of dub.


I don't see anything that wouldn't be fixable. In fact, there are really 
few things that are not possible now, even if not in the most 
comfortable way.



 Jun 04 2015



  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/04/2015 04:39 AM, Sönke Ludwig wrote:

 Of course you could make the system completely build
 tool agnostic, but then you'd basically lose interoperability between
 packages, as each package might choose its own build tool.


With this PR and the upcoming follow-up to add 
--format=(list|some_compiler|whatever), there is NO issue with package 
interoperability (as long as a package's "import-paths" is correct):

https://github.com/D-Programming-Language/dub/pull/572

Note that even WITHOUT dub, packages from different buildsystems 
*already* interop just fine. Even in C-land. All you ever need for 
packages to work together is the appropriate "build this package" 
command and the import/lib paths for the compiler/linker. Buildsystem is 
a package-local issue, not a cross-package issue.



 Jun 04 2015



  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 04.06.2015 um 17:03 schrieb Nick Sabalausky:

 On 06/04/2015 04:39 AM, Sönke Ludwig wrote:

 Of course you could make the system completely build
 tool agnostic, but then you'd basically lose interoperability between
 packages, as each package might choose its own build tool.


 With this PR and the upcoming follow-up to add
 --format=(list|some_compiler|whatever), there is NO issue with package
 interoperability (as long as a package's "import-paths" is correct):

 https://github.com/D-Programming-Language/dub/pull/572


Just to get this straight: Supporting external build tools like that and 
by writing the proper generator has always been the goal of DUB (being 
build tool agnostic). The built-in builder is basically mainly there for 
convenience (although there is no reason not to improve it to the point 
where it is a worthy competitor). But you still use DUB's build 
description in that case.

What I really meant (and didn't really say) was if you completely remove 
all build description related fields from dub.json - then you lose 
interoperability.


 Note that even WITHOUT dub, packages from different buildsystems
 *already* interop just fine. Even in C-land. All you ever need for
 packages to work together is the appropriate "build this package"
 command and the import/lib paths for the compiler/linker. Buildsystem is
 a package-local issue, not a cross-package issue.


That's not what I had in mind with "interoperability". That would rather 
be gluing them together manually. Of course you can always do that, but 
then you lose a lot of potential for avoiding useless/redundant work. It 
means you are doing the job that the tool should (could) do.

This also gets especially interesting and annoying when compiler flags 
(e.g. version identifiers) don't match, or the dependencies require 
advanced compiler flags. I really think C/C++ is the last thing we 
should take as our role model when it comes to this topic (apart from 
C's strict ABI perhaps).



 Jun 04 2015



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/04/2015 11:33 AM, Sönke Ludwig wrote:

 Am 04.06.2015 um 17:03 schrieb Nick Sabalausky:

 On 06/04/2015 04:39 AM, Sönke Ludwig wrote:

 Of course you could make the system completely build
 tool agnostic, but then you'd basically lose interoperability between
 packages, as each package might choose its own build tool.


 With this PR and the upcoming follow-up to add
 --format=(list|some_compiler|whatever), there is NO issue with package
 interoperability (as long as a package's "import-paths" is correct):

 https://github.com/D-Programming-Language/dub/pull/572


 Just to get this straight: Supporting external build tools like that and
 by writing the proper generator has always been the goal of DUB (being
 build tool agnostic). The built-in builder is basically mainly there for
 convenience (although there is no reason not to improve it to the point
 where it is a worthy competitor). But you still use DUB's build
 description in that case.

 What I really meant (and didn't really say) was if you completely remove
 all build description related fields from dub.json - then you lose
 interoperability.


Oh, right. I have no issue with dub *having* a built-in buildsystem. 
(And it is fine and convenient for a lot of things, even if not 
everything.) I've just hit roadblocks trying to use other buildsystems 
instead and am only just now getting around to addressing those issues.



 Jun 04 2015



prev sibling  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 02:20 AM, Nick Sabalausky wrote:

 On 05/31/2015 11:48 PM, Manu via Digitalmars-d wrote:

 As for dub, I'd use it if it worked like a package manager; dub get
 libcurl-d libqt-d zlib-d libsdl2-d etc
 I have no use for it as a build system, and therefore it's expression
 of dependencies is no use to me. I just want something that works the
 same way as '-dev' packages already work perfectly well in linux, that
 is, they fetch headers and libs, and put them in a standard location
 that all the tooling can find.


 Ditto. Dub's great if you let it be your buildsystem, but as soon as you
 want to use it as a package-manager-only it becomes an uphill battle
 with dub fighting back every step of the way (I speak from experience).
 That problem is worse if your project is a library that you want
 fetchable through dub.

 That's why I've been working on some PRs to fix all that. The first step
 is here (building/expanding on a flag another contributer recently added):

 https://github.com/D-Programming-Language/dub/pull/572

 (Although I think I may need to change --data=libs and separate system
 libs from dependency libs since linkers expect those to be passed in
 differently.)

 The next step will be to add a flag to have that data get output in a
 compiler-ready argument list (your choice of compiler, of course).
 That'll let you do something comparable to "pkg-config --cflags --libs".


All the above is now included in the PR:

https://github.com/D-Programming-Language/dub/pull/572

Working example:

------------------------------
$ cd /home/nick/proj/dub

$ dub describe --data=main-source-file --data=options \
--data=versions --data=import-paths
'/home/nick/proj/dub/source/app.d' -debug -g -w -version=DubUseCurl 
-version=Have_dub '-I/home/nick/proj/dub/source/'

$ dub describe --compiler=ldc --data=options --data=versions
-d-debug -g -w -oq -od=.dub/obj -oq -od=.dub/obj -d-version=DubUseCurl 
-d-version=Have_dub

$ dub describe --data-list --data=options --data=versions
debugMode
debugInfo
warningsAsErrors

DubUseCurl
Have_dub
------------------------------

Of course, "dub describe --help" for the full list of options.

Only thing not included yet is this:


 Then finally, a shortcut flag so you can just do something kinda like:

 rdmd --build-only `dub describe --cmdargs=rdmd` src/main.d

 And all your dependencies will be properly referenced (-I..., linker
 flags, etc).





 Jun 05 2015



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/05/2015 12:45 PM, Nick Sabalausky wrote:

 https://github.com/D-Programming-Language/dub/pull/572

 Working example:

 ------------------------------
 $ cd /home/nick/proj/dub

 $ dub describe --data=main-source-file --data=options \
 --data=versions --data=import-paths
 '/home/nick/proj/dub/source/app.d' -debug -g -w -version=DubUseCurl
 -version=Have_dub '-I/home/nick/proj/dub/source/'

 $ dub describe --compiler=ldc --data=options --data=versions
 -d-debug -g -w -oq -od=.dub/obj -oq -od=.dub/obj -d-version=DubUseCurl
 -d-version=Have_dub

 $ dub describe --data-list --data=options --data=versions
 debugMode
 debugInfo
 warningsAsErrors

 DubUseCurl
 Have_dub
 ------------------------------


It also works even if your project doesn't have a dub.json :)

$ mkdir ~/my-proj
$ cd ~/my-proj
$ dub fetch vibe-d
$ dub describe vibe-d --data=versions --data=import-paths
-version=VibeLibeventDriver -version=Have_vibe_d -version=Have_libevent 
-version=Have_openssl 
'-I/home/nick/.dub/packages/vibe-d-0.7.23-beta.1/source/' 
'-I/home/nick/.dub/packages/libevent-2.0.1_2.0.16/' 
'-I/home/nick/.dub/packages/openssl-1.1.4_1.0.1g/'



 Jun 05 2015



prev sibling next sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com>  writes:


On 01/06/2015 04:48, Manu via Digitalmars-d wrote:

 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff (I saw it precedented a few times
 before, but it doesn't seem that great), but I want a standard place
 that stuff bundled by linux package managers can agree on.


If you use DUB to build, isn't this transparent anyways?

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros



 Jun 05 2015



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/05/2015 02:56 PM, Bruno Medeiros wrote:

 On 01/06/2015 04:48, Manu via Digitalmars-d wrote:

 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff (I saw it precedented a few times
 before, but it doesn't seem that great), but I want a standard place
 that stuff bundled by linux package managers can agree on.


 If you use DUB to build, isn't this transparent anyways?


Yes, but dub's buildsystem isn't always suitable for all projects.



 Jun 05 2015



prev sibling  parent reply Marco Leise <Marco.Leise gmx.de>  writes:


Am Mon, 1 Jun 2015 13:48:21 +1000
schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:


 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff [=E2=80=A6].


I would like to mention that 40 out of 62 ppl by the end of
2013 did agreed on /usr/include/dlang :
http://forum.dlang.org/thread/20131112205019.12585bbd marco-leise

That's a few more than "nobody". Today I would include a
"other" choice at least, but it wouldn't have changed the
picture much.

--=20
Marco



 Jun 08 2015



  parent reply Manu via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On 9 June 2015 at 15:52, Marco Leise via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Am Mon, 1 Jun 2015 13:48:21 +1000
 schrieb Manu via Digitalmars-d <digitalmars-d puremagic.com>:


 Please declare a standard unix location for D 'includes'. Nobody
 agrees where in the filesystem D files should be.
 I use /usr/include/d2/ for my stuff […].


 I would like to mention that 40 out of 62 ppl by the end of
 2013 did agreed on /usr/include/dlang :
 http://forum.dlang.org/thread/20131112205019.12585bbd marco-leise

 That's a few more than "nobody". Today I would include a
 "other" choice at least, but it wouldn't have changed the
 picture much.


There was no 'd2' option, which is the only one I've ever encountered.
Is there a convention in use? Does anyone else bother with this? I
just want someone to decide and publish it somewhere... I don't care
what's chosen, but we really need to just agree on something.



 Jun 09 2015



  parent reply "Dicebot" <public dicebot.lv>  writes:


I don't see how it matters as long as packaging is done properly. 
Developer should not ever be allowed to write to /usr anyway.



 Jun 09 2015



  parent  "John Colvin" <john.loughran.colvin gmail.com>  writes:


On Tuesday, 9 June 2015 at 14:09:03 UTC, Dicebot wrote:

 I don't see how it matters as long as packaging is done 
 properly. Developer should not ever be allowed to write to /usr 
 anyway.


+1

I don't expect to be able to rely on the location of anything 
under /usr except maybe /usr/bin/env and I don't see why D should 
be any different.



 Jun 09 2015



prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not 
certain exactly what work he had in mind for that though, versus post-1.0.



 May 31 2015



 next sibling parent  "ponce" <contact gam3sfrommars.fr>  writes:


On Monday, 1 June 2015 at 06:35:38 UTC, Nick Sabalausky wrote:

 On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


 IIRC, I think Sonke wanted to hold off on that until a dub 1.0. 
 I'm not certain exactly what work he had in mind for that 
 though, versus post-1.0.


DUB has worked equally well and almost unchanged for me in 2015.



 May 31 2015



prev sibling  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 01.06.2015 um 08:35 schrieb Nick Sabalausky:

 On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


 IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not
 certain exactly what work he had in mind for that though, versus post-1.0.


We have collected some things that would be good to have done. There are 
no big issues left, though, except for making the API stable and 
supporting the improved build description language. This should be done 
within another compiler release cycle.



 Jun 04 2015



  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/04/2015 04:43 AM, Sönke Ludwig wrote:

 Am 01.06.2015 um 08:35 schrieb Nick Sabalausky:

 On 05/31/2015 07:01 PM, Andrei Alexandrescu wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


 IIRC, I think Sonke wanted to hold off on that until a dub 1.0. I'm not
 certain exactly what work he had in mind for that though, versus
 post-1.0.


 We have collected some things that would be good to have done. There are
 no big issues left, though,


Cool.


 except for making the API stable and
 supporting the improved build description language.


Is there anything you need from me on the latter? (If a particular 
matter quiets down or I get distracted, I'm liable to forget about it.)



 Jun 04 2015



  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 04.06.2015 um 17:08 schrieb Nick Sabalausky:

 On 06/04/2015 04:43 AM, Sönke Ludwig wrote:
(...)

 except for making the API stable and
 supporting the improved build description language.


 Is there anything you need from me on the latter? (If a particular
 matter quiets down or I get distracted, I'm liable to forget about it.)


I think Dicebot did some work recently (or intended to do?). It's 
actually not that much work overall. I could also pick up what's there 
and finalize things over the next days.



 Jun 04 2015



  parent reply "Dicebot" <public dicebot.lv>  writes:


On Thursday, 4 June 2015 at 15:39:02 UTC, Sönke Ludwig wrote:

 Am 04.06.2015 um 17:08 schrieb Nick Sabalausky:

 On 06/04/2015 04:43 AM, Sönke Ludwig wrote:
(...)

 except for making the API stable and
 supporting the improved build description language.


 Is there anything you need from me on the latter? (If a 
 particular
 matter quiets down or I get distracted, I'm liable to forget 
 about it.)


 I think Dicebot did some work recently (or intended to do?). 
 It's actually not that much work overall. I could also pick up 
 what's there and finalize things over the next days.


I got it to the point where I could build simple hello-world 
using dub.sdl description but haven't worked on it since then. 
This issue is neither interesting nor important to me, thus 
finding motivation is rather hard. Dump of last sources can be 
found here : https://github.com/Dicebot/dub/tree/sdl



 Jun 04 2015



  parent  =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 04.06.2015 um 19:55 schrieb Dicebot:

 I got it to the point where I could build simple hello-world using
 dub.sdl description but haven't worked on it since then. This issue is
 neither interesting nor important to me, thus finding motivation is
 rather hard. Dump of last sources can be found here :
 https://github.com/Dicebot/dub/tree/sdl


I'll pick it up from there in a few days.



 Jun 04 2015



prev sibling next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>  writes:


Since this thread seems to have turned into a wishlist for dub 
features, I'm going to add one:

Undeprecate dependencies on git branches. They are really not 
different from dependencies with fuzzy "~>" versions. For both, 
the exact selected version can be stored in dub.selections.json.

Use case:
Making a fix to a third-party project and sharing that fix with 
other team members. Having to create a temporary project on 
code.dlang.org just for that purpose is both a waste of time for 
developers and a waste of resources of community infrastructure.



 Jun 01 2015



 next sibling parent  Jacob Carlborg <doob me.com>  writes:


On 2015-06-01 17:16, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 Since this thread seems to have turned into a wishlist for dub features,
 I'm going to add one:

 Undeprecate dependencies on git branches. They are really not different
 from dependencies with fuzzy "~>" versions. For both, the exact selected
 version can be stored in dub.selections.json.


That's a good point, as long as people understand how to use 
dub.selections.json [1]. But I would rather not allow branches in the 
registry. I would prefer that a git URL could be added directly to 
dub.json instead.

[1] https://github.com/github/gitignore/pull/1444

-- 
/Jacob Carlborg



 Jun 02 2015



prev sibling  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 01.06.2015 um 17:16 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?= 
<schuetzm gmx.net>":

 Since this thread seems to have turned into a wishlist for dub features,
 I'm going to add one:

 Undeprecate dependencies on git branches. They are really not different
 from dependencies with fuzzy "~>" versions. For both, the exact selected
 version can be stored in dub.selections.json.

 Use case:
 Making a fix to a third-party project and sharing that fix with other
 team members. Having to create a temporary project on code.dlang.org
 just for that purpose is both a waste of time for developers and a waste
 of resources of community infrastructure.


They are only deprecated within dub.json. You can still use them within 
dub.selections.json. But really, branch based dependencies are simply a 
broken feature. They are completely different to version ranges, because 
they are not at all comparable (to other branches or versions) - at 
least without completely committing to GIT and starting to walk the GIT 
commit graph.

You can also check out certain branches locally and use "dub add-local" 
or "dub add-path" to let DUB use it instead of the publicly registered one.



 Jun 04 2015



  parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>  writes:


On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote:

 Am 01.06.2015 um 17:16 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?= 
 <schuetzm gmx.net>":

 Since this thread seems to have turned into a wishlist for dub 
 features,
 I'm going to add one:

 Undeprecate dependencies on git branches. They are really not 
 different
 from dependencies with fuzzy "~>" versions. For both, the 
 exact selected
 version can be stored in dub.selections.json.

 Use case:
 Making a fix to a third-party project and sharing that fix 
 with other
 team members. Having to create a temporary project on 
 code.dlang.org
 just for that purpose is both a waste of time for developers 
 and a waste
 of resources of community infrastructure.


 They are only deprecated within dub.json. You can still use 
 them within dub.selections.json.


This is exactly the wrong way round. dub.selections.json must 
only contain exact versions. That's its purpose after all.

See Ruby's bundler as an example:
http://bundler.io/v1.9/rationale.html
Especially the parts about Gemfile.lock, which is their 
equivalent of dub.selections.json.


 But really, branch based dependencies are simply a broken 
 feature. They are completely different to version ranges, 
 because they are not at all comparable (to other branches or 
 versions) - at least without completely committing to GIT and 
 starting to walk the GIT commit graph.


Why would you even want to compare them, except for equality? 
There are only the following cases AFAICS, none of which require 
an order:

1) Package is not yet in dub.selections.json:
=> Clone/update repo, checkout the branch, and put the exact 
commit id into dub.selections.json.
2) Package is already included in dub.selections.json:
=> All fine, nothing needs to be done.
3) An upgrade of the package has been requested:
=> Same as 1)

There could be situations during dependency resolution where you 
want to check whether a given commit is part of a branch, but 
that's a different (and easier) problem.


 You can also check out certain branches locally and use "dub 
 add-local" or "dub add-path" to let DUB use it instead of the 
 publicly registered one.


This is not an option in a larger team. Everyone would have to do 
that manually, or you'd need to write a custom script to do what 
is actually the package manager's task.



 Jun 04 2015



 next sibling parent  Jacob Carlborg <doob me.com>  writes:


On 2015-06-04 12:30, "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>" 
wrote:

 On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote:

 Am 01.06.2015 um 17:16 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?=
 <schuetzm gmx.net>":





 You can also check out certain branches locally and use "dub
 add-local" or "dub add-path" to let DUB use it instead of the publicly
 registered one.


 This is not an option in a larger team. Everyone would have to do that
 manually, or you'd need to write a custom script to do what is actually
 the package manager's task.


I would prefer to just stick a Git URL + branch in the dub.json file, 
just as with the Gemfile in Bundler.

-- 
/Jacob Carlborg



 Jun 04 2015



prev sibling  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 04.06.2015 um 12:30 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?= 
<schuetzm gmx.net>":

 On Thursday, 4 June 2015 at 08:51:02 UTC, Sönke Ludwig wrote:

 Am 01.06.2015 um 17:16 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?=
 <schuetzm gmx.net>":

 Since this thread seems to have turned into a wishlist for dub features,
 I'm going to add one:

 Undeprecate dependencies on git branches. They are really not different
 from dependencies with fuzzy "~>" versions. For both, the exact selected
 version can be stored in dub.selections.json.

 Use case:
 Making a fix to a third-party project and sharing that fix with other
 team members. Having to create a temporary project on code.dlang.org
 just for that purpose is both a waste of time for developers and a waste
 of resources of community infrastructure.


 They are only deprecated within dub.json. You can still use them
 within dub.selections.json.


 This is exactly the wrong way round. dub.selections.json must only
 contain exact versions. That's its purpose after all.


A branch *is* an exact version as far as DUB goes (just one that changes 
over time). But to make use of any commit relationships within a branch, 
it would have to gain specific knowledge of GIT or other version 
management systems.

Adding support for commit hashes in dub.selections.json is still TBD, 
but would be another alternative.

BTW, my primary concern with all of these things is retaining a healthy 
public package eco system. I personally suspect that this is much more 
important for the language as a whole rather than pleasing every single 
taste. This doesn't mean that the latter isn't a worthwhile goal, but to 
me the former has always been more important. What is missing is a 
proposal how to solve both issues at the same time.


 See Ruby's bundler as an example:
 http://bundler.io/v1.9/rationale.html
 Especially the parts about Gemfile.lock, which is their equivalent of
 dub.selections.json.


 But really, branch based dependencies are simply a broken feature.
 They are completely different to version ranges, because they are not
 at all comparable (to other branches or versions) - at least without
 completely committing to GIT and starting to walk the GIT commit graph.


 Why would you even want to compare them, except for equality? (...)


For example for conflict resolution. This is the foremost reason why the 
*have* to be banned from dub.json. You can't really know if a branch 
matches/is compatible with any particular version (range) or another 
branch, so you have to either make an arbitrary guess (always prefer the 
branch; always prefer the version; always prefer "master", but a tagged 
version otherwise; etc.), or always issue a version conflict error.

The practical effect that was already very visible in the existing 
ecosystem was that packages tended to separate into two clusters. One 
cluster that used branch dependencies and another that used version 
dependencies. Both incompatible with each other. It also meant that a 
lot of people didn't bother to make any version tags at all, which is 
understandable, but also a very bad influence to the stability of the 
ecosystem. Semantic versioning, if used correctly, is a very important 
factor in that regard.

The compromise solution to allow branch based dependencies only in the 
version selection file doesn't really have drawbacks, but dodges all of 
those issues by moving the decision to the root of the dependency tree.


 You can also check out certain branches locally and use "dub
 add-local" or "dub add-path" to let DUB use it instead of the publicly
 registered one.


 This is not an option in a larger team. Everyone would have to do that
 manually, or you'd need to write a custom script to do what is actually
 the package manager's task.


Using dub.selections.json should be the right tool then. You could of 
course also use things like git submodules + path based dependencies in 
this case, there is no need to force everything into one schema 
(especially when it doesn't fit well, such as versions and branches).



 Jun 04 2015



  parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net>  writes:


On Thursday, 4 June 2015 at 13:47:41 UTC, Sönke Ludwig wrote:

 A branch *is* an exact version as far as DUB goes (just one 
 that changes over time).


Well, that's the problem then.


 But to make use of any commit relationships within a branch, it 
 would have to gain specific knowledge of GIT or other version 
 management systems.

 Adding support for commit hashes in dub.selections.json is 
 still TBD, but would be another alternative.


I'd say it's the only consistent way.


 BTW, my primary concern with all of these things is retaining a 
 healthy public package eco system. I personally suspect that 
 this is much more important for the language as a whole rather 
 than pleasing every single taste. This doesn't mean that the 
 latter isn't a worthwhile goal, but to me the former has always 
 been more important. What is missing is a proposal how to solve 
 both issues at the same time.


I can understand that. But from what I've seen with rubygems, it 
is not a problem in practice. You're totally right that released 
packages shouldn't have such dependencies. A warning message in 
dub or a check on code.dlang.org is totally okay IMO.


 For example for conflict resolution. This is the foremost 
 reason why the *have* to be banned from dub.json. You can't 
 really know if a branch matches/is compatible with any 
 particular version (range) or another branch, so you have to 
 either make an arbitrary guess (always prefer the branch; 
 always prefer the version; always prefer "master", but a tagged 
 version otherwise; etc.), or always issue a version conflict 
 error.


Definitely always prefer the branch. The goal is to let the 
developer override the decisions made by upstream library authors.


 The practical effect that was already very visible in the 
 existing ecosystem was that packages tended to separate into 
 two clusters. One cluster that used branch dependencies and 
 another that used version dependencies. Both incompatible with 
 each other. It also meant that a lot of people didn't bother to 
 make any version tags at all, which is understandable, but also 
 a very bad influence to the stability of the ecosystem. 
 Semantic versioning, if used correctly, is a very important 
 factor in that regard.


Yes, but that's a social problem. We can nudge people in the 
right direction by printing a warning message, or even by 
rejecting it completely on code.dlang.org.


 The compromise solution to allow branch based dependencies only 
 in the version selection file doesn't really have drawbacks, 
 but dodges all of those issues by moving the decision to the 
 root of the dependency tree.


I does have obvious drawbacks: If dub.selections.json contains 
non-exact versions, different developers can get slightly 
different dependencies. dub.selections.json must fully determine 
the versions that are going to be used, otherwise it has failed 
its purpose.


 You can also check out certain branches locally and use "dub
 add-local" or "dub add-path" to let DUB use it instead of the 
 publicly
 registered one.


 This is not an option in a larger team. Everyone would have to 
 do that
 manually, or you'd need to write a custom script to do what is 
 actually
 the package manager's task.


 Using dub.selections.json should be the right tool then.


As explained above, it isn't.


 You could of course also use things like git submodules + path 
 based dependencies in this case, there is no need to force 
 everything into one schema (especially when it doesn't fit 
 well, such as versions and branches).


Yes, and I'm sure I can come up with even more workarounds ;-)



 Jun 05 2015



  parent  =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 05.06.2015 um 11:56 schrieb "Marc =?UTF-8?B?U2Now7x0eiI=?= 
<schuetzm gmx.net>":

 On Thursday, 4 June 2015 at 13:47:41 UTC, Sönke Ludwig wrote:

 A branch *is* an exact version as far as DUB goes (just one that
 changes over time).


 Well, that's the problem then.


 But to make use of any commit relationships within a branch, it would
 have to gain specific knowledge of GIT or other version management
 systems.

 Adding support for commit hashes in dub.selections.json is still TBD,
 but would be another alternative.


 I'd say it's the only consistent way.


 BTW, my primary concern with all of these things is retaining a
 healthy public package eco system. I personally suspect that this is
 much more important for the language as a whole rather than pleasing
 every single taste. This doesn't mean that the latter isn't a
 worthwhile goal, but to me the former has always been more important.
 What is missing is a proposal how to solve both issues at the same time.


 I can understand that. But from what I've seen with rubygems, it is not
 a problem in practice. You're totally right that released packages
 shouldn't have such dependencies. A warning message in dub or a check on
 code.dlang.org is totally okay IMO.


Well, enforcing it on code.dlang.org would indeed be an option. I still 
think that using the version backed approach below, or VCS facilities is 
superior in general, though. There is not much objective reason to 
duplicate that functionality in the package manager AFAICS.


 For example for conflict resolution. This is the foremost reason why
 the *have* to be banned from dub.json. You can't really know if a
 branch matches/is compatible with any particular version (range) or
 another branch, so you have to either make an arbitrary guess (always
 prefer the branch; always prefer the version; always prefer "master",
 but a tagged version otherwise; etc.), or always issue a version
 conflict error.


 Definitely always prefer the branch. The goal is to let the developer
 override the decisions made by upstream library authors.


That's what dub.selections.json is for! It applies only to the top level 
project, which can make such decisions. On the other hand, if you always 
prefer the branch during *dependency resolution*, some deeply buried 
dependency can simply break your build (or any upstream build) because 
it chose to use an incompatible branch based dependency.


 The practical effect that was already very visible in the existing
 ecosystem was that packages tended to separate into two clusters. One
 cluster that used branch dependencies and another that used version
 dependencies. Both incompatible with each other. It also meant that a
 lot of people didn't bother to make any version tags at all, which is
 understandable, but also a very bad influence to the stability of the
 ecosystem. Semantic versioning, if used correctly, is a very important
 factor in that regard.


 Yes, but that's a social problem. We can nudge people in the right
 direction by printing a warning message, or even by rejecting it
 completely on code.dlang.org.


 The compromise solution to allow branch based dependencies only in the
 version selection file doesn't really have drawbacks, but dodges all
 of those issues by moving the decision to the root of the dependency
 tree.


 I does have obvious drawbacks: If dub.selections.json contains non-exact
 versions, different developers can get slightly different dependencies.
 dub.selections.json must fully determine the versions that are going to
 be used, otherwise it has failed its purpose.


 You can also check out certain branches locally and use "dub
 add-local" or "dub add-path" to let DUB use it instead of the publicly
 registered one.


 This is not an option in a larger team. Everyone would have to do that
 manually, or you'd need to write a custom script to do what is actually
 the package manager's task.


 Using dub.selections.json should be the right tool then.


 As explained above, it isn't.


If you want commit granularity then that's a different thing. The 
intended way to support this is to still use version tags - DUB will 
then associate a branch with the latest matching version tag, so that 
normal version based dependencies can be used.

If that case applies, dub.selections.json will contain something like 
{"somedep": "1.0.0+commit.14.f23a56gb"}. Currently this information 
isn't used to drive GIT to fetch the right tag, but a ticket for that is 
open and this could be a viable middleground solution.


 You could of course also use things like git submodules + path based
 dependencies in this case, there is no need to force everything into
 one schema (especially when it doesn't fit well, such as versions and
 branches).


 Yes, and I'm sure I can come up with even more workarounds ;-)


It's not really a workaround. The package manager has a version based 
dependency handling logic that has the proper semantics for this kind of 
task. Trying to squeeze branches and commits into this isn't necessarily 
a good idea or even necessary at all. The idea here is to let you do 
such things and not get in the way, but why should the package manager 
duplicate functionality of the VCS when that can solve the task just 
fine? It adds additional and redundant complexity for no real benefit.



 Jun 10 2015



prev sibling  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 5/31/15 7:01 PM, Andrei Alexandrescu wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


Just a reminder to everyone that at one point, we wanted the compiler to 
do kind of what 'dub' does -- fetch packages as dependencies for a 
project. Remember this DIP? http://wiki.dlang.org/DIP11

The problem I have with dub is it makes you as a consumer of libraries 
live within its realm. You have to make a dub file to use it, and that's 
not a small burden.

That being said, can we get a tool that does what DIP11 asked for by 
using dub? We do have quite a large repository of code that's 
dub-enabled, and I have no problem requiring extra steps for *libraries* 
to make them usable with this system. Basically, a builder that says 
"hey, if dmd (or whatever) cannot find an import, see if dub can do it, 
and then use dub to download the package".

-Steve



 Jun 01 2015



  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 11:16 AM, Steven Schveighoffer wrote:

 On 5/31/15 7:01 PM, Andrei Alexandrescu wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


 Just a reminder to everyone that at one point, we wanted the compiler to
 do kind of what 'dub' does -- fetch packages as dependencies for a
 project. Remember this DIP? http://wiki.dlang.org/DIP11


That sort of did what dub does, but badly. Among other things, it 
directly encouraged developers to hardcode their exact dependencies. It 
would have been a mess. Dub's package management side already handles 
that DIP's goals WAY better. The *only* issue now is trying to opt-out 
of dub's buildsystem side without causing problems (and I absolutely 
believe that to be easily solvable, see my other posts).

There is one part of that old proposal that I have come to like though, 
and wouldn't mind seeing: There was an idea to have DMD automatically 
invoke an external tool


 The problem I have with dub is it makes you as a consumer of libraries
 live within its realm. You have to make a dub file to use it, and that's
 not a small burden.


The dub.json file is an absolutely trivial "burden". It's a really, 
REALLY simple little file. Or at least, it should be...

The ONLY real burden comes when trying to teach dub how to build your 
project using dub's own buildsystem. If your project doesn't do 
everything "the dub way" then it certainly can be a bother. And I'm 
pushing very hard to relieve projects of that requirement.

But really, the following is NOT a burden unless extreme lazyness is in 
play:

{
     "name": "my-project",
     "dependencies": {
         "libblah": "~>1.7.0",
         "libfoobar": "~>2.1.0"
     }
}

Yes, a very, very, *VERY* tiny "burden".

And then to also get dub's buildsystem out of the way:

$ mkdir dummy-src-dir

$ touch dummy-src-dir/dummy.d

$ echo "void main() {}" > dummy-src-dir/dummy.d

{
     "name": "my-project",
     "dependencies": {
         "libblah": "~>1.7.0",
         "libfoobar": "~>2.1.0"
     },

     "sourcePaths": "dummy-src-dir/",
     "pre-build-command-posix": "./buildscript",
     "pre-build-command-win": "buildscript"
}

Done. Trivial. No real burden. (Well, aside from JSON's irritating lack 
of comments or trailing commas.)



 Jun 01 2015



 next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/1/15 1:36 PM, Nick Sabalausky wrote:

 On 06/01/2015 11:16 AM, Steven Schveighoffer wrote:

 On 5/31/15 7:01 PM, Andrei Alexandrescu wrote:

 Let's make this part of 2.068:

 https://issues.dlang.org/show_bug.cgi?id=14636

 It's preapproved. Who would want to work on it?


 Just a reminder to everyone that at one point, we wanted the compiler to
 do kind of what 'dub' does -- fetch packages as dependencies for a
 project. Remember this DIP? http://wiki.dlang.org/DIP11


 That sort of did what dub does, but badly. Among other things, it
 directly encouraged developers to hardcode their exact dependencies. It
 would have been a mess. Dub's package management side already handles
 that DIP's goals WAY better. The *only* issue now is trying to opt-out
 of dub's buildsystem side without causing problems (and I absolutely
 believe that to be easily solvable, see my other posts).


I'm not defending DIP11, just that the fact that it could be done 
without requiring an extra "dependencies" file. One thing I absolutely 
LOVE about git, is that when I do git status, if it sees a 
not-just-cloned repository, it gives me all the commands I would need to 
run to do all the various things. I rarely have to go to the git manual. 
I'd love to see something like the following:


 dmd x.d


Error: Your import of some.lib.module didn't work. Running dub 
--with-some-cool-arg some.lib.module would fetch the correct module from 
code.dlang.org

 dub --with-some-cool-arg some.lib.module


Dub : got it, fetched.

 dmd x.d
 echo 'yay, it worked! Too bad this is a fantasy :('



 The problem I have with dub is it makes you as a consumer of libraries
 live within its realm. You have to make a dub file to use it, and that's
 not a small burden.


 The dub.json file is an absolutely trivial "burden". It's a really,
 REALLY simple little file. Or at least, it should be...


Sorry, I suppose it's a small burden if you use dub frequently. I don't. 
Look what happens when I initialize a dub project:

{
	"name": "testdub",
	"description": "A minimal D application.",
	"copyright": "Copyright © 2015, steves",
	"authors": ["steves"],
	"dependencies": {
	}
}

Cool, I can figure out how to massage all the existing strings, except, 
hm... dependencies. What goes in there?

Now, it's time to get out the dub manual. annoying. Where is the comment 
that says "put your dependencies in here, they should look like this: 
..."???

And dub uses versions. How does that work? Again no comment.

And where does the e.g. "name" get stuck into the executable? Oh, it 
doesn't. Why is it generated? I don't freaking know (except that it was 
the directory I was in when running dub init).

I'm sure dub is great for people who want to learn dub. For those who 
don't, it should work in a minimal way, and ESPECIALLY if we are to 
include it with the distribution.

-Steve



 Jun 01 2015



 next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 02:49 PM, Steven Schveighoffer wrote:

 On 6/1/15 1:36 PM, Nick Sabalausky wrote:

 That sort of did what dub does, but badly. Among other things, it
 directly encouraged developers to hardcode their exact dependencies. It
 would have been a mess. Dub's package management side already handles
 that DIP's goals WAY better. The *only* issue now is trying to opt-out
 of dub's buildsystem side without causing problems (and I absolutely
 believe that to be easily solvable, see my other posts).


 I'm not defending DIP11, just that the fact that it could be done
 without requiring an extra "dependencies" file. One thing I absolutely
 LOVE about git, is that when I do git status, if it sees a
 not-just-cloned repository, it gives me all the commands I would need to
 run to do all the various things. I rarely have to go to the git manual.
 I'd love to see something like the following:

  > dmd x.d
 Error: Your import of some.lib.module didn't work. Running dub
 --with-some-cool-arg some.lib.module would fetch the correct module from
 code.dlang.org
  > dub --with-some-cool-arg some.lib.module
 Dub : got it, fetched.
  > dmd x.d
  > echo 'yay, it worked! Too bad this is a fantasy :('


Yea, true, that would be nice.

However, once the dub PRs I'm working on now are finished, then you 
should be able to do something like this:


 dmd x.d


Error: Your import of some.lib.module didn't work.

 dub fetch libfoobar
 dmd `dub describe libfoobar --dmd-args` x.d


And it should even be possible to add support for this stuff, even 
without a dub.json for the current project:


 dmd x.d


Error: Your import of some.lib.module didn't work.

 dub find-module some.lib.module


(searches the repo)
libfoobar

 dmd `dub use libfoobar [optional-version] --dmd-args` x.d


(Auto-fetches libfoobar if it hasn't already been fetched, and then 
outputs the necessary CLI args for DMD to use.)



 The problem I have with dub is it makes you as a consumer of libraries
 live within its realm. You have to make a dub file to use it, and that's
 not a small burden.


 The dub.json file is an absolutely trivial "burden". It's a really,
 REALLY simple little file. Or at least, it should be...


 Sorry, I suppose it's a small burden if you use dub frequently. I don't.
 Look what happens when I initialize a dub project:

 {
      "name": "testdub",
      "description": "A minimal D application.",
      "copyright": "Copyright © 2015, steves",
      "authors": ["steves"],
      "dependencies": {
      }
 }

 Cool, I can figure out how to massage all the existing strings, except,
 hm... dependencies. What goes in there?

 Now, it's time to get out the dub manual. annoying. Where is the comment
 that says "put your dependencies in here, they should look like this:
 ..."???

 And dub uses versions. How does that work? Again no comment.


There's really not much to remember. All you really need to remember is 
this:

"dependencies": {
     "libname": "==1.3.0"
}

That's really it.

There are fancier things you *can* do, but that right there is really 
all you need, nothing more.

Once dub supports a format that can actually freaking handle comments 
(*grumble grumble* stupid json *grumble grumble*), then dub init's 
generated file could be way more self-descriptive:

"dependencies": {
     // Uncomment these to use them:

     // Use an exact version:
     //     "name-of-lib": "==1.3.4",

     // Restrict to a certain minor version,
     // equivalent to ">=1.3.4 <1.4.0":
     //     "name-of-lib": "~>1.3.4",
}


 And where does the e.g. "name" get stuck into the executable? Oh, it
 doesn't. Why is it generated? I don't freaking know (except that it was
 the directory I was in when running dub init).


Name is just the name of the project. To identify it. Just like a Github 
project name. Or, pretty much anything called a "name", really. Every 
project has a name, dub or not. Not really hard to remember. There's no 
magic going on there.

And again, dub needs a format here that actually freaking supports comments.


 I'm sure dub is great for people who want to learn dub. For those who
 don't, it should work in a minimal way, and ESPECIALLY if we are to
 include it with the distribution.


It could be improved in some ways, but I think dub.json is way simpler 
then non-users really expect it to be. It just needs to communicate the 
minimal basics a little better, so people don't have to go trying to 
find <http://code.dlang.org/package-format> and then get intimidated by 
all the (mostly non-essential) stuff there. And of course, it needs the 
PRs I'm doing to make it way more friendly to non-dub buildsystems.



 Jun 01 2015



  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/1/15 3:53 PM, Nick Sabalausky wrote:

 On 06/01/2015 02:49 PM, Steven Schveighoffer wrote:

 On 6/1/15 1:36 PM, Nick Sabalausky wrote:

 That sort of did what dub does, but badly. Among other things, it
 directly encouraged developers to hardcode their exact dependencies. It
 would have been a mess. Dub's package management side already handles
 that DIP's goals WAY better. The *only* issue now is trying to opt-out
 of dub's buildsystem side without causing problems (and I absolutely
 believe that to be easily solvable, see my other posts).


 I'm not defending DIP11, just that the fact that it could be done
 without requiring an extra "dependencies" file. One thing I absolutely
 LOVE about git, is that when I do git status, if it sees a
 not-just-cloned repository, it gives me all the commands I would need to
 run to do all the various things. I rarely have to go to the git manual.
 I'd love to see something like the following:

  > dmd x.d
 Error: Your import of some.lib.module didn't work. Running dub
 --with-some-cool-arg some.lib.module would fetch the correct module from
 code.dlang.org
  > dub --with-some-cool-arg some.lib.module
 Dub : got it, fetched.
  > dmd x.d
  > echo 'yay, it worked! Too bad this is a fantasy :('


 Yea, true, that would be nice.

 However, once the dub PRs I'm working on now are finished, then you
 should be able to do something like this:

  > dmd x.d
 Error: Your import of some.lib.module didn't work.
  > dub fetch libfoobar
  > dmd `dub describe libfoobar --dmd-args` x.d

 And it should even be possible to add support for this stuff, even
 without a dub.json for the current project:

  > dmd x.d
 Error: Your import of some.lib.module didn't work.
  > dub find-module some.lib.module
 (searches the repo)
 libfoobar
  > dmd `dub use libfoobar [optional-version] --dmd-args` x.d
 (Auto-fetches libfoobar if it hasn't already been fetched, and then
 outputs the necessary CLI args for DMD to use.)


I would like to see more automation like this, even if it means there 
are some generated meta-files. But I'd like it to be dmd and dub working 
together. For instance, dmd should check to see if dub can has already 
fetched the dependency and use it (maybe with a switch enabled). Having 
to do the whole "`dub use blah blah`" on the command line is hacky.

A protocol to be used between dmd and dub (or any other package fetcher) 
would be a good first step.


 And where does the e.g. "name" get stuck into the executable? Oh, it
 doesn't. Why is it generated? I don't freaking know (except that it was
 the directory I was in when running dub init).


 Name is just the name of the project. To identify it. Just like a Github
 project name. Or, pretty much anything called a "name", really. Every
 project has a name, dub or not. Not really hard to remember. There's no
 magic going on there.


I think I mistakenly hastily chose "name" as the first thing that stuck 
out as not being used, but it's actually probably the only one that *is* 
used (to name the resulting binary), which actually makes sense. Pick 
any of the other ones :)


 And again, dub needs a format here that actually freaking supports
 comments.


Yes, definitely. Does it have to be straight json?

-Steve



 Jun 01 2015



  parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote:


 A protocol to be used between dmd and dub (or any other package fetcher)
 would be a good first step.


that was the thing i once proposed. see, we have a powerful scripting=20
language inside DMD: D! yet we never used all it's power to do something=20
really exciting -- like, for example, preparing command lines for=20
external package fetching tool and parsing the answers. instead of=20
providing a simple module for that, it all goes to be hardcoded.

i extended CTFE engine with simple file i/o functions and "system" call=20
with config-defined executables (something like sudoers list) just to see=20
if it will be usable. and it's surprisingly fun even with all the limits=20
and without hooks for module imports and so on.

this system can be extended to allow writing arbitrary subcommands (like=20
git). just import the corresponding subcommand module, if any, and CTFE=20
it's invocation point. bingo! the system that can be extended without=20
recompiling the compiler. and user can add subcommands on per-project=20
base!

sadly, this seems to get no "wow!"s. :-(=



 Jun 02 2015



 next sibling parent reply Rikki Cattermole <alphaglosined gmail.com>  writes:


On 2/06/2015 8:54 p.m., ketmar wrote:

 On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote:


 A protocol to be used between dmd and dub (or any other package fetcher)
 would be a good first step.


 that was the thing i once proposed. see, we have a powerful scripting
 language inside DMD: D! yet we never used all it's power to do something
 really exciting -- like, for example, preparing command lines for
 external package fetching tool and parsing the answers. instead of
 providing a simple module for that, it all goes to be hardcoded.

 i extended CTFE engine with simple file i/o functions and "system" call
 with config-defined executables (something like sudoers list) just to see
 if it will be usable. and it's surprisingly fun even with all the limits
 and without hooks for module imports and so on.

 this system can be extended to allow writing arbitrary subcommands (like
 git). just import the corresponding subcommand module, if any, and CTFE
 it's invocation point. bingo! the system that can be extended without
 recompiling the compiler. and user can add subcommands on per-project
 base!

 sadly, this seems to get no "wow!"s. :-(


I say wow.

Okay not really I know it is possible and not all the hard.
Personally I think it is a rather an awesome possibility.

Really what I want is to add the ability to add on the compiler side 
symbols that can be CTFE'd.
Registered at start of runtime.

Add on shared library support and wamo. You can add stuff like this 
pretty arbitrary :)



 Jun 02 2015



  parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Tue, 02 Jun 2015 21:12:14 +1200, Rikki Cattermole wrote:


 I say wow.
=20
 Okay not really I know it is possible and not all the hard. Personally I
 think it is a rather an awesome possibility.


adding some CTFE functions is not that hard at all. it's not documented,=20
but one can look at "import()" and "pragma()" to see how they're doing=20
their things and do the same. that's what i did, and it took me one day=20
to write a working PoC.

i promised to make a series of articles on that for TWiD, but still=20
didn't done that. mea maxima culpa.



 Really what I want is to add the ability to add on the compiler side
 symbols that can be CTFE'd.
 Registered at start of runtime.


i'm afraid i didn't get it. can you provide a sample?



 Add on shared library support and wamo. You can add stuff like this
 pretty arbitrary :)


you mean adding symbols with plugin-like system? i'm not sure that it=20
will be easy. anyway, with well-defined CTFE API to access file system=20
and execute commands (with restrictions, of course, so D programs will=20
not go wild doing "rm -rf ~/" ;-) one can write modules that contains=20
functions that acts like "filters", executing external binaries and=20
massaging input/output.

with some hooks added (like "call this CTFE function (if it present) when=20
module importing fails") and ability to add -I/-J pathes in that hooks=20
(restricted to some "library root dir") one can write "CTFE rdmd with=20
automatic package downloading".

i think i have to make a draft and PoC for that, so people can try it and=20
see how k00l it is. ;-)=



 Jun 02 2015



  parent reply Rikki Cattermole <alphaglosined gmail.com>  writes:


On 2/06/2015 9:32 p.m., ketmar wrote:

 On Tue, 02 Jun 2015 21:12:14 +1200, Rikki Cattermole wrote:


 I say wow.

 Okay not really I know it is possible and not all the hard. Personally I
 think it is a rather an awesome possibility.


 adding some CTFE functions is not that hard at all. it's not documented,
 but one can look at "import()" and "pragma()" to see how they're doing
 their things and do the same. that's what i did, and it took me one day
 to write a working PoC.

 i promised to make a series of articles on that for TWiD, but still
 didn't done that. mea maxima culpa.



 Really what I want is to add the ability to add on the compiler side
 symbols that can be CTFE'd.
 Registered at start of runtime.


 i'm afraid i didn't get it. can you provide a sample?



 Add on shared library support and wamo. You can add stuff like this
 pretty arbitrary :)


 you mean adding symbols with plugin-like system? i'm not sure that it
 will be easy. anyway, with well-defined CTFE API to access file system
 and execute commands (with restrictions, of course, so D programs will
 not go wild doing "rm -rf ~/" ;-) one can write modules that contains
 functions that acts like "filters", executing external binaries and
 massaging input/output.

 with some hooks added (like "call this CTFE function (if it present) when
 module importing fails") and ability to add -I/-J pathes in that hooks
 (restricted to some "library root dir") one can write "CTFE rdmd with
 automatic package downloading".

 i think i have to make a draft and PoC for that, so people can try it and
 see how k00l it is. ;-)


Essentially its compiler plugins that can add news types + 
free-functions as if it was D code provided by source code only also 
able to modify e.g. AST directly.
So while in of itself would not provide new language features, it could 
add some very nice behavior to existing ones.

So what you want with commands ext. would be done in a shared library. 
It have e.g. a struct added into object.d(i).

__CEIFileSystem.delete("./tmp/something");

Of course adding these plugins should be pretty explicit. Package 
maintainers for example shouldn't auto install any. There should be no 
way to auto install them.

Basically dmd-plugin-dub-bin would be an example package. That would add 
dub support directly into dmd.

----------
 __CEIDub.Dependency("vibe-d", ">=0.7.22")
void main() {
	// what have you
}
----------

Or:

----------
static assert(__CEIDub.dependency("vibe.d", ">=0.7.22"), "Requires 
vibe-d 0.7.22 or newer.");

void main() {
	// what have you
}
----------

That way it can be used for e.g. static-if and template if.

Of course I'm sure Walter would go nuts at such an idea, if it was 
seriously proposed.
So perhaps dmd doesn't support it. Instead a new version of dmd is also 
shipped (dmdext) with this enabled ;)

With the community agreeing that we should try to keep as close to dmd 
as possible and only reverting to dmdext + plugins to help newbies and 
where it just wouldn't be possible to not have an external tool. Which 
most likely uses multiple compilation or something else not so nice.



 Jun 02 2015



  parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Tue, 02 Jun 2015 21:51:54 +1200, Rikki Cattermole wrote:


 Essentially its compiler plugins that can add news types +
 free-functions as if it was D code provided by source code only also
 able to modify e.g. AST directly.
 So while in of itself would not provide new language features, it could
 add some very nice behavior to existing ones.
=20
 So what you want with commands ext. would be done in a shared library.
 It have e.g. a struct added into object.d(i).
=20
 __CEIFileSystem.delete("./tmp/something");
=20
 Of course adding these plugins should be pretty explicit. Package
 maintainers for example shouldn't auto install any. There should be no
 way to auto install them.
=20
 Basically dmd-plugin-dub-bin would be an example package. That would add
 dub support directly into dmd.
=20
 ----------
  __CEIDub.Dependency("vibe-d", ">=3D0.7.22")
 void main() {
 	// what have you
 }
 ----------
=20
 Or:
=20
 ----------
 static assert(__CEIDub.dependency("vibe.d", ">=3D0.7.22"), "Requires
 vibe-d 0.7.22 or newer.");
=20
 void main() {
 	// what have you
 }
 ----------
=20
 That way it can be used for e.g. static-if and template if.
=20
 Of course I'm sure Walter would go nuts at such an idea, if it was
 seriously proposed.
 So perhaps dmd doesn't support it. Instead a new version of dmd is also
 shipped (dmdext) with this enabled ;)
=20
 With the community agreeing that we should try to keep as close to dmd
 as possible and only reverting to dmdext + plugins to help newbies and
 where it just wouldn't be possible to not have an external tool. Which
 most likely uses multiple compilation or something else not so nice.


actually, that should be doable with DDMD on non-windows systems*, i=20
believe. DDMD -- to avoid writing plugins in other languages. ;-) and=20
with DDMD plugins will have full access to frontend internals, including=20
AST manipulation.

* non-windows, as dll support on windows is still not here, i believe.


the downside of this approach is that it's highly compiler-specific. i.e.=20
one will force to have different plugins for GDC/LDC/SDC/etc. ;-) and=20
with pure CTFE API without plugins, only with external executables, it=20
will be compiler-agnostic, any compiler using DMDFE will get it for free.=20
only SDC will be forced to rewrite the CTFE part, but other than that all=20
CTFE D code will work the same.

i mean, 'cmon, we have such great built-in scripting engine, let's use it!

your samples can be built ontop of my "subcommand" proposal, for example.=20
let's say that compiler will try to automatically do static import=20
"DMD.subcommands.dub" when programmer writing something like
`DMD.dub.Dependency("vibe-d", ">=3D0.7.22");`, falling back to "DMD.failure
(name, args)" if there is no "dub" subcommand package.

and then subcommand can execute dub, parse it's output and do what it=20
wants. or one can make a local override for subcommand -- without even=20
compiling separate plugin.

that's how i see it.
=



 Jun 02 2015



  parent reply Rikki Cattermole <alphaglosined gmail.com>  writes:


On 2/06/2015 10:15 p.m., ketmar wrote:

 On Tue, 02 Jun 2015 21:51:54 +1200, Rikki Cattermole wrote:


 Essentially its compiler plugins that can add news types +
 free-functions as if it was D code provided by source code only also
 able to modify e.g. AST directly.
 So while in of itself would not provide new language features, it could
 add some very nice behavior to existing ones.

 So what you want with commands ext. would be done in a shared library.
 It have e.g. a struct added into object.d(i).

 __CEIFileSystem.delete("./tmp/something");

 Of course adding these plugins should be pretty explicit. Package
 maintainers for example shouldn't auto install any. There should be no
 way to auto install them.

 Basically dmd-plugin-dub-bin would be an example package. That would add
 dub support directly into dmd.

 ----------
  __CEIDub.Dependency("vibe-d", ">=0.7.22")
 void main() {
 	// what have you
 }
 ----------

 Or:

 ----------
 static assert(__CEIDub.dependency("vibe.d", ">=0.7.22"), "Requires
 vibe-d 0.7.22 or newer.");

 void main() {
 	// what have you
 }
 ----------

 That way it can be used for e.g. static-if and template if.

 Of course I'm sure Walter would go nuts at such an idea, if it was
 seriously proposed.
 So perhaps dmd doesn't support it. Instead a new version of dmd is also
 shipped (dmdext) with this enabled ;)

 With the community agreeing that we should try to keep as close to dmd
 as possible and only reverting to dmdext + plugins to help newbies and
 where it just wouldn't be possible to not have an external tool. Which
 most likely uses multiple compilation or something else not so nice.


 actually, that should be doable with DDMD on non-windows systems*, i
 believe. DDMD -- to avoid writing plugins in other languages. ;-) and
 with DDMD plugins will have full access to frontend internals, including
 AST manipulation.

 * non-windows, as dll support on windows is still not here, i believe.


I've been able to do it a little bit already with my Web server. Haven't 
hit its limit so far. In other words as long as exceptions aren't used 
and its never unloaded, should be ok. Anyway, do you really think we 
would get that sort of support for a couple of releases?


 the downside of this approach is that it's highly compiler-specific. i.e.
 one will force to have different plugins for GDC/LDC/SDC/etc. ;-) and
 with pure CTFE API without plugins, only with external executables, it
 will be compiler-agnostic, any compiler using DMDFE will get it for free.
 only SDC will be forced to rewrite the CTFE part, but other than that all
 CTFE D code will work the same.


Yeah it is, but it would also mean better customization of the build 
process!


 i mean, 'cmon, we have such great built-in scripting engine, let's use it!


Indeed lets. But can we also not go modifying the language? I'm just 
suggesting injecting of symbols that execute compiler side actions. Your 
suggesting language changes. Even if it is a new pragma.


 your samples can be built ontop of my "subcommand" proposal, for example.
 let's say that compiler will try to automatically do static import
 "DMD.subcommands.dub" when programmer writing something like
 `DMD.dub.Dependency("vibe-d", ">=0.7.22");`, falling back to "DMD.failure
 (name, args)" if there is no "dub" subcommand package.

 and then subcommand can execute dub, parse it's output and do what it
 wants. or one can make a local override for subcommand -- without even
 compiling separate plugin.

 that's how i see it.


We really need to toy with these ideas properly and implement each. Then 
its just a matter of time to convince the higher ups that it should be 
merged.



 Jun 02 2015



  parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Tue, 02 Jun 2015 22:30:47 +1200, Rikki Cattermole wrote:


 * non-windows, as dll support on windows is still not here, i believe.


=20
 I've been able to do it a little bit already with my Web server. Haven't
 hit its limit so far. In other words as long as exceptions aren't used
 and its never unloaded, should be ok. Anyway, do you really think we
 would get that sort of support for a couple of releases?


it seems that nobody on windows really needs that. ;-)



 Indeed lets. But can we also not go modifying the language? I'm just
 suggesting injecting of symbols that execute compiler side actions. Your
 suggesting language changes. Even if it is a new pragma.


actually, no language changes at all. CTFE is already here, and it is=20
used in semantic analysis stage. there are no changes to the language,=20
besides adding some hooks and defining an API. it's invisible from the=20
user's POV and doesn't require changing of specs. now CTFE is started=20
explicitly, by using mixins and template evaluation. with my idea it will=20
be started implicitly when some condition is met. sure, that conditions=20
must be documented, but it will not change the language drastically.



 We really need to toy with these ideas properly and implement each. Then
 its just a matter of time to convince the higher ups that it should be
 merged.


yes, i believe that this is the best approach. implement both and let the=20
best solution win in a honest competition. ;-)

i don't feel very creative right now, but implementing that idea is in my=20
TODO list, and it's not at the bottom of list. so eventually (month? two?=20
ten years?) i'll write a PoC.=



 Jun 02 2015



  parent reply Rikki Cattermole <alphaglosined gmail.com>  writes:


On 3/06/2015 1:33 a.m., ketmar wrote:

 On Tue, 02 Jun 2015 22:30:47 +1200, Rikki Cattermole wrote:


 * non-windows, as dll support on windows is still not here, i believe.


 I've been able to do it a little bit already with my Web server. Haven't
 hit its limit so far. In other words as long as exceptions aren't used
 and its never unloaded, should be ok. Anyway, do you really think we
 would get that sort of support for a couple of releases?


 it seems that nobody on windows really needs that. ;-)



 Indeed lets. But can we also not go modifying the language? I'm just
 suggesting injecting of symbols that execute compiler side actions. Your
 suggesting language changes. Even if it is a new pragma.


 actually, no language changes at all. CTFE is already here, and it is
 used in semantic analysis stage. there are no changes to the language,
 besides adding some hooks and defining an API. it's invisible from the
 user's POV and doesn't require changing of specs. now CTFE is started
 explicitly, by using mixins and template evaluation. with my idea it will
 be started implicitly when some condition is met. sure, that conditions
 must be documented, but it will not change the language drastically.


Nope, it can be just another function call.

------
auto something(string text);

 something("...")
void another() {

}
------

Where something is specified on the compiler side.
UDA's + static assert imply compile time literals, which is what we want.

Something that is important to note is, these symbols like something 
defined by the compiler should never hit the backend. If they are 
referenced they should be forced to be called.


 We really need to toy with these ideas properly and implement each. Then
 its just a matter of time to convince the higher ups that it should be
 merged.


 yes, i believe that this is the best approach. implement both and let the
 best solution win in a honest competition. ;-)

 i don't feel very creative right now, but implementing that idea is in my
 TODO list, and it's not at the bottom of list. so eventually (month? two?
 ten years?) i'll write a PoC.


I think we want something very similar. We just haven't quite merged our 
ideas just yet.



 Jun 02 2015



  parent  ketmar <ketmar ketmar.no-ip.org>  writes:


On Wed, 03 Jun 2015 01:48:13 +1200, Rikki Cattermole wrote:


 Nope, it can be just another function call.
=20
 ------
 auto something(string text);
=20
  something("...")
 void another() {
=20
 }
 ------
=20
 Where something is specified on the compiler side.
 UDA's + static assert imply compile time literals, which is what we
 want.


for me UDA syntax that actually calling some function seems=20
counterintuitive. like "well, i know that it's a simple declaration...=20
WUT?! how it managed to execute the code?!"



 I think we want something very similar. We just haven't quite merged our
 ideas just yet.


yes, seems that it's the same goal, but slightly different approaches. i=20
think i should do a draft for API and write some explanations first, so=20
it can be discussed in details. 'cause now i have a foggy picture in my=20
head which is hard to describe.=



 Jun 02 2015



prev sibling next sibling parent reply "Atila Neves" <atila.neves gmail.com>  writes:


On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:

 On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote:


 A protocol to be used between dmd and dub (or any other 
 package fetcher)
 would be a good first step.


 that was the thing i once proposed. see, we have a powerful 
 scripting
 language inside DMD: D! yet we never used all it's power to do 
 something
 really exciting -- like, for example, preparing command lines 
 for
 external package fetching tool and parsing the answers. instead 
 of
 providing a simple module for that, it all goes to be hardcoded.


https://github.com/atilaneves/reggae

Atila



 Jun 02 2015



 next sibling parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Tue, 02 Jun 2015 09:15:12 +0000, Atila Neves wrote:


 On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:

 On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer wrote:


 A protocol to be used between dmd and dub (or any other package
 fetcher)
 would be a good first step.


 that was the thing i once proposed. see, we have a powerful scripting
 language inside DMD: D! yet we never used all it's power to do
 something really exciting -- like, for example, preparing command lines
 for external package fetching tool and parsing the answers. instead of
 providing a simple module for that, it all goes to be hardcoded.


=20
 https://github.com/atilaneves/reggae
=20
 Atila


hm. looks interesting, albeit a little too noisy for my taste.=



 Jun 02 2015



  parent reply "Atila Neves" <atila.neves gmail.com>  writes:


On Tuesday, 2 June 2015 at 09:33:24 UTC, ketmar wrote:

 On Tue, 02 Jun 2015 09:15:12 +0000, Atila Neves wrote:


 On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:

 On Mon, 01 Jun 2015 16:05:17 -0400, Steven Schveighoffer 
 wrote:


 A protocol to be used between dmd and dub (or any other 
 package
 fetcher)
 would be a good first step.


 that was the thing i once proposed. see, we have a powerful 
 scripting
 language inside DMD: D! yet we never used all it's power to do
 something really exciting -- like, for example, preparing 
 command lines
 for external package fetching tool and parsing the answers. 
 instead of
 providing a simple module for that, it all goes to be 
 hardcoded.


 
 https://github.com/atilaneves/reggae
 
 Atila


 hm. looks interesting, albeit a little too noisy for my taste.


It depends on what you want to do: the idea is to use high-level 
utility functions as much as possible, while at the same time 
offering low-level primitives that those high-level ones are 
based on.

If it can be made simpler, I'd really like to know.

Atila



 Jun 02 2015



  parent reply ketmar <ketmar ketmar.no-ip.org>  writes:


On Tue, 02 Jun 2015 12:12:06 +0000, Atila Neves wrote:


 https://github.com/atilaneves/reggae
=20
 Atila


 hm. looks interesting, albeit a little too noisy for my taste.


=20
 It depends on what you want to do: the idea is to use high-level utility
 functions as much as possible, while at the same time offering low-level
 primitives that those high-level ones are based on.
=20
 If it can be made simpler, I'd really like to know.


not that i have something better to offer. it's readable and looks fine.=20
besides, i'm really sux at design things. ;-)=



 Jun 02 2015



  parent  "Atila Neves" <atila.neves gmail.com>  writes:


Well, there's this thread now:

http://forum.dlang.org/thread/ranqlmrjornlvopsuris forum.dlang.org#post-ranqlmrjornlvopsuris:40forum.dlang.org

Atila

On Tuesday, 2 June 2015 at 13:22:25 UTC, ketmar wrote:

 On Tue, 02 Jun 2015 12:12:06 +0000, Atila Neves wrote:


 https://github.com/atilaneves/reggae
 
 Atila


 hm. looks interesting, albeit a little too noisy for my taste.


 
 It depends on what you want to do: the idea is to use 
 high-level utility
 functions as much as possible, while at the same time offering 
 low-level
 primitives that those high-level ones are based on.
 
 If it can be made simpler, I'd really like to know.


 not that i have something better to offer. it's readable and 
 looks fine.
 besides, i'm really sux at design things. ;-)





 Jun 02 2015



prev sibling  parent reply "Dicebot" <public dicebot.lv>  writes:


On Tuesday, 2 June 2015 at 09:15:13 UTC, Atila Neves wrote:

 https://github.com/atilaneves/reggae

 Atila


If this works up to the expectation, I will only need to write a 
simple code.dlang.org package fetcher and will never need to use 
dub again :)



 Jun 02 2015



  parent  "Atila Neves" <atila.neves gmail.com>  writes:


On Tuesday, 2 June 2015 at 18:56:22 UTC, Dicebot wrote:

 On Tuesday, 2 June 2015 at 09:15:13 UTC, Atila Neves wrote:

 https://github.com/atilaneves/reggae

 Atila


 If this works up to the expectation, I will only need to write 
 a simple code.dlang.org package fetcher and will never need to 
 use dub again :)


Destroy away!

If it doesn't work like you like, let me know.

Atila



 Jun 02 2015



prev sibling next sibling parent reply "Wyatt" <wyatt.epp gmail.com>  writes:


On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:

 sadly, this seems to get no "wow!"s. :-(


If no one knows about it, "wow!"s will not happen.  Make PRs.

-Wyatt



 Jun 02 2015



  parent  ketmar <ketmar ketmar.no-ip.org>  writes:


On Tue, 02 Jun 2015 13:44:10 +0000, Wyatt wrote:


 On Tuesday, 2 June 2015 at 08:54:46 UTC, ketmar wrote:

 sadly, this seems to get no "wow!"s. :-(


=20
 If no one knows about it, "wow!"s will not happen.  Make PRs.


i mentioned the idea several times in NG, it fades almost without any=20
attention. honestly, i have alot of hobby projects, so before writing=20
such complex things (and it's fairly complex, considering design and=20
documentation) i want to know if there will be enough attention to it.

as people seems to pass it by, i have no motivation of doing that work. i=20
have alot of things i want to add to Aliced, so no "wow, that would be=20
nice to try!" replies means "move that down in my TODO list".=



 Jun 02 2015



prev sibling  parent reply Jacob Carlborg <doob me.com>  writes:


On 2015-06-02 10:54, ketmar wrote:


 that was the thing i once proposed. see, we have a powerful scripting
 language inside DMD: D! yet we never used all it's power to do something
 really exciting -- like, for example, preparing command lines for
 external package fetching tool and parsing the answers. instead of
 providing a simple module for that, it all goes to be hardcoded.

 i extended CTFE engine with simple file i/o functions and "system" call
 with config-defined executables (something like sudoers list) just to see
 if it will be usable. and it's surprisingly fun even with all the limits
 and without hooks for module imports and so on.

 this system can be extended to allow writing arbitrary subcommands (like
 git). just import the corresponding subcommand module, if any, and CTFE
 it's invocation point. bingo! the system that can be extended without
 recompiling the compiler. and user can add subcommands on per-project
 base!

 sadly, this seems to get no "wow!"s. :-(


I think it's pretty cool. But I like the approach SDC has taken even 
more. Using the LLVM JIT to run CTFE code.

-- 
/Jacob Carlborg



 Jun 03 2015



  parent  ketmar <ketmar ketmar.no-ip.org>  writes:


On Wed, 03 Jun 2015 13:45:06 +0200, Jacob Carlborg wrote:


 On 2015-06-02 10:54, ketmar wrote:
=20

 that was the thing i once proposed. see, we have a powerful scripting
 language inside DMD: D! yet we never used all it's power to do
 something really exciting -- like, for example, preparing command lines
 for external package fetching tool and parsing the answers. instead of
 providing a simple module for that, it all goes to be hardcoded.

 i extended CTFE engine with simple file i/o functions and "system" call
 with config-defined executables (something like sudoers list) just to
 see if it will be usable. and it's surprisingly fun even with all the
 limits and without hooks for module imports and so on.

 this system can be extended to allow writing arbitrary subcommands
 (like git). just import the corresponding subcommand module, if any,
 and CTFE it's invocation point. bingo! the system that can be extended
 without recompiling the compiler. and user can add subcommands on
 per-project base!

 sadly, this seems to get no "wow!"s. :-(


=20
 I think it's pretty cool. But I like the approach SDC has taken even
 more. Using the LLVM JIT to run CTFE code.


so SDC will benefit from JITting, as it's not required to modify CTFE=20
evaluator in any way to implement my idea. just adding some more=20
"internal functions" (like `import()`), which are nothing more than "call=20
`interpret`, use results). porting that entry points to SDC will be=20
trivial.=



 Jun 03 2015



prev sibling next sibling parent reply Jacob Carlborg <doob me.com>  writes:


On 2015-06-01 20:49, Steven Schveighoffer wrote:


 I'm not defending DIP11, just that the fact that it could be done
 without requiring an extra "dependencies" file.


One usually needs a file to set all compiler and linker flags and other 
kinds of configurations. Dub allows you to do this as well in 
cross-platform way. So when you already have all that, just put the 
dependencies there and be done with it.

-- 
/Jacob Carlborg



 Jun 02 2015



  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/2/15 3:12 AM, Jacob Carlborg wrote:

 On 2015-06-01 20:49, Steven Schveighoffer wrote:


 I'm not defending DIP11, just that the fact that it could be done
 without requiring an extra "dependencies" file.


 One usually needs a file to set all compiler and linker flags and other
 kinds of configurations. Dub allows you to do this as well in
 cross-platform way. So when you already have all that, just put the
 dependencies there and be done with it.


Actually, I don't. dmd *.d generally works fine.

-Steve



 Jun 02 2015



  parent  Jacob Carlborg <doob me.com>  writes:


On 2015-06-02 16:52, Steven Schveighoffer wrote:


 Actually, I don't. dmd *.d generally works fine.


I if it works for you, great. It doesn't work for me. And that shell 
globbing doesn't work on Windows.

-- 
/Jacob Carlborg



 Jun 03 2015



prev sibling  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 01.06.2015 um 20:49 schrieb Steven Schveighoffer:

 (...)
 Sorry, I suppose it's a small burden if you use dub frequently. I don't.
 Look what happens when I initialize a dub project:

 {
      "name": "testdub",
      "description": "A minimal D application.",
      "copyright": "Copyright © 2015, steves",
      "authors": ["steves"],
      "dependencies": {
      }
 }

 Cool, I can figure out how to massage all the existing strings, except,
 hm... dependencies. What goes in there?


As of recently, you can also directly specify dependencies to the "dub 
init" command, for example "dub init myproject tango derelict-gl". There 
was also the suggestion to make dub init (without arguments) 
interactive, which would make this a really easy matter.



 Jun 04 2015



  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/4/15 4:58 AM, Sönke Ludwig wrote:

 Am 01.06.2015 um 20:49 schrieb Steven Schveighoffer:

 (...)
 Sorry, I suppose it's a small burden if you use dub frequently. I don't.
 Look what happens when I initialize a dub project:

 {
      "name": "testdub",
      "description": "A minimal D application.",
      "copyright": "Copyright © 2015, steves",
      "authors": ["steves"],
      "dependencies": {
      }
 }

 Cool, I can figure out how to massage all the existing strings, except,
 hm... dependencies. What goes in there?


 As of recently, you can also directly specify dependencies to the "dub
 init" command, for example "dub init myproject tango derelict-gl".


That's nice. But it still isn't self-explanatory. Nor additive.


 There
 was also the suggestion to make dub init (without arguments)
 interactive, which would make this a really easy matter.


This would be better, but I still think a way to add dependencies should 
be supported on the command line. Something like dub depend someproject. 
You don't always know what a project will need at the beginning.

Of course, you could just say "well, edit the json file!". You really 
need a format that supports comments...

In any case, this doesn't negate the concerns others have raised.

I don't mean to bash dub, it's better than nothing. But I haven't used 
it for any real project yet. And when I did use it, it was not a 
straightforward experience.

-Steve



 Jun 04 2015



 next sibling parent  "wobbles" <grogan.colin gmail.com>  writes:


On Thursday, 4 June 2015 at 13:07:50 UTC, Steven Schveighoffer 
wrote:

 That's nice. But it still isn't self-explanatory. Nor additive.


Do you think it'd be better to have a list of flags to specify 
dependancies?
Like :
dub init myProj -d tango -d derelict-gl --dependency=vibe-d


 This would be better, but I still think a way to add 
 dependencies should be supported on the command line. Something 
 like dub depend someproject. You don't always know what a 
 project will need at the beginning.


Agree here, this is something I plan on working on soon.


 Of course, you could just say "well, edit the json file!". You 
 really need a format that supports comments...

 In any case, this doesn't negate the concerns others have 
 raised.

 I don't mean to bash dub, it's better than nothing. But I 
 haven't used it for any real project yet. And when I did use 
 it, it was not a straightforward experience.

 -Steve





 Jun 04 2015



prev sibling  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer:

 On 6/4/15 4:58 AM, Sönke Ludwig wrote:

 As of recently, you can also directly specify dependencies to the "dub
 init" command, for example "dub init myproject tango derelict-gl".


 That's nice. But it still isn't self-explanatory. Nor additive.


A command to add/remove dependencies might be useful. But the question 
is where to stop. Does it make sense to have a command to edit the 
description? Or add compiler flags? Or manage configurations or sub 
packages? It also gets a little hairy w.r.t. keeping possible user 
formatting in the package description file, especially once support for 
comments gets added.


 There
 was also the suggestion to make dub init (without arguments)
 interactive, which would make this a really easy matter.


 This would be better, but I still think a way to add dependencies should
 be supported on the command line. Something like dub depend someproject.
 You don't always know what a project will need at the beginning.

 Of course, you could just say "well, edit the json file!". You really
 need a format that supports comments...


Agreed, that is really needed. It's almost on top of the TODO list.


 In any case, this doesn't negate the concerns others have raised.

 I don't mean to bash dub, it's better than nothing. But I haven't used
 it for any real project yet. And when I did use it, it was not a
 straightforward experience.


The important part is to not stop with voicing criticism, but to 
actively help to improve the situation. Opening tickets or "voting" on 
existing ones, or even making an enhancement proposal or an actual PR 
would be the most constructive.

It's a little sad how some people seem to perceive DUB as being static 
and unable to evolve. That is not the case. It has started out with a 
small core functionality to get something that benefits a lot of people 
right away. But everything has been done in a way that allows to extend 
or generalize things later.

Not everyone will agree with its philosophy to provide high level 
declarative abstractions instead of a low level system that can be used 
to create abstractions within the build language, but I'm very positive 
that this approach is much more useful for the vast majority of users 
(and especially for newcomers). Retrofitting existing build approaches 
can of course sometimes come with friction, but even that should be 
solvable one way or another in most cases.

In any case, probably all of the mentioned criticism so far seems to be 
based on nothing more than missing functionality or bugs and nothing 
that would go against the existing design. An exception is support for 
other high level tasks, such as "deployment" targets or similar. This 
has never been the scope and I don't think we should go that way.

On a related note, it's also a pity that Reggae mixes incremental build 
improvements (from which DUB itself could greatly profit, too - just as 
a Ninja generator for DUB would be a nice feature) with a separate, 
layered build description. I mean there is of course no reason to not 
have alternative approaches for build descriptions available in general, 
but when mixed with a public package repository, it just leads to 
fragmentation.

Sorry for the slightly OT reply, most of this isn't targeted at you, I 
just went through the newsgroup posts for the first time after a while 
and needed to get this out.



 Jun 04 2015



 next sibling parent reply "Atila Neves" <atila.neves gmail.com>  writes:


 On a related note, it's also a pity that Reggae mixes 
 incremental build improvements (from which DUB itself could 
 greatly profit, too - just as a Ninja generator for DUB would 
 be a nice feature) with a separate, layered build description. 
 I mean there is of course no reason to not have alternative 
 approaches for build descriptions available in general, but 
 when mixed with a public package repository, it just leads to 
 fragmentation.


I can add a ninja generator to dub if you want, with the default 
being per-package compilation (since right now for dub it's one 
file or everything at once).

But I wrote reggae because:

1. Declarative is preferable but imperative is needed. My (our?) 
favourite imperative language is D, so I want to specify builds 
in it.
2. I want a build tool that builds on dub but doesn't require it
3. I wanted it to make it easy to link with C and C++

As I've mentioned before, I know the kind of things I'd want to 
do with the build system if I had a large and complicated enough 
project, and I know I wouldn't be able to do it easily using dub 
alone. As I've also mentioned before, building with dub is just 
fine for most people.

I don't know if fragmentation would be an issue. The packages are 
still dub packages and I for one will use dub.json/sdl to list my 
dependencies even if reggae is actually generating the build.

Atila



 Jun 04 2015



  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 04.06.2015 um 17:47 schrieb Atila Neves:

 On a related note, it's also a pity that Reggae mixes incremental
 build improvements (from which DUB itself could greatly profit, too -
 just as a Ninja generator for DUB would be a nice feature) with a
 separate, layered build description. I mean there is of course no
 reason to not have alternative approaches for build descriptions
 available in general, but when mixed with a public package repository,
 it just leads to fragmentation.


 I can add a ninja generator to dub if you want, with the default being
 per-package compilation (since right now for dub it's one file or
 everything at once).


That would be great! If the per-package mode has such practical 
benefits, we could also simply make it the general default, for all 
generators that can support it. However, I just briefly looked at your 
benchmark results, the performance boost seems to be solely because of 
parallel compiler invocations. We already had issues with per-module 
parallel compilation and out-of-memory errors, so this optimization 
seems to be only suitable for certain projects or build machines. So at 
least with the current compiler memory requirements I'm not sure if this 
would be a good default build mode.


 But I wrote reggae because:

 1. Declarative is preferable but imperative is needed. My (our?)
 favourite imperative language is D, so I want to specify builds in it.


This is of course perfectly fine and reasonable (although I personally 
find the D syntax to be a bit too verbose for this task, but thats 
really just personal taste - oh, and JSON is awful, too ;). The problems 
just start to creep in as soon as public DUB packages start to depend on 
Reggae - then we'll possibly get another split in the ecosystem. I'd say 
we should generally try to focus on a single standard solution, however 
that looks and however the tool is split into executables or packages.


 2. I want a build tool that builds on dub but doesn't require it


Fair enough.


 3. I wanted it to make it easy to link with C and C++


This really also needs to be solved within DUB (i.e. without resorting 
to preBuildCommands). That is going to be the next priority after 
getting the 1.0.0 features ready.


 As I've mentioned before, I know the kind of things I'd want to do with
 the build system if I had a large and complicated enough project, and I
 know I wouldn't be able to do it easily using dub alone. As I've also
 mentioned before, building with dub is just fine for most people.


Getting some of those use cases on a Wiki page or something would be 
great. There have been some ideas to approach the procedural aspect from 
a different angle - using procedural DUB plugins that can be invoked 
from within the declarative build description. This would have the 
advantage that a lot of information can be gathered about a package 
without executing procedural code (with the possible accompanying 
security risks and performance issues or the efforts needed to mitigate 
those).


 I don't know if fragmentation would be an issue. The packages are still
 dub packages and I for one will use dub.json/sdl to list my dependencies
 even if reggae is actually generating the build.


Yeah that would definitely not be an issue. I just fear (maybe 
unnecessarily) that people might start to put packages in the registry 
that can *only* be built using Reggae (or some other build tool). At 
least for libraries that would really be bad.



 Jun 10 2015



 next sibling parent reply "Atila Neves" <atila.neves gmail.com>  writes:


 That would be great! If the per-package mode has such practical 
 benefits, we could also simply make it the general default, for 
 all generators that can support it. However, I just briefly 
 looked at your benchmark results, the performance boost seems 
 to be solely because of parallel compiler invocations. We 
 already had issues with per-module parallel compilation and 
 out-of-memory errors, so this optimization seems to be only 
 suitable for certain projects or build machines. So at least 
 with the current compiler memory requirements I'm not sure if 
 this would be a good default build mode.


 From mine and Andrei's results it seems to be a much better 
default.


 This is of course perfectly fine and reasonable (although I 
 personally find the D syntax to be a bit too verbose for this 
 task, but thats really just personal taste - oh, and JSON is 
 awful, too ;). The problems just start to creep in as soon as 
 public DUB packages start to depend on Reggae - then we'll 
 possibly get another split in the ecosystem. I'd say we should 
 generally try to focus on a single standard solution, however 
 that looks and however the tool is split into executables or 
 packages.


I'm working on the syntax ;)


 As I've mentioned before, I know the kind of things I'd want 
 to do with
 the build system if I had a large and complicated enough 
 project, and I
 know I wouldn't be able to do it easily using dub alone. As 
 I've also
 mentioned before, building with dub is just fine for most 
 people.


 Getting some of those use cases on a Wiki page or something 
 would be great.


I'd have to go back to my old work project and read all the CMake 
code...


 I don't know if fragmentation would be an issue. The packages 
 are still
 dub packages and I for one will use dub.json/sdl to list my 
 dependencies
 even if reggae is actually generating the build.


 Yeah that would definitely not be an issue. I just fear (maybe 
 unnecessarily) that people might start to put packages in the 
 registry that can *only* be built using Reggae (or some other 
 build tool). At least for libraries that would really be bad.


I think that if a project is a dub package, then it needs to be 
able to be built with dub. If anything on code.dlang.org doesn't 
(correctly) build with `dub build` then that would be incredibly 
wrong.

Also, I'm not as sure as you that reggae will catch on that much 
:)

Atila



 Jun 10 2015



  parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/10/2015 09:06 AM, Atila Neves wrote:

 I think that if a project is a dub package, then it needs to be able to
 be built with dub. If anything on code.dlang.org doesn't (correctly)
 build with `dub build` then that would be incredibly wrong.


Right. And to summarize what I said elsewhere about that: "Building WITH 
dub" need NOT imply "building using dub's internal buildsystem". It only 
needs to mean "The CORRECT buildsystem can be trigged via 'dub build', 
whatever buildsystem that may be".

That's why the current situation is backwards and CAUSES splits rather 
than preventing them: We can always trigger dub's internal buildsystem 
via "dub build", but trying to get "dub build" to trigger the CORRECT 
one INSTEAD of built-in (whenever a different one is needed) is a world 
of hurt.



 Jun 10 2015



prev sibling next sibling parent  Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/10/2015 08:11 AM, Sönke Ludwig wrote:

 Am 04.06.2015 um 17:47 schrieb Atila Neves:

 I don't know if fragmentation would be an issue. The packages are still
 dub packages and I for one will use dub.json/sdl to list my dependencies
 even if reggae is actually generating the build.


 Yeah that would definitely not be an issue. I just fear (maybe
 unnecessarily) that people might start to put packages in the registry
 that can *only* be built using Reggae (or some other build tool). At
 least for libraries that would really be bad.


I've found the opposite to be true.

We can *already* get packages in code.dlang.org that work with a custom 
buildscript, but then fail when doing "dub build". I know this because 
I've *already* done it by accident multiple times. And then I get bug 
reports of "X or Y build problem under dub".

But if we can easily tell dub via dub.json, "Hey, don't use your 
internal build system to build this, I've stuck the appropriate command 
in pre/postBuildCommands", then "dub build" will automatically just 
simply work. Major bonus compatibility points the more information we 
can pass from dub to the custom command (ex: "Which 'built type' did the 
user request with '--build='?" or...well, everything provided by dub 
describe --data).

Remember, libraries do NOT need to use the same build system to be used 
together. As long as there's a common interface to...

1. Trigger a build, and

2. Get the lists of import paths and output binaries (and maybe any 
other required flags),

...then everything's golden. This is the job of a package manager. Each 
library author tells the package manager:

1. "When someone asks you to build me, here's how you do it" and

2. "When someone asks you for my import paths and output binaries 
(etc.), this is what you tell them...".


can tell it "Don't go trying to build this yourself". (I'm wondering if 

my --data= pull request.

Ok, only one possible issue left: Source libraries (as opposed to binary 
libraries) *could* be an issue just because of the differences in 
collecting the list of files to be compiled. But that's solvable two ways:

1. If your lib *requires* something like RDMD-style dependency 
collecting, then the lib must be a binary lib.

Or much, much better yet: 2. Option in dub.json to say "when you collect 
this package's list of source files, do it RDMD-style". Sure, that may 
not be appropriate for ALL packages, but that's exactly why it'd be a 
per-package OPTION.

And there it is, the *existing* issues of splitting the dub ecosystem 
are thus eliminated, without creating new splits.



 Jun 10 2015



prev sibling  parent  Jacob Carlborg <doob me.com>  writes:


On 2015-06-10 14:11, Sönke Ludwig wrote:


 Getting some of those use cases on a Wiki page or something would be
 great. There have been some ideas to approach the procedural aspect from
 a different angle - using procedural DUB plugins that can be invoked
 from within the declarative build description. This would have the
 advantage that a lot of information can be gathered about a package
 without executing procedural code (with the possible accompanying
 security risks and performance issues or the efforts needed to mitigate
 those).


Orbit, the package manager I was working on, uses Ruby as the 
description language. It uses the more traditional way of uploading 
packages to the registry, basically a ZIP with with a special directory 
layout with every file necessary to build the package.

With this model it'a straight forward to "compile" the Ruby code to JSON 
or YAML when building the package locally (same thing can be done with 
D). On the server side, the registry only needs to read the JSON file to 
get the metadata of the package. When installing the package it would 
use the Ruby file again (or JSON description, never got that far), to be 
able to run pre and post actions. This part shouldn't be any less secure 
than the current pre/post commands.

-- 
/Jacob Carlborg



 Jun 10 2015



prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/4/15 11:07 AM, Sönke Ludwig wrote:

 Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer:

 On 6/4/15 4:58 AM, Sönke Ludwig wrote:

 As of recently, you can also directly specify dependencies to the "dub
 init" command, for example "dub init myproject tango derelict-gl".


 That's nice. But it still isn't self-explanatory. Nor additive.


 A command to add/remove dependencies might be useful. But the question
 is where to stop. Does it make sense to have a command to edit the
 description? Or add compiler flags? Or manage configurations or sub
 packages? It also gets a little hairy w.r.t. keeping possible user
 formatting in the package description file, especially once support for
 comments gets added.


Editing strings that don't affect the build doesn't seem like a 
necessary thing. An interactive init would probably solve that.

However, things like adding or removing a dependency would be very nice 
without having to know the format of the file. I'm trying to think of 
things that a simple first-time user would use dub for, aside from 
adding dependencies. Perhaps creating a debug build. Other than that, I 
think editing the file for nitty gritty details is probably OK.

The use case I'm thinking of is basically:

import somepackage.somemodule;
...

compile -> can't find somepackage.somemodule.
did you mean somepackage.somemodule from libgeneric on code.dlang.org?
    (url)
    to add: dub depend add libgeneric

dub depend add libgeneric => found it, added it, added dependency

I don't have to look up anything on code.dlang.org, I don't have to do 
anything, dub figures it all out from the description file online.

I think of things like linux shell on my linuxmint installation (ubuntu 
based), which uses some cool things:

user> blah
No command 'blah' found, did you mean:
  Command 'blam' from package 'blam' (universe)


 Of course, you could just say "well, edit the json file!". You really
 need a format that supports comments...


 Agreed, that is really needed. It's almost on top of the TODO list.


That is good!


 In any case, this doesn't negate the concerns others have raised.

 I don't mean to bash dub, it's better than nothing. But I haven't used
 it for any real project yet. And when I did use it, it was not a
 straightforward experience.


 The important part is to not stop with voicing criticism, but to
 actively help to improve the situation. Opening tickets or "voting" on
 existing ones, or even making an enhancement proposal or an actual PR
 would be the most constructive.


Not using dub frequently means I have a very sparse relationship with 
it. It's kind of that chicken and egg thing. I'm sure if I started 
having to use dub for a project, I would become more involved :)


 Sorry for the slightly OT reply, most of this isn't targeted at you, I
 just went through the newsgroup posts for the first time after a while
 and needed to get this out.


No problem, please keep up the dialogue!

-Steve



 Jun 04 2015



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2015-06-04 19:44, Steven Schveighoffer wrote:


 The use case I'm thinking of is basically:

 import somepackage.somemodule;
 ...

 compile -> can't find somepackage.somemodule.
 did you mean somepackage.somemodule from libgeneric on code.dlang.org?
     (url)
     to add: dub depend add libgeneric

 dub depend add libgeneric => found it, added it, added dependency

 I don't have to look up anything on code.dlang.org, I don't have to do
 anything, dub figures it all out from the description file online.


Sounds like a nice feature but how would that work in practice? I mean, 
there's nothing that connects a particular D module or package to a 
particular Dub package.

-- 
/Jacob Carlborg



 Jun 05 2015



  parent reply Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/5/15 5:07 AM, Jacob Carlborg wrote:

 On 2015-06-04 19:44, Steven Schveighoffer wrote:


 The use case I'm thinking of is basically:

 import somepackage.somemodule;
 ...

 compile -> can't find somepackage.somemodule.
 did you mean somepackage.somemodule from libgeneric on code.dlang.org?
     (url)
     to add: dub depend add libgeneric

 dub depend add libgeneric => found it, added it, added dependency

 I don't have to look up anything on code.dlang.org, I don't have to do
 anything, dub figures it all out from the description file online.


 Sounds like a nice feature but how would that work in practice? I mean,
 there's nothing that connects a particular D module or package to a
 particular Dub package.


The source of the package does. code.dlang.org can create this 
automatically.

-Steve



 Jun 05 2015



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2015-06-05 15:17, Steven Schveighoffer wrote:


 The source of the package does. code.dlang.org can create this
 automatically.


I see two problems with this:

1. Does the registry (code.dlang.org) really know which files a Dub 
package contains? I would guess the registry mostly contains information 
where to find the source for a package, i.e. GitHub

Rubygems, for example, on the other hand would be able to do this 
because a Ruby gem (package) is ZIP archive with all the sources for the 
package included. Also, when building a gem the package description is 
"compiled", that is, all shell globs are expanded to locate all files.

2. It's possible to have a file with a completely different module name in D

-- 
/Jacob Carlborg



 Jun 06 2015



  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/06/2015 09:16 AM, Jacob Carlborg wrote:

 On 2015-06-05 15:17, Steven Schveighoffer wrote:


 The source of the package does. code.dlang.org can create this
 automatically.


 I see two problems with this:

 1. Does the registry (code.dlang.org) really know which files a Dub
 package contains? I would guess the registry mostly contains information
 where to find the source for a package, i.e. GitHub

 Rubygems, for example, on the other hand would be able to do this
 because a Ruby gem (package) is ZIP archive with all the sources for the
 package included. Also, when building a gem the package description is
 "compiled", that is, all shell globs are expanded to locate all files.

 2. It's possible to have a file with a completely different module name
 in D


Even if that stuff isn't currently known to code.dlang.org, it'd all be 
easy for it to obtain:

$ mkdir some_temp && cd some_temp
$ git clone package_URL dir && cd dir
$ dmd (flags dub already knows) -v | grep import

Would only need to do that when it detects a new version tag (which it 
already detects).



 Jun 07 2015



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2015-06-07 17:27, Nick Sabalausky wrote:


 Even if that stuff isn't currently known to code.dlang.org, it'd all be
 easy for it to obtain:

 $ mkdir some_temp && cd some_temp
 $ git clone package_URL dir && cd dir
 $ dmd (flags dub already knows) -v | grep import

 Would only need to do that when it detects a new version tag (which it
 already detects).


So the registry basically needs to compile all packages? Isn't it better 
to "compile" and upload packages then?

-- 
/Jacob Carlborg



 Jun 07 2015



  parent reply =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 07.06.2015 um 19:57 schrieb Jacob Carlborg:

 On 2015-06-07 17:27, Nick Sabalausky wrote:


 Even if that stuff isn't currently known to code.dlang.org, it'd all be
 easy for it to obtain:

 $ mkdir some_temp && cd some_temp
 $ git clone package_URL dir && cd dir
 $ dmd (flags dub already knows) -v | grep import

 Would only need to do that when it detects a new version tag (which it
 already detects).


 So the registry basically needs to compile all packages? Isn't it better
 to "compile" and upload packages then?


Isn't just the list of *modules* needed and not the list of *imports*? 
The registry can easily query the list of files of a package using the 
GitHub API. The package description is obviously already known, too, so 
this would definitely be possible to do without compiling anything.



 Jun 07 2015



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2015-06-08 08:04, Sönke Ludwig wrote:


 Isn't just the list of *modules* needed and not the list of *imports*?
 The registry can easily query the list of files of a package using the
 GitHub API. The package description is obviously already known, too, so
 this would definitely be possible to do without compiling anything.


There's still the issue with a module not having a matching filename.

-- 
/Jacob Carlborg



 Jun 07 2015



  parent  =?UTF-8?B?U8O2bmtlIEx1ZHdpZw==?= <sludwig rejectedsoftware.com>  writes:


Am 08.06.2015 um 08:47 schrieb Jacob Carlborg:

 On 2015-06-08 08:04, Sönke Ludwig wrote:


 Isn't just the list of *modules* needed and not the list of *imports*?
 The registry can easily query the list of files of a package using the
 GitHub API. The package description is obviously already known, too, so
 this would definitely be possible to do without compiling anything.


 There's still the issue with a module not having a matching filename.


I think that will only work with a target type "sourceLibrary" anyway 
(since such modules always have to be on the compiler command line), so 
it will always be at least somewhat problematic. But there is already 
some code [1] to extract the module name for "dub test", so that would 
still be easy to do without actually invoking the compiler.

[1]: 
https://github.com/D-Programming-Language/dub/blob/b3ea9e9f026bd096c60b4d1de55e2ea62bd313cd/source/dub/dub.d#L777



 Jun 09 2015



prev sibling  parent reply "ponce" <contact gam3sfrommars.fr>  writes:


On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote:

 Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer:

 This would be better, but I still think a way to add 
 dependencies should
 be supported on the command line. Something like dub depend 
 someproject.
 You don't always know what a project will need at the 
 beginning.




imho if a program consumes a particular configuration file, it 
should not also write it.
Counter examples: Visual Studio projects, MFC, IRC bouncers, and 
other technologies from Hell.



 On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote:
 Sorry for the slightly OT reply, most of this isn't targeted at 
 you, I just went through the newsgroup posts for the first time 
 after a while and needed to get this out.


Be sure your work is appreciated, I couldn't imagine working 
without DUB everyday.

As always the NG isn't the masses but some vocal subset of D 
users.

Personally I won't use any library which is not on the DUB 
registry, learning how to build something is disposable knowledge.



 Jun 04 2015



  parent  Steven Schveighoffer <schveiguy yahoo.com>  writes:


On 6/4/15 3:49 PM, ponce wrote:

 On Thursday, 4 June 2015 at 15:07:58 UTC, Sönke Ludwig wrote:

 Am 04.06.2015 um 15:07 schrieb Steven Schveighoffer:

 This would be better, but I still think a way to add dependencies should
 be supported on the command line. Something like dub depend someproject.
 You don't always know what a project will need at the beginning.




 imho if a program consumes a particular configuration file, it should
 not also write it.
 Counter examples: Visual Studio projects, MFC, IRC bouncers, and other
 technologies from Hell.


That program is not consuming a configuration file, it's editing it. 
Call it dub-depend if you want.

-Steve



 Jun 04 2015



prev sibling  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 01:36 PM, Nick Sabalausky wrote:

 There is one part of that old proposal that I have come to like though,
 and wouldn't mind seeing: There was an idea to have DMD automatically
 invoke an external tool


Oops, forgot to finish writing that thought...

As part of that old proposal, there was an idea to have DMD, when 
importing a module it couldn't find, automatically invoke an external 
tool which would tell DMD where to find the module. That way, a tool 
like RDMD would NOT need to invoke DMD twice (once to find the 
dependencies, and again to actually compile the dependencies).

I've sort of come around more to that particular idea (not the entire 
proposal though), although actually using it would likely increase 
compile times on windows (notoriously slow at launching processes, just 
look at dustmite on win), and frankly it'd probably be better just to 
incorporate rdmd's dependency searching as an optional feature of dmd.



 Jun 01 2015



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/1/15 12:16 PM, Nick Sabalausky wrote:

 On 06/01/2015 01:36 PM, Nick Sabalausky wrote:

 There is one part of that old proposal that I have come to like though,
 and wouldn't mind seeing: There was an idea to have DMD automatically
 invoke an external tool


 Oops, forgot to finish writing that thought...

 As part of that old proposal, there was an idea to have DMD, when
 importing a module it couldn't find, automatically invoke an external
 tool which would tell DMD where to find the module. That way, a tool
 like RDMD would NOT need to invoke DMD twice (once to find the
 dependencies, and again to actually compile the dependencies).


rdmd could collect dependencies while performing the actual compilation. 
Then, if the dependencies are stable (compare with the saved ones), all 
done in one run. Otherwise, discard the output and rebuild.

This can be implemented today. Would you want to work on it?

Andrei



 Jun 01 2015



 next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 03:46 PM, Andrei Alexandrescu wrote:

 rdmd could collect dependencies while performing the actual compilation.
 Then, if the dependencies are stable (compare with the saved ones), all
 done in one run. Otherwise, discard the output and rebuild.


Hmm, actually, that's a good idea.


 This can be implemented today. Would you want to work on it?


I've gotten rusty on RDMD's internals, but I'll take a look, see what I 
can manage.



 Jun 01 2015



 next sibling parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 6/1/15 12:59 PM, Nick Sabalausky wrote:

 I'll take a look, see what I
 can manage.


Thanks! -- Andrei

P.S. Note the crafty editing of the quote :o).



 Jun 01 2015



prev sibling  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/01/2015 03:59 PM, Nick Sabalausky wrote:

 On 06/01/2015 03:46 PM, Andrei Alexandrescu wrote:

 rdmd could collect dependencies while performing the actual compilation.
 Then, if the dependencies are stable (compare with the saved ones), all
 done in one run. Otherwise, discard the output and rebuild.


 Hmm, actually, that's a good idea.


 This can be implemented today. Would you want to work on it?


 I've gotten rusty on RDMD's internals, but I'll take a look, see what I
 can manage.


Unfortunately, I've hit a bit of a problem: "dmd -v" doesn't emit an 
"import  module_name (path/to)" line for imports that were provided on 
the command line.

Aside from the minor issue of it complicating parts of the code, this 
makes it impossible (AFAICT) to distinguish between "the deps have not 
changed" vs "a former dep is no longer imported and should no longer be 
compiled".



 Jun 01 2015



  parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com>  writes:


On 06/02/2015 01:03 AM, Nick Sabalausky wrote:

 On 06/01/2015 03:59 PM, Nick Sabalausky wrote:

 On 06/01/2015 03:46 PM, Andrei Alexandrescu wrote:

 rdmd could collect dependencies while performing the actual compilation.
 Then, if the dependencies are stable (compare with the saved ones), all
 done in one run. Otherwise, discard the output and rebuild.


 Hmm, actually, that's a good idea.


 This can be implemented today. Would you want to work on it?


 I've gotten rusty on RDMD's internals, but I'll take a look, see what I
 can manage.


 Unfortunately, I've hit a bit of a problem: "dmd -v" doesn't emit an
 "import  module_name (path/to)" line for imports that were provided on
 the command line.

 Aside from the minor issue of it complicating parts of the code, this
 makes it impossible (AFAICT) to distinguish between "the deps have not
 changed" vs "a former dep is no longer imported and should no longer be
 compiled".


Related to this, does anyone happen to recall why rdmd uses "dmd -v" to 
get dependencies instead of "dmd -deps"? IIRC, it used to use -deps back 
at one time.



 Jun 02 2015



  parent  Jacob Carlborg <doob me.com>  writes:


On 2015-06-02 17:01, Nick Sabalausky wrote:


 Related to this, does anyone happen to recall why rdmd uses "dmd -v" to
 get dependencies instead of "dmd -deps"? IIRC, it used to use -deps back
 at one time.


I would assumed it uses "-v" because it existed before "-deps"? I guess 
"-deps" was explicitly create because of the rdmd use case.

-- 
/Jacob Carlborg



 Jun 03 2015



prev sibling  parent  "Atila Neves" <atila.neves gmail.com>  writes:


 rdmd could collect dependencies while performing the actual 
 compilation. Then, if the dependencies are stable (compare with 
 the saved ones), all done in one run. Otherwise, discard the 
 output and rebuild.


I wrote a silly tool that gathers dependencies as it compiles. 
Essentially, it calls dmd -v and parses the result as rdmd does. 
The net effect is something akin to gcc's -MF -MD combo.

It's part of the build system I keep pestering everyone about. I 
had to write it because Ninja depends on it for proper dependency 
tracking. Make needs it too, but the Makefile code I have to 
generate is best not looked at. Make's fault, not mine.

Atila



 Jun 01 2015



prev sibling  parent  "Rikki Cattermole" <alphaglosined gmail.com>  writes:


On Monday, 1 June 2015 at 19:16:31 UTC, Nick Sabalausky wrote:

 On 06/01/2015 01:36 PM, Nick Sabalausky wrote:

 There is one part of that old proposal that I have come to 
 like though,
 and wouldn't mind seeing: There was an idea to have DMD 
 automatically
 invoke an external tool


 Oops, forgot to finish writing that thought...

 As part of that old proposal, there was an idea to have DMD, 
 when importing a module it couldn't find, automatically invoke 
 an external tool which would tell DMD where to find the module. 
 That way, a tool like RDMD would NOT need to invoke DMD twice 
 (once to find the dependencies, and again to actually compile 
 the dependencies).

 I've sort of come around more to that particular idea (not the 
 entire proposal though), although actually using it would 
 likely increase compile times on windows (notoriously slow at 
 launching processes, just look at dustmite on win), and frankly 
 it'd probably be better just to incorporate rdmd's dependency 
 searching as an optional feature of dmd.


Of course there is always shared libraries as an alternative to 
executables.



 Jun 01 2015