www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [gsoc] DUB - Ideas

reply Seb <seb wilzba.ch> writes:
So while we have quite a large list of potential GSoC projects 
[1], I think that many of these ideas are still a bit too 
abstract for potential students.

Hence, I will try to introduce this new [gsoc] series aimed at 
(1) discussing potential ideas in depth and (2) maybe finding new 
ideas. I encourage everyone (students or just curious D 
enthusiasts) to start more of these [gsoc] threads, s.t. our 
potential students can learn more about the proposed projects 
(students: please don't be shy).

The first item on my list is D's package manager DUB.

My personal favorite "missing feature" in DUB is a direct git 
support. It would be really awesome if we would implement a 
"GitRegistry" for packages, s.t. one can directly depend on git 
repositories:

dependency "my-package" version="git://.../my-branch#my-commit"

There's also a DEP about this feature [2] (though the proposed 
format there isn't set in stone). DUB's architecture is already 
fairly modular, so as a start students could look into creating a 
GitRegistry supplier [3]. There's also some previous work [4] 
which can of course be used for inspiration too. While initially, 
the `git` CLI binary can be used, it's not guaranteed that the 
user will have this installed (especially on Windows), so one 
would either need to use e.g. libgit2 (see [5] for a D binding) 
or implement Git's protocol itself (see e.g. [6]).

 community: what features do you miss the most from DUB?

Disclaimer: The span of the GSoC is three months of full time 
work and proposals from students should take this into 
consideration. The DUB project is a bit special as some 
individual ideas might take less than three months of work, but 
in this case one could just combine two or three DUB 
features/ideas into one project.

[1] https://wiki.dlang.org/GSOC_2019_Ideas
[2] https://github.com/dlang/dub/wiki/DEP6
[3] 
https://github.com/dlang/dub/blob/master/source/dub/packagesuppliers/packagesupplier.d
[4] https://github.com/dlang/dub/pull/1403
[5] https://github.com/s-ludwig/dlibgit
[6] 
https://github.com/git/git/blob/master/Documentation/technical/http-protocol.txt
Feb 28 2019
next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

 So while we have quite a large list of potential GSoC projects 
 [1], I think that many of these ideas are still a bit too 
 abstract for potential students.

 [...]


Yes beeing able to specify git and also http dependencies would 
be great.

Also there is the cross compilation support missing 
(https://github.com/dlang/dub/pull/1541).

Another topic is watch command for dub, this would be also really 
great.

Kind regards
Andre
Feb 28 2019
prev sibling next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 2/28/19 10:50 AM, Seb wrote:

 
  community: what features do you miss the most from DUB?
 


Oh my...where to start...?

- Need a clear, obvious, dead-simple, 100% reliable way to tell DUB to 
keep its grubby hands out of my project's compilation. Tell it "Run this 
command to compile, run this other command to generate docs, etc., 
here's where you can find the resulting output, and otherwise, ***stay 
out of it!!***". Both the sdl/json config and the launched process 
should support appropriate envvars so I can cleanly pass-through CLI 
information like "Make docs instead of do build" or "use LDC" to my 
scripts without complicating them too much.

- Need to be able to tell DUB "I want *THIS* compiler flag 
included/omitted", WITHOUT triggering a stupid big ugly warning, or 
having to manually translate the CLI flag into DUB-speak. Both GDC and 
LDC come with compatibility wrappers for DMD's CLI, so an additional 
abstraction layer here is neither necessary nor welcome.

- Compile times that don't make a mockery of DMD.

- DUB's default stdout output needs to STFU by default.

- DUB and the compilers themselves should be bona-fide dependencies just 
like anything else.

- Needs good, solid support for non-source dependencies, system 
dependencies, and tools that the build itself depends on, not just libs 
your source depends on.

- This following workflow needs to be a BASIC STANDARD FULLY-ROBUST 
use-case with the same priority and importance as running "dub" to build 
a project: I have a non-DUB project and I want to include a library 
(say, Vibe.D) via DUB. I have my CLI command(s) to invoke the compiler, 
and all I have to do to use Vibe.d (or whatever) is insert this as part 
of my compiler invocation:

dmd blah blah `dub include vibe-d:~0.8.3` blah blah

And boom, -I... paths, libs, Have_vibe_d, etc., all guaranteed working, 
correct and ready to go.

At one point, I did a lot of work to make that happen, and I made some 
progress, but it was just so contrary to the way dub wanted to work I 
finally gave up.

- There's probably more I'm forgetting at the moment...
Feb 28 2019
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Feb 28, 2019 at 12:45:56PM -0500, Nick Sabalausky (Abscissa) via
Digitalmars-d wrote:

 On 2/28/19 10:50 AM, Seb wrote:

  community: what features do you miss the most from DUB?




[...]

 Oh my...where to start...?
 
 - Need a clear, obvious, dead-simple, 100% reliable way to tell DUB to
 keep its grubby hands out of my project's compilation. Tell it "Run
 this command to compile, run this other command to generate docs,
 etc., here's where you can find the resulting output, and otherwise,
 ***stay out of it!!***". Both the sdl/json config and the launched
 process should support appropriate envvars so I can cleanly
 pass-through CLI information like "Make docs instead of do build" or
 "use LDC" to my scripts without complicating them too much.


+1.



 - Need to be able to tell DUB "I want *THIS* compiler flag
 included/omitted", WITHOUT triggering a stupid big ugly warning, or
 having to manually translate the CLI flag into DUB-speak. Both GDC and
 LDC come with compatibility wrappers for DMD's CLI, so an additional
 abstraction layer here is neither necessary nor welcome.


+1.



 - Compile times that don't make a mockery of DMD.


+100.  I think dub has improved in this area the last time I checked,
but still, there's room for improvement.



 - DUB's default stdout output needs to STFU by default.


+1. Though to be fair, dub is already fairly quiet by defaut, compared
to some other programs that tend to get run during builds.



 - DUB and the compilers themselves should be bona-fide dependencies
 just like anything else.


+100.  I don't have a problem with special handling of certain types of
dependencies, but the user needs to be empowered to write the same
functionality without hacking dub's source code.  Empowerment before
convenience is my motto.



 - Needs good, solid support for non-source dependencies, system
 dependencies, and tools that the build itself depends on, not just
 libs your source depends on.


A generic dependency resolution system that the user can directly access
would address this point.



 - This following workflow needs to be a BASIC STANDARD FULLY-ROBUST
 use-case with the same priority and importance as running "dub" to
 build a project: I have a non-DUB project and I want to include a
 library (say, Vibe.D) via DUB. I have my CLI command(s) to invoke the
 compiler, and all I have to do to use Vibe.d (or whatever) is insert
 this as part of my compiler invocation:
 
 dmd blah blah `dub include vibe-d:~0.8.3` blah blah
 
 And boom, -I... paths, libs, Have_vibe_d, etc., all guaranteed
 working, correct and ready to go.
 
 At one point, I did a lot of work to make that happen, and I made some
 progress, but it was just so contrary to the way dub wanted to work I
 finally gave up.


[...]

I tried doing a non-dub project with dub dependencies before, and ended
up with so much frustration that I threw in the towel and resorted to
the hack of creating an empty dummy dub project in a subdirectory, whose
sole purpose was to declare dub dependencies. I'd only run dub from the
subdirectory when one or more dependencies need to be updated, but the
actual compilation / linking is done by a different, sane build system
that pulls in the objects dub built.

The fact that such hacks are needed just to make dub work the way I want
it to, is a sign that something is fundamentally broken with its design.
Again, it's an issue of user empowerment, or lack thereof: in theory it
should be a tool that provides pluggable functionalities that the user
can integrate into whatever else he may be using.  Unfortunately, its
current design assumes the universe revolves around it, and requires
everything else to orbit around how it works. When that doesn't happen
it either works poorly, requiring contorted workarounds, or doesn't work
at all, needing to boxed off into its own sub-universe in which it's the
center of revolution before it is willing to play nice.  It's a hammer
that wields the user, rather than the other way round.  Most of the
above points are consequences of this fundamental point.


T

-- 
No! I'm not in denial!
Feb 28 2019
next sibling parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 2/28/19 8:38 PM, H. S. Teoh wrote:

 
 It's a hammer
 that wields the user, rather than the other way round.  Most of the
 above points are consequences of this fundamental point.


+1, Definitely true...plus, I like this metaphor.

Though, to be fair, that description applies to most software these 
days. But that's a whole other rant...
Feb 28 2019
prev sibling parent sarn <sarn theartofmachinery.com> writes:
On Friday, 1 March 2019 at 01:38:13 UTC, H. S. Teoh wrote:

 On Thu, Feb 28, 2019 at 12:45:56PM -0500, Nick Sabalausky 
 (Abscissa) via Digitalmars-d wrote:

 - This following workflow needs to be a BASIC STANDARD 
 FULLY-ROBUST use-case with the same priority and importance as 
 running "dub" to build a project: I have a non-DUB project and 
 I want to include a library (say, Vibe.D) via DUB. I have my 
 CLI command(s) to invoke the compiler, and all I have to do to 
 use Vibe.d (or whatever) is insert this as part of my compiler 
 invocation:
 
 dmd blah blah `dub include vibe-d:~0.8.3` blah blah
 
 And boom, -I... paths, libs, Have_vibe_d, etc., all guaranteed 
 working, correct and ready to go.
 
 At one point, I did a lot of work to make that happen, and I 
 made some progress, but it was just so contrary to the way dub 
 wanted to work I finally gave up.


 [...]

 I tried doing a non-dub project with dub dependencies before, 
 and ended up with so much frustration that I threw in the towel 
 and resorted to the hack of creating an empty dummy dub project 
 in a subdirectory, whose sole purpose was to declare dub 
 dependencies. I'd only run dub from the subdirectory when one 
 or more dependencies need to be updated, but the actual 
 compilation / linking is done by a different, sane build system 
 that pulls in the objects dub built.


I've also had this use case, and also ended up hacking around it 
using a dummy empty project.

I wrote a tool that parsed the output of "dub describe", but it 
only mostly works because that stuff is a confusing mess.
Mar 01 2019
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

  community: what features do you miss the most from DUB?



- SOUNDNESS: dub.selections.json doesn't encode deps for all 
comfigurations. The design isn't sound.

- configurations-based dependencies, or OS-based dependencies: if 
that is possible at all

- remove optional dependencies, removedub fetch/remove (it should 
never matter what is "installed"), Optional dependencies can be 
done with a dynamic loader anyway, i na way that is SOUND,

- remove SDLang support ("a stratagic mistake"), we are paying 
for this since it was introduced, with CONFUSION and having to 
double the recommendations on recipe files ("if you use JSON, use 
this... if you use SDLang, blah...")

- remove `dub describe` (no IDE really relies on it AFAIK), do 
not check the network by default which makes `dub describe` 
unusable in the first place

- build faster

- choose to keep one: overrides, add-local or add-path... too 
many way to have user-defined modifications

- colors, with way easier error messages! DUB IS THE PRIMARY 
INTERFACE TO D.

- remove `dub search`, you can as well type code.dlang.org in the 
browser, and it will be a better experience

There are too many features that scare people in DUB!
Mar 01 2019
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:


 - remove SDLang support ("a stratagic mistake"), we are paying 
 for this since it was introduced, with CONFUSION and having to 
 double the recommendations on recipe files ("if you use JSON, 
 use this... if you use SDLang, blah...")


I'll stake my flag on that hill! Remove JSON if anything :-) It's 
ugly, it's noisy, and it doesn't allow comments. SDLang is much 
cleaner and human-friendly.
Mar 01 2019
next sibling parent Andrea Fontana <nospam example.org> writes:
On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:

 On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:


 - remove SDLang support ("a stratagic mistake"), we are paying 
 for this since it was introduced, with CONFUSION and having to 
 double the recommendations on recipe files ("if you use JSON, 
 use this... if you use SDLang, blah...")


 I'll stake my flag on that hill! Remove JSON if anything :-) 
 It's ugly, it's noisy, and it doesn't allow comments. SDLang is 
 much cleaner and human-friendly.


I agree. I started using json and then I switched to sdl. I had 
some difficulties but now I vote for sdl and I don't like the 
idea to switch back to json format.
Mar 01 2019
prev sibling next sibling parent reply bauss <jj_1337 live.dk> writes:
On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:

 On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:


 - remove SDLang support ("a stratagic mistake"), we are paying 
 for this since it was introduced, with CONFUSION and having to 
 double the recommendations on recipe files ("if you use JSON, 
 use this... if you use SDLang, blah...")


 I'll stake my flag on that hill! Remove JSON if anything :-) 
 It's ugly, it's noisy, and it doesn't allow comments. SDLang is 
 much cleaner and human-friendly.


No. I like json more instead of sdl. My whole build process is 
build upon json so it makes it easy to work with everything else 
that isn't dub related.

And there are json versions that allow comments, just switch to 
json5.

https://json5.org/
Mar 01 2019
parent Mike Parker <aldacron gmail.com> writes:
On Friday, 1 March 2019 at 11:11:56 UTC, bauss wrote:


 No. I like json more instead of sdl. My whole build process is 
 build upon json so it makes it easy to work with everything 
 else that isn't dub related.

 And there are json versions that allow comments, just switch to 
 json5.

 https://json5.org/


Well, if they would make it less ugly and noisy, then it would be 
useful!

This shows why we should continue to support both.
Mar 01 2019
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:

 I'll stake my flag on that hill! Remove JSON if anything :-) 
 It's ugly, it's noisy, and it doesn't allow comments. SDLang is 
 much cleaner and human-friendly.


Let's disagree!
- JSON schemas
- lots of parsers, you can parse it at compile-time
- it's well known
- the "best language" for 20 lines recipe files has very little 
upside
Mar 01 2019
parent reply JN <666total wp.pl> writes:
On Friday, 1 March 2019 at 16:21:19 UTC, Guillaume Piolat wrote:

 On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:

 I'll stake my flag on that hill! Remove JSON if anything :-) 
 It's ugly, it's noisy, and it doesn't allow comments. SDLang 
 is much cleaner and human-friendly.


 Let's disagree!
 - JSON schemas
 - lots of parsers, you can parse it at compile-time
 - it's well known
 - the "best language" for 20 lines recipe files has very little 
 upside


It's obvious no one will win the SDL vs JSON debate. The only 
solution is to dump both and switch to YAML :)
Mar 01 2019
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 1 March 2019 at 17:22:47 UTC, JN wrote:

 It's obvious no one will win the SDL vs JSON debate. The only 
 solution is to dump both and switch to YAML :)


Believe it or not, there is a proposal for a third format (the 
'tree' format).
https://github.com/dlang/dub/wiki/DEP7

* Obligatory reference to https://xkcd.com/927/ *

I wish one of the two got dropped. I'm using SDL now since it's 
officially recommended, but I'd prefer to only keep JSON since it 
is a clear format that newcomers don't need to familiarize with, 
while SDL (still) to me looks like a bunch of tags floating 
around without clear structure.

Keeping only SDL is fine to me too, it's better than having two 
competing standards.
Mar 01 2019
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 3/1/19 1:13 PM, Dennis wrote:

 On Friday, 1 March 2019 at 17:22:47 UTC, JN wrote:

 It's obvious no one will win the SDL vs JSON debate. The only solution 
 is to dump both and switch to YAML :)


 
 Believe it or not, there is a proposal for a third format (the 'tree' 
 format).
 https://github.com/dlang/dub/wiki/DEP7


Not that I think there's even a slightest chance DUB will ever gain 
another format, but that page contains some questionable claims about 
SDL, and at least one claim that's just downright false:

It says:
----------------------------------
[Tree's] Comparison to SDL:
     Easy fast syntax
     Syntax highlighting in few editors (Idea, VSCode, Atom, SynWrite)
     More implementations (D,TS/JS)
----------------------------------

The third claim is flat out wrong. SDL has implementations for D, Java, 
Python and Ruby (that I know of). In fact. the D implementation is the 
youngest of those four. (Sometimes I get the impression that people 
think I created SDLang, and did so for D, despite my attempts to dispel 
that notion. I was looking for a sane alternative to XML/JSON/YAML, 
found SDL, and wanted to use it in D. Just like how I once looked for a 
sane alternative to C++ and Java2/4, and found D.)

The second claim is tenuous at best: Even without a dedicated "SDLang 
highlighter", I can go right through my editor's list of supported 
highlighters and most of them do an entirely useable job highlighting 
SDLang. It's not exactly a complicated language: It's basically just 
custom identifiers and a few literals and grouping symbols. Just like 
JSON/etc, there's just not much semantic or syntactic detail there TO 
highlight.

The first one is a highly subjective matter of opinion. SDL and Tree 
(based on what I can tell from the example provided) *both* have very 
simple easy syntaxes that put XML, JSON and YAML to shame. Comparing the 
two directly against each other, well, they're pretty much on par with 
each other in this regard, any advantage to be had is slight at best.

Someone else mentioned TOML, too. Just looked it up. That's also a very 
nice data language too, just like Tree and SDLang are. Hell, there's so 
much out there that just puts the XML/JSON/YAML popular kids clique to 
shame. I find it absolutely disgusting, shameful and frankly, 
professionally negligent, that a *technical* field such as ours 
routinely prioritizes and even promotes following the cool kids' table 
over actual, freaking technical merit.
Mar 02 2019
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Saturday, 2 March 2019 at 18:22:01 UTC, Nick Sabalausky 
(Abscissa) wrote:

 Someone else mentioned TOML, too. Just looked it up. That's 
 also a very nice data language too, just like Tree and SDLang 
 are. Hell, there's so much out there that just puts the 
 XML/JSON/YAML popular kids clique to shame. I find it 
 absolutely disgusting, shameful and frankly, professionally 
 negligent, that a *technical* field such as ours routinely 
 prioritizes and even promotes following the cool kids' table 
 over actual, freaking technical merit.



Well, to me the ONLY reasons DUB use SDLang and cargo use TOML is 
obviously... to make things as different as possible! It's just 
trying to be different for being different's sake.

Sociology explains this way more than any technical merit, we are 
prone to this in the D community because well it's an alternative 
language, it selects for people who likes alternatives.

This is the elephant in the room, why else would we argue (with 
so much passion) the technical merit of 20-lines recipe files?

And TOML is really tragic, I don't know if you have tried using 
it, I have and reverted to XML.
Mar 03 2019
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 3/3/19 7:01 AM, Guillaume Piolat wrote:

 
 Well, to me the ONLY reasons DUB use SDLang and cargo use TOML is 
 obviously... to make things as different as possible! It's just trying 
 to be different for being different's sake.
 
 Sociology explains this way more than any technical merit, we are prone 
 to this in the D community because well it's an alternative language, it 
 selects for people who likes alternatives.
 
 This is the elephant in the room, why else would we argue (with so much 
 passion) the technical merit of 20-lines recipe files?


1. Because I've already wasted enough of my life suffering shitty tools.

2. Because no amount of insignificance will ever turn an inferior choice 
into a superior one.

3. Because any coup needs to start somewhere.

As for the elephant, it's only a photo on the wall...


 And TOML is really tragic, I don't know if you have tried using it, I 
 have and reverted to XML.


I haven't. This thread is the first I've heard of it. That's unfortunate 
to hear as it *seems* very nice at a glance. What issues did you have?
Mar 03 2019
parent Guillaume Piolat <first.last gmail.com> writes:
On Monday, 4 March 2019 at 06:27:33 UTC, Nick Sabalausky 
(Abscissa) wrote:


 I haven't. This thread is the first I've heard of it. That's 
 unfortunate to hear as it *seems* very nice at a glance. What 
 issues did you have?



TOML make difficult for humans to write "Array of Tables"
https://github.com/toml-lang/toml

because it's where the flat idea break down.
Array of Tables of Array of Tables would be increasingly 
unworkable, whereas in JSON nesting happens normally.
Mar 04 2019
prev sibling parent Ralph Amissah <ralph.amissah gmail.com> writes:
On 2019-03-01 17:22, Digitalmars-d wrote:

 On Friday, 1 March 2019 at 16:21:19 UTC, Guillaume Piolat wrote:

 On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:

 I'll stake my flag on that hill! Remove JSON if anything :-) It's
 ugly, it's noisy, and it doesn't allow comments. SDLang is much
 cleaner and human-friendly.


 
 Let's disagree!
 - JSON schemas
 - lots of parsers, you can parse it at compile-time
 - it's well known
 - the "best language" for 20 lines recipe files has very little upside


 
 It's obvious no one will win the SDL vs JSON debate. The only solution is to
 dump both and switch to YAML :)


or to keep bicycle-shedding: TOML
- (even) "nicer" than SDL
-  more sane than YAML
(one thing Rust perhaps got better)

https://forum.dlang.org/search?q=toml&scope=group%3AdigitalmarsD
Mar 01 2019
prev sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:

 - colors, with way easier error messages! DUB IS THE PRIMARY 
 INTERFACE TO D.


For ref, there was this initiative by gdelazzari in June.
https://forum.dlang.org/post/vefplpvwvrcbednsufli forum.dlang.org
Mar 01 2019
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

 So while we have quite a large list of potential GSoC projects 
 [1], I think that many of these ideas are still a bit too 
 abstract for potential students.

 Hence, I will try to introduce this new [gsoc] series aimed at 
 (1) discussing potential ideas in depth and (2) maybe finding 
 new ideas. I encourage everyone (students or just curious D 
 enthusiasts) to start more of these [gsoc] threads, s.t. our 
 potential students can learn more about the proposed projects 
 (students: please don't be shy).

 The first item on my list is D's package manager DUB.

 My personal favorite "missing feature" in DUB is a direct git 
 support. It would be really awesome if we would implement a 
 "GitRegistry" for packages, s.t. one can directly depend on git 
 repositories:

 dependency "my-package" version="git://.../my-branch#my-commit"

 There's also a DEP about this feature [2] (though the proposed 
 format there isn't set in stone). DUB's architecture is already 
 fairly modular, so as a start students could look into creating 
 a GitRegistry supplier [3]. There's also some previous work [4] 
 which can of course be used for inspiration too. While 
 initially, the `git` CLI binary can be used, it's not 
 guaranteed that the user will have this installed (especially 
 on Windows), so one would either need to use e.g. libgit2 (see 
 [5] for a D binding) or implement Git's protocol itself (see 
 e.g. [6]).

  community: what features do you miss the most from DUB?


Some of these might have been done/fixed since the last time I 
ran into them. I wouldn't know because I keep not using dub 
except to make my packages available to everyone else.

1. Bugs getting fixed.

2. Why oh why oh why does it build dependent packages 
serially?????

3. Build per D package by default instead of all-at-once, with 
proper dependency tracking.

4. Ability to use a real build system where needed.

5. `dub describe` actually working, or, even better, an easy to 
use API to query the build. Did you know that the output of `dub 
describe -c unittest` doesn't have the string "-unittest" 
anywhere, much less in "dflags"? Now you do.

6. Not having to write the same code over and over again to check 
if dub dependencies are already downloaded, somehow running 
faster than dub's own checks even if I write said code in Emacs 
Lisp.

7. Ability to avoid the "lib" prefix for libraries on Linux 
(think Python shared objects).

8. Possibility of using a shell other than CMD on Windows 
(preBuildCommands, etc.).

9. Be able to run two dub instances in parallel without crashing 
the linker.

10. A more flexible registry engine so that one can use code that 
lives in a private gitlab instance.


Off the top of my head, that's it.
Mar 01 2019
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Friday, 1 March 2019 at 14:16:17 UTC, Atila Neves wrote:

 5. `dub describe` actually working, or, even better, an easy to 
 use API to query the build. Did you know that the output of 
 `dub describe -c unittest` doesn't have the string "-unittest" 
 anywhere, much less in "dflags"? Now you do.


That is actually to some extent expected behaviour. -c unittest 
only refers to the unittest configuration, not the unittest 
*build*.

dub describe -c unittest -b unittest

will give you "unittests" under targets[0].buildSettings.options

Of course those options still need to be converted in to the 
relevant compiler flags.
Mar 01 2019
parent Atila Neves <atila.neves gmail.com> writes:
On Friday, 1 March 2019 at 14:28:54 UTC, John Colvin wrote:

 On Friday, 1 March 2019 at 14:16:17 UTC, Atila Neves wrote:

 5. `dub describe` actually working, or, even better, an easy 
 to use API to query the build. Did you know that the output of 
 `dub describe -c unittest` doesn't have the string "-unittest" 
 anywhere, much less in "dflags"? Now you do.


 That is actually to some extent expected behaviour. -c unittest 
 only refers to the unittest configuration, not the unittest 
 *build*.

 dub describe -c unittest -b unittest

 will give you "unittests" under targets[0].buildSettings.options

 Of course those options still need to be converted in to the 
 relevant compiler flags.


Bad example from me. Replace "-unittest" with "-g -debug" for the 
debug build, which is the default.

Why anyone would want to use the "unittest" configuration (which 
is treated differently by dub) in a non-unittest build is beyond 
me.
Mar 01 2019
prev sibling next sibling parent Radu <void null.pt> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

 So while we have quite a large list of potential GSoC projects 
 [1], I think that many of these ideas are still a bit too 
 abstract for potential students.

 [...]


- Cross compiling support https://github.com/dlang/dub/issues/1523
- Usable with private repos 
https://github.com/dlang/dub/issues/1499
Mar 01 2019
prev sibling next sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:


  community: what features do you miss the most from DUB?


A properly working preGenerateCommands 
https://github.com/dlang/dub/issues/1474

- Bastiaan.
Mar 01 2019
prev sibling next sibling parent reply jpvalley <vikram.mullick yale.edu> writes:
Hi, all! I discovered D looking through the GSoC organizations 
page, and I'm really intrigued by it. I installed the compiler 
and messed around with it a bit, but I don't really have much 
wisdom as to what could make a good project.

One of the ideas I came up with was Python extension modules for 
D. Does this seem like something that would be useful/feasible? 
I'm planning on doing more research when I get on spring break, 
but I was hoping to bounce this idea off people who would know 
better.

Cheers!
Mar 02 2019
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 03/03/2019 3:55 AM, jpvalley wrote:

 Hi, all! I discovered D looking through the GSoC organizations page, and 
 I'm really intrigued by it. I installed the compiler and messed around 
 with it a bit, but I don't really have much wisdom as to what could make 
 a good project.
 
 One of the ideas I came up with was Python extension modules for D. Does 
 this seem like something that would be useful/feasible? I'm planning on 
 doing more research when I get on spring break, but I was hoping to 
 bounce this idea off people who would know better.
 
 Cheers!


https://github.com/ariovistus/pyd

If you want to talk to us more casually you can do so, by joining IRC 
(FreeNode), Slack or Discord (https://discord.gg/3aUX5nv).

Happy coding!
Mar 02 2019
prev sibling next sibling parent Jon Degenhardt <jond noreply.com> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

  community: what features do you miss the most from DUB?


Dub support for building code.dlang.org packages with LDC's LTO. 
The idea would be to support something analogous to the way LDC 
provides LTO support for druntime and phobos. See 
https://github.com/ldc-developers/ldc/releases/tag/v1.9.0 for a 
description.

Building with LTO against druntime/phobos is a substantial 
performance win for certain classes of programs. This would 
likely hold true for a number of code.dlang.org packages as well. 
My benchmarks involving druntime/phobos are here: 
https://github.com/eBay/tsv-utils/blob/master/docs/lto-pgo-study.md.

LDC supports druntime/phobos by shipping these prebuilt as LTO 
libraries (with IR code). For  packages it probably makes more 
sense to build them from source after being downloaded.

This is probably not a long project, but it might fit well as 
part of a suite of Dub changes. Dub may well have the necessary 
facilities already, I'm not sure, but this is not supported in a 
straightforward manner. There are likely some open questions, for 
example, should the package provider mark whether package 
supports LTO or not.
Mar 02 2019
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

 My personal favorite "missing feature" in DUB is a direct git 
 support. It would be really awesome if we would implement a 
 "GitRegistry" for packages, s.t. one can directly depend on git 
 repositories:

 dependency "my-package" version="git://.../my-branch#my-commit"


One can already just add a git submodule and point the dub 
dependency declaration at the resulting directory.  What features 
are you missing that aren't covered by this?
Mar 03 2019
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 3 March 2019 at 13:24:21 UTC, Joseph Rushton Wakeling 
wrote:

 One can already just add a git submodule and point the dub 
 dependency declaration at the resulting directory.


Dub doesn't support git submodules (because it downloads the Zip 
archive from GitHub/GitLab).
So `dub fetch my-package` won't fetch any submodules. This is 
already a huge annoyance, but the mentioned point was about 
supporting git checkouts in dub, s.t. you can have Dub 
dependencies from any git source like in almost any other major 
package manager without them needing to be registered on the dub 
registry or to be manually checked out.
Mar 03 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 3 March 2019 at 13:43:15 UTC, Seb wrote:

 On Sunday, 3 March 2019 at 13:24:21 UTC, Joseph Rushton 
 Wakeling wrote:

 One can already just add a git submodule and point the dub 
 dependency declaration at the resulting directory.


 Dub doesn't support git submodules (because it downloads the 
 Zip archive from GitHub/GitLab).


I think we may be talking at cross-purposes.  Here's what I was 
suggesting w.r.t. using git repos as a source for dub 
dependencies:

(1) add a submodule to your current project (let's use mir-glas 
as an example):
     git submodule add --name mir-glas 
https://github.com/libmir/mir-glas.git mir-glas

(2) add a dependency to dub.json pointing at the submodule 
directory:

         "dependencies": {
             "mir-glas": { "path": "./mir-glas" }
         }

This is only really needed though if one wants to create 
dependencies on projects that are not public DUB packages.  I 
think this ought to also support nested submodules, though I have 
not tried it.

Now OTOH if I understand your concerns right:


 So `dub fetch my-package` won't fetch any submodules. This is 
 already a huge annoyance, but the mentioned point was about 
 supporting git checkouts in dub, s.t. you can have Dub 
 dependencies from any git source like in almost any other major 
 package manager without them needing to be registered on the 
 dub registry or to be manually checked out.


... you're concerned about the case where a DUB package relies on 
submodules for its complete source code?  I wonder if you could 
be more specific about the exact use case you have in mind, and 
why one would want to use submodules in that way?

What strikes me is that all the use cases one could think of 
probably come down to different examples of this: "How does one 
make DUB aware of and able to handle a codebase that does not 
include its own DUB package config?"

I'd suggest that considering that problem independent of any 
particular VCS might be more illuminating than asking how to make 
DUB git-repo-aware.
Mar 03 2019
parent reply Atila Neves <atila.neves gmail.com> writes:
On Sunday, 3 March 2019 at 15:59:44 UTC, Joseph Rushton Wakeling 
wrote:

 On Sunday, 3 March 2019 at 13:43:15 UTC, Seb wrote:

 On Sunday, 3 March 2019 at 13:24:21 UTC, Joseph Rushton 
 Wakeling wrote:

 One can already just add a git submodule and point the dub 
 dependency declaration at the resulting directory.


 Dub doesn't support git submodules (because it downloads the 
 Zip archive from GitHub/GitLab).


 I think we may be talking at cross-purposes.  Here's what I was 
 suggesting w.r.t. using git repos as a source for dub 
 dependencies:

 (1) add a submodule to your current project (let's use mir-glas 
 as an example):
     git submodule add --name mir-glas 
 https://github.com/libmir/mir-glas.git mir-glas

 (2) add a dependency to dub.json pointing at the submodule 
 directory:

         "dependencies": {
             "mir-glas": { "path": "./mir-glas" }
         }

 This is only really needed though if one wants to create 
 dependencies on projects that are not public DUB packages.  I 
 think this ought to also support nested submodules, though I 
 have not tried it.


This doesn't work. Worse, it actually does work if you test 
adding this dub package (the one that has mir-glas as a 
dependency) locally by using 
`path="/path/to/thing/needing/mir-glass"`. Then you tag a new 
version, try using a version dependency and... oops. Check out 
all the v0.2.x releases for autowrap here:

https://github.com/kaleidicassociates/autowrap/releases

Notice how they're all from two different days. That was me 
trying to make pyd a dependency as a git submodule. I have to 
give up and go up to v0.3.0 since none of those tags work.
Mar 04 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 4 March 2019 at 10:19:29 UTC, Atila Neves wrote:

 This doesn't work.


Exactly which bits of it?  I have actual projects that use git 
submodules to provide some dependencies for dub.  There are no 
nested submodules, though.

A constraint is that I'm tied to a _specific_ version of the 
dependency (determined by what commit or tag the git submodule 
checks out), but that's not actually a problem for the use-case 
(generally for production code I find it better to have precise 
control over dependency versions).
Mar 04 2019
next sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 4 March 2019 at 13:12:09 UTC, Joseph Rushton Wakeling 
wrote:

 On Monday, 4 March 2019 at 10:19:29 UTC, Atila Neves wrote:

 This doesn't work.


 Exactly which bits of it?  I have actual projects that use git 
 submodules to provide some dependencies for dub.  There are no 
 nested submodules, though.

 A constraint is that I'm tied to a _specific_ version of the 
 dependency (determined by what commit or tag the git submodule 
 checks out), but that's not actually a problem for the use-case 
 (generally for production code I find it better to have precise 
 control over dependency versions).


Have you published any of these projects on the dub registry?
It wouldn't work for any potential users as dub wouldn't download 
the git submodules for you.
Mar 04 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 4 March 2019 at 13:26:43 UTC, Seb wrote:

 Have you published any of these projects on the dub registry?
 It wouldn't work for any potential users as dub wouldn't 
 download the git submodules for you.


No, but that's the point: it's a way of integrating _private_ 
internal projects into DUB.  I wouldn't use it for public DUB 
packages, because what's the point?

The question is why you want a public DUB package to rely on 
submodules for parts of its code.  What's the use-case?  You 
still haven't described it.
Mar 04 2019
parent reply Seb <seb wilzba.ch> writes:
On Monday, 4 March 2019 at 13:52:03 UTC, Joseph Rushton Wakeling 
wrote:

 The question is why you want a public DUB package to rely on 
 submodules for parts of its code.  What's the use-case?  You 
 still haven't described it.


You fix a bug in let's say Vibe.d (or any other library for that 
matter). You don't want to wait 6 months until your bug fix gets 
merged and released (if ever). I have run into this many many 
times.
In some cases I had to result to registering a duplicate repo on 
the registry as there was a breaking change in DMD and I (and the 
users of a depending library) wanted to continue using it.
Note that this shortcoming of dub was also the main reason why 
Dlang Community has been created.
Mar 04 2019
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 4 March 2019 at 14:04:41 UTC, Seb wrote:

 You fix a bug in let's say Vibe.d (or any other library for 
 that matter). You don't want to wait 6 months until your bug 
 fix gets merged and released (if ever). I have run into this 
 many many times.


Perfectly reasonable use-case, the question is whether the right 
solution is to tie DUB to direct support for git submodules.

I'll slightly rephrase my earlier remarks: this and other cases I 
can think of all seem to come down to:

     How does one make DUB aware of and able to handle a codebase 
that
     is not registered in the main DUB package repo and possibly 
does
     not even define its own DUB package config?

What I would suggest is that rather than building in direct 
support for git, which is only one VCS, we consider instead how 
to make it possible to specify dependencies in terms of two kinds 
of instructions:

   * how to fetch a dependency's code and place it in a directory 
determined
     either by DUB itself or by another setting in the `dub.json` 
file

     - this might be done by a git clone or checkout, or by 
fetching and
       unzipping an archive file, or by a number of other options

   * how to define DUB packaging/dependency settings if the 
dependency does
     not define them in its own codebase

That should be a more flexible approach that would allow the same 
outcomes.
Mar 04 2019
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 4 March 2019 at 13:12:09 UTC, Joseph Rushton Wakeling 
wrote:

 On Monday, 4 March 2019 at 10:19:29 UTC, Atila Neves wrote:

 This doesn't work.


 Exactly which bits of it?


Try starting a new dub project that depends on autowrap v0.2.0.


 I have actual projects that use git submodules to provide some 
 dependencies for dub.  There are no nested submodules, though.


This might be the crucial difference - it might be a submodules 
thing.
Mar 05 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 5 March 2019 at 10:44:04 UTC, Atila Neves wrote:

 Try starting a new dub project that depends on autowrap v0.2.0.


Indeed if you just add a regular dub dependency this won't work 
(because dub doesn't know how to check out the pyd submodule).  
But try instead:

* in your main project add a submodule for autowrap (checking out 
v0.2.0)

* in your dub config specify the dependency as

       "autowrap": { "path": "./path/to/autowrap_submodule" }

(or whatever the SDL equivalent is).

That works.
Mar 06 2019
next sibling parent reply Seb <seb wilzba.ch> writes:
On Wednesday, 6 March 2019 at 12:23:07 UTC, Joseph Rushton 
Wakeling wrote:

 On Tuesday, 5 March 2019 at 10:44:04 UTC, Atila Neves wrote:

 Try starting a new dub project that depends on autowrap v0.2.0.


 Indeed if you just add a regular dub dependency this won't work 
 (because dub doesn't know how to check out the pyd submodule).  
 But try instead:

 * in your main project add a submodule for autowrap (checking 
 out v0.2.0)

 * in your dub config specify the dependency as

       "autowrap": { "path": "./path/to/autowrap_submodule" }

 (or whatever the SDL equivalent is).

 That works.


Nope it only works locally. It does not work for anyone using it 
via dub. As mentioned earlier, try fetching this package (or any 
package with submodules) from the dub registry.
Mar 06 2019
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 6 March 2019 at 12:29:17 UTC, Seb wrote:

 Nope it only works locally. It does not work for anyone using 
 it via dub. As mentioned earlier, try fetching this package (or 
 any package with submodules) from the dub registry.


Yes.  But that's because you're doing something fundamentally 
inappropriate for the DUB package repo, which is to have source 
code dependencies defined somewhere (the git submodule system) 
that the DUB repo inherently cannot know about.

Basically if you want to have git submodules in the mix then you 
have to organize the entire dependency hierarchy for everything 
involved via git submodules rather than via the DUB package repo. 
  The git submodule issue isn't a blocker towards DUB being able 
to handle these dependencies, it's a blocker on the DUB package 
repo containing packages whose source code dependencies are not 
known to the DUB package repo.  Those are not quite the same 
things.

In other words, if you choose to bypass DUB's supply mechanisms 
and hand control to something else (like git), then you need to 
respect that separation of concerns: handle the supply via the 
tool you asked to provide supply, and just give DUB sufficient 
config to build the code once it has been supplied.

And it may be important to have that separation of concerns in 
your face -- after all, if it's trivial to just set a DUB package 
dependency to point at an arbitrary git repo, what's to stop me 
setting up a convenience library that depends on a git fork of 
vibe, which includes dodgy tracking components?

I trust that _your_ fork of vibe is well-motivated and only has 
bugfixes, but I'm not sure that I would trust that approach in 
general, and I am not so sure it's a good idea to open it up as a 
possibility in the DUB package repo.

After all, if you can't rely on upstream vibe to get out timely 
bugfix releases, maybe the problem is one of process rather than 
of DUB itself.  I'm not sure it isn't better to fix that process 
rather than open up a Pandora's box of dependency options to work 
around it.
Mar 06 2019
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 6 March 2019 at 12:23:07 UTC, Joseph Rushton 
Wakeling wrote:

 On Tuesday, 5 March 2019 at 10:44:04 UTC, Atila Neves wrote:

 Try starting a new dub project that depends on autowrap v0.2.0.


 Indeed if you just add a regular dub dependency this won't work 
 (because dub doesn't know how to check out the pyd submodule).  
 But try instead:

 * in your main project add a submodule for autowrap (checking 
 out v0.2.0)

 * in your dub config specify the dependency as

       "autowrap": { "path": "./path/to/autowrap_submodule" }

 (or whatever the SDL equivalent is).

 That works.


I don't want to do that. I want to use dub as intended, and have 
it figure out versions for me. I also want to make my code 
available for others, have them list my package in their dub.sdl 
and for Bob to subsequently be their proverbial uncle.

It's bad enough I can't use dub as a build system, I'm certainly 
not going to manually do its job as a package manager!  
Especially if one of the submodules depends on a dub package that 
depends on a dub package that depends on a dub package...
Mar 06 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 6 March 2019 at 14:27:56 UTC, Atila Neves wrote:

 I don't want to do that. I want to use dub as intended


I would suggest that DUB stopped being used as intended the 
moment someone submitted a package to the DUB repo with source 
code dependencies that were not also packages in the DUB repo.
Mar 06 2019
parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 6 March 2019 at 23:38:20 UTC, Joseph Rushton 
Wakeling wrote:

 On Wednesday, 6 March 2019 at 14:27:56 UTC, Atila Neves wrote:

 I don't want to do that. I want to use dub as intended


 I would suggest that DUB stopped being used as intended the 
 moment someone submitted a package to the DUB repo with source 
 code dependencies that were not also packages in the DUB repo.


A fair point.

At the time I was trying to pin the pyd version because there was 
a bug in the dub registry that prevented me from doing so in the 
regular fashion. Then the bug got fixed and I moved on.
Mar 07 2019
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 7 March 2019 at 16:01:18 UTC, Atila Neves wrote:

 A fair point.

 At the time I was trying to pin the pyd version because there 
 was a bug in the dub registry that prevented me from doing so 
 in the regular fashion. Then the bug got fixed and I moved on.


It wasn't meant as a personal attack!  Sorry if it came over that 
way.

I get the motivation, but it feels somewhat dodgy to allow 
packages in the DUB repository to depend on arbitrary git forks.  
What might make more sense is to allow a way for the build call 
(or settings in `dub.selections.json`) to easily override 
dependency locations, which can be used for temporary workaround 
like this.  What do you think?

It's obviously not great if e.g. one has a library package that 
triggers a bug in one of its dependencies and has no possibility 
of an in-code workaround, but it seems more secure and consistent 
to require dependency overrides to be chosen by the user rather 
than allowed in any circumstances.
Mar 17 2019
next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Sunday, 17 March 2019 at 13:54:54 UTC, Joseph Rushton Wakeling 
wrote:

 On Thursday, 7 March 2019 at 16:01:18 UTC, Atila Neves wrote:

 A fair point.

 At the time I was trying to pin the pyd version because there 
 was a bug in the dub registry that prevented me from doing so 
 in the regular fashion. Then the bug got fixed and I moved on.


 It wasn't meant as a personal attack!  Sorry if it came over 
 that way.

 I get the motivation, but it feels somewhat dodgy to allow 
 packages in the DUB repository to depend on arbitrary git 
 forks.  What might make more sense is to allow a way for the 
 build call (or settings in `dub.selections.json`) to easily 
 override dependency locations, which can be used for temporary 
 workaround like this.  What do you think?

 It's obviously not great if e.g. one has a library package that 
 triggers a bug in one of its dependencies and has no 
 possibility of an in-code workaround, but it seems more secure 
 and consistent to require dependency overrides to be chosen by 
 the user rather than allowed in any circumstances.


One can already override locations in dub.selections.json, the 
problem there is automating it so downstream users don't have to 
do anything.

The whole thing was just a workaround for a dub registry bug that 
got fixed anyway, so I don't think it's too important going 
forward.
Mar 18 2019
prev sibling parent Seb <seb wilzba.ch> writes:
On Sunday, 17 March 2019 at 13:54:54 UTC, Joseph Rushton Wakeling 
wrote:

 On Thursday, 7 March 2019 at 16:01:18 UTC, Atila Neves wrote:

 A fair point.

 At the time I was trying to pin the pyd version because there 
 was a bug in the dub registry that prevented me from doing so 
 in the regular fashion. Then the bug got fixed and I moved on.


 It wasn't meant as a personal attack!  Sorry if it came over 
 that way.

 I get the motivation, but it feels somewhat dodgy to allow 
 packages in the DUB repository to depend on arbitrary git 
 forks.  What might make more sense is to allow a way for the 
 build call (or settings in `dub.selections.json`) to easily 
 override dependency locations, which can be used for temporary 
 workaround like this.  What do you think?


The dub.selections.json isn't used for dependencies.
All major package managers have built-in VCS support for a good 
reason:

https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support
https://docs.npmjs.com/files/package.json#git-urls-as-dependencies
https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html
https://golang.github.io/dep/docs/ensure-mechanics.html
...

Especially in the D ecosystem there are a ton of good, 
unmaintained projects which just require a small change to work 
with e.g. latest DMD and you can't get that into "upstream", 
because the author isn't there anymore or very inactive.
I understand that in a perfect world we wouldn't have to do this, 
but in practice this is a lot better than registering dummy fork 
packages on the Dub registry every time.

An example to what the current limitation leads to:

https://github.com/dlang-tour/core/pull/487
Mar 19 2019
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

  community: what features do you miss the most from DUB?


(1) Support for library maintainers' test requirements

Specifically: when a library has dependencies of its own, one 
wants to be sure the library stays compatible with the stated 
minimum supported version, because changing that could be a 
breaking change for downstreams.

With this in mind it would be useful to have:

   * a command to downgrade all dependencies in dub.selections.json
     to the minimum supported (including dependencies of 
dependencies)

   * a flag for build and test commands to request that the minimum
     versions of all dependencies be used

   * a flag to specify the maximum kind of version change `dub 
upgrade`
     can perform: patch, minor, or major.

(2) Cleanly and clearly separate between test options and build 
options

Specifically, this involves fixing the problems with `dub test` 
when the `--build` flag is used.

Example: if one runs `dub test --build=release` then intuitively 
one would expect (unit)tests to be run with release flags and 
optimizations enabled. In fact no unittests are run, because 
`--build=release` mandates a specific set of D flags that does 
not include `-unittest`.
Mar 03 2019
next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 3 March 2019 at 17:03:20 UTC, Joseph Rushton Wakeling 
wrote:

 Example: if one runs `dub test --build=release` then 
 intuitively one would expect (unit)tests to be run with release 
 flags and optimizations enabled. In fact no unittests are run, 
 because `--build=release` mandates a specific set of D flags 
 that does not include `-unittest`.


As a work-around you can use buildTypes today: 
https://p0nce.github.io/d-idioms/#Unittests-with-optimizations-:-the-Final-Frontier

I think it would be a good idea to have this in vanilla DUB.
Mar 03 2019
prev sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 3/3/19 12:03 PM, Joseph Rushton Wakeling wrote:

 On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

  community: what features do you miss the most from DUB?


 
 (1) Support for library maintainers' test requirements
 
 Specifically: when a library has dependencies of its own, one wants to 
 be sure the library stays compatible with the stated minimum supported 
 version, because changing that could be a breaking change for downstreams.
 
 With this in mind it would be useful to have:
 
    * a command to downgrade all dependencies in dub.selections.json
      to the minimum supported (including dependencies of dependencies)
 
    * a flag for build and test commands to request that the minimum
      versions of all dependencies be used
 
    * a flag to specify the maximum kind of version change `dub upgrade`
      can perform: patch, minor, or major.


Big "ditto" on this stuff. Althouth I'd change it to just "Support for 
library maintainers' requirements" in general.

I've already had to deal with that stuff you mention by hacking it all 
into my travis setup scripts and then have a custom test runner script 
invoke `dub test`.

For example, take a look at mysql-native's root dirctory:
https://github.com/mysql-d/mysql-native

Notice that not only is `dub.selections.json` included in version 
control, but I also have another file: `dub.selections.vibe-0.8.3.json`. 
That's because some compiler versions require vibe v0.7.x and NOT 
v0.8.x, but newer compilers require v0.8.x and up. So 
`dub.selections.json` is carefully set (by hand) to exactly my minimum 
dependency versions. And `dub.selections.vibe-0.8.3.json` is carefully 
set to the minimum versions required by the newer compilers.

So then, in my travis setup script (ci_setup.d), I check for the envvar 
DUB_SELECT. If it exists, I overrwite `dub.selections.json` with 
`dub.selections.$(DUB_SELECT).json`:
https://github.com/mysql-d/mysql-native/blob/2f9d71ca026dc675d07bfb4dd20ce44e07c9aac5/ci_setup.d#L37

The travis setup script also checks for another envvar, DUB_UPGRADE. If 
it's true, then I run "dub upgrade".

So then, in my travis build matrix, I set up jobs both with and without 
DUB_SELECT/DUB_UPGRADE, as needed:

DUB_SELECT: 
https://github.com/mysql-d/mysql-native/blob/2f9d71ca026dc675d07bfb4dd20ce44e07c9aac5/.travis.yml#L15

DUB_UPGRADE: 
https://github.com/mysql-d/mysql-native/blob/2f9d71ca026dc675d07bfb4dd20ce44e07c9aac5/.travis.yml#L119

Resulting tests: https://travis-ci.org/mysql-d/mysql-native/builds/497662010

BUT WAIT, THERE'S MORE WORKAROUND! Notice that when DUB_UPGRADE *isn't 
set, I still run `dub upgrade`, but with the `--missing-only` flag:
https://github.com/mysql-d/mysql-native/blob/2f9d71ca026dc675d07bfb4dd20ce44e07c9aac5/ci_setup.d#L59

I needed to do that (which is an incredibly unintuitive way to pre-fetch 
all the dependencies without upgrading any), because the frequent dub 
server outages were causing jobs to be marked as "failed the tests" when 
really they were just "job error".

One thing I absolutely refuse to do is manage a library that demands all 
users always "upgrade all the things or STFU". I only believe in 
dropping support when it's necessary, not "because I can" or "because I 
know my users' needs better than they do and I say they don't need 
version latest-1". So, I need to deal with this stuff.

So, yea, some of this stuff *technically* *can* be done, but it would be 
REALLY nice if I didn't have to jump through all these hoops and hacks.

And yes, once again, all this ultimately comes down to DUB demanding to 
always be the center of the world.
Mar 03 2019
parent reply Jacob Carlborg <doob me.com> writes:
On 2019-03-04 06:32, Nick Sabalausky (Abscissa) wrote:


 Notice that not only is `dub.selections.json` included in version 
 control, but I also have another file: `dub.selections.vibe-0.8.3.json`. 
 That's because some compiler versions require vibe v0.7.x and NOT 
 v0.8.x, but newer compilers require v0.8.x and up. So 
 `dub.selections.json` is carefully set (by hand) to exactly my minimum 
 dependency versions. And `dub.selections.vibe-0.8.3.json` is carefully 
 set to the minimum versions required by the newer compilers.


Usually `dub.selections.json` should not be committed for libraries, 
only for applications.

With the latest version of Dub it's now possible to specify which 
compiler is required [1]. That could potentially have solved your 
problem. I'm not sure if this means that Dub will try to pick an older 
version of the dependency if the compiler requirement doesn't match.

[1] https://dlang.org/changelog/2.085.0.html#toolchain_requirements

-- 
/Jacob Carlborg
Mar 04 2019
parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 3/4/19 12:19 PM, Jacob Carlborg wrote:

 On 2019-03-04 06:32, Nick Sabalausky (Abscissa) wrote:
 

 Notice that not only is `dub.selections.json` included in version 
 control, but I also have another file: 
 `dub.selections.vibe-0.8.3.json`. That's because some compiler 
 versions require vibe v0.7.x and NOT v0.8.x, but newer compilers 
 require v0.8.x and up. So `dub.selections.json` is carefully set (by 
 hand) to exactly my minimum dependency versions. And 
 `dub.selections.vibe-0.8.3.json` is carefully set to the minimum 
 versions required by the newer compilers.


 
 Usually `dub.selections.json` should not be committed for libraries, 
 only for applications.


Well, its absolutely mandatory if you want your CI builds to be 
reproducible. (As they SHOULD be, unless you enjoy completely 
unnecessary potential problems.)

Although, come to think of it, I guess I could just rename it to 
`dub.selections.ci-default.json` and have my travis setup script 
automatically rename that to `dub.selections.json` if I haven't chosen 
any other alternative dub.selections for the current CI job.

But I'm not sure I see the harm in including `dub.selections.json`, 
though. If you're using the given project as a lib in your own project, 
dub doesn't actually do anything with the `dub.selections.json`s from 
the dependencies...or does it? 'Cause *that* would be bad...(Though I 
can't imagine *why* it would intentionally work that way.)


 With the latest version of Dub it's now possible to specify which 
 compiler is required [1]. That could potentially have solved your 
 problem.
 [1] https://dlang.org/changelog/2.085.0.html#toolchain_requirements


Yea, just saw that myself. I'm definitely thrilled to see it added (for 
other reasons), I wish it had been added years ago, but I don't see how 
it would help the cases I mentioned above.


 I'm not sure if this means that Dub will try to pick an older 
 version of the dependency if the compiler requirement doesn't match.


Hmm, maybe, I hadn't thought of that. I certainly hope so. But in any 
case, the vide-d versions supported by mysql-native lack the 
"toolchainRequirements" directive (as they predate it), so it doesn't 
help mysql-native's existing cases here, at least not anytime in the 
near future. It also doesn't help with needing to test on the minimum
supported dependency versions.
Mar 04 2019
prev sibling next sibling parent Domain <dont_email empty.com> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

  community: what features do you miss the most from DUB?


Nested subpackages:
https://github.com/dlang/dub/issues/843
Mar 04 2019
prev sibling parent bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

 So while we have quite a large list of potential GSoC projects 
 [1], I think that many of these ideas are still a bit too 
 abstract for potential students.

 [...]



Support staged build and install for ease packaging process.
Usually build tools support staged install by using the special 
environment variable DESTDIR. It is a coding standard: 
https://www.gnu.org/prep/standards/html_node/DESTDIR.html

Be able to run dub on offline computer and looking dependencies 
following this order: (i) /usr/{lib,include,bin}, (ii) 
/usr/local/{lib,include,bin}, (iii) 
/${HOME}/.local/{lib,include,bin}
Mar 19 2019