www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - [gsoc] DUB - Ideas

reply Seb <seb wilzba.ch> writes:
So while we have quite a large list of potential GSoC projects 
[1], I think that many of these ideas are still a bit too 
abstract for potential students.

Hence, I will try to introduce this new [gsoc] series aimed at 
(1) discussing potential ideas in depth and (2) maybe finding new 
ideas. I encourage everyone (students or just curious D 
enthusiasts) to start more of these [gsoc] threads, s.t. our 
potential students can learn more about the proposed projects 
(students: please don't be shy).

The first item on my list is D's package manager DUB.

My personal favorite "missing feature" in DUB is a direct git 
support. It would be really awesome if we would implement a 
"GitRegistry" for packages, s.t. one can directly depend on git 
repositories:

dependency "my-package" version="git://.../my-branch#my-commit"

There's also a DEP about this feature [2] (though the proposed 
format there isn't set in stone). DUB's architecture is already 
fairly modular, so as a start students could look into creating a 
GitRegistry supplier [3]. There's also some previous work [4] 
which can of course be used for inspiration too. While initially, 
the `git` CLI binary can be used, it's not guaranteed that the 
user will have this installed (especially on Windows), so one 
would either need to use e.g. libgit2 (see [5] for a D binding) 
or implement Git's protocol itself (see e.g. [6]).

 community: what features do you miss the most from DUB?

Disclaimer: The span of the GSoC is three months of full time 
work and proposals from students should take this into 
consideration. The DUB project is a bit special as some 
individual ideas might take less than three months of work, but 
in this case one could just combine two or three DUB 
features/ideas into one project.

[1] https://wiki.dlang.org/GSOC_2019_Ideas
[2] https://github.com/dlang/dub/wiki/DEP6
[3] 
https://github.com/dlang/dub/blob/master/source/dub/packagesuppliers/packagesupplier.d
[4] https://github.com/dlang/dub/pull/1403
[5] https://github.com/s-ludwig/dlibgit
[6] 
https://github.com/git/git/blob/master/Documentation/technical/http-protocol.txt
Feb 28
next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
 So while we have quite a large list of potential GSoC projects 
 [1], I think that many of these ideas are still a bit too 
 abstract for potential students.

 [...]
Yes beeing able to specify git and also http dependencies would be great. Also there is the cross compilation support missing (https://github.com/dlang/dub/pull/1541). Another topic is watch command for dub, this would be also really great. Kind regards Andre
Feb 28
prev sibling next sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 2/28/19 10:50 AM, Seb wrote:
 
  community: what features do you miss the most from DUB?
 
Oh my...where to start...? - Need a clear, obvious, dead-simple, 100% reliable way to tell DUB to keep its grubby hands out of my project's compilation. Tell it "Run this command to compile, run this other command to generate docs, etc., here's where you can find the resulting output, and otherwise, ***stay out of it!!***". Both the sdl/json config and the launched process should support appropriate envvars so I can cleanly pass-through CLI information like "Make docs instead of do build" or "use LDC" to my scripts without complicating them too much. - Need to be able to tell DUB "I want *THIS* compiler flag included/omitted", WITHOUT triggering a stupid big ugly warning, or having to manually translate the CLI flag into DUB-speak. Both GDC and LDC come with compatibility wrappers for DMD's CLI, so an additional abstraction layer here is neither necessary nor welcome. - Compile times that don't make a mockery of DMD. - DUB's default stdout output needs to STFU by default. - DUB and the compilers themselves should be bona-fide dependencies just like anything else. - Needs good, solid support for non-source dependencies, system dependencies, and tools that the build itself depends on, not just libs your source depends on. - This following workflow needs to be a BASIC STANDARD FULLY-ROBUST use-case with the same priority and importance as running "dub" to build a project: I have a non-DUB project and I want to include a library (say, Vibe.D) via DUB. I have my CLI command(s) to invoke the compiler, and all I have to do to use Vibe.d (or whatever) is insert this as part of my compiler invocation: dmd blah blah `dub include vibe-d:~0.8.3` blah blah And boom, -I... paths, libs, Have_vibe_d, etc., all guaranteed working, correct and ready to go. At one point, I did a lot of work to make that happen, and I made some progress, but it was just so contrary to the way dub wanted to work I finally gave up. - There's probably more I'm forgetting at the moment...
Feb 28
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Feb 28, 2019 at 12:45:56PM -0500, Nick Sabalausky (Abscissa) via
Digitalmars-d wrote:
 On 2/28/19 10:50 AM, Seb wrote:
  community: what features do you miss the most from DUB?
[...]
 Oh my...where to start...?
 
 - Need a clear, obvious, dead-simple, 100% reliable way to tell DUB to
 keep its grubby hands out of my project's compilation. Tell it "Run
 this command to compile, run this other command to generate docs,
 etc., here's where you can find the resulting output, and otherwise,
 ***stay out of it!!***". Both the sdl/json config and the launched
 process should support appropriate envvars so I can cleanly
 pass-through CLI information like "Make docs instead of do build" or
 "use LDC" to my scripts without complicating them too much.
+1.
 - Need to be able to tell DUB "I want *THIS* compiler flag
 included/omitted", WITHOUT triggering a stupid big ugly warning, or
 having to manually translate the CLI flag into DUB-speak. Both GDC and
 LDC come with compatibility wrappers for DMD's CLI, so an additional
 abstraction layer here is neither necessary nor welcome.
+1.
 - Compile times that don't make a mockery of DMD.
+100. I think dub has improved in this area the last time I checked, but still, there's room for improvement.
 - DUB's default stdout output needs to STFU by default.
+1. Though to be fair, dub is already fairly quiet by defaut, compared to some other programs that tend to get run during builds.
 - DUB and the compilers themselves should be bona-fide dependencies
 just like anything else.
+100. I don't have a problem with special handling of certain types of dependencies, but the user needs to be empowered to write the same functionality without hacking dub's source code. Empowerment before convenience is my motto.
 - Needs good, solid support for non-source dependencies, system
 dependencies, and tools that the build itself depends on, not just
 libs your source depends on.
A generic dependency resolution system that the user can directly access would address this point.
 - This following workflow needs to be a BASIC STANDARD FULLY-ROBUST
 use-case with the same priority and importance as running "dub" to
 build a project: I have a non-DUB project and I want to include a
 library (say, Vibe.D) via DUB. I have my CLI command(s) to invoke the
 compiler, and all I have to do to use Vibe.d (or whatever) is insert
 this as part of my compiler invocation:
 
 dmd blah blah `dub include vibe-d:~0.8.3` blah blah
 
 And boom, -I... paths, libs, Have_vibe_d, etc., all guaranteed
 working, correct and ready to go.
 
 At one point, I did a lot of work to make that happen, and I made some
 progress, but it was just so contrary to the way dub wanted to work I
 finally gave up.
[...] I tried doing a non-dub project with dub dependencies before, and ended up with so much frustration that I threw in the towel and resorted to the hack of creating an empty dummy dub project in a subdirectory, whose sole purpose was to declare dub dependencies. I'd only run dub from the subdirectory when one or more dependencies need to be updated, but the actual compilation / linking is done by a different, sane build system that pulls in the objects dub built. The fact that such hacks are needed just to make dub work the way I want it to, is a sign that something is fundamentally broken with its design. Again, it's an issue of user empowerment, or lack thereof: in theory it should be a tool that provides pluggable functionalities that the user can integrate into whatever else he may be using. Unfortunately, its current design assumes the universe revolves around it, and requires everything else to orbit around how it works. When that doesn't happen it either works poorly, requiring contorted workarounds, or doesn't work at all, needing to boxed off into its own sub-universe in which it's the center of revolution before it is willing to play nice. It's a hammer that wields the user, rather than the other way round. Most of the above points are consequences of this fundamental point. T -- No! I'm not in denial!
Feb 28
next sibling parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 2/28/19 8:38 PM, H. S. Teoh wrote:
 
 It's a hammer
 that wields the user, rather than the other way round.  Most of the
 above points are consequences of this fundamental point.
+1, Definitely true...plus, I like this metaphor. Though, to be fair, that description applies to most software these days. But that's a whole other rant...
Feb 28
prev sibling parent sarn <sarn theartofmachinery.com> writes:
On Friday, 1 March 2019 at 01:38:13 UTC, H. S. Teoh wrote:
 On Thu, Feb 28, 2019 at 12:45:56PM -0500, Nick Sabalausky 
 (Abscissa) via Digitalmars-d wrote:
 - This following workflow needs to be a BASIC STANDARD 
 FULLY-ROBUST use-case with the same priority and importance as 
 running "dub" to build a project: I have a non-DUB project and 
 I want to include a library (say, Vibe.D) via DUB. I have my 
 CLI command(s) to invoke the compiler, and all I have to do to 
 use Vibe.d (or whatever) is insert this as part of my compiler 
 invocation:
 
 dmd blah blah `dub include vibe-d:~0.8.3` blah blah
 
 And boom, -I... paths, libs, Have_vibe_d, etc., all guaranteed 
 working, correct and ready to go.
 
 At one point, I did a lot of work to make that happen, and I 
 made some progress, but it was just so contrary to the way dub 
 wanted to work I finally gave up.
[...] I tried doing a non-dub project with dub dependencies before, and ended up with so much frustration that I threw in the towel and resorted to the hack of creating an empty dummy dub project in a subdirectory, whose sole purpose was to declare dub dependencies. I'd only run dub from the subdirectory when one or more dependencies need to be updated, but the actual compilation / linking is done by a different, sane build system that pulls in the objects dub built.
I've also had this use case, and also ended up hacking around it using a dummy empty project. I wrote a tool that parsed the output of "dub describe", but it only mostly works because that stuff is a confusing mess.
Mar 01
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
  community: what features do you miss the most from DUB?
- SOUNDNESS: dub.selections.json doesn't encode deps for all comfigurations. The design isn't sound. - configurations-based dependencies, or OS-based dependencies: if that is possible at all - remove optional dependencies, removedub fetch/remove (it should never matter what is "installed"), Optional dependencies can be done with a dynamic loader anyway, i na way that is SOUND, - remove SDLang support ("a stratagic mistake"), we are paying for this since it was introduced, with CONFUSION and having to double the recommendations on recipe files ("if you use JSON, use this... if you use SDLang, blah...") - remove `dub describe` (no IDE really relies on it AFAIK), do not check the network by default which makes `dub describe` unusable in the first place - build faster - choose to keep one: overrides, add-local or add-path... too many way to have user-defined modifications - colors, with way easier error messages! DUB IS THE PRIMARY INTERFACE TO D. - remove `dub search`, you can as well type code.dlang.org in the browser, and it will be a better experience There are too many features that scare people in DUB!
Mar 01
next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:

 - remove SDLang support ("a stratagic mistake"), we are paying 
 for this since it was introduced, with CONFUSION and having to 
 double the recommendations on recipe files ("if you use JSON, 
 use this... if you use SDLang, blah...")
I'll stake my flag on that hill! Remove JSON if anything :-) It's ugly, it's noisy, and it doesn't allow comments. SDLang is much cleaner and human-friendly.
Mar 01
next sibling parent Andrea Fontana <nospam example.org> writes:
On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:
 On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:

 - remove SDLang support ("a stratagic mistake"), we are paying 
 for this since it was introduced, with CONFUSION and having to 
 double the recommendations on recipe files ("if you use JSON, 
 use this... if you use SDLang, blah...")
I'll stake my flag on that hill! Remove JSON if anything :-) It's ugly, it's noisy, and it doesn't allow comments. SDLang is much cleaner and human-friendly.
I agree. I started using json and then I switched to sdl. I had some difficulties but now I vote for sdl and I don't like the idea to switch back to json format.
Mar 01
prev sibling next sibling parent reply bauss <jj_1337 live.dk> writes:
On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:
 On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:

 - remove SDLang support ("a stratagic mistake"), we are paying 
 for this since it was introduced, with CONFUSION and having to 
 double the recommendations on recipe files ("if you use JSON, 
 use this... if you use SDLang, blah...")
I'll stake my flag on that hill! Remove JSON if anything :-) It's ugly, it's noisy, and it doesn't allow comments. SDLang is much cleaner and human-friendly.
No. I like json more instead of sdl. My whole build process is build upon json so it makes it easy to work with everything else that isn't dub related. And there are json versions that allow comments, just switch to json5. https://json5.org/
Mar 01
parent Mike Parker <aldacron gmail.com> writes:
On Friday, 1 March 2019 at 11:11:56 UTC, bauss wrote:

 No. I like json more instead of sdl. My whole build process is 
 build upon json so it makes it easy to work with everything 
 else that isn't dub related.

 And there are json versions that allow comments, just switch to 
 json5.

 https://json5.org/
Well, if they would make it less ugly and noisy, then it would be useful! This shows why we should continue to support both.
Mar 01
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:
 I'll stake my flag on that hill! Remove JSON if anything :-) 
 It's ugly, it's noisy, and it doesn't allow comments. SDLang is 
 much cleaner and human-friendly.
Let's disagree! - JSON schemas - lots of parsers, you can parse it at compile-time - it's well known - the "best language" for 20 lines recipe files has very little upside
Mar 01
parent reply JN <666total wp.pl> writes:
On Friday, 1 March 2019 at 16:21:19 UTC, Guillaume Piolat wrote:
 On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:
 I'll stake my flag on that hill! Remove JSON if anything :-) 
 It's ugly, it's noisy, and it doesn't allow comments. SDLang 
 is much cleaner and human-friendly.
Let's disagree! - JSON schemas - lots of parsers, you can parse it at compile-time - it's well known - the "best language" for 20 lines recipe files has very little upside
It's obvious no one will win the SDL vs JSON debate. The only solution is to dump both and switch to YAML :)
Mar 01
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 1 March 2019 at 17:22:47 UTC, JN wrote:
 It's obvious no one will win the SDL vs JSON debate. The only 
 solution is to dump both and switch to YAML :)
Believe it or not, there is a proposal for a third format (the 'tree' format). https://github.com/dlang/dub/wiki/DEP7 * Obligatory reference to https://xkcd.com/927/ * I wish one of the two got dropped. I'm using SDL now since it's officially recommended, but I'd prefer to only keep JSON since it is a clear format that newcomers don't need to familiarize with, while SDL (still) to me looks like a bunch of tags floating around without clear structure. Keeping only SDL is fine to me too, it's better than having two competing standards.
Mar 01
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 3/1/19 1:13 PM, Dennis wrote:
 On Friday, 1 March 2019 at 17:22:47 UTC, JN wrote:
 It's obvious no one will win the SDL vs JSON debate. The only solution 
 is to dump both and switch to YAML :)
Believe it or not, there is a proposal for a third format (the 'tree' format). https://github.com/dlang/dub/wiki/DEP7
Not that I think there's even a slightest chance DUB will ever gain another format, but that page contains some questionable claims about SDL, and at least one claim that's just downright false: It says: ---------------------------------- [Tree's] Comparison to SDL: Easy fast syntax Syntax highlighting in few editors (Idea, VSCode, Atom, SynWrite) More implementations (D,TS/JS) ---------------------------------- The third claim is flat out wrong. SDL has implementations for D, Java, Python and Ruby (that I know of). In fact. the D implementation is the youngest of those four. (Sometimes I get the impression that people think I created SDLang, and did so for D, despite my attempts to dispel that notion. I was looking for a sane alternative to XML/JSON/YAML, found SDL, and wanted to use it in D. Just like how I once looked for a sane alternative to C++ and Java2/4, and found D.) The second claim is tenuous at best: Even without a dedicated "SDLang highlighter", I can go right through my editor's list of supported highlighters and most of them do an entirely useable job highlighting SDLang. It's not exactly a complicated language: It's basically just custom identifiers and a few literals and grouping symbols. Just like JSON/etc, there's just not much semantic or syntactic detail there TO highlight. The first one is a highly subjective matter of opinion. SDL and Tree (based on what I can tell from the example provided) *both* have very simple easy syntaxes that put XML, JSON and YAML to shame. Comparing the two directly against each other, well, they're pretty much on par with each other in this regard, any advantage to be had is slight at best. Someone else mentioned TOML, too. Just looked it up. That's also a very nice data language too, just like Tree and SDLang are. Hell, there's so much out there that just puts the XML/JSON/YAML popular kids clique to shame. I find it absolutely disgusting, shameful and frankly, professionally negligent, that a *technical* field such as ours routinely prioritizes and even promotes following the cool kids' table over actual, freaking technical merit.
Mar 02
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Saturday, 2 March 2019 at 18:22:01 UTC, Nick Sabalausky 
(Abscissa) wrote:
 Someone else mentioned TOML, too. Just looked it up. That's 
 also a very nice data language too, just like Tree and SDLang 
 are. Hell, there's so much out there that just puts the 
 XML/JSON/YAML popular kids clique to shame. I find it 
 absolutely disgusting, shameful and frankly, professionally 
 negligent, that a *technical* field such as ours routinely 
 prioritizes and even promotes following the cool kids' table 
 over actual, freaking technical merit.
Well, to me the ONLY reasons DUB use SDLang and cargo use TOML is obviously... to make things as different as possible! It's just trying to be different for being different's sake. Sociology explains this way more than any technical merit, we are prone to this in the D community because well it's an alternative language, it selects for people who likes alternatives. This is the elephant in the room, why else would we argue (with so much passion) the technical merit of 20-lines recipe files? And TOML is really tragic, I don't know if you have tried using it, I have and reverted to XML.
Mar 03
parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 3/3/19 7:01 AM, Guillaume Piolat wrote:
 
 Well, to me the ONLY reasons DUB use SDLang and cargo use TOML is 
 obviously... to make things as different as possible! It's just trying 
 to be different for being different's sake.
 
 Sociology explains this way more than any technical merit, we are prone 
 to this in the D community because well it's an alternative language, it 
 selects for people who likes alternatives.
 
 This is the elephant in the room, why else would we argue (with so much 
 passion) the technical merit of 20-lines recipe files?
1. Because I've already wasted enough of my life suffering shitty tools. 2. Because no amount of insignificance will ever turn an inferior choice into a superior one. 3. Because any coup needs to start somewhere. As for the elephant, it's only a photo on the wall...
 And TOML is really tragic, I don't know if you have tried using it, I 
 have and reverted to XML.
I haven't. This thread is the first I've heard of it. That's unfortunate to hear as it *seems* very nice at a glance. What issues did you have?
Mar 03
parent Guillaume Piolat <first.last gmail.com> writes:
On Monday, 4 March 2019 at 06:27:33 UTC, Nick Sabalausky 
(Abscissa) wrote:

 I haven't. This thread is the first I've heard of it. That's 
 unfortunate to hear as it *seems* very nice at a glance. What 
 issues did you have?
TOML make difficult for humans to write "Array of Tables" https://github.com/toml-lang/toml because it's where the flat idea break down. Array of Tables of Array of Tables would be increasingly unworkable, whereas in JSON nesting happens normally.
Mar 04
prev sibling parent Ralph Amissah <ralph.amissah gmail.com> writes:
On 2019-03-01 17:22, Digitalmars-d wrote:
 On Friday, 1 March 2019 at 16:21:19 UTC, Guillaume Piolat wrote:
 On Friday, 1 March 2019 at 10:24:45 UTC, Mike Parker wrote:
 I'll stake my flag on that hill! Remove JSON if anything :-) It's
 ugly, it's noisy, and it doesn't allow comments. SDLang is much
 cleaner and human-friendly.
Let's disagree! - JSON schemas - lots of parsers, you can parse it at compile-time - it's well known - the "best language" for 20 lines recipe files has very little upside
It's obvious no one will win the SDL vs JSON debate. The only solution is to dump both and switch to YAML :)
or to keep bicycle-shedding: TOML - (even) "nicer" than SDL - more sane than YAML (one thing Rust perhaps got better) https://forum.dlang.org/search?q=toml&scope=group%3AdigitalmarsD
Mar 01
prev sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Friday, 1 March 2019 at 09:41:55 UTC, Guillaume Piolat wrote:
 - colors, with way easier error messages! DUB IS THE PRIMARY 
 INTERFACE TO D.
For ref, there was this initiative by gdelazzari in June. https://forum.dlang.org/post/vefplpvwvrcbednsufli forum.dlang.org
Mar 01
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
 So while we have quite a large list of potential GSoC projects 
 [1], I think that many of these ideas are still a bit too 
 abstract for potential students.

 Hence, I will try to introduce this new [gsoc] series aimed at 
 (1) discussing potential ideas in depth and (2) maybe finding 
 new ideas. I encourage everyone (students or just curious D 
 enthusiasts) to start more of these [gsoc] threads, s.t. our 
 potential students can learn more about the proposed projects 
 (students: please don't be shy).

 The first item on my list is D's package manager DUB.

 My personal favorite "missing feature" in DUB is a direct git 
 support. It would be really awesome if we would implement a 
 "GitRegistry" for packages, s.t. one can directly depend on git 
 repositories:

 dependency "my-package" version="git://.../my-branch#my-commit"

 There's also a DEP about this feature [2] (though the proposed 
 format there isn't set in stone). DUB's architecture is already 
 fairly modular, so as a start students could look into creating 
 a GitRegistry supplier [3]. There's also some previous work [4] 
 which can of course be used for inspiration too. While 
 initially, the `git` CLI binary can be used, it's not 
 guaranteed that the user will have this installed (especially 
 on Windows), so one would either need to use e.g. libgit2 (see 
 [5] for a D binding) or implement Git's protocol itself (see 
 e.g. [6]).

  community: what features do you miss the most from DUB?
Some of these might have been done/fixed since the last time I ran into them. I wouldn't know because I keep not using dub except to make my packages available to everyone else. 1. Bugs getting fixed. 2. Why oh why oh why does it build dependent packages serially????? 3. Build per D package by default instead of all-at-once, with proper dependency tracking. 4. Ability to use a real build system where needed. 5. `dub describe` actually working, or, even better, an easy to use API to query the build. Did you know that the output of `dub describe -c unittest` doesn't have the string "-unittest" anywhere, much less in "dflags"? Now you do. 6. Not having to write the same code over and over again to check if dub dependencies are already downloaded, somehow running faster than dub's own checks even if I write said code in Emacs Lisp. 7. Ability to avoid the "lib" prefix for libraries on Linux (think Python shared objects). 8. Possibility of using a shell other than CMD on Windows (preBuildCommands, etc.). 9. Be able to run two dub instances in parallel without crashing the linker. 10. A more flexible registry engine so that one can use code that lives in a private gitlab instance. Off the top of my head, that's it.
Mar 01
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Friday, 1 March 2019 at 14:16:17 UTC, Atila Neves wrote:
 5. `dub describe` actually working, or, even better, an easy to 
 use API to query the build. Did you know that the output of 
 `dub describe -c unittest` doesn't have the string "-unittest" 
 anywhere, much less in "dflags"? Now you do.
That is actually to some extent expected behaviour. -c unittest only refers to the unittest configuration, not the unittest *build*. dub describe -c unittest -b unittest will give you "unittests" under targets[0].buildSettings.options Of course those options still need to be converted in to the relevant compiler flags.
Mar 01
parent Atila Neves <atila.neves gmail.com> writes:
On Friday, 1 March 2019 at 14:28:54 UTC, John Colvin wrote:
 On Friday, 1 March 2019 at 14:16:17 UTC, Atila Neves wrote:
 5. `dub describe` actually working, or, even better, an easy 
 to use API to query the build. Did you know that the output of 
 `dub describe -c unittest` doesn't have the string "-unittest" 
 anywhere, much less in "dflags"? Now you do.
That is actually to some extent expected behaviour. -c unittest only refers to the unittest configuration, not the unittest *build*. dub describe -c unittest -b unittest will give you "unittests" under targets[0].buildSettings.options Of course those options still need to be converted in to the relevant compiler flags.
Bad example from me. Replace "-unittest" with "-g -debug" for the debug build, which is the default. Why anyone would want to use the "unittest" configuration (which is treated differently by dub) in a non-unittest build is beyond me.
Mar 01
prev sibling next sibling parent Radu <void null.pt> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
 So while we have quite a large list of potential GSoC projects 
 [1], I think that many of these ideas are still a bit too 
 abstract for potential students.

 [...]
- Cross compiling support https://github.com/dlang/dub/issues/1523 - Usable with private repos https://github.com/dlang/dub/issues/1499
Mar 01
prev sibling next sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:

  community: what features do you miss the most from DUB?
A properly working preGenerateCommands https://github.com/dlang/dub/issues/1474 - Bastiaan.
Mar 01
prev sibling next sibling parent reply jpvalley <vikram.mullick yale.edu> writes:
Hi, all! I discovered D looking through the GSoC organizations 
page, and I'm really intrigued by it. I installed the compiler 
and messed around with it a bit, but I don't really have much 
wisdom as to what could make a good project.

One of the ideas I came up with was Python extension modules for 
D. Does this seem like something that would be useful/feasible? 
I'm planning on doing more research when I get on spring break, 
but I was hoping to bounce this idea off people who would know 
better.

Cheers!
Mar 02
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 03/03/2019 3:55 AM, jpvalley wrote:
 Hi, all! I discovered D looking through the GSoC organizations page, and 
 I'm really intrigued by it. I installed the compiler and messed around 
 with it a bit, but I don't really have much wisdom as to what could make 
 a good project.
 
 One of the ideas I came up with was Python extension modules for D. Does 
 this seem like something that would be useful/feasible? I'm planning on 
 doing more research when I get on spring break, but I was hoping to 
 bounce this idea off people who would know better.
 
 Cheers!
https://github.com/ariovistus/pyd If you want to talk to us more casually you can do so, by joining IRC (FreeNode), Slack or Discord (https://discord.gg/3aUX5nv). Happy coding!
Mar 02
prev sibling next sibling parent Jon Degenhardt <jond noreply.com> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
  community: what features do you miss the most from DUB?
Dub support for building code.dlang.org packages with LDC's LTO. The idea would be to support something analogous to the way LDC provides LTO support for druntime and phobos. See https://github.com/ldc-developers/ldc/releases/tag/v1.9.0 for a description. Building with LTO against druntime/phobos is a substantial performance win for certain classes of programs. This would likely hold true for a number of code.dlang.org packages as well. My benchmarks involving druntime/phobos are here: https://github.com/eBay/tsv-utils/blob/master/docs/lto-pgo-study.md. LDC supports druntime/phobos by shipping these prebuilt as LTO libraries (with IR code). For packages it probably makes more sense to build them from source after being downloaded. This is probably not a long project, but it might fit well as part of a suite of Dub changes. Dub may well have the necessary facilities already, I'm not sure, but this is not supported in a straightforward manner. There are likely some open questions, for example, should the package provider mark whether package supports LTO or not.
Mar 02
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
 My personal favorite "missing feature" in DUB is a direct git 
 support. It would be really awesome if we would implement a 
 "GitRegistry" for packages, s.t. one can directly depend on git 
 repositories:

 dependency "my-package" version="git://.../my-branch#my-commit"
One can already just add a git submodule and point the dub dependency declaration at the resulting directory. What features are you missing that aren't covered by this?
Mar 03
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 3 March 2019 at 13:24:21 UTC, Joseph Rushton Wakeling 
wrote:
 One can already just add a git submodule and point the dub 
 dependency declaration at the resulting directory.
Dub doesn't support git submodules (because it downloads the Zip archive from GitHub/GitLab). So `dub fetch my-package` won't fetch any submodules. This is already a huge annoyance, but the mentioned point was about supporting git checkouts in dub, s.t. you can have Dub dependencies from any git source like in almost any other major package manager without them needing to be registered on the dub registry or to be manually checked out.
Mar 03
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 3 March 2019 at 13:43:15 UTC, Seb wrote:
 On Sunday, 3 March 2019 at 13:24:21 UTC, Joseph Rushton 
 Wakeling wrote:
 One can already just add a git submodule and point the dub 
 dependency declaration at the resulting directory.
Dub doesn't support git submodules (because it downloads the Zip archive from GitHub/GitLab).
I think we may be talking at cross-purposes. Here's what I was suggesting w.r.t. using git repos as a source for dub dependencies: (1) add a submodule to your current project (let's use mir-glas as an example): git submodule add --name mir-glas https://github.com/libmir/mir-glas.git mir-glas (2) add a dependency to dub.json pointing at the submodule directory: "dependencies": { "mir-glas": { "path": "./mir-glas" } } This is only really needed though if one wants to create dependencies on projects that are not public DUB packages. I think this ought to also support nested submodules, though I have not tried it. Now OTOH if I understand your concerns right:
 So `dub fetch my-package` won't fetch any submodules. This is 
 already a huge annoyance, but the mentioned point was about 
 supporting git checkouts in dub, s.t. you can have Dub 
 dependencies from any git source like in almost any other major 
 package manager without them needing to be registered on the 
 dub registry or to be manually checked out.
... you're concerned about the case where a DUB package relies on submodules for its complete source code? I wonder if you could be more specific about the exact use case you have in mind, and why one would want to use submodules in that way? What strikes me is that all the use cases one could think of probably come down to different examples of this: "How does one make DUB aware of and able to handle a codebase that does not include its own DUB package config?" I'd suggest that considering that problem independent of any particular VCS might be more illuminating than asking how to make DUB git-repo-aware.
Mar 03
parent reply Atila Neves <atila.neves gmail.com> writes:
On Sunday, 3 March 2019 at 15:59:44 UTC, Joseph Rushton Wakeling 
wrote:
 On Sunday, 3 March 2019 at 13:43:15 UTC, Seb wrote:
 On Sunday, 3 March 2019 at 13:24:21 UTC, Joseph Rushton 
 Wakeling wrote:
 One can already just add a git submodule and point the dub 
 dependency declaration at the resulting directory.
Dub doesn't support git submodules (because it downloads the Zip archive from GitHub/GitLab).
I think we may be talking at cross-purposes. Here's what I was suggesting w.r.t. using git repos as a source for dub dependencies: (1) add a submodule to your current project (let's use mir-glas as an example): git submodule add --name mir-glas https://github.com/libmir/mir-glas.git mir-glas (2) add a dependency to dub.json pointing at the submodule directory: "dependencies": { "mir-glas": { "path": "./mir-glas" } } This is only really needed though if one wants to create dependencies on projects that are not public DUB packages. I think this ought to also support nested submodules, though I have not tried it.
This doesn't work. Worse, it actually does work if you test adding this dub package (the one that has mir-glas as a dependency) locally by using `path="/path/to/thing/needing/mir-glass"`. Then you tag a new version, try using a version dependency and... oops. Check out all the v0.2.x releases for autowrap here: https://github.com/kaleidicassociates/autowrap/releases Notice how they're all from two different days. That was me trying to make pyd a dependency as a git submodule. I have to give up and go up to v0.3.0 since none of those tags work.
Mar 04
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 4 March 2019 at 10:19:29 UTC, Atila Neves wrote:
 This doesn't work.
Exactly which bits of it? I have actual projects that use git submodules to provide some dependencies for dub. There are no nested submodules, though. A constraint is that I'm tied to a _specific_ version of the dependency (determined by what commit or tag the git submodule checks out), but that's not actually a problem for the use-case (generally for production code I find it better to have precise control over dependency versions).
Mar 04
next sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 4 March 2019 at 13:12:09 UTC, Joseph Rushton Wakeling 
wrote:
 On Monday, 4 March 2019 at 10:19:29 UTC, Atila Neves wrote:
 This doesn't work.
Exactly which bits of it? I have actual projects that use git submodules to provide some dependencies for dub. There are no nested submodules, though. A constraint is that I'm tied to a _specific_ version of the dependency (determined by what commit or tag the git submodule checks out), but that's not actually a problem for the use-case (generally for production code I find it better to have precise control over dependency versions).
Have you published any of these projects on the dub registry? It wouldn't work for any potential users as dub wouldn't download the git submodules for you.
Mar 04
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 4 March 2019 at 13:26:43 UTC, Seb wrote:
 Have you published any of these projects on the dub registry?
 It wouldn't work for any potential users as dub wouldn't 
 download the git submodules for you.
No, but that's the point: it's a way of integrating _private_ internal projects into DUB. I wouldn't use it for public DUB packages, because what's the point? The question is why you want a public DUB package to rely on submodules for parts of its code. What's the use-case? You still haven't described it.
Mar 04
parent reply Seb <seb wilzba.ch> writes:
On Monday, 4 March 2019 at 13:52:03 UTC, Joseph Rushton Wakeling 
wrote:
 The question is why you want a public DUB package to rely on 
 submodules for parts of its code.  What's the use-case?  You 
 still haven't described it.
You fix a bug in let's say Vibe.d (or any other library for that matter). You don't want to wait 6 months until your bug fix gets merged and released (if ever). I have run into this many many times. In some cases I had to result to registering a duplicate repo on the registry as there was a breaking change in DMD and I (and the users of a depending library) wanted to continue using it. Note that this shortcoming of dub was also the main reason why Dlang Community has been created.
Mar 04
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Monday, 4 March 2019 at 14:04:41 UTC, Seb wrote:
 You fix a bug in let's say Vibe.d (or any other library for 
 that matter). You don't want to wait 6 months until your bug 
 fix gets merged and released (if ever). I have run into this 
 many many times.
Perfectly reasonable use-case, the question is whether the right solution is to tie DUB to direct support for git submodules. I'll slightly rephrase my earlier remarks: this and other cases I can think of all seem to come down to: How does one make DUB aware of and able to handle a codebase that is not registered in the main DUB package repo and possibly does not even define its own DUB package config? What I would suggest is that rather than building in direct support for git, which is only one VCS, we consider instead how to make it possible to specify dependencies in terms of two kinds of instructions: * how to fetch a dependency's code and place it in a directory determined either by DUB itself or by another setting in the `dub.json` file - this might be done by a git clone or checkout, or by fetching and unzipping an archive file, or by a number of other options * how to define DUB packaging/dependency settings if the dependency does not define them in its own codebase That should be a more flexible approach that would allow the same outcomes.
Mar 04
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 4 March 2019 at 13:12:09 UTC, Joseph Rushton Wakeling 
wrote:
 On Monday, 4 March 2019 at 10:19:29 UTC, Atila Neves wrote:
 This doesn't work.
Exactly which bits of it?
Try starting a new dub project that depends on autowrap v0.2.0.
 I have actual projects that use git submodules to provide some 
 dependencies for dub.  There are no nested submodules, though.
This might be the crucial difference - it might be a submodules thing.
Mar 05
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 5 March 2019 at 10:44:04 UTC, Atila Neves wrote:
 Try starting a new dub project that depends on autowrap v0.2.0.
Indeed if you just add a regular dub dependency this won't work (because dub doesn't know how to check out the pyd submodule). But try instead: * in your main project add a submodule for autowrap (checking out v0.2.0) * in your dub config specify the dependency as "autowrap": { "path": "./path/to/autowrap_submodule" } (or whatever the SDL equivalent is). That works.
Mar 06
next sibling parent reply Seb <seb wilzba.ch> writes:
On Wednesday, 6 March 2019 at 12:23:07 UTC, Joseph Rushton 
Wakeling wrote:
 On Tuesday, 5 March 2019 at 10:44:04 UTC, Atila Neves wrote:
 Try starting a new dub project that depends on autowrap v0.2.0.
Indeed if you just add a regular dub dependency this won't work (because dub doesn't know how to check out the pyd submodule). But try instead: * in your main project add a submodule for autowrap (checking out v0.2.0) * in your dub config specify the dependency as "autowrap": { "path": "./path/to/autowrap_submodule" } (or whatever the SDL equivalent is). That works.
Nope it only works locally. It does not work for anyone using it via dub. As mentioned earlier, try fetching this package (or any package with submodules) from the dub registry.
Mar 06
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 6 March 2019 at 12:29:17 UTC, Seb wrote:
 Nope it only works locally. It does not work for anyone using 
 it via dub. As mentioned earlier, try fetching this package (or 
 any package with submodules) from the dub registry.
Yes. But that's because you're doing something fundamentally inappropriate for the DUB package repo, which is to have source code dependencies defined somewhere (the git submodule system) that the DUB repo inherently cannot know about. Basically if you want to have git submodules in the mix then you have to organize the entire dependency hierarchy for everything involved via git submodules rather than via the DUB package repo. The git submodule issue isn't a blocker towards DUB being able to handle these dependencies, it's a blocker on the DUB package repo containing packages whose source code dependencies are not known to the DUB package repo. Those are not quite the same things. In other words, if you choose to bypass DUB's supply mechanisms and hand control to something else (like git), then you need to respect that separation of concerns: handle the supply via the tool you asked to provide supply, and just give DUB sufficient config to build the code once it has been supplied. And it may be important to have that separation of concerns in your face -- after all, if it's trivial to just set a DUB package dependency to point at an arbitrary git repo, what's to stop me setting up a convenience library that depends on a git fork of vibe, which includes dodgy tracking components? I trust that _your_ fork of vibe is well-motivated and only has bugfixes, but I'm not sure that I would trust that approach in general, and I am not so sure it's a good idea to open it up as a possibility in the DUB package repo. After all, if you can't rely on upstream vibe to get out timely bugfix releases, maybe the problem is one of process rather than of DUB itself. I'm not sure it isn't better to fix that process rather than open up a Pandora's box of dependency options to work around it.
Mar 06
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 6 March 2019 at 12:23:07 UTC, Joseph Rushton 
Wakeling wrote:
 On Tuesday, 5 March 2019 at 10:44:04 UTC, Atila Neves wrote:
 Try starting a new dub project that depends on autowrap v0.2.0.
Indeed if you just add a regular dub dependency this won't work (because dub doesn't know how to check out the pyd submodule). But try instead: * in your main project add a submodule for autowrap (checking out v0.2.0) * in your dub config specify the dependency as "autowrap": { "path": "./path/to/autowrap_submodule" } (or whatever the SDL equivalent is). That works.
I don't want to do that. I want to use dub as intended, and have it figure out versions for me. I also want to make my code available for others, have them list my package in their dub.sdl and for Bob to subsequently be their proverbial uncle. It's bad enough I can't use dub as a build system, I'm certainly not going to manually do its job as a package manager! Especially if one of the submodules depends on a dub package that depends on a dub package that depends on a dub package...
Mar 06
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 6 March 2019 at 14:27:56 UTC, Atila Neves wrote:
 I don't want to do that. I want to use dub as intended
I would suggest that DUB stopped being used as intended the moment someone submitted a package to the DUB repo with source code dependencies that were not also packages in the DUB repo.
Mar 06
parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 6 March 2019 at 23:38:20 UTC, Joseph Rushton 
Wakeling wrote:
 On Wednesday, 6 March 2019 at 14:27:56 UTC, Atila Neves wrote:
 I don't want to do that. I want to use dub as intended
I would suggest that DUB stopped being used as intended the moment someone submitted a package to the DUB repo with source code dependencies that were not also packages in the DUB repo.
A fair point. At the time I was trying to pin the pyd version because there was a bug in the dub registry that prevented me from doing so in the regular fashion. Then the bug got fixed and I moved on.
Mar 07
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 7 March 2019 at 16:01:18 UTC, Atila Neves wrote:
 A fair point.

 At the time I was trying to pin the pyd version because there 
 was a bug in the dub registry that prevented me from doing so 
 in the regular fashion. Then the bug got fixed and I moved on.
It wasn't meant as a personal attack! Sorry if it came over that way. I get the motivation, but it feels somewhat dodgy to allow packages in the DUB repository to depend on arbitrary git forks. What might make more sense is to allow a way for the build call (or settings in `dub.selections.json`) to easily override dependency locations, which can be used for temporary workaround like this. What do you think? It's obviously not great if e.g. one has a library package that triggers a bug in one of its dependencies and has no possibility of an in-code workaround, but it seems more secure and consistent to require dependency overrides to be chosen by the user rather than allowed in any circumstances.
Mar 17
next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Sunday, 17 March 2019 at 13:54:54 UTC, Joseph Rushton Wakeling 
wrote:
 On Thursday, 7 March 2019 at 16:01:18 UTC, Atila Neves wrote:
 A fair point.

 At the time I was trying to pin the pyd version because there 
 was a bug in the dub registry that prevented me from doing so 
 in the regular fashion. Then the bug got fixed and I moved on.
It wasn't meant as a personal attack! Sorry if it came over that way. I get the motivation, but it feels somewhat dodgy to allow packages in the DUB repository to depend on arbitrary git forks. What might make more sense is to allow a way for the build call (or settings in `dub.selections.json`) to easily override dependency locations, which can be used for temporary workaround like this. What do you think? It's obviously not great if e.g. one has a library package that triggers a bug in one of its dependencies and has no possibility of an in-code workaround, but it seems more secure and consistent to require dependency overrides to be chosen by the user rather than allowed in any circumstances.
One can already override locations in dub.selections.json, the problem there is automating it so downstream users don't have to do anything. The whole thing was just a workaround for a dub registry bug that got fixed anyway, so I don't think it's too important going forward.
Mar 18
prev sibling parent Seb <seb wilzba.ch> writes:
On Sunday, 17 March 2019 at 13:54:54 UTC, Joseph Rushton Wakeling 
wrote:
 On Thursday, 7 March 2019 at 16:01:18 UTC, Atila Neves wrote:
 A fair point.

 At the time I was trying to pin the pyd version because there 
 was a bug in the dub registry that prevented me from doing so 
 in the regular fashion. Then the bug got fixed and I moved on.
It wasn't meant as a personal attack! Sorry if it came over that way. I get the motivation, but it feels somewhat dodgy to allow packages in the DUB repository to depend on arbitrary git forks. What might make more sense is to allow a way for the build call (or settings in `dub.selections.json`) to easily override dependency locations, which can be used for temporary workaround like this. What do you think?
The dub.selections.json isn't used for dependencies. All major package managers have built-in VCS support for a good reason: https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support https://docs.npmjs.com/files/package.json#git-urls-as-dependencies https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html https://golang.github.io/dep/docs/ensure-mechanics.html ... Especially in the D ecosystem there are a ton of good, unmaintained projects which just require a small change to work with e.g. latest DMD and you can't get that into "upstream", because the author isn't there anymore or very inactive. I understand that in a perfect world we wouldn't have to do this, but in practice this is a lot better than registering dummy fork packages on the Dub registry every time. An example to what the current limitation leads to: https://github.com/dlang-tour/core/pull/487
Mar 19
prev sibling next sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
  community: what features do you miss the most from DUB?
(1) Support for library maintainers' test requirements Specifically: when a library has dependencies of its own, one wants to be sure the library stays compatible with the stated minimum supported version, because changing that could be a breaking change for downstreams. With this in mind it would be useful to have: * a command to downgrade all dependencies in dub.selections.json to the minimum supported (including dependencies of dependencies) * a flag for build and test commands to request that the minimum versions of all dependencies be used * a flag to specify the maximum kind of version change `dub upgrade` can perform: patch, minor, or major. (2) Cleanly and clearly separate between test options and build options Specifically, this involves fixing the problems with `dub test` when the `--build` flag is used. Example: if one runs `dub test --build=release` then intuitively one would expect (unit)tests to be run with release flags and optimizations enabled. In fact no unittests are run, because `--build=release` mandates a specific set of D flags that does not include `-unittest`.
Mar 03
next sibling parent Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 3 March 2019 at 17:03:20 UTC, Joseph Rushton Wakeling 
wrote:
 Example: if one runs `dub test --build=release` then 
 intuitively one would expect (unit)tests to be run with release 
 flags and optimizations enabled. In fact no unittests are run, 
 because `--build=release` mandates a specific set of D flags 
 that does not include `-unittest`.
As a work-around you can use buildTypes today: https://p0nce.github.io/d-idioms/#Unittests-with-optimizations-:-the-Final-Frontier I think it would be a good idea to have this in vanilla DUB.
Mar 03
prev sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 3/3/19 12:03 PM, Joseph Rushton Wakeling wrote:
 On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
  community: what features do you miss the most from DUB?
(1) Support for library maintainers' test requirements Specifically: when a library has dependencies of its own, one wants to be sure the library stays compatible with the stated minimum supported version, because changing that could be a breaking change for downstreams. With this in mind it would be useful to have:   * a command to downgrade all dependencies in dub.selections.json     to the minimum supported (including dependencies of dependencies)   * a flag for build and test commands to request that the minimum     versions of all dependencies be used   * a flag to specify the maximum kind of version change `dub upgrade`     can perform: patch, minor, or major.
Big "ditto" on this stuff. Althouth I'd change it to just "Support for library maintainers' requirements" in general. I've already had to deal with that stuff you mention by hacking it all into my travis setup scripts and then have a custom test runner script invoke `dub test`. For example, take a look at mysql-native's root dirctory: https://github.com/mysql-d/mysql-native Notice that not only is `dub.selections.json` included in version control, but I also have another file: `dub.selections.vibe-0.8.3.json`. That's because some compiler versions require vibe v0.7.x and NOT v0.8.x, but newer compilers require v0.8.x and up. So `dub.selections.json` is carefully set (by hand) to exactly my minimum dependency versions. And `dub.selections.vibe-0.8.3.json` is carefully set to the minimum versions required by the newer compilers. So then, in my travis setup script (ci_setup.d), I check for the envvar DUB_SELECT. If it exists, I overrwite `dub.selections.json` with `dub.selections.$(DUB_SELECT).json`: https://github.com/mysql-d/mysql-native/blob/2f9d71ca026dc675d07bfb4dd20ce44e07c9aac5/ci_setup.d#L37 The travis setup script also checks for another envvar, DUB_UPGRADE. If it's true, then I run "dub upgrade". So then, in my travis build matrix, I set up jobs both with and without DUB_SELECT/DUB_UPGRADE, as needed: DUB_SELECT: https://github.com/mysql-d/mysql-native/blob/2f9d71ca026dc675d07bfb4dd20ce44e07c9aac5/.travis.yml#L15 DUB_UPGRADE: https://github.com/mysql-d/mysql-native/blob/2f9d71ca026dc675d07bfb4dd20ce44e07c9aac5/.travis.yml#L119 Resulting tests: https://travis-ci.org/mysql-d/mysql-native/builds/497662010 BUT WAIT, THERE'S MORE WORKAROUND! Notice that when DUB_UPGRADE *isn't set, I still run `dub upgrade`, but with the `--missing-only` flag: https://github.com/mysql-d/mysql-native/blob/2f9d71ca026dc675d07bfb4dd20ce44e07c9aac5/ci_setup.d#L59 I needed to do that (which is an incredibly unintuitive way to pre-fetch all the dependencies without upgrading any), because the frequent dub server outages were causing jobs to be marked as "failed the tests" when really they were just "job error". One thing I absolutely refuse to do is manage a library that demands all users always "upgrade all the things or STFU". I only believe in dropping support when it's necessary, not "because I can" or "because I know my users' needs better than they do and I say they don't need version latest-1". So, I need to deal with this stuff. So, yea, some of this stuff *technically* *can* be done, but it would be REALLY nice if I didn't have to jump through all these hoops and hacks. And yes, once again, all this ultimately comes down to DUB demanding to always be the center of the world.
Mar 03
parent reply Jacob Carlborg <doob me.com> writes:
On 2019-03-04 06:32, Nick Sabalausky (Abscissa) wrote:

 Notice that not only is `dub.selections.json` included in version 
 control, but I also have another file: `dub.selections.vibe-0.8.3.json`. 
 That's because some compiler versions require vibe v0.7.x and NOT 
 v0.8.x, but newer compilers require v0.8.x and up. So 
 `dub.selections.json` is carefully set (by hand) to exactly my minimum 
 dependency versions. And `dub.selections.vibe-0.8.3.json` is carefully 
 set to the minimum versions required by the newer compilers.
Usually `dub.selections.json` should not be committed for libraries, only for applications. With the latest version of Dub it's now possible to specify which compiler is required [1]. That could potentially have solved your problem. I'm not sure if this means that Dub will try to pick an older version of the dependency if the compiler requirement doesn't match. [1] https://dlang.org/changelog/2.085.0.html#toolchain_requirements -- /Jacob Carlborg
Mar 04
parent "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 3/4/19 12:19 PM, Jacob Carlborg wrote:
 On 2019-03-04 06:32, Nick Sabalausky (Abscissa) wrote:
 
 Notice that not only is `dub.selections.json` included in version 
 control, but I also have another file: 
 `dub.selections.vibe-0.8.3.json`. That's because some compiler 
 versions require vibe v0.7.x and NOT v0.8.x, but newer compilers 
 require v0.8.x and up. So `dub.selections.json` is carefully set (by 
 hand) to exactly my minimum dependency versions. And 
 `dub.selections.vibe-0.8.3.json` is carefully set to the minimum 
 versions required by the newer compilers.
Usually `dub.selections.json` should not be committed for libraries, only for applications.
Well, its absolutely mandatory if you want your CI builds to be reproducible. (As they SHOULD be, unless you enjoy completely unnecessary potential problems.) Although, come to think of it, I guess I could just rename it to `dub.selections.ci-default.json` and have my travis setup script automatically rename that to `dub.selections.json` if I haven't chosen any other alternative dub.selections for the current CI job. But I'm not sure I see the harm in including `dub.selections.json`, though. If you're using the given project as a lib in your own project, dub doesn't actually do anything with the `dub.selections.json`s from the dependencies...or does it? 'Cause *that* would be bad...(Though I can't imagine *why* it would intentionally work that way.)
 With the latest version of Dub it's now possible to specify which 
 compiler is required [1]. That could potentially have solved your 
 problem.
 [1] https://dlang.org/changelog/2.085.0.html#toolchain_requirements
Yea, just saw that myself. I'm definitely thrilled to see it added (for other reasons), I wish it had been added years ago, but I don't see how it would help the cases I mentioned above.
 I'm not sure if this means that Dub will try to pick an older 
 version of the dependency if the compiler requirement doesn't match.
Hmm, maybe, I hadn't thought of that. I certainly hope so. But in any case, the vide-d versions supported by mysql-native lack the "toolchainRequirements" directive (as they predate it), so it doesn't help mysql-native's existing cases here, at least not anytime in the near future. It also doesn't help with needing to test on the minimum supported dependency versions.
Mar 04
prev sibling next sibling parent Domain <dont_email empty.com> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
  community: what features do you miss the most from DUB?
Nested subpackages: https://github.com/dlang/dub/issues/843
Mar 04
prev sibling parent bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Thursday, 28 February 2019 at 15:50:49 UTC, Seb wrote:
 So while we have quite a large list of potential GSoC projects 
 [1], I think that many of these ideas are still a bit too 
 abstract for potential students.

 [...]
Support staged build and install for ease packaging process. Usually build tools support staged install by using the special environment variable DESTDIR. It is a coding standard: https://www.gnu.org/prep/standards/html_node/DESTDIR.html Be able to run dub on offline computer and looking dependencies following this order: (i) /usr/{lib,include,bin}, (ii) /usr/local/{lib,include,bin}, (iii) /${HOME}/.local/{lib,include,bin}
Mar 19