www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - I just created a dub package. Frankly, the whole thign is backward.

reply deadalnix <deadalnix gmail.com> writes:
So, I have this nice little project, sdc. It builds using make, 
and everything was good. People wanted it to be distributed via 
dub, so be it, let's create a dub.json file.

Ho, you wanted to distribute a package? Well, now you have a new 
build system. make was working great for you? Too bad. Can dub 
run make? It doesn't look like it but who knows. I don't.

Ok, now I have two build systems, including one I never asked for 
nor have any need for, it is now time to register the package in 
the dub registry. I need to create an account, so far so good. I 
can now register the package.

Not so quick, the registry tells me. `A "license" field in the 
package description file is missing or empty. Check your 
dub.json.`. I check the dub.json and indeed, it has no license 
field. I had a suspicion, it was created minutes ago and my 
memory is still working. I'm not sure what to put in there. Will 
dub complain if I put a license it doesn't know about? The 
project is under the MIT licence, so I decide the put the value 
at `MIT` and push the change, hoping for the best.

The registry seems happy with the license change. I still don't 
if it expected something specific or if I could have put anything 
in there, either way, it seems to be happy with MIT. On the other 
hand, on the other hand, `The repository must have at least one 
tagged version (SemVer format with a "v" prefix, e.g. "v1.0.0" or 
"v0.0.1") to be published on the registry. Please add a proper 
tag using "git tag" or equivalent means and see 
http://semver.org/ for more information.` .

This must be a new requirement that came up over the last 5 
minutes, because nothing told me about this before. I tag master 
as `v0.0.1` and push the new tag. I'm glad dub is now not only 
replacing my build system, it is also now dictating the release 
process. It turns out I have several applications in that 
repository, but thanks to dub, I don't have to ask myself the 
question of whether I want to use the same version number for all 
of them or not.

Now the package exist, and while I never asked, I also found 
myself having to commit to a new build system and a release 
strategy.

Let me blunt: this is bullshit. While I make fun of the poor UX 
and its guessing game, this ultimately is something that isn't a 
blocker. However, this whole my way or the highway attitude to 
everything is everything i don't want in a software.

I want tool to work for me, not work for the tool. I want to 
publish a package, not commit to a new build system. I never 
asked to be told how to assign version number to the different 
applications int he repo, or even whether I should put several 
application in the same repo or not. dub told me what to do to a 
degree that is almost disrespectful and is a big "fuck you" from 
the D ecosystem to anyone who want to integrate with it.

Why do we use a package manager that expect everyone to use it to 
work in a certain way? It is blatantly obvious that many 
organization out there are not going to use the exact setup dub 
expects. So what do they do if they want to adopt D? Not use dub 
at all? It's an option, but everything is distributed via dub 
nowadays in the D ecosystem. Use dub to build everything? Fork 
their own version of dub? No, the only sensible path forward for 
pretty much any organization stumbling on this bullshit is to not 
use D at all because none of the crap we put out there is going 
to play nicely with what they already have.

Anyways, enough said. I'm don't think this post is going to 
dramatically change things, but it needed to be said. The total 
disregard for users is so endemic I cannot wrap my head around 
how this has been embraced by the community.
Apr 23 2022
next sibling parent monkyyy <crazymonkyyy gmail.com> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 correct opinions on dub


make biuld.d the standard

https://github.com/crazymonkyyy/shmup/blob/master/biuld.d

I'm sure like; after 3 iterations of people copying and pasting a 
script then editing it to their needs there could be a nice 
template that has 3x the features of dub with all the flexibility 
90% of people need.

and if you want to use make, you could just parse  a "make 
-version" commands output to check if it will do something 
sane/complain at the user to install make.
Apr 23 2022
prev sibling next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Apr 23, 2022 at 08:15:27PM +0000, deadalnix via Digitalmars-d wrote:

 So, I have this nice little project, sdc. It builds using make, and
 everything was good. People wanted it to be distributed via dub, so be it,
 let's create a dub.json file.


[...]

 Let me blunt: this is bullshit. While I make fun of the poor UX and its
 guessing game, this ultimately is something that isn't a blocker. However,
 this whole my way or the highway attitude to everything is everything i
 don't want in a software.


This is why I have not used dub at all in spite of having used D for a
decade, except for one aborted project where I used a dummy empty
project whose sole purpose was to pull in vibe.d and its tangle of
dependencies. I already have my own build system which works better than
dub, yet it insists that I have to conform to its rules and doesn't even
have an escape hatch for integrating with foreign build systems. IOW,
it's a walled garden. I have no patience for walled gardens; life is too
short for that.

[...]

 Anyways, enough said. I'm don't think this post is going to
 dramatically change things, but it needed to be said. The total
 disregard for users is so endemic I cannot wrap my head around how
 this has been embraced by the community.


FWIW, I have *not* embraced dub. I love D (the language), though.


T

-- 
Старый друг лучше новых двух.
Apr 23 2022
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 ...
 This must be a new requirement that came up over the last 5 
 minutes, because nothing told me about this before. I tag 
 master as `v0.0.1` and push the new tag. I'm glad dub is now 
 not only replacing my build system, it is also now dictating 
 the release process. It turns out I have several applications 
 in that repository, but thanks to dub, I don't have to ask 
 myself the question of whether I want to use the same version 
 number for all of them or not.


That imho is indeed strange. Perhaps a bug report?
The only explanation I find, is that dub needs to know what 
commits are actually considered as releases.


 Anyways, enough said. I'm don't think this post is going to 
 dramatically change things, but it needed to be said. The total 
 disregard for users is so endemic I cannot wrap my head around 
 how this has been embraced by the community.


So what can be done for dub then?
Apr 24 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 24/04/2022 8:18 PM, Alexandru Ermicioi wrote:

 The only explanation I find, is that dub needs to know what commits are 
 actually considered as releases.


It also needs to understand the versions themselves, hence SemVer.
Apr 24 2022
parent reply Daniel N <no public.email> writes:
On Sunday, 24 April 2022 at 08:19:36 UTC, rikki cattermole wrote:

 On 24/04/2022 8:18 PM, Alexandru Ermicioi wrote:

 The only explanation I find, is that dub needs to know what 
 commits are actually considered as releases.


 It also needs to understand the versions themselves, hence 
 SemVer.


Why force 'v' prefix though? That's incompatible with the 
standard used at my company. So we could not use dub even if we 
wanted to.
Apr 24 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 24/04/2022 8:52 PM, Daniel N wrote:

 Why force 'v' prefix though? That's incompatible with the standard used 
 at my company. So we could not use dub even if we wanted to.


I have no idea and I'm not sure there is going to be a one size fits all 
solution without adding issues into the version checking.

But in saying that, as long as what dub expects and what standard you 
follow does not conflict, just tag multiple times for the same commit. 
There shouldn't be an issue there.
Apr 24 2022
parent reply Paul Backus <snarwin gmail.com> writes:
On Sunday, 24 April 2022 at 09:47:25 UTC, rikki cattermole wrote:

 On 24/04/2022 8:52 PM, Daniel N wrote:

 Why force 'v' prefix though? That's incompatible with the 
 standard used at my company. So we could not use dub even if 
 we wanted to.


 I have no idea and I'm not sure there is going to be a one size 
 fits all solution without adding issues into the version 
 checking.

 But in saying that, as long as what dub expects and what 
 standard you follow does not conflict, just tag multiple times 
 for the same commit. There shouldn't be an issue there.


The registry could also allow you to create releases manually. 
Like, you type the version number and the tag/commit hash you 
want to use for that version into a form, and then whenever 
somebody asks for version x.y.z, it gives them the commit you 
specified.
Apr 24 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 25/04/2022 1:51 AM, Paul Backus wrote:

 The registry could also allow you to create releases manually. Like, you 
 type the version number and the tag/commit hash you want to use for that 
 version into a form, and then whenever somebody asks for version x.y.z, 
 it gives them the commit you specified.


For a while now I've been wanting the dub registry to be augmented with 
a proper artifact server.

After that its just a matter of teaching both dub and the registry the 
order of the versions for the given package.
Apr 24 2022
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 I want tool to work for me, not work for the tool.



DUB gives me a chance to download and run your program and just 
type "dub".

While if I have to understand your build system first (especially 
if it's a _custom_ build system), install the right version of 
make, etc.. I will probably go on with my life and not even try.
Apr 24 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 24/04/2022 9:44 PM, Guillaume Piolat wrote:

 On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 I want tool to work for me, not work for the tool.


 
 
 DUB gives me a chance to download and run your program and just type "dub".
 
 While if I have to understand your build system first (especially if 
 it's a _custom_ build system), install the right version of make, etc.. 
 I will probably go on with my life and not even try.


Is it dmc make? Cygwin's make, Microsoft's nmake? MingW's make...

So many options, and not a one may be on the PATH variable when dub 
executes.

At least with dub, unless you did something stupid involving shell 
commands or non-D dependencies, or didn't test on a given platform it 
should "just work".
Apr 24 2022
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Sunday, 24 April 2022 at 09:56:13 UTC, rikki cattermole wrote:

 On 24/04/2022 9:44 PM, Guillaume Piolat wrote:

 On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 I want tool to work for me, not work for the tool.


 
 
 DUB gives me a chance to download and run your program and 
 just type "dub".
 
 While if I have to understand your build system first 
 (especially if it's a _custom_ build system), install the 
 right version of make, etc.. I will probably go on with my 
 life and not even try.


 Is it dmc make? Cygwin's make, Microsoft's nmake? MingW's 
 make...

 So many options, and not a one may be on the PATH variable when 
 dub executes.


dub contains a dependency manager.

This is *EXACTLY* the problem that a dependency manager solves.


 At least with dub, unless you did something stupid involving 
 shell commands or non-D dependencies, or didn't test on a given 
 platform it should "just work".


Any project that is big enough so that `dmd -i` doesn't cut it 
anymore will do one of these things, most likely both.
Apr 24 2022
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Sunday, 24 April 2022 at 09:56:13 UTC, rikki cattermole wrote:

 Is it dmc make? Cygwin's make, Microsoft's nmake? MingW's 
 make...


GNU-make is dominating, but then you have cmake and the like with 
their overcomplicating "autoconfig-setups". Most D authors 
probably cannot test their build setup on enough platforms to 
iron out all possible build quirks…

Dub might be a valuable resource for getting attraction to new 
projects, but perhaps insufficient for complex projects.
Apr 25 2022
parent reply user1234 <user1234 12.de> writes:
On Monday, 25 April 2022 at 10:38:28 UTC, Ola Fosheim Grøstad 
wrote:

 On Sunday, 24 April 2022 at 09:56:13 UTC, rikki cattermole 
 wrote:

 Is it dmc make? Cygwin's make, Microsoft's nmake? MingW's 
 make...


 GNU-make is dominating, but then you have cmake and the like 
 with their overcomplicating "autoconfig-setups". Most D authors 
 probably cannot test their build setup on enough platforms to 
 iron out all possible build quirks…

 Dub might be a valuable resource for getting attraction to new 
 projects, but perhaps insufficient for complex projects.


common have you took a look at how is complex the recipe for what 
lead to this whole discussion ? it is not one of the complex 
projects you imagine.
Apr 25 2022
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 April 2022 at 10:41:35 UTC, user1234 wrote:

 common have you took a look at how is complex the recipe for 
 what lead to this whole discussion ? it is not one of the 
 complex projects you imagine.


My point was just this: gnu make is available on most developer 
setups (or easy to get hold of) so using that from Dub ought to 
be easy, but most applications that use make also use an 
auto-config layer on top of that. I doubt most Dub authors will 
be able to make good use of auto-config, so encouraging the use 
of make might create more issues than it solves. Not because make 
is complicated, but because autoconfig setups might require 
testing on many platforms.

It might be better to build on top of a package manager for C/C++ 
and find a way to reuse that in a way that autoconfigs an 
existing C/C++ package in such a way that it works with "import 
C".

D needs to focus on reducing the need for manpower, so reusing 
other eco systems makes good sense, but how to do it? That is far 
from clear, but there are some emerging options in the C/C++ 
community that could be investigated.
Apr 25 2022
prev sibling parent reply bauss <jj_1337 live.dk> writes:
On Monday, 25 April 2022 at 10:41:35 UTC, user1234 wrote:

 On Monday, 25 April 2022 at 10:38:28 UTC, Ola Fosheim Grøstad 
 wrote:

 On Sunday, 24 April 2022 at 09:56:13 UTC, rikki cattermole 
 wrote:

 Is it dmc make? Cygwin's make, Microsoft's nmake? MingW's 
 make...


 GNU-make is dominating, but then you have cmake and the like 
 with their overcomplicating "autoconfig-setups". Most D 
 authors probably cannot test their build setup on enough 
 platforms to iron out all possible build quirks…

 Dub might be a valuable resource for getting attraction to new 
 projects, but perhaps insufficient for complex projects.


 common have you took a look at how is complex the recipe for 
 what lead to this whole discussion ? it is not one of the 
 complex projects you imagine.


This discussion would never have happened if the project was just 
created with dub in mind from the beginning, like if it was built 
using dub since the beginning and not with make, or at the very 
least both.
Apr 25 2022
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 25 April 2022 at 11:18:57 UTC, bauss wrote:

 This discussion would never have happened if the project was 
 just created with dub in mind from the beginning, like if it 
 was built using dub since the beginning and not with make, or 
 at the very least both.


Good, what's you plan to get every project restarted from scratch 
using dub?
Apr 25 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Apr 25, 2022 at 04:05:10PM +0000, deadalnix via Digitalmars-d wrote:

 On Monday, 25 April 2022 at 11:18:57 UTC, bauss wrote:

 This discussion would never have happened if the project was just
 created with dub in mind from the beginning, like if it was built
 using dub since the beginning and not with make, or at the very
 least both.


 
 Good, what's you plan to get every project restarted from scratch
 using dub?


One ideological struggle I have with dub is that it imposes its
worldview upon your project, instead of fitting itself into what already
exists, like many other systems do.

I've worked with integrating software with OS package managers, for
example, and they often involve simply adding some files, or maybe a
subdirectory or two, to your existing source tree, and the packaging
tools pick it up from there.  None of them involve mandatory,
dictated-from-above reorganization of the source tree structure to fit
its ideals (such a thing would have quickly become too onerous to use --
people would just give up trying to cater to the system, and the
repository would just stagnate from a dearth of people who are willing
to go far enough to satisfy its whims).

Or take the old workhorse Make, which everybody (including myself)
hates. It does not require you to reorganize your source tree to fits
its worldview; all it needs is to add a Makefile with the right
contents, and it Just Works(tm).  The same can be said for a multitude
of other Make-replacement build systems.

But not so for dub. It demands that you conform to its conventions,
otherwise it flat out doesn't work (or refuses to).  It doesn't play
well with external tools: it's theoretically *possible* to invoke other
tools if you ask really nicely, but it's glued on as an afterthought,
the core design obviously does not acknowledge such usage.  It does not
support customization -- you either do things its way, or the highway.

And most importantly of all -- it does not integrate nicely with
existing codebases that are already working with a different system.
This is especially bad for a small ecosystem that's trying to grow. When
the hurdle to adoption involves such efforts as restructuring your
entire danged source tree to fit dub's completely (and needlessly)
arbitrary requirements, guess how many eager projects are rushing to
contribute.


T

-- 
A program should be written to model the concepts of the task it performs
rather than the physical world or a process because this maximizes the
potential for it to be applied to tasks that are conceptually similar and, more
important, to tasks that have not yet been conceived. -- Michael B. Allen
Apr 25 2022
parent Mike Parker <aldacron gmail.com> writes:
On Monday, 25 April 2022 at 17:07:18 UTC, H. S. Teoh wrote:


 I've worked with integrating software with OS package managers, 
 for example, and they often involve simply adding some files, 
 or maybe a subdirectory or two, to your existing source tree, 
 and the packaging tools pick it up from there.  None of them 
 involve mandatory, dictated-from-above reorganization of the 
 source tree structure to fit its ideals (such a thing would 
 have quickly become too onerous to use -- people would just 
 give up trying to cater to the system, and the repository would 
 just stagnate from a dearth of people who are willing to go far 
 enough to satisfy its whims).

 Or take the old workhorse Make, which everybody (including 
 myself) hates. It does not require you to reorganize your 
 source tree to fits its worldview; all it needs is to add a 
 Makefile with the right contents, and it Just Works(tm).  The 
 same can be said for a multitude of other Make-replacement 
 build systems.

 But not so for dub. It demands that you conform to its 
 conventions, otherwise it flat out doesn't work (or refuses 
 to).  It doesn't play well with external tools: it's 
 theoretically *possible* to invoke other tools if you ask 
 really nicely, but it's glued on as an afterthought, the core 
 design obviously does not acknowledge such usage.  It does not 
 support customization -- you either do things its way, or the 
 highway.



dub does *not* require you to use a specific structure for your 
source tree. It has a default that it looks for if none is 
specified ("source" or "src" subdirectories), but you have 
options to set it up however you'd like with the directives 
"sourcePaths", "sourceFiles", and "importPaths". You can output 
your binary to "targetPath" and use "workingDirectory" to specify 
where the executable should be run. It's missing a "libPaths" 
directive, but you can set that up for specific linkers via 
"lflags".
Apr 25 2022
prev sibling parent monkyyy <crazymonkyyy gmail.com> writes:
On Monday, 25 April 2022 at 11:18:57 UTC, bauss wrote:

 This discussion would never have happened if the project was 
 just created with dub in mind from the beginning,


I started with dub for all raylib projects, got annoyed with the 
lack of multi executables per project that share several files 
like oh idk type definitions without the whole song a dance.

Replaced it all with 150 lines of code. Dub doesn't narrow 
opinions on organization dont fit everyone.

Dub bad, I should have listened to the wise adr sooner
Apr 26 2022
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 24 April 2022 at 09:44:40 UTC, Guillaume Piolat wrote:

 On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 I want tool to work for me, not work for the tool.



 DUB gives me a chance to download and run your program and just 
 type "dub".

 While if I have to understand your build system first 
 (especially if it's a _custom_ build system), install the right 
 version of make, etc.. I will probably go on with my life and 
 not even try.


Just have dub ship the right version of make and alike. Problem 
solved.
Apr 24 2022
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Apr 24, 2022 at 03:30:06PM +0000, deadalnix via Digitalmars-d wrote:

 On Sunday, 24 April 2022 at 09:44:40 UTC, Guillaume Piolat wrote:

 On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 I want tool to work for me, not work for the tool.


 
 
 DUB gives me a chance to download and run your program and just type
 "dub".
 
 While if I have to understand your build system first (especially if
 it's a _custom_ build system), install the right version of make,
 etc..  I will probably go on with my life and not even try.


 
 Just have dub ship the right version of make and alike. Problem
 solved.


+1.  Had dub been designed with the proper build capabilities, it would
have been a simple of matter of: "Project P uses build system B" -> dub
downloads B, downloads P, and builds P with B. You can even version it:
"Project P version V1 uses build system B version V2" -> dub downloads P
version V1, downloads P version V2, and builds P with B.

This can't be *that* hard to do, can it?


T

-- 
"The number you have dialed is imaginary. Please rotate your phone 90 degrees
and try again."
Apr 24 2022
prev sibling next sibling parent user1234 <user1234 12.de> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 So, I have this nice little project, sdc. It builds using make, 
 and everything was good. People wanted it to be distributed via 
 dub, so be it, let's create a dub.json file.


Writing a makefile from sratch for the first time is not easier 
than writing a DUB recipe. I think 2 over 3 of your complains are 
invalid (license, semver), you should have read the manual 
before. It is clearly specified that they are mandatory fields, 
[here](https://dub.pm/package-format-json).

The only thing that's valid is the shared version for the other 
application.
Apr 24 2022
prev sibling next sibling parent reply norm <norm.rowtree gmail.com> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 So, I have this nice little project, sdc. It builds using make, 
 and everything was good. People wanted it to be distributed via 
 dub, so be it, let's create a dub.json file.

 [...]


At my company we use Conan for D because the majority of code is 
C++. Our D projects are Conan packages using Makefiles or CMake, 
just like our C++ projects. It is a pity D decided to try and 
reinvent package management when there are battle hardened 
options already available that are flexible enough to just work 
with D.

Conan is NOT perfect but to date it has just worked and is much 
easier when integrating D with C/C++ code and mixed build systems.
Apr 24 2022
parent reply drug <drug2004 bk.ru> writes:
On 25.04.2022 07:54, norm wrote:

 On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 So, I have this nice little project, sdc. It builds using make, and 
 everything was good. People wanted it to be distributed via dub, so be 
 it, let's create a dub.json file.

 [...]


 
 At my company we use Conan for D because the majority of code is C++. 
 Our D projects are Conan packages using Makefiles or CMake, just like 
 our C++ projects. It is a pity D decided to try and reinvent package 
 management when there are battle hardened options already available that 
 are flexible enough to just work with D.
 
 Conan is NOT perfect but to date it has just worked and is much easier 
 when integrating D with C/C++ code and mixed build systems.


Dub is much more easier than conan (we use it all over the place). 
Although conan works with binary artifacts in contrast to dub working 
with source code so dub and conan can't be compared easily.
Apr 25 2022
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 April 2022 at 14:48:47 UTC, drug wrote:

 Dub is much more easier than conan (we use it all over the 
 place). Although conan works with binary artifacts in contrast 
 to dub working with source code so dub and conan can't be 
 compared easily.


I am very sceptical of distributing binaries without very 
restrictive maintainers. Microsoft has something called 
[vcpkg](https://devblogs.microsoft.com/cppblog/vcpkg-april-2022-release
is-now-available/). I have no idea how that works, but they might be big enough
to sustain it.
Apr 25 2022
prev sibling parent reply norm <norm.rowtree gmail.com> writes:
On Monday, 25 April 2022 at 14:48:47 UTC, drug wrote:

 On 25.04.2022 07:54, norm wrote:

 On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 So, I have this nice little project, sdc. It builds using 
 make, and everything was good. People wanted it to be 
 distributed via dub, so be it, let's create a dub.json file.

 [...]


 
 At my company we use Conan for D because the majority of code 
 is C++. Our D projects are Conan packages using Makefiles or 
 CMake, just like our C++ projects. It is a pity D decided to 
 try and reinvent package management when there are battle 
 hardened options already available that are flexible enough to 
 just work with D.
 
 Conan is NOT perfect but to date it has just worked and is 
 much easier when integrating D with C/C++ code and mixed build 
 systems.


 Dub is much more easier than conan (we use it all over the 
 place). Although conan works with binary artifacts in contrast 
 to dub working with source code so dub and conan can't be 
 compared easily.


Conan works with either packaged binaries or packaged sources. I 
think the comparison is valid. I also disagree that dub is 
easier. Sure it is much simpler to get started for trivial 
projects but with any non-trivial project you end up hitting its 
walled garden pretty hard.
Apr 25 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 26 April 2022 at 01:56:42 UTC, norm wrote:

 Conan works with either packaged binaries or packaged sources. 
 I think the comparison is valid. I also disagree that dub is 
 easier. Sure it is much simpler to get started for trivial 
 projects but with any non-trivial project you end up hitting 
 its walled garden pretty hard.


My (limited) understanding of Conan is that Conan-central 
compiles packages to binaries for various configurations (over 
100 configurations for C++), but that it also allows for 
packaging of precompiled binaries. Isn't this a security hazard?

Is it possible to configure Conan so that it only compiles from 
sources and never downloads binaries?
Apr 26 2022
parent reply norm <norm.rowtree gmail.com> writes:
On Tuesday, 26 April 2022 at 07:37:42 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 26 April 2022 at 01:56:42 UTC, norm wrote:

 Conan works with either packaged binaries or packaged sources. 
 I think the comparison is valid. I also disagree that dub is 
 easier. Sure it is much simpler to get started for trivial 
 projects but with any non-trivial project you end up hitting 
 its walled garden pretty hard.


 My (limited) understanding of Conan is that Conan-central 
 compiles packages to binaries for various configurations (over 
 100 configurations for C++), but that it also allows for 
 packaging of precompiled binaries. Isn't this a security hazard?


No more than anything else like maven, linux package managers, 
chocolatey on windows. I hear vcpkg also supports prebuilt 
binaries now but I think it is only for private registries, 
they're not hosting them. Vcpkg is focused on build from source 
not because of security concerns but ABI and compiler 
compatibility.

It is also fairly easy, much more than pip or maven, to set up 
your own package registry server so it never goes out to he 
official hosted service for packages.


 Is it possible to configure Conan so that it only compiles from 
 sources and never downloads binaries?


When you invoke conan, or in your config file, you can tell it to 
only build from source so it will not download the binary package 
even if available.
Apr 26 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 26 April 2022 at 23:48:03 UTC, norm wrote:

 No more than anything else like maven, linux package managers, 
 chocolatey on windows. I hear vcpkg also supports prebuilt 
 binaries now but I think it is only for private registries, 
 they're not hosting them.


I think it comes down to credible management and vetting 
strategies. My impression from Debian stable (when I used it 
regularly) was that they took security very seriously and made it 
hard for rogue binaries to make it into the repo. It also helps 
to have a million critical and skilled users that are eagerly 
looking for flaws!

In contrast, I am much more critical of pip/Python and either 
avoid less known packages from pip or vet the code before 
executing it.


 When you invoke conan, or in your config file, you can tell it 
 to only build from source so it will not download the binary 
 package even if available.


Thanks, I might have look at Conan then… :)

(I guess it is ok if the binaries are reproducible and built on a 
secure cloud solution and not by individual contributors, but it 
still sounds like a risk to my ears.)
Apr 26 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 27 April 2022 at 06:48:48 UTC, Ola Fosheim Grøstad 
wrote:

 When you invoke conan, or in your config file, you can tell it 
 to only build from source so it will not download the binary 
 package even if available.


 Thanks, I might have look at Conan then… :)

 (I guess it is ok if the binaries are reproducible and built on 
 a secure cloud solution and not by individual contributors, but 
 it still sounds like a risk to my ears.)



Seems like vcpkg has expanded their scope, by automatically 
downloading compilers, linkers and SDKs for embedded development:

https://devblogs.microsoft.com/cppblog/vcpkg-artifacts/


  «The experience is in preview and currently focused on 
 embedded developers. We will expand the scope in the future to 
 include any developers targeting Linux, macOS, or Windows.»


Not sure how they plan to ensure the quality/security of this 
approach, but I guess Microsoft can afford to pay staff to do the 
auditing (if they do it).

It would be a grave mistake to try to replicate any of this for a 
small player like the D community. It would be better to get LDC 
and other dependencies into vcpkg.
Apr 27 2022
parent reply norm <nr gmail.com> writes:
On Wednesday, 27 April 2022 at 11:05:01 UTC, Ola Fosheim Grøstad 
wrote:

 On Wednesday, 27 April 2022 at 06:48:48 UTC, Ola Fosheim 
 Grøstad wrote:

 When you invoke conan, or in your config file, you can tell 
 it to only build from source so it will not download the 
 binary package even if available.


 Thanks, I might have look at Conan then… :)

 (I guess it is ok if the binaries are reproducible and built 
 on a secure cloud solution and not by individual contributors, 
 but it still sounds like a risk to my ears.)



 Seems like vcpkg has expanded their scope, by automatically 
 downloading compilers, linkers and SDKs for embedded 
 development:

 https://devblogs.microsoft.com/cppblog/vcpkg-artifacts/


  «The experience is in preview and currently focused on 
 embedded developers. We will expand the scope in the future to 
 include any developers targeting Linux, macOS, or Windows.»


 Not sure how they plan to ensure the quality/security of this 
 approach, but I guess Microsoft can afford to pay staff to do 
 the auditing (if they do it).

 It would be a grave mistake to try to replicate any of this for 
 a small player like the D community. It would be better to get 
 LDC and other dependencies into vcpkg.


+1

It would be good if D could get a foot in this door. vcpkg is 
growing in popularity in the C++ community and getting some 
really nice features. D compiler support would provide some great 
marketing exposure.

Afaik vckg is tied to CMake so if you want to use GNU make it may 
require some wrangling.
Apr 27 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 27 April 2022 at 23:22:23 UTC, norm wrote:

 Afaik vckg is tied to CMake so if you want to use GNU make it 
 may require some wrangling.


I am no expert on build systems, make does what I need most of 
the time, but CMake is becoming the de facto standard in the 
C/C++ world and displacing other autoconfig solutions, so I am 
not sure if there is another viable path that is future proof?

I understand the ease-of-use argument for Dub and pure D 
projects. The problem is that small languages are more likely to 
be incorporated as part of a multi-language project and then 
dealing with many different package/build systems become a 
liability and not an advantage. You might end up being forced to 
embed the Dub source into your project… and that is just way too 
clunky. It is not the kind of resistance you want people to face 
when evaluating language options.

So you got me thinking that you are onto something with your 
initial comment about Conan. Since it is possible to set up 
private servers for Conan and vcpkg, I would assume that one 
could fork either one of those and create a D version that also 
access the central C/C++ hub. Then lobby for the D version to be 
merged into upstream over time, that would give you full 
integration with C/C++ build strategies…

People could still use Dub, what you want is some sort of 
"syndication" solution so that Dub converts Dub packages and 
uploads them to other systems. (Kinda like blogging solutions 
that allows you to publish the same message to Twitter, Facebook, 
SMS and your own website at the push of one button).
Apr 28 2022
prev sibling next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 Why do we use a package manager that expect everyone to use it 
 to work in a certain way? It is blatantly obvious that many 
 organization out there are not going to use the exact setup dub 
 expects. So what do they do if they want to adopt D? Not use 
 dub at all? It's an option, but everything is distributed via 
 dub nowadays in the D ecosystem. Use dub to build everything? 
 Fork their own version of dub? No, the only sensible path 
 forward for pretty much any organization stumbling on this 
 bullshit is to not use D at all because none of the crap we put 
 out there is going to play nicely with what they already have.


Just shell out to dub as required. That's what we do.

I disagree with this post. Dub's requirements seem eminently 
sensible to me, if you publish source code without a license, 
you're basically setting a copyright trap for the unwary. I 
disagree with the v1.2.3 format but it's explicitly called out as 
a common choice in the semver spec. And dub's goal should be to 
be simple first, composable second. Why make dub shell out to 
another build system? If you want to use another build system, 
just build with that system. Dub is off in its own little world 
of D packages, and that's fine, it limits the breakage surface, 
encourages internal standardization and makes it easy to get 
started for a newcomer. Imagine if you run 'dub build' and it 
told you to install meson (because some dependency needed it), 
you'd be weirded out, right? "So is Dub the D build system or 
meson?" Imagine needing to install *multiple* build systems. I'm 
glad that dub doesn't enable that sort of mess.

It's okay to not have a feature sometimes.
Apr 24 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 April 2022 at 06:43:47 UTC, FeepingCreature wrote:

 I disagree with this post. Dub's requirements seem eminently 
 sensible to me, if you publish source code without a license, 
 you're basically setting a copyright trap for the unwary.


I don't know. I recently spent quite a bit of time with Faust and 
its standard library has different licenses for different 
functions (GPL, LGPL, MIT etc). This makes perfect sense when you 
want to use the best tools (DSP algorithms) for the job and have 
them all at your fingertips. So it isn't obvious that having one 
license for a library make sense. For some projects it would make 
more sense to have a "license filter" of some sort at the 
compiler-level.

So, for instance instead of having one FFT function in your 
"numerical" package you can have an FFT interface and have many 
FFT implementations (with various licenses) in the same 
"numerical" package. Then you can easily test many of them for 
performance/quality in your application with little effort.

In general, it is a problem if D has a package system that 
authors don't use. It would probably make more sense to use what 
is/becomes common for C++/C and build on top of that. Larger D 
projects will use a lot of C++/C code anyway.


 system, just build with that system. Dub is off in its own 
 little world of D packages, and that's fine, it limits the 
 breakage surface, encourages internal standardization and makes 
 it easy to get started for a newcomer.


Ok, so you have to decide what the purpose of Dub is. If it is 
primarily for newbies, then you focus more on quality testing 
packages before making them public, requiring good documentation 
and making sure that they are removed when outdated.

If it is for more advanced projects you have to make sure that 
everybody wants to use it. If one commonly used library choses 
not to use it, then it becomes unworkable.

In Java, Javascript, Go and Rust you have a culture of not using 
much FFI, so they are not really comparable to a language where 
you use a lot of FFI. More importantly they have a large enough 
user base to maintain libraries over time. In some of these 
languages you cannot avoid using their leading package 
managers/build systems, it is close to mandatory. And some of the 
builds that these package managers lead to adds bloat.

Anyway, there is a reason for why there is an increasing demand 
for header-only libraries in C++. People learn the hard way that 
libraries that require setup will suck up time, increase 
complexity and usually add bloat, regardless of build system used…

*Fo D I don't really think it matters either way, as the main 
issue seems to be to get enough maintained high quality libraries 
to drive the adoption in the first place.*

The best strategic option right now seems to be to have 
maintained C-library bindings that uses "import C" and ship them 
with the compiler (funded by the D foundation) and provide an 
install-option that uses the operating system's package manager 
(linux distribution, macports, homebrew etc) so that the install 
becomes 100% build-free.

I personally try to avoid libraries, regardless of language, that 
require building beyond the standard compiler features. I think 
most pure D libraries should be nothing more than a github fetch 
and it would be better to reuse a system that is likely to be 
dominating C/C++ builds in the future, or rather to use a fork of 
such a system that adds information useful for D builds in a 
registry, such as classifying existing C-libraries as "import C" 
compatible.
Apr 25 2022
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 25 April 2022 at 09:20:23 UTC, Ola Fosheim Grøstad 
wrote:

 Ok, so you have to decide what the purpose of Dub is. If it is 
 primarily for newbies


The reason seasoned developers use things like DUB is not ease of 
building, but debt control.

With Semver you will get bug fixes while you sleep.
For example, if I fix issue 
https://github.com/AuburnSounds/Dplug/issues/642 and release a 
new DUB tag, you get this bug fix without even knowing the 
problem existed, at your next "dub upgrade". For the user of the 
library, it doesn't enter their consciousness and they can go on 
with their lives not caring about what this is, or which commit 
they should look at.

Because dependencies are nested, and reused across 
people/organizations, this happens at various levels hence 
improving cost effectiveness of development.

Things like DUB and NPM replaces attention by trust.
Apr 25 2022
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 April 2022 at 11:15:59 UTC, Guillaume Piolat wrote:

 For example, if I fix issue 
 https://github.com/AuburnSounds/Dplug/issues/642 and release a 
 new DUB tag, you get this bug fix without even knowing the 
 problem existed, at your next "dub upgrade". For the user of 
 the library, it doesn't enter their consciousness and they can 
 go on with their lives not caring about what this is, or which 
 commit they should look at.


This can be problematic, sometimes people hardcode a way around 
library bugs and things will go wrong if they are silently fixed. 
I'd rather not having bugs silently fixed if I use a library for 
something significant.


 Because dependencies are nested, and reused across 
 people/organizations, this happens at various levels hence 
 improving cost effectiveness of development.

 Things like DUB and NPM replaces attention by trust.


In theory. The problem is when there are breaking changes in a 
package used by a package used by a package. This can happen if 
the versioning-condition wasn't fixed, or if a package's package 
requirements are updated and the application relies on features 
of an indirect dependency (e.g. you import a "webserver package" 
and depend on the interface of objects from an indirectly 
imported "json package").

Such things happen, like I had this experience with a widely used 
web server library for Python called "Flask" that imports a 
library called "itsdangerous". The condition Flask used for 
"itsdangerous" was >= some.required.version. So it worked well on 
local testing, but when it was uploaded to the cloud the cloud 
service retrieved a more recent version of "itsdangerous" and it 
failed to run.

In the npm environment the problem is much larger as javascript 
authors go out of their way by importing various libraries for 
even the most trivial functionality. If you use a larger 
javascript project you risk having many different versions of the 
same type of functionality because different packages pull in 
different (but similiar) dependencies! So I usually choose not to 
import npm-packages, only larger ones that are maintained by 
professional teams (e.g. angular), then those teams can deal with 
the deep-import npm-dependency-sillyness so I don't get burned.

For Python it usually works ok, but I think that is because the 
programming environment in Python is very stable ("virtual 
machine") and commonly packages usually don't change drastically 
most of the time. Also the user base is so large that breaking 
changes are detected. You can make the same arguments for Go, as 
it is also to a large extent a "self sufficient virtual 
machine/environment".

Now, you might not see these issues in the use of Dub until there 
are many packages that depends on each other.
Apr 25 2022
next sibling parent reply bauss <jj_1337 live.dk> writes:
On Monday, 25 April 2022 at 11:47:17 UTC, Ola Fosheim Grøstad 
wrote:

 Such things happen, like I had this experience with a widely 
 used web server library for Python called "Flask" that imports 
 a library called "itsdangerous". The condition Flask used for 
 "itsdangerous" was >= some.required.version. So it worked well 
 on local testing, but when it was uploaded to the cloud the 
 cloud service retrieved a more recent version of "itsdangerous" 
 and it failed to run.


Another great example, or even more infamous example would be 
this:

https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/

There are a couple more examples like the above, most notably it 
has happened to NPM the most.
Apr 25 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 April 2022 at 12:03:13 UTC, bauss wrote:

 Another great example, or even more infamous example would be 
 this:

 https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/

 There are a couple more examples like the above, most notably 
 it has happened to NPM the most.


You also have the issue of checking indirect imports for bad or 
malicious code. Which is why I avoid npm for the most part, at 
some point it will be used for "cyber warfare" or mischief. It is 
only a matter of time as the attack vector is so accessible and 
obvious.

(It is very difficult to vet code you build on when it is pulled 
in by package managers.)
Apr 25 2022
prev sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 25 April 2022 at 11:47:17 UTC, Ola Fosheim Grøstad 
wrote:

 On Monday, 25 April 2022 at 11:15:59 UTC, Guillaume Piolat 
 wrote:

 For example, if I fix issue 
 https://github.com/AuburnSounds/Dplug/issues/642 and release a 
 new DUB tag, you get this bug fix without even knowing the 
 problem existed, at your next "dub upgrade". For the user of 
 the library, it doesn't enter their consciousness and they can 
 go on with their lives not caring about what this is, or which 
 commit they should look at.


 This can be problematic, sometimes people hardcode a way around 
 library bugs and things will go wrong if they are silently 
 fixed. I'd rather not having bugs silently fixed if I use a 
 library for something significant.


 Because dependencies are nested, and reused across 
 people/organizations, this happens at various levels hence 
 improving cost effectiveness of development.

 Things like DUB and NPM replaces attention by trust.


 In theory.


Not in theory.
If you won't trust anyone you can live in your own packages and 
it's all in your control.
But by foregoing some degree of control you save on debt.


 The problem is when there are breaking changes in a package 
 used by a package used by a package.


Then it is a breach of trust and upstream should stop depending 
on it.


 Such things happen, like I had this experience with a widely 
 used web server library for Python called "Flask" that imports 
 a library called "itsdangerous". The condition Flask used for 
 "itsdangerous" was >= some.required.version. So it worked well 
 on local testing, but when it was uploaded to the cloud the 
 cloud service retrieved a more recent version of "itsdangerous" 
 and it failed to run.


You should never use >= without <=. It is so basic I'm surprised 
a popular framework would do it. Nothing prevents people from 
using Semver correctly. Like OOP.



 In the npm environment the problem is much larger as javascript 
 authors go out of their way by importing various libraries for 
 even the most trivial functionality.


Yeah we don't have that in D.
But I've seen something similar in Rust, hundreds of packages 
because small stdlib.



 If you use a larger javascript project


Thankfully this concerns the Javascript ecosystem, and not the D 
ecosystem.


 Now, you might not see these issues in the use of Dub until 
 there are many packages that depends on each other.


And at this point (IF this happens with D) this isn't a problem 
with DUB, but a community problem.
Apr 25 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Monday, 25 April 2022 at 12:11:48 UTC, Guillaume Piolat wrote:

 If you won't trust anyone you can live in your own packages and 
 it's all in your control.
 But by foregoing some degree of control you save on debt.


You can use dplug Dub for prototyping and switch to using a 
github fork if you release it, then you get to track all changes 
in detail on your own private fork.

Most programs are "personal prototypes", so one can use both, 
sure.


 You should never use >= without <=. It is so basic I'm 
 surprised a popular framework would do it. Nothing prevents 
 people from using Semver correctly. Like OOP.


All I can say is that I wasn't pleased when small trivial tweak 
on the webserver turned into a longer debugging-session…  (I 
wasn't planning on upgrading anything, just wanted to add a line 
of code.)
Apr 25 2022
parent reply Guillaume Piolat <first.last gmail.com> writes:
On Monday, 25 April 2022 at 12:45:39 UTC, Ola Fosheim Grøstad 
wrote:

 All I can say is that I wasn't pleased when small trivial tweak 
 on the webserver turned into a longer debugging-session…  (I 
 wasn't planning on upgrading anything, just wanted to add a 
 line of code.)


If you interpret your anecdote differently, it says that 
framework Flask subscribed to a >= dependency and got exactly 
what it asked for: unlimited breaking changes.
A solution would have been to make the build system even more 
restrictive :) and disallow that anti-pattern, in order to 
improve your life, as a user of the dependency.
Because there is one maintainer that needs to make a change, and 
much more users that will suffer.

Similarly, it seems in this thread people misunderstand what the 
declarative restrictions of DUB are about.

Yes, with make you can do "everything".

But can you write a macOS tool that takes a Makefile and build an 
Universal Binary from a make recipe that builds a single 
architecture binary? You can't because a Makefile can do anything 
and it fully imperative, nothing describes where the binary is, 
or architectures.
But with dub you can do that, because it's all declarative.

Can you write a tool that analyze all your dependent packages, 
get number of open issues and compute total LOC of the project? 
yes.
Can you get all import paths of the binary and dependencies, and 
generate an IDE project from that? yes.
Can you get a new architecture for free with the same recipe? yes.

cf: least power for the job => 
https://www.w3.org/2001/tag/doc/leastPower.html
Apr 25 2022
next sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 25 April 2022 at 19:17:37 UTC, Guillaume Piolat wrote:

 Similarly, it seems in this thread people misunderstand what 
 the declarative restrictions of DUB are about.


Dude, we understand.

That condescending attitude is exactly how dub got as bad as it 
got.
Apr 25 2022
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Monday, 25 April 2022 at 19:17:37 UTC, Guillaume Piolat wrote:

 cf: least power for the job => 
 https://www.w3.org/2001/tag/doc/leastPower.html


The least power for the job definitively implies not inserting a 
build system in the package manager.
Apr 25 2022
parent reply Mike Parker <aldacron gmail.com> writes:
On Monday, 25 April 2022 at 20:12:19 UTC, deadalnix wrote:


 The least power for the job definitively implies not inserting 
 a build system in the package manager.


I'm posting in reply to deadalnix, but my comments are not 
directed at him specifically.

dub exists because it filled a huge, gaping hole in the D 
ecosystem.

Go back far enough and you'll find "build", later renamed "bud", 
by Derek Parnell. This was the first D build tool the community 
rallied around. Several D users adopted it.

https://www.digitalmars.com/d/archives/digitalmars/D/40914.html

Gregor Richards wasn't happy with bud, so he created "Rebuild" as 
an alternative build tool, but added a package manager on top of 
it called "DSSS" (D Shared Software System). Rebuild actually 
parsed source files looking for custom pragmas, and IIRC DSSS 
could install packages from multiple locations (svn, git, etc). 
Several D libraries users adopted it.

https://forum.dlang.org/thread/f06hsi$ng1$2 digitalmars.com

So let me tell you, this sounds but it was also a serious PITA as 
a library maintainer. You had the bud crowd demanding support for 
bud, the dsss crowd demanding support for dsss, and the crowd who 
refused to use either because make was good enough. At one point 
I had three platform-specific make files, a `dsss.conf`, and a 
`forbud.txt` (I eventually replaced some of that with a 
`buildme.d file`).

There was no standard way of building D projects, and it sucked. 
Then Derek and Gregor disappeared, and suddenly these build 
systems everyone is using are no longer supported. No one takes 
them over, so we're in the Dark Ages again. Then dub comes along 
and saves the day.

It caught on quickly. Then people start thinking: "What happens 
if Sönke disappears? This should be an official D thing." So dub 
gets added to the installation. And here we are.

Without dub, we would not have the ecosystem that we do have. 
Personally, I love it. It makes using dependencies a snap. I'm 
happy with the default source tree expectations, and I don't mind 
compiling C libraries separately when I need them.

But I understand that no everyone is going to be happy with it (I 
don't understand the level of hatred some people direct at it, 
but I just chalk it up as "they aren't happy with it" and move 
on).

The question is, what exactly is dub missing and how can it be 
improved? Which perceived shortcomings are reasonable to address 
and which are not? And assuming we can answer those questions, 
who is going to address them?

We tried to secure funding a while back for someone to oversee 
development more actively since Sönke is too busy to spend the 
time on it that he used to, but that didn't happen. And no one on 
the core team has the time needed to steer the project. So until 
one of those things changes, the only way it's going to happen is 
if one or more people step up and make it happen.

So I propose someone, anyone, who has an interest in seeing dub 
evolve raise their hand and take on the task of gathering 
requirements. Not dive in and start refactoring code, not sit 
down and write a new build tool, but to gather the gripes and 
complaints from bitch sessions here and on Discord and distill 
them into lists of needs, wants, and nice-to-haves. (I'd do it 
myself, but I just don't have the time.)

Then that list can be a starting point for a plan to move 
forward. I'm sure I can put together a team of interested parties 
willing to donate a little of their time to provide input on 
formulating that plan. Then we can look at how to actually get 
the work done, e.g., putting out a task list that contributors 
can spend time on, picking out more complicated tasks for 
contract work funded from our ecosystem fund, etc.

So please, someone take all this energy directed at telling us 
all how terrible dub is and take the first step toward 
redirecting at turning dub into something more of us are willing 
to use.
Apr 25 2022
next sibling parent Arjan <arjan ask.me.to> writes:
On Tuesday, 26 April 2022 at 01:21:58 UTC, Mike Parker wrote:

 Without dub, we would not have the ecosystem that we do have.


I think so too.


 Personally, I love it. It makes using dependencies a snap. I'm 
 happy with the default source tree expectations, and I don't 
 mind compiling C libraries separately when I need them.


Had to put off my cmake/ninja glasses first to get rid of some 
prejudice, cmake btw is not heaven either, but does the job and 
has and is still improving. Dub has already improved a lot over 
time I do not experience any big issues with it since quite some 
time though my use cases are quite straight fwd.


 So I propose someone, anyone, who has an interest in seeing dub 
 evolve raise their hand and take on the task of gathering 
 requirements. Not dive in and start refactoring code, not sit 
 down and write a new build tool, but to gather the gripes and 
 complaints from bitch sessions here and on Discord and distill 
 them into lists of needs, wants, and nice-to-haves. (I'd do it 
 myself, but I just don't have the time.)

 Then that list can be a starting point for a plan to move 
 forward. I'm sure I can put together a team of interested 
 parties willing to donate a little of their time to provide 
 input on formulating that plan. Then we can look at how to 
 actually get the work done, e.g., putting out a task list that 
 contributors can spend time on, picking out more complicated 
 tasks for contract work funded from our ecosystem fund, etc.

 So please, someone take all this energy directed at telling us 
 all how terrible dub is and take the first step toward 
 redirecting at turning dub into something more of us are 
 willing to use.


This is helpful. Thanks for doing this Mike.
Apr 26 2022
prev sibling next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 26 April 2022 at 01:21:58 UTC, Mike Parker wrote:

 On Monday, 25 April 2022 at 20:12:19 UTC, deadalnix wrote:


 The least power for the job definitively implies not inserting 
 a build system in the package manager.


 The question is, what exactly is dub missing and how can it be 
 improved? Which perceived shortcomings are reasonable to 
 address and which are not? And assuming we can answer those 
 questions, who is going to address them?


Excellent question. My answer would be that it doesn't miss 
anything, to the contrary. If it was limiting itself to fetching 
dependencies and allow to publish new version of these, then it 
would be perfect.

But in addition of doing that, it is telling me how to build, how 
to do releases and so on. That was not a void i needed filling, I 
had tools to do this just fine. Now dub is getting into the way 
by interfering with say tools.


 So please, someone take all this energy directed at telling us 
 all how terrible dub is and take the first step toward 
 redirecting at turning dub into something more of us are 
 willing to use.


I do not want to improve dub, I do not want to even use it. I'm 
more or less dragged along because this is what people expect in 
the ecosystem. It causes way more problems for me that it solves. 
If that was up to me, I wouldn't fix it, I would just stop using 
it.
Apr 26 2022
next sibling parent Loara <loara noreply.com> writes:
On Tuesday, 26 April 2022 at 10:48:49 UTC, deadalnix wrote:

 I do not want to improve dub, I do not want to even use it. I'm 
 more or less dragged along because this is what people expect 
 in the ecosystem. It causes way more problems for me that it 
 solves. If that was up to me, I wouldn't fix it, I would just 
 stop using it.


Then don't use it, use instead your custom build system. If your 
project is open source someone else will port it on 
dub/Conan/Cmake/AUR/Microsoft Store/some other exotic repository. 
You don't need to use a tool you don't like.

If peoplewant something from you that you don't want then it's up 
to them to fit their needs if your project is free software.
Apr 26 2022
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/26/22 6:48 AM, deadalnix wrote:

 On Tuesday, 26 April 2022 at 01:21:58 UTC, Mike Parker wrote:

 So please, someone take all this energy directed at telling us all how 
 terrible dub is and take the first step toward redirecting at turning 
 dub into something more of us are willing to use.


 
 I do not want to improve dub, I do not want to even use it. I'm more or 
 less dragged along because this is what people expect in the ecosystem. 
 It causes way more problems for me that it solves. If that was up to me, 
 I wouldn't fix it, I would just stop using it.


There's no requirement to use dub. If you want to build your own 
ecosystem, or don't want to use this one, then don't. But saying "nobody 
should use it, it should be just wild-west without any direction" is not 
helpful.

This means you lose those benefits of being there. Your call.

I can say with certainty that I won't be leaving dub, even though I have 
several issues with it. But literally none of the things you complain 
about are any concern to me, in fact, I consider them to be features I want.

I would like to see dub redone in a way that is approachable. My biggest 
problem is that the code is impenetrable. So even if I want to fix 
problems, I can't. Even in this scenario, I'd keep all the things you 
complain about, so that isn't really relevant to you.

-Steve
Apr 26 2022
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 26 April 2022 at 17:02:21 UTC, Steven Schveighoffer 
wrote:

 There's no requirement to use dub.


Strictly speaking, there isn't.

Practically speaking, because dub doesn't really play nice with 
anything else, you end having to either:
1/ Not use any 3rd party lib.
2/ Copy 3rd party libs in your source tree and redo the build.
3/ Convert to our lord and savior dub.
4/ Have dub build the 3rd party lib and pile on hacks to recover 
the results in your own build.

All of these option are bad, thanks to widespread dub adoption in 
the D ecosystem.
Apr 26 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 7:02 AM, deadalnix wrote:

 2/ Copy 3rd party libs in your source tree and redo the build.
 4/ Have dub build the 3rd party lib and pile on hacks to recover the 
 results in your own build.


Something I want to see is a proper artifact repository.

For non-D that should allow us to upload binaries to it, and have dub 
automatically download and copy it based upon the target.

I think such a service would go a long way to give a much better 
experience for people.
Apr 26 2022
parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 26 April 2022 at 19:06:47 UTC, rikki cattermole wrote:

 Something I want to see is a proper artifact repository.

 For non-D that should allow us to upload binaries to it, and 
 have dub automatically download and copy it based upon the 
 target.

 I think such a service would go a long way to give a much 
 better experience for people.


None of these approaches work, because the core problem is that 
the eco system is too small to sustain a secure package system 
detached from the rest of the world. I would never use it. Too 
dangerous and exploitable. One big security vulnerability that I 
don't want to think about.

There is no point in dealing with C packages when others do it, 
one should at most only maintain a thin set of additional files 
to cover those areas where "import C" falls short.

We are now getting dedicated languages that compile directly to 
LLVM IR (with claims of better code-gen than compiling to C), 
things are in a flux and it is increasingly expensive to roll 
your own.

Of course, if the goal was for D to be self-sufficient then that 
would be different, but that does not seem to be the strategy.

*If the official strategy is to mix D with other languages then 
that has to be reflected in the official strategy for building 
the eco system.*

Or change the strategy.
Apr 26 2022
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 7:27 AM, Ola Fosheim Grøstad wrote:

 None of these approaches work, because the core problem is that the eco 
 system is too small to sustain a secure package system detached from the 
 rest of the world. I would never use it. Too dangerous and exploitable. 
 One big security vulnerability that I don't want to think about.


Unless we build literally everything in D, people will continue to ship 
their own binaries for platforms such as Windows. So that their 
library/binding "just works".

But in saying that, you do raise a good point. A long term strategy may 
be to require only our CI to build + upload binaries. This would make it 
reproducible since its script based.
Apr 26 2022
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Tuesday, 26 April 2022 at 19:27:38 UTC, Ola Fosheim Grøstad 
wrote:

 *If the official strategy is to mix D with other languages then 
 that has to be reflected in the official strategy for building 
 the eco system.*

 Or change the strategy.


Ding ding ding!
Apr 26 2022
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/26/22 3:02 PM, deadalnix wrote:

 On Tuesday, 26 April 2022 at 17:02:21 UTC, Steven Schveighoffer wrote:

 There's no requirement to use dub.


 
 Strictly speaking, there isn't.
 
 Practically speaking, because dub doesn't really play nice with anything 
 else, you end having to either:
 1/ Not use any 3rd party lib.
 2/ Copy 3rd party libs in your source tree and redo the build.
 3/ Convert to our lord and savior dub.
 4/ Have dub build the 3rd party lib and pile on hacks to recover the 
 results in your own build.
 
 All of these option are bad, thanks to widespread dub adoption in the D 
 ecosystem.


The solution is not to bring everyone down to your misery though. "If I 
can't have a good experience with what I want, nobody should" isn't the 
right answer.

-Steve
Apr 26 2022
next sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 26 April 2022 at 19:29:31 UTC, Steven Schveighoffer 
wrote:

 The solution is not to bring everyone down to your misery 
 though. "If I can't have a good experience with what I want, 
 nobody should" isn't the right answer.


Of course not, but Dub has the position of being the official 
tool for building the eco system. If it wasn't then there would 
be no argument, I think.

If Walter is betting heavily on C-interop then that should be 
reflected in the official tooling. The goal should be to provide 
easy access to all C-packages, also the ones that are more 
esoteric. The only way to achieve that is to let (non D) 
outsiders do the hard work and then you need to do something 
different.
Apr 26 2022
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/26/22 3:41 PM, Ola Fosheim Grøstad wrote:

 On Tuesday, 26 April 2022 at 19:29:31 UTC, Steven Schveighoffer wrote:

 The solution is not to bring everyone down to your misery though. "If 
 I can't have a good experience with what I want, nobody should" isn't 
 the right answer.


 
 Of course not, but Dub has the position of being the official tool for 
 building the eco system. If it wasn't then there would be no argument, I 
 think.
 
 If Walter is betting heavily on C-interop then that should be reflected 
 in the official tooling. The goal should be to provide easy access to 
 all C-packages, also the ones that are more esoteric. The only way to 
 achieve that is to let (non D) outsiders do the hard work and then you 
 need to do something different.
 
 


No amount of ImportC is going to encompass building all existing C projects.

Dub needs to play friendly with artifacts from other build systems. I 
have run into headaches with using other non-D projects also. But it 
doesn't have to play friendly with alternatives to building D code IMO.

But if you can all find a way to build dub projects in a way that's 
usable outside building with dub (e.g. `dub --produce-makefile` or 
whatnot), that's fine by me. Just don't make me change how I build my 
projects.

-Steve
Apr 26 2022
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Apr 26, 2022 at 04:29:54PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:
[...]

 Dub needs to play friendly with artifacts from other build systems. I
 have run into headaches with using other non-D projects also. But it
 doesn't have to play friendly with alternatives to building D code
 IMO.
 
 But if you can all find a way to build dub projects in a way that's
 usable outside building with dub (e.g. `dub --produce-makefile` or
 whatnot), that's fine by me. Just don't make me change how I build my
 projects.


[...]

I'm really tempted to create a build tool that parses dub.sdl, et al, in
the source tree and maps them internally to its own representation that
replicates dub's actions, so existing dub projects would Just Work(tm),
while exposing a different interface for more advanced use cases. Or
perhaps extend dub.sdl for that purpose.


T

-- 
Never step over a puddle, always step around it. Chances are that whatever made
it is still dripping.
Apr 26 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 8:41 AM, H. S. Teoh wrote:

 On Tue, Apr 26, 2022 at 04:29:54PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:
 [...]

 Dub needs to play friendly with artifacts from other build systems. I
 have run into headaches with using other non-D projects also. But it
 doesn't have to play friendly with alternatives to building D code
 IMO.

 But if you can all find a way to build dub projects in a way that's
 usable outside building with dub (e.g. `dub --produce-makefile` or
 whatnot), that's fine by me. Just don't make me change how I build my
 projects.


 [...]
 
 I'm really tempted to create a build tool that parses dub.sdl, et al, in
 the source tree and maps them internally to its own representation that
 replicates dub's actions, so existing dub projects would Just Work(tm),
 while exposing a different interface for more advanced use cases. Or
 perhaps extend dub.sdl for that purpose.


Don't parse the files themselves.

Use $(dub describe) instead.

Note: dub is capable of generating for other build systems including make.
Apr 26 2022
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 8:52 AM, rikki cattermole wrote:

 Note: dub is capable of generating for other build systems including make.


Okay looks like it doesn't support make (oops).

What it does support:

visuald - VisualD project files
sublimetext - SublimeText project file
cmake - CMake build scripts
build - Builds the package directly
Apr 26 2022
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 08:52:06AM +1200, rikki cattermole via Digitalmars-d
wrote:

 
 On 27/04/2022 8:41 AM, H. S. Teoh wrote:


[...]

 I'm really tempted to create a build tool that parses dub.sdl, et
 al, in the source tree and maps them internally to its own
 representation that replicates dub's actions, so existing dub
 projects would Just Work(tm), while exposing a different interface
 for more advanced use cases. Or perhaps extend dub.sdl for that
 purpose.


 
 Don't parse the files themselves.
 
 Use $(dub describe) instead.


[...]

I know `dub describe` can do it, but the point is for said tool to run
on a dub project with zero additional effort from the user, and to do so
even if dub is absent.  So even non-dub users can use dub packages
directly.


T

-- 
It is not the employer who pays the wages. Employers only handle the money. It
is the customer who pays the wages. -- Henry Ford
Apr 26 2022
parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Tuesday, 26 April 2022 at 21:16:00 UTC, H. S. Teoh wrote:
[...]

 even if dub is absent.


What scenario would that be? Dub is distributed with the compiler.

-- Bastiaan.
Apr 27 2022
prev sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 26 April 2022 at 20:29:54 UTC, Steven Schveighoffer 
wrote:

 No amount of ImportC is going to encompass building all 
 existing C projects.


Yes, I of course agree with this, but one could make it a goal to 
only require thin shims that require minimal amounts of 
maintenance. If you only require thin shims then some sort of 
collaborative effort could go a long way to keep it up to date 
(e.g. you create a partial shim for your project and I improve on 
it when I need it).

It does require some organizing efforts though.


 But if you can all find a way to build dub projects in a way 
 that's usable outside building with dub (e.g. `dub 
 --produce-makefile` or whatnot), that's fine by me. Just don't 
 make me change how I build my projects.


Ok, but the question is what the official strategy ought to be. 
Should official strategy be to roll-your-own (Dub etc), or should 
it be to build on what others have created.

Where do you get more bangs for the bucks? By pouring resources 
into your own corner of the universe or by enhancing an existing 
system to have solid D support (e.g. Conan, Cmake etc).

There is a marketing issue too. For newbies, Dub and "idiomatic" 
packages might be the best option. If you want to win over 
experienced developers, maybe Cmake/Conan or some other 
comparable setup focusing on brining in existing frameworks would 
be more convincing.

What is the target audience? People who want to stay in their own 
corner of the universe and build something unique and easy to 
use, or people who want to engage with the larger software field?
Apr 26 2022
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 26 April 2022 at 19:29:31 UTC, Steven Schveighoffer 
wrote:

 The solution is not to bring everyone down to your misery 
 though. "If I can't have a good experience with what I want, 
 nobody should" isn't the right answer.

 -Steve


I'm not bringing anyone anywhere. You want to use make? Good. 
cmake/ninja? Good. Scons? Good. A custom build script? Good.

I really don't what you use.

I *DO* care about dub because dub doesn't provide me with a good 
way to work with it without dubifying everything.

See I lead a project that uses cmake/ninja. It has dependencies 
using autotools, cmake, qmake, and even some that uses their own 
thing. None of that causes any significant problems.

But dub don't just want to build the thing or do dependency 
management. Dub is the true way to enlightenment. You have to do 
it all the dub way or be in a world of pain.

All I'm asking is that you sub guy recognize for other the same 
thing as you are now requesting for yourself: don't impose a way 
of doing things on me because it is the one true wayâ„¢.
Apr 26 2022
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Apr 26, 2022 at 09:38:31PM +0000, deadalnix via Digitalmars-d wrote:
[...]

 I *DO* care about dub because dub doesn't provide me with a good way
 to work with it without dubifying everything.
 
 See I lead a project that uses cmake/ninja. It has dependencies using
 autotools, cmake, qmake, and even some that uses their own thing. None
 of that causes any significant problems.
 
 But dub don't just want to build the thing or do dependency
 management. Dub is the true way to enlightenment. You have to do it
 all the dub way or be in a world of pain.


[...]

What dub needs is a way to integrate with other build systems that
doesn't require "dubifying" everything turtles all the way down.

For the first time today, I decided to read the dub package format docs
all the way from top to bottom without skipping anything.  I've thought
of a few ideas of how the configuration file may be extended in a
non-intrusive way to allow for better integration with other build
tools.  I'm going to mull over it some more and then post some of my
ideas.

We're gonna bust open this walled garden into something that plays nicer
with its neighbours.


T

-- 
"The whole problem with the world is that fools and fanatics are always so
certain of themselves, but wiser people so full of doubts." -- Bertrand
Russell. "How come he didn't put 'I think' at the end of it?" -- Anonymous
Apr 26 2022
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/26/22 5:38 PM, deadalnix wrote:

 On Tuesday, 26 April 2022 at 19:29:31 UTC, Steven Schveighoffer wrote:

 The solution is not to bring everyone down to your misery though. "If 
 I can't have a good experience with what I want, nobody should" isn't 
 the right answer.


 
 I'm not bringing anyone anywhere. You want to use make? Good. 
 cmake/ninja? Good. Scons? Good. A custom build script? Good.
 
 I really don't what you use.
 
 I *DO* care about dub because dub doesn't provide me with a good way to 
 work with it without dubifying everything.


I'm perfectly fine with projects not using dub. Don't listen to people 
who cry about dub usage.


 See I lead a project that uses cmake/ninja. It has dependencies using 
 autotools, cmake, qmake, and even some that uses their own thing. None 
 of that causes any significant problems.


And I lead projects that use dub. People have asked me to include meson 
build files in some cases. And I've obliged, but I hate maintaining it 
(most important reason -- meson build file must be manually updated to 
match git tag). I'd rather just get rid of them, and if you don't want 
to use dub to build my projects, then roll your own fork.


 
 But dub don't just want to build the thing or do dependency management. 
 Dub is the true way to enlightenment. You have to do it all the dub way 
 or be in a world of pain.


Don't use dub, no enlightenment necessary.


 
 All I'm asking is that you sub guy recognize for other the same thing as 
 you are now requesting for yourself: don't impose a way of doing things 
 on me because it is the one true wayâ„¢.


I'm not. I said this from my first post on this unreasonably long 
thread, I don't care what you use, D does not require you to use dub to 
build.

-Steve
Apr 26 2022
parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 26 April 2022 at 22:08:27 UTC, Steven Schveighoffer 
wrote:

 I'm perfectly fine with projects not using dub. Don't listen to 
 people who cry about dub usage.


You are missing the important part. Because dub do not play nice 
with anything else, that means we might as well use a different 
language. My D code can never talk to your D code without an 
absurd amount of time, energy and effort expanded.

You apparently had a small taste of that with meson. I ended 
discarding meson as an option for reason similar to dub - and 
reasons you are experiencing now: meson doesn't play nice with 
other kids.

Except meson is kinda asocial, while dub is more like the 
Westboro Baptist Church.
Apr 26 2022
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/26/22 6:50 PM, deadalnix wrote:

 On Tuesday, 26 April 2022 at 22:08:27 UTC, Steven Schveighoffer wrote:

 I'm perfectly fine with projects not using dub. Don't listen to people 
 who cry about dub usage.


 
 You are missing the important part. Because dub do not play nice with 
 anything else, that means we might as well use a different language. My 
 D code can never talk to your D code without an absurd amount of time, 
 energy and effort expanded.


Define "absurd". You want to include `version` flags that might be in my 
dub file? They are in there, not hard to read. You want to include the 
library? Build with `dub build` in my repo, and then you have the lib 
available. All my files are in my `source` directory. Is that hard to use?

Dub does not use extraneous build pieces. All it does is construct a 
valid compilation line. You can mimic that line without (IMO) an absurd 
amount of effort.

With dmd -i, you can pretty much just ignore dub and just use the proper 
`-I` directories, I can't imagine this will be extremely difficult.

None of this is to say that it's not work, or that it's seamless. But 
it's not any less seamless than if dub didn't exist.


 You apparently had a small taste of that with meson. I ended discarding 
 meson as an option for reason similar to dub - and reasons you are 
 experiencing now: meson doesn't play nice with other kids.


Nah, my experience was never with *using* meson, my experience was with 
CI and meson. In other words, I'd update my library, tag it, and it's 
ready for dub. But meson *also* needs me to duplicate the tag in its 
build file, or it doesn't work. That is junk. Imagine this workflow:

1. create your update
2. Use CI on your PR, it works
3. Merge to master
4. Tag for release
5. Master CI now breaks because meson wasn't satisfied.

i.e. you *can't run CI until you've tagged the release*.

I asked, can you make Meson automatically use the tagged version? Answer 
is no. So I really am regretting merging that file, and probably will 
just take it out of CI unless someone cares.


 Except meson is kinda asocial, while dub is more like the Westboro 
 Baptist Church.


This is an insanely overexxagerated analogy.

-Steve
Apr 26 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 27 April 2022 at 00:51:08 UTC, Steven Schveighoffer 
wrote:

 Except meson is kinda asocial, while dub is more like the 
 Westboro Baptist Church.


 This is an insanely overexxagerated analogy.


What it boils down to is that it is beneficial for the eco system 
that all D users have an equal opportunity to share and market 
their projects through the official channels.

Maybe Dub should be a brick in a larger system, rather than the 
opposite. That way people with simple needs can stick to Dub and 
people who draw on non-D resources can get equal opportunities.

Some sort of extensible agnostic meta-packaging system that can 
draw in resources from Conan, Dub etc.
Apr 27 2022
prev sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Tuesday, 26 April 2022 at 21:38:31 UTC, deadalnix wrote:

 See I lead a project that uses cmake/ninja. It has dependencies 
 using autotools, cmake, qmake, and even some that uses their 
 own thing. None of that causes any significant problems.

 But dub don't just want to build the thing or do dependency 
 management. Dub is the true way to enlightenment. You have to 
 do it all the dub way or be in a world of pain.

 All I'm asking is that you sub guy recognize for other the same 
 thing as you are now requesting for yourself: don't impose a 
 way of doing things on me because it is the one true wayâ„¢.


We use a non-dub build flow. But integrating dub packages work 
fine. We just grab dub.selections.json, and run `dub build` for 
every dependency, then grab the library path from `dub describe` 
and add it into our DMD build command. I don't understand why 
people say dub is all-or-nothing.
Apr 28 2022
next sibling parent Tejas <notrealemail gmail.com> writes:
On Thursday, 28 April 2022 at 08:21:41 UTC, FeepingCreature wrote:

 On Tuesday, 26 April 2022 at 21:38:31 UTC, deadalnix wrote:

 See I lead a project that uses cmake/ninja. It has 
 dependencies using autotools, cmake, qmake, and even some that 
 uses their own thing. None of that causes any significant 
 problems.

 But dub don't just want to build the thing or do dependency 
 management. Dub is the true way to enlightenment. You have to 
 do it all the dub way or be in a world of pain.

 All I'm asking is that you sub guy recognize for other the 
 same thing as you are now requesting for yourself: don't 
 impose a way of doing things on me because it is the one true 
 wayâ„¢.


 We use a non-dub build flow. But integrating dub packages work 
 fine. We just grab dub.selections.json, and run `dub build` for 
 every dependency, then grab the library path from `dub 
 describe` and add it into our DMD build command. I don't 
 understand why people say dub is all-or-nothing.


I think the problem is that he's **providing** his stuff to the 
dub ecosystem, not integrating other packages from it, forcing 
him to use dub's versioning scheme and build system since dub 
insists on being the package manager and build system both for 
any library that wants to get published on code.dlang.org

I may be wrong though
Apr 28 2022
prev sibling parent reply Tobias Pankrath <tobias pankrath.net> writes:
On Thursday, 28 April 2022 at 08:21:41 UTC, FeepingCreature wrote:

 On Tuesday, 26 April 2022 at 21:38:31 UTC, deadalnix wrote:

 We use a non-dub build flow. But integrating dub packages work 
 fine. We just grab dub.selections.json, and run `dub build` for 
 every dependency, then grab the library path from `dub 
 describe` and add it into our DMD build command. I don't 
 understand why people say dub is all-or-nothing.


Do you do this for every build or per compiler? `dub describe` 
doesn't work that well if you use multiple compilers.
Apr 29 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 30/04/2022 12:52 AM, Tobias Pankrath wrote:

 Do you do this for every build or per compiler? `dub describe` doesn't 
 work that well if you use multiple compilers.


How can ``dub describe`` be improved for multiple compilers?
Apr 29 2022
parent reply Tobias Pankrath <tobias pankrath.net> writes:
On Friday, 29 April 2022 at 16:33:57 UTC, rikki cattermole wrote:

 On 30/04/2022 12:52 AM, Tobias Pankrath wrote:

 Do you do this for every build or per compiler? `dub describe` 
 doesn't work that well if you use multiple compilers.


 How can ``dub describe`` be improved for multiple compilers?


`dub describe` specifically should provide paths to the actual 
artifacts
instead of the symlinked locations, which are the same for all
build configurations.

While integrating it with nix, I have wished for:

1. Better control over where dub places and searches for stuff, 
although `HOME=. dub ..` works.

2. A way to just download a projects and all it's dependencies 
and place it somewhere.

3. That dub describe points to the actual build artifacts instead 
of symlinked locations
used by differnet locations.

If you look at the dub integration code of meson, 3/4 of it is 
finding stuff that dub is supposed to have already built after it 
parsed the output of `dub describe`.
Apr 29 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 30/04/2022 4:50 AM, Tobias Pankrath wrote:

 On Friday, 29 April 2022 at 16:33:57 UTC, rikki cattermole wrote:

 On 30/04/2022 12:52 AM, Tobias Pankrath wrote:

 Do you do this for every build or per compiler? `dub describe` 
 doesn't work that well if you use multiple compilers.


 How can ``dub describe`` be improved for multiple compilers?


 
 `dub describe` specifically should provide paths to the actual artifacts
 instead of the symlinked locations, which are the same for all
 build configurations.
 
 While integrating it with nix, I have wished for:
 
 1. Better control over where dub places and searches for stuff, although 
 `HOME=. dub ..` works.
 
 2. A way to just download a projects and all it's dependencies and place 
 it somewhere.


$ dub fetch vibelog --cache=local
$ dub build vibelog --cache=local

That'll do it, but you shouldn't need to build to do all the dependencies...

https://github.com/dlang/dub/issues/1930

Okay so:

$ dub describe vibelog --cache=local

Will also do the job.


 3. That dub describe points to the actual build artifacts instead of 
 symlinked locations
 used by differnet locations.


targetPath	string	The destination path of the output binary - this 
setting does not support platform suffixes

That needs to support platform suffixes for what you are wanting.
Describe would just follow suit.

On Windows the artifacts are copied not symlinked. So the paths are 
correct (even if not desirable).
Apr 29 2022
parent reply Tobias Pankrath <tobias pankrath.net> writes:
On Friday, 29 April 2022 at 17:02:55 UTC, rikki cattermole wrote:

 On 30/04/2022 4:50 AM, Tobias Pankrath wrote:

 On Friday, 29 April 2022 at 16:33:57 UTC, rikki cattermole 
 wrote:

 On 30/04/2022 12:52 AM, Tobias Pankrath wrote:

 Do you do this for every build or per compiler? `dub 
 describe` doesn't work that well if you use multiple 
 compilers.


 How can ``dub describe`` be improved for multiple compilers?


 
 `dub describe` specifically should provide paths to the actual 
 artifacts
 instead of the symlinked locations, which are the same for all
 build configurations.
 
 While integrating it with nix, I have wished for:
 
 1. Better control over where dub places and searches for 
 stuff, although `HOME=. dub ..` works.
 
 2. A way to just download a projects and all it's dependencies 
 and place it somewhere.


 $ dub fetch vibelog --cache=local
 $ dub build vibelog --cache=local


That doesn't really work, you need `HOME=.`.


 That'll do it, but you shouldn't need to build to do all the 
 dependencies...

 https://github.com/dlang/dub/issues/1930


I've started a PR for that some years ago, but gave up 
https://github.com/dlang/dub/pull/1898


 Okay so:

 $ dub describe vibelog --cache=local

 Will also do the job.


 3. That dub describe points to the actual build artifacts 
 instead of symlinked locations
 used by differnet locations.


 targetPath	string	The destination path of the output binary - 
 this setting does not support platform suffixes

 That needs to support platform suffixes for what you are 
 wanting.
 Describe would just follow suit.

 On Windows the artifacts are copied not symlinked. So the paths 
 are correct (even if not desirable).


That doesn't change the issue that `dub describe --compiler=ldc` 
says that a file it at <path/to/file>, but then you do `dub build 
--compiler=dmd` and suddenly <path/to/file> is something 
different.
Apr 29 2022
parent FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 29 April 2022 at 17:48:44 UTC, Tobias Pankrath wrote:

 That doesn't change the issue that `dub describe 
 --compiler=ldc` says that a file it at <path/to/file>, but then 
 you do `dub build --compiler=dmd` and suddenly <path/to/file> 
 is something different.


I wish I could say I had a good answer here. But we just make a 
local copy of the result file.
May 04 2022
prev sibling next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Apr 26, 2022 at 01:21:58AM +0000, Mike Parker via Digitalmars-d wrote:
[...]

 dub exists because it filled a huge, gaping hole in the D ecosystem.
 
 Go back far enough and you'll find "build", later renamed "bud", by
 Derek Parnell. This was the first D build tool the community rallied
 around.


[...]

 Gregor Richards wasn't happy with bud, so he created "Rebuild" as an
 alternative build tool, but added a package manager on top of it
 called "DSSS" (D Shared Software System).


[...]

 At one point I had three platform-specific make files, a `dsss.conf`,
 and a `forbud.txt` (I eventually replaced some of that with a
 `buildme.d file`).
 
 There was no standard way of building D projects, and it sucked. Then
 Derek and Gregor disappeared, and suddenly these build systems
 everyone is using are no longer supported. No one takes them over, so
 we're in the Dark Ages again. Then dub comes along and saves the day.
 
 It caught on quickly. Then people start thinking: "What happens if
 Sönke disappears? This should be an official D thing." So dub gets
 added to the installation. And here we are.


Thanks for the summary, that was very insightful.


[...]

 The question is, what exactly is dub missing and how can it be
 improved?  Which perceived shortcomings are reasonable to address and
 which are not?  And assuming we can answer those questions, who is
 going to address them?


The primary functionality missing from dub that I rely on is explicit
build dependency graph specification. (Note: I'm not talking about
*package* dependencies, which dub actually does an OK job at, but
*build* dependencies.)

What I mean is this: my projects often involve a main executable, which
is the primary target of the project, plus several helpers, which are
either secondary targets sharing most of the same sources, or code
generators that create one or more targets required to compile the main
executable.  Occasionally, there may also be auxiliary targets like HTML
pages, procedurally-generated images, and other resources.

Currently, for building all these targets, I use SCons (yes, yes, I
know). It's not perfect, but it does what I want: it has no problem
handling complex build jobs like building a subset of source files into
helper utility U1, running U1 in order to generate some extra D source
files, then compiling the results into an executable.  It can also, *at
the same time*, compile a different subset of source files into helper
utility U2, run U2 on a set of HTML templates and generating HTML files,
then running a PHP postprocessing script on said HTML files, then
install these files into a staging directory hierarchy, and then
creating a tarball to be uploaded to the target webserver.

As far as I know -- and if I'm wrong I'd be happy to be corrected -- dub
is unable to handle the above (at least not natively -- I'd have to
write my own code for building the non-D parts of the build AFAIK, which
defeats the purpose of having a build system in the first place).

This is my primary complaint about dub.

There are also secondary issues, like:

- Network dependence (I'd *really* like for it *not* to depend on
  internet access being available by default, only when I ask it to).
  IIRC there's some switch or option that does this, it would be nice if
  there was a local setting I could toggle to make this automatic.

- Performance: is there an option to skip the expensive NP-complete
  dependency resolution step at the beginning for faster turnaround
  time? When I'm debugging something I do *not* want dub to do anything
  except recompile local source, no network access, no package
  dependency resolution, nothing, just *build* the darned thing and
  leave it at that.

- Reproducibility: if I change one source file out of a directory of 50,
  I want the build system to be able to detect that one change,
  determine the *minimum* sequence of actions to update current targets,
  and run only those actions. After running these actions, the targets
  should be in EXACTLY the same state as if I had rebuilt the entire
  workspace from a clean checkout. And this should NOT be dependent on
  the current state of the workspace (it should know to overwrite stale
  intermediates, etc., so that the final targets are in the correct
  state).

- Selective builds: sometimes during development I only want to rebuild
  one target out of possibly 5 or 10. There has to be an easy way only
  build a subset of targets (and it should perform the *minimum* actions
  needed to achieve this, in a completely reproducible way, as described
  above).

- Parallel building: if the actions required to build targets T1 and T2
  are not dependent on each other, it should know to run them in
  parallel so that the build is completed as quickly as possible. This
  should be baked-in behaviour that does not require any additional
  effort from the user.


If dub can be made to do all of the above, I'd be happy to adopt it for
my projects.


T

-- 
I'm still trying to find a pun for "punishment"...
Apr 26 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 3:35 AM, H. S. Teoh wrote:

 The question is, what exactly is dub missing and how can it be
 improved?  Which perceived shortcomings are reasonable to address and
 which are not?  And assuming we can answer those questions, who is
 going to address them?


 
 The primary functionality missing from dub that I rely on is explicit
 build dependency graph specification. (Note: I'm not talking about
 *package* dependencies, which dub actually does an OK job at, but
 *build* dependencies.)
 
 What I mean is this: my projects often involve a main executable, which
 is the primary target of the project, plus several helpers, which are
 either secondary targets sharing most of the same sources, or code
 generators that create one or more targets required to compile the main
 executable.  Occasionally, there may also be auxiliary targets like HTML
 pages, procedurally-generated images, and other resources.
 
 Currently, for building all these targets, I use SCons (yes, yes, I
 know). It's not perfect, but it does what I want: it has no problem
 handling complex build jobs like building a subset of source files into
 helper utility U1, running U1 in order to generate some extra D source
 files, then compiling the results into an executable.  It can also, *at
 the same time*, compile a different subset of source files into helper
 utility U2, run U2 on a set of HTML templates and generating HTML files,
 then running a PHP postprocessing script on said HTML files, then
 install these files into a staging directory hierarchy, and then
 creating a tarball to be uploaded to the target webserver.
 
 As far as I know -- and if I'm wrong I'd be happy to be corrected -- dub
 is unable to handle the above (at least not natively -- I'd have to
 write my own code for building the non-D parts of the build AFAIK, which
 defeats the purpose of having a build system in the first place).


Pre build commands.

For D stuff in dub something like this works fine.

"preBuildCommands": ["dub run package:tool -- args"]

But what you are describing is something automatic, which is not 
currently supported.


 This is my primary complaint about dub.
 
 There are also secondary issues, like:
 
 - Network dependence (I'd *really* like for it *not* to depend on
    internet access being available by default, only when I ask it to).
    IIRC there's some switch or option that does this, it would be nice if
    there was a local setting I could toggle to make this automatic.


https://dub.pm/settings

So yeah settings file already supports this.


 - Performance: is there an option to skip the expensive NP-complete
    dependency resolution step at the beginning for faster turnaround
    time? When I'm debugging something I do *not* want dub to do anything
    except recompile local source, no network access, no package
    dependency resolution, nothing, just *build* the darned thing and
    leave it at that.


I've had a look at this, it would take a good bit of refactoring to 
split this out into dub.selections.json *I think*.

But yeah you're right, if nothing has changed it should be cached.


 - Reproducibility: if I change one source file out of a directory of 50,
    I want the build system to be able to detect that one change,
    determine the *minimum* sequence of actions to update current targets,
    and run only those actions. After running these actions, the targets
    should be in EXACTLY the same state as if I had rebuilt the entire
    workspace from a clean checkout. And this should NOT be dependent on
    the current state of the workspace (it should know to overwrite stale
    intermediates, etc., so that the final targets are in the correct
    state).


I was questioning if the problem here is the compiler stuff, but its not.

However, I don't think that this should be the default. Processing all 
of those dates, caching them... yeah won't be cheap either.
Apr 26 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 03:57:24AM +1200, rikki cattermole via Digitalmars-d
wrote:

 On 27/04/2022 3:35 AM, H. S. Teoh wrote:


[...]

 What I mean is this: my projects often involve a main executable,
 which is the primary target of the project, plus several helpers,
 which are either secondary targets sharing most of the same sources,
 or code generators that create one or more targets required to
 compile the main executable.  Occasionally, there may also be
 auxiliary targets like HTML pages, procedurally-generated images,
 and other resources.




[...]

 As far as I know -- and if I'm wrong I'd be happy to be corrected --
 dub is unable to handle the above (at least not natively -- I'd have
 to write my own code for building the non-D parts of the build
 AFAIK, which defeats the purpose of having a build system in the
 first place).


 
 Pre build commands.
 
 For D stuff in dub something like this works fine.
 
 "preBuildCommands": ["dub run package:tool -- args"]


Does this mean I have to create an entire subpackage just for this
purpose?  Or in fact, one subpackage per auxiliary target?  If so, that
would seem needlessly cumbersome for something that, in my mind, is a
trivial additional node in the build graph.

Also, treating these auxiliary build targets as second-class citizens
doesn't really sit right with me. I mean, after all, it all boils down
to "build sources S1, S2, ... into targets T1, T2, ... by running
command(s) C1, C2, ...".  What if I decide to insert a postprocessing
step in the middle of one of these build chains?  E.g., after creating a
HTML file, before installing it to the staging area, I decide that I
want to run a HTML tidying utility on it?  Does that mean I have to
create another subpackage to represent this extra step?



 But what you are describing is something automatic, which is not
 currently supported.


What do you mean by "automatic"?  These targets are generally not
automatically inferrable, i.e., I'm not expecting that if I say "build
xyz.html" dub would magically know that in order to build HTML files it
needs to compile a.d, b.d, c.d into abc.exe and run abc.exe on
xyz.template in order to produce xyz.html.  Obviously these build steps
must be explicitly stated somewhere.

But I do expect that build products generated by these steps would be
smoothly integrated into the build, i.e., if "code.template" is
preprocessed by some tool "helper.exe" to produce "code.d", then there
should be a way to compile "code.d" into the main executable as well.


[...]

 - Network dependence (I'd *really* like for it *not* to depend on
   internet access being available by default, only when I ask it
   to).  IIRC there's some switch or option that does this, it would
   be nice if there was a local setting I could toggle to make this
   automatic.


 
 https://dub.pm/settings
 
 So yeah settings file already supports this.


Which setting disables network lookup by default?  Glancing at that
page, it's not obvious which setting it is and what value I should set
it to.



 - Performance: is there an option to skip the expensive NP-complete
   dependency resolution step at the beginning for faster turnaround
   time? When I'm debugging something I do *not* want dub to do
   anything except recompile local source, no network access, no
   package dependency resolution, nothing, just *build* the darned
   thing and leave it at that.


 
 I've had a look at this, it would take a good bit of refactoring to
 split this out into dub.selections.json *I think*.
 
 But yeah you're right, if nothing has changed it should be cached.


Not just that, when I'm recompiling a project during debugging, I don't
want dub to look at the network *at all*.  I don't care if upstream has
released a critical zero-day exploit fix, I do NOT want the code to
suddenly change from under me when I'm trying to trace down a segfault.
I want it to just build the sources that are currently on the local
machine, and that's it.

Also, sometimes if I'm on the road without internet access, I do not
want to suddenly become unable to build my project.



 - Reproducibility: if I change one source file out of a directory of
   50, I want the build system to be able to detect that one change,
   determine the *minimum* sequence of actions to update current
   targets, and run only those actions. After running these actions,
   the targets should be in EXACTLY the same state as if I had
   rebuilt the entire workspace from a clean checkout. And this
   should NOT be dependent on the current state of the workspace (it
   should know to overwrite stale intermediates, etc., so that the
   final targets are in the correct state).


 
 I was questioning if the problem here is the compiler stuff, but its
 not.
 
 However, I don't think that this should be the default. Processing all
 of those dates, caching them... yeah won't be cheap either.


Two comments here:

1) Dates should NOT be used as the basis for detecting changes, because
   it's not reliable. Preferably some kind of checksum should be used (a
   cheap one like md5 or CRC would do -- we don't need strong crypto
   strength here). Why?  Because sometimes, an updated timestamp does
   *not* mean the file actually changed.

   For example, if I `git checkout` a branch to look at something and
   switch back later, the file may have been touched during the switch,
   but afterwards its contents are identical to when it was last built.
   In this case, targets that depend on that file do not need to be
   rebuilt; it can be skipped entirely. This can sometimes lead to
   better performance, e.g., if a commonly-imported module is touched in
   this way, but you realize it hasn't actually changed, you can prune
   away large parts of the build graph.

2) The performance issue has already been solved, see for example:

	https://gittup.org/tup/

   The underlying idea is: *don't* scan the entire source tree to detect
   changes, use modern OS facilities (inotify, FileSystemWatcher, etc.)
   to let the OS tell you when something changes.  This allows the build
   time to be O(n), where n is the size of the change, rather than O(N)
   where N is the size of the workspace.  This is important for
   scalability to large projects where N is usually significantly larger
   than n.


T

-- 
An elephant: A mouse built to government specifications. -- Robert Heinlein
Apr 26 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 4:41 AM, H. S. Teoh wrote:

 On Wed, Apr 27, 2022 at 03:57:24AM +1200, rikki cattermole via Digitalmars-d
wrote:

 On 27/04/2022 3:35 AM, H. S. Teoh wrote:


 [...]

 What I mean is this: my projects often involve a main executable,
 which is the primary target of the project, plus several helpers,
 which are either secondary targets sharing most of the same sources,
 or code generators that create one or more targets required to
 compile the main executable.  Occasionally, there may also be
 auxiliary targets like HTML pages, procedurally-generated images,
 and other resources.




 [...]

 As far as I know -- and if I'm wrong I'd be happy to be corrected --
 dub is unable to handle the above (at least not natively -- I'd have
 to write my own code for building the non-D parts of the build
 AFAIK, which defeats the purpose of having a build system in the
 first place).


 Pre build commands.

 For D stuff in dub something like this works fine.

 "preBuildCommands": ["dub run package:tool -- args"]


 
 Does this mean I have to create an entire subpackage just for this
 purpose?  Or in fact, one subpackage per auxiliary target?  If so, that
 would seem needlessly cumbersome for something that, in my mind, is a
 trivial additional node in the build graph.
 
 Also, treating these auxiliary build targets as second-class citizens
 doesn't really sit right with me. I mean, after all, it all boils down
 to "build sources S1, S2, ... into targets T1, T2, ... by running
 command(s) C1, C2, ...".  What if I decide to insert a postprocessing
 step in the middle of one of these build chains?  E.g., after creating a
 HTML file, before installing it to the staging area, I decide that I
 want to run a HTML tidying utility on it?  Does that mean I have to
 create another subpackage to represent this extra step?


No and no.

My example used dub as that is known to exist on the system, but you can 
use whatever you want.

Dub supports single file packages, so you the package in question could 
be only one D file, unrelated to the package you are building.

There are post build commands too. Its not just pre build.


 But what you are describing is something automatic, which is not
 currently supported.


 
 What do you mean by "automatic"?  These targets are generally not
 automatically inferrable, i.e., I'm not expecting that if I say "build
 xyz.html" dub would magically know that in order to build HTML files it
 needs to compile a.d, b.d, c.d into abc.exe and run abc.exe on
 xyz.template in order to produce xyz.html.  Obviously these build steps
 must be explicitly stated somewhere.
 
 But I do expect that build products generated by these steps would be
 smoothly integrated into the build, i.e., if "code.template" is
 preprocessed by some tool "helper.exe" to produce "code.d", then there
 should be a way to compile "code.d" into the main executable as well.


What I mean by automatic, is some sort of directive to tell dub to run a 
package that you depend upon that is executable.


 [...]

 - Network dependence (I'd *really* like for it *not* to depend on
    internet access being available by default, only when I ask it
    to).  IIRC there's some switch or option that does this, it would
    be nice if there was a local setting I could toggle to make this
    automatic.


 https://dub.pm/settings

 So yeah settings file already supports this.


 
 Which setting disables network lookup by default?  Glancing at that
 page, it's not obvious which setting it is and what value I should set
 it to.


Set skipRegistry to configured or standard.

 From there you will have to enable registries when you need them via 
cli. ``dub --skip-registry=none``

Upgrade only might be a good addition however.
Apr 26 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 04:55:39AM +1200, rikki cattermole via Digitalmars-d
wrote:

 On 27/04/2022 4:41 AM, H. S. Teoh wrote:

 On Wed, Apr 27, 2022 at 03:57:24AM +1200, rikki cattermole via Digitalmars-d
wrote:




[...]

 Pre build commands.
 
 For D stuff in dub something like this works fine.
 
 "preBuildCommands": ["dub run package:tool -- args"]


 
 Does this mean I have to create an entire subpackage just for this
 purpose?  Or in fact, one subpackage per auxiliary target?  If so,
 that would seem needlessly cumbersome for something that, in my
 mind, is a trivial additional node in the build graph.
 
 Also, treating these auxiliary build targets as second-class
 citizens doesn't really sit right with me. I mean, after all, it all
 boils down to "build sources S1, S2, ... into targets T1, T2, ... by
 running command(s) C1, C2, ...".  What if I decide to insert a
 postprocessing step in the middle of one of these build chains?
 E.g., after creating a HTML file, before installing it to the
 staging area, I decide that I want to run a HTML tidying utility on
 it?  Does that mean I have to create another subpackage to represent
 this extra step?


 
 No and no.
 
 My example used dub as that is known to exist on the system, but you
 can use whatever you want.


But doesn't that mean it will lose out on the benefits of having a build
system in the first place?  E.g., if the source file for the helper
utility wasn't changed, there's no need to run it again. But dub
wouldn't know better, since I specified preBuildCommands it would always
run it, correct?  So it would rebuild the helper utility every single
time?



 Dub supports single file packages, so you the package in question
 could be only one D file, unrelated to the package you are building.


That's problematic.  Quite often the auxiliary utilities in my projects
share a good amount of code with the main executable (which is why I
don't put them in a separate workspace to begin with).  Can dub handle
building multiple executables from different subsets of source files in
the same source tree?



 There are post build commands too. Its not just pre build.


Are post build commands unconditionally run too?


[...]

 What I mean by automatic, is some sort of directive to tell dub to run
 a package that you depend upon that is executable.


I see.

But why should I need to create a separate package anyway?  The entire
reason I build my helper programs from the same source tree is because
they logically belong together and complement each other.  Why can't I
have multiple targets per package, and why can't I use one target to
build another within the same package?

It feels just like what deadalnix said: the cases dub handles can be
handled with `dmd -i`, but the cases that can't be handled by `dmd -i`
can't be handled by dub either.  It unfortunately only reinforces my
perception that dub the package manager is good, but dub the build
system sucks.  It makes me want to use dub only for package
dependencies, but a different build system.


[...]

 Which setting disables network lookup by default?  Glancing at that
 page, it's not obvious which setting it is and what value I should
 set it to.


 
 Set skipRegistry to configured or standard.
 
 From there you will have to enable registries when you need them via
 cli.  ``dub --skip-registry=none``


Thanks, this is helpful.



 Upgrade only might be a good addition however.


Doesn't dub already do this? Or am I not understanding this right?


T

-- 
What's a "hot crossed bun"? An angry rabbit.
Apr 26 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 5:19 AM, H. S. Teoh wrote:

 On Wed, Apr 27, 2022 at 04:55:39AM +1200, rikki cattermole via Digitalmars-d
wrote:

 On 27/04/2022 4:41 AM, H. S. Teoh wrote:

 On Wed, Apr 27, 2022 at 03:57:24AM +1200, rikki cattermole via Digitalmars-d
wrote:




 [...]

 Pre build commands.

 For D stuff in dub something like this works fine.

 "preBuildCommands": ["dub run package:tool -- args"]


 Does this mean I have to create an entire subpackage just for this
 purpose?  Or in fact, one subpackage per auxiliary target?  If so,
 that would seem needlessly cumbersome for something that, in my
 mind, is a trivial additional node in the build graph.

 Also, treating these auxiliary build targets as second-class
 citizens doesn't really sit right with me. I mean, after all, it all
 boils down to "build sources S1, S2, ... into targets T1, T2, ... by
 running command(s) C1, C2, ...".  What if I decide to insert a
 postprocessing step in the middle of one of these build chains?
 E.g., after creating a HTML file, before installing it to the
 staging area, I decide that I want to run a HTML tidying utility on
 it?  Does that mean I have to create another subpackage to represent
 this extra step?


 No and no.

 My example used dub as that is known to exist on the system, but you
 can use whatever you want.


 
 But doesn't that mean it will lose out on the benefits of having a build
 system in the first place?  E.g., if the source file for the helper
 utility wasn't changed, there's no need to run it again. But dub
 wouldn't know better, since I specified preBuildCommands it would always
 run it, correct?  So it would rebuild the helper utility every single
 time?


Dub is able to cache things.

In context I don't know how much can be done prevent work, since your 
own tools would need to do caching as well.


 Dub supports single file packages, so you the package in question
 could be only one D file, unrelated to the package you are building.


 
 That's problematic.  Quite often the auxiliary utilities in my projects
 share a good amount of code with the main executable (which is why I
 don't put them in a separate workspace to begin with).  Can dub handle
 building multiple executables from different subsets of source files in
 the same source tree?


Can be.

Single file packages just like any other package can depend upon another 
by path, rather than version lookup.

I do this plenty, it works fine. Quite often I swap out to a full blown 
package, since single file packages can't have multiple files in it 
(hence the name).


 There are post build commands too. Its not just pre build.


 
 Are post build commands unconditionally run too?


Probably.


 [...]

 What I mean by automatic, is some sort of directive to tell dub to run
 a package that you depend upon that is executable.


 
 I see.
 
 But why should I need to create a separate package anyway?  The entire
 reason I build my helper programs from the same source tree is because
 they logically belong together and complement each other.  Why can't I
 have multiple targets per package, and why can't I use one target to
 build another within the same package?


Yes this is a pretty big sticking point.

By in large this is a genuine issue that I believe needs resolving.

However in 10 years I haven't needed it, I only just recently got around 
to improving shared library support so it actually worked as I needed 
that ;)


 Which setting disables network lookup by default?  Glancing at that
 page, it's not obvious which setting it is and what value I should
 set it to.


 Set skipRegistry to configured or standard.

  From there you will have to enable registries when you need them via
 cli.  ``dub --skip-registry=none``


 
 Thanks, this is helpful.
 
 

 Upgrade only might be a good addition however.


 
 Doesn't dub already do this? Or am I not understanding this right?


I don't know. I don't use this feature.

https://github.com/dlang/dub/issues/2244
Apr 26 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 05:27:01AM +1200, rikki cattermole via Digitalmars-d
wrote:

 
 On 27/04/2022 5:19 AM, H. S. Teoh wrote:


[...]

 But doesn't that mean it will lose out on the benefits of having a
 build system in the first place?  E.g., if the source file for the
 helper utility wasn't changed, there's no need to run it again. But
 dub wouldn't know better, since I specified preBuildCommands it
 would always run it, correct?  So it would rebuild the helper
 utility every single time?


 
 Dub is able to cache things.
 
 In context I don't know how much can be done prevent work, since your
 own tools would need to do caching as well.


Right, which brings me back to square one. Dub already knows how to
cache stuff, but I can't leverage any of it because it was arbitrarily
decided that dub would only work with D source files, and therefore it
only caches D artifacts.  If I want to build HTML files from templates,
for example, I couldn't leverage dub's caching ability; I have to roll
my own and reinvent the square wheel.  If I want to build PNG files from
data files, I'd also have to roll my own cache.  Eventually, I'd want to
avoid reinventing my own cache for every artifact I want to build, so
I'd look for a system that offers me that capability.  Such a system
would obviously also be able to build D programs and cache them as
artifacts, since building D programs would be a specific case of the
more general task of building a target from sources and caching it.
Which in turn means that dub has just become redundant: I might as well
just use that other system for *all* my build needs, including building
D programs, and it would let me do everything I need without needing to
work with two different, overlapping systems (dub + other build system).

That's my current line of reasoning, and why I haven't seriously
invested in dub.


[...]

 Single file packages just like any other package can depend upon
 another by path, rather than version lookup.
 
 I do this plenty, it works fine. Quite often I swap out to a full
 blown package, since single file packages can't have multiple files in
 it (hence the name).


[...]

 But why should I need to create a separate package anyway?  The
 entire reason I build my helper programs from the same source tree
 is because they logically belong together and complement each other.
 Why can't I have multiple targets per package, and why can't I use
 one target to build another within the same package?


 
 Yes this is a pretty big sticking point.
 
 By in large this is a genuine issue that I believe needs resolving.


Whether dub ends up implementing this or not, there is a larger picture
I feel is being missed here.

Even if dub eventually allows building multiple executables from
different subsets of the same source tree, that's still limited to only
D artifacts.  It still doesn't allow the user to leverage its dependency
resolution, building, and caching capabilities for non-D artifacts.  If
I were to adopt dub at that point, I'd still have to roll my own
dependency management, building, and caching for non-D artifacts.  Which
would drive me to use a different build system that *can* offer me those
capabilities. Which in turn makes dub a redundant factor in the
equation.

Strictly speaking, dependency resolution isn't specific to building D
programs.  There's no technical reason why it should be limited only to
D.  Similarly, building a D program is merely a more specific instance
of the general task of transforming a set of input files into one or
more output targets by running some command (compiler in the case of D,
but I seriously doubt dub actually cares what executable is being run in
the place of the compiler, as long as it produces the expected outputs).
Caching is also a general concept that isn't tied specifically to
compiling D programs.

These general capabilities are arbitrarily restricted in dub, for
whatever reason, to only building D artifacts. As a consequence, dub is
unable to handle many things which, given the algorithms that it
*already implements*, it ought to able to handle.  The user is unable to
leverage these algorithms, which dub already uses internally, to other
tasks in his project.  This imposes arbitrary limitations on dub and
forces users to seek for more general solutions.

If there was a way to remove these limitations from dub so that it can
be used for more general build needs, I would be happy to use it.  But
as things stand, it's not meeting my build needs.



 However in 10 years I haven't needed it, I only just recently got
 around to improving shared library support so it actually worked as I
 needed that ;)


I'd be happy to contribute toward making dub more general, like I
outlined above.  But my current perception is that the dub maintainers
aren't interested in moving in this direction. I'd be happy to be proven
wrong, though!


[...]

 Upgrade only might be a good addition however.


 
 Doesn't dub already do this? Or am I not understanding this right?


 
 I don't know. I don't use this feature.
 
 https://github.com/dlang/dub/issues/2244


I think I'm missing something here. How does "upgrade only" relate to
the our present discussion?


T

-- 
Windows 95 was a joke, and Windows 98 was the punchline.
Apr 26 2022
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 6:14 AM, H. S. Teoh wrote:

 However in 10 years I haven't needed it, I only just recently got
 around to improving shared library support so it actually worked as I
 needed that ;)


 
 I'd be happy to contribute toward making dub more general, like I
 outlined above.  But my current perception is that the dub maintainers
 aren't interested in moving in this direction. I'd be happy to be proven
 wrong, though!


Yeah, there is a strong desire in the community to make dub support 
other build systems. It has been known for many years that this will 
need to happen at some point.

It is its greatest weakness due to it not supporting other languages. 
Which happens to be why it also works very well for D only projects *sigh*.

Note: there really isn't any maintainers, that is the problem. Dub needs 
someone to step up to the plate to manage it.

If we had someone who was able to review code and merge stuff for dub 
without waiting months (total exaggeration) a lot of these issues might 
go away (man power wise).

Of course this will be needed once ImportC starts to be mature enough to 
use (i.e. integrated preprocessor).


 [...]

 Upgrade only might be a good addition however.


 Doesn't dub already do this? Or am I not understanding this right?


 I don't know. I don't use this feature.

 https://github.com/dlang/dub/issues/2244


 
 I think I'm missing something here. How does "upgrade only" relate to
 the our present discussion?



$ dub upgrade

That should be the only command that should call out to the web service 
by default, unless required.

This command reruns dependency resolution and things like that, which 
were already cached.
Apr 26 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 06:25:40AM +1200, rikki cattermole via Digitalmars-d
wrote:

 On 27/04/2022 6:14 AM, H. S. Teoh wrote:


[...]

 I'd be happy to contribute toward making dub more general, like I
 outlined above.  But my current perception is that the dub
 maintainers aren't interested in moving in this direction. I'd be
 happy to be proven wrong, though!


 
 Yeah, there is a strong desire in the community to make dub support
 other build systems. It has been known for many years that this will
 need to happen at some point.


Well, maybe now is the time to make that happen!



 It is its greatest weakness due to it not supporting other languages.
 Which happens to be why it also works very well for D only projects
 *sigh*.
 
 Note: there really isn't any maintainers, that is the problem. Dub
 needs someone to step up to the plate to manage it.
 
 If we had someone who was able to review code and merge stuff for dub
 without waiting months (total exaggeration) a lot of these issues
 might go away (man power wise).


Wait. If there are no maintainers, then how does *anything* get merged
at all?!



 Of course this will be needed once ImportC starts to be mature enough
 to use (i.e. integrated preprocessor).


Well, so it looks like the time is ripe to break dub out of its original
limitations and extend its scope to other languages. (And other
build-like tasks, in general. ;-))


[...]

 I think I'm missing something here. How does "upgrade only" relate
 to the our present discussion?


 
 
 $ dub upgrade
 
 That should be the only command that should call out to the web
 service by default, unless required.
 
 This command reruns dependency resolution and things like that, which
 were already cached.


Ah, I see.  That makes sense.


T

-- 
It won't be covered in the book. The source code has to be useful for
something, after all. -- Larry Wall
Apr 26 2022
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 6:47 AM, H. S. Teoh wrote:

 It is its greatest weakness due to it not supporting other languages.
 Which happens to be why it also works very well for D only projects
 *sigh*.

 Note: there really isn't any maintainers, that is the problem. Dub
 needs someone to step up to the plate to manage it.

 If we had someone who was able to review code and merge stuff for dub
 without waiting months (total exaggeration) a lot of these issues
 might go away (man power wise).


 Wait. If there are no maintainers, then how does*anything*  get merged
 at all?!


Looks like WebFreak has been the main person merging stuff in the last 
year (and a lot of it has been mine :D).
Apr 26 2022
prev sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Tuesday, 26 April 2022 at 18:47:44 UTC, H. S. Teoh wrote:

 Well, so it looks like the time is ripe to break dub out of its 
 original limitations and extend its scope to other languages. 
 (And other build-like tasks, in general. ;-))


Would be nice if it will use plugin based arch, so that even 
currently supported features could be extracted into plugins, 
just like gradle or maven if you're from java world, and be easy 
for the public to add additional custom functionality as plugins 
to it without messing the internals.
Apr 26 2022
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 27/04/2022 7:15 AM, Alexandru Ermicioi wrote:

 On Tuesday, 26 April 2022 at 18:47:44 UTC, H. S. Teoh wrote:

 Well, so it looks like the time is ripe to break dub out of its 
 original limitations and extend its scope to other languages. (And 
 other build-like tasks, in general. ;-))


 
 Would be nice if it will use plugin based arch, so that even currently 
 supported features could be extracted into plugins, just like gradle or 
 maven if you're from java world, and be easy for the public to add 
 additional custom functionality as plugins to it without messing the 
 internals.


Right now shared library support is simply not at the level required to 
do this.

Dmd is the biggest lagger atm. But seriously export is a real design 
problem. I really dislike it affecting visibility when it is a linker 
directive.
Apr 26 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 07:23:16AM +1200, rikki cattermole via Digitalmars-d
wrote:

 
 On 27/04/2022 7:15 AM, Alexandru Ermicioi wrote:

 On Tuesday, 26 April 2022 at 18:47:44 UTC, H. S. Teoh wrote:

 Well, so it looks like the time is ripe to break dub out of its
 original limitations and extend its scope to other languages. (And
 other build-like tasks, in general. ;-))


 
 Would be nice if it will use plugin based arch, so that even
 currently supported features could be extracted into plugins, just
 like gradle or maven if you're from java world, and be easy for the
 public to add additional custom functionality as plugins to it
 without messing the internals.




Totally.  A plugin-based system like gradle would work.



 Right now shared library support is simply not at the level required
 to do this.


[...]

But why does a plugin system need to be based on shared libraries?

Why not expand dub's DSL to be able to encode the kind of functionality
needed to write external plugins, then rewrite existing functionality in
terms of that DSL as a plugin?  Then others can also write plugins in
the same DSL, and you wouldn't need to recompile dub just to add a
plugin.  Just download the plugin description file into some standard
dub directory, and dub would pick it up upon startup.

It's really not that complex.  At the core of it, dub has two main
functions:

1) Package management.
2) Building packages.

I don't think (1) needs to be touched; aside from some performance
optimizations and usage streamlining, dub does a decent enough job
already; the core logic doesn't need to be changed.

For (2), currently there's a lot of hard-coded functionality that's
specific to D. But at the core of it, it essentially boils down to a
build graph consisting of sources (currently D source files) and targets
(currently D executables / libaries) as nodes, and commands that
transform sources into targets (currently invocations of the D compiler)
as edges.

All that's needed is to factor out this underlying build graph structure
and make it a generic build graph resolution system, and rephrasing the
D-specific parts in terms of this generic code in a built-in plugin.
Then expose external interfaces for specifying build graph nodes and
edges to external plugins.


T

-- 
GEEK = Gatherer of Extremely Enlightening Knowledge
Apr 26 2022
parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Tuesday, 26 April 2022 at 20:30:59 UTC, H. S. Teoh wrote:

 On Wed, Apr 27, 2022 at 07:23:16AM +1200, rikki cattermole via 
 Digitalmars-d wrote:

 
 On 27/04/2022 7:15 AM, Alexandru Ermicioi wrote:

 On Tuesday, 26 April 2022 at 18:47:44 UTC, H. S. Teoh wrote:

 Well, so it looks like the time is ripe to break dub out 
 of its original limitations and extend its scope to other 
 languages. (And other build-like tasks, in general. ;-))


 
 Would be nice if it will use plugin based arch, so that even 
 currently supported features could be extracted into 
 plugins, just like gradle or maven if you're from java 
 world, and be easy for the public to add additional custom 
 functionality as plugins to it without messing the internals.




 Totally.  A plugin-based system like gradle would work.



 Right now shared library support is simply not at the level 
 required to do this.


 [...]

 But why does a plugin system need to be based on shared 
 libraries?


I think the reason is that dub will have to compile itself with 
declared plugins first in order to be able to execute them, if no 
shared libs are used.


 Why not expand dub's DSL to be able to encode the kind of 
 functionality needed to write external plugins, then rewrite 
 existing functionality in terms of that DSL as a plugin?  Then 
 others can also write plugins in the same DSL, and you wouldn't 
 need to recompile dub just to add a plugin.  Just download the 
 plugin description file into some standard dub directory, and 
 dub would pick it up upon startup.


Dub is more like maven from java world. I.e. it uses declarative 
approach to define a project and how to build it.

Code based config build systems in D are more like gradle, since 
gradle dsl is just plain groovy with some ast macros.

Btw, the idea with artifact repo support in dub is good. We 
should apply it to dub repos and replace current fetching from 
git with it. Then we could upload semi built packs as well as 
fully built ones.

The point of semi built ones being that they should be 
compile-able in straightforward manner by people downloading 
them, with everything else pre-built.

Then all this dissatisfaction might disappear, since you could 
use any build system you'd want for your lib, and should be able 
to integrate semi built format easily with your tool of choice 
too if you use a lib from dub repository.
Apr 27 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 08:14:21AM +0000, Alexandru Ermicioi via Digitalmars-d
wrote:

 On Tuesday, 26 April 2022 at 20:30:59 UTC, H. S. Teoh wrote:


[...]

 Why not expand dub's DSL to be able to encode the kind of
 functionality needed to write external plugins, then rewrite
 existing functionality in terms of that DSL as a plugin?  Then
 others can also write plugins in the same DSL, and you wouldn't need
 to recompile dub just to add a plugin.  Just download the plugin
 description file into some standard dub directory, and dub would
 pick it up upon startup.


 
 Dub is more like maven from java world. I.e. it uses declarative
 approach to define a project and how to build it.


Why can't a plugin DSL be declarative too?



 Code based config build systems in D are more like gradle, since
 gradle dsl is just plain groovy with some ast macros.


[...]

IMO, it's better to have a declarative language for writing plugins, so
that you do not have to recompile dub to install a plugin.


T

-- 
"A one-question geek test. If you get the joke, you're a geek: Seen on a
California license plate on a VW Beetle: 'FEATURE'..." -- Joshua D.
Wachs - Natural Intelligence, Inc.
Apr 27 2022
parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Wednesday, 27 April 2022 at 14:37:35 UTC, H. S. Teoh wrote:

 Dub is more like maven from java world. I.e. it uses 
 declarative approach to define a project and how to build it.


 Why can't a plugin DSL be declarative too?


It can, I was just thinking that dub is more like maven in java 
world, and for some reason was thinking that you'd extend config 
syntax to plugins (which is wrong).

It would be worth to check if existing language can be employed 
instead of custom dsl. It should lessen the need of maintenance 
then. Perhaps interpreter for D, or javascript?


 Code based config build systems in D are more like gradle, 
 since gradle dsl is just plain groovy with some ast macros.


 [...]

 IMO, it's better to have a declarative language for writing 
 plugins, so that you do not have to recompile dub to install a 
 plugin.


That's where shared library jumps in, to avoid dub recompilation.
The idea is that dub once seeing that a project requires plugin 
x, will download it from official repo, and load it and let it 
run at right events. This though, means shared lib use should be 
without hiccups in D.

If shared lib hiccups are not fixed or avoided in dub, then 
following can be done for a plugin based system:
1. Complete recompile of dub for each project combination of 
plugins, in order to run them.
2. Some form of microservices where each plugin is a standalone 
process.
3. Use of a interpreted language, declarative or imperative (your 
proposal).

Personally I'd prefer plugins as shared libs, and a well defined 
api for dub core through which plugins could interact and perform 
their job, while for configuration being done through existing 
sdl and json formats.

It would still be interesting to see what you can come up with 
custom DSL :)
Apr 28 2022
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Apr 28, 2022 at 10:06:37PM +0000, Alexandru Ermicioi via Digitalmars-d
wrote:

 On Wednesday, 27 April 2022 at 14:37:35 UTC, H. S. Teoh wrote:

 Dub is more like maven from java world. I.e. it uses declarative
 approach to define a project and how to build it.


 
 Why can't a plugin DSL be declarative too?


 
 It can, I was just thinking that dub is more like maven in java world,
 and for some reason was thinking that you'd extend config syntax to
 plugins (which is wrong).


Config syntax could be extended in such a way that plugins use a
different subtree in the config.



 It would be worth to check if existing language can be employed
 instead of custom dsl. It should lessen the need of maintenance then.
 Perhaps interpreter for D, or javascript?


I don't like the idea of an embedded interpreter. If you really need
code-based plugins, might as well just use dynamic libraries, or build
the core primitives into dub and expose some kind of external
declarative API for it.


[...]

 IMO, it's better to have a declarative language for writing plugins,
 so that you do not have to recompile dub to install a plugin.


 
 That's where shared library jumps in, to avoid dub recompilation.
 The idea is that dub once seeing that a project requires plugin x,
 will download it from official repo, and load it and let it run at
 right events.  This though, means shared lib use should be without
 hiccups in D.


I don't like the idea of using binary plugins for several reasons.

First, do we *really* need code-based plugins?  If the user really,
absolutely needs a programmatic way to express what he wants (and I have
my doubts about this), why can't we expose, say, some hooks that he can
run shell scripts (or full-fledged D programs if he insists) from?

Second, we will have to precompile and distribute plugins per platform,
which adds maintenance burden on our already thin manpower.

Third, we will need some kind of cryptographic signature mechanism for
ensuring the authenticity of plugins (we don't want dub to be the
vehicle for a security exploit by, e.g., a MITM attack that replaces
plugin code with malicious code).

Fourth, ongoing maintenance will be much harder than a declarative DSL
for plugins because of Turing completeness.  It will also be harder to
tell what exactly a plugin is doing. Anyone could write a dub plugin
that does who-knows-what when you run it. A declarative plugin is much
easier to review and understand/maintain.



 If shared lib hiccups are not fixed or avoided in dub, then following
 can be done for a plugin based system:
 1. Complete recompile of dub for each project combination of plugins,
 in order to run them.


Or use a DSL and avoid recompilation / cross-platform / security issues
altogether. ;-)



 2. Some form of microservices where each plugin is a standalone
 process.


I think this is total overkill. At least for the initial plan of making
dub support non-D builds.



 3. Use of a interpreted language, declarative or imperative (your
 proposal).


[...]

Actually, my proposal is far less ambitious than an interpreted
language. :-D  The main idea is to refactor dub into some generic
primitives for dependency resolution / building, and use config
directives to customize these primitives such that they do the right
thing for non-D builds.

I may be a little too optimistic here, though. :-D


T

-- 
My program has no bugs! Only undocumented features...
Apr 28 2022
prev sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Tuesday, 26 April 2022 at 18:14:20 UTC, H. S. Teoh wrote:

 Right, which brings me back to square one. Dub already knows 
 how to cache stuff, but I can't leverage any of it because it 
 was arbitrarily decided that dub would only work with D source 
 files, and therefore it only caches D artifacts.  If I want to 
 build HTML files from templates, for example, I couldn't 
 leverage dub's caching ability; I have to roll my own and 
 reinvent the square wheel.  If I want to build PNG files from 
 data files, I'd also have to roll my own cache.  Eventually, 
 I'd want to avoid reinventing my own cache for every artifact I 
 want to build, so I'd look for a system that offers me that 
 capability.  Such a system would obviously also be able to 
 build D programs and cache them as artifacts, since building D 
 programs would be a specific case of the more general task of 
 building a target from sources and caching it. Which in turn 
 means that dub has just become redundant: I might as well just 
 use that other system for *all* my build needs, including 
 building D programs, and it would let me do everything I need 
 without needing to work with two different, overlapping systems 
 (dub + other build system).

 That's my current line of reasoning, and why I haven't 
 seriously invested in dub.


So, perhaps dub itself should be used as part of your more 
general build system to build D code only?
Perhaps, there should be some integrations of of dub with other 
popular tools, so such use would be less painful.
That is ofc, in case you want your code available for others to 
use where you work, or by public.
Apr 26 2022
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Apr 26, 2022 at 07:06:50PM +0000, Alexandru Ermicioi via Digitalmars-d
wrote:

 On Tuesday, 26 April 2022 at 18:14:20 UTC, H. S. Teoh wrote:

 
 Right, which brings me back to square one. Dub already knows how to
 cache stuff, but I can't leverage any of it because it was
 arbitrarily decided that dub would only work with D source files,
 and therefore it only caches D artifacts.  If I want to build HTML
 files from templates, for example, I couldn't leverage dub's caching
 ability; I have to roll my own and reinvent the square wheel.  If I
 want to build PNG files from data files, I'd also have to roll my
 own cache.  Eventually, I'd want to avoid reinventing my own cache
 for every artifact I want to build, so I'd look for a system that
 offers me that capability.  Such a system would obviously also be
 able to build D programs and cache them as artifacts, since building
 D programs would be a specific case of the more general task of
 building a target from sources and caching it.  Which in turn means
 that dub has just become redundant: I might as well just use that
 other system for *all* my build needs, including building D
 programs, and it would let me do everything I need without needing
 to work with two different, overlapping systems (dub + other build
 system).
 
 That's my current line of reasoning, and why I haven't seriously
 invested in dub.


 
 So, perhaps dub itself should be used as part of your more general
 build system to build D code only?


But why would I even need dub when the more general build system also
handles building D code just fine?  Unless the boss dictates that I
absolutely have to use dub *somewhere*, just for the sake of using dub,
then it's just a redundant gear doing what other gears are already
doing. Might as well economize and take it out completely.



 Perhaps, there should be some integrations of of dub with other
 popular tools, so such use would be less painful.


If we want more widespread adoption of dub, I would say this is
absolutely essential.



 That is ofc, in case you want your code available for others to use
 where you work, or by public.


[...]

The problem right now is, even if I want my code to be available to
others, dub may not be able to do it.

E.g., if part of my D source files is autogenerated from data files
using an auxiliary utility (also written in D), then dub wouldn't even
know how to build such a thing. I can call dub from my build system and
it'd work, but anyone else who uses dub to download my package would not
be able to use it because dub doesn't know how to build it, and that
person would not know to run another build system to make it work.

To make it work, I'd have to jump through lots of hoops and write a
whole bunch of extra scripts to be called from dub's "preBuildCommands",
that essentially duplicates what my other build system is already doing.
It's a lot of redundant bending over backwards just to work around dub's
limitations.  Unless my boss is holding a gun to my head and demanding
that I make it work, the chances of me putting in this effort is nil.

Dub needs to expand its horizons and break out of its own walled garden.


T

-- 
Time flies like an arrow. Fruit flies like a banana.
Apr 26 2022
prev sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Tuesday, 26 April 2022 at 17:19:24 UTC, H. S. Teoh wrote:

 Can dub handle building multiple executables from different 
 subsets of source files in the same source tree?


Yes, you can define them as subpackages in the config. See my 
terminal emulator for example:

https://github.com/adamdruppe/terminal-emulator/blob/master/dub.json

dub run adr-terminalemulator:mactabs

for example.

The makefile is in many ways simpler though

https://github.com/adamdruppe/terminal-emulator/blob/master/Makefile

(and that makefile precedes dmd -i, which is how i'd do it now)
Apr 26 2022
prev sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 26 April 2022 at 01:21:58 UTC, Mike Parker wrote:

 On Monday, 25 April 2022 at 20:12:19 UTC, deadalnix wrote:


 The least power for the job definitively implies not inserting 
 a build system in the package manager.


 I'm posting in reply to deadalnix, but my comments are not 
 directed at him specifically.

 dub exists because it filled a huge, gaping hole in the D 
 ecosystem.

 Go back far enough and you'll find "build", later renamed 
 "bud", by Derek Parnell. This was the first D build tool the 
 community rallied around. Several D users adopted it.

 https://www.digitalmars.com/d/archives/digitalmars/D/40914.html

 Gregor Richards wasn't happy with bud, so he created "Rebuild" 
 as an alternative build tool, but added a package manager on 
 top of it called "DSSS" (D Shared Software System). Rebuild 
 actually parsed source files looking for custom pragmas, and 
 IIRC DSSS could install packages from multiple locations (svn, 
 git, etc). Several D libraries users adopted it.

 https://forum.dlang.org/thread/f06hsi$ng1$2 digitalmars.com

 So let me tell you, this sounds but it was also a serious PITA 
 as a library maintainer. You had the bud crowd demanding 
 support for bud, the dsss crowd demanding support for dsss, and 
 the crowd who refused to use either because make was good 
 enough. At one point I had three platform-specific make files, 
 a `dsss.conf`, and a `forbud.txt` (I eventually replaced some 
 of that with a `buildme.d file`).

 There was no standard way of building D projects, and it 
 sucked. Then Derek and Gregor disappeared, and suddenly these 
 build systems everyone is using are no longer supported. No one 
 takes them over, so we're in the Dark Ages again. Then dub 
 comes along and saves the day.

 It caught on quickly. Then people start thinking: "What happens 
 if Sönke disappears? This should be an official D thing." So 
 dub gets added to the installation. And here we are.

 Without dub, we would not have the ecosystem that we do have. 
 Personally, I love it. It makes using dependencies a snap. I'm 
 happy with the default source tree expectations, and I don't 
 mind compiling C libraries separately when I need them.

 But I understand that no everyone is going to be happy with it 
 (I don't understand the level of hatred some people direct at 
 it, but I just chalk it up as "they aren't happy with it" and 
 move on).

 The question is, what exactly is dub missing and how can it be 
 improved? Which perceived shortcomings are reasonable to 
 address and which are not? And assuming we can answer those 
 questions, who is going to address them?


dub misses a good tutorial and good example files for different 
scenarios. I tried to integrate it in my C project (granted, 
which is very oddly structured). I managed to get quite far but 
the issue was that I had to check a lot of projects on github to 
see examples of different options. One thing for example I didn't 
manage to get to work correctly was to specify separate directory 
for the object files (in the C project we have debug and 
optimized builds and to avoid rebuilding each time, the `.o` 
files are in their own directory). dub always put them where it 
did the linking. I'm sure dub allows to set it up somehow but the 
documentation is very unclear on that.
May 02 2022
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 02/05/2022 10:16 PM, Patrick Schluter wrote:

 I'm sure dub allows to set it up somehow but the documentation is very 
 unclear on that.


The documentation is clear. It does not support this (currently).

targetPath	string	The destination path of the output binary - this 
setting does not support platform suffixes
May 02 2022
prev sibling parent deadalnix <deadalnix gmail.com> writes:
On Monday, 25 April 2022 at 11:15:59 UTC, Guillaume Piolat wrote:

 Things like DUB and NPM replaces attention by trust.


The big difference between RPM and dub is that one solve that 
problem for me while imposing it's worldview on me like it's on a 
mission from god and the other just works.
Apr 25 2022
prev sibling next sibling parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
 my way or the highway attitude


Just view it from the other side: You are typically publishing a package 
for an audience to use. If you insist on using your own choice of build 
system, you also dictate that upon the users of your package - forcing 
them to install and use that system, as well as forcing custom 
integration of any build artifacts into development dependencies (aka 
C/C++ dependency hell).

Having said that, you can define a custom command to run make if you 
really want to.

The way versions are managed is very convenient for publishing if DUB is 
your primary target, but may admittedly get in the way if you have other 
systems. Nothing, except the need to put the necessary work into the 
registry, speaks against supporting other mechanisms, though.

This applies to many of the current shortcomings, such as the error 
diagnostics when adding packages. A lot of things are not meant to be 
limitations, but simply represent what people were willing to invest or 
deemed sufficient.
Apr 25 2022
prev sibling next sibling parent reply JN <666total wp.pl> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 Ho, you wanted to distribute a package? Well, now you have a 
 new build system. make was working great for you? Too bad. Can 
 dub run make? It doesn't look like it but who knows. I don't.


Does your makefile support other platforms than Linux? How about 
Windows? Makefiles aren't even native for Windows. I remember 
before dub came about I had to recreate the Unix-like ecosystem 
in MSYS/Cygwin just to have a chance to compile a D library. 
These days I do dub build and most projects compile with no issue 
on Windows.

Yes, it is limiting, and that is it's strength. With boundaries 
comes reliability and consistency. If I want to reuse a piece of 
code I want to dub add it to a project and start using it, not 
chase down dependencies and try to make its code structure fit 
into mine.

And the general practice seems to show it's the right approach. 
Cargo is a massive success in Rust and C++ is constantly trying 
to get their package manager story right.
Apr 26 2022
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Tuesday, 26 April 2022 at 11:00:26 UTC, JN wrote:

 Yes, it is limiting, and that is it's strength. With boundaries 
 comes reliability and consistency.


Don't fart on me and tell me it's Channel.

Here the simple truth. Either the build is
1/ simple in which case `dmd -i` is enough and dub superfluous.
2/ complex in which case dub's "strength" is that it is too 
limiting to do it.

You are getting real D users, people who have been using it for 
many many years, telling you dub is creating more problem for 
them that it solves.

And for the windows build, I'm glad you brought that up. As 
pointed out, I have several application in that repo. Some of 
them build on windows, other don't. Ever since I added dub, the 
ones that used to build on windows still build on windows, and 
the one that don't don't build with dub.

A grand total of zero progress has been made thanks to dub's 
amazing "strength" on this front. Zero, niltch, nada, niente.
Apr 26 2022
parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 26 April 2022 at 11:37:51 UTC, deadalnix wrote:

 On Tuesday, 26 April 2022 at 11:00:26 UTC, JN wrote:

 Yes, it is limiting, and that is it's strength. With 
 boundaries comes reliability and consistency.


 Don't fart on me and tell me it's Channel.

 Here the simple truth. Either the build is
 1/ simple in which case `dmd -i` is enough and dub superfluous.
 2/ complex in which case dub's "strength" is that it is too 
 limiting to do it.

 You are getting real D users, people who have been using it for 
 many many years, telling you dub is creating more problem for 
 them that it solves.


You're approaching this from the wrong perspective. Dub isn't for 
packaging an arbitrary project as effortlessly as possible. Use 
rpm/apt/ports/flatpak/snap/nix/chocolatey/etc for that.

Dub is for packaging a pure D application, library or wrapper so 
that anyone with a D compiler can compile your D code as easily 
as possible. The idea is that the user does not need anything but 
Dub and a D compiler to build a DUB package. If you require 
something beyond that, you're using Dub for a purpose it's not 
intended to. It's more about the user and less about the packager.
May 07 2022
parent reply Egor Pugin <egor.pugin gmail.com> writes:
On Saturday, 7 May 2022 at 20:58:54 UTC, Dukc wrote:

 snip


Hi,

I see this thread is discussing package management/buils systems.
I have some experience in this area in C++ and will do some notes.

---

**1. Declarative artificial build descriptions**

See https://github.com/cppan
'specs' repo contains many examples, but they are serialized and 
not very useful as is (replace all '\\n' with '\n' in 'cppan' 
field).
You can check this build script with cmake inserions 
-https://github.com/cppan/cppan/blob/v1/cppan.yml
Cmake was used as a driver here.

Declarative approach showed itself unuseful in complex cases (C 
libs, C programs with complex build systems).
Relying on other tools also is a dependency, also very dangerous.

---

**2. Plugin-based compiled build descriptions**

Second (and existing) approach is compiled C++ code into shared 
object plugins.

https://software-network.org/
https://github.com/SoftwareNetwork
https://github.com/SoftwareNetwork/sw (main repo)
https://github.com/SoftwareNetwork/specifications (storage of 
specs, specs are visible on the website also)

Imperative approach and C++ itself give more space when 
describing builds, but there are limitations also.
Compilation can be quick with PCH (implemented and used) and 
probably with C++20 modules (not yet implemented).
Linking of these plugins take visible amount of time and a lot of 
RAM. So, parallel builds of these plugins are slow on small 
systems.

In general I see following problems with this approach.
We get now full tree of any build. How do we store build 
settings? We must be able to tweak any parameter, any definition, 
and compile/link option in any of downstream libs.
I used simple object-based config tree like json/yaml and this is 
the problem.
With such approach flexibility is the problem, we cannot pass 
custom data structures as config to plugins and order to use them.

Some scalability is present, but not great because of some bad 
implementation decisions.
Low or very low flexibility. It is hard to ask dependencyA to 
build with option X.

You can check the website, some application is possible, but 
still it is not that smooth.
Project have some integrations with cmake. Build capabilities as 
make/ninja.

---

**3. [WIP] Next iteration, current ideas**

Build description of a project should be compiled into an 
executable binary that contains whole build tree config.
Every target(package) must have its build config in the form of 
polymorphic object so it can change any property directly, not in 
any json/yaml/xml-like tree structure.

Not giving more details here because they are still only in form 
of ideas.

---

**Ending**

I see some mentions or preferences of some people for p.1. or p.2.
But curious readers may try to repeat steps 1 or 2 get their own 
experience which takes couple of years or point what was done 
wrong in their opinion.

Quick comments about some existing high level C++ tools (build 
systems (BS), build system generators (BSG) and package managers 
(PM)).

**cmake** - BSG; very bad DSL; not viable in long term?
**meson** - BSG; better because of python-like DSL. But bad 
because not imperative, not real Python (very bad - look at their 
parser). Questionable system of project settings. Need more 
practical work on their PM.
**premake** - BSG; Lua. Lua is fine.
**conan** - PM; python dsl is good, but it's hard to work with 
every underlying BS projects have.
**vcpkg** - PM; what DSL? still no package versions? There was 
some work. Are package versions ready? Many decisions are 
questionable also. Project is pushed with MS power, has a lot of 
users.
**xmake** - BS+BSG+PM; Lua as build desc; looks good, no deps on 
BS/PM, needs evaluation. Lua is fine.
**make** - BS; just works. Easy for simple configs, harder for 
large and huge projects. P.3 future projects must aim for its 
simplicity in describing simple builds.
**ninja** - BS; works. What about dynamic build graphs 
(Fortran/C++ modules)?
**presented here**
**cppan** - BSGG: yaml->cmake; cmake is bad, building something 
on the top of it is worse.
**sw** - BS+BSG+PM; C++ as build desc; complex native plugins 
building; low flexibility on huge build graphs.
**other langs**
**D(dub,...?)** - do not know much. You should tell better.
**rust(cargo)** - BS(?)+PM(?); smooth integration, does it job.
**java(...)** - established set of tools; ecosystem is fine.


Not implementing BS+BSG+PM combo is bad.
You will get dependency on make/ninja/PM etc.
Viable project must do everything itself.
May 07 2022
parent reply Tejas <notrealemail gmail.com> writes:
On Saturday, 7 May 2022 at 21:36:40 UTC, Egor Pugin wrote:

 On Saturday, 7 May 2022 at 20:58:54 UTC, Dukc wrote:

 [...]


 Hi,

 I see this thread is discussing package management/buils 
 systems.
 I have some experience in this area in C++ and will do some 
 notes.

 [...]


What do you think about using **conda** as a package manager? 
It's _completely_ decoupled from build systems(as well as which 
programming language you use, from what I can tell)
May 07 2022
parent reply Egor Pugin <egor.pugin gmail.com> writes:
On Sunday, 8 May 2022 at 03:47:41 UTC, Tejas wrote:

 What do you think about using **conda** as a package manager? 
 It's _completely_ decoupled from build systems(as well as which 
 programming language you use, from what I can tell)


Looks like just binary package manager.
I do not see how it helps building software.
It can make an environment for you, but how to integrate it into 
cmake or make a build file for make/ninja?
May 08 2022
parent reply Tejas <notrealemail gmail.com> writes:
On Sunday, 8 May 2022 at 09:24:18 UTC, Egor Pugin wrote:

 On Sunday, 8 May 2022 at 03:47:41 UTC, Tejas wrote:

 What do you think about using **conda** as a package manager? 
 It's _completely_ decoupled from build systems(as well as 
 which programming language you use, from what I can tell)


 Looks like just binary package manager.
 I do not see how it helps building software.
 It can make an environment for you, but how to integrate it 
 into cmake or make a build file for make/ninja?


Yeah, that's actually what I like about it: it's completely 
seperate from any build system and programming language

Over here, you can see how people don't like `dub`, and would 
like to use something else while still managing to provide 
packages to the community as a whole

I'd reckon that people build their code using whatever software 
they prefer, but publish their code on `conda` for easy usage, 
allowing the flexibility of their preferred tool, while 
maintaining the easy accessibility of their package.
May 08 2022
next sibling parent Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Sunday, 8 May 2022 at 10:46:52 UTC, Tejas wrote:

 I'd reckon that people build their code using whatever software 
 they prefer, but publish their code on `conda` for easy usage, 
 allowing the flexibility of their preferred tool, while 
 maintaining the easy accessibility of their package.


You should be able to do this even now. Just integrate dub into 
your build system, and that is it.
May 08 2022
prev sibling parent Egor Pugin <egor.pugin gmail.com> writes:
On Sunday, 8 May 2022 at 10:46:52 UTC, Tejas wrote:

 Over here, you can see how people don't like `dub`, and would 
 like to use something else while still managing to provide 
 packages to the community as a whole

 I'd reckon that people build their code using whatever software 
 they prefer, but publish their code on `conda` for easy usage, 
 allowing the flexibility of their preferred tool, while 
 maintaining the easy accessibility of their package.


After my previous work I do not see this as scalable approach.
Having different build tools makes it harder to publish, 
distribute and control.

My idea is that everything related must be in one tool.
Having one tool allows you to control everything.
May 08 2022
prev sibling parent reply Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Tuesday, 26 April 2022 at 11:00:26 UTC, JN wrote:

 On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 Ho, you wanted to distribute a package? Well, now you have a 
 new build system. make was working great for you? Too bad. Can 
 dub run make? It doesn't look like it but who knows. I don't.


 Does your makefile support other platforms than Linux? How 
 about Windows? Makefiles aren't even native for Windows.


Is this still an issue? Doesn't 
[winget](https://winget.run/pkg/GnuWin32/Make) fix this?
Apr 26 2022
parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Tuesday, 26 April 2022 at 12:47:56 UTC, Ola Fosheim Grøstad 
wrote:

 On Tuesday, 26 April 2022 at 11:00:26 UTC, JN wrote:

 On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 Ho, you wanted to distribute a package? Well, now you have a 
 new build system. make was working great for you? Too bad. 
 Can dub run make? It doesn't look like it but who knows. I 
 don't.


 Does your makefile support other platforms than Linux? How 
 about Windows? Makefiles aren't even native for Windows.


 Is this still an issue? Doesn't 
 [winget](https://winget.run/pkg/GnuWin32/Make) fix this?


Indeed, not native. Are you suggesting Dub should include winget?

-- Bastiaan.
Apr 27 2022
next sibling parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Wednesday, 27 April 2022 at 11:09:33 UTC, Bastiaan Veelo wrote:

 Indeed, not native. Are you suggesting Dub should include 
 winget?


What do you mean by «not native»? Microsoft is pushing it to 
developers?

https://docs.microsoft.com/en-us/windows/package-manager/winget/
Apr 27 2022
prev sibling parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 27 April 2022 at 11:09:33 UTC, Bastiaan Veelo wrote:

 Indeed, not native. Are you suggesting Dub should include 
 winget?


If dub could resolve non-D dependencies, it could actually grab 
winget as part of its resolution. Potentially a very interesting 
angle.
Apr 27 2022
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 03:42:50PM +0000, Adam D Ruppe via Digitalmars-d wrote:

 On Wednesday, 27 April 2022 at 11:09:33 UTC, Bastiaan Veelo wrote:

 Indeed, not native. Are you suggesting Dub should include winget?


 
 If dub could resolve non-D dependencies, it could actually grab winget
 as part of its resolution. Potentially a very interesting angle.


If dub could be expanded to accept a superset of its narrowminded
interpretation of SemVer, its dependency resolution algorithm could be
reused for resolving non-D dependencies too.  I think this is a fruitful
direction to go.  Dub seriously needs to get over its walled garden
mentality and become more open to interacting with the outside world.

This would most certainly lead to very interesting developments. The
package management part of dub could be decoupled from the build system,
for example, by making build systems part of the dependency system so
that alternative build systems can be installed and used directly by
dub.  Cross-language capability could be implemented by making compiler
toolchains part of the dependency system -- you'd simply depend on, say,
a C compiler or a Java compiler pseudo-package, and you could build
native C/Java code directly inside dub and link it to D code (e.g., via
jni.d as part of the Java compiler pseudo-package).

There's a whole world out there beyond the fences of the walled garden,
just waiting to be explored.


T

-- 
The problem with the world is that everybody else is stupid.
Apr 27 2022
next sibling parent Adam D Ruppe <destructionator gmail.com> writes:
On Wednesday, 27 April 2022 at 20:38:53 UTC, H. S. Teoh wrote:

 There's a whole world out there beyond the fences of the walled 
 garden, just waiting to be explored.


Aye. I've written a bit in my blog about these ideas btw:

dub's problems and some general ideas for dub 2.0:
http://dpldocs.info/this-week-in-d/Blog.Posted_2022_03_28.html

a bit more thought on fallback dependencies and system 
integration:
http://dpldocs.info/this-week-in-d/Blog.Posted_2022_04_25.html


I think my topic next week might be more general, about how so 
many of the D stuff right now puts burden on library developers 
without benefiting them. Maintaining dub.json gives me absolutely 
nothing and is a bit of a pain to keep working. A lot of my views 
lately have just been reducing the cost - I already have a system 
that works really well, so I don't expect to benefit from any of 
these concepts. So I'm all about reducing the cost.

But there might be potential to actually give some return on 
investment if all this worked; I could make the postgres.d, for 
example, find a libpq or get the user to download it. (Though 
again, I kinda already can with runtime dynamic loading. But meh.)

The one time dub delivered a little was the android package and 
even there it limited me more than it benefited when compared to 
dmd -i. alas.
Apr 27 2022
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 27 April 2022 at 20:38:53 UTC, H. S. Teoh wrote:

 [snip]
 This would most certainly lead to very interesting 
 developments. The package management part of dub could be 
 decoupled from the build system, for example, by making build 
 systems part of the dependency system so that alternative build 
 systems can be installed and used directly by dub.  [snip]


That's been talked about for awhile...maybe it would take someone 
forking dub and just releasing one or the other component.
Apr 27 2022
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Wed, Apr 27, 2022 at 10:20:23PM +0000, jmh530 via Digitalmars-d wrote:

 On Wednesday, 27 April 2022 at 20:38:53 UTC, H. S. Teoh wrote:

 [snip]
 This would most certainly lead to very interesting developments. The
 package management part of dub could be decoupled from the build
 system, for example, by making build systems part of the dependency
 system so that alternative build systems can be installed and used
 directly by dub.  [snip]


 
 That's been talked about for awhile...maybe it would take someone
 forking dub and just releasing one or the other component.


I've thought about doing exactly this, in fact.  I haven't actually
looked at the code in detail yet, but based on the documentation of the
configuration files, I'm thinking to extend the concept of
configurations to cover non-D build rules.

Essentially, the current concept of a "package" is more-or-less
equivalent to a subgraph in a generic build graph where the inputs are a
particular collection of D source files, and the output is a D
executable or library.  Thanks to dub's built-in backward compatibility
measure of ignoring unknown directives, one could retroactively say that
current packages have implicit settings whose default values are those
that designate the package as a D build, with the current semantics.
When these (new, but retroactively interpreted as implicit) settings are
modified, they change the semantics into non-D builds.  Thus, the
concept of a package can be extended to non-D builds in a backward
compatible way that does not require modifying existing dub packages.

I'm not sure how amenable the current dub code is to this modification,
but worse comes to worst, one can always rewrite this part of the code
from ground up, following the current specifications for the existing
configuration parameters, and adding new parameters with the new
semantics. Possibly reuse dub's test suite to ensure backwards
compatibility.  Or perhaps even the nuclear option of rewriting the code
from scratch while maintaining the ability to parse existing
dub.json/dub.sdl files so that existing packages can transition with
zero change.


T

-- 
Don't drink and derive. Alcohol and algebra don't mix.
Apr 27 2022
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 28/04/2022 8:38 AM, H. S. Teoh wrote:

 If dub could be expanded to accept a superset of its narrowminded
 interpretation of SemVer, its dependency resolution algorithm could be
 reused for resolving non-D dependencies too.  I think this is a fruitful
 direction to go.  Dub seriously needs to get over its walled garden
 mentality and become more open to interacting with the outside world.


Dependency resolution algorithm specialization:

https://github.com/dlang/dub/blob/059f7a2a16cf7a50a5c68dc295c796482116ff87/source/dub/dub.d#L1793

Semver matching ext.

https://github.com/dlang/dub/blob/master/source/dub/dependency.d#L453
Apr 27 2022
prev sibling next sibling parent monkyyy <crazymonkyyy gmail.com> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

anti dub the topic


Is there some possible compromise with dub and biuld.d?

Like if biuld.d exists and has a "linux run this" comment, dub 
will attempt to run it; maybe define some syntax there to make it 
flexible.
May 02 2022
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Saturday, 23 April 2022 at 20:15:27 UTC, deadalnix wrote:

 Why do we use a package manager that expect everyone to use it 
 to work in a certain way? It is blatantly obvious that many 
 organization out there are not going to use the exact setup dub 
 expects. So what do they do if they want to adopt D? Not use 
 dub at all? It's an option, but everything is distributed via 
 dub nowadays in the D ecosystem. Use dub to build everything? 
 Fork their own version of dub? No, the only sensible path 
 forward for pretty much any organization stumbling on this 
 bullshit is to not use D at all because none of the crap we put 
 out there is going to play nicely with what they already have.


The problem is that if we don't impose restrictions on how Dub 
packages can build, it's a complete Wild West from user 
perspective. Dub packages would install, or require you to 
install countless third-party build systems or even package 
managers, and they would likely set and read environmental 
variables willy-nilly.

Also unlike general purpose package managers, Dub needs to be 
portable to the Windows world. This is another reason you can't 
just have it build however it happens to work on your system.

Same goes to the version scheme. If you could have versions like 
"1-experimental-0", Dub would have no way to know how it's 
supposed to sort different versions from oldest to newest. 
Forcing SemVer solves that.
May 07 2022
parent Ola Fosheim =?UTF-8?B?R3LDuHN0YWQ=?= <ola.fosheim.grostad gmail.com> writes:
On Saturday, 7 May 2022 at 20:31:00 UTC, Dukc wrote:

 The problem is that if we don't impose restrictions on how Dub 
 packages can build, it's a complete Wild West from user 
 perspective.


Yes, I think this makes sense. If it is true that Dub can 
generate *cmake* builds then it means that Dub packages can later 
be converted to other package managers that does not use Dub. If 
this actually is possible with Dub today or could be made 
possible, in a transparent way, then that would be a very 
important quality to obtain/retain.
May 08 2022