www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - dub -> meson

reply 9il <ilyayaroshenko gmail.com> writes:
Hi all,

'DUB - call to arms' post by Seb
https://forum.dlang.org/post/unimbbqxosxcmjdhtzch forum.dlang.org

Hope we will move other direction and extend Meson build system 
with better D support and make a new Dlang registry.

Reasons to move forward with Meson:

0. Progressive and large community
1. Has clear python source code: faster to extend, easier to 
maintain.
2. D+Meson already used in production: last year it is a major 
build system for all of Mir and its commercial applications.
3. Meson language is super cool. It is much better than dub for 
commercial projects, for large projects, for library distribution.
4. Ability to compile and configure D code with other languages 
like C/C++.
5. With Meson we don't need to catch up Rust, we can do better 
than Rust with fewer resources.


Best,
Ilya
Apr 15
next sibling parent reply Julian <julian.fondren gmail.com> writes:
On Tuesday, 16 April 2019 at 01:45:26 UTC, 9il wrote:
 Hi all,

 'DUB - call to arms' post by Seb
 https://forum.dlang.org/post/unimbbqxosxcmjdhtzch forum.dlang.org

 Hope we will move other direction and extend Meson build system 
 with better D support and make a new Dlang registry.
Is it just a build system? Isn't a package manager also needed? Why a *new* registry?
 Reasons to move forward with Meson:

 1. Has clear python source code: faster to extend, easier to 
 maintain.
Oh. Well it was certainly a mistake to try use D for something like dub. I've thought of D as a kind of competitor to Python but I sure don't want slow to extend, hard to maintain code.
 4. Ability to compile and configure D code with other languages 
 like C/C++.
That's a surprising oversight. Was it intentional?
Apr 15
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 16/04/2019 2:15 PM, Julian wrote:
 
 4. Ability to compile and configure D code with other languages like 
 C/C++.
That's a surprising oversight. Was it intentional?
Out of scope. Dub was only meant to handle D programs using minimal external dependencies and for this purpose it has succeeded.
Apr 15
parent reply 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 16 April 2019 at 02:30:25 UTC, rikki cattermole wrote:
 On 16/04/2019 2:15 PM, Julian wrote:
 
 4. Ability to compile and configure D code with other 
 languages like C/C++.
That's a surprising oversight. Was it intentional?
Out of scope. Dub was only meant to handle D programs using minimal external dependencies and for this purpose it has succeeded.
I don't think so. For example, dub does not allow to compile selected dependencies in release builds, while the current project in debug mode. This is a common case for numeric libraries like mir-glas (blas), mir-optim and etc. So, we need to call dub as prebuild command https://github.com/libmir/mir-glas/blob/master/dub.json#L28 For simple system dependencies dub force users to define configuration monsters like that for mir-blas: https://github.com/libmir/mir-blas/blob/master/dub.sdl Maybe configurations for vibe.d and other packages looks no so monstrous, however for my uses cases, it adds a lot of pain. Also, this pain in inherited by dependencies: https://github.com/libmir/mir-lapack/blob/master/dub.sdl https://github.com/kaleidicassociates/lubeck/blob/master/dub.sdl
Apr 15
parent reply JN <666total wp.pl> writes:
On Tuesday, 16 April 2019 at 05:11:29 UTC, 9il wrote:
 I don't think so. For example, dub does not allow to compile 
 selected dependencies in release builds, while the current 
 project in debug mode. This is a common case for numeric 
 libraries like mir-glas (blas), mir-optim and etc.
 So, we need to call dub as prebuild command 
 https://github.com/libmir/mir-glas/blob/master/dub.json#L28

 For simple system dependencies dub force users to define 
 configuration monsters like that for mir-blas:
 https://github.com/libmir/mir-blas/blob/master/dub.sdl

 Maybe configurations for vibe.d and other packages looks no so 
 monstrous, however for my uses cases, it adds a lot of pain. 
 Also, this pain in inherited by dependencies:
 https://github.com/libmir/mir-lapack/blob/master/dub.sdl
 https://github.com/kaleidicassociates/lubeck/blob/master/dub.sdl
Those "configuration monsters" don't look that bad, compared to your average Ant or CMake build script (can't speak for Meson, never used it). The problem with using anything else is that we're back to square one. If you push Meson as an official build tool, soon people will want to bring CMake, SCons, Gradle, makefiles etc. into the mix. Which will be problematic, especially on Windows. Right now you can download dub and hope that most of stuff will build, without having to chase down dependencies and additional build tools. And after a while someone will propose, why do we use all these tools, why not create a single build tool, written in D, that can support these usecases. And then we'll reinvent dub again :)
Apr 15
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d wrote:
[...]
 The problem with using anything else is that we're back to square one.
 If you push Meson as an official build tool, soon people will want to
 bring CMake, SCons, Gradle, makefiles etc. into the mix. Which will be
 problematic, especially on Windows. Right now you can download dub and
 hope that most of stuff will build, without having to chase down
 dependencies and additional build tools. And after a while someone
 will propose, why do we use all these tools, why not create a single
 build tool, written in D, that can support these usecases. And then
 we'll reinvent dub again :)
The problem is that while dub fills a very important role in the D ecosystem, it also has some unfortunate design decisions that has since become major limitations as the ecosystem grows. It originally targeted a very specific, and narrow use case, and the design was fine in that context. But as the D ecosystem grows to encompass wider use cases, the inherent limitations in that design has since become a major issue, esp. with those of us who need to do stuff beyond just "pull in dependencies, build everything". For example, for me, supporting D codegen via external tools (themselves needing to be compiled and run from the same source tree) and mixed-language codebases (requiring multiple different toolchains) is absolutely essential. Since dub has no way of doing this, it simply isn't even a viable option for me. Performance is also a big showstopper -- although it has improved since I last checked, the code-compile-test turnaround time is still far too slow for everyday usage. What we really need is a dub replacement that has a different, more flexible design that can accomodate things that people want to do, that the current dub simply cannot due to its design limitations. There are some promising ideas in the other thread about improving dub, and I hope that discussion will eventually lead to a redesign / revamp of dub that makes it more viable for my use case. T -- The problem with the world is that everybody else is stupid.
Apr 16
next sibling parent psycha0s <box mail.com> writes:
On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
 What we really need is a dub replacement that has a different, 
 more flexible design that can accomodate things that people 
 want to do, that the current dub simply cannot due to its 
 design limitations.
I absolutely agree. Trying keep backward compatibility is a good intention but sometimes you just have to redo everything from scratch using accumulated experience. It can be painful but this is evolution. Otherwise it can lead to stagnation.
Apr 16
prev sibling next sibling parent JN <666total wp.pl> writes:
On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
 On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d 
 wrote: [...]
 What we really need is a dub replacement that has a different, 
 more flexible design that can accomodate things that people 
 want to do, that the current dub simply cannot due to its 
 design limitations.  There are some promising ideas in the 
 other thread about improving dub, and I hope that discussion 
 will eventually lead to a redesign / revamp of dub that makes 
 it more viable for my use case.


 T
Dub replacement doesn't sound good. D ecosystem is fragmented enough, no need to add additional split dub/dub replacement. I think it'd be best to just start with dub and try to improve on it. Cargo could be used as an inspiration, looks to me like in Cargo, CMake and Meson and other build systems are added as dependencies (https://docs.rs/meson/1.0.0/meson/ https://crates.io/crates/cmake ). Perhaps something like this could be adapted for dub. Some kind of plugin system, with optional cmake and meson plugins.
Apr 16
prev sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
 On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d 
 wrote: [...]

 What we really need is a dub replacement that has a different, 
 more flexible design that can accomodate things that people 
 want to do, that the current dub simply cannot due to its 
 design limitations.  There are some promising ideas in the 
 other thread about improving dub, and I hope that discussion 
 will eventually lead to a redesign / revamp of dub that makes 
 it more viable for my use case.


 T
IMO, the best course of action is: 1.0. Create a generic foundation based on DAG for describing targets and their relations. This can start as a completely independent project with zero dependencies. 1.1. On top of this core functionality we can add an interoperability layer that allows exporting this abstract representation to other build system formats. 1.2. We can reuse the existing build systems written in D as a build backend. 2.0. Integrate the new build system foundation in Dub 2.0: 2.1 Replace dub's current execution code with this new core while keeping backwards compatibility with existing dub projects 2.2 `dub build` consists of two general steps: 1) fetching dependencies 2) building each project in the set { dependencies..., top-level project } in topological order. (Of course there are many other (sub)steps, which I'm mentioning here for simplicity.) Now that in 2.1 we replaced Dub's existing execution code with the new build foundation we have the option to not execute those steps directly, but instead use Dub as a meta build-system which can generate build description for other build systems to execute. 2.3 Allow describing complex builds based on the foundation layer. This can be done either declaratively, directly in dub.json / dub.sdl, or dub can call to a D script that generates a JSON describing the build dependencies that the core described in 1.1 can read. What do you think? Are you willing to help with the design and/or implemention on this initiative?
Apr 16
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 17/04/2019 6:43 PM, Petar Kirov [ZombineDev] wrote:
 On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
 On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d wrote: 
 [...]

 What we really need is a dub replacement that has a different, more 
 flexible design that can accomodate things that people want to do, 
 that the current dub simply cannot due to its design limitations.  
 There are some promising ideas in the other thread about improving 
 dub, and I hope that discussion will eventually lead to a redesign / 
 revamp of dub that makes it more viable for my use case.


 T
IMO, the best course of action is: 1.0. Create a generic foundation based on DAG for describing targets and their relations. This can start as a completely independent project with zero dependencies. 1.1. On top of this core functionality we can add an interoperability layer that allows exporting this abstract representation to other build system formats. 1.2. We can reuse the existing build systems written in D as a build backend. 2.0. Integrate the new build system foundation in Dub 2.0: 2.1 Replace dub's current execution code with this new core while keeping backwards compatibility with existing dub projects 2.2 `dub build` consists of two general steps: 1) fetching dependencies 2) building each project in the set { dependencies..., top-level project } in topological order. (Of course there are many other (sub)steps, which I'm mentioning here for simplicity.) Now that in 2.1 we replaced Dub's existing execution code with the new build foundation we have the option to not execute those steps directly, but instead use Dub as a meta build-system which can generate build description for other build systems to execute. 2.3 Allow describing complex builds based on the foundation layer. This can be done either declaratively, directly in dub.json / dub.sdl, or dub can call to a D script that generates a JSON describing the build dependencies that the core described in 1.1 can read.
That's going to require a ton of work and refactoring. But it would be well worth funding after "1.0".
Apr 16
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 17 April 2019 at 06:43:35 UTC, Petar Kirov 
[ZombineDev] wrote:
 On Tuesday, 16 April 2019 at 16:42:24 UTC, H. S. Teoh wrote:
 On Tue, Apr 16, 2019 at 06:13:30AM +0000, JN via Digitalmars-d 
 wrote: [...]

 What we really need is a dub replacement that has a different, 
 more flexible design that can accomodate things that people 
 want to do, that the current dub simply cannot due to its 
 design limitations.  There are some promising ideas in the 
 other thread about improving dub, and I hope that discussion 
 will eventually lead to a redesign / revamp of dub that makes 
 it more viable for my use case.


 T
IMO, the best course of action is: 1.0. Create a generic foundation based on DAG for describing targets and their relations. This can start as a completely independent project with zero dependencies.
I considered this; I even named the project. However, after I read this: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=2ahUKEwjNgZLl3dbhAhXN66YKHXbEBkEQFjAAegQIBBAC&url=https%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fresearch%2Fuploads%2Fprod%2F2018%2F03%2Fbuild-systems-final.pdf&usg=AOvVaw2j71hXjOoEQLNNjvEOp_RQ I'm not sure just describing a DAG is enough. I've started playing about with the same concepts in D here: https://github.com/atilaneves/rocksteady
 1.1. On top of this core functionality we can add an 
 interoperability layer that allows exporting this abstract 
 representation to other build system formats.
This might not always be possible. First, because of what I said above, and secondly, the feature sets sometimes don't match. I had to give up trying to express some concepts in the reggae tup backend because they're not doable there. And ninja requires modifications to the compiler that I solved by writing a compiler wrapper.
 2.1 Replace dub's current execution code with this new core 
 while keeping backwards compatibility with existing dub projects
I'm already doing that.
 2.2 `dub build` consists of two general steps: 1) fetching 
 dependencies
Which has nothing to do with building and should be done as a separate step.
 2) building each project in the set { dependencies..., 
 top-level project } in topological order.
The builds are independent, they should be done in parallel.
 2.3 Allow describing complex builds based on the foundation 
 layer. This can be done either declaratively, directly in 
 dub.json / dub.sdl, or dub can call to a D script that 
 generates a JSON describing the build dependencies that the 
 core described in 1.1 can read.
This would work for many build requirements, but not all.
 What do you think? Are you willing to help with the design 
 and/or implemention on this initiative?
Already doing it.
Apr 17
parent "Schrom, Brian T" <Brian.Schrom pnnl.gov> writes:
If reimplementing build systems is being investigated, the
jam(perforce)/bjam(b2,boost)/boost.build/rypple searches should dig up a lot of
well thought through issues for very generic and cross platform considerations.
Apr 17
prev sibling parent 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 16 April 2019 at 02:15:13 UTC, Julian wrote:
 On Tuesday, 16 April 2019 at 01:45:26 UTC, 9il wrote:
 Hi all,

 'DUB - call to arms' post by Seb
 https://forum.dlang.org/post/unimbbqxosxcmjdhtzch forum.dlang.org

 Hope we will move other direction and extend Meson build 
 system with better D support and make a new Dlang registry.
Is it just a build system? Isn't a package manager also needed? Why a *new* registry?
The current registry is based on dub.sdl, dub.json. They can be supported as a plugin though.
 Reasons to move forward with Meson:

 1. Has clear python source code: faster to extend, easier to 
 maintain.
Oh. Well it was certainly a mistake to try use D for something like dub. I've thought of D as a kind of competitor to Python but I sure don't want slow to extend, hard to maintain code.
 4. Ability to compile and configure D code with other 
 languages like C/C++.
That's a surprising oversight. Was it intentional?
I mean dub can compile only D sources. Meson can compile integrated projects, where some code is C, some C++, some D and etc. It is a killer feature to involve new companies. .
Apr 15
prev sibling next sibling parent reply Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 16 April 2019 at 01:45:26 UTC, 9il wrote:
 Hi all,

 'DUB - call to arms' post by Seb
 https://forum.dlang.org/post/unimbbqxosxcmjdhtzch forum.dlang.org

 Hope we will move other direction and extend Meson build system 
 with better D support and make a new Dlang registry.

 Reasons to move forward with Meson:

 0. Progressive and large community
 1. Has clear python source code: faster to extend, easier to 
 maintain.
 2. D+Meson already used in production: last year it is a major 
 build system for all of Mir and its commercial applications.
 3. Meson language is super cool. It is much better than dub for 
 commercial projects, for large projects, for library 
 distribution.
 4. Ability to compile and configure D code with other languages 
 like C/C++.
 5. With Meson we don't need to catch up Rust, we can do better 
 than Rust with fewer resources.


 Best,
 Ilya
Don't you think we send following message with this step: You cannot use the D Programming Language without installing Python. Python is able to do things, D is not able to do. In my opinion this is a rather bad advertising): Kind regards Andre
Apr 16
parent 9il <ilyayaroshenko gmail.com> writes:
On Tuesday, 16 April 2019 at 08:25:17 UTC, Andre Pany wrote:
 On Tuesday, 16 April 2019 at 01:45:26 UTC, 9il wrote:

 Don't you think we send following message with this step:
 You cannot use the D Programming Language without installing 
 Python. Python is able to do things, D is not able to do.


 In my opinion this is a rather bad advertising):

 Kind regards
 Andre
Well, bad advertising is advertising, I don't think it is really bad. --Ilya
Apr 16
prev sibling next sibling parent drug <drug2004 bk.ru> writes:
On 16.04.2019 4:45, 9il wrote:
 Hi all,
 
 'DUB - call to arms' post by Seb
 https://forum.dlang.org/post/unimbbqxosxcmjdhtzch forum.dlang.org
 
 Hope we will move other direction and extend Meson build system with 
 better D support and make a new Dlang registry.
 
 Reasons to move forward with Meson:
 
 0. Progressive and large community
 1. Has clear python source code: faster to extend, easier to maintain.
 2. D+Meson already used in production: last year it is a major build 
 system for all of Mir and its commercial applications.
 3. Meson language is super cool. It is much better than dub for 
 commercial projects, for large projects, for library distribution.
 4. Ability to compile and configure D code with other languages like C/C++.
 5. With Meson we don't need to catch up Rust, we can do better than Rust 
 with fewer resources.
 
 
 Best,
 Ilya
Dub suppor for Meson (and other ones) is really useful thing, but it should be optional advanced feature.
Apr 16
prev sibling next sibling parent Andre Pany <andre s-e-a-p.de> writes:
On Tuesday, 16 April 2019 at 01:45:26 UTC, 9il wrote:
 Hi all,

 'DUB - call to arms' post by Seb
 https://forum.dlang.org/post/unimbbqxosxcmjdhtzch forum.dlang.org

 Hope we will move other direction and extend Meson build system 
 with better D support and make a new Dlang registry.

 Reasons to move forward with Meson:

 0. Progressive and large community
 1. Has clear python source code: faster to extend, easier to 
 maintain.
 2. D+Meson already used in production: last year it is a major 
 build system for all of Mir and its commercial applications.
 3. Meson language is super cool. It is much better than dub for 
 commercial projects, for large projects, for library 
 distribution.
 4. Ability to compile and configure D code with other languages 
 like C/C++.
 5. With Meson we don't need to catch up Rust, we can do better 
 than Rust with fewer resources.


 Best,
 Ilya
Just for the records, an alternative registry already exists. You can store your dub packages in Maven (Nexus, Artifactory, ...) and set the Maven url as Dub registry. Works like a charme, Dub does here a perfect job. Kind regards Andre
Apr 16
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 16 April 2019 at 01:45:26 UTC, 9il wrote:
 Hi all,

 'DUB - call to arms' post by Seb
 https://forum.dlang.org/post/unimbbqxosxcmjdhtzch forum.dlang.org

 Hope we will move other direction and extend Meson build system 
 with better D support and make a new Dlang registry.
Why would we need a new registry? Even if we did, how would we convince the existing users that switching would be in their best interests?
 Reasons to move forward with Meson:

 0. Progressive and large community
 1. Has clear python source code: faster to extend, easier to 
 maintain.
This wouldn't be too important since said extension should really only be done once.
 2. D+Meson already used in production: last year it is a major 
 build system for all of Mir and its commercial applications.
My usage of meson with D revealed how bad its dub integration was.
 3. Meson language is super cool. It is much better than dub for 
 commercial projects, for large projects, for library 
 distribution.
Anything is better than dub as a build system, including raw make.
 4. Ability to compile and configure D code with other languages 
 like C/C++.
AFAIK, reggae, CMake and Premake can all do this as well.
 5. With Meson we don't need to catch up Rust, we can do better 
 than Rust with fewer resources.
I don't know what this means. They seem to be doing fine with cargo.
Apr 16
parent Russel Winder <russel winder.org.uk> writes:
On Tue, 2019-04-16 at 12:19 +0000, Atila Neves via Digitalmars-d wrote:
[=E2=80=A6]
=20
 Anything is better than dub as a build system, including raw make.
That's going a bit far, and is fighting talk. But let's not even think about that argument here. :-) [=E2=80=A6]
=20
 AFAIK, reggae, CMake and Premake can all do this as well.
SCons, and I believe Meson, are also able to do mixed language builds very well. I started a Dub add-in to SCons to handle dependencies but I got disheartened by the naming strategy of Dub's build of dependencies and gave up. I'd be happy to restart work on D support for SCons if it had any chance of being used, and SCons Dub support if there was a deterministic algorithm for finding Dub builds. [=E2=80=A6] --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk
Apr 16