www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Build using D

reply NaN <divide by.zero> writes:
Why not actually use D as build system for D projects? I assume 
DUB is written in D, but why not just have DUB features in a 
library and use a D build script?

rdmd build.d

Is it a dumb idea?
Apr 16 2020
next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:

 Why not actually use D as build system for D projects? I assume 
 DUB is written in D, but why not just have DUB features in a 
 library and use a D build script?

 rdmd build.d

 Is it a dumb idea?


You might be interested in https://github.com/atilaneves/reggae
Apr 16 2020
parent ted <spam spam.spam> writes:
On 17/4/20 2:47 am, Cym13 wrote:

 On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:

 Why not actually use D as build system for D projects? I assume DUB is 
 written in D, but why not just have DUB features in a library and use 
 a D build script?

 rdmd build.d

 Is it a dumb idea?


 
 You might be interested in https://github.com/atilaneves/reggae


Also have a look at https://github.com/GrahamStJack/bottom-up-build
(used to build/develop mission-critical software)
Apr 16 2020
prev sibling next sibling parent Cogitri <oss cogitri.dev> writes:
On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:

 Why not actually use D as build system for D projects? I assume 
 DUB is written in D, but why not just have DUB features in a 
 library and use a D build script?

 rdmd build.d

 Is it a dumb idea?


Using a fully featured programming language for the build system 
usually leads to everyone doing things in a slightly different 
(and usually very brittle) way. See e.g. build.rs from the Rust 
world - you need different environment variables for each package 
to use system packages instead of using some random version the 
-sys package distributes, half of the packages don't even use 
pkg-config to detect native libs and just assume you have 
everything in the same spot as the distro the developer uses and 
generally everything is prone to breaking if you have a slightly 
different setup than the developers.
Really, I think it the less custom and "clever" a buildsystem is, 
the better.

Personally, I think it's best to just use dub or meson.
Apr 16 2020
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 4/16/20 1:11 PM, NaN wrote:

 Why not actually use D as build system for D projects? I assume DUB is 
 written in D, but why not just have DUB features in a library and use a 
 D build script?
 
 rdmd build.d
 
 Is it a dumb idea?
 
 


DMD doesn't think so:

https://github.com/dlang/dmd/blob/master/src/build.d

And it's fast too. I really like it better than the makefile bs before.

-Steve
Apr 16 2020
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Apr 16, 2020 at 01:45:51PM -0400, Steven Schveighoffer via
Digitalmars-d wrote:

 On 4/16/20 1:11 PM, NaN wrote:

 Why not actually use D as build system for D projects? I assume DUB
 is written in D, but why not just have DUB features in a library and
 use a D build script?
 
 rdmd build.d
 
 Is it a dumb idea?


 
 DMD doesn't think so:
 
 https://github.com/dlang/dmd/blob/master/src/build.d
 
 And it's fast too. I really like it better than the makefile bs
 before.


[...]

Just about *anything* is better than the makefile bs. :-P


T

-- 
Prosperity breeds contempt, and poverty breeds consent. -- Suck.com
Apr 16 2020
prev sibling parent NaN <divide by.zero> writes:
On Thursday, 16 April 2020 at 17:45:51 UTC, Steven Schveighoffer 
wrote:

 On 4/16/20 1:11 PM, NaN wrote:

 Why not actually use D as build system for D projects? I 
 assume DUB is written in D, but why not just have DUB features 
 in a library and use a D build script?
 
 rdmd build.d
 
 Is it a dumb idea?
 
 


 DMD doesn't think so:

 https://github.com/dlang/dmd/blob/master/src/build.d

 And it's fast too. I really like it better than the makefile bs 
 before.


How much of that build.d could be factored out into a separate 
build utilities module? I was thinking not that people would roll 
their own completely custom build script, more like you'd have a 
bunch of utilities and be able to have a concise build script 
that was very flexible and easy to add extra features. So it's 
quick and easy if people want it to be, but can be extended 
easily, plus its D, so it's not another language / bunch of 
rules/quirks to learn.
Apr 16 2020
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:

 Why not actually use D as build system for D projects? I assume 
 DUB is written in D, but why not just have DUB features in a 
 library and use a D build script?

 rdmd build.d

 Is it a dumb idea?


It's a bad idea.

A declarative build description trades being power for being 
conventional and deriving rules that works on every project 
(through "lack of power").

For example, dub test, dub dustmite etc... are impossible with 
powerful immediate build (as opposed to declarative).

Your "unique" build combination has to be justified.
If a standard build system cannot express your program, _perhaps_ 
it's up to you to simplify your build, rather than the build 
system to adapt to unique complexity.

That said, there is always a large class of programs that cannot 
easily be built, that domain is cursed.
Apr 17 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Apr 17, 2020 at 08:24:06AM +0000, Guillaume Piolat via Digitalmars-d
wrote:
[...]

 A declarative build description trades being power for being
 conventional and deriving rules that works on every project (through
 "lack of power").


IMNSHO, this is a flawed argument.  The basics of a build system is a
directed acyclic graph, with build sources, any intermediate targets,
and targets represented by the nodes, and derivation rules represented
by the edges. This model works for every project, but does not trade for
any "lack of power".

The problem with most modern build systems is that instead of tackling
this essential problem structure they try to "simplify" it by tacking on
various assumptions and conventions, which impose artificial limitations
on the system.  Just like the old version control systems like RCS/CVS,
Subversion, etc., they assume a particular workflow / revision history
structure, and therefore when you encounter a case that wasn't
considered in the original design, it's a pain to work around (CVS has
incredibly long times just for creating a branch, for example; SVN is
much better in that it's just a copy, but still requires server
roundtrips, and merges can sometimes be nightmarish).  Modern systems
like Git directly address the underlying structure of a revision graph
-- it's a directed acyclic graph, you see -- and therefore allows
extremely efficient branching/merging, and other graph manipulations
unimaginable in other revision control systems.

It's about time the same thing happened in the build system world.
Something that tackles the underlying DAG directly, and provides modern
guarantees like 100% build reproducibility, ability to handle any task
that fits into the DAG structure,[1] not just some canned operations
like "derive executable from C++ sources", and build times proportional
to the size of change rather than the size of the entire workspace.

I have a website project, for example, where sources include
mathematical model files which get transformed into POVRay input files
that are fed to povray to produce images, then post-processed by
imagemagick to produce .png's which are then installed onto a webserver
-- nowhere is a "compiler" involved but the structure of the task
remains the same: input -> transform -> output.  Another project takes
3D model data as input, and transforms them into runtime data and
accessor code in the form of D source code, that then gets
cross-compiled to ARM and packaged into an APK for Android.

In all of these cases, the details of the input -> output transformation
differs widely, but they nonetheless all fit into the framework of
dependency resolution over a DAG.  This is not even a tough problem like
satisfying versioned dependencies, an NP-complete problem.  Current
technology is well able to handle it with ease.

The only real problem that stands in the way is, IMNSHO, inertia of
developers too used to their aging tools to try something new, and the
lack of a well-known project to propel the prospective build system from
obscurity into the limelight (e.g., without Linux, I honestly doubt most
people would've even heard of git -- Torvalds probably wouldn't have
bothered to write it in the first place).



 For example, dub test, dub dustmite etc... are impossible with
 powerful immediate build (as opposed to declarative).


I don't agree with this.  A "powerful" build system can still be fully
declarative, and amenable to auto-testing, bisecting, etc.. Nothing
about the DAG model requires that your build must be expressed in an
imperative language.  You just have to think outside the box a bit to
arrive at a workable solution.



 Your "unique" build combination has to be justified.  If a standard
 build system cannot express your program, _perhaps_ it's up to you to
 simplify your build, rather than the build system to adapt to unique
 complexity.


I can't agree with this.  Why should I change the structure of my build
just because some build system stuck with a dated paradigm can't handle
its "complexity"?  Nothing can be more elementary that the concept of
transforming a bunch of inputs into some set of outputs via some
specified transformation.  I'm not even asking for something so complex
as solving an NP-complete problem in order to satisfy versioned
dependencies.  If a build system can't handle something as simple as
that, _perhaps_ that build system ought to be replaced with something
more competent! ;-)

(And I note, ironically, that dub *does* include a solver for the
NP-complete problem of satisfying versioned dependencies, *yet* it is
unable to handle something so simple as compiling a helper program from
some subset of input files, running said program to generate some more
source files, and compiling said source files into the final executable.
Note that each of these steps is again a simple input -> transform ->
output step.  Something capable of solving an NP-complete problem ought
to find such a thing beyond trivial!  Yet it cannot because of the
artificial restrictions arbitrarily placed upon it -- *needless*
arbitrary restrictions IMNSHO.)



 That said, there is always a large class of programs that cannot
 easily be built, that domain is cursed.


It's only cursed if one refuses to drop one's outmoded way of thinking
and embrace a better paradigm. ;-)


T

-- 
What's an anagram of "BANACH-TARSKI"?  BANACH-TARSKI BANACH-TARSKI.
Apr 29 2020
parent reply ted <spam spam.spam> writes:
On 30/4/20 2:39 am, H. S. Teoh wrote:

 On Fri, Apr 17, 2020 at 08:24:06AM +0000, Guillaume Piolat via Digitalmars-d
wrote:
 [...]

 A declarative build description trades being power for being
 conventional and deriving rules that works on every project (through
 "lack of power").


 
 IMNSHO, this is a flawed argument.  The basics of a build system is a
 directed acyclic graph, with build sources, any intermediate targets,
 and targets represented by the nodes, and derivation rules represented
 by the edges. This model works for every project, but does not trade for
 any "lack of power".
 


<snip>


 
 It's about time the same thing happened in the build system world.
 Something that tackles the underlying DAG directly, and provides modern
 guarantees like 100% build reproducibility, ability to handle any task
 that fits into the DAG structure,[1] not just some canned operations
 like "derive executable from C++ sources", and build times proportional
 to the size of change rather than the size of the entire workspace.
 


<snip>


I agree 100% - which is the exact approach taken by 
https://github.com/GrahamStJack/bottom-up-build

--ted
Apr 29 2020
parent Panke <tobias pankrath.net> writes:
On Thursday, 30 April 2020 at 05:20:28 UTC, ted wrote:

 I agree 100% - which is the exact approach taken by 
 https://github.com/GrahamStJack/bottom-up-build

 --ted


It's also what bazel does, which has a little bit more support 
behind it. I've started to investigate how it could interact with 
the dub registry, but have not gotten back to it yet.
Apr 30 2020
prev sibling parent guai <guai fake.mail> writes:
On Thursday, 16 April 2020 at 17:11:39 UTC, NaN wrote:

 Is it a dumb idea?


This is what gradle does for java. It's made around java API and 
groovy/kotlin DSL on top of it.
In most cases users do not go beyond some declarative dsl 
describing project name, applying plugins, adding simple tasks 
and their dependencies from standard ones; where files lay, where 
to pull libs from, credentials and so on.
But when this isn't enough one can write their own hooks in 
groovy or kotlin, or even write a plugin.
And I find it better since in fully declarative maven once you 
hit some nonstandard need (which large projects eventually do) 
you fallback to painful programming in xml.
With gradle you just write and debug another program, which you 
already know how to do, just using their API, and not digging 
some alien declarative stuff, and how it actually being converted 
to code.
Apr 30 2020