www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D based build system to replace current makefile

reply Timothee Cour <thelastmammoth gmail.com> writes:
We should move to a D-based build system to build
dmd/druntime/phobos/d-programming-language/tools.

Reasons, in case it's not obvious:

1) DRY: makefile is full of repetitions
2) cross platform: different makefiles for different architectures are
needed (even win32 vs win64!)
3) safety: makefile happily ignores that $VAR variables are not set, and
can create havoc; likewise tools/update.sh is a bash script and may not
stop on 1st error.
4) tools/update.sh doesn't work out of the box on OSX last i checked (I
made a pull request for that some time back:
https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)

Even a bad D-based build system will be better than a makefile, but it's
not hard to write a reasonable one at least for the particular task of
building dmd/druntime/phobos/d-programming-language/tools. It'll lower the
entry point for people to contribute to dmd/phobos by making rebuilding
trivial.

Here's how it'd work:

1) a mixed D-based/makefile build that'll work 'from scratch' via
bootstrapping:
1a) dmd: first uses a makefile to build dmd+druntime
1b) rdmd built from a D file using dmd
1c) phobos, d-programming-language, tools built from a D file using rdmd

2) a pure D-based build that'll work using an existing dmd binary (eg
distributed binary)
that's the one most will use.
May 30 2013
next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 30 May 2013 00:49:57 -0700
Timothee Cour <thelastmammoth gmail.com> wrote:

 
 Here's how it'd work:
 
 1) a mixed D-based/makefile build that'll work 'from scratch' via
 bootstrapping:
 1a) dmd: first uses a makefile to build dmd+druntime
 1b) rdmd built from a D file using dmd
 1c) phobos, d-programming-language, tools built from a D file using
 rdmd
 


Related:
https://github.com/D-Programming-Language/installer/pull/18#issuecomment-18662826
May 30 2013
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 30 May 2013 06:13:54 -0400
Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> wrote:


 On Thu, 30 May 2013 00:49:57 -0700
 Timothee Cour <thelastmammoth gmail.com> wrote:

 
 Here's how it'd work:
 
 1) a mixed D-based/makefile build that'll work 'from scratch' via
 bootstrapping:
 1a) dmd: first uses a makefile to build dmd+druntime
 1b) rdmd built from a D file using dmd
 1c) phobos, d-programming-language, tools built from a D file using
 rdmd
 


 
 Related:
 https://github.com/D-Programming-Language/installer/pull/18#issuecomment-18662826
 


Erm, I mean: https://github.com/D-Programming-Language/installer/pull/18
May 30 2013
prev sibling next sibling parent reply Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 30 May 2013 00:49:57 -0700
Timothee Cour <thelastmammoth gmail.com> wrote:


 We should move to a D-based build system to build
 dmd/druntime/phobos/d-programming-language/tools.
 
 Reasons, in case it's not obvious:
 
 1) DRY: makefile is full of repetitions
 2) cross platform: different makefiles for different architectures are
 needed (even win32 vs win64!)
 3) safety: makefile happily ignores that $VAR variables are not set,
 and can create havoc; likewise tools/update.sh is a bash script and
 may not stop on 1st error.
 4) tools/update.sh doesn't work out of the box on OSX last i checked
 (I made a pull request for that some time back:
 https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)
 
 Even a bad D-based build system will be better than a makefile, but
 it's not hard to write a reasonable one at least for the particular
 task of building dmd/druntime/phobos/d-programming-language/tools.
 It'll lower the entry point for people to contribute to dmd/phobos by
 making rebuilding trivial.
 
 Here's how it'd work:
 
 1) a mixed D-based/makefile build that'll work 'from scratch' via
 bootstrapping:
 1a) dmd: first uses a makefile to build dmd+druntime
 1b) rdmd built from a D file using dmd
 1c) phobos, d-programming-language, tools built from a D file using
 rdmd
 
 2) a pure D-based build that'll work using an existing dmd binary (eg
 distributed binary)
 that's the one most will use.
 


FWIW, phobos needs to be built before rdmd because rdmd relies on
phobos.

Also, while I like the idea of replacing makefiles with a D solution,
I'm unsure whether it's really a good idea because that may make it
more difficult to bootstrap DMD to run on new platforms.

However, I do think that wrapping the invocations of all the makefiles
is a good idea and would still have the benefit of making it harder to
make mistakes such as (like you said) setting up the %VARs wrong.

In fact I've started working on such stuff already (And DVM can
already compile DMD, too, although it's possible it may still have
some room for improvement).
May 30 2013
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, May 30, 2013 16:47:49 Nick Sabalausky wrote:

 Also, while I like the idea of replacing makefiles with a D solution,
 I'm unsure whether it's really a good idea because that may make it
 more difficult to bootstrap DMD to run on new platforms.


I would point out that we're going to have that problem anyway once we move 
the compiler to D. At some point, already having a fully functional D compiler 
_will_ be required in order to build dmd, druntime, and Phobos. If we switched 
to using D for writing the build scripts before switching the compiler to D, 
we'd simply be introducing that circular dependency earlier. So, that 
shouldn't ultimately be much of a factor in whether we make the switch.

Whether we switch to using D scripts for building dmd, druntime, and Phobos 
should depend on whether the result would be more maintainable and still be 
able to reasonably do everything that we need our build system to do. And 
while many of us really aren't very good at make and find the Makefiles that we 
have to be rather obtuse, folks that are more proficient at make might not 
think that making the switch to a D script would make sense. And I really 
don't know what the tradeoffs would be.

I did go with writing my build script in D for a recent project which was 
generating a document written in ddoc, and it's working out quite well to do 
that, but I don't know whether it ultimately makes sense to do the same for 
the dmd toolchain.

- Jonathan M Davis
May 30 2013
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On Thu, 30 May 2013 16:56:48 -0400
"Jonathan M Davis" <jmdavisProg gmx.com> wrote:

 
 And while many of us really aren't very good at make
 and find the Makefiles that we have to be rather obtuse, folks that
 are more proficient at make might not think that making the switch to
 a D script would make sense. And I really don't know what the
 tradeoffs would be.
 


I normally despise make, but FWIW I do find DMD's makefiles to be
*vastly* nicer than others, probably because they doesn't really use the
DRY-ness features that usually cause make to be such a pain.
May 30 2013
prev sibling parent "Flamaros" <flamaros.xavier gmail.com> writes:
On Thursday, 30 May 2013 at 07:50:11 UTC, Timothee Cour wrote:

 We should move to a D-based build system to build
 dmd/druntime/phobos/d-programming-language/tools.

 Reasons, in case it's not obvious:

 1) DRY: makefile is full of repetitions
 2) cross platform: different makefiles for different 
 architectures are
 needed (even win32 vs win64!)
 3) safety: makefile happily ignores that $VAR variables are not 
 set, and
 can create havoc; likewise tools/update.sh is a bash script and 
 may not
 stop on 1st error.
 4) tools/update.sh doesn't work out of the box on OSX last i 
 checked (I
 made a pull request for that some time back:
 https://github.com/timotheecour/d-programming-language.org/commit/557a2befa74ddfe99ee5c0e12e7c3d028f27d276)

 Even a bad D-based build system will be better than a makefile, 
 but it's
 not hard to write a reasonable one at least for the particular 
 task of
 building dmd/druntime/phobos/d-programming-language/tools. 
 It'll lower the
 entry point for people to contribute to dmd/phobos by making 
 rebuilding
 trivial.

 Here's how it'd work:

 1) a mixed D-based/makefile build that'll work 'from scratch' 
 via
 bootstrapping:
 1a) dmd: first uses a makefile to build dmd+druntime
 1b) rdmd built from a D file using dmd
 1c) phobos, d-programming-language, tools built from a D file 
 using rdmd

 2) a pure D-based build that'll work using an existing dmd 
 binary (eg
 distributed binary)
 that's the one most will use.


You may want take a look at Qbs, it will the next build system of 
Qt in replacement of QMake.
http://qt-project.org/wiki/Qbs-Quick-Reference

Using a full featured language is good cause, there is no 
restriction and build aren't just build, it's also 
packaging/testing issues.
Doing something that just have to call compiler binaries is 
finaly the simple thing, and provide more control to users.
May 30 2013