www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Best build tool for D projects

reply Chris Molozian <chris cmoz.me> writes:
Hey All,

This is my first post to the mailing list, I'm an avid follower of D's 
development and am currently using it to develop a compiler for my 
thesis work. One of the goals of this stage of the development work is 
to provide a simple build environment to compile the codebase on Linux, 
Windows and Mac OS X. The only complex aspects of the build process is 
compiling the LLVM-D bindings and linking to LLVM.

I'm evaluating build tools for this purpose and have concluded (correct 
me if I'm wrong) that the D-orientated build tools: Bud 
<http://www.dsource.org/projects/build> and DSSS 
<http://www.dsource.org/projects/dsss> are abandoned. I'm not sure 
whether development on xfBuild 
<https://bitbucket.org/h3r3tic/xfbuild/overview> is still going on.

I'd like to use a tool that is easy for testers to install on their 
system (preferably pre-built binaries are available) and use to compile 
my work. I've been looking at C/C++ build tools and have narrowed it 
down to these:

    * Jam (ftjam) <http://www.freetype.org/jam/index.html>,
      cross-platform and platform independent build language. Lots of
      variants with the same name, therefore finding it hard to find
      good tutorials and documentation.
    * Boost.Build (bjam) <http://www.boost.org/boost-build2/>, not sure
      how it differs to ftjam.
    * Cook <http://miller.emu.id.au/pmiller/software/cook/>, can't find
      whether it can be built for use on Windows. No pre-built Windows
      binary. Very extensive documentation, although I think the default
      build file name is silly "Howto.cook" :-) .

After all this preamble I guess what I'm asking is... what (if any) 
cross-platform build tools does everyone use with their D projects? Any 
feedback on experiences with any of the build tools I've mentioned is 
also greatly appreciated. If you can suggest any alternatives, please do.

If you've read this far, thanks for taking the time to read it :-) and 
sorry for the long message.

Cheers,

Chris

PS: I've seen the CMakeD <http://www.dsource.org/projects/cmaked> 
module, I know a lot of people recommend CMake for cross-platform builds 
and that the KDE guys use it. I have tried to like it... but settled on 
hating it. The procedural language is daft and ugly and I loathe the 
CMakeLists.txt file that goes in each directory. I've already ruled it out.
May 13 2011
next sibling parent reply "Eric Poggel (JoeCoder)" <dnewsgroup2 yage3d.net> writes:
On 5/13/2011 1:38 PM, Chris Molozian wrote:

 After all this preamble I guess what I'm asking is... what (if any)
 cross-platform build tools does everyone use with their D projects? Any
 feedback on experiences with any of the build tools I've mentioned is
 also greatly appreciated. If you can suggest any alternatives, please do.


I'm certainly biased because I wrote it, but I use cdc for all of my D 
building.  At one point it was working with dmd1, dmd2, gdc1, and ldc on 
windows and linux (I don't own a mac) with tango or phobos.  But it's 
been a while since I've tested it.

http://dsource.org/projects/cdc

It accepts the same arguments and works the same as dmd except it also 
accepts folder names for arguments, compiling in all .d files it finds 
there.  And it's a single file, making it script friendly.  You can do 
something like this:

dmd -run cdc.d project_folder -of../bin/myproject.exe

cdc.d also has a nicely organized api for easily creating your own build 
script.  I use this with Yage for building derelict into a lib in order 
to reduce compilation time.  See the main function in this file.  The 
rest is unmodified from the cdc project:

http://dsource.org/projects/yage/browser/trunk/build/buildyage.d
May 13 2011
parent reply Chris Molozian <chris cmoz.me> writes:
Thanks for the link to the project, I'll try it out. It's a shame 
there's no documentation for CDC but the Yage project is considerably 
more complex than my needs from a build tool, it's build script will be 
useful to read.

On 13/05/11 21:11, Eric Poggel (JoeCoder) wrote:

 On 5/13/2011 1:38 PM, Chris Molozian wrote:

 After all this preamble I guess what I'm asking is... what (if any)
 cross-platform build tools does everyone use with their D projects? Any
 feedback on experiences with any of the build tools I've mentioned is
 also greatly appreciated. If you can suggest any alternatives, please
 do.


 I'm certainly biased because I wrote it, but I use cdc for all of my D
 building.  At one point it was working with dmd1, dmd2, gdc1, and ldc
 on windows and linux (I don't own a mac) with tango or phobos.  But
 it's been a while since I've tested it.

 http://dsource.org/projects/cdc

 It accepts the same arguments and works the same as dmd except it also
 accepts folder names for arguments, compiling in all .d files it finds
 there.  And it's a single file, making it script friendly.  You can do
 something like this:

 dmd -run cdc.d project_folder -of../bin/myproject.exe

 cdc.d also has a nicely organized api for easily creating your own
 build script.  I use this with Yage for building derelict into a lib
 in order to reduce compilation time.  See the main function in this
 file.  The rest is unmodified from the cdc project:

 http://dsource.org/projects/yage/browser/trunk/build/buildyage.d
May 14 2011
parent "Eric Poggel (JoeCoder)" <dnewsgroup2 yage3d.net> writes:
On 5/14/2011 6:39 AM, Chris Molozian wrote:

 It's a shame there's no documentation for CDC


There's an html documentation file (generated from the source) in the 
download zip file.
May 15 2011
prev sibling next sibling parent reply Trass3r <un known.com> writes:
Am 13.05.2011, 19:38 Uhr, schrieb Chris Molozian <chris cmoz.me>:

 I'm not sure whether development on xfBuild
 <https://bitbucket.org/h3r3tic/xfbuild/overview> is still going on.


Well activity has certainly decreased lately but it still works fine  
(incremental builds regularly crash though, but according to h3r3tic's  
blog that's dmd's fault)
May 13 2011
parent Chris Molozian <chris cmoz.me> writes:
Thanks for the update on xfBuild.

On 13/05/11 21:28, Trass3r wrote:

 Am 13.05.2011, 19:38 Uhr, schrieb Chris Molozian <chris cmoz.me>:

 I'm not sure whether development on xfBuild
 <https://bitbucket.org/h3r3tic/xfbuild/overview> is still going on.


 Well activity has certainly decreased lately but it still works fine 
 (incremental builds regularly crash though, but according to h3r3tic's 
 blog that's dmd's fault)
May 14 2011
prev sibling next sibling parent reply "Nick Sabalausky" <a a.a> writes:
"Chris Molozian" <chris cmoz.me> wrote in message 
news:mailman.137.1305310099.14074.digitalmars-d puremagic.com...

 Hey All,

 This is my first post to the mailing list, I'm an avid follower of D's
 development and am currently using it to develop a compiler for my
 thesis work. One of the goals of this stage of the development work is
 to provide a simple build environment to compile the codebase on Linux,
 Windows and Mac OS X. The only complex aspects of the build process is
 compiling the LLVM-D bindings and linking to LLVM.



Bu(il)d, rebuild and dsss are all old, out-of-date and abandoned.

Xfbuild is abandoned (unless someone else has picked up the reins since the 
original author left), and it doesn't work for D2.

I hate make with a passion, so I won't go near anything that requires make 
(such as CMake).

There are other good build systems that aren't targeted directly at D, like 
Rake, SCons and A-A-P. But A-A-P's documentation is very difficult to 
understand. Rake and SCons introduce a dependency on Ruby and Python, 
respectively. And none of them (to my knowledge) will find dependencies 
automatically, you have to keep track of every single D source file 
individually, which is a pointless pain in the ass.

But there is a good option: RDMD. There's a version of RDMD that comes with 
DMD, but it has a lot of bugs that have been around forever, and some of 
them are real blockers for using RDMD as a general D build tool. So I'd 
recommend using the patched version of RDMD I created:

http://www.dsource.org/projects/semitwist/wiki/WikiStart#RDMDSeemaTadBitrotted

Works great for me.

If you need to do more than just simply build a D executable with a specific 
configuration, I'd recommend using one of the following and just have them 
invoke RDMD instead of invoking DMD directly.

- Rake: I kind of like it despite the fact that it's Ruby instead of D.

- SCons: You use Python. A lot of people like it.

- STBuild: Included in my SemiTwist D Tools project:

http://www.dsource.org/projects/semitwist/

STBuild is nowhere near as fancy or powerful as Rake or SCons, and there's 
no real documentation other than the built-in "--help" screen. But it's 
simple, and it's good if you don't have any custom build steps and all you 
need to do is manage debug and release builds of multiple D-based targets. 
It uses RDMD internally (It can optionally use rebuild or xfbuild instead if 
you really want, and will even convert most commandline parameters between 
rdmd/rebuild/xfbuild). I have to be honest, though, my plan is to completely 
re-write STBuild as a D equivalent of Rake or SCons. But anyway, here are 
some example configuration files for STBuild:

http://www.dsource.org/projects/semitwist/browser/trunk/stbuild.conf
http://www.dsource.org/projects/goldie/browser/trunk/stbuild.conf
May 13 2011
next sibling parent Russel Winder <russel russel.org.uk> writes:
On Fri, 2011-05-13 at 17:56 -0400, Nick Sabalausky wrote:
[ . . . ]

 There are other good build systems that aren't targeted directly at D, li=


ke=20

 Rake, SCons and A-A-P. But A-A-P's documentation is very difficult to=20
 understand. Rake and SCons introduce a dependency on Ruby and Python,=20
 respectively. And none of them (to my knowledge) will find dependencies=


=20

 automatically, you have to keep track of every single D source file=20
 individually, which is a pointless pain in the ass.


SCons capabilities can be fixed if there is some willingness to get
things fixed and for people to use it.  If no-one other than me is ever
going to use the D tool of SCons there is little point in putting
forward a programme of evolution of said D tool -- since it works fine
for my needs just now.

The built-in SCons D tool is a little out of date -- see earlier email
about the latest DMD release -- all development work should be done with
the separate tool fork on BitBucket
https://bitbucket.org/russel/scons_dmd_new

This BitBucket project has an issues store which is undoubtedly the best
way of getting some activity going -- along with people interested in
doing some Python coding to help this D support, forking the repository
making changes and sending in pull requests.  If issues relate directly
to SCons rather than the D tool they can be bounced to the SCons issue
tracker.

[ . . . ]

=20
 - Rake: I kind of like it despite the fact that it's Ruby instead of D.


Rake is very much Make but as an internal DSL with the power of a good
dynamic language.  As far as I am aware there is little support for C, C
++, LaTeX, D, Go, Java, Scala, etc. you have to role everything
yourself.  Ruby, Gems, etc. are very well supported.


 - SCons: You use Python. A lot of people like it.


I have to admit I do.  But then I have a special sort of use
environment.

[ . . . ]

And then there is Waf.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel russel.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
May 14 2011
prev sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 13/05/2011 22:56, Nick Sabalausky wrote:

 Xfbuild is abandoned (unless someone else has picked up the reins since the
 original author left), and it doesn't work for D2.


xfbuild works perfectly well for D2, it just requires D1/Tango to 
compile it. There are pre-compiled binaries available.

I'm using redo to build my D stuff - https://github.com/apenwarr/redo. 
Ideally I'd port it to D then I could write my build scripts in D, I've 
not had chance though.

-- 
Robert
http://octarineparrot.com/
May 14 2011
next sibling parent Robert Clipsham <robert octarineparrot.com> writes:
On 14/05/2011 20:16, Robert Clipsham wrote:

 I'm using redo to build my D stuff - https://github.com/apenwarr/redo.
 Ideally I'd port it to D then I could write my build scripts in D, I've
 not had chance though.


To clarify, you can write build scripts in D using it as it is now - 
they just won't look very pretty.

-- 
Robert
http://octarineparrot.com/
May 14 2011
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"Robert Clipsham" <robert octarineparrot.com> wrote in message 
news:iqmkl7$1m4a$1 digitalmars.com...

 On 13/05/2011 22:56, Nick Sabalausky wrote:

 Xfbuild is abandoned (unless someone else has picked up the reins since 
 the
 original author left), and it doesn't work for D2.


 xfbuild works perfectly well for D2, it just requires D1/Tango to compile 
 it. There are pre-compiled binaries available.


I wasn't able to get it to work on even a trivial main:

https://bitbucket.org/h3r3tic/xfbuild/issue/20/strange-error-on-simple-main-when-using#comment-232209

Always worked fine for me on D1/Tango though (Aside from the occasional 
problem with incremental build that someone already mentioned - but that was 
always trivial enough to fix: just do a clean and a full re-build.)
May 14 2011
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2011-05-13 19:38, Chris Molozian wrote:

 Hey All,

 This is my first post to the mailing list, I'm an avid follower of D's
 development and am currently using it to develop a compiler for my
 thesis work. One of the goals of this stage of the development work is
 to provide a simple build environment to compile the codebase on Linux,
 Windows and Mac OS X. The only complex aspects of the build process is
 compiling the LLVM-D bindings and linking to LLVM.

 I'm evaluating build tools for this purpose and have concluded (correct
 me if I'm wrong) that the D-orientated build tools: Bud
 <http://www.dsource.org/projects/build> and DSSS
 <http://www.dsource.org/projects/dsss> are abandoned. I'm not sure
 whether development on xfBuild
 <https://bitbucket.org/h3r3tic/xfbuild/overview> is still going on.

 I'd like to use a tool that is easy for testers to install on their
 system (preferably pre-built binaries are available) and use to compile
 my work. I've been looking at C/C++ build tools and have narrowed it
 down to these:

     * Jam (ftjam) <http://www.freetype.org/jam/index.html>,
       cross-platform and platform independent build language. Lots of
       variants with the same name, therefore finding it hard to find
       good tutorials and documentation.
     * Boost.Build (bjam) <http://www.boost.org/boost-build2/>, not sure
       how it differs to ftjam.
     * Cook <http://miller.emu.id.au/pmiller/software/cook/>, can't find
       whether it can be built for use on Windows. No pre-built Windows
       binary. Very extensive documentation, although I think the default
       build file name is silly "Howto.cook" :-) .

 After all this preamble I guess what I'm asking is... what (if any)
 cross-platform build tools does everyone use with their D projects? Any
 feedback on experiences with any of the build tools I've mentioned is
 also greatly appreciated. If you can suggest any alternatives, please do.

 If you've read this far, thanks for taking the time to read it :-) and
 sorry for the long message.

 Cheers,

 Chris

 PS: I've seen the CMakeD <http://www.dsource.org/projects/cmaked>
 module, I know a lot of people recommend CMake for cross-platform builds
 and that the KDE guys use it. I have tried to like it... but settled on
 hating it. The procedural language is daft and ugly and I loathe the
 CMakeLists.txt file that goes in each directory. I've already ruled it out.


I'm still using DSSS.

-- 
/Jacob Carlborg
May 15 2011