www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - The annoying D build system

reply Jacob Carlborg <doob me.com> writes:
Is it just me that thinks the build system of D (DMD, Phobos, druntime, 
and so on) is really annoying. I'm mostly thinking of the directory 
structure that the makefiles expect.

For example, when building Phobos it will also, automatically, build 
druntime, which it expects to find in the parent directory.

Today I tried to build the dlang.org web site. The first thing that hit 
me was that it tried to build a complete release out of DMD by default. 
Sure that can be consider my fault since I didn't check what targets the 
makefile would run by default. But this makefile will also expect DMD, 
Phobos and so on to be in the parent directory. It will also expect that 
its own folder is named "d-programming-language.org". I renamed that to 
"dlang.org" since it's shorter and that's what the site is called these 
days, but that completely broke the makefile. The makefile is looking 
explicitly for a folder named "d-programming-language.org", i.e. 
"../d-programming-language.org/std.ddoc".

I think it's really annoying, and slightly scary, that the build system 
breaks out of its own directory, doing things in the parent directory 
and expects a certain parent directory structure.

-- 
/Jacob Carlborg
Nov 19 2012
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/19/12, Jacob Carlborg <doob me.com> wrote:

 Is it just me that thinks the build system of D (DMD, Phobos, druntime,
 and so on) is really annoying. I'm mostly thinking of the directory
 structure that the makefiles expect.


Well on win32 I have two directories on my drive, "dmd" and "dmd-git".
"dmd" is an official release, while "dmd-git" is git-head. The dir
structure is:

dmd-git\dmd2\src\dmd  <- git-head
dmd-git\dmd2\src\druntime <- git-head
dmd-git\dmd2\src\phobos <- git-head
dmd-git\dmd2\windows\  ... <- binaries, etc

"dmd-git" is in the PATH env variable before "dmd" so it always
overrides it. When I want to switch between using git-head and an
official release I append an underscore to "dmd-git" (a shortcut key
can do the trick using autohotkey).

It's the simplest system I could think of..
Nov 19 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-11-19 11:34, Andrej Mitrovic wrote:


 Well on win32 I have two directories on my drive, "dmd" and "dmd-git".
 "dmd" is an official release, while "dmd-git" is git-head. The dir
 structure is:

 dmd-git\dmd2\src\dmd  <- git-head
 dmd-git\dmd2\src\druntime <- git-head
 dmd-git\dmd2\src\phobos <- git-head
 dmd-git\dmd2\windows\  ... <- binaries, etc

 "dmd-git" is in the PATH env variable before "dmd" so it always
 overrides it. When I want to switch between using git-head and an
 official release I append an underscore to "dmd-git" (a shortcut key
 can do the trick using autohotkey).

 It's the simplest system I could think of..


I think you misunderstood what I was trying to say, or I am 
misunderstanding you.

-- 
/Jacob Carlborg
Nov 19 2012
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/20/12, Jacob Carlborg <doob me.com> wrote:

 I think you misunderstood what I was trying to say, or I am
 misunderstanding you.


I'm showing the path of least resistance for how to build either
dmd/druntime/phobos/dlang-org


 One solution that instantly pops in my mind is using git submodules.


They are buggy on Windows. The last time I've used them Git started
showing thousands of error messages, crashed and erased my repo and
commit history, but one of the last few commits were left in the
reflog so I managed to recover my files. It also copied the submodule
repo directory into randomly selected folders in an upper directory.

Maybe it's just an msysgit thing, but I'm never touching submodules again.
Nov 20 2012
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-11-20 09:21, Andrej Mitrovic wrote:


 I'm showing the path of least resistance for how to build either
 dmd/druntime/phobos/dlang-org


The build systems for the repositories are still breaking out of their 
top level directory.


 They are buggy on Windows. The last time I've used them Git started
 showing thousands of error messages, crashed and erased my repo and
 commit history, but one of the last few commits were left in the
 reflog so I managed to recover my files. It also copied the submodule
 repo directory into randomly selected folders in an upper directory.

 Maybe it's just an msysgit thing, but I'm never touching submodules again.


I haven't had any problems with submodules so far. I don't really 
understand why they wouldn't work as good on Windows as the other platforms.

-- 
/Jacob Carlborg
Nov 20 2012
prev sibling parent =?ISO-8859-1?Q?Alex_R=F8nne_Petersen?= <alex lycus.org> writes:
On 20-11-2012 09:21, Andrej Mitrovic wrote:

 On 11/20/12, Jacob Carlborg <doob me.com> wrote:

 I think you misunderstood what I was trying to say, or I am
 misunderstanding you.


 I'm showing the path of least resistance for how to build either
 dmd/druntime/phobos/dlang-org


 One solution that instantly pops in my mind is using git submodules.


 They are buggy on Windows. The last time I've used them Git started
 showing thousands of error messages, crashed and erased my repo and
 commit history, but one of the last few commits were left in the
 reflog so I managed to recover my files. It also copied the submodule
 repo directory into randomly selected folders in an upper directory.

 Maybe it's just an msysgit thing, but I'm never touching submodules again.


Er... what?

I certainly cannot say I've had the issues you describe. Submodules have 
always worked flawlessly on Windows for me.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Nov 20 2012
prev sibling next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 11/19/12, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:

 dmd-git\dmd2\src\dmd  <- git-head
 dmd-git\dmd2\src\druntime <- git-head
 dmd-git\dmd2\src\phobos <- git-head
 dmd-git\dmd2\windows\  ... <- binaries, etc


Also:
dmd-git\dmd2\src\d-programming-language.org
Nov 19 2012
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/19/12 3:29 AM, Jacob Carlborg wrote:

 Is it just me that thinks the build system of D (DMD, Phobos, druntime,
 and so on) is really annoying. I'm mostly thinking of the directory
 structure that the makefiles expect.

 For example, when building Phobos it will also, automatically, build
 druntime, which it expects to find in the parent directory.


That's because phobos is dependent on druntime.


 Today I tried to build the dlang.org web site. The first thing that hit
 me was that it tried to build a complete release out of DMD by default.


That is because the current site docs must include the current druntime 
and phobos docs. In turn, those must be built with the currently 
released compiler. To automate all that, the build makes sure the latest 
released compiler is built and then the docs for the latest released 
libs are built.

Before that we were in the situation that updating the site was putting 
whatever was in the Phobos head (or worse, what was on the machine the 
site was built on).


 Sure that can be consider my fault since I didn't check what targets the
 makefile would run by default. But this makefile will also expect DMD,
 Phobos and so on to be in the parent directory. It will also expect that
 its own folder is named "d-programming-language.org". I renamed that to
 "dlang.org" since it's shorter and that's what the site is called these
 days, but that completely broke the makefile. The makefile is looking
 explicitly for a folder named "d-programming-language.org", i.e.
 "../d-programming-language.org/std.ddoc".


That's the name of the github project.


 I think it's really annoying, and slightly scary, that the build system
 breaks out of its own directory, doing things in the parent directory
 and expects a certain parent directory structure.


The makefiles have evolved organically out of clear and simple 
necessities. Also, they could use some love because there was more focus 
on solving specific automation issues, rather than on a globally nice 
system.

So if you have ideas on how to improve the above - great! Just don't 
fall for Chesterton's fence fallacy.


Andrei
Nov 19 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-11-19 21:00, Andrei Alexandrescu wrote:


 That's the name of the github project.


Yeah, but I haven't seen any other project that is actually dependent on 
the name of its own folder.


 So if you have ideas on how to improve the above - great! Just don't
 fall for Chesterton's fence fallacy.


One solution that instantly pops in my mind is using git submodules. 
Create a new repository, say "dlang", add the other repositories as git 
submoduels to "dlang". Then the build system can do whatever it wants in 
the "dlang" directory and its sub directories, but it shouldn't escape 
the "dlang" directory.

Since Make is one of the worst thing I know, I'm not very enthusiastic 
about improving the build system.

-- 
/Jacob Carlborg
Nov 19 2012