www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Replacing Make for the DMD build

reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
A direct question to Walter and Andrei really.

If someone, let us say Russel Winder, create a CMake/Ninja and/or
Meson/Ninja build for DMD, is there any chance of it being allowed to
replace the Make system?

If the answer is no, then Russel will obviously not waste his time
doing something that will not be accepted.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 15 2017
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2017-06-16 08:30, Russel Winder via Digitalmars-d wrote:

 A direct question to Walter and Andrei really.
 
 If someone, let us say Russel Winder, create a CMake/Ninja and/or
 Meson/Ninja build for DMD, is there any chance of it being allowed to
 replace the Make system?
 
 If the answer is no, then Russel will obviously not waste his time
 doing something that will not be accepted.


I would guess it's more likely that Make would be replaced by a build 
script written in D than any of the above.

-- 
/Jacob Carlborg
Jun 16 2017
next sibling parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Friday, 16 June 2017 at 07:00:10 UTC, Jacob Carlborg wrote:

 On 2017-06-16 08:30, Russel Winder via Digitalmars-d wrote:

 A direct question to Walter and Andrei really.
 
 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or
 Meson/Ninja build for DMD, is there any chance of it being 
 allowed to
 replace the Make system?
 
 If the answer is no, then Russel will obviously not waste his 
 time
 doing something that will not be accepted.


 I would guess it's more likely that Make would be replaced by a 
 build script written in D than any of the above.


+1!

Along with a simple sh/bat generated script that simply builds 
everything, without checking the dependencies ...

/P
Jun 16 2017
prev sibling next sibling parent bachmeier <no spam.net> writes:
On Friday, 16 June 2017 at 07:00:10 UTC, Jacob Carlborg wrote:

 On 2017-06-16 08:30, Russel Winder via Digitalmars-d wrote:

 A direct question to Walter and Andrei really.
 
 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or
 Meson/Ninja build for DMD, is there any chance of it being 
 allowed to
 replace the Make system?
 
 If the answer is no, then Russel will obviously not waste his 
 time
 doing something that will not be accepted.


 I would guess it's more likely that Make would be replaced by a 
 build script written in D than any of the above.


More likely than that is that there is discussion of writing a 
build script in D, someone starts working on it, and two years 
later there are forum posts about whatever happened to that 
project.

I would much prefer to using an existing build system. There are 
so many higher priorities for the community.
Jun 16 2017
prev sibling parent Seb <seb wilzba.ch> writes:
On Friday, 16 June 2017 at 07:00:10 UTC, Jacob Carlborg wrote:

 On 2017-06-16 08:30, Russel Winder via Digitalmars-d wrote:

 A direct question to Walter and Andrei really.
 
 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or
 Meson/Ninja build for DMD, is there any chance of it being 
 allowed to
 replace the Make system?
 
 If the answer is no, then Russel will obviously not waste his 
 time
 doing something that will not be accepted.


 I would guess it's more likely that Make would be replaced by a 
 build script written in D than any of the above.


This was attempted before:

https://github.com/dlang/phobos/pull/4194
Jun 16 2017
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Friday, 16 June 2017 at 06:30:01 UTC, Russel Winder wrote:

 A direct question to Walter and Andrei really.


While they would ultimately decide this, it's more likely that 
something championed by the D contributors will get in.


 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or Meson/Ninja build for DMD, is there any chance of it 
 being allowed to replace the Make system?


Allow me to veto CMake.  I've been using it with ldc and I hate 
it almost as much as Make.  As for Meson, never dealt with it 
much, do you have an example for D code we can look at?  The 
trivial samples on their site look fine and python is supported 
everywhere we want, but I'd like to see what it'd look like for a 
non-trivial D project.

One issue that came up is that whatever we replace Make with 
would have to generate Makefiles as a fallback, back in the 
previous thread about using Atila's build system, Reggae.


 If the answer is no, then Russel will obviously not waste his 
 time doing something that will not be accepted.


Even if you do the work, like Atila did, you may not get a clear 
answer.  Better to just do it, if you believe in it.  At least 
you can get it merged as an alternative to Make and if people use 
it, the two can be switched over someday.
Jun 16 2017
next sibling parent Suliman <evermind live.ru> writes:
Also looks good https://github.com/jasonwhite/button
Jun 16 2017
prev sibling next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Friday, 16 June 2017 at 13:16:06 UTC, Joakim wrote:

 One issue that came up is that whatever we replace Make with 
 would have to generate Makefiles as a fallback, back in the 
 previous thread about using Atila's build system, Reggae.


I very much support this idea, Reggae is great. Also it supports 
multiple backends including make, ninja, and binary (which means 
no dependency, reggae does it all). It's been arround for long 
enough to be considered stable and quite frankly Attila being 
behind it alone makes me believe it is quality software. It's 
already been used for phobos so we know it can handle at least 
that kind of building complexity. It seems like the obvious 
choice for me if make is to be let go.
Jun 16 2017
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Fri, 2017-06-16 at 19:20 +0000, Cym13 via Digitalmars-d wrote:

 On Friday, 16 June 2017 at 13:16:06 UTC, Joakim wrote:

 One issue that came up is that whatever we replace Make with=20
 would have to generate Makefiles as a fallback, back in the=20
 previous thread about using Atila's build system, Reggae.


=20
 I very much support this idea, Reggae is great. Also it supports=20
 multiple backends including make, ninja, and binary (which means=20
 no dependency, reggae does it all). It's been arround for long=20
 enough to be considered stable and quite frankly Attila being=20
 behind it alone makes me believe it is quality software. It's=20
 already been used for phobos so we know it can handle at least=20
 that kind of building complexity. It seems like the obvious=20
 choice for me if make is to be let go.


Reggae instead of CMake with Ninja sounds a good option. The upside of
CMake is you can use CLion (it uses Make underneath for now but will
soon allow Ninja).

Given D is not well resourced, working with toolchains that are well
resourced is a good move. Hence I think putting effort into IntelliJ-
DLanguage a good idea. Currently it uses Dub, maybe it should support
Reggae.

CLion make C++ programming less painful than it might otherwise be,
Gogland does something similar for Go, likewise IDEA with the Rust
plugin for Rust (uses Cargo). IDEA with IntelliJ-DLanguage has to be a
good move.=20

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 17 2017
prev sibling parent reply Mike Wey <mike-wey example.com> writes:
On 06/16/2017 03:16 PM, Joakim wrote:

 As for Meson, never dealt with it much, do you have an example for D 
 code we can look at?


The meson files for tilix might be a good example:
https://github.com/gnunn1/tilix/tree/master/experimental/meson

-- 
Mike Wey
Jun 16 2017
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 2017-06-17 at 00:07 +0200, Mike Wey via Digitalmars-d wrote:

 On 06/16/2017 03:16 PM, Joakim wrote:

 As for Meson, never dealt with it much, do you have an example for
 D=20
 code we can look at?


=20
 The meson files for tilix might be a good example:
 https://github.com/gnunn1/tilix/tree/master/experimental/meson


I use Meson for building Me TV:

https://github.com/russel/Me-TV

The CMake build is only there so as to use CLion.


=20


--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 17 2017
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/15/2017 11:30 PM, Russel Winder via Digitalmars-d wrote:

 A direct question to Walter and Andrei really.
 
 If someone, let us say Russel Winder, create a CMake/Ninja and/or
 Meson/Ninja build for DMD, is there any chance of it being allowed to
 replace the Make system?
 
 If the answer is no, then Russel will obviously not waste his time
 doing something that will not be accepted.


It's highly unlikely it would be accepted:

1. make is ubiquitous. It's not something we have to scrounge to find on 
platform X, it's already there. People already are familiar with it (even if 
they hate it).

2. we're in the D business, not the project build business. It's easier to get 
past that "first 5 minutes" if everything about D other than D itself is 
familiar and conventional.

3. to steal from Churchill, `make` is the worst form of build system except for 
all the others

4. much as I dislike make, the time spent wrestling with it is a vanishingly 
tiny slice of time compared to what spent on the rest of D. Getting that time 
slice to zero will have no effect on productivity, and I'm not convinced that a 
newer build system will even reduce that time slice at all (see point 5).

5. D has a more complex build process than it should. Using another build
system 
won't make that complexity go away.

6. unlike the choice to use github, there is no clear winner in the `make` 
category of build tools. If the industry has moved on from make to X, then we 
should, too. But it has not.

7. the current makefiles for DMD suffer from over-engineering, i.e. making 
simple things complicated and excessively using obscure features of make. This 
isn't really the fault of make.
Jun 17 2017
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Saturday, 17 June 2017 at 19:20:54 UTC, Walter Bright wrote:

 On 6/15/2017 11:30 PM, Russel Winder via Digitalmars-d wrote:

 A direct question to Walter and Andrei really.
 
 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or
 Meson/Ninja build for DMD, is there any chance of it being 
 allowed to
 replace the Make system?
 
 If the answer is no, then Russel will obviously not waste his 
 time
 doing something that will not be accepted.


 It's highly unlikely it would be accepted:

 1. make is ubiquitous. It's not something we have to scrounge 
 to find on platform X, it's already there. People already are 
 familiar with it (even if they hate it).

 2. we're in the D business, not the project build business. 
 It's easier to get past that "first 5 minutes" if everything 
 about D other than D itself is familiar and conventional.

 3. to steal from Churchill, `make` is the worst form of build 
 system except for all the others

 4. much as I dislike make, the time spent wrestling with it is 
 a vanishingly tiny slice of time compared to what spent on the 
 rest of D. Getting that time slice to zero will have no effect 
 on productivity, and I'm not convinced that a newer build 
 system will even reduce that time slice at all (see point 5).

 5. D has a more complex build process than it should. Using 
 another build system won't make that complexity go away.

 6. unlike the choice to use github, there is no clear winner in 
 the `make` category of build tools. If the industry has moved 
 on from make to X, then we should, too. But it has not.

 7. the current makefiles for DMD suffer from over-engineering, 
 i.e. making simple things complicated and excessively using 
 obscure features of make. This isn't really the fault of make.


A lot of this is simply saying Make is popular so we should just 
stick with it: I hate that mindset.  It is the same mindset D has 
to combat with C or C++, and that was exhibited when you spoke 
against the SDL format for dub.

I understand what you're saying, that D should pick its battles, 
but it's possible to do that and not just go with the status quo 
for everything other than D.  It is often necessary to get rid of 
defaults in many other categories, like Make or XML, and where 
possible D should try to make the best choices, without getting 
too far out there in NIH-land or its own D island.

I don't think choosing an outside Python build system that GNOME 
is in the process of switching to qualifies as either of those, 
as Python is almost as ubiquitous as make.  If we were to use 
Meson, we'd gain a lot in ease of use and lose almost nothing in 
platform availability.

All that said, I'll reiterate what I said earlier to Russel, and 
what I'd say to Atila too: don't aim to replace Make, just aim to 
provide an alternative in the dmd/phobos repos.  If we find that 
everybody is using that instead of Make, we'll just switch over 
to it naturally someday.
Jun 17 2017
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/17/2017 1:25 PM, Joakim wrote:

 A lot of this is simply saying Make is popular so we should just stick with
it: 
 I hate that mindset.  It is the same mindset D has to combat with C or C++,
and 
 that was exhibited when you spoke against the SDL format for dub.


It's not quite the same. I spend 99.99% of my time programming working with 
code, not makefiles. Productivity gains from using a better language have a 
major effect. A better make simply does not.



 D should pick its battles,


That's a good summary of the situation.



 All that said, I'll reiterate what I said earlier to Russel, and what I'd say
to 
 Atila too: don't aim to replace Make, just aim to provide an alternative in
the 
 dmd/phobos repos.  If we find that everybody is using that instead of Make, 
 we'll just switch over to it naturally someday.


Maintaining two parallel build systems is having the downsides of both and the 
benefits of neither. Then there's the endless vim-vs-emacs debate about which 
alternative build system is better. It's not a battle we need to or can afford 
to invest in.

Let's please stay focused on things that will move the needle.
Jun 17 2017
next sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 17 June 2017 at 21:49:29 UTC, Walter Bright wrote:

 It's not quite the same. I spend 99.99% of my time programming 
 working with code, not makefiles.


I'd say somewhere around 10% of the time I've spent on D pull 
requests has been wasted because of the makefiles. It is a 
barrier to entry in new contributors adding new modules.

Though, I think it is just because the ones in there are 
overcomplicated and crappy. My ideal makefile is less than ten 
lines long and I see no reason why dmd, phobos, druntime, and the 
  dlang.org website can't get close to that. Heck, `dmd **/*.d` 
almost actually works...
Jun 17 2017
prev sibling next sibling parent Joakim <dlang joakim.fea.st> writes:
On Saturday, 17 June 2017 at 21:49:29 UTC, Walter Bright wrote:

 On 6/17/2017 1:25 PM, Joakim wrote:

 A lot of this is simply saying Make is popular so we should 
 just stick with it: I hate that mindset.  It is the same 
 mindset D has to combat with C or C++, and that was exhibited 
 when you spoke against the SDL format for dub.


 It's not quite the same. I spend 99.99% of my time programming 
 working with code, not makefiles. Productivity gains from using 
 a better language have a major effect. A better make simply 
 does not.


I agree that most time spent coding doesn't involve the build 
system, but as Adam says, you say that as someone with a long 
history with Make.  For noobs, it represents a barrier to 
contributing, one that we should strive to lower as much as 
possible.


 D should pick its battles,


 That's a good summary of the situation.



 All that said, I'll reiterate what I said earlier to Russel, 
 and what I'd say to Atila too: don't aim to replace Make, just 
 aim to provide an alternative in the dmd/phobos repos.  If we 
 find that everybody is using that instead of Make, we'll just 
 switch over to it naturally someday.


 Maintaining two parallel build systems is having the downsides 
 of both and the benefits of neither. Then there's the endless 
 vim-vs-emacs debate about which alternative build system is 
 better. It's not a battle we need to or can afford to invest in.

 Let's please stay focused on things that will move the needle.


There's an easy way out of that morass.  Anybody who wants to 
submit an alternate build file for their build system must also 
provide a point of contact/support for that build system, so that 
any questions or update requests would be directed at them.

Make would remain the default and you make clear that it's the 
_only_ one officially supported.  Eventually, you prune out all 
the build files that don't work that well and aren't being 
supported by their proponents.  If they like, the project 
contributors can eventually decide to switch over to the new 
build system as the default, if it has been proven for some time 
to both work better and be well-supported.

All this would require you to do nothing, just perhaps make a 
decision someday down the line if and when a build system has 
been proven and others want to make it the default.  Would you be 
okay with starting this pragmatic approach, by okaying the commit 
of a Meson or Reggae build file to these repos and seeing where 
it goes?
Jun 17 2017
prev sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Saturday, 17 June 2017 at 21:49:29 UTC, Walter Bright wrote:

 It's not quite the same. I spend 99.99% of my time programming 
 working with code, not makefiles.


Martin and Sebastian can correct me if I'm wrong, but unless I 
am, Martin, Sebastian, and myself spend a scary amount of time 
wrestling with makefiles. We have had bad production issues due 
to makefiles, such as missing dependencies causing inconsistent 
or broken builds (notably observable when something works only 
without -j), typos or undefined variables resulting in bugs being 
silently ignored (hey, I filed a PR to fix one of those not half 

switched off (which happened more than once!), the win32.mak / 
win64.mak / posix.mak mess, super-ugly hacks due to limitations 
of Make that slow down the build unconditionally and/or make 
everything much more fragile and complicated... and I could go on.

Which is not to say that I think we need to get off makefiles 
ASAP. Migrating to anything else is going to undoubtedly incur a 
massive migration cost and for a long time, a big maintenance 
cost (especially if it involves dogfooding). I agree with many of 
your arguments as well.

I think you're not encountering much of the problems with 
makefiles because you're mainly dealing with DMD, whose build 
process is relatively simple - and even there, the build process 
is mostly maintained these days by other people. In comparison, 
the build process for dlang.org has been getting much more 
complicated with time (partially because of some technical debt 
and fragmentation, but also simply because we want to do more 
things like nice runnable examples, various pre-processing / 
post-processing / validation, building the spec from the compiler 
source, etc.

If anyone wants to SERIOUSLY propose a plan to migrate from 
makefiles, I'd be interested to look at it. However, its benefits 
obviously must outweigh the maintenance costs of current 
makefiles, as well as not raising the contribution barrier too 
much.

Some requirements would be:

- Obvious syntax - making a change such as adding a new target or 
dependency should be obvious just from looking at the existing 
code.

- Performance - shouldn't have considerable overhead; but also, 
it shouldn't take 10 seconds to "rebuild" the build tool itself 
after every modification.

- Cross-platform support - this may seem obvious, but it should 
support at least all platforms DMD supports. One of the 
suggestions for replacement in a similar recent thread 
(http://forum.dlang.org/post/ohkkqj$21lg$1 digitalmars.com), 
buildsome, shows no indication of any Windows support at all, not 
even plans to work on it.

- Minimal dependencies - ideally it should work with as few as 
possible external dependencies that one would typically need to 
install to build D. Windows is very often overlooked when 
evaluating this requirement, for example although Python is 
ubiquitous on POSIX systems, it's much less so on Windows.

That doesn't leave many candidates. reggae might be the closest 
to satisfying all of the above, though I haven't looked too 
closely at it. For one thing, I don't really understand the need 
for build system generators - seems like an extra step and 
imposing an implementation detail on the user.
Jun 17 2017
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 6/17/2017 6:20 PM, Vladimir Panteleev wrote:

 On Saturday, 17 June 2017 at 21:49:29 UTC, Walter Bright wrote:

 It's not quite the same. I spend 99.99% of my time programming working with 
 code, not makefiles.


 
 Martin and Sebastian can correct me if I'm wrong, but unless I am, Martin, 
 Sebastian, and myself spend a scary amount of time wrestling with makefiles.


I spent hours today attempting to figure out why, after I refreshed my copy of 
druntime with HEAD, the druntime unittests would no longer run. I am still at a 
dead end.

The biggest issue with understanding the makefiles is a near total lack of any 
helpful comments. I stepped up a bit with this:

   https://github.com/dlang/druntime/pull/1843

but so far I have not been able to figure out what is going on with the 
druntime/test directory. I don't even know why it exists.

Makefiles need to be documented just like code. Even simple things like what 
variables are inputs to the makefile. What public targets are in the makefile 
(rather than private targets). What those internally defined variables are for, 
and what are their possible values. What the sed scripts are doing. What's the 
difference between SRCS and MANIFEST. Etc.

Make can be a bear to use. But I believe that is mostly due to the convention
of 
never documenting anything in it (I see this commonly, not just for our 
project!). I'm guilty of that myself.

David Held, a while back, did do some work to fix this. You can see the results 
at the beginning of:

   https://github.com/dlang/phobos/blob/master/posix.mak

though it has suffered from years of neglect, barnacles and bit rot.

We must try and do better with this. We can start by refusing to accept
makefile 
PRs that add new constructions without documentation, just like we don't accept 
new functions without Ddoc comments.
Jun 18 2017
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2017-06-18 at 02:31 -0700, Walter Bright via Digitalmars-d
wrote:

 [=E2=80=A6]
=20
 The biggest issue with understanding the makefiles is a near total
 lack of any=20
 helpful comments. I stepped up a bit with this:


Wrong diagnosis. The biggest issue with the makefiles is that they are
hand constructed. There is also the problem that they are makefiles.

CMake =E2=86=92 Ninja
Meson =E2=86=92 Ninja
Raggae =E2=86=92 Ninja

is the 2010s state of the art for build.


 [=E2=80=A6]
=20
 We must try and do better with this. We can start by refusing to
 accept makefile=20
 PRs that add new constructions without documentation, just like we
 don't accept=20
 new functions without Ddoc comments.


Wrong solution to the problem.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 19 2017
prev sibling parent Atila Neves <atila.neves gmail.com> writes:
On Sunday, 18 June 2017 at 01:20:21 UTC, Vladimir Panteleev wrote:

 On Saturday, 17 June 2017 at 21:49:29 UTC, Walter Bright wrote:

 [...]


 Martin and Sebastian can correct me if I'm wrong, but unless I 
 am, Martin, Sebastian, and myself spend a scary amount of time 
 wrestling with makefiles. We have had bad production issues due 
 to makefiles, such as missing dependencies causing inconsistent 
 or broken builds (notably observable when something works only 
 without -j), typos or undefined variables resulting in bugs 
 being silently ignored (hey, I filed a PR to fix one of those 

 completely switched off (which happened more than once!), the 
 win32.mak / win64.mak / posix.mak mess, super-ugly hacks due to 
 limitations of Make that slow down the build unconditionally 
 and/or make everything much more fragile and complicated... and 
 I could go on.

 [...]


reggae has a binary backend: i.e. it produces an executable that 
when run does the build. There are no dependencies on any 
external tools at that point, you only need a D compiler and 
you're in business. Which you already need to build dmd, druntime 
and phobos anyway.

Atila
Jun 19 2017
prev sibling next sibling parent Mike B Johnson <Mikey Ikes.com> writes:
On Saturday, 17 June 2017 at 19:20:54 UTC, Walter Bright wrote:

 On 6/15/2017 11:30 PM, Russel Winder via Digitalmars-d wrote:

 A direct question to Walter and Andrei really.
 
 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or
 Meson/Ninja build for DMD, is there any chance of it being 
 allowed to
 replace the Make system?
 
 If the answer is no, then Russel will obviously not waste his 
 time
 doing something that will not be accepted.


 It's highly unlikely it would be accepted:

 1. make is ubiquitous. It's not something we have to scrounge 
 to find on platform X, it's already there. People already are 
 familiar with it (even if they hate it).

 2. we're in the D business, not the project build business. 
 It's easier to get past that "first 5 minutes" if everything 
 about D other than D itself is familiar and conventional.

 3. to steal from Churchill, `make` is the worst form of build 
 system except for all the others

 4. much as I dislike make, the time spent wrestling with it is 
 a vanishingly tiny slice of time compared to what spent on the 
 rest of D. Getting that time slice to zero will have no effect 
 on productivity, and I'm not convinced that a newer build 
 system will even reduce that time slice at all (see point 5).

 5. D has a more complex build process than it should. Using 
 another build system won't make that complexity go away.

 6. unlike the choice to use github, there is no clear winner in 
 the `make` category of build tools. If the industry has moved 
 on from make to X, then we should, too. But it has not.

 7. the current makefiles for DMD suffer from over-engineering, 
 i.e. making simple things complicated and excessively using 
 obscure features of make. This isn't really the fault of make.


These reasons are terrible.

1. So! Just because something is "universal" doesn't mean it is 
good. Look at any major religion. Usually those that believe in 
it tend to use as proof that there are so many other believers.

2. If you are in the D business you are in the make business.

3. Which, you are saying the exact same as make is the best. Yet, 
those others are just make clones that are not any worse(as they 
make it better). It is a contradictory argument.

4. You are not taking in to account everyone elses time. Sure, 
you might only save a penny, but if 7B people saved a penny, that 
would be 70 million dollars. That is not an insignificant amount.

5. It might, how do you know, have you tried?

6. This is not logical, they haven't moved precisely because they 
are thinking like you are. "If X hasn't moved from Y then why 
should we" everyone exclaims.

7. Then come up with something better instead of making 
excuses... which is what it all boils down to. You could have 
simply said, in a more logical way: "I do not want to spent the 
time to create/switch to something new and better because it is 
not worth it to me"... as that is all you have said(obfuscating 
in to a 7 point bullet list of meaningless il-arguments doesn't 
change that).


My suggestion is, that, if someone else wants to "waste" their 
time creating something that is new and better then they should 
be "allowed" and "supported" in doing so with the only caveat 
that they must prove, at the end of the day, that it is better. 
If it is not, then they "wasted" their time. If it is, then 
everyone is happier and more productive. The question is, are we 
fighting over pennies or millions?
Jun 18 2017
prev sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 2017-06-17 at 12:20 -0700, Walter Bright via Digitalmars-d
wrote:

=20


[=E2=80=A6]

 It's highly unlikely it would be accepted:


So it is highly unlikely I am going to offer to do any work on it then.

I am extremely disappointed in the attitude displayed by your reply, it
shows a lack of interest in doing things better. This is ironic in the
extreme given it is the build infrastructure for a compiler for
programming language supposedly to help make things better for
programmers.


 1. make is ubiquitous. It's not something we have to scrounge to find
 on=20
 platform X, it's already there. People already are familiar with it
 (even if=20
 they hate it).


"is ubiquitous" is a lazy answer. Python is ubiquitous to the same
extent. Of course you still have to install make so actually it isn't
as ubiquitous as the comment implies.


 2. we're in the D business, not the project build business. It's
 easier to get=20
 past that "first 5 minutes" if everything about D other than D itself
 is=20
 familiar and conventional.


You may be in the D business and already know the Make-based framework
intimately, but it is a barrier to anyone else wanting to contribute.
The inference must thus be that you don't want new contributors to the
compiler. That's fine, but a dead end for the future of D.


 3. to steal from Churchill, `make` is the worst form of build system
 except for=20
 all the others


Argumentation by false analogy.


 4. much as I dislike make, the time spent wrestling with it is a
 vanishingly=20
 tiny slice of time compared to what spent on the rest of D. Getting
 that time=20
 slice to zero will have no effect on productivity, and I'm not
 convinced that a=20
 newer build system will even reduce that time slice at all (see point
 5).


As you have no experience you have no data point. Everyone else, who
does have actual data is moaning like crazy about staying with a pure
Make/Shell system.

The world has moved forward in build in the last 40 years, can I
suggest you try and learn a little bit about it.


 5. D has a more complex build process than it should. Using another
 build system=20
 won't make that complexity go away.


Yes it will.


 6. unlike the choice to use github, there is no clear winner in the
 `make`=20
 category of build tools. If the industry has moved on from make to X,
 then we=20
 should, too. But it has not.


Yes it has. I suspect you are focussing too much on the D compiler
internals and not enough on the programmer development environment and
tools progress that has been made in the last 20 years.


 7. the current makefiles for DMD suffer from over-engineering, i.e.
 making=20
 simple things complicated and excessively using obscure features of
 make. This=20
 isn't really the fault of make.


Again faulty reasoning. Replacing a system is a way of getting rid of
crap and getting something better.

I am not a fan of fashion or fadism in software development, but
neither am I a fan of ignoring all progress so as pretend nothing has
changed. There is nothing wrong with a project changing its build
system on a regular, although not too frequent, basis.

D should be there at the cutting edge of software development in its
own environment as an integral part of the pitch to the developers.
Rust and Go have done this =E2=80=93 well Go has a problem with vendoring, =
but
like most of the Go community, we'll pretend it isn't a big problem.

Reggae is D's pitch in the CMake and Meson class of meta-build tools.
Why aren't all the D compiler and tool developments using it? Or if
Reggae is a step too far because it is a D program offering D (or
Python, Lua,=E2=80=A6) specifications of build, use Meson.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 19 2017
next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Monday, 19 June 2017 at 08:06:54 UTC, Russel Winder wrote:

 On Sat, 2017-06-17 at 12:20 -0700, Walter Bright via 
 Digitalmars-d wrote:

 


 […]

 It's highly unlikely it would be accepted:


 So it is highly unlikely I am going to offer to do any work on 
 it then.


If you can present a replacement build infrastructure for D which 
satisfies our needs and shows dramatic, measurable improvements, 
I (and other contributors, I don't doubt) will help push it 
through.

I'm writing this post just as I'm running a bisection to find out 
what broke DMD's win64.mak makefile again.
Jun 19 2017
parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 2017-06-19 at 09:53 +0000, Vladimir Panteleev via Digitalmars-d
wrote:

=20
 [=E2=80=A6]



 If you can present a replacement build infrastructure for D which=20
 satisfies our needs and shows dramatic, measurable improvements,=20
 I (and other contributors, I don't doubt) will help push it=20
 through.


[=E2=80=A6]

So if I do a Reggae and/or Meson build for DMD, is there the possibility
that it will be added to the repository against Walters express wishes?

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder     t:+44 20 7585 2200   voip:sip:
russel.winder ekiga.net
41 Buckmaster Road   m:+44 7770 465 077   xmpp:russel winder.org.uk
London SW11 1EN, UK  w: www.russel.org.uk skype:russel_winder
Jun 19 2017
next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 19 June 2017 at 13:38:52 UTC, Russel Winder wrote:

 On Mon, 2017-06-19 at 09:53 +0000, Vladimir Panteleev via 
 Digitalmars-d wrote:

 
 […]



 If you can present a replacement build infrastructure for D 
 which satisfies our needs and shows dramatic, measurable 
 improvements, I (and other contributors, I don't doubt) will 
 help push it through.


 […]

 So if I do a Reggae and/or Meson build for DMD, is there the 
 possibility that it will be added to the repository against 
 Walters express wishes?


I replicated the _entirety_ [1] of Phobos's posix.mak (that 
Makefile does a _lot_) with reggae and that never happened, so I 
guess the answer is "vanishingly small".


Atila


[1] https://github.com/dlang/phobos/pull/4194
Jun 19 2017
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 2017-06-19 at 13:44 +0000, Atila Neves via Digitalmars-d wrote:

=20


[=E2=80=A6]

 I replicated the _entirety_ [1] of Phobos's posix.mak (that=20
 Makefile does a _lot_) with reggae and that never happened, so I=20
 guess the answer is "vanishingly small".
=20


In the end as long as the D front end, druntime, and phobos get to the LDC
folk so as to create ldc packages for Debian, Fedora, and Homebrew (*), I
don't actually care what sort of pain they have with their workflow. It doe=
s
seem though that the barrier has now been raised, it's not just dmd, it all
of dmsd, druntime, phobos and dlang.org as a big bang addition before even
getting to review. I really can't see me doing that. One repository at a
time maybe, but it sounds as though your experience of Reggae and Phobos is
"DMD developers use Make and deal with the pain". Sad.


(*) I've had to switch from MacPorts to Homebrew on my MBP (**), which is
both sad, and interesting.=20

(**) OSX really is crap compared to Linux/GNOME, how do people stand it?
<rhetorical-question-i-think/>

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder     t:+44 20 7585 2200   voip:sip:
russel.winder ekiga.net
41 Buckmaster Road   m:+44 7770 465 077   xmpp:russel winder.org.uk
London SW11 1EN, UK  w: www.russel.org.uk skype:russel_winder
Jun 19 2017
prev sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Monday, 19 June 2017 at 13:38:52 UTC, Russel Winder wrote:

 On Mon, 2017-06-19 at 09:53 +0000, Vladimir Panteleev via 
 Digitalmars-d wrote:

 
 […]



 If you can present a replacement build infrastructure for D 
 which satisfies our needs and shows dramatic, measurable 
 improvements, I (and other contributors, I don't doubt) will 
 help push it through.


 […]

 So if I do a Reggae and/or Meson build for DMD, is there the 
 possibility that it will be added to the repository against 
 Walters express wishes?


DMD is insufficient, and is not the hardest makefile to port. Any 
serious proposal would need to cover all core repositories - dmd, 
druntime, phobos, tools, and dlang.org. We would need to evaluate 
any proposals thoroughly, and it will likely involve a trial 
period during which both will be maintained in parallel before 
any switch-over occurs.

If we can draw strong conclusions of considerable benefits for 
the replacement, then I think we should be able to reach an 
agreement.

However, be warned: our makefiles are pretty darn tangled. This 
is very likely more than a weekend project.
Jun 19 2017
parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 2017-06-19 at 13:44 +0000, Vladimir Panteleev via Digitalmars-d
wrote:

 [=E2=80=A6]
=20
 DMD is insufficient, and is not the hardest makefile to port. Any=20
 serious proposal would need to cover all core repositories - dmd,=20
 druntime, phobos, tools, and dlang.org. We would need to evaluate=20
 any proposals thoroughly, and it will likely involve a trial=20
 period during which both will be maintained in parallel before=20
 any switch-over occurs.


One doesn't port Makefiles, one writes a new build that achieves the same
final outcomes.

If those 5 are so interconnected why are they in separate repositories? It
should be entirely feasible to replace, and evaluate the replacement of, th=
e
builds of each of the repositories independently.

To demand it is an all or nothing, is to set a bar to high for volunteer
labour.

Interestingly I bet the Make-based build didn't have to undergo such review=
.
;-)


 If we can draw strong conclusions of considerable benefits for=20
 the replacement, then I think we should be able to reach an=20
 agreement.
=20
 However, be warned: our makefiles are pretty darn tangled. This=20
 is very likely more than a weekend project.


That the Makefiles are tangled does not imply a proper build should be.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder     t:+44 20 7585 2200   voip:sip:
russel.winder ekiga.net
41 Buckmaster Road   m:+44 7770 465 077   xmpp:russel winder.org.uk
London SW11 1EN, UK  w: www.russel.org.uk skype:russel_winder
Jun 19 2017
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Monday, 19 June 2017 at 14:15:22 UTC, Russel Winder wrote:

 On Mon, 2017-06-19 at 13:44 +0000, Vladimir Panteleev via 
 Digitalmars-d wrote:

 […]
 
 DMD is insufficient, and is not the hardest makefile to port. 
 Any serious proposal would need to cover all core repositories 
 - dmd, druntime, phobos, tools, and dlang.org. We would need 
 to evaluate any proposals thoroughly, and it will likely 
 involve a trial period during which both will be maintained in 
 parallel before any switch-over occurs.


 One doesn't port Makefiles, one writes a new build that 
 achieves the same final outcomes.


We will likely need to continue providing shim makefiles which 
invoke the replacement build system for the foreseeable future. 
The big problem here is that all make targets are essentially 
part of their public interface, so if we are to avoid breaking 
anyone's scripts or workflow, they must be kept working. 
Otherwise, I don't disagree.


 If those 5 are so interconnected why are they in separate 
 repositories?


Legacy reasons; prohibitively high cost of migrating to a 
monorepo; higher granularity of specialization for maintainers 
and reviewers of various codebase components, as well as 
assigning permissions to them.


 It should be entirely feasible to replace, and evaluate the 
 replacement of, the builds of each of the repositories 
 independently.


The current build system has a number of problems caused by the 
components being in separate repositories. For example, if you 
change a file in Phobos and want to rebuild the documentation 
(dlang.org), the latter won't know that only a single file 
changed in the former, because the dlang.org makefile is not 
aware of the rules governing the building of Phobos files. 
Furthermore, both the dlang.org makefile and the Phobos makefile 
have a target for building Phobos documentation, but they work in 
different ways and produce different results. A replacement build 
system that can achieve correct interoperability as described 
above would score a lot of points towards being accepted.

More importantly, if we accept a proposal for three repos and on 
the fourth one we run into a blocker (or even the replacement 
simply being very unsatisfying), then that would put us in an 
unpleasant situation. So, I think that requiring working 
replacements for all components makes sense as a prerequisite for 
any single component being switched over to the new build system.


 To demand it is an all or nothing, is to set a bar to high for 
 volunteer labour.


My labour is just as volunteer as yours. I'm not demanding that 
someone presents a complete solution; but neither am I going to 
write one from scratch myself. Which is why I said I'd be 
interested in further discussion and proposals.

Will you have some time to chat about Reggae on IRC tomorrow? I 
should be around about 12 hours from now for about 12 hours since 
then.


 Interestingly I bet the Make-based build didn't have to undergo 
 such review. ;-)


It certainly did not. However, migrating from the status quo to 
the status quo is a no-op and has a cost of zero. Here, we need 
to balance the cost of the migration against the benefits of the 
proposal.


 That the Makefiles are tangled does not imply a proper build 
 should be.


There are many moving parts, and we will keep wanting to add more.
Jun 19 2017
next sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 2017-06-19 at 14:52 +0000, Vladimir Panteleev via Digitalmars-d=20
wrote:

=20


[=E2=80=A6]

 We will likely need to continue providing shim makefiles which=C2=A0
 invoke the replacement build system for the foreseeable future.=C2=A0
 The big problem here is that all make targets are essentially=C2=A0
 part of their public interface, so if we are to avoid breaking=C2=A0
 anyone's scripts or workflow, they must be kept working.=C2=A0
 Otherwise, I don't disagree.


The usual approach is to leave the old system in place, put the new
system in place, and run in parallel. Then people can remove their
dependency on the old system over time. Then you deprecate the old
system, and then remove it. The new system does not have to replicate
any part of the old system, it just has to produce the end products.

[=E2=80=A6]

 The current build system has a number of problems caused by the=C2=A0
 components being in separate repositories. For example, if you=C2=A0
 change a file in Phobos and want to rebuild the documentation=C2=A0
 (dlang.org), the latter won't know that only a single file=C2=A0
 changed in the former, because the dlang.org makefile is not=C2=A0
 aware of the rules governing the building of Phobos files.=C2=A0
 Furthermore, both the dlang.org makefile and the Phobos makefile=C2=A0
 have a target for building Phobos documentation, but they work in=C2=A0
 different ways and produce different results. A replacement build=C2=A0
 system that can achieve correct interoperability as described=C2=A0
 above would score a lot of points towards being accepted.
=20
 More importantly, if we accept a proposal for three repos and on=C2=A0
 the fourth one we run into a blocker (or even the replacement=C2=A0
 simply being very unsatisfying), then that would put us in an=C2=A0
 unpleasant situation. So, I think that requiring working=C2=A0
 replacements for all components makes sense as a prerequisite for=C2=A0
 any single component being switched over to the new build system.


GStreamer seems like a good model here as they have dealt with this
exact same situation (well not exact, they do not have the dlang.org
problem in the same way). As part of their trialling Meson to replace
Autotools, they created a build repository that pulls in all the other
repositories such that the whole structure has a well-defined
architecture, all paths are known. Thus you get a nice simplification
that enables a global build as well as individual repository builds.

Of course they have made the decision to trial on the real
repositories, with the option to delete it all and return to using
Autotools. This has a major benefit, it is the real repositories being
worked on and anyone can chip in at any time.

This approach also leads to earlier "buy in" from more people. Though
in GStreamer case Meson is the natural step on from Autotools, so there
is perhaps less argumentation over options, before heading to a trial.

[=E2=80=A6]

 Will you have some time to chat about Reggae on IRC tomorrow? I=C2=A0
 should be around about 12 hours from now for about 12 hours since=C2=A0
 then.


Possibly, what time are you thinking of? I haven't used IRC, well not
in over 20 years anyway, so I would need data on connecting.=20

[=E2=80=A6]
--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 20 2017
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Tuesday, 20 June 2017 at 07:24:26 UTC, Russel Winder wrote:

 The usual approach is to leave the old system in place, put the 
 new system in place, and run in parallel. Then people can 
 remove their dependency on the old system over time. Then you 
 deprecate the old system, and then remove it. The new system 
 does not have to replicate any part of the old system, it just 
 has to produce the end products.


Yep; same as what was done with ddmd.


 Possibly, what time are you thinking of? I haven't used IRC, 
 well not in over 20 years anyway, so I would need data on 
 connecting.


Sorry Russell, I thought I was replying to Atila :) But you are 
of course welcome on IRC. The D channel is on Freenode (#d). If 
you don't want to install an IRC client, there is a web portal at 
http://webchat.freenode.net/.
Jun 20 2017
parent Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 20 June 2017 at 09:08:32 UTC, Vladimir Panteleev 
wrote:

 On Tuesday, 20 June 2017 at 07:24:26 UTC, Russel Winder wrote:

 [...]


 Yep; same as what was done with ddmd.


 [...]


 Sorry Russell, I thought I was replying to Atila :) But you are 
 of course welcome on IRC. The D channel is on Freenode (#d). If 
 you don't want to install an IRC client, there is a web portal 
 at http://webchat.freenode.net/.


In that case, what were you replying? :P

I'm serious, I'm lost now.

Atila
Jun 20 2017
prev sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
It is also worth noting for deployment purposes in package management
systems, that it is important that all the build tools necessary have
to be in the distribution in order for something to be accepted into
the distribution.

So for example Make is acceptable, but Reggae (at least currently)
would not be. Scons is not liked in some circles, for purely ancient
unthinking prejudice reasons :-(, CMake and Meson are very liked.

DMD has not had this problem before because licences meant it would not
be packaged by many distributions. With the new licensing regime maybe
this can change?

On the other hand we have LDC already packaged because, CMake =E2=80=93 why=
 do
people need DMD.

And they will soon have GDC properly, so no need for DMD <slight-
trollish-wording-i-agree/>

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 22 2017
prev sibling parent reply "Nick Sabalausky (Abscissa)" <SeeWebsiteToContactMe semitwist.com> writes:
On 06/19/2017 04:06 AM, Russel Winder via Digitalmars-d wrote:

 
 Reggae is D's pitch in the CMake and Meson class of meta-build tools.
 Why aren't all the D compiler and tool developments using it?


I'm convinced a big part of that is because DUB is ubiquitous and 
incredibly helpful in the D world for package management, but plays very 
poorly with any build system that isn't DUB's internal one.

I've blown enormous amounts of time trying to get my projects (that need 
DUB for dependencies) to use alternate buildsystems without DUB getting 
in the way, and never really succeeded.

A. Figuring out how to obtain all the necessary import paths, link 
paths, etc to USE a DUB-repo-provided package (especially vibe) on the 
compiler command-line, and be sure that I'm actualy getting everything I 
need and that it won't be broken when the dependency is updated...

B. Figuring out how to tell dub "quit trying to compile/link my files 
yourself" without screwing up part "A" above in the process...

...it's just a mess, and despite the rediculous amounts of effort I've 
put into trying to get that all working sanely, even I mostly just gave 
up, just use DUB to build everything (even though it's slow), and shy 
away from attempting anything that's part of the supposedly "0.1%" of 
use-cases (such as including any libs or components that aren't D) which 
DUB, by design, doesn't attempt to address.
Jun 20 2017
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 20 June 2017 at 19:06:05 UTC, Nick Sabalausky 
(Abscissa) wrote:

 On 06/19/2017 04:06 AM, Russel Winder via Digitalmars-d wrote:

 
 Reggae is D's pitch in the CMake and Meson class of meta-build 
 tools.
 Why aren't all the D compiler and tool developments using it?


 I'm convinced a big part of that is because DUB is ubiquitous 
 and incredibly helpful in the D world for package management, 
 but plays very poorly with any build system that isn't DUB's 
 internal one.


While the dub documentation is not always the best, it seems to 
me to be in a better state than reggae's. I've heard about reggae 
a bit on the forum, but I never really made any attempt to try to 
use it. dub seems a lot simpler for small projects.

Maybe Atila could do a blog post with some simple examples and 
compare/contrast with dub?
Jun 20 2017
parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 20 June 2017 at 19:38:14 UTC, jmh530 wrote:

 On Tuesday, 20 June 2017 at 19:06:05 UTC, Nick Sabalausky 
 (Abscissa) wrote:

 On 06/19/2017 04:06 AM, Russel Winder via Digitalmars-d wrote:

 
 Reggae is D's pitch in the CMake and Meson class of 
 meta-build tools.
 Why aren't all the D compiler and tool developments using it?


 I'm convinced a big part of that is because DUB is ubiquitous 
 and incredibly helpful in the D world for package management, 
 but plays very poorly with any build system that isn't DUB's 
 internal one.


 While the dub documentation is not always the best, it seems to 
 me to be in a better state than reggae's. I've heard about 
 reggae a bit on the forum, but I never really made any attempt 
 to try to use it. dub seems a lot simpler for small projects.

 Maybe Atila could do a blog post with some simple examples and 
 compare/contrast with dub?


I'm not the best at documentation. Funnily enough, I made an 
effort with reggae, which might just show how bad I am at this.

There's not much to compare/constrast - dub is a package manager 
that also builds your code, as long as your requirements are 
simple, it doesn't have a DAG. reggae is a build system. You 
wouldn't be able to replace the Makefiles with dub. You _would_ 
be able to build phobos, but that's not all the Makefiles do.

Atila
Jun 21 2017
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 21 June 2017 at 14:11:58 UTC, Atila Neves wrote:

 I'm not the best at documentation. Funnily enough, I made an 
 effort with reggae, which might just show how bad I am at this.


Ha, well maybe ask one of the users then?


 There's not much to compare/constrast - dub is a package 
 manager that also builds your code, as long as your 
 requirements are simple, it doesn't have a DAG. reggae is a 
 build system. You wouldn't be able to replace the Makefiles 
 with dub. You _would_ be able to build phobos, but that's not 
 all the Makefiles do.

 Atila


This is what I was thinking: start with a simple project, show 
how you can build it with dub or with reggae, then show a 
slightly more complicated project that dub cannot handle well and 
show how you can use reggae (and integrate it with dub).
Jun 21 2017
parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 2017-06-21 at 14:48 +0000, jmh530 via Digitalmars-d wrote:

=20


[=E2=80=A6]

 This is what I was thinking: start with a simple project, show=C2=A0
 how you can build it with dub or with reggae, then show a=C2=A0
 slightly more complicated project that dub cannot handle well and=C2=A0
 show how you can use reggae (and integrate it with dub).


Or keep the Dub repository, and ditch dub the tool.

The analogy from the JVM world.

Maven took over from Ant providing a new build tool and a repository of
artefacts. Everyone stared using Maven and Ant effectively passed away
(*). Then came Gradle. It used the Maven repository for artefacts but
replaced Maven the build system which is slowly passing away (*). But
Maven had problems, and so we get Artifactory and Bintray. Maven is
still there people still add artefacts but generally via JCentre (a
Bintray instance) using JCentre as the first search point.

Dub has moved D along a good path, but does it need evolving. Does it
need replacing? I am not a fan of Dub the build system (**) but the
idea of a repository if source packages is a good move. Even C++ now
does this with Conan. But Conan is a package manager not a build
system. Should Dub the package manager emerge from Dub the build
system? Is this possible now?

Clearly the "Big Problem" is that the JVM world has profit-making
entities involved seeking business opportunities. D has very little of
this.

Given that D has very little commercial effort available, minimisation
of goals for improvement are needed. I'd posit:

1. Make using Dub only as a package getter better known and used. Make
it easier for SCons, Meson, CMake, etc. to use Dub the repository
without using Dub the builder.

2. Add Git, Mercurial (, and Bazaar/Breezy ?) repository getting to
Dub, if it is not there, or make it easier to do this if it is.

3. Add some form of review and curating to Dub packages. PyPI is a free
for all and there is a lot of crap and ancient rubbish in it that needs
removing. sadly this will never happen, but at least there is a
strongly opinionated metric publicly known and displayed for each
package that allows you to decide whether a package is just dross.

I am prepared to put time into Dub, if that helps, just as I am putting
time into SCons, and possibly Meson.


(*) A bit like COBOL passed away, sadly there is still a lot of it
about.

(**) Unless it evolves and becomes as good as Cargo is for Rust.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 22 2017
prev sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 2017-06-21 at 14:11 +0000, Atila Neves via Digitalmars-d wrote:

 [=E2=80=A6]
=20
 I'm not the best at documentation. Funnily enough, I made an=C2=A0
 effort with reggae, which might just show how bad I am at this.


Hopefully the era of programmers boasting how crap they are at
documentation is over. Documentation is important to code and to the
uses of code. Some programmers may not be good at writing documentation
in the language required, so get a ghost writer, or at least a sub-
editor.


 There's not much to compare/constrast - dub is a package manager=C2=A0
 that also builds your code, as long as your requirements are=C2=A0
 simple, it doesn't have a DAG. reggae is a build system. You=C2=A0
 wouldn't be able to replace the Makefiles with dub. You _would_=C2=A0
 be able to build phobos, but that's not all the Makefiles do.


Reggae, like CMake and Meson, is a meta-build system, the actual build
is done by Make, Ninja, Tup,=E2=80=A6 We are currently in the era of meta-b=
uild=20
systems.

Having been directly involved in the Gant =E2=86=92 Gradle period of build =
for
the JVM (*) I can state categorically that any build system not using
some form of DAG will be replaced, and fairly quickly. If Dub doesn't
use a DAG, it has managed to last longer than perhaps it should.


(*) Gradle also does native build, because clients of Gradle Inc
demanded it.
--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 22 2017
prev sibling parent Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 2017-06-20 at 15:06 -0400, Nick Sabalausky (Abscissa) via
Digitalmars-d wrote:

 [=E2=80=A6]
=20
 I'm convinced a big part of that is because DUB is ubiquitous and=C2=A0
 incredibly helpful in the D world for package management, but plays
 very=C2=A0
 poorly with any build system that isn't DUB's internal one.


Either Dub needs to be made as good as Cargo is for Rust, so no-one
bothers to use any other build system, or Dub needs to be made a little
more supportive of non-Dub build frameworks.

I could go with the former, but no-one appears to be doing anything
about it.

For the moment I am trying to get SCons to play nicely with Dub as a
package manager, and it sucks, at least using it as a subprocess to get
and build packages.

I am tempted to see if something analogous can be done in Meson.

The status quo is not good, and as they say, something needs to be
done.


 I've blown enormous amounts of time trying to get my projects (that
 need=C2=A0
 DUB for dependencies) to use alternate buildsystems without DUB
 getting=C2=A0
 in the way, and never really succeeded.


SCons + Dub is working fine for me for unit-threaded, but that is about
as far as I have dared go so far.

[=E2=80=A6]

In many ways I would love to simply give up making SCons work with Dub
because Dub is as good for D/C/C++ systems as Cargo is for Rust, but it
isn't. :-(


--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Jun 22 2017
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Friday, 16 June 2017 at 06:30:01 UTC, Russel Winder wrote:

 A direct question to Walter and Andrei really.

 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or Meson/Ninja build for DMD, is there any chance of it 
 being allowed to replace the Make system?

 If the answer is no, then Russel will obviously not waste his 
 time doing something that will not be accepted.


Why?

Why replacing a rock-stable Make with build-system-X that most 
likely adds another dependency. I am with Walter on this one. - 
We should continue using Make unless there is a real need for 
something else. DMD's makefiles are really simple!
Jun 19 2017
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2017-06-19 11:19, Dejan Lekic wrote:


 Why replacing a rock-stable Make with build-system-X that most likely 
 adds another dependency.


Where did you get the rock-stable part from?

http://forum.dlang.org/post/euslavyxzcaclrpiaumd forum.dlang.org

-- 
/Jacob Carlborg
Jun 19 2017
prev sibling parent reply meppl <mephisto nordhoff-online.de> writes:
On Monday, 19 June 2017 at 09:19:32 UTC, Dejan Lekic wrote:

 On Friday, 16 June 2017 at 06:30:01 UTC, Russel Winder wrote:

 A direct question to Walter and Andrei really.

 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or Meson/Ninja build for DMD, is there any chance of it 
 being allowed to replace the Make system?

 If the answer is no, then Russel will obviously not waste his 
 time doing something that will not be accepted.


 Why?

 Why replacing a rock-stable Make with build-system-X that most 
 likely adds another dependency. I am with Walter on this one. - 
 We should continue using Make unless there is a real need for 
 something else. DMD's makefiles are really simple!


is there a point in disallowing several alternate build systems 
residing in the dmd repository?
If it is just allowed to upload README-files and make-files of 
alternate build systems etc, it would not be necessary to waste 
time with this discussion here.
Jun 19 2017
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Monday, June 19, 2017 1:45:27 PM MDT meppl via Digitalmars-d wrote:

 On Monday, 19 June 2017 at 09:19:32 UTC, Dejan Lekic wrote:

 On Friday, 16 June 2017 at 06:30:01 UTC, Russel Winder wrote:

 A direct question to Walter and Andrei really.

 If someone, let us say Russel Winder, create a CMake/Ninja
 and/or Meson/Ninja build for DMD, is there any chance of it
 being allowed to replace the Make system?

 If the answer is no, then Russel will obviously not waste his
 time doing something that will not be accepted.


 Why?

 Why replacing a rock-stable Make with build-system-X that most
 likely adds another dependency. I am with Walter on this one. -
 We should continue using Make unless there is a real need for
 something else. DMD's makefiles are really simple!


 is there a point in disallowing several alternate build systems
 residing in the dmd repository?
 If it is just allowed to upload README-files and make-files of
 alternate build systems etc, it would not be necessary to waste
 time with this discussion here.


Having alternate build systems means maintaining more than one build system.
The main reason that a number of us would like to see make replaced is to
_reduce_ the maintenance requirements, not increase them.

- Jonathan M Davis
Jun 20 2017
parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 20 June 2017 at 21:26:02 UTC, Jonathan M Davis wrote:

 On Monday, June 19, 2017 1:45:27 PM MDT meppl via Digitalmars-d 
 wrote:

 On Monday, 19 June 2017 at 09:19:32 UTC, Dejan Lekic wrote:

 On Friday, 16 June 2017 at 06:30:01 UTC, Russel Winder wrote:

 A direct question to Walter and Andrei really.

 If someone, let us say Russel Winder, create a CMake/Ninja 
 and/or Meson/Ninja build for DMD, is there any chance of it 
 being allowed to replace the Make system?

 If the answer is no, then Russel will obviously not waste 
 his time doing something that will not be accepted.


 Why?

 Why replacing a rock-stable Make with build-system-X that 
 most likely adds another dependency. I am with Walter on 
 this one. - We should continue using Make unless there is a 
 real need for something else. DMD's makefiles are really 
 simple!


 is there a point in disallowing several alternate build systems
 residing in the dmd repository?
 If it is just allowed to upload README-files and make-files of
 alternate build systems etc, it would not be necessary to waste
 time with this discussion here.


 Having alternate build systems means maintaining more than one 
 build system. The main reason that a number of us would like to 
 see make replaced is to _reduce_ the maintenance requirements, 
 not increase them.

 - Jonathan M Davis


No, as I noted earlier in this thread, all you have to do is keep 
the Makefiles up to date, and dump the maintenance burden for 
Meson/reggae build scripts on their proponents, who in turn don't 
have to keep them up to date.  Once the project contributors have 
solid experience with the Make alternatives, you consider making 
a switch.
Jun 20 2017
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 16/06/2017 7:30 AM, Russel Winder via Digitalmars-d wrote:

 A direct question to Walter and Andrei really.
 
 If someone, let us say Russel Winder, create a CMake/Ninja and/or
 Meson/Ninja build for DMD, is there any chance of it being allowed to
 replace the Make system?
 
 If the answer is no, then Russel will obviously not waste his time
 doing something that will not be accepted.


Just an FYI:
Eventually dmd-fe will be split up to be able to work as a library 
(on-going work). At which point in time dub will need to be available 
for use. Now dub should be able to generate any other descriptor that 
you need.

This is where I would put my energy, but it is up to you on where you 
see things going and helps you out the most.
Jun 19 2017