www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - GNU Make fork with D support

reply Brian <bcallah openbsd.org> writes:
Hello --

Some of the long tail of the dis/assembler thread got me thinking 
about D tooling, or, more precisely, external tools that could 
help aid D usage in other projects.

I stumbled upon this page in the wiki:
https://wiki.dlang.org/Build_systems

And noticed that the only thing that provides plain ol' Makefiles 
is Reggae. But that is listed as alpha software.

So I went ahead and added D support to GNU Make: 
https://github.com/ibara/gmake-d
I am unsure how welcoming such a change would be to upstream, but 
this is an entirely vanilla GNU Make just a few additions (the 
file dlang.diff in the repo will tell you all the changes 
needed). Clone, build, install, and away you go.

~Brian
Apr 23 2021
next sibling parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 23 April 2021 at 12:34:09 UTC, Brian wrote:

 So I went ahead and added D support to GNU Make: 
 https://github.com/ibara/gmake-d
 I am unsure how welcoming such a change would be to upstream, 
 but this is an entirely vanilla GNU Make just a few additions 
 (the file dlang.diff in the repo will tell you all the changes 
 needed). Clone, build, install, and away you go.

 ~Brian


Rather than add these rules and variable definitions to GNU 
`make`'s hard-coded internal lists, it seems like it would be 
much easier to distribute them in a file like `dlang.mk`, which 
users could then `include` into their own makefiles.
Apr 23 2021
parent reply Brian <bcallah openbsd.org> writes:
On Friday, 23 April 2021 at 15:01:01 UTC, Paul Backus wrote:

 On Friday, 23 April 2021 at 12:34:09 UTC, Brian wrote:

 So I went ahead and added D support to GNU Make: 
 https://github.com/ibara/gmake-d
 I am unsure how welcoming such a change would be to upstream, 
 but this is an entirely vanilla GNU Make just a few additions 
 (the file dlang.diff in the repo will tell you all the changes 
 needed). Clone, build, install, and away you go.

 ~Brian


 Rather than add these rules and variable definitions to GNU 
 `make`'s hard-coded internal lists, it seems like it would be 
 much easier to distribute them in a file like `dlang.mk`, which 
 users could then `include` into their own makefiles.


Maybe? In general I don't like those kinds of approaches, but I'd 
be willing to add such things to the repo.

IME, if you give people even the slightest reason not to do 
something, they won't do it. Asking them to include something 
that gmake could have built-in is just going to get people to say 
"I don't wanna use D then, it's not even a part of gmake."

Yes, that could be a criticism of this approach too. But I'd at 
least be willing to try to upstream it if it's something the 
community wants as a means towards amelioration of that criticism.

~Brian
Apr 23 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 23 April 2021 at 16:15:30 UTC, Brian wrote:

 Maybe? In general I don't like those kinds of approaches, but 
 I'd be willing to add such things to the repo.

 IME, if you give people even the slightest reason not to do 
 something, they won't do it. Asking them to include something 
 that gmake could have built-in is just going to get people to 
 say "I don't wanna use D then, it's not even a part of gmake."


My impression is that `make` users are used to having to write 
their own makefiles, even for languages like C and C++ that have 
some built-in support, so I would not expect this to be a big 
deal. And conversely, anyone looking for a build system that 
"just works" out-of-the-box is unlikely to choose `make` to begin 
with.

Regardless, I am not saying that you should not attempt to get 
this merged upstream--but in the interim, I think most D 
programmers would find it much easier to download and `include` a 
makefile fragment than to install an entire separate version of 
`make` from source.
Apr 23 2021
next sibling parent reply Dennis <dkorpel gmail.com> writes:
On Friday, 23 April 2021 at 16:37:15 UTC, Paul Backus wrote:

 Regardless, I am not saying that you should not attempt to get 
 this merged upstream--but in the interim, I think most D 
 programmers would find it much easier to download and `include` 
 a makefile fragment than to install an entire separate version 
 of `make` from source.


+1

My impression of `make` was that it's generating generic target 
files using generic shell commands with generic files as input. I 
didn't know there was special support for languages. That being 
the case, my preference would be:

1. upstream native support
2. additional file with rules for D
3. special D version of `make`

The problem with a separate executable is that it's not only 
extra work for me as the author of a project, but I have to ask 
everyone else who wants to build my code to also install it, 
while a `dlang.mk` can be checked in so the extra work is only 
done once.

In any case, thanks for this work Brian! I hope you can get this 
upstream.
Apr 23 2021
parent reply bachmeier <no spam.net> writes:
On Friday, 23 April 2021 at 17:29:06 UTC, Dennis wrote:

 On Friday, 23 April 2021 at 16:37:15 UTC, Paul Backus wrote:

 Regardless, I am not saying that you should not attempt to get 
 this merged upstream--but in the interim, I think most D 
 programmers would find it much easier to download and 
 `include` a makefile fragment than to install an entire 
 separate version of `make` from source.


 +1

 My impression of `make` was that it's generating generic target 
 files using generic shell commands with generic files as input. 
 I didn't know there was special support for languages. That 
 being the case, my preference would be:

 1. upstream native support
 2. additional file with rules for D
 3. special D version of `make`


I guess my first preference would be the D compilers handling it. 
Just as `dub init` creates a new Dub project, `dmd dlang.mk` 
dumps dlang.mk in the directory and a Makefile that includes it.
Apr 23 2021
parent reply Brian <bcallah openbsd.org> writes:
On Friday, 23 April 2021 at 19:58:53 UTC, bachmeier wrote:

 I guess my first preference would be the D compilers handling 
 it. Just as `dub init` creates a new Dub project, `dmd 
 dlang.mk` dumps dlang.mk in the directory and a Makefile that 
 includes it.


I suspect that makes this a larger conversation which requires 
the enrollment of the compiler writers, possibly on the order of 
a DIP (which I can author if desired).

~Brian
Apr 23 2021
next sibling parent reply bachmeier <no spam.net> writes:
On Friday, 23 April 2021 at 20:24:24 UTC, Brian wrote:

 On Friday, 23 April 2021 at 19:58:53 UTC, bachmeier wrote:

 I guess my first preference would be the D compilers handling 
 it. Just as `dub init` creates a new Dub project, `dmd 
 dlang.mk` dumps dlang.mk in the directory and a Makefile that 
 includes it.


 I suspect that makes this a larger conversation which requires 
 the enrollment of the compiler writers, possibly on the order 
 of a DIP (which I can author if desired).

 ~Brian


I'm not sure how that decision would be made. It shouldn't 
require a DIP since it's not a language change.
Apr 23 2021
parent Brian <bcallah openbsd.org> writes:
On Friday, 23 April 2021 at 22:33:28 UTC, bachmeier wrote:

 On Friday, 23 April 2021 at 20:24:24 UTC, Brian wrote:

 On Friday, 23 April 2021 at 19:58:53 UTC, bachmeier wrote:

 I guess my first preference would be the D compilers handling 
 it. Just as `dub init` creates a new Dub project, `dmd 
 dlang.mk` dumps dlang.mk in the directory and a Makefile that 
 includes it.


 I suspect that makes this a larger conversation which requires 
 the enrollment of the compiler writers, possibly on the order 
 of a DIP (which I can author if desired).

 ~Brian


 I'm not sure how that decision would be made. It shouldn't 
 require a DIP since it's not a language change.


Perhaps I misunderstood the process then, since DIP specifically 
mentions tooling changes, and this would be a tooling change that 
(probably) all the compilers would have to get behind to get much 
tractions.
Apr 23 2021
prev sibling parent Max Haughton <maxhaton gmail.com> writes:
On Friday, 23 April 2021 at 20:24:24 UTC, Brian wrote:

 On Friday, 23 April 2021 at 19:58:53 UTC, bachmeier wrote:

 I guess my first preference would be the D compilers handling 
 it. Just as `dub init` creates a new Dub project, `dmd 
 dlang.mk` dumps dlang.mk in the directory and a Makefile that 
 includes it.


 I suspect that makes this a larger conversation which requires 
 the enrollment of the compiler writers, possibly on the order 
 of a DIP (which I can author if desired).

 ~Brian


Building D code does not require a DIP.


`dmd --smart-makefile-output=dlang.mk` or similar would be a 
better API
Apr 23 2021
prev sibling parent reply Brian <bcallah openbsd.org> writes:
On Friday, 23 April 2021 at 16:37:15 UTC, Paul Backus wrote:

 Regardless, I am not saying that you should not attempt to get 
 this merged upstream--but in the interim, I think most D 
 programmers would find it much easier to download and `include` 
 a makefile fragment than to install an entire separate version 
 of `make` from source.


As I said, willing to accept such files.
Apr 23 2021
parent reply Paul Backus <snarwin gmail.com> writes:
On Friday, 23 April 2021 at 18:40:24 UTC, Brian wrote:

 On Friday, 23 April 2021 at 16:37:15 UTC, Paul Backus wrote:

 Regardless, I am not saying that you should not attempt to get 
 this merged upstream--but in the interim, I think most D 
 programmers would find it much easier to download and 
 `include` a makefile fragment than to install an entire 
 separate version of `make` from source.


 As I said, willing to accept such files.


https://gist.github.com/pbackus/8f9fe3a26eac463af553e22a50476cf0
Apr 23 2021
parent Brian <bcallah openbsd.org> writes:
Added, thanks:
https://github.com/ibara/gmake-d/commit/847345aee0121031f4fe56e31f32293ba3c1ebd6

~Brian
Apr 23 2021
prev sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Friday, 23 April 2021 at 12:34:09 UTC, Brian wrote:

 So I went ahead and added D support to GNU Make: 
 https://github.com/ibara/gmake-d


It looks like it doesn't handle inter-module dependencies yet.

You might be interested in looking at `rdmd`'s `--makedepend` and 
`--makedepfile` switches. These output a module dependency graph 
in a format that can be included into a makefile, thus allowing 
safe and correct rebuilds of D programs.

However, generally, single-object compilation doesn't scale very 
well. This area has been explored quite thoroughly over the past 
two decades, and I think the best middle ground is that which Dub 
uses: build things one package at a time, and then link them 
together. Issues with which object files which template 
instantiations go into, and the quality of generated `.di` files, 
have been the obstacles in significantly improving the building 
of D programs, and I don't think sufficient progress has occurred 
there lately either.
Apr 23 2021
next sibling parent Brian <bcallah openbsd.org> writes:
On Friday, 23 April 2021 at 22:48:56 UTC, Vladimir Panteleev 
wrote:

 On Friday, 23 April 2021 at 12:34:09 UTC, Brian wrote:

 So I went ahead and added D support to GNU Make: 
 https://github.com/ibara/gmake-d


 It looks like it doesn't handle inter-module dependencies yet.


True. It only handles the simple case right now.


 You might be interested in looking at `rdmd`'s `--makedepend` 
 and `--makedepfile` switches. These output a module dependency 
 graph in a format that can be included into a makefile, thus 
 allowing safe and correct rebuilds of D programs.


This is good to know.


 However, generally, single-object compilation doesn't scale 
 very well. This area has been explored quite thoroughly over 
 the past two decades, and I think the best middle ground is 
 that which Dub uses: build things one package at a time, and 
 then link them together. Issues with which object files which 
 template instantiations go into, and the quality of generated 
 `.di` files, have been the obstacles in significantly improving 
 the building of D programs, and I don't think sufficient 
 progress has occurred there lately either.


I agree with you here. But I also generally think that we need to 
"meet people where they are" and not where we want them to be.

~Brian
Apr 23 2021
prev sibling parent Paul Backus <snarwin gmail.com> writes:
On Friday, 23 April 2021 at 22:48:56 UTC, Vladimir Panteleev 
wrote:

 On Friday, 23 April 2021 at 12:34:09 UTC, Brian wrote:

 So I went ahead and added D support to GNU Make: 
 https://github.com/ibara/gmake-d


 It looks like it doesn't handle inter-module dependencies yet.


Neither do any of GNU make's other built-in rules (see e.g. [1]). 
They are not intended to be comprehensive.

[1] 
http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
Apr 23 2021