digitalmars.D - <rant> D's makefiles... </rant>
- Atila Neves (9/9) Apr 13 2015 I did a `git up` on dmd, druntime and phobos, but phobos wouldn't
- Russel Winder via Digitalmars-d (18/28) Apr 13 2015 Make was a revelation and a revolution, in 1977. Things have progressed...
- deadalnix (7/27) Apr 13 2015 To be fair, I've seen a lot of new build systems, but not a lot
- weaselcat (3/9) Apr 13 2015 For the amount of complaining about make, there sure has been
- Russel Winder via Digitalmars-d (10/13) Apr 15 2015 Isn't that the whole point, something better than make is not make, it
- Paulo Pinto (7/35) Apr 13 2015 Except it is not, unless you are writing pure POSIX makefiles and
- Rikki Cattermole (9/18) Apr 13 2015 Unfortunately druntime has a few issues that means that we can't
- Dicebot (1/1) Apr 13 2015 I want to replace those with D scripts for a reason.
- H. S. Teoh via Digitalmars-d (10/11) Apr 13 2015 Now that dmd is dependent on an existing version of dmd to build, it
- Andrei Alexandrescu (10/18) Apr 13 2015 A thought: any makefile replacement should make sure there's an
- Atila Neves (12/32) Apr 14 2015 The "-j" option should be there for tweaking only, by default I
- Russel Winder via Digitalmars-d (13/17) Apr 14 2015 N-1 not N on a system you are using as a workstation. N on a compiler
- John Colvin (10/23) Apr 14 2015 That really depends on how your process/thread scheduler is set
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (5/17) Apr 14 2015 That's the typical distinction between batch and interactive
- "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= (2/6) Apr 14 2015 RAM.
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (4/10) Apr 14 2015 Well, if RAM is the problem, it is pure coincidence that it
- Jacob Carlborg (5/8) Apr 14 2015 I have four cores (if I recall correctly) and I always compile DMD with
- deadalnix (6/19) Apr 14 2015 Sounds like your scheduler is messing up.
- Andrei Alexandrescu (5/31) Apr 14 2015 Cool. BTW threads should be more than cores there, you have more devices...
- Atila Neves (4/5) Apr 13 2015 Me too. Let's see if we can actually agree on how they'd end up
- deadalnix (3/12) Apr 13 2015 I'd bet you had a dmd.conf left somewhere. I have one, all my
I did a `git up` on dmd, druntime and phobos, but phobos wouldn't build. Weird, but I had to fork phobos anyway, let me do that. Still wouldn't build. I tried `make clean`, can't remember exactly what happened but that didn't work either. In the end I `rm -rf`ed dmd and druntime, recloned them, rebuilt them and _then_ phobos could build. Feels like when I wrote Makefiles by hand, I'd forgotten it could get this bad. Atila
Apr 13 2015
On Mon, 2015-04-13 at 13:12 +0000, Atila Neves via Digitalmars-d wrote:I did a `git up` on dmd, druntime and phobos, but phobos wouldn't=20 build. Weird, but I had to fork phobos anyway, let me do that.=20 Still wouldn't build. I tried `make clean`, can't remember=20 exactly what happened but that didn't work either. =20 In the end I `rm -rf`ed dmd and druntime, recloned them, rebuilt=20 them and _then_ phobos could build. =20 Feels like when I wrote Makefiles by hand, I'd forgotten it could=20 get this bad.Make was a revelation and a revolution, in 1977. Things have progressed in= =20 the last 37 years. Sadly a large number of C, C++, D, etc. programmers seem= =20 to be convinced that build technology reached a pinnacle in 1992 and=20 stopped progressing. =20 --=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D Dr Russel Winder t:+44 20 7585 2200 voip:sip: russel.winder ekiga.net 41 Buckmaster Road m:+44 7770 465 077 xmpp:russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype:russel_winder
Apr 13 2015
On Monday, 13 April 2015 at 13:56:17 UTC, Russel Winder wrote:On Mon, 2015-04-13 at 13:12 +0000, Atila Neves via Digitalmars-d wrote:To be fair, I've seen a lot of new build systems, but not a lot of better build system. I keep coming back to makefile, not because it is good, but because it is not worse than most of the shit I tried, and at least it is available everywhere and you find documentation about it on the net easily for everyone of its obscure corners.I did a `git up` on dmd, druntime and phobos, but phobos wouldn't build. Weird, but I had to fork phobos anyway, let me do that. Still wouldn't build. I tried `make clean`, can't remember exactly what happened but that didn't work either. In the end I `rm -rf`ed dmd and druntime, recloned them, rebuilt them and _then_ phobos could build. Feels like when I wrote Makefiles by hand, I'd forgotten it could get this bad.Make was a revelation and a revolution, in 1977. Things have progressed in the last 37 years. Sadly a large number of C, C++, D, etc. programmers seem to be convinced that build technology reached a pinnacle in 1992 and stopped progressing.
Apr 13 2015
On Monday, 13 April 2015 at 21:27:47 UTC, deadalnix wrote:To be fair, I've seen a lot of new build systems, but not a lot of better build system. I keep coming back to makefile, not because it is good, but because it is not worse than most of the shit I tried, and at least it is available everywhere and you find documentation about it on the net easily for everyone of its obscure corners.For the amount of complaining about make, there sure has been little process in making a better make.
Apr 13 2015
Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> On Mon, 2015-04-13 at 21:35 +0000, weaselcat via Digitalmars-d wrote:[…]For the amount of complaining about make, there sure has been little process in making a better make.Isn't that the whole point, something better than make is not make, it is something else. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 15 2015
On Monday, 13 April 2015 at 21:27:47 UTC, deadalnix wrote:On Monday, 13 April 2015 at 13:56:17 UTC, Russel Winder wrote:Except it is not, unless you are writing pure POSIX makefiles and constraining yourself only to POSIX external commands and their respective arguments. This is exactly why tools like Autoconf came into existence. -- PauloOn Mon, 2015-04-13 at 13:12 +0000, Atila Neves via Digitalmars-d wrote:To be fair, I've seen a lot of new build systems, but not a lot of better build system. I keep coming back to makefile, not because it is good, but because it is not worse than most of the shit I tried, and at least it is available everywhere and you find documentation about it on the net easily for everyone of its obscure corners.I did a `git up` on dmd, druntime and phobos, but phobos wouldn't build. Weird, but I had to fork phobos anyway, let me do that. Still wouldn't build. I tried `make clean`, can't remember exactly what happened but that didn't work either. In the end I `rm -rf`ed dmd and druntime, recloned them, rebuilt them and _then_ phobos could build. Feels like when I wrote Makefiles by hand, I'd forgotten it could get this bad.Make was a revelation and a revolution, in 1977. Things have progressed in the last 37 years. Sadly a large number of C, C++, D, etc. programmers seem to be convinced that build technology reached a pinnacle in 1992 and stopped progressing.
Apr 13 2015
On 14/04/2015 1:12 a.m., Atila Neves wrote:I did a `git up` on dmd, druntime and phobos, but phobos wouldn't build. Weird, but I had to fork phobos anyway, let me do that. Still wouldn't build. I tried `make clean`, can't remember exactly what happened but that didn't work either. In the end I `rm -rf`ed dmd and druntime, recloned them, rebuilt them and _then_ phobos could build. Feels like when I wrote Makefiles by hand, I'd forgotten it could get this bad. AtilaUnfortunately druntime has a few issues that means that we can't outright remake druntime + phobos as dub packages. Which is what we really need to happen. And sooner or later the only things that can't be transferred over are some c libs in phobos, a few assembly files in druntime and of course dmd's backend. Oh and don't forget documentation generation... That won't play nicely with dub egggg.
Apr 13 2015
I want to replace those with D scripts for a reason.
Apr 13 2015
On Mon, Apr 13, 2015 at 02:24:57PM +0000, Dicebot via Digitalmars-d wrote:I want to replace those with D scripts for a reason.Now that dmd is dependent on an existing version of dmd to build, it should be no problem to replace the makefiles with D code. Everytime something like this happens I just see all the flaws of makefiles all over again, but unfortunately some people just refuse the acknowledge that makefiles are going the way of the dinosaur and it's time to move on. T -- Why ask rhetorical questions? -- JC
Apr 13 2015
On 4/13/15 7:34 AM, H. S. Teoh via Digitalmars-d wrote:On Mon, Apr 13, 2015 at 02:24:57PM +0000, Dicebot via Digitalmars-d wrote:A thought: any makefile replacement should make sure there's an equivalent to "-j". My personal feeling: the build is really low on the priority list no matter how you sort it - except "least impact stuff first" :o) - and replacing makefiles with even the perfect solution won't improve anything measurably. Atila: "make clean" should work. Must have been something weird on your system. AndreiI want to replace those with D scripts for a reason.Now that dmd is dependent on an existing version of dmd to build, it should be no problem to replace the makefiles with D code. Everytime something like this happens I just see all the flaws of makefiles all over again, but unfortunately some people just refuse the acknowledge that makefiles are going the way of the dinosaur and it's time to move on.
Apr 13 2015
On Monday, 13 April 2015 at 20:41:20 UTC, Andrei Alexandrescu wrote:On 4/13/15 7:34 AM, H. S. Teoh via Digitalmars-d wrote:The "-j" option should be there for tweaking only, by default I expect a modern tool to just use as many threads as I have cores. Which of course is exactly what Ninja does.On Mon, Apr 13, 2015 at 02:24:57PM +0000, Dicebot via Digitalmars-d wrote:A thought: any makefile replacement should make sure there's an equivalent to "-j".I want to replace those with D scripts for a reason.Now that dmd is dependent on an existing version of dmd to build, it should be no problem to replace the makefiles with D code. Everytime something like this happens I just see all the flaws of makefiles all over again, but unfortunately some people just refuse the acknowledge that makefiles are going the way of the dinosaur and it's time to move on.Atila: "make clean" should work. Must have been something weird on your system.Maybe there was something weird, but said weirdness was caused by the existing build system. If "make clean" is ever needed, it's proof the build system is broken. CMake doesn't even generate a "clean" target. Of course, this is made a lot easier by the fact that CMake encourages and enables out-of-tree builds, so if a clean is needed just delete the build directory. Atila
Apr 14 2015
On Tue, 2015-04-14 at 08:53 +0000, Atila Neves via Digitalmars-d wrote:[…]The "-j" option should be there for tweaking only, by default I expect a modern tool to just use as many threads as I have cores. Which of course is exactly what Ninja does.N-1 not N on a system you are using as a workstation. N on a compiler server, no problem. Waf chooses N and it makes the workstation unusable whilst the compilation is happening. […] -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: russel winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Apr 14 2015
On Tuesday, 14 April 2015 at 12:07:18 UTC, Russel Winder wrote:On Tue, 2015-04-14 at 08:53 +0000, Atila Neves via Digitalmars-d wrote:That really depends on how your process/thread scheduler is set up. I often work with all my cores maxed out, the UI all runs smooth enough, but that's on OS X. On linux I occasionally ran in to a bit of slowness with firefox, but terminals, vim, emacs etc. all worked fine, but then that's with xmonad as a WM, so perhaps my experience isn't typical. Also, I have often got faster compile-times using more more processes than cores, which I put down to disk latency hiding but I could be wrong.[…]The "-j" option should be there for tweaking only, by default I expect a modern tool to just use as many threads as I have cores. Which of course is exactly what Ninja does.N-1 not N on a system you are using as a workstation. N on a compiler server, no problem. Waf chooses N and it makes the workstation unusable whilst the compilation is happening. […]
Apr 14 2015
On Tuesday, 14 April 2015 at 12:07:18 UTC, Russel Winder wrote:On Tue, 2015-04-14 at 08:53 +0000, Atila Neves via Digitalmars-d wrote:That's the typical distinction between batch and interactive processes; today's schedulers shouldn't have trouble handling this. If they do, there's still process priorities/nice values to adjust.[…]The "-j" option should be there for tweaking only, by default I expect a modern tool to just use as many threads as I have cores. Which of course is exactly what Ninja does.N-1 not N on a system you are using as a workstation. N on a compiler server, no problem. Waf chooses N and it makes the workstation unusable whilst the compilation is happening.
Apr 14 2015
On Tuesday, 14 April 2015 at 12:21:19 UTC, Marc Schütz wrote:That's the typical distinction between batch and interactive processes; today's schedulers shouldn't have trouble handling this. If they do, there's still process priorities/nice values to adjust.RAM.
Apr 14 2015
On Tuesday, 14 April 2015 at 13:58:31 UTC, Ola Fosheim Grøstad wrote:On Tuesday, 14 April 2015 at 12:21:19 UTC, Marc Schütz wrote:Well, if RAM is the problem, it is pure coincidence that it should be ok with N-1 jobs, while N is already one too many.That's the typical distinction between batch and interactive processes; today's schedulers shouldn't have trouble handling this. If they do, there's still process priorities/nice values to adjust.RAM.
Apr 14 2015
On 2015-04-14 14:07, Russel Winder via Digitalmars-d wrote:Waf chooses N and it makes the workstation unusable whilst the compilation is happening. […]I have four cores (if I recall correctly) and I always compile DMD with -j 16 without any problems. -- /Jacob Carlborg
Apr 14 2015
On Tuesday, 14 April 2015 at 12:07:18 UTC, Russel Winder wrote:On Tue, 2015-04-14 at 08:53 +0000, Atila Neves via Digitalmars-d wrote:Sounds like your scheduler is messing up. Compiling is CPU intensive when UI isn't. Meaning the compiling process should burns its CPU time while UI process is mostly idle. Meaning UI thread should take over when it needs to, as per most scheduler policies.[…]The "-j" option should be there for tweaking only, by default I expect a modern tool to just use as many threads as I have cores. Which of course is exactly what Ninja does.N-1 not N on a system you are using as a workstation. N on a compiler server, no problem. Waf chooses N and it makes the workstation unusable whilst the compilation is happening. […]
Apr 14 2015
On 4/14/15 1:53 AM, Atila Neves wrote:On Monday, 13 April 2015 at 20:41:20 UTC, Andrei Alexandrescu wrote:Cool. BTW threads should be more than cores there, you have more devices working in parallel than the cores.On 4/13/15 7:34 AM, H. S. Teoh via Digitalmars-d wrote:The "-j" option should be there for tweaking only, by default I expect a modern tool to just use as many threads as I have cores. Which of course is exactly what Ninja does.On Mon, Apr 13, 2015 at 02:24:57PM +0000, Dicebot via Digitalmars-d wrote:A thought: any makefile replacement should make sure there's an equivalent to "-j".I want to replace those with D scripts for a reason.Now that dmd is dependent on an existing version of dmd to build, it should be no problem to replace the makefiles with D code. Everytime something like this happens I just see all the flaws of makefiles all over again, but unfortunately some people just refuse the acknowledge that makefiles are going the way of the dinosaur and it's time to move on.No need to argue against make's shortcomings with me :o). AndreiAtila: "make clean" should work. Must have been something weird on your system.Maybe there was something weird, but said weirdness was caused by the existing build system. If "make clean" is ever needed, it's proof the build system is broken. CMake doesn't even generate a "clean" target. Of course, this is made a lot easier by the fact that CMake encourages and enables out-of-tree builds, so if a clean is needed just delete the build directory.
Apr 14 2015
On Monday, 13 April 2015 at 14:24:58 UTC, Dicebot wrote:I want to replace those with D scripts for a reason.Me too. Let's see if we can actually agree on how they'd end up looking, huh? ;) Atila
Apr 13 2015
On Monday, 13 April 2015 at 13:12:23 UTC, Atila Neves wrote:I did a `git up` on dmd, druntime and phobos, but phobos wouldn't build. Weird, but I had to fork phobos anyway, let me do that. Still wouldn't build. I tried `make clean`, can't remember exactly what happened but that didn't work either. In the end I `rm -rf`ed dmd and druntime, recloned them, rebuilt them and _then_ phobos could build. Feels like when I wrote Makefiles by hand, I'd forgotten it could get this bad. AtilaI'd bet you had a dmd.conf left somewhere. I have one, all my build are broken.
Apr 13 2015