digitalmars.D.learn - How to reduce compile times?
- grauzone (26/26) Mar 21 2009 I'm using dsss (with dmd under Linux) to compile my project, and
- Christopher Wright (4/8) Mar 21 2009 oneatatime = [yes|no]
- grauzone (6/16) Mar 21 2009 Yes, this causes random linker errors.
- Jarrett Billingsley (2/5) Mar 21 2009 -full
- grauzone (7/14) Mar 21 2009 Sorry for being so helpless, but where do I add this to the rebuild
- Clay Smith (3/21) Mar 21 2009 http://svn.dsource.org/projects/arclib/trunk/arclib/arc/dsss.conf
- Jarrett Billingsley (2/6) Mar 22 2009 Why would you recompile if you haven't changed anything? :P
- Robert Fraser (3/10) Mar 23 2009 If you have a build script that invokes the compiler among many other
- grauzone (4/11) Mar 23 2009 I want to build system to recompile the program automatically before I
- torhu (7/23) Mar 21 2009 Those errors shouldn't happen if you compile one file at a time, I
- Brian (8/10) Mar 21 2009 out of curiosity, how much code do you actually have? im using D for
I'm using dsss (with dmd under Linux) to compile my project, and build/compile times are reaching unacceptable highs. What are some tips to speed up the build process? For example, I could imagine that heavy use of templates and CTFE slows the compiler down. Maybe string mixins with CTFE on the module level are most expensive: here, the compiler has to do heavy weight semantic analysis every time a module is imported from another module (maybe even recursively). But I don't really know; it depends too much on how the compiler works internally. Without some kind of "build profiler", I have no clue what is actually causing slow downs. It's like groping in the dark! The only thing that comes near to this seems to be to insert a "pragma(msg, blabla)" at random points in your modules. Then you recompile the project and count how many times the message is outputted. Actually, the whole issue seems to boil down to reducing module dependencies. But then again, your "main" module recursively depends from _all_ used modules in your project. What I personally found useful in practice is to reduce the number of interdependent modules by separating them into interface and implementation modules, e.g. by using delegates or inheritance. Then you can "hide" the implementation from most modules by removing the static dependencies. But this only works in some cases, and is always inconvenient. PS: another thing that possibly would bring a speed gain would be to make dsss compile the whole project in one run, instead of invoking a new dmd process for each source file. How do I need to change the rebuild configuration to achieve this?
Mar 21 2009
grauzone wrote:PS: another thing that possibly would bring a speed gain would be to make dsss compile the whole project in one run, instead of invoking a new dmd process for each source file. How do I need to change the rebuild configuration to achieve this?oneatatime = [yes|no] You want 'no'. This will occasionally produce issues with ModuleInfo not being defined with some dmd versions, I think. Or something like that.
Mar 21 2009
Christopher Wright wrote:grauzone wrote:Yes, this causes random linker errors. What I need is to make dsss completely recompile the project, even if only a single source file was modified. This way, no errors should occur, and it would still be faster than with oneatatime=yes. (Damn that crappy support for incremental compilation.)PS: another thing that possibly would bring a speed gain would be to make dsss compile the whole project in one run, instead of invoking a new dmd process for each source file. How do I need to change the rebuild configuration to achieve this?oneatatime = [yes|no] You want 'no'. This will occasionally produce issues with ModuleInfo not being defined with some dmd versions, I think. Or something like that.
Mar 21 2009
On Sat, Mar 21, 2009 at 2:50 PM, grauzone <none example.net> wrote:What I need is to make dsss completely recompile the project, even if only a single source file was modified. This way, no errors should occur, and it would still be faster than with oneatatime=yes.-full
Mar 21 2009
Jarrett Billingsley wrote:On Sat, Mar 21, 2009 at 2:50 PM, grauzone <none example.net> wrote:Sorry for being so helpless, but where do I add this to the rebuild configfile? Also, I noticed that "dsss build -full" seems to be the way to pass this flag on the command line. But the project is recompiled even when no file was modified at all. This is not good: it should only recompile if something has changed.What I need is to make dsss completely recompile the project, even if only a single source file was modified. This way, no errors should occur, and it would still be faster than with oneatatime=yes.-full
Mar 21 2009
Clay Smith <clayasaurus gmail.com> grauzone wrote:Jarrett Billingsley wrote:http://svn.dsource.org/projects/arclib/trunk/arclib/arc/dsss.conf buildflags = -fullOn Sat, Mar 21, 2009 at 2:50 PM, grauzone <none example.net> wrote:Sorry for being so helpless, but where do I add this to the rebuild configfile? Also, I noticed that "dsss build -full" seems to be the way to pass this flag on the command line. But the project is recompiled even when no file was modified at all. This is not good: it should only recompile if something has changed.What I need is to make dsss completely recompile the project, even if only a single source file was modified. This way, no errors should occur, and it would still be faster than with oneatatime=yes.-full
Mar 21 2009
On Sat, Mar 21, 2009 at 3:45 PM, grauzone <none example.net> wrote:Also, I noticed that "dsss build -full" seems to be the way to pass this flag on the command line. But the project is recompiled even when no file was modified at all. This is not good: it should only recompile if something has changed.Why would you recompile if you haven't changed anything? :P
Mar 22 2009
Jarrett Billingsley wrote:On Sat, Mar 21, 2009 at 3:45 PM, grauzone <none example.net> wrote:If you have a build script that invokes the compiler among many other things, a continuous integration system, etc.Also, I noticed that "dsss build -full" seems to be the way to pass this flag on the command line. But the project is recompiled even when no file was modified at all. This is not good: it should only recompile if something has changed.Why would you recompile if you haven't changed anything? :P
Mar 23 2009
Jarrett Billingsley wrote:On Sat, Mar 21, 2009 at 3:45 PM, grauzone <none example.net> wrote:I want to build system to recompile the program automatically before I execute it. Doing this manually is not an option, because it's annoying and I could forget it.Also, I noticed that "dsss build -full" seems to be the way to pass this flag on the command line. But the project is recompiled even when no file was modified at all. This is not good: it should only recompile if something has changed.Why would you recompile if you haven't changed anything? :P
Mar 23 2009
On 21.03.2009 19:50, grauzone wrote:Christopher Wright wrote:Those errors shouldn't happen if you compile one file at a time, I believe. On the other hand, dsss' incremental compilation feature never seems to work for me.grauzone wrote:Yes, this causes random linker errors.PS: another thing that possibly would bring a speed gain would be to make dsss compile the whole project in one run, instead of invoking a new dmd process for each source file. How do I need to change the rebuild configuration to achieve this?oneatatime = [yes|no] You want 'no'. This will occasionally produce issues with ModuleInfo not being defined with some dmd versions, I think. Or something like that.What I need is to make dsss completely recompile the project, even if only a single source file was modified. This way, no errors should occur, and it would still be faster than with oneatatime=yes. (Damn that crappy support for incremental compilation.)I use bud, which builds everything with a single run of dmd, but uses incremental compilation. If I get linker errors, I just run my cleanup script and try again. Or add -full to bud's command line.
Mar 21 2009
I use bud, which builds everything with a single run of dmd, but uses incremental compilation. If I get linker errors, I just run my cleanup script and try again. Or add -full to bud's command line.In my case, this practically always causes linker errors. Of course I don't know why.
Mar 21 2009
grauzone wrote:In my case, this practically always causes linker errors. Of course I don't know why.You could change the order of object files by linking. if you have -> gdmd bbb.o aaa.o ccc.o -o binrary (-L..... -l....) change to -> gdmd aaa.o bbb.o ccc.o -o binrary (-L..... -l....) or change to -> gdmd aaa.o bbb.o ccc.o aaa.o bbb.o ccc.o -o binrary (-L..... -l....) -- Xu, Qian (stanleyxu) http://stanleyxu2005.blogspot.com
Mar 22 2009
On Sat, 21 Mar 2009 15:44:41 +0100, grauzone wrote:I'm using dsss (with dmd under Linux) to compile my project, and build/compile times are reaching unacceptable highs.out of curiosity, how much code do you actually have? im using D for something with ~12,000 lines of code right now, spread among 40 files or so, with a somewhat excessive use of CTFE and templates all over. a full rebuild takes about 5 seconds with incremental builds taking 1 or 2 seconds in most cases. i just wanted to know what "excessively high" means P.S. using dmd 1.036, rebuild 0.78, phobos, linux
Mar 21 2009
Brian wrote:On Sat, 21 Mar 2009 15:44:41 +0100, grauzone wrote:65906 physical lines of code (+ some Tango .di imports + some small external libraries), maybe 200+ files, takes 1m10s to build when using normal dsss. With -full and oneatatime=no, compile time goes down to 6-7 seconds. It's not that template- and CTFE-heavy. Incremental builds can take relatively long (depending where the changed files are in the dependency tree), and it's really annoying.I'm using dsss (with dmd under Linux) to compile my project, and build/compile times are reaching unacceptable highs.out of curiosity, how much code do you actually have? im using D for something with ~12,000 lines of code right now, spread among 40 files or so, with a somewhat excessive use of CTFE and templates all over. a full rebuild takes about 5 seconds with incremental builds taking 1 or 2 seconds in most cases. i just wanted to know what "excessively high" means P.S. using dmd 1.036, rebuild 0.78, phobos, linux
Mar 21 2009
grauzone wrote:Brian wrote:I think it is normal. try to copy all files to a ram disk and then compile them again. -- Xu, Qian (stanleyxu) http://stanleyxu2005.blogspot.comOn Sat, 21 Mar 2009 15:44:41 +0100, grauzone wrote:65906 physical lines of code (+ some Tango .di imports + some small external libraries), maybe 200+ files, takes 1m10s to build when using normal dsss. With -full and oneatatime=no, compile time goes down to 6-7 seconds. It's not that template- and CTFE-heavy. Incremental builds can take relatively long (depending where the changed files are in the dependency tree), and it's really annoying.I'm using dsss (with dmd under Linux) to compile my project, and build/compile times are reaching unacceptable highs.out of curiosity, how much code do you actually have? im using D for something with ~12,000 lines of code right now, spread among 40 files or so, with a somewhat excessive use of CTFE and templates all over. a full rebuild takes about 5 seconds with incremental builds taking 1 or 2 seconds in most cases. i just wanted to know what "excessively high" means P.S. using dmd 1.036, rebuild 0.78, phobos, linux
Mar 22 2009