www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Best Builder to Use

reply eris <jvburnes gmail.com> writes:
Hi all

I'm trying to finish up a 0.9 release of a new open source package that's
taken a fair amount of my spare time lately, but I can't seem to get it past
the final build stage.

- I'm using linux and recent versions of ldc, tango and Descent.  I'm using D
language 1.x.
- I was using simple build scripts, then simple make files, then DSSS, but now
I've run into a problem.
- Part of my code creates a fairly flexible class factory that allows you do
select a component to build at runtime.  This is critical for the dynamic
nature of the app.  The code uses a mixin to configure the factory at compile
time, depending on the "blueprints" that I want the factory to use.
- DSSS can't "see through" the mixin to the dependencies it creates at
compile-time.
- I could manually build/link the unbuilt code, but I might as well go back to
scripts at that point.
- DSSS doesn't look like it's being maintained (which brings up another
subject, I'll avoid here).
- DSSS isn't documented well enough to even figure out if I could  force the
dependencies.

That gives me a few options:

1. Stop using mixins.  If DSSS can't figure it out, maybe it's bad programming
practice.  Not a particularly attractive option for my application.  If
Descent can see the dependencies, why can't DSSS?
2. Use some other method to pre-configure the factories other than manually
coding them every time. (Possible, though mixins seemed to be the best way.)
3. Use a different build engine.  At the moment that looks like 'rebuild',
'bud', 'xfbuild', make, cmaked and a few others.

So this is where I'm at.  The code is just now becoming complex enough that a
complete tool set will be required, so I want to take some time for this
decision.

Here are my thought's so far..

a) xfbuild and dsss are similar.  Perhaps xfbuild could see the dependency.
If I could get a sense of committment of the developer and it's capabilities I
might use it (especially if it worked with linux and ldc).  Is there a way to
force dependencies in DSSS or xfbuild?  If so, I haven't seen them.

b) Auto build tools like DSSS are great when they work, but useless if they
can't see a critical dependency.  Which makes me lean towards 'makefile'
generators like cmake.  If the D module for cmake can't see a particular
dependency, at least I could force the issue in the generated makefile.  Only
issue with cmaked is that it looks like it hasn't been maintained in a long
time either.  How many changes to D have occurred since then?

c) Plain makefiles by hand.  Sometimes the most obvious is the best and I
could probably get by using a makefile for a little longer, but it's not
particularly conducive to rapid development right now.

In short, I don't mind hacking together a toolset, but there comes a time when
you are spending more time hunting down and hacking tools than programming
your application.  It's all fun, but I'd like to keep up a modicum of
productivity.

Any thoughts?

BTW: For all of the development I've done so far, I'd like to thank the devs
and tool teams that have held up so far...

a) Walter, of course for the language.  It's amazing what he's been able to do
without funding from a major corp for this work.
b) The LDC/LLVM team.  Wonderful compiler.  It would be nice if it was
available for the Windows guys (maybe it is now).
c) Team Tango.  You have such a wide variety of tools and algorithms that
occasionally I write an obscure little class only to find out you've already
written it for me.
d) everyone else who contributes.  D is a great language.  Everything we
contribute adds to the critical mass.
e) is there any way in which we can get the more critical core tools funded at
least a little bit?  Maybe some sort of "street performer" protocol,
donations, university tenure for core D developers, Google summer of code
contributions?
Mar 12 2010
next sibling parent Trass3r <un known.com> writes:
 - DSSS doesn't look like it's being maintained (which brings up another
 subject, I'll avoid here).


Nope, it's abandoned. The dependencies are determined by rebuild which is  
based on stone age dmd code.


 a) xfbuild and dsss are similar.  Perhaps xfbuild could see the  
 dependency.
 If I could get a sense of committment of the developer and it's  
 capabilities I
 might use it (especially if it worked with linux and ldc).  Is there a  
 way to
 force dependencies in DSSS or xfbuild?  If so, I haven't seen them.


xfBuild determines the dependencies by using a special file created by dmd.
So if dmd is able to detect them, xfBuild will compile them as well.
Mar 12 2010
prev sibling next sibling parent reply "Andrei Alexandrescu" <SeeWebsiteForEmail erdani.org> writes:
On Fri, 12 Mar 2010 11:20:25 -0600, eris <jvburnes gmail.com> wrote:


 Hi all

 I'm trying to finish up a 0.9 release of a new open source package that's
 taken a fair amount of my spare time lately, but I can't seem to get it  
 past
 the final build stage.

 - I'm using linux and recent versions of ldc, tango and Descent.  I'm  
 using D
 language 1.x.
 - I was using simple build scripts, then simple make files, then DSSS,  
 but now
 I've run into a problem.
 - Part of my code creates a fairly flexible class factory that allows  
 you do
 select a component to build at runtime.  This is critical for the dynamic
 nature of the app.  The code uses a mixin to configure the factory at  
 compile
 time, depending on the "blueprints" that I want the factory to use.
 - DSSS can't "see through" the mixin to the dependencies it creates at
 compile-time.


Nice work! If you think the process of figuring out the mixin can be  
automated, you may want to post it as a bug to dmd's bugzilla and I'll try  
to have rdmd support it.

Andrei
Mar 12 2010
parent reply eris <jvburnes gmail.com> writes:
Andrei,

I don't see why you couldn't see through it, unless you weren't expanding the
mixin.

The component factory pulls it blueprints from a directory path created by the
mixin.  This path is imported in the namespace of the factory, thus limiting the
factory to creating components from a specific set of blueprints.

Even though I haven't really cleaned up the code for an "embarrassment-free"
release, here is the specific class:

// creates a runtime component factory from "blueprint.all"
// creates a catalog hash of the module's classinfo also (better way of doing
this?)

// mixin template
template ComponentFactory(char[] blueprint)
{
	const char[] ComponentFactory = `

		class `~blueprint~`Factory
		{
			import dendrite.component.` ~ blueprint ~ `.all;
			HashMap!(char[],ClassInfo) catalog;
			this(char[] blueprint="` ~ blueprint ~ `")
			{
				catalog = new HashMap!(char[],ClassInfo);
				auto minfo = new ModuleInfo();
				foreach (mod; minfo)
				{
					if (mod.name == blueprint) {
						debug (1) Stdout("creating component catalog:").newline;
						foreach (classInfo; mod.localClasses)
						{
							catalog[classInfo.name] = classInfo;
							debug (1) Stdout("  ")(classInfo.name).newline;
						}
					}
				}
			}

			Component createComponent(char[] name)
			{
				debug (2) Stdout("creating component: ")(name).newline;
				return cast(Component)catalog[name].create();
			}
		}
	`;
}

// in-line declaration

mixin(ComponentFactory!("Minimal"));

// usage

// create the factory

factory = new MinimalFactory();

// use it to create a component

auto intGenerator = factory.createComponent("IntStreamer");

// attach it to the simulation backplane

backplane.activate(intGenerator);
Mar 12 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 03/12/2010 12:10 PM, eris wrote:

 Andrei,

 I don't see why you couldn't see through it, unless you weren't expanding the
mixin.


[snip]

Thanks. One more question - does

dmd -v your_root_module

list the appropriate imports?


Andrei
Mar 12 2010
parent reply eris <jvburnes gmail.com> writes:
Andrei,

Here's the relevant output

---
import    dendrite.process.model.IMailbox      
(dendrite/process/model/IMailbox.d)
import    dendrite.process.model.IBackplane    
(dendrite/process/model/IBackplane.d)
import    dendrite.process.MicroProcess (dendrite/process/MicroProcess.d)
import    dendrite.component.Component  (dendrite/component/Component.d)
import    dendrite.component.model.IComponent
(dendrite/component/model/IComponent.d)
import    dendrite.process.scheduler.model.IScheduler
(dendrite/process/scheduler/model/IScheduler.d)
import    dendrite.process.scheduler.RoundRobin
(dendrite/process/scheduler/RoundRobin.d)
import    dendrite.component.ComponentFactory
(dendrite/component/ComponentFactory.d)
import    dendrite.reactor.EvController (dendrite/reactor/EvController.d)
import    tango.stdc.posix.termios     
(/usr/include/d/tango/stdc/posix/termios.d)
import    ev.d  (ev/d.d)
import    ev.c  (ev/c.d)
semantic  blane_demo
import    dendrite.component.Minimal.all       
(dendrite/component/Minimal/all.d)
import    dendrite.component.Minimal.IntStreamer
(dendrite/component/Minimal/IntStreamer.d)
---

As you can see, soon after the ComponentFactory, dmd does a semantic pass and
deduces that it needs to import/build the Minimal blueprints (at this time only
IntStreamer).

So it looks like dmd is prepared to build it.  When DSSS passed the objects to
the
gcc link stage the only thing it couldn't resolve was the IntStreamer component.

eris
Mar 12 2010
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 03/12/2010 02:19 PM, eris wrote:

 Andrei,

 Here's the relevant output

 ---
 import    dendrite.process.model.IMailbox      
(dendrite/process/model/IMailbox.d)
 import    dendrite.process.model.IBackplane    
(dendrite/process/model/IBackplane.d)
 import    dendrite.process.MicroProcess (dendrite/process/MicroProcess.d)
 import    dendrite.component.Component  (dendrite/component/Component.d)
 import    dendrite.component.model.IComponent
 (dendrite/component/model/IComponent.d)
 import    dendrite.process.scheduler.model.IScheduler
 (dendrite/process/scheduler/model/IScheduler.d)
 import    dendrite.process.scheduler.RoundRobin
 (dendrite/process/scheduler/RoundRobin.d)
 import    dendrite.component.ComponentFactory
 (dendrite/component/ComponentFactory.d)
 import    dendrite.reactor.EvController (dendrite/reactor/EvController.d)
 import    tango.stdc.posix.termios     
(/usr/include/d/tango/stdc/posix/termios.d)
 import    ev.d  (ev/d.d)
 import    ev.c  (ev/c.d)
 semantic  blane_demo
 import    dendrite.component.Minimal.all       
(dendrite/component/Minimal/all.d)
 import    dendrite.component.Minimal.IntStreamer
 (dendrite/component/Minimal/IntStreamer.d)
 ---

 As you can see, soon after the ComponentFactory, dmd does a semantic pass and
 deduces that it needs to import/build the Minimal blueprints (at this time only
 IntStreamer).

 So it looks like dmd is prepared to build it.  When DSSS passed the objects to
the
 gcc link stage the only thing it couldn't resolve was the IntStreamer
component.

 eris


In that case I think you may want to give rdmd a shot. Run this:

rdmd --build-only your_root_module

If something doesn't work, it may be worth a bug report.


Andrei
Mar 12 2010
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
eris:

 b) The LDC/LLVM team.  Wonderful compiler.  It would be nice if it was
 available for the Windows guys (maybe it is now).


It's a wonderful compiler indeed. I invite every one to try it few times.
There is no Windows version unless someone writes the exceptions for Windows
for LLVM. LLVM is mostly worked on for Macs/Linuxes.



 e) is there any way in which we can get the more critical core tools funded at
 least a little bit?  Maybe some sort of "street performer" protocol,
 donations, university tenure for core D developers, Google summer of code
 contributions?


Google summer of Code is a good idea. So I suggest Walter and Andrei (and other
people) to write few proposals of "D2 jobs" to send to Google. Something a
little more complex than just fixing a bug, but not as much work as writing a
new GC, and not as hard as designing a nice unique/lend system for the D type
system. Something that can be done in two months by a not overly intelligent
person.

Things like:
- adding Windows exceptions to LLVM;
- adding runtime integral overflows to D (LLVM has safe integral operations
already);
- creating a good D2 version of LDC (there is some work done on this already);
- Adding pinned/unpinned objects to D GC and its type system;
- Adding good Actors/Agents or transactional memory to Phobos2;
- Introducing staged compilation in the dmd front-end to perform the CTFE;
- etc.

Bye,
bearophile
Mar 12 2010
prev sibling parent reply "Nick Sabalausky" <a a.a> writes:
"eris" <jvburnes gmail.com> wrote in message 
news:hndt4p$303g$1 digitalmars.com...

 3. Use a different build engine.  At the moment that looks like 'rebuild',
 'bud', 'xfbuild', make, cmaked and a few others.


'rebuild' is the part of dsss. When you use dsss to build something, you're 
using rebuild...which is abandoned. Plus, the best version of rebuild is 
0.76 (and not 0.78) which is difficult to get ahold of (extremely difficult 
for linux).

Like Trass3r said, xfbuild will do what you want just fine. I use it.

I don't know much about 'bud', other than it's pretty old, but some people 
use it (I know bearophile does), so I assume it must have some virtues.

As far as 'make', well, personally, I hate all forms of 'make' with a 
passion (but that's just me).

As alternatives to 'make', there's also A-A-P, Python SCons and Ruby Rake. 
None of them will do automatic D dependencies (SCons and Rake definitely 
don't, and I don't think A-A-P does, but I could be wrong), but you can use 
them to do much more than just compile D stuff (and you can use them to 
invoke 'xfbuild' if you want). A-A-P has a lot of good points, although I 
personally find the documentation confusing as hell, so I don't really 
understand how to use it particularly well. SCons and Rake, IMO beat the 
pants off of make, or at least if you're willing to write the build scripts 
in Python or Ruby and require your users to have it installed.
Mar 12 2010
next sibling parent eris <jvburnes gmail.com> writes:
 Like Trass3r said, xfbuild will do what you want just fine. I use it.
 I don't know much about 'bud', other than it's pretty old, but some people
 use it (I know bearophile does), so I assume it must have some virtues.
 As far as 'make', well, personally, I hate all forms of 'make' with a
 passion (but that's just me).
 As alternatives to 'make', there's also A-A-P, Python SCons and Ruby Rake.
 None of them will do automatic D dependencies (SCons and Rake definitely
 don't, and I don't think A-A-P does, but I could be wrong), but you can use
 them to do much more than just compile D stuff (and you can use them to
 invoke 'xfbuild' if you want). A-A-P has a lot of good points, although I
 personally find the documentation confusing as hell, so I don't really
 understand how to use it particularly well. SCons and Rake, IMO beat the
 pants off of make, or at least if you're willing to write the build scripts
 in Python or Ruby and require your users to have it installed.


Good ideas there Nick.

Okay, so rebuild/bud are ancient.

Make is admittedly a low-level tool, but time-tested and ubiquitous.  If you can
feed make, then not such a big deal.

Haven't looked at AAP or Rake.  I was looking for something that at least had an
autobuild front-end, with back-end tweaking in case of failure.

SCons I am familiar with and, while I love Python, SCons seemed too large and
unwieldy.  I was working on a D front-end for bam, but I didn't know how much
work
it would be and bam is a little light even for me.

I did find a darned interesting build utility. It's light but not too light,
well
documented, supports arbitrary build scripting in Lua and can be embedded into
the
source package by actually autocompiling itself during the config process.  It's
called prime-mover. Even though it defaults to explicit dependency mode, it also
supports intelligent parser/dependency builder plugins out of the box.  Nothing
for D yet, but they do have a C/C++ plugin. I think you could probably do a D
plugin without too much trouble.

It works just fine on all of the unixen, but I think the only downside is for
the
Windows people since it embeds itself into a shell script for the end user.
Otherwise you could build it explicitly.

We'll see.

Until then xfbuild.

I'll play with it and let you know.
Mar 12 2010
prev sibling parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Nick Sabalausky wrote:

 As alternatives to 'make', there's also A-A-P, Python SCons and Ruby Ra=


ke.=20

 None of them will do automatic D dependencies (SCons and Rake definitel=


y=20

 don't, and I don't think A-A-P does, but I could be wrong),
=20


	Actually, SCons does support D dependencies (I should know, I wrote
the current code for it), but the support is pretty basic. All it
does is look for "import" statements, locate the corresponding
source files and add them as dependencies. It does not take static
ifs into account, and it won't see import statements generated by a
mixin (but it will allow you to add dependencies manually).

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Mar 12 2010