• pragma (91/91) Dec 19 2004 With all the talk around here lately about improvements to dmake, with c...
• John Reimer (3/128) Dec 19 2004 Excellent! I'll take a look!
• Derek Parnell (7/19) Dec 19 2004 I like the pragma-tool and similar ideas. I'll take a look at your code

pragma <pragma_member pathlink.com> writes:

With all the talk around here lately about improvements to dmake, with custom
pragmas and such, I thought I should go ahead and let the group know that I too
have been working on a similar project.  To that end, I've pushed DBuild through
to an early beta for public consumption.  Since I'm using it in conjunction with
my development of DSP, it's available in the same source repo:

http://svn.dsource.org/svn/projects/dsp/trunk/misc/dbuild/
http://svn.dsource.org/svn/projects/dsp/downloads/dbuild.zip

Tip: take a look at the dbuild.id file in case dmd is not on the path.

Feedback is always welcome.


-A Quick Overview-

DBuild, like dmake and digc, navigates a tree of source files from a single
root, then launches dmd based on the information it gathers.  What is different,
is that it more closely matches to 'make' in that one can specify *any* tool to
run for a given kind of file.

For example, the following is a configuration statement in Dbuild:

pragma(tool,"d","obj","$root$\\","dmd $in$ -c -release -of$out$ -od$outdir$");

The parameters (after 'tool') are: source extension, output extension, output
directory, and the tool's path/commandline.  The dollar-sign expressions are
substitutions used to help make tool declarations a little more flexible.  For
example $root$ is the directory where the tool is started, $in$ is the source
file, and $out$ is the output file (composed from $in$, the output extension and
the output directory).

All these different parameters are important, since the tool keeps track of tool
'products' as it goes along.  This is what enables DBuild to compose a target
file.

pragma(tool,"*","exe","$root$\\","dmd $*.obj$ $*.lib$ -release -of$out$
-od$outdir$");
pragma(target,"myprogram.exe");

These two lines specify a dmd command line for creating an exe, from all the
obj and .lib files garnered from the rest of the source tree.  The second
pragma, 'target', is the trigger for the use of the '*' rule; it matches based
on the output file extension.

To use other files as products for tools in DBuild, simply use the 'dependency'
pragma to add them to the product list.

pragma(dependency,"foo.lib");
pragma(dependency,"exports.def"); // useful for dll compilation

All of this is so that Dbuild remains open-ended for tools aside from dmd.  For
example, we could build documentation using a tool (ddoc) that would spit out an
xml file for each .d file, then complete a cross-reference and html files as an
explicit target:

pragma(tool,"d","xml","$root$\\doc\\","ddoc $in$ -of$out$"); 
pragma(tool,"*","html","$root$\\doc\\","ddoc $*.xml$ -of$out$");
pragma(target,"$root$\\doc\\index.html");

DBuild makes no assumptions about what libraries to include or exclude.  To that
end, the 'ignore' statement helps weed out namespaces that you don't want to
include in your build.  Simply provide the prefix(es) for any namespace and off
you go:

pragma(ignore,"std.");
pragma(ignore,"concurrent.");
pragma(ignore,"mango.");

It also doesn't assume where to find things.  Use the 'path' statement to direct
dbuild where to go:

pragma(path,"\\");
pragma(path,"d:\\progra~1\\d\\dmd\\src\\phobos\\");

Also, DBuild honors the 'msg' pragma and will echo these statements to the
console while it hides successful tool output.

pragma(msg,"hello world");

Lastly, these are all (except for 'msg') non-standard pragmas, so enclose them
in a version statement to make sure that dmd doesn't trip on them:

version(dbuild){ /*only dbuild will see this*/ }


-What's Missing & Broken-

At a minimum, dbuild is self-hosting (can build itself) and has been tested
against rather tivial source trees.  It doesn't take advantage of response files
when talking to DMD, so if things fail on large projects, it may be due to
limited environment space.

I'm not proud of the tool syntax statements, and would like to improve them for
santiy's sake.  I'll likely remove the 'from' extension from the delcaration
since its barely even used, and will probably use another pragma name to
differentiate between a d-sourcefile tool and a target tool.  Also, it's not
checking the date/time of inputs and outputs in the same fashion as make; its an
obvious place to improve things and pick up some speed.  Lastly, multiple tool
definitions, with the same output extension, do not override correctly.  

Lastly: the "_timer" pragmas do not report the correct time, and "_debug" is
only so useful. 

-Under the Hood-

DBuild uses an experimental D-based spirit library to define enough of the D
language grammar to do the job (*).  The parsing engine uses these grammar
definitions to properly recognize version, debug and pragma statements.  It also
understands comments and strings, plus the various escape sequences typically
found in strings (eg. \n \r \t, etc).

That having been said, the code is an undocumented mess, which is in dire need
of refactoring.  The DFileParser uses a state-stack class which is a horrid mess
of arrays and accessors.  The D-Spirit classes need some more testing to ensure
proper traversal of a given grammar.  Also, the event hooks in the d-grammar
need some work to assist readability.

* This was made possible, in no small part, to the minimal Spirit-style library
I stumbled upon online some time ago... Ben, was that you?  The parser is not
templatized in the same manner as the C++ library; its somewhat slower than I
would like.  Also, the lack of certain operators in D makes for some /very/
interesting grammar definitions.

- Pragma at yahoo

John Reimer <brk_6502 yahoo.com> writes:

pragma wrote:
 With all the talk around here lately about improvements to dmake, with custom
 pragmas and such, I thought I should go ahead and let the group know that I too
 have been working on a similar project.  To that end, I've pushed DBuild
through
 to an early beta for public consumption.  Since I'm using it in conjunction
with
 my development of DSP, it's available in the same source repo:
 
 http://svn.dsource.org/svn/projects/dsp/trunk/misc/dbuild/
 http://svn.dsource.org/svn/projects/dsp/downloads/dbuild.zip
 
 Tip: take a look at the dbuild.id file in case dmd is not on the path.
 
 Feedback is always welcome.
 
 
 -A Quick Overview-
 
 DBuild, like dmake and digc, navigates a tree of source files from a single
 root, then launches dmd based on the information it gathers.  What is
different,
 is that it more closely matches to 'make' in that one can specify *any* tool to
 run for a given kind of file.
 
 For example, the following is a configuration statement in Dbuild:
 
 pragma(tool,"d","obj","$root$\\","dmd $in$ -c -release -of$out$ -od$outdir$");
 
 The parameters (after 'tool') are: source extension, output extension, output
 directory, and the tool's path/commandline.  The dollar-sign expressions are
 substitutions used to help make tool declarations a little more flexible.  For
 example $root$ is the directory where the tool is started, $in$ is the source
 file, and $out$ is the output file (composed from $in$, the output extension
and
 the output directory).
 
 All these different parameters are important, since the tool keeps track of
tool
 'products' as it goes along.  This is what enables DBuild to compose a target
 file.
 
 pragma(tool,"*","exe","$root$\\","dmd $*.obj$ $*.lib$ -release -of$out$
 -od$outdir$");
 pragma(target,"myprogram.exe");
 
 These two lines specify a dmd command line for creating an exe, from all the
 obj and .lib files garnered from the rest of the source tree.  The second
 pragma, 'target', is the trigger for the use of the '*' rule; it matches based
 on the output file extension.
 
 To use other files as products for tools in DBuild, simply use the 'dependency'
 pragma to add them to the product list.
 
 pragma(dependency,"foo.lib");
 pragma(dependency,"exports.def"); // useful for dll compilation
 
 All of this is so that Dbuild remains open-ended for tools aside from dmd.  For
 example, we could build documentation using a tool (ddoc) that would spit out
an
 xml file for each .d file, then complete a cross-reference and html files as an
 explicit target:
 
 pragma(tool,"d","xml","$root$\\doc\\","ddoc $in$ -of$out$"); 
 pragma(tool,"*","html","$root$\\doc\\","ddoc $*.xml$ -of$out$");
 pragma(target,"$root$\\doc\\index.html");
 
 DBuild makes no assumptions about what libraries to include or exclude.  To
that
 end, the 'ignore' statement helps weed out namespaces that you don't want to
 include in your build.  Simply provide the prefix(es) for any namespace and off
 you go:
 
 pragma(ignore,"std.");
 pragma(ignore,"concurrent.");
 pragma(ignore,"mango.");
 
 It also doesn't assume where to find things.  Use the 'path' statement to
direct
 dbuild where to go:
 
 pragma(path,"\\");
 pragma(path,"d:\\progra~1\\d\\dmd\\src\\phobos\\");
 
 Also, DBuild honors the 'msg' pragma and will echo these statements to the
 console while it hides successful tool output.
 
 pragma(msg,"hello world");
 
 Lastly, these are all (except for 'msg') non-standard pragmas, so enclose them
 in a version statement to make sure that dmd doesn't trip on them:
 
 version(dbuild){ /*only dbuild will see this*/ }
 
 
 -What's Missing & Broken-
 
 At a minimum, dbuild is self-hosting (can build itself) and has been tested
 against rather tivial source trees.  It doesn't take advantage of response
files
 when talking to DMD, so if things fail on large projects, it may be due to
 limited environment space.
 
 I'm not proud of the tool syntax statements, and would like to improve them for
 santiy's sake.  I'll likely remove the 'from' extension from the delcaration
 since its barely even used, and will probably use another pragma name to
 differentiate between a d-sourcefile tool and a target tool.  Also, it's not
 checking the date/time of inputs and outputs in the same fashion as make; its
an
 obvious place to improve things and pick up some speed.  Lastly, multiple tool
 definitions, with the same output extension, do not override correctly.  
 
 Lastly: the "_timer" pragmas do not report the correct time, and "_debug" is
 only so useful. 
 
 -Under the Hood-
 
 DBuild uses an experimental D-based spirit library to define enough of the D
 language grammar to do the job (*).  The parsing engine uses these grammar
 definitions to properly recognize version, debug and pragma statements.  It
also
 understands comments and strings, plus the various escape sequences typically
 found in strings (eg. \n \r \t, etc).
 
 That having been said, the code is an undocumented mess, which is in dire need
 of refactoring.  The DFileParser uses a state-stack class which is a horrid
mess
 of arrays and accessors.  The D-Spirit classes need some more testing to ensure
 proper traversal of a given grammar.  Also, the event hooks in the d-grammar
 need some work to assist readability.
 
 * This was made possible, in no small part, to the minimal Spirit-style library
 I stumbled upon online some time ago... Ben, was that you?  The parser is not
 templatized in the same manner as the C++ library; its somewhat slower than I
 would like.  Also, the lack of certain operators in D makes for some /very/
 interesting grammar definitions.
 
 - Pragma at yahoo

Excellent!  I'll take a look!

- John

Derek Parnell <derek psych.ward> writes:

On Mon, 20 Dec 2004 05:34:13 +0000 (UTC), pragma wrote:

 With all the talk around here lately about improvements to dmake, with custom
 pragmas and such, I thought I should go ahead and let the group know that I too
 have been working on a similar project.  To that end, I've pushed DBuild
through
 to an early beta for public consumption.  Since I'm using it in conjunction
with
 my development of DSP, it's available in the same source repo:
 
 http://svn.dsource.org/svn/projects/dsp/trunk/misc/dbuild/
 http://svn.dsource.org/svn/projects/dsp/downloads/dbuild.zip
 
 Tip: take a look at the dbuild.id file in case dmd is not on the path.
 
 Feedback is always welcome.

I like the pragma-tool and similar ideas. I'll take a look at your code
next week, just too busy now. 

-- 
Derek
Melbourne, Australia
20/12/2004 4:55:55 PM