www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - pluggable preprocessor

reply B.G. <B.G._member pathlink.com> writes:
There was a thread about D embedded in xml instead of html, 

Embedding D in any of those is of less interest to me personally, at least now, 
but I think it were GREAT if there was a possibility to write any source code
preprocessor as a plugin for compiler, in form of a dll/so and have a command
line option to use this or that filter.

This can be helpful for a variaty of cool dirty tricks. I would for instance
write myself one that would emit some code, resulting in a program capable of
printing a nice cross-platform exception stacktrace information. Or one that
would emit code to make [a == b when a is null] not to segfault but instead
alarm me utilizing all imaginable means.

It'd be nice to have for the development stage, without a need of uglifying the
code by versioning such fundamental things.

What do you think?
May 03 2005
next sibling parent reply Thomas Kuehne <thomas-dloop kuehne.thisisspam.cn> writes:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

B.G schrieb am Tue, 3 May 2005 10:33:06 +0000 (UTC):

 There was a thread about D embedded in xml instead of html, 

 Embedding D in any of those is of less interest to me personally, at least
now, 
 but I think it were GREAT if there was a possibility to write any source code
 preprocessor as a plugin for compiler, in form of a dll/so and have a command
 line option to use this or that filter.

 This can be helpful for a variaty of cool dirty tricks. I would for instance
 write myself one that would emit some code, resulting in a program capable of
 printing a nice cross-platform exception stacktrace information. Or one that
 would emit code to make [a == b when a is null] not to segfault but instead
 alarm me utilizing all imaginable means.

 It'd be nice to have for the development stage, without a need of uglifying the
 code by versioning such fundamental things.

 What do you think?


I'm not sure why you'd like to plug the preprocessor into the compiler.
You could write a very simple wraper that hands every sourcefile to your
preprocessor before running dmd.

sample:


Your a.pre can now contain all kinds of D source and C preprocessor
statements. 

Thomas


-----BEGIN PGP SIGNATURE-----

iD8DBQFCd1e63w+/yD4P9tIRAnN4AKC3wqypRHqvd+9U5W3r8pLx4htKggCePeqp
+ruDarrc88sI1MVUZXvJCKE=
=oums
-----END PGP SIGNATURE-----
May 03 2005
parent "B.G." <gbatyan gmx.net> writes:
Thomas Kuehne wrote:

 -----BEGIN PGP SIGNED MESSAGE-----
 Hash: SHA1
 
 B.G schrieb am Tue, 3 May 2005 10:33:06 +0000 (UTC):
 

There was a thread about D embedded in xml instead of html, 

Embedding D in any of those is of less interest to me personally, at least now, 
but I think it were GREAT if there was a possibility to write any source code
preprocessor as a plugin for compiler, in form of a dll/so and have a command
line option to use this or that filter.

This can be helpful for a variaty of cool dirty tricks. I would for instance
write myself one that would emit some code, resulting in a program capable of
printing a nice cross-platform exception stacktrace information. Or one that
would emit code to make [a == b when a is null] not to segfault but instead
alarm me utilizing all imaginable means.

It'd be nice to have for the development stage, without a need of uglifying the
code by versioning such fundamental things.

What do you think?


 
 
 I'm not sure why you'd like to plug the preprocessor into the compiler.
 You could write a very simple wraper that hands every sourcefile to your
 preprocessor before running dmd.
 
 sample:

 
 Your a.pre can now contain all kinds of D source and C preprocessor
 statements. 
 
 Thomas


OK, for PREPROCESSING tasks, you're right, it's ok to do that with 
external methods, although I'd still be more happy if there were at 
least a possibility to have a more efficient method than scripts, pipes, 
greps, etc.

But actually, a _preprocessor_ is a bad underdefinition of what I've 
actually ment here. Sorry for misexpressing myself.

What I've ment is that external code could be inserted at arbitrary 
position into the pipleline of compilation and work directly with 
internal representation of the parsed file on syntactical and/or 
semantical level. That'd be cool.




 
 
 -----BEGIN PGP SIGNATURE-----
 
 iD8DBQFCd1e63w+/yD4P9tIRAnN4AKC3wqypRHqvd+9U5W3r8pLx4htKggCePeqp
 +ruDarrc88sI1MVUZXvJCKE=
 =oums
 -----END PGP SIGNATURE-----
May 13 2005
prev sibling next sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
B.G. wrote:

 There was a thread about D embedded in xml instead of html, 
 
 Embedding D in any of those is of less interest to me personally, at least
now, 
 but I think it were GREAT if there was a possibility to write any source code
 preprocessor as a plugin for compiler, in form of a dll/so and have a command
 line option to use this or that filter.
 
 This can be helpful for a variaty of cool dirty tricks. I would for instance
 write myself one that would emit some code, resulting in a program capable of
 printing a nice cross-platform exception stacktrace information. Or one that
 would emit code to make [a == b when a is null] not to segfault but instead
 alarm me utilizing all imaginable means.
 
 It'd be nice to have for the development stage, without a need of uglifying the
 code by versioning such fundamental things.
 
 What do you think?
 
 


I think one of the design goals for D was to completly eliminate the pre 
proecessor.

I don't see how #ifdef #ifndel #elif etc is nicer than version(windows){ 
code }
May 03 2005
parent Paul Bonser <misterpib gmail.com> writes:
Hasan Aljudy wrote:
...

 What do you think?


 I think one of the design goals for D was to completly eliminate the pre
 proecessor.
 
 I don't see how #ifdef #ifndel #elif etc is nicer than version(windows){
 code }


I'm pretty sure that's not what he meant. There was some discussion of
this topic earlier, and many (some?) of us came to the conclusion that
it would be nice to have the ability to write custom plug-ins for
various reasons ( It was suggested that this functionality be built into
Build). I suggested a intermediate file, such as .pd or something, so
that no files will be replaced (only new ones made, which are the ones
actually passed to the compiler).

Anyway, I'd totally use it to (automatically) add full-ish reflection
capabilities to my programs, something that I would never even try to do
with a C preprocessor.

-- 
-PIB

--
"C++ also supports the notion of *friends*: cooperative classes that
are permitted to see each other's private parts." - Grady Booch
May 03 2005
prev sibling next sibling parent "G.Vidal" <gyvidal wanadoo.fr> writes:
 What do you think?


I posted a thread some time ago about it.
Go to the BUILD's forum at dsource.org and check the "pragma(plugin)"
thread.

Itcha.
May 03 2005
prev sibling parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
What I'd rather see, for the specific purpose of the below, is a 
"pluggable" compiler.  What I mean is, basically, that I could have some 
code like the below in my own application:

void compile(char[] bytes, char[] out_filename)
{
	D_compiler_options opt;
	opt.inline = 1;
	opt.optimize = 1;

	D_compiler_process(toStringz(bytes), toStringz(out_filename), &opt);
}

Which would, clearly, compile the data pointed to by *bytes to the 
object file out_filename.  The syntax is crappy and just for example.

This would make it very easy to write a program which, upon access to a 
file, checked its mtime and if necessary recompiled to an object 
file/dll.  If a dll, it could then dynamically load this new dll and run 
the program.

This has many benefits:
   - for make/build systems, this would be more efficient; load the 
compiler once, and compile multiple files - instead of loading the 
compiler for each file, as many do right now.
   - for IDEs/interpreters, this would allow much more streamlined 
integration; the D code could provide error messages, line numbers, and 
such without annoying/slow analysis of output.
   - for web services, this allows for more ease and efficiency if the 
code is parsed into a (native) D program dll.

This is very like mod_php vs. php-cgi.  You see, PHP can either be 
integrated into Apache in a way that Apache just calls a function to 
execute a PHP file, or it can pipe the php executable and send the 
output to the client manually.  Clearly, the integrated method (the way 
you can't use with most compilers) is the most efficient and desirable one.

I know that, for some reason, a lot of people are fans of piping.  As 
much as I love grep, gawk, bash scripts, make, and similar tools... 
there's a penalty.  Just like function calls, it's faster to avoid the 
call altogether.

Anyway, this is somewhat off topic, and for that I apologize.  I think 
it's on the same track, though.

-[Unknown]



 There was a thread about D embedded in xml instead of html, 
 
 Embedding D in any of those is of less interest to me personally, at least
now, 
 but I think it were GREAT if there was a possibility to write any source code
 preprocessor as a plugin for compiler, in form of a dll/so and have a command
 line option to use this or that filter.
May 15 2005
parent reply "B.G." <gbatyan gmx.net> writes:
Unknown W. Brackets wrote:

 What I'd rather see, for the specific purpose of the below, is a 
 "pluggable" compiler.  What I mean is, basically, that I could have some 
 code like the below in my own application:
 
 void compile(char[] bytes, char[] out_filename)
 {
     D_compiler_options opt;
     opt.inline = 1;
     opt.optimize = 1;
 
     D_compiler_process(toStringz(bytes), toStringz(out_filename), &opt);
 }
 
 Which would, clearly, compile the data pointed to by *bytes to the 
 object file out_filename.  The syntax is crappy and just for example.
 
 This would make it very easy to write a program which, upon access to a 
 file, checked its mtime and if necessary recompiled to an object 
 file/dll.  If a dll, it could then dynamically load this new dll and run 
 the program.
 
 This has many benefits:
   - for make/build systems, this would be more efficient; load the 
 compiler once, and compile multiple files - instead of loading the 
 compiler for each file, as many do right now.
   - for IDEs/interpreters, this would allow much more streamlined 
 integration; the D code could provide error messages, line numbers, and 
 such without annoying/slow analysis of output.
   - for web services, this allows for more ease and efficiency if the 
 code is parsed into a (native) D program dll.
 
 This is very like mod_php vs. php-cgi.  You see, PHP can either be 
 integrated into Apache in a way that Apache just calls a function to 
 execute a PHP file, or it can pipe the php executable and send the 
 output to the client manually.  Clearly, the integrated method (the way 
 you can't use with most compilers) is the most efficient and desirable one.
 
 I know that, for some reason, a lot of people are fans of piping.  As 
 much as I love grep, gawk, bash scripts, make, and similar tools... 
 there's a penalty.  Just like function calls, it's faster to avoid the 
 call altogether.
 
 Anyway, this is somewhat off topic, and for that I apologize.  I think 
 it's on the same track, though.
 


For me personally it is on-topic enough :)

This would definitely be interesting, there are vast possibilities how 
this could be used, easy integration into IDE's, faster development 
cycle of big projects, maybe someday something like D Server Pages, 
compiling and loading modified pages on the fly, etc.

BTW, I'm curious how good is DMD at memory management? Well, at least 
the open source part? Would a program using DMD sources for parsing D 
have any memory leaks (taken dmdfe as the base for example)?

Although, I'd wouldn't probably dare to inlude GCC into my executable 
anyway :)
May 15 2005
parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
Why not?  We have GD, which is pretty big... zlib, SDL, OpenGL, 
libmysqld, and all measure of other large libraries.  What's scary about 
a libgcc or libdmd?

I would expect that DMD doesn't leak.  I know it has memory management 
stuff in there, and if it does leak right now I should hope Walter would 
be more than willing to fix it ;).

-[Unknown]



 BTW, I'm curious how good is DMD at memory management? Well, at least 
 the open source part? Would a program using DMD sources for parsing D 
 have any memory leaks (taken dmdfe as the base for example)?
 
 Although, I'd wouldn't probably dare to inlude GCC into my executable 
 anyway :)
May 15 2005
parent reply "B.G." <gbatyan gmx.net> writes:
Unknown W. Brackets wrote:

 Why not?  We have GD, which is pretty big... zlib, SDL, OpenGL, 
 libmysqld, and all measure of other large libraries.  What's scary about 
 a libgcc or libdmd?


http://gcc.gnu.org/onlinedocs/gccint/Libgcc.html
Do you mean this libgcc? As far as I've understood the documentation 
this library has nothing to do with a 'pluggable compiler'. It's just a 
way for gcc to reference some code from a shared library instead of 
emitting directly into executable.

If it's not the library you have ment, can you give me the link for the 
right one?


 I would expect that DMD doesn't leak.  I know it has memory management 
 stuff in there, and if it does leak right now I should hope Walter would 
 be more than willing to fix it ;).


I believe there will be a day, DMD compiler (including it's open source 
part)
will be converted from  C++ to D. This will have LOTS of positive 
implications for development of the tools based on the open source DMD 
part. And as a nice 'side effect' any leaks will go away for granted.

Walter, do you plan to completely move DMD from C++ to D?
If yes - when?


 -[Unknown]
 
 

 BTW, I'm curious how good is DMD at memory management? Well, at least 
 the open source part? Would a program using DMD sources for parsing D 
 have any memory leaks (taken dmdfe as the base for example)?

 Although, I'd wouldn't probably dare to inlude GCC into my executable 
 anyway :)
May 16 2005
next sibling parent "Unknown W. Brackets" <unknown simplemachines.org> writes:
Actually, I meant a theoretically libgcc.  I was not aware one existed, 
and the one you linked to is really not at all what I meant.

Let's try "libgcccompiler" :P?

-[Unknown]



 Do you mean this libgcc? As far as I've understood the documentation 
 this library has nothing to do with a 'pluggable compiler'. It's just a 
 way for gcc to reference some code from a shared library instead of 
 emitting directly into executable.
 
 If it's not the library you have ment, can you give me the link for the 
 right one?
 

 I would expect that DMD doesn't leak.  I know it has memory management 
 stuff in there, and if it does leak right now I should hope Walter 
 would be more than willing to fix it ;).


 
 
 I believe there will be a day, DMD compiler (including it's open source 
 part)
 will be converted from  C++ to D. This will have LOTS of positive 
 implications for development of the tools based on the open source DMD 
 part. And as a nice 'side effect' any leaks will go away for granted.
May 16 2005
prev sibling parent J C Calvarese <technocrat7 gmail.com> writes:
In article <d69u4b$1ifl$1 digitaldaemon.com>, B.G. says...

I believe there will be a day, DMD compiler (including it's open source 
part)
will be converted from  C++ to D. This will have LOTS of positive 
implications for development of the tools based on the open source DMD 
part. And as a nice 'side effect' any leaks will go away for granted.

Walter, do you plan to completely move DMD from C++ to D?
If yes - when?


From http://www.prowiki.org/wiki4d/wiki.cgi?FaqRoadmap#Self-CompilingCompiler:

(unofficial FAQ)

Q: Will DMD ever be converting into a self-compiling compiler?

A: Maybe someday in the future. D is certainly powerful enough to write a D
compiler.

In the meantime, many other issues have a higher priority. (Part of the
challenge to convert the current C++ sources to D is that the interface to his
backend isn't trivial. There are a number of .h files, the back end is in C++,
and it would be a fair chunk of work to interface it.)

Walter has far too many other things to do right now.

Sources:
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D:19899
http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D:19929

jcc7
May 16 2005