www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Asmjit - JIT asm compiler (for C++)

reply Bill Baxter <wbaxter gmail.com> writes:
Don't know if this is of interest to anyone here, but I've seen this
on another mailing list recently:
http://code.google.com/p/asmjit/

C++ library to jit-compile assembly functions.
It uses function syntax to do it's think like "a.push(ebp);"
That's pretty cool already, but I was thinking with D you could just
write the assembly function directly as a string mixin.  Or really
(even in C++ ) you could just parse the darn assembly function at
runtime.   But then you don't find out about your coding errors till
runtime.  With compile time parsing you get to keep the compile time
check to make sure the asm instructions are at least typo-free.

Also kind of relevant to the other thread here about compiling
different versions of the same function for different target
processors.

--bb
Feb 02 2009
next sibling parent reply grauzone <none example.net> writes:
Normal Programmer: look here, I have some cool stuff that could be 
useful, especially because it works done at runtime.

D Programmer: LET'S DO IT IN COMPILE TIME!

SCNR
Feb 02 2009
parent reply Bill Baxter <wbaxter gmail.com> writes:
On Tue, Feb 3, 2009 at 10:09 AM, grauzone <none example.net> wrote:

 Normal Programmer: look here, I have some cool stuff that could be useful,
 especially because it works done at runtime.

 D Programmer: LET'S DO IT IN COMPILE TIME!


Yeh forget this just-in-time compilation at run time stuff!  Let's
compile code at /compile time/!   Er.. wait no.

--bb
Feb 02 2009
parent BCS <none anon.com> writes:
Hello Bill,


 On Tue, Feb 3, 2009 at 10:09 AM, grauzone <none example.net> wrote:
 

 Normal Programmer: look here, I have some cool stuff that could be
 useful, especially because it works done at runtime.
 
 D Programmer: LET'S DO IT IN COMPILE TIME!
 


 Yeh forget this just-in-time compilation at run time stuff!  Let's
 compile code at /compile time/!   Er.. wait no.
 
 --bb
 


and then their are these guys: 
http://www.povray.org/documentation/view/3.6.1/124/

and this by Tomasz Stachowiak: http://h3.team0xf.com/ctrace/
Feb 02 2009
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Bill Baxter Wrote:

 C++ library to jit-compile assembly functions.


You may be interested in this too, that also optionally performs some of the
loop optimizations done by good Fortran compilers:
http://www.corepy.org/

Bye,
bearophile
Feb 02 2009
parent reply Bill Baxter <wbaxter gmail.com> writes:
On Tue, Feb 3, 2009 at 10:15 AM, bearophile <bearophileHUGS lycos.com> wrote:

 Bill Baxter Wrote:

 C++ library to jit-compile assembly functions.


 You may be interested in this too, that also optionally performs some of the
loop optimizations done by good Fortran compilers:
 http://www.corepy.org/


Interesting.  Seems like this idea is popping up all over the place
these days.  I think the MESA OpenGL guys were doing something like
this to support shaders, too.  I also could have sworn I read
something about Jitting things in some Qt graphics framework.  But I
can't find anything about either at the moment.  And for some reason
something about Apple+LLVM+JIT also is tumbling around in my mind.

--bb
Feb 02 2009
next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Bill Baxter wrote:

 [snip]

 And for some reason
 something about Apple+LLVM+JIT also is tumbling around in my mind.
 
 --bb


I believe Apple was using LLVM to do software-acceleration in its OpenGL
stack, specifically for shaders, I believe.

I have a sneaking suspicion that they're also using it for OpenCL (with
clang as the front-end.)

  -- Daniel
Feb 02 2009
parent Chad J <gamerchad __spam.is.bad__gmail.com> writes:
Daniel Keep wrote:

 
 Bill Baxter wrote:

 [snip]

 And for some reason
 something about Apple+LLVM+JIT also is tumbling around in my mind.

 --bb


 
 I believe Apple was using LLVM to do software-acceleration in its OpenGL
 stack, specifically for shaders, I believe.
 
 I have a sneaking suspicion that they're also using it for OpenCL (with
 clang as the front-end.)
 
   -- Daniel


Yeah, LLVM seems popular for this stuff these days.

LLVM+Clang for OpenCL is happening in the Linux world too:
http://www.phoronix.com/scan.php?page=news_item&px=NzAzMw
http://zrusin.blogspot.com/2009/02/opencl.html

- Chad
Feb 02 2009
prev sibling parent Sandeep Kakarlapudi <sandeep.iitkgpspammenot gmail.com> writes:
Bill Baxter Wrote:

 Interesting.  Seems like this idea is popping up all over the place
 these days.  I think the MESA OpenGL guys were doing something like
 this to support shaders, too.  I also could have sworn I read
 something about Jitting things in some Qt graphics framework.  But I
 can't find anything about either at the moment.  And for some reason
 something about Apple+LLVM+JIT also is tumbling around in my mind.
 
 --bb


Code JIT-ing is very common when dealing which shaders because shaders
essentially are hot inner loops. Even when executing shaders on GPUs the
shaders are often JIT optimized - if the optimized code is 1 instruction less
it still is huge savings. 

Here's are some slides on using LLVM for OpenGL :
http://llvm.org/devmtg/2007-05/10-Lattner-OpenGL.pdf
What they seem to be doing is achieving compilation by specializing the glsl
interpreter for each shader using the LLVM optimizer. ( See:
http://en.wikipedia.org/wiki/Partial_evaluation ) This leads to more
maintainable code and still produces optimal machine code. 

While runtime assemblers do have some benefit I think ultimately run-time
specialization kind of approaches will have wider adoption because of ease of
maintenance. The programmer does not have to master the ISA of the CPU. So
having it as part of the standard library would certainly accelerate the
adoption! 

Sandeep
Feb 03 2009
prev sibling next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
Bill Baxter schrieb:


 That's pretty cool already, but I was thinking with D you could just
 write the assembly function directly as a string mixin.  Or really
 (even in C++ ) you could just parse the darn assembly function at
 runtime.


other JIT assemblers:

http://homepage1.nifty.com/herumi/soft/xbyak_e.html
http://www.transgaming.com/products/swiftshader/


 With compile time parsing you get to keep the compile time
 check to make sure the asm instructions are at least typo-free.
 Also kind of relevant to the other thread here about compiling
 different versions of the same function for different target
 processors.


that makes no sense for an "JIT" assembler
to find code errors in assembly or produce different version at 
compiletime ---> use version-flag and the inline assembler :-)
Feb 02 2009
next sibling parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Feb 3, 2009 at 4:27 PM, dennis luehring <dl.soluz gmx.net> wrote:

 Bill Baxter schrieb:


 That's pretty cool already, but I was thinking with D you could just
 write the assembly function directly as a string mixin.  Or really
 (even in C++ ) you could just parse the darn assembly function at
 runtime.


 other JIT assemblers:

 http://homepage1.nifty.com/herumi/soft/xbyak_e.html
 http://www.transgaming.com/products/swiftshader/


 With compile time parsing you get to keep the compile time
 check to make sure the asm instructions are at least typo-free.
 Also kind of relevant to the other thread here about compiling
 different versions of the same function for different target
 processors.


 that makes no sense for an "JIT" assembler
 to find code errors in assembly or produce different version at compiletime
 ---> use version-flag and the inline assembler :-)


It would let you find out if you typed "psuh" instead of "push" in
your chunk of assembly code that you will only be compiling at runtime
in the event it is appropriate for that platform.  Maybe it's not much
better than just doing what the asmjit guys are doing, though.
a.push(ebp);  is already close to looking like assembly, and the
compiler will already check that for typos and proper argument counts
just fine.

--bb
Feb 02 2009
prev sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
dennis luehring schrieb:

 Bill Baxter schrieb:
 

 That's pretty cool already, but I was thinking with D you could just
 write the assembly function directly as a string mixin.  Or really
 (even in C++ ) you could just parse the darn assembly function at
 runtime.


 other JIT assemblers:
 
 http://homepage1.nifty.com/herumi/soft/xbyak_e.html
 http://www.transgaming.com/products/swiftshader/


but what i realy like to see (later) is a JIT compiler for D itselfe
like a mixin but at runtime ... ideas?
Feb 02 2009
parent Bill Baxter <wbaxter gmail.com> writes:
On Tue, Feb 3, 2009 at 4:59 PM, dennis luehring <dl.soluz gmx.net> wrote:

 dennis luehring schrieb:

 Bill Baxter schrieb:


 That's pretty cool already, but I was thinking with D you could just
 write the assembly function directly as a string mixin.  Or really
 (even in C++ ) you could just parse the darn assembly function at
 runtime.


 other JIT assemblers:

 http://homepage1.nifty.com/herumi/soft/xbyak_e.html
 http://www.transgaming.com/products/swiftshader/


 but what i realy like to see (later) is a JIT compiler for D itselfe
 like a mixin but at runtime ... ideas?


Maybe LLVM can do this at some point.  I think the apple shader thing
mentioned above jit compiles some sort of C-like language.  OpenCL
will probably be doing the same.

--bb
Feb 03 2009
prev sibling next sibling parent reply Robert Fraser <fraserofthenight gmail.com> writes:
Bill Baxter wrote:

 Don't know if this is of interest to anyone here, but I've seen this
 on another mailing list recently:
 http://code.google.com/p/asmjit/
 
 C++ library to jit-compile assembly functions.
 It uses function syntax to do it's think like "a.push(ebp);"
 That's pretty cool already, but I was thinking with D you could just
 write the assembly function directly as a string mixin.  Or really
 (even in C++ ) you could just parse the darn assembly function at
 runtime.   But then you don't find out about your coding errors till
 runtime.  With compile time parsing you get to keep the compile time
 check to make sure the asm instructions are at least typo-free.
 
 Also kind of relevant to the other thread here about compiling
 different versions of the same function for different target
 processors.
 
 --bb


Semi on-topic, does anyone know of a a JIT assembler for D (or a good 
one for C it wouldn't take too long to make a header for?.... Of course, 
DMD+Windows has linking issues)
Feb 03 2009
parent reply Bill Baxter <wbaxter gmail.com> writes:
On Tue, Feb 3, 2009 at 5:24 PM, Robert Fraser
<fraserofthenight gmail.com> wrote:

 Bill Baxter wrote:

 Don't know if this is of interest to anyone here, but I've seen this
 on another mailing list recently:
 http://code.google.com/p/asmjit/

 C++ library to jit-compile assembly functions.
 It uses function syntax to do it's think like "a.push(ebp);"
 That's pretty cool already, but I was thinking with D you could just
 write the assembly function directly as a string mixin.  Or really
 (even in C++ ) you could just parse the darn assembly function at
 runtime.   But then you don't find out about your coding errors till
 runtime.  With compile time parsing you get to keep the compile time
 check to make sure the asm instructions are at least typo-free.

 Also kind of relevant to the other thread here about compiling
 different versions of the same function for different target
 processors.

 --bb


 Semi on-topic, does anyone know of a a JIT assembler for D (or a good one
 for C it wouldn't take too long to make a header for?.... Of course,
 DMD+Windows has linking issues)


Maybe TCC fits the bill for the C?  http://bellard.org/tcc/
"""
With libtcc, you can use TCC as a backend for dynamic code generation.
"""

--bb
Feb 03 2009
next sibling parent Robert Fraser <fraserofthenight gmail.com> writes:
Bill Baxter wrote:

 On Tue, Feb 3, 2009 at 5:24 PM, Robert Fraser
 <fraserofthenight gmail.com> wrote:

 Bill Baxter wrote:

 Don't know if this is of interest to anyone here, but I've seen this
 on another mailing list recently:
 http://code.google.com/p/asmjit/

 C++ library to jit-compile assembly functions.
 It uses function syntax to do it's think like "a.push(ebp);"
 That's pretty cool already, but I was thinking with D you could just
 write the assembly function directly as a string mixin.  Or really
 (even in C++ ) you could just parse the darn assembly function at
 runtime.   But then you don't find out about your coding errors till
 runtime.  With compile time parsing you get to keep the compile time
 check to make sure the asm instructions are at least typo-free.

 Also kind of relevant to the other thread here about compiling
 different versions of the same function for different target
 processors.

 --bb


 Semi on-topic, does anyone know of a a JIT assembler for D (or a good one
 for C it wouldn't take too long to make a header for?.... Of course,
 DMD+Windows has linking issues)


 
 Maybe TCC fits the bill for the C?  http://bellard.org/tcc/
 """
 With libtcc, you can use TCC as a backend for dynamic code generation.
 """
 
 --bb


Cool, thanks for the link. That looks good, but it is a complete 
compiler, so might be sort of slow. Is there anything more 
lightweight/designed for JITs that works on x86+Windows? There's libjit, 
but that's Unix only AFAICT. Thanks.
Feb 03 2009
prev sibling parent Jesse Phillips <jessekphillips gmail.com> writes:
On Tue, 03 Feb 2009 17:36:25 +0900, Bill Baxter wrote:


 On Tue, Feb 3, 2009 at 5:24 PM, Robert Fraser
 <fraserofthenight gmail.com> wrote:

 Bill Baxter wrote:

 Don't know if this is of interest to anyone here, but I've seen this
 on another mailing list recently:
 http://code.google.com/p/asmjit/

 C++ library to jit-compile assembly functions. It uses function syntax
 to do it's think like "a.push(ebp);" That's pretty cool already, but I
 was thinking with D you could just write the assembly function
 directly as a string mixin.  Or really (even in C++ ) you could just
 parse the darn assembly function at runtime.   But then you don't find
 out about your coding errors till runtime.  With compile time parsing
 you get to keep the compile time check to make sure the asm
 instructions are at least typo-free.

 Also kind of relevant to the other thread here about compiling
 different versions of the same function for different target
 processors.

 --bb


 Semi on-topic, does anyone know of a a JIT assembler for D (or a good
 one for C it wouldn't take too long to make a header for?.... Of
 course, DMD+Windows has linking issues)


 
 Maybe TCC fits the bill for the C?  http://bellard.org/tcc/ """
 With libtcc, you can use TCC as a backend for dynamic code generation.
 """
 
 --bb


:( here I thought it was short for Totally Cool Compiler.
Feb 03 2009
prev sibling parent Don <nospam nospam.com> writes:
Bill Baxter wrote:

 Don't know if this is of interest to anyone here, but I've seen this
 on another mailing list recently:
 http://code.google.com/p/asmjit/
 
 C++ library to jit-compile assembly functions.
 It uses function syntax to do it's think like "a.push(ebp);"
 That's pretty cool already, but I was thinking with D you could just
 write the assembly function directly as a string mixin.


Yes. I've been tempted to do that on a small scale to get around some of 
DMD's asm bugs. In fact, if we had a standard 'db' asm instruction, it 
would be possible to write a CTFE assembler for compilers which don't 
support it.

Mostly OT: Is there any way to use 64-bit asm instructions on 32-bit XP?
In the Win3.1 days, it was possible to use 32-bit asm, if you fiddled 
around enough. I imagine that it was possible only because Win3.1 was a 
joke of an OS. I've never seen anything similar for 64-bit; am I correct 
in thinking that it's impossible?
Feb 03 2009