www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to include Python Modules in a D Program?

reply "Jon" <SchaduwBlink hotmail.com> writes:
Hi,

I am fairly new to D, and I would like to write a program that uses a bunch 
of modules that were written in Python. This is the reverse of what I have 
read where people use D written dlls in a python program. I have a lot of 
modules that would take too long to translate to D: a lot of files and 
several thousand lines of code each. So, it would be nice if I could use 
these modules with the D program. There are just some features of D that I 
want to use and I can only get away with these features if the program is a 
compiled program. Sometimes the python modules will not be used, for 
example, there is a safe mode that uses no external libraries at all in case 
something is broken, then you can at least start the program to back things 
up properly. The only way to do that is to have a compiled binary. Plus, I 
want to learn more about D application writing and not dll writing. :P I 
would appreciate it if someone could explain it very thoroughly.

Thanks,
Jon
Sep 24 2007
next sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Jon wrote:

 Hi,
 
 I am fairly new to D, and I would like to write a program that uses a 
 bunch of modules that were written in Python. This is the reverse of 
 what I have read where people use D written dlls in a python program. I 
 have a lot of modules that would take too long to translate to D: a lot 
 of files and several thousand lines of code each. So, it would be nice 
 if I could use these modules with the D program. There are just some 
 features of D that I want to use and I can only get away with these 
 features if the program is a compiled program. Sometimes the python 
 modules will not be used, for example, there is a safe mode that uses no 
 external libraries at all in case something is broken, then you can at 
 least start the program to back things up properly. The only way to do 
 that is to have a compiled binary. Plus, I want to learn more about D 
 application writing and not dll writing. :P I would appreciate it if 
 someone could explain it very thoroughly.
 
 Thanks,
 Jon


There is a translation of the Python C/API.  That should be all you need.
For general instructions on embedding python in a C application see: 
http://www.python.org/doc/ext/embedding.html
For the D translation of the CPython API, see PyD (Specifically you need 
this module 
http://dsource.org/projects/pyd/browser/trunk/infrastructure/python/python.d)

Are you looking for something higher-level?

--bb
Sep 24 2007
parent reply "Jon" <SchaduwBlink hotmail.com> writes:
What would be higher level than this?

I was hoping for something that would be easy to use and allow me to include 
all the python modules. Is there an example of some kind for python module 
you linked? I just import that in my D program and then link to all the 
modules?

What would be the best course of action for this many modules? Maybe write a 
module in D that used that python module and all it does is wrap the python 
modules to cleaner code? I looked briefly at the PyD site, but they tend to 
have examples showing how to include D modules into Python. :( Did I miss 
something?

Thanks for replying and taking an interest in helping me. :)

Oh, by the way, I use GDC on Linux. It is the latest GDC version. I am also 
using Python v2.5.x

"Bill Baxter"wrote in message...

 Jon wrote:

 Hi,

 I am fairly new to D, and I would like to write a program that uses a 
 bunch of modules that were written in Python. This is the reverse of what 
 I have read where people use D written dlls in a python program. I have a 
 lot of modules that would take too long to translate to D: a lot of files 
 and several thousand lines of code each. So, it would be nice if I could 
 use these modules with the D program. There are just some features of D 
 that I want to use and I can only get away with these features if the 
 program is a compiled program. Sometimes the python modules will not be 
 used, for example, there is a safe mode that uses no external libraries 
 at all in case something is broken, then you can at least start the 
 program to back things up properly. The only way to do that is to have a 
 compiled binary. Plus, I want to learn more about D application writing 
 and not dll writing. :P I would appreciate it if someone could explain it 
 very thoroughly.

 Thanks,
 Jon


 There is a translation of the Python C/API.  That should be all you need.
 For general instructions on embedding python in a C application see: 
 http://www.python.org/doc/ext/embedding.html
 For the D translation of the CPython API, see PyD (Specifically you need 
 this module 
 http://dsource.org/projects/pyd/browser/trunk/infrastructure/python/python.d)

 Are you looking for something higher-level?

 --bb
Sep 24 2007
parent reply Kirk McDonald <kirklin.mcdonald gmail.com> writes:
Jon wrote:

 What would be higher level than this?
 
 I was hoping for something that would be easy to use and allow me to 
 include all the python modules. Is there an example of some kind for 
 python module you linked? I just import that in my D program and then 
 link to all the modules?
 
 What would be the best course of action for this many modules? Maybe 
 write a module in D that used that python module and all it does is wrap 
 the python modules to cleaner code? I looked briefly at the PyD site, 
 but they tend to have examples showing how to include D modules into 
 Python. :( Did I miss something?
 
 Thanks for replying and taking an interest in helping me. :)
 
 Oh, by the way, I use GDC on Linux. It is the latest GDC version. I am 
 also using Python v2.5.x
 


Pyd is geared towards /extending/ Python with D code. It wraps much of 
the raw Python/C API with a great deal of template trickery. It does 
this pretty well, if I may say so.

Pyd also comes with an extension to Python's distutils, which makes 
building these extension modules relatively painless.

You want to /embed/ Python in your D program. You can certainly do this. 
The Python/C API has support for this. However, it is frequently easier 
to extend rather than embed. This page covers many of the arguments why:

http://www.twistedmatrix.com/users/glyph/rant/extendit.html

If you decide to go the embedding route, you'll basically end up using 
the Python/C API directly for most things. You can still use Pyd, 
though. It provides a number of very useful utilities for tying D to 
Python (e.g. value conversion functions), even if you can't use its 
high-level function and class wrapping interface. The PydObject class in 
particular may be useful. (Though some careful attention will need to be 
paid to how the D GC and the Python interpreter interact.)

Even if you don't use Pyd, you will need the D bindings to the Python/C 
API, the most complete version of which is part of the Pyd project. Bill 
already linked to these. I will link to them again:

http://dsource.org/projects/pyd/browser/trunk/infrastructure/python/python.d

Building such applications is not covered by Pyd. Usually, you just need 
to pass in the right version flags and link against the Python runtime.

-- 
Kirk McDonald
http://kirkmcdonald.blogspot.com
Pyd: Connecting D and Python
http://pyd.dsource.org
Sep 24 2007
parent reply "Jon" <SchaduwBlink hotmail.com> writes:
It sounds like either way is not that easy. -_- There wouldn't be a python 
to D convertor anywhere would there? Something like the .h to .d convertor?

"Kirk McDonald" <kirklin.mcdonald gmail.com> wrote in message 
news:fd9g3b$2sbh$1 digitalmars.com...

 Jon wrote:

 What would be higher level than this?

 I was hoping for something that would be easy to use and allow me to 
 include all the python modules. Is there an example of some kind for 
 python module you linked? I just import that in my D program and then 
 link to all the modules?

 What would be the best course of action for this many modules? Maybe 
 write a module in D that used that python module and all it does is wrap 
 the python modules to cleaner code? I looked briefly at the PyD site, but 
 they tend to have examples showing how to include D modules into Python. 
 :( Did I miss something?

 Thanks for replying and taking an interest in helping me. :)

 Oh, by the way, I use GDC on Linux. It is the latest GDC version. I am 
 also using Python v2.5.x


 Pyd is geared towards /extending/ Python with D code. It wraps much of the 
 raw Python/C API with a great deal of template trickery. It does this 
 pretty well, if I may say so.

 Pyd also comes with an extension to Python's distutils, which makes 
 building these extension modules relatively painless.

 You want to /embed/ Python in your D program. You can certainly do this. 
 The Python/C API has support for this. However, it is frequently easier to 
 extend rather than embed. This page covers many of the arguments why:

 http://www.twistedmatrix.com/users/glyph/rant/extendit.html

 If you decide to go the embedding route, you'll basically end up using the 
 Python/C API directly for most things. You can still use Pyd, though. It 
 provides a number of very useful utilities for tying D to Python (e.g. 
 value conversion functions), even if you can't use its high-level function 
 and class wrapping interface. The PydObject class in particular may be 
 useful. (Though some careful attention will need to be paid to how the D 
 GC and the Python interpreter interact.)

 Even if you don't use Pyd, you will need the D bindings to the Python/C 
 API, the most complete version of which is part of the Pyd project. Bill 
 already linked to these. I will link to them again:

 http://dsource.org/projects/pyd/browser/trunk/infrastructure/python/python.d

 Building such applications is not covered by Pyd. Usually, you just need 
 to pass in the right version flags and link against the Python runtime.

 -- 
 Kirk McDonald
 http://kirkmcdonald.blogspot.com
 Pyd: Connecting D and Python
 http://pyd.dsource.org
Sep 24 2007
next sibling parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Jon" <SchaduwBlink hotmail.com> wrote in message 
news:fd9inf$2voj$1 digitalmars.com...

 It sounds like either way is not that easy. -_- There wouldn't be a python 
 to D convertor anywhere would there? Something like the .h to .d 
 convertor?


No offense, but apples to oranges.  Converting a C header file to D is 
mostly a mechanical process -- there is (usually) no actual code to convert, 
just function and type declarations.  D is also (almost) a superset of C, 
meaning that semantically there's not much difference between C and D code.

Automatic translation of a language such as Python to D would be an 
incredibly complex undertaking.  Think writing a Python to native code 
compiler which preserved all the language semantics at runtime -- that's 
basically what you'd be doing, although not all the way to machine code, 
just to D code.  The languages are just too different for one to be 
converted to the other.  This is why Pyrex exists.  Probably about the 
closest you could _practically_ get would be something like Pyrex that 
outputted D code instead of C code.
Sep 24 2007
prev sibling next sibling parent Kirk McDonald <kirklin.mcdonald gmail.com> writes:
Jon wrote:

 It sounds like either way is not that easy. -_- There wouldn't be a 
 python to D convertor anywhere would there? Something like the .h to .d 
 convertor?
 


Please don't top-post.

No. There's no sensible way to directly use Python code in D code. 
Python is dynamically typed. It is highly introspective. It has 
eval/exec. Things such as integers, functions, and classes are 
first-class objects in Python. Python distinguishes between runtime and 
compile-time much differently than D does. (Much more happens at runtime 
in Python than in D.)

What you /can/ do, and what the PydObject class in Pyd does, is write a 
class wrapping a PyObject* which overloads all of the operators, and 
delegates them off to the appropriate Python/C API functions. A small 
example of PydObject's use can be found in the presentation I gave at 
the D conference:

http://pyd.dsource.org/dconf2007/presentation.html

Just search for "PydObject" on that page. Note that this demonstrates 
the use of PydObject when extending Python with Pyd. Using it when 
embedding Python will look a little different.

-- 
Kirk McDonald
http://kirkmcdonald.blogspot.com
Pyd: Connecting D and Python
http://pyd.dsource.org
Sep 24 2007
prev sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
Jon wrote:

 It sounds like either way is not that easy. -_- 


No, it's not 'import my.python.modules.*' easy.
But it's also not all that complicated, just a little tedious.


 There wouldn't be a 
 python to D convertor anywhere would there? Something like the .h to .d 
 convertor?


No, and there probably never will be because the languages are just too 
different.


 "Kirk McDonald" <kirklin.mcdonald gmail.com> wrote in message 
 news:fd9g3b$2sbh$1 digitalmars.com...

 Jon wrote:

 What would be higher level than this?

 I was hoping for something that would be easy to use and allow me to 
 include all the python modules. Is there an example of some kind for 
 python module you linked? I just import that in my D program and then 
 link to all the modules?

 What would be the best course of action for this many modules? Maybe 
 write a module in D that used that python module and all it does is 
 wrap the python modules to cleaner code? I looked briefly at the PyD 
 site, but they tend to have examples showing how to include D modules 
 into Python. :( Did I miss something?

 Thanks for replying and taking an interest in helping me. :)

 Oh, by the way, I use GDC on Linux. It is the latest GDC version. I 
 am also using Python v2.5.x


 Pyd is geared towards /extending/ Python with D code. It wraps much of 
 the raw Python/C API with a great deal of template trickery. It does 
 this pretty well, if I may say so.

 Pyd also comes with an extension to Python's distutils, which makes 
 building these extension modules relatively painless.

 You want to /embed/ Python in your D program. You can certainly do 
 this. The Python/C API has support for this. However, it is frequently 
 easier to extend rather than embed. This page covers many of the 
 arguments why:

 http://www.twistedmatrix.com/users/glyph/rant/extendit.html

 If you decide to go the embedding route, you'll basically end up using 
 the Python/C API directly for most things. You can still use Pyd, 
 though. It provides a number of very useful utilities for tying D to 
 Python (e.g. value conversion functions), even if you can't use its 
 high-level function and class wrapping interface. The PydObject class 
 in particular may be useful. (Though some careful attention will need 
 to be paid to how the D GC and the Python interpreter interact.)

 Even if you don't use Pyd, you will need the D bindings to the 
 Python/C API, the most complete version of which is part of the Pyd 
 project. Bill already linked to these. I will link to them again:

 http://dsource.org/projects/pyd/browser/trunk/infrastructure/python/python.d 


 Building such applications is not covered by Pyd. Usually, you just 
 need to pass in the right version flags and link against the Python 
 runtime.

 -- 
 Kirk McDonald
 http://kirkmcdonald.blogspot.com
 Pyd: Connecting D and Python
 http://pyd.dsource.org
Sep 24 2007
parent reply bearophile <bearophileHUGS lycos.com> writes:
Bill Baxter Wrote:

 No, and there probably never will be because the languages are just too 
 different.


I help the development of ShedSkin (sourceforge.net/projects/shedskin) with
Mark, it can convert pure, but implicitly statically typed Python programs into
optimized C++ code. Modifying it for D is well possible.

bearophile
Sep 26 2007
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
bearophile wrote:

 Bill Baxter Wrote:

 No, and there probably never will be because the languages are just too 
 different.


 
 I help the development of ShedSkin (sourceforge.net/projects/shedskin) with
Mark, it can convert pure, but implicitly statically typed Python programs into
optimized C++ code. Modifying it for D is well possible.


"Currently, these programs cannot freely use the Python standard library."

But ok, point taken.  Maybe some day there will be something that will 
be able to convert whatever python code you want to throw at it.  I 
guess there are lisp compilers that solve a similarly challenging problem.

--bb
Sep 26 2007
parent "Jon" <SchaduwBlink hotmail.com> writes:
Thanks for the info on this. I will definately take a look at this program. 
:)

Thanks for all your help all. Development is a little slow, but I am 
learning.
"Bill Baxter" <dnewsgroup billbaxter.com> wrote in message 
news:fddeb1$1rqb$1 digitalmars.com...

 bearophile wrote:

 Bill Baxter Wrote:

 No, and there probably never will be because the languages are just too 
 different.


 I help the development of ShedSkin (sourceforge.net/projects/shedskin) 
 with Mark, it can convert pure, but implicitly statically typed Python 
 programs into optimized C++ code. Modifying it for D is well possible.


 "Currently, these programs cannot freely use the Python standard library."

 But ok, point taken.  Maybe some day there will be something that will be 
 able to convert whatever python code you want to throw at it.  I guess 
 there are lisp compilers that solve a similarly challenging problem.

 --bb
Sep 30 2007
prev sibling parent reply BCS <BCS pathlink.com> writes:
Jon wrote:

 Hi,
 
 I am fairly new to D, and I would like to write a program that uses a 
 bunch of modules that were written in Python. This is the reverse of 
 what I have read where people use D written dlls in a python program. I 
 have a lot of modules that would take too long to translate to D: a lot 
 of files and several thousand lines of code each. So, it would be nice 
 if I could use these modules with the D program. There are just some 
 features of D that I want to use and I can only get away with these 
 features if the program is a compiled program. Sometimes the python 
 modules will not be used, for example, there is a safe mode that uses no 
 external libraries at all in case something is broken, then you can at 
 least start the program to back things up properly. The only way to do 
 that is to have a compiled binary. Plus, I want to learn more about D 
 application writing and not dll writing. :P I would appreciate it if 
 someone could explain it very thoroughly.
 
 Thanks,
 Jon


http://pyd.dsource.org/
Sep 24 2007
parent BCS <BCS pathlink.com> writes:
BCS wrote:

 Jon wrote:
 

 Hi,

 I am fairly new to D, and I would like to write a program that uses a 
 bunch of modules that were written in Python. This is the reverse of 
 what I have read where people use D written dlls in a python program. 
 I have a lot of modules that would take too long to translate to D: a 
 lot of files and several thousand lines of code each. So, it would be 
 nice if I could use these modules with the D program. There are just 
 some features of D that I want to use and I can only get away with 
 these features if the program is a compiled program. Sometimes the 
 python modules will not be used, for example, there is a safe mode 
 that uses no external libraries at all in case something is broken, 
 then you can at least start the program to back things up properly. 
 The only way to do that is to have a compiled binary. Plus, I want to 
 learn more about D application writing and not dll writing. :P I would 
 appreciate it if someone could explain it very thoroughly.

 Thanks,
 Jon


 
 
 http://pyd.dsource.org/


I'm fairly sure that this will have something for you
Sep 24 2007