www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - bigint <-> python long

reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
Hey.

Investigating the possibility of providing this conversion in pyd.

Python provides an api for accessing the underlying bytes.

std.bigint seemingly doesn't. Am I missing anything?
Sep 05 2012
next sibling parent reply "Paul D. Anderson" <paul.d.removethis.anderson comcast.andthis.net> writes:
On Wednesday, 5 September 2012 at 18:13:40 UTC, Ellery Newcomer 
wrote:

 Hey.

 Investigating the possibility of providing this conversion in 
 pyd.

 Python provides an api for accessing the underlying bytes.

 std.bigint seemingly doesn't. Am I missing anything?


No, I don't believe so. AFAIK there is no public access to the 
underlying array, but I think it is a good idea.

I suspect the reason for not disclosing the details is to 
disallow anyone putting the data into an invalid state. But 
read-only access would be safe.
Sep 05 2012
next sibling parent "Paul D. Anderson" <paul.d.removethis.anderson comcast.andthis.net> writes:
On Wednesday, 5 September 2012 at 19:23:11 UTC, Paul D. Anderson 
wrote:


 No, I don't believe so. AFAIK there is no public access to the 
 underlying array, but I think it is a good idea.


I meant to say I think that access to the array is a good idea, 
not the lack of access. Words are hard!
Sep 05 2012
prev sibling parent Don Clugston <dac nospam.com> writes:
On 05/09/12 21:23, Paul D. Anderson wrote:

 On Wednesday, 5 September 2012 at 18:13:40 UTC, Ellery Newcomer wrote:

 Hey.

 Investigating the possibility of providing this conversion in pyd.

 Python provides an api for accessing the underlying bytes.

 std.bigint seemingly doesn't. Am I missing anything?


 No, I don't believe so. AFAIK there is no public access to the
 underlying array, but I think it is a good idea.

 I suspect the reason for not disclosing the details is to disallow
 anyone putting the data into an invalid state. But read-only access
 would be safe.


No, it's just not disclosed because I didn't know the best way to do it.
I didn't want to put something in unless I was sure it was correct.
(And a key part of that, is what is required to implement BigFloat).
Sep 06 2012
prev sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:


 Investigating the possibility of providing this conversion in 
 pyd.


Are you updating Pyd? :-)

Bye,
bearophile
Sep 05 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 05:02 PM, bearophile wrote:

 Ellery Newcomer:


 Investigating the possibility of providing this conversion in pyd.


 Are you updating Pyd? :-)

 Bye,
 bearophile


Yep. Have any suggestions for supported conversion out of the box? From 
the standard library, I already have Complex, Tuple, and now BigInt. Was 
thinking maybe the datetime stuff..

Also, which of the following looks more appealing to you?


wrap_class!(Foo, Def!(Foo.bar, PyName!"__len__"))();

wrap_class!(Foo, Len!(Foo.bar))();


they wrap Foo and give it an overriden __len__; the latter is how it is 
done at the moment (with similar templates for the other operators), but 
boost::python does something akin to the former. PyName is in the former 
because Def takes too many optional string arguments.
Sep 05 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:


 Yep.


Oh, good.



 Have any suggestions for supported conversion out of the box?


There are several important cases, like:

Some D lazy ranges <==> Python lazy iterators/generators

array.array <==> D arrays

NumPy arrays <==> D arrays



 Also, which of the following looks more appealing to you?


I don't know.

Bye,
bearophile
Sep 05 2012
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 07:10 PM, bearophile wrote:

 Some D lazy ranges <==> Python lazy iterators/generators


I'll look into this one.


 array.array <==> D arrays


just checked, looks like we have it:

PyStmts(q"{from array import array; a = array('i', [44,33,22,11]);}", 
"testing");
assert(PyEval!(int[])("a", "testing") == [44,33,22,11]);

I think if the python object is iterable, it can be converted to array.

Matrices might pose a problem. But user can define custom conversions if 
need be.


 NumPy arrays <==> D arrays


I have never used NumPy. Are its arrays special?


 Also, which of the following looks more appealing to you?


 I don't know.


ok, I'll be lazy then.
Sep 05 2012
next sibling parent reply Russel Winder <russel winder.org.uk> writes:
Sorry, I missed earlier bits of this thread=E2=80=A6

On Wed, 2012-09-05 at 19:37 -0700, Ellery Newcomer wrote:

 On 09/05/2012 07:10 PM, bearophile wrote:

 Some D lazy ranges <=3D=3D> Python lazy iterators/generators


=20
 I'll look into this one.
=20

 array.array <=3D=3D> D arrays


=20
 just checked, looks like we have it:
=20
 PyStmts(q"{from array import array; a =3D array('i', [44,33,22,11]);}",=


=20

 "testing");
 assert(PyEval!(int[])("a", "testing") =3D=3D [44,33,22,11]);
=20
 I think if the python object is iterable, it can be converted to array.


I am guessing this is interfacing to CPython, remember there is also
PyPy and ActiveState, they have different ways of doing things.  Well
the ActiveState C API will be very close to the CPython C API, but PyPy
(which is the best Python 2.7 just now) doesn't have a C API since it is
written in RPython.=20


 Matrices might pose a problem. But user can define custom conversions if=


=20

 need be.
=20

 NumPy arrays <=3D=3D> D arrays


=20
 I have never used NumPy. Are its arrays special?


Oh yes.

NumPy is basically a subsystem that Python applications make calls into.
Although the data structure can be accessed and amended, algorithms on
the data structures should never be written in Python, they should
always be function calls into the NumPy framework.


 Also, which of the following looks more appealing to you?


 I don't know.


=20
 ok, I'll be lazy then.
=20


--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Sep 06 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/06/2012 12:07 AM, Russel Winder wrote:

 I am guessing this is interfacing to CPython, remember there is also
 PyPy and ActiveState, they have different ways of doing things.  Well
 the ActiveState C API will be very close to the CPython C API, but PyPy
 (which is the best Python 2.7 just now) doesn't have a C API since it is
 written in RPython.


Yep, CPython for now. If activestate supports the CPython C API, then 
supporting it will probably just be a matter of linkage.

Googling, it looks like PyPy is/was trying to grow some CPython C API 
compatibility, so hopefully we can squeeze it in at some point.


 Oh yes.

 NumPy is basically a subsystem that Python applications make calls into.
 Although the data structure can be accessed and amended, algorithms on
 the data structures should never be written in Python, they should
 always be function calls into the NumPy framework.


I had a look at numpy, and at least ndarray supports the new style 
buffer interface (which is pretty freaking complicated), so convertion 
is totally doable. I'll fold in support as soon as I can get dmd to stop 
stabbing me in the face.


just used your scons fork to build the pyd embedded unittests. works 
pretty nice
Sep 07 2012
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:


 I had a look at numpy, and at least ndarray supports the new 
 style buffer interface (which is pretty freaking complicated), 
 so convertion is totally doable. I'll fold in support as soon 
 as I can get dmd to stop stabbing me in the face.


If you support NumPy well and efficiently through Pyd, I think 
some people will start using the D language just for this :-)

NumPy is good but it usually forces you to program vectorially, 
like Matlab. Sometimes this is not nice, so people write some 
routines in Fortran, C, etc. With Pyd some scientific programmers 
maybe will want to use D instead.

Bye,
bearophile
Sep 07 2012
parent Russel Winder <russel winder.org.uk> writes:
On Sat, 2012-09-08 at 00:58 +0200, bearophile wrote:
[=E2=80=A6]

 If you support NumPy well and efficiently through Pyd, I think=20
 some people will start using the D language just for this :-)
=20
 NumPy is good but it usually forces you to program vectorially,=20
 like Matlab. Sometimes this is not nice, so people write some=20
 routines in Fortran, C, etc. With Pyd some scientific programmers=20
 maybe will want to use D instead.


The idea of a Python/NumPy/D combination, especially if connected to
OpenCL does have great appeal. NumPy and D for computation and Python
for data visualization and coordination would be an improvement over the
currently canonical Python/NumPy/C/C++/Fortran. But there is a large
inertia to the system that has built up in the bioinformatics and HEP
communities.
--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Sep 08 2012
prev sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Fri, 2012-09-07 at 15:21 -0700, Ellery Newcomer wrote:

 On 09/06/2012 12:07 AM, Russel Winder wrote:


[=E2=80=A6]

 just used your scons fork to build the pyd embedded unittests. works=20
 pretty nice


Splendid :-)

If you see any problems or glitches, let me know and/or post a bug
report.  Although I said use the BitBucket issue system, that was a hang
over from the days of a totally separate package for the D system. Now
that D support is being evolved as a fork, it has been pointed out to me
I should have pointed people to the SCons issue tracker
http://scons.tigris.org/project_issues.html for posting reports about D
tooling issues as they are in the SCons core.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Sep 08 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/08/2012 03:09 AM, Russel Winder wrote:

 On Fri, 2012-09-07 at 15:21 -0700, Ellery Newcomer wrote:

 On 09/06/2012 12:07 AM, Russel Winder wrote:


 […]

 just used your scons fork to build the pyd embedded unittests. works
 pretty nice


 Splendid :-)


Okay, here: https://bitbucket.org/ariovistus/deimos-elfutils/overview

I have some code with a working makefile and a nonworking SConstruct file.

I believe the issue is the header files have pragma(lib, X) in them, and 
a single call to dmd links the appropriate lib in, but scons' link step 
loses that information.

Do you have any intention of supporting pragma(lib) in scons?
Sep 08 2012
parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2012-09-08 at 07:20 -0700, Ellery Newcomer wrote:
[=E2=80=A6]

 Okay, here: https://bitbucket.org/ariovistus/deimos-elfutils/overview
=20
 I have some code with a working makefile and a nonworking SConstruct file=


.

=20
 I believe the issue is the header files have pragma(lib, X) in them, and=


=20

 a single call to dmd links the appropriate lib in, but scons' link step=


=20

 loses that information.
=20
 Do you have any intention of supporting pragma(lib) in scons?


If that is valid Dv2 and SCons doesn't deal with it then the SCons D
tools are broken and need fixing.

Is there a tiny project replicating this that I can turn into a
unit/system test. The red so caused will necessitate action :-)

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Sep 08 2012
parent reply Johannes Pfau <nospam example.com> writes:
Am Sat, 08 Sep 2012 16:25:49 +0100
schrieb Russel Winder <russel winder.org.uk>:


 On Sat, 2012-09-08 at 07:20 -0700, Ellery Newcomer wrote:
 [=E2=80=A6]

 Okay, here:
 https://bitbucket.org/ariovistus/deimos-elfutils/overview
=20
 I have some code with a working makefile and a nonworking
 SConstruct file.
=20
 I believe the issue is the header files have pragma(lib, X) in
 them, and a single call to dmd links the appropriate lib in, but
 scons' link step loses that information.
=20
 Do you have any intention of supporting pragma(lib) in scons?


=20
 If that is valid Dv2 and SCons doesn't deal with it then the SCons D
 tools are broken and need fixing.
=20
 Is there a tiny project replicating this that I can turn into a
 unit/system test. The red so caused will necessitate action :-)
=20


Please note that pragma(lib) is an evil feature. For example it will
never work in gdc.
Sep 09 2012
next sibling parent Russel Winder <russel winder.org.uk> writes:
On Sun, 2012-09-09 at 10:15 +0200, Johannes Pfau wrote:
[=E2=80=A6]

 Please note that pragma(lib) is an evil feature. For example it will
 never work in gdc.


So this is a DMD-only (*) feature and not a feature of the D programming
language per se?

(*) and hence LDC.
--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Sep 09 2012
prev sibling next sibling parent reply " bearophile" <bearophileHUGS lycos.com> writes:
Johannes Pfau:


 pragma(lib) is an evil feature. For example it will never work 
 in gdc.


Do you know why it's impossible to implement it in GDC?

And if it's impossible to implement in GDC, then maybe it's
better to ask Walter to deprecate its usage in general.

Bye,
bearophile
Sep 09 2012
parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 09 Sep 2012 13:53:23 +0200
schrieb " bearophile" <bearophileHUGS lycos.com>:


 
 Do you know why it's impossible to implement it in GDC?
 
 And if it's impossible to implement in GDC, then maybe it's
 better to ask Walter to deprecate its usage in general.
 
 Bye,
 bearophile


IIRC this is the reason:
GCC programs are always split into 2 executables, for gdc it's 'gdc'
and 'cc1d' (for c it's gcc and cc1, etc).

This allows the gcc executable to compile d sources, you can actually
call gcc file.d and it'll invoke cc1d.

cc1d parses the source files and compiles code, so cc1d is the
tool which knows about pragma(lib). but cc1d is not responsible for
linking, linking is done by gdc. And IIRC there's no easy, portable way
to pass the information from cc1d to gdc.

So putting linker flags in source file doesn't fit gcc very well.

Note that it may not work with dmd either when linking incrementally.
Putting linker flags into source files is just not a good idea:

test1.d
----
pragma(lib, "curl");
----

test2.d
----
import std.net.curl;
import test1;

void main()
{
    Curl c;
}
----

dmd test1.d test2.d //OK
dmd test1.d -c
dmd test2.d test1.o //FAIL
Sep 09 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Johannes Pfau:


 Putting linker flags into source files is just not a good idea:


Thank you for the explanation. Then maybe we should deprecate 
pragma(lib).

Bye,
bearophile
Sep 09 2012
next sibling parent "Adam D. Ruppe" <destructionator gmail.com> writes:
On Sunday, 9 September 2012 at 14:26:48 UTC, bearophile wrote:

 Thank you for the explanation. Then maybe we should deprecate 
 pragma(lib).


If it doesn't work, there's no difference between having it and 
not - you have to do it yourself anyway. So deprecation 
accomplishes nothing except breaking it for the people who *do* 
use it.

At least the pragma could be printed out to the user (treat it 
like pragma(msg) where lib isn't supported?) or read by some 
other tool even if the compiler doesn't use it.
Sep 09 2012
prev sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/09/2012 07:27 AM, bearophile wrote:

 Johannes Pfau:


 Putting linker flags into source files is just not a good idea:


 Thank you for the explanation. Then maybe we should deprecate 


pragma(lib).

 Bye,
 bearophile


Similar to pragmas in C and C++, D pragmas allow language extensions: 
"Pragmas are a way to pass special information to the compiler and to 
add vendor specific extensions to D."

   http://dlang.org/pragma.html

Ali
Sep 09 2012
parent reply Russel Winder <russel winder.org.uk> writes:
On Sun, 2012-09-09 at 08:05 -0700, Ali =C3=87ehreli wrote:
[=E2=80=A6]

 Similar to pragmas in C and C++, D pragmas allow language extensions:=20
 "Pragmas are a way to pass special information to the compiler and to=20
 add vendor specific extensions to D."
=20
    http://dlang.org/pragma.html


This states that:

	pragma(lib, "blah.lib")

is a part of the D language and not just a DMD extension.  This means
that the code is platform specific (due to extensions embedded in code)
as well as predicating that the toolchain will deal with explicit
toolchain information in the source code.

Personally I would say that putting dependencies such as this in the
source code is anathema, it is the role of the build framework to deal
with this. That way you can deal with the platform specific extensions
and naming as well as toolchain issues. These dependencies are link time
not source time ones.

So I am with Johannes, pragma(lib,=E2=80=A6) is "evil" and should be remove=
d.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Sep 09 2012
parent =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 09/09/2012 08:48 AM, Russel Winder wrote:

 On Sun, 2012-09-09 at 08:05 -0700, Ali Çehreli wrote:



     http://dlang.org/pragma.html


 This states that:

 	pragma(lib, "blah.lib")

 is a part of the D language and not just a DMD extension.


I hadn't read that far. :*)


 Personally I would say that putting dependencies such as this in the
 source code is anathema, it is the role of the build framework to deal
 with this.


Agreed.

Ali
Sep 09 2012
prev sibling parent reply Brad Roberts <braddr puremagic.com> writes:
On 9/9/2012 1:15 AM, Johannes Pfau wrote:

 Am Sat, 08 Sep 2012 16:25:49 +0100
 schrieb Russel Winder <russel winder.org.uk>:
 

 On Sat, 2012-09-08 at 07:20 -0700, Ellery Newcomer wrote:
 […]

 Okay, here:
 https://bitbucket.org/ariovistus/deimos-elfutils/overview

 I have some code with a working makefile and a nonworking
 SConstruct file.

 I believe the issue is the header files have pragma(lib, X) in
 them, and a single call to dmd links the appropriate lib in, but
 scons' link step loses that information.

 Do you have any intention of supporting pragma(lib) in scons?


 If that is valid Dv2 and SCons doesn't deal with it then the SCons D
 tools are broken and need fixing.

 Is there a tiny project replicating this that I can turn into a
 unit/system test. The red so caused will necessitate action :-)


 
 Please note that pragma(lib) is an evil feature. For example it will
 never work in gdc.
 


It's not impossible and never is rather defeatist.  Using the frontend as is
and grabing the json output, part of which
includes the pragmas, would be easy.  Then invoking gdc with the appropriate
options to get the library linked in.  rdmd
is a good example of this sort of process.
Sep 09 2012
parent reply Johannes Pfau <nospam example.com> writes:
Am Sun, 09 Sep 2012 12:55:19 -0700
schrieb Brad Roberts <braddr puremagic.com>:


 On 9/9/2012 1:15 AM, Johannes Pfau wrote:

 Am Sat, 08 Sep 2012 16:25:49 +0100
 schrieb Russel Winder <russel winder.org.uk>:
=20

 On Sat, 2012-09-08 at 07:20 -0700, Ellery Newcomer wrote:
 [=E2=80=A6]

 Okay, here:
 https://bitbucket.org/ariovistus/deimos-elfutils/overview

 I have some code with a working makefile and a nonworking
 SConstruct file.

 I believe the issue is the header files have pragma(lib, X) in
 them, and a single call to dmd links the appropriate lib in, but
 scons' link step loses that information.

 Do you have any intention of supporting pragma(lib) in scons?


 If that is valid Dv2 and SCons doesn't deal with it then the SCons
 D tools are broken and need fixing.

 Is there a tiny project replicating this that I can turn into a
 unit/system test. The red so caused will necessitate action :-)


=20
 Please note that pragma(lib) is an evil feature. For example it will
 never work in gdc.
=20


=20
 It's not impossible and never is rather defeatist.  Using the
 frontend as is and grabing the json output, part of which includes
 the pragmas, would be easy.  Then invoking gdc with the appropriate
 options to get the library linked in.  rdmd is a good example of this
 sort of process.
=20
=20


Is there a special flag to enable pragmas for the json output? It does
not work with gdc right now, but it should be possible to make it work.

Sorry, I should have said 'It'll _probably_ never be supported in gdc'.
There are some possible solutions but:

* It must be good enough to get approved when gdc is merged into gcc.
  (remember it must be portable and gpl and you can't use
  stdout/stdin...)
* Someone would have to implement the solution. I guess Iain had his
  reasons not to implement it so somebody else would have to do that.

Of course you can always try to make it work with external build
tools. But a solution _in_ gdc seems not very likely.


I don't want to badmouth the pragma(lib) feature, in some cases it's
nice to have (mainly building simple script-like programs with few
source files). But for bigger projects, pragma(lib) makes things
difficult (incremental compilation; build tools usually check if a
library is available before trying to link against it so they can put
out a nice warning. pragma(lib) in dmd subverts this feature; can't
specify linker path with pragma lib, can't specify static vs dynamic
linking, ...).

The C/C++ architecture splits compilation and linking. Trying to
conflate those concepts as pragma(lib) does, might even be a good
idea(other languages have done it for some time now). But as we have to
deal with tools that were designed for C/C++ (linkers, gcc) we'll
always hit some issues with pragma(lib).
Sep 10 2012
parent Johannes Pfau <nospam example.com> writes:
Am Mon, 10 Sep 2012 14:48:30 +0200
schrieb Johannes Pfau <nospam example.com>:


 Sorry, I should have said 'It'll _probably_ never be supported in
 gdc'. There are some possible solutions but:
 
 * It must be good enough to get approved when gdc is merged into gcc.
   (remember it must be portable and gpl and you can't use
   stdout/stdin...)
 * Someone would have to implement the solution. I guess Iain had his
   reasons not to implement it so somebody else would have to do that.
 
 Of course you can always try to make it work with external build
 tools. But a solution _in_ gdc seems not very likely.


For reference: Here's the gdc bug report for pragma(lib):
http://d.puremagic.com/issues/show_bug.cgi?id=1690

Filed 2007, closed 2012 as RESOLVED/WONTFIX.
Sep 10 2012
prev sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:


 array.array <==> D arrays


 just checked, looks like we have it:

 PyStmts(q"{from array import array; a = array('i', 
 [44,33,22,11]);}", "testing");
 assert(PyEval!(int[])("a", "testing") == [44,33,22,11]);

 I think if the python object is iterable, it can be converted 
 to array.


array.array are special, they aren't Python lists. array.array 
contains uniform data, so conversion to D arrays is a memcpy (or 
it's nearly nothing if you don't copy the data).

Bye,
bearophile
Sep 06 2012
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/06/2012 04:11 AM, bearophile wrote:

 Ellery Newcomer:


 array.array <==> D arrays


 just checked, looks like we have it:

 PyStmts(q"{from array import array; a = array('i', [44,33,22,11]);}",
 "testing");
 assert(PyEval!(int[])("a", "testing") == [44,33,22,11]);

 I think if the python object is iterable, it can be converted to array.


 array.array are special, they aren't Python lists. array.array contains
 uniform data, so conversion to D arrays is a memcpy (or it's nearly
 nothing if you don't copy the data).

 Bye,
 bearophile


I see. The docs for array.array suggest that it implements the buffer 
interface, but it doesn't seem to implement new or old style buffers, at 
least according to PyObject_CheckBuffer and PyBuffer_Check.

I think I'll add support for new style buffers anyways. a memoryview 
would be good, too.

Guess I'll hack together a special case for array using buffer_info.
Sep 06 2012
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-09-06 04:10, bearophile wrote:


 There are several important cases, like:

 Some D lazy ranges <==> Python lazy iterators/generators

 array.array <==> D arrays

 NumPy arrays <==> D arrays


Associative arrays?

-- 
/Jacob Carlborg
Sep 05 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 11:19 PM, Jacob Carlborg wrote:

 On 2012-09-06 04:10, bearophile wrote:


 There are several important cases, like:

 Some D lazy ranges <==> Python lazy iterators/generators

 array.array <==> D arrays

 NumPy arrays <==> D arrays


 Associative arrays?


check.

https://bitbucket.org/ariovistus/pyd/wiki/TypeConversion
Sep 06 2012
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/06/2012 09:48 AM, Ellery Newcomer wrote:

 On 09/05/2012 11:19 PM, Jacob Carlborg wr

 Associative arrays?


 check.


eh, that was check as in yes, not check as in look it up yourself.

didn't seem ambiguous at the time.
Sep 07 2012
prev sibling next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 07:10 PM, bearophile wrote:

 NumPy arrays <==> D arrays


I've been thinking about this one a bit more, and I am not sure it 
belongs in pyd.

First, the conversion is not symmetric. One can convert a numpy.ndarray 
to a d array like so:

PyObject* ndarray;
double[][] matrix = d_type!(double[][])(ndarray);

however, going back

PyObject* res = _py(matrix);

It is not at all clear that the user wants res to be a numpy.ndarray. 
The problem is partially that d arrays would be overloaded to a few too 
many things (list, str, array, any iterable, any buffer). That last one 
is a doozy. d_type never actually touches ndarray's type, so _py can 
hardly know what to use to convert matrix. (what if ndarray is actually 
a foo.BizBar matrix?)

I could just specialize _py for numpy.ndarrays, defaulting to lists of 
lists (which is what we do already), but I kinda want a specialized type 
for numpy.ndarrays.

Also, all these conversions imply data copying; is this reasonable for 
numpy arrays?

It is easy enough to get a void* and shape information out of the 
ndarray, but building a decent matrix type out of them is not trivial. 
Is there a good matrix library for D that would be suitable for this?

Oh yeah, also: rectangular matrices. For static arrays, the conversion 
is 1 memcpy. For dynamic arrays: lots of memcpys. I suppose I could 
abuse slicing much.
Sep 10 2012
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Ellery Newcomer:


 I've been thinking about this one a bit more, and I am not sure 
 it belongs in pyd.


I understand. The point of Pyd is to interface D and Python, 
while NumPy is something external. So if you find difficulties 
just keep it out. Adding it later is possible.

Bye,
bearophile
Sep 10 2012
parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/10/2012 12:11 PM, bearophile wrote:

 I understand. The point of Pyd is to interface D and Python, while NumPy
 is something external. So if you find difficulties just keep it out.
 Adding it later is possible.


Thing is, pyd will convert a ndarray to d array already, it just won't 
do it as quickly as it could if it made use of the underlying c array, and

_py(d_type!(double[][])(ndarray))

will result in a list of lists.

So it's really a question of should I add more oddness to an already odd 
situation.


<OT> Bugger, I'm going to have to go through pyd and replace all usages 
of str with unicode. </OT>
Sep 10 2012
parent reply Russel Winder <russel winder.org.uk> writes:
On Mon, 2012-09-10 at 15:54 -0700, Ellery Newcomer wrote:
[=E2=80=A6]

 <OT> Bugger, I'm going to have to go through pyd and replace all usages=


=20

 of str with unicode. </OT>


Python 2 and Python 3 are totally different in this regard. I don't have
a obvious proposal to make to avoid having PyD for Python 2 and a
different PyD for Python 3, but the six package might have some hints as
it is intended to support creating Python codebases guaranteed to run
under Python 2 and Python 3.

It is a pity the world doesn't just spontaneously switch to Python 3 so
that Python 2 is just a "we used to use that" technology."

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Sep 10 2012
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/10/2012 10:50 PM, Russel Winder wrote:

 Python 2 and Python 3 are totally different in this regard. I don't have
 a obvious proposal to make to avoid having PyD for Python 2 and a
 different PyD for Python 3, but the six package might have some hints as
 it is intended to support creating Python codebases guaranteed to run
 under Python 2 and Python 3.


Pyd doesn't really have a python codebase, I was talking mostly about 
PyString_AsString -> PyUnicode_Whatever, since even for Python 2, 
unicode is much more appropriate for anything interfacing with D.

For getting pyd to support python 3, its mostly a matter of choosing the 
right C API functions, and anyways I have version identifiers I can rely 
on if there is divergence.

Wait, CeleriD is python. I might need that six package after all. Thanks 
for the tip.
Sep 11 2012
prev sibling parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 09/05/2012 07:10 PM, bearophile wrote:

 Ellery Newcomer:


 Yep.


 Oh, good.



 Have any suggestions for supported conversion out of the box?


 There are several important cases, like:

 Some D lazy ranges <==> Python lazy iterators/generators

 array.array <==> D arrays

 NumPy arrays <==> D arrays


Welp. I started on NumPy arrays <== D arrays, and it turned out to be 
pretty easy. It's in its own function; maybe I'll put it in pyd.extras 
or something. But now I have just about all of the above cases working.

Bearophile: would you be interested in contributing some code showcasing 
what we can do with numpy? Just, say, a D function operating on D arrays 
that does something that maybe numpy doesn't have built in.
Sep 11 2012