www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Calypso and the future of D

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I think it's important that we enable Calypso 
(https://github.com/Syniurge/Calypso) and related tooling that 
interfaces D with other languages, notably C++.

A key topic in 2015 for D is playing well with C++. A good C++ interface 
will give access to a host of mature C++ libraries, starting of course 
with the C++ standard library. More importantly, it will provide a 
smooth migration path for organizations that want to do development in D 
whilst taking advantage of their legacy code.

I'd like to open the topic of "what can we do in core D to make Calypso 
better".

But first, I want to get better acquainted with Calypso and raise 
awareness of it with coworkers and the larger community. To my dismay, 
the github homepage provides exactly zero "look and feel" use examples 
beyond the mechanics of building Calypso itself.

To Calypso's creator: is it possible to beef up the documentation of 
Calypso with a few use cases? Ideally there'd be a soup-to-nuts step by 
step guide of making a C++ library (either artificial or extant) 
interfacing with D. Also: what things on the D side would be necessary 
to make the interface better? Exceptions would be an obvious topic on 
which we have an attack already (more on it later).


Thanks,

Andrei
Jan 22 2015
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jan 22, 2015 at 04:24:44PM -0800, Andrei Alexandrescu via Digitalmars-d
wrote:

 I think it's important that we enable Calypso
 (https://github.com/Syniurge/Calypso) and related tooling that interfaces D
 with other languages, notably C++.


[...]

 But first, I want to get better acquainted with Calypso and raise
 awareness of it with coworkers and the larger community. To my dismay,
 the github homepage provides exactly zero "look and feel" use examples
 beyond the mechanics of building Calypso itself.


What exactly is Calypso and what is it supposed to do? The README is
unhelpful in this respect. All I can tell is that it's a fork of ldc.
But why? What does it afford beyond stock ldc? No explanation is given.


T

-- 
Don't drink and derive. Alcohol and algebra don't mix.
Jan 22 2015
parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 23 January 2015 at 00:31:54 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Thu, Jan 22, 2015 at 04:24:44PM -0800, Andrei Alexandrescu 
 via Digitalmars-d wrote:

 I think it's important that we enable Calypso
 (https://github.com/Syniurge/Calypso) and related tooling that 
 interfaces D
 with other languages, notably C++.


 [...]

 But first, I want to get better acquainted with Calypso and 
 raise
 awareness of it with coworkers and the larger community. To my 
 dismay,
 the github homepage provides exactly zero "look and feel" use 
 examples
 beyond the mechanics of building Calypso itself.


 What exactly is Calypso and what is it supposed to do? The 
 README is
 unhelpful in this respect. All I can tell is that it's a fork 
 of ldc.
 But why? What does it afford beyond stock ldc? No explanation 
 is given.


 T


http://forum.dlang.org/post/nsjafpymezlqdknmnkhi forum.dlang.org
Jan 22 2015
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Jan 23, 2015 at 12:33:18AM +0000, Dicebot via Digitalmars-d wrote:

 On Friday, 23 January 2015 at 00:31:54 UTC, H. S. Teoh via Digitalmars-d
 wrote:


[...]

What exactly is Calypso and what is it supposed to do? The README is
unhelpful in this respect. All I can tell is that it's a fork of ldc.
But why? What does it afford beyond stock ldc? No explanation is
given.


T


 
 http://forum.dlang.org/post/nsjafpymezlqdknmnkhi forum.dlang.org


Thanks!

It would really help if this info was in the README... not just to me,
but to a casual observer.


T

-- 
Microsoft is to operating systems & security ... what McDonalds is to gourmet
cooking.
Jan 22 2015
prev sibling next sibling parent Paul O'Neil <redballoon36 gmail.com> writes:
First, a general increase in quality & test coverage.  If C++ interop is
a priority, it makes sense to focus on ironing out what's already there.
 I've filed a few bugs over the last 2 months or so and I don't think
I'm trying anything too far out there.  Some of these are  internal
compiler errors that probably should have been detected earlier in
compilation.  The new C++ keyword in bugzilla may help here.

I'm curious to know about others' experience with the namespace design
is.  I'm skeptical, but am not too familiar with the reasoning yet.  It
does lead to problems like
https://issues.dlang.org/show_bug.cgi?id=13183 .  Does anyone have a
example of why this worked great for them?


-- 
Paul O'Neil
Github / IRC: todayman
Jan 22 2015
prev sibling next sibling parent "weaselcat" <weaselcat gmail.com> writes:
On Friday, 23 January 2015 at 00:24:45 UTC, Andrei Alexandrescu 
wrote:

 I think it's important that we enable Calypso 
 (https://github.com/Syniurge/Calypso) and related tooling that


Safe to assume this will be LDC only?
Jan 22 2015
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-23 01:24, Andrei Alexandrescu wrote:

 I think it's important that we enable Calypso
 (https://github.com/Syniurge/Calypso) and related tooling that
 interfaces D with other languages, notably C++.


I could always use some help with DStep [1]. For those who doesn't know, 
it's an automated binding generator for C and Objective-C. It doesn't 
support C++ yet.

For other languages I've recently opened a pull request with the first 
steps for interfacing with Objective-C [2]. Help with reviewing is 
appreciated.

[1] https://github.com/jacob-carlborg/dstep
[2] https://github.com/D-Programming-Language/dmd/pull/4321

-- 
/Jacob Carlborg
Jan 22 2015
prev sibling next sibling parent reply Max Klyga <max.klyga gmail.com> writes:
On 2015-01-23 00:24:44 +0000, Andrei Alexandrescu said:


 I think it's important that we enable Calypso 
 (https://github.com/Syniurge/Calypso) and related tooling that 
 interfaces D with other languages, notably C++.
 
 A key topic in 2015 for D is playing well with C++. A good C++ 
 interface will give access to a host of mature C++ libraries, starting 
 of course with the C++ standard library. More importantly, it will 
 provide a smooth migration path for organizations that want to do 
 development in D whilst taking advantage of their legacy code.
 
 I'd like to open the topic of "what can we do in core D to make Calypso 
 better".
 
 But first, I want to get better acquainted with Calypso and raise 
 awareness of it with coworkers and the larger community. To my dismay, 
 the github homepage provides exactly zero "look and feel" use examples 
 beyond the mechanics of building Calypso itself.
 
 To Calypso's creator: is it possible to beef up the documentation of 
 Calypso with a few use cases? Ideally there'd be a soup-to-nuts step by 
 step guide of making a C++ library (either artificial or extant) 
 interfacing with D. Also: what things on the D side would be necessary 
 to make the interface better? Exceptions would be an obvious topic on 
 which we have an attack already (more on it later).
 
 
 Thanks,
 
 Andrei


I remember Walter arguing against this type of solutions because it 
requires a C++ compiler to be bundled with D.
Does that mean that now we are adopting this path instead of 'extern(C++)' ?
Jan 23 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/23/2015 12:34 AM, Max Klyga wrote:

 I remember Walter arguing against this type of solutions because it requires a
 C++ compiler to be bundled with D.


Yes.


 Does that mean that now we are adopting this path instead of 'extern(C++)' ?


No. I presume it will work much like htod.exe, and extern(C++) will be what 
Calypso generates.
Jan 23 2015
parent "Dicebot" <public dicebot.lv> writes:
On Friday, 23 January 2015 at 10:52:26 UTC, Walter Bright wrote:

 On 1/23/2015 12:34 AM, Max Klyga wrote:

 I remember Walter arguing against this type of solutions 
 because it requires a
 C++ compiler to be bundled with D.


 Yes.


 Does that mean that now we are adopting this path instead of 
 'extern(C++)' ?


 No. I presume it will work much like htod.exe, and extern(C++) 
 will be what Calypso generates.


AFAICS it is not the way Calypso works right now, it does direct 
imports of C++ headers from D modules via `modmap (C++) 
"headerofyourdll.h"` without generating intermediate D binding 
module:
http://forum.dlang.org/post/nuiihxfcgjnaougivker forum.dlang.org
Jan 23 2015
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/23/15 12:34 AM, Max Klyga wrote:

 I remember Walter arguing against this type of solutions because it
 requires a C++ compiler to be bundled with D.


It doesn't require as much as makes it usable if you have it. Calypso 
requires dmd + clang++ is the natural dependency direction.


 Does that mean that now we are adopting this path instead of
 'extern(C++)' ?


I see it more like "with the help of" rather than "instead of". The 
extern(C++) feature is porcelain over a bunch of compiler internals that 
make it possible to use C++ in conjunction with D.

We should foster and support "plugin" tools like Calypso that add value 
to the D environment.


Andrei
Jan 23 2015
prev sibling next sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
On Friday, 23 January 2015 at 00:24:45 UTC, Andrei Alexandrescu 
wrote:

 I think it's important that we enable Calypso 
 (https://github.com/Syniurge/Calypso) and related tooling that 
 interfaces D with other languages, notably C++.

 A key topic in 2015 for D is playing well with C++. A good C++ 
 interface will give access to a host of mature C++ libraries, 
 starting of course with the C++ standard library. More 
 importantly, it will provide a smooth migration path for 
 organizations that want to do development in D whilst taking 
 advantage of their legacy code.


I'm very glad to see this coment from you so that we can get a 
more fully fleshed out vision for where D is heading with C++ 
interop. I for instance have developed Smidgen 
(https://github.com/alynch4047/smidgen) which was to wrap Qt. 
However, I halted development (having made very good progress, I 
believe) due to feeling that it could easily be superceded by a 
more direct interface to C++ such as Calypso seems to be.

These efforts do amount to a lot of work and need commitment (to 
their architecture) at the top level too. For instance, I needed 
weak references which I have had to work around, without any 
feeling that the workaround would continue to work with future 
changes to the GC. I forget the details but I also had concerns 
relating to GCed objects being relocated.

Calypso sounds fantastic but seems very tied to one compiler - we 
all need to know if yourself and Walter are content with that, 
for instance.

Getting a plan out there so that everyone can pull in one 
direction would be great.
Jan 23 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/23/2015 2:42 AM, Abdulhaq wrote:

 Calypso sounds fantastic but seems very tied to one compiler - we all need to
 know if yourself and Walter are content with that, for instance.


Yes, it's tied to clang++. It may not even work on all the platforms we
support. 
But that's no matter for now.
Jan 23 2015
next sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
On Friday, 23 January 2015 at 10:53:47 UTC, Walter Bright wrote:

 On 1/23/2015 2:42 AM, Abdulhaq wrote:

 Calypso sounds fantastic but seems very tied to one compiler - 
 we all need to
 know if yourself and Walter are content with that, for 
 instance.


 Yes, it's tied to clang++. It may not even work on all the 
 platforms we support. But that's no matter for now.


When you say "for now", does that imply that at some time in the 
future it may matter, in which case isn't it better to get these 
issues thrashed out now? Is this a potential dead end?
Jan 23 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/23/2015 3:22 AM, Abdulhaq wrote:

 On Friday, 23 January 2015 at 10:53:47 UTC, Walter Bright wrote:

 Yes, it's tied to clang++. It may not even work on all the platforms we
 support. But that's no matter for now.


 When you say "for now", does that imply that at some time in the future it may
 matter, in which case isn't it better to get these issues thrashed out now? Is
 this a potential dead end?


We don't need a perfect solution immediately. We do need a solution that's 
better than nothing, we can build on that as required.
Jan 23 2015
parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Friday, 23 January 2015 at 11:26:07 UTC, Walter Bright wrote:

 On 1/23/2015 3:22 AM, Abdulhaq wrote:

 On Friday, 23 January 2015 at 10:53:47 UTC, Walter Bright 
 wrote:

 Yes, it's tied to clang++. It may not even work on all the 
 platforms we
 support. But that's no matter for now.


 When you say "for now", does that imply that at some time in 
 the future it may
 matter, in which case isn't it better to get these issues 
 thrashed out now? Is
 this a potential dead end?


 We don't need a perfect solution immediately. We do need a 
 solution that's better than nothing, we can build on that as 
 required.


I actually agree (though I am nowhere near as well informed as 
most others on this forum) that Calypso is the way to go, however 
ISTM there are large implications that go with that decision. The 
infrastructure that Calyspo depends on is not something, IMO, 
that you "can build on as required" to address missing platforms, 
for instance. Gazing into my crystal ball I'd say that 5 years 
down the road there will be many D libraries with significant 
dependencies on C++ (Calypso bound) to the extent that the great 
unwashed will view D as a language that is only practicable on 
platforms supported by clang++. For me that is fine, but I 
suspect not fine for others. For instance, I would view Qt and 
VTK as key bindings. Others will want numeric libraries etc. etc..

I should also point out that D doesn't have 'nothing' in terms of 
alternatives, there exist other more traditional binding-based 
technologies that could flourish with 'official' support. Since I 
have a hand in one of those I should point out that I think 
Calypso, if it does what I think it does, will work better in 
terms of integration, speed etc. and from my personal perspective 
is therefore better. My goal is to have Qt, VTK, linear algebra, 
matrices etc., available for D (linux and Windows) and I'm not 
fussed exactly how it is done.
Jan 23 2015
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-23 11:52, Walter Bright wrote:


 Yes, it's tied to clang++. It may not even work on all the platforms we
 support. But that's no matter for now.


At least Clang supports all platforms we support.

-- 
/Jacob Carlborg
Jan 23 2015
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/23/15 2:42 AM, Abdulhaq wrote:

 Calypso sounds fantastic but seems very tied to one compiler - we all
 need to know if yourself and Walter are content with that, for instance.


We should make the life of plugin writers easy. They do get to decide 
what other prerequisites their tools have.

Andrei
Jan 23 2015
parent reply Jacob Carlborg <doob me.com> writes:
On 2015-01-23 17:02, Andrei Alexandrescu wrote:


 We should make the life of plugin writers easy. They do get to decide
 what other prerequisites their tools have.


Are you suggesting to have a general plugin architecture in DMD? That 
would be very cool, especially if they can be installed as Dub packages. 
Although I'm guessing there will be complains that it might create 
different flavors of D. And how will multiple plugins interact with each 
other?

-- 
/Jacob Carlborg
Jan 23 2015
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/23/15 8:57 AM, Jacob Carlborg wrote:

 On 2015-01-23 17:02, Andrei Alexandrescu wrote:


 We should make the life of plugin writers easy. They do get to decide
 what other prerequisites their tools have.


 Are you suggesting to have a general plugin architecture in DMD?


It's something worth thinking about.


 That
 would be very cool, especially if they can be installed as Dub packages.
 Although I'm guessing there will be complains that it might create
 different flavors of D. And how will multiple plugins interact with each
 other?


As of now I'm unclear on the details. Supporting plugins seems to be an 
interesting recent trend in compilers.


Andrei
Jan 23 2015
prev sibling next sibling parent reply "Elie Morisse" <syniurge gmail.com> writes:
On Friday, 23 January 2015 at 00:24:45 UTC, Andrei Alexandrescu 
wrote:

 I think it's important that we enable Calypso 
 (https://github.com/Syniurge/Calypso) and related tooling that 
 interfaces D with other languages, notably C++.

 A key topic in 2015 for D is playing well with C++. A good C++ 
 interface will give access to a host of mature C++ libraries, 
 starting of course with the C++ standard library. More 
 importantly, it will provide a smooth migration path for 
 organizations that want to do development in D whilst taking 
 advantage of their legacy code.

 I'd like to open the topic of "what can we do in core D to make 
 Calypso better".

 But first, I want to get better acquainted with Calypso and 
 raise awareness of it with coworkers and the larger community. 
 To my dismay, the github homepage provides exactly zero "look 
 and feel" use examples beyond the mechanics of building Calypso 
 itself.

 To Calypso's creator: is it possible to beef up the 
 documentation of Calypso with a few use cases? Ideally there'd 
 be a soup-to-nuts step by step guide of making a C++ library 
 (either artificial or extant) interfacing with D. Also: what 
 things on the D side would be necessary to make the interface 
 better? Exceptions would be an obvious topic on which we have 
 an attack already (more on it later).


Andrei thanks for finally sharing your thoughts on the matter :-)

I'll see what I can do this week-end to expand examples and write 
a tutorial.

As for what could be done in core D, I haven't struck anything 
blocking while writing Calypso. Exception catching is next after 
the first light of my Ogre3D demo, Clang will probably simplify 
handling of C++ exceptions a lot.
Jan 23 2015
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/23/15 10:27 AM, Elie Morisse wrote:

 Exception catching is next after the first light of my Ogre3D demo,
 Clang will probably simplify handling of C++ exceptions a lot.


On that front, a coworker gave me a simple idea for integration.

1. All C++ non-polymorphic types thrown cannot be caught in D

2. We (or Calypso) define D class counterparts for polymorphic types 
thrown from C++, notably std::exception and subtypes. N.B. those are D 
classes, not D structs

3. D code can catch std::exception. The only caveat is that the caught 
exception cannot survive the catch statement. Given that calling into 
C++ is not the paramount of safety in the first place, I'd say that's a 
reasonable compromise.

Example:

----
import core.stdcpp.vector;
import core.stdcpp.exception;

core.stdcpp.exception g;

void fun(core.stdcpp.vector!int v)
{
     try
     {
         v.push_back(42);
     }
     catch (core.stdcpp.exception e) // fine, can catch
     {
         g = e; // ouch, will dangle
     }
}
----

Thoughts?


Andrei
Jan 23 2015
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2015-01-23 19:43, Andrei Alexandrescu wrote:


 On that front, a coworker gave me a simple idea for integration.

 1. All C++ non-polymorphic types thrown cannot be caught in D

 2. We (or Calypso) define D class counterparts for polymorphic types
 thrown from C++, notably std::exception and subtypes. N.B. those are D
 classes, not D structs

 3. D code can catch std::exception. The only caveat is that the caught
 exception cannot survive the catch statement. Given that calling into
 C++ is not the paramount of safety in the first place, I'd say that's a
 reasonable compromise.

 Example:

 ----
 import core.stdcpp.vector;
 import core.stdcpp.exception;

 core.stdcpp.exception g;

 void fun(core.stdcpp.vector!int v)
 {
      try
      {
          v.push_back(42);
      }
      catch (core.stdcpp.exception e) // fine, can catch
      {
          g = e; // ouch, will dangle
      }
 }
 ----

 Thoughts?


Even with these restrictions C++ exceptions are quite complicated. I've 
probably said this before but in 64bit Objective-C (which uses the same 
exception model as C++) you can only catch C++ exceptions with the 
catch-all block which won't let you access the actual exception:

 try {
     v.push_back(42);
}
 catch (...) { }

Not sure if that makes it particular easier to implement.

-- 
/Jacob Carlborg
Jan 23 2015
prev sibling parent "Kagamin" <spam here.lot> writes:
On Friday, 23 January 2015 at 18:43:30 UTC, Andrei Alexandrescu 
wrote:

 2. We (or Calypso) define D class counterparts for polymorphic 
 types thrown from C++, notably std::exception and subtypes. 
 N.B. those are D classes, not D structs


Wouldn't it be easier with structs? They lack only one feature - 
inheritance, while classes lack many features necessary to value 
types: being a value type and deterministic destruction.
Jan 25 2015
prev sibling parent reply "Elie Morisse" <syniurge gmail.com> writes:
The README should be clearer now about what Calypso is supposed 
to do, links to the showcase example which I improved and 
expanded with a template partial and explicit spec example, and 
explains how to build it and link it to a C++ library.


(thanks to the simple partial spec example a nasty oversight 
about partial specializations was also fixed in the process, 

importing Ogre)
Jan 24 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/24/2015 5:26 PM, Elie Morisse wrote:

 The README should be clearer now about what Calypso is supposed to do, links to
 the showcase example which I improved and expanded with a template partial and
 explicit spec example, and explains how to build it and link it to a C++
library.


 (thanks to the simple partial spec example a nasty oversight about partial

 of the snags I hit while importing Ogre)


I'm sorry, but it still far from clear just what Calypso does.

Suppose I have a C++ file, test.cpp, that contains:

     int foo(unsigned *p);

How do I use Calypso to interface with that from D:

     ... what happens here ? ...
     uint x;
     foo(&x);


Next, in the README example, it says:

   $ clang++ -std=c++11 -c showcase.cpp -o showcase.cpp.o
   $ ar rcs libshowcase.a showcase.cpp.o
   $ ldc2 -cpp-args -std=c++11 -Llibshowcase.a -L-lstdc++ showcase.d

Where's Calypso in that?
Jan 25 2015
next sibling parent "Kelly" <wilsonk cpsc.ucalgary.ca> writes:
On Sunday, 25 January 2015 at 10:18:34 UTC, Walter Bright wrote:

 I'm sorry, but it still far from clear just what Calypso does.

 Suppose I have a C++ file, test.cpp, that contains:

     int foo(unsigned *p);

 How do I use Calypso to interface with that from D:

     ... what happens here ? ...
     uint x;
     foo(&x);


 Next, in the README example, it says:

   $ clang++ -std=c++11 -c showcase.cpp -o showcase.cpp.o
   $ ar rcs libshowcase.a showcase.cpp.o
   $ ldc2 -cpp-args -std=c++11 -Llibshowcase.a -L-lstdc++ 
 showcase.d

 Where's Calypso in that?



Calypso is part ldc2 and, as far as I understand it, is invoked 
via the "-cpp-args" command line arg.

I believe the simplest example that shows the use of your 
function above is as follows:

-----------------------------------------
test.h
-----------------------------------------

namespace test {
   int foo(unsigned int *p);
}


-----------------------------------------
test.cpp
-----------------------------------------

#include "test.h"
int test::foo(unsigned int *p)
{
   return *p * 2;
}

-----------------------------------------
test.d
-----------------------------------------

modmap (C++) "test.h";

import (C++) test._;  //this imports global vars, funcs and 
typedefs
import std.stdio;

void main()
{
   uint x = 4;
   writeln("foo = ", foo(&x));
}

-----------------------------------------
BUILD
-----------------------------------------


clang++ -std=c++11 -c test.cpp -o test.cpp.o
ar rcs libtest.a test.cpp.o
/bin/rm calypso_cache*
ldc2 -v -cpp-args -std=c++11 -Llibtest.a -L-lstdc++ test.d

-----------------------------------------
-----------------------------------------

The above was cut-n-pasted from a test dir. Tested on Linux. 
Produces a 'test' executable that prints 'foo = 8' when run.

I have tested more complex examples and things seem to work quite 
well. The one big caveat that I have run into so far with Calypso 
is that everything must be in a namespace...that is why test.h 
has the somewhat superfluous 'test' namespace. It must be there 
to use Calypso, as far as I can see.

This can be an issue when trying to import 'C++' libraries if 
they just use .h files like glorified C header files with some 
classes in them, or something like that, and they don't 
encapsulate everything in a unique namespace.

I am sure Elie will chime in, but I am definitely impressed and 
inspired by what he has accomplished here. Hopefully I haven't 
butchered the above explanation too much since I am not a Calypso 
expert  :)

Thanks,
Kelly
Jan 25 2015
prev sibling parent reply "Elie Morisse" <syniurge gmail.com> writes:
On Sunday, 25 January 2015 at 10:18:34 UTC, Walter Bright wrote:

 Next, in the README example, it says:

   $ clang++ -std=c++11 -c showcase.cpp -o showcase.cpp.o
   $ ar rcs libshowcase.a showcase.cpp.o
   $ ldc2 -cpp-args -std=c++11 -Llibshowcase.a -L-lstdc++ 
 showcase.d

 Where's Calypso in that?


On Sunday, 25 January 2015 at 11:12:50 UTC, Kelly wrote:

 Calypso is part ldc2 and, as far as I understand it, is invoked 
 via the "-cpp-args" command line arg.


-cpp-args is only to pass arguments to Clang while generating the 
precompiled header.

Calypso registers itself as a "language plugin", and when parse.c 
encounters « import (ABC) xxx.yyy; » it asks the registered 
plugins if one of them handles the "ABC" language id. If that's 
the case it lets the plugin create the Import symbol by itself, 
for instance Calypso creates a cpp::Import which derives from 
Import and has a completely different load() method, which won't 
look for modules in .d files but inside the PCH generated by 
Clang.

Here's the LangPlugin interface:

class LangPlugin
{
public:
     // returns -1 if said lang isn't handled by this plugin, or 
its id number
     // to be passed to createImport otherwise
     virtual int doesHandleModmap(const utf8_t *lang) = 0;

     virtual Modmap *createModmap(int langId,
         Loc loc, Expression *arg) = 0;

     // returns -1 if said tree isn't handled by this plugin, or 
its id number
     // to be passed to createImport otherwise
     virtual int doesHandleImport(const utf8_t *tree) = 0;

     virtual Import *createImport(int treeId,
         Loc loc, Identifiers *packages, Identifier *id,
         Identifier *aliasId, int isstatic) = 0;

     // ===== - - - - - ===== //

     virtual Expression *getRightThis(Loc loc, Scope *sc, 
AggregateDeclaration *ad,
         Expression *e1, Declaration *var, int flag = 0) = 0;

     // ===== - - - - - ===== //

     virtual FuncDeclaration *buildDtor(AggregateDeclaration *ad, 
Scope *sc) = 0;
     virtual FuncDeclaration *buildCpCtor(StructDeclaration *sd, 
Scope *sc) = 0;

     // ===== - - - - - ===== //

      virtual CodeGen *codegen() = 0;
};

getRightThis, buildDtor and buildCpCtor are needed because they 
"override" the global functions with the same name.

About Andrei's query about a general plugin system I don't know 
how one could be architectured. The hooks created for Calypso are 
specific to C++, so the help I could get is simply to be open to 
those hooks (which aren't really intrusive and don't uglify the 
code except for one or two that could probably be done 
differently: 
https://github.com/Syniurge/Calypso/commit/debd83f49363bf6805573d141a62b25d068d8a14)

The problem with arbitrarily limiting where the hooks could occur 
is that it might force the plugins to copy and paste redundantly 
the base functions, which is bad especially for big semantic() 
functions.

This probably deserves more thought, maybe there's a more elegant 
way waiting to be found.
Jan 25 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/25/2015 7:16 AM, Elie Morisse wrote:

 On Sunday, 25 January 2015 at 10:18:34 UTC, Walter Bright wrote:

 Next, in the README example, it says:

   $ clang++ -std=c++11 -c showcase.cpp -o showcase.cpp.o
   $ ar rcs libshowcase.a showcase.cpp.o
   $ ldc2 -cpp-args -std=c++11 -Llibshowcase.a -L-lstdc++ showcase.d

 Where's Calypso in that?


 On Sunday, 25 January 2015 at 11:12:50 UTC, Kelly wrote:

 Calypso is part ldc2 and, as far as I understand it, is invoked via the
 "-cpp-args" command line arg.


 -cpp-args is only to pass arguments to Clang while generating the precompiled
 header.


You see, nobody understands where Calypso is! Is it part of clang, part of ldc, 
a standalone tool?

I feel like Noah:

https://www.youtube.com/watch?v=bputeFGXEjA 1:44



 Calypso registers itself as a "language plugin", and when parse.c encounters


parse.c is not part of my project. Where is parse.c?


 « import (ABC) xxx.yyy; » it asks the registered plugins if one of them
handles
 the "ABC" language id. If that's the case it lets the plugin create the Import
 symbol by itself, for instance Calypso creates a cpp::Import which derives from
 Import and has a completely different load() method, which won't look for
 modules in .d files but inside the PCH generated by Clang.


A language plugin plugging into where?



 Here's the LangPlugin interface:

 class LangPlugin
 {
 public:
      // returns -1 if said lang isn't handled by this plugin, or its id number
      // to be passed to createImport otherwise
      virtual int doesHandleModmap(const utf8_t *lang) = 0;

      virtual Modmap *createModmap(int langId,
          Loc loc, Expression *arg) = 0;

      // returns -1 if said tree isn't handled by this plugin, or its id number
      // to be passed to createImport otherwise
      virtual int doesHandleImport(const utf8_t *tree) = 0;

      virtual Import *createImport(int treeId,
          Loc loc, Identifiers *packages, Identifier *id,
          Identifier *aliasId, int isstatic) = 0;

      // ===== - - - - - ===== //

      virtual Expression *getRightThis(Loc loc, Scope *sc, AggregateDeclaration
*ad,
          Expression *e1, Declaration *var, int flag = 0) = 0;

      // ===== - - - - - ===== //

      virtual FuncDeclaration *buildDtor(AggregateDeclaration *ad, Scope *sc) =
0;
      virtual FuncDeclaration *buildCpCtor(StructDeclaration *sd, Scope *sc) =
0;

      // ===== - - - - - ===== //

       virtual CodeGen *codegen() = 0;
 };

 getRightThis, buildDtor and buildCpCtor are needed because they "override" the
 global functions with the same name.

 About Andrei's query about a general plugin system I don't know how one could
be
 architectured. The hooks created for Calypso are specific to C++, so the help I
 could get is simply to be open to those hooks (which aren't really intrusive
and
 don't uglify the code except for one or two that could probably be done
 differently:
 https://github.com/Syniurge/Calypso/commit/debd83f49363bf6805573d141a62b25d068d8a14)


 The problem with arbitrarily limiting where the hooks could occur is that it
 might force the plugins to copy and paste redundantly the base functions, which
 is bad especially for big semantic() functions.

 This probably deserves more thought, maybe there's a more elegant way waiting
to
 be found.
Jan 25 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Sunday, 25 January 2015 at 20:45:26 UTC, Walter Bright wrote:

 Calypso registers itself as a "language plugin", and when 
 parse.c encounters


 parse.c is not part of my project. Where is parse.c?


here? 
https://github.com/D-Programming-Language/dmd/blob/master/src/parse.c
Jan 25 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/25/2015 1:36 PM, John Colvin wrote:

 On Sunday, 25 January 2015 at 20:45:26 UTC, Walter Bright wrote:

 Calypso registers itself as a "language plugin", and when parse.c encounters


 parse.c is not part of my project. Where is parse.c?


 here? https://github.com/D-Programming-Language/dmd/blob/master/src/parse.c


Not what I meant.

What does parse.c have to do with the user of Calypso?

How does the user use Calypso? Where is Calypso? How do I run Calypso?

A user of Calypso will be baffled when encountering user documentation of 
Calypso that explains it in terms of dmd internal source code.
Jan 25 2015
parent reply "Elie Morisse" <syniurge gmail.com> writes:
On Sunday, 25 January 2015 at 22:45:31 UTC, Walter Bright wrote:

 Not what I meant.

 What does parse.c have to do with the user of Calypso?

 How does the user use Calypso? Where is Calypso? How do I run 
 Calypso?

 A user of Calypso will be baffled when encountering user 
 documentation of Calypso that explains it in terms of dmd 
 internal source code.


Is the new README any better? Once I succeed making the nature of 
Calypso intelligible does the showcase example seem 
self-explaining enough to make the usage straightforward?
Jan 25 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/25/2015 3:07 PM, Elie Morisse wrote:

 On Sunday, 25 January 2015 at 22:45:31 UTC, Walter Bright wrote:

 Not what I meant.

 What does parse.c have to do with the user of Calypso?

 How does the user use Calypso? Where is Calypso? How do I run Calypso?

 A user of Calypso will be baffled when encountering user documentation of
 Calypso that explains it in terms of dmd internal source code.


 Is the new README any better?


No. It doesn't answer any of my questions.


 Once I succeed making the nature of Calypso
 intelligible does the showcase example seem self-explaining enough to make the
 usage straightforward?


I'm obviously terrible at communicating. Let me try again. Assume that I wrote 
Calypso, and I was explaining it to you:

-----------------------------------------

Given the C++ header file foo.h:

     void bar(unsigned *);

and the C++ source file foo.cpp:

     void bar(unsigned *p) { }

I want to call bar() from my D code in test.d:

     void main() {
       uint x;
       bar(&x);
     }

Here's how to do it with Calypso:

     calypso foo.h

which will generate the file foo.d. add an import to test.d:

     import foo;

     void main() {
       uint x;
       bar(&x);
     }

To compile and link:

     clang++ foo.cpp -c
     dmd test foo.o

Which generates the program 'test' which can be run.
Jan 25 2015
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 1/25/15 4:37 PM, Walter Bright wrote:

 On 1/25/2015 3:07 PM, Elie Morisse wrote:

 On Sunday, 25 January 2015 at 22:45:31 UTC, Walter Bright wrote:

 Not what I meant.

 What does parse.c have to do with the user of Calypso?

 How does the user use Calypso? Where is Calypso? How do I run Calypso?

 A user of Calypso will be baffled when encountering user
 documentation of
 Calypso that explains it in terms of dmd internal source code.


 Is the new README any better?


 No. It doesn't answer any of my questions.


Might be a great idea if a person who went through the steps of 
installing and using Calypso would issue a pull request for the README. 
-- Andrei
Jan 25 2015
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2015-01-26 01:37, Walter Bright wrote:


 I'm obviously terrible at communicating. Let me try again. Assume that I
 wrote Calypso, and I was explaining it to you:

 -----------------------------------------

 Given the C++ header file foo.h:

      void bar(unsigned *);

 and the C++ source file foo.cpp:

      void bar(unsigned *p) { }

 I want to call bar() from my D code in test.d:

      void main() {
        uint x;
        bar(&x);
      }

 Here's how to do it with Calypso:

      calypso foo.h

 which will generate the file foo.d. add an import to test.d:

      import foo;

      void main() {
        uint x;
        bar(&x);
      }

 To compile and link:

      clang++ foo.cpp -c
      dmd test foo.o

 Which generates the program 'test' which can be run.


It works something like this:

Given the C++ header file foo.h:

     void bar(unsigned *);

and the C++ source file foo.cpp:

     void bar(unsigned *p) { }

I want to call bar() from my D code in test.d:

     void main() {
       uint x;
       bar(&x);
     }

Here's how to do it with Calypso:

     module test;

     modmap (C++) "foo.h";
     import (C++) _ : bar;

     void main() {
       uint x;
       bar(&x);
     }

To compile and link:

     clang++ foo.cpp -c
     ldc test.d foo.o

Which generates the program 'test' which can be run.

Calypso is not a separate tool. It's a fork of LDC which allows you to 
directly import/include a C++ header files and use the declarations from 
within D. No bindings or intermediate files are necessary.

-- 
/Jacob Carlborg
Jan 26 2015
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/26/2015 12:18 AM, Jacob Carlborg wrote:

 It works something like this:

 Given the C++ header file foo.h:

      void bar(unsigned *);

 and the C++ source file foo.cpp:

      void bar(unsigned *p) { }

 I want to call bar() from my D code in test.d:

      void main() {
        uint x;
        bar(&x);
      }

 Here's how to do it with Calypso:

      module test;

      modmap (C++) "foo.h";
      import (C++) _ : bar;

      void main() {
        uint x;
        bar(&x);
      }

 To compile and link:

      clang++ foo.cpp -c
      ldc test.d foo.o

 Which generates the program 'test' which can be run.

 Calypso is not a separate tool. It's a fork of LDC which allows you to directly
 import/include a C++ header files and use the declarations from within D. No
 bindings or intermediate files are necessary.


Thank you. That really, really needs to go into the README.md, especially the 
last paragraph.
Jan 26 2015
parent reply "Ulrich =?UTF-8?B?S8O8dHRsZXIi?= <kuettler gmail.com> writes:
On Monday, 26 January 2015 at 08:59:34 UTC, Walter Bright wrote:

 On 1/26/2015 12:18 AM, Jacob Carlborg wrote:

 Calypso is not a separate tool. It's a fork of LDC which 
 allows you to directly
 import/include a C++ header files and use the declarations 
 from within D. No
 bindings or intermediate files are necessary.


 Thank you. That really, really needs to go into the README.md, 
 especially the last paragraph.


Now I feel sorry for Elie. He contributed a great idea, put in a 
huge amount of work, still has miles to go, tried hard to be 
helpful and D's very own mastermind does not seem to care enough 
to read his announcement. His very first sentence reads

"I have the pleasure to announce to you all the existence of a 
modified LDC able to interface directly to C++ libraries, wiping 
out the need to write bindings:"

The idea is just brilliant. I for one am excited about the 
prospect. Lets please discuss how to get the plugin into D, 
sooner rather than later. No language is an island. (Apologies to 
Nick Hornby and everyone else offended.)
Jan 26 2015
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 1/26/2015 1:58 AM, "Ulrich =?UTF-8?B?S8O8dHRsZXIi?= <kuettler gmail.com>"
wrote:

 The idea is just brilliant. I for one am excited about the prospect. Lets
please
 discuss how to get the plugin into D, sooner rather than later. No language is
 an island. (Apologies to Nick Hornby and everyone else offended.)


I am pretty excited as well about Calypso and think it can be very effective in 
making D more usable.

Thank you, Elie!
Jan 26 2015
parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Monday, 26 January 2015 at 10:16:04 UTC, Walter Bright wrote:

 On 1/26/2015 1:58 AM, "Ulrich =?UTF-8?B?S8O8dHRsZXIi?= 
 <kuettler gmail.com>" wrote:

 The idea is just brilliant. I for one am excited about the 
 prospect. Lets please
 discuss how to get the plugin into D, sooner rather than 
 later. No language is
 an island. (Apologies to Nick Hornby and everyone else 
 offended.)


 I am pretty excited as well about Calypso and think it can be 
 very effective in making D more usable.

 Thank you, Elie!


The modmap seems a little wonky to me, a pragma(calypso, ...) 
seems more appropriate.

Any chance that we can get a plugin system in dmd that allows us 
to register pragmas, so that calypso can be used?
Jan 26 2015
prev sibling parent "bachmeier" <no spam.com> writes:
On Monday, 26 January 2015 at 09:58:25 UTC, Ulrich Küttler wrote:

 On Monday, 26 January 2015 at 08:59:34 UTC, Walter Bright wrote:

 On 1/26/2015 12:18 AM, Jacob Carlborg wrote:

 Calypso is not a separate tool. It's a fork of LDC which 
 allows you to directly
 import/include a C++ header files and use the declarations 
 from within D. No
 bindings or intermediate files are necessary.


 Thank you. That really, really needs to go into the README.md, 
 especially the last paragraph.


 Now I feel sorry for Elie. He contributed a great idea, put in 
 a huge amount of work, still has miles to go, tried hard to be 
 helpful and D's very own mastermind does not seem to care 
 enough to read his announcement. His very first sentence reads

 "I have the pleasure to announce to you all the existence of a 
 modified LDC able to interface directly to C++ libraries, 
 wiping out the need to write bindings:"

 The idea is just brilliant. I for one am excited about the 
 prospect. Lets please discuss how to get the plugin into D, 
 sooner rather than later. No language is an island. (Apologies 
 to Nick Hornby and everyone else offended.)


A fast way to kill the language would be to add "great new 
features" that are not properly documented. Walter's comments 
were correct IMO. I was not at all clear about those things, but 
I wasn't going to say so.
Jan 26 2015
prev sibling parent reply "Elie Morisse" <syniurge gmail.com> writes:
On Monday, 26 January 2015 at 00:37:02 UTC, Walter Bright wrote:


Excuse the creator's bias :)

I agree that Jacob's description makes things more clear, added!



 Jan 26 2015



  parent reply "Laeeth Isharc" <Laeeth.nospam nospam-laeeth.com>  writes:


On Monday, 26 January 2015 at 16:40:38 UTC, Elie Morisse wrote:

 On Monday, 26 January 2015 at 00:37:02 UTC, Walter Bright wrote:


 Excuse the creator's bias :)

 I agree that Jacob's description makes things more clear, added!


I posted some thoughts on web docs writeup of C+= interface here.

http://forum.dlang.org/thread/fmjehcyzhnirybmnjloj forum.dlang.org#post-fmjehcyzhnirybmnjloj:40forum.dlang.org

Do you think we could make binaries of calypso available for 
download ?  I know it is alpha, but it is so strategically 
important, and just getting clang to build is not always so easy 
given the stuff in gcc 4.9 with size__align_t ?

We should mention calypso in the web docs, and link to it too 
(with appropriate caveats).



 Jan 26 2015



  parent reply "Elie Morisse" <syniurge gmail.com>  writes:


On Monday, 26 January 2015 at 19:35:11 UTC, Laeeth Isharc wrote:

 I posted some thoughts on web docs writeup of C+= interface 
 here.

 http://forum.dlang.org/thread/fmjehcyzhnirybmnjloj forum.dlang.org#post-fmjehcyzhnirybmnjloj:40forum.dlang.org

 Do you think we could make binaries of calypso available for 
 download ?  I know it is alpha, but it is so strategically 
 important, and just getting clang to build is not always so 
 easy given the stuff in gcc 4.9 with size__align_t ?

 We should mention calypso in the web docs, and link to it too 
 (with appropriate caveats).


It feel slightly too early, Calypso shouldn't be far from getting 
most of the C++ standard library to work but it still chokes on 
most tangles of templates (afaics it all boils down to a few 
issues). Also operators, function templates and merging latest 
LDC would be nice and not a lot of work away (last merge was 
mid-October, before better Win64 ABI support).

As soon as this is done I'll make builds for Windows and Linux 
users.



 Jan 27 2015



  parent reply "Laeeth Isharc" <laeethnospam nospamlaeeth.com>  writes:


On Tuesday, 27 January 2015 at 16:39:40 UTC, Elie Morisse wrote:

 On Monday, 26 January 2015 at 19:35:11 UTC, Laeeth Isharc wrote:

 I posted some thoughts on web docs writeup of C+= interface 
 here.

 http://forum.dlang.org/thread/fmjehcyzhnirybmnjloj forum.dlang.org#post-fmjehcyzhnirybmnjloj:40forum.dlang.org

 Do you think we could make binaries of calypso available for 
 download ?  I know it is alpha, but it is so strategically 
 important, and just getting clang to build is not always so 
 easy given the stuff in gcc 4.9 with size__align_t ?

 We should mention calypso in the web docs, and link to it too 
 (with appropriate caveats).


 It feel slightly too early, Calypso shouldn't be far from 
 getting most of the C++ standard library to work but it still 
 chokes on most tangles of templates (afaics it all boils down 
 to a few issues). Also operators, function templates and 
 merging latest LDC would be nice and not a lot of work away 
 (last merge was mid-October, before better Win64 ABI support).

 As soon as this is done I'll make builds for Windows and Linux 
 users.


Thanks.  People like to play with things when it is easy, and 
bear in mind what the present alternative is.  But I respect your 
desire to maintain a standard of quality.



 Jan 27 2015



  parent  Swoorup Joshi <swoorupjoshi gmail.com>  writes:


On Tuesday, 27 January 2015 at 19:34:03 UTC, Laeeth Isharc wrote:

 On Tuesday, 27 January 2015 at 16:39:40 UTC, Elie Morisse wrote:

 On Monday, 26 January 2015 at 19:35:11 UTC, Laeeth Isharc 
 wrote:

 I posted some thoughts on web docs writeup of C+= interface 
 here.

 http://forum.dlang.org/thread/fmjehcyzhnirybmnjloj forum.dlang.org#post-fmjehcyzhnirybmnjloj:40forum.dlang.org

 Do you think we could make binaries of calypso available for 
 download ?  I know it is alpha, but it is so strategically 
 important, and just getting clang to build is not always so 
 easy given the stuff in gcc 4.9 with size__align_t ?

 We should mention calypso in the web docs, and link to it too 
 (with appropriate caveats).


 It feel slightly too early, Calypso shouldn't be far from 
 getting most of the C++ standard library to work but it still 
 chokes on most tangles of templates (afaics it all boils down 
 to a few issues). Also operators, function templates and 
 merging latest LDC would be nice and not a lot of work away 
 (last merge was mid-October, before better Win64 ABI support).

 As soon as this is done I'll make builds for Windows and Linux 
 users.


 Thanks.  People like to play with things when it is easy, and 
 bear in mind what the present alternative is.  But I respect 
 your desire to maintain a standard of quality.


What is the progress on this project? I am planning of working on 
a game project that involves using physx and offcourse D. It 
looks like there would be a lot of work just writing D 
boilerplate code to interface with C++ and alot of unknowns it 
seems.

With this realized as a clang plugin, its supposed to 
automatically generate the bindings for D?



 Dec 15 2016