www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Qt's MOC getting replicated in D for Calypso

reply Elie Morisse <syniurge gmail.com> writes:
Hi all,

I now have a working D replacement for moc:

   
https://github.com/Syniurge/Calypso/blob/master/tests/calypso/qt5/moc

For those unfamiliar with Qt, moc (the Meta-Object Compiler) is a 
tool that generates additional code for Qt classes, code then 
used by the signal/slot system, properties and other Qt-specific 
extensions. While C++ moc is not going away for probably a very 
long time 
(http://woboq.com/blog/reflection-in-cpp-and-qt-moc.html), D's 
compile-time reflection makes it possible to replicate moc's 
functionality entirely with templates and CTFE, with the help of 
some Calypso features: C++ template instantiation, overriding C++ 
virtual methods, and member function pointers.

Calypso had been partially supporting Qt5 for a while and there 
was a Qt5 demo written by Kelly Wilson, but to work around the 
lack of moc he had to write an intermediate C++ class with 
virtual slots and the signal <-> slot connections were done in 
C++ code.

Now everything is in D:

   
https://github.com/Syniurge/Calypso/blob/master/tests/calypso/qt5/qt5demo.d

The moc package is as faithful as possible to Qt 5.5.0's moc. 
It's not finished yet (missing: properties, class info) but 
signals, slots and connections are working, making Qt 5 
development in D much more practical than before.
Feb 17 2016
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 02/17/2016 06:47 PM, Elie Morisse wrote:


 I now have a working D replacement for moc:

 https://github.com/Syniurge/Calypso/blob/master/tests/calypso/qt5/moc

 For those unfamiliar with Qt, moc (the Meta-Object Compiler) is a tool
 that generates additional code for Qt classes, code then used by the
 signal/slot system, properties and other Qt-specific extensions. While
 C++ moc is not going away


Congratulations! Any project that can get rid of moc is a big 
achievement. :) Folks at CopperSpice had done the same in C++ with their 
Qt replacement:

   http://www.copperspice.com/

Ali
Feb 17 2016
prev sibling next sibling parent reply ZombineDev <valid_email he.re> writes:
On Thursday, 18 February 2016 at 02:47:37 UTC, Elie Morisse wrote:

 Hi all,

 I now have a working D replacement for moc:

   
 https://github.com/Syniurge/Calypso/blob/master/tests/calypso/qt5/moc

 For those unfamiliar with Qt, moc (the Meta-Object Compiler) is 
 a tool that generates additional code for Qt classes, code then 
 used by the signal/slot system, properties and other 
 Qt-specific extensions. While C++ moc is not going away for 
 probably a very long time 
 (http://woboq.com/blog/reflection-in-cpp-and-qt-moc.html), D's 
 compile-time reflection makes it possible to replicate moc's 
 functionality entirely with templates and CTFE, with the help 
 of some Calypso features: C++ template instantiation, 
 overriding C++ virtual methods, and member function pointers.

 Calypso had been partially supporting Qt5 for a while and there 
 was a Qt5 demo written by Kelly Wilson, but to work around the 
 lack of moc he had to write an intermediate C++ class with 
 virtual slots and the signal <-> slot connections were done in 
 C++ code.

 Now everything is in D:

   
 https://github.com/Syniurge/Calypso/blob/master/tests/calypso/qt5/qt5demo.d

 The moc package is as faithful as possible to Qt 5.5.0's moc. 
 It's not finished yet (missing: properties, class info) but 
 signals, slots and connections are working, making Qt 5 
 development in D much more practical than before.


Congratulations! This is an extremely promising result!

BTW, what do you think about DMD's recent advancement in this 
area? Can you reuse some of this work for Calypso? When do you 
think you'll be ready to start upstreaming some of your work to 
LDC and/or DMD?

https://github.com/D-Programming-Language/dmd/pull/5261 - 
Parameter types should have namespace std mangling too

https://github.com/D-Programming-Language/dmd/pull/5262 - Add 
versioned-out branch to support new C++-11 implementations of 
std::string

https://github.com/D-Programming-Language/druntime/pull/1470 - 
Add catching C++ exception support to dwarfeh.d

https://github.com/D-Programming-Language/druntime/pull/1473 - 
add core.stdcpp.exception and core.stdcpp.typeinfo

https://github.com/D-Programming-Language/dmd/pull/5330 - fix 
Issue 15389 - extern(C++) forward referencing problem

https://github.com/D-Programming-Language/dmd/pull/5333 - fix 
Issue 15519 - Circular imports leads to fwd ref error with 
aliased imports

https://github.com/D-Programming-Language/dmd/pull/5342 - C++ EH: 
initial front end work

https://github.com/D-Programming-Language/dmd/pull/5361 - fix 
Issue 15579 - extern(C++) interfaces/multiple-inheritance

https://github.com/D-Programming-Language/dmd/pull/5372 - fix 
Issue 15610 - extern(C++) multiple inheritance - calling with 
wrong 'this' ptr

https://github.com/D-Programming-Language/dmd/pull/5397 - fix 
Issue 15644 - Switch object layout ABI to MI style

https://github.com/D-Programming-Language/dmd/pull/5402 - fix 
Issue 15647 - Casting from one C++ interface in a hierarchy to 
another is a noop

https://github.com/D-Programming-Language/dmd/pull/5403 - fix 
Issue 15626 - extern(C++) calling crash
Feb 17 2016
parent reply Elie Morisse <syniurge gmail.com> writes:
On Thursday, 18 February 2016 at 03:07:22 UTC, Ali Çehreli wrote:

 Congratulations! Any project that can get rid of moc is a big 
 achievement. :) Folks at CopperSpice had done the same in C++ 
 with their Qt replacement:

   http://www.copperspice.com/

 Ali


Nice! I've never heard of them, interesting how they proved the 
moc maintainer partly wrong.

On Thursday, 18 February 2016 at 03:36:20 UTC, ZombineDev wrote:

 Congratulations! This is an extremely promising result!

 BTW, what do you think about DMD's recent advancement in this 
 area? Can you reuse some of this work for Calypso? When do you 
 think you'll be ready to start upstreaming some of your work to 
 LDC and/or DMD?

 https://github.com/D-Programming-Language/dmd/pull/5261 - 
 Parameter types should have namespace std mangling too

 https://github.com/D-Programming-Language/dmd/pull/5262 - Add 
 versioned-out branch to support new C++-11 implementations of 
 std::string

 https://github.com/D-Programming-Language/druntime/pull/1470 - 
 Add catching C++ exception support to dwarfeh.d

 https://github.com/D-Programming-Language/druntime/pull/1473 - 
 add core.stdcpp.exception and core.stdcpp.typeinfo

 https://github.com/D-Programming-Language/dmd/pull/5330 - fix 
 Issue 15389 - extern(C++) forward referencing problem

 https://github.com/D-Programming-Language/dmd/pull/5333 - fix 
 Issue 15519 - Circular imports leads to fwd ref error with 
 aliased imports

 https://github.com/D-Programming-Language/dmd/pull/5342 - C++ 
 EH: initial front end work

 https://github.com/D-Programming-Language/dmd/pull/5361 - fix 
 Issue 15579 - extern(C++) interfaces/multiple-inheritance

 https://github.com/D-Programming-Language/dmd/pull/5372 - fix 
 Issue 15610 - extern(C++) multiple inheritance - calling with 
 wrong 'this' ptr

 https://github.com/D-Programming-Language/dmd/pull/5397 - fix 
 Issue 15644 - Switch object layout ABI to MI style

 https://github.com/D-Programming-Language/dmd/pull/5402 - fix 
 Issue 15647 - Casting from one C++ interface in a hierarchy to 
 another is a noop

 https://github.com/D-Programming-Language/dmd/pull/5403 - fix 
 Issue 15626 - extern(C++) calling crash


Most of Calypso lives independently from the C++ support in DMD, 
and instead queries Clang whenever possible on C++ matters.

It was actually catching C++ exceptions some time before DMD 
(https://github.com/Syniurge/Calypso/tree/master/tests/calypso/eh), has no C++
multiple inheritance layout issue since that's handled by Clang, and the only
way extern(C++) affects Calypso is by not reverting the order of function
parameters so these functions can be correctly called by C++ code (function
pointers, overridden virtual methods), and if I understand correctly this is
only necessary because there's a discrepancy between DMD and what the docs say:
http://forum.dlang.org/thread/zogygbvfszssudpaejwo forum.dlang.org?page=2


When do you think you'll be ready to start upstreaming some of 
your work to LDC and/or DMD?


My big concern is the switch to DDMD. It seems that the 
transition is going pretty smoothly for LDC, but what about 
Calypso's extensions to DMD? Calypso classes deriving from DMD 
ones will have to be converted to D and those classes happen to 
sollicit Clang's C++ API a lot, so I'm counting on C++ Calypso to 
build D Calypso meaning it has to be pretty solid at that point 
to support both Clang and LLVM.

Anyway as long as both LDC and Calypso haven't both caught up 
with DDMD it's probably too early for upstreaming.

The downside is that Calypso is marginalized, I don't get many 
testers (<- euphemism, only wilsonk helped a lot and right now 
I'm about the only one testing). The neglecting of MSVC, the 
"still experimental" label and the lack of releases probably 
don't help in that regard. It's not a big deal though since I 
have enough on my hands, coming next is the LDC 0.17 merge, and 
rework on the PCH generation that should increase the speed 3x 
according to cachegrind.
Feb 18 2016
next sibling parent Craig Dillabaugh <craig.dillabaugh gmail.com> writes:
On Thursday, 18 February 2016 at 23:48:44 UTC, Elie Morisse wrote:

 On Thursday, 18 February 2016 at 03:07:22 UTC, Ali Çehreli 
 wrote:

 [...]


 Nice! I've never heard of them, interesting how they proved the 
 moc maintainer partly wrong.

 [...]


Any interest in a Google Summer of Code project ... there is 
still a tiny bit of time (deadline is today 17:00 UTC, but I 
suspect the Idea's page can continue to receive updates after 
that.)

http://wiki.dlang.org/GSOC_2016_Ideas
Feb 19 2016
prev sibling parent David Nadlinger <code klickverbot.at> writes:
On Thursday, 18 February 2016 at 23:48:44 UTC, Elie Morisse wrote:

 Calypso classes deriving from DMD ones will have to be 
 converted to D […]


This is not necessarily true. You should be able to inherit from 
an extern(C++) class just fine on the C++ side. Of course, Walter 
is busy converting various internals to D, but it should be 
possible to keep the parts you need accessible from C++.

Please make sure let us know if we can do anything on the LDC 
side to make the transition easier.

  — David
Feb 21 2016
prev sibling parent reply "Nicolas F." <ddev fratti.ch> writes:
This is really cool and an interesting project, though I've got one
concern: How will this fit in with the rest of the C++ efforts done
upstream? As I see it the goal here is to spearhead a working Qt <-> D
interaction, but how would this be used in production? Would Calypso
simply be run to generate bindings, or is the goal to upstream these
changes and make them an officially supported feature?
Feb 20 2016
parent reply Elie Morisse <syniurge gmail.com> writes:
On Saturday, 20 February 2016 at 17:34:48 UTC, Nicolas F. wrote:

 This is really cool and an interesting project, though I've got 
 one concern: How will this fit in with the rest of the C++ 
 efforts done upstream? (...) or is the goal to upstream these 
 changes and make them an officially supported feature?


The two efforts are independent, and the main issue with 
Calypso's approach: it's tied to LDC, LLVM and Clang. Although I 
had a slight hope that the approach would get recognized as 
allowing perfect interfacing with C++ incl. things unthinkable 
with the « from scratch » approach (like C++ template 
instantiation) and give D an edge that would probably be 
sufficient to make lots and lots of people switch from C++ to D, 
as long as DMD is there and a GDC/GCC version isn't proved 
feasible there's no question about whether this approach should 
get officially endorsed or not, and nevertheless the current 
efforts towards better C++ support in DMD should still yield 
important results.

Calypso will exist as a LDC plugin, and yes code using Calypso 
features will only be build-able by LDC+Calypso.


 As I see it the goal here is to spearhead a working Qt <-> D 
 interaction, but how would this be used in production? Would 
 Calypso simply be run to generate bindings


The goal of Calypso is to make any C++ library of any complexity 
usable in D straightaway, and there's no binding involved.

moc was a barrier for Qt because it only parses C++ code, and 
Qt's C++ API can hardly be used without the code moc generates.
Feb 21 2016
parent reply Brad Roberts via Digitalmars-d-announce writes:
On 2/21/2016 9:09 AM, Elie Morisse via Digitalmars-d-announce wrote:

 On Saturday, 20 February 2016 at 17:34:48 UTC, Nicolas F. wrote:

 This is really cool and an interesting project, though I've got one
 concern: How will this fit in with the rest of the C++ efforts done
 upstream? (...) or is the goal to upstream these changes and make them
 an officially supported feature?


 The two efforts are independent, and the main issue with Calypso's
 approach: it's tied to LDC, LLVM and Clang. Although I had a slight hope
 that the approach would get recognized as allowing perfect interfacing
 with C++ incl. things unthinkable with the « from scratch » approach
 (like C++ template instantiation) and give D an edge that would probably
 be sufficient to make lots and lots of people switch from C++ to D, as
 long as DMD is there and a GDC/GCC version isn't proved feasible there's
 no question about whether this approach should get officially endorsed
 or not, and nevertheless the current efforts towards better C++ support
 in DMD should still yield important results.

 Calypso will exist as a LDC plugin, and yes code using Calypso features
 will only be build-able by LDC+Calypso.


 As I see it the goal here is to spearhead a working Qt <-> D
 interaction, but how would this be used in production? Would Calypso
 simply be run to generate bindings


 The goal of Calypso is to make any C++ library of any complexity usable
 in D straightaway, and there's no binding involved.

 moc was a barrier for Qt because it only parses C++ code, and Qt's C++
 API can hardly be used without the code moc generates.


Is there anything preventing Calypso from turning into a code and 
interface generator?  Making it an application that is part of the build 
rather than a plug in to ldc would make it available to both dmd and gdc 
users, no?
Feb 21 2016
next sibling parent reply Kagamin <spam here.lot> writes:
On Sunday, 21 February 2016 at 17:21:51 UTC, Brad Roberts wrote:

 Is there anything preventing Calypso from turning into a code 
 and interface generator?  Making it an application that is part 
 of the build rather than a plug in to ldc would make it 
 available to both dmd and gdc users, no?


That's DStep: https://github.com/jacob-carlborg/dstep
Feb 21 2016
parent reply bachmeier <no spam.net> writes:
On Sunday, 21 February 2016 at 22:23:10 UTC, Kagamin wrote:

 On Sunday, 21 February 2016 at 17:21:51 UTC, Brad Roberts wrote:

 Is there anything preventing Calypso from turning into a code 
 and interface generator?  Making it an application that is 
 part of the build rather than a plug in to ldc would make it 
 available to both dmd and gdc users, no?


 That's DStep: https://github.com/jacob-carlborg/dstep


I don't think that works for C++, and it's not complete.
Feb 21 2016
next sibling parent Kagamin <spam here.lot> writes:
On Sunday, 21 February 2016 at 23:30:14 UTC, bachmeier wrote:

 I don't think that works for C++, and it's not complete.


At least it's intended to generate bindings, Calypso does very 
different thing. So if one wants bindings generation, it would be 
easier to implement missing functionality in DStep.
Feb 22 2016
prev sibling parent reply Dicebot <public dicebot.lv> writes:
On 02/22/2016 01:30 AM, bachmeier wrote:

 On Sunday, 21 February 2016 at 22:23:10 UTC, Kagamin wrote:

 On Sunday, 21 February 2016 at 17:21:51 UTC, Brad Roberts wrote:

 Is there anything preventing Calypso from turning into a code and
 interface generator?  Making it an application that is part of the
 build rather than a plug in to ldc would make it available to both
 dmd and gdc users, no?


 That's DStep: https://github.com/jacob-carlborg/dstep


 
 I don't think that works for C++, and it's not complete.


The very reason why Calypso doesn't work with C++ so well is also the
reason why you won't be able to generate bindings easily - it calls C++
code directly without creating intermediate D interface in any form.
Feb 22 2016
next sibling parent bachmeier <no spam.com> writes:
On Monday, 22 February 2016 at 10:20:35 UTC, Dicebot wrote:


 The very reason why Calypso doesn't work with C++ so well is 
 also the reason why you won't be able to generate bindings 
 easily - it calls C++ code directly without creating 
 intermediate D interface in any form.


I wanted to clarify that it doesn't currently claim to be a tool 
for creating C++ bindings, so as not to mislead someone trying 
out the language.
Feb 22 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
On 02/22/2016 12:20 PM, Dicebot wrote:

 The very reason why Calypso doesn't work with C++ so well is also the
 reason why you won't be able to generate bindings easily - it calls C++
 code directly without creating intermediate D interface in any form.


Typo: "very reason why Calypso DOES work with C++ so well"
Feb 22 2016
prev sibling next sibling parent Kagamin <spam here.lot> writes:
Though fully automatic generation of bindings will be very 
difficult because DMD uses semantics deviating from that of C++, 
which will require some heuristic analysis.
Feb 21 2016
prev sibling parent Chris Wright <dhasenan gmail.com> writes:
On Sun, 21 Feb 2016 09:21:51 -0800, Brad Roberts via
Digitalmars-d-announce wrote:


 Making it an application that is part of the build
 rather than a plug in to ldc would make it available to both dmd and gdc
 users, no?


And it would sidestep issues where I run dstep on one platform and try to 
use the results on another, but some #defines are incorrect for the 
second platform.

Of course, that's true for any build-time conversion scheme. Running dstep 
as part of a build would be equivalent.

In terms of practicality, I'd sooner turn to CastXML. It's easier to 
inspect than clang AST.
Feb 21 2016