www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - C++ Binding Generator

reply Paul O'Neil <redballoon36 gmail.com> writes:
I've been thinking about how to generate bindings for C++ (esp. Qt!) for 
a little while now.  Unfortuneatly, I won't have time to implement 
anything until my semester is over, but with the recent talk of 
projects, maybe someone else will pick it up before then.

Qt bindings come up every so often; they are not going to be written 
manually.  The existing Qdd[1] was a port of Qt Jambi[2], which was the 
Java binding.  This makes some sense, since Java is also an 
object-oriented language that isn't C++.  Qt Jambi consists of two 
parts: a generator and generated code.  The generator reads the Qt 
headers as well as XML files that describe how to map the C++ into Java. 
  The generator creates both C++ and Java files that get compiled into a 
JNI library.  Given that D is conceptually closer to C++ than Java, I 
think that binding to D should be easier than Java.  Another issue here 
is that both Qt Jambi and QtD target Qt 4 and not version 5.

Also relevant is that the libclang bindings in Deimos[3] are 2 years and 
several versions old (3.1 -> 3.4 is out).  This is probably due to the 
lack of interest in libclang from D programmers (No surprise.  Who wants 
to write code in D to parse C(++)?).  It binds to libclang's C interface 
and may have been generated with DStep.

Here's my proposed path forward:
Write a binding generator in C++ that uses libclang to parse headers and 
auxillary XML / json / etc. files to guide the generation of D source. 
Clang provides data structures for traversing the C++ AST.  This tool 
could be used to bind Qt 5 and libclang so the conversion tool can 
eventually be migrated to D.  My hope is that using clang to avoid 
writing another C++ parser and D's similarity to C++ make this an easier 
(though still significant) undertaking than the Java bindings.

I suspect that there are several patterns in C++ and D that are 
implemented using slightly different constructions in each of them.  For 
instance, figuring out which C++ classes should map to D structs. 
Automatically identifying these or finding a description language for 
them would be a huge benefit to automation.

On the other hand, I don't yet see a way to map generic containers (such 
as QList<T>) into D yet.  But Qt Jambi found some way to solve this 
problem even when dealing with type erasure in Java.

Thoughts?

Thanks for reading my wall of text.
Paul O'Neil

[1]
QtD is not actively maintained.  It existed on
BitBucket: https://bitbucket.org/qtd/repo
before someone else forked it and moved to
GitHub: https://github.com/qtd-developers/qtd

[2]
Homepage (appears dead): http://qt-jambi.org/
Gitorious (shows recent activity): https://qt.gitorious.org/qt-jambi

[3]
Github: https://github.com/D-Programming-Deimos/libclang

[4]
Clang: http://clang.llvm.org
An example of using Clang to find CXXRecordDecl (classes, unions, 
structs, and the like):
http://clang.llvm.org/docs/RAVFrontendAction.html
Feb 14 2014
next sibling parent reply "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Friday, 14 February 2014 at 23:52:43 UTC, Paul O'Neil wrote:

 I've been thinking about how to generate bindings for C++ (esp. 
 Qt!) for a little while now.  Unfortuneatly, I won't have time 
 to implement anything until my semester is over, but with the 
 recent talk of projects, maybe someone else will pick it up 
 before then.


Why wouldn't you want to expand on DStep and add support for C++, 
it already uses clang?
Feb 14 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-02-15 04:04, Jesse Phillips wrote:


 Why wouldn't you want to expand on DStep and add support for C++, it
 already uses clang?


Yes, it's using libclang.

-- 
/Jacob Carlborg
Feb 15 2014
prev sibling next sibling parent reply "michaelc37" <michaelc37 msn.com> writes:
 [1]
 QtD is not actively maintained.  It existed on
 BitBucket: https://bitbucket.org/qtd/repo
 before someone else forked it and moved to
 GitHub: https://github.com/qtd-developers/qtd


Probably worth mentioning.
There are also some recent startup efforts to generate qt 
bindings based on smoke qt.

https://github.com/w0rp/dqt
https://github.com/GlobecSys/smoke.d

However, its qt4.
Feb 14 2014
next sibling parent Russel Winder <russel winder.org.uk> writes:
On Sat, 2014-02-15 at 03:36 +0000, michaelc37 wrote:

 [1]
 QtD is not actively maintained.  It existed on
 BitBucket: https://bitbucket.org/qtd/repo
 before someone else forked it and moved to
 GitHub: https://github.com/qtd-developers/qtd




There is https://bitbucket.org/michaelc37/qtd-experimental as well. This
was a bit active for a while but has gone quiet recently.


 Probably worth mentioning.
 There are also some recent startup efforts to generate qt 
 bindings based on smoke qt.
 
 https://github.com/w0rp/dqt
 https://github.com/GlobecSys/smoke.d
 
 However, its qt4.


If using Qt let's stick to Qt5 and not create brand new legacy APIs.

In the Pythonverse, Riverbank have some mechanism for creating their
Python bindings, but I bet the tool is proprietary. PySide used some
binding generation tool that appears to have to be rewritten for Qt5 so
they are staying with Qt4. This has stopped the PyQt → PySide rush for
licencing reasons, people are just using PyQt5 and finding ways round
the licencing problems.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Feb 14 2014
prev sibling parent reply "w0rp" <devw0rp gmail.com> writes:
On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:

 [1]
 QtD is not actively maintained.  It existed on
 BitBucket: https://bitbucket.org/qtd/repo
 before someone else forked it and moved to
 GitHub: https://github.com/qtd-developers/qtd


 Probably worth mentioning.
 There are also some recent startup efforts to generate qt 
 bindings based on smoke qt.

 https://github.com/w0rp/dqt
 https://github.com/GlobecSys/smoke.d

 However, its qt4.


I'm working on this slowly. I'll post more when I actually have 
something worth looking at.

My library takes the SMOKE data generated by smokeqt and then 
does at least 90% of the work in D code to manipulate it, for 
generating D source files and loading C++ function pointers at 
runtime so you can wrap C++ classes in D classes. If you have the 
SMOKE data already in a shared library already I recommend this, 
as you can get much better results this way. There is some 
additional C++ code for two reasons. First, because smokeqt 
doesn't generate bindings for QString, which is weird and 
annoying, so I have to provide functions with C calling 
conventions written in C++ for creating QStrings. The second 
reason is that the functions for creating and deleting the QtGUI 
and QtCore SMOKE structs just don't seem to work when you call 
them directly from D, but do work when I wrap them again in 
functions with C calling conventions.

Right now I have a generator which spits out a full list of 
methods and classes, also enums with the right values at the 
moment, and I need to do more work to get it to map from C++ type 
to D type, generate required import lines for tying things 
together, and then generate the snippets of code in methods to 
tie to the code I wrote before for loading the function pointers 
on startup. After that it should work, but not well enough to be 
that useful.

I'll try Qt5 later. The main reason I'm trying to get Qt4 working 
at the moment is that I have experience with Qt4, but not with 
Qt5, and from my understanding a lot of the advantages of Qt5 
come from QML, and I don't quite understand yet how QML would tie 
together with a D API.

If I manage to get my bindings to work, I could see how you could 
take that and move on to creating a generalised C++ binding 
generator, but what you get would never be that great. It would 
be much better to have some kind of support in the language or 
runtime for doing the remaining things D can't do at the moment, 
like matching C++ function calling conventions, calling virtual 
methods, I don't know how you'd get multiple inheritance or 
stack-based classes to work. I get why all of this stuff just 
doesn't exist already, because it's really hard to implement and 
requires a lot of careful thought about it.
Feb 16 2014
next sibling parent reply Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/02/2014 01:36, w0rp a écrit :

 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 working at the
 moment is that I have experience with Qt4, but not with Qt5, and from my
 understanding a lot of the advantages of Qt5 come from QML, and I don't
 quite understand yet how QML would tie together with a D API.


If someone is interested by Qt mainly for QML, maybe the better way is 
to help us on DQuick, cause it's really similar, but we try to take 
advantages of D to do something more friendly.
Feb 16 2014
next sibling parent reply "w0rp" <devw0rp gmail.com> writes:
On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:

 Le 17/02/2014 01:36, w0rp a écrit :

 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 
 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 
 working at the
 moment is that I have experience with Qt4, but not with Qt5, 
 and from my
 understanding a lot of the advantages of Qt5 come from QML, 
 and I don't
 quite understand yet how QML would tie together with a D API.


 If someone is interested by Qt mainly for QML, maybe the better 
 way is to help us on DQuick, cause it's really similar, but we 
 try to take advantages of D to do something more friendly.


Writing a native D GUI library instead of wrapping a C++ one is a 
good idea. My big problem with the wrapper around Qt I'm building 
is that it will never run as efficiently as a Qt application 
written in C++ would, so that might turn some people away. The 
main advantage of writing the wrapper is pretty much that it's 
just less work, as creating a cross platform toolkit with an API 
as nice as Qt's takes a lot of time and effort.
Feb 16 2014
next sibling parent reply "eles" <eles eles.com> writes:
On Monday, 17 February 2014 at 07:48:51 UTC, w0rp wrote:

 On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand 
 wrote:

 Le 17/02/2014 01:36, w0rp a écrit :

 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 
 wrote:







 Writing a native D GUI library instead of wrapping a C++ one is 
 a good idea. My big problem with the wrapper around Qt I'm 
 building is that it will never run as efficiently as a Qt 
 application written in C++ would, so that might turn some 
 people away. The main advantage of writing the wrapper is 
 pretty much that it's just less work, as creating a cross 
 platform toolkit with an API as nice as Qt's takes a lot of 
 time and effort.


I agree with this but, still, a wrapper around Qt 5 (not 4!) 
would be a huge thing, especially if one wants to write Android 
Qt-based applications.
Feb 17 2014
parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/02/2014 09:18, eles a écrit :

 On Monday, 17 February 2014 at 07:48:51 UTC, w0rp wrote:

 On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:

 Le 17/02/2014 01:36, w0rp a écrit :

 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:







 Writing a native D GUI library instead of wrapping a C++ one is a good
 idea. My big problem with the wrapper around Qt I'm building is that
 it will never run as efficiently as a Qt application written in C++
 would, so that might turn some people away. The main advantage of
 writing the wrapper is pretty much that it's just less work, as
 creating a cross platform toolkit with an API as nice as Qt's takes a
 lot of time and effort.


 I agree with this but, still, a wrapper around Qt 5 (not 4!) would be a
 huge thing, especially if one wants to write Android Qt-based applications.


I can tell you Qt 5 miss some features for Android or have some bugs. I 
wrote some java bindings to solve Qt issues. It's not a big deal.
The main issue is the port of D and phobos on Android.

With DQuick we target openGL 2.1 to be compatible with openGL ES. We 
have iOS and Android in mind too.
Feb 17 2014
prev sibling parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/02/2014 08:48, w0rp a écrit :

 On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:

 Le 17/02/2014 01:36, w0rp a écrit :

 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 working at the
 moment is that I have experience with Qt4, but not with Qt5, and from my
 understanding a lot of the advantages of Qt5 come from QML, and I don't
 quite understand yet how QML would tie together with a D API.


 If someone is interested by Qt mainly for QML, maybe the better way is
 to help us on DQuick, cause it's really similar, but we try to take
 advantages of D to do something more friendly.


 Writing a native D GUI library instead of wrapping a C++ one is a good
 idea. My big problem with the wrapper around Qt I'm building is that it
 will never run as efficiently as a Qt application written in C++ would,
 so that might turn some people away. The main advantage of writing the
 wrapper is pretty much that it's just less work, as creating a cross
 platform toolkit with an API as nice as Qt's takes a lot of time and
 effort.


Yep, that why we are focus on the smallest part of Qt needed to create 
GUI. IMO all features of QtCore have to be directly in phobos (Network, 
serialization, signals,...), so only Widgets and QtQuick API stay here.

The advantage of QtQuick his the modernity and it stay really small, 
it's mainly a script engine (property binding) coupled with a 2D modern 
renderer based on rasterization.
We took lua as script language cause it can support property bindings 
without modifications. And the renderer just wait for contributions, the 
minimal functional target isn't really hard to get.

But polish stay hard to do.
Feb 17 2014
prev sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:

 Le 17/02/2014 01:36, w0rp a écrit :

 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 
 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 
 working at the
 moment is that I have experience with Qt4, but not with Qt5, 
 and from my
 understanding a lot of the advantages of Qt5 come from QML, 
 and I don't
 quite understand yet how QML would tie together with a D API.


 If someone is interested by Qt mainly for QML, maybe the better 
 way is to help us on DQuick, cause it's really similar, but we 
 try to take advantages of D to do something more friendly.


 From my personal point of view my main interest is in the 
programmatic Qt constructions (ie. Core & Widget module, not QML) 
as that's my itch that needs scratching. I'm not applying any 
effort ATM into the QML side of Qt. I think that a native DQuick 
implementation would be great and would probably provide a very 
compelling alternative to a wrapped QML (if that ever gets done 
by anybody) so I hope you get lets of help from anyone interested 
in that side of things.
Feb 17 2014
parent Xavier Bigand <flamaros.xavier gmail.com> writes:
Le 17/02/2014 09:54, Abdulhaq a écrit :

 On Monday, 17 February 2014 at 01:16:44 UTC, Xavier Bigand wrote:

 Le 17/02/2014 01:36, w0rp a écrit :

 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:
 I'll try Qt5 later. The main reason I'm trying to get Qt4 working at the
 moment is that I have experience with Qt4, but not with Qt5, and from my
 understanding a lot of the advantages of Qt5 come from QML, and I don't
 quite understand yet how QML would tie together with a D API.


 If someone is interested by Qt mainly for QML, maybe the better way is
 to help us on DQuick, cause it's really similar, but we try to take
 advantages of D to do something more friendly.


  From my personal point of view my main interest is in the programmatic
 Qt constructions (ie. Core & Widget module, not QML) as that's my itch
 that needs scratching. I'm not applying any effort ATM into the QML side
 of Qt. I think that a native DQuick implementation would be great and
 would probably provide a very compelling alternative to a wrapped QML
 (if that ever gets done by anybody) so I hope you get lets of help from
 anyone interested in that side of things.


I am completely agree with you, binding of Qt must be focused on Widgets 
and the core module.

I am dreaming to be able to convince some QtQuick developers to 
contribute to DQuick, we already have some things working better.
I am thinking to the property binding loop detection on which we are 
able to print the full call stack, where QML engine is only able to give 
the item not the property affected.
And with the power of D it's not necessary to modify existent code, here 
with Qt wrappers are needed.
Feb 17 2014
prev sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 17 February 2014 at 00:36:58 UTC, w0rp wrote:

 On Saturday, 15 February 2014 at 03:36:29 UTC, michaelc37 wrote:

 [1]
 QtD is not actively maintained.  It existed on
 BitBucket: https://bitbucket.org/qtd/repo
 before someone else forked it and moved to
 GitHub: https://github.com/qtd-developers/qtd


 Probably worth mentioning.
 There are also some recent startup efforts to generate qt 
 bindings based on smoke qt.

 https://github.com/w0rp/dqt
 https://github.com/GlobecSys/smoke.d

 However, its qt4.


 I'm working on this slowly. I'll post more when I actually have 
 something worth looking at.

 My library takes the SMOKE data generated by smokeqt and then 
 does at least 90% of the work in D code to manipulate it, for 
 generating D source files and loading C++ function pointers at 
 runtime so you can wrap C++ classes in D classes.


I'm sure we're facing similar problems from time to time, we 
should probably have a chat about that. Here's to hoping we both 
get a successful Qt wrapping and suddenly there is a choice of Qt 
wrapper!

Here's a great article about the problems with C++ bindings and 
Qt in particular:

http://setanta.wordpress.com/binding-c/
Feb 17 2014
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-02-15 00:52, Paul O'Neil wrote:


 Also relevant is that the libclang bindings in Deimos[3] are 2 years and
 several versions old (3.1 -> 3.4 is out).  This is probably due to the
 lack of interest in libclang from D programmers (No surprise.  Who wants
 to write code in D to parse C(++)?).


I originally create those bindings to be used by DStep. But never bother 
updating them after that. DStep contains its own bindings. I haven't 
updated the bindings to anything later than 3.1 since I don't use any of 
these features yet.


 It binds to libclang's C interface  and may have been generated with DStep.


Actually, I just did some simple search-and-replace on the original C 
header. libclang is the easiest binding I've created :)


 Here's my proposed path forward:
 Write a binding generator in C++ that uses libclang to parse headers and
 auxillary XML / json / etc. files to guide the generation of D source.
 Clang provides data structures for traversing the C++ AST.  This tool
 could be used to bind Qt 5 and libclang so the conversion tool can
 eventually be migrated to D.  My hope is that using clang to avoid
 writing another C++ parser and D's similarity to C++ make this an easier
 (though still significant) undertaking than the Java bindings.


Why not contribute to DStep?


 I suspect that there are several patterns in C++ and D that are
 implemented using slightly different constructions in each of them.  For
 instance, figuring out which C++ classes should map to D structs.
 Automatically identifying these or finding a description language for
 them would be a huge benefit to automation.

 On the other hand, I don't yet see a way to map generic containers (such
 as QList<T>) into D yet.  But Qt Jambi found some way to solve this
 problem even when dealing with type erasure in Java.


You could hard code a couple of these containers. QList<T> could be 
converted to T[] or a linked list. I've done that with a couple of other 
types: like "BOOL" is translated to "bool" and "wchar_t" is translated 
to "wchar".

-- 
/Jacob Carlborg
Feb 15 2014
parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Saturday, 15 February 2014 at 11:25:09 UTC, Jacob Carlborg 
wrote:

 and "wchar_t" is translated to "wchar".


That doesn't seem correct, it should be `wchar` on Windows only, 
and `dchar` on other systems, i.e. it should translate to 
core.stdc.stddef.wchar_t.
Feb 15 2014
parent Jacob Carlborg <doob me.com> writes:
On 2014-02-15 12:54, Jakob Ovrum wrote:


 That doesn't seem correct, it should be `wchar` on Windows only, and
 `dchar` on other systems, i.e. it should translate to
 core.stdc.stddef.wchar_t.


I was mistaken. Currently DStep recognizes the "wchar_t" typedef, then 
it checks the size of it, translating to either "wchar" or "dchar". This 
is correct but will not generate cross-platform bindings. 
core.stdc.stddef.wchar_t seems like a better choice. Thanks.

-- 
/Jacob Carlborg
Feb 15 2014
prev sibling next sibling parent "TC" <chalucha gmail.com> writes:
Can SWIG[1] be used to help here? It even supports D somehow. As 
I understand, Python uses SIP[2] to create Qt and other bindings.

[1]http://www.swig.org/
[2]http://www.riverbankcomputing.co.uk/software/sip/intro
Feb 15 2014
prev sibling next sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
I'm working on a C++ binding generator that is coming on well,
called Smidgen (first there was SWIG, then sip, now Smidgen...).
I have a small demo working with both VTK and the beginnings of
Qt5 GUI. I have to pick it back up (after moving house with my
family) so would like help on various things such as building it
on Windows (currently Linux 64bit only but it's only the build
system that is Linux specific, and it's cmake so should be easy
to get going on Windows).

The code is tested and just-about one-click build. Currently it's
in a subversion repository but I plan to move to github. I'd love
someone to use Smidgen to try wrapping a small C++ library as I
think it would make the job quite easy, if anyone reads this
(since I have not formally announced the thing yet) then please
let me know and I'll get the code out there).

I have targeted the sip format binding specification (as used in
sip / pyqt) so that I then get a Qt binding on the cheap (but it
will therefore be GPL only). It is going well and as I said I
have wrapped a small part of Qt GUI using it (and the PyQt sip
files) which demonstrates working:

* Using the PyQt sip files is working and I am parsing nearly all
sip tags - extending to the rest of the PyQt sip files should not
be _too_ arduous

* Subclassing a wrapped C++ class, and overriding protected and
virtual functions (e.g. paint method and mouseMoveEvent)

* Nested classes (e.g. Qt::Connection)

* Multiple packages / modules

* Enums half-done

* Sip Transfer, TransferThis and TransferBack annotations for
method arguments (which govern object ownership)

* Multiple C++ inheritance => multiple pointers (still needs a
little bit of work but generally there)

* An automated route for type conversion e.g. QString <-> D
string - needs a bit more work for e.g. QList

My TODO List
============


* non-primitive Typedefs

* Primitive types C -> D conversion
     - *.conf file
      %CToDType long = long
      %CToDType unsigned char = ubyte  (half done)

* Sip If clauses for timelines
      %Timeline {Qt_5_0_0 Qt_5_0_1 Qt_5_0_2}  (half done)


* KeepReference for arguments + tests for Transfer etc. - Easy,
in class with KeepRef
    e.g. View.setModel(model /KeepReference/) it has an extra
attribute
        -   void* setModel_SMIKeepRef
        then in setModel() {
          View_setModel_SMIX23(model.wrappedObject);
          setModel_SMIKeepRef = model;
        }

        This will make D keep a reference to the model as long as
the View instance is alive.
        Each view will have its own reference so the total can go
above 1 for a given model.

* getCastPointerForInterface can be easily improved by not
switching on a name but instead
    each class has a separate variable for each base class pointer,
that is populated in
    the constructor

    - each class has one extra pointer per interface implemented
      - override virtual void*[] getExtraPointers() {
          void*[] extraPointers = super.getExtraPointers();
          extraPointers ~= wrappedObject_Calculator;
          return extraPointers;
      }

    - in constructor this() {
          wrappedObject_Calculator =
castRectAsCalculator(wrappedObject);
      }

    - in destructor ~this() {
          deregisterWrappedObject(wrappedObject);
          deregisterWrappedObejct(wrappedObject_Calculator);
      }

* instance_wrapper et al., need to also register base class
pointers

* getWrappedObject / getClassName - how to handle this in a
x-module fashion.

* Add support for wrapping members

* Add QTest support

* int arguments that take a default enum value & enum defaults

* char** -> Use this _idea_
      this(string[] args)
      {
          //        if (m_instance != null)
          //            throw new
RuntimeException("QCoreApplication can only be initialized once");

          argc = cast(int)args.length;
          argv = toStringzArray(args);
          this(&argc, argv);

          //        m_instance.aboutToQuit.connect(m_instance,
"disposeOfMyself()");
      }

* Threading? - wrappedObjects[] should be shared as with CPP
instance tracker. Use signal.d's
    WeakRef and InvisibleAddress

* qRegisterMetaType??

* Add %UsesConverter to package.sip, so that the correct includes
are in the package and
    we don't get problems trying to include e.g. widget.h in 
core.so

* Converted types that are returned are not passed through
getWrapper - might cause
    duplicate D objects around same C++ object e.g. QList<QWidget>?
Is this a problem?

* In CPP wrapper, the conversion from argument can have a memory
leak (e.g. new string created
   and not deleted)  BETTER - pass new type in as a reference if
possible

* Enums don't have C++ value if it was specified - need a CPP
program that writes
    a text file e.g. Color::Red 124
                     Color::Green 234
    and then write enum wrappers which include the values using the
text lookup (at build time
    of wrapper NOT at runtime of final program)

* Operator overloading - do nicely in D

* Free function operator overloads in CPP

* Handle array arguments

* Multiple inheritance - if a method returns an instance of an
interface and it has
    been created by CPP then the *Impl is returned - it is not
currently typechecked.
    We should instead check if it's already on a registered object.

* Interfaces - allow virtual calls on the interface

* Lifetime management - VTK?

* VTK/ Qt differences - have factory methods for Method, Klass,
Package etc., and
    different wrapping flavours can have differing factories. Needs
API stability.

* Virtual functions, overloading etc. - will the right call be
made if from an inherited class I make a call
    on an inherited D class method for a virtual method - will it
call the C++ base equivalent,
    thereby not calling the correct virtual call on the derived
class? See GOTW 5,
    calling virtual funcs on a base pointer will not call the
derived class method.

* Answer to above Q - yes, so need to implement wrappers for all
inherited virtual functions
    even if not mentioned in SIP file

* Signals and slots

* Enums declared inside classes

* In CPP wrapper class remove factory function/private
constructor - it is not reqd.

* Nested classes and multiple inheritance combination probably
not working (e.g. Klass.getWrappedClassName)
Feb 15 2014
next sibling parent reply "michaelc37" <michaelc37 msn.com> writes:
 The code is tested and just-about one-click build. Currently 
 it's
 in a subversion repository but I plan to move to github. I'd 
 love
 someone to use Smidgen to try wrapping a small C++ library as I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then please
 let me know and I'll get the code out there).


yes github please.. i'd like to check this out.
Feb 15 2014
next sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
 yes github please.. i'd like to check this out.


OK I'm in the process of migrating the code to github. I was in 
such a hurry yesterday evening I didn't list the features 
properly:

FEATURES
========

* All D
* Understandable, maintainable code
* Wraps protected and virtual methods, allows virtual methods to 
be overridden in D
* Mixin classes in target C++ library supported
* Allows custom type conversions between C++ and D types
* C++ enums mapped to D enums and are type checked in D
* Wraps nested C++ classes
* Tested
* Based on the sip format. This is well proven and allows 
simplified maintenance of wrappers for multiple versions of the 
target library.  (All larger target libraries will need some 
ongoing maintenance of the wrapper regardless of the wrapping 
technology).

Once all the code is in github I'll just check that a totally 
clean build is working, and then give you a heads up. FYI the 
repository is at

https://github.com/alynch4047/smidgen/tree/master/morsel
Feb 16 2014
next sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
 https://github.com/alynch4047/smidgen/tree/master/morsel


that should of course be

https://github.com/alynch4047/smidgen
Feb 16 2014
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/16/14, Abdulhaq <alynch4047 gmail.com> wrote:

 that should of course be

 https://github.com/alynch4047/smidgen


Nice. I've had a codegen project before which is currently stalled:
https://github.com/AndrejMitrovic/dgen

Some docs I wrote about peculiarities of wrapping C++:
https://github.com/AndrejMitrovic/dgen/wiki

Unfortunately the tool has a long history, it went through several
rewrites. From basing the typeinfo extracted from doxygen (initially
the tool was only supposed to be used to generate wxWidgets bindings),
to later basing it on GCCXML, to later forking GCCXML and adding my
own features (https://github.com/AndrejMitrovic/gccxml), to finally
realizing I won't be able to get all C++ typeinfo from GCCXML because
it's based on an old GCC parser. It was an uphill battle.

Ultimately I would have to re-purpose the tool to use Clang.

I did have some initial success. I've managed to autowrap the TagLib
C++ library, some small XML libraries, but ultimately the tool needs
proper typeinfo that GCCXML doesn't provide.

Clang itself has a C API, but from what I've read online it doesn't
export enough C++ typeinfo either, so I'd have to use the Clang C++
interface instead (maybe libtooling or something of that sort).

I might get back to working on that tool one day, I don't know yet.
I've got a big queue of things to work on.

Your project looks nice though.
Feb 17 2014
next sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 17 February 2014 at 09:46:15 UTC, Andrej Mitrovic 
wrote:

 Nice. I've had a codegen project before which is currently 
 stalled:
 https://github.com/AndrejMitrovic/dgen

 Some docs I wrote about peculiarities of wrapping C++:
 https://github.com/AndrejMitrovic/dgen/wiki


Interesting reading, it seems we've run into the same problems 
(not surprising I suppose) and even come up with the same 
solutions (virtual functions, return-by-value).


 Unfortunately the tool has a long history, it went through 
 several
 rewrites. From basing the typeinfo extracted from doxygen 
 (initially
 the tool was only supposed to be used to generate wxWidgets 
 bindings),
 to later basing it on GCCXML, to later forking GCCXML and 
 adding my
 own features (https://github.com/AndrejMitrovic/gccxml), to 
 finally
 realizing I won't be able to get all C++ typeinfo from GCCXML 
 because
 it's based on an old GCC parser. It was an uphill battle.

 Ultimately I would have to re-purpose the tool to use Clang.

 I did have some initial success. I've managed to autowrap the 
 TagLib
 C++ library, some small XML libraries, but ultimately the tool 
 needs
 proper typeinfo that GCCXML doesn't provide.

 Clang itself has a C API, but from what I've read online it 
 doesn't
 export enough C++ typeinfo either, so I'd have to use the Clang 
 C++
 interface instead (maybe libtooling or something of that sort).


I feel your pain! Because my main target library was Qt I was 
able to bypass the whole typeinfo extraction issue by basing the 
binding generator on sip files and then primarily using the PyQt 
sip files - with the penalty that the PyQt wrapping will 
therefore be GPL. However, seeing how hard it can be to extract 
the typeinfo automatically (and the object ownership information 
still will need to be done manually) I'm very glad that I fell 
into doing it that way. A feature evolution for me would be to 
try and bolt on an automated typeinfo extractor front end. 
Riverbank already have one for generating sip files but it's 
practically undocumented and hard to use.



 I might get back to working on that tool one day, I don't know 
 yet.
 I've got a big queue of things to work on.


Same here - so much to do and so little time! If I'm not careful 
I end up with analysis paralysis and just browse HN and these 
forums instead of catching up on my reading list or coding 
something up. Speaking of which....
Feb 17 2014
next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/17/14, Abdulhaq <alynch4047 gmail.com> wrote:

 Interesting reading, it seems we've run into the same problems
 (not surprising I suppose) and even come up with the same
 solutions (virtual functions, return-by-value).


And then there are things like trying to allow access to protected
methods (meaning you have to re-declare them public for the C wrappers
to access them), which isn't easy because simply re-declaring them
only works in /some/ contests (specifically when overloads come into
play it's a mess). And then having to handle namespaces.. etc etc.
It's very easy to burn out. :)

I can't even recall how I planned to support multiple inheritance. I
think I was planning on using interfaces, but to keep all classes
neatly castable up the hirearchy at the D side it would probably mean
having to make /all/ classes inherit from interfaces.

And then you have to worry about memory management. Fun stuff..
Feb 17 2014
parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 17 February 2014 at 22:58:57 UTC, Andrej Mitrovic 
wrote:

 And then there are things like trying to allow access to 
 protected
 methods (meaning you have to re-declare them public for the C 
 wrappers
 to access them), which isn't easy because simply re-declaring 
 them
 only works in /some/ contests (specifically when overloads come 
 into
 play it's a mess). And then having to handle namespaces.. etc 
 etc.
 It's very easy to burn out. :)


There are many problems to solve, it take a lot of thought and 
time. I've ploughed all my spare time into it for a couple of 
months, then I moved house. Now I've got to get back in to it 
again :-).


 I can't even recall how I planned to support multiple 
 inheritance. I
 think I was planning on using interfaces, but to keep all 
 classes
 neatly castable up the hirearchy at the D side it would 
 probably mean
 having to make /all/ classes inherit from interfaces.


I'm only trying to handle mixin-style multiple inheritance, and 
doing it with interfaces for the mixins. I've got to finish up 
dealing with the fact that the C++ pointer can change for the 
same object with multiple inheritance, but it's just about done.


 And then you have to worry about memory management. Fun stuff..


It's so depressing when it all seems golden and then the 
application using the wrapper crashes! Anyway I believe I've got 
that aspect sorted out now. I wish D had proper support for weak 
references.

I'm glad to have this conversation it's given me renewed energy 
to finish it off...
Feb 18 2014
prev sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/17/14, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:

 On 2/17/14, Abdulhaq <alynch4047 gmail.com> wrote:

 Interesting reading, it seems we've run into the same problems
 (not surprising I suppose) and even come up with the same
 solutions (virtual functions, return-by-value).


 And then there are things like trying to allow access to protected
 methods


It looks like I already described this in the wiki, I'm repeating
myself here. :)
Feb 17 2014
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2014-02-17 10:45, Andrej Mitrovic wrote:


 Clang itself has a C API, but from what I've read online it doesn't
 export enough C++ typeinfo either, so I'd have to use the Clang C++
 interface instead (maybe libtooling or something of that sort).


Why not contribute to Clang by adding the missing functionality. Then 
contributing to DStep ;)

-- 
/Jacob Carlborg
Feb 17 2014
prev sibling parent reply "Joao Matos" <joao tritao.eu> writes:
On Monday, 17 February 2014 at 09:46:15 UTC, Andrej Mitrovic 
wrote:

 On 2/16/14, Abdulhaq <alynch4047 gmail.com> wrote:

 that should of course be

 https://github.com/alynch4047/smidgen


 Nice. I've had a codegen project before which is currently 
 stalled:
 https://github.com/AndrejMitrovic/dgen

 Some docs I wrote about peculiarities of wrapping C++:
 https://github.com/AndrejMitrovic/dgen/wiki


Nice tools, I've also been working on a C++ binding tool myself 


https://github.com/mono/CppSharp

It is based on the Clang C++ parser. All the hard work of parsing 
and C++ AST support is done, so in theory a D backend could be 
added, if the days had more hours I'd work on one myself :)


binding generator I've seen referenced before in this thread) has 



https://github.com/ddobrev/QtSharp
Feb 17 2014
parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 2/18/14, Joao Matos <joao tritao.eu> wrote:

 Nice tools, I've also been working on a C++ binding tool myself


 https://github.com/mono/CppSharp


Does this use Clang's C++ API or libclang which is a C API? If the

not very easily. Was it hard to do? I guess I'm looking for any
experiences you can share.
Feb 17 2014
parent "Joao Matos" <joao tritao.eu> writes:
On Tuesday, 18 February 2014 at 07:49:20 UTC, Andrej Mitrovic 
wrote:

 On 2/18/14, Joao Matos <joao tritao.eu> wrote:

 Nice tools, I've also been working on a C++ binding tool myself


 https://github.com/mono/CppSharp


 Does this use Clang's C++ API or libclang which is a C API? If 
 the

 least
 not very easily. Was it hard to do? I guess I'm looking for any
 experiences you can share.


I actually started with the libclang C API but found out that it 
was too limited, so I switched to the C++ one.

Well, .NET can actually call directly into C++ via C++/CLI (it's 
limited to Windows though) so that's what the first parser relied 
on. We now have a second parser that uses pure C++ and uses 

on all platforms.

And yes, it's been pretty hard to do, binding C++ _is_ hard, 
though D should have a big advantage here, it's semantics are 
closer to C++ than many other languages.
Feb 18 2014
prev sibling parent reply "Moritz Maxeiner" <moritz ucworks.org> writes:
On Sunday, 16 February 2014 at 14:10:02 UTC, Abdulhaq wrote:

 yes github please.. i'd like to check this out.


 OK I'm in the process of migrating the code to github. I was in 
 such a hurry yesterday evening I didn't list the features 
 properly:

 FEATURES
 ========

 * All D
 * Understandable, maintainable code
 * Wraps protected and virtual methods, allows virtual methods 
 to be overridden in D
 * Mixin classes in target C++ library supported
 * Allows custom type conversions between C++ and D types
 * C++ enums mapped to D enums and are type checked in D
 * Wraps nested C++ classes
 * Tested
 * Based on the sip format. This is well proven and allows 
 simplified maintenance of wrappers for multiple versions of the 
 target library.  (All larger target libraries will need some 
 ongoing maintenance of the wrapper regardless of the wrapping 
 technology).

 Once all the code is in github I'll just check that a totally 
 clean build is working, and then give you a heads up. FYI the 
 repository is at

 https://github.com/alynch4047/smidgen/tree/master/morsel


This looks very nice!
Two questions, though:

1) Is it possible to use an already existing C API with morsel? 
The reason I'm asking is, because LLVM classes contain many 
methods where an automatic conversion is likely to cause memory 
leaks to appear. This is the main reason why I decided to use the 
C-API for my llvm-d project, instead of attempting to use 
automatic tools like SWIG (that and the fact that LLVM did not 
export C++ symbols properly for a Windows DLL, so you needed to 
use the C API anyway).

2) What is the main difference between using morsel and SWIG, 
other than the language used to specify the interface?
Feb 18 2014
parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
On Tuesday, 18 February 2014 at 11:05:20 UTC, Moritz Maxeiner 
wrote:

 This looks very nice!
 Two questions, though:

 1) Is it possible to use an already existing C API with morsel? 
 The reason I'm asking is, because LLVM classes contain many 
 methods where an automatic conversion is likely to cause memory 
 leaks to appear. This is the main reason why I decided to use 
 the C-API for my llvm-d project, instead of attempting to use 
 automatic tools like SWIG (that and the fact that LLVM did not 
 export C++ symbols properly for a Windows DLL, so you needed to 
 use the C API anyway).


At the moment smidgen (morsel is the example C++ library I use 
for testing) only wraps C++ classes, on the TODO is of course to 
wrap C++ top level functions etc. It makes a lot of sense to add 
support for C functions too, but to be totally honest I have no 
personal need for that ATM so it would come after everything 
else. I would class smidgen as at beta stage at the moment in 
terms of the functionality it currently supports. It needs to be 
(ab)used by some other developers testing it out on some small 
APIs, and then once I feel it has settled down others could 
contribute code to support C APIs.

Regarding memory leaks, you're right, to wrap a C++ library there 
has to be manual effort on the part of the developer to tell the 
wrapper where object ownership is transferred etc. This cannot be 
automated. Smidgen is based on the sip format and it's pretty 
easy to annotate the API to indicate where ownership is 
transferred.


 2) What is the main difference between using morsel and SWIG, 
 other than the language used to specify the interface?


I don't know much about SWIG but the biggest difference would 
have to be SWIG's great maturity and extensive feature set. 
Smidgen has its own plusses too though:

* It's pure D and the code is tested and maintainable - i.e. if 
it doesn't work for someone they could probably extend/change it 
so that it did

* It's a very thin wrapper around the client library, so I 
suspect it's a bit faster at runtime than the more generic SWIG

* Did I mention it's in D :-)

I should stress that this is beta code and needs a bit of a 
hammering on smaller libraries before anyone spends a lot of 
effort with it. However, a smaller library would not take long to 
wrap using smidgen, so little time would be wasted if it didn't 
work out.
Feb 18 2014
parent "Moritz Maxeiner" <moritz ucworks.org> writes:
On Tuesday, 18 February 2014 at 11:52:24 UTC, Abdulhaq wrote:

 At the moment smidgen (morsel is the example C++ library I use 
 for testing) only wraps C++ classes, on the TODO is of course 
 to wrap C++ top level functions etc. It makes a lot of sense to 
 add support for C functions too, but to be totally honest I 
 have no personal need for that ATM so it would come after 
 everything else. I would class smidgen as at beta stage at the 
 moment in terms of the functionality it currently supports. It 
 needs to be (ab)used by some other developers testing it out on 
 some small APIs, and then once I feel it has settled down 
 others could contribute code to support C APIs.

 Regarding memory leaks, you're right, to wrap a C++ library 
 there has to be manual effort on the part of the developer to 
 tell the wrapper where object ownership is transferred etc. 
 This cannot be automated. Smidgen is based on the sip format 
 and it's pretty easy to annotate the API to indicate where 
 ownership is transferred.


 2) What is the main difference between using morsel and SWIG, 
 other than the language used to specify the interface?


 I don't know much about SWIG but the biggest difference would 
 have to be SWIG's great maturity and extensive feature set. 
 Smidgen has its own plusses too though:

 * It's pure D and the code is tested and maintainable - i.e. if 
 it doesn't work for someone they could probably extend/change 
 it so that it did

 * It's a very thin wrapper around the client library, so I 
 suspect it's a bit faster at runtime than the more generic SWIG

 * Did I mention it's in D :-)



Thanks for the comprehensive write-up, I'll have a look at
smidgen the next time I'm fiddling with llvm-d, then. It seems
promising, but I'm not yet entirely convinced that it'd fit well
with LLVM: About 70% of their C++ API not in the backend is
essentially helpers classes to deal with things either already
present or not needed in languages like D. The problem is that
the methods you actually want to access very often require
arguments with the type being one of these helper classes. And
there's no benefit of having these classes in D other than to use
instances of them as arguments to the rest of the LLVM API, since
their functionality is already present in D or not needed.
Anyway, I'll try it out when I have time and see if it beats
writing the wrapper manually.
Feb 18 2014
prev sibling parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:

 The code is tested and just-about one-click build. Currently 
 it's
 in a subversion repository but I plan to move to github. I'd 
 love
 someone to use Smidgen to try wrapping a small C++ library as I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then please
 let me know and I'll get the code out there).


 yes github please.. i'd like to check this out.


OK I believe that smidgen is now fully hosted on github.

Using smidgen as it stands I would hope that you could wrap a 
simple C++ library. However, no-one else has tried it yet so 
there are bound to be a few teething problems, please feel free 
to email me (anyone trying it out or just taking a look) at 
alynch4047 at gmail dot com if you have any questions / problems. 
Alternatively perhaps we could use one of the forums here??

I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll 
try to do that over the next day or two.
Feb 16 2014
next sibling parent reply "ed" <growlercab gmail.com> writes:
On Sunday, 16 February 2014 at 20:05:27 UTC, Abdulhaq wrote:

 On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:

 The code is tested and just-about one-click build. Currently 
 it's
 in a subversion repository but I plan to move to github. I'd 
 love
 someone to use Smidgen to try wrapping a small C++ library as 
 I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then 
 please
 let me know and I'll get the code out there).


 yes github please.. i'd like to check this out.


 OK I believe that smidgen is now fully hosted on github.

 Using smidgen as it stands I would hope that you could wrap a 
 simple C++ library. However, no-one else has tried it yet so 
 there are bound to be a few teething problems, please feel free 
 to email me (anyone trying it out or just taking a look) at 
 alynch4047 at gmail dot com if you have any questions / 
 problems. Alternatively perhaps we could use one of the forums 
 here??

 I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll 
 try to do that over the next day or two.


D wrappings for VTK would be great!

I have a SWIG version of D-VTK but it is a little flaky and soooo 
slow to generate. I've just started work developing D wrapper for 
VTK based on their Java wrapper but it's very early stages 
(started yesterday in fact).

If you've got a VTK wrapping that generates D code I'd love to 
check it out.

Cheers,
ed
Feb 16 2014
parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
 I have a SWIG version of D-VTK but it is a little flaky and 
 soooo slow to generate. I've just started work developing D 
 wrapper for VTK based on their Java wrapper but it's very early 
 stages (started yesterday in fact).

 If you've got a VTK wrapping that generates D code I'd love to 
 check it out.


The smidgen binding generator is (I believe) sufficiently feature 
rich to wrap all of VTK, but I've only wrapped a half dozen 
classes as proof of concept for now (I've been focussing on Qt5). 
My plan on the VTK side has been to use the automated binding 
generator in VTK to produce the *.sip (similar to *.h) files that 
smidgen needs.

VTK is quite a large library so I would expect the wrapper to be 
a bit slow to generate, however it shouldn't be flaky when 
running. I would suspect that you've got a glitch in object 
ownership somewhere? My plan has been that the automatic VTK 
binding generator for e.g. python knows about object ownership, 
and so I can tap in to that to generate the correct smidgen 
annotations that are needed for that aspect of the binding. I've 
also found that you can get problems when wrapped methods return 
instances typed as base classes that need casting to more 
specific types.

I'm wondering if we can pool our efforts somehow? Because I need 
to finish the last parts of smidgen (e.g. enums with specific 
values and subtleties with C++ multiple inheritance)and the Qt5 
wrapping side of it I couldn't divert much from my current 
trajectory, but if you get a chance then have a look at what 
smidgen is doing and see if you think you'd get any benefit using 
it (and you'd have my efforts working to extend smidgen for you 
if needed). As soon as I've uploaded the current smidgen-vtk I'll 
notify here.

Another way we might be able to cooperate is getting a full 
understanding of the current built-in VTK binding generator 
technology.

Is your SWIG wrapping online somewhere? - I'd like to take a look.
Feb 17 2014
parent reply "ed" <growlercab gmail.com> writes:
On Monday, 17 February 2014 at 08:49:09 UTC, Abdulhaq wrote:

 I have a SWIG version of D-VTK but it is a little flaky and 
 soooo slow to generate. I've just started work developing D 
 wrapper for VTK based on their Java wrapper but it's very 
 early stages (started yesterday in fact).

 If you've got a VTK wrapping that generates D code I'd love to 
 check it out.


 The smidgen binding generator is (I believe) sufficiently 
 feature rich to wrap all of VTK, but I've only wrapped a half 
 dozen classes as proof of concept for now (I've been focussing 
 on Qt5). My plan on the VTK side has been to use the automated 
 binding generator in VTK to produce the *.sip (similar to *.h) 
 files that smidgen needs.

 VTK is quite a large library so I would expect the wrapper to 
 be a bit slow to generate, however it shouldn't be flaky when 
 running. I would suspect that you've got a glitch in object 
 ownership somewhere? My plan has been that the automatic VTK 
 binding generator for e.g. python knows about object ownership, 
 and so I can tap in to that to generate the correct smidgen 
 annotations that are needed for that aspect of the binding. 
 I've also found that you can get problems when wrapped methods 
 return instances typed as base classes that need casting to 
 more specific types.

 I'm wondering if we can pool our efforts somehow? Because I 
 need to finish the last parts of smidgen (e.g. enums with 
 specific values and subtleties with C++ multiple 
 inheritance)and the Qt5 wrapping side of it I couldn't divert 
 much from my current trajectory, but if you get a chance then 
 have a look at what smidgen is doing and see if you think you'd 
 get any benefit using it (and you'd have my efforts working to 
 extend smidgen for you if needed). As soon as I've uploaded the 
 current smidgen-vtk I'll notify here.

 Another way we might be able to cooperate is getting a full 
 understanding of the current built-in VTK binding generator 
 technology.

 Is your SWIG wrapping online somewhere? - I'd like to take a 
 look.


https://github.com/lyrebirdsw/vtkdbind

The SWIG bindings were left in a very bad state so they won't 
work at the moment. It should generate the D code and possibly 
compile but I don't think it will link I sort of gave up on it 
and decided to try the VTK wrapper generator.

The last bindings I generated are in the repo under 
swigd/bindings/d

Oh and some of the scripts will have a copyright in them, ignore 
it. It is just the automatic header our editors at work stamp in. 
It is all opensource, consider it BSD, I just haven't bothered 
putting any license in the files themselves.

As for the VTK wrapper bindings they are convoluted but I'm 
making some headway. It is spread across C++/CMake files to 
complete the full process. The C++ creates a parser which 
extracts the required details, CMake then feeds the output of 
this into the wrapper generator.

I have it generating half D / half Java at the moment. If I get 
it all D and compiling I'll upload it.

Cheers,
ed
Feb 17 2014
parent reply "ed" <growlercab gmail.com> writes:
On Tuesday, 18 February 2014 at 06:44:19 UTC, ed wrote:
[snip]


 Is your SWIG wrapping online somewhere? - I'd like to take a 
 look.


 https://github.com/lyrebirdsw/vtkdbind

 The SWIG bindings were left in a very bad state so they won't 
 work at the moment. It should generate the D code and possibly 
 compile but I don't think it will link I sort of gave up on it 
 and decided to try the VTK wrapper generator.

 The last bindings I generated are in the repo under 
 swigd/bindings/d

 Oh and some of the scripts will have a copyright in them, 
 ignore it. It is just the automatic header our editors at work 
 stamp in. It is all opensource, consider it BSD, I just haven't 
 bothered putting any license in the files themselves.

 As for the VTK wrapper bindings they are convoluted but I'm 
 making some headway. It is spread across C++/CMake files to 
 complete the full process. The C++ creates a parser which 
 extracts the required details, CMake then feeds the output of 
 this into the wrapper generator.

 I have it generating half D / half Java at the moment. If I get 
 it all D and compiling I'll upload it.

 Cheers,
 ed


Oh let me say this is real hack code I wrote for experimentation 
only :D
Feb 18 2014
parent reply "Abdulhaq" <alynch4047 gmail.com> writes:
On Tuesday, 18 February 2014 at 08:32:07 UTC, ed wrote:

 On Tuesday, 18 February 2014 at 06:44:19 UTC, ed wrote:
 [snip]


 Is your SWIG wrapping online somewhere? - I'd like to take a 
 look.


 https://github.com/lyrebirdsw/vtkdbind

 The SWIG bindings were left in a very bad state so they won't 
 work at the moment. It should generate the D code and possibly 
 compile but I don't think it will link I sort of gave up on it 
 and decided to try the VTK wrapper generator.

 The last bindings I generated are in the repo under 
 swigd/bindings/d

 Oh and some of the scripts will have a copyright in them, 
 ignore it. It is just the automatic header our editors at work 
 stamp in. It is all opensource, consider it BSD, I just 
 haven't bothered putting any license in the files themselves.

 As for the VTK wrapper bindings they are convoluted but I'm 
 making some headway. It is spread across C++/CMake files to 
 complete the full process. The C++ creates a parser which 
 extracts the required details, CMake then feeds the output of 
 this into the wrapper generator.

 I have it generating half D / half Java at the moment. If I 
 get it all D and compiling I'll upload it.

 Cheers,
 ed


 Oh let me say this is real hack code I wrote for 
 experimentation only :D


Thanks for taking the effort to upload it! If you do get the VTK 
wrapper approach going then please do upload that. I'd really 
like to know how the wrapper generator deals with object 
ownership transferal, or is it using some kind of reference 
counting?
Feb 18 2014
parent reply "ed" <growlercab gmail.com> writes:
On Tuesday, 18 February 2014 at 09:15:11 UTC, Abdulhaq wrote:

 On Tuesday, 18 February 2014 at 08:32:07 UTC, ed wrote:

 On Tuesday, 18 February 2014 at 06:44:19 UTC, ed wrote:
 [snip]


 Is your SWIG wrapping online somewhere? - I'd like to take a 
 look.


 https://github.com/lyrebirdsw/vtkdbind

 The SWIG bindings were left in a very bad state so they won't 
 work at the moment. It should generate the D code and 
 possibly compile but I don't think it will link I sort of 
 gave up on it and decided to try the VTK wrapper generator.

 The last bindings I generated are in the repo under 
 swigd/bindings/d

 Oh and some of the scripts will have a copyright in them, 
 ignore it. It is just the automatic header our editors at 
 work stamp in. It is all opensource, consider it BSD, I just 
 haven't bothered putting any license in the files themselves.

 As for the VTK wrapper bindings they are convoluted but I'm 
 making some headway. It is spread across C++/CMake files to 
 complete the full process. The C++ creates a parser which 
 extracts the required details, CMake then feeds the output of 
 this into the wrapper generator.

 I have it generating half D / half Java at the moment. If I 
 get it all D and compiling I'll upload it.

 Cheers,
 ed


 Oh let me say this is real hack code I wrote for 
 experimentation only :D


 Thanks for taking the effort to upload it! If you do get the 
 VTK wrapper approach going then please do upload that. I'd 
 really like to know how the wrapper generator deals with object 
 ownership transferal, or is it using some kind of reference 
 counting?


Well I only know the Java wrapper as that is the one I'm trying 
to adapt. It uses reference counting and a custom garbage 
collector in the Java layer (vtkJavaGarbageCollector).

I'm looking at adapting this over to D. So far all goes well, 
Java -->> D is straight forward. It will get interesting though 
when things statr compiling :)

I'll post here if I get any further...

Cheers,
ed

vtkJava
Feb 18 2014
parent "ed" <growlercab gmail.com> writes:
On Tuesday, 18 February 2014 at 09:28:07 UTC, ed wrote:
[snip]

 Java -->> D is straight forward. It will get interesting though 
 when things statr compiling :)


s/compiling/linking/
Feb 18 2014
prev sibling parent reply Manu <turkeyman gmail.com> writes:
On 17 February 2014 06:05, Abdulhaq <alynch4047 gmail.com> wrote:


 On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:


 The code is tested and just-about one-click build. Currently it's
 in a subversion repository but I plan to move to github. I'd love
 someone to use Smidgen to try wrapping a small C++ library as I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then please
 let me know and I'll get the code out there).

  yes github please.. i'd like to check this out.




 OK I believe that smidgen is now fully hosted on github.

 Using smidgen as it stands I would hope that you could wrap a simple C++
 library. However, no-one else has tried it yet so there are bound to be a
 few teething problems, please feel free to email me (anyone trying it out
 or just taking a look) at alynch4047 at gmail dot com if you have any
 questions / problems. Alternatively perhaps we could use one of the forums
 here??

 I have not yet uploaded the initial Qt5 wrapping (or VTK), I'll try to do
 that over the next day or two.


This is awesome. I have a comprehensive game engine, and I've been looking
for a solution to auto-generate my D bindings, but so far, I haven't found
anything capable of doing a proper job.
I'll definitely be checking this out. I maintain bindings by hand
currently, and I'm often fighting with keeping them in sync.

I have a couple of interesting requirements, I'll be curious to see how it
works out in practise.
Feb 23 2014
parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Monday, 24 February 2014 at 07:40:42 UTC, Manu wrote:

 On 17 February 2014 06:05, Abdulhaq <alynch4047 gmail.com> 
 wrote:


 On Sunday, 16 February 2014 at 02:37:50 UTC, michaelc37 wrote:


 The code is tested and just-about one-click build. Currently 
 it's
 in a subversion repository but I plan to move to github. I'd 
 love
 someone to use Smidgen to try wrapping a small C++ library 
 as I
 think it would make the job quite easy, if anyone reads this
 (since I have not formally announced the thing yet) then 
 please
 let me know and I'll get the code out there).

  yes github please.. i'd like to check this out.




 OK I believe that smidgen is now fully hosted on github.

 Using smidgen as it stands I would hope that you could wrap a 
 simple C++
 library. However, no-one else has tried it yet so there are 
 bound to be a
 few teething problems, please feel free to email me (anyone 
 trying it out
 or just taking a look) at alynch4047 at gmail dot com if you 
 have any
 questions / problems. Alternatively perhaps we could use one 
 of the forums
 here??

 I have not yet uploaded the initial Qt5 wrapping (or VTK), 
 I'll try to do
 that over the next day or two.


 This is awesome. I have a comprehensive game engine, and I've 
 been looking
 for a solution to auto-generate my D bindings, but so far, I 
 haven't found
 anything capable of doing a proper job.
 I'll definitely be checking this out. I maintain bindings by 
 hand
 currently, and I'm often fighting with keeping them in sync.

 I have a couple of interesting requirements, I'll be curious to 
 see how it
 works out in practise.


It would be great if you could give it a run, it's beta software 
so please email me if you run into any problems.
Feb 24 2014
prev sibling parent "Abdulhaq" <alynch4047 gmail.com> writes:
On Saturday, 15 February 2014 at 19:57:40 UTC, Abdulhaq wrote:

 I'm working on a C++ binding generator that is coming on well,
 called Smidgen (first there was SWIG, then sip, now Smidgen...).
 I have a small demo working with both VTK and the beginnings of
 Qt5 GUI. I have to pick it back up (after moving house with my
 family) so would like help on various things such as building it
 on Windows (currently Linux 64bit only but it's only the build
 system that is Linux specific, and it's cmake so should be easy
 to get going on Windows).


I've tidied a few things up and the Qt5 example is on github at

https://github.com/alynch4047/sqt

It's only wrapping about 20 classes or so, from QtCore and 
QtWidget. I could easily include many more but it would slow down 
development of new binding features so I've resisted the 
temptation.

The main thing missing iro Qt is signals and slots - I know how I 
will do it, I just a need a couple of days free time.
Feb 22 2014
prev sibling next sibling parent "Dejan Lekic" <dejan.lekic gmail.com> writes:
C++ binding generator is definitely something we will use. 
However, I would rather have a good, robust tool that can be used 
to generate bindings/wrappers to C libraries. That is of equal 
(or even higher I think) importance as C++ binding generator.
Feb 18 2014
prev sibling parent "w0rp" <devw0rp gmail.com> writes:
https://github.com/w0rp/dqt

I have been working on my generator over the past few weeks, and 
extensively this weekend. It took many hours of work, but I just 
got generation of D source files from SMOKE data to work. Here is 
what it does and roughly how it works.

1. Earlier on, SMOKE is by a third party to generate a dynamic 
library. This library contains data held in a "Smoke" class 
defined in C++, which functions provided for creating, deleting, 
and getting a reference to the pointer for the class with what 
are *supposed* to be functions with C linkage. (I had to wrap 
them anyway with my own functions due to issues.)
2. Because the class members in Smoke are all C primitives, I can 
match the layout with a struct in D and load everything from the 
struct in D code. At the time of generation, I copy everything I 
need out of the struct into a saner representation in D, which I 
have called "SmokeContainer."
3. SmokeContainer is supposed to tell you everything you need to 
know to make generation work. So I use that data and pass it to 
my class which handles generation of D source files, 
"SmokeGenerator." It does all kinds of work to translate C++ 
classes from the SMOKE data to D code, with some ability for 
customisation like ignoring certain types or classes, replacing 
classes with different ones in D or just wrappers for generating 
them, etc.
4. The SmokeGenerator spits out the D source files and requires 
having a "SmokeLoader" defined in prefix D file, which loads the 
same SMOKE data again at runtime for finding the class method 
pointers needed to call C++ functions and so on.
5. You run a D program which uses the generated D source files 
and now you have C++ code being run from D.

While my project is actually called DQt above and the motivation 
for it is to get Qt working in D, I have written it all in such a 
way as to make the Qt specific part of the code and the general 
case SMOKE binding stuff loosely coupled. The only Qt specific 
stuff now lies in some C++ code needed for certain circumstances, 
(like wrapping QString) a few pre-baked D source files used as 
finishing touches for the generation specific to Qt, and the 
current test programs which generate code and run Hello World 
respectively. I shouldn't have too much trouble moving most of 
this stuff into a "dsmoke" repository instead.

If you want to try out running Hello World from the generated 
code yourself, and you're willing to work around my sloppy 
project structure, do this.

1. Build smokegen and smokeqt. I wrote two guides for Linux and 
Windows on this.
    1. Linux: 
https://github.com/w0rp/dqt/blob/master/doc/smoke_linux_build_guide.rst
    2. Windows: 
https://github.com/w0rp/dqt/blob/master/doc/smoke_win32_build_guide.rst
2. Use CMake to build my code. I think 'cmake . && make' will do 
the job on Linux, if you're in my project directory.
3. Put 'libsmoke_cwrapper' in your library path. I make it easy 
on myself by copying it out of the lib/ directory into src/ and 
by running 'export LD_LIBRARY_PATH=.' Not the best way, but it's 
a quick way to get it going.
4. 'rdmd -g -Llibsmoke_cwrapper.so -d -debug -w new_test.d' Will 
run the generator and spit some stuff into src/dqt for now.
5. 'rdmd -Llibsmoke_cwrapper.so -d smoke_test.d' will now run the 
Hello World program, supposing everything else went well.

I am going to continue working on this. There are some things to 
iron out and a lot of interesting work ahead. Hopefully I can get 
a somewhat usable Qt in D library out of this.
Feb 23 2014