www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - More automated interfacing of D with C codebases

reply "CapitalLetterC" <zarathustra.zoroaster gmail.com> writes:
This will be my first posting here, but I've been obsessed with D 
since before there was a D2 standard. Despite that rather long 
period of obsession, it's only just now that I've started 
seriously using D as I attempt to code-up some POC projects to 
demonstrate that the language is mature enough for serious 
development, or at least the kind of development we're doing, 
anyway.

So, that out of the way, I suppose I should apologize in advance 
for what must amount to a pretty dumb question, since I truly 
couldn't have been the first to think of this: with regard to 
interfacing D with C libraries, why would we require a project 
like Deimos which ensures that each codebase is ported, almost 
entirely, by hand? Theoretically, shouldn't there be some kind of 
autotools-like method to establish something like, config.d.in 
which is then used via abstractions to properly configure the 
rest of the D wrapper? Obviously, D doesn't have a preprocessor, 
but couldn't one synthesize that with a series of judicious 
version() blocks? Then, whenever someone goes to build the 
application on their system, you'd have something to a series of 
autoconf-like tests to fill in the proper sections of of 
config.d.in and get out a config.d suitable for their system.

I'm really sorry if there's some stupidly obvious reason this 
couldn't work and I'm just not seeing it, or if this is some n00b 
question you get all the time.
Oct 15 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-15 13:51, CapitalLetterC wrote:

 This will be my first posting here, but I've been obsessed with D since
 before there was a D2 standard. Despite that rather long period of
 obsession, it's only just now that I've started seriously using D as I
 attempt to code-up some POC projects to demonstrate that the language is
 mature enough for serious development, or at least the kind of
 development we're doing, anyway.

 So, that out of the way, I suppose I should apologize in advance for
 what must amount to a pretty dumb question, since I truly couldn't have
 been the first to think of this: with regard to interfacing D with C
 libraries, why would we require a project like Deimos which ensures that
 each codebase is ported, almost entirely, by hand? Theoretically,
 shouldn't there be some kind of autotools-like method to establish
 something like, config.d.in which is then used via abstractions to
 properly configure the rest of the D wrapper? Obviously, D doesn't have
 a preprocessor, but couldn't one synthesize that with a series of
 judicious version() blocks? Then, whenever someone goes to build the
 application on their system, you'd have something to a series of
 autoconf-like tests to fill in the proper sections of of config.d.in and
 get out a config.d suitable for their system.

 I'm really sorry if there's some stupidly obvious reason this couldn't
 work and I'm just not seeing it, or if this is some n00b question you
 get all the time.


The header files need to be ported so D can find the declarations of the 
functions and types. There are a couple of tools that can do most of the 
work, but not completely. Macros are specially hard to port automatically.

https://github.com/jacob-carlborg/dstep

-- 
/Jacob Carlborg
Oct 15 2012
parent reply "CapitalLetterC" <zarathustra.zoroaster gmail.com> writes:
On Monday, 15 October 2012 at 12:54:51 UTC, Jacob Carlborg wrote:

 On 2012-10-15 13:51, CapitalLetterC wrote:

 This will be my first posting here, but I've been obsessed 
 with D since
 before there was a D2 standard. Despite that rather long 
 period of
 obsession, it's only just now that I've started seriously 
 using D as I
 attempt to code-up some POC projects to demonstrate that the 
 language is
 mature enough for serious development, or at least the kind of
 development we're doing, anyway.

 So, that out of the way, I suppose I should apologize in 
 advance for
 what must amount to a pretty dumb question, since I truly 
 couldn't have
 been the first to think of this: with regard to interfacing D 
 with C
 libraries, why would we require a project like Deimos which 
 ensures that
 each codebase is ported, almost entirely, by hand? 
 Theoretically,
 shouldn't there be some kind of autotools-like method to 
 establish
 something like, config.d.in which is then used via 
 abstractions to
 properly configure the rest of the D wrapper? Obviously, D 
 doesn't have
 a preprocessor, but couldn't one synthesize that with a series 
 of
 judicious version() blocks? Then, whenever someone goes to 
 build the
 application on their system, you'd have something to a series 
 of
 autoconf-like tests to fill in the proper sections of of 
 config.d.in and
 get out a config.d suitable for their system.

 I'm really sorry if there's some stupidly obvious reason this 
 couldn't
 work and I'm just not seeing it, or if this is some n00b 
 question you
 get all the time.


 The header files need to be ported so D can find the 
 declarations of the functions and types. There are a couple of 
 tools that can do most of the work, but not completely. Macros 
 are specially hard to port automatically.

 https://github.com/jacob-carlborg/dstep


Actually, I understand perfectly well why the headers need to be 
ported to D code, what I was more wondering is if anybody knew of 
or had ever tried to use some kind of build system to automate 
the process to the greatest extent possible. So, for example, as 
a result of the configuration process certain features may be 
implicitly or explicitly enabled/disabled, without probing the 
individual user's configuration, you wouldn't know that, so how 
does one maximize portability, in that case? If you simply go to 
support the maximal set of features, you may wind up with a bunch 
of codepaths that go nowhere, likewise, if you choose only to 
support the minimal set of functionality, you may not wind up 
with code that's not useful to anyone, yourself included. 
Granted, I hadn't thought of the difficulty in porting things 
like macros, but I'd imagine that, in the absolute worst case, 
you could synthesize their behavior with templates.

But there are all sorts of issues in that same vein, like how 
certain structs get put together, what types are aliased with 
what, etc. That's why I was wondering if you could handle this 
analogously to how C does, with a config file that gets setup by 
a kind of configuration system.
Oct 15 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-15 19:20, CapitalLetterC wrote:


 But there are all sorts of issues in that same vein, like how certain
 structs get put together, what types are aliased with what, etc. That's
 why I was wondering if you could handle this analogously to how C does,
 with a config file that gets setup by a kind of configuration system.


I think that would be possible but I haven't seen anyone program in D 
like that.

-- 
/Jacob Carlborg
Oct 15 2012
parent reply "CapitalLetterC" <zarathustra.zoroaster gmail.com> writes:
On Monday, 15 October 2012 at 20:09:03 UTC, Jacob Carlborg wrote:

 I think that would be possible but I haven't seen anyone 
 program in D like that.


One of the primary reasons I was concerned that there might be 
some serious flaw I'm overlooking. Oh well, that's a good enough 
excuse for me to begin experimentation. Thanks! :D
Oct 15 2012
parent reply "Brad Lanam" <brad.lanam.comp -NOSPAM-gmail.com> writes:
I don't think this is exactly what you want either, but it's 
certainly a lot easier than a manual process:

http://gentoo.com/di/mkconfig.html
(On Freecode) https://freecode.com/projects/mkconfig

I just now released a new version.  Freecode will be updated 
within 24 hours.
Check the examples/dtcltk/ directory.

There's also a simpler example within the D/ subdirectory in:
   http://gentoo.com/di/

Macros have to have their signature specified. e.g.:

     cmacro Tcl_InitMemory tcl.h void 'Tcl_Interp *'
     cmacro ScreenOfDisplay X11/Xlib.h 'Screen *' 'Display' int

  -- Brad
Oct 21 2012
parent reply "timotheecour" <thelastmammoth gmail.com> writes:
Manually porting of C/C++ libraries shouldn't be the way to go (a
major problem being when said library gets updated, human errors,
porting to different architectures etc).

I've had good experience using SWIG to automate the job for me,
and used it successfully to port the following libraries to D:

* ncurses
* opencv
* sfml
to name a few.

I believe this is the way forward, as opposed to maintaining
(necessary stale) ports such as Deimos.

Swig isn't perfect however, and sometimes will bail out on
constructs such as:
C++ classes with multiple inheritance, or a few obscure C or C++
constructs. Even in that case, I find it much easier to tweak the
swig interface file to get desired output rather than undertake
the huge manual conversion task. For sfml, the interface file is
just about 40 lines for example.

Improving such kind of automated tools is where we should be
concentrating efforts, not on maintaining deimos. One such
improvement I made was to allow conversion of C++ templates to D
templates (works in large majority of cases), which makes the
library much easier to use than using name mangling schemes. More
improvements would be welcome (eg support for static library
generation as opposed to dynamic load libraries), reducing the
need for tweaking swig interface files etc.
Oct 21 2012
next sibling parent "Brad Lanam" <brad.lanam.comp -NOSPAM-gmail.com> writes:
On Sunday, 21 October 2012 at 19:45:53 UTC, timotheecour wrote:

 I've had good experience using SWIG to automate the job for me,
 and used it successfully to port the following libraries to D:


Yes.  SWIG should be a lot faster than my tool.
Also I don't have any units to handle C++ code.

Completely agree. It has to be automated.


 Swig isn't perfect however, and sometimes will bail out on
 constructs such as:
 C++ classes with multiple inheritance, or a few obscure C or C++
 constructs. Even in that case, I find it much easier to tweak


If I recall, I ran into some obscure C constructs in the Tcl/Tk/X 
headers that I have never seen before.  I believe I had to 
various aliases to my "interface file" to handle those.  Also use 
of D reserved words had to be handled:

   substitute new new_
   substitute class class_
   substitute 'function;' 'function_;'

And some things just plain won't work in D:


   substitute '_XPrivate .private9, .private10;' \
             '_XPrivate * private9, private10;'
Oct 21 2012
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-21 21:45, timotheecour wrote:

 Manually porting of C/C++ libraries shouldn't be the way to go (a
 major problem being when said library gets updated, human errors,
 porting to different architectures etc).


SWIG requires some kind of interface files, which I assume one must have 
to manually write. Then what's the difference?


 I've had good experience using SWIG to automate the job for me,
 and used it successfully to port the following libraries to D:

 * ncurses
 * opencv
 * sfml
 to name a few.

 I believe this is the way forward, as opposed to maintaining
 (necessary stale) ports such as Deimos.

 Swig isn't perfect however, and sometimes will bail out on
 constructs such as:
 C++ classes with multiple inheritance, or a few obscure C or C++
 constructs. Even in that case, I find it much easier to tweak the
 swig interface file to get desired output rather than undertake
 the huge manual conversion task. For sfml, the interface file is
 just about 40 lines for example.

 Improving such kind of automated tools is where we should be
 concentrating efforts, not on maintaining deimos. One such
 improvement I made was to allow conversion of C++ templates to D
 templates (works in large majority of cases), which makes the
 library much easier to use than using name mangling schemes. More
 improvements would be welcome (eg support for static library
 generation as opposed to dynamic load libraries), reducing the
 need for tweaking swig interface files etc.


I don't think SWIG is the right approach because:

* You have to create these interface files
* It seem to not handle all code
* No support for Objective-C

BTW, how does SWIG handle macros and documentation?

-- 
/Jacob Carlborg
Oct 21 2012
next sibling parent reply "timotheecour" <thelastmammoth gmail.com> writes:
 SWIG requires some kind of interface files, which I assume one 
 must have to manually write. Then what's the difference?


These can be as small as:


sfml.i:
%{
#include <SFML/Audio.hpp>
%}
%include SFML/Config.hpp
%include SFML/Audio/Export.hpp

In practice, some additional tweaks may be needed to support 
certain constructs like multiple inheritance, etc, but nothing 
daunting (~ 50 lines of code for my sfml interface file)


 I don't think SWIG is the right approach because:
 * You have to create these interface files


=>
see above. Interface files can be very small.


 * It seem to not handle all code


I was still able to port large libraries like sfml and opencv, 
using tweaks to the interface files and enhancing swig to support 
C++template to D template conversion. There's a small number of 
cases where swig currently chokes, some of which could be easily 
fixed if there's enough demand.


 * No support for Objective-C


=>
http://www.swig.org/Doc1.1/HTML/SWIG.html#n7
One of SWIG's most recent additions is support for Objective-C 
parsing. This is currently an experimental feature that may be 
improved in future releases.


 BTW, how does SWIG handle macros and documentation?


You'd have to write a C/C++ function that does what the macro 
does in the interface file, which would in turn be converted to a 
D function.

For exporting C documentation to D, I've asked on stackoverflow: 
http://stackoverflow.com/questions/12413957/export-comments-in-swig-wrapper
I'm not sure it's supported ATM, but that's definitely something 
possible since swig has a full blown C++ parser.
Oct 22 2012
next sibling parent "timotheecour" <thelastmammoth gmail.com> writes:
correction regarding macros:

myheader.h:
#define MEX_INFORMATION_VERSION 1

interface file myheader.i:
%{
%include "myheader.h"
%}
%include myheader.h

generated d file will contain:

int MEX_INFORMATION_VERSION()  property {
   auto ret = swig.mex_im.swig_mex_MEX_INFORMATION_VERSION_get();
   return ret;
}

which allows one to use the macro from D. Certainly not the best 
way (an enum would be preferable) but it works. Again, swig could 
be improved. Also if static libraries could be generated, the 
overhead above wouldn't be such a big deal with link time 
optimization.

For functional macros, I'm not sure, but once can write a C 
function that calls the macro, and then swig will convert to D 
function. Ideally, swig would directly convert it to a templated 
function.


I think maintaining a repository of interface files would be a 
better idea than maintaining manually converted C files. The 
former would be much smaller & easier to maintain, and for 99% 
cases stay in sync with library updates. The latter would end up 
as a graveyard of never up to date partially-portable C ports 
that no-one will want to maintain.
Oct 22 2012
prev sibling next sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-10-22 19:27, timotheecour wrote:

 SWIG requires some kind of interface files, which I assume one must
 have to manually write. Then what's the difference?


 These can be as small as:


 sfml.i:
 %{
 #include <SFML/Audio.hpp>
 %}
 %include SFML/Config.hpp
 %include SFML/Audio/Export.hpp

 In practice, some additional tweaks may be needed to support certain
 constructs like multiple inheritance, etc, but nothing daunting (~ 50
 lines of code for my sfml interface file)


 I don't think SWIG is the right approach because:
 * You have to create these interface files


 =>
 see above. Interface files can be very small.


 * It seem to not handle all code


 I was still able to port large libraries like sfml and opencv, using
 tweaks to the interface files and enhancing swig to support C++template
 to D template conversion. There's a small number of cases where swig
 currently chokes, some of which could be easily fixed if there's enough
 demand.


 * No support for Objective-C


 =>
 http://www.swig.org/Doc1.1/HTML/SWIG.html#n7
 One of SWIG's most recent additions is support for Objective-C parsing.
 This is currently an experimental feature that may be improved in future
 releases.


 BTW, how does SWIG handle macros and documentation?


 You'd have to write a C/C++ function that does what the macro does in
 the interface file, which would in turn be converted to a D function.

 For exporting C documentation to D, I've asked on stackoverflow:
 http://stackoverflow.com/questions/12413957/export-comments-in-swig-wrapper
 I'm not sure it's supported ATM, but that's definitely something
 possible since swig has a full blown C++ parser.


Ok, I see.

If SWIG has a full blown C++ parser, then why is it choking on some 
code? Or does it need a semantic analyzer as well.

-- 
/Jacob Carlborg
Oct 22 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-10-22 19:27, timotheecour wrote:


 http://www.swig.org/Doc1.1/HTML/SWIG.html#n7
 One of SWIG's most recent additions is support for Objective-C parsing.
 This is currently an experimental feature that may be improved in future
 releases.


Will this generate C code or can it be made to generate something like this:

http://michelf.ca/projects/d-objc/syntax/

-- 
/Jacob Carlborg
Oct 22 2012
prev sibling parent reply "Brad Lanam" <brad.lanam.comp -NOSPAM-gmail.com> writes:
On Monday, 22 October 2012 at 06:28:12 UTC, Jacob Carlborg wrote:

 On 2012-10-21 21:45, timotheecour wrote:

 Manually porting of C/C++ libraries shouldn't be the way to go 
 (a
 major problem being when said library gets updated, human 
 errors,
 porting to different architectures etc).


 SWIG requires some kind of interface files, which I assume one 
 must have to manually write. Then what's the difference?


Huge difference.  It's not necessary to figure out corresponding 
type sizes on every system.  I only need to figure out what needs 
to be in there.  And once all the work of creating the interface 
file is done, the work of porting to another architecture or 
operating system or compiler is much reduced.

It's not necessary to figure out which api calls are macros on 
arch XYZ or are simply not there:

cdcl Tcl_IncrRefCount
clib Tcl_IncrRefCount

for the macro.
if lib_Tcl_IncrRefCount ! _clib_Tcl_IncrRefCount
  cmacro Tcl_IncrRefCount tcl.h void 'Tcl_Obj *'
endif

My example Tck/Tk conversion doesn't handle 8.6, but the effort 
to add it will be very minimal.  All of the hard work of 
discovering the needed typedefs has been done.


 [...]
 I don't think SWIG is the right approach because:

 * You have to create these interface files
 * It seem to not handle all code


I don't know SWIG, but the idea is the same for my tool.

If you try to automate processing of the pre-processed headers, 
all of the included system headers will be processed and too much 
that isn't needed will be output. The included system definitions 
will conflict with D definitions.

I'm not sure that handling *all* code is necessary.  How often do 
you see
this sort of thing in C code?
     _XPrivate .private9, .private10;
Rather than spend effort on handling *every* possible obscure 
C/C++/ObjC construct, it's easier to simply handle those in the 
interface file.

Other constructs that are more common can be added to SWIG or to 
my tool.
They're both open source.

   -- Brad
Oct 22 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-22 20:38, Brad Lanam wrote:


 Huge difference.  It's not necessary to figure out corresponding type
 sizes on every system.  I only need to figure out what needs to be in
 there.  And once all the work of creating the interface file is done,
 the work of porting to another architecture or operating system or
 compiler is much reduced.

 It's not necessary to figure out which api calls are macros on arch XYZ
 or are simply not there:

 cdcl Tcl_IncrRefCount
 clib Tcl_IncrRefCount

 macro.
 if lib_Tcl_IncrRefCount ! _clib_Tcl_IncrRefCount
   cmacro Tcl_IncrRefCount tcl.h void 'Tcl_Obj *'
 endif

 My example Tck/Tk conversion doesn't handle 8.6, but the effort to add
 it will be very minimal.  All of the hard work of discovering the needed
 typedefs has been done.


 [...]
 I don't think SWIG is the right approach because:

 * You have to create these interface files
 * It seem to not handle all code


 I don't know SWIG, but the idea is the same for my tool.

 If you try to automate processing of the pre-processed headers, all of
 the included system headers will be processed and too much that isn't
 needed will be output. The included system definitions will conflict
 with D definitions.

 I'm not sure that handling *all* code is necessary.  How often do you see
 this sort of thing in C code?
      _XPrivate .private9, .private10;
 Rather than spend effort on handling *every* possible obscure C/C++/ObjC
 construct, it's easier to simply handle those in the interface file.

 Other constructs that are more common can be added to SWIG or to my tool.
 They're both open source.


I prefer the approach I took when I created DStep, it uses libclang. The 
tool can parse all C(++) and Objective-C(++) code that clang can. 
Although it currently won't generate code for everything, at least it 
doesn't choke any code.

I also get all bug fixes and new languages features for free when 
they're added to clang.

No need to create interface files.

-- 
/Jacob Carlborg
Oct 22 2012
next sibling parent reply "Brad Lanam" <brad.lanam.comp -NOSPAM-gmail.com> writes:
On Monday, 22 October 2012 at 19:12:46 UTC, Jacob Carlborg wrote:

 On 2012-10-22 20:38, Brad Lanam wrote:
 I prefer the approach I took when I created DStep, it uses 
 libclang. The tool can parse all C(++) and Objective-C(++) code 
 that clang can. Although it currently won't generate code for 
 everything, at least it doesn't choke any code.

 I also get all bug fixes and new languages features for free 
 when they're added to clang.

 No need to create interface files.


Nothing wrong with that.
I had different goals -- portability and legacy systems.  My tool 
is all bourne shell and awk.  There aren't many systems it does 
not work on (e.g. ULTRIX shell runs out of memory).

I should also state my tool was written as a build configuration 
tool (like autoconf/iffe/dist), not a conversion tool.  SWIG is 
better supported, faster and supports more languages.

If you need to write code that runs on multiple systems and works 
with low level system calls (rpc, et.al.), my tool might be a 
better choice.

This sort of thing (from the D compiler):

#if __sun
#include        <alloca.h>
#endif

Is nuts.  I thought everyone had gotten away from that sort of 
thing (it died out in the mid-eighties!), but I still see it in 
code everywhere.  The system should be tested for its 
capabilities and what it has available, not whether it is a sun 
machine.

   -- Brad
Oct 22 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-22 21:48, Brad Lanam wrote:


 Nothing wrong with that.
 I had different goals -- portability and legacy systems.  My tool is all
 bourne shell and awk.  There aren't many systems it does not work on
 (e.g. ULTRIX shell runs out of memory).


How can you mention bourne shell and portability in the same sentence? I 
doesn't work on Windows (yes I know about cygwin and mingw). Clang does 
work on Windows, I just haven't been able to compile DStep for Windows 
yet due to optlink not cooperating.

Sure, if you're satisfied with Posix then I guess that's fine. But it's 
not really easy to build cross-platform code with shell script.


 I should also state my tool was written as a build configuration tool
 (like autoconf/iffe/dist), not a conversion tool.  SWIG is better
 supported, faster and supports more languages.

 If you need to write code that runs on multiple systems and works with
 low level system calls (rpc, et.al.), my tool might be a better choice.


Why would that make a difference.


 This sort of thing (from the D compiler):

 #if __sun
 #include        <alloca.h>
 #endif

 Is nuts.  I thought everyone had gotten away from that sort of thing (it
 died out in the mid-eighties!), but I still see it in code everywhere.
 The system should be tested for its capabilities and what it has
 available, not whether it is a sun machine.


DMD (DMC) is from the eighties. It sill uses the .c for its C++ files.

-- 
/Jacob Carlborg
Oct 22 2012
parent reply "Brad Lanam" <brad.lanam.comp -NOSPAM-gmail.com> writes:
On Tuesday, 23 October 2012 at 06:35:06 UTC, Jacob Carlborg wrote:

 On 2012-10-22 21:48, Brad Lanam wrote:
 How can you mention bourne shell and portability in the same 
 sentence? I doesn't work on Windows (yes I know about cygwin 
 and mingw). Clang does work on Windows, I just haven't been 
 able to compile DStep for Windows yet due to optlink not 
 cooperating.

 Sure, if you're satisfied with Posix then I guess that's fine. 
 But it's not really easy to build cross-platform code with 
 shell script.


I'm sorry, I don't see how that follows.  My scripts work on All 
variants of Linux (2.4, 2.6), Solaris (2.6 - 11), AIX, Tru64, 
HP-UX, All *BSD, Mac OSX, Haiku, QNX, SCO, Syllable, UnixWare, 
Windows Cygwin.  My 'di' program builds on all of the above with 
a simple 'make' as it uses my build tool.  And I don't need to 
install any special tools.  These are just the systems I have 
access to.

How is that not cross-platform?  How is that not portable?  How 
is that not easy?

If I had a D compiler on all those platforms, the D version of 
the 'di' program would build everywhere with just a 'make'.


 If you need to write code that runs on multiple systems and 
 works with
 low level system calls (rpc, et.al.), my tool might be a 
 better choice.


 Why would that make a difference.


Not all systems are alike.

       static if (_c_args_setmntent == 1)
       {
         f = setmntent (toStringz(DI_MOUNT_FILE));
       } else static if (_c_args_setmntent == 2) {
         f = setmntent (toStringz(DI_MOUNT_FILE), toStringz("r"));
       }

If I understand your tool correctly, you might convert the 
mount.h or mntent.h file [but how does your program decide which 
one and even if they exist?].  But then how are you going to 
change your code to handle the above?  You've got a *lot* of work 
to do if you want a cross platform tool.

I recommend SWIG+CMAKE.  Or my 'mkconfig' tool.
Oct 23 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-10-23 19:42, Brad Lanam wrote:


 I'm sorry, I don't see how that follows.  My scripts work on All
 variants of Linux (2.4, 2.6), Solaris (2.6 - 11), AIX, Tru64, HP-UX, All
 *BSD, Mac OSX, Haiku, QNX, SCO, Syllable, UnixWare, Windows Cygwin.  My
 'di' program builds on all of the above with a simple 'make' as it uses
 my build tool.  And I don't need to install any special tools.  These
 are just the systems I have access to.


All of these don't use the same shell. Requiring to install bash would 
mean you do need to install special tools. Not really special but 
additional tools. But I would count Cygwin as a special tool for Windows.


 How is that not cross-platform?  How is that not portable?  How is that
 not easy?

 If I had a D compiler on all those platforms, the D version of the 'di'
 program would build everywhere with just a 'make'.


 If you need to write code that runs on multiple systems and works with
 low level system calls (rpc, et.al.), my tool might be a better choice.


 Why would that make a difference.


 Not all systems are alike.

        static if (_c_args_setmntent == 1)
        {
          f = setmntent (toStringz(DI_MOUNT_FILE));
        } else static if (_c_args_setmntent == 2) {
          f = setmntent (toStringz(DI_MOUNT_FILE), toStringz("r"));
        }

 If I understand your tool correctly, you might convert the mount.h or
 mntent.h file [but how does your program decide which one and even if
 they exist?].  But then how are you going to change your code to handle
 the above?  You've got a *lot* of work to do if you want a cross
 platform tool.

 I recommend SWIG+CMAKE.  Or my 'mkconfig' tool.


Well then, how does your tool work?

-- 
/Jacob Carlborg
Oct 23 2012
parent reply "Brad Lanam" <brad.lanam.comp -NOSPAM-gmail.com> writes:
On Tuesday, 23 October 2012 at 19:10:29 UTC, Jacob Carlborg wrote:

 On 2012-10-23 19:42, Brad Lanam wrote:
 All of these don't use the same shell. Requiring to install 
 bash would mean you do need to install special tools. Not 
 really special but additional tools. But I would count Cygwin 
 as a special tool for Windows.


Oh, maybe were you thinking that bash is the bourne shell?  It's 
the
bourne-again shell, a rewrite of the bourne shell.  Solaris sh is 
probably the closest to the original bourne shell.

My tool runs with any bourne shell compatible shell.  It works
with bash2, bash3, bash4, ksh, ksh88, ksh93, Solaris sh, Tru64 sh,
ash, dash, mksh, pdksh. HP-UX sh.


 If you need to write code that runs on multiple systems and 
 works with
 low level system calls (rpc, et.al.), my tool might be a 
 better choice.


 Why would that make a difference.


 Not all systems are alike.

       static if (_c_args_setmntent == 1)
       {
         f = setmntent (toStringz(DI_MOUNT_FILE));
       } else static if (_c_args_setmntent == 2) {
         f = setmntent (toStringz(DI_MOUNT_FILE), 
 toStringz("r"));
       }

 If I understand your tool correctly, you might convert the 
 mount.h or
 mntent.h file [but how does your program decide which one and 
 even if
 they exist?].  But then how are you going to change your code 
 to handle
 the above?  You've got a *lot* of work to do if you want a 
 cross
 platform tool.

 I recommend SWIG+CMAKE.  Or my 'mkconfig' tool.


 Well then, how does your tool work?


You put directives in the "interface file" (to use the SWIG term) 
to test which capabilities the system has.  It creates an output 
file (.sh, .c, .d). Depending on those capabilities, the code can 
use #if or static if statements to provide code that works for 
the system you are building on.

In the case of the C/D language interface, the directives will 
extract typedefs, structures, C function declarations, macros, 
#defines.
Oct 23 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-10-23 21:43, Brad Lanam wrote:


 Oh, maybe were you thinking that bash is the bourne shell?  It's the
 bourne-again shell, a rewrite of the bourne shell.  Solaris sh is
 probably the closest to the original bourne shell.


I thought I said bash somewhere, but perhaps you didn't.


 My tool runs with any bourne shell compatible shell.  It works
 with bash2, bash3, bash4, ksh, ksh88, ksh93, Solaris sh, Tru64 sh,
 ash, dash, mksh, pdksh. HP-UX sh.


I see.


 You put directives in the "interface file" (to use the SWIG term) to
 test which capabilities the system has.  It creates an output file (.sh,
 .c, .d). Depending on those capabilities, the code can use #if or static
 if statements to provide code that works for the system you are building
 on.

 In the case of the C/D language interface, the directives will extract
 typedefs, structures, C function declarations, macros, #defines.


Ok, I see.

-- 
/Jacob Carlborg
Oct 23 2012
prev sibling parent reply "Brad Lanam" <brad.lanam.comp -NOSPAM-gmail.com> writes:
On Monday, 22 October 2012 at 19:12:46 UTC, Jacob Carlborg wrote:

 On 2012-10-22 20:38, Brad Lanam wrote:
 I prefer the approach I took when I created DStep, it uses 
 libclang. The tool can parse all C(++) and Objective-C(++) code 
 that clang can. Although it currently won't generate code for 
 everything, at least it doesn't choke any code.

 I also get all bug fixes and new languages features for free 
 when they're added to clang.

 No need to create interface files.


Nothing wrong with that.
I had different goals -- portability and legacy systems.  My tool 
is all bourne shell and awk.  There aren't many systems it does 
not work on (e.g. ULTRIX shell runs out of memory).

I should also state my tool was written as a build configuration 
tool (like autoconf/iffe/dist), not a conversion tool.  SWIG is 
better supported, faster and supports more languages.

If you need to write code that runs on multiple systems and works 
with low level system calls (rpc, et.al.), my tool might be a 
better choice.

This sort of thing (from the D compiler):

#if __sun
#include        <alloca.h>
#endif

Is nuts.  I thought everyone had gotten away from that sort of 
thing (it died out in the mid-eighties!), but I still see it in 
code everywhere.  The system should be tested for its 
capabilities and what it has available, not whether it is a sun 
machine.

   -- Brad
Oct 22 2012
parent reply "timotheecour" <thelastmammoth gmail.com> writes:
If you try to automate processing of the pre-processed headers,
 all of the included system headers will be processed and too 
 much that isn't needed will be output.


Swig has lots of customization to control what gets translated. 
For example, we recursively translate the contents of header 
files or just the contents minus without looking into the include 
statements (the default):

b.h:
void funb();
a.h:
#include b.h
void funa();

%{
%include "a.h"
%}
%include a.h

=> the default is NOT to inlude funb(), which is inside an 
include statement.


 If SWIG has a full blown C++ parser, then why is it choking on 
 some




code?
One example is when there's no D equivalent, eg multiple 
inheritance. But there are more:
actually in the example above, if I understand correctly, the 
stuff inside the %{ %} uses the full blown C++ parser, whereas 
the stuff after has some swig specificities and sometimes can't 
handle certain rare constructs if one uses the lazy approach that 
I've taken of translating the whole header file (%include a.h):

some stuff it choked on:
A) typedef Matx<_Tp, MIN(m, n), 1> diag_type;
B) void addParam(   int (Algorithm::*getter)()=0)



 Will this generate C code or can it be made to generate 
 something like this:
 http://michelf.ca/projects/d-objc/syntax/


I haven't tried, I'd be curious to know.
Oct 22 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-10-23 07:09, timotheecour wrote:

 If you try to automate processing of the pre-processed headers,
 all of the included system headers will be processed and too much that
 isn't needed will be output.


 Swig has lots of customization to control what gets translated. For
 example, we recursively translate the contents of header files or just
 the contents minus without looking into the include statements (the
 default):

 b.h:
 void funb();
 a.h:
 #include b.h
 void funa();

 %{
 %include "a.h"
 %}
 %include a.h

 => the default is NOT to inlude funb(), which is inside an include
 statement.


I'm only translating what's in the "main" header file, i.e. the one you 
gave on the command line. I have access to source information of where a 
given symbol is declared, if it doesn't match the main header file it's 
not translated.


 If SWIG has a full blown C++ parser, then why is it choking on some




 code?
 One example is when there's no D equivalent, eg multiple inheritance.
 But there are more:
 actually in the example above, if I understand correctly, the stuff
 inside the %{ %} uses the full blown C++ parser, whereas the stuff after
 has some swig specificities and sometimes can't handle certain rare
 constructs if one uses the lazy approach that I've taken of translating
 the whole header file (%include a.h):

 some stuff it choked on:
 A) typedef Matx<_Tp, MIN(m, n), 1> diag_type;
 B) void addParam(   int (Algorithm::*getter)()=0)


If there's something I can't translate to D I just skip it. I could 
output a warning, some kind of static assert in the code or whatever. 
The tool can still handle the file.

-- 
/Jacob Carlborg
Oct 22 2012