www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - #include C headers in D code

reply Atila Neves <atila.neves gmail.com> writes:
Here's my blog post about my project that allows directly 
#including C headers in D*

https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/

The summary is that, modulo bugs, things like this work:

     #include <stdio.h>
     void main() { printf("Hello world\n".ptr); }

So far it's successfully compiled whilst #including pthread, 
libcurl, openssl and others. The blog and the github README have 
more information, and feel free to reply to this with questions.

dub: http://code.dlang.org/packages/dpp
reddit: 
https://www.reddit.com/r/programming/comments/8axj53/include_c_headers_in_d_code/
hacker news: It's in there somewhere, search around.

Atila

* Technically, "D + #include directives + C macros"
Apr 09 2018
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 [snip]


Fantastic.
Apr 09 2018
parent Cym13 <cpicard openmailbox.org> writes:
On Monday, 9 April 2018 at 11:48:51 UTC, jmh530 wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 [snip]


 Fantastic.


This is so cool, I think it should be featured on the D blog as 
well (maybe when you get to your goal of having C++ features 
working consistently).

More exposure is better exposure.
Apr 09 2018
prev sibling next sibling parent SrMordred <patric.dexheimer gmail.com> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/

 The summary is that, modulo bugs, things like this work:

     #include <stdio.h>
     void main() { printf("Hello world\n".ptr); }

 So far it's successfully compiled whilst #including pthread, 
 libcurl, openssl and others. The blog and the github README 
 have more information, and feel free to reply to this with 
 questions.

 dub: http://code.dlang.org/packages/dpp
 reddit: 
 https://www.reddit.com/r/programming/comments/8axj53/include_c_headers_in_d_code/
 hacker news: It's in there somewhere, search around.

 Atila

 * Technically, "D + #include directives + C macros"


This is awesome!
i´ll try soon as possible :)
Apr 09 2018
prev sibling next sibling parent Colin <grogan.colin gmail.com> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/

 The summary is that, modulo bugs, things like this work:

     #include <stdio.h>
     void main() { printf("Hello world\n".ptr); }

 So far it's successfully compiled whilst #including pthread, 
 libcurl, openssl and others. The blog and the github README 
 have more information, and feel free to reply to this with 
 questions.

 dub: http://code.dlang.org/packages/dpp
 reddit: 
 https://www.reddit.com/r/programming/comments/8axj53/include_c_headers_in_d_code/
 hacker news: It's in there somewhere, search around.

 Atila

 * Technically, "D + #include directives + C macros"


This is very cool.

Gonna try it against gl.h tonight!
Apr 09 2018
prev sibling next sibling parent reply kinke <noone nowhere.com> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/


Certainly interesting. Is there a good reason why you didn't look 
into Calypso though? It's restricted to LDC, okay, but its 
approach of fusioning C++ and D parts at the AST and LLVM-IR 
level seems far more promising to me - no need to translate the 
C(++) parts to D and let the underlying D compiler parse & 
analyze all that generated code again. And Calypso's C++ support 
is apparently mature enough to interface with Qt5 (!)...
Apr 09 2018
next sibling parent bachmeier <no spam.net> writes:
On Monday, 9 April 2018 at 18:15:33 UTC, kinke wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/


 Certainly interesting. Is there a good reason why you didn't 
 look into Calypso though? It's restricted to LDC, okay, but its 
 approach of fusioning C++ and D parts at the AST and LLVM-IR 
 level seems far more promising to me - no need to translate the 
 C(++) parts to D and let the underlying D compiler parse & 
 analyze all that generated code again. And Calypso's C++ 
 support is apparently mature enough to interface with Qt5 (!)...


Aren't there limitations on what Calypso can do? I thought I read 
that it can't be made to work with all C code, but maybe I'm 
wrong... Of course, only working with LDC is itself a big 
restriction.
Apr 09 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 9 April 2018 at 18:15:33 UTC, kinke wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/


 Certainly interesting. Is there a good reason why you didn't 
 look into Calypso though? It's restricted to LDC, okay, but its 
 approach of fusioning C++ and D parts at the AST and LLVM-IR 
 level seems far more promising to me - no need to translate the 
 C(++) parts to D and let the underlying D compiler parse & 
 analyze all that generated code again. And Calypso's C++ 
 support is apparently mature enough to interface with Qt5 (!)...


This whole idea came about because a couple of years ago I was in 
a team that was responsible for legacy code written in C which 
had no tests. To write tests, we needed both to write the tests 
themselves and some fakes/mocks/doubles for the code that we 
depended on - essentially a whole operating system. I wasn't 
going to write that in C, since life is too short for that.

I wanted to write it in D, and even tried running dstep on one of 
the dependency headers we had - to no avail. Manual conversion 
was out of the question - too many headers, all them as coupled 
together as possible. So I sighed and started writing some C++14. 
I hated to admit it, but that mess of headers was only really 
usable from C or C++ (Objective C wasn't in the running). I still 
feel bad I picked C++ over D, but it was the right choice.

Why not Calypso? It's a hard enough task to convince a team of C 
programmers who don't know C++ that C++ was a better choice than 
C. It's a herculean task to convince them of D, a language they 
know even less and that they've never heard about (the first 
question is always "but which companies are using D?").

If you add to all that "No, really, it's ok, there's this project 
that forked one of the compilers. No, it's not the reference 
compiler. There's just this bit of non-standard syntax to learn 
that's neither C nor D", then the chances of convincing any 
"normal" engineer are 0.

C++ got popular because it lowered the barrier of adoption to 0. 
I want that for D.  Because of psychological reasons, I don't 
think Calypso is the way to do it, however much it's succeeded 
technically. And the whole point of dpp is convince the average 
coder and their managers.
Apr 09 2018
next sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 9 April 2018 at 19:36:23 UTC, Atila Neves wrote:

 On Monday, 9 April 2018 at 18:15:33 UTC, kinke wrote:

 [...]


 This whole idea came about because a couple of years ago I was 
 in a team that was responsible for legacy code written in C 
 which had no tests. To write tests, we needed both to write the 
 tests themselves and some fakes/mocks/doubles for the code that 
 we depended on - essentially a whole operating system. I wasn't 
 going to write that in C, since life is too short for that.

 [...]


Speaking of dstep - it has greatly improved over the last years.
I'm interested what stop you from using it or improving it 
(instead of rolling your own tool)?
Apr 09 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 9 April 2018 at 20:19:35 UTC, Seb wrote:

 On Monday, 9 April 2018 at 19:36:23 UTC, Atila Neves wrote:

 On Monday, 9 April 2018 at 18:15:33 UTC, kinke wrote:

 [...]


 This whole idea came about because a couple of years ago I was 
 in a team that was responsible for legacy code written in C 
 which had no tests. To write tests, we needed both to write 
 the tests themselves and some fakes/mocks/doubles for the code 
 that we depended on - essentially a whole operating system. I 
 wasn't going to write that in C, since life is too short for 
 that.

 [...]


 Speaking of dstep - it has greatly improved over the last years.
 I'm interested what stop you from using it or improving it 
 (instead of rolling your own tool)?


I tried using dstep as the translation engine to avoid 
reinventing the wheel. I'm lazy, I don't want to write code that 
doesn't neeed to be written!

Unfortunately, dstep makes assumptions that are valid for it's 
trying to do that aren't for what I am, and ultimately made it 
very hard. The refactoring that would be needed would have been 
more work than doing it from scratch, so I sighed, cracked my 
knuckles, and wrote a bunch of code I didn't really want to.

I did copy the part of its test suite that was applicable though.
Apr 10 2018
parent Joakim <dlang joakim.fea.st> writes:
On Tuesday, 10 April 2018 at 09:36:39 UTC, Atila Neves wrote:

 On Monday, 9 April 2018 at 20:19:35 UTC, Seb wrote:

 On Monday, 9 April 2018 at 19:36:23 UTC, Atila Neves wrote:

 On Monday, 9 April 2018 at 18:15:33 UTC, kinke wrote:

 [...]


 This whole idea came about because a couple of years ago I 
 was in a team that was responsible for legacy code written in 
 C which had no tests. To write tests, we needed both to write 
 the tests themselves and some fakes/mocks/doubles for the 
 code that we depended on - essentially a whole operating 
 system. I wasn't going to write that in C, since life is too 
 short for that.

 [...]


 Speaking of dstep - it has greatly improved over the last 
 years.
 I'm interested what stop you from using it or improving it 
 (instead of rolling your own tool)?


 I tried using dstep as the translation engine to avoid 
 reinventing the wheel. I'm lazy, I don't want to write code 
 that doesn't neeed to be written!


Nice work. I prefer the DStep approach of translating the headers 
to D, but obviously it's not going to be as easy for the user as 
this, so this approach has its uses too.

Regarding "reinventing the wheel," I liked this recent blog post 
giving reasons why it can be a viable approach:

https://blog.cerebralab.com/#!/blog/11
Apr 10 2018
prev sibling parent reply Kagamin <spam here.lot> writes:
On Monday, 9 April 2018 at 19:36:23 UTC, Atila Neves wrote:

 If you add to all that "No, really, it's ok, there's this 
 project that forked one of the compilers. No, it's not the 
 reference compiler. There's just this bit of non-standard 
 syntax to learn that's neither C nor D", then the chances of 
 convincing any "normal" engineer are 0.


It is the reference compiler though (which is the frontend), the 
backend is different, but they don't want the dmc backend do 
they? Also recently it started to use pragma and import syntax, 
which are both D.
Apr 11 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, April 11, 2018 09:23:29 Kagamin via Digitalmars-d-announce 
wrote:

 On Monday, 9 April 2018 at 19:36:23 UTC, Atila Neves wrote:

 If you add to all that "No, really, it's ok, there's this
 project that forked one of the compilers. No, it's not the
 reference compiler. There's just this bit of non-standard
 syntax to learn that's neither C nor D", then the chances of
 convincing any "normal" engineer are 0.


 It is the reference compiler though (which is the frontend), the
 backend is different, but they don't want the dmc backend do
 they? Also recently it started to use pragma and import syntax,
 which are both D.


Yes, the frontend is shared, but you don't just use the frontend. You use
the whole compiler. dmd is the reference compiler and what your average
programmer coming to D is going to expect to be using (at least initially).
And telling folks that they have to use a language feature that is not
supported by the reference compiler is not going to go over well with a lot
of people. It would be one thing to tell them that they should use ldc,
because it generates faster code. That doesn't involve forking the language.
Your code would then still work just fine with dmd. It would just be slower.
It's quite another thing to tell them to use a feature that dmd doesn't
support. That _would_ be forking the language, and it would mean writing
programs that would not work with the reference compiler. Many folks are not
going to be happy with the idea of using a fork rather than the real deal.
Some folks will probably be fine with it, but in general, it just plain
looks bad.

It's one thing for someone who is familiar with D to weigh the options and
decide that being tied to ldc is okay. It's quite another to tell someone
who isn't familiar with D that in order to use D, they have to use a feature
which only works with a specific compiler that is not the reference compiler
and which will likely never work with the reference compiler.

- Jonathan M Davis
Apr 11 2018
next sibling parent reply bachmeier <no spam.net> writes:
On Wednesday, 11 April 2018 at 09:45:07 UTC, Jonathan M Davis 
wrote:


 It's one thing for someone who is familiar with D to weigh the 
 options and decide that being tied to ldc is okay. It's quite 
 another to tell someone who isn't familiar with D that in order 
 to use D, they have to use a feature which only works with a 
 specific compiler that is not the reference compiler and which 
 will likely never work with the reference compiler.


It also wouldn't work with GDC. Given that GDC has been added to 
GCC, it would be a bad idea to tell people they need to use LDC 
to work with C code.
Apr 11 2018
next sibling parent reply drug <drug2004 bk.ru> writes:
11.04.2018 15:22, bachmeier пишет:

 On Wednesday, 11 April 2018 at 09:45:07 UTC, Jonathan M Davis wrote:
 ... Given that GDC has been added to GCC...


Is it true? I don't see anything like that here 
https://gcc.gnu.org/gcc-8/changes.html
Apr 11 2018
parent reply Uknown <sireeshkodali1 gmail.com> writes:
On Wednesday, 11 April 2018 at 13:17:23 UTC, drug wrote:

 11.04.2018 15:22, bachmeier пишет:

 On Wednesday, 11 April 2018 at 09:45:07 UTC, Jonathan M Davis 
 wrote:
 ... Given that GDC has been added to GCC...


 Is it true? I don't see anything like that here 
 https://gcc.gnu.org/gcc-8/changes.html


Here's relevant news from Phoronix:

https://www.phoronix.com/scan.php?page=news_item&px=D-Frontend-For-GCC

Here's the relevant announcement:
https://gcc.gnu.org/ml/gcc/2017-06/msg00111.html
Apr 11 2018
parent reply drug <drug2004 bk.ru> writes:
11.04.2018 16:26, Uknown пишет:

 On Wednesday, 11 April 2018 at 13:17:23 UTC, drug wrote:

 11.04.2018 15:22, bachmeier пишет:

 On Wednesday, 11 April 2018 at 09:45:07 UTC, Jonathan M Davis wrote:
 ... Given that GDC has been added to GCC...


 Is it true? I don't see anything like that here 
 https://gcc.gnu.org/gcc-8/changes.html


 
 Here's relevant news from Phoronix:
 
 https://www.phoronix.com/scan.php?page=news_item&px=D-Frontend-For-GCC
 
 Here's the relevant announcement:
 https://gcc.gnu.org/ml/gcc/2017-06/msg00111.html


I've read it. Unfortunately it doesn't answer my question. I've heard 
there were some problems.
Apr 11 2018
next sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
AFAIK, GDC does not make it, so hopefully it will be merge with gcc 9

On Wed, Apr 11, 2018 at 3:44 PM, drug via Digitalmars-d-announce <
digitalmars-d-announce puremagic.com> wrote:


 11.04.2018 16:26, Uknown =D0=BF=D0=B8=D1=88=D0=B5=D1=82:

 On Wednesday, 11 April 2018 at 13:17:23 UTC, drug wrote:

 11.04.2018 15:22, bachmeier =D0=BF=D0=B8=D1=88=D0=B5=D1=82:


 On Wednesday, 11 April 2018 at 09:45:07 UTC, Jonathan M Davis wrote:
 ... Given that GDC has been added to GCC...


 Is it true? I don't see anything like that here
 https://gcc.gnu.org/gcc-8/changes.html


 Here's relevant news from Phoronix:

 https://www.phoronix.com/scan.php?page=3Dnews_item&px=3DD-Frontend-For-G=




CC

 Here's the relevant announcement:
 https://gcc.gnu.org/ml/gcc/2017-06/msg00111.html


 I've read it. Unfortunately it doesn't answer my question. I've heard
 there were some problems.
Apr 11 2018
prev sibling parent Johannes Pfau <nospam example.com> writes:
Am Wed, 11 Apr 2018 16:44:32 +0300 schrieb drug:


 11.04.2018 16:26, Uknown пишет:

 On Wednesday, 11 April 2018 at 13:17:23 UTC, drug wrote:

 11.04.2018 15:22, bachmeier пишет:

 On Wednesday, 11 April 2018 at 09:45:07 UTC, Jonathan M Davis wrote:
 ... Given that GDC has been added to GCC...


 Is it true? I don't see anything like that here
 https://gcc.gnu.org/gcc-8/changes.html


 
 Here's relevant news from Phoronix:
 
 https://www.phoronix.com/scan.php?page=news_item&px=D-Frontend-For-GCC
 
 Here's the relevant announcement:
 https://gcc.gnu.org/ml/gcc/2017-06/msg00111.html


 I've read it. Unfortunately it doesn't answer my question. I've heard
 there were some problems.


IIRC copyright stuff once again stalled further discussion and the 
relevant GCC guys are not responding:
https://www.mail-archive.com/gcc-patches gcc.gnu.org/msg186124.html


-- 
Johannes
Apr 12 2018
prev sibling parent Kagamin <spam here.lot> writes:
On Wednesday, 11 April 2018 at 12:22:56 UTC, bachmeier wrote:

 It also wouldn't work with GDC. Given that GDC has been added 
 to GCC, it would be a bad idea to tell people they need to use 
 LDC to work with C code.


Currently porting a project to linux, stuff is so severely 
outdated, pretty much no dependency was satisfied  by the distro, 
in wich case for D it would be better to install ldc just to not 
touch gdc :)
Granted, it's only a problem for active projects, but what 
projects are inactive?
Apr 11 2018
prev sibling parent Kagamin <spam here.lot> writes:
On Wednesday, 11 April 2018 at 09:45:07 UTC, Jonathan M Davis 
wrote:

 It's one thing for someone who is familiar with D to weigh the 
 options and decide that being tied to ldc is okay. It's quite 
 another to tell someone who isn't familiar with D that in order 
 to use D, they have to use a feature which only works with a 
 specific compiler that is not the reference compiler and which 
 will likely never work with the reference compiler.


I'd say, you can focus on negative aspects if the auditory will 
analyze them, but it's a bad strategy if you only want to 
overcome prejudice.
Apr 11 2018
prev sibling next sibling parent Stefan <dl ng.rocks> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*


neat!
Apr 09 2018
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/9/2018 4:03 AM, Atila Neves wrote:

 Here's my blog post about my project that allows directly #including C headers 
 in D*


Very nice work, and great article!
Apr 10 2018
parent Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 10 April 2018 at 08:45:06 UTC, Walter Bright wrote:

 On 4/9/2018 4:03 AM, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*


 Very nice work, and great article!


Thanks!
Apr 10 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/9/18 7:03 AM, Atila Neves wrote:

 Here's my blog post about my project that allows directly #including C 
 headers in D*
 
 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/ 
 
 
 The summary is that, modulo bugs, things like this work:
 
      #include <stdio.h>
      void main() { printf("Hello world\n".ptr); }
 
 So far it's successfully compiled whilst #including pthread, libcurl, 
 openssl and others. The blog and the github README have more 
 information, and feel free to reply to this with questions.
 
 dub: http://code.dlang.org/packages/dpp
 reddit: 
 https://www.reddit.com/r/programming/comments/8axj53/include_
_headers_in_d_code/ 
 
 hacker news: It's in there somewhere, search around.
 
 Atila
 
 * Technically, "D + #include directives + C macros"


Awesome. Can't say I will use it, as I don't use C much, but I 
understand how difficult a task this is.

If you get to the point where you can #include <vector>, it will be 
doubly impressive!

-Steve
Apr 10 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 10 April 2018 at 13:53:34 UTC, Steven Schveighoffer 
wrote:

 On 4/9/18 7:03 AM, Atila Neves wrote:

 [...]


 Awesome. Can't say I will use it, as I don't use C much, but I 
 understand how difficult a task this is.


Thanks!


 If you get to the point where you can #include <vector>, it 
 will be doubly impressive!


Not *if*, *when*. ;)

Atila
Apr 10 2018
next sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/10/18 12:51 PM, Atila Neves wrote:

 On Tuesday, 10 April 2018 at 13:53:34 UTC, Steven Schveighoffer wrote:



 If you get to the point where you can #include <vector>, it will be 
 doubly impressive!


 
 Not *if*, *when*. ;)


I hope you are right, but I remain skeptical :)

If I may throw a curveball back at you, I recall a question at last 
year's dconf during the talk about interfacing with C++ STL about using 
emplace_back instead of push_back. And the questioner who happened to be 
sitting next to me said "meh, push_back isn't very useful, nobody uses 
push_back any more" :P

-Steve
Apr 10 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 10 April 2018 at 18:01:37 UTC, Steven Schveighoffer 
wrote:

 On 4/10/18 12:51 PM, Atila Neves wrote:

 On Tuesday, 10 April 2018 at 13:53:34 UTC, Steven 
 Schveighoffer wrote:



 If you get to the point where you can #include <vector>, it 
 will be doubly impressive!


 
 Not *if*, *when*. ;)


 I hope you are right, but I remain skeptical :)

 If I may throw a curveball back at you, I recall a question at 
 last year's dconf during the talk about interfacing with C++ 
 STL about using emplace_back instead of push_back. And the 
 questioner who happened to be sitting next to me said "meh, 
 push_back isn't very useful, nobody uses push_back any more" :P

 -Steve


Haha, I remember. I do plan on dealing with emplace_back, but I 
have no idea how just yet and I was hoping nobody was going to 
call me on it until then. Busted! :P

Atila
Apr 10 2018
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/10/18 2:36 PM, Atila Neves wrote:


 
 Haha, I remember. I do plan on dealing with emplace_back, but I have no 
 idea how just yet and I was hoping nobody was going to call me on it 
 until then. Busted! :P
 


I think we all agree you aren't going to instantiate C++ templates in D 
(and who would want to).

But since you are using a wrapper to call the compiler, and invoking 
clang anyway, generating a file to compile which calls the template to 
generate the object file for the method, and then using the symbol 
generated to bind to the method might be possible. Of course, you first 
have to compile your D code to see what templates to generate! So this 
is another extra step.

It could be as easy as generating a mock C++ template for vector, that 
then has a symbol that d++ recognizes and both replaces with a real call 
AND helps you figure out what C++ template to generate. -vcg-ast may 
help here.

Whatever this is going to be, it ain't going to be fast...

-Steve
Apr 10 2018
parent Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 10 April 2018 at 19:28:09 UTC, Steven Schveighoffer 
wrote:

 On 4/10/18 2:36 PM, Atila Neves wrote:


 
 Haha, I remember. I do plan on dealing with emplace_back, but 
 I have no idea how just yet and I was hoping nobody was going 
 to call me on it until then. Busted! :P
 


 I think we all agree you aren't going to instantiate C++ 
 templates in D (and who would want to).


Do we? Do I? :P

(I've actually seriously considered it)


 But since you are using a wrapper to call the compiler, and 
 invoking clang anyway, generating a file to compile which calls 
 the template to generate the object file for the method, and 
 then using the symbol generated to bind to the method might be 
 possible. Of course, you first have to compile your D code to 
 see what templates to generate! So this is another extra step.


That's one of my ideas.


 It could be as easy as generating a mock C++ template for 
 vector, that then has a symbol that d++ recognizes and both 
 replaces with a real call AND helps you figure out what C++ 
 template to generate. -vcg-ast may help here.


The details elude me right now, but that seems to make sense.


 Whatever this is going to be, it ain't going to be fast...


Compiling C++ isn't known to be fast. ;)

It might be possible to limit the damage by confining the C++ 
#includes to one D package or module so it doesn't get rebuilt 
every time.

Atila
Apr 10 2018
prev sibling parent reply Seb <seb wilzba.ch> writes:
On Tuesday, 10 April 2018 at 16:51:57 UTC, Atila Neves wrote:

 If you get to the point where you can #include <vector>, it 
 will be doubly impressive!


 Not *if*, *when*. ;)

 Atila


FYI people have been fighting with this for a long time:

https://github.com/dlang/druntime/pull/1316
Apr 10 2018
next sibling parent David Nadlinger <code klickverbot.at> writes:
On Tuesday, 10 April 2018 at 20:32:05 UTC, Seb wrote:

 On Tuesday, 10 April 2018 at 16:51:57 UTC, Atila Neves wrote:

 If you get to the point where you can #include <vector>, it 
 will be doubly impressive!


 Not *if*, *when*. ;)

 Atila


 FYI people have been fighting with this for a long time:

 https://github.com/dlang/druntime/pull/1316


FYI this appears to work just fine in Calypso: 
https://github.com/Syniurge/Calypso/blob/master/tests/calypso/libstdc%2B%2B/vector.d

  — David
Apr 10 2018
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 10 April 2018 at 20:32:05 UTC, Seb wrote:

 On Tuesday, 10 April 2018 at 16:51:57 UTC, Atila Neves wrote:

 If you get to the point where you can #include <vector>, it 
 will be doubly impressive!


 Not *if*, *when*. ;)

 Atila


 FYI people have been fighting with this for a long time:

 https://github.com/dlang/druntime/pull/1316


The beauty of using libclang is that name mangling issues don't 
exist. :)

It's still not going to be easy, but that's one worry I won't 
have to bother with.

Atila
Apr 10 2018
parent reply Jacob Carlborg <doob me.com> writes:
On Tuesday, 10 April 2018 at 23:44:46 UTC, Atila Neves wrote:


 The beauty of using libclang is that name mangling issues don't 
 exist. :)


How is that not going to be an issue? Are you adding 
`pragma(mangle)` everywhere?

--
/Jacob Carlborg
Apr 10 2018
parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 11 April 2018 at 06:21:47 UTC, Jacob Carlborg wrote:

 On Tuesday, 10 April 2018 at 23:44:46 UTC, Atila Neves wrote:


 The beauty of using libclang is that name mangling issues 
 don't exist. :)


 How is that not going to be an issue? Are you adding 
 `pragma(mangle)` everywhere?


Yes.

I don't know how to deal with class templates yet though, since 
none of their member functions have mangled symbols until they're 
instantiated. Which obviously doesn't happen at the declaration.
Apr 11 2018
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 09/04/2018 11:03 PM, Atila Neves wrote:

 Here's my blog post about my project that allows directly #including C 
 headers in D*
 
 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/ 
 
 
 The summary is that, modulo bugs, things like this work:
 
      #include <stdio.h>
      void main() { printf("Hello world\n".ptr); }
 
 So far it's successfully compiled whilst #including pthread, libcurl, 
 openssl and others. The blog and the github README have more 
 information, and feel free to reply to this with questions.
 
 dub: http://code.dlang.org/packages/dpp
 reddit: 
 https://www.reddit.com/r/programming/comments/8axj53/include_
_headers_in_d_code/ 
 
 hacker news: It's in there somewhere, search around.
 
 Atila
 
 * Technically, "D + #include directives + C macros"


Any chance objectice-c as well?
Apr 10 2018
parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 11 April 2018 at 06:12:49 UTC, rikki cattermole 
wrote:

 On 09/04/2018 11:03 PM, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*
 
 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/
 
 
 The summary is that, modulo bugs, things like this work:
 
      #include <stdio.h>
      void main() { printf("Hello world\n".ptr); }
 
 So far it's successfully compiled whilst #including pthread, 
 libcurl, openssl and others. The blog and the github README 
 have more information, and feel free to reply to this with 
 questions.
 
 dub: http://code.dlang.org/packages/dpp
 reddit: 
 https://www.reddit.com/r/programming/comments/8axj53/include_c_headers_in_d_code/
 
 hacker news: It's in there somewhere, search around.
 
 Atila
 
 * Technically, "D + #include directives + C macros"


 Any chance objectice-c as well?


I don't know Objective-C. The only way I can think of to even get 
close to getting it to work is by copying the relevant tests from 
dstep. I also don't think the language is nearly as important as 
C and C++ in terms of fostering D adoption.

So probably no unless someone sends PRs my way.
Apr 11 2018
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/


How do you deal with macros containing invalid D code, i.e. 
`#define foo(T) sizeof(T)`?

--
/Jacob Carlborg
Apr 10 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 11 April 2018 at 06:24:38 UTC, Jacob Carlborg wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/


 How do you deal with macros containing invalid D code, i.e. 
 `#define foo(T) sizeof(T)`?


https://github.com/atilaneves/dpp/issues/22
https://github.com/atilaneves/dpp/blob/60f98e4fee2fac0117ac430216ef9c5c25c511fe/tests/issues.d#L229
https://github.com/atilaneves/dpp/blob/60f98e4fee2fac0117ac430216ef9c5c25c511fe/source/dpp/cursor/macro_.d#L55

I did the best I could having seen some macros. It's likely there 
are cases I've missed, or that maybe the translation in the link 
above doesn't work even for what it's supposed to be doing (I 
have no confidence about catching all the C casts for instance).

If there are other cases, I'll fix them as they're encountered. 
It's possible some of them can't be fixed and the user will have 
to work around them. Right now I have a feeling it will probably 
be ok. Time will tell (assuming I have users!).
Apr 11 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/11/2018 3:25 AM, Atila Neves wrote:

 I did the best I could having seen some macros. It's likely there are cases
I've 
 missed, or that maybe the translation in the link above doesn't work even for 
 what it's supposed to be doing (I have no confidence about catching all the C 
 casts for instance).
 
 If there are other cases, I'll fix them as they're encountered. It's possible 
 some of them can't be fixed and the user will have to work around them. Right 
 now I have a feeling it will probably be ok. Time will tell (assuming I have 
 users!).



That's right. There is no general solution. One can only look for common 
patterns and do those. For example,

   #define X 15

is a common pattern and can be reliably rewritten as:

   enum X = 15;
Apr 11 2018
parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 11 April 2018 at 18:36:56 UTC, Walter Bright wrote:

 On 4/11/2018 3:25 AM, Atila Neves wrote:

 I did the best I could having seen some macros. It's likely 
 there are cases I've missed, or that maybe the translation in 
 the link above doesn't work even for what it's supposed to be 
 doing (I have no confidence about catching all the C casts for 
 instance).
 
 If there are other cases, I'll fix them as they're 
 encountered. It's possible some of them can't be fixed and the 
 user will have to work around them. Right now I have a feeling 
 it will probably be ok. Time will tell (assuming I have 
 users!).



 That's right. There is no general solution. One can only look 
 for common patterns and do those. For example,

   #define X 15

 is a common pattern and can be reliably rewritten as:

   enum X = 15;


If I understand it correctly, dpp doesn't do that.

Instead, it runs the pre-processor on the source code, just like 
in C, so

// test.dpp
#define X 15
int foo() { return X; }

becomes

// test.d
int foo() { return 15; }

The upside of this approach: all macros just work, unless they 
use C (not C pre-processor, C proper) features that dpp can't 
handle. `sizeof(...)` is a special case that is handled in 
dpp.cursor.macro.translateToD and more could be added.

The downside: macros can't be directly used outside .dpp files.
Apr 12 2018
parent Nemanja Boric <4burgos gmail.com> writes:
On Thursday, 12 April 2018 at 11:43:51 UTC, John Colvin wrote:

 On Wednesday, 11 April 2018 at 18:36:56 UTC, Walter Bright 
 wrote:

 On 4/11/2018 3:25 AM, Atila Neves wrote:

 [...]



 That's right. There is no general solution. One can only look 
 for common patterns and do those. For example,

   #define X 15

 is a common pattern and can be reliably rewritten as:

   enum X = 15;


 If I understand it correctly, dpp doesn't do that.

 Instead, it runs the pre-processor on the source code, just 
 like in C, so

 // test.dpp
 #define X 15
 int foo() { return X; }

 becomes

 // test.d
 int foo() { return 15; }

 The upside of this approach: all macros just work, unless they 
 use C (not C pre-processor, C proper) features that dpp can't 
 handle. `sizeof(...)` is a special case that is handled in 
 dpp.cursor.macro.translateToD and more could be added.

 The downside: macros can't be directly used outside .dpp files.


Yes, I assumed it actually "expands" the macros, whereas it 
actually runs
the preprocessor on dpp files: 
https://github.com/atilaneves/dpp/issues/30

I can see it perfectly matches the Atila's usecase, and I'm 
curious
what are the other usecases for dpp?
Apr 12 2018
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*


BTW, you can steal the config script [1] from DStep to help 
detect locations of LLVM/libclang. It also supports static 
linking. Supports manually specifying the path to LLVM if needed.

[1] 
https://github.com/jacob-carlborg/dstep/blob/master/configure.d

--
/Jacob Carlborg
Apr 11 2018
parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 11 April 2018 at 14:26:04 UTC, Jacob Carlborg wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*


 BTW, you can steal the config script [1] from DStep to help 
 detect locations of LLVM/libclang. It also supports static 
 linking. Supports manually specifying the path to LLVM if 
 needed.

 [1] 
 https://github.com/jacob-carlborg/dstep/blob/master/configure.d

 --
 /Jacob Carlborg


Thanks!
Apr 13 2018
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*


I don't know the exact details of your project but can't you just:

1. Copy the includes
2. Paste them into a C file
3. Run DStep on the C file
4. Replace the includes in the first file with the result from 
DStep

This would require changing DStep to always return `false` here 
[1]. Or perhaps run the preprocessor to expand the includes and 
then run DStep.

[1] 
https://github.com/jacob-carlborg/dstep/blob/master/dstep/translator/Translator.d#L326

--
/Jacob Carlborg
Apr 11 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 11 April 2018 at 14:33:26 UTC, Jacob Carlborg wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*


 I don't know the exact details of your project but can't you 
 just:

 1. Copy the includes
 2. Paste them into a C file
 3. Run DStep on the C file
 4. Replace the includes in the first file with the result from 
 DStep

 This would require changing DStep to always return `false` here 
 [1]. Or perhaps run the preprocessor to expand the includes and 
 then run DStep.

 [1] 
 https://github.com/jacob-carlborg/dstep/blob/master/dstep/translator/Translator.d#L326

 --
 /Jacob Carlborg


That wouldn't have the same semantics as I want.

I tried using dstep as a library. It didn't work.
Apr 13 2018
parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Friday, 13 April 2018 at 10:31:43 UTC, Atila Neves wrote:

 On Wednesday, 11 April 2018 at 14:33:26 UTC, Jacob Carlborg 
 wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*


 I don't know the exact details of your project but can't you 
 just:

 1. Copy the includes
 2. Paste them into a C file
 3. Run DStep on the C file
 4. Replace the includes in the first file with the result from 
 DStep

 This would require changing DStep to always return `false` 
 here [1]. Or perhaps run the preprocessor to expand the 
 includes and then run DStep.

 [1] 
 https://github.com/jacob-carlborg/dstep/blob/master/dstep/translator/Translator.d#L326

 --
 /Jacob Carlborg


 That wouldn't have the same semantics as I want.

 I tried using dstep as a library. It didn't work.


You also mentioned this in the reddit thread, though I'm still 
curious to understand what's difference in the semantics between 
the approach you have taken and the approach Jacob proposed.
Apr 13 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 14 April 2018 at 04:07:12 UTC, Petar Kirov 
[ZombineDev] wrote:

 On Friday, 13 April 2018 at 10:31:43 UTC, Atila Neves wrote:

 On Wednesday, 11 April 2018 at 14:33:26 UTC, Jacob Carlborg 
 wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*


 I don't know the exact details of your project but can't you 
 just:

 1. Copy the includes
 2. Paste them into a C file
 3. Run DStep on the C file
 4. Replace the includes in the first file with the result 
 from DStep

 This would require changing DStep to always return `false` 
 here [1]. Or perhaps run the preprocessor to expand the 
 includes and then run DStep.

 [1] 
 https://github.com/jacob-carlborg/dstep/blob/master/dstep/translator/Translator.d#L326

 --
 /Jacob Carlborg


 That wouldn't have the same semantics as I want.

 I tried using dstep as a library. It didn't work.


 You also mentioned this in the reddit thread, though I'm still 
 curious to understand what's difference in the semantics 
 between the approach you have taken and the approach Jacob 
 proposed.


You can use the C macros in the headers that you #include in your 
dpp file.

dstep has a lot of code for translating macros. I don't want to 
translate macros at all, but it's deeply intertwined with 
translating everything else. This bug just can't happen with the 
dpp approach:

https://github.com/jacob-carlborg/dstep/issues/166

In the idea above it assumes that all #include directives are 
together at the top of the file. They probably are, but they 
might not be for some reason. I can't remember the specifics, but 
dstep by default ignores declarations from other headers because 
the idea is to translate this one particular header. I don't want 
that either. I also don't want D code generated that has an 
`import` to a D file that's actually a translation of another C 
header.

I want it to work like it does in C++. dstep doesn't set out to 
do that, which is fine, but contorting it to make it do what I 
wanted it to was more trouble than it was worth.

Believe me, if I can avoid writing code, I do. Writing 
translations from scratch was not a decision I made lightly. In 
the end it turned out to be a lot less work than I thought though 
(just over 500 SLOC).


Atila
Apr 16 2018
parent reply Jacob Carlborg <doob me.com> writes:
On Monday, 16 April 2018 at 11:20:51 UTC, Atila Neves wrote:


 You can use the C macros in the headers that you #include in 
 your dpp file.

 dstep has a lot of code for translating macros. I don't want to 
 translate macros at all, but it's deeply intertwined with 
 translating everything else.


There's a command line switch to disable that, 
`--translate-macros=false`. Or, alternatively, run the 
preprocessor first.


 I can't remember the specifics, but dstep by default ignores 
 declarations from other headers because the idea is to 
 translate this one particular header.


That's a simple change: replace these lines [1] with `return 
false;`. If that's something you need we can make it configurable.

[1] 
https://github.com/jacob-carlborg/dstep/blob/97870ac5167f09e8acf17f8754c32636492b237f/dstep/translator/Translator.d#L326-L329

--
/Jacob Carlborg
Apr 16 2018
parent Atila Neves <atila.neves gmail.com> writes:
On Monday, 16 April 2018 at 12:26:12 UTC, Jacob Carlborg wrote:

 On Monday, 16 April 2018 at 11:20:51 UTC, Atila Neves wrote:


 You can use the C macros in the headers that you #include in 
 your dpp file.

 dstep has a lot of code for translating macros. I don't want 
 to translate macros at all, but it's deeply intertwined with 
 translating everything else.


 There's a command line switch to disable that, 
 `--translate-macros=false`. Or, alternatively, run the 
 preprocessor first.


I missed that.


 I can't remember the specifics, but dstep by default ignores 
 declarations from other headers because the idea is to 
 translate this one particular header.


 That's a simple change: replace these lines [1] with `return 
 false;`. If that's something you need we can make it 
 configurable.

 [1] 
 https://github.com/jacob-carlborg/dstep/blob/97870ac5167f09e8acf17f8754c32636492b237f/dstep/translator/Translator.d#L326-L329


No need to make it configureable now - I did the translation work 
already and made a point of copying in dstep's tests to guarantee 
the same level of support. Given how small it ended up being, I'd 
trade ~500SLOC over a dependency nearly every time.

I also like the end result better (unsurprising since I wrote 
it), even though I didn't quite achieve the level of testability 
I wanted.
Apr 16 2018
prev sibling next sibling parent reply rumbu <rumbu rumbu.ro> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/


Cannot manage to build it on Windows:

D:\git\dpp>dub build
WARNING: A deprecated branch based version specification is used 
for the dependency libclang. Please use numbered versions 
instead. Also note that you can still use the dub.selections.json 
file to override a certain dependency to use a branch instead.
Performing "debug" build using dmd for x86.
libclang ~master: target for configuration "library" is up to 
date.
dpp 0.0.1+commit.41.g60f98e4: building configuration 
"executable"...
Linking...
OPTLINK (R) for Win32  Release 8.00.17
Copyright (C) Digital Mars 1989-2013  All rights reserved.
http://www.digitalmars.com/ctg/optlink.html
OPTLINK : Warning 183: Extension not .RES : clang.lib
C:\Users\[]\AppData\Roaming\dub\packages\libclang-master\libclang\.dub\build\library-debug-windows-x86-dmd_2079-78261F5A299D700FEEC2C0E7B511
1C1\libclang.lib(1) : Error 52: .DEF Syntax Error


D:\git\dpp>dub build --arch=x86_64
WARNING: A deprecated branch based version specification is used 
for the dependency libclang. Please use numbered versions 
instead. Also note that you can still use the dub.selections.json 
file to override a certain dependency to use a branch instead.
Performing "debug" build using dmd for x86_64.
libclang ~master: target for configuration "library" is up to 
date.
dpp 0.0.1+commit.41.g60f98e4: building configuration 
"executable"...
Linking...
LINK : fatal error LNK1104: cannot open file 'clang.lib'
Error: linker exited with status 1104
dmd failed with exit code 1.
Apr 11 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 11 April 2018 at 14:57:51 UTC, rumbu wrote:

 On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

[...]


 Cannot manage to build it on Windows:

 D:\git\dpp>dub build
 WARNING: A deprecated branch based version specification is 
 used for the dependency libclang. Please use numbered versions 
 instead. Also note that you can still use the 
 dub.selections.json file to override a certain dependency to 
 use a branch instead.
 Performing "debug" build using dmd for x86.
 libclang ~master: target for configuration "library" is up to 
 date.
 dpp 0.0.1+commit.41.g60f98e4: building configuration 
 "executable"...
 Linking...
 OPTLINK (R) for Win32  Release 8.00.17
 Copyright (C) Digital Mars 1989-2013  All rights reserved.
 http://www.digitalmars.com/ctg/optlink.html
 OPTLINK : Warning 183: Extension not .RES : clang.lib
 C:\Users\[]\AppData\Roaming\dub\packages\libclang-master\libclang\.dub\build\library-debug-windows-x86-dmd_2079-78261F5A299D700FEEC2C0E7B511
1C1\libclang.lib(1) : Error 52: .DEF Syntax Error

 [...]


I'll have to take a look at Jacob's configure.d to find out where 
libclang is installed on Windows.

It'll fail even if it builds though, since there won't be a 
`cpp.exe` on Windows for it to call. I don't even know what the 
equivalent would be with Visual Studio.
Apr 13 2018
parent Jacob Carlborg <doob me.com> writes:
On 2018-04-13 12:33, Atila Neves wrote:


 I'll have to take a look at Jacob's configure.d to find out where 
 libclang is installed on Windows.


Unfortunately the configuration script is only for Posix.

-- 
/Jacob Carlborg
Apr 13 2018
prev sibling parent Chris <wendlec tcd.ie> writes:
On Monday, 9 April 2018 at 11:03:48 UTC, Atila Neves wrote:

 Here's my blog post about my project that allows directly 
 #including C headers in D*

 https://atilanevesoncode.wordpress.com/2018/04/09/include-c-headers-in-d-code/

 The summary is that, modulo bugs, things like this work:

     #include <stdio.h>
     void main() { printf("Hello world\n".ptr); }

 So far it's successfully compiled whilst #including pthread, 
 libcurl, openssl and others. The blog and the github README 
 have more information, and feel free to reply to this with 
 questions.

 dub: http://code.dlang.org/packages/dpp
 reddit: 
 https://www.reddit.com/r/programming/comments/8axj53/include_c_headers_in_d_code/
 hacker news: It's in there somewhere, search around.

 Atila

 * Technically, "D + #include directives + C macros"


This is really cool. This can be _very_ useful in situations 
where you don't want to translate all the headers to D, 
especially when experimenting with C libs. Really cool. Thanks.
Apr 16 2018