www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DMD 2.111.0 - Now with more ImportC!

reply Adam Wilson <flyboynw gmail.com> writes:
During BeerConf this month I was finally able to deliver on a 
promise I made back at DConf 2023. The new ODBC 4.0 ImportC 
generated bindings have finally landed in Phobos!

Why is this such a big deal? Because for the first time D will 
ship bindings for a major library that are generated using 
ImportC. This represents a major milestone on our long-term 
mission of being able to use ImportC to import bindings for all 
of our major Operating System targets. There will be more bugs to 
crush along the way, but ImportC has finally achieved 
production-grade quality usability.

The final result is that out of nearly 2000 symbols, only two had 
to be hand-ported. One was a complex enum with multiple bitwise 
OR's. The other is a multi-line string enum. It is unlikely that 
either of these will be supported by ImportC in the near future.

The process to reach this milestone involved crushing a number of 
bugs. And there are still a couple of bugs that need to be fixed 
to reach optimal conversion, one of which was an arcane lexer bug 
that Walter has already put out a PR for.

Specifically they are:
https://github.com/dlang/dmd/pull/20914 (Lexer skips to EoF when 
reading a 0.)
https://github.com/dlang/dmd/issues/20911 (Forward Declared 
Structs are emitted with the struct and a "forward declaration" 
that conflicts.)

There remains an outstanding issue with C union types not being 
able to be emitted into DI files, but this one is rather arcane 
and likely to require a complex solution, if possible at all.

These bindings are generated using ImportC in conjunction with a 
small tool I developed (here: 
https://github.com/LightBender/odbc-d). Initially, I wrote the 
tool to work around bugs in ImportC, but as we've closed out 
those bugs that need has reduced.

However, one use of the tool that does not appear likely to 
change is removing unwanted symbols. For example, in the 
canonical ODBC headers from Microsoft, there are symbols for 
optional Windows-specific and Visual Studio-specific capabilities 
that are unavailable across platforms. I was able to use the tool 
to remove these and it creates a much cleaner binding. 
Additionally, because ImportC modules were never intended to be 
written to files, the DI emitter for ImportC bindings emits a 
bunch of garbage that the importer generates. Using this tool I 
was able to circumvent those symbols.

I would stress that the tool approach will not be useful for all 
cases as there are C header constructs that are not expressible 
in D. But in practice this appears to be a highly infrequent 
situation.

If you have any questions about this process and the future of 
ImportC bindings (which will play a critical role in the future 
of DRT and Phobos) please ask away!

In the mean time, my congratulations to Walter and the rest of 
the D compiler hackers who made the ImportC dream a practical 
reality!
Feb 25
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 25 February 2025 at 09:04:49 UTC, Adam Wilson wrote:
 During BeerConf this month I was finally able to deliver on a 
 promise I made back at DConf 2023. The new ODBC 4.0 ImportC 
 generated bindings have finally landed in Phobos!

 [snip]
Cool. Might make for a good blog post.
Feb 25
prev sibling next sibling parent Mike Shah <mshah.475 gmail.com> writes:
On Tuesday, 25 February 2025 at 09:04:49 UTC, Adam Wilson wrote:
 During BeerConf this month I was finally able to deliver on a 
 promise I made back at DConf 2023. The new ODBC 4.0 ImportC 
 generated bindings have finally landed in Phobos!

 [...]
Awesome work Adam and team!
Feb 25
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Tuesday, 25 February 2025 at 09:04:49 UTC, Adam Wilson wrote:
 These bindings are generated using ImportC in conjunction with 
 a small tool I developed (here: 
 https://github.com/LightBender/odbc-d). Initially, I wrote the 
 tool to work around bugs in ImportC, but as we've closed out 
 those bugs that need has reduced.
Using ImportC _implies_ writing a "small tool" to make the output somewhat useful. It will always be like that unfortunately...
Feb 26
next sibling parent Adam Wilson <flyboynw gmail.com> writes:
On Wednesday, 26 February 2025 at 12:17:17 UTC, Dejan Lekic wrote:
 Using ImportC _implies_ writing a "small tool" to make the 
 output somewhat useful. It will always be like that 
 unfortunately...
Not necessarily. If all you need is a quick import directly into the code, then ImportC works just fine. Sure the result has a bit of mess in it, but it's not like you have to see the mess, it's just there in the background. But if you do want something cleaner, then you'd be writing a tool in any case, but with ImportC the tool ends up being far simpler because ImportC does all of the lexing, parsing, and AST transforms. All we have to do is trim the inputs down to what we actually want, you'd be doing that with other tools as well. And while the tool is used to work around bugs, this serves as a way to isolate and reduce bugs to a minimal test case, which makes it easier to fix the bug, then once the bug is fixed you simply delete the fix from the tool. ImportC gives us options. If we need to quickly import something directly into the code ImportC supports that, even if what it generates is a little messy. But if you want to distribute bindings with your library, you write a simple tool that does the cleanups and runs ImportC. Either case has it's pro's and con's but both are supported. Nothing else can do that, which makes ImportC a superior tool.
Feb 26
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
There will never be perfect translation from C to D, as semantics do differ and 
there are endless nutburger C extensions.

But ImportC shows that even with being imperfect, it is a tremendous
improvement 
over manual conversion.
Feb 27
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Thank you, Adam!
Feb 27