www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ImportC now supports function-like macros

reply Walter Bright <newshound2 digitalmars.com> writes:
#importc now successfully converts function-like C macros to #dlang function 
templates:

```C
#define ADD(a, b) a + b
```

is converted to a D function template:

```D
auto ADD(T1, T2)(T1 a, T2 b) { return a + b; }
```

which opens up a lot more automatic support for C macro use.

https://github.com/dlang/dmd/pull/16199

Thanks to  ibuclaw,  dkorpel,  thewilsonator,  Herringway,  ntrel,  tim-dlang 
for helping me get it over the finish line.
Feb 19
next sibling parent Alex Bryan <abryancs gmail.com> writes:
On Tuesday, 20 February 2024 at 02:07:45 UTC, Walter Bright wrote:

 #importc now successfully converts function-like C macros to 
 #dlang function templates:

 ```C
 #define ADD(a, b) a + b
 ```

 is converted to a D function template:

 ```D
 auto ADD(T1, T2)(T1 a, T2 b) { return a + b; }
 ```

 which opens up a lot more automatic support for C macro use.

 https://github.com/dlang/dmd/pull/16199

 Thanks to  ibuclaw,  dkorpel,  thewilsonator,  Herringway, 
  ntrel,  tim-dlang for helping me get it over the finish line.


Yes!!!! Thanks to all for all of the hard work
Feb 19
prev sibling next sibling parent reply Bradley Chatha <sealabjaster gmail.com> writes:
On Tuesday, 20 February 2024 at 02:07:45 UTC, Walter Bright wrote:

 Thanks to  ibuclaw,  dkorpel,  thewilsonator,  Herringway, 
  ntrel,  tim-dlang for helping me get it over the finish line.


Great work (TM).
Feb 20
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/20/2024 2:13 PM, Bradley Chatha wrote:

 Great work (TM).


Thank you. I've known how to make it work for a while, and finally got around
to 
getting it done. It's very satisfying to see it work.
Feb 20
prev sibling next sibling parent Imperatorn <johan_forsberg_86 hotmail.com> writes:
On Tuesday, 20 February 2024 at 02:07:45 UTC, Walter Bright wrote:

 #importc now successfully converts function-like C macros to 
 #dlang function templates:

 ```C
 #define ADD(a, b) a + b
 ```

 is converted to a D function template:

 ```D
 auto ADD(T1, T2)(T1 a, T2 b) { return a + b; }
 ```

 which opens up a lot more automatic support for C macro use.

 https://github.com/dlang/dmd/pull/16199

 Thanks to  ibuclaw,  dkorpel,  thewilsonator,  Herringway, 
  ntrel,  tim-dlang for helping me get it over the finish line.


Thanks 😇
Feb 21
prev sibling next sibling parent Carl Sturtivant <sturtivant gmail.com> writes:
On Tuesday, 20 February 2024 at 02:07:45 UTC, Walter Bright wrote:

 #importc now successfully converts function-like C macros to 
 #dlang function templates:

 ```C
 #define ADD(a, b) a + b
 ```

 is converted to a D function template:

 ```D
 auto ADD(T1, T2)(T1 a, T2 b) { return a + b; }
 ```

 which opens up a lot more automatic support for C macro use.

 https://github.com/dlang/dmd/pull/16199

 Thanks to  ibuclaw,  dkorpel,  thewilsonator,  Herringway, 
  ntrel,  tim-dlang for helping me get it over the finish line.


Amazing! Congratulations to all concerned.
Feb 21
prev sibling next sibling parent reply Carl Sturtivant <sturtivant gmail.com> writes:
On Tuesday, 20 February 2024 at 02:07:45 UTC, Walter Bright wrote:

 #importc now successfully converts function-like C macros to 
 #dlang function templates:

 ```C
 #define ADD(a, b) a + b
 ```

 is converted to a D function template:

 ```D
 auto ADD(T1, T2)(T1 a, T2 b) { return a + b; }
 ```


ImportC is now fantastically useful! However, does Exception 
connected behavior limit this?

The ImportC documentation says for [Exception 
Handling](https://dlang.org/spec/importc.html#limitations) that 
"ImportC is assumed to never throw exceptions. setjmp and longjmp 
are not supported".

I am unclear under what circumstances D throwing exceptions in 
the presence of ImportC works.

Reading that sentence carefully, it does not seem to imply that 
exceptions cannot occur in D code linked to C code compiled with 
ImportC in all cases. Of course if executing D code hasn't got 
ImportC code in its call chain then exception handling will work. 
But there are two other cases.

1. I imagine that if C code calls D code which throws and the 
exception is caught in D code without unwinding the C part of the 
call stack this is OK. In some sense ImportC did not throw an 
exception in this case, even though it executed a call chain to 
do so, which could be interpreted as ImportC throwing an 
exception.

2. What if D throws an exception that would supposedly unwind the 
C part of the call stack and be caught from D further up? I might 
guess this counts as ImportC throwing an exception  and so is not 
OK, but I hope I am wrong.

Could you please explain the actual situation of ImportC in these 
cases?
Feb 21
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/21/2024 7:40 AM, Carl Sturtivant wrote:

 Could you please explain the actual situation of ImportC in these cases?


You can throw an exception in D code called by C code that is caught by D code.

C code does not have any constructs that require unwinding support, other than 
setjmp/longjmp which is not supported by ImportC.
Feb 21
parent Carl Sturtivant <sturtivant gmail.com> writes:
On Wednesday, 21 February 2024 at 18:41:17 UTC, Walter Bright 
wrote:

 On 2/21/2024 7:40 AM, Carl Sturtivant wrote:

 Could you please explain the actual situation of ImportC in 
 these cases?


 You can throw an exception in D code called by C code that is 
 caught by D code.

 C code does not have any constructs that require unwinding 
 support, other than setjmp/longjmp which is not supported by 
 ImportC.


Fantastic! I think we could informally call this no limitation at 
all.
Feb 21
prev sibling parent reply Dakota <dakota gmail.com> writes:
On Tuesday, 20 February 2024 at 02:07:45 UTC, Walter Bright wrote:

 #importc now successfully converts function-like C macros to 
 #dlang function templates:

 ```C
 #define ADD(a, b) a + b
 ```

 is converted to a D function template:

 ```D
 auto ADD(T1, T2)(T1 a, T2 b) { return a + b; }
 ```

 which opens up a lot more automatic support for C macro use.

 https://github.com/dlang/dmd/pull/16199

 Thanks to  ibuclaw,  dkorpel,  thewilsonator,  Herringway, 
  ntrel,  tim-dlang for helping me get it over the finish line.



Hi Walter, Thanks for the great work.


I want to ask you to speedup the process for this ImportC issue: 
https://issues.dlang.org/show_bug.cgi?id=23926

I am use importC for a huge C header(700KB), with dmd nightly 
most of code work well.

without fix 
https://issues.dlang.org/show_bug.cgi?id=23926), there 
is no easy way to fix this huge c code header that change 
constantly by upstream, this header is generate by amalgamation 
scripts.


There is 14K lines code, and so much function that accept 
nont-const struct pointer mixed const struct pointer.  It is not 
impractical to add so much `cast(Message*)` into D code.

Please let me know what I can do to speedup the process to fix 
this problem.

Thanks in advance.
Feb 27
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/27/2024 9:26 PM, Dakota wrote:

 It is not impractical to add so much 
 `cast(Message*)` into D code.


Hmm, interesting approach!
Feb 29
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 2/29/2024 11:22 AM, Walter Bright wrote:

 On 2/27/2024 9:26 PM, Dakota wrote:

 It is not impractical to add so much `cast(Message*)` into D code.


 
 Hmm, interesting approach!
 


I'm wondering if you could try adding this:

     #define const

and see how far that goes!
Feb 29
parent reply Dakota <dakota gmail.com> writes:
On Thursday, 29 February 2024 at 19:25:05 UTC, Walter Bright 
wrote:

 On 2/29/2024 11:22 AM, Walter Bright wrote:

 On 2/27/2024 9:26 PM, Dakota wrote:

 It is not impractical to add so much `cast(Message*)` into D 
 code.


 
 Hmm, interesting approach!
 


 I'm wondering if you could try adding this:

     #define const

 and see how far that goes!


On Thursday, 29 February 2024 at 19:25:05 UTC, Walter Bright 
wrote:

 On 2/29/2024 11:22 AM, Walter Bright wrote:

 On 2/27/2024 9:26 PM, Dakota wrote:

 It is not impractical to add so much `cast(Message*)` into D 
 code.


 
 Hmm, interesting approach!
 


 I'm wondering if you could try adding this:

     #define const

 and see how far that goes!


This cloud work but I don't want go this way. (loss const prone 
to errors in my d code, because there is a lot `property` read 
should return const, and the returned const need pass into other 
function chain. remove const is not a solution)


I suggestion add a dmd flags to change the behavior, like 
--importC-const-pointer=. so the user can choice what kind 
behavior they want.

some project use transitive const in C, so they can work with 
this new options.

One more options is translate it into a user-defined template, 
this is more flexible. user can return diff result base on the 
name. for example:

```c
typedef struct Message Message;
int tryEncode(const Message* msg);
```
=>

```sh
struct Message;
int tryEncode(importConstPointer!(Message)* msg);
```
Mar 01
next sibling parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
On Friday, 1 March 2024 at 09:52:34 UTC, Dakota wrote:

 On Thursday, 29 February 2024 at 19:25:05 UTC, Walter Bright 
 wrote:

 On 2/29/2024 11:22 AM, Walter Bright wrote:

 On 2/27/2024 9:26 PM, Dakota wrote:

 It is not impractical to add so much `cast(Message*)` into D 
 code.


 
 Hmm, interesting approach!
 


 I'm wondering if you could try adding this:

     #define const

 and see how far that goes!


 On Thursday, 29 February 2024 at 19:25:05 UTC, Walter Bright 
 wrote:

 On 2/29/2024 11:22 AM, Walter Bright wrote:

 On 2/27/2024 9:26 PM, Dakota wrote:

 It is not impractical to add so much `cast(Message*)` into D 
 code.


 
 Hmm, interesting approach!
 


 I'm wondering if you could try adding this:

     #define const

 and see how far that goes!


 This cloud work but I don't want go this way. (loss const prone 
 to errors in my d code, because there is a lot `property` read 
 should return const, and the returned const need pass into 
 other function chain. remove const is not a solution)


 I suggestion add a dmd flags to change the behavior, like 
 --importC-const-pointer=. so the user can choice what kind 
 behavior they want.

 some project use transitive const in C, so they can work with 
 this new options.

 One more options is translate it into a user-defined template, 
 this is more flexible. user can return diff result base on the 
 name. for example:

 ```c
 typedef struct Message Message;
 int tryEncode(const Message* msg);
 ```
 =>

 ```sh
 struct Message;
 int tryEncode(importConstPointer!(Message)* msg);
 ```


Unnecessary template

Perhaps it should be marked `inout`?
Mar 01
parent Dakota <dakota gmail.com> writes:
On Friday, 1 March 2024 at 11:16:56 UTC, ryuukk_ wrote:

 I suggestion add a dmd flags to change the behavior, like 
 --importC-const-pointer=. so the user can choice what kind 
 behavior they want.

 some project use transitive const in C, so they can work with 
 this new options.

 One more options is translate it into a user-defined template, 
 this is more flexible. user can return diff result base on the 
 name. for example:

 ```c
 typedef struct Message Message;
 int tryEncode(const Message* msg);
 ```
 =>

 ```sh
 struct Message;
 int tryEncode(importConstPointer!(Message)* msg);
 ```


 Unnecessary template

 Perhaps it should be marked `inout`?



In some case `inout` make sense, in other case it should be const 
and not allow accept non-const. so I think a options for dmd or 
template will let user do the choice.
Mar 01
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
The trouble is that D's semantic code simply does not support non-transitive 
const, this is worked fundamentally into the type system. Any other method I
can 
think of is going to have seams and warts.
Mar 02
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, March 2, 2024 12:01:40 PM MST Walter Bright via Digitalmars-d 
wrote:

 The trouble is that D's semantic code simply does not support non-transitive
 const, this is worked fundamentally into the type system. Any other method
 I can think of is going to have seams and warts.


Honestly, given C's semanticts for const, I'm tempted to argue that it isn't
even correct to treat anything as const when passing it to extern(C)
functions. Ultimately, it's an  system thing, because D's type system
guarantees aren't in place, but fundamentally, if you pass a const pointer
to C from D, you're not getting any actual promises about it - to the point
that you're effectively casting it to mutable when passing it.

- Jonathan M Davis
Mar 02
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/2/2024 7:05 PM, Jonathan M Davis wrote:

 Honestly, given C's semanticts for const, I'm tempted to argue that it isn't
 even correct to treat anything as const when passing it to extern(C)
 functions. Ultimately, it's an  system thing, because D's type system
 guarantees aren't in place, but fundamentally, if you pass a const pointer
 to C from D, you're not getting any actual promises about it - to the point
 that you're effectively casting it to mutable when passing it.


Hence my suggestion of:

     #define const

when having difficulties with non-transitive const!
Mar 03
parent Dakota <dakota gmail.com> writes:
On Monday, 4 March 2024 at 02:22:51 UTC, Walter Bright wrote:

 Hence my suggestion of:

     #define const

 when having difficulties with non-transitive const!



Thanks again for all the explain and tips.


from a C library user perspective:

1) the struct pointer is represents a resource handle
2) all resource is create and destroy by the library
3) to access the resource property, use api function
4) resource has attribute of const, and non-const type. and they 
are create and pass in and out into function call chain (a lot 
mix const and non-const)
5) From the perspective of a library user, a resource pointer is 
an opaque type.


in your point of view, the const pointer must behavior 
transitive, not what the c library expected.  this make no sense 
for an opaque type. (you dont have the struct definition some 
time).  You are try interacting with the c library the D ways, 
not C ways.


So until we get headconst, let user made the choice or force 
convert into const(type)* will be my choice.


right now I will try some libcang solution,  importC not work for 
me here.
Mar 03
prev sibling parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 03/03/2024 8:01 AM, Walter Bright wrote:

 The trouble is that D's semantic code simply does not support 
 non-transitive const, this is worked fundamentally into the type system. 
 Any other method I can think of is going to have seams and warts.


My suspicion is that headconst will be implemented in D at some point, 
its a question of when not if.

So all this talk about what to do because of us not having it is a tad 
amusing to me.
Mar 03