www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ImportC and nothrow/ nogc?

reply Manu <turkeyman gmail.com> writes:
I just tried using ImportC for the first time ever, but I was surprised
when I immediately received a sea of errors calling the C symbols from
`nothrow  nogc` functions.
My entire program is `nothrow  nogc`... I assumed ImportC would be
supremely suitable in this context, but apparently not...

Is there something I've missed? Is there a plan for this?
I found just one single very short forum thread...

This is classic D experience; so much work has been done on this, and then
the moment I try and use something I encounter a showstopping oversight >_<
Aug 17
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was 
 surprised
 when I immediately received a sea of errors calling the C 
 symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC would 
 be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on 
 this, and then the moment I try and use something I encounter a 
 showstopping oversight >_<
Well, C may call C++ which may throw. It is very unlikely to call the GC however. I don't think it would be hard to add a switch that "`extern(C)` functions being called from import C are ` nogc`".
Aug 18
parent reply Gregor =?UTF-8?B?TcO8Y2ts?= <gregormueckl gmx.de> writes:
On Sunday, 18 August 2024 at 09:42:16 UTC, Nicholas Wilson wrote:
 On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was 
 surprised
 when I immediately received a sea of errors calling the C 
 symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC 
 would be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on 
 this, and then the moment I try and use something I encounter 
 a showstopping oversight >_<
Well, C may call C++ which may throw. It is very unlikely to call the GC however. I don't think it would be hard to add a switch that "`extern(C)` functions being called from import C are ` nogc`".
Here's what I'm thinking: The C function interface has no notion of C++ exceptions (AFAIK it's following C specs, not C++), so C++ exceptions should not even be a consideration at that point. I do not think that throwing a C++ exception up a stack that contains extern(C) functions is even properly defined behavior on the C++ side. In other words, all ImportC declarations should be implicitly nothrow. Any usage of the D GC from a C interface could only come in two flavors: D code exported through a C interface or a C interface implemented in another language that uses the D GC through a library interface. I believe that both cases should be treated as if the use of the D GC is an implementation detail and an implicit nogc is warranted.
Aug 19
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 20/08/2024 12:24 PM, Gregor Mückl wrote:
 On Sunday, 18 August 2024 at 09:42:16 UTC, Nicholas Wilson wrote:
 On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was surprised
 when I immediately received a sea of errors calling the C symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC would be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on this, and 
 then the moment I try and use something I encounter a showstopping 
 oversight >_<
Well, C may call C++ which may throw. It is very unlikely to call the GC however. I don't think it would be hard to add a switch that "`extern(C)` functions being called from import C are ` nogc`".
Here's what I'm thinking: The C function interface has no notion of C++ exceptions (AFAIK it's following C specs, not C++), so C++ exceptions should not even be a consideration at that point. I do not think that throwing a C++ exception up a stack that contains extern(C) functions is even properly defined behavior on the C++ side. In other words, all ImportC declarations should be implicitly nothrow.
Its defined for MSVC: https://learn.microsoft.com/en-us/cpp/cpp/structured-exception-handling-c-cpp?view=msvc-170 "Structured exception handling (SEH) is a Microsoft extension to C and C++ to handle certain exceptional code situations, such as hardware faults, gracefully." I'm pretty sure for other platforms it is also defined as part of unwinding, although I can't be bothered to hunt for the specification. It's not about C, it's about languages that are not C++ (consider JIT's for instance).
Aug 19
next sibling parent Manu <turkeyman gmail.com> writes:
SEH is not C or C++, it's a Microsoft thing with no standards to speak of.
It seems irrelevant to this conversation to me.

On Tue, 20 Aug 2024 at 10:36, Richard (Rikki) Andrew Cattermole via
Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 20/08/2024 12:24 PM, Gregor M=C3=BCckl wrote:
 On Sunday, 18 August 2024 at 09:42:16 UTC, Nicholas Wilson wrote:
 On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was surpris=
ed
 when I immediately received a sea of errors calling the C symbols fro=
m
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC would be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on this, and
 then the moment I try and use something I encounter a showstopping
 oversight >_<
Well, C may call C++ which may throw. It is very unlikely to call the GC however. I don't think it would be hard to add a switch that "`extern(C)` functions being called from import C are ` nogc`".
Here's what I'm thinking: The C function interface has no notion of C++ exceptions (AFAIK it's following C specs, not C++), so C++ exceptions should not even be a consideration at that point. I do not think that throwing a C++ exception up a stack that contains extern(C) functions is even properly defined behavior on the C++ side. In other words, all ImportC declarations should be implicitly nothrow.
Its defined for MSVC: https://learn.microsoft.com/en-us/cpp/cpp/structured-exception-handling-c=
-cpp?view=3Dmsvc-170
 "Structured exception handling (SEH) is a Microsoft extension to C and
 C++ to handle certain exceptional code situations, such as hardware
 faults, gracefully."

 I'm pretty sure for other platforms it is also defined as part of
 unwinding, although I can't be bothered to hunt for the specification.
 It's not about C, it's about languages that are not C++ (consider JIT's
 for instance).
Aug 20
prev sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
On Tuesday, 20 August 2024 at 00:30:46 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 On 20/08/2024 12:24 PM, Gregor Mückl wrote:
 On Sunday, 18 August 2024 at 09:42:16 UTC, Nicholas Wilson 
 wrote:
 On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 [...]
Well, C may call C++ which may throw. It is very unlikely to call the GC however. I don't think it would be hard to add a switch that "`extern(C)` functions being called from import C are ` nogc`".
Here's what I'm thinking: The C function interface has no notion of C++ exceptions (AFAIK it's following C specs, not C++), so C++ exceptions should not even be a consideration at that point. I do not think that throwing a C++ exception up a stack that contains extern(C) functions is even properly defined behavior on the C++ side. In other words, all ImportC declarations should be implicitly nothrow.
Its defined for MSVC: https://learn.microsoft.com/en-us/cpp/cpp/structured-exception-handling-c-cpp?view=msvc-170 "Structured exception handling (SEH) is a Microsoft extension to C and C++ to handle certain exceptional code situations, such as hardware faults, gracefully." I'm pretty sure for other platforms it is also defined as part of unwinding, although I can't be bothered to hunt for the specification. It's not about C, it's about languages that are not C++ (consider JIT's for instance).
That is a special case, given how C++ is relevant in the Microsoft ecosystem, and SEH existence, you won't find that in other platforms, not even on UNIX where C++ was born alongside C. I am not even sure that is even covered on Itanium ABI that most modern compilers have meanwhile adopted on remaining UNIX platforms.
Aug 21
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 22/08/2024 6:54 PM, Paulo Pinto wrote:
 That is a special case, given how C++ is relevant in the Microsoft 
 ecosystem, and SEH existence, you won't find that in other platforms, 
 not even on UNIX where C++ was born alongside C.
 
 I am not even sure that is even covered on Itanium ABI that most modern 
 compilers have meanwhile adopted on remaining UNIX platforms.
https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_exceptions.html "C language code that is expecting to interoperate with C++ should be compiled with -fexceptions. This will make debugging a C language function called as part of C++-induced stack unwinding possible." Not quite as special as it may first appear. Unwinding tables are just not turned on by default.
Aug 22
parent IchorDev <zxinsworld gmail.com> writes:
On Thursday, 22 August 2024 at 07:38:08 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 Unwinding tables are just not turned on by default.
And so ImportC functions should probably also be `nothrow` by default, and then we should finally merge an implementation of [DIP1029](https://github.com/dlang/DIPs/blob/master/DIPs/accepted/DIP1029.md). The 6 months of stalling ljmf’s implementation went through was just cruel for such a trivial feature.
Aug 22
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, 20 Aug 2024 at 10:26, Gregor M=C3=BCckl via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Sunday, 18 August 2024 at 09:42:16 UTC, Nicholas Wilson wrote:
 On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was
 surprised
 when I immediately received a sea of errors calling the C
 symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC
 would be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on
 this, and then the moment I try and use something I encounter
 a showstopping oversight >_<
Well, C may call C++ which may throw. It is very unlikely to call the GC however. I don't think it would be hard to add a switch that "`extern(C)` functions being called from import C are ` nogc`".
Here's what I'm thinking: The C function interface has no notion of C++ exceptions (AFAIK it's following C specs, not C++), so C++ exceptions should not even be a consideration at that point. I do not think that throwing a C++ exception up a stack that contains extern(C) functions is even properly defined behavior on the C++ side. In other words, all ImportC declarations should be implicitly nothrow. Any usage of the D GC from a C interface could only come in two flavors: D code exported through a C interface or a C interface implemented in another language that uses the D GC through a library interface. I believe that both cases should be treated as if the use of the D GC is an implementation detail and an implicit nogc is warranted.
Exactly... these points are my precise assumption going in here. Also, considering the possibility that D code is called from C code via a function pointer; you have to wonder how the C code received a function pointer in the first place? If the C code is nothrow nogc, then it would be impossible to supply a function pointer that was not also nothrow nogc to the C code in the first place.
Aug 20
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/20/2024 2:12 AM, Manu wrote:
 Exactly... these points are my precise assumption going in here.
 Also, considering the possibility that D code is called from C code via a 
 function pointer; you have to wonder how the C code received a function
pointer 
 in the first place? If the C code is nothrow  nogc, then it would be
impossible 
 to supply a function pointer that was not also nothrow  nogc to the C code in 
 the first place.
``` // D file: extern (C) void daFunc() { throw new Exception(); } ``` ``` // C file: extern void daFunc(); void hahahahaha() { daFunc(); } ``` No function pointers required. The D <=> C interoperability goes both ways.
Aug 21
parent reply Manu <turkeyman gmail.com> writes:
On Thu, 22 Aug 2024 at 05:51, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/20/2024 2:12 AM, Manu wrote:
 Exactly... these points are my precise assumption going in here.
 Also, considering the possibility that D code is called from C code via
a
 function pointer; you have to wonder how the C code received a function
pointer
 in the first place? If the C code is nothrow  nogc, then it would be
impossible
 to supply a function pointer that was not also nothrow  nogc to the C
code in
 the first place.
``` // D file: extern (C) void daFunc() { throw new Exception(); } ``` ``` // C file: extern void daFunc(); void hahahahaha() { daFunc(); } ``` No function pointers required. The D <=> C interoperability goes both ways.
That's not a C program, that's a D program with some C code sprinkled in. But right... a mixture of extern(C) in D code, and also ImportC used in conjunction, where the ImportC code makes explicit use of symbols that it expects to find externally... Feels unlikely, pretty contrived; why would you be using extern(C) if you are also using ImportC? They're kinda mutually exclusive for my interest. I wouldn't use ImportC if I was satisfied to write extern(C) bindings. I mean, being overly cautious like this is not in the spirit of using C code at all! C code will _always_ introduce these sorts of risks 100% without question. If you're stressed about this, while also having no agency to control your environment or test your programs validity, you kinda have no business linking to C code in the first place. But all that as it is, so... we add a compiler flag to control whether these 2 attributes are applied to ImportC declarations or not? The current situation is not reasonable. You've done all this work and made all this hype, the first time I want to make use of it, it turns out it's not actually usable by the exact sort of code that seems the most likely to want to use the feature in the first place! C doesn't throw or GC, period. Of course I can contort a C environment to do whatever I want; but the argument that C can throw or can GC alloc implies that the C code is not actually C code... it is infact actually just a small piece of connective tissue in some non-C project. A C library that is a self-contained package does not call C++ code or D code. We need a way to assert this case. I think a compile option which asserts this case seems fine. Maybe apply the arg to a directory; in the event there are multiple C libraries being included in the project, it only applies to C code under that path, like some self-contained libraries?
Aug 21
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/21/2024 10:27 PM, Manu wrote:
 That's not a C program, that's a D program with some C code sprinkled in.
 But right... a mixture of extern(C) in D code, and also ImportC used in 
 conjunction, where the ImportC code makes explicit use of symbols that it 
 expects to find externally...
 Feels unlikely, pretty contrived; why would you be using extern(C) if you are 
 also using ImportC? They're kinda mutually exclusive for my interest. I
wouldn't 
 use ImportC if I was satisfied to write extern(C) bindings.
I've used it in hybrid C and D programs. It also comes about when one is "transitioning" C code to D code. Enabling use of D code in a C project without needing a D main() entry point is a nice plus.
 I mean, being overly cautious like this is not in the spirit of using C code
at 
 all! C code will _always_ introduce these sorts of risks 100% without
question. 
 If you're stressed about this, while also having no agency to control your 
 environment or test your programs validity, you kinda have no business linking 
 to C code in the first place.
Some years back, I got into a terrific disagreement with everyone else in the D community when I wanted C declarations to default to trusted :-/
 But all that as it is, so... we add a compiler flag to control whether these 2 
 attributes are applied to ImportC declarations or not?
I'm reluctant to add more flags as every one of them is a hack, and an admission of language design failure. A compiler should "just work".
 The current situation is not reasonable. You've done all this work and made
all 
 this hype, the first time I want to make use of it, it turns out it's not 
 actually usable by the exact sort of code that seems the most likely to want
to 
 use the feature in the first place!
 C doesn't throw
Yes, it does. setjmp()/longjmp() are in the C Standard 7.13.
 or GC, period.
 Of course I can contort a C environment to do 
 whatever I want; but the argument that C can throw or can GC alloc implies
that 
 the C code is not actually C code... it is infact actually just a small piece
of 
 connective tissue in some non-C project.
 A C library that is a self-contained package does not call C++ code or D code. 
 We need a way to assert this case. I think a compile option which asserts this 
 case seems fine.
 Maybe apply the arg to a directory; in the event there are multiple C
libraries 
 being included in the project, it only applies to C code under that path, like 
 some self-contained libraries?
I understand your concern. Perhaps we can approach it from a different direction - why do you need the code to be nothrow and nogc? I don't know if this would work for you, but you could also try -betterC, which is implicitly nogc. For reference, the following enables calling a throwing function from a nothrow function: ``` nothrow int horse(int x, int y) { try { battery(); } catch (Exception e) { } return x = y + 3; } int battery() { throw new Exception("hello"); } ```
Aug 23
parent reply Manu <turkeyman gmail.com> writes:
On Sat, 24 Aug 2024 at 12:28, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/21/2024 10:27 PM, Manu wrote:
 That's not a C program, that's a D program with some C code sprinkled in.
 But right... a mixture of extern(C) in D code, and also ImportC used in
 conjunction, where the ImportC code makes explicit use of symbols that
it
 expects to find externally...
 Feels unlikely, pretty contrived; why would you be using extern(C) if
you are
 also using ImportC? They're kinda mutually exclusive for my interest. I
wouldn't
 use ImportC if I was satisfied to write extern(C) bindings.
I've used it in hybrid C and D programs. It also comes about when one is "transitioning" C code to D code. Enabling use of D code in a C project without needing a D main() entry point is a nice plus.
 I mean, being overly cautious like this is not in the spirit of using C
code at
 all! C code will _always_ introduce these sorts of risks 100% without
question.
 If you're stressed about this, while also having no agency to control
your
 environment or test your programs validity, you kinda have no business
linking
 to C code in the first place.
Some years back, I got into a terrific disagreement with everyone else in the D community when I wanted C declarations to default to trusted :-/
I might have been inclined to back you on that one... that said though, the real solution to that category of problem is to accept that we need annotated scopes. We need to have: trusted { some_code; } I don't think people would feel particularly inconvenienced by C calls being unsafe if it's trivial to write an unsafe block.
 But all that as it is, so... we add a compiler flag to control whether
 these 2
 attributes are applied to ImportC declarations or not?
I'm reluctant to add more flags as every one of them is a hack, and an admission of language design failure. A compiler should "just work".
Look, ImportC is a gigantic effort, and it's safe to say that with this limitation, it is a critical design failure! I literally can't use it at all. The most useful case for ImportC I've ever encountered is precisely the case that's explicitly prohibited by ImportC... if that's not a design failure, what is?
 The current situation is not reasonable. You've done all this work and
 made all
 this hype, the first time I want to make use of it, it turns out it's
not
 actually usable by the exact sort of code that seems the most likely to
want to
 use the feature in the first place!
 C doesn't throw
Yes, it does. setjmp()/longjmp() are in the C Standard 7.13.
That's way outside the language. nothrow has nothing to say about this.
 or GC, period.

 Of course I can contort a C environment to do
 whatever I want; but the argument that C can throw or can GC alloc
implies that
 the C code is not actually C code... it is infact actually just a small
piece of
 connective tissue in some non-C project.
 A C library that is a self-contained package does not call C++ code or D
code.
 We need a way to assert this case. I think a compile option which
asserts this
 case seems fine.
 Maybe apply the arg to a directory; in the event there are multiple C
libraries
 being included in the project, it only applies to C code under that
path, like
 some self-contained libraries?
I understand your concern. Perhaps we can approach it from a different direction - why do you need the code to be nothrow and nogc?
That's just what I'm into. You just can't have a GC when you have only 200K of ram in total. I will not link the GC. Similarly, exceptions are just worthless noise. They don't make the language better, and I'm trying to tighten what I can control. Exceptions aren't the same in terms of being ultra-limiting though, you can still make use of a lib in some leaf function; just as long as you catch before returning to the main ecosystem, which I'm quite happy to strictly enforce. You shouldn't have created nothrow and nogc if you didn't intend for me to use them... I don't know if this would work for you, but you could also try -betterC,
 which
 is implicitly  nogc.
Yeah, it's a tricky migration. with nothrow and nogc I'm finding I can make progress incrementally, but not the case with BetterC. Also, there's things that you lose with BetterC that I don't object to. I like classinfo, and I like dynamic casts, and several other things I noticed when I tried it out that I can't remember right now. For reference, the following enables calling a throwing function from a
 nothrow
 function:

 ```
 nothrow int horse(int x, int y)
 {
      try
      {
          battery();
      }
      catch (Exception e)
      {
      }
      return x = y + 3;
 }

 int battery()
 {
      throw new Exception("hello");
 }
 ```
Yes, I mentioned that above... but I'm not going to do that at every call to the CRT or the OS. I know perfectly well that malloc() won't throw. I'm trying to systematically minimise waste.
Aug 24
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 25/08/2024 5:10 AM, Manu wrote:
 I might have been inclined to back you on that one... that said though, 
 the real solution to that category of problem is to accept that we need 
 annotated scopes.
 We need to have:
 
  trusted {
    some_code;
 }
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Aug 24
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew Cattermole via
Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 I might have been inclined to back you on that one... that said though,
 the real solution to that category of problem is to accept that we need
 annotated scopes.
 We need to have:

  trusted {
    some_code;
 }
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Yes, this isn't a new conversation! We've all been begging for this since literally the day it landed ;)
Aug 24
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Saturday, 24 August 2024 at 17:43:38 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew Cattermole 
 via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 [...]
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Just wrote a trusted function and call it: that's the sane way to do it and respect code reviewer hard job.
Aug 25
parent reply Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 19:56, Paolo Invernizzi via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:43:38 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew Cattermole
 via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 [...]
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Just wrote a trusted function and call it: that's the sane way to do it and respect code reviewer hard job.
...so, because I'm going to make one single unsafe function call inside of some function, I should eject all other related or unrelated safety checks for the entire surrounding context?
Aug 25
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Sunday, 25 August 2024 at 10:32:31 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:56, Paolo Invernizzi via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:43:38 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew 
 Cattermole via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 [...]
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Just wrote a trusted function and call it: that's the sane way to do it and respect code reviewer hard job.
...so, because I'm going to make one single unsafe function call inside of some function, I should eject all other related or unrelated safety checks for the entire surrounding context?
No, you should isolate the unsafe part of code into a function, explain why it’s unsafe, the intent of the code, the expected parameter values and the expected returns, so that reviewer can check that the interface is really memory safe. Then call this extremely simple function from the rest of safe code safe code.
Aug 25
parent reply Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 21:31, Paolo Invernizzi via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 10:32:31 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:56, Paolo Invernizzi via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:43:38 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew
 Cattermole via Digitalmars-d <digitalmars-d puremagic.com>
 wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 [...]
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Just wrote a trusted function and call it: that's the sane way to do it and respect code reviewer hard job.
...so, because I'm going to make one single unsafe function call inside of some function, I should eject all other related or unrelated safety checks for the entire surrounding context?
No, you should isolate the unsafe part of code into a function, explain why it=E2=80=99s unsafe, the intent of the code, the expected parameter values and the expected returns, so that reviewer can check that the interface is really memory safe. Then call this extremely simple function from the rest of safe code safe code.
So, this then: extern(C) int myUnsafeFunction(int x, int y); trusted int myPointlessWrapper(int x, int y) { return myUnsafeFunction (x, y); } safe mySuperSafeFunction(...) { //... lots of code int r =3D myPointlessWrapper(arg1, arg2); //... lots of code } Brilliant.
Aug 25
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Sunday, 25 August 2024 at 11:40:06 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 21:31, Paolo Invernizzi via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 10:32:31 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:56, Paolo Invernizzi via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:43:38 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew 
 Cattermole via Digitalmars-d 
 <digitalmars-d puremagic.com> wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 [...]
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Just wrote a trusted function and call it: that's the sane way to do it and respect code reviewer hard job.
...so, because I'm going to make one single unsafe function call inside of some function, I should eject all other related or unrelated safety checks for the entire surrounding context?
No, you should isolate the unsafe part of code into a function, explain why it’s unsafe, the intent of the code, the expected parameter values and the expected returns, so that reviewer can check that the interface is really memory safe. Then call this extremely simple function from the rest of safe code safe code.
So, this then: extern(C) int myUnsafeFunction(int x, int y); trusted int myPointlessWrapper(int x, int y) { return myUnsafeFunction (x, y); } safe mySuperSafeFunction(...) { //... lots of code int r = myPointlessWrapper(arg1, arg2); //... lots of code } Brilliant.
That’s sidestepping the problem from your side: there’s no explanation why myUnsafeFunction has an unsafe interface, under what condition it’s unsafe, and what it’s supposed to do. Neither any check that it’s parameters are valid and they are not turning into memory unsafety. A reviewer for sure will ask you to add that information and assurance about memory safety of calling myUnsadeFunction If the extern(C) function is intese memory safe, there should be a way to mark the declaration safe, also with importC, and there we can reason about a solution.
Aug 25
parent reply Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 21:56, Paolo Invernizzi via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 11:40:06 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 21:31, Paolo Invernizzi via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 10:32:31 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:56, Paolo Invernizzi via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:43:38 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew
 Cattermole via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 [...]
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Just wrote a trusted function and call it: that's the sane way to do it and respect code reviewer hard job.
...so, because I'm going to make one single unsafe function call inside of some function, I should eject all other related or unrelated safety checks for the entire surrounding context?
No, you should isolate the unsafe part of code into a function, explain why it=E2=80=99s unsafe, the intent of the code, the expected parameter values and the expected returns, so that reviewer can check that the interface is really memory safe. Then call this extremely simple function from the rest of safe code safe code.
So, this then: extern(C) int myUnsafeFunction(int x, int y); trusted int myPointlessWrapper(int x, int y) { return myUnsafeFunction (x, y); } safe mySuperSafeFunction(...) { //... lots of code int r =3D myPointlessWrapper(arg1, arg2); //... lots of code } Brilliant.
That=E2=80=99s sidestepping the problem from your side: there=E2=80=99s n=
o
 explanation why myUnsafeFunction has an unsafe interface, under
 what condition it=E2=80=99s unsafe, and what it=E2=80=99s supposed to do.=
Neither
 any check that it=E2=80=99s parameters are valid and they are not turning
 into memory unsafety.

 A reviewer for sure will ask you to add that information and
 assurance about memory safety of calling myUnsadeFunction

 If the extern(C) function is intese memory safe, there should be
 a way to mark the declaration  safe, also with importC, and there
 we can reason about a solution.
I think you might have missed the point here... we're talking about calling C libraries. It already has its interface nicely presented and documented, with expected arguments and returns all written out. We just want to call a function, that's all. There's no safety 'risk' here to comment on, unless you consider using a C library to be a risk in principle.
Aug 25
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Sunday, 25 August 2024 at 12:47:13 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 21:56, Paolo Invernizzi via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 11:40:06 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 21:31, Paolo Invernizzi via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 10:32:31 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:56, Paolo Invernizzi via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:43:38 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew
 Cattermole via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 [...]
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Just wrote a trusted function and call it: that's the sane way to do it and respect code reviewer hard job.
...so, because I'm going to make one single unsafe function call inside of some function, I should eject all other related or unrelated safety checks for the entire surrounding context?
No, you should isolate the unsafe part of code into a function, explain why it’s unsafe, the intent of the code, the expected parameter values and the expected returns, so that reviewer can check that the interface is really memory safe. Then call this extremely simple function from the rest of safe code safe code.
So, this then: extern(C) int myUnsafeFunction(int x, int y); trusted int myPointlessWrapper(int x, int y) { return myUnsafeFunction (x, y); } safe mySuperSafeFunction(...) { //... lots of code int r = myPointlessWrapper(arg1, arg2); //... lots of code } Brilliant.
That’s sidestepping the problem from your side: there’s no explanation why myUnsafeFunction has an unsafe interface, under what condition it’s unsafe, and what it’s supposed to do. Neither any check that it’s parameters are valid and they are not turning into memory unsafety. A reviewer for sure will ask you to add that information and assurance about memory safety of calling myUnsadeFunction If the extern(C) function is intese memory safe, there should be a way to mark the declaration safe, also with importC, and there we can reason about a solution.
I think you might have missed the point here... we're talking about calling C libraries. It already has its interface nicely presented and documented, with expected arguments and returns all written out. We just want to call a function, that's all. There's no safety 'risk' here to comment on, unless you consider using a C library to be a risk in principle.
Using a C library IS a risk in principle, if it includes memory unsafe function interfaces. The memory unsafe C interface ( system) must be called from a trusted function that ensure that proper parameters and conditions are properly set to not trigger memory unsafely: that's basically the whole point of trusted. If you instead mean that SOME extern(C) function is non memory unsafe, and that have been discovered reading its documentation, you could declare the C declaration (in the D module) directly as safe. That's what happened to be before importC. I further agree with you that would be good to have a clever way to declare safe declarations for SOME function automatically declared by importC. BTW, nice to see you here again in the forum, Manu! :-P Further, I agree
Aug 25
parent reply Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 23:25, Paolo Invernizzi via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 12:47:13 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 21:56, Paolo Invernizzi via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 11:40:06 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 21:31, Paolo Invernizzi via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 10:32:31 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:56, Paolo Invernizzi via
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:43:38 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 03:31, Richard (Rikki) Andrew
 Cattermole via Digitalmars-d
 <digitalmars-d puremagic.com> wrote:

 On 25/08/2024 5:10 AM, Manu wrote:
 [...]
I've been considering something along these lines. Specifically, `` trusted`` does not mean the entire body shouldn't be verified. It just means that you are going to do something naughty that needs looking at. So you need annotated scopes inside of it, to do the naughty thing.
Just wrote a trusted function and call it: that's the sane way to do it and respect code reviewer hard job.
...so, because I'm going to make one single unsafe function call inside of some function, I should eject all other related or unrelated safety checks for the entire surrounding context?
No, you should isolate the unsafe part of code into a function, explain why it=E2=80=99s unsafe, the intent of the code, the expected parameter values and the expected returns, so that reviewer can check that the interface is really memory safe. Then call this extremely simple function from the rest of safe code safe code.
So, this then: extern(C) int myUnsafeFunction(int x, int y); trusted int myPointlessWrapper(int x, int y) { return myUnsafeFunction (x, y); } safe mySuperSafeFunction(...) { //... lots of code int r =3D myPointlessWrapper(arg1, arg2); //... lots of code } Brilliant.
That=E2=80=99s sidestepping the problem from your side: there=E2=80=99=
s no
 explanation why myUnsafeFunction has an unsafe interface, under
 what condition it=E2=80=99s unsafe, and what it=E2=80=99s supposed to =
do.
 Neither
 any check that it=E2=80=99s parameters are valid and they are not
 turning
 into memory unsafety.

 A reviewer for sure will ask you to add that information and
 assurance about memory safety of calling myUnsadeFunction

 If the extern(C) function is intese memory safe, there should
 be a way to mark the declaration  safe, also with importC, and
 there we can reason about a solution.
I think you might have missed the point here... we're talking about calling C libraries. It already has its interface nicely presented and documented, with expected arguments and returns all written out. We just want to call a function, that's all. There's no safety 'risk' here to comment on, unless you consider using a C library to be a risk in principle.
Using a C library IS a risk in principle, if it includes memory unsafe function interfaces. The memory unsafe C interface ( system) must be called from a trusted function that ensure that proper parameters and conditions are properly set to not trigger memory unsafely: that's basically the whole point of trusted.
Right, but in ~95% of cases C functions don't have anything about them that makes them memory unsafe, or any relevant validation or concerning API material... they're just de-facto unsafe. Here's some great functions that I plucked from top of my head: float abs(float); int rand(); void sleep(int); int fclose(FILE); pid_t getpid(); It's obviously stupid to write a wrapper function for every single C function we use. That's entirely self-defeating. Why would anybody import a C header file if they have to write as many function wrappers as C function prototypes imported? Might as well have just written the prototypes... it's actually LESS code. And then there's maybe kinda contentious ones, like: FILE fopen(char* filename, char* mode); Now, clearly there's a null-termination concern here, but that's not actually a category of issue that safe is concerned with as far as I know. Generally, the user just wants to do this: unsafe { FILE f =3D fopen(filename.toStringz, "rb"); } There's no value in writing a whole function wrapper to automate a call to toStringz(). I'm not even sure that call is actually unsafe anyway. toStringz() isn't unsafe...? I further agree with you that would be good to have a clever way
 to declare  safe declarations for SOME function automatically
 declared by importC.
That list I wrote above and a million functions of that nature; they don't have any interaction with safe... declaring them system is nothing more than a nuisance. Nothing is added by not trusting them. If there's some API tell-tales that can definitively determine the safety of a function, then maybe it could make an educated guess... but failing that, then there's a serious practical conundrum here. BTW, nice to see you here again in the forum, Manu! :-P

=F0=9F=8E=89 =F0=9F=8E=89 =F0=9F=8E=89
Aug 25
next sibling parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Sunday, 25 August 2024 at 16:58:53 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 23:25, Paolo Invernizzi via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 12:47:13 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 21:56, Paolo Invernizzi via 
 Digitalmars-d < digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 11:40:06 UTC, Manu wrote:
 [...]
That’s sidestepping the problem from your side: there’s no explanation why myUnsafeFunction has an unsafe interface, under what condition it’s unsafe, and what it’s supposed to do. Neither any check that it’s parameters are valid and they are not turning into memory unsafety. A reviewer for sure will ask you to add that information and assurance about memory safety of calling myUnsadeFunction If the extern(C) function is intese memory safe, there should be a way to mark the declaration safe, also with importC, and there we can reason about a solution.
I think you might have missed the point here... we're talking about calling C libraries. It already has its interface nicely presented and documented, with expected arguments and returns all written out. We just want to call a function, that's all. There's no safety 'risk' here to comment on, unless you consider using a C library to be a risk in principle.
Using a C library IS a risk in principle, if it includes memory unsafe function interfaces. The memory unsafe C interface ( system) must be called from a trusted function that ensure that proper parameters and conditions are properly set to not trigger memory unsafely: that's basically the whole point of trusted.
Right, but in ~95% of cases C functions don't have anything about them that makes them memory unsafe, or any relevant validation or concerning API material... they're just de-facto unsafe. Here's some great functions that I plucked from top of my head: float abs(float); int rand(); void sleep(int); int fclose(FILE); pid_t getpid(); It's obviously stupid to write a wrapper function for every single C function we use. That's entirely self-defeating. Why would anybody import a C header file if they have to write as many function wrappers as C function prototypes imported? Might as well have just written the prototypes... it's actually LESS code.
If you know that 100% of the C function prototypes you are importing are memory safe, then yes, that's yes, having them imported as system is an annoyance, and DLang can be improved with some way to pragma the compiler to importC the file as safe On the other side, if you know that you have 95% of memory safety C functions, and 5% of memory unsafe function, marking that 5% as trusted or safe is just safe-washing, it's not acceptable in a code base that aims to be serious about memory safety.
 And then there's maybe kinda contentious ones, like:

 FILE fopen(char* filename, char* mode);

 Now, clearly there's a null-termination concern here, but 
 that's not actually a category of issue that  safe is concerned 
 with as far as I know. Generally, the user just wants to do 
 this:

 unsafe {
   FILE f = fopen(filename.toStringz, "rb");
 }

 There's no value in writing a whole function wrapper to 
 automate a call to
 toStringz().
 I'm not even sure that call is actually unsafe anyway. 
 toStringz() isn't
 unsafe...?
Indeed we do that all the time, to provide some kind of D-way to call C-API. As you see, the inconvenience mark it's just a matter of taste, an opinion.
 I further agree with you that would be good to have a clever way
 to declare  safe declarations for SOME function automatically 
 declared by importC.
That list I wrote above and a million functions of that nature; they don't have any interaction with safe... declaring them system is nothing more than a nuisance. Nothing is added by not trusting them. If there's some API tell-tales that can definitively determine the safety of a function, then maybe it could make an educated guess... but failing that, then there's a serious practical conundrum here.
The point here is that's not something like 'maybe my process is memory safe, by educated guess', it's not a matter of being practical or not. If memory safeties is a value, than it should be threaded seriously also if it involves not practical way of ensuring it. Any addition to alleviate the burden is welcomed, especially for import C, but without throwing the safety baby with the water.
Aug 26
prev sibling parent Dukc <ajieskola gmail.com> writes:
Manu kirjoitti 25.8.2024 klo 19.58:
 There's no value in writing a whole function wrapper to automate a call 
 to toStringz().
 I'm not even sure that call is actually unsafe anyway. toStringz() isn't 
 unsafe...?
Well, `toStringz` copies the string to append a zero. Besides, `fopen` takes char pointers, not arrays, unlike `toStringz`. So this would compile: `fopen(new char('u'), new char('b'))` .
Aug 26
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/24/2024 10:29 AM, Richard (Rikki) Andrew Cattermole wrote:
 So you need annotated scopes inside of it, to do the naughty thing.
We did that with trusted lambdas. The result was safe turned into a joke.
Aug 25
parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 26/08/2024 5:42 PM, Walter Bright wrote:
 On 8/24/2024 10:29 AM, Richard (Rikki) Andrew Cattermole wrote:
 So you need annotated scopes inside of it, to do the naughty thing.
We did that with trusted lambdas. The result was safe turned into a joke.
That doesn't sound like what I was suggesting there. To clarify: ```d void func() trusted { trusted { unsafe_thing; } safe_only_thing; unsafe_thing; // Error } ``` Not: ```d void func() safe { trusted { unsafe_thing; } } ```
Aug 25
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Mixing safe and trusted blocks into the same function just doesn't work. Timon 
pointed out why in another post in this thread.

I take that back. It appears to work, but it does not, as trusted code needs to 
provide a safe interface to it. That doesn't happen if it's just a "suppress
the 
error message" hack.
Aug 26
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 26/08/2024 7:25 PM, Walter Bright wrote:
 Mixing safe and trusted blocks into the same function just doesn't work. 
 Timon pointed out why in another post in this thread.
 
 I take that back. It appears to work, but it does not, as trusted code 
 needs to provide a safe interface to it. That doesn't happen if it's 
 just a "suppress the error message" hack.
I agree there is a certain amount of uneasiness of logic that would be doable with it. Not worth an ideas forum post, without this figured out and if it was part of a bigger design. The problem is though, there shouldn't be a fundamental difference between a function body, and a scope within a function body. So I'll agree that it appears to work, but there is likely holes in it.
Aug 26
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 8/24/24 19:10, Manu wrote:
 
     Some years back, I got into a terrific disagreement with everyone
     else in the D
     community when I wanted C declarations to default to  trusted :-/
 
 
 I might have been inclined to back you on that one...
Well, you'd have been wrong. Just remove safe at that point.
Aug 24
parent reply Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 10:56, Timon Gehr via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/24/24 19:10, Manu wrote:
     Some years back, I got into a terrific disagreement with everyone
     else in the D
     community when I wanted C declarations to default to  trusted :-/


 I might have been inclined to back you on that one...
Well, you'd have been wrong. Just remove safe at that point.
Well I think the proper solution is to allow the user to declare unsafe (ie, trusted) code blocks, and make unsafe calls from within those blocks, which would include calls to C code.
Aug 24
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 8/25/24 04:04, Manu wrote:
 On Sun, 25 Aug 2024 at 10:56, Timon Gehr via Digitalmars-d <digitalmars- 
 d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 8/24/24 19:10, Manu wrote:
      >
      >     Some years back, I got into a terrific disagreement with everyone
      >     else in the D
      >     community when I wanted C declarations to default to  trusted :-/
      >
      >
      > I might have been inclined to back you on that one...
 
     Well, you'd have been wrong. Just remove  safe at that point.
 
 
 Well I think the proper solution is to allow the user to declare unsafe 
 (ie,  trusted) code blocks, and make unsafe calls from within those 
 blocks, which would include calls to C code.
` trusted` does not mean unsafe. ` trusted` means memory safe but not automatically checked. Some `extern(C)` functions can be marked ` trusted`, it's just a really bad default as memory-unsafe interfaces abound in C. Anyway, I fully agree there would be more ergonomic ways to design those features.
Aug 24
parent reply Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 13:21, Timon Gehr via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/25/24 04:04, Manu wrote:
 On Sun, 25 Aug 2024 at 10:56, Timon Gehr via Digitalmars-d <digitalmars-
 d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:

     On 8/24/24 19:10, Manu wrote:
      >
      >     Some years back, I got into a terrific disagreement with
everyone
      >     else in the D
      >     community when I wanted C declarations to default to  trusted
:-/
      >
      >
      > I might have been inclined to back you on that one...

     Well, you'd have been wrong. Just remove  safe at that point.


 Well I think the proper solution is to allow the user to declare unsafe
 (ie,  trusted) code blocks, and make unsafe calls from within those
 blocks, which would include calls to C code.
` trusted` does not mean unsafe. ` trusted` means memory safe but not automatically checked. Some `extern(C)` functions can be marked ` trusted`, it's just a really bad default as memory-unsafe interfaces abound in C. Anyway, I fully agree there would be more ergonomic ways to design those features.
Yeah the 'trusted' terminology doesn't actually map well to the operation that the programmer usually wants to perform, which is "I am about to do something unsafe, but I'm trying to assert confidence that I knew what I was doing". An 'unsafe' scope is almost always what the user wants. trusted sits awkwardly conceptually; I see provably-safe and unsafe, and that's it. Any middling is a programmers effort to make use of unsafe machinery and assert that they thought they did it right; while leaving a trail that you can search for in the code.
Aug 24
parent Walter Bright <newshound2 digitalmars.com> writes:
 trusted means the function has an  safe interface. What the function does 
internally is not checked. For example:

```
 trusted int[] foo()
{
     int* p = cast(int*)malloc(10 * int.sizeof);
     assert(p);
     int[] a = p[0 .. 10];
     return a;
}
```
has a memory safe interface. The following:

```
 trusted int[] foo()
{
     int* p = cast(int*)malloc(10 * int.sizeof);
     assert(p);
     int[] a = p[0 .. 11]; // not memory safe
     return a;
}
```
does not.
Aug 25
prev sibling parent reply Dom DiSc <dominikus scherkl.de> writes:
On Saturday, 24 August 2024 at 17:10:39 UTC, Manu wrote:
 We need to have:

  trusted {
   some_code;
 }
We have: () trusted { some_code; }(); Which works kind of the same, beside it's ugly as hell. Until we have real trusted blocks, I use this (heavily).
Aug 25
parent reply Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 19:26, Dom DiSc via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:10:39 UTC, Manu wrote:
 We need to have:

  trusted {
   some_code;
 }
We have: () trusted { some_code; }(); Which works kind of the same, beside it's ugly as hell. Until we have real trusted blocks, I use this (heavily).
..."kind of the same" int normal_scope(int x, int y) { int t; // normal scope { t = x + y; } return t; } assume CS:.text._D7example12normal_scopeFiiZi push RBP mov RBP,RSP sub RSP,020h mov -010h[RBP],EDI mov -8[RBP],ESI mov dword ptr -018h[RBP],0 mov EAX,-8[RBP] add EAX,-010h[RBP] mov -018h[RBP],EAX leave ret int stupid_hack(int x, int y) { int t; // stupid hack () trusted { t = x + y; }(); return t; } assume CS:.text._D7example11stupid_hackFiiZi push RBP mov RBP,RSP sub RSP,020h mov -010h[RBP],EDI mov -8[RBP],ESI mov dword ptr -018h[RBP],0 mov RDI,RBP call qword ptr pure nothrow nogc trusted void example.stupid_hack(int, int).__lambda4() GOTPCREL[RIP] mov EAX,-018h[RBP] leave ret assume CS:.text.pure nothrow nogc trusted void example.stupid_hack(int, int).__lambda4() pure nothrow nogc trusted void example.stupid_hack(int, int).__lambda4(): push RBP mov RBP,RSP sub RSP,010h mov -8[RBP],RDI mov RAX,-8[RBP] mov ECX,-8[RAX] add ECX,-010h[RAX] mov -018h[RAX],ECX leave ret add [RAX],AL I hope we can agree that this is definitely not 'kind of the same'... And that's to say nothing about the damage it causes to the debug info, and the ability to breakpoint and step through the code in a sane way. Completely unacceptable hack. I won't give this pattern the dignity of my fingertips approval under any circumstances. It should not be legitimised.
Aug 25
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 25 August 2024 at 10:28:57 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:26, Dom DiSc via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:10:39 UTC, Manu wrote:
I hope we can agree that this is definitely not 'kind of the same'... And that's to say nothing about the damage it causes to the debug info, and the ability to breakpoint and step through the code in a sane way. Completely unacceptable hack. I won't give this pattern the dignity of my fingertips approval under any circumstances. It should not be legitimised.
was that with DMD without optimisations, or LDC with optimisations? If this is the former, that's probably that's to be expected.
Aug 25
next sibling parent Manu <turkeyman gmail.com> writes:
On Sun, 25 Aug 2024 at 20:46, Nicholas Wilson via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On Sunday, 25 August 2024 at 10:28:57 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:26, Dom DiSc via Digitalmars-d <
 digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:10:39 UTC, Manu wrote:
I hope we can agree that this is definitely not 'kind of the same'... And that's to say nothing about the damage it causes to the debug info, and the ability to breakpoint and step through the code in a sane way. Completely unacceptable hack. I won't give this pattern the dignity of my fingertips approval under any circumstances. It should not be legitimised.
was that with DMD without optimisations, or LDC with optimisations? If this is the former, that's probably that's to be expected.
Of course it's expected; that's why I'm saying it's a completely insane pattern. Code quality without optimisations is extremely important, for the reasons I said above; debugability, breakability, stepability. If you abandon debug code quality, you end up with rust; completely unworkable, with no tools except printf like it's 1980-something. The amount of time you spend working on debug code vs release code is like 10,000:1. Optimisation quality is important obviously, but debug code quality should never be overlooked.
Aug 25
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/25/2024 3:43 AM, Nicholas Wilson wrote:
 was that with DMD without optimisations, or LDC with optimisations?
 If this is the former, that's probably that's to be expected.
Kinda unfair to run dmd without optimizations and compare with ldc with? Here's dmd with -O -inline: ``` _D5test211stupid_hackFiiZi: 0000: 48 83 EC 18 sub RSP,018h 0004: 89 7C 24 08 mov 8[RSP],EDI 0008: 89 74 24 10 mov 010h[RSP],ESI 000c: C7 04 24 00 00 00 00 mov [RSP],0 0013: 03 74 24 08 add ESI,8[RSP] 0017: 89 34 24 mov [RSP],ESI 001a: 89 F0 mov EAX,ESI 001c: 48 83 C4 18 add RSP,018h 0020: C3 ret ```
Aug 25
prev sibling parent reply Dom DiSc <dominikus scherkl.de> writes:
On Sunday, 25 August 2024 at 10:28:57 UTC, Manu wrote:
 On Sun, 25 Aug 2024 at 19:26, Dom DiSc via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:

 On Saturday, 24 August 2024 at 17:10:39 UTC, Manu wrote:
 We need to have:

  trusted {
   some_code;
 }
We have: () trusted { some_code; }(); Which works kind of the same, beside it's ugly as hell. Until we have real trusted blocks, I use this (heavily).
..."kind of the same"
[...]
 I hope we can agree that this is definitely not 'kind of the 
 same'...

 And that's to say nothing about the damage it causes to the 
 debug info, and the ability to breakpoint and step through the 
 code in a sane way. Completely unacceptable hack. I won't give 
 this pattern the dignity of my fingertips approval under any 
 circumstances. It should not be legitimised.
Of course this should be better optimized, but I use this almost always only to call a system function, which get less overhead (or maybe even DMD is able to recognize a single call in a call, and optimize it away). But I fully agree, this should be replaced by true trusted blocks, the sooner the better.
Aug 25
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 8/25/24 13:06, Dom DiSc wrote:
 
 But I fully agree, this should be replaced by true trusted blocks, the 
 sooner the better.
You cannot have a "trusted block". It just does not work. The interface to any trusted thing has to be clearly delineated.
Aug 25
parent reply Manu <turkeyman gmail.com> writes:
On Mon, 26 Aug 2024, 07:56 Timon Gehr via Digitalmars-d, <
digitalmars-d puremagic.com> wrote:

 On 8/25/24 13:06, Dom DiSc wrote:
 But I fully agree, this should be replaced by true trusted blocks, the
 sooner the better.
You cannot have a "trusted block". It just does not work. The interface to any trusted thing has to be clearly delineated.
Well it obviously does work in some sense, because it's the de facto standard that people generally expect in numerous languages. People are going to have it one way or another; whether it's a ridiculous hack like `() trusted { ... }();` or otherwise. It's what other languages with this sort of thing do. We don't have a better offering to motivate people to deviate from their patterns. Resisting that degrades D.

Aug 25
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 8/26/24 03:16, Manu wrote:
 On Mon, 26 Aug 2024, 07:56 Timon Gehr via Digitalmars-d, <digitalmars- 
 d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:
 
     On 8/25/24 13:06, Dom DiSc wrote:
      >
      > But I fully agree, this should be replaced by true trusted
     blocks, the
      > sooner the better.
 
     You cannot have a "trusted block". It just does not work. The interface
     to any trusted thing has to be clearly delineated.
 
 
 Well it obviously does work in some sense, because it's the de facto 
 standard that people generally expect in numerous languages.
 ...
No, it's laid out differently.
 People are going to have it one way or another; whether it's a 
 ridiculous hack like `() trusted { ... }();` or otherwise. It's what 
 other languages with this sort of thing do.
 ...
No, e.g. Rust just does not distinguish trusted and untrusted safe functions.
 We don't have a better offering to motivate people to deviate from their 
 patterns.
 
 Resisting that degrades D.
 
It should not be called "trusted" then. Fundamentally, I agree on the substance (though probably, even better designs are possible). However, let's not invent another idiosyncratic keyword now.
Aug 26
parent reply Manu <turkeyman gmail.com> writes:
On Tue, 27 Aug 2024 at 04:06, Timon Gehr via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/26/24 03:16, Manu wrote:
 On Mon, 26 Aug 2024, 07:56 Timon Gehr via Digitalmars-d, <digitalmars-
 d puremagic.com <mailto:digitalmars-d puremagic.com>> wrote:

     On 8/25/24 13:06, Dom DiSc wrote:
      >
      > But I fully agree, this should be replaced by true trusted
     blocks, the
      > sooner the better.

     You cannot have a "trusted block". It just does not work. The
interface
     to any trusted thing has to be clearly delineated.


 Well it obviously does work in some sense, because it's the de facto
 standard that people generally expect in numerous languages.
 ...
No, it's laid out differently.
 People are going to have it one way or another; whether it's a
 ridiculous hack like `() trusted { ... }();` or otherwise. It's what
 other languages with this sort of thing do.
 ...
No, e.g. Rust just does not distinguish trusted and untrusted safe functions.
Rust does: unsafe { my_unsafe_code; } Perfect. unsafe { my_unsafe_code; } Perfect.
 We don't have a better offering to motivate people to deviate from their
 patterns.

 Resisting that degrades D.
It should not be called "trusted" then.
Absolutely, 100%. Fundamentally, I agree on the substance (though probably, even better
 designs are possible). However, let's not invent another idiosyncratic
 keyword now.
What 'another keyword' are you talking about? I think the problem that you're pointing at here is that we're using ` trusted` as a hack as if it meant what we all want; `unsafe` as in other languages. I understand that D's 'concept' is not quite like that, but nobody ever wanted the definition that D presents that I'm aware of, and all we want is an unsafe block. I don't see any reason for trusted... we only need safe to invoke the escape analysis, and then the ability to write an unsafe block inside a function; which people are abusing the trusted keyword to achieve. Why is the design we have better than the established designs?
Aug 26
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/26/2024 11:14 PM, Manu wrote:
 Why is the design we have better than the established designs?
The idea is not to simply mark code as unsafe, but to encapsulate it within a safe interface. It's a different approach. Yes, it is (initially) more work, but will pay off with more readability and maintainability. It is worth it. A number of D features have a bias towards encapsulation rather than interleaving things. The established designs are inferior, which is why we have D.
Aug 27
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Another approach is to compile the .c file with the -di flag, which will 
translate the C code to D code and emit it into .di file. It doesn't do this 
perfectly, but usually gets it good enough that any rough edges can be edited 
and fixed.

Then you can add  nogc and nothrow.
Aug 23
parent reply Manu <turkeyman gmail.com> writes:
On Sat, 24 Aug 2024 at 12:28, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 Another approach is to compile the .c file with the -di flag, which will
 translate the C code to D code and emit it into .di file. It doesn't do
 this
 perfectly, but usually gets it good enough that any rough edges can be
 edited
 and fixed.

 Then you can add  nogc and nothrow.
*sigh* ... I'm not going to do that. ImportC either works, or it doesn't... and it currently doesn't work, and I'm just going to move on. By far the easiest and least ridiculous feeling thing can do right now is to just prototype the occasional C library call inline immediately before the call, which is what I've moved on with. Fortunately my call volume is low.
Aug 24
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/24/2024 10:15 AM, Manu wrote:
 *sigh* ... I'm not going to do that.
 ImportC either works, or it doesn't... and it currently doesn't work, and I'm 
 just going to move on.
 By far the easiest and least ridiculous feeling thing can do right now is to 
 just prototype the occasional C library call inline immediately before the
call, 
 which is what I've moved on with.
 Fortunately my call volume is low.
You're right, that will work. I tried to make C calls default to trusted. Every single person in the community who voiced an opinion on it was dead set against it. But here you are, saying who cares what C does. I've got a cannon on one side aimed at me, and a flak gun on the other side. What do you suggest I do?
Aug 25
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 8/26/24 07:37, Walter Bright wrote:
 On 8/24/2024 10:15 AM, Manu wrote:
 *sigh* ... I'm not going to do that.
 ImportC either works, or it doesn't... and it currently doesn't work, 
 and I'm just going to move on.
 By far the easiest and least ridiculous feeling thing can do right now 
 is to just prototype the occasional C library call inline immediately 
 before the call, which is what I've moved on with.
 Fortunately my call volume is low.
You're right, that will work. I tried to make C calls default to trusted. Every single person in the community who voiced an opinion on it was dead set against it. But here you are, saying who cares what C does. I've got a cannon on one side aimed at me, and a flak gun on the other side. What do you suggest I do?
Marking every C function ` trusted` would not solve his problem. What would solve his problem is if ImportC supported C extensions that let you mark groups of declarations using D attributes. Then you can put those in the C files that you import.
Aug 26
next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, 27 Aug 2024 at 04:17, Timon Gehr via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/26/24 07:37, Walter Bright wrote:
 On 8/24/2024 10:15 AM, Manu wrote:
 *sigh* ... I'm not going to do that.
 ImportC either works, or it doesn't... and it currently doesn't work,
 and I'm just going to move on.
 By far the easiest and least ridiculous feeling thing can do right now
 is to just prototype the occasional C library call inline immediately
 before the call, which is what I've moved on with.
 Fortunately my call volume is low.
You're right, that will work. I tried to make C calls default to trusted. Every single person in the community who voiced an opinion on it was dead set against it. But here you are, saying who cares what C does. I've got a cannon on one side aimed at me, and a flak gun on the other side. What do you suggest I do?
Marking every C function ` trusted` would not solve his problem. What would solve his problem is if ImportC supported C extensions that let you mark groups of declarations using D attributes. Then you can put those in the C files that you import.
I could live with that... though I don't necessarily agree. I stand with Walter in principle on this; C prototypes should just be trusted. If you import and link a C library; you are 'trusting' it by definition. You're making use of some middleware, and you're not in control of the code... it is as it is, and you decided to use it anyway. You either trust it (and link it into your software), or you don't. Nobody gains anything from causing a hassle every time you want to call into a library that you've opted in to. The *whole point* is to use the library; what possible advantage could there be to making that a pain in the arse? There's one edge case I can think of; source-available functions. If the C source is available in a header, then we can infer safe-ty rather than blindly trust it. Can we direct this thread back towards nothrow and nogc? This wasn't actually a conversation about trusted...
Aug 26
next sibling parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 27 August 2024 at 06:32:06 UTC, Manu wrote:
 On Tue, 27 Aug 2024 at 04:17, Timon Gehr via Digitalmars-d < 
 digitalmars-d puremagic.com> wrote:

 On 8/26/24 07:37, Walter Bright wrote:
 On 8/24/2024 10:15 AM, Manu wrote:
 *sigh* ... I'm not going to do that.
 ImportC either works, or it doesn't... and it currently 
 doesn't work,
 and I'm just going to move on.
 By far the easiest and least ridiculous feeling thing can 
 do right now
 is to just prototype the occasional C library call inline 
 immediately
 before the call, which is what I've moved on with.
 Fortunately my call volume is low.
You're right, that will work. I tried to make C calls default to trusted. Every single person in the community who voiced an opinion on it was dead set against it. But here you are, saying who cares what C does. I've got a cannon on one side aimed at me, and a flak gun on the other side. What do you suggest I do?
Marking every C function ` trusted` would not solve his problem. What would solve his problem is if ImportC supported C extensions that let you mark groups of declarations using D attributes. Then you can put those in the C files that you import.
I could live with that... though I don't necessarily agree. I stand with Walter in principle on this; C prototypes should just be trusted.
Again, a big NO
 If you import and link a C library; you are 'trusting' it by 
 definition.
Again, a big NO
 You're making use of some middleware, and you're not in control
 of the code... it is as it is, and you decided to use it anyway.
trusted relates to _memory safety_, nothing more, it's not about the distinction between crap codebases and excellent codebases. You are in control of reading the documentation of the external C function You are in control of understanding that some C function is NOT memory safe. You are in control that you CAN'T call the function from safe code without sanitising its input with some other trusted D function. You still mix memory safety with other completely different things.
 You either trust it (and link it into your software), or you 
 don't.
That's nonsense, again, see above
 Nobody gains anything from causing a hassle every time you want 
  to call
 into a library that you've opted in to. The *whole point* is to 
 use the
 library; what possible advantage could there be to making that 
 a pain in the arse?
Best effort towards memory safety. If you don't care and are searching for no hassles just don't use safe at all in your codebase
 Can we direct this thread back towards nothrow and  nogc? This 
 wasn't actually a conversation about  trusted...
It seems to me actually that there's still a lot of confusion about safe / trusted / system from time to time, as your thread is showing. So I think this digression is just prophylaxis against the trusted virus :-P
Aug 27
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/26/2024 11:32 PM, Manu wrote:
 Can we direct this thread back towards nothrow and  nogc?
 This wasn't actually a conversation about  trusted...
They are closely related enough that the same issues apply.
Aug 27
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/26/2024 11:12 AM, Timon Gehr wrote:
 Marking every C function ` trusted` would not solve his problem. What would 
 solve his problem is if ImportC supported C extensions that let you mark
groups 
 of declarations using D attributes. Then you can put those in the C files that 
 you import.
People are very reluctant to modify their C files, as they usually come from outside their organization. I've done a lot of work on ImportC to get it to work with unmodified C code. Note that Manu rejected my suggestion to use ImportC to generate .di files of the function prototypes, and then edit in the attributes. This is equivalent to your suggestion.
Aug 27
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 8/28/24 00:04, Walter Bright wrote:
 On 8/26/2024 11:12 AM, Timon Gehr wrote:
 Marking every C function ` trusted` would not solve his problem. What 
 would solve his problem is if ImportC supported C extensions that let 
 you mark groups of declarations using D attributes. Then you can put 
 those in the C files that you import.
People are very reluctant to modify their C files, as they usually come from outside their organization. I've done a lot of work on ImportC to get it to work with unmodified C code. Note that Manu rejected my suggestion to use ImportC to generate .di files of the function prototypes, and then edit in the attributes. This is equivalent to your suggestion.
No, this is completely different to my suggestion.
Aug 29
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Mon, 26 Aug 2024 at 15:41, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/24/2024 10:15 AM, Manu wrote:
 *sigh* ... I'm not going to do that.
 ImportC either works, or it doesn't... and it currently doesn't work,
and I'm
 just going to move on.
 By far the easiest and least ridiculous feeling thing can do right now
is to
 just prototype the occasional C library call inline immediately before
the call,
 which is what I've moved on with.
 Fortunately my call volume is low.
You're right, that will work. I tried to make C calls default to trusted. Every single person in the community who voiced an opinion on it was dead set against it. But here you are, saying who cares what C does. I've got a cannon on one side aimed at me, and a flak gun on the other side. What do you suggest I do?
We're not talking about trusted, we're talking about nothrow and nogc... the community voices that have presented here so far seem to agree; I don't think you're being pressured by competing opinions. I *suggest* you make the change, OR consider adding a command line arg to override it; and optionally make that argument apply to an isolated source tree if you're feeling particularly conservative. (I wouldn't do that, but the more conservative might consider it) The situation as it stands is a literal tragedy... you put a lot of work into this. Realistically speaking; C calls are not going to throw or gcalloc. You can contrive a situation, but you have to go miles out of your way to arrange that. Any C library that operates via callbacks will have an API that also has those attributes attached; in a nothrow nogc C interface, the user can't provide a callback that can throw or GC. The only risk vector is as you described earlier; *mixed use* of the API and using a binary back-door. The whole point of ImportC, is to use the API. And if someone does a binary-back-door... who cares? That's called a BUG. They're playing with fire already! C doesn't have any such type safety, and they shouldn't expect it to. They know what they did; they did it intentionally, surely knew what the risk factors were, and they are naturally expected to not write such bugs into their program.
Aug 26
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/26/2024 10:54 PM, Manu wrote:
 The whole point of ImportC, is to use the API.
Initially, yes. But the mission creep has already happened, and why not?
 And if someone does a binary-back-door... who cares? That's called a BUG. 
 They're playing with fire already! C doesn't have any such type safety, and
they 
 shouldn't expect it to.
 They know what they did; they did it intentionally, surely knew what the risk 
 factors were, and they are naturally expected to not write such bugs into
their 
 program.
The author of the C code likely has no idea that the caller from D exists let alone that it would require that the C code not call any D functions.
Aug 27
parent reply Manu <turkeyman gmail.com> writes:
On Wed, 28 Aug 2024 at 08:12, Walter Bright via Digitalmars-d <
digitalmars-d puremagic.com> wrote:

 On 8/26/2024 10:54 PM, Manu wrote:
 The whole point of ImportC, is to use the API.
Initially, yes. But the mission creep has already happened, and why not?
Where has it happened? The mission isn't even off the ground. I thought I'd have a go, and it's a complete non-starter. So no, I really just want to use the C API; it's called ImportC; surely that's _literally the point_.
 And if someone does a binary-back-door... who cares? That's called a BUG.
 They're playing with fire already! C doesn't have any such type safety,
and they
 shouldn't expect it to.
 They know what they did; they did it intentionally, surely knew what the
risk
 factors were, and they are naturally expected to not write such bugs
into their
 program.
The author of the C code likely has no idea that the caller from D exists let alone that it would require that the C code not call any D functions.
The author of the library expects you to use the library via the API they provide... their API is C code; if C code is nothrow nogc, then the callback you provide is necessarily nothrow and nogc also. I really can't see the fuss here...
Aug 28
next sibling parent reply "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 29/08/2024 1:41 AM, Manu wrote:
      > And if someone does a binary-back-door... who cares? That's
     called a BUG.
      > They're playing with fire already! C doesn't have any such type
     safety, and they
      > shouldn't expect it to.
      > They know what they did; they did it intentionally, surely knew
     what the risk
      > factors were, and they are naturally expected to not write such
     bugs into their
      > program.
 
     The author of the C code likely has no idea that the caller from D
     exists let
     alone that it would require that the C code not call any D functions.
 
 
 The author of the library expects you to use the library via the API 
 they provide... their API is C code; if C code is nothrow  nogc, then 
 the callback you provide is necessarily nothrow and  nogc also.
 I really can't see the fuss here...
I've just had a thought, an old idea of mine is called contract invalidation. What it does is if you have an argument for say a function pointer that doesn't have an attribute, the attribute comes off the called function. Useful for opApply, but it might be perfect here too. ImportC can default to nothrow, nogc, system. If you pass in a callback that throws or uses GC, then so does the function. Its system, so compiler isn't making any guarantees in terms of safety anyway. I know Timon wants something more powerful than this, but... it could work here.
Aug 28
parent Manu <turkeyman gmail.com> writes:
On Wed, 28 Aug 2024 at 23:52, Richard (Rikki) Andrew Cattermole via
Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On 29/08/2024 1:41 AM, Manu wrote:
      > And if someone does a binary-back-door... who cares? That's
     called a BUG.
      > They're playing with fire already! C doesn't have any such type
     safety, and they
      > shouldn't expect it to.
      > They know what they did; they did it intentionally, surely knew
     what the risk
      > factors were, and they are naturally expected to not write such
     bugs into their
      > program.

     The author of the C code likely has no idea that the caller from D
     exists let
     alone that it would require that the C code not call any D functions.


 The author of the library expects you to use the library via the API
 they provide... their API is C code; if C code is nothrow  nogc, then
 the callback you provide is necessarily nothrow and  nogc also.
 I really can't see the fuss here...
I've just had a thought, an old idea of mine is called contract invalidation. What it does is if you have an argument for say a function pointer that doesn't have an attribute, the attribute comes off the called function. Useful for opApply, but it might be perfect here too. ImportC can default to nothrow, nogc, system. If you pass in a callback that throws or uses GC, then so does the function. Its system, so compiler isn't making any guarantees in terms of safety anyway. I know Timon wants something more powerful than this, but... it could work here.
That's an interesting idea in other contexts; but I don't see it's relevant here. This is not an engineering problem that needs to be solved.
Aug 28
prev sibling parent Lance Bachmeier <no spam.net> writes:
On Wednesday, 28 August 2024 at 13:41:47 UTC, Manu wrote:

 The author of the C code likely has no idea that the caller 
 from D exists
 let
 alone that it would require that the C code not call any D 
 functions.
The author of the library expects you to use the library via the API they provide... their API is C code; if C code is nothrow nogc, then the callback you provide is necessarily nothrow and nogc also. I really can't see the fuss here...
Strong agree. While technically Walter is correct, it would require doing something that doesn't make a lot of sense, at the cost of what started this thread.
Aug 28
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, August 26, 2024 11:54:06 PM MDT Manu via Digitalmars-d wrote:
 We're not talking about  trusted, we're talking about nothrow and  nogc...
 the community voices that have presented here so far seem to agree; I don't
 think you're being pressured by competing opinions.
 I *suggest* you make the change, OR consider adding a command line arg to
 override it; and optionally make that argument apply to an isolated source
 tree if you're feeling particularly conservative. (I wouldn't do that, but
 the more conservative might consider it)
 The situation as it stands is a literal tragedy... you put a lot of work
 into this.

 Realistically speaking; C calls are not going to throw or gcalloc. You can
 contrive a situation, but you have to go miles out of your way to arrange
 that.
 Any C library that operates via callbacks will have an API that also has
 those attributes attached; in a nothrow  nogc C interface, the user can't
 provide a callback that can throw or GC. The only risk vector is as you
 described earlier; *mixed use* of the API and using a binary back-door.
 The whole point of ImportC, is to use the API.

 And if someone does a binary-back-door... who cares? That's called a BUG.
 They're playing with fire already! C doesn't have any such type safety, and
 they shouldn't expect it to.
 They know what they did; they did it intentionally, surely knew what the
 risk factors were, and they are naturally expected to not write such bugs
 into their program.
Yeah. We already have the ability to declare C bindings without importC. The normal use case for importC is to "import" actual C code without having to write and maintain a bunch of bindings, and realistically, that code is going to be implemented as nothrow and nogc, because normal C code is nothrow and nogc. Yes, it's _technically_ possible to have C functions using D features and import them with importC, but who is actually going to do that in practice? Not treating importC functions as nothrow and nogc is hurting the intended use case for importC without really enabling much. It seems to me that if anyone really wants to have C code using the GC or throwing exceptions, they can just use C bindings like they've been able to do for many years, whereas having importC assume nothrow and nogc will make the feature work much better for the problem that it's actually trying to solve. If need be, we can put it as a caveat in the importC documentation that extern(C) functions implemented with D which should not be treated as nothrow or nogc cannot be used with importC, but I question that anyone will ever feel constrained by that. The current situation is a pessimization which makes importC worse for its actual use case for fear of causing problems for a use case which might not even ever exist in practice. It seems to me that it would be much better to just assume nothrow and nogc and document that behavior, allowing the few (if any) who want something else to just write their own C bindings. As it is, they're probably writing their own extern(C) functions anyway rather than looking to import C headers, because their code is D rather than C. - Jonathan M Davis
Aug 26
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/26/2024 11:29 PM, Jonathan M Davis wrote:
 Yeah. We already have the ability to declare C bindings without importC. The
 normal use case for importC is to "import" actual C code without having to
 write and maintain a bunch of bindings, and realistically, that code is
 going to be implemented as nothrow and  nogc, because normal C code is
 nothrow and  nogc. Yes, it's _technically_ possible to have C functions
 using D features and import them with importC, but who is actually going to
 do that in practice?
Anyone adding D functions to a C code base. This is the "gradual migration" strategy. I've used it, and C/C++ users do it a lot between C and C++.
 Not treating importC functions as nothrow and  nogc is hurting the intended
 use case for importC without really enabling much. It seems to me that if
 anyone really wants to have C code using the GC or throwing exceptions, they
 can just use C bindings like they've been able to do for many years, whereas
 having importC assume nothrow and  nogc will make the feature work much
 better for the problem that it's actually trying to solve.
You're right, until someone does it and it breaks in some bizarre way. You and Manu are right, this is a problem. There's got to be a better, correct solution.
Aug 27
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/19/2024 5:24 PM, Gregor Mückl wrote:
 The C function interface has no notion of C++ exceptions (AFAIK it's following
C 
 specs, not C++),
C's setjmp/longjmp use the same exception throwing method.
Aug 21
parent reply Dukc <ajieskola gmail.com> writes:
Walter Bright kirjoitti 21.8.2024 klo 22.42:
 On 8/19/2024 5:24 PM, Gregor Mückl wrote:
 The C function interface has no notion of C++ exceptions (AFAIK it's 
 following C specs, not C++),
C's setjmp/longjmp use the same exception throwing method.
OTOH a C function `longjmp`ing is like a D function throwing unrecoverable errors. It's still `nothrow`, it's just that `setjmp`ing (like catching unrecoverables) is unsafe, since it can break the D type system by skipping destructors and `finally` blocks.
Aug 23
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/23/2024 11:23 PM, Dukc wrote:
 OTOH a C function `longjmp`ing is like a D function throwing unrecoverable 
 errors. It's still `nothrow`, it's just that `setjmp`ing (like catching 
 unrecoverables) is unsafe, since it can break the D type system by skipping 
 destructors and `finally` blocks.
The fact that C++ compilers that translate C++ to C code use setjmp/longjmp to implement EH is sufficient to explain that. If that's not enough, take a look at how it is implemented. It uses the same EH scheme as C++ does.
Aug 25
prev sibling next sibling parent Tim <tim.dlang t-online.de> writes:
On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was 
 surprised
 when I immediately received a sea of errors calling the C 
 symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC would 
 be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on 
 this, and then the moment I try and use something I encounter a 
 showstopping oversight >_<
ImportC supports the following syntax for `nothrow` functions: ``` __declspec(nothrow) void f() { } __attribute__((nothrow)) void f2() { } ``` Unfortunately this only works for `nothrow` and not ` nogc` and needs to be added to every declaration.
Aug 18
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
C code can call C++ code which can throw.

C code can also call D code with can throw and/or gc.

D attributes are all subtractive, meaning they remove capability from the
called 
function. But C code can do anything.
Aug 21
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Wednesday, 21 August 2024 at 19:41:00 UTC, Walter Bright wrote:
 C code can call C++ code which can throw.
Does this actually interact with D code properly? Like, does this work? ```d void fn() nothrow { try { cFunctionThatCallsCplusplusThatThrows(); } catch(Exception e) { } } ```
 C code can also call D code with can throw and/or gc.

 D attributes are all subtractive, meaning they remove 
 capability from the called function. But C code can do anything.
I don't think you understand the problem here. `printf` doesn't use the GC. It doesn't throw. This is not conjecture or philosophy, it objectively, explicitly does not do this. Yet, if I use importC, I can't call `printf` from a `nothrow nogc` function. Consider that I don't want to have to mark a function as using the GC and throwing because *`printf` made me do it*. If I import `core.stdc.stdio`, then I can use it, but that basically means importC has failed if you have to do that. In addition, I have already run into situations where I import C code that includes stdio.h, so even if I want to use `core.stdc.stdio`, I can't. This must be fixed. There is no other option. If this is not fixed, you cannot use importC to actually import C. https://issues.dlang.org/show_bug.cgi?id=23812 -Steve
Aug 26
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/26/2024 10:26 AM, Steven Schveighoffer wrote:
 Does this actually interact with D code properly? Like, does this work?
Nope. D doesn't emit EH stack frames for nothrow functions.
 I don't think you understand the problem here.
 
 `printf` doesn't use the GC. It doesn't throw. This is not conjecture or 
 philosophy, it objectively, explicitly does not do this.
I know everything there is to know about printf. I've implemented a Standard compliant one :-) I've also proposed an exemption to make printf usable from safe code.
 This must be fixed. There is no other option. If this is not fixed, you cannot 
 use importC to actually import C.
I remember when you were adamantly against C code defaulting to trusted. Assuming nothrow and nogc is in the same category. I agree this is a problem.
Aug 27
next sibling parent Bruce Carneal <bcarneal gmail.com> writes:
On Tuesday, 27 August 2024 at 22:22:12 UTC, Walter Bright wrote:
 On 8/26/2024 10:26 AM, Steven Schveighoffer wrote:
 Does this actually interact with D code properly? Like, does 
 this work?
...
 I remember when you were adamantly against C code defaulting to 
  trusted. Assuming nothrow and  nogc is in the same category.
WRT trusted by default, lets not go there again, please. It was a heated and extremely lopsided argument last time around and things have not gotten better for the contrarian viewpoint since.
 I agree this is a problem.
Yes. I like the .di solution proposed for nothrow/ nogc . I used .di a while back to "port" cuda.h , but maybe there's something even better.
Aug 27
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Tuesday, 27 August 2024 at 22:22:12 UTC, Walter Bright wrote:
 On 8/26/2024 10:26 AM, Steven Schveighoffer wrote:
 I don't think you understand the problem here.
 
 `printf` doesn't use the GC. It doesn't throw. This is not 
 conjecture or philosophy, it objectively, explicitly does not 
 do this.
I know everything there is to know about printf. I've implemented a Standard compliant one :-)
If you are arguing it can possibly use the GC or throw an exception, then I'd like to hear more about how the standard allows that!
 I've also proposed an exemption to make printf usable from 
  safe code.
`printf` is already usable from ` safe` code through interpolation overloads.
 This must be fixed. There is no other option. If this is not 
 fixed, you cannot use importC to actually import C.
I remember when you were adamantly against C code defaulting to trusted.
I still am.
 Assuming nothrow and  nogc is in the same category.
It's not. C doesn't have a GC nor does it have an ability to throw D exceptions, whereas it can possibly have a safe interface. But this isn't even close to the same problem anyway -- what we were talking about with "make C prototypes safe by default" were *`extern(C)` prototypes written in D*. That is, these weren't in the C language, which has no concept of marking for any D attributes. It is a bad default, and probably just kills D as a memory safe language, but it could at least be worked around if you cared enough. The problem here isn't the default, the problem is the inability to mark because the language doesn't support it. While making the default ` nogc nothrow` is at least consistent with C (as is making C functions ` system`), a problem with that approach is that unlike ` system`, there is no attribute to cancel ` nogc` or `nothrow`. So if we made the default attributes for importC code `nothrow nogc`, then there would be no way to cancel that default.
 I agree this is a problem.
Yes, it is causing problems with people who want to write correct attributes in their D code. As I linked to in my project, I have to leave my D code that does not throw or use the GC unmarked, because of `printf` from importC. This makes importC unusable unless you just don't care about any attributes. -Steve
Aug 27
prev sibling next sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was 
 surprised
 when I immediately received a sea of errors calling the C 
 symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC would 
 be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...
I found the same. I'm still using ImportC on my project, temporarily as -betterC and commenting out the ` nogc nothrow` attributes but this must be fixed at some point. https://issues.dlang.org/show_bug.cgi?id=23812 https://github.com/schveiguy/draylib/blob/7de20a4cd2359d4df01acc159f953aa6dbe04347/source/raylib/rtext.d#L3 -Steve
Aug 26
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was 
 surprised
 when I immediately received a sea of errors calling the C 
 symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC would 
 be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on 
 this, and then the moment I try and use something I encounter a 
 showstopping oversight >_<
I missed most of this somehow, but after reading the posts in this thread, here's what I think: * Automatically making declarations obtained via ImportC default to ` trusted` is a terrible idea. * Making declarations obtained via ImportC uncallable from D code annotated with ` trusted nothrow` with no escape hatch is also a terrible idea. This should work: trusted { import mycfile; } Or, at least, *something* like this should work. I don't think it's acceptable to ask people to have to translate (by hand or otherwise) C code to be able to add attributes to it. Whatever attributes my D code has or not, I should be able to call C code via ImportC if I'm convinced the C code is compatible. *Especially* for ` nogc nothrow`. Could a "C" function throw? Yes. Is that going to be common? No. The same goes for allocating with the GC or not.
Aug 29
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 8/29/24 18:01, Atila Neves wrote:
 * Making declarations obtained via ImportC uncallable from D code 
 annotated with ` trusted nothrow` with no escape hatch is also a 
 terrible idea.
https://forum.dlang.org/post/v0urjh$58i$1 digitalmars.com
Aug 29
prev sibling parent reply Tim <tim.dlang t-online.de> writes:
On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was 
 surprised
 when I immediately received a sea of errors calling the C 
 symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC would 
 be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on 
 this, and then the moment I try and use something I encounter a 
 showstopping oversight >_<
Here is a pull request for a new ImportC pragma, which allows to set `nothrow`, ` nogc` and `pure`: https://github.com/dlang/dmd/pull/16820
Aug 29
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Thursday, 29 August 2024 at 18:39:23 UTC, Tim wrote:
 On Sunday, 18 August 2024 at 04:43:34 UTC, Manu wrote:
 I just tried using ImportC for the first time ever, but I was 
 surprised
 when I immediately received a sea of errors calling the C 
 symbols from
 `nothrow  nogc` functions.
 My entire program is `nothrow  nogc`... I assumed ImportC 
 would be
 supremely suitable in this context, but apparently not...

 Is there something I've missed? Is there a plan for this?
 I found just one single very short forum thread...

 This is classic D experience; so much work has been done on 
 this, and then the moment I try and use something I encounter 
 a showstopping oversight >_<
Here is a pull request for a new ImportC pragma, which allows to set `nothrow`, ` nogc` and `pure`: https://github.com/dlang/dmd/pull/16820
That requires modification of the C header. maybe using the attribute on the import is a better idea. Perhaps it is reasonable to have both, but I can imagine people not wanting to change their headers.
Aug 29
next sibling parent Tim <tim.dlang t-online.de> writes:
On Thursday, 29 August 2024 at 19:32:06 UTC, Stefan Koch wrote:
 That requires modification of the C header.
 maybe using the attribute on the import is a better idea.
 Perhaps it is reasonable to have both, but I can imagine people 
 not wanting to change their headers.
A modification of the header is not needed. It is enough to create a wrapper C file, which uses the pragma and includes the real header.
Aug 29
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On Thursday, 29 August 2024 at 19:32:06 UTC, Stefan Koch wrote:

 That requires modification of the C header.
 maybe using the attribute on the import is a better idea.
 Perhaps it is reasonable to have both, but I can imagine people 
 not wanting to change their headers.
You can't import headers with importC. You have to import C files. It is already necessary to create shim C files. So in the shim, you set the attributes, and import the headers you need. -Steve
Aug 29
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, August 29, 2024 1:50:10 PM MDT Steven Schveighoffer via 
Digitalmars-d wrote:
 On Thursday, 29 August 2024 at 19:32:06 UTC, Stefan Koch wrote:
 That requires modification of the C header.
 maybe using the attribute on the import is a better idea.
 Perhaps it is reasonable to have both, but I can imagine people
 not wanting to change their headers.
You can't import headers with importC. You have to import C files.
I was _really_ confused when I found that out, and I have no clue why it was done that way, since you don't normally #include .c files.
 It is already necessary to create shim C files.

 So in the shim, you set the attributes, and import the headers
 you need.
I do think that it's kind of dumb that you can't just import C headers with importC, since it seems like that would be the obvious thing that you're trying to do, but putting something in a shim file to affect the attributes isn't terrible when you already have to have one. - Jonathan M Davis
Aug 29
prev sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Thursday, 29 August 2024 at 19:50:10 UTC, Steven Schveighoffer 
wrote:
 On Thursday, 29 August 2024 at 19:32:06 UTC, Stefan Koch wrote:

 That requires modification of the C header.
 maybe using the attribute on the import is a better idea.
 Perhaps it is reasonable to have both, but I can imagine 
 people not wanting to change their headers.
You can't import headers with importC. You have to import C files. It is already necessary to create shim C files. So in the shim, you set the attributes, and import the headers you need.
Except you can't, because it's a C file, not D. What am I missing?
Aug 30
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Friday, 30 August 2024 at 10:20:07 UTC, Atila Neves wrote:
 On Thursday, 29 August 2024 at 19:50:10 UTC, Steven 
 Schveighoffer wrote:

 [snip]

 Except you can't, because it's a C file, not D. What am I 
 missing?
There was a lot of back and forth last year about having importC recognize .h files. Initial bug report here: https://issues.dlang.org/show_bug.cgi?id=23479 PRs about it: https://github.com/dlang/dmd/pulls?q=is%3Apr+23479+ The result is that if there is some C header that you want to use, you have to make a .c file to include it and then import that with importC.
Aug 30
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On Friday, 30 August 2024 at 10:20:07 UTC, Atila Neves wrote:
 On Thursday, 29 August 2024 at 19:50:10 UTC, Steven 
 Schveighoffer wrote:
 On Thursday, 29 August 2024 at 19:32:06 UTC, Stefan Koch wrote:

 That requires modification of the C header.
 maybe using the attribute on the import is a better idea.
 Perhaps it is reasonable to have both, but I can imagine 
 people not wanting to change their headers.
You can't import headers with importC. You have to import C files. It is already necessary to create shim C files. So in the shim, you set the attributes, and import the headers you need.
Except you can't, because it's a C file, not D. What am I missing?
From the original message that Stefan responded to:
 Here is a pull request for a new ImportC pragma, which allows 
 to set `nothrow`, ` nogc` and `pure`:
 https://github.com/dlang/dmd/pull/16820
-Steve
Aug 30