www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Is trusted the wrong direction?

reply Jesse Phillips <Jesse.K.Phillips+D gmail.com> writes:
Trusted code is the gateway for safe code to call into system 
code. The code is suppose to indicate the layer for verify safe 
use of system code.

 safe
void main()
{
     foo();
}

 trusted
void foo () {
     auto fish = 5;
     auto m = &fish;
}

However the code itself is system code and the language does not 
restrict operations like in safe.

I'm wondering if trusted should operate in the same world as 
safe, with the benefit of calling  system code.

Now the concern would be that there would just be an additional 
system function between safe and the desired system code.

Would it make sense to analyze some existing trusted methods and 
suggest a change?
Nov 08
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/9/19 1:55 AM, Jesse Phillips wrote:
 Trusted code is the gateway for safe code to call into system code. The 
 code is suppose to indicate the layer for verify safe use of system code.
 
  safe
 void main()
 {
      foo();
 }
 
  trusted
 void foo () {
      auto fish = 5;
      auto m = &fish;
 }
 
 However the code itself is system code and the language does not 
 restrict operations like in safe.
 
 I'm wondering if trusted should operate in the same world as safe, with 
 the benefit of calling  system code.
 
 Now the concern would be that there would just be an additional system 
 function between safe and the desired system code.
 
 Would it make sense to analyze some existing trusted methods and suggest 
 a change?
I wrote an article about trusted. It's definitely overused in a lot of places. I won't rehash what I said in the article, so please have a read. When to use trusted is not always a straightforward and easy set of rules. https://dlang.org/blog/2016/09/28/how-to-write-trusted-code-in-d/ -Steve
Nov 09
parent reply Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
I always thought trusted functions shouldn't be a thing. Almost 
never a whole function need to be trusted, but only a few lines 
of code. What we need instead are trusted blocks. Those can be 
simulated with anonymous nested functions, but the syntax is ugly 
as hell while complete trusted functions should be forbidden.
Nov 09
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Saturday, November 9, 2019 9:22:05 AM MST Dominikus Dittes Scherkl via 
Digitalmars-d wrote:
 I always thought trusted functions shouldn't be a thing. Almost
 never a whole function need to be trusted, but only a few lines
 of code. What we need instead are trusted blocks. Those can be
 simulated with anonymous nested functions, but the syntax is ugly
 as hell while complete trusted functions should be forbidden.
Really, the fact that trusted is at the function level is just an unnecessary complication. From the caller's perspective, safe and trusted are identical. Whether the function has been vetted for memory safety by the compiler or by a programmer doesn't matter to the caller. So, the fact that there's an API difference just causes problems (though fortunately, it's really just metaprogramming that's negatively affected by it). I think that it would definitely be a good idea to add trusted blocks of some kind to the language, but what that should look like is an open question. I also don't know how easy it would be to convince Walter to accept such a DIP, since I don't know what he currently thinks about the issue. For the moment, using trusted lambdas is the best that we have, but yeah, it's ridiculously verbose and ugly. It also can cause bugs when you forget to call the lambda. Either way, using trusted on non-lambda functions should usually be discouraged. Sometimes, it does make sense to use trusted on large blocks of code, but most of the time, it doesn't, and it makes it a lot harder to figure out what exactly was system that needed to be trusted when a lot of code was trusted at once. - Jonathan M Davis
Nov 09
parent Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Saturday, 9 November 2019 at 18:33:09 UTC, Jonathan M Davis 
wrote:
 Really, the fact that  trusted is at the function level is just 
 an unnecessary complication. From the caller's perspective, 
  safe and  trusted are identical.
Yes. Three state enums are always clumsy, especially so if they are useless - and even not obviously so but you have to think about it every time to figure that out.
 Sometimes, it does make sense to use  trusted on large blocks 
 of code, but most of the time, it doesn't,
Really almost never.
 and it makes it a lot harder to figure out what exactly was 
  system that needed to be  trusted when a lot of code was 
  trusted at once.
And even if that is really necessary, it would be no problem to declare a trusted block containing the whole function body. But from the outside the function is safe, that's all that matters to the caller.
Nov 09
prev sibling next sibling parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Saturday, 9 November 2019 at 16:22:05 UTC, Dominikus Dittes 
Scherkl wrote:
 I always thought trusted functions shouldn't be a thing. Almost 
 never a whole function need to be trusted, but only a few lines 
 of code. What we need instead are trusted blocks. Those can be 
 simulated with anonymous nested functions, but the syntax is 
 ugly as hell while complete trusted functions should be 
 forbidden.
Had an idea how to make more nice trusted blocks: ```d import std.stdio; T trusted (T)(T delegate() system dg) trusted { return dg(); } void main() safe { writeln("Hello D"); ({ writeln("C ", cast(void*) 1); }).trusted; } ``` looks a lot better than anonymous functions that are called right away after being defined. Best regards, Alexandru.
Nov 09
next sibling parent reply Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Saturday, 9 November 2019 at 20:38:47 UTC, Alexandru Ermicioi 
wrote:
 ```d
 import std.stdio;

 T trusted (T)(T delegate()  system dg)  trusted {
    return dg();
 }

 void main()  safe
 {
     writeln("Hello D");

     ({
         writeln("C ", cast(void*) 1);
     }).trusted;
 }
 ```

 looks a lot better than anonymous functions that are called 
 right away after being defined.
Hmm. Only slightly better syntax, but more important doesn't solve the other half of the problem: Forbidding non-lambda trusted functions. The syntax I would prefer for trusted blocks is simply trusted { } And everything between the trusted and the opening bracket (like function declarations) would be forbidden.
Nov 09
parent reply anon5886 <Me nowhe.re> writes:
On Saturday, 9 November 2019 at 21:13:22 UTC, Dominikus Dittes 
Scherkl wrote:
 On Saturday, 9 November 2019 at 20:38:47 UTC, Alexandru 
 Ermicioi wrote:
 ```d
 import std.stdio;

 T trusted (T)(T delegate()  system dg)  trusted {
    return dg();
 }

 void main()  safe
 {
     writeln("Hello D");

     ({
         writeln("C ", cast(void*) 1);
     }).trusted;
 }
 ```

 looks a lot better than anonymous functions that are called 
 right away after being defined.
Hmm. Only slightly better syntax, but more important doesn't solve the other half of the problem: Forbidding non-lambda trusted functions. The syntax I would prefer for trusted blocks is simply trusted { } And everything between the trusted and the opening bracket (like function declarations) would be forbidden.
Implementation is trivial... I got tit to work in a small hour but the problem is that for now the safety really only works at the function level so the function properties have to be patch for the duration of the trusted block. If someone wants to play a bit: --- From 8b4fc62610b54375b4824be28478dc11bd0023cf Mon Sep 17 00:00:00 2001 From: Me <Me nowhe.re> Date: Sun, 10 Nov 2019 00:27:25 +0100 Subject: [PATCH] Allow partial trusting of a BlockStatment - prevent usage of trusted delegates - allow to infere safe on func template containing these blocks - reduce the scope of the manual verification, supposed to be done for trusted --- src/dmd/parse.d | 23 ++++++++++++++++++++++- src/dmd/statement.d | 1 + src/dmd/statementsem.d | 17 +++++++++++++++++ 3 files changed, 40 insertions(+), 1 deletion(-) diff --git a/src/dmd/parse.d b/src/dmd/parse.d index 04cfc34e3..a57224b55 100644 --- a/src/dmd/parse.d +++ b/src/dmd/parse.d -5383,6 +5383,27 final class Parser(AST) : Lexer switch (token.value) { + case TOK.at: + auto n = peek(&token); + // trusted { } + // --- + // ScopeBlockStatement: + // Attributes? BlockStatement + // + if (n.value == TOK.identifier && isBuiltinAtAttribute(n.ident) == STC.trusted + && peekNext2 == TOK.leftCurly) + { + nextToken(); + nextToken(); + AST.Statement r = parseStatement(flags); + if (r.stmt == AST.STMT.Scope) + { + (cast(AST.ScopeStatement) r).stc = AST.STC.trusted; + // printf("trusted block parsed...\n"); + } + return r; + } + goto Ldeclaration; case TOK.identifier: { /* A leading identifier can be a declaration, label, or expression. -5565,7 +5586,7 final class Parser(AST) : Lexer case TOK.pure_: case TOK.ref_: case TOK.gshared: - case TOK.at: + //case TOK.at: case TOK.struct_: case TOK.union_: case TOK.class_: diff --git a/src/dmd/statement.d b/src/dmd/statement.d index ea05cb75a..4a081e5a9 100644 --- a/src/dmd/statement.d +++ b/src/dmd/statement.d -1009,6 +1009,7 extern (C++) class ScopeStatement : Statement { Statement statement; Loc endloc; // location of closing curly bracket + STC stc; // indicate the block temporary attributes, trusted only extern (D) this(const ref Loc loc, Statement statement, Loc endloc) { diff --git a/src/dmd/statementsem.d b/src/dmd/statementsem.d index 9276ffddb..ab1f1d234 100644 --- a/src/dmd/statementsem.d +++ b/src/dmd/statementsem.d -420,6 +420,9 private extern (C++) final class StatementSemanticVisitor : Visitor override void visit(ScopeStatement ss) { //printf("ScopeStatement::semantic(sc = %p)\n", sc); + + + if (ss.statement) { ScopeDsymbol sym = new ScopeDsymbol(); -427,6 +430,20 private extern (C++) final class StatementSemanticVisitor : Visitor sym.endlinnum = ss.endloc.linnum; sc = sc.push(sym); + // patch the scope to apply a temporary trusting for trusted { } + TypeFunction tf = sc.func ? sc.func.type.toTypeFunction() : null; + const TRUST tr = tf ? tf.trust : TRUST.init; + if (ss.stc == STC.trusted && tr != TRUST.init) + { + tf.trust = TRUST.trusted; + //printf("set block statement scope and parent func as trusted...\n"); + } + scope(exit) + { + if (tf) + tf.trust = tr; + } + Statements* a = ss.statement.flatten(sc); if (a) { -- 2.17.2 ---
Nov 09
parent Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Saturday, 9 November 2019 at 23:44:06 UTC, anon5886 wrote:
 On Saturday, 9 November 2019 at 21:13:22 UTC, Dominikus Dittes 
 Scherkl wrote:
 On Saturday, 9 November 2019 at 20:38:47 UTC, Alexandru 
 Ermicioi wrote:
 ```d
 import std.stdio;

 T trusted (T)(T delegate()  system dg)  trusted {
    return dg();
 }

 void main()  safe
 {
     writeln("Hello D");

     ({
         writeln("C ", cast(void*) 1);
     }).trusted;
 }
 ```

 looks a lot better than anonymous functions that are called 
 right away after being defined.
Hmm. Only slightly better syntax, but more important doesn't solve the other half of the problem: Forbidding non-lambda trusted functions. The syntax I would prefer for trusted blocks is simply trusted { } And everything between the trusted and the opening bracket (like function declarations) would be forbidden.
Implementation is trivial... I got this to work in a small hour but the problem is that for now the safety really only works at the function level so the function properties have to be patch for the duration of the trusted block.
I don't see a problem with that. It's exactly what I expected need to be done. Cool. I'll test this.
 - prevent usage of trusted delegates
 - allow to infere  safe on func template containing these blocks
 - reduce the scope of the manual verification, supposed to be 
 done for  trusted
Nov 09
prev sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Saturday, 9 November 2019 at 20:38:47 UTC, Alexandru Ermicioi 
wrote:
 On Saturday, 9 November 2019 at 16:22:05 UTC, Dominikus Dittes 
 Scherkl wrote:
 I always thought trusted functions shouldn't be a thing. 
 Almost never a whole function need to be trusted, but only a 
 few lines of code. What we need instead are trusted blocks. 
 Those can be simulated with anonymous nested functions, but 
 the syntax is ugly as hell while complete trusted functions 
 should be forbidden.
Had an idea how to make more nice trusted blocks: ```d import std.stdio; T trusted (T)(T delegate() system dg) trusted { return dg(); } void main() safe { writeln("Hello D"); ({ writeln("C ", cast(void*) 1); }).trusted; } ``` looks a lot better than anonymous functions that are called right away after being defined. Best regards, Alexandru.
That's exactly the kind of stuff to avoid, in my opinion: for a code reviewer point of view, this is just obfuscation, while the author should write trusted code in the more pedantic and clear way, to facilitate the reviewer analysis. The whole point, in trusted code, is bond to the fact that the code should just be just plain easy to manually be checked. /Paolo
Nov 10
parent reply Alexandru Ermicioi <alexandru.ermicioi gmail.com> writes:
On Sunday, 10 November 2019 at 11:08:29 UTC, Paolo Invernizzi 
wrote:
 On Saturday, 9 November 2019 at 20:38:47 UTC, Alexandru 
 Ermicioi wrote:
 On Saturday, 9 November 2019 at 16:22:05 UTC, Dominikus Dittes 
 Scherkl wrote:
 I always thought trusted functions shouldn't be a thing. 
 Almost never a whole function need to be trusted, but only a 
 few lines of code. What we need instead are trusted blocks. 
 Those can be simulated with anonymous nested functions, but 
 the syntax is ugly as hell while complete trusted functions 
 should be forbidden.
Had an idea how to make more nice trusted blocks: ```d import std.stdio; T trusted (T)(T delegate() system dg) trusted { return dg(); } void main() safe { writeln("Hello D"); ({ writeln("C ", cast(void*) 1); }).trusted; } ``` looks a lot better than anonymous functions that are called right away after being defined. Best regards, Alexandru.
That's exactly the kind of stuff to avoid, in my opinion: for a code reviewer point of view, this is just obfuscation, while the author should write trusted code in the more pedantic and clear way, to facilitate the reviewer analysis. The whole point, in trusted code, is bond to the fact that the code should just be just plain easy to manually be checked. /Paolo
How is this more obfuscating than: ```d (() trusted => writeln("C", cast(void*) 1))(); ``` In my case as reviewer, I'd find this example more hard to reason due to high amount of braces here. Original solution could work in existing version of d language. I' m not stating that changes to facilitate more clear and less trusted code should not be implemented on language level given library solution. Trusted blocks would make sense when we're trying to minimize amount of such code. On this note, if trusted is to be added to block statements, then all annotation related functionality should be considered whether to add or not to block statements or any type os statement in general to keep annotation behavior uniform. Best regards, Alexandru.
Nov 16
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Saturday, 16 November 2019 at 20:24:59 UTC, Alexandru Ermicioi 
wrote:
 On Sunday, 10 November 2019 at 11:08:29 UTC, Paolo Invernizzi 
 wrote:
 On Saturday, 9 November 2019 at 20:38:47 UTC, Alexandru 
 Ermicioi wrote:
 On Saturday, 9 November 2019 at 16:22:05 UTC, Dominikus 
 Dittes Scherkl wrote:
 I always thought trusted functions shouldn't be a thing. 
 Almost never a whole function need to be trusted, but only a 
 few lines of code. What we need instead are trusted blocks. 
 Those can be simulated with anonymous nested functions, but 
 the syntax is ugly as hell while complete trusted functions 
 should be forbidden.
Had an idea how to make more nice trusted blocks: ```d import std.stdio; T trusted (T)(T delegate() system dg) trusted { return dg(); } void main() safe { writeln("Hello D"); ({ writeln("C ", cast(void*) 1); }).trusted; } ``` looks a lot better than anonymous functions that are called right away after being defined. Best regards, Alexandru.
That's exactly the kind of stuff to avoid, in my opinion: for a code reviewer point of view, this is just obfuscation, while the author should write trusted code in the more pedantic and clear way, to facilitate the reviewer analysis. The whole point, in trusted code, is bond to the fact that the code should just be just plain easy to manually be checked. /Paolo
How is this more obfuscating than: ```d (() trusted => writeln("C", cast(void*) 1))(); ``` In my case as reviewer, I'd find this example more hard to reason due to high amount of braces here. Original solution could work in existing version of d language. I' m not stating that changes to facilitate more clear and less trusted code should not be implemented on language level given library solution. Trusted blocks would make sense when we're trying to minimize amount of such code.
I remember a discussion some time ago about the overusing of trusted-as-a-super-thin-wrapper around sytem calls, made by Andrei and Walter over some parts of Phobos, and that proposed work just walks towards that same direction. The point is, I'm against trusted blocks, as I think it's more clear to have a fundamental minimal aggregate of code functionality: the function, as it's right now, especially for a reviewer. The function has everything needed, documentation, contracts, everything is optimised, to clearly explain what should be the input, the output, and what's the intention of the code inside of if. The mere fact that a reviewer must pay attention not only to trusted, but 'trusted' as a template, or why not '__trusted', or 'this_is_trusted', and so on, it's just opening a can of worms when you review unfamiliar codebase. With a lesser impact, I think that this also holds with an implementation on language level. I underline, that are only personal opinions, I understand your point around the issue.
 On this note, if trusted is to be added to block statements, 
 then all annotation related functionality should be considered 
 whether to add or not to block statements or any type os 
 statement in general to keep annotation behavior uniform.
The difference is that pure, nothrow or nogc are mechanically checked, so if attributes are scattered around the codebase, also inside code blocks, there's not an impact: the compiler happily digests them. That's not true for trusted ... The introduction of pure, nothrow or nogc blocks can be an interesting option to investigate, especially for nogc, but I guess that the latter is linked to the GC vs allocator work (DIP 1025, to be clear). Cheers, Paolo
Nov 17
parent reply Dominikus Dittes Scherkl <dominikus scherkl.de> writes:
On Sunday, 17 November 2019 at 14:37:16 UTC, Paolo Invernizzi 
wrote:

 I remember a discussion some time ago about the overusing of 
  trusted-as-a-super-thin-wrapper around  sytem calls, made by 
 Andrei and Walter over some parts of Phobos, and that proposed 
 work just walks towards that same direction.
I fully agree that a library wrapper around trusted lambdas are a bad idea. They neither provide the desired short and readable syntax nor do they solve the three-state enum around safety. But the small compiler-change proposed by anon5886 is really very nice.
 The point is, I'm against  trusted blocks, as I think it's more 
 clear to have a fundamental minimal aggregate of code 
 functionality: the function, as it's right now, especially for 
 a reviewer.
This will not change. The function keeps it info: it is safe, so it has to provide a memory-safe interface. It's only sightly more obvious to the reviewer, because he doesn't have to remember that trusted is only and alias for safe, from the caller point of view.
 The mere fact that a reviewer must pay attention not only to 
  trusted, but 'trusted' as a template, or why not '__trusted', 
 or 'this_is_trusted', and so on, it's just opening a can of 
 worms when you review unfamiliar codebase.
But this is exactly NOT he case. If he reviews a function that is marked safe his alarm bells only need to ring, if the function contains a trusted block. Nothing else. There are no trusted templates or macros or other __-stuff anymore. But of course the whole function must be treated with care, if it contains a trusted block, no change there. But the parts that need to be trusted should be as sparse as possible, and a short and clear syntax helps in doing this. Editors can highlight trusted blocks heavily and ugly, so you will automatically try to keep those sections as small as possible. And no newbie is irritated anymore what this third thing between safe and system should be.
Nov 17
parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Sunday, 17 November 2019 at 16:41:16 UTC, Dominikus Dittes 
Scherkl wrote:
 On Sunday, 17 November 2019 at 14:37:16 UTC, Paolo Invernizzi 
 wrote:

 The point is, I'm against  trusted blocks, as I think it's 
 more clear to have a fundamental minimal aggregate of code 
 functionality: the function, as it's right now, especially for 
 a reviewer.
This will not change. The function keeps it info: it is safe, so it has to provide a memory-safe interface.
It's "safe" because a human inspected it and I'm trusting the human. That's a huge difference from "safe" because it was certified by the compiler automatically. I think that the internal trusted block simply hide this information, the caller need to check the body.
 It's only sightly more obvious to the reviewer, because he 
 doesn't have to remember that  trusted is only and alias for 
  safe, from the caller point of view.
Well, I hope that a reviewer checking trusted code knows the difference very well :-P
 The mere fact that a reviewer must pay attention not only to 
  trusted, but 'trusted' as a template, or why not '__trusted', 
 or 'this_is_trusted', and so on, it's just opening a can of 
 worms when you review unfamiliar codebase.
But this is exactly NOT he case. If he reviews a function that is marked safe his alarm bells only need to ring, if the function contains a trusted block. Nothing else. There are no trusted templates or macros or other __-stuff anymore. But of course the whole function must be treated with care, if it contains a trusted block, no change there. But the parts that need to be trusted should be as sparse as possible, and a short and clear syntax helps in doing this.
I think documentation, contract and a clear encapsulation in a function help the reviewer more ... but that's only my opinion, not a fact.
 Editors can highlight  trusted blocks heavily and ugly, so you 
 will automatically try to keep those sections as small as 
 possible.
I don't see anything ugly in trusted, it's simply a necessary form to connect safe to system.
 And no newbie is irritated anymore what this third thing 
 between safe and system should be.
I've not seen any complain on that in learn forum, but maybe you have a better visibility than me on that. I find the safe-trusted-system triade very intuitive, for sure more than other recently proposed D features. But again, I understand your point of trying to minimise the amount of trusted code around.
Nov 18
prev sibling parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 11/9/19 11:22 AM, Dominikus Dittes Scherkl wrote:
 I always thought trusted functions shouldn't be a thing. Almost never a 
 whole function need to be trusted, but only a few lines of code. What we 
 need instead are trusted blocks. Those can be simulated with anonymous 
 nested functions, but the syntax is ugly as hell while complete trusted 
 functions should be forbidden.
Yeah, it would be nicer. But the sad part is that a safe function with trusted pieces STILL has to be completely manually verified. Because a trusted lambda can muck with the guarantees of the safe parts inside the lambda. In essence, a function boundary is the correct place, because that's where the safety guarantees are defined. What a safe function with trusted parts DOES do, is help the focus of the review. You can look at the trusted lambda and reason about what possibly safety problems could arise from it, then check all the safe code to see if those cases happen. I'd still be in favor of a less verbose trusted block syntax. Not only for the brevity, but also because relying on the optimizer/inliner to properly write the code seems like a code smell. -Steve
Nov 11