www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Encapsulating trust

reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
Quite recently a lot of work has been done to make most of Phobos usable 
in  safe code.

While a very welcome effort, it caused a number of doubts in particular 
due to the boilerplate required to isolate a small amount of unsafe 
operations and slap " trusted" over it.

See e.g. Denis argument:
https://github.com/D-Programming-Language/phobos/pull/2465

There were proposals for language changes along the lines of having 
 trusted block alike to debug/version blocks, but nothing ever came out 
of them.

Without language support I decided it worth a shot to create a universal 
wrappers to establish a consistent convention. A use of such wrapper 
should indicate that a  system function call or language feature was 
hand-verified.

Names and complete set of primitives are up for debate, but here is the 
start:

https://gist.github.com/DmitryOlshansky/bc02f369c8a63818bd07

A bit of usage:

import core.stdc.string;
import trusted;

void main()  safe
{

     char[] msg = "Hello!".dup;
     char[] msg2 = msg;
     import trusted; // may also use static import for absolute clarity
     assert(call!memcmp(addrOf(msg[0]), addrOf(msg2[0]), msg.length) == 0);
}


What do you guys think?

-- 
Dmitry Olshansky
Aug 31 2014
next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Sunday, 31 August 2014 at 13:47:42 UTC, Dmitry Olshansky wrote:

 What do you guys think?


I'd say add "trusted" to those function names:
"trustedCall"
"trustedAddrOf"

Because:
- "call" could mean a lot of things. It's not imidiatly obvious 
that it is meant for being trusted.
- "addrOf" is *also* used as an alternative for "&", especially 
in generic code, when you need the address of an attribute, as 
that attribute could actually be a property function. EG:
auto p = addrOf(r.front);
Nothing here implies trust.

Also, implementation wise, wouldn't it be possible to instead 
make `call` a template aliases itself away to `fun`, but with 
different attributes? The `auto fun(Args)(auto ref Args args)` 
tube approach has the disadvantages that it:
- prevents return by ref
- fails perfect forwarding of lvalues into rvalues.
Aug 31 2014
next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 31 Aug 2014 20:36:59 +0000
monarch_dodra via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 On Sunday, 31 August 2014 at 13:47:42 UTC, Dmitry Olshansky wrote:

 What do you guys think?


=20
 I'd say add "trusted" to those function names:
 "trustedCall"
 "trustedAddrOf"


i first think like this too, but then i remembered that "call" is in
module "trusted", so one who can document the code is free to write
"trusted.call". and one who want something short can use simple "call".
Sep 01 2014
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
01-Sep-2014 00:36, monarch_dodra пишет:

 On Sunday, 31 August 2014 at 13:47:42 UTC, Dmitry Olshansky wrote:

 What do you guys think?


 I'd say add "trusted" to those function names:
 "trustedCall"
 "trustedAddrOf"


Would we ever use our module system? I agree though that being easily 
grep-able is useful for these functions.

assumeSafe?


 Because:
 - "call" could mean a lot of things. It's not imidiatly obvious that it
 is meant for being trusted.
 - "addrOf" is *also* used as an alternative for "&", especially in
 generic code, when you need the address of an attribute, as that
 attribute could actually be a property function. EG:
 auto p = addrOf(r.front);
 Nothing here implies trust.


Local import of 'trusted' right there might imply trust. Selective, 
preferably static.


 Also, implementation wise, wouldn't it be possible to instead make
 `call` a template aliases itself away to `fun`, but with different
 attributes? The `auto fun(Args)(auto ref Args args)` tube approach has
 the disadvantages that it:
 - prevents return by ref


auto ref return FTW


 - fails perfect forwarding of lvalues into rvalues.


?

void inc(ref int a)
{
     a += 1;
}
     int b;
     call!inc(b);
     assert(b == 1);

Works fine.

-- 
Dmitry Olshansky
Sep 01 2014
parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 1 September 2014 at 07:13:47 UTC, Dmitry Olshansky 
wrote:

 auto ref return FTW


I thought you had avoided that on purpose, in the sense that 
generic auto-ref input *and* output has been proven unsafe, in 
the sense that there are tons of ways for the compiler to 
accidently return a ref to something that is actually local.

unaryFun!"a[0]" or unaryFun"a.field" is a perfect example of that.
Or, say:
int tube(int a){return a;}
ref int tube(ref int a){return a;}
call!tube(5); //Here


 - fails perfect forwarding of lvalues into rvalues.


 ?

 void inc(ref int a)
 {
     a += 1;
 }
     int b;
     call!inc(b);
     assert(b == 1);

 Works fine.


Yes, but:
call!inc(5) will *also* succeed.

But background on this issue would be certain functions, such as 
"emplace" that could elide postblit entirely when asked to 
emplace from an rvalue. The issue is that such functions that use 
auto-ref have a tendency to lose that information.

We could use "std.algorithm.forward", but that function is 
currently too expensive.

That said, there are no real functions that would exploit this 
"perfect forwarding" anyways. I filed:
https://issues.dlang.org/show_bug.cgi?id=12683
https://issues.dlang.org/show_bug.cgi?id=12684
After Andrei filed:
https://issues.dlang.org/show_bug.cgi?id=12628

Which would be the first steps to really start making more 
efficient use of rvalues in D.
Sep 01 2014
prev sibling next sibling parent "Dicebot" <public dicebot.lv> writes:
I am not convinced it warrants dedicated helper as opposed to 
just fixing compiler to inline those lambdas.
Sep 01 2014
prev sibling next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Dmitry Olshansky"  wrote in message news:ltv91u$2mtc$1 digitalmars.com...


 Quite recently a lot of work has been done to make most of Phobos usable 
 in  safe code.

 While a very welcome effort, it caused a number of doubts in particular 
 due to the boilerplate required to isolate a small amount of unsafe 
 operations and slap " trusted" over it.

 See e.g. Denis argument:
 https://github.com/D-Programming-Language/phobos/pull/2465

 There were proposals for language changes along the lines of having 
  trusted block alike to debug/version blocks, but nothing ever came out of 
 them.

 Without language support I decided it worth a shot to create a universal 
 wrappers to establish a consistent convention. A use of such wrapper 
 should indicate that a  system function call or language feature was 
 hand-verified.



 What do you guys think?


I think this is an abuse of  trusted.  It takes unsafe operations, and 
re-presents them as  safe options by renaming them.  Now, you can do use 
 system things in  safe functions without the compiler detecting it.

Take the `addrOf` function for example.  This is equivalent to adding a new 
version of the '&' operation, which does exactly the same thing except it's 
allowed in  safe code.  The `addrOf` function should _not_ be  trusted, 
because when used from  safe code it can cause escape the address of a local 
etc, just like '&' can.

Because these functions violate the meaning of  trusted, that  trusted 
functions must be  safe although the compiler can't prove them  safe, you've 
increased the surface of  trusted.  Now an audit of  trusted code must 
include all functions that call `addrOf` and friends.

I don't think this is a good idea.  Each time  trusted is used, it should be 
on a function that is completely  safe to call.  I think this is worth more 
than the cost in verbosity.

Lambdas and nested functions are special in that they can't be called from 
other code, so they only have to be  safe in the context of the enclosing 
function.  They do still need to make sure they don't violate  safe, 
otherwise the entire enclosing function will need to be manually checked.

eg
void fun(int a)  safe
{
...
    p =  trusted () { return &a; }
...
}

This function is now essentially  trusted, because although the unsafe '&' 
operation was inside the trusted block, the  safe function now has a pointer 
it should not have been able to get.
Sep 01 2014
next sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 1 September 2014 at 16:36:04 UTC, Daniel Murphy wrote:

 I don't think this is a good idea.  Each time  trusted is used, 
 it should be on a function that is completely  safe to call.  I 
 think this is worth more than the cost in verbosity.

 Lambdas and nested functions are special in that they can't be 
 called from other code, so they only have to be  safe in the 
 context of the enclosing function.  They do still need to make 
 sure they don't violate  safe, otherwise the entire enclosing 
 function will need to be manually checked.

 eg
 void fun(int a)  safe
 {
 ...
    p =  trusted () { return &a; }
 ...
 }

 This function is now essentially  trusted, because although the 
 unsafe '&' operation was inside the trusted block, the  safe 
 function now has a pointer it should not have been able to get.


I feels like you are missing the point of the  trusted lambda 
construct, in that is meant to be used in generic code, where you 
know a *piece* of a function is provably safe (eg:  trusted), but 
not all of it: The rest of the code depends on the inferred 
attributes of the parameter-dependent code.

If your function is not generic, then just mark it as  trusted, 
and then that's that.

Another alternative I had proposed was one of being able to 
simply create blocks with attributes. EG:

void foo(T)(T t)
{
     t.doSomething(); //May or may not be safe.

     nothrow
     {
         ... //Do "critical" code that can't throw here.
     }

      trusted
     {
         ... //This slice of code is trusted.
     }

      safe  nogc
     {
         ... //Have the compiler enforce only  safe and  ngc code 
goes here.
     }
     return t;
}
Sep 01 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 1 September 2014 at 17:48:59 UTC, monarch_dodra wrote:

 I feels like you are missing the point of the  trusted lambda 
 construct, in that is meant to be used in generic code, where 
 you know a *piece* of a function is provably safe (eg: 
  trusted), but not all of it: The rest of the code depends on 
 the inferred attributes of the parameter-dependent code.

 If your function is not generic, then just mark it as  trusted, 
 and then that's that.


I totally disagree. Marking whole function  trusted (unless those 
are extern(C)) is an abomination we should try to get rid of. 
Trusted lambda must encapsulate minimal amount of code possible 
together with all data validation if necessary.Anything else 
simply does not scale with maintenance and is likely to introduce 
holes in  safe.
Sep 01 2014
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Monday, 1 September 2014 at 17:59:07 UTC, Dicebot wrote:

 On Monday, 1 September 2014 at 17:48:59 UTC, monarch_dodra 
 wrote:

 I feels like you are missing the point of the  trusted lambda 
 construct, in that is meant to be used in generic code, where 
 you know a *piece* of a function is provably safe (eg: 
  trusted), but not all of it: The rest of the code depends on 
 the inferred attributes of the parameter-dependent code.

 If your function is not generic, then just mark it as 
  trusted, and then that's that.


 I totally disagree. Marking whole function  trusted (unless 
 those are extern(C)) is an abomination we should try to get rid 
 of. Trusted lambda must encapsulate minimal amount of code 
 possible together with all data validation if 
 necessary.Anything else simply does not scale with maintenance 
 and is likely to introduce holes in  safe.


I meant it mostly in that the proposal to mark the entire 
function as  trusted isn't even *applicable* to template 
functions.

I agree with you.
Sep 01 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"monarch_dodra"  wrote in message 
news:wvslfmdauwupzioryhgq forum.dlang.org...


 I meant it mostly in that the proposal to mark the entire function as 
  trusted isn't even *applicable* to template functions.


That isn't what I was proposing.  Using a trusted nested or lambda function 
is fine, but it should not be possible to violate  safe by changing only the 
enclosing function.

eg making a trustedMemcpy is just as much a violation of  safe as marking 
the actual memcpy as  safe.  It still relies on the calling function being 
manually verified not to call it incorrectly.  If your calling function 
needs to be manually verified, it should be  trusted.  If it can't be 
manually verified, it should be  system.
Sep 01 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Monday, 1 September 2014 at 18:29:32 UTC, Daniel Murphy wrote:

 "monarch_dodra"  wrote in message 
 news:wvslfmdauwupzioryhgq forum.dlang.org...


 I meant it mostly in that the proposal to mark the entire 
 function as  trusted isn't even *applicable* to template 
 functions.


 That isn't what I was proposing.  Using a trusted nested or 
 lambda function is fine, but it should not be possible to 
 violate  safe by changing only the enclosing function.

 eg making a trustedMemcpy is just as much a violation of  safe 
 as marking the actual memcpy as  safe.  It still relies on the 
 calling function being manually verified not to call it 
 incorrectly.  If your calling function needs to be manually 
 verified, it should be  trusted.  If it can't be manually 
 verified, it should be  system.


Idea is that you move not only actual unsafe operation into 
 trusted lambda but also any additional context that makes it 
effectively  safe, separating it from the rest of the function.

It is not that trivial to spot though so some mistakes are 
inevitable.
Sep 01 2014
parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Dicebot"  wrote in message news:uwalapqtroynalabxtwo forum.dlang.org... 


 Idea is that you move not only actual unsafe operation into 
  trusted lambda but also any additional context that makes it 
 effectively  safe, separating it from the rest of the function.
 
 It is not that trivial to spot though so some mistakes are 
 inevitable.


Absolutely.  This thread's proposal is a complete departure from that idea.
Sep 01 2014
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
01-Sep-2014 20:36, Daniel Murphy пишет:

 "Dmitry Olshansky"  wrote in message news:ltv91u$2mtc$1 digitalmars.com...


 Quite recently a lot of work has been done to make most of Phobos
 usable in  safe code.

 While a very welcome effort, it caused a number of doubts in
 particular due to the boilerplate required to isolate a small amount
 of unsafe operations and slap " trusted" over it.

 See e.g. Denis argument:
 https://github.com/D-Programming-Language/phobos/pull/2465

 There were proposals for language changes along the lines of having
  trusted block alike to debug/version blocks, but nothing ever came
 out of them.

 Without language support I decided it worth a shot to create a
 universal wrappers to establish a consistent convention. A use of such
 wrapper should indicate that a  system function call or language
 feature was hand-verified.



 What do you guys think?


 I think this is an abuse of  trusted.  It takes unsafe operations, and
 re-presents them as  safe options by renaming them.  Now, you can do use
  system things in  safe functions without the compiler detecting it.

 Take the `addrOf` function for example.  This is equivalent to adding a
 new version of the '&' operation, which does exactly the same thing
 except it's allowed in  safe code.  The `addrOf` function should _not_
 be  trusted, because when used from  safe code it can cause escape the
 address of a local etc, just like '&' can.



 Because these functions violate the meaning of  trusted, that  trusted
 functions must be  safe although the compiler can't prove them  safe,
 you've increased the surface of  trusted.  Now an audit of  trusted code
 must include all functions that call `addrOf` and friends.


Only these that import stdx.trusted. Trivial to check.


 I don't think this is a good idea.  Each time  trusted is used, it
 should be on a function that is completely  safe to call.  I think this
 is worth more than the cost in verbosity.


I'd be damned but it's made precisely because taking address is 
frequently needed in a function that is otherwise  safe. Alternative of 
marking the whole thing as  trusted ain't going in the right direction.

Instead of writing the same ugly shit (pardon local functions and 
lambda+call) everywhere, let just compose a set of markers (2-3 
functions) that indicate something as trusted block.

Obviously one can easily abuse it, much like  trusted can be easily 
abuse today, especially with templates where it's easy to fail in trap 
of putting  trusted on the whole function and trust pretty much any 3rd 
party type to be safe.


 Lambdas and nested functions are special in that they can't be called
 from other code, so they only have to be  safe in the context of the
 enclosing function.  They do still need to make sure they don't violate
  safe, otherwise the entire enclosing function will need to be manually
 checked.


They have in common is lot of noise that distracts and obfuscates the 
thing that already needs our full attention making it both bigger and 
harder to follow.


 eg
 void fun(int a)  safe
 {
 ...
     p =  trusted () { return &a; }
 ...
 }

 This function is now essentially  trusted, because although the unsafe
 '&' operation was inside the trusted block, the  safe function now has a
 pointer it should not have been able to get.


Careful there - a trusted lambda must ensure that pointer is fine, 
although the means of getting it are  system. The same review-driven 
thing about  trusted remains.

-- 
Dmitry Olshansky
Sep 02 2014
parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Dmitry Olshansky"  wrote in message news:lu4jld$m0b$1 digitalmars.com...


 Only these that import stdx.trusted. Trivial to check.


Sure, if this type of function only exists in that module (and there are no 
public imports of it).  But you've still made it so you now have to inspect 
 safe functions for non- safe code.


 I'd be damned but it's made precisely because taking address is frequently 
 needed in a function that is otherwise  safe. Alternative of marking the 
 whole thing as  trusted ain't going in the right direction.


Of course marking the whole thing as  trusted is wrong.


 Instead of writing the same ugly shit (pardon local functions and 
 lambda+call) everywhere, let just compose a set of markers (2-3 functions) 
 that indicate something as trusted block.


If you're writing a lambda to take the address of a variable, then you're 
using  trusted wrong.  Neither should be done.


 Obviously one can easily abuse it, much like  trusted can be easily abuse 
 today, especially with templates where it's easy to fail in trap of 
 putting  trusted on the whole function and trust pretty much any 3rd party 
 type to be safe.


That's what we have code review for.


 They have in common is lot of noise that distracts and obfuscates the 
 thing that already needs our full attention making it both bigger and 
 harder to follow.


Yes, they are noisy.  But that noise does make them stand out more.


 Careful there - a trusted lambda must ensure that pointer is fine, 
 although the means of getting it are  system. The same review-driven thing 
 about  trusted remains.


A trusted lambda must give the same guarantees any other  safe of  trusted 
function gives - that when called from inside a  safe function it will not 
violate memory safety with any possible arguments.

I'm not saying that you should mark entire functions as  trusted because 
parts of them are not  safe.  You should instead extract the unsafe parts 
into a nested function or lambda (or external function) that provides a 
completely  safe interface.

This way the _only_ code than needs  safeness review is inside functions 
marked as  trusted.
Sep 02 2014
prev sibling next sibling parent Denis Shelomovskij <verylonglogin.reg gmail.com> writes:
31.08.2014 17:47, Dmitry Olshansky пишет:

 Quite recently a lot of work has been done to make most of Phobos usable
 in  safe code.

 While a very welcome effort, it caused a number of doubts in particular
 due to the boilerplate required to isolate a small amount of unsafe
 operations and slap " trusted" over it.

 See e.g. Denis argument:
 https://github.com/D-Programming-Language/phobos/pull/2465

 There were proposals for language changes along the lines of having
  trusted block alike to debug/version blocks, but nothing ever came out
 of them.

 Without language support I decided it worth a shot to create a universal
 wrappers to establish a consistent convention. A use of such wrapper
 should indicate that a  system function call or language feature was
 hand-verified.

 Names and complete set of primitives are up for debate, but here is the
 start:

 https://gist.github.com/DmitryOlshansky/bc02f369c8a63818bd07

 A bit of usage:

 import core.stdc.string;
 import trusted;

 void main()  safe
 {

      char[] msg = "Hello!".dup;
      char[] msg2 = msg;
      import trusted; // may also use static import for absolute clarity
      assert(call!memcmp(addrOf(msg[0]), addrOf(msg2[0]), msg.length) == 0);
 }


 What do you guys think?


The language works fine for me as it is in this aspect. E.g. functions 
from `std.file` like `read` should be marked as ` trusted` because this 
is what ` trusted` is for, a function operating unsafe stuff and 
providing a safe interface to it.

Currently the only problem is with templates like `std.array.Appender` 
when we want the compiler to infer attributes from user type and have to 
carefully wrap our code in ` trusted` blocks leaving calls to user code 
unwrapped.

Yes, my opinion here is the same as Daniel's one.


About related `std.file` pulls:

I wrote my original comment [1] to the pull because it blows the code 
size and reduce its readability by using ` trusted` nested functions 
(lambdas not used just because they currently can't be inlined, but it's 
silly as these functions work with disk and thus slow anyway).

This introduces a bad anti-pattern in the library and should be 
reverted. Just imagine a D newbie who just started leaning the language 
and is reading `std.file` module to see how easy can he use native API 
in D (e.g. I was such newbie). With all these pulls merged (current 
HEAD) he will be terrified of the code ugliness and either decide this 
is the only way to use native API in D (wrap every native function in 
` trusted` nested function in ` safe` function) or will spend 
significant time discovering the history of this module and who is 
guilty in such bad design (e.g. I spend a lot of time when learned D 
because of `TypeTuple` used with expressions which completely confused me).


[1] 
https://github.com/D-Programming-Language/phobos/pull/2465#issuecomment-53950146

-- 
Денис В. Шеломовский
Denis V. Shelomovskij
Sep 01 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/31/2014 6:47 AM, Dmitry Olshansky wrote:

 import core.stdc.string;
 import trusted;

 void main()  safe
 {

      char[] msg = "Hello!".dup;
      char[] msg2 = msg;
      import trusted; // may also use static import for absolute clarity
      assert(call!memcmp(addrOf(msg[0]), addrOf(msg2[0]), msg.length) == 0);
 }


I don't agree with the notion of having primitives that provide escapes from 
 safe - it means the interleaving of  safe and  system code becomes too easy, 
and too easy to miss.

I also don't agree with the notion of having  trusted blocks of the form:

      trusted {
         ... system code ...
     }

We already have a mechanism to do that -  trusted nested functions. The code 
example becomes:

   void main()  safe {
      char[] msg = "Hello!".dup;
      char[] msg2 = msg;

      void checkEquals(const char[] msg, const char[] msg2) pure  trusted {
        assert(msg.length == msg2.length);
        assert(memcmp(msg.ptr, msg2.ptr, msg.length) == 0);
      }

      checkEquals(msg, msg2);
   }

Granted, this can be abused to subvert the  safe/ trusted/ system system, but 
that has always been the case. Note that I added an assert to make
checkEquals() 
actually trustworthy. The QA code reviewer can still grep for  trusted and flag 
them for special review.

These sorts of nested functions should inline and pose no extra overhead.
Sep 01 2014
next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Mon, 01 Sep 2014 17:03:01 -0700
Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 I also don't agree with the notion of having  trusted blocks of the
 form:
=20
       trusted {
          ... system code ...
      }
=20
 We already have a mechanism to do that -  trusted nested functions.


why don't add some sugar here? nothing will stop a dedicated person if
he wants to write "bad code". yet making life harder for thouse who
knows what they doing is... anti-human. this discriminating
knowledgeable people, forcing them obey the rules that was written for
fools. yet that rules will not stop fools of doing foolish things.

lose-lose.
Sep 01 2014
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 2 September 2014 at 06:44:13 UTC, ketmar via 
Digitalmars-d wrote:

 why don't add some sugar here? nothing will stop a dedicated 
 person if
 he wants to write "bad code". yet making life harder for thouse 
 who
 knows what they doing is... anti-human. this discriminating
 knowledgeable people, forcing them obey the rules that was 
 written for
 fools. yet that rules will not stop fools of doing foolish 
 things.

 lose-lose.


Because by adding dedicated syntax you save 2 (two!) symbols and 
gain nothing else. And get all the associated compiler 
maintenance costs which add up with each smallest feature. Not 
worth it.
Sep 02 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 07:03:05 +0000
Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Because by adding dedicated syntax you save 2 (two!) symbols and=20
 gain nothing else.


except code readability. anti-human tendencies.


 And get all the associated compiler=20
 maintenance costs which add up with each smallest feature.


so let's stop adding features at all. and extending Phobos too. we are
so scared by adding even simple things.

bwah, i can write that patch in a hour, and it will require very little
maintenance: it will break when lambdas break, and that is not often
(and such changes breaks alot of other things anyway).

p.s. sure i'll not write it, 'cause i'm not ready to fork D and my site
with "D powerpatches" aren't up yet. ;-)
Sep 02 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 2 September 2014 at 07:41:25 UTC, ketmar via 
Digitalmars-d wrote:

 And get all the associated compiler maintenance costs which 
 add up with each smallest feature.


 so let's stop adding features at all. and extending Phobos too. 
 we are
 so scared by adding even simple things.


This is what we generally do. Features get added only if they 
enable something fundamentally new that can't be reasonably 
workarounded in Phobos (which does not have such cumulative 
maintenance implications). This is rather strict policy.
Sep 02 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 07:48:15 +0000
Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 This is rather strict policy.


and that is why my "local D" diverges more and more from "mainline D".
'cause i know where to download C++ compiler if i want language which
not trying to make my life easier.

one of the main things that dragged me to D was "made by humans for
humans". and now D tries to wear C++ anti-human shoes -- exactly what i
wanted to avoid by "migrating to D".

D is far superior to C++, but D starting to make C++ mistakes. not
fixing legacy syntax, not adding new handy features, abusing keywords
(enum constants anyone? why 'enum'?! ah, 'const' is already taken)...
`foreach (auto i; 0..42)`? no-no-no! it worth nothing that newcomers
will be surprised by `foreach (i; 0..42)`, asking "where that 'i'
variable was declared?" and that 'auto' is an ideal candidate here,
as 'type placeholder for variable declaration'. nameless args for
"foreach"? no-no-no, you can't write `foreach (int, string; smth)` --
invent silly names for unused args. `foreach (; 0..42)`? heretics!
nobody will understand that arcane operator! allowing ' ' before 'pure'
and 'nothrow'? no-no-no, we can't declare D mature without legacy! and
so on...

ah, and "foreach_reverse" abomination... i don't even want to talk
about this *keyword*.

please note that i'm not trying to say that D developers doing
everything wrong nor that they are incompetent. D is great. but we can
make it even better. just stop buying "enterprise need stability" bs:
we have enough "enterprise-stable" languages already, let's make one
that is attractive to programmers.
Sep 02 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
ketmar:


 and that is why my "local D" diverges more and more from 
 "mainline D".


This is a bad idea, it leads to an increase of your 
dissatisfaction, until you stop using D :-(

Bye,
bearophile
Sep 02 2014
next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 08:40:35 +0000
bearophile via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 and that is why my "local D" diverges more and more from=20
 "mainline D".


 This is a bad idea, it leads to an increase of your=20
 dissatisfaction, until you stop using D :-(


no, it just leads me to being isolated ('cause "my" D still able to
compile any "mainline D" code, and "mainline D" is unable to compile
mine code). but "being isolated" is the thing i used to for decades.

this makes me happier too, 'cause i have a better language to write
*my* code. my dissatisfaction increases when i see legacy and
"anti-human" in "mainline D" which is not going to be fixed. there i
have only two choises: either become more and more frustrated and drop D
or fix that at least in "my D" and be happy. i chose second way.

'cause i don't want to go back to C or C++: i can't live without
template magic, mixins, CTFE, UFCS and other D wonders anymore! ;-)

and i don't mind keeping my patches up-to-date: it's not hard and even
fun sometimes.
Sep 02 2014
prev sibling parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 2 September 2014 at 08:40:37 UTC, bearophile wrote:

 ketmar:


 and that is why my "local D" diverges more and more from 
 "mainline D".


 This is a bad idea, it leads to an increase of your 
 dissatisfaction, until you stop using D :-(


I am interested in ketmar's patches. Modifying the parser is a 
small change and you can have multiple parsers in the same 
compiler. Since D files start with "module " you can discriminate 
the source code by looking at the head of the file.

I've been sketching a modified syntax for parts of D for a while 
now so ketmar's work is going to make the implementation work 
easier for me. Thus, I applaud his rebellious efforts. :-)
Sep 02 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 2 September 2014 at 08:24:42 UTC, ketmar via 
Digitalmars-d wrote:

 please note that i'm not trying to say that D developers doing
 everything wrong nor that they are incompetent. D is great. but 
 we can
 make it even better. just stop buying "enterprise need 
 stability" bs:
 we have enough "enterprise-stable" languages already, let's 
 make one
 that is attractive to programmers.


I am by all means regular advocate of changes but your are 
pushing it to other radically harmful attitude. There is a huge 
difference between maintaining own set of patches that work for 
cases you personally use language for and maintaining feature 
constantly in the publicly available upstream. Things break in 
most unexpected ways when exposes to even small / medium user 
bases, we have it all the time.

For example, this specific syntax is absolutely guaranteed to 
result in weird issues because it is ambiguous with already 
existing one (that applies attributes to declarations). In the 
end it gives you _nothing_. Saving two characters for already 
short idiom that is not even supposed to be easy to use is just a 
joke. There is simply no way something as trivial as that is 
going to pull its weight.

There is a limited amount of maintenance effort we can invest 
into adding new stuff to language and it needs to be used wisely.
Sep 02 2014
next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 13:15:01 +0000
Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 For example, this specific syntax is absolutely guaranteed to=20
 result in weird issues because it is ambiguous with already=20
 existing one (that applies attributes to declarations).


please write a sample. i'm not trying to troll, i really don't
understand what you mean here. you can't have ' nothrow' and ' pure'
UDAs anyway.
Sep 02 2014
prev sibling next sibling parent "Andrew Godfrey" <X y.com> writes:
On Tuesday, 2 September 2014 at 13:15:02 UTC, Dicebot wrote:

 On Tuesday, 2 September 2014 at 08:24:42 UTC, ketmar via 
 Digitalmars-d wrote:

 please note that i'm not trying to say that D developers doing
 everything wrong nor that they are incompetent. D is great. 
 but we can
 make it even better. just stop buying "enterprise need 
 stability" bs:
 we have enough "enterprise-stable" languages already, let's 
 make one
 that is attractive to programmers.


 I am by all means regular advocate of changes but your are 
 pushing it to other radically harmful attitude. There is a huge 
 difference between maintaining own set of patches that work for 
 cases you personally use language for and maintaining feature 
 constantly in the publicly available upstream. Things break in 
 most unexpected ways when exposes to even small / medium user 
 bases, we have it all the time.

 For example, this specific syntax is absolutely guaranteed to 
 result in weird issues because it is ambiguous with already 
 existing one (that applies attributes to declarations). In the 
 end it gives you _nothing_. Saving two characters for already 
 short idiom that is not even supposed to be easy to use is just 
 a joke. There is simply no way something as trivial as that is 
 going to pull its weight.

 There is a limited amount of maintenance effort we can invest 
 into adding new stuff to language and it needs to be used 
 wisely.


Something I'm feeling as a newcomer to D reading these threads, 
is that the community has already thought of a number of 
guidelines for Phobos development, but they are buried in the 
forum. I also saw one which is controversial - the idea that 
embedded systems programmers should be able to use some subset of 
Phobos without bloating their executable. I'd have no idea 
whether we are trying to maintain that property of Phobos where 
it already exists, or ignore it. Similarly, there are a lot of 
diverse goals/use cases people have in mind - the std.logger 
thread has some.

It feels like Phobos has a good amount of 'breadth'' but could do 
with more "depth". Have we already tried some exercises which 
would flesh out more "Phobos requirements" (I.e. That we aspire 
to for existing modules and require for new ones?)
Sep 02 2014
prev sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 13:15:01 +0000
Dicebot via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 For example, this specific syntax is absolutely guaranteed to=20
 result in weird issues because it is ambiguous with already=20
 existing one (that applies attributes to declarations).


ah, you talking about ` trusted { ... }`, right? why it is ambiguous? i
applied one restriction to it:  trusted block must always be
"{}"-blocks. that removes ambiguity altogether.

it's not about saving some chars, it's about not writing arcane
constructs like:

  ()  trusted {
    ...
  }();

this looks weird.

but hell with that "trusted blocks", how about foreach i mentioned?
not accepting 'auto' in `foreach (auto i; 0..42)` is just illogical.
here we saving 5 chars for... for what? `foreach (i; 0..42)` looks
exactly like `for (i =3D 0; i < 42; ++i)`, yet it means something
completely different.

i understand that we can't make 'auto' mandatory there, but why don't
make it at least acceptable? (and then add a deprecation warning, and
then finally make 'auto' mandatory)

as the time passing it becomes increasingly hard to fix this "cosmetic
issues", as there are more and more people writing D code. i don't
believe that waiting for another year or three and then just saying
"alas, this train is gone long ago so now we must live with this
legacy" is a good solution.

yes, i'm relatively new in D world and my position may look like
extremist one, but all i want is to make D better. that's why i'm
writing those posts instead of sitting silently in my shell. i know
that we can't change C or C++, but i believe that we can change D. not
by throwing in random features, but by making D more consistent, better
looking and more pleasant to use.
Sep 02 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 2 September 2014 at 17:19:28 UTC, ketmar via 
Digitalmars-d wrote:

 yes, i'm relatively new in D world and my position may look like
 extremist one, but all i want is to make D better. that's why 
 i'm
 writing those posts instead of sitting silently in my shell. i 
 know
 that we can't change C or C++, but i believe that we can change 
 D. not
 by throwing in random features, but by making D more 
 consistent, better
 looking and more pleasant to use.


After following these forums for quite some time I think the 
roll-your-own-as-proof-of-concept is the best approach. If it is 
better and the compiler is capable of taking mainstream D as well 
as your own syntax then others will follow and there will be 
pressure to implement it in the main branch. There is just waaaay 
to much resistance to improve the D syntax to waste time arguing 
for it.

Time is better spent arguing for more performant/better semantics 
in the mainline and fix syntax in private builds until the wind 
changes direction… (and an alternative build that is better, yet 
remains compatible, is the best way to change direction the wind 
is blowing).

I'm rooting for you! :-)
Sep 02 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 19:15:46 +0000
via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 After following these forums for quite some time I think the=20
 roll-your-own-as-proof-of-concept is the best approach.


that's why we need "D powerpatches" site. i'm working on it, albeit
slowly. i see it as "patch repository", with descriptions, dependency
tracking ("patch xyz needs patch zxy applied first"), web interface and
CLI tool (something like very lightweight apt). maybe even with
"private repositories" which can be created by users.
so-called "dub for patches". ;-)

something like:

  dppm list
  dppm search
  dppm get uda_nothrow_pure trusted_block -- to get "combined patch"

and so on. maybe even with comment system for patchsets.

this is ambitious project, i know. but long time ago D was just a dream
too. and this project will help me to improve my nearly non-existent
vibe.d skills. ;-)
Sep 02 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 2 September 2014 at 19:38:47 UTC, ketmar via 
Digitalmars-d wrote:

 this is ambitious project, i know. but long time ago D was just 
 a dream
 too. and this project will help me to improve my nearly 
 non-existent
 vibe.d skills. ;-)


Another approach would be to cooperate on building an 
infrastructure where it is easy to have multiple parsers in the 
same build then have the ability so select parser and configure 
the parser syntax in the head of the file:

module.parser1( attrs,simd,stuff) modulename;
or
module.parser2(pythonstyle, otherstuff) modulename;

That way you can even create a DSL and use it in select modules.

The main problem is string mixins where you probably will have to 
retain classic D syntax?
Sep 02 2014
next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 20:04:51 +0000
via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Another approach would be to cooperate on building an=20
 infrastructure where it is easy to have multiple parsers in the=20
 same build then have the ability so select parser and configure=20
 the parser syntax in the head of the file:


i'm still waiting for AST macros (DIP50). ;-)

actually, changing parsers is not that fun, 'cause writing parser needs
"inner knowledge" about compiler and it's AST. and writing complete
parser is very tedious. and you will not be able to write "whitespace
sensitive" parser anyway, 'cause it needs new scanner and lexer.

in other words: i'm not ready for this challenge yet. i'd better stick
with my patches and leave custom parsers for somebody else. such change
will diverge ALOT from "mainline D" and syncing with git head will be
even more time-consuming than doing full-time 8 hr. work. and i have
nothing to sell right now to keep me alive and well-functioning without
payed work. ;-)
Sep 02 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 2 September 2014 at 20:31:18 UTC, ketmar via 
Digitalmars-d wrote:

 actually, changing parsers is not that fun, 'cause writing 
 parser needs
 "inner knowledge" about compiler and it's AST. and writing 
 complete
 parser is very tedious. and you will not be able to write 
 "whitespace
 sensitive" parser anyway, 'cause it needs new scanner and lexer.


If it changes a lot then you can do it as a builtin preprocessor 
that compiles to D then hand it over to the regular parser.


 in other words: i'm not ready for this challenge yet. i'd 
 better stick
 with my patches and leave custom parsers for somebody else.


Mhmmm, and here I thought I could lure you into paving the way 
for my own ambitious syntactical changes… ;-)
Sep 02 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 20:35:05 +0000
via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 If it changes a lot then you can do it as a builtin preprocessor=20
 that compiles to D then hand it over to the regular parser.


but you can do it as an external tool. if new parser is too "far way"
from D, this is easier than patching the D compiler. and we have
Dscanner to parse D code. my patches are relatively small and it's
easier to integrate 'em into existing D parser.

but yes, you will lose CTFE this way.
Sep 02 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 2 September 2014 at 20:54:55 UTC, ketmar via 
Digitalmars-d wrote:

 but you can do it as an external tool. if new parser is too 
 "far way"
 from D, this is easier than patching the D compiler. and we have
 Dscanner to parse D code. my patches are relatively small and


The change-log for the ASTs don't look too bad to me when 
glancing over it. If you want to affect the syntax then you 
probably will need to release binary builds as well as patches. 
Maybe following the major revisions of ldc might be less work 
than tracking dmd.
Sep 02 2014
next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 03 Sep 2014 02:51:24 +0000
via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 If you want to affect the syntax then you=20
 probably will need to release binary builds as well as patches.


nononono, i don't want to go into it. ;-)
building DMD is easy and fast. and i'm not sure that i can redistribute
DMD binaries anyway.


 Maybe following the major revisions of ldc might be less work=20
 than tracking dmd.


i like to see what's coming, so i'm updating and rebuilding DMD at
least once per day. it's easy to keep everything in sync with
relativelty small changes.

besides, i don't like LLVM. ;-)
Sep 03 2014
parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 3 September 2014 at 09:13:41 UTC, ketmar via 
Digitalmars-d wrote:

 i like to see what's coming, so i'm updating and rebuilding DMD 
 at
 least once per day. it's easy to keep everything in sync with
 relativelty small changes.


Whoa, great to see that you are enthusiastic!


 besides, i don't like LLVM. ;-)


So you will have to build your own backend too! :]
Out of curiosity, what is it about LLVM that you don't like?


p.s. eventually i will come to AST and sematics too. fear 
everyone! ;-)


That's the spirit! Looking forward too it. :-)
Sep 03 2014
parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 03 Sep 2014 09:43:47 +0000
via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 besides, i don't like LLVM. ;-)


 So you will have to build your own backend too! :]


sometimes i'm thinking about writing backend that generates C++ code
(not human-readable, with templates expanded and so on, just to use
already available C++ compiler).


 Out of curiosity, what is it about LLVM that you don't like?


license and apples. ;-) i prefer gcc. i don't know gcc internals, so
can't help Iain, but i'm backporting some of my patches and mainline
bugfixes to current gdc (2.065). when we got 2.066 landed i'll try to
keep patches for both DMD and GDC in sync. sorry, no LDC support is
planned.


 That's the spirit!


i just have no life. ;-)
Sep 03 2014
prev sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 03 Sep 2014 02:51:24 +0000
via Digitalmars-d <digitalmars-d puremagic.com> wrote:

p.s. eventually i will come to AST and sematics too. fear everyone! ;-)
Sep 03 2014
prev sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 2 Sep 2014 23:31:05 +0300
ketmar via Digitalmars-d <digitalmars-d puremagic.com> wrote:

p.s. i REALLY need to take English lessons.
Sep 02 2014
prev sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 2 September 2014 at 06:44:13 UTC, ketmar via 
Digitalmars-d wrote:

 On Mon, 01 Sep 2014 17:03:01 -0700
 Walter Bright via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:


 I also don't agree with the notion of having  trusted blocks 
 of the
 form:
 
       trusted {
          ... system code ...
      }
 
 We already have a mechanism to do that -  trusted nested 
 functions.


 why don't add some sugar here? nothing will stop a dedicated 
 person if
 he wants to write "bad code". yet making life harder for thouse 
 who
 knows what they doing is... anti-human. this discriminating
 knowledgeable people, forcing them obey the rules that was 
 written for
 fools. yet that rules will not stop fools of doing foolish 
 things.

 lose-lose.


And thus we end with the security exploits and computer errors C 
has brought into the world.
Sep 02 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 09:24:37 +0000
Paulo Pinto via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 And thus we end with the security exploits and computer errors C=20
 has brought into the world.


ok, so we should disable  trusted nested functions then, 'cause
 trusted blocks are just syntactic sugar for them. and pointers --
pointers are dangerous! and system calls -- system calls are dangerous!
and manual memory management -- manual memory management is dangerous!
and... wait, what do you mean by "D is not java"?

how having handy sugar for the thing that is *already* in language can
hurt us here?
Sep 02 2014
parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 2 September 2014 at 09:36:50 UTC, ketmar via 
Digitalmars-d wrote:

 On Tue, 02 Sep 2014 09:24:37 +0000
 Paulo Pinto via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:


 And thus we end with the security exploits and computer errors 
 C has brought into the world.


 ok, so we should disable  trusted nested functions then, 'cause
  trusted blocks are just syntactic sugar for them. and pointers 
 --
 pointers are dangerous! and system calls -- system calls are 
 dangerous!
 and manual memory management -- manual memory management is 
 dangerous!
 and... wait, what do you mean by "D is not java"?

 how having handy sugar for the thing that is *already* in 
 language can
 hurt us here?


I sure like D isn't Java.

In principle sounds very nice to "thrust the programmer", but 
this only works for single coders, or teams with top level coders.

In the sad reality of every day computing tasks, the code is as 
good as the worst developer on the team.

So yes, manual memory management goes astray when such developers 
keep rotating in teams with medium size code bases.

Happy pointer chasing, like C does, leads to all sort of nice 
bugs in the same development scenarios.

Specially bad, given that the alternatives at the time, for the 
same use cases, didn't require such pointer chasing like C does.

I like quite much that D follows Ada and Modula-3 model of 
explicit system/trusted/safe code.

--
Paulo
Sep 02 2014
parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 11:08:25 +0000
Paulo Pinto via Digitalmars-d <digitalmars-d puremagic.com> wrote:

let me ask it again:
how, in the name of hell, having handy sugar for the thing that is
*already* in the language can hurt us here?
Sep 02 2014
parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 2 September 2014 at 11:30:43 UTC, ketmar via 
Digitalmars-d wrote:

 On Tue, 02 Sep 2014 11:08:25 +0000
 Paulo Pinto via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:

 let me ask it again:
 how, in the name of hell, having handy sugar for the thing that 
 is
 *already* in the language can hurt us here?


In this particular case:

Because it _is_ handy. It shouldn't be. It's supposed to be ugly, 
to make you think twice whether you actually want to use it.

Besides, as was already mentioned, 'grep -r  trusted' wouldn't 
work anymore.
Sep 02 2014
next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 11:37:38 +0000
via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Because it _is_ handy. It shouldn't be. It's supposed to be ugly,=20
 to make you think twice whether you actually want to use it.


where i can vote for making pointer syntax unusable too? something like
'void
yes_i_really_want_to_declare_pointer_here_secret_password_47385hjfhiegeioje=
wqon
________star________ p;'?
Sep 02 2014
prev sibling next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 11:37:38 +0000
via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Because it _is_ handy. It shouldn't be. It's supposed to be ugly,=20
 to make you think twice whether you actually want to use it.


what i'm trying to say it that D either *system* language or java-like
crap that tries to artificially "protect" the programmer and fails.

such king of "uglification for protection" doesn't work: it annoys
people who know what are they doing and will not stop idiots anyway,
'cause idiots are very inventive in their ways of doing idiocity.
Sep 02 2014
parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 2 September 2014 at 11:55:58 UTC, ketmar via 
Digitalmars-d wrote:

 On Tue, 02 Sep 2014 11:37:38 +0000
 via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Because it _is_ handy. It shouldn't be. It's supposed to be 
 ugly, to make you think twice whether you actually want to use 
 it.


 what i'm trying to say it that D either *system* language or 
 java-like
 crap that tries to artificially "protect" the programmer and 
 fails.

 such king of "uglification for protection" doesn't work: it 
 annoys
 people who know what are they doing and will not stop idiots 
 anyway,
 'cause idiots are very inventive in their ways of doing 
 idiocity.


For some strange reason human life critical systems are written 
in Ada, SPARK, MISRA C dialect
Sep 02 2014
next sibling parent reply ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 02 Sep 2014 12:18:08 +0000
Paulo Pinto via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 For some strange reason human life critical systems are written=20
 in Ada, SPARK, MISRA C dialect


ah, sorry, i was thinking that this is NG for D language... mea culpa.
Sep 02 2014
parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 2 September 2014 at 13:42:05 UTC, ketmar via 
Digitalmars-d wrote:

 On Tue, 02 Sep 2014 12:18:08 +0000
 Paulo Pinto via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:


 For some strange reason human life critical systems are 
 written in Ada, SPARK, MISRA C dialect


 ah, sorry, i was thinking that this is NG for D language... mea 
 culpa.


And I though we were all grown ups and not school kids, looking 
at the level of your posts to other members.

Stepping out of this discussion.
Sep 02 2014
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Paulo Pinto:


 For some strange reason human life critical systems are written 
 in Ada, SPARK, MISRA C dialect


If D designers and developers work to make D better for those 
usages, someday D could be used to replace the MISRA C and 
perhaps even some less strict usages of Ada.

Bye,
bearophile
Sep 02 2014
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
02-Sep-2014 15:37, "Marc Schütz" <schuetzm gmx.net>" пишет:

 On Tuesday, 2 September 2014 at 11:30:43 UTC, ketmar via Digitalmars-d
 wrote:

 On Tue, 02 Sep 2014 11:08:25 +0000
 Paulo Pinto via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 let me ask it again:
 how, in the name of hell, having handy sugar for the thing that is
 *already* in the language can hurt us here?


 In this particular case:

 Because it _is_ handy. It shouldn't be. It's supposed to be ugly, to
 make you think twice whether you actually want to use it.

 Besides, as was already mentioned, 'grep -r  trusted' wouldn't work
 anymore.


Making things ugly doesn't make them safe or easier to verify.
Somehow people expect the opposite, but just take a look at e.g. OpenSSL :)

Slapping  trusted across whole functions just blurs the scope of system 
code (where? what was system? or maybe it's that pointer ... it's really 
hard to analyze afterwards).

-- 
Dmitry Olshansky
Sep 02 2014
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Tuesday, 2 September 2014 at 14:10:39 UTC, Dmitry Olshansky 
wrote:

 02-Sep-2014 15:37, "Marc Schütz" <schuetzm gmx.net>" пишет:

 On Tuesday, 2 September 2014 at 11:30:43 UTC, ketmar via 
 Digitalmars-d
 wrote:

 let me ask it again:
 how, in the name of hell, having handy sugar for the thing 
 that is
 *already* in the language can hurt us here?


 In this particular case:

 Because it _is_ handy. It shouldn't be. It's supposed to be 
 ugly, to
 make you think twice whether you actually want to use it.

 Besides, as was already mentioned, 'grep -r  trusted' wouldn't 
 work
 anymore.


 Making things ugly doesn't make them safe or easier to verify.
 Somehow people expect the opposite, but just take a look at 
 e.g. OpenSSL :)

 Slapping  trusted across whole functions just blurs the scope 
 of system code (where? what was system? or maybe it's that 
 pointer ... it's really hard to analyze afterwards).


I agree, it needs to be as fine-grained as possible. I just 
happen to believe that the suggested template wrappers are not a 
good idea.

Note that my post was in response to the question how "having 
handy sugar [...] can hurt us here". That doesn't automatically 
mean that the alternatives are perfect.
Sep 02 2014
prev sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Dmitry Olshansky"  wrote in message news:lu4j4v$leu$1 digitalmars.com...


 Making things ugly doesn't make them safe or easier to verify.
 Somehow people expect the opposite, but just take a look at e.g. OpenSSL 
 :)


No, but making unsafe code ugly makes the safe alternatives look more 
attractive, and hopefully more likely to be used.  It also makes the unsafe 
code stand out more, so it is less likely to be overlooked.


 Slapping  trusted across whole functions just blurs the scope of system 
 code (where? what was system? or maybe it's that pointer ... it's really 
 hard to analyze afterwards).


Nobody is suggesting this.
Sep 02 2014
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
02-Sep-2014 04:03, Walter Bright пишет:

 On 8/31/2014 6:47 AM, Dmitry Olshansky wrote:

 import core.stdc.string;
 import trusted;

 void main()  safe
 {

      char[] msg = "Hello!".dup;
      char[] msg2 = msg;
      import trusted; // may also use static import for absolute clarity
      assert(call!memcmp(addrOf(msg[0]), addrOf(msg2[0]), msg.length)
 == 0);
 }


 I don't agree with the notion of having primitives that provide escapes
 from  safe - it means the interleaving of  safe and  system code becomes
 too easy, and too easy to miss.


Make distinctive name like assumeSafe and it's going to be trivially 
grepable.


 I also don't agree with the notion of having  trusted blocks of the form:

       trusted {
          ... system code ...
      }



 We already have a mechanism to do that -  trusted nested functions. The
 code example becomes:


So there is need, but somehow requires a bunch of useless boilerplate, 
like repeating arguments and inventing creative names for local functions.

    void main()  safe {
       char[] msg = "Hello!".dup;
       char[] msg2 = msg;

       void checkEquals(const char[] msg, const char[] msg2) pure  trusted {
         assert(msg.length == msg2.length);
         assert(memcmp(msg.ptr, msg2.ptr, msg.length) == 0);
       }


So you think adding boilerplate will make function more easily 
verifiable? Time and statistics proven that more LOCs ==> more bugs. 
Especially highly repetitive patterns, because nobody actually reads them.



-- 
Dmitry Olshansky
Sep 02 2014
next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Dmitry Olshansky"  wrote in message news:lu4iup$l9v$1 digitalmars.com...

    void main()  safe {
       char[] msg = "Hello!".dup;
       char[] msg2 = msg;

       void checkEquals(const char[] msg, const char[] msg2) pure 
  trusted {
         assert(msg.length == msg2.length);
         assert(memcmp(msg.ptr, msg2.ptr, msg.length) == 0);
       }


 So you think adding boilerplate will make function more easily verifiable? 
 Time and statistics proven that more LOCs ==> more bugs.


Yes, that function is more easily verifiable for  safety, because any 
violation _must_ be inside the  trusted function.  If the  safe violating 
helpers were used, main would effectively be  trusted and more lines would 
need to be reviewed.


 Especially highly repetitive patterns, because nobody actually reads them.


If you have highly repetitive patterns, they should be factored out into 
reusable functions just like always.
Sep 02 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/2/2014 10:11 AM, Daniel Murphy wrote:

 Yes, that function is more easily verifiable for  safety, because any violation
 _must_ be inside the  trusted function.


The 'pure' attribute helps with that.



 If the  safe violating helpers were
 used, main would effectively be  trusted and more lines would need to be
reviewed.


Yes, that says it better than I did.
Sep 02 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/2/2014 7:07 AM, Dmitry Olshansky wrote:

  Make distinctive name like assumeSafe and it's going to be trivially grepable.


We already have a distinctive name,  trusted. Adding an panopoly of more names 
make it not so trivially greppable.



 So there is need, but somehow requires a bunch of useless boilerplate, like
 repeating arguments and inventing creative names for local functions.


In the example I gave, it wasn't actually necessary to repeat the arguments. 
This will work as well:

       void checkEquals()  trusted {
          assert(msg.length == msg2.length);
          assert(memcmp(msg.ptr, msg2.ptr, msg.length) == 0);
        }

But the parameterized one was better encapsulated by being pure.



 So you think adding boilerplate will make function more easily verifiable?


Encapsulating the trusted operation, not the components of the trusted 
operation, make it more easily verifiable.
Sep 02 2014
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
31-Aug-2014 17:47, Dmitry Olshansky пишет:

 Quite recently a lot of work has been done to make most of Phobos usable
 in  safe code.


...

 What do you guys think?


Probably a lot of people missed the point that if we standardize a few 
idioms (dangerous but at least centralized) we at least can conveniently 
contain the "abuse" of  trusted to the select standard module. Else it 
*will* be abused in a multitude of ways anyway.

-- 
Dmitry Olshansky
Sep 02 2014
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 2 September 2014 at 14:33:53 UTC, Dmitry Olshansky 
wrote:

 31-Aug-2014 17:47, Dmitry Olshansky пишет:

 Quite recently a lot of work has been done to make most of 
 Phobos usable
 in  safe code.


 ...

 What do you guys think?


 Probably a lot of people missed the point that if we 
 standardize a few idioms (dangerous but at least centralized) 
 we at least can conveniently contain the "abuse" of  trusted to 
 the select standard module. Else it *will* be abused in a 
 multitude of ways anyway.


I think it's probably hard to appreciate where you are coming 
from, until you've reviewed code for things such as Appender 
and/or emplace. I swear there was 1 point where roughly 25% of 
the lines of code in that thing where wrapped in a trusted lambda.

1 issue I find with your proposal, is (personally), I've seldom 
had to *call* unsafe functions in a trusted fashion, but rather, 
had to do unsafe *things*:

if (capacity > slice.length)
     slice =  trusted(){return slice.ptr[0 .. slice.length + 
1];}();

In such context, "call!" wouldn't help much. That said, there are 
also plenty of cases where we call memcpy (just grep 
"trustedMemcpy" in phobos), where your proposal would help.

Also: There's already a help "addressOf" somewhere in phobos. 
It's meant mostly to take the address of property return values. 
Instead of providing "addressOf" in std.trusted, you could simply 
do a "call!" of the not-trusted generic "addressOf". Just a 
thought.
Sep 02 2014
prev sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Dmitry Olshansky"  wrote in message news:lu4kgh$mpm$1 digitalmars.com...


 Probably a lot of people missed the point that if we standardize a few 
 idioms (dangerous but at least centralized) we at least can conveniently 
 contain the "abuse" of  trusted to the select standard module. Else it 
 *will* be abused in a multitude of ways anyway.



 we at least can conveniently contain the "abuse" of  trusted to the select 
 standard module


This is Wrong!  Any function that uses these wrappers is abusing  trusted.

eg:

import stdx.trusted;

int* func(int x)  safe
{
    return addrOf(x);
}

This functions is  safe, but happily returns an invalid pointer.  This is 
possible because addrOf violates the requirement that  trusted functions 
must be completely  safe to call from an  safe function.

Say we have a very useful function like this:

void doThing(bool corruptRandomMemory)  system { ... }

So if the parameter is true, it will cause memory corruption, otherwise it 
will not do anything that violates  safe.

This is a valid  trusted wrapper:

void doThingTrusted()  trusted { doThing(false); }

This is not:

void doThingTrusted(bool b)  trusted { doThing(b); }

Having syntax (or a wrapper function) to do the second wrapping 
automatically would violate  safe.  If it was syntax, it would be banned in 
 safe.  If it's a wrapping method like the proposed 'call', then it is a 
program error for it to be marked  trusted.
Sep 02 2014
next sibling parent "monarch_dodra" <monarchdodra gmail.com> writes:
On Tuesday, 2 September 2014 at 17:20:06 UTC, Daniel Murphy wrote:

 This is Wrong!  Any function that uses these wrappers is 
 abusing  trusted.

 eg:

 import stdx.trusted;

 int* func(int x)  safe
 {
    return addrOf(x);
 }

 This functions is  safe, but happily returns an invalid 
 pointer.  This is possible because addrOf violates the 
 requirement that  trusted functions must be completely  safe to 
 call from an  safe function.


That's a good point.


 Having syntax (or a wrapper function) to do the second wrapping 
 automatically would violate  safe.  If it was syntax, it would 
 be banned in  safe. If it's a wrapping method like the proposed 
 'call', then it is a program error for it to be marked  trusted.


Good points too.

A very logical conclusion.
Sep 02 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/2/2014 10:20 AM, Daniel Murphy wrote:

 Having syntax (or a wrapper function) to do the second wrapping automatically
 would violate  safe.  If it was syntax, it would be banned in  safe.  If it's a
 wrapping method like the proposed 'call', then it is a program error for it to
 be marked  trusted.


I agree with monarch_data, this is the executive summary, the salient point,
the 
money shot, etc.
Sep 02 2014
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
02-Sep-2014 22:50, Walter Bright пишет:

 On 9/2/2014 10:20 AM, Daniel Murphy wrote:

 Having syntax (or a wrapper function) to do the second wrapping
 automatically
 would violate  safe.  If it was syntax, it would be banned in  safe.
 If it's a
 wrapping method like the proposed 'call', then it is a program error
 for it to
 be marked  trusted.


 I agree with monarch_data, this is the executive summary, the salient
 point, the money shot, etc.


Well, whatever. Let's wait to see where our code base goes.

-- 
Dmitry Olshansky
Sep 02 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/2/2014 12:40 PM, Dmitry Olshansky wrote:

 Well, whatever. Let's wait to see where our code base goes.



:-)
Sep 02 2014
prev sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On Tuesday, 2 September 2014 at 18:50:19 UTC, Walter Bright wrote:

 On 9/2/2014 10:20 AM, Daniel Murphy wrote:

 Having syntax (or a wrapper function) to do the second 
 wrapping automatically
 would violate  safe.  If it was syntax, it would be banned in 
  safe.  If it's a
 wrapping method like the proposed 'call', then it is a program 
 error for it to
 be marked  trusted.


 I agree with monarch_data, this is the executive summary, the 
 salient point, the money shot, etc.


+1 from me as the self-proclaimed resident  safe-ty philosopher. 
We might want to think about fixing the immediately invoked 
delegate literal inlining problem.

David
Sep 02 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 2 September 2014 at 20:18:24 UTC, David Nadlinger 
wrote:

 I agree with monarch_data, this is the executive summary, the 
 salient point, the money shot, etc.


 +1 from me as the self-proclaimed resident  safe-ty 
 philosopher. We might want to think about fixing the 
 immediately invoked delegate literal inlining problem.

 David


I wouldn't expect LDC or GDC to have problem with that. Do they ?
Sep 02 2014
parent reply Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 Sep 2014 04:00, "deadalnix via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 On Tuesday, 2 September 2014 at 20:18:24 UTC, David Nadlinger wrote:

 I agree with monarch_data, this is the executive summary, the salient






point, the money shot, etc.

 +1 from me as the self-proclaimed resident  safe-ty philosopher. We




might want to think about fixing the immediately invoked delegate literal
inlining problem.

 David



 I wouldn't expect LDC or GDC to have problem with that. Do they ?


For the time being, they are as expensive as virtual calls.

Iain.
Sep 02 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Wednesday, 3 September 2014 at 06:37:15 UTC, Iain Buclaw via
Digitalmars-d wrote:

 For the time being, they are as expensive as virtual calls.

 Iain.


WUT ???

How come ? The address of the call is know at compile time, the
optimizer should see through this, no ?
Sep 03 2014
parent reply "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 3 September 2014 at 21:00:13 UTC, deadalnix wrote:

 On Wednesday, 3 September 2014 at 06:37:15 UTC, Iain Buclaw via
 Digitalmars-d wrote:

 For the time being, they are as expensive as virtual calls.

 Iain.


 WUT ???

 How come ? The address of the call is know at compile time, the
 optimizer should see through this, no ?


Iain is selling his compiler short again. ;)

---
int foo(int[] a)  safe {
   if (a.length == 0) return -1;
   return (()  trusted => a.ptr[0])();
}
---
compiles to exactly what you would expect when optimizations are 
enabled with both GDC and LDC.

However, as long as DMD can't handle this idiom, there will be 
push-back regarding use in Phobos. Additionally, we should think 
about always inlining such literals even on -O0.

Cheers,
David
Sep 03 2014
parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 3 Sep 2014 23:00, "David Nadlinger via Digitalmars-d" <
digitalmars-d puremagic.com> wrote:

 On Wednesday, 3 September 2014 at 21:00:13 UTC, deadalnix wrote:

 On Wednesday, 3 September 2014 at 06:37:15 UTC, Iain Buclaw via
 Digitalmars-d wrote:

 For the time being, they are as expensive as virtual calls.

 Iain.



 WUT ???

 How come ? The address of the call is know at compile time, the
 optimizer should see through this, no ?



 Iain is selling his compiler short again. ;)


Always sell short, so they never complain when it does one better (also
means you don't overshoot expectations ;)

I am of course going off memory, and there are some very obvious cases with
delegates that never get optimized.

Iain.
Sep 03 2014