www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DIP 1006 - Preliminary Review Round 1

reply Mike Parker <aldacron gmail.com> writes:
DIP 1006 is titled "Providing more selective control over 
contracts".

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

All review-related feedback on and discussion of the DIP should 
occur in this thread. The review period will end at 11:59 PM ET 
on April 26 (3:59 AM GMT), or when I make a post declaring it 
complete.

At the end of Round 1, if further review is deemed necessary, the 
DIP will be scheduled for another round. Otherwise, it will be 
queued for the formal review and evaluation by the language 
authors.

Thanks in advance to all who participate.

Destroy!
Apr 12 2017
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 12/04/2017 12:25 PM, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP should occur in
 this thread. The review period will end at 11:59 PM ET on April 26 (3:59
 AM GMT), or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, the DIP
 will be scheduled for another round. Otherwise, it will be queued for
 the formal review and evaluation by the language authors.

 Thanks in advance to all who participate.

 Destroy!
How exactly does this affect unittests? From what I can see, in none mode unittests won't have any asserts, which is clearly a problem.
Apr 12 2017
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Wednesday, 12 April 2017 at 11:32:37 UTC, rikki cattermole 
wrote:
 On 12/04/2017 12:25 PM, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP 
 should occur in
 this thread. The review period will end at 11:59 PM ET on 
 April 26 (3:59
 AM GMT), or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP
 will be scheduled for another round. Otherwise, it will be 
 queued for
 the formal review and evaluation by the language authors.

 Thanks in advance to all who participate.

 Destroy!
How exactly does this affect unittests? From what I can see, in none mode unittests won't have any asserts, which is clearly a problem.
This is a -release style optimisation and would not be expected to be in dev/testing. Also unittest asserts call a different druntime function (IIRC) and therefore _should_ logically be under the control of -unittest and not affected by dip1006 OR be an invalid flag combination.
Apr 12 2017
parent rikki cattermole <rikki cattermole.co.nz> writes:
On 12/04/2017 12:48 PM, Nicholas Wilson wrote:
 On Wednesday, 12 April 2017 at 11:32:37 UTC, rikki cattermole wrote:
 On 12/04/2017 12:25 PM, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP should occur in
 this thread. The review period will end at 11:59 PM ET on April 26 (3:59
 AM GMT), or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, the DIP
 will be scheduled for another round. Otherwise, it will be queued for
 the formal review and evaluation by the language authors.

 Thanks in advance to all who participate.

 Destroy!
How exactly does this affect unittests? From what I can see, in none mode unittests won't have any asserts, which is clearly a problem.
This is a -release style optimisation and would not be expected to be in dev/testing. Also unittest asserts call a different druntime function (IIRC) and therefore _should_ logically be under the control of -unittest and not affected by dip1006 OR be an invalid flag combination.
Please make it explicit that asserts are only in invariant and in/out blocks are affected. With a note about unittests, just to remove this possible incorrect implementation detail. I am not happy with the args passed to the switch, but I can't think of an alternative names, so I am happy with how the DIP is including the above statements.
Apr 12 2017
prev sibling parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 12 April 2017 at 11:32:37 UTC, rikki cattermole 
wrote:
 On 12/04/2017 12:25 PM, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP 
 should occur in
 this thread. The review period will end at 11:59 PM ET on 
 April 26 (3:59
 AM GMT), or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP
 will be scheduled for another round. Otherwise, it will be 
 queued for
 the formal review and evaluation by the language authors.

 Thanks in advance to all who participate.

 Destroy!
How exactly does this affect unittests? From what I can see, in none mode unittests won't have any asserts, which is clearly a problem.
Good point. I would say that `-contracts=none` and `-unittest` should behave the same as `-release` and `-unittest`, and currently it only turns asserts on ( https://github.com/dlang/dmd/blob/ac3225a025b578d45ff39a40dda35006fb455a37/src/ddm /mars.d#L1100-L1109 ). I'll add a note about it in the DIP.
Apr 12 2017
parent Martin Nowak <code dawg.eu> writes:
On Wednesday, 12 April 2017 at 15:02:49 UTC, Mathias Lang wrote:
 I would say that `-contracts=none` and `-unittest` should 
 behave the same as `-release` and `-unittest`, and currently it 
 only turns asserts on ( 
 https://github.com/dlang/dmd/blob/ac3225a025b578d45ff39a40dda35006fb455a37/src/ddm
/mars.d#L1100-L1109 ).
 I'll add a note about it in the DIP.
As said above, asserts in unittests are different, so we could separate enabling asserts in unittests and in the rest of the program.
Nov 21 2017
prev sibling next sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on April 26 (3:59 AM GMT), or when I make a post declaring it 
 complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it will 
 be queued for the formal review and evaluation by the language 
 authors.

 Thanks in advance to all who participate.

 Destroy!
Is this supposed to affect release build, debug build or both of them?
Apr 12 2017
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 12.04.2017 13:25, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP should occur in
 this thread. The review period will end at 11:59 PM ET on April 26 (3:59
 AM GMT), or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, the DIP
 will be scheduled for another round. Otherwise, it will be queued for
 the formal review and evaluation by the language authors.

 Thanks in advance to all who participate.

 Destroy!
The DIP should probably specify in detail how this interacts with -release. I guess the difference between "-contracts=none -boundscheck=safeonly" and" -release" is that failing assertions are UB only with the latter? What happens if I pass both -contracts=none and -release?
Apr 12 2017
parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 12 April 2017 at 12:52:42 UTC, Timon Gehr wrote:
 On 12.04.2017 13:25, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP 
 should occur in
 this thread. The review period will end at 11:59 PM ET on 
 April 26 (3:59
 AM GMT), or when I make a post declaring it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP
 will be scheduled for another round. Otherwise, it will be 
 queued for
 the formal review and evaluation by the language authors.

 Thanks in advance to all who participate.

 Destroy!
The DIP should probably specify in detail how this interacts with -release. I guess the difference between "-contracts=none -boundscheck=safeonly" and" -release" is that failing assertions are UB only with the latter? What happens if I pass both -contracts=none and -release?
Addressing Daniel Kozak's question as well here: Providing `-release` on the command line has the following effect: - Disable invariants, - Disable in and out, - Disable assert, - Disable switch error [1] Which `-contract` just allows more control on. TL;DR: It affects all build. It's a subset of `-release`, so passing `-contracts=whatever` and `-release` has the same effect as passing `-release`. [1] Switch error: Compiler code here: https://github.com/dlang/dmd/blob/0158d32fcbbdda16b3f79e73f2f1f8c13afb9f6d/src/ddmd/statementsem.d#L2120-L2145 When a `switch has no `default,` in debug it'll insert a `default` which throws a `core.exception : SwitchError`, and in release the generated `default` will just be an `HLT` (same as `assert(0)`).
Apr 12 2017
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 12.04.2017 17:16, Mathias Lang wrote:

 Addressing Daniel Kozak's question as well here:

 Providing `-release` on the command line has the following effect:
 - Disable invariants,
 - Disable in and out,
 - Disable assert,
Actually, (at least) asserts are turned into compiler hints (i.e. potential UB).
 - Disable switch error [1]
 ...
I have missed that one. Note that it also implies -boundscheck=safeonly.
 Which `-contract` just allows more control on.

 TL;DR: It affects all build. It's a subset of `-release`, so passing
 `-contracts=whatever` and `-release` has the same effect as passing
 `-release`.
It's not a subset unless "disable" can mean "turn failures into undefined behaviour".
Apr 12 2017
prev sibling next sibling parent reply Andrea Fontana <nospam example.com> writes:
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on April 26 (3:59 AM GMT), or when I make a post declaring it 
 complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it will 
 be queued for the formal review and evaluation by the language 
 authors.

 Thanks in advance to all who participate.

 Destroy!
Why not --disable-contracts=invariant --disable-contracts=inout,invariants --disable-contracts=asserts,inout And so on?
Apr 12 2017
parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 12 April 2017 at 13:45:08 UTC, Andrea Fontana wrote:
 On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP 
 should occur in this thread. The review period will end at 
 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post 
 declaring it complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it 
 will be queued for the formal review and evaluation by the 
 language authors.

 Thanks in advance to all who participate.

 Destroy!
Why not --disable-contracts=invariant --disable-contracts=inout,invariants --disable-contracts=asserts,inout And so on?
It was a conscious decision to provide something simple to use, over something which allowed more control (good old KISS). If a use case for it develop in the future, the addition will be trivial.
Apr 12 2017
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Wednesday, 12 April 2017 at 15:37:14 UTC, Mathias Lang wrote:
 It was a conscious decision to provide something simple to use, 
 over something which allowed more control (good old KISS). If a 
 use case for it develop in the future, the addition will be 
 trivial.
Well, it's not simple to use if it doesn't fulfil your use-case. ;-) With that in mind, it would seem simpler overall to not make assumptions about use-cases, and just allow the user a free choice of what kinds of contract they disable: --disable-contracts=invariant,in,out,assert,all (Yes, I'm intentionally suggesting allowing `--disable-contracts=in`, `--disable-contracts=out`, and `--disable-contracts=in,out`.)
Apr 12 2017
prev sibling next sibling parent reply Lewis <musicaljelly gmail.com> writes:
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md

 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on April 26 (3:59 AM GMT), or when I make a post declaring it 
 complete.

 At the end of Round 1, if further review is deemed necessary, 
 the DIP will be scheduled for another round. Otherwise, it will 
 be queued for the formal review and evaluation by the language 
 authors.

 Thanks in advance to all who participate.

 Destroy!
I have to ask the newbie question, just to make sure we're not missing anything obvious. Why can't we fix invariants so that they're pay-for-what-you-use? In other words, is there a way we can make sure _d_invariant is never called (or early-outs) for classes that don't use invariants?
Apr 12 2017
parent reply Mathias Lang <mathias.lang sociomantic.com> writes:
On Wednesday, 12 April 2017 at 16:22:00 UTC, Lewis wrote:
 I have to ask the newbie question, just to make sure we're not 
 missing anything obvious. Why can't we fix invariants so that 
 they're pay-for-what-you-use? In other words, is there a way we 
 can make sure _d_invariant is never called (or early-outs) for 
 classes that don't use invariants?
There's no newbie question :) Sadly it is not possible. Consider the following hierarchy: ``` class Mother : Object { public int i; public void myFunction () {} } class Daughter1 : Mother { invariant () { assert(i != 0); } } class Daughter2 : Mother {} ``` The `Mother` class needs to insert invariant checks at the beginning and the end of `myFunction` to account for the possibility of a derived class defining invariant (here `Daughter1`). However those checks are superfluous if no `invariant` is defined, as it's the case with `Daughter2`. However the base class cannot know this in advance (because it might be in a library and already compiler, for example).
Apr 12 2017
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/12/17 12:34 PM, Mathias Lang wrote:
 On Wednesday, 12 April 2017 at 16:22:00 UTC, Lewis wrote:
 I have to ask the newbie question, just to make sure we're not missing
 anything obvious. Why can't we fix invariants so that they're
 pay-for-what-you-use? In other words, is there a way we can make sure
 _d_invariant is never called (or early-outs) for classes that don't
 use invariants?
There's no newbie question :) Sadly it is not possible. Consider the following hierarchy: ``` class Mother : Object { public int i; public void myFunction () {} } class Daughter1 : Mother { invariant () { assert(i != 0); } } class Daughter2 : Mother {} ``` The `Mother` class needs to insert invariant checks at the beginning and the end of `myFunction` to account for the possibility of a derived class defining invariant (here `Daughter1`). However those checks are superfluous if no `invariant` is defined, as it's the case with `Daughter2`. However the base class cannot know this in advance (because it might be in a library and already compiler, for example).
You can check the vtable address against the known Object.invariant address. To explain further (no idea what the symbol names are, but you get the idea): executeInvariant(Object o) { static so = new Object; // has default invariant if((&so.__invariant).funcptr != &(so.__invariant).funcptr) // need to call invariant, not the default o.__invariant(); } I'm sure the compiler can do this without any indirections or virtual calls. Better yet, just put null in the Object.invariant vtable entry. Then it's really easy! -Steve
Apr 27 2017
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Apr 12, 2017 at 11:25:09AM +0000, Mike Parker via Digitalmars-d wrote:
 DIP 1006 is titled "Providing more selective control over contracts".
 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md
 
 All review-related feedback on and discussion of the DIP should occur
 in this thread. The review period will end at 11:59 PM ET on April 26
 (3:59 AM GMT), or when I make a post declaring it complete.
 
 At the end of Round 1, if further review is deemed necessary, the DIP
 will be scheduled for another round. Otherwise, it will be queued for
 the formal review and evaluation by the language authors.
 
 Thanks in advance to all who participate.
 
 Destroy!
Overall, I support the idea of this DIP. However, as others have mentioned, it needs to make it clear whether/how `-contracts=assert` here interacts with unittests. According to the discussion, apparently a different druntime function is used for asserts in unittests? If so, this needs to be clearly stated in the DIP. T -- People demand freedom of speech to make up for the freedom of thought which they avoid. -- Soren Aabye Kierkegaard (1813-1855)
Apr 12 2017
parent Olivier FAURE <olivier.faure epitech.eu> writes:
On Wednesday, 12 April 2017 at 17:16:33 UTC, H. S. Teoh wrote:
 Overall, I support the idea of this DIP.

 However, as others have mentioned, it needs to make it clear 
 whether/how `-contracts=assert` here interacts with unittests. 
 According to the discussion, apparently a different druntime 
 function is used for asserts in unittests? If so, this needs to 
 be clearly stated in the DIP.
Agreed. The simplest behavior I could imagine for this switch is "override everything else". That is, no matter which other switch is used (release, unittests), the -contracts switch has the final say on which tests are enabled and which are suppressed.
Apr 26 2017
prev sibling next sibling parent reply Mike Parker <aldacron gmail.com> writes:
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on April 26 (3:59 AM GMT), or when I make a post declaring it 
 complete.
Reminder: There's one week remaining.
Apr 19 2017
parent Mike Parker <aldacron gmail.com> writes:
On Wednesday, 19 April 2017 at 13:28:03 UTC, Mike Parker wrote:
 On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 All review-related feedback on and discussion of the DIP 
 should occur in this thread. The review period will end at 
 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post 
 declaring it complete.
Reminder: There's one week remaining.
Final reminder: The review period ends in a little over 24 hours.
Apr 25 2017
prev sibling next sibling parent Mike Parker <aldacron gmail.com> writes:
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 All review-related feedback on and discussion of the DIP should 
 occur in this thread. The review period will end at 11:59 PM ET 
 on April 26 (3:59 AM GMT), or when I make a post declaring it 
 complete.
The review period has ended. Thanks to everyone who provided feedback.
Apr 27 2017
prev sibling next sibling parent reply Martin Nowak <code dawg.eu> writes:
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".

 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md
Has come up a couple of times and it's a good idea to allow more control over which checks are enabled. I find the suggested switch levels a bit counter-intuitive and would suggest -release=assert,in,out,invariant to be the equivalent of the current -release while allowing to enable any subset, e.g. -release=assert,in. The effect of -release=assert -release=in would be to enable both. Using -release= also avoids any confusion about the interaction of -release and -contracts=.
Nov 21 2017
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Tuesday, 21 November 2017 at 14:15:30 UTC, Martin Nowak wrote:
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md
Has come up a couple of times and it's a good idea to allow more control over which checks are enabled. I find the suggested switch levels a bit counter-intuitive and would suggest -release=assert,in,out,invariant to be the equivalent of the current -release while allowing to enable any subset
I like the idea of specializing the meaning of the -release flag via optional arguments. One suggestion: replace -release=assert with -release=body, so in the above, you would have: -release=body,in,out,invariant ... which has the nice intuitive property of specifying _which bits of code_ release criteria will be applied to. In other words, -release=body would result in asserts being removed from function bodies _and only there_. That would make clearer that we're not removing asserts from e.g. unittests (or indeed contracts or invariants).
Nov 26 2017
next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 26/11/2017 11:59 AM, Joseph Rushton Wakeling wrote:
 On Tuesday, 21 November 2017 at 14:15:30 UTC, Martin Nowak wrote:
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md
Has come up a couple of times and it's a good idea to allow more control over which checks are enabled. I find the suggested switch levels a bit counter-intuitive and would suggest   -release=assert,in,out,invariant to be the equivalent of the current   -release while allowing to enable any subset
I like the idea of specializing the meaning of the -release flag via optional arguments. One suggestion: replace -release=assert with -release=body, so in the above, you would have:     -release=body,in,out,invariant ... which has the nice intuitive property of specifying _which bits of code_ release criteria will be applied to. In other words, -release=body would result in asserts being removed from function bodies _and only there_.  That would make clearer that we're not removing asserts from e.g. unittests (or indeed contracts or invariants).
Agreed that looks good +1
Nov 26 2017
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 26 November 2017 at 12:09:37 UTC, rikki cattermole 
wrote:
 On 26/11/2017 11:59 AM, Joseph Rushton Wakeling wrote:
 One suggestion: replace -release=assert with -release=body, so 
 in the above, you would have:
 
      -release=body,in,out,invariant
 
 ... which has the nice intuitive property of specifying _which 
 bits of code_ release criteria will be applied to.
 
 In other words, -release=body would result in asserts being 
 removed from function bodies _and only there_.  That would 
 make clearer that we're not removing asserts from e.g. 
 unittests (or indeed contracts or invariants).
Agreed that looks good +1
What would be the appropriate way to follow up on that idea? The last I saw DIP 1006 was undergoing formal review, but the end of that period seems to have passed with no further follow-up. I could always write up an alternative DIP, but I don't know if that would be the most constructive way forward.
Nov 27 2017
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 27/11/2017 7:20 PM, Joseph Rushton Wakeling wrote:
 On Sunday, 26 November 2017 at 12:09:37 UTC, rikki cattermole wrote:
 On 26/11/2017 11:59 AM, Joseph Rushton Wakeling wrote:
 One suggestion: replace -release=assert with -release=body, so in the 
 above, you would have:

      -release=body,in,out,invariant

 ... which has the nice intuitive property of specifying _which bits 
 of code_ release criteria will be applied to.

 In other words, -release=body would result in asserts being removed 
 from function bodies _and only there_.  That would make clearer that 
 we're not removing asserts from e.g. unittests (or indeed contracts 
 or invariants).
Agreed that looks good +1
What would be the appropriate way to follow up on that idea?  The last I saw DIP 1006 was undergoing formal review, but the end of that period seems to have passed with no further follow-up. I could always write up an alternative DIP, but I don't know if that would be the most constructive way forward.
Contact Mike&Author of the DIP.
Nov 27 2017
prev sibling parent Mike Parker <aldacron gmail.com> writes:
On Monday, 27 November 2017 at 19:20:53 UTC, Joseph Rushton 
Wakeling wrote:

 What would be the appropriate way to follow up on that idea?  
 The last I saw DIP 1006 was undergoing formal review, but the 
 end of that period seems to have passed with no further 
 follow-up.
Formal reviews have two stages. The first stage is just a courtesy to allow last minute feedback before I ship the DIP to Walter & Andrei. After I send it to them, I await their decision and then I report it here. In this case, only the first stage was completed. It's in their hands and as soon as I receive a decision I'll report it here.
 I could always write up an alternative DIP, but I don't know if 
 that would be the most constructive way forward.
I would say that's the thing to do, but it's probably best to wait until we get a decision on 1006.
Nov 27 2017
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On Sunday, 26 November 2017 at 11:59:28 UTC, Joseph Rushton 
Wakeling wrote:
 One suggestion: replace -release=assert with -release=body, so 
 in the above, you would have:

     -release=body,in,out,invariant
Doesn't really work that way, we can disable assertions, in contracts, out contracts, and invariants. But not assertions in some contexts while leaving them enabled in other contexts. At least not without modifying all related codegen and introducing context queries (e.g. think mixin templates). FWIW -release=assert,in,out,invariant fits out needs well enough. Just the use-case that someone wants to disable asserts in functions but still wants to use contracts, required to use a replacement for assert in contracts and invariants.
Mar 03
parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 3 March 2018 at 16:33:00 UTC, Martin Nowak wrote:
 Doesn't really work that way, we can disable assertions, in 
 contracts, out contracts, and invariants. But not assertions in 
 some contexts while leaving them enabled in other contexts. At 
 least not without modifying all related codegen and introducing 
 context queries (e.g. think mixin templates).
That's a shame, but presumably the fine-grainedness could be extended at some point. Question: what would -release=assert do to unittests? Would it not touch them at all? Or would it disable all asserts including in unittests?
 FWIW -release=assert,in,out,invariant fits out needs well 
 enough.
 Just the use-case that someone wants to disable asserts in 
 functions but still wants to use contracts, required to use a 
 replacement for assert in contracts and invariants.
Yea, there are obviously workarounds. I think the main concern from my side is to not have hierarchical assumptions about what gets turned on or off, and AFAICS -release=assert,in,out-invariant pretty much fits that.
Mar 05
parent Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 5 March 2018 at 18:44:54 UTC, Joseph Rushton Wakeling 
wrote:
 On Saturday, 3 March 2018 at 16:33:00 UTC, Martin Nowak wrote:
 Doesn't really work that way, we can disable assertions, in 
 contracts, out contracts, and invariants. But not assertions 
 in some contexts while leaving them enabled in other contexts. 
 At least not without modifying all related codegen and 
 introducing context queries (e.g. think mixin templates).
That's a shame, but presumably the fine-grainedness could be extended at some point. Question: what would -release=assert do to unittests? Would it not touch them at all? Or would it disable all asserts including in unittests?
From memory, it would turn off asserts even in unittests. You could raise a bug against gdc for that as it's a reasonable suggestion.
 FWIW -release=assert,in,out,invariant fits out needs well 
 enough.
 Just the use-case that someone wants to disable asserts in 
 functions but still wants to use contracts, required to use a 
 replacement for assert in contracts and invariants.
Yea, there are obviously workarounds. I think the main concern from my side is to not have hierarchical assumptions about what gets turned on or off, and AFAICS -release=assert,in,out-invariant pretty much fits that.
N.B: GDC has -f[no]-release, -f[no-]assert, -f[no-]invariant, -f[no-]preconditions, and -f[no-]postconditions (-f[no-]in and -f[no-]out were removed as they are a little too vague). And it doesn't matter which order you pass them in, if an option is explicitly set, then they do not get turned on/off by -frelease.
Mar 05
prev sibling next sibling parent Martin Nowak <code dawg.eu> writes:
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".
Possible implementation https://github.com/dlang/dmd/pull/7980 FYI
Mar 03
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/12/2017 4:25 AM, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over contracts".
 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md
Currently, we have 3 switches that affect the asserts: `release`, `boundscheck`, and `unittest`. The documentation for these new additions is completely confusing. No mention is made of interactions with the latter two. No mention is made on what happens if more than one of these switches is used, or what if the same switch appears multiple times. I'd like to see a chart enumerating all the behaviors influenced by these switches, the default behavior, and which switch settings do what. Each switch, when applied, should specify what happens to each behavior: 1. turned on 2. turned off 3. not affected and that the switches are applied in the order they appear on the command line. The use of comma-separated arguments is something I've argued against for other switches. The use of `-release=in -release=out` should be fine and is less confusing/buggy to implement. The idea that `-release=in` actually turns *off* `in` is completely topsy-turvy. Ideally, the whole thing should be simplified to its fundamentals: 1. leave the old switches as they are 2. Add a new switch, let's say "check": -check turn on all checks -check=on turn on all checks -check=off turn off all checks -check=xxx turn on check xxx -check=yyy turn on check yyy simple, easy to explain, easy to comprehend, easy to implement, easy to extend with new checks.
Mar 03
parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 4 March 2018 at 00:32:20 UTC, Walter Bright wrote:
 On 4/12/2017 4:25 AM, Mike Parker wrote:
 DIP 1006 is titled "Providing more selective control over 
 contracts".
 
 https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md
Currently, we have 3 switches that affect the asserts: `release`, `boundscheck`, and `unittest`. The documentation for these new additions is completely confusing. No mention is made of interactions with the latter two. No mention is made on what happens if more than one of these switches is used, or what if the same switch appears multiple times. I'd like to see a chart enumerating all the behaviors influenced by these switches, the default behavior, and which switch settings do what. Each switch, when applied, should specify what happens to each behavior: 1. turned on 2. turned off 3. not affected and that the switches are applied in the order they appear on the command line. The use of comma-separated arguments is something I've argued against for other switches. The use of `-release=in -release=out` should be fine and is less confusing/buggy to implement.
Why? Implementation is trivial (unit testable no less!) , see https://github.com/dlang/dmd/pull/7863 and extensible to other args.
 The idea that `-release=in` actually turns *off* `in` is 
 completely topsy-turvy.

 Ideally, the whole thing should be simplified to its 
 fundamentals:

 1. leave the old switches as they are
 2. Add a new switch, let's say "check":

    -check     turn on all checks
    -check=on  turn on all checks
    -check=off turn off all checks
    -check=xxx turn on check xxx
    -check=yyy turn on check yyy

 simple, easy to explain, easy to comprehend, easy to implement, 
 easy to extend with new checks.
I think aggregation is a good idea. So is phrasing in the positive sense, except for it looks like xxx and yyy are _off_ by default, where surely we want all checks _on_ by default.
Mar 03
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/3/2018 5:29 PM, Nicholas Wilson wrote:
 The use of comma-separated arguments is something I've argued against for 
 other switches. The use of `-release=in -release=out` should be fine and is 
 less confusing/buggy to implement.
Why?  Implementation is trivial (unit testable no less!) , see https://github.com/dlang/dmd/pull/7863 and extensible to other args.
It's not trivial. First, it's 100 lines of code (with no comments) just for that, and there are templates and behaviors with memory allocation. Moving along that path, we're gradually reinventing std.getopt which is 1814 lines (with comments). Generally, people will be driving dmd with a makefile, dmd.conf, or other response file. It's complicated enough already, and gets constantly worse. Once in, we're stuck with it forever. I just don't feel it is worthwhile spending time on this.
 I think aggregation is a good idea. So is phrasing in the positive sense,
except 
 for it looks like xxx and yyy are _off_ by default, where surely we want all 
 checks _on_ by default.
The default is that they're all on. So to just have one on, first turn them all off then turn on the desired ones.
Mar 03
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 4 March 2018 at 04:30:31 UTC, Walter Bright wrote:
 It's not trivial. First, it's 100 lines of code (with no 
 comments) just for that, and there are templates and behaviors 
 with memory allocation. Moving along that path, we're gradually 
 reinventing std.getopt which is 1814 lines (with comments).
I disagree, it is 100 (slightly less) lines of modular code and can be used for -i,-I,-J,-check and possibly a whole lot more. Yes there are no comments but it's not exactly difficult to follow. You get that from modular code. parseCommandLine is 746! and is much more complicated. I'm very glad that in LDC we have access to LLVM command line arguments which are purely declarative.
 Generally, people will be driving dmd with a makefile, 
 dmd.conf, or other response file.
You still have to write those.
It's complicated enough already, and gets constantly worse.
And it won't get simpler unless anything is done about it!
 Once in, we're stuck with it forever.
We can (and do) deprecate things.
 I just don't feel it is worthwhile spending time on this.
Then the situation will not improve. It is a shame to lose this excellent work.
 I think aggregation is a good idea. So is phrasing in the 
 positive sense, except for it looks like xxx and yyy are _off_ 
 by default, where surely we want all checks _on_ by default.
The default is that they're all on.
Good.
 So to just have one on, first turn them all off then turn on 
 the desired ones.
Not so good. That is the opposite pattern of use I would expect. I would much rather turn them off individually, because I might forget one or a new release might add more and then it is off by default. Not to mention it result in way more args and the intention is much less clear . Looking at a command line invocation of DMD could you figure out which checks are off by looking at: ` -checks=off -checks=boundscheck -checks=in -checks=out -checks=unittest` don't forget they could be interspersed between other args _and_ be in the conf file. The types of checks are a closed set, it makes sense to specify them all at once.
Mar 03
prev sibling parent reply rjframe <dlang ryanjframe.com> writes:
On Sat, 03 Mar 2018 20:30:31 -0800, Walter Bright wrote:

 The default is that they're all on. So to just have one on, first turn
 them all off then turn on the desired ones.
Would I be correct to interpret this as "turn them all off with -release"? E.g., "dmd -check=in a.d" is unnecessary; it's equivalent to "dmd a.d" "dmd -release -check=in a.d" turns off all checks except in contracts. If adding any check turns everything else off, that seems strange; that said, wouldn't -check=off be equivalent to -release? Do we need -check=off and -check=on, since it just duplicates the -release toggle? I do like adding the -check switch because it reads far more naturally than -release=in, etc. But as far as readability, only using -check once -release is passed to disable everything makes sense, as using -check shouldn't disable other checks, and -release shows intention better than starting with -check=off --Ryan
Mar 04
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/4/2018 4:05 AM, rjframe wrote:
 Would I be correct to interpret this as "turn them all off with -release"?
Array bounds checking is left on with -release.
Mar 04
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04.03.2018 21:40, Walter Bright wrote:
 On 3/4/2018 4:05 AM, rjframe wrote:
 Would I be correct to interpret this as "turn them all off with 
 -release"?
Array bounds checking is left on with -release.
Not necessarily. If the code contains an explicit assertion that the index is in bounds, then, according to the language specification, the bounds check may be removed with -release. I find the reasoning in terms of "on"/"off" confusing anyway. Does "off" mean "contract/assertion removed", or does it mean "failure is UB"?
Mar 04
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/4/2018 1:16 PM, Timon Gehr wrote:
 On 04.03.2018 21:40, Walter Bright wrote:
 On 3/4/2018 4:05 AM, rjframe wrote:
 Would I be correct to interpret this as "turn them all off with -release"?
Array bounds checking is left on with -release.
Not necessarily. If the code contains an explicit assertion that the index is in bounds, then, according to the language specification, the bounds check may be removed with -release.
D, as all languages that I know of do implicitly or explicitly, generates code based on the "as if" rule.
 I find the reasoning in terms of "on"/"off" confusing anyway.
 Does "off" mean "contract/assertion removed", or does it mean "failure is UB"?
"Off" means the check is removed. If the check does not hold, the program enters an invalid state, whether or not the check was actually done. An invalid state means subsequent execution is UB.
Mar 04
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04.03.2018 22:49, Walter Bright wrote:
 On 3/4/2018 1:16 PM, Timon Gehr wrote:
 On 04.03.2018 21:40, Walter Bright wrote:
 On 3/4/2018 4:05 AM, rjframe wrote:
 Would I be correct to interpret this as "turn them all off with 
 -release"?
Array bounds checking is left on with -release.
Not necessarily. If the code contains an explicit assertion that the index is in bounds, then, according to the language specification, the bounds check may be removed with -release.
D, as all languages that I know of do implicitly or explicitly, generates code based on the "as if" rule. ...
Impossible. You wrote a Java compiler. All languages that use your "as if" rule are memory unsafe. Zero languages that use the "as if" rule have any memory safe subset that includes assertions. In D, assert is safe, and it should remain safe.
 
 I find the reasoning in terms of "on"/"off" confusing anyway.
 Does "off" mean "contract/assertion removed", or does it mean "failure 
 is UB"?
"Off" means the check is removed. If the check does not hold, the program enters an invalid state, whether or not the check was actually done. An invalid state means subsequent execution is UB.
Why is potential memory corruption to be expected when using safe language features with a flag to disable contract checks? This makes no sense. This is not useful behavior. There are convenient ways to support potentially unsound compilation hints that do not do this. Contracts and compilation hints should be orthogonal. Contracts should be potentially safe, compilation hints should be system always. Note that _actual removal_ is the only use case of 'disabling contracts' that I care about, and I think many D programmers who use "off" will also have this behavior in mind. Yet this is not even an option. At the very least, the DIP should be up-front about this. I'm still not even sure that Mathias Lang intended the UB semantics.
Mar 04
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/4/2018 3:06 PM, Timon Gehr wrote:
 On 04.03.2018 22:49, Walter Bright wrote:
 Not necessarily. If the code contains an explicit assertion that the index is 
 in bounds, then, according to the language specification, the bounds check 
 may be removed with -release.
D, as all languages that I know of do implicitly or explicitly, generates code based on the "as if" rule. ...
Impossible. You wrote a Java compiler.
Even in Java, the compiler generates code that, from the user's point of view, behaves "as if" the code was actually what was specified. For a trivial example, replacing x*2 with x<<1. Not having this means no optimizations can be done.
 All languages that use your "as if" rule are memory unsafe.
 Zero languages that use the "as if" rule have any memory safe subset that 
 includes assertions.
 In D, assert is  safe, and it should remain  safe.
 I find the reasoning in terms of "on"/"off" confusing anyway.
 Does "off" mean "contract/assertion removed", or does it mean "failure is UB"?
"Off" means the check is removed. If the check does not hold, the program enters an invalid state, whether or not the check was actually done. An invalid state means subsequent execution is UB.
Why is potential memory corruption to be expected when using safe language features with a flag to disable contract checks?
Because the checks provide extra information to the compiler that it can use to generate better code. If that extra information is not true, then the better code will be invalid. Memory safety is only one class of errors in a program. If the program has entered a state that is not accounted for by the programmer, the rest of the program's execution will be not predictable.
 This makes no sense. This is 
 not useful behavior. There are convenient ways to support potentially unsound 
 compilation hints that do not do this. Contracts and compilation hints should
be 
 orthogonal. Contracts should be potentially  safe, compilation hints should be 
  system always.
 
 Note that _actual removal_ is the only use case of 'disabling contracts' that
I 
 care about, and I think many D programmers who use "off" will also have this 
 behavior in mind. Yet this is not even an option.
I don't see much use for this behavior, unless you want to continue running the program after an assert failure, which I cannot recommend and the language is not designed to support. But you can always do something like: version (ignore_asserts) { } else { assert(...); } which would optionally remove both the runtime check and any compiler use of the assert. Or you could use https://dlang.org/library/std/exception/enforce.html which has no influence on compiler semantics.
 At the very least, the DIP should be up-front about this.
 I'm still not even sure that Mathias Lang intended the UB semantics.
It being UB was my doing, not Mathias'. DIP1006 is not redefining the semantics of what assert does.
Mar 05
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
The idea behind removal of the runtime checks is as a performance optimization 
done on a debugged program. It's like turning on or off array bounds checking. 
Many leave asserts and array bounds checking on even in released code to ensure 
memory safety.

At a minimum, turning it off and on will illuminate just what the checks are 
costing you.

It's at the option of the programmer.
Mar 05
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05.03.2018 11:30, Walter Bright wrote:
 The idea behind removal of the runtime checks is as a performance 
 optimization done on a debugged program.
Optimizing performance is fine, but why pessimize safety? The hints will usually not make a significant difference in performance anyway. I guess it is fine to have a compiler option that is all speed no safety, but it should not be the only option.
 It's like turning on or off array bounds checking.
It is not. void main() safe{ int[] x=[]; writeln(x[0]); // range violation even with -release // defined behavior even with -boundscheck=off (!) } If I now add an assertion, I suddenly get UB: void main() safe{ int[] x=[]; assert(0<x.length); // obviously this should hold, or next line is invalid writeln(x[0]); // UB with -release } I did not make the code any more wrong by adding the assertion. Why should I get more UB?
 Many leave asserts and array bounds checking on 
 even in released code to ensure memory safety.
 ...
Maybe the requirements change and it is now too costly to leave contracts on in release mode, or the number of contracts in the code base slowly accumulates until we reach a point where the total cost is too large, or we replace a library, and the new version has costly contracts, etc. Now we have the following options: - Leave contracts in -- fail performance requirements. - Remove contracts -- fail safety requirements. - Track down all 'assert's, even those in external libraries, and replace them by a custom home-cooked solution that is incompatible with everyone else's -- fail maintainability requirements. To me this situation is ridiculous.
 At a minimum, turning it off and on will illuminate just what the checks 
 are costing you.
 ...
Well, no. If the bug is elusive enough to not have shown up in debug mode, it probably won't be seen early during -release testing, and even if it does, the UB may mask it. (Note that when the program becomes faster, the likelihood of timing-dependent bugs showing up may change.) I.e., if something goes wrong, it is likely that you won't see the safety costs until it is too late.
 It's at the option of the programmer.
It is not, but I want it to be. That's all I want. [1] I'm just saying there should be the following option: - Remove contracts -- sufficient performance and retain memory safety. FWIW, this is what all contract systems that I'm aware of do, except D, and maybe C asserts in certain implementations (if you want to call that contracts). [1] Well, maybe add a system "__assume" intrinsic.
Mar 05
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/5/2018 11:34 AM, Timon Gehr wrote:
 On 05.03.2018 11:30, Walter Bright wrote:
The hints will usually not make a significant difference in performance anyway.
Reasonable people will disagree on what is significant or not.
 It's like turning on or off array bounds checking.
It is not. void main() safe{      int[] x=[];      writeln(x[0]); // range violation even with -release                     // defined behavior even with -boundscheck=off (!)
It is not defined behavior with -boundscheck=off.
 }
 
 If I now add an assertion, I suddenly get UB:
 
 void main() safe{
      int[] x=[];
      assert(0<x.length); // obviously this should hold, or next line is
invalid
      writeln(x[0]); // UB with -release
 }
 
 I did not make the code any more wrong by adding the assertion.
 Why should I get more UB?
Because you put in an assert that did not hold, and disabled the check.
 Now we have the 
 following options:
 
 - Leave contracts in -- fail performance requirements.
 
 - Remove contracts -- fail safety requirements.
 
 - Track down all 'assert's, even those in external libraries, and replace them 
 by a custom home-cooked solution that is incompatible with everyone else's -- 
 fail maintainability requirements.
 
 To me this situation is ridiculous.
It's completely under the control of the programmer. I know you disagree with that notion. You can even create your own `myassert` to produced your desired semantics.
 FWIW, this is what all contract systems that I'm aware of do, except D, and 
 maybe C asserts in certain implementations (if you want to call that
contracts).
D is better (!). (C's asserts are not part of the language, so impart no semantics to the compiler.)
Mar 05
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On 03/05/2018 10:11 PM, Walter Bright wrote:
 On 3/5/2018 11:34 AM, Timon Gehr wrote:
[...]
       int[] x=[];
       writeln(x[0]); // range violation even with -release
                      // defined behavior even with
-boundscheck=off (!)
It is not defined behavior with -boundscheck=off.
Dereferencing null is not defined with -boundscheck=off?
Mar 05
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05.03.2018 22:24, ag0aep6g wrote:
 On 03/05/2018 10:11 PM, Walter Bright wrote:
 On 3/5/2018 11:34 AM, Timon Gehr wrote:
[...]
       int[] x=[];
       writeln(x[0]); // range violation even with -release
                      // defined behavior even with
-boundscheck=off (!)
It is not defined behavior with -boundscheck=off.
Dereferencing null is not defined with -boundscheck=off?
This was my bad. It's not dereferencing null. The compiler is free to assume 0<x.length, which means it is allowed to think that the main function is dead code. Anyway, a similar point can be made by considering contracts that say that specific values are non-null. They will turn null values into UB even though without them, null dereferences would have been defined to crash.
Mar 05
parent reply ag0aep6g <anonymous example.com> writes:
On 03/05/2018 11:57 PM, Timon Gehr wrote:
 On 05.03.2018 22:24, ag0aep6g wrote:
 On 03/05/2018 10:11 PM, Walter Bright wrote:
[...]
 It is not defined behavior with -boundscheck=off.
Dereferencing null is not defined with -boundscheck=off?
This was my bad. It's not dereferencing null. The compiler is free to assume 0<x.length, which means it is allowed to think that the main function is dead code.
How is it free to assume that? This was the full snippet (before I mutilated it in my quote): ---- void main() safe{ int[] x=[]; writeln(x[0]); // range violation even with -release // defined behavior even with -boundscheck=off (!) } ---- There is no `assert(0<x.length);` in this one. -release doesn't do anything, because there are no contracts, no asserts, and main is safe. -boundscheck=off just makes it so that the length isn't checked before x.ptr is dereferenced. x.ptr is null, so the code is defined to dereference null, no? If -boundscheck=off somehow does introduce UB here, we have the weird situation that using `x.ptr[0]` is more safe than in this scenario than `x[0]`. Because surely `x.ptr[0]` is a null dereference that's not affected by -boundscheck=off, right?
Mar 05
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06.03.2018 00:52, ag0aep6g wrote:
 On 03/05/2018 11:57 PM, Timon Gehr wrote:
 On 05.03.2018 22:24, ag0aep6g wrote:
 On 03/05/2018 10:11 PM, Walter Bright wrote:
[...]
 It is not defined behavior with -boundscheck=off.
Dereferencing null is not defined with -boundscheck=off?
This was my bad. It's not dereferencing null. The compiler is free to assume 0<x.length, which means it is allowed to think that the main function is dead code.
How is it free to assume that? ...
By Walter's definition. -boundscheck=off makes the compiler assume that all array accesses are within bounds. ("off" is a misleading term.)
 This was the full snippet (before I mutilated it in my quote):
 
 ----
 void main() safe{
       int[] x=[];
       writeln(x[0]); // range violation even with -release
                      // defined behavior even with
-boundscheck=off (!)
 }
 ----
 
 There is no `assert(0<x.length);` in this one. -release doesn't do 
 anything, because there are no contracts, no asserts, and main is  safe. 
 -boundscheck=off just makes it so that the length isn't checked before 
 x.ptr is dereferenced.
It's not checked, but the compiler may still assume that it has actually been checked. The story is similar to asserts.
 x.ptr is null, so the code is defined to 
 dereference null, no?
 
 If -boundscheck=off somehow does introduce UB here, we have the weird 
 situation that using `x.ptr[0]` is more safe than in this scenario than 
 `x[0]`. Because surely `x.ptr[0]` is a null dereference that's not 
 affected by -boundscheck=off, right?
Yes, I think that's a good point (though it hinges on the assumption that x.ptr[i] is equivalent to *(x.ptr+i), which I'm not sure the specification states explicitly).
Mar 05
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05.03.2018 22:11, Walter Bright wrote:
 On 3/5/2018 11:34 AM, Timon Gehr wrote:
 On 05.03.2018 11:30, Walter Bright wrote:
 The hints will usually not make a significant difference in 
 performance anyway.
Reasonable people will disagree on what is significant or not. ...
My point exactly! Hence, compiler flag.
 ...
 I did not make the code any more wrong by adding the assertion.
 Why should I get more UB?
Because you put in an assert that did not hold, and disabled the check. ...
(Maybe let's assume it was not me who did it, to stop the whole silly "you deserve what you got because you made a mistake" notion.) Again, my question is not about the _mechanics_ of the status quo. I know it very well. It's the rationale that matters.
 
 Now we have the following options:

 - Leave contracts in -- fail performance requirements.

 - Remove contracts -- fail safety requirements.

 - Track down all 'assert's, even those in external libraries, and 
 replace them by a custom home-cooked solution that is incompatible 
 with everyone else's -- fail maintainability requirements.

 To me this situation is ridiculous.
It's completely under the control of the programmer. I know you disagree with that notion.
I don't. I can use a manual patch of the compiler that has the additionally required flags and replicate the official packaging effort and make everyone who wants to compile my programs use that version. I just don't want to, as it seems silly. It would be a lot better if the standard DMD compiler had the flags. Do you disagree that there should be an additional option to ignore contracts completely?
 You can even create your own `myassert` to produced 
 your desired semantics.
 ...
That's the third option above. It's not a practical solution. Putting the flag into a compiler fork is trivial by comparison.
 
 FWIW, this is what all contract systems that I'm aware of do, except 
 D, and maybe C asserts in certain implementations (if you want to call 
 that contracts).
D is better (!). (C's asserts are not part of the language, so impart no semantics to the compiler.)
(That's why I said "in certain implementations".)
Mar 05
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Monday, 5 March 2018 at 10:30:12 UTC, Walter Bright wrote:
 The idea behind removal of the runtime checks is as a 
 performance optimization done on a debugged program. It's like 
 turning on or off array bounds checking. Many leave asserts and 
 array bounds checking on even in released code to ensure memory 
 safety.

 At a minimum, turning it off and on will illuminate just what 
 the checks are costing you.

 It's at the option of the programmer.
void safeCode1(int a, ref int[2] b) safe { assert(a < 2); b[a] = 0; } So, if I compile this with `-release -O`, the compiler is free to remove the bounds-check, which will cause a buffer overrun if `a
 1`. Ok.
void safeCode2(int a, ref int[2] b) safe { b[a] = 0; } And here the compiler is *not* free to remove the bounds check. This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.
Mar 05
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/5/2018 2:30 PM, John Colvin wrote:
 This just feels bad. Adding extra failsafes for my debug program shouldn't
make 
 my release program less safe.
Then use `enforce()`.
Mar 05
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06.03.2018 03:05, Walter Bright wrote:
 On 3/5/2018 2:30 PM, John Colvin wrote:
 This just feels bad. Adding extra failsafes for my debug program 
 shouldn't make my release program less safe.
Then use `enforce()`.
That makes no sense at all. Enforce is for conditions that are expected to fail in exceptional circumstances. It does not document the same intent as assert does and it cannot be ignored in release builds. Anyway, "do not use assert" is not the solution, as I have explained many times now.
Mar 06
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/6/2018 12:45 AM, Timon Gehr wrote:
 Anyway, "do not use assert" is not the solution, as I have explained many
times 
 now.
My interpretation is you want D assert to behave like C assert. C assert and enforce are purely creatures of the library, with semantics defined by their library implementation, and have no effect on the core language. I recommend creating your own library assert, call it 'check' for example, and give it the semantics you wish. You can even have it expand to nothing for release builds. Creating library asserts is why D has special support for __FILE__ and __LINE__ like C does, and for the same reasons.
Mar 06
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06.03.2018 10:02, Walter Bright wrote:
 On 3/6/2018 12:45 AM, Timon Gehr wrote:
 Anyway, "do not use assert" is not the solution, as I have explained 
 many times now.
My interpretation is you want D assert to behave like C assert.
This interpretation is wrong. I, as well as other people, want a compiler option to make the compiler ignore D asserts and contracts. Not more, not less.
 C assert 
 and enforce are purely creatures of the library, with semantics defined 
 by their library implementation, and have no effect on the core language.
 ...
That's a meaningless proposition. The current D -release assert behavior can be implemented in a library (just trigger UB in the false branch).
 I recommend creating your own library assert, call it 'check' for 
 example, and give it the semantics you wish. You can even have it expand 
 to nothing for release builds.
 ...
Well, awesome. Now I need to make everyone on the project as well as external libraries, such as Phobos, use my 'check' function, when the contract documentation tells them to use assert and does not even hint at the downsides. No, thanks. I'd rather fork the compiler. But I have explained this (and further reasons) already. I suspect you did not read my posts. A few others have. I'll try to keep this one shorter.
 Creating library asserts is why D has special support for __FILE__ and 
 __LINE__ like C does, and for the same reasons.
What I want is special support for sane built-in assert semantics using a compiler flag. That does not mean that there cannot _also_ be a flag to unleash the nasal demons upon unworthy programmers who were stupid enough to collaborate with someone who imported an external library that was written by somebody who had a bad day one time and left in a wrong assertion. Again: There is no reason why we need to force one behavior over the other. This should be configurable.
Mar 06
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 6 March 2018 at 23:50:20 UTC, Timon Gehr wrote:
 On 06.03.2018 10:02, Walter Bright wrote:
 On 3/6/2018 12:45 AM, Timon Gehr wrote:
 Anyway, "do not use assert" is not the solution, as I have 
 explained many times now.
My interpretation is you want D assert to behave like C assert.
This interpretation is wrong. I, as well as other people, want a compiler option to make the compiler ignore D asserts and contracts. Not more, not less.
Is it an option having the compiler not to remove the asserts from safe code, like bounds check? Just to understand, otherwise, if the assert is removed and it does not hold, you are in UB, so the request is to guarantee memory safety in a UB state, right? The point I don't grasp is why keep running in UB in safe is acceptable, while memory corruption not.
 Creating library asserts is why D has special support for 
 __FILE__ and __LINE__ like C does, and for the same reasons.
What I want is special support for sane built-in assert semantics using a compiler flag.
and that's a reasonable request, that, IMHO, does not hurts anybody....
 That does not mean that there cannot _also_ be a flag to 
 unleash the nasal demons upon unworthy programmers who were 
 stupid enough to collaborate with someone who imported an 
 external library that was written by somebody who had a bad day 
 one time and left in a wrong assertion.
That's a process management problem, I think. Debug version of external libraries can be requested, and the author can be reported about the bad-day wrong assert. I know, if I can catch it...
 Again: There is no reason why we need to force one behavior 
 over the other. This should be configurable.
I'm all in for having the maximum flexibility in configuration, but I would stick with Walter as keeping its idea as the default. /Paolo
Mar 07
parent reply ag0aep6g <anonymous example.com> writes:
On Wednesday, 7 March 2018 at 08:58:50 UTC, Paolo Invernizzi 
wrote:
 Just to understand, otherwise, if the assert is removed and it 
 does not hold, you are in UB,
You're not. Just let the compiler treat the code as if the asserts weren't there. If the resulting code has UB, it won't compile, because safe code is statically checked to not have UB.
 so the request is to guarantee memory safety in a UB state, 
 right?
I don't think anyone is asking for that. The request is for no UB in safe code.
Mar 07
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 7 March 2018 at 13:32:37 UTC, ag0aep6g wrote:
 On Wednesday, 7 March 2018 at 08:58:50 UTC, Paolo Invernizzi 
 wrote:
 Just to understand, otherwise, if the assert is removed and it 
 does not hold, you are in UB,
You're not. Just let the compiler treat the code as if the asserts weren't there. If the resulting code has UB, it won't compile, because safe code is statically checked to not have UB.
 so the request is to guarantee memory safety in a UB state, 
 right?
I don't think anyone is asking for that. The request is for no UB in safe code.
Are we asking to statically check things like: Assign Expressions [1] Undefined Behavior: if the lvalue and rvalue have partially overlapping storage if the lvalue and rvalue's storage overlaps exactly but the types are different Is that doable, in practise? [1] https://dlang.org/spec/expression.html#assign_expressions /Paolo
Mar 07
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, March 07, 2018 14:01:31 Paolo Invernizzi via Digitalmars-d 
wrote:
 On Wednesday, 7 March 2018 at 13:32:37 UTC, ag0aep6g wrote:
 On Wednesday, 7 March 2018 at 08:58:50 UTC, Paolo Invernizzi

 wrote:
 Just to understand, otherwise, if the assert is removed and it
 does not hold, you are in UB,
You're not. Just let the compiler treat the code as if the asserts weren't there. If the resulting code has UB, it won't compile, because safe code is statically checked to not have UB.
 so the request is to guarantee memory safety in a UB state,
 right?
I don't think anyone is asking for that. The request is for no UB in safe code.
Are we asking to statically check things like: Assign Expressions [1] Undefined Behavior: if the lvalue and rvalue have partially overlapping storage if the lvalue and rvalue's storage overlaps exactly but the types are different Is that doable, in practise? [1] https://dlang.org/spec/expression.html#assign_expressions
In places where the compiler can statically check things, it does. In the places where it can't, it either introduces runtime checks (e.g. array bounds checking), or it treats the code as system, forcing the programmer to ensure that the code is safe, since the compiler can't determine whether it is or not. Either way, we then get the guarantee that safe code is memory safe so long as trusted is used correctly. - Jonathan M Davis
Mar 07
prev sibling next sibling parent ag0aep6g <anonymous example.com> writes:
On 03/07/2018 03:01 PM, Paolo Invernizzi wrote:
 On Wednesday, 7 March 2018 at 13:32:37 UTC, ag0aep6g wrote:
[...]
 I don't think anyone is asking for that. The request is for no UB in 
  safe code.
Are we asking to statically check things like: Assign Expressions [1] Undefined Behavior:   if the lvalue and rvalue have partially overlapping storage   if the lvalue and rvalue's storage overlaps exactly but the types are different
If it can't be guaranteed that some code has defined behavior, then it's not allowed in an safe function (or it should not be allowed). We are not asking for all valid code to be safe. Guaranteeing no UB is exactly safe's purpose. The spec says: "Safe functions are functions that are statically checked to exhibit no possibility of undefined behavior." [1]
 Is that doable, in practise?
If you think that's not doable, what do you think safe should aim for? [1] https://dlang.org/spec/function.html#function-safety
Mar 07
prev sibling parent ag0aep6g <anonymous example.com> writes:
On 03/07/2018 03:01 PM, Paolo Invernizzi wrote:
 Are we asking to statically check things like:
 
 Assign Expressions [1]
 Undefined Behavior:
    if the lvalue and rvalue have partially overlapping storage
    if the lvalue and rvalue's storage overlaps exactly but the types are 
 different
A simple way to get overlapping storage is with a union. Unfortunately, DMD accepts this: ---- struct S { union { int i; byte b; float f; struct { byte b2; align(1) int i2; } } } void main() safe { S s; s.i = s.b; /* Partially overlapping, different types. */ s.f = s.i; /* Exactly overlapping, different types. */ s.i = s.i2; /* Partially overlapping, same type. */ } ---- I've filed an issue: https://issues.dlang.org/show_bug.cgi?id=18568 If you have more examples of UB in safe functions, don't hesitate to file them as bugs.
Mar 07
prev sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
 On 3/5/2018 2:30 PM, John Colvin wrote:
 This just feels bad. Adding extra failsafes for my debug 
 program shouldn't make my release program less safe.
Then use `enforce()`.
So, to clarify, adding asserts to my code makes my release builds violate safe?
Mar 06
next sibling parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 6 March 2018 at 16:30:09 UTC, John Colvin wrote:
 On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
 On 3/5/2018 2:30 PM, John Colvin wrote:
 This just feels bad. Adding extra failsafes for my debug 
 program shouldn't make my release program less safe.
Then use `enforce()`.
So, to clarify, adding asserts to my code makes my release builds violate safe?
Only if the assert does not hold, you have _not_ tested it, and a future optimiser will use the assert expression for some hints that generated broken code. At the end, I'm with Walter: you should have tested the code with the assert enabled, and you should have noticed that the assert it's not holding _before_ running the code in release without them. /Paolo
Mar 06
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, March 06, 2018 17:01:04 Paolo Invernizzi via Digitalmars-d 
wrote:
 On Tuesday, 6 March 2018 at 16:30:09 UTC, John Colvin wrote:
 On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
 On 3/5/2018 2:30 PM, John Colvin wrote:
 This just feels bad. Adding extra failsafes for my debug
 program shouldn't make my release program less safe.
Then use `enforce()`.
So, to clarify, adding asserts to my code makes my release builds violate safe?
Only if the assert does not hold, you have _not_ tested it, and a future optimiser will use the assert expression for some hints that generated broken code. At the end, I'm with Walter: you should have tested the code with the assert enabled, and you should have noticed that the assert it's not holding _before_ running the code in release without them.
The problem with this approach is that even if you have tested your code well, sometimes you miss things, and then you have bugs. And that's life, but with safe, you're not supposed to then up with memory safety problems. That's why array bounds checking is not removed in safe code even with -release. Yes, well-tested code would hopefully not need the checks anymore, but bugs still creep in, and if the array bounds checking is removed, then you can end up with memory corruption. This issue is really no different from array bounds checking, and I don't see any reason to treat it any differently. safe could should be guaranteed to be safe so long as trusted is used correctly. The compiler should not be assuming that the programmer has done a perfect job of testing and caught every possible bug and that there's no way that an assertion would have failed if it were still compiled in and that it is thus safe to introduce optimizations which would not be memory safe if the assertion would have failed. If the compiler does that, then safe is being violated, which the compiler should never be doing. - Jonathan M Davis
Mar 06
prev sibling parent ag0aep6g <anonymous example.com> writes:
On 03/06/2018 06:01 PM, Paolo Invernizzi wrote:
 Only if the assert does not hold, you have _not_ tested it,
In other words: only if you have a bug in your code. If safe is only safe as long you don't have bugs, it's no different from system. So -release turns safe code into system code, if it contains asserts.
 and a future 
 optimiser will use the assert expression for some hints that generated 
 broken code
Mar 06
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, March 06, 2018 16:30:09 John Colvin via Digitalmars-d wrote:
 On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
 On 3/5/2018 2:30 PM, John Colvin wrote:
 This just feels bad. Adding extra failsafes for my debug
 program shouldn't make my release program less safe.
Then use `enforce()`.
So, to clarify, adding asserts to my code makes my release builds violate safe?
If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous. If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner. - Jonathan M Davis
Mar 06
parent reply 12345swordy <alexanderheistermann gmail.com> writes:
On Tuesday, 6 March 2018 at 17:24:35 UTC, Jonathan M Davis wrote:
 On Tuesday, March 06, 2018 16:30:09 John Colvin via 
 Digitalmars-d wrote:
 On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
 On 3/5/2018 2:30 PM, John Colvin wrote:
 This just feels bad. Adding extra failsafes for my debug 
 program shouldn't make my release program less safe.
Then use `enforce()`.
So, to clarify, adding asserts to my code makes my release builds violate safe?
If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous. If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner. - Jonathan M Davis
I think a reasonable compromise is to introduce a new system attribute such as unsafeoptimize to tell the programmer that this function may have it's safe attribute removed when making optimizations based on the asserts. We have trusted attribute for a good reason here.
Mar 06
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 6 March 2018 at 17:34:08 UTC, 12345swordy wrote:
 On Tuesday, 6 March 2018 at 17:24:35 UTC, Jonathan M Davis 
 wrote:
 On Tuesday, March 06, 2018 16:30:09 John Colvin via 
 Digitalmars-d wrote:
 On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:
 [...]
So, to clarify, adding asserts to my code makes my release builds violate safe?
If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous. If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner. - Jonathan M Davis
I think a reasonable compromise is to introduce a new system attribute such as unsafeoptimize to tell the programmer that this function may have it's safe attribute removed when making optimizations based on the asserts. We have trusted attribute for a good reason here.
I simply don't understand why enforce or a custom check can't be used safe code, if you want that behaviour. If the program must HALT on this or that, well, what is better than an explicit piece of unremovable code that do that? Instead of writing 'assert', one should write 'enforce'. /Paolo
Mar 06
next sibling parent 12345swordy <alexanderheistermann gmail.com> writes:
On Tuesday, 6 March 2018 at 18:49:42 UTC, Paolo Invernizzi wrote:
 I simply don't understand why enforce or a custom check can't 
 be used  safe code, if you want that behaviour.

 /Paolo
Easy: the custom check itself could be wrong the custom check itself is right, but the implementation is wrong even though it may passed your unit tests.
Mar 06
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, March 06, 2018 18:49:42 Paolo Invernizzi via Digitalmars-d 
wrote:
 I simply don't understand why enforce or a custom check can't be
 used  safe code, if you want that behaviour.

 If the program must HALT on this or that, well, what is better
 than an explicit piece of unremovable code that do that? Instead
 of writing 'assert', one should write 'enforce'.
1. It's quite common for folks to want to add debugging checks that are compiled out in -release. That's exactly what assert is for in pretty much every lanugage that has it. It's what folks expect to use and what your average programmer will use without thinking about safety issues at all. It's what everyone uses right now, and I'm pretty sure that almost everyone using it has no clue that Walter considers it okay for assertions to introduce optimizations which are not memory safe, and if/when he does do so, a lot of D code will suddenly have safe code which is not memory safe. Such problems will hopefully be hit rarely, because hopefully, the code will have been well-tested, and the assertions will have found all of the related bugs, but there's every possibility that some bugs will manage to not be caught, thereby resulting in safe code being unsafe. No one is going to be looking to use solutions other than assertions for what assertions are for unless we start telling everyone to avoid assertions, because they make safe code unsafe. And honestly, if assertions make safe code unsafe, I don't see a good argument for using them at all. If I didn't care about code being safe, I wouldn't be using safe. safe is supposed to guarantee that the code is memory safe. 2. I think that it's fundamentally a terrible idea to allow built-in language features to violate safe. Aside from issues related to trusted being misused, safe code should be guaranteed to be memory safe, and it should be considered a bug any time that it isn't. That's why safe exists. No one should have to be looking at safe code to track down memory safety problems. And if they do, then safe is not doing it's job. Array bounds checks are left in safe code for exactly these reasons. I'm all for introducing optimizations that do not violate safe, but if we allow safe code to be unsafe, then why do we even have it? - Jonathan M Davis
Mar 06
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Tuesday, 6 March 2018 at 20:03:11 UTC, Jonathan M Davis wrote:
 On Tuesday, March 06, 2018 18:49:42 Paolo Invernizzi via 
 Digitalmars-d wrote:
 I simply don't understand why enforce or a custom check can't 
 be used  safe code, if you want that behaviour.

 If the program must HALT on this or that, well, what is better 
 than an explicit piece of unremovable code that do that? 
 Instead of writing 'assert', one should write 'enforce'.
1. It's quite common for folks to want to add debugging checks that are compiled out in -release. That's exactly what assert is for in pretty much every lanugage that has it. It's what folks expect to use and what your average programmer will use without thinking about safety issues at all. It's what everyone uses right now, and I'm pretty sure that almost everyone using it has no clue that Walter considers it okay for assertions to introduce optimizations which are not memory safe, and if/when he does do so, a lot of D code will suddenly have safe code which is not memory safe. Such problems will hopefully be hit rarely, because hopefully, the code will have been well-tested, and the assertions will have found all of the related bugs, but there's every possibility that some bugs will manage to not be caught, thereby resulting in safe code being unsafe. No one is going to be looking to use solutions other than assertions for what assertions are for unless we start telling everyone to avoid assertions, because they make safe code unsafe. And honestly, if assertions make safe code unsafe, I don't see a good argument for using them at all. If I didn't care about code being safe, I wouldn't be using safe. safe is supposed to guarantee that the code is memory safe.
Understood. Are asking that UB should not include memory corruptions?
 2. I think that it's fundamentally a terrible idea to allow 
 built-in language features to violate  safe. Aside from issues 
 related to  trusted being misused,  safe code should be 
 guaranteed to be memory safe, and it should be considered a bug 
 any time that it isn't. That's why  safe exists. No one should 
 have to be looking at  safe code to track down memory safety 
 problems. And if they do, then  safe is not doing it's job. 
 Array bounds checks are left in  safe code for exactly these 
 reasons.
So, the request is to just leave assert active as a default in safe code, like the bounds checks?
 I'm all for introducing optimizations that do not violate 
  safe, but if we allow  safe code to be unsafe, then why do we 
 even have it?
So, the reasoning is that UB should not lead to memory corruption, right? /P
Mar 07
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, March 07, 2018 08:39:30 Paolo Invernizzi via Digitalmars-d 
wrote:
 On Tuesday, 6 March 2018 at 20:03:11 UTC, Jonathan M Davis wrote:
 On Tuesday, March 06, 2018 18:49:42 Paolo Invernizzi via

 Digitalmars-d wrote:
 I simply don't understand why enforce or a custom check can't
 be used  safe code, if you want that behaviour.

 If the program must HALT on this or that, well, what is better
 than an explicit piece of unremovable code that do that?
 Instead of writing 'assert', one should write 'enforce'.
1. It's quite common for folks to want to add debugging checks that are compiled out in -release. That's exactly what assert is for in pretty much every lanugage that has it. It's what folks expect to use and what your average programmer will use without thinking about safety issues at all. It's what everyone uses right now, and I'm pretty sure that almost everyone using it has no clue that Walter considers it okay for assertions to introduce optimizations which are not memory safe, and if/when he does do so, a lot of D code will suddenly have safe code which is not memory safe. Such problems will hopefully be hit rarely, because hopefully, the code will have been well-tested, and the assertions will have found all of the related bugs, but there's every possibility that some bugs will manage to not be caught, thereby resulting in safe code being unsafe. No one is going to be looking to use solutions other than assertions for what assertions are for unless we start telling everyone to avoid assertions, because they make safe code unsafe. And honestly, if assertions make safe code unsafe, I don't see a good argument for using them at all. If I didn't care about code being safe, I wouldn't be using safe. safe is supposed to guarantee that the code is memory safe.
Understood. Are asking that UB should not include memory corruptions?
I'm saying that whatever optimizations are enabled by assertions should be guaranteed to not include memory corruptions.
 2. I think that it's fundamentally a terrible idea to allow
 built-in language features to violate  safe. Aside from issues
 related to  trusted being misused,  safe code should be
 guaranteed to be memory safe, and it should be considered a bug
 any time that it isn't. That's why  safe exists. No one should
 have to be looking at  safe code to track down memory safety
 problems. And if they do, then  safe is not doing it's job.
 Array bounds checks are left in  safe code for exactly these
 reasons.
So, the request is to just leave assert active as a default in safe code, like the bounds checks?
No. I'm saying that no optimizations should be enabled which introduce potential memory corruption. Assertions should have zero impact on whether code is safe or not unless the code in the condition or which is generating the message for the assertion is system, and it's no more reasonable to assume that an assertion is going to pass than it is to assume that bounds checking won't fail. Regardless, the key thing here is that safe code should be guaranteed to be safe so long as trusted code is vetted properly. It should _never_ be possible for the compiler to introduce memory safety issues into safe code.
 I'm all for introducing optimizations that do not violate
  safe, but if we allow  safe code to be unsafe, then why do we
 even have it?
So, the reasoning is that UB should not lead to memory corruption, right?
The reasoning is that no safe code should ever have memory corruptions in it unless it calls trusted code that was incorrectly vetted by the programmer. The compiler is supposed to guarantee that safe code is safe just like it's supposed to guarantee that a const variable isn't mutated or that a pure function doesn't access mutable, global variables. And as such, introducing anything into safe code which could be memory unsafe is a violation of the compiler's responsibility. - Jonathan M Davis
Mar 07
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 7 March 2018 at 09:11:10 UTC, Jonathan M Davis 
wrote:

 So, the request is to just leave assert active as a default in 
  safe code, like the bounds checks?
 No. I'm saying that no optimizations should be enabled which 
 introduce potential memory corruption. Assertions should have 
 zero impact on whether code is  safe or not unless the code in 
 the condition or which is generating the message for the 
 assertion is  system, and it's no more reasonable to assume 
 that an assertion is going to pass than it is to assume that 
 bounds checking won't fail. Regardless, the key thing here is 
 that  safe code should be guaranteed to be  safe so long as 
  trusted code is vetted properly. It should _never_ be possible 
 for the compiler to introduce memory safety issues into  safe 
 code.
 So, the reasoning is that UB should not lead to memory 
 corruption, right?
 The reasoning is that no  safe code should ever have memory 
 corruptions in it unless it calls  trusted code that was 
 incorrectly vetted by the programmer. The compiler is supposed 
 to guarantee that  safe code is  safe just like it's supposed 
 to guarantee that a const variable isn't mutated or that a pure 
 function doesn't access mutable, global variables. And as such, 
 introducing anything into  safe code which could be memory 
 unsafe is a violation of the compiler's responsibility.
Jonathan, I understand your reasoning, but it's not what I'm asking: are we asking for UB that do not lead to memory corruption? /Paolo
Mar 07
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, March 07, 2018 09:22:40 Paolo Invernizzi via Digitalmars-d 
wrote:
 On Wednesday, 7 March 2018 at 09:11:10 UTC, Jonathan M Davis

 wrote:
 So, the request is to just leave assert active as a default in
  safe code, like the bounds checks?
No. I'm saying that no optimizations should be enabled which introduce potential memory corruption. Assertions should have zero impact on whether code is safe or not unless the code in the condition or which is generating the message for the assertion is system, and it's no more reasonable to assume that an assertion is going to pass than it is to assume that bounds checking won't fail. Regardless, the key thing here is that safe code should be guaranteed to be safe so long as trusted code is vetted properly. It should _never_ be possible for the compiler to introduce memory safety issues into safe code.
 So, the reasoning is that UB should not lead to memory
 corruption, right?
The reasoning is that no safe code should ever have memory corruptions in it unless it calls trusted code that was incorrectly vetted by the programmer. The compiler is supposed to guarantee that safe code is safe just like it's supposed to guarantee that a const variable isn't mutated or that a pure function doesn't access mutable, global variables. And as such, introducing anything into safe code which could be memory unsafe is a violation of the compiler's responsibility.
Jonathan, I understand your reasoning, but it's not what I'm asking: are we asking for UB that do not lead to memory corruption?
I'm saying that safe code must not be violated by the compiler. Beyond that I'm not arguing about UB one way or the other. If UB must be disallowed to avoid violating safe, then it must be disallowed. If some form of UB can be allowed, because it's restricted in a manner that it can't violate safe but may do something else stupid because the assertion would have failed if it weren't compiled out, I don't care. If an assertion would have failed if it weren't compiled out, then you have a bug regardless, and if the code is buggier because of an optimization, then that's fine with me. You have a bug either way. What isn't fine is that that result violate safe, because that would defeat the entire purpose of safe and make it far, far more difficult to track down safety problems. Right now, since no optimizations like Walter has been talking about are done by the compiler, if you have memory corruption, you know that you only have to look at system and trusted code to find it, whereas with the unsafe optimizations that Walter is talking about, it then becomes possible that you're going to have to look through the entire program to find the problem. And right now, you can be sure that you don't have safety problems in safe code if you use trusted correctly, whereas with what Walter is talking about, simply adding an assertion could add safety problems to your code. - Jonathan M Davis
Mar 07
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 7 March 2018 at 11:52:05 UTC, Jonathan M Davis 
wrote:
 On Wednesday, March 07, 2018 09:22:40 Paolo Invernizzi via 
 Digitalmars-d wrote:
 On Wednesday, 7 March 2018 at 09:11:10 UTC, Jonathan M Davis

 wrote:
 So, the request is to just leave assert active as a default 
 in  safe code, like the bounds checks?
No. I'm saying that no optimizations should be enabled which introduce potential memory corruption. Assertions should have zero impact on whether code is safe or not unless the code in the condition or which is generating the message for the assertion is system, and it's no more reasonable to assume that an assertion is going to pass than it is to assume that bounds checking won't fail. Regardless, the key thing here is that safe code should be guaranteed to be safe so long as trusted code is vetted properly. It should _never_ be possible for the compiler to introduce memory safety issues into safe code.
 So, the reasoning is that UB should not lead to memory 
 corruption, right?
The reasoning is that no safe code should ever have memory corruptions in it unless it calls trusted code that was incorrectly vetted by the programmer. The compiler is supposed to guarantee that safe code is safe just like it's supposed to guarantee that a const variable isn't mutated or that a pure function doesn't access mutable, global variables. And as such, introducing anything into safe code which could be memory unsafe is a violation of the compiler's responsibility.
Jonathan, I understand your reasoning, but it's not what I'm asking: are we asking for UB that do not lead to memory corruption?
I'm saying that safe code must not be violated by the compiler. Beyond that I'm not arguing about UB one way or the other.
And that's clear.
 If UB must be disallowed to avoid violating  safe, then it must 
 be disallowed.
And how to do this, in practise I mean?
 If some form of UB can be allowed, because it's restricted in a 
 manner that it can't violate  safe but may do something else 
 stupid because the assertion would have failed if it weren't 
 compiled out, I don't care.
And that's the original question: are we asking for UB that do not lead to memory corruption?
 If an assertion would have failed if it weren't compiled out, 
 then you have a bug regardless, and if the code is buggier 
 because of an optimization, then that's fine with me. You have 
 a bug either way.
Agreed.
 What isn't fine is that that result violate  safe, because that 
 would defeat the entire purpose of  safe and make it far, far 
 more difficult to track down  safety problems.
So, see above, the original question, agreed.
 Right now, since no optimizations like Walter has been talking 
 about are done by the compiler, if you have memory corruption, 
 you know that you only have to look at  system and  trusted 
 code to find it, whereas with the unsafe optimizations that 
 Walter is talking about, it then becomes possible that you're 
 going to have to look through the entire program to find the 
 problem.
Or you can just turn on assertion, right? If we have corrupted memory in release, there's a bug, somewhere, in the logic or in the implementation of the logic. As you have told, we must audit system and trusted, we can imagine to use static checkers or some strange beast like that. But, while doing that, I think that the most common practise is keep running the code with assertion on, do you agree?
 And right now, you can be sure that you don't have  safety 
 problems in  safe code if you use  trusted correctly, whereas 
 with what Walter is talking about, simply adding an assertion 
 could add  safety problems to your code.
Nope, not adding an assertion, but having the process in UB state. And we are back again to the original question. /Paolo
Mar 07
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, March 07, 2018 13:24:19 Paolo Invernizzi via Digitalmars-d 
wrote:
 Right now, since no optimizations like Walter has been talking
 about are done by the compiler, if you have memory corruption,
 you know that you only have to look at  system and  trusted
 code to find it, whereas with the unsafe optimizations that
 Walter is talking about, it then becomes possible that you're
 going to have to look through the entire program to find the
 problem.
Or you can just turn on assertion, right? If we have corrupted memory in release, there's a bug, somewhere, in the logic or in the implementation of the logic. As you have told, we must audit system and trusted, we can imagine to use static checkers or some strange beast like that. But, while doing that, I think that the most common practise is keep running the code with assertion on, do you agree?
That would make assertions a lot worse to use, because then they would be in production code slowing it down. Also, as it stands, -release is not supposed to violate safe. To do that, you have to use -boundscheck=off to turn off bounsd checking. That was a very purposeful design decision, because we did not want -release to violate safe, and if the compiler is allowed to add optimizations which are unsafe based on assertions, then that completely destroys the ability to have safe code with -release. And if we were going to do that, why did we leave array bounds checking on with -release? Assertions are to help debug code, and most people disable them in production with zero expectation that that's going to result in their safe code suddenly becoming unsafe. It's a huge change if -release makes code unsafe, and IMHO, doing so would make assertions an immediate code smell and that assertions should then never be used except by folks who are willing to leave them in all the time. I don't see how it can be argued that allowing assertions to introduce unsafe behavior into safe code is not in complete violation of what safe is supposed to do and guarantee. It's already bad enough that we talk about how memory safe D code is when safe isn't the default, but to then completely bypass safe like this seems unconscionable to me. That would be like deciding that we're now going to introduce cast_cast and mutable into the language and allow const's guarantees to be violated whenever the programmer feels like it. safe needs to actually be guaranteed to be safe or it's worthless. - Jonathan M Davis
Mar 07
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis 
wrote:
 On Wednesday, March 07, 2018 13:24:19 Paolo Invernizzi via 
 Digitalmars-d wrote:
 [...]
That would make assertions a lot worse to use, because then they would be in production code slowing it down. Also, as it stands, -release is not supposed to violate safe. To do that, you have to use -boundscheck=off to turn off bounsd checking. That was a very purposeful design decision, because we did not want -release to violate safe, and if the compiler is allowed to add optimizations which are unsafe based on assertions, then that completely destroys the ability to have safe code with -release. And if we were going to do that, why did we leave array bounds checking on with -release? [...]
Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /Paolo
Mar 07
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, March 07, 2018 14:08:35 Paolo Invernizzi via Digitalmars-d 
wrote:
 On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis

 wrote:
 On Wednesday, March 07, 2018 13:24:19 Paolo Invernizzi via

 Digitalmars-d wrote:
 [...]
That would make assertions a lot worse to use, because then they would be in production code slowing it down. Also, as it stands, -release is not supposed to violate safe. To do that, you have to use -boundscheck=off to turn off bounsd checking. That was a very purposeful design decision, because we did not want -release to violate safe, and if the compiler is allowed to add optimizations which are unsafe based on assertions, then that completely destroys the ability to have safe code with -release. And if we were going to do that, why did we leave array bounds checking on with -release? [...]
Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions?
safe is all about guaranteeing memory safety. That's it's entire job. No more, no less. What happens with UB beyond that is irrelevant. If satisfying the requirement that safe code be memory safe means that UB cannot be allowed in safe code, then UB cannot be allowed in safe code. If there is some form of UB that is constrained enough that it's guaranteed that it can't violate memory safety, then I don't see any reason why it can't be in safe code any more than it can't be in system code, because it's not violating the guarantees that safe is intended to provide - that the code is memory safe. Other language rules may make UB illegal or explicitly allow it for one reason or another (e.g. it's supposed to be guaranteed that function arguments are evaluated left-to-right, though I'm not sure if that's ever been implemented like it's supposed to be), but in the case of safe, it's all about what's memory safe. And what is or isn't allowed with regards to UB in safe therefore has to be a function of what is required to guarantee that the code is memory safe. - Jonathan M Davis
Mar 07
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07.03.2018 15:08, Paolo Invernizzi wrote:
 On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis wrote:
 On Wednesday, March 07, 2018 13:24:19 Paolo Invernizzi via 
 Digitalmars-d wrote:
 [...]
That would make assertions a lot worse to use, because then they would be in production code slowing it down. Also, as it stands, -release is not supposed to violate safe. To do that, you have to use -boundscheck=off to turn off bounsd checking. That was a very purposeful design decision, because we did not want -release to violate safe, and if the compiler is allowed to add optimizations which are unsafe based on assertions, then that completely destroys the ability to have safe code with -release. And if we were going to do that, why did we leave array bounds checking on with -release? [...]
Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /Paolo
UB is unconstrained by definition. If it is constrained, it is not UB.
Mar 07
parent reply Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 7 March 2018 at 15:26:01 UTC, Timon Gehr wrote:
 On 07.03.2018 15:08, Paolo Invernizzi wrote:
 On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis 
 wrote:
 [...]
Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /Paolo
UB is unconstrained by definition. If it is constrained, it is not UB.
That! Thanks! So, safe code is code where UB should not be possible? Is it pragmatically possible to reach that goal? /Paolo
Mar 07
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07.03.2018 16:30, Paolo Invernizzi wrote:
 On Wednesday, 7 March 2018 at 15:26:01 UTC, Timon Gehr wrote:
 On 07.03.2018 15:08, Paolo Invernizzi wrote:
 On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis wrote:
 [...]
Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /Paolo
UB is unconstrained by definition. If it is constrained, it is not UB.
That! Thanks! So, safe code is code where UB should not be possible?
Yes. (If you have UB, memory corruption is one of the allowed outcomes, but safe should not allow memory corruption. So safe needs to at least ban UB. According to Walter safe needs to ban memory corruption but not more. Therefore, as memory corruption leads to UB (it is impractical to specify anything else, at least for writes), safe bans UB and nothing else.) Also note that even in system code I don't want asserts to cause UB in release. There should be an system facility for this purpose. (The situation is different than with bounds checks: if bounds checks fail, there will always be a bad memory access, which is UB, but with asserts it's always possible that the assert itself was wrong and the code itself will not trigger UB.)
 Is it pragmatically possible to reach that goal?
 
 /Paolo
Yes. (Languages can be type safe and type systems can be arbitrarily powerful.) It is certainly possible to _aim_ for that goal, which Walter and Andrei have done on other occasions.
Mar 07
parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 7 March 2018 at 16:04:50 UTC, Timon Gehr wrote:
 On 07.03.2018 16:30, Paolo Invernizzi wrote:
 On Wednesday, 7 March 2018 at 15:26:01 UTC, Timon Gehr wrote:
 On 07.03.2018 15:08, Paolo Invernizzi wrote:
 On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis 
 wrote:
 [...]
Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /Paolo
UB is unconstrained by definition. If it is constrained, it is not UB.
That! Thanks! So, safe code is code where UB should not be possible?
Yes. (If you have UB, memory corruption is one of the allowed outcomes, but safe should not allow memory corruption. So safe needs to at least ban UB. According to Walter safe needs to ban memory corruption but not more. Therefore, as memory corruption leads to UB (it is impractical to specify anything else, at least for writes), safe bans UB and nothing else.) Also note that even in system code I don't want asserts to cause UB in release. There should be an system facility for this purpose. (The situation is different than with bounds checks: if bounds checks fail, there will always be a bad memory access, which is UB, but with asserts it's always possible that the assert itself was wrong and the code itself will not trigger UB.)
 Is it pragmatically possible to reach that goal?
 
 /Paolo
Yes. (Languages can be type safe and type systems can be arbitrarily powerful.) It is certainly possible to _aim_ for that goal, which Walter and Andrei have done on other occasions.
Thanks Timon, now everything it's definitely more clear to me on that matter. I think a good compromise is to just totally ignore assert in release during optimization as a default, and use a compiler flag to let the optimiser peek to them, if a user want to explore that potential gain. Just like '-boundscheck'. Thanks to all for having clarified that point to me (and hopefully to others also!) /Paolo
Mar 07
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 06.03.2018 19:49, Paolo Invernizzi wrote:
 
 I simply don't understand why enforce or a custom check can't be used 
  safe code, if you want that behaviour.
 ...
I have explained why. UB is non-modular and you don't (want to) control all the code that you use. Also, enforce cannot be disabled. And no, keeping the check or introducing UB are not the only sensible options.
Mar 06
parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Wednesday, 7 March 2018 at 00:11:33 UTC, Timon Gehr wrote:
 On 06.03.2018 19:49, Paolo Invernizzi wrote:
 
 I simply don't understand why enforce or a custom check can't 
 be used  safe code, if you want that behaviour.
 ...
I have explained why. UB is non-modular and you don't (want to) control all the code that you use.
As I've written, you should ask for debug versions of every closed source library that it's used in a project, it's always a win-only things to do.
 Also, enforce cannot be disabled.
Because if you use it instead of assert, the enforce has just detected a bug, so it's just doing the sane thing to join that cases: stop the execution because the process is entering UB.
 And no, keeping the check or introducing UB are not the only 
 sensible options.
Just to understand, as I've already asked: do we aim at UB that guarantees no memory corruptions? /Paolo
Mar 07
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, March 05, 2018 02:30:12 Walter Bright via Digitalmars-d wrote:
 The idea behind removal of the runtime checks is as a performance
 optimization done on a debugged program. It's like turning on or off
 array bounds checking. Many leave asserts and array bounds checking on
 even in released code to ensure memory safety.

 At a minimum, turning it off and on will illuminate just what the checks
 are costing you.

 It's at the option of the programmer.
Except that disabling array bounds checking negates the guarantees of safe, and you have to go to the extra effort of supplying a compiler flag to tell the compiler that you don't care about the compiler guaranteeing safe. And your suggesting that with assertions, safe's guarantees can be removed. You never catch everything when debugging or running your program prior to releasing it. Assertions can help considerably in catching problems, but there is zero guarantee that that they've caught all of the problems. They're disabled in release mode on the theory that the program has been tested enough that that assertions are unlikely to fail, but they could still have failed if they remained. It's one thing to remove the assertions and let your program behave badly as a result whenever you didn't catch all of the problems. It's quite another to have the compiler make your code safe code system on the assumption that the assertion would have passed. Personally, I don't mind if the compiler is able to make optimization decisions based on assertions - the code is buggy anyway if the assertion would have failed - but I very much mind if those optimizations introduce behavior that could violate safe. Because then, instead of just having a bug, you have memory corruption problem, which could be really hard to detect and track down, and it would be essentially invisible, because most programmers who understand safe and trusted, would assume that they needed to check the system code and trusted code for the problem, when it could now be sitting in safe code just because one of the developers added an assertion to help catch a bug during testing. If assertions make it possible for safe code to actually be system, that seems like a serious bug to me. IIRC, Andrei made a huge deal with you several years ago about how array bounds checking needed to be kept in safe code or safe meant nothing. I don't see how this situation is any different. safe code needs to guaranteed to be safe so long as trusted is used correctly, otherwise, safe is arguably meaningless - certainly, it then doesn't restrict the code that you have to look through for memory safety problems, and that's one of the main purposes of safe. If you want to introduce a compiler flag like -boundscheck=off that makes it so that compiling out assertions can introduce system behavior, then fine. But removing assertions with -release and removing safety guarantees are two completely separate issues. I sure don't want assertions in my production builds; I'd be forced to use them a lot less if they were going to be in my production builds. But I also don't want to introduce safety issues into my production builds by using assertions. You keep talking about how important safety is to D and how much better it makes it than C, and then you go and talk about invisibly removing safety guarantees with common, built-in features like assertions. If assertions removed safety guaranatees, I would be forced to never use them in my code, which IMHO is ridiculous and is certainly going to make my code worse. And IMHO, suggesting that someone should then use a user-defined assertion isn't a good solution either, because then the built-in default that everyone is going to use unless someone more knowledgable tells them not to is an safety problem. If you want to be able to optimize based on assertions then fine, but please, please, please do not allow the compiler to introduce safety problems when adding optimizations - preferably not even in system code, because realistically, almost no one is going to have a clue what safety problems could be introduced by an optimization based on an assertion, making it impossible for the programmer to verify that system code can be marked trusted. And most programmers won't know that assertions could even introduce optimizations that weren't memory safe. I honestly don't see how we can claim that safe code is guaranteed to be memory safe if assertions - or any other D feature other than the programmer using trusted incorrectly - could introduce anything that is not guaranteed to be memory safe. - Jonathan M Davis
Mar 06
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/6/2018 1:58 AM, Jonathan M Davis wrote:
 [...]
The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code. It's been like that for D since day 1. It has always been documented to do that. It has been discussed many times in this n.g. over the years with loooong threads. I designed it that way so that D could potentially produce better code than other languages in ways they could not match. There is no other purpose to making it a core language feature. It's fine if you want an assert-like feature to have other semantics - just define one called 'check', give it the semantics you want, and put it in the library. As I mentioned to Timon, support for that sort of thing is why D has special support for __LINE__ and __FILE__. Or just don't turn off assert checking. Personally, I use asserts in a way that they add little overhead so they can remain active in the release build. It's entirely under your control.
Mar 06
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, March 06, 2018 02:17:42 Walter Bright via Digitalmars-d wrote:
 On 3/6/2018 1:58 AM, Jonathan M Davis wrote:
 [...]
The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code. It's been like that for D since day 1. It has always been documented to do that. It has been discussed many times in this n.g. over the years with loooong threads. I designed it that way so that D could potentially produce better code than other languages in ways they could not match. There is no other purpose to making it a core language feature. It's fine if you want an assert-like feature to have other semantics - just define one called 'check', give it the semantics you want, and put it in the library. As I mentioned to Timon, support for that sort of thing is why D has special support for __LINE__ and __FILE__. Or just don't turn off assert checking. Personally, I use asserts in a way that they add little overhead so they can remain active in the release build. It's entirely under your control.
I don't mind if assertions allow the compiler to better optimize. What I object to quite strongly is about the compiler being allowed to introduce optimizations that violate safety guarantees. Built-in features should not result in safe code invisibly becoming system. That violates the entire purpose of safe. I don't see how this situation is any different from the array bounds checks, and those aren't compiled out in safe code - even when assertions are compiled out. - Jonathan M Davis
Mar 06
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 06.03.2018 11:17, Walter Bright wrote:
 On 3/6/2018 1:58 AM, Jonathan M Davis wrote:
 [...]
The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code.
It does not need to be a core language feature for that. It seems that you did the right thing for the wrong reasons.
 It's 
 been like that for D since day 1. It has always been documented to do 
 that. It has been discussed many times in this n.g. over the years with 
 loooong threads. I designed it that way so that D could potentially 
 produce better code than other languages in ways they could not match.
 
 There is no other purpose to making it a core language feature.
 ...
Yes. The (only!) purpose is standardization. asserts and contracts are a core language feature for similar reasons why unit tests are a core language feature.
 ...
 
 Or just don't turn off assert checking. Personally, I use asserts in a 
 way that they add little overhead so they can remain active in the 
 release build.
 ...
Not everybody runs only their own code.
 It's entirely under your control.
It is not. Could we please make it that way?
Mar 06
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 05.03.2018 11:25, Walter Bright wrote:
 On 3/4/2018 3:06 PM, Timon Gehr wrote:
 On 04.03.2018 22:49, Walter Bright wrote:
 Not necessarily. If the code contains an explicit assertion that the 
 index is in bounds, then, according to the language specification, 
 the bounds check may be removed with -release.
D, as all languages that I know of do implicitly or explicitly, generates code based on the "as if" rule. ...
Impossible. You wrote a Java compiler.
Even in Java, the compiler generates code that, from the user's point of view, behaves "as if" the code was actually what was specified. For a trivial example, replacing x*2 with x<<1. Not having this means no optimizations can be done. ...
I guess I misunderstood what you meant when you said "as if". I thought you meant that for all languages you know, when assertions are disabled, the compiler behaves "as if" the check was actually there and was known to succeed, even though the check is actually not there and may have failed if it was.
 
 All languages that use your "as if" rule are memory unsafe.
 Zero languages that use the "as if" rule have any memory safe subset 
 that includes assertions.
 In D, assert is  safe, and it should remain  safe.
 I find the reasoning in terms of "on"/"off" confusing anyway.
 Does "off" mean "contract/assertion removed", or does it mean 
 "failure is UB"?
"Off" means the check is removed. If the check does not hold, the program enters an invalid state, whether or not the check was actually done. An invalid state means subsequent execution is UB.
Why is potential memory corruption to be expected when using safe language features with a flag to disable contract checks?
Because the checks provide extra information to the compiler that it can use to generate better code. If that extra information is not true, then the better code will be invalid. ...
My question is not why it is the case technically, I was asking for a _rationale_ for this apparently silly behavior. I.e., why is this a good idea from the point of view of language design? Again: assert is safe. Compiler hints are system. Why should assert give compiler hints?
 Memory safety is only one class of errors in a program. If the program 
 has entered a state that is not accounted for by the programmer, the 
 rest of the program's execution will be not predictable.
 ...
But the whole point of having memory safety is to not have UB when the programmer screwed up. Behavior not foreseen by the programmer (a bug) is not the same as behavior unconstrained by the language specification (UB).
 
 This makes no sense. This is not useful behavior. There are convenient 
 ways to support potentially unsound compilation hints that do not do 
 this. Contracts and compilation hints should be orthogonal. Contracts 
 should be potentially  safe, compilation hints should be  system always.

 Note that _actual removal_ is the only use case of 'disabling 
 contracts' that I care about, and I think many D programmers who use 
 "off" will also have this behavior in mind. Yet this is not even an 
 option.
I don't see much use for this behavior, unless you want to continue running the program after an assert failure, which I cannot recommend and the language is not designed to support.
'in'-contracts catch AssertError when being composed. How can the language not be designed to support that? Except for this case, the assertion is not _supposed_ to fail for my use cases, and I don't really need the language to explicitly "support that use case". The situation is the following: - I usually don't want UB in programs I am working on. I want the runtime behavior of the programs to be determined by the source code, such that every behavior observed in the wild (intended or unintended) can be traced back to the source code (potentially in a non-deterministic way, e.g. void initialization of an integer constant). This should be the case always, even if me or someone else on my team made a mistake. The safe D subset is supposed to give this guarantee. What good is safe if it does not guarantee absence of buffer overrun attacks? - Checking assertions can be too costly, so it should be possible to disable the check. - Using existing assertions as compiler hints is not necessary. (Without having checked it, I'm sure that LDC/GDC have a more suitable intrinsic for this already.) As far as I can discern, forcing disabled asserts to give compiler hints has no upsides.
 But you can always do 
 something like:
 
     version (ignore_asserts) { } else { assert(...); }
 ...
I know. Actually version(assert) assert(...); also works. However, this is too verbose, especially in contracts. I'd like a solution that does not require me to change the source code. Ideally, I just want the Java behavior (with reversed defaults).
 which would optionally remove both the runtime check and any compiler 
 use of the assert. Or you could use 
 https://dlang.org/library/std/exception/enforce.html which has no 
 influence on compiler semantics.
 ...
(enforce is _completely unrelated_ to the current discussion.) Contracts are designed to be used with built-in assert. Built-in assert is convenient. If the point of 'assert' is to give compiler hints (for me this is not even a use case of assert at all), then it should be system.
 
 At the very least, the DIP should be up-front about this.
 I'm still not even sure that Mathias Lang intended the UB semantics.
It being UB was my doing, not Mathias'. DIP1006 is not redefining the semantics of what assert does.
This is not really about assert semantics, this is about the semantics of "disabling the check". There was no "-check=off" flag before. The DIP uses terminology such as "disable assertions" as opposed to "disable assertion checks (but introduce compiler hints)". Contracts are supposed to increase safety, not to reduce it (ask Bertrand Meyer).
Mar 05
next sibling parent reply Iain Buclaw <ibuclaw gdcproject.org> writes:
On Monday, 5 March 2018 at 15:48:12 UTC, Timon Gehr wrote:
 - Using existing assertions as compiler hints is not necessary. 
 (Without having checked it, I'm sure that LDC/GDC have a more 
 suitable intrinsic for this already.)

 As far as I can discern, forcing disabled asserts to give 
 compiler hints has no upsides.
In the simple cases, or in anything that looks like a unittest/testsuite, probably not. There are likely going to be more aggressive optimizations however if CFA can see that a variable will never be outside a given range, i.e: --- int[5] arr; if (len < 0 || len >= 5) { unreachable(); // in non-release code, this would throw a RangeError. } return arr[len]; --- From this, we aggressively assume that len is a valid index of arr. Something that happens in optimized non-release builds, but in release builds we must accommodate for the possibility of a range error.
Mar 05
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05.03.2018 20:41, Iain Buclaw wrote:
 On Monday, 5 March 2018 at 15:48:12 UTC, Timon Gehr wrote:
 - Using existing assertions as compiler hints is not necessary. 
 (Without having checked it, I'm sure that LDC/GDC have a more suitable 
 intrinsic for this already.)

 As far as I can discern, forcing disabled asserts to give compiler 
 hints has no upsides.
In the simple cases, or in anything that looks like a unittest/testsuite, probably not. There are likely going to be more aggressive optimizations however if CFA can see that a variable will never be outside a given range, i.e: ...
(Note that by "forcing", I mean withholding other options from the user. I'm not saying that using information from asserts can never be useful, just that it can just as well be harmful, and therefore it is unwise to not allow disabling them. I was saying that there are no upsides to not having a flag that actually removes assertions.)
 ---
 int[5] arr;
 
 if (len < 0 || len >= 5)
 {
      unreachable();  // in non-release code, this would throw a RangeError.
 }
 
 return arr[len];
 ---
 
  From this, we aggressively assume that len is a valid index of arr.  
 Something that happens in optimized non-release builds, but in release 
 builds we must accommodate for the possibility of a range error.
I think this particular case is a bit less questionable than doing the same for general assertions (for instance, in safe code, -release will not actually remove the bounds check unless there is some relevant assertion somewhere). In any case, I don't argue strongly against a flag that turns all assertions into compiler hints, I just think there should also be a flag that disables them safely. Also, maybe -release should commit to either disregarding safe completely or respecting it completely.
Mar 05
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 3/5/2018 7:48 AM, Timon Gehr wrote:
 Again: assert is  safe. Compiler hints are  system. Why should assert give 
 compiler hints?
Asserts give expressions that must be true. Why not take advantage of them? See Spec# which based an entire language around that notion: https://en.wikipedia.org/wiki/Spec_Sharp Some possible optimizations based on this are: 1. elimination of array bounds checks 2. elimination of null pointer checks 3. by knowing a variable can take on a limited range of values, a cheaper data type can be substituted 4. elimination of checks for 'default' switch values 5. elimination of overflow checks dmd's optimizer currently does not extract any information from assert's. But why shut the door on that possibility?
 But the whole point of having memory safety is to not have UB when the 
 programmer screwed up. Behavior not foreseen by the programmer (a bug) is not 
 the same as behavior unconstrained by the language specification (UB).
It's the programmer's option to leave those runtime checks in if he wants to.
 'in'-contracts catch AssertError when being composed. How can the language not 
 be designed to support that?
That is indeed an issue. It's been proposed that in-contracts throw a different exception, say "ContractException" so that it is not UB when they fail. There's a bugzilla ER on this. (It's analogous to asserts in unittests not having UB after they fail.)
 - I usually don't want UB in programs I am working on. I want the runtime 
 behavior of the programs to be determined by the source code, such that every 
 behavior observed in the wild (intended or unintended) can be traced back to
the 
 source code (potentially in a non-deterministic way, e.g. void initialization
of 
 an integer constant). This should be the case always, even if me or someone
else 
 on my team made a mistake. The  safe D subset is supposed to give this 
 guarantee. What good is  safe if it does not guarantee absence of buffer
overrun 
 attacks?
It guarantees it at the option of the programmer via a command line switch.
 - Using existing assertions as compiler hints is not necessary. (Without
having 
 checked it, I'm sure that LDC/GDC have a more suitable intrinsic for this
already.)
 
 As far as I can discern, forcing disabled asserts to give compiler hints has
no 
 upsides.
I suspect that if: compiler_hint(i < 10); were added, there would be nothing but confusion as to its correct usage vs assert vs enforce. There's already enough confusion about the latter two. In fact, I can pretty much guarantee it will be rarely used correctly.
 I know. Actually version(assert) assert(...); also works. However, this is too 
 verbose, especially in contracts.
You can wrap it in a template.
 I'd like a solution that does not require me 
 to change the source code. Ideally, I just want the Java behavior (with
reversed 
 defaults).
But you'll have to change the code to compiler_hint().
 (enforce is _completely unrelated_ to the current discussion.)
It does just what you ask (for the regular assert case).
 It being UB was my doing, not Mathias'. DIP1006 is not redefining the 
 semantics of what assert does.
This is not really about assert semantics, this is about the semantics of "disabling the check".
It is very much about the semantics of assert.
 There was no "-check=off" flag before.
Yes there was, it's the "release" flag.
 The DIP uses terminology such as "disable assertions" as opposed to "disable 
 assertion checks (but introduce compiler hints)".
Yes, the language could be more precise, but I couldn't blame Mathias for that. I also disagree with the word "hint", because it implies things like "this branch is more likely to be taken" to guide code generation decisions, rather than "assume X is absolutely always incontrovertibly true and you can bet the code on it".
Mar 05
next sibling parent ag0aep6g <anonymous example.com> writes:
On 03/05/2018 09:55 PM, Walter Bright wrote:
 On 3/5/2018 7:48 AM, Timon Gehr wrote:
 Again: assert is  safe. Compiler hints are  system. Why should assert 
 give compiler hints?
Asserts give expressions that must be true. Why not take advantage of them?
Because it's exactly what safe is not supposed to do. You're trusting the programmer to get their asserts right. Trusting the programmer to get it right is system. [...]> It's the programmer's option to leave those runtime checks in if he
 wants to.
As far as I understand, Timon only asks for a third option: to simply compile the code as if the asserts weren't there, without assuming that they would pass. That way you get a speedup from the omitted asserts, but you don't get UB from a mistaken assert. This is not an unreasonable thing to want, is it? You say that DMD does not currently use assert information, so -release currently does this. [...]
 There was no "-check=off" flag before.
Yes there was, it's the "release" flag.
But the controversial aspect is not implemented. And it will be very surprising if you ever do implement it. I'm actually pretty shocked that -release is described that way. It makes a point of keeping bounds checks in safe code. The reason is that it would be unsafe to remove them. What's the point of that when safety is compromised anyway by assuming that asserts would pass?
Mar 05
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 05.03.2018 21:55, Walter Bright wrote:
 On 3/5/2018 7:48 AM, Timon Gehr wrote:
 Again: assert is  safe. Compiler hints are  system. Why should assert 
 give compiler hints?
Asserts give expressions that must be true.
"Trust the programmer" does not always scale.
 Why not take advantage of them?
For some use cases it might be fine, but not for others, because you can't know whether the program and the assertions are really consistent. Basically, I think the flags should be: -check-{assert,invariant,precondition,postcondition,...}={on,off,assume} E.g.: $ dmd -check-assert=on test.d # throw on assertion failure $ dmd -check-assert=off test.d # ignore assertions $ dmd -check-assert=assume test.d # assertions are assumptions for code generation Then the spec says that "assume" is potentially dangerous and can break safe-ty guarantees, like -boundscheck=off.
 See Spec# which based an entire language around that notion:
 
   https://en.wikipedia.org/wiki/Spec_Sharp
 ...
Spec# is the opposite of what you claim. It verifies statically that the conditions actually hold. Also, it is type safe. (I.e. no UB.)
 Some possible optimizations based on this are:
 
 1. elimination of array bounds checks
 2. elimination of null pointer checks
 3. by knowing a variable can take on a limited range of values, a 
 cheaper data type can be substituted
 4. elimination of checks for 'default' switch values
 5. elimination of overflow checks
 
 dmd's optimizer currently does not extract any information from 
 assert's. But why shut the door on that possibility?
 ...
We should not do that, and it is not what I am arguing for. Sorry if that did not come across clearly.
 
 But the whole point of having memory safety is to not have UB when the 
 programmer screwed up. Behavior not foreseen by the programmer (a bug) 
 is not the same as behavior unconstrained by the language 
 specification (UB).
It's the programmer's option to leave those runtime checks in if he wants to. ...
My point is that either leaving them in or turning failures into UB are too few options. Also, safe is a bit of a joke if there is no way to _disable contracts_ without nullifying the guarantees it's supposed to give.
 
 'in'-contracts catch AssertError when being composed. How can the 
 language not be designed to support that?
That is indeed an issue. It's been proposed that in-contracts throw a different exception, say "ContractException" so that it is not UB when they fail. There's a bugzilla ER on this. (It's analogous to asserts in unittests not having UB after they fail.) ...
This is ugly, but I don't think there is a better solution.
 
 - I usually don't want UB in programs I am working on. I want the 
 runtime behavior of the programs to be determined by the source code, 
 such that every behavior observed in the wild (intended or unintended) 
 can be traced back to the source code (potentially in a 
 non-deterministic way, e.g. void initialization of an integer 
 constant). This should be the case always, even if me or someone else 
 on my team made a mistake. The  safe D subset is supposed to give this 
 guarantee. What good is  safe if it does not guarantee absence of 
 buffer overrun attacks?
It guarantees it at the option of the programmer via a command line switch. ...
You mean, leave in checks?
 
 - Using existing assertions as compiler hints is not necessary. 
 (Without having checked it, I'm sure that LDC/GDC have a more suitable 
 intrinsic for this already.)

 As far as I can discern, forcing disabled asserts to give compiler 
 hints has no upsides.
I suspect that if:     compiler_hint(i < 10); were added, there would be nothing but confusion as to its correct usage vs assert vs enforce. There's already enough confusion about the latter two.
I have never understood why. The use cases of assert and enforce are disjoint.
 In fact, I can pretty much guarantee it will be rarely used correctly.
 ...
Me too, but that's mostly because it will be rarely used.
 
 I know. Actually version(assert) assert(...); also works. However, 
 this is too verbose, especially in contracts.
You can wrap it in a template. ...
That won't work for in contracts if they start catching ContractException instead of AssertError. Also, I think we'd actually like to _shorten_ the contract syntax (there is another DIP on this). For other uses, a function suffices, but I ideally want to keep using standard 'assert'. Everybody already knows what 'assert' means.
 
 I'd like a solution that does not require me to change the source 
 code. Ideally, I just want the Java behavior (with reversed defaults).
But you'll have to change the code to compiler_hint(). ...
I don't, because I don't want that behavior. Others who want that behavior also should not have to. This should be a compilation switch.
 
 (enforce is _completely unrelated_ to the current discussion.)
It does just what you ask (for the regular assert case). ...
No. "enforce" does not document that the programmer thinks that the condition will never fail. It's not a contract. Hence, enforcements are never removed from the executable, because it would not make sense to do so, so they definitely do not fit my requirements.
 
 It being UB was my doing, not Mathias'. DIP1006 is not redefining the 
 semantics of what assert does.
This is not really about assert semantics, this is about the semantics of "disabling the check".
It is very much about the semantics of assert.
 There was no "-check=off" flag before.
Yes there was, it's the "release" flag. ...
Depends on what you mean by "off". :o) In my book, "-release" is "-check-assert=assume -boundscheck=safeonly". A very strange combination!
 
 The DIP uses terminology such as "disable assertions" as opposed to 
 "disable assertion checks (but introduce compiler hints)".
Yes, the language could be more precise, but I couldn't blame Mathias for that.
I don't understand why that would be necessary. The point of the preliminary DIP review is not to blame the author for the DIP's shortcomings, it's to collect feedback on the DIP to make it better, ideally in an interactive way. It is also very well possible that Mathias was simply unaware of the UB behavior with -release.
 I also disagree with the word "hint", because it implies 
 things like "this branch is more likely to be taken" to guide code 
 generation decisions, rather than "assume X is absolutely always 
 incontrovertibly true and you can bet the code on it".
 
Makes sense. I'll call them "assumptions" from now on.
Mar 05
prev sibling parent Kagamin <spam here.lot> writes:
On Sunday, 4 March 2018 at 12:05:08 UTC, rjframe wrote:
 E.g., "dmd -check=in a.d" is unnecessary; it's equivalent to 
 "dmd a.d" "dmd -release -check=in a.d" turns off all checks 
 except in contracts.
I think, -check should specify hierarchic modes like in the DIP, that would work independently of -release switch (override).
Mar 04