digitalmars.D - DIP 1006 - Preliminary Review Round 1
- Mike Parker (13/13) Apr 12 2017 DIP 1006 is titled "Providing more selective control over
- rikki cattermole (4/14) Apr 12 2017 How exactly does this affect unittests?
- Nicholas Wilson (7/31) Apr 12 2017 This is a -release style optimisation and would not be expected
- rikki cattermole (7/32) Apr 12 2017 Please make it explicit that asserts are only in invariant and in/out
- Mathias Lang (8/32) Apr 12 2017 Good point.
- Martin Nowak (4/9) Nov 21 2017 As said above, asserts in unittests are different, so we could
- Daniel Kozak (3/16) Apr 12 2017 Is this supposed to affect release build, debug build or both of
- Timon Gehr (5/15) Apr 12 2017 The DIP should probably specify in detail how this interacts with -relea...
- Mathias Lang (17/44) Apr 12 2017 Addressing Daniel Kozak's question as well here:
- Timon Gehr (7/19) Apr 12 2017 Actually, (at least) asserts are turned into compiler hints (i.e.
- Andrea Fontana (6/19) Apr 12 2017 Why not
- Mathias Lang (5/29) Apr 12 2017 It was a conscious decision to provide something simple to use,
- Joseph Rushton Wakeling (10/14) Apr 12 2017 Well, it's not simple to use if it doesn't fulfil your use-case.
- Lewis (6/19) Apr 12 2017 I have to ask the newbie question, just to make sure we're not
- Mathias Lang (16/21) Apr 12 2017 There's no newbie question :)
- Steven Schveighoffer (15/36) Apr 27 2017 You can check the vtable address against the known Object.invariant addr...
- H. S. Teoh via Digitalmars-d (9/24) Apr 12 2017 Overall, I support the idea of this DIP.
- Olivier FAURE (6/12) Apr 26 2017 Agreed.
- Mike Parker (2/6) Apr 19 2017 Reminder: There's one week remaining.
- Mike Parker (2/10) Apr 25 2017 Final reminder: The review period ends in a little over 24 hours.
- Mike Parker (3/7) Apr 27 2017 The review period has ended. Thanks to everyone who provided
- Martin Nowak (15/18) Nov 21 2017 Has come up a couple of times and it's a good idea to allow more
- Joseph Rushton Wakeling (12/21) Nov 26 2017 I like the idea of specializing the meaning of the -release flag
- rikki cattermole (2/33) Nov 26 2017 Agreed that looks good +1
- Joseph Rushton Wakeling (7/21) Nov 27 2017 What would be the appropriate way to follow up on that idea? The
- rikki cattermole (2/25) Nov 27 2017 Contact Mike&Author of the DIP.
- Mike Parker (10/16) Nov 27 2017 Formal reviews have two stages. The first stage is just a
- Martin Nowak (11/14) Mar 03 2018 Doesn't really work that way, we can disable assertions, in
- Joseph Rushton Wakeling (10/20) Mar 05 2018 That's a shame, but presumably the fine-grainedness could be
- Iain Buclaw (10/30) Mar 05 2018 From memory, it would turn off asserts even in unittests. You
- Martin Nowak (2/4) Mar 03 2018 Possible implementation https://github.com/dlang/dmd/pull/7980 FYI
- Walter Bright (27/30) Mar 03 2018 Currently, we have 3 switches that affect the asserts: `release`, `bound...
- Nicholas Wilson (7/45) Mar 03 2018 Why? Implementation is trivial (unit testable no less!) , see
- Walter Bright (11/19) Mar 03 2018 It's not trivial. First, it's 100 lines of code (with no comments) just ...
- Nicholas Wilson (28/43) Mar 03 2018 I disagree, it is 100 (slightly less) lines of modular code and
- rjframe (13/15) Mar 04 2018 Would I be correct to interpret this as "turn them all off with -release...
- Walter Bright (2/3) Mar 04 2018 Array bounds checking is left on with -release.
- Timon Gehr (7/12) Mar 04 2018 Not necessarily. If the code contains an explicit assertion that the
- Walter Bright (6/17) Mar 04 2018 D, as all languages that I know of do implicitly or explicitly, generate...
- Timon Gehr (17/40) Mar 04 2018 Impossible. You wrote a Java compiler.
- Walter Bright (19/53) Mar 05 2018 Even in Java, the compiler generates code that, from the user's point of...
- Walter Bright (7/7) Mar 05 2018 The idea behind removal of the runtime checks is as a performance optimi...
- Timon Gehr (44/54) Mar 05 2018 Optimizing performance is fine, but why pessimize safety? The hints will...
- Walter Bright (9/45) Mar 05 2018 It is not defined behavior with -boundscheck=off.
- ag0aep6g (3/9) Mar 05 2018 Dereferencing null is not defined with -boundscheck=off?
- Timon Gehr (8/18) Mar 05 2018 This was my bad. It's not dereferencing null. The compiler is free to
- ag0aep6g (20/29) Mar 05 2018 How is it free to assume that?
- Timon Gehr (8/43) Mar 05 2018 By Walter's definition. -boundscheck=off makes the compiler assume that
- Timon Gehr (16/58) Mar 05 2018 (Maybe let's assume it was not me who did it, to stop the whole silly
- John Colvin (15/24) Mar 05 2018 void safeCode1(int a, ref int[2] b) @safe
- Walter Bright (2/4) Mar 05 2018 Then use `enforce()`.
- Timon Gehr (6/11) Mar 06 2018 That makes no sense at all. Enforce is for conditions that are expected
- Walter Bright (9/11) Mar 06 2018 My interpretation is you want D assert to behave like C assert. C assert...
- Timon Gehr (20/35) Mar 06 2018 This interpretation is wrong. I, as well as other people, want a
- Paolo Invernizzi (17/37) Mar 07 2018 Is it an option having the compiler not to remove the asserts
- ag0aep6g (7/11) Mar 07 2018 You're not. Just let the compiler treat the code as if the
- Paolo Invernizzi (10/22) Mar 07 2018 Are we asking to statically check things like:
- Jonathan M Davis (9/34) Mar 07 2018 In places where the compiler can statically check things, it does. In th...
- ag0aep6g (10/22) Mar 07 2018 If it can't be guaranteed that some code has defined behavior, then it's...
- ag0aep6g (30/37) Mar 07 2018 A simple way to get overlapping storage is with a union. Unfortunately,
- John Colvin (3/7) Mar 06 2018 So, to clarify, adding asserts to my code makes my release builds
- Paolo Invernizzi (9/17) Mar 06 2018 Only if the assert does not hold, you have _not_ tested it, and a
- Jonathan M Davis (19/36) Mar 06 2018 The problem with this approach is that even if you have tested your code
- ag0aep6g (5/9) Mar 06 2018 In other words: only if you have a bug in your code.
- Jonathan M Davis (12/20) Mar 06 2018 If the compiler actually optimized based on assertions, then yes, but no...
- 12345swordy (6/31) Mar 06 2018 I think a reasonable compromise is to introduce a new system
- Paolo Invernizzi (7/38) Mar 06 2018 I simply don't understand why enforce or a custom check can't be
- 12345swordy (5/8) Mar 06 2018 Easy:
- Jonathan M Davis (30/35) Mar 06 2018 1. It's quite common for folks to want to add debugging checks that are
- Paolo Invernizzi (8/49) Mar 07 2018 Understood. Are asking that UB should not include memory
- Jonathan M Davis (21/71) Mar 07 2018 I'm saying that whatever optimizations are enabled by assertions should ...
- Paolo Invernizzi (6/29) Mar 07 2018 Jonathan, I understand your reasoning, but it's not what I'm
- Jonathan M Davis (23/54) Mar 07 2018 I'm saying that @safe code must not be violated by the compiler. Beyond ...
- Paolo Invernizzi (18/81) Mar 07 2018 And that's clear.
- Jonathan M Davis (27/41) Mar 07 2018 That would make assertions a lot worse to use, because then they would b...
- Paolo Invernizzi (8/22) Mar 07 2018 Jonathan, I understand your point, but still I can't find an
- Jonathan M Davis (19/43) Mar 07 2018 @safe is all about guaranteeing memory safety. That's it's entire job. N...
- Timon Gehr (2/27) Mar 07 2018 UB is unconstrained by definition. If it is constrained, it is not UB.
- Paolo Invernizzi (5/20) Mar 07 2018 That! Thanks!
- Timon Gehr (16/38) Mar 07 2018 Yes. (If you have UB, memory corruption is one of the allowed outcomes,
- Paolo Invernizzi (11/52) Mar 07 2018 Thanks Timon, now everything it's definitely more clear to me on
- Timon Gehr (4/8) Mar 06 2018 I have explained why. UB is non-modular and you don't (want to) control
- Paolo Invernizzi (10/20) Mar 07 2018 As I've written, you should ask for debug versions of every
- Jonathan M Davis (62/69) Mar 06 2018 Except that disabling array bounds checking negates the guarantees of @s...
- Walter Bright (15/16) Mar 06 2018 The entire point of making assert a core language feature was so that th...
- Jonathan M Davis (10/27) Mar 06 2018 I don't mind if assertions allow the compiler to better optimize. What I
- Timon Gehr (8/28) Mar 06 2018 It does not need to be a core language feature for that.
- Leandro Lucarella (7/22) Jun 12 2018 As a side node, we have a `verify()` exactly for this:
- Timon Gehr (51/123) Mar 05 2018 I guess I misunderstood what you meant when you said "as if". I thought
- Iain Buclaw (19/24) Mar 05 2018 In the simple cases, or in anything that looks like a
- Timon Gehr (13/44) Mar 05 2018 (Note that by "forcing", I mean withholding other options from the user....
- Walter Bright (34/67) Mar 05 2018 Asserts give expressions that must be true. Why not take advantage of th...
- ag0aep6g (19/29) Mar 05 2018 Because it's exactly what @safe is not supposed to do. You're trusting
- Timon Gehr (45/156) Mar 05 2018 For some use cases it might be fine, but not for others, because you
- Kagamin (3/6) Mar 04 2018 I think, -check should specify hierarchic modes like in the DIP,
DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!
Apr 12 2017
On 12/04/2017 12:25 PM, Mike Parker wrote:DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!How exactly does this affect unittests? From what I can see, in none mode unittests won't have any asserts, which is clearly a problem.
Apr 12 2017
On Wednesday, 12 April 2017 at 11:32:37 UTC, rikki cattermole wrote:On 12/04/2017 12:25 PM, Mike Parker wrote:This is a -release style optimisation and would not be expected to be in dev/testing. Also unittest asserts call a different druntime function (IIRC) and therefore _should_ logically be under the control of -unittest and not affected by dip1006 OR be an invalid flag combination.DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!How exactly does this affect unittests? From what I can see, in none mode unittests won't have any asserts, which is clearly a problem.
Apr 12 2017
On 12/04/2017 12:48 PM, Nicholas Wilson wrote:On Wednesday, 12 April 2017 at 11:32:37 UTC, rikki cattermole wrote:Please make it explicit that asserts are only in invariant and in/out blocks are affected. With a note about unittests, just to remove this possible incorrect implementation detail. I am not happy with the args passed to the switch, but I can't think of an alternative names, so I am happy with how the DIP is including the above statements.On 12/04/2017 12:25 PM, Mike Parker wrote:This is a -release style optimisation and would not be expected to be in dev/testing. Also unittest asserts call a different druntime function (IIRC) and therefore _should_ logically be under the control of -unittest and not affected by dip1006 OR be an invalid flag combination.DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!How exactly does this affect unittests? From what I can see, in none mode unittests won't have any asserts, which is clearly a problem.
Apr 12 2017
On Wednesday, 12 April 2017 at 11:32:37 UTC, rikki cattermole wrote:On 12/04/2017 12:25 PM, Mike Parker wrote:Good point. I would say that `-contracts=none` and `-unittest` should behave the same as `-release` and `-unittest`, and currently it only turns asserts on ( https://github.com/dlang/dmd/blob/ac3225a025b578d45ff39a40dda35006fb455a37/src/ddm /mars.d#L1100-L1109 ). I'll add a note about it in the DIP.DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!How exactly does this affect unittests? From what I can see, in none mode unittests won't have any asserts, which is clearly a problem.
Apr 12 2017
On Wednesday, 12 April 2017 at 15:02:49 UTC, Mathias Lang wrote:I would say that `-contracts=none` and `-unittest` should behave the same as `-release` and `-unittest`, and currently it only turns asserts on ( https://github.com/dlang/dmd/blob/ac3225a025b578d45ff39a40dda35006fb455a37/src/ddm /mars.d#L1100-L1109 ). I'll add a note about it in the DIP.As said above, asserts in unittests are different, so we could separate enabling asserts in unittests and in the rest of the program.
Nov 21 2017
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!Is this supposed to affect release build, debug build or both of them?
Apr 12 2017
On 12.04.2017 13:25, Mike Parker wrote:DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!The DIP should probably specify in detail how this interacts with -release. I guess the difference between "-contracts=none -boundscheck=safeonly" and" -release" is that failing assertions are UB only with the latter? What happens if I pass both -contracts=none and -release?
Apr 12 2017
On Wednesday, 12 April 2017 at 12:52:42 UTC, Timon Gehr wrote:On 12.04.2017 13:25, Mike Parker wrote:Addressing Daniel Kozak's question as well here: Providing `-release` on the command line has the following effect: - Disable invariants, - Disable in and out, - Disable assert, - Disable switch error [1] Which `-contract` just allows more control on. TL;DR: It affects all build. It's a subset of `-release`, so passing `-contracts=whatever` and `-release` has the same effect as passing `-release`. [1] Switch error: Compiler code here: https://github.com/dlang/dmd/blob/0158d32fcbbdda16b3f79e73f2f1f8c13afb9f6d/src/ddmd/statementsem.d#L2120-L2145 When a `switch has no `default,` in debug it'll insert a `default` which throws a `core.exception : SwitchError`, and in release the generated `default` will just be an `HLT` (same as `assert(0)`).DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!The DIP should probably specify in detail how this interacts with -release. I guess the difference between "-contracts=none -boundscheck=safeonly" and" -release" is that failing assertions are UB only with the latter? What happens if I pass both -contracts=none and -release?
Apr 12 2017
On 12.04.2017 17:16, Mathias Lang wrote:Actually, (at least) asserts are turned into compiler hints (i.e. potential UB).Addressing Daniel Kozak's question as well here: Providing `-release` on the command line has the following effect: - Disable invariants, - Disable in and out, - Disable assert,- Disable switch error [1] ...I have missed that one. Note that it also implies -boundscheck=safeonly.Which `-contract` just allows more control on. TL;DR: It affects all build. It's a subset of `-release`, so passing `-contracts=whatever` and `-release` has the same effect as passing `-release`.It's not a subset unless "disable" can mean "turn failures into undefined behaviour".
Apr 12 2017
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!Why not --disable-contracts=invariant --disable-contracts=inout,invariants --disable-contracts=asserts,inout And so on?
Apr 12 2017
On Wednesday, 12 April 2017 at 13:45:08 UTC, Andrea Fontana wrote:On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:It was a conscious decision to provide something simple to use, over something which allowed more control (good old KISS). If a use case for it develop in the future, the addition will be trivial.DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!Why not --disable-contracts=invariant --disable-contracts=inout,invariants --disable-contracts=asserts,inout And so on?
Apr 12 2017
On Wednesday, 12 April 2017 at 15:37:14 UTC, Mathias Lang wrote:It was a conscious decision to provide something simple to use, over something which allowed more control (good old KISS). If a use case for it develop in the future, the addition will be trivial.Well, it's not simple to use if it doesn't fulfil your use-case. ;-) With that in mind, it would seem simpler overall to not make assumptions about use-cases, and just allow the user a free choice of what kinds of contract they disable: --disable-contracts=invariant,in,out,assert,all (Yes, I'm intentionally suggesting allowing `--disable-contracts=in`, `--disable-contracts=out`, and `--disable-contracts=in,out`.)
Apr 12 2017
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!I have to ask the newbie question, just to make sure we're not missing anything obvious. Why can't we fix invariants so that they're pay-for-what-you-use? In other words, is there a way we can make sure _d_invariant is never called (or early-outs) for classes that don't use invariants?
Apr 12 2017
On Wednesday, 12 April 2017 at 16:22:00 UTC, Lewis wrote:I have to ask the newbie question, just to make sure we're not missing anything obvious. Why can't we fix invariants so that they're pay-for-what-you-use? In other words, is there a way we can make sure _d_invariant is never called (or early-outs) for classes that don't use invariants?There's no newbie question :) Sadly it is not possible. Consider the following hierarchy: ``` class Mother : Object { public int i; public void myFunction () {} } class Daughter1 : Mother { invariant () { assert(i != 0); } } class Daughter2 : Mother {} ``` The `Mother` class needs to insert invariant checks at the beginning and the end of `myFunction` to account for the possibility of a derived class defining invariant (here `Daughter1`). However those checks are superfluous if no `invariant` is defined, as it's the case with `Daughter2`. However the base class cannot know this in advance (because it might be in a library and already compiler, for example).
Apr 12 2017
On 4/12/17 12:34 PM, Mathias Lang wrote:On Wednesday, 12 April 2017 at 16:22:00 UTC, Lewis wrote:You can check the vtable address against the known Object.invariant address. To explain further (no idea what the symbol names are, but you get the idea): executeInvariant(Object o) { static so = new Object; // has default invariant if((&so.__invariant).funcptr != &(so.__invariant).funcptr) // need to call invariant, not the default o.__invariant(); } I'm sure the compiler can do this without any indirections or virtual calls. Better yet, just put null in the Object.invariant vtable entry. Then it's really easy! -SteveI have to ask the newbie question, just to make sure we're not missing anything obvious. Why can't we fix invariants so that they're pay-for-what-you-use? In other words, is there a way we can make sure _d_invariant is never called (or early-outs) for classes that don't use invariants?There's no newbie question :) Sadly it is not possible. Consider the following hierarchy: ``` class Mother : Object { public int i; public void myFunction () {} } class Daughter1 : Mother { invariant () { assert(i != 0); } } class Daughter2 : Mother {} ``` The `Mother` class needs to insert invariant checks at the beginning and the end of `myFunction` to account for the possibility of a derived class defining invariant (here `Daughter1`). However those checks are superfluous if no `invariant` is defined, as it's the case with `Daughter2`. However the base class cannot know this in advance (because it might be in a library and already compiler, for example).
Apr 27 2017
On Wed, Apr 12, 2017 at 11:25:09AM +0000, Mike Parker via Digitalmars-d wrote:DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.md All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete. At the end of Round 1, if further review is deemed necessary, the DIP will be scheduled for another round. Otherwise, it will be queued for the formal review and evaluation by the language authors. Thanks in advance to all who participate. Destroy!Overall, I support the idea of this DIP. However, as others have mentioned, it needs to make it clear whether/how `-contracts=assert` here interacts with unittests. According to the discussion, apparently a different druntime function is used for asserts in unittests? If so, this needs to be clearly stated in the DIP. T -- People demand freedom of speech to make up for the freedom of thought which they avoid. -- Soren Aabye Kierkegaard (1813-1855)
Apr 12 2017
On Wednesday, 12 April 2017 at 17:16:33 UTC, H. S. Teoh wrote:Overall, I support the idea of this DIP. However, as others have mentioned, it needs to make it clear whether/how `-contracts=assert` here interacts with unittests. According to the discussion, apparently a different druntime function is used for asserts in unittests? If so, this needs to be clearly stated in the DIP.Agreed. The simplest behavior I could imagine for this switch is "override everything else". That is, no matter which other switch is used (release, unittests), the -contracts switch has the final say on which tests are enabled and which are suppressed.
Apr 26 2017
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete.Reminder: There's one week remaining.
Apr 19 2017
On Wednesday, 19 April 2017 at 13:28:03 UTC, Mike Parker wrote:On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:Final reminder: The review period ends in a little over 24 hours.All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete.Reminder: There's one week remaining.
Apr 25 2017
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:All review-related feedback on and discussion of the DIP should occur in this thread. The review period will end at 11:59 PM ET on April 26 (3:59 AM GMT), or when I make a post declaring it complete.The review period has ended. Thanks to everyone who provided feedback.
Apr 27 2017
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.mdHas come up a couple of times and it's a good idea to allow more control over which checks are enabled. I find the suggested switch levels a bit counter-intuitive and would suggest -release=assert,in,out,invariant to be the equivalent of the current -release while allowing to enable any subset, e.g. -release=assert,in. The effect of -release=assert -release=in would be to enable both. Using -release= also avoids any confusion about the interaction of -release and -contracts=.
Nov 21 2017
On Tuesday, 21 November 2017 at 14:15:30 UTC, Martin Nowak wrote:I like the idea of specializing the meaning of the -release flag via optional arguments. One suggestion: replace -release=assert with -release=body, so in the above, you would have: -release=body,in,out,invariant ... which has the nice intuitive property of specifying _which bits of code_ release criteria will be applied to. In other words, -release=body would result in asserts being removed from function bodies _and only there_. That would make clearer that we're not removing asserts from e.g. unittests (or indeed contracts or invariants).https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.mdHas come up a couple of times and it's a good idea to allow more control over which checks are enabled. I find the suggested switch levels a bit counter-intuitive and would suggest -release=assert,in,out,invariant to be the equivalent of the current -release while allowing to enable any subset
Nov 26 2017
On 26/11/2017 11:59 AM, Joseph Rushton Wakeling wrote:On Tuesday, 21 November 2017 at 14:15:30 UTC, Martin Nowak wrote:Agreed that looks good +1I like the idea of specializing the meaning of the -release flag via optional arguments. One suggestion: replace -release=assert with -release=body, so in the above, you would have: -release=body,in,out,invariant ... which has the nice intuitive property of specifying _which bits of code_ release criteria will be applied to. In other words, -release=body would result in asserts being removed from function bodies _and only there_. That would make clearer that we're not removing asserts from e.g. unittests (or indeed contracts or invariants).https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.mdHas come up a couple of times and it's a good idea to allow more control over which checks are enabled. I find the suggested switch levels a bit counter-intuitive and would suggest -release=assert,in,out,invariant to be the equivalent of the current -release while allowing to enable any subset
Nov 26 2017
On Sunday, 26 November 2017 at 12:09:37 UTC, rikki cattermole wrote:On 26/11/2017 11:59 AM, Joseph Rushton Wakeling wrote:What would be the appropriate way to follow up on that idea? The last I saw DIP 1006 was undergoing formal review, but the end of that period seems to have passed with no further follow-up. I could always write up an alternative DIP, but I don't know if that would be the most constructive way forward.One suggestion: replace -release=assert with -release=body, so in the above, you would have: -release=body,in,out,invariant ... which has the nice intuitive property of specifying _which bits of code_ release criteria will be applied to. In other words, -release=body would result in asserts being removed from function bodies _and only there_. That would make clearer that we're not removing asserts from e.g. unittests (or indeed contracts or invariants).Agreed that looks good +1
Nov 27 2017
On 27/11/2017 7:20 PM, Joseph Rushton Wakeling wrote:On Sunday, 26 November 2017 at 12:09:37 UTC, rikki cattermole wrote:Contact Mike&Author of the DIP.On 26/11/2017 11:59 AM, Joseph Rushton Wakeling wrote:What would be the appropriate way to follow up on that idea? The last I saw DIP 1006 was undergoing formal review, but the end of that period seems to have passed with no further follow-up. I could always write up an alternative DIP, but I don't know if that would be the most constructive way forward.One suggestion: replace -release=assert with -release=body, so in the above, you would have: -release=body,in,out,invariant ... which has the nice intuitive property of specifying _which bits of code_ release criteria will be applied to. In other words, -release=body would result in asserts being removed from function bodies _and only there_. That would make clearer that we're not removing asserts from e.g. unittests (or indeed contracts or invariants).Agreed that looks good +1
Nov 27 2017
On Monday, 27 November 2017 at 19:20:53 UTC, Joseph Rushton Wakeling wrote:What would be the appropriate way to follow up on that idea? The last I saw DIP 1006 was undergoing formal review, but the end of that period seems to have passed with no further follow-up.Formal reviews have two stages. The first stage is just a courtesy to allow last minute feedback before I ship the DIP to Walter & Andrei. After I send it to them, I await their decision and then I report it here. In this case, only the first stage was completed. It's in their hands and as soon as I receive a decision I'll report it here.I could always write up an alternative DIP, but I don't know if that would be the most constructive way forward.I would say that's the thing to do, but it's probably best to wait until we get a decision on 1006.
Nov 27 2017
On Sunday, 26 November 2017 at 11:59:28 UTC, Joseph Rushton Wakeling wrote:One suggestion: replace -release=assert with -release=body, so in the above, you would have: -release=body,in,out,invariantDoesn't really work that way, we can disable assertions, in contracts, out contracts, and invariants. But not assertions in some contexts while leaving them enabled in other contexts. At least not without modifying all related codegen and introducing context queries (e.g. think mixin templates). FWIW -release=assert,in,out,invariant fits out needs well enough. Just the use-case that someone wants to disable asserts in functions but still wants to use contracts, required to use a replacement for assert in contracts and invariants.
Mar 03 2018
On Saturday, 3 March 2018 at 16:33:00 UTC, Martin Nowak wrote:Doesn't really work that way, we can disable assertions, in contracts, out contracts, and invariants. But not assertions in some contexts while leaving them enabled in other contexts. At least not without modifying all related codegen and introducing context queries (e.g. think mixin templates).That's a shame, but presumably the fine-grainedness could be extended at some point. Question: what would -release=assert do to unittests? Would it not touch them at all? Or would it disable all asserts including in unittests?FWIW -release=assert,in,out,invariant fits out needs well enough. Just the use-case that someone wants to disable asserts in functions but still wants to use contracts, required to use a replacement for assert in contracts and invariants.Yea, there are obviously workarounds. I think the main concern from my side is to not have hierarchical assumptions about what gets turned on or off, and AFAICS -release=assert,in,out-invariant pretty much fits that.
Mar 05 2018
On Monday, 5 March 2018 at 18:44:54 UTC, Joseph Rushton Wakeling wrote:On Saturday, 3 March 2018 at 16:33:00 UTC, Martin Nowak wrote:From memory, it would turn off asserts even in unittests. You could raise a bug against gdc for that as it's a reasonable suggestion.Doesn't really work that way, we can disable assertions, in contracts, out contracts, and invariants. But not assertions in some contexts while leaving them enabled in other contexts. At least not without modifying all related codegen and introducing context queries (e.g. think mixin templates).That's a shame, but presumably the fine-grainedness could be extended at some point. Question: what would -release=assert do to unittests? Would it not touch them at all? Or would it disable all asserts including in unittests?N.B: GDC has -f[no]-release, -f[no-]assert, -f[no-]invariant, -f[no-]preconditions, and -f[no-]postconditions (-f[no-]in and -f[no-]out were removed as they are a little too vague). And it doesn't matter which order you pass them in, if an option is explicitly set, then they do not get turned on/off by -frelease.FWIW -release=assert,in,out,invariant fits out needs well enough. Just the use-case that someone wants to disable asserts in functions but still wants to use contracts, required to use a replacement for assert in contracts and invariants.Yea, there are obviously workarounds. I think the main concern from my side is to not have hierarchical assumptions about what gets turned on or off, and AFAICS -release=assert,in,out-invariant pretty much fits that.
Mar 05 2018
On Wednesday, 12 April 2017 at 11:25:09 UTC, Mike Parker wrote:DIP 1006 is titled "Providing more selective control over contracts".Possible implementation https://github.com/dlang/dmd/pull/7980 FYI
Mar 03 2018
On 4/12/2017 4:25 AM, Mike Parker wrote:DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.mdCurrently, we have 3 switches that affect the asserts: `release`, `boundscheck`, and `unittest`. The documentation for these new additions is completely confusing. No mention is made of interactions with the latter two. No mention is made on what happens if more than one of these switches is used, or what if the same switch appears multiple times. I'd like to see a chart enumerating all the behaviors influenced by these switches, the default behavior, and which switch settings do what. Each switch, when applied, should specify what happens to each behavior: 1. turned on 2. turned off 3. not affected and that the switches are applied in the order they appear on the command line. The use of comma-separated arguments is something I've argued against for other switches. The use of `-release=in -release=out` should be fine and is less confusing/buggy to implement. The idea that `-release=in` actually turns *off* `in` is completely topsy-turvy. Ideally, the whole thing should be simplified to its fundamentals: 1. leave the old switches as they are 2. Add a new switch, let's say "check": -check turn on all checks -check=on turn on all checks -check=off turn off all checks -check=xxx turn on check xxx -check=yyy turn on check yyy simple, easy to explain, easy to comprehend, easy to implement, easy to extend with new checks.
Mar 03 2018
On Sunday, 4 March 2018 at 00:32:20 UTC, Walter Bright wrote:On 4/12/2017 4:25 AM, Mike Parker wrote:Why? Implementation is trivial (unit testable no less!) , see https://github.com/dlang/dmd/pull/7863 and extensible to other args.DIP 1006 is titled "Providing more selective control over contracts". https://github.com/dlang/DIPs/blob/master/DIPs/DIP1006.mdCurrently, we have 3 switches that affect the asserts: `release`, `boundscheck`, and `unittest`. The documentation for these new additions is completely confusing. No mention is made of interactions with the latter two. No mention is made on what happens if more than one of these switches is used, or what if the same switch appears multiple times. I'd like to see a chart enumerating all the behaviors influenced by these switches, the default behavior, and which switch settings do what. Each switch, when applied, should specify what happens to each behavior: 1. turned on 2. turned off 3. not affected and that the switches are applied in the order they appear on the command line. The use of comma-separated arguments is something I've argued against for other switches. The use of `-release=in -release=out` should be fine and is less confusing/buggy to implement.The idea that `-release=in` actually turns *off* `in` is completely topsy-turvy. Ideally, the whole thing should be simplified to its fundamentals: 1. leave the old switches as they are 2. Add a new switch, let's say "check": -check turn on all checks -check=on turn on all checks -check=off turn off all checks -check=xxx turn on check xxx -check=yyy turn on check yyy simple, easy to explain, easy to comprehend, easy to implement, easy to extend with new checks.I think aggregation is a good idea. So is phrasing in the positive sense, except for it looks like xxx and yyy are _off_ by default, where surely we want all checks _on_ by default.
Mar 03 2018
On 3/3/2018 5:29 PM, Nicholas Wilson wrote:It's not trivial. First, it's 100 lines of code (with no comments) just for that, and there are templates and behaviors with memory allocation. Moving along that path, we're gradually reinventing std.getopt which is 1814 lines (with comments). Generally, people will be driving dmd with a makefile, dmd.conf, or other response file. It's complicated enough already, and gets constantly worse. Once in, we're stuck with it forever. I just don't feel it is worthwhile spending time on this.The use of comma-separated arguments is something I've argued against for other switches. The use of `-release=in -release=out` should be fine and is less confusing/buggy to implement.Why? Implementation is trivial (unit testable no less!) , see https://github.com/dlang/dmd/pull/7863 and extensible to other args.I think aggregation is a good idea. So is phrasing in the positive sense, except for it looks like xxx and yyy are _off_ by default, where surely we want all checks _on_ by default.The default is that they're all on. So to just have one on, first turn them all off then turn on the desired ones.
Mar 03 2018
On Sunday, 4 March 2018 at 04:30:31 UTC, Walter Bright wrote:It's not trivial. First, it's 100 lines of code (with no comments) just for that, and there are templates and behaviors with memory allocation. Moving along that path, we're gradually reinventing std.getopt which is 1814 lines (with comments).I disagree, it is 100 (slightly less) lines of modular code and can be used for -i,-I,-J,-check and possibly a whole lot more. Yes there are no comments but it's not exactly difficult to follow. You get that from modular code. parseCommandLine is 746! and is much more complicated. I'm very glad that in LDC we have access to LLVM command line arguments which are purely declarative.Generally, people will be driving dmd with a makefile, dmd.conf, or other response file.You still have to write those.It's complicated enough already, and gets constantly worse.And it won't get simpler unless anything is done about it!Once in, we're stuck with it forever.We can (and do) deprecate things.I just don't feel it is worthwhile spending time on this.Then the situation will not improve. It is a shame to lose this excellent work.Good.I think aggregation is a good idea. So is phrasing in the positive sense, except for it looks like xxx and yyy are _off_ by default, where surely we want all checks _on_ by default.The default is that they're all on.So to just have one on, first turn them all off then turn on the desired ones.Not so good. That is the opposite pattern of use I would expect. I would much rather turn them off individually, because I might forget one or a new release might add more and then it is off by default. Not to mention it result in way more args and the intention is much less clear . Looking at a command line invocation of DMD could you figure out which checks are off by looking at: ` -checks=off -checks=boundscheck -checks=in -checks=out -checks=unittest` don't forget they could be interspersed between other args _and_ be in the conf file. The types of checks are a closed set, it makes sense to specify them all at once.
Mar 03 2018
On Sat, 03 Mar 2018 20:30:31 -0800, Walter Bright wrote:The default is that they're all on. So to just have one on, first turn them all off then turn on the desired ones.Would I be correct to interpret this as "turn them all off with -release"? E.g., "dmd -check=in a.d" is unnecessary; it's equivalent to "dmd a.d" "dmd -release -check=in a.d" turns off all checks except in contracts. If adding any check turns everything else off, that seems strange; that said, wouldn't -check=off be equivalent to -release? Do we need -check=off and -check=on, since it just duplicates the -release toggle? I do like adding the -check switch because it reads far more naturally than -release=in, etc. But as far as readability, only using -check once -release is passed to disable everything makes sense, as using -check shouldn't disable other checks, and -release shows intention better than starting with -check=off --Ryan
Mar 04 2018
On 3/4/2018 4:05 AM, rjframe wrote:Would I be correct to interpret this as "turn them all off with -release"?Array bounds checking is left on with -release.
Mar 04 2018
On 04.03.2018 21:40, Walter Bright wrote:On 3/4/2018 4:05 AM, rjframe wrote:Not necessarily. If the code contains an explicit assertion that the index is in bounds, then, according to the language specification, the bounds check may be removed with -release. I find the reasoning in terms of "on"/"off" confusing anyway. Does "off" mean "contract/assertion removed", or does it mean "failure is UB"?Would I be correct to interpret this as "turn them all off with -release"?Array bounds checking is left on with -release.
Mar 04 2018
On 3/4/2018 1:16 PM, Timon Gehr wrote:On 04.03.2018 21:40, Walter Bright wrote:D, as all languages that I know of do implicitly or explicitly, generates code based on the "as if" rule.On 3/4/2018 4:05 AM, rjframe wrote:Not necessarily. If the code contains an explicit assertion that the index is in bounds, then, according to the language specification, the bounds check may be removed with -release.Would I be correct to interpret this as "turn them all off with -release"?Array bounds checking is left on with -release.I find the reasoning in terms of "on"/"off" confusing anyway. Does "off" mean "contract/assertion removed", or does it mean "failure is UB"?"Off" means the check is removed. If the check does not hold, the program enters an invalid state, whether or not the check was actually done. An invalid state means subsequent execution is UB.
Mar 04 2018
On 04.03.2018 22:49, Walter Bright wrote:On 3/4/2018 1:16 PM, Timon Gehr wrote:Impossible. You wrote a Java compiler. All languages that use your "as if" rule are memory unsafe. Zero languages that use the "as if" rule have any memory safe subset that includes assertions. In D, assert is safe, and it should remain safe.On 04.03.2018 21:40, Walter Bright wrote:D, as all languages that I know of do implicitly or explicitly, generates code based on the "as if" rule. ...On 3/4/2018 4:05 AM, rjframe wrote:Not necessarily. If the code contains an explicit assertion that the index is in bounds, then, according to the language specification, the bounds check may be removed with -release.Would I be correct to interpret this as "turn them all off with -release"?Array bounds checking is left on with -release.Why is potential memory corruption to be expected when using safe language features with a flag to disable contract checks? This makes no sense. This is not useful behavior. There are convenient ways to support potentially unsound compilation hints that do not do this. Contracts and compilation hints should be orthogonal. Contracts should be potentially safe, compilation hints should be system always. Note that _actual removal_ is the only use case of 'disabling contracts' that I care about, and I think many D programmers who use "off" will also have this behavior in mind. Yet this is not even an option. At the very least, the DIP should be up-front about this. I'm still not even sure that Mathias Lang intended the UB semantics.I find the reasoning in terms of "on"/"off" confusing anyway. Does "off" mean "contract/assertion removed", or does it mean "failure is UB"?"Off" means the check is removed. If the check does not hold, the program enters an invalid state, whether or not the check was actually done. An invalid state means subsequent execution is UB.
Mar 04 2018
On 3/4/2018 3:06 PM, Timon Gehr wrote:On 04.03.2018 22:49, Walter Bright wrote:Even in Java, the compiler generates code that, from the user's point of view, behaves "as if" the code was actually what was specified. For a trivial example, replacing x*2 with x<<1. Not having this means no optimizations can be done.Impossible. You wrote a Java compiler.Not necessarily. If the code contains an explicit assertion that the index is in bounds, then, according to the language specification, the bounds check may be removed with -release.D, as all languages that I know of do implicitly or explicitly, generates code based on the "as if" rule. ...All languages that use your "as if" rule are memory unsafe. Zero languages that use the "as if" rule have any memory safe subset that includes assertions. In D, assert is safe, and it should remain safe.Because the checks provide extra information to the compiler that it can use to generate better code. If that extra information is not true, then the better code will be invalid. Memory safety is only one class of errors in a program. If the program has entered a state that is not accounted for by the programmer, the rest of the program's execution will be not predictable.Why is potential memory corruption to be expected when using safe language features with a flag to disable contract checks?I find the reasoning in terms of "on"/"off" confusing anyway. Does "off" mean "contract/assertion removed", or does it mean "failure is UB"?"Off" means the check is removed. If the check does not hold, the program enters an invalid state, whether or not the check was actually done. An invalid state means subsequent execution is UB.This makes no sense. This is not useful behavior. There are convenient ways to support potentially unsound compilation hints that do not do this. Contracts and compilation hints should be orthogonal. Contracts should be potentially safe, compilation hints should be system always. Note that _actual removal_ is the only use case of 'disabling contracts' that I care about, and I think many D programmers who use "off" will also have this behavior in mind. Yet this is not even an option.I don't see much use for this behavior, unless you want to continue running the program after an assert failure, which I cannot recommend and the language is not designed to support. But you can always do something like: version (ignore_asserts) { } else { assert(...); } which would optionally remove both the runtime check and any compiler use of the assert. Or you could use https://dlang.org/library/std/exception/enforce.html which has no influence on compiler semantics.At the very least, the DIP should be up-front about this. I'm still not even sure that Mathias Lang intended the UB semantics.It being UB was my doing, not Mathias'. DIP1006 is not redefining the semantics of what assert does.
Mar 05 2018
The idea behind removal of the runtime checks is as a performance optimization done on a debugged program. It's like turning on or off array bounds checking. Many leave asserts and array bounds checking on even in released code to ensure memory safety. At a minimum, turning it off and on will illuminate just what the checks are costing you. It's at the option of the programmer.
Mar 05 2018
On 05.03.2018 11:30, Walter Bright wrote:The idea behind removal of the runtime checks is as a performance optimization done on a debugged program.Optimizing performance is fine, but why pessimize safety? The hints will usually not make a significant difference in performance anyway. I guess it is fine to have a compiler option that is all speed no safety, but it should not be the only option.It's like turning on or off array bounds checking.It is not. void main() safe{ int[] x=[]; writeln(x[0]); // range violation even with -release // defined behavior even with -boundscheck=off (!) } If I now add an assertion, I suddenly get UB: void main() safe{ int[] x=[]; assert(0<x.length); // obviously this should hold, or next line is invalid writeln(x[0]); // UB with -release } I did not make the code any more wrong by adding the assertion. Why should I get more UB?Many leave asserts and array bounds checking on even in released code to ensure memory safety. ...Maybe the requirements change and it is now too costly to leave contracts on in release mode, or the number of contracts in the code base slowly accumulates until we reach a point where the total cost is too large, or we replace a library, and the new version has costly contracts, etc. Now we have the following options: - Leave contracts in -- fail performance requirements. - Remove contracts -- fail safety requirements. - Track down all 'assert's, even those in external libraries, and replace them by a custom home-cooked solution that is incompatible with everyone else's -- fail maintainability requirements. To me this situation is ridiculous.At a minimum, turning it off and on will illuminate just what the checks are costing you. ...Well, no. If the bug is elusive enough to not have shown up in debug mode, it probably won't be seen early during -release testing, and even if it does, the UB may mask it. (Note that when the program becomes faster, the likelihood of timing-dependent bugs showing up may change.) I.e., if something goes wrong, it is likely that you won't see the safety costs until it is too late.It's at the option of the programmer.It is not, but I want it to be. That's all I want. [1] I'm just saying there should be the following option: - Remove contracts -- sufficient performance and retain memory safety. FWIW, this is what all contract systems that I'm aware of do, except D, and maybe C asserts in certain implementations (if you want to call that contracts). [1] Well, maybe add a system "__assume" intrinsic.
Mar 05 2018
On 3/5/2018 11:34 AM, Timon Gehr wrote:On 05.03.2018 11:30, Walter Bright wrote:Reasonable people will disagree on what is significant or not.The hints will usually not make a significant difference in performance anyway.It is not defined behavior with -boundscheck=off.It's like turning on or off array bounds checking.It is not. void main() safe{ int[] x=[]; writeln(x[0]); // range violation even with -release // defined behavior even with -boundscheck=off (!)} If I now add an assertion, I suddenly get UB: void main() safe{ int[] x=[]; assert(0<x.length); // obviously this should hold, or next line is invalid writeln(x[0]); // UB with -release } I did not make the code any more wrong by adding the assertion. Why should I get more UB?Because you put in an assert that did not hold, and disabled the check.Now we have the following options: - Leave contracts in -- fail performance requirements. - Remove contracts -- fail safety requirements. - Track down all 'assert's, even those in external libraries, and replace them by a custom home-cooked solution that is incompatible with everyone else's -- fail maintainability requirements. To me this situation is ridiculous.It's completely under the control of the programmer. I know you disagree with that notion. You can even create your own `myassert` to produced your desired semantics.FWIW, this is what all contract systems that I'm aware of do, except D, and maybe C asserts in certain implementations (if you want to call that contracts).D is better (!). (C's asserts are not part of the language, so impart no semantics to the compiler.)
Mar 05 2018
On 03/05/2018 10:11 PM, Walter Bright wrote:On 3/5/2018 11:34 AM, Timon Gehr wrote:[...]Dereferencing null is not defined with -boundscheck=off?int[] x=[]; writeln(x[0]); // range violation even with -release // defined behavior even with -boundscheck=off (!)It is not defined behavior with -boundscheck=off.
Mar 05 2018
On 05.03.2018 22:24, ag0aep6g wrote:On 03/05/2018 10:11 PM, Walter Bright wrote:This was my bad. It's not dereferencing null. The compiler is free to assume 0<x.length, which means it is allowed to think that the main function is dead code. Anyway, a similar point can be made by considering contracts that say that specific values are non-null. They will turn null values into UB even though without them, null dereferences would have been defined to crash.On 3/5/2018 11:34 AM, Timon Gehr wrote:[...]Dereferencing null is not defined with -boundscheck=off?int[] x=[]; writeln(x[0]); // range violation even with -release // defined behavior even with -boundscheck=off (!)It is not defined behavior with -boundscheck=off.
Mar 05 2018
On 03/05/2018 11:57 PM, Timon Gehr wrote:On 05.03.2018 22:24, ag0aep6g wrote:[...]On 03/05/2018 10:11 PM, Walter Bright wrote:How is it free to assume that? This was the full snippet (before I mutilated it in my quote): ---- void main() safe{ int[] x=[]; writeln(x[0]); // range violation even with -release // defined behavior even with -boundscheck=off (!) } ---- There is no `assert(0<x.length);` in this one. -release doesn't do anything, because there are no contracts, no asserts, and main is safe. -boundscheck=off just makes it so that the length isn't checked before x.ptr is dereferenced. x.ptr is null, so the code is defined to dereference null, no? If -boundscheck=off somehow does introduce UB here, we have the weird situation that using `x.ptr[0]` is more safe than in this scenario than `x[0]`. Because surely `x.ptr[0]` is a null dereference that's not affected by -boundscheck=off, right?This was my bad. It's not dereferencing null. The compiler is free to assume 0<x.length, which means it is allowed to think that the main function is dead code.It is not defined behavior with -boundscheck=off.Dereferencing null is not defined with -boundscheck=off?
Mar 05 2018
On 06.03.2018 00:52, ag0aep6g wrote:On 03/05/2018 11:57 PM, Timon Gehr wrote:By Walter's definition. -boundscheck=off makes the compiler assume that all array accesses are within bounds. ("off" is a misleading term.)On 05.03.2018 22:24, ag0aep6g wrote:[...]On 03/05/2018 10:11 PM, Walter Bright wrote:How is it free to assume that? ...This was my bad. It's not dereferencing null. The compiler is free to assume 0<x.length, which means it is allowed to think that the main function is dead code.It is not defined behavior with -boundscheck=off.Dereferencing null is not defined with -boundscheck=off?This was the full snippet (before I mutilated it in my quote): ---- void main() safe{ int[] x=[]; writeln(x[0]); // range violation even with -release // defined behavior even with -boundscheck=off (!) } ---- There is no `assert(0<x.length);` in this one. -release doesn't do anything, because there are no contracts, no asserts, and main is safe. -boundscheck=off just makes it so that the length isn't checked before x.ptr is dereferenced.It's not checked, but the compiler may still assume that it has actually been checked. The story is similar to asserts.x.ptr is null, so the code is defined to dereference null, no? If -boundscheck=off somehow does introduce UB here, we have the weird situation that using `x.ptr[0]` is more safe than in this scenario than `x[0]`. Because surely `x.ptr[0]` is a null dereference that's not affected by -boundscheck=off, right?Yes, I think that's a good point (though it hinges on the assumption that x.ptr[i] is equivalent to *(x.ptr+i), which I'm not sure the specification states explicitly).
Mar 05 2018
On 05.03.2018 22:11, Walter Bright wrote:On 3/5/2018 11:34 AM, Timon Gehr wrote:My point exactly! Hence, compiler flag.On 05.03.2018 11:30, Walter Bright wrote:Reasonable people will disagree on what is significant or not. ...The hints will usually not make a significant difference in performance anyway....(Maybe let's assume it was not me who did it, to stop the whole silly "you deserve what you got because you made a mistake" notion.) Again, my question is not about the _mechanics_ of the status quo. I know it very well. It's the rationale that matters.I did not make the code any more wrong by adding the assertion. Why should I get more UB?Because you put in an assert that did not hold, and disabled the check. ...I don't. I can use a manual patch of the compiler that has the additionally required flags and replicate the official packaging effort and make everyone who wants to compile my programs use that version. I just don't want to, as it seems silly. It would be a lot better if the standard DMD compiler had the flags. Do you disagree that there should be an additional option to ignore contracts completely?Now we have the following options: - Leave contracts in -- fail performance requirements. - Remove contracts -- fail safety requirements. - Track down all 'assert's, even those in external libraries, and replace them by a custom home-cooked solution that is incompatible with everyone else's -- fail maintainability requirements. To me this situation is ridiculous.It's completely under the control of the programmer. I know you disagree with that notion.You can even create your own `myassert` to produced your desired semantics. ...That's the third option above. It's not a practical solution. Putting the flag into a compiler fork is trivial by comparison.(That's why I said "in certain implementations".)FWIW, this is what all contract systems that I'm aware of do, except D, and maybe C asserts in certain implementations (if you want to call that contracts).D is better (!). (C's asserts are not part of the language, so impart no semantics to the compiler.)
Mar 05 2018
On Monday, 5 March 2018 at 10:30:12 UTC, Walter Bright wrote:The idea behind removal of the runtime checks is as a performance optimization done on a debugged program. It's like turning on or off array bounds checking. Many leave asserts and array bounds checking on even in released code to ensure memory safety. At a minimum, turning it off and on will illuminate just what the checks are costing you. It's at the option of the programmer.void safeCode1(int a, ref int[2] b) safe { assert(a < 2); b[a] = 0; } So, if I compile this with `-release -O`, the compiler is free to remove the bounds-check, which will cause a buffer overrun if `a1`. Ok.void safeCode2(int a, ref int[2] b) safe { b[a] = 0; } And here the compiler is *not* free to remove the bounds check. This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.
Mar 05 2018
On 3/5/2018 2:30 PM, John Colvin wrote:This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.Then use `enforce()`.
Mar 05 2018
On 06.03.2018 03:05, Walter Bright wrote:On 3/5/2018 2:30 PM, John Colvin wrote:That makes no sense at all. Enforce is for conditions that are expected to fail in exceptional circumstances. It does not document the same intent as assert does and it cannot be ignored in release builds. Anyway, "do not use assert" is not the solution, as I have explained many times now.This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.Then use `enforce()`.
Mar 06 2018
On 3/6/2018 12:45 AM, Timon Gehr wrote:Anyway, "do not use assert" is not the solution, as I have explained many times now.My interpretation is you want D assert to behave like C assert. C assert and enforce are purely creatures of the library, with semantics defined by their library implementation, and have no effect on the core language. I recommend creating your own library assert, call it 'check' for example, and give it the semantics you wish. You can even have it expand to nothing for release builds. Creating library asserts is why D has special support for __FILE__ and __LINE__ like C does, and for the same reasons.
Mar 06 2018
On 06.03.2018 10:02, Walter Bright wrote:On 3/6/2018 12:45 AM, Timon Gehr wrote:This interpretation is wrong. I, as well as other people, want a compiler option to make the compiler ignore D asserts and contracts. Not more, not less.Anyway, "do not use assert" is not the solution, as I have explained many times now.My interpretation is you want D assert to behave like C assert.C assert and enforce are purely creatures of the library, with semantics defined by their library implementation, and have no effect on the core language. ...That's a meaningless proposition. The current D -release assert behavior can be implemented in a library (just trigger UB in the false branch).I recommend creating your own library assert, call it 'check' for example, and give it the semantics you wish. You can even have it expand to nothing for release builds. ...Well, awesome. Now I need to make everyone on the project as well as external libraries, such as Phobos, use my 'check' function, when the contract documentation tells them to use assert and does not even hint at the downsides. No, thanks. I'd rather fork the compiler. But I have explained this (and further reasons) already. I suspect you did not read my posts. A few others have. I'll try to keep this one shorter.Creating library asserts is why D has special support for __FILE__ and __LINE__ like C does, and for the same reasons.What I want is special support for sane built-in assert semantics using a compiler flag. That does not mean that there cannot _also_ be a flag to unleash the nasal demons upon unworthy programmers who were stupid enough to collaborate with someone who imported an external library that was written by somebody who had a bad day one time and left in a wrong assertion. Again: There is no reason why we need to force one behavior over the other. This should be configurable.
Mar 06 2018
On Tuesday, 6 March 2018 at 23:50:20 UTC, Timon Gehr wrote:On 06.03.2018 10:02, Walter Bright wrote:Is it an option having the compiler not to remove the asserts from safe code, like bounds check? Just to understand, otherwise, if the assert is removed and it does not hold, you are in UB, so the request is to guarantee memory safety in a UB state, right? The point I don't grasp is why keep running in UB in safe is acceptable, while memory corruption not.On 3/6/2018 12:45 AM, Timon Gehr wrote:This interpretation is wrong. I, as well as other people, want a compiler option to make the compiler ignore D asserts and contracts. Not more, not less.Anyway, "do not use assert" is not the solution, as I have explained many times now.My interpretation is you want D assert to behave like C assert.and that's a reasonable request, that, IMHO, does not hurts anybody....Creating library asserts is why D has special support for __FILE__ and __LINE__ like C does, and for the same reasons.What I want is special support for sane built-in assert semantics using a compiler flag.That does not mean that there cannot _also_ be a flag to unleash the nasal demons upon unworthy programmers who were stupid enough to collaborate with someone who imported an external library that was written by somebody who had a bad day one time and left in a wrong assertion.That's a process management problem, I think. Debug version of external libraries can be requested, and the author can be reported about the bad-day wrong assert. I know, if I can catch it...Again: There is no reason why we need to force one behavior over the other. This should be configurable.I'm all in for having the maximum flexibility in configuration, but I would stick with Walter as keeping its idea as the default. /Paolo
Mar 07 2018
On Wednesday, 7 March 2018 at 08:58:50 UTC, Paolo Invernizzi wrote:Just to understand, otherwise, if the assert is removed and it does not hold, you are in UB,You're not. Just let the compiler treat the code as if the asserts weren't there. If the resulting code has UB, it won't compile, because safe code is statically checked to not have UB.so the request is to guarantee memory safety in a UB state, right?I don't think anyone is asking for that. The request is for no UB in safe code.
Mar 07 2018
On Wednesday, 7 March 2018 at 13:32:37 UTC, ag0aep6g wrote:On Wednesday, 7 March 2018 at 08:58:50 UTC, Paolo Invernizzi wrote:Are we asking to statically check things like: Assign Expressions [1] Undefined Behavior: if the lvalue and rvalue have partially overlapping storage if the lvalue and rvalue's storage overlaps exactly but the types are different Is that doable, in practise? [1] https://dlang.org/spec/expression.html#assign_expressions /PaoloJust to understand, otherwise, if the assert is removed and it does not hold, you are in UB,You're not. Just let the compiler treat the code as if the asserts weren't there. If the resulting code has UB, it won't compile, because safe code is statically checked to not have UB.so the request is to guarantee memory safety in a UB state, right?I don't think anyone is asking for that. The request is for no UB in safe code.
Mar 07 2018
On Wednesday, March 07, 2018 14:01:31 Paolo Invernizzi via Digitalmars-d wrote:On Wednesday, 7 March 2018 at 13:32:37 UTC, ag0aep6g wrote:In places where the compiler can statically check things, it does. In the places where it can't, it either introduces runtime checks (e.g. array bounds checking), or it treats the code as system, forcing the programmer to ensure that the code is safe, since the compiler can't determine whether it is or not. Either way, we then get the guarantee that safe code is memory safe so long as trusted is used correctly. - Jonathan M DavisOn Wednesday, 7 March 2018 at 08:58:50 UTC, Paolo Invernizzi wrote:Are we asking to statically check things like: Assign Expressions [1] Undefined Behavior: if the lvalue and rvalue have partially overlapping storage if the lvalue and rvalue's storage overlaps exactly but the types are different Is that doable, in practise? [1] https://dlang.org/spec/expression.html#assign_expressionsJust to understand, otherwise, if the assert is removed and it does not hold, you are in UB,You're not. Just let the compiler treat the code as if the asserts weren't there. If the resulting code has UB, it won't compile, because safe code is statically checked to not have UB.so the request is to guarantee memory safety in a UB state, right?I don't think anyone is asking for that. The request is for no UB in safe code.
Mar 07 2018
On 03/07/2018 03:01 PM, Paolo Invernizzi wrote:On Wednesday, 7 March 2018 at 13:32:37 UTC, ag0aep6g wrote:[...]If it can't be guaranteed that some code has defined behavior, then it's not allowed in an safe function (or it should not be allowed). We are not asking for all valid code to be safe. Guaranteeing no UB is exactly safe's purpose. The spec says: "Safe functions are functions that are statically checked to exhibit no possibility of undefined behavior." [1]I don't think anyone is asking for that. The request is for no UB in safe code.Are we asking to statically check things like: Assign Expressions [1] Undefined Behavior: if the lvalue and rvalue have partially overlapping storage if the lvalue and rvalue's storage overlaps exactly but the types are differentIs that doable, in practise?If you think that's not doable, what do you think safe should aim for? [1] https://dlang.org/spec/function.html#function-safety
Mar 07 2018
On 03/07/2018 03:01 PM, Paolo Invernizzi wrote:Are we asking to statically check things like: Assign Expressions [1] Undefined Behavior: if the lvalue and rvalue have partially overlapping storage if the lvalue and rvalue's storage overlaps exactly but the types are differentA simple way to get overlapping storage is with a union. Unfortunately, DMD accepts this: ---- struct S { union { int i; byte b; float f; struct { byte b2; align(1) int i2; } } } void main() safe { S s; s.i = s.b; /* Partially overlapping, different types. */ s.f = s.i; /* Exactly overlapping, different types. */ s.i = s.i2; /* Partially overlapping, same type. */ } ---- I've filed an issue: https://issues.dlang.org/show_bug.cgi?id=18568 If you have more examples of UB in safe functions, don't hesitate to file them as bugs.
Mar 07 2018
On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:On 3/5/2018 2:30 PM, John Colvin wrote:So, to clarify, adding asserts to my code makes my release builds violate safe?This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.Then use `enforce()`.
Mar 06 2018
On Tuesday, 6 March 2018 at 16:30:09 UTC, John Colvin wrote:On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:Only if the assert does not hold, you have _not_ tested it, and a future optimiser will use the assert expression for some hints that generated broken code. At the end, I'm with Walter: you should have tested the code with the assert enabled, and you should have noticed that the assert it's not holding _before_ running the code in release without them. /PaoloOn 3/5/2018 2:30 PM, John Colvin wrote:So, to clarify, adding asserts to my code makes my release builds violate safe?This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.Then use `enforce()`.
Mar 06 2018
On Tuesday, March 06, 2018 17:01:04 Paolo Invernizzi via Digitalmars-d wrote:On Tuesday, 6 March 2018 at 16:30:09 UTC, John Colvin wrote:The problem with this approach is that even if you have tested your code well, sometimes you miss things, and then you have bugs. And that's life, but with safe, you're not supposed to then up with memory safety problems. That's why array bounds checking is not removed in safe code even with -release. Yes, well-tested code would hopefully not need the checks anymore, but bugs still creep in, and if the array bounds checking is removed, then you can end up with memory corruption. This issue is really no different from array bounds checking, and I don't see any reason to treat it any differently. safe could should be guaranteed to be safe so long as trusted is used correctly. The compiler should not be assuming that the programmer has done a perfect job of testing and caught every possible bug and that there's no way that an assertion would have failed if it were still compiled in and that it is thus safe to introduce optimizations which would not be memory safe if the assertion would have failed. If the compiler does that, then safe is being violated, which the compiler should never be doing. - Jonathan M DavisOn Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:Only if the assert does not hold, you have _not_ tested it, and a future optimiser will use the assert expression for some hints that generated broken code. At the end, I'm with Walter: you should have tested the code with the assert enabled, and you should have noticed that the assert it's not holding _before_ running the code in release without them.On 3/5/2018 2:30 PM, John Colvin wrote:So, to clarify, adding asserts to my code makes my release builds violate safe?This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.Then use `enforce()`.
Mar 06 2018
On 03/06/2018 06:01 PM, Paolo Invernizzi wrote:Only if the assert does not hold, you have _not_ tested it,In other words: only if you have a bug in your code. If safe is only safe as long you don't have bugs, it's no different from system. So -release turns safe code into system code, if it contains asserts.and a future optimiser will use the assert expression for some hints that generated broken code
Mar 06 2018
On Tuesday, March 06, 2018 16:30:09 John Colvin via Digitalmars-d wrote:On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous. If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner. - Jonathan M DavisOn 3/5/2018 2:30 PM, John Colvin wrote:So, to clarify, adding asserts to my code makes my release builds violate safe?This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.Then use `enforce()`.
Mar 06 2018
On Tuesday, 6 March 2018 at 17:24:35 UTC, Jonathan M Davis wrote:On Tuesday, March 06, 2018 16:30:09 John Colvin via Digitalmars-d wrote:I think a reasonable compromise is to introduce a new system attribute such as unsafeoptimize to tell the programmer that this function may have it's safe attribute removed when making optimizations based on the asserts. We have trusted attribute for a good reason here.On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous. If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner. - Jonathan M DavisOn 3/5/2018 2:30 PM, John Colvin wrote:So, to clarify, adding asserts to my code makes my release builds violate safe?This just feels bad. Adding extra failsafes for my debug program shouldn't make my release program less safe.Then use `enforce()`.
Mar 06 2018
On Tuesday, 6 March 2018 at 17:34:08 UTC, 12345swordy wrote:On Tuesday, 6 March 2018 at 17:24:35 UTC, Jonathan M Davis wrote:I simply don't understand why enforce or a custom check can't be used safe code, if you want that behaviour. If the program must HALT on this or that, well, what is better than an explicit piece of unremovable code that do that? Instead of writing 'assert', one should write 'enforce'. /PaoloOn Tuesday, March 06, 2018 16:30:09 John Colvin via Digitalmars-d wrote:I think a reasonable compromise is to introduce a new system attribute such as unsafeoptimize to tell the programmer that this function may have it's safe attribute removed when making optimizations based on the asserts. We have trusted attribute for a good reason here.On Tuesday, 6 March 2018 at 02:05:58 UTC, Walter Bright wrote:If the compiler actually optimized based on assertions, then yes, but not right now. As I understand it, the problem is theoretical at the moment, because the compiler does not yet optimize based on assertions, but once it does, if it's allowed to introduce optimizations that would be not be memory safe if the assertion would have failed if it hadn't been compiled out, then safe will be violated, and at that point, I would be telling everyone to never use assertions, because they're too dangerous. If we can restrict the compiler to optimizations that are memory safe, then I don't see a problem, but clearly, Walter is not in agreement that the optimizations should be restricted in that manner. - Jonathan M Davis[...]So, to clarify, adding asserts to my code makes my release builds violate safe?
Mar 06 2018
On Tuesday, 6 March 2018 at 18:49:42 UTC, Paolo Invernizzi wrote:I simply don't understand why enforce or a custom check can't be used safe code, if you want that behaviour. /PaoloEasy: the custom check itself could be wrong the custom check itself is right, but the implementation is wrong even though it may passed your unit tests.
Mar 06 2018
On Tuesday, March 06, 2018 18:49:42 Paolo Invernizzi via Digitalmars-d wrote:I simply don't understand why enforce or a custom check can't be used safe code, if you want that behaviour. If the program must HALT on this or that, well, what is better than an explicit piece of unremovable code that do that? Instead of writing 'assert', one should write 'enforce'.1. It's quite common for folks to want to add debugging checks that are compiled out in -release. That's exactly what assert is for in pretty much every lanugage that has it. It's what folks expect to use and what your average programmer will use without thinking about safety issues at all. It's what everyone uses right now, and I'm pretty sure that almost everyone using it has no clue that Walter considers it okay for assertions to introduce optimizations which are not memory safe, and if/when he does do so, a lot of D code will suddenly have safe code which is not memory safe. Such problems will hopefully be hit rarely, because hopefully, the code will have been well-tested, and the assertions will have found all of the related bugs, but there's every possibility that some bugs will manage to not be caught, thereby resulting in safe code being unsafe. No one is going to be looking to use solutions other than assertions for what assertions are for unless we start telling everyone to avoid assertions, because they make safe code unsafe. And honestly, if assertions make safe code unsafe, I don't see a good argument for using them at all. If I didn't care about code being safe, I wouldn't be using safe. safe is supposed to guarantee that the code is memory safe. 2. I think that it's fundamentally a terrible idea to allow built-in language features to violate safe. Aside from issues related to trusted being misused, safe code should be guaranteed to be memory safe, and it should be considered a bug any time that it isn't. That's why safe exists. No one should have to be looking at safe code to track down memory safety problems. And if they do, then safe is not doing it's job. Array bounds checks are left in safe code for exactly these reasons. I'm all for introducing optimizations that do not violate safe, but if we allow safe code to be unsafe, then why do we even have it? - Jonathan M Davis
Mar 06 2018
On Tuesday, 6 March 2018 at 20:03:11 UTC, Jonathan M Davis wrote:On Tuesday, March 06, 2018 18:49:42 Paolo Invernizzi via Digitalmars-d wrote:Understood. Are asking that UB should not include memory corruptions?I simply don't understand why enforce or a custom check can't be used safe code, if you want that behaviour. If the program must HALT on this or that, well, what is better than an explicit piece of unremovable code that do that? Instead of writing 'assert', one should write 'enforce'.1. It's quite common for folks to want to add debugging checks that are compiled out in -release. That's exactly what assert is for in pretty much every lanugage that has it. It's what folks expect to use and what your average programmer will use without thinking about safety issues at all. It's what everyone uses right now, and I'm pretty sure that almost everyone using it has no clue that Walter considers it okay for assertions to introduce optimizations which are not memory safe, and if/when he does do so, a lot of D code will suddenly have safe code which is not memory safe. Such problems will hopefully be hit rarely, because hopefully, the code will have been well-tested, and the assertions will have found all of the related bugs, but there's every possibility that some bugs will manage to not be caught, thereby resulting in safe code being unsafe. No one is going to be looking to use solutions other than assertions for what assertions are for unless we start telling everyone to avoid assertions, because they make safe code unsafe. And honestly, if assertions make safe code unsafe, I don't see a good argument for using them at all. If I didn't care about code being safe, I wouldn't be using safe. safe is supposed to guarantee that the code is memory safe.2. I think that it's fundamentally a terrible idea to allow built-in language features to violate safe. Aside from issues related to trusted being misused, safe code should be guaranteed to be memory safe, and it should be considered a bug any time that it isn't. That's why safe exists. No one should have to be looking at safe code to track down memory safety problems. And if they do, then safe is not doing it's job. Array bounds checks are left in safe code for exactly these reasons.So, the request is to just leave assert active as a default in safe code, like the bounds checks?I'm all for introducing optimizations that do not violate safe, but if we allow safe code to be unsafe, then why do we even have it?So, the reasoning is that UB should not lead to memory corruption, right? /P
Mar 07 2018
On Wednesday, March 07, 2018 08:39:30 Paolo Invernizzi via Digitalmars-d wrote:On Tuesday, 6 March 2018 at 20:03:11 UTC, Jonathan M Davis wrote:I'm saying that whatever optimizations are enabled by assertions should be guaranteed to not include memory corruptions.On Tuesday, March 06, 2018 18:49:42 Paolo Invernizzi via Digitalmars-d wrote:Understood. Are asking that UB should not include memory corruptions?I simply don't understand why enforce or a custom check can't be used safe code, if you want that behaviour. If the program must HALT on this or that, well, what is better than an explicit piece of unremovable code that do that? Instead of writing 'assert', one should write 'enforce'.1. It's quite common for folks to want to add debugging checks that are compiled out in -release. That's exactly what assert is for in pretty much every lanugage that has it. It's what folks expect to use and what your average programmer will use without thinking about safety issues at all. It's what everyone uses right now, and I'm pretty sure that almost everyone using it has no clue that Walter considers it okay for assertions to introduce optimizations which are not memory safe, and if/when he does do so, a lot of D code will suddenly have safe code which is not memory safe. Such problems will hopefully be hit rarely, because hopefully, the code will have been well-tested, and the assertions will have found all of the related bugs, but there's every possibility that some bugs will manage to not be caught, thereby resulting in safe code being unsafe. No one is going to be looking to use solutions other than assertions for what assertions are for unless we start telling everyone to avoid assertions, because they make safe code unsafe. And honestly, if assertions make safe code unsafe, I don't see a good argument for using them at all. If I didn't care about code being safe, I wouldn't be using safe. safe is supposed to guarantee that the code is memory safe.No. I'm saying that no optimizations should be enabled which introduce potential memory corruption. Assertions should have zero impact on whether code is safe or not unless the code in the condition or which is generating the message for the assertion is system, and it's no more reasonable to assume that an assertion is going to pass than it is to assume that bounds checking won't fail. Regardless, the key thing here is that safe code should be guaranteed to be safe so long as trusted code is vetted properly. It should _never_ be possible for the compiler to introduce memory safety issues into safe code.2. I think that it's fundamentally a terrible idea to allow built-in language features to violate safe. Aside from issues related to trusted being misused, safe code should be guaranteed to be memory safe, and it should be considered a bug any time that it isn't. That's why safe exists. No one should have to be looking at safe code to track down memory safety problems. And if they do, then safe is not doing it's job. Array bounds checks are left in safe code for exactly these reasons.So, the request is to just leave assert active as a default in safe code, like the bounds checks?The reasoning is that no safe code should ever have memory corruptions in it unless it calls trusted code that was incorrectly vetted by the programmer. The compiler is supposed to guarantee that safe code is safe just like it's supposed to guarantee that a const variable isn't mutated or that a pure function doesn't access mutable, global variables. And as such, introducing anything into safe code which could be memory unsafe is a violation of the compiler's responsibility. - Jonathan M DavisI'm all for introducing optimizations that do not violate safe, but if we allow safe code to be unsafe, then why do we even have it?So, the reasoning is that UB should not lead to memory corruption, right?
Mar 07 2018
On Wednesday, 7 March 2018 at 09:11:10 UTC, Jonathan M Davis wrote:So, the request is to just leave assert active as a default in safe code, like the bounds checks?No. I'm saying that no optimizations should be enabled which introduce potential memory corruption. Assertions should have zero impact on whether code is safe or not unless the code in the condition or which is generating the message for the assertion is system, and it's no more reasonable to assume that an assertion is going to pass than it is to assume that bounds checking won't fail. Regardless, the key thing here is that safe code should be guaranteed to be safe so long as trusted code is vetted properly. It should _never_ be possible for the compiler to introduce memory safety issues into safe code.So, the reasoning is that UB should not lead to memory corruption, right?The reasoning is that no safe code should ever have memory corruptions in it unless it calls trusted code that was incorrectly vetted by the programmer. The compiler is supposed to guarantee that safe code is safe just like it's supposed to guarantee that a const variable isn't mutated or that a pure function doesn't access mutable, global variables. And as such, introducing anything into safe code which could be memory unsafe is a violation of the compiler's responsibility.Jonathan, I understand your reasoning, but it's not what I'm asking: are we asking for UB that do not lead to memory corruption? /Paolo
Mar 07 2018
On Wednesday, March 07, 2018 09:22:40 Paolo Invernizzi via Digitalmars-d wrote:On Wednesday, 7 March 2018 at 09:11:10 UTC, Jonathan M Davis wrote:I'm saying that safe code must not be violated by the compiler. Beyond that I'm not arguing about UB one way or the other. If UB must be disallowed to avoid violating safe, then it must be disallowed. If some form of UB can be allowed, because it's restricted in a manner that it can't violate safe but may do something else stupid because the assertion would have failed if it weren't compiled out, I don't care. If an assertion would have failed if it weren't compiled out, then you have a bug regardless, and if the code is buggier because of an optimization, then that's fine with me. You have a bug either way. What isn't fine is that that result violate safe, because that would defeat the entire purpose of safe and make it far, far more difficult to track down safety problems. Right now, since no optimizations like Walter has been talking about are done by the compiler, if you have memory corruption, you know that you only have to look at system and trusted code to find it, whereas with the unsafe optimizations that Walter is talking about, it then becomes possible that you're going to have to look through the entire program to find the problem. And right now, you can be sure that you don't have safety problems in safe code if you use trusted correctly, whereas with what Walter is talking about, simply adding an assertion could add safety problems to your code. - Jonathan M DavisJonathan, I understand your reasoning, but it's not what I'm asking: are we asking for UB that do not lead to memory corruption?So, the request is to just leave assert active as a default in safe code, like the bounds checks?No. I'm saying that no optimizations should be enabled which introduce potential memory corruption. Assertions should have zero impact on whether code is safe or not unless the code in the condition or which is generating the message for the assertion is system, and it's no more reasonable to assume that an assertion is going to pass than it is to assume that bounds checking won't fail. Regardless, the key thing here is that safe code should be guaranteed to be safe so long as trusted code is vetted properly. It should _never_ be possible for the compiler to introduce memory safety issues into safe code.So, the reasoning is that UB should not lead to memory corruption, right?The reasoning is that no safe code should ever have memory corruptions in it unless it calls trusted code that was incorrectly vetted by the programmer. The compiler is supposed to guarantee that safe code is safe just like it's supposed to guarantee that a const variable isn't mutated or that a pure function doesn't access mutable, global variables. And as such, introducing anything into safe code which could be memory unsafe is a violation of the compiler's responsibility.
Mar 07 2018
On Wednesday, 7 March 2018 at 11:52:05 UTC, Jonathan M Davis wrote:On Wednesday, March 07, 2018 09:22:40 Paolo Invernizzi via Digitalmars-d wrote:And that's clear.On Wednesday, 7 March 2018 at 09:11:10 UTC, Jonathan M Davis wrote:I'm saying that safe code must not be violated by the compiler. Beyond that I'm not arguing about UB one way or the other.Jonathan, I understand your reasoning, but it's not what I'm asking: are we asking for UB that do not lead to memory corruption?So, the request is to just leave assert active as a default in safe code, like the bounds checks?No. I'm saying that no optimizations should be enabled which introduce potential memory corruption. Assertions should have zero impact on whether code is safe or not unless the code in the condition or which is generating the message for the assertion is system, and it's no more reasonable to assume that an assertion is going to pass than it is to assume that bounds checking won't fail. Regardless, the key thing here is that safe code should be guaranteed to be safe so long as trusted code is vetted properly. It should _never_ be possible for the compiler to introduce memory safety issues into safe code.So, the reasoning is that UB should not lead to memory corruption, right?The reasoning is that no safe code should ever have memory corruptions in it unless it calls trusted code that was incorrectly vetted by the programmer. The compiler is supposed to guarantee that safe code is safe just like it's supposed to guarantee that a const variable isn't mutated or that a pure function doesn't access mutable, global variables. And as such, introducing anything into safe code which could be memory unsafe is a violation of the compiler's responsibility.If UB must be disallowed to avoid violating safe, then it must be disallowed.And how to do this, in practise I mean?If some form of UB can be allowed, because it's restricted in a manner that it can't violate safe but may do something else stupid because the assertion would have failed if it weren't compiled out, I don't care.And that's the original question: are we asking for UB that do not lead to memory corruption?If an assertion would have failed if it weren't compiled out, then you have a bug regardless, and if the code is buggier because of an optimization, then that's fine with me. You have a bug either way.Agreed.What isn't fine is that that result violate safe, because that would defeat the entire purpose of safe and make it far, far more difficult to track down safety problems.So, see above, the original question, agreed.Right now, since no optimizations like Walter has been talking about are done by the compiler, if you have memory corruption, you know that you only have to look at system and trusted code to find it, whereas with the unsafe optimizations that Walter is talking about, it then becomes possible that you're going to have to look through the entire program to find the problem.Or you can just turn on assertion, right? If we have corrupted memory in release, there's a bug, somewhere, in the logic or in the implementation of the logic. As you have told, we must audit system and trusted, we can imagine to use static checkers or some strange beast like that. But, while doing that, I think that the most common practise is keep running the code with assertion on, do you agree?And right now, you can be sure that you don't have safety problems in safe code if you use trusted correctly, whereas with what Walter is talking about, simply adding an assertion could add safety problems to your code.Nope, not adding an assertion, but having the process in UB state. And we are back again to the original question. /Paolo
Mar 07 2018
On Wednesday, March 07, 2018 13:24:19 Paolo Invernizzi via Digitalmars-d wrote:That would make assertions a lot worse to use, because then they would be in production code slowing it down. Also, as it stands, -release is not supposed to violate safe. To do that, you have to use -boundscheck=off to turn off bounsd checking. That was a very purposeful design decision, because we did not want -release to violate safe, and if the compiler is allowed to add optimizations which are unsafe based on assertions, then that completely destroys the ability to have safe code with -release. And if we were going to do that, why did we leave array bounds checking on with -release? Assertions are to help debug code, and most people disable them in production with zero expectation that that's going to result in their safe code suddenly becoming unsafe. It's a huge change if -release makes code unsafe, and IMHO, doing so would make assertions an immediate code smell and that assertions should then never be used except by folks who are willing to leave them in all the time. I don't see how it can be argued that allowing assertions to introduce unsafe behavior into safe code is not in complete violation of what safe is supposed to do and guarantee. It's already bad enough that we talk about how memory safe D code is when safe isn't the default, but to then completely bypass safe like this seems unconscionable to me. That would be like deciding that we're now going to introduce cast_cast and mutable into the language and allow const's guarantees to be violated whenever the programmer feels like it. safe needs to actually be guaranteed to be safe or it's worthless. - Jonathan M DavisRight now, since no optimizations like Walter has been talking about are done by the compiler, if you have memory corruption, you know that you only have to look at system and trusted code to find it, whereas with the unsafe optimizations that Walter is talking about, it then becomes possible that you're going to have to look through the entire program to find the problem.Or you can just turn on assertion, right? If we have corrupted memory in release, there's a bug, somewhere, in the logic or in the implementation of the logic. As you have told, we must audit system and trusted, we can imagine to use static checkers or some strange beast like that. But, while doing that, I think that the most common practise is keep running the code with assertion on, do you agree?
Mar 07 2018
On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis wrote:On Wednesday, March 07, 2018 13:24:19 Paolo Invernizzi via Digitalmars-d wrote:Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /Paolo[...]That would make assertions a lot worse to use, because then they would be in production code slowing it down. Also, as it stands, -release is not supposed to violate safe. To do that, you have to use -boundscheck=off to turn off bounsd checking. That was a very purposeful design decision, because we did not want -release to violate safe, and if the compiler is allowed to add optimizations which are unsafe based on assertions, then that completely destroys the ability to have safe code with -release. And if we were going to do that, why did we leave array bounds checking on with -release? [...]
Mar 07 2018
On Wednesday, March 07, 2018 14:08:35 Paolo Invernizzi via Digitalmars-d wrote:On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis wrote:safe is all about guaranteeing memory safety. That's it's entire job. No more, no less. What happens with UB beyond that is irrelevant. If satisfying the requirement that safe code be memory safe means that UB cannot be allowed in safe code, then UB cannot be allowed in safe code. If there is some form of UB that is constrained enough that it's guaranteed that it can't violate memory safety, then I don't see any reason why it can't be in safe code any more than it can't be in system code, because it's not violating the guarantees that safe is intended to provide - that the code is memory safe. Other language rules may make UB illegal or explicitly allow it for one reason or another (e.g. it's supposed to be guaranteed that function arguments are evaluated left-to-right, though I'm not sure if that's ever been implemented like it's supposed to be), but in the case of safe, it's all about what's memory safe. And what is or isn't allowed with regards to UB in safe therefore has to be a function of what is required to guarantee that the code is memory safe. - Jonathan M DavisOn Wednesday, March 07, 2018 13:24:19 Paolo Invernizzi via Digitalmars-d wrote:Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions?[...]That would make assertions a lot worse to use, because then they would be in production code slowing it down. Also, as it stands, -release is not supposed to violate safe. To do that, you have to use -boundscheck=off to turn off bounsd checking. That was a very purposeful design decision, because we did not want -release to violate safe, and if the compiler is allowed to add optimizations which are unsafe based on assertions, then that completely destroys the ability to have safe code with -release. And if we were going to do that, why did we leave array bounds checking on with -release? [...]
Mar 07 2018
On 07.03.2018 15:08, Paolo Invernizzi wrote:On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis wrote:UB is unconstrained by definition. If it is constrained, it is not UB.On Wednesday, March 07, 2018 13:24:19 Paolo Invernizzi via Digitalmars-d wrote:Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /Paolo[...]That would make assertions a lot worse to use, because then they would be in production code slowing it down. Also, as it stands, -release is not supposed to violate safe. To do that, you have to use -boundscheck=off to turn off bounsd checking. That was a very purposeful design decision, because we did not want -release to violate safe, and if the compiler is allowed to add optimizations which are unsafe based on assertions, then that completely destroys the ability to have safe code with -release. And if we were going to do that, why did we leave array bounds checking on with -release? [...]
Mar 07 2018
On Wednesday, 7 March 2018 at 15:26:01 UTC, Timon Gehr wrote:On 07.03.2018 15:08, Paolo Invernizzi wrote:That! Thanks! So, safe code is code where UB should not be possible? Is it pragmatically possible to reach that goal? /PaoloOn Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis wrote:UB is unconstrained by definition. If it is constrained, it is not UB.[...]Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /Paolo
Mar 07 2018
On 07.03.2018 16:30, Paolo Invernizzi wrote:On Wednesday, 7 March 2018 at 15:26:01 UTC, Timon Gehr wrote:Yes. (If you have UB, memory corruption is one of the allowed outcomes, but safe should not allow memory corruption. So safe needs to at least ban UB. According to Walter safe needs to ban memory corruption but not more. Therefore, as memory corruption leads to UB (it is impractical to specify anything else, at least for writes), safe bans UB and nothing else.) Also note that even in system code I don't want asserts to cause UB in release. There should be an system facility for this purpose. (The situation is different than with bounds checks: if bounds checks fail, there will always be a bad memory access, which is UB, but with asserts it's always possible that the assert itself was wrong and the code itself will not trigger UB.)On 07.03.2018 15:08, Paolo Invernizzi wrote:That! Thanks! So, safe code is code where UB should not be possible?On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis wrote:UB is unconstrained by definition. If it is constrained, it is not UB.[...]Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /PaoloIs it pragmatically possible to reach that goal? /PaoloYes. (Languages can be type safe and type systems can be arbitrarily powerful.) It is certainly possible to _aim_ for that goal, which Walter and Andrei have done on other occasions.
Mar 07 2018
On Wednesday, 7 March 2018 at 16:04:50 UTC, Timon Gehr wrote:On 07.03.2018 16:30, Paolo Invernizzi wrote:Thanks Timon, now everything it's definitely more clear to me on that matter. I think a good compromise is to just totally ignore assert in release during optimization as a default, and use a compiler flag to let the optimiser peek to them, if a user want to explore that potential gain. Just like '-boundscheck'. Thanks to all for having clarified that point to me (and hopefully to others also!) /PaoloOn Wednesday, 7 March 2018 at 15:26:01 UTC, Timon Gehr wrote:Yes. (If you have UB, memory corruption is one of the allowed outcomes, but safe should not allow memory corruption. So safe needs to at least ban UB. According to Walter safe needs to ban memory corruption but not more. Therefore, as memory corruption leads to UB (it is impractical to specify anything else, at least for writes), safe bans UB and nothing else.) Also note that even in system code I don't want asserts to cause UB in release. There should be an system facility for this purpose. (The situation is different than with bounds checks: if bounds checks fail, there will always be a bad memory access, which is UB, but with asserts it's always possible that the assert itself was wrong and the code itself will not trigger UB.)On 07.03.2018 15:08, Paolo Invernizzi wrote:That! Thanks! So, safe code is code where UB should not be possible?On Wednesday, 7 March 2018 at 13:55:11 UTC, Jonathan M Davis wrote:UB is unconstrained by definition. If it is constrained, it is not UB.[...]Jonathan, I understand your point, but still I can't find an answer to clarify my doubts. Are we asking for no UB in safe code? Are we asking for UB in safe code but constrained to no memory corruptions? /PaoloIs it pragmatically possible to reach that goal? /PaoloYes. (Languages can be type safe and type systems can be arbitrarily powerful.) It is certainly possible to _aim_ for that goal, which Walter and Andrei have done on other occasions.
Mar 07 2018
On 06.03.2018 19:49, Paolo Invernizzi wrote:I simply don't understand why enforce or a custom check can't be used safe code, if you want that behaviour. ...I have explained why. UB is non-modular and you don't (want to) control all the code that you use. Also, enforce cannot be disabled. And no, keeping the check or introducing UB are not the only sensible options.
Mar 06 2018
On Wednesday, 7 March 2018 at 00:11:33 UTC, Timon Gehr wrote:On 06.03.2018 19:49, Paolo Invernizzi wrote:As I've written, you should ask for debug versions of every closed source library that it's used in a project, it's always a win-only things to do.I simply don't understand why enforce or a custom check can't be used safe code, if you want that behaviour. ...I have explained why. UB is non-modular and you don't (want to) control all the code that you use.Also, enforce cannot be disabled.Because if you use it instead of assert, the enforce has just detected a bug, so it's just doing the sane thing to join that cases: stop the execution because the process is entering UB.And no, keeping the check or introducing UB are not the only sensible options.Just to understand, as I've already asked: do we aim at UB that guarantees no memory corruptions? /Paolo
Mar 07 2018
On Monday, March 05, 2018 02:30:12 Walter Bright via Digitalmars-d wrote:The idea behind removal of the runtime checks is as a performance optimization done on a debugged program. It's like turning on or off array bounds checking. Many leave asserts and array bounds checking on even in released code to ensure memory safety. At a minimum, turning it off and on will illuminate just what the checks are costing you. It's at the option of the programmer.Except that disabling array bounds checking negates the guarantees of safe, and you have to go to the extra effort of supplying a compiler flag to tell the compiler that you don't care about the compiler guaranteeing safe. And your suggesting that with assertions, safe's guarantees can be removed. You never catch everything when debugging or running your program prior to releasing it. Assertions can help considerably in catching problems, but there is zero guarantee that that they've caught all of the problems. They're disabled in release mode on the theory that the program has been tested enough that that assertions are unlikely to fail, but they could still have failed if they remained. It's one thing to remove the assertions and let your program behave badly as a result whenever you didn't catch all of the problems. It's quite another to have the compiler make your code safe code system on the assumption that the assertion would have passed. Personally, I don't mind if the compiler is able to make optimization decisions based on assertions - the code is buggy anyway if the assertion would have failed - but I very much mind if those optimizations introduce behavior that could violate safe. Because then, instead of just having a bug, you have memory corruption problem, which could be really hard to detect and track down, and it would be essentially invisible, because most programmers who understand safe and trusted, would assume that they needed to check the system code and trusted code for the problem, when it could now be sitting in safe code just because one of the developers added an assertion to help catch a bug during testing. If assertions make it possible for safe code to actually be system, that seems like a serious bug to me. IIRC, Andrei made a huge deal with you several years ago about how array bounds checking needed to be kept in safe code or safe meant nothing. I don't see how this situation is any different. safe code needs to guaranteed to be safe so long as trusted is used correctly, otherwise, safe is arguably meaningless - certainly, it then doesn't restrict the code that you have to look through for memory safety problems, and that's one of the main purposes of safe. If you want to introduce a compiler flag like -boundscheck=off that makes it so that compiling out assertions can introduce system behavior, then fine. But removing assertions with -release and removing safety guarantees are two completely separate issues. I sure don't want assertions in my production builds; I'd be forced to use them a lot less if they were going to be in my production builds. But I also don't want to introduce safety issues into my production builds by using assertions. You keep talking about how important safety is to D and how much better it makes it than C, and then you go and talk about invisibly removing safety guarantees with common, built-in features like assertions. If assertions removed safety guaranatees, I would be forced to never use them in my code, which IMHO is ridiculous and is certainly going to make my code worse. And IMHO, suggesting that someone should then use a user-defined assertion isn't a good solution either, because then the built-in default that everyone is going to use unless someone more knowledgable tells them not to is an safety problem. If you want to be able to optimize based on assertions then fine, but please, please, please do not allow the compiler to introduce safety problems when adding optimizations - preferably not even in system code, because realistically, almost no one is going to have a clue what safety problems could be introduced by an optimization based on an assertion, making it impossible for the programmer to verify that system code can be marked trusted. And most programmers won't know that assertions could even introduce optimizations that weren't memory safe. I honestly don't see how we can claim that safe code is guaranteed to be memory safe if assertions - or any other D feature other than the programmer using trusted incorrectly - could introduce anything that is not guaranteed to be memory safe. - Jonathan M Davis
Mar 06 2018
On 3/6/2018 1:58 AM, Jonathan M Davis wrote:[...]The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code. It's been like that for D since day 1. It has always been documented to do that. It has been discussed many times in this n.g. over the years with loooong threads. I designed it that way so that D could potentially produce better code than other languages in ways they could not match. There is no other purpose to making it a core language feature. It's fine if you want an assert-like feature to have other semantics - just define one called 'check', give it the semantics you want, and put it in the library. As I mentioned to Timon, support for that sort of thing is why D has special support for __LINE__ and __FILE__. Or just don't turn off assert checking. Personally, I use asserts in a way that they add little overhead so they can remain active in the release build. It's entirely under your control.
Mar 06 2018
On Tuesday, March 06, 2018 02:17:42 Walter Bright via Digitalmars-d wrote:On 3/6/2018 1:58 AM, Jonathan M Davis wrote:I don't mind if assertions allow the compiler to better optimize. What I object to quite strongly is about the compiler being allowed to introduce optimizations that violate safety guarantees. Built-in features should not result in safe code invisibly becoming system. That violates the entire purpose of safe. I don't see how this situation is any different from the array bounds checks, and those aren't compiled out in safe code - even when assertions are compiled out. - Jonathan M Davis[...]The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code. It's been like that for D since day 1. It has always been documented to do that. It has been discussed many times in this n.g. over the years with loooong threads. I designed it that way so that D could potentially produce better code than other languages in ways they could not match. There is no other purpose to making it a core language feature. It's fine if you want an assert-like feature to have other semantics - just define one called 'check', give it the semantics you want, and put it in the library. As I mentioned to Timon, support for that sort of thing is why D has special support for __LINE__ and __FILE__. Or just don't turn off assert checking. Personally, I use asserts in a way that they add little overhead so they can remain active in the release build. It's entirely under your control.
Mar 06 2018
On 06.03.2018 11:17, Walter Bright wrote:On 3/6/2018 1:58 AM, Jonathan M Davis wrote:It does not need to be a core language feature for that. It seems that you did the right thing for the wrong reasons.[...]The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code.It's been like that for D since day 1. It has always been documented to do that. It has been discussed many times in this n.g. over the years with loooong threads. I designed it that way so that D could potentially produce better code than other languages in ways they could not match. There is no other purpose to making it a core language feature. ...Yes. The (only!) purpose is standardization. asserts and contracts are a core language feature for similar reasons why unit tests are a core language feature.... Or just don't turn off assert checking. Personally, I use asserts in a way that they add little overhead so they can remain active in the release build. ...Not everybody runs only their own code.It's entirely under your control.It is not. Could we please make it that way?
Mar 06 2018
On Tuesday, 6 March 2018 at 10:17:42 UTC, Walter Bright wrote:On 3/6/2018 1:58 AM, Jonathan M Davis wrote:As a side node, we have a `verify()` exactly for this: https://github.com/sociomantic-tsunami/ocean/blob/v3.x.x/src/ocean/core/Verify.d (and because in some circumstances we need to be able to catch programming errors in servers that are low risk to catch and just ignore the current request and continue with our lives, with asserts we can't do that because they throw an Error).[...]The entire point of making assert a core language feature was so that the compiler could take advantage of it to generate better code. It's been like that for D since day 1. It has always been documented to do that. It has been discussed many times in this n.g. over the years with loooong threads. I designed it that way so that D could potentially produce better code than other languages in ways they could not match. There is no other purpose to making it a core language feature. It's fine if you want an assert-like feature to have other semantics - just define one called 'check', give it the semantics you want, and put it in the library. As I mentioned to Timon, support for that sort of thing is why D has special support for __LINE__ and __FILE__.
Jun 12 2018
On 05.03.2018 11:25, Walter Bright wrote:On 3/4/2018 3:06 PM, Timon Gehr wrote:I guess I misunderstood what you meant when you said "as if". I thought you meant that for all languages you know, when assertions are disabled, the compiler behaves "as if" the check was actually there and was known to succeed, even though the check is actually not there and may have failed if it was.On 04.03.2018 22:49, Walter Bright wrote:Even in Java, the compiler generates code that, from the user's point of view, behaves "as if" the code was actually what was specified. For a trivial example, replacing x*2 with x<<1. Not having this means no optimizations can be done. ...Impossible. You wrote a Java compiler.Not necessarily. If the code contains an explicit assertion that the index is in bounds, then, according to the language specification, the bounds check may be removed with -release.D, as all languages that I know of do implicitly or explicitly, generates code based on the "as if" rule. ...My question is not why it is the case technically, I was asking for a _rationale_ for this apparently silly behavior. I.e., why is this a good idea from the point of view of language design? Again: assert is safe. Compiler hints are system. Why should assert give compiler hints?All languages that use your "as if" rule are memory unsafe. Zero languages that use the "as if" rule have any memory safe subset that includes assertions. In D, assert is safe, and it should remain safe.Because the checks provide extra information to the compiler that it can use to generate better code. If that extra information is not true, then the better code will be invalid. ...Why is potential memory corruption to be expected when using safe language features with a flag to disable contract checks?I find the reasoning in terms of "on"/"off" confusing anyway. Does "off" mean "contract/assertion removed", or does it mean "failure is UB"?"Off" means the check is removed. If the check does not hold, the program enters an invalid state, whether or not the check was actually done. An invalid state means subsequent execution is UB.Memory safety is only one class of errors in a program. If the program has entered a state that is not accounted for by the programmer, the rest of the program's execution will be not predictable. ...But the whole point of having memory safety is to not have UB when the programmer screwed up. Behavior not foreseen by the programmer (a bug) is not the same as behavior unconstrained by the language specification (UB).'in'-contracts catch AssertError when being composed. How can the language not be designed to support that? Except for this case, the assertion is not _supposed_ to fail for my use cases, and I don't really need the language to explicitly "support that use case". The situation is the following: - I usually don't want UB in programs I am working on. I want the runtime behavior of the programs to be determined by the source code, such that every behavior observed in the wild (intended or unintended) can be traced back to the source code (potentially in a non-deterministic way, e.g. void initialization of an integer constant). This should be the case always, even if me or someone else on my team made a mistake. The safe D subset is supposed to give this guarantee. What good is safe if it does not guarantee absence of buffer overrun attacks? - Checking assertions can be too costly, so it should be possible to disable the check. - Using existing assertions as compiler hints is not necessary. (Without having checked it, I'm sure that LDC/GDC have a more suitable intrinsic for this already.) As far as I can discern, forcing disabled asserts to give compiler hints has no upsides.This makes no sense. This is not useful behavior. There are convenient ways to support potentially unsound compilation hints that do not do this. Contracts and compilation hints should be orthogonal. Contracts should be potentially safe, compilation hints should be system always. Note that _actual removal_ is the only use case of 'disabling contracts' that I care about, and I think many D programmers who use "off" will also have this behavior in mind. Yet this is not even an option.I don't see much use for this behavior, unless you want to continue running the program after an assert failure, which I cannot recommend and the language is not designed to support.But you can always do something like: version (ignore_asserts) { } else { assert(...); } ...I know. Actually version(assert) assert(...); also works. However, this is too verbose, especially in contracts. I'd like a solution that does not require me to change the source code. Ideally, I just want the Java behavior (with reversed defaults).which would optionally remove both the runtime check and any compiler use of the assert. Or you could use https://dlang.org/library/std/exception/enforce.html which has no influence on compiler semantics. ...(enforce is _completely unrelated_ to the current discussion.) Contracts are designed to be used with built-in assert. Built-in assert is convenient. If the point of 'assert' is to give compiler hints (for me this is not even a use case of assert at all), then it should be system.This is not really about assert semantics, this is about the semantics of "disabling the check". There was no "-check=off" flag before. The DIP uses terminology such as "disable assertions" as opposed to "disable assertion checks (but introduce compiler hints)". Contracts are supposed to increase safety, not to reduce it (ask Bertrand Meyer).At the very least, the DIP should be up-front about this. I'm still not even sure that Mathias Lang intended the UB semantics.It being UB was my doing, not Mathias'. DIP1006 is not redefining the semantics of what assert does.
Mar 05 2018
On Monday, 5 March 2018 at 15:48:12 UTC, Timon Gehr wrote:- Using existing assertions as compiler hints is not necessary. (Without having checked it, I'm sure that LDC/GDC have a more suitable intrinsic for this already.) As far as I can discern, forcing disabled asserts to give compiler hints has no upsides.In the simple cases, or in anything that looks like a unittest/testsuite, probably not. There are likely going to be more aggressive optimizations however if CFA can see that a variable will never be outside a given range, i.e: --- int[5] arr; if (len < 0 || len >= 5) { unreachable(); // in non-release code, this would throw a RangeError. } return arr[len]; --- From this, we aggressively assume that len is a valid index of arr. Something that happens in optimized non-release builds, but in release builds we must accommodate for the possibility of a range error.
Mar 05 2018
On 05.03.2018 20:41, Iain Buclaw wrote:On Monday, 5 March 2018 at 15:48:12 UTC, Timon Gehr wrote:(Note that by "forcing", I mean withholding other options from the user. I'm not saying that using information from asserts can never be useful, just that it can just as well be harmful, and therefore it is unwise to not allow disabling them. I was saying that there are no upsides to not having a flag that actually removes assertions.)- Using existing assertions as compiler hints is not necessary. (Without having checked it, I'm sure that LDC/GDC have a more suitable intrinsic for this already.) As far as I can discern, forcing disabled asserts to give compiler hints has no upsides.In the simple cases, or in anything that looks like a unittest/testsuite, probably not. There are likely going to be more aggressive optimizations however if CFA can see that a variable will never be outside a given range, i.e: ...--- int[5] arr; if (len < 0 || len >= 5) { unreachable(); // in non-release code, this would throw a RangeError. } return arr[len]; --- From this, we aggressively assume that len is a valid index of arr. Something that happens in optimized non-release builds, but in release builds we must accommodate for the possibility of a range error.I think this particular case is a bit less questionable than doing the same for general assertions (for instance, in safe code, -release will not actually remove the bounds check unless there is some relevant assertion somewhere). In any case, I don't argue strongly against a flag that turns all assertions into compiler hints, I just think there should also be a flag that disables them safely. Also, maybe -release should commit to either disregarding safe completely or respecting it completely.
Mar 05 2018
On 3/5/2018 7:48 AM, Timon Gehr wrote:Again: assert is safe. Compiler hints are system. Why should assert give compiler hints?Asserts give expressions that must be true. Why not take advantage of them? See https://en.wikipedia.org/wiki/Spec_Sharp Some possible optimizations based on this are: 1. elimination of array bounds checks 2. elimination of null pointer checks 3. by knowing a variable can take on a limited range of values, a cheaper data type can be substituted 4. elimination of checks for 'default' switch values 5. elimination of overflow checks dmd's optimizer currently does not extract any information from assert's. But why shut the door on that possibility?But the whole point of having memory safety is to not have UB when the programmer screwed up. Behavior not foreseen by the programmer (a bug) is not the same as behavior unconstrained by the language specification (UB).It's the programmer's option to leave those runtime checks in if he wants to.'in'-contracts catch AssertError when being composed. How can the language not be designed to support that?That is indeed an issue. It's been proposed that in-contracts throw a different exception, say "ContractException" so that it is not UB when they fail. There's a bugzilla ER on this. (It's analogous to asserts in unittests not having UB after they fail.)- I usually don't want UB in programs I am working on. I want the runtime behavior of the programs to be determined by the source code, such that every behavior observed in the wild (intended or unintended) can be traced back to the source code (potentially in a non-deterministic way, e.g. void initialization of an integer constant). This should be the case always, even if me or someone else on my team made a mistake. The safe D subset is supposed to give this guarantee. What good is safe if it does not guarantee absence of buffer overrun attacks?It guarantees it at the option of the programmer via a command line switch.- Using existing assertions as compiler hints is not necessary. (Without having checked it, I'm sure that LDC/GDC have a more suitable intrinsic for this already.) As far as I can discern, forcing disabled asserts to give compiler hints has no upsides.I suspect that if: compiler_hint(i < 10); were added, there would be nothing but confusion as to its correct usage vs assert vs enforce. There's already enough confusion about the latter two. In fact, I can pretty much guarantee it will be rarely used correctly.I know. Actually version(assert) assert(...); also works. However, this is too verbose, especially in contracts.You can wrap it in a template.I'd like a solution that does not require me to change the source code. Ideally, I just want the Java behavior (with reversed defaults).But you'll have to change the code to compiler_hint().(enforce is _completely unrelated_ to the current discussion.)It does just what you ask (for the regular assert case).It is very much about the semantics of assert.It being UB was my doing, not Mathias'. DIP1006 is not redefining the semantics of what assert does.This is not really about assert semantics, this is about the semantics of "disabling the check".There was no "-check=off" flag before.Yes there was, it's the "release" flag.The DIP uses terminology such as "disable assertions" as opposed to "disable assertion checks (but introduce compiler hints)".Yes, the language could be more precise, but I couldn't blame Mathias for that. I also disagree with the word "hint", because it implies things like "this branch is more likely to be taken" to guide code generation decisions, rather than "assume X is absolutely always incontrovertibly true and you can bet the code on it".
Mar 05 2018
On 03/05/2018 09:55 PM, Walter Bright wrote:On 3/5/2018 7:48 AM, Timon Gehr wrote:Because it's exactly what safe is not supposed to do. You're trusting the programmer to get their asserts right. Trusting the programmer to get it right is system. [...]> It's the programmer's option to leave those runtime checks in if heAgain: assert is safe. Compiler hints are system. Why should assert give compiler hints?Asserts give expressions that must be true. Why not take advantage of them?wants to.As far as I understand, Timon only asks for a third option: to simply compile the code as if the asserts weren't there, without assuming that they would pass. That way you get a speedup from the omitted asserts, but you don't get UB from a mistaken assert. This is not an unreasonable thing to want, is it? You say that DMD does not currently use assert information, so -release currently does this. [...]But the controversial aspect is not implemented. And it will be very surprising if you ever do implement it. I'm actually pretty shocked that -release is described that way. It makes a point of keeping bounds checks in safe code. The reason is that it would be unsafe to remove them. What's the point of that when safety is compromised anyway by assuming that asserts would pass?There was no "-check=off" flag before.Yes there was, it's the "release" flag.
Mar 05 2018
On 05.03.2018 21:55, Walter Bright wrote:On 3/5/2018 7:48 AM, Timon Gehr wrote:"Trust the programmer" does not always scale.Again: assert is safe. Compiler hints are system. Why should assert give compiler hints?Asserts give expressions that must be true.Why not take advantage of them?For some use cases it might be fine, but not for others, because you can't know whether the program and the assertions are really consistent. Basically, I think the flags should be: -check-{assert,invariant,precondition,postcondition,...}={on,off,assume} E.g.: generation Then the spec says that "assume" is potentially dangerous and can break safe-ty guarantees, like -boundscheck=off.https://en.wikipedia.org/wiki/Spec_Sharp ...conditions actually hold. Also, it is type safe. (I.e. no UB.)Some possible optimizations based on this are: 1. elimination of array bounds checks 2. elimination of null pointer checks 3. by knowing a variable can take on a limited range of values, a cheaper data type can be substituted 4. elimination of checks for 'default' switch values 5. elimination of overflow checks dmd's optimizer currently does not extract any information from assert's. But why shut the door on that possibility? ...We should not do that, and it is not what I am arguing for. Sorry if that did not come across clearly.My point is that either leaving them in or turning failures into UB are too few options. Also, safe is a bit of a joke if there is no way to _disable contracts_ without nullifying the guarantees it's supposed to give.But the whole point of having memory safety is to not have UB when the programmer screwed up. Behavior not foreseen by the programmer (a bug) is not the same as behavior unconstrained by the language specification (UB).It's the programmer's option to leave those runtime checks in if he wants to. ...This is ugly, but I don't think there is a better solution.'in'-contracts catch AssertError when being composed. How can the language not be designed to support that?That is indeed an issue. It's been proposed that in-contracts throw a different exception, say "ContractException" so that it is not UB when they fail. There's a bugzilla ER on this. (It's analogous to asserts in unittests not having UB after they fail.) ...You mean, leave in checks?- I usually don't want UB in programs I am working on. I want the runtime behavior of the programs to be determined by the source code, such that every behavior observed in the wild (intended or unintended) can be traced back to the source code (potentially in a non-deterministic way, e.g. void initialization of an integer constant). This should be the case always, even if me or someone else on my team made a mistake. The safe D subset is supposed to give this guarantee. What good is safe if it does not guarantee absence of buffer overrun attacks?It guarantees it at the option of the programmer via a command line switch. ...I have never understood why. The use cases of assert and enforce are disjoint.- Using existing assertions as compiler hints is not necessary. (Without having checked it, I'm sure that LDC/GDC have a more suitable intrinsic for this already.) As far as I can discern, forcing disabled asserts to give compiler hints has no upsides.I suspect that if: compiler_hint(i < 10); were added, there would be nothing but confusion as to its correct usage vs assert vs enforce. There's already enough confusion about the latter two.In fact, I can pretty much guarantee it will be rarely used correctly. ...Me too, but that's mostly because it will be rarely used.That won't work for in contracts if they start catching ContractException instead of AssertError. Also, I think we'd actually like to _shorten_ the contract syntax (there is another DIP on this). For other uses, a function suffices, but I ideally want to keep using standard 'assert'. Everybody already knows what 'assert' means.I know. Actually version(assert) assert(...); also works. However, this is too verbose, especially in contracts.You can wrap it in a template. ...I don't, because I don't want that behavior. Others who want that behavior also should not have to. This should be a compilation switch.I'd like a solution that does not require me to change the source code. Ideally, I just want the Java behavior (with reversed defaults).But you'll have to change the code to compiler_hint(). ...No. "enforce" does not document that the programmer thinks that the condition will never fail. It's not a contract. Hence, enforcements are never removed from the executable, because it would not make sense to do so, so they definitely do not fit my requirements.(enforce is _completely unrelated_ to the current discussion.)It does just what you ask (for the regular assert case). ...Depends on what you mean by "off". :o) In my book, "-release" is "-check-assert=assume -boundscheck=safeonly". A very strange combination!It is very much about the semantics of assert.It being UB was my doing, not Mathias'. DIP1006 is not redefining the semantics of what assert does.This is not really about assert semantics, this is about the semantics of "disabling the check".There was no "-check=off" flag before.Yes there was, it's the "release" flag. ...I don't understand why that would be necessary. The point of the preliminary DIP review is not to blame the author for the DIP's shortcomings, it's to collect feedback on the DIP to make it better, ideally in an interactive way. It is also very well possible that Mathias was simply unaware of the UB behavior with -release.The DIP uses terminology such as "disable assertions" as opposed to "disable assertion checks (but introduce compiler hints)".Yes, the language could be more precise, but I couldn't blame Mathias for that.I also disagree with the word "hint", because it implies things like "this branch is more likely to be taken" to guide code generation decisions, rather than "assume X is absolutely always incontrovertibly true and you can bet the code on it".Makes sense. I'll call them "assumptions" from now on.
Mar 05 2018
On Sunday, 4 March 2018 at 12:05:08 UTC, rjframe wrote:E.g., "dmd -check=in a.d" is unnecessary; it's equivalent to "dmd a.d" "dmd -release -check=in a.d" turns off all checks except in contracts.I think, -check should specify hierarchic modes like in the DIP, that would work independently of -release switch (override).
Mar 04 2018