www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - assume, assert, enforce, safe

reply Walter Bright <newshound2 digitalmars.com> writes:
I'd like to sum up my position and intent on all this.

1. I can discern no useful, practical difference between the notions of assume 
and assert.

2. The compiler can make use of assert expressions to improve optimization,
even 
in -release mode.

3. Use of assert to validate input is utterly wrong and will not be supported. 
Use such constructs at your own risk.

4. An assert failure is a non-recoverable error. The compiler may assume that 
execution does not proceed after one is tripped. The language does allow 
attempts to shut a program down gracefully after one is tripped, but that must 
not be misconstrued as assuming that the program is in a valid state at that
point.

5. assert(0); is equivalent to a halt, and the compiler won't remove it.

6. enforce() is meant to check for input errors (environmental errors are 
considered input).

7. using enforce() to check for program bugs is utterly wrong. enforce() is a 
library creation, the core language does not recognize it.

8.  safe is a guarantee of memory safety. It is not a guarantee that a program 
passes all its assert expressions. -release does not disable  safe.

9. -noboundscheck does disable  safe's array bounds checks, however, the 
compiler may assume that the array index is within bounds after use, even 
without the array bounds check.


I am not terribly good at writing formal legalese specifications for this. I 
welcome PR's to improve the specification along these lines, if you find any 
Aha! Gotcha! issues in it. Of course, implementation errors for this in DMD 
should be reported on bugzilla.
Jul 30 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/30/14, 3:01 PM, Walter Bright wrote:

 7. using enforce() to check for program bugs is utterly wrong. enforce()
 is a library creation, the core language does not recognize it.


GAAAAAA!
Jul 30 2014
prev sibling next sibling parent reply Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 31/07/14 00:01, Walter Bright via Digitalmars-d wrote:

 3. Use of assert to validate input is utterly wrong and will not be supported.
 Use such constructs at your own risk.

 ...

 6. enforce() is meant to check for input errors (environmental errors are
 considered input).

 7. using enforce() to check for program bugs is utterly wrong. enforce() is a
 library creation, the core language does not recognize it.


A question on that.

There are various places in Phobos where enforce() statements are used to 
validate function input or class constructor parameters.  For example, in 
std.random.LinearCongruentialEngine, we have:

     void seed(UIntType x0 = 1)  safe pure
     {
         static if (c == 0)
         {
             enforce(x0, "Invalid (zero) seed for "
                     ~ LinearCongruentialEngine.stringof);
         }
         _x = modulus ? (x0 % modulus) : x0;
         popFront();
     }

while in RandomSample we have this method which is called inside the
constructor:

     private void initialize(size_t howMany, size_t total)
     {
         _available = total;
         _toSelect = howMany;
         enforce(_toSelect <= _available,
                 text("RandomSample: cannot sample ", _toSelect,
                      " items when only ", _available, " are available"));
         static if (hasLength!Range)
         {
             enforce(_available <= _input.length,
                     text("RandomSample: specified ", _available,
                          " items as available when input contains only ",
                          _input.length));
         }
     }

The point is that using these enforce() statements means that these methods 
cannot be nothrow, which doesn't seem particularly nice if it can be avoided. 
Now, on the one hand, one could say that, quite obviously, these methods cannot 
control their input.  But on the other hand, it's reasonable to say that these 
methods' input can and should never be anything other than 100% controlled by 
the programmer.

My take is that, for this reason, these should be asserts and not enforce() 
statements.  What are your thoughts on the matter?
Jul 30 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/30/14, 3:39 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:

 On 31/07/14 00:01, Walter Bright via Digitalmars-d wrote:

 7. using enforce() to check for program bugs is utterly wrong.
 enforce() is a
 library creation, the core language does not recognize it.


 A question on that.

 There are various places in Phobos where enforce() statements are used
 to validate function input or class constructor parameters.


Yah, Phobos is a bit inconsistent about that. TDPL discusses the matter: 
if a library is deployed in separation from the program(s) it serves, it 
may as well handle arguments as "input". That's what e.g. the Windows 
API is doing - it consistently considers all function arguments 
"inputs", scrubs them, and returns error codes for all invalid inputs it 
detects. In contracts, the traditional libc/Unix interface does little 
checking, even a strlen(NULL) will segfault.

Phobos is somewhere in the middle - sometimes it verifies arguments with 
enforce(), some other times it just uses assert().


Andrei
Jul 30 2014
parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, 30 July 2014 at 22:58:01 UTC, Andrei Alexandrescu 
wrote:

 On 7/30/14, 3:39 PM, Joseph Rushton Wakeling via Digitalmars-d 
 wrote:

 On 31/07/14 00:01, Walter Bright via Digitalmars-d wrote:

 7. using enforce() to check for program bugs is utterly wrong.
 enforce() is a
 library creation, the core language does not recognize it.


 A question on that.

 There are various places in Phobos where enforce() statements 
 are used
 to validate function input or class constructor parameters.


 Yah, Phobos is a bit inconsistent about that. TDPL discusses 
 the matter: if a library is deployed in separation from the 
 program(s) it serves, it may as well handle arguments as 
 "input". That's what e.g. the Windows API is doing - it 
 consistently considers all function arguments "inputs", scrubs 
 them, and returns error codes for all invalid inputs it 
 detects. In contracts, the traditional libc/Unix interface does 
 little checking, even a strlen(NULL) will segfault.

 Phobos is somewhere in the middle - sometimes it verifies 
 arguments with enforce(), some other times it just uses 
 assert().


Yeah, we're not entirely consistent with it. However, if it would 
definitely be a program bug for an argument to not pass a 
particular condition, then it should be an assertion, and if it's 
definitely likely to be program input (e.g. this is frequently 
the case with strings), then exceptions are the appropriate 
approach. It's the cases where it's not obviously program input 
that's more debatable. Forcing checks and throwing exceptions 
incurs overhead, but it can significantly reduce programming 
bugs, because it doesn't put the onus on the programmer to verify 
the arguments. Using assertions is more performant but can 
significantly increase the risk of bugs - particularly when the 
assertions will all be compiled out when Phobos is compiled into 
a library unless the function is templated.

I know that Walter favors using assertions everywhere and then 
providing functions which do the checks so that the programmer 
can check and then throw if appropriate, but the check isn't 
forced. Personally, I much prefer being defensive and to default 
to checking the input and throwing on bad input but to provide a 
way to avoid the check if you've already validated the input and 
don't want the cost of the check. For instance, many of the 
functions in std.datetime throw (particularly constructors), 
because it's being defensive, but it's on my todo list to add 
functions to bypass some of the checks (e.g. a function which 
constructs the type without doing any checks in addition to 
having the normal constructors). Regardless, I think that using 
assertions as the go-to solution for validating function 
arguments is generally going to result in a lot more programming 
bugs. I'd much prefer to default to being safe but provide 
backdoors for speed when you need it (which is generally how D 
does things).

But regardless of which approach you prefer, there are some cases 
where it's pretty clear whether an assertion or exception should 
be used, and there are other cases where it's highly debatable - 
primarily depending on whether you want to treat a function's 
arguments as user input or rely on the programmer to do all of 
the necessary validations first.

- Jonathan M Davis
Jul 30 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/30/2014 3:39 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:

 My take is that, for this reason, these should be asserts and not enforce()
 statements.  What are your thoughts on the matter?


An excellent question.

First, note that enforce() returns a recoverable exception, and assert() a 
non-recoverable error.

Logically, this means that enforce() is for scrubbing input, and assert() is
for 
detecting program bugs. I'm pretty brutal in asserting (!) that program bugs
are 
fatal, non-recoverable errors. I've been fighting this battle for decades, i.e. 
repeated proposals by well-meaning programmers who believe their programs can 
safely recover from an unknown, invalid state.

Pragmatically, enforce() is significantly more expensive as it uses the GC and 
callers must support exception safety. Also, take a look at the quantity 
generated code for the enforce() in the examples, and see how expensive it is. 
All those lovely messages generate a lot of bloat.

Phobos functions that are designed to scrub input must document so. Otherwise, 
they should assert.

For LinearCongruentialEngine() and initialize(), passing invalid arguments are 
programming bugs, and so they should be asserting.
Jul 30 2014
next sibling parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 31 July 2014 at 06:57:15 UTC, Walter Bright wrote:

 For LinearCongruentialEngine() and initialize(), passing 
 invalid arguments are programming bugs, and so they should be 
 asserting.


Isn't phobos compiled in release mode? And since those asserts 
are never compiled, what purpose do they serve?
Jul 31 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 31 July 2014 at 13:10:58 UTC, Kagamin wrote:

 On Thursday, 31 July 2014 at 06:57:15 UTC, Walter Bright wrote:

 For LinearCongruentialEngine() and initialize(), passing 
 invalid arguments are programming bugs, and so they should be 
 asserting.


 Isn't phobos compiled in release mode? And since those asserts 
 are never compiled, what purpose do they serve?


The whole type is templated, so the assertions will be compiled 
in based on whether the user's code is compiled with -released or 
not. But it is true that in any non-templated Phobos code, 
assertions are only good for checking Phobos during its unit 
tests if Phobos is compiled in release mode. IIRC though, there's 
a -defaultlib switch (or something close to that) which allows 
you to tell the compiler to use a different version of Phobos, so 
you can tell it to use a non-release build of Phobos, but I don't 
remember if we provide such a build with the released compiler or 
not. Regardless, most code probably won't use it, so while it's a 
nice idea and useful for those who put in the effort to use it, 
it's definitely true that most code won't benefit from it, 
seriously reducing the value of any assertions in non-templated 
Phobos code which are intended to check user code rather than 
Phobos itself.

- Jonathan M Davis
Jul 31 2014
parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 31 July 2014 at 19:31:51 UTC, Jonathan M Davis wrote:

 The whole type is templated, so the assertions will be compiled 
 in based on whether the user's code is compiled with -released 
 or not.


Sounds tricky. Doesn't the compiler optimize template 
instantiation? If it finds an already compiled template instance 
somewhere, it will use it instead of generating new one.
Jul 31 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 1 August 2014 at 06:53:17 UTC, Kagamin wrote:

 On Thursday, 31 July 2014 at 19:31:51 UTC, Jonathan M Davis 
 wrote:

 The whole type is templated, so the assertions will be 
 compiled in based on whether the user's code is compiled with 
 -released or not.


 Sounds tricky. Doesn't the compiler optimize template 
 instantiation? If it finds an already compiled template 
 instance somewhere, it will use it instead of generating new 
 one.


Since all template instantiations must happen when you compile 
your program rather than in any libraries you're linking against, 
why would it matter? If you compile your program without 
-release, then all assertions in all templates that your program 
uses will be enabled. The only cases where it wouldn't be would 
be when a library that you're linking against uses that template 
internally, but then it's that library's code and not yours, so 
it's probably out of your hands to deal with assertion failures 
in it anyway. Any template reuse that occurred would be reuse 
within your program and thus use the same flags. The only way 
that I can think of that that could be screwed up is if you're 
compiling each module separately and use different flags for 
each, but then each module which was compiled with -release 
wouldn't have assertions and those that weren't would. The 
compiler would never try to share any template instantiations 
across them. They wouldn't even be involved with each other until 
the linker was run on them, so the compiler wouldn't even have 
the chance to try and share anything between them.

- Jonathan M Davis
Aug 01 2014
next sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Fri, 01 Aug 2014 07:01:48 +0000
schrieb "Jonathan M Davis" <jmdavisProg gmx.com>:


 Since all template instantiations must happen when you compile 
 your program rather than in any libraries you're linking against, 
 why would it matter? If you compile your program without 


That's not true, template instances emit weak symbols. If you link
against a compiled library which also has instantiated the same
templates this library provides the same weak symbols. Which symbols are
actually used in the final executable is up to the linker and not
standardized.
Aug 01 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Johannes Pfau"  wrote in message news:lrfqqt$1jem$1 digitalmars.com...


 Which symbols are
 actually used in the final executable is up to the linker and not
 standardized.


Isn't it?  dmd will set it up so the definitions in the library will only 
get pulled in if undefined, and this will usually mean any definitions in 
the main object files will override the library ones.  It should work this 
way on all platforms.

Of course, definitions in different object files will be subject to comdat 
folding and that is not standardized.
Aug 01 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Fri, 1 Aug 2014 23:20:02 +1000
schrieb "Daniel Murphy" <yebbliesnospam gmail.com>:


 "Johannes Pfau"  wrote in message
 news:lrfqqt$1jem$1 digitalmars.com...
 

 Which symbols are
 actually used in the final executable is up to the linker and not
 standardized.


 
 Isn't it?  dmd will set it up so the definitions in the library will
 only get pulled in if undefined, and this will usually mean any
 definitions in the main object files will override the library ones.
 It should work this way on all platforms.


Do you know if linkers actually guarantee that behaviour? AFAICS dmd
doesn't do anything special, it always emits weak symbols and just calls
gcc to link. The linker usually uses the first symbol it sees, but I
have not seen any guarantees for that.

Also what happens if your main application doesn't use the template,
but two libraries use the same template. Then which instances are
actually used? I'd guess those from the 'first' library?


 Of course, definitions in different object files will be subject to
 comdat folding and that is not standardized.
Aug 01 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Johannes Pfau"  wrote in message news:lrgar7$1vrr$1 digitalmars.com...


 Do you know if linkers actually guarantee that behaviour? AFAICS dmd
 doesn't do anything special, it always emits weak symbols and just calls
 gcc to link. The linker usually uses the first symbol it sees, but I
 have not seen any guarantees for that.


Yes, they do.  Or at least, it's the classic linker behaviour and I expect 
they all implement it.  It's relied upon for nofloat and a few other things. 
It's one of the reasons linkers are hard to parallelize.

dmd does split single d modules into multiple object modules when compiling 
with -lib, although I think that's fairly standard.


 Also what happens if your main application doesn't use the template,
 but two libraries use the same template. Then which instances are
 actually used? I'd guess those from the 'first' library?


Well, the first one that it encounters after getting the undefined 
reference.

eg
module application
{
undef x
...
}


lib1:
module a
{
undef tmpl
def a
}
module tmpl
{
def tmpl
}

lib2:
module x
{
undef tmpl
def x
}
module tmpl
{
def tmpl
}

Here nothing that the application references will cause the tmpl in lib1 to 
get pulled in, but when it processes lib2 it will pull in x, and then tmpl.
Aug 01 2014
parent "Kagamin" <spam here.lot> writes:
On Friday, 1 August 2014 at 16:04:21 UTC, Daniel Murphy wrote:

 "Johannes Pfau"  wrote in message 
 news:lrgar7$1vrr$1 digitalmars.com...


 Do you know if linkers actually guarantee that behaviour? 
 AFAICS dmd
 doesn't do anything special, it always emits weak symbols and 
 just calls
 gcc to link. The linker usually uses the first symbol it sees, 
 but I
 have not seen any guarantees for that.


 Yes, they do.  Or at least, it's the classic linker behaviour 
 and I expect they all implement it.  It's relied upon for 
 nofloat and a few other things. It's one of the reasons linkers 
 are hard to parallelize.


Though your linker specification is not the same as Johannes said.


 Also what happens if your main application doesn't use the 
 template,
 but two libraries use the same template. Then which instances 
 are
 actually used? I'd guess those from the 'first' library?


 Well, the first one that it encounters after getting the 
 undefined reference.


That's just how ld ended up working, and it's known to be pretty 
stupid in comparison with other linkers. And it's used rather 
stupidly, e.g. gcc just repeats libs several times to ensure ld 
sees all symbols it needs.


 eg
 module application
 {
 undef x
 ...
 }


 lib1:
 module a
 {
 undef tmpl
 def a
 }
 module tmpl
 {
 def tmpl
 }

 lib2:
 module x
 {
 undef tmpl
 def x
 }
 module tmpl
 {
 def tmpl
 }

 Here nothing that the application references will cause the 
 tmpl in lib1 to get pulled in, but when it processes lib2 it 
 will pull in x, and then tmpl.


lib2:
module x
{
undef tmpl
def x
}

phobos:
module tmpl
{
def tmpl
}

Then tmpl is picked from phobos instead of your code. Template 
emission optimization is a simple tool: if you realistically link 
against a module, which already has an instantiated template, 
template emission can be skipped and the linker will reasonably 
link whatever it finds.
Aug 03 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Friday, 1 August 2014 at 07:01:49 UTC, Jonathan M Davis wrote:

 Since all template instantiations must happen when you compile 
 your program rather than in any libraries you're linking 
 against, why would it matter?


AFAIK, there's no distinction between a library and a program in 
D, only modules and packages. How would it differentiate them? 
And I didn't see any documented guarantee for template 
instantiation optimization behavior.


 The compiler would never try to share any template 
 instantiations across them. They wouldn't even be involved with 
 each other until the linker was run on them, so the compiler 
 wouldn't even have the chance to try and share anything between 
 them.


Nobody cares about intermediate files, only what gets into the 
final executable matters, and that is decided by the linker, and 
it will remove redundant template instances.
Aug 01 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 7:33 AM, Kagamin wrote:

 Nobody cares about intermediate files,


I do because it influences build speed, which is a major feature of D.
Aug 01 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 31 July 2014 at 06:57:15 UTC, Walter Bright wrote:

 On 7/30/2014 3:39 PM, Joseph Rushton Wakeling via Digitalmars-d 
 wrote:

 My take is that, for this reason, these should be asserts and 
 not enforce() statements.  What are your thoughts on the 
 matter?


 An excellent question.

 First, note that enforce() returns a recoverable exception, and 
 assert() a non-recoverable error.

 Logically, this means that enforce() is for scrubbing input, 
 and assert() is for detecting program bugs. I'm pretty brutal 
 in asserting (!) that program bugs are fatal, non-recoverable 
 errors. I've been fighting this battle for decades, i.e. 
 repeated proposals by well-meaning programmers who believe 
 their programs can safely recover from an unknown, invalid 
 state.


It may be worth drawing a distinction between serial programs
(ie. a typical desktop app) and parallel programs (ie. server
code).  For example, in a server program, a logic error caused by
one particular type of request often doesn't invalidate the state
of the entire process.  More often, it just prevents further
processing of that one request.  Much like how an error in one
thread of a multithreaded program with no shared data does not
corrupt the state of the entire system.

In short, what one terms a "process" is really somewhat fluid.
What is the distinction between a multi-threaded application
without sharing and multiple instances of the same process all
running individually?  In Erlang, a "process" is really just a
thread being run by the VM, and each process is effectively a
class instance.  All errors are fatal, but they're only fatal for
that one logical process.  It doesn't take down the whole VM.

Now in a language like D that allows direct memory access, one
could argue that any logic error may theoretically corrupt the
entire program, and I presume this is where you stand.  But more
often in my experience, some artifact of the input data or
environmental factors end up pushing execution through a path
that wasn't adequately tested, and results in a fully recoverable
error (initiated by bad program logic) so long as the error is
detected in a timely manner.  These are issues I want caught and
signaled in the most visible manner possible so the logic error
can be fixed, but I don't always want to immediately halt the
entire process and terminate potentially thousands of entirely
correct in-progress transactions.

Perhaps these are all issues that should be marked by enforce
throwing a ProgramLogicException rather than assert with an an
AssertError, but at that point it's almost bikeshedding.
Thoughts?
Jul 31 2014
next sibling parent Daniel Gibson <metalcaedes gmail.com> writes:
Am 31.07.2014 23:21, schrieb Sean Kelly:

 On Thursday, 31 July 2014 at 06:57:15 UTC, Walter Bright wrote:

 On 7/30/2014 3:39 PM, Joseph Rushton Wakeling via Digitalmars-d wrote:

 My take is that, for this reason, these should be asserts and not
 enforce() statements.  What are your thoughts on the matter?


 An excellent question.

 First, note that enforce() returns a recoverable exception, and
 assert() a non-recoverable error.

 Logically, this means that enforce() is for scrubbing input, and
 assert() is for detecting program bugs. I'm pretty brutal in asserting
 (!) that program bugs are fatal, non-recoverable errors. I've been
 fighting this battle for decades, i.e. repeated proposals by
 well-meaning programmers who believe their programs can safely recover
 from an unknown, invalid state.


 It may be worth drawing a distinction between serial programs
 (ie. a typical desktop app) and parallel programs (ie. server
 code).  For example, in a server program, a logic error caused by
 one particular type of request often doesn't invalidate the state
 of the entire process.  More often, it just prevents further
 processing of that one request.  Much like how an error in one
 thread of a multithreaded program with no shared data does not
 corrupt the state of the entire system.

 In short, what one terms a "process" is really somewhat fluid.
 What is the distinction between a multi-threaded application
 without sharing and multiple instances of the same process all
 running individually?  In Erlang, a "process" is really just a
 thread being run by the VM, and each process is effectively a
 class instance.  All errors are fatal, but they're only fatal for
 that one logical process.  It doesn't take down the whole VM.

 Now in a language like D that allows direct memory access, one
 could argue that any logic error may theoretically corrupt the
 entire program, and I presume this is where you stand.  But more
 often in my experience, some artifact of the input data or
 environmental factors end up pushing execution through a path
 that wasn't adequately tested, and results in a fully recoverable
 error (initiated by bad program logic) so long as the error is
 detected in a timely manner.  These are issues I want caught and
 signaled in the most visible manner possible so the logic error
 can be fixed, but I don't always want to immediately halt the
 entire process and terminate potentially thousands of entirely
 correct in-progress transactions.

 Perhaps these are all issues that should be marked by enforce
 throwing a ProgramLogicException rather than assert with an an
 AssertError, but at that point it's almost bikeshedding.
 Thoughts?


I agree.
Also: During development I'd be fine with this terminating the program 
(ideally dumping the core) so the error is immediately noticed, but in 
release mod I'm not, so a construct that halts in debug mode and throws 
an exception or maybe just returns false (or one construct for each of 
the cases) in release mode would be helpful.

Cheers,
Daniel
Jul 31 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 2:21 PM, Sean Kelly wrote:

 Thoughts?


If a process detects a logic error, then that process is in an invalid state 
that is unanticipated and unknown to the developer. The only correct solution
is 
to halt that process, and all processes that share memory with it.

Anything less is based on faith and hope. If it is medical, flight control, or 
banking software, I submit that operating on faith and hope is not acceptable.

If it's a dvr or game, who cares :-) My dvr crashes regularly needing a power 
off reboot.
Jul 31 2014
next sibling parent "Kagamin" <spam here.lot> writes:
On Friday, 1 August 2014 at 04:12:40 UTC, Walter Bright wrote:

 On 7/31/2014 2:21 PM, Sean Kelly wrote:

 Thoughts?


 If a process detects a logic error, then that process is in an 
 invalid state that is unanticipated and unknown to the 
 developer. The only correct solution is to halt that process, 
 and all processes that share memory with it.

 Anything less is based on faith and hope. If it is medical, 
 flight control, or banking software, I submit that operating on 
 faith and hope is not acceptable.


The release version of software is compiled in release mode, 
asserts are removed and everything will continue to operate just 
fine, thankfully.
Aug 01 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 01/08/2014 05:12, Walter Bright wrote:

 On 7/31/2014 2:21 PM, Sean Kelly wrote:

 Thoughts?


 If a process detects a logic error, then that process is in an invalid
 state that is unanticipated and unknown to the developer. The only
 correct solution is to halt that process, and all processes that share
 memory with it.

 Anything less is based on faith and hope. If it is medical, flight
 control, or banking software, I submit that operating on faith and hope
 is not acceptable.

 If it's a dvr or game, who cares :-) My dvr crashes regularly needing a
 power off reboot.


"If it's a game, who cares" -> Oh let's see... let's say I'm playing a 
game, and then there's a bug (which happens often). What would I prefer 
to happen:

* a small (or big) visual glitch, like pixels out of place, corrupted 
textures, or 3D model of an object becoming deformed, or the physics of 
some object behaving erratically, or some broken animation.

* the whole game crashes, and I lose all my progress?

So Walter, which one do you think I prefer? Me, and the rest of the 
million gamers out there?

So yeah, we care, and we are the consumers of an industry worth more 
than the movie industry. As a software developer, to dismiss these 
concerns is silly and unprofessional.

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Sep 18 2014
next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Thu, 18 Sep 2014 17:05:31 +0100
Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 * a small (or big) visual glitch, like pixels out of place, corrupted=20
 textures, or 3D model of an object becoming deformed, or the physics
 of some object behaving erratically, or some broken animation.


or the whole game renders itself unbeatable due to some correpted data,
but you have no way to know it until you made it to the Final Boss and
can never win that fight. ah, so charming!
Sep 18 2014
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Sep 18, 2014 at 05:05:31PM +0100, Bruno Medeiros via Digitalmars-d
wrote:

 On 01/08/2014 05:12, Walter Bright wrote:

On 7/31/2014 2:21 PM, Sean Kelly wrote:

Thoughts?


If a process detects a logic error, then that process is in an
invalid state that is unanticipated and unknown to the developer. The
only correct solution is to halt that process, and all processes that
share memory with it.

Anything less is based on faith and hope. If it is medical, flight
control, or banking software, I submit that operating on faith and
hope is not acceptable.

If it's a dvr or game, who cares :-) My dvr crashes regularly needing
a power off reboot.


 
 "If it's a game, who cares" -> Oh let's see... let's say I'm playing a
 game, and then there's a bug (which happens often). What would I
 prefer to happen:
 
 * a small (or big) visual glitch, like pixels out of place, corrupted
 textures, or 3D model of an object becoming deformed, or the physics
 of some object behaving erratically, or some broken animation.
 
 * the whole game crashes, and I lose all my progress?


[...]

What if the program has a bug that corrupts your save game file, but
because the program ignores these logic errors, it just bumbles onward
and destroys all your progress *without* you even knowing about it until
much later?

(I have actually experienced this firsthand, btw. I found it *far* more
frustrating than losing all my progress -- at least I can restore the
game to the last savepoint, and have confidence that it isn't
irreparably corrupted! I almost threw the computer out the window once
when after a game crash I restored the savefile, only to discover a few
hours later that due to a corruption in the savefile, it was impossible
to win the game after all. Logic errors should *never* have made it past
the point of detection.)


T

-- 
People tell me I'm stubborn, but I refuse to accept it!
Sep 18 2014
next sibling parent Jacob Carlborg <doob me.com> writes:
On 18/09/14 18:49, H. S. Teoh via Digitalmars-d wrote:


 What if the program has a bug that corrupts your save game file, but
 because the program ignores these logic errors, it just bumbles onward
 and destroys all your progress *without* you even knowing about it until
 much later?


Happened to me with Assassin's Creed 3, twice. It did crash and when I 
started the game again the save files where corrupted. Although, I have 
no way of knowing if it crashed when the bug happened or much later.

-- 
/Jacob Carlborg
Sep 18 2014
prev sibling parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 18/09/2014 17:49, H. S. Teoh via Digitalmars-d wrote:

 On Thu, Sep 18, 2014 at 05:05:31PM +0100, Bruno Medeiros via Digitalmars-d
wrote:

 On 01/08/2014 05:12, Walter Bright wrote:

 On 7/31/2014 2:21 PM, Sean Kelly wrote:

 Thoughts?


 If a process detects a logic error, then that process is in an
 invalid state that is unanticipated and unknown to the developer. The
 only correct solution is to halt that process, and all processes that
 share memory with it.

 Anything less is based on faith and hope. If it is medical, flight
 control, or banking software, I submit that operating on faith and
 hope is not acceptable.

 If it's a dvr or game, who cares :-) My dvr crashes regularly needing
 a power off reboot.


 "If it's a game, who cares" -> Oh let's see... let's say I'm playing a
 game, and then there's a bug (which happens often). What would I
 prefer to happen:

 * a small (or big) visual glitch, like pixels out of place, corrupted
 textures, or 3D model of an object becoming deformed, or the physics
 of some object behaving erratically, or some broken animation.

 * the whole game crashes, and I lose all my progress?


 [...]

 What if the program has a bug that corrupts your save game file, but
 because the program ignores these logic errors, it just bumbles onward
 and destroys all your progress *without* you even knowing about it until
 much later?

 (I have actually experienced this firsthand, btw. I found it *far* more
 frustrating than losing all my progress -- at least I can restore the
 game to the last savepoint, and have confidence that it isn't
 irreparably corrupted! I almost threw the computer out the window once
 when after a game crash I restored the savefile, only to discover a few
 hours later that due to a corruption in the savefile, it was impossible
 to win the game after all. Logic errors should *never* have made it past
 the point of detection.)


 T


I never implied a program should try to recover or ignore *all* bugs 
that an assertion triggers. Sometimes it is better to crash the whole 
program straight away, yes. But sometimes ignoring the bug and keep 
going (or throwing an exception) is perfectly fine, and far better than 
the alternative (full program crash).
And since when you're working with assertion checks you know if they 
relate to game logic (the equivalent to "business" model), or to 
presentation (UI, 3D, graphics, sound, etc.), you could define if the 
program tries to continue running or not.

Under Walter's idea, you'd always crash the program.

Even if I had to choose between an occasional savefile corruption that 
would render a game impossible to win, and having my games crash every 
time ANY bug happened, I would still prefer the former. Because of all 
the games I played in my life (I'm 31 and I like gaming, so I've played 
quite a few), I've encountered maybe 3 games where that happened, that I 
can remember (*). But I've seen visual glitches in nearly all games I've 
played, and in maybe 60-75% of them those glitches happened *often 
enough* that it would render the game virtually unplayable.

BTW, the same doesn't apply just to games. Same thing for other desktop 
applications: IDEs, browsers, etc.. For example, in an IDE a bug in a 
parser might bring some visual or behavior glitch ing the current 
editor, but the whole IDE doesn't need to crash because of that!


* If anyone is curious:
Elder Scrolls 2 - Daggerfall: one of the possible endings, the one I was 
trying to go for, became impossible to achieve due to a main quest bug.
Fallout 2: an big side quest in a town became completely unresolvable.
Ultima VII- The Black Gate : with a certain magical spell you can sneak 
into a cave by a point that was meant only for exit. The problem is that 
the cave was only meant to be entered in a different place, and much 
later in the storyline. As such, in the cave you start to find 
dialogues, and notes that are clues to a *murder* that hadn't even 
happened yet! Unfortunately I only realized I was not meant to be the 
cave after several hours of gameplay (battling high level monsters took 
a while)... and even after that, I not only lost those hours, but I had 
a big twist and a massive chunk of the game's storyline spoiled...

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Sep 23 2014
next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Tue, 23 Sep 2014 15:19:56 +0100
Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 Even if I had to choose between an occasional savefile corruption
 that would render a game impossible to win, and having my games crash
 every time ANY bug happened, I would still prefer the former.


it game will crash on ANY bug, it will contain alot less bugs.
'cause even the dumbiest QA full of monkeys will not let the game which
crashes once in a ten seconds to go out of the door.

yet game that renders itself unbeatable after some bug can sucessfully
pass even the very good QA.

so it's easy: bug --> crash. oh, this game has only "savepoint" save
system and last save was long ago? designers of this game are lazy
dumbs, don't buy their games anymore.
Sep 23 2014
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Sep 23, 2014 at 09:25:51PM +0300, ketmar via Digitalmars-d wrote:

 On Tue, 23 Sep 2014 15:19:56 +0100
 Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 

 Even if I had to choose between an occasional savefile corruption
 that would render a game impossible to win, and having my games
 crash every time ANY bug happened, I would still prefer the former.


 it game will crash on ANY bug, it will contain alot less bugs.  'cause
 even the dumbiest QA full of monkeys will not let the game which
 crashes once in a ten seconds to go out of the door.


+1. When the game blatantly crashes on every little thing, there is
pressure on the coders to actually fix the problem. Ignored errors ==
convenient excuse for coders to evade work ("it's *only* a minor display
glitch!", "the deadline's tomorrow, we don't have time to fix this",
"it's 5pm, I need to go home and feed my goldfish", "besides, it still
works anyway, more or less", etc.).


T

-- 
Life is complex. It consists of real and imaginary parts. -- YHL
Sep 23 2014
parent reply Bruno Medeiros <bruno.do.medeiros+dng gmail.com> writes:
On 23/09/2014 19:35, H. S. Teoh via Digitalmars-d wrote:

it game will crash on ANY bug, it will contain alot less bugs.  'cause
even the dumbiest QA full of monkeys will not let the game which
crashes once in a ten seconds to go out of the door.






When doing QA, that's a whole different thing than when playing normally.


 +1. When the game blatantly crashes on every little thing, there is
 pressure on the coders to actually fix the problem. Ignored errors ==
 convenient excuse for coders to evade work ("it's*only*  a minor display
 glitch!", "the deadline's tomorrow, we don't have time to fix this",
 "it's 5pm, I need to go home and feed my goldfish", "besides, it still
 works anyway, more or less", etc.).


I would invite you to buy a *retail copy* of Elder Scrolls 3 : Morrowind 
for PC and try playing that. The game did exactly what Walter and you 
guys suggested: when an assertion tripped, it would crash straight away 
to the desktop, with only a message dialog saying that an assertion had 
failed (with info on the source and line of the assertion, and an extra 
message).
I couldn't play that game, it would crash too often. And saving wasn't 
quick enough that I could in practice be manually saving every 15 min. 
or so.
Only when several iterations of patches game out, several months after 
release, did the game become stable enough to be played enjoyably.
(in fairness, if the assertions where about core game logic, I too would 
prefer a crash rather then possible corruption of game state. But IIRC 
there was a few assertions crashes that seemed related to textures, 
directx, or shader stuff - for that I could have lived without a game crash)

Sometimes even with known bugs the game is shipped, because of 
management pressure and/or huge delays already. (especially with 
pressures to do multi-platform releases).

-- 
Bruno Medeiros
https://twitter.com/brunodomedeiros
Sep 29 2014
next sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Monday, 29 September 2014 at 13:01:19 UTC, Bruno Medeiros 
wrote:

 On 23/09/2014 19:35, H. S. Teoh via Digitalmars-d wrote:

it game will crash on ANY bug, it will contain alot less 
bugs.  'cause
even the dumbiest QA full of monkeys will not let the game 
which
crashes once in a ten seconds to go out of the door.






 When doing QA, that's a whole different thing than when playing 
 normally.


 +1. When the game blatantly crashes on every little thing, 
 there is
 pressure on the coders to actually fix the problem. Ignored 
 errors ==
 convenient excuse for coders to evade work ("it's*only*  a 
 minor display
 glitch!", "the deadline's tomorrow, we don't have time to fix 
 this",
 "it's 5pm, I need to go home and feed my goldfish", "besides, 
 it still
 works anyway, more or less", etc.).


 I would invite you to buy a *retail copy* of Elder Scrolls 3 : 
 Morrowind for PC and try playing that. The game did exactly 
 what Walter and you guys suggested: when an assertion tripped, 
 it would crash straight away to the desktop, with only a 
 message dialog saying that an assertion had failed (with info 
 on the source and line of the assertion, and an extra message).
 I couldn't play that game, it would crash too often. And saving 
 wasn't quick enough that I could in practice be manually saving 
 every 15 min. or so.
 Only when several iterations of patches game out, several 
 months after release, did the game become stable enough to be 
 played enjoyably.
 (in fairness, if the assertions where about core game logic, I 
 too would prefer a crash rather then possible corruption of 
 game state. But IIRC there was a few assertions crashes that 
 seemed related to textures, directx, or shader stuff - for that 
 I could have lived without a game crash)

 Sometimes even with known bugs the game is shipped, because of 
 management pressure and/or huge delays already. (especially 
 with pressures to do multi-platform releases).


And the right attitude as consumer is to ask for the money back, 
not to wait patiently for the day those bugs eventually get fixed.

If everyone did that, the quality in IT would be much better.

--
Paulo
Sep 29 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 29 September 2014 at 13:41:33 UTC, Paulo  Pinto wrote:

 And the right attitude as consumer is to ask for the money 
 back, not to wait patiently for the day those bugs eventually 
 get fixed.

 If everyone did that, the quality in IT would be much better.


The mobile freemium model where you try before you buy kind of 
offsets this position though. But if the game saves frequently 
and reloads quickly it is less annoying.

After watching the initial years of MMO I also think people are 
willing to live with bugs, nerfing and other upsetting events 
until the Next Big Thing comes along. There probably is some 
value in having something common to complain about. :-)
Sep 29 2014
prev sibling parent Brad Roberts via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 9/29/2014 6:01 AM, Bruno Medeiros via Digitalmars-d wrote:

 I would invite you to buy a *retail copy* of Elder Scrolls 3 : Morrowind
 for PC and try playing that. The game did exactly what Walter and you
 guys suggested: when an assertion tripped, it would crash straight away
 to the desktop, with only a message dialog saying that an assertion had
 failed (with info on the source and line of the assertion, and an extra
 message).
 I couldn't play that game, it would crash too often. And saving wasn't
 quick enough that I could in practice be manually saving every 15 min.
 or so.
 Only when several iterations of patches game out, several months after
 release, did the game become stable enough to be played enjoyably.
 (in fairness, if the assertions where about core game logic, I too would
 prefer a crash rather then possible corruption of game state. But IIRC
 there was a few assertions crashes that seemed related to textures,
 directx, or shader stuff - for that I could have lived without a game
 crash)

 Sometimes even with known bugs the game is shipped, because of
 management pressure and/or huge delays already. (especially with
 pressures to do multi-platform releases).


The problem isn't the use of asserts and quick exiting of the app, 
though.  The problem is releasing an app of horrible quality.  The 
assert based reporting just makes it super obvious.  There's no way to 
say how bad the game would have been with "try and continue anyway" code.
Sep 29 2014
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Sep 18, 2014 at 07:13:48PM +0300, ketmar via Digitalmars-d wrote:

 On Thu, 18 Sep 2014 17:05:31 +0100
 Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> wrote:
 

 * a small (or big) visual glitch, like pixels out of place,
 corrupted textures, or 3D model of an object becoming deformed, or
 the physics of some object behaving erratically, or some broken
 animation.


 or the whole game renders itself unbeatable due to some correpted
 data, but you have no way to know it until you made it to the Final
 Boss and can never win that fight. ah, so charming!


Exactly!!!!

Seriously, this philosophy of ignoring supposedly "minor" bugs in
software is what led to the sad state of software today, where nothing
is reliable and people have come to expect that software will inevitably
crash, and that needing to reboot an OS every now and then just to keep
things working is acceptable. Yet, strangely enough, people will scream
bloody murder if a car behaved like that.


T

-- 
Skill without imagination is craftsmanship and gives us many useful
objects such as wickerwork picnic baskets.  Imagination without skill
gives us modern art. -- Tom Stoppard
Sep 18 2014
next sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 18 September 2014 at 16:55:33 UTC, H. S. Teoh via
Digitalmars-d wrote:

 On Thu, Sep 18, 2014 at 07:13:48PM +0300, ketmar via 
 Digitalmars-d wrote:

 On Thu, 18 Sep 2014 17:05:31 +0100
 Bruno Medeiros via Digitalmars-d <digitalmars-d puremagic.com> 
 wrote:
 

 * a small (or big) visual glitch, like pixels out of place,
 corrupted textures, or 3D model of an object becoming 
 deformed, or
 the physics of some object behaving erratically, or some 
 broken
 animation.


 or the whole game renders itself unbeatable due to some 
 correpted
 data, but you have no way to know it until you made it to the 
 Final
 Boss and can never win that fight. ah, so charming!


 Exactly!!!!

 Seriously, this philosophy of ignoring supposedly "minor" bugs 
 in
 software is what led to the sad state of software today, where 
 nothing
 is reliable and people have come to expect that software will 
 inevitably
 crash, and that needing to reboot an OS every now and then just 
 to keep
 things working is acceptable. Yet, strangely enough, people 
 will scream
 bloody murder if a car behaved like that.


 T


Fully agree. I support the idea that software companies should be
accountable by their products the same way as other industries.

Just like when you buy a car the car dealer will be accountable
for any defect.

Sure it will make software development a bit more expensive, but
it will also end with a lot of cowboy programming, updates that
only work in full moon, ...

--
Paulo
Sep 19 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 18 September 2014 at 16:55:33 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 Seriously, this philosophy of ignoring supposedly "minor" bugs 
 in
 software is what led to the sad state of software today, where 
 nothing
 is reliable and people have come to expect that software will 
 inevitably
 crash, and that needing to reboot an OS every now and then just 
 to keep
 things working is acceptable. Yet, strangely enough, people 
 will scream
 bloody murder if a car behaved like that.


A car did behave like that--the Toyota Prius.  It would randomly 
accelerate out of control because of a bug in the car's software. 
  People died.  And yes, bloody murder was screamed.

I used to work for the telephone company, and telcos are a bit 
weird because they're considered an essential service.  If 
martial law were enacted in the US, telephone workers would still 
be allowed on the street because the telephone system must work.  
There was one instance when I was working that I had to get to 
the switch for maintenance during a horrible storm where the 
police had closed the roads.  No problem--they let me through.  
Network services are rapidly approaching this point.  People 
depend on the internet for communication these days as much or 
more than they do on telephone calls.  I suspect/hope that it 
won't be long before communications software is held to standards 
similar to telephone, which will require a huge adjustment on the 
part of programmers.  People in this industry still tend to not 
think of things in terms of 5+ "nines" of uptime for their 
service, despite that being the expectation of their users.
Sep 19 2014
parent "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 19 September 2014 at 13:56:14 UTC, Sean Kelly wrote:

 I suspect/hope that it won't be long before communications 
 software is held to standards similar to telephone, which will 
 require a huge adjustment on the part of programmers.  People 
 in this industry still tend to not think of things in terms of 
 5+ "nines" of uptime for their service, despite that being the 
 expectation of their users.


Oh, just to clarify one thing though.  The typical way that a lot 
of network services work for reasons of performance is via 
asynchronous event-based models (fancy versions of the Unix 
poll() routine).  If a logic error results in a transaction 
failure and I can be sure that the scope of this error is limited 
to the state of that one transaction, I don't necessarily want to 
terminate the process as it will result in thousands of failed 
transactions.  Functional languages like Erlang tend to support 
this model really well, because a "process" is actually just a 
thread of execution in the VM.  The "process" (thread) is 
terminated and the VM soldiers on because there's no logical way 
for the state of the VM to have been corrupted.

I'm still unsure how I feel about D in this regard.  Clients will 
retry if a request fails, and so terminating the entire process 
and failing thousands of transactions would be bad but possibly 
still invisible to users.  At least unless the logic error occurs 
in a common path of execution, in which case terminating the 
entire process on a logic error becomes a potential attack vector 
to bring down the entire system.

I suspect that what I'll be doing with D is moving away from 
asserts and throwing Errors to give myself the latitude to handle 
situations in whatever manner I feel is appropriate.  I'm still 
not sure about this, but I'm leaning towards feeling that the D 
approach is possibly too draconian for multi-user systems.  
Perhaps it's possible to split each transaction off to be handled 
by a separate process (via unix domain sockets, for example, or 
IPC), but I suspect this won't scale to the degree that's needed. 
  This is how web servers used to work, for example, and I'm 
pretty sure none of them do this any more.  But I want to think 
more on the problem, as I do like the motivation behind these 
rules.
Sep 19 2014
prev sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 18 September 2014 at 16:05:32 UTC, Bruno Medeiros 
wrote:

 "If it's a game, who cares" -> Oh let's see... let's say I'm 
 playing a game, and then there's a bug (which happens often). 
 What would I prefer to happen:

 * a small (or big) visual glitch, like pixels out of place, 
 corrupted textures, or 3D model of an object becoming deformed, 
 or the physics of some object behaving erratically, or some 
 broken animation.

 * the whole game crashes, and I lose all my progress?

 So Walter, which one do you think I prefer? Me, and the rest of 
 the million gamers out there?


Don't worry.  A shipped game would have been built with -release 
set because it's a release build.  No one cares about the 
reliability of released software, only test builds. /rant
Sep 19 2014
prev sibling parent "IgorStepanov" <wazar mail.ru> writes:
 The point is that using these enforce() statements means that 
 these methods cannot be nothrow, which doesn't seem 
 particularly nice if it can be avoided. Now, on the one hand, 
 one could say that, quite obviously, these methods cannot 
 control their input.  But on the other hand, it's reasonable to 
 say that these methods' input can and should never be anything 
 other than 100% controlled by the programmer.

 My take is that, for this reason, these should be asserts and 
 not enforce() statements.  What are your thoughts on the matter?


Assert's are disable by -release option. If you want to do check, 
and don't want to raise exception you can define special function 
like this:
import std.stdio;

void enforceNoThrow(bool pred, string msg, string file = 
__FILE__, uint line = __LINE__)
{
     if (!pred)
     {
         stderr.writefln("Fatal error: %s at %s:%s", msg, file, 
line);
         assert(0);
     }

}

void doSomething(int min, int middle, int max)
{
     enforceNoThrow(middle >= min && middle <= max, "middle out of 
bounds");
}

doSomething(1, 5, 3); //prints error message into stderr and 
terminate programm execution

This way is better for library code then asserts. Asserts should 
be used for catching internal bugs, but incorrect code is 
external bug (for extern library)
Sep 18 2014
prev sibling next sibling parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 7/30/14, 7:01 PM, Walter Bright wrote:

 I'd like to sum up my position and intent on all this.

 7. using enforce() to check for program bugs is utterly wrong. enforce()
 is a library creation, the core language does not recognize it.


What do you suggest to use to check program bugs?
Jul 30 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jul 30, 2014 at 08:05:35PM -0300, Ary Borenszweig via Digitalmars-d
wrote:

 On 7/30/14, 7:01 PM, Walter Bright wrote:

I'd like to sum up my position and intent on all this.

7. using enforce() to check for program bugs is utterly wrong. enforce()
is a library creation, the core language does not recognize it.


 
 What do you suggest to use to check program bugs?


This is what assert is designed for.

But if you don't want to check ever to be removed, currently you have to
write:

	if (!requiredCondition)
		assert(0); // compiler never removes this

which IMO is relatively clear, and the use of assert(0) for forceful
termination is consistent with existing practice in D code.


T

-- 
Старый друг лучше новых двух.
Jul 30 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 30 July 2014 at 23:50:51 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 But if you don't want to check ever to be removed, currently 
 you have to
 write:

 	if (!requiredCondition)
 		assert(0); // compiler never removes this

 which IMO is relatively clear, and the use of assert(0) for 
 forceful
 termination is consistent with existing practice in D code.


Not helping.

```
import std.stdio;

void foo()
in { writeln("in contract"); }
body { }

void main() { foo(); }
```

Compile with -release and check the output.
Jul 30 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/30/2014 6:16 PM, Dicebot wrote:

 On Wednesday, 30 July 2014 at 23:50:51 UTC, H. S. Teoh via Digitalmars-d wrote:

 But if you don't want to check ever to be removed, currently you have to
 write:

     if (!requiredCondition)
         assert(0); // compiler never removes this

 which IMO is relatively clear, and the use of assert(0) for forceful
 termination is consistent with existing practice in D code.


 Not helping.

 ```
 import std.stdio;

 void foo()
 in { writeln("in contract"); }
 body { }

 void main() { foo(); }
 ```

 Compile with -release and check the output.


What do you expect to happen?
Jul 31 2014
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 31 July 2014 at 07:42:16 UTC, Walter Bright wrote:

 On 7/30/2014 6:16 PM, Dicebot wrote:

 On Wednesday, 30 July 2014 at 23:50:51 UTC, H. S. Teoh via 
 Digitalmars-d wrote:

 But if you don't want to check ever to be removed, currently 
 you have to
 write:

    if (!requiredCondition)
        assert(0); // compiler never removes this

 which IMO is relatively clear, and the use of assert(0) for 
 forceful
 termination is consistent with existing practice in D code.


 Not helping.

 ```
 import std.stdio;

 void foo()
 in { writeln("in contract"); }
 body { }

 void main() { foo(); }
 ```

 Compile with -release and check the output.


 What do you expect to happen?


It acts as defined in spec, nothing unexpected here. I was 
referring to H. S. Teoh proposed workaround to keep assertions in 
release mode - it does not work with contracts because those are 
eliminated completely, not just assertions inside.
Jul 31 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

 On 7/30/14, 7:01 PM, Walter Bright wrote:

 I'd like to sum up my position and intent on all this.

 7. using enforce() to check for program bugs is utterly wrong. enforce()
 is a library creation, the core language does not recognize it.


 What do you suggest to use to check program bugs?


assert()
Jul 31 2014
parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 7/31/14, 4:37 AM, Walter Bright wrote:

 On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

 On 7/30/14, 7:01 PM, Walter Bright wrote:

 I'd like to sum up my position and intent on all this.

 7. using enforce() to check for program bugs is utterly wrong. enforce()
 is a library creation, the core language does not recognize it.


 What do you suggest to use to check program bugs?


 assert()


Then you are potentially releasing programs with bugs that are of 
undefined behavior, instead of halting the program immediately.
Jul 31 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 31, 2014 at 03:43:48PM -0300, Ary Borenszweig via Digitalmars-d
wrote:

 On 7/31/14, 4:37 AM, Walter Bright wrote:

On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

On 7/30/14, 7:01 PM, Walter Bright wrote:

I'd like to sum up my position and intent on all this.

7. using enforce() to check for program bugs is utterly wrong.
enforce() is a library creation, the core language does not
recognize it.


What do you suggest to use to check program bugs?


assert()


 
 Then you are potentially releasing programs with bugs that are of
 undefined behavior, instead of halting the program immediately.


Isn't that already what you're doing with the current behaviour of
assert? Not only in D, but also in C/C++.


T

-- 
When solving a problem, take care that you do not become part of the problem.
Jul 31 2014
next sibling parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 7/31/14, 4:34 PM, H. S. Teoh via Digitalmars-d wrote:

 On Thu, Jul 31, 2014 at 03:43:48PM -0300, Ary Borenszweig via Digitalmars-d
wrote:

 On 7/31/14, 4:37 AM, Walter Bright wrote:

 On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

 On 7/30/14, 7:01 PM, Walter Bright wrote:

 I'd like to sum up my position and intent on all this.

 7. using enforce() to check for program bugs is utterly wrong.
 enforce() is a library creation, the core language does not
 recognize it.


 What do you suggest to use to check program bugs?


 assert()


 Then you are potentially releasing programs with bugs that are of
 undefined behavior, instead of halting the program immediately.


 Isn't that already what you're doing with the current behaviour of
 assert? Not only in D, but also in C/C++.


 T


I don't program in those languages, and if I did I would always use 
exceptions (at least in C++). I don't want to compromise the safety of 
my programs and if they fail I want to get a clean backtrace, not some 
random undefined behaviour resulting in a segfault.
Jul 31 2014
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 31 July 2014 at 19:36:34 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Thu, Jul 31, 2014 at 03:43:48PM -0300, Ary Borenszweig via 
 Digitalmars-d wrote:

 On 7/31/14, 4:37 AM, Walter Bright wrote:

On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

On 7/30/14, 7:01 PM, Walter Bright wrote:

I'd like to sum up my position and intent on all this.

7. using enforce() to check for program bugs is utterly 
wrong.
enforce() is a library creation, the core language does not
recognize it.


What do you suggest to use to check program bugs?


assert()


 
 Then you are potentially releasing programs with bugs that are 
 of
 undefined behavior, instead of halting the program immediately.


 Isn't that already what you're doing with the current behaviour 
 of
 assert? Not only in D, but also in C/C++.


Yes, but I think that his point was that he wants a way to check 
programming bugs in release mode, and Walter was saying not to 
use enforce for checking programming bugs. So, that leaves the 
question of how to check them in release mode, since assertions 
won't work in release mode. But the answer to that is normally to 
not compile in release mode. And I believe that dmd gives enough 
control over that that you can get everything that -release does 
without disabling assertions using flags other than -release.

- Jonathan M Davis
Jul 31 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 31, 2014 at 07:39:54PM +0000, Jonathan M Davis via Digitalmars-d
wrote:

 On Thursday, 31 July 2014 at 19:36:34 UTC, H. S. Teoh via Digitalmars-d
 wrote:

On Thu, Jul 31, 2014 at 03:43:48PM -0300, Ary Borenszweig via
Digitalmars-d wrote:




[...]

What do you suggest to use to check program bugs?


assert()


Then you are potentially releasing programs with bugs that are of
undefined behavior, instead of halting the program immediately.


Isn't that already what you're doing with the current behaviour of
assert? Not only in D, but also in C/C++.


 
 Yes, but I think that his point was that he wants a way to check
 programming bugs in release mode, and Walter was saying not to use
 enforce for checking programming bugs. So, that leaves the question of
 how to check them in release mode, since assertions won't work in
 release mode. But the answer to that is normally to not compile in
 release mode. And I believe that dmd gives enough control over that
 that you can get everything that -release does without disabling
 assertions using flags other than -release.


[...]

Ah, I see.

But doesn't that just mean that you shouldn't use -release, period?
AFAIK, the only thing -release does it to remove various safety checks,
like array bounds checks, asserts, contracts (which are generally
written using asserts), etc.. I'd think that Ary wouldn't want any of
these disabled, so he shouldn't use -release at all. There's already -O
and -inline to enable what people generally expect from a release build,
so -release wouldn't really be needed at all.

Right?


T

-- 
Some ideas are so stupid that only intellectuals could believe them. -- George
Orwell
Jul 31 2014
parent reply Ary Borenszweig <ary esperanto.org.ar> writes:
On 7/31/14, 4:54 PM, H. S. Teoh via Digitalmars-d wrote:

 On Thu, Jul 31, 2014 at 07:39:54PM +0000, Jonathan M Davis via Digitalmars-d
wrote:

 On Thursday, 31 July 2014 at 19:36:34 UTC, H. S. Teoh via Digitalmars-d
 wrote:

 On Thu, Jul 31, 2014 at 03:43:48PM -0300, Ary Borenszweig via
 Digitalmars-d wrote:




 [...]

 What do you suggest to use to check program bugs?


 assert()


 Then you are potentially releasing programs with bugs that are of
 undefined behavior, instead of halting the program immediately.


 Isn't that already what you're doing with the current behaviour of
 assert? Not only in D, but also in C/C++.


 Yes, but I think that his point was that he wants a way to check
 programming bugs in release mode, and Walter was saying not to use
 enforce for checking programming bugs. So, that leaves the question of
 how to check them in release mode, since assertions won't work in
 release mode. But the answer to that is normally to not compile in
 release mode. And I believe that dmd gives enough control over that
 that you can get everything that -release does without disabling
 assertions using flags other than -release.


 [...]

 Ah, I see.

 But doesn't that just mean that you shouldn't use -release, period?
 AFAIK, the only thing -release does it to remove various safety checks,
 like array bounds checks, asserts, contracts (which are generally
 written using asserts), etc.. I'd think that Ary wouldn't want any of
 these disabled, so he shouldn't use -release at all. There's already -O
 and -inline to enable what people generally expect from a release build,
 so -release wouldn't really be needed at all.

 Right?


 T


That's exactly my point, thank you for summing that up :-)

I don't see the point of having a "-release" flag. It should be renamed 
to "-a-bit-faster-but-unsafe".

I think there are other languages that do quite well in terms of 
performance without disabling bounds checks and other stuff.
Jul 31 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 31, 2014 at 05:17:13PM -0300, Ary Borenszweig via Digitalmars-d
wrote:

 On 7/31/14, 4:54 PM, H. S. Teoh via Digitalmars-d wrote:


[...]

But doesn't that just mean that you shouldn't use -release, period?
AFAIK, the only thing -release does it to remove various safety
checks, like array bounds checks, asserts, contracts (which are
generally written using asserts), etc.. I'd think that Ary wouldn't
want any of these disabled, so he shouldn't use -release at all.
There's already -O and -inline to enable what people generally expect
from a release build, so -release wouldn't really be needed at all.

Right?


T


 
 That's exactly my point, thank you for summing that up :-)
 
 I don't see the point of having a "-release" flag. It should be
 renamed to "-a-bit-faster-but-unsafe".


It's probably named -release because traditionally, turning off asserts
and bounds checks is what is normally done when building a C/C++ project
for release. (C/C++ don't have built-in bounds checks, but I've seen
projects that use macros for manually implementing this, which are
#define'd away when building for release.)



 I think there are other languages that do quite well in terms of
 performance without disabling bounds checks and other stuff.


It depends on what you're doing. If you have assert's in CPU intensive
inner loops, turning them off can make a big difference in performance.


T

-- 
Almost all proofs have bugs, but almost all theorems are true. -- Paul Pedersen
Jul 31 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 10:33 PM, H. S. Teoh via Digitalmars-d wrote:

 On Thu, Jul 31, 2014 at 05:17:13PM -0300, Ary Borenszweig via Digitalmars-d
wrote:

On 7/31/14, 4:54 PM, H. S. Teoh via Digitalmars-d wrote:




 [...]

But doesn't that just mean that you shouldn't use -release, period?
AFAIK, the only thing -release does it to remove various safety
checks, like array bounds checks, asserts, contracts (which are
generally written using asserts), etc.. I'd think that Ary wouldn't
want any of these disabled, so he shouldn't use -release at all.
There's already -O and -inline to enable what people generally expect


from a release build, so -release wouldn't really be needed at all.

Right?


T




That's exactly my point, thank you for summing that up:-)

I don't see the point of having a "-release" flag. It should be
renamed to "-a-bit-faster-but-unsafe".




 It's probably named -release because traditionally, turning off asserts
 and bounds checks is what is normally done when building a C/C++ project
 for release. (C/C++ don't have built-in bounds checks, but I've seen
 projects that use macros for manually implementing this, which are
 #define'd away when building for release.)


The suggestion is not: 'make -release disable assertions.' it is 
'failing assertions are undefined behaviour in -release'.
Jul 31 2014
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 31 July 2014 at 18:43:49 UTC, Ary Borenszweig wrote:

 On 7/31/14, 4:37 AM, Walter Bright wrote:

 On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

 On 7/30/14, 7:01 PM, Walter Bright wrote:

 I'd like to sum up my position and intent on all this.

 7. using enforce() to check for program bugs is utterly 
 wrong. enforce()
 is a library creation, the core language does not recognize 
 it.


 What do you suggest to use to check program bugs?


 assert()


 Then you are potentially releasing programs with bugs that are 
 of undefined behavior, instead of halting the program 
 immediately.


Then don't build with -release. You can even build with 
-boundscheck=safe if you want to turn off bounds checking in 
 system code like -release does. IIRC, the only things that 
-release does are disable assertions, disable contracts, turn 
assert(0) into a halt instruction, and disable bounds checking in 
 system and  trusted code. So, if you want to keep the assertions 
and contracts and whatnot in, just don't use -release and use 
-boundscheck=safe to get the bounds checking changes that 
-release does.

- Jonathan M Davis
Jul 31 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 09:37 PM, Jonathan M Davis wrote:

 On Thursday, 31 July 2014 at 18:43:49 UTC, Ary Borenszweig wrote:

 On 7/31/14, 4:37 AM, Walter Bright wrote:

 On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

 On 7/30/14, 7:01 PM, Walter Bright wrote:

 I'd like to sum up my position and intent on all this.

 7. using enforce() to check for program bugs is utterly wrong.
 enforce()
 is a library creation, the core language does not recognize it.


 What do you suggest to use to check program bugs?


 assert()


 Then you are potentially releasing programs with bugs that are of
 undefined behavior, instead of halting the program immediately.


 Then don't build with -release. You can even build with
 -boundscheck=safe if you want to turn off bounds checking in  system
 code like -release does. IIRC, the only things that -release does are
 disable assertions,


No, according to the OP -release assumes assertions to be true.


 disable contracts, turn assert(0) into a halt
 instruction, and disable bounds checking in  system and  trusted code.
 So, if you want to keep the assertions and contracts and whatnot in,


Unfortunately, if used pervasively, assertions and contracts and whatnot 
may actually hog the speed of a program in a way that breaks the deal.

Disabling assertions (and whatnot), assuming assertions to be true (and 
disabling whatnot) and leaving all assertions and whatnot in are 
different trade-offs, of which assuming all assertions to be true is the 
most dangerous one. Why hide this behaviour in '-release'?


 just don't use -release and use -boundscheck=safe to get the bounds
 checking changes that -release does.

 - Jonathan M Davis


This leaves assertions and contracts in though.
Jul 31 2014
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 31 July 2014 at 20:49:18 UTC, Timon Gehr wrote:

 On 07/31/2014 09:37 PM, Jonathan M Davis wrote:

 disable contracts, turn assert(0) into a halt
 instruction, and disable bounds checking in  system and 
  trusted code.
 So, if you want to keep the assertions and contracts and 
 whatnot in,


 Unfortunately, if used pervasively, assertions and contracts 
 and whatnot may actually hog the speed of a program in a way 
 that breaks the deal.

 Disabling assertions (and whatnot), assuming assertions to be 
 true (and disabling whatnot) and leaving all assertions and 
 whatnot in are different trade-offs, of which assuming all 
 assertions to be true is the most dangerous one. Why hide this 
 behaviour in '-release'?


I'm afraid that I don't see the problem. If you want assertions 
left in in your release/production builds, then don't use 
-release. If you want them removed, then use -release. Are you 
objecting to the fact that the compiler can do further 
optimizations based on the fact that the assertions are presumed 
to be true if they're removed? I really don't see any problem 
with that. You're screwed regardless if the assertions would have 
failed. By their very nature, if an assertion would have failed, 
your program is an invalid state, so if you don't want to risk 
having code run that would have failed an assertion, then just 
don't compile with -release. And if you are willing to assume 
that the assertions won't fail and have them disabled in your 
release build, then you might as well gain any extra 
optimizations that can be made from assuming that the assertion 
is true. You're already making that assumption anyway and 
potentially letting your program enter an invalid state.


 just don't use -release and use -boundscheck=safe to get the 
 bounds
 checking changes that -release does.

 - Jonathan M Davis


 This leaves assertions and contracts in though.


Which is precisely what Ary was looking to do. He wanted to use 
-release but not have assertions removed, and using 
-boundscheck=safe instead of -release does that (though further 
posts by him seem to indicate that he doesn't want the  system 
bounds checking removed either, which means that he should just 
skip -release and all related flags entirely).

- Jonathan M Davis
Jul 31 2014
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 01 Aug 2014 01:07:55 -0400, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Thursday, 31 July 2014 at 20:49:18 UTC, Timon Gehr wrote:

 On 07/31/2014 09:37 PM, Jonathan M Davis wrote:

 disable contracts, turn assert(0) into a halt
 instruction, and disable bounds checking in  system and  trusted code.
 So, if you want to keep the assertions and contracts and whatnot in,


 Unfortunately, if used pervasively, assertions and contracts and  
 whatnot may actually hog the speed of a program in a way that breaks  
 the deal.

 Disabling assertions (and whatnot), assuming assertions to be true (and  
 disabling whatnot) and leaving all assertions and whatnot in are  
 different trade-offs, of which assuming all assertions to be true is  
 the most dangerous one. Why hide this behaviour in '-release'?


 I'm afraid that I don't see the problem. If you want assertions left in  
 in your release/production builds, then don't use -release. If you want  
 them removed, then use -release. Are you objecting to the fact that the  
 compiler can do further optimizations based on the fact that the  
 assertions are presumed to be true if they're removed? I really don't  
 see any problem with that. You're screwed regardless if the assertions  
 would have failed. By their very nature, if an assertion would have  
 failed, your program is an invalid state, so if you don't want to risk  
 having code run that would have failed an assertion, then just don't  
 compile with -release. And if you are willing to assume that the  
 assertions won't fail and have them disabled in your release build, then  
 you might as well gain any extra optimizations that can be made from  
 assuming that the assertion is true. You're already making that  
 assumption anyway and potentially letting your program enter an invalid  
 state.


This is not tenable. I realized when developing a library with classes,  
that not doing -release, means every virtual call is preceded and followed  
by a call to assert(obj), which calls Object.invariant -- a virtual call.  
Even if you don't define any invariant, it's still called (maybe final  
classes are better, but I'm not sure).

The cost for this is tremendous. You may as well not use classes.

-Steve
Sep 15 2014
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 16 September 2014 at 00:33:47 UTC, Steven 
Schveighoffer wrote:

 You may as well not use classes.


Always a good idea ;)
Sep 17 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 16 September 2014 at 00:33:47 UTC, Steven 
Schveighoffer wrote:

 The cost for this is tremendous. You may as well not use 
 classes.


Looks like ldc has a separate option to turn off invariants.
Sep 18 2014
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Thu, 18 Sep 2014 08:57:20 -0400, Kagamin <spam here.lot> wrote:


 On Tuesday, 16 September 2014 at 00:33:47 UTC, Steven Schveighoffer  
 wrote:

 The cost for this is tremendous. You may as well not use classes.


 Looks like ldc has a separate option to turn off invariants.


That's a good thing. I'm actually thinking this should be in DMD as well.  
invariants, always called by default, when almost no object defines one,  
is really costly. It's also a virtual call, which means any inlining is  
not possible.

-Steve
Sep 18 2014
prev sibling parent reply "eles" <eles215 gzk.dot> writes:
On Thursday, 31 July 2014 at 20:49:18 UTC, Timon Gehr wrote:

 On 07/31/2014 09:37 PM, Jonathan M Davis wrote:

 On Thursday, 31 July 2014 at 18:43:49 UTC, Ary Borenszweig 
 wrote:

 On 7/31/14, 4:37 AM, Walter Bright wrote:

 On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

 On 7/30/14, 7:01 PM, Walter Bright wrote:











 Disabling assertions (and whatnot), assuming assertions to be 
 true (and disabling whatnot) and leaving all assertions and 
 whatnot in are different trade-offs, of which assuming all 
 assertions to be true is the most dangerous one. Why hide this 
 behaviour in '-release'?


But assertions are *always* assumed to be true. The sole 
difference is that in Debug mode actual code generation for their 
check is disabled (not exactly the assertions, but code testing 
them).

The compiler makes no guarantee on the code that comes after an 
assert other than it will work correctly *if the assertion 
holds*. This is true for the code generated in both Debug and 
Release builds.

The difference is in the level of optimization (to play nice with 
gdb) and checks: in the Debug mode, that "assert-compelling 
generated code" never gets to actually execute as the program 
chocks at the assertion point (that is, just before entering the 
code).
Jul 31 2014
parent "eles" <eles215 gzk.dot> writes:
On Friday, 1 August 2014 at 05:27:48 UTC, eles wrote:

 On Thursday, 31 July 2014 at 20:49:18 UTC, Timon Gehr wrote:

 On 07/31/2014 09:37 PM, Jonathan M Davis wrote:

 On Thursday, 31 July 2014 at 18:43:49 UTC, Ary Borenszweig 
 wrote:

 On 7/31/14, 4:37 AM, Walter Bright wrote:

 On 7/30/2014 4:05 PM, Ary Borenszweig wrote:

 On 7/30/14, 7:01 PM, Walter Bright wrote:











 Disabling assertions (and whatnot), assuming assertions to be 
 true (and disabling whatnot) and leaving all assertions and 
 whatnot in are different trade-offs, of which assuming all 
 assertions to be true is the most dangerous one. Why hide this 
 behaviour in '-release'?


 But assertions are *always* assumed to be true. The sole 
 difference is that in Debug mode actual code generation for


*Release mode
Jul 31 2014
prev sibling next sibling parent reply =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
Walter Bright <newshound2 digitalmars.com> wrote: 

 2. The compiler can make use of assert expressions to improve
 optimization, even in -release mode.


I can see the benefits of that, but I consider it very dangerous.

It similar to undefined behavior in C/C++. There the 'assume/assert' is
implicit not explicit, but it's still the same effect.

If the assume/assert is hidden somewhere in a function you basically
introduce new traps for UB.

Initially I was strong proponent of such optimizations:
(a + a)/2 can be optimized to just a for signed integers, that's nice, the
classic example. This inserts an implicit assume(a < INT_MAX/2).

My opinion suddenly changed when I realized that such assumptions (explicit
or implicit) can also propagate up/backwards and leak into a bigger
context.
A wrong assumption can introduce bugs in seemingly unrelated parts of the
program that would actually be correct on their own.

With relatively 'dumb' compilers, this is not a big problem, but optimizers
are more and more clever and will take profit of such assumptions if they
can.

Tobi
Jul 30 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/30/14, 4:51 PM, Tobias Müller wrote:

 With relatively 'dumb' compilers, this is not a big problem, but optimizers
 are more and more clever and will take profit of such assumptions if they
 can.


That's true, and it seems like a growing trend. Relevant threads:

https://groups.google.com/a/isocpp.org/forum/#!topic/std-proposals/9S5jNRW-5wY

http://www.spinics.net/lists/gcchelp/msg41714.html

Recent versions of gcc and clang have become increasingly aggressive 
about optimizing code by taking advantage of making undefined behavior 
_really_ undefined. There's been a couple of posts in the news recently 
that I can't find at the moment.


Andrei
Jul 30 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/30/14, 5:29 PM, Andrei Alexandrescu wrote:

 On 7/30/14, 4:51 PM, Tobias Müller wrote:

 With relatively 'dumb' compilers, this is not a big problem, but
 optimizers
 are more and more clever and will take profit of such assumptions if they
 can.


 That's true, and it seems like a growing trend. Relevant threads:

 https://groups.google.com/a/isocpp.org/forum/#!topic/std-proposals/9S5jNRW-5wY


 http://www.spinics.net/lists/gcchelp/msg41714.html

 Recent versions of gcc and clang have become increasingly aggressive
 about optimizing code by taking advantage of making undefined behavior
 _really_ undefined. There's been a couple of posts in the news recently
 that I can't find at the moment.


I think I found it: http://www.redfelineninja.org.uk/daniel/?p=307

Andreu
Jul 30 2014
parent reply Johannes Pfau <nospam example.com> writes:
Am Wed, 30 Jul 2014 17:32:10 -0700
schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:


 On 7/30/14, 5:29 PM, Andrei Alexandrescu wrote:

 On 7/30/14, 4:51 PM, Tobias M=C3=BCller wrote:

 With relatively 'dumb' compilers, this is not a big problem, but
 optimizers
 are more and more clever and will take profit of such assumptions
 if they can.


 That's true, and it seems like a growing trend. Relevant threads:

 https://groups.google.com/a/isocpp.org/forum/#!topic/std-proposals/9S5j=




NRW-5wY

 http://www.spinics.net/lists/gcchelp/msg41714.html

 Recent versions of gcc and clang have become increasingly aggressive
 about optimizing code by taking advantage of making undefined
 behavior _really_ undefined. There's been a couple of posts in the
 news recently that I can't find at the moment.


=20
 I think I found it: http://www.redfelineninja.org.uk/daniel/?p=3D307
=20
 Andreu
=20


Also this:

Linus Torvalds On GCC 4.9: Pure & Utter Crap
http://www.phoronix.com/scan.php?page=3Dnews_item&px=3DMTc1MDQ

(This actually is a GCC bug, but valid behaviour for normal C++ code.
GCC only broke the compiler switch to explicitly force non-standard
behaviour)
Jul 31 2014
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 31.07.2014 15:32, schrieb Johannes Pfau:

 Am Wed, 30 Jul 2014 17:32:10 -0700
 schrieb Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>:


 On 7/30/14, 5:29 PM, Andrei Alexandrescu wrote:

 On 7/30/14, 4:51 PM, Tobias Müller wrote:

 With relatively 'dumb' compilers, this is not a big problem, but
 optimizers
 are more and more clever and will take profit of such assumptions
 if they can.


 That's true, and it seems like a growing trend. Relevant threads:

 https://groups.google.com/a/isocpp.org/forum/#!topic/std-proposals/9S5jNRW-5wY


 http://www.spinics.net/lists/gcchelp/msg41714.html

 Recent versions of gcc and clang have become increasingly aggressive
 about optimizing code by taking advantage of making undefined
 behavior _really_ undefined. There's been a couple of posts in the
 news recently that I can't find at the moment.


 I think I found it: http://www.redfelineninja.org.uk/daniel/?p=307

 Andreu


 Also this:

 Linus Torvalds On GCC 4.9: Pure & Utter Crap
 http://www.phoronix.com/scan.php?page=news_item&px=MTc1MDQ

 (This actually is a GCC bug, but valid behaviour for normal C++ code.
 GCC only broke the compiler switch to explicitly force non-standard
 behaviour)


And don't forget this (rather old) case: 
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
(I really don't get why anyone would want such an optimization: I want 
an optimizer to use clever inlining, use SSE etc where it makes sense 
and stuff like that - but not to remove code I wrote.)

Cheers,
Daniel
Jul 31 2014
next sibling parent reply Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/31/14 15:44, Daniel Gibson via Digitalmars-d wrote:

 And don't forget this (rather old) case:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
 (I really don't get why anyone would want such an optimization: I want an
optimizer to use clever inlining, use SSE etc where it makes sense and stuff
like that - but not to remove code I wrote.)


That is actually not a bug, but a perfectly valid optimization. The
compiler isn't clairvoyant and can not know that some data that you
wrote, but never read back, matters.

The solution is to tell the compiler that you really need that newly
(over-)written data. Eg

   asm {"" : : "m" (*cast(typeof(password[0])[9999999]*)password.ptr); }

(yes, stdizing compiler barriers would be a good idea)

artur
Jul 31 2014
next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 31.07.2014 17:26, schrieb Artur Skawina via Digitalmars-d:

 On 07/31/14 15:44, Daniel Gibson via Digitalmars-d wrote:

 And don't forget this (rather old) case:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
 (I really don't get why anyone would want such an optimization: I want an
optimizer to use clever inlining, use SSE etc where it makes sense and stuff
like that - but not to remove code I wrote.)


 That is actually not a bug, but a perfectly valid optimization. The
 compiler isn't clairvoyant and can not know that some data that you
 wrote, but never read back, matters.


I don't want the compiler to care about that. When I tell it to write 
something, I want it to do that, even if it might look like nonsense (if 
anything, it could create a warning).


 The solution is to tell the compiler that you really need that newly
 (over-)written data. Eg

     asm {"" : : "m" (*cast(typeof(password[0])[9999999]*)password.ptr); }


inline asm is not portable


 (yes, stdizing compiler barriers would be a good idea)


C11 defines a memset_s which is guaranteed not to be optimized away..
that's 9 years after that bugreport and will probably never be supported 
by MSVC (they don't even support C99).
One could write a memset_s oneself.. that does a memset, reads the data 
and writes a char of it or something to a global variable (hoping that 
the compiler won't optimize that to "just set that variable to 0").

The thing is: I don't want a compiler to remove code I wrote just 
because it "thinks" it's superfluous.
It could tell me about it as a warning, but it shouldn't just silently 
do it. If removing code makes my code faster, I can do it myself.

Cheers,
Daniel
Jul 31 2014
next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 31 July 2014 at 15:37:23 UTC, Daniel Gibson wrote:

 If removing code makes my code faster, I can do it myself.


No, in general you can't. Opportunities for dead code elimination 
often only present themselves after inlining and/or in generic 
(as in templates) code.

David
Jul 31 2014
prev sibling next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Daniel Gibson:


    asm {"" : : "m" 
 (*cast(typeof(password[0])[9999999]*)password.ptr); }


 inline asm is not portable


See:
https://d.puremagic.com/issues/show_bug.cgi?id=10661



 C11 defines a memset_s which is guaranteed not to be optimized 
 away..
 that's 9 years after that bugreport and will probably never be 
 supported by MSVC (they don't even support C99).


See:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366877%28v=vs.85%29.aspx

Bye,
bearophile
Jul 31 2014
prev sibling next sibling parent reply Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/31/14 17:37, Daniel Gibson via Digitalmars-d wrote:

 Am 31.07.2014 17:26, schrieb Artur Skawina via Digitalmars-d:

 On 07/31/14 15:44, Daniel Gibson via Digitalmars-d wrote:

 And don't forget this (rather old) case:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
 (I really don't get why anyone would want such an optimization: I want an
optimizer to use clever inlining, use SSE etc where it makes sense and stuff
like that - but not to remove code I wrote.)


 That is actually not a bug, but a perfectly valid optimization. The
 compiler isn't clairvoyant and can not know that some data that you
 wrote, but never read back, matters.


 
 I don't want the compiler to care about that. When I tell it to write
something, I want it to do that, even if it might look like nonsense (if
anything, it could create a warning).


This approach becomes completely impractical once generic code (templates)
enters the picture. But even for simple cases, it does not work. Keep in
mind that D initializes every object by default...



 The solution is to tell the compiler that you really need that newly
 (over-)written data. Eg

     asm {"" : : "m" (*cast(typeof(password[0])[9999999]*)password.ptr); }


 
 inline asm is not portable


That's why a portable compiler barrier interface is needed.
But this was just an example showing a zero-cost solution. A portable
fallback is always possible (the bug report was about C code -- there,
a loop that reads the data and stores a copy into a volatile location
would work).

artur
Jul 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 9:02 AM, Artur Skawina via Digitalmars-d wrote:

 The solution is to tell the compiler that you really need that newly
 (over-)written data. Eg

      asm {"" : : "m" (*cast(typeof(password[0])[9999999]*)password.ptr); }


 inline asm is not portable


 That's why a portable compiler barrier interface is needed.
 But this was just an example showing a zero-cost solution. A portable
 fallback is always possible (the bug report was about C code -- there,
 a loop that reads the data and stores a copy into a volatile location
 would work).


This is not a "barrier" operation. There you are thinking of atomic operations. 
This is a case of a "volatile" operation, and this supports it for D:

   https://github.com/D-Programming-Language/druntime/pull/892

Of course, someone has to actually pull it!
Jul 31 2014
parent Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 08/01/14 00:08, Walter Bright via Digitalmars-d wrote:

 On 7/31/2014 9:02 AM, Artur Skawina via Digitalmars-d wrote:

 The solution is to tell the compiler that you really need that newly
 (over-)written data. Eg

      asm {"" : : "m" (*cast(typeof(password[0])[9999999]*)password.ptr); }


 inline asm is not portable


 That's why a portable compiler barrier interface is needed.
 But this was just an example showing a zero-cost solution. A portable
 fallback is always possible (the bug report was about C code -- there,
 a loop that reads the data and stores a copy into a volatile location
 would work).


 
 This is not a "barrier" operation. There you are thinking of atomic
operations. This is a case of a "volatile" operation, and this supports it for
D:
 
   https://github.com/D-Programming-Language/druntime/pull/892


It's a _compiler_ barrier and has nothing to do with atomic ops or
volatile. It simply tells the compiler (in this case) 'i'm going
to read the data in the memory locations pointed to by the password.ptr'.
That means that the compiler has to make sure that the data is there,
before the `asm` executes; it can not assume that the stores are dead
and can not optimize them away. The actual (emitted) asm does nothing.
It's just a way to communicate to the compiler that the data is needed.
Since in this case the point was just to overwrite /other/ security
sensitive data present at this location, nothing else is necessary. We
don't actually care about the new content, we only pretend we do, so
that the compiler isn't able to optimize across this barrier.

Exposing compiler barriers in a portable way in D would definitively be
a good idea. Relatively decent implementations can be easily done, for
example, the above functionality can be achieved via a pure function
that takes a reference to a static array. The function would do nothing,
just immediately return to the caller; it'd just need to be opaque from
the optimizers POV. This version wouldn't be zero-cost, like the example
above, but still very cheap (usually just a call+ret sequence), correct
and enough for many not perf-sensitive use cases like the one described
in that bug report.

artur
Jul 31 2014
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/31/14, 8:37 AM, Daniel Gibson wrote:

 When I tell it to write something, I want it to do that, even if it
 might look like nonsense (if anything, it could create a warning).


I'm afraid those days are long gone by now. -- Andrei
Jul 31 2014
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 31.07.2014 18:29, schrieb Andrei Alexandrescu:

 On 7/31/14, 8:37 AM, Daniel Gibson wrote:

 When I tell it to write something, I want it to do that, even if it
 might look like nonsense (if anything, it could create a warning).


 I'm afraid those days are long gone by now. -- Andrei


At least for C..
It sucks not to be able to predict the behavior of sane-looking code 
without knowing the language standard in all details by heart.

I'd prefer if D  only did optimizations that are safe and don't change 
the behavior of the code and thus was less painful to use than C and C++ 
that need deep understanding of complicated standards to get right ("the 
last thing D needs is someone like Scott Meyers"?)

Examples for eliminations that I'd consider safe:
* if something is checked twice (e.g. due to inlining) it's okay to only 
keep the first check
* if a variable is certainly not read before the first assignment, the 
standard initialization could be optimized away.. same for multiple 
assignments without reads in between, *but*
  - that would still look strange in a debugger when you'd expect 
another value
  - don't we have int x = void; to prevent default initialization?
* turning multiplies into shifts and additions is totally fine if it 
does not change the behavior, even in corner cases.
* optimizing out variables

It's a major PITA to debug problems that only happen in release builds.

Cheers,
Daniel
Jul 31 2014
next sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 31 July 2014 at 17:40:45 UTC, Daniel Gibson wrote:

 Am 31.07.2014 18:29, schrieb Andrei Alexandrescu:

 On 7/31/14, 8:37 AM, Daniel Gibson wrote:

 When I tell it to write something, I want it to do that, even 
 if it
 might look like nonsense (if anything, it could create a 
 warning).


 I'm afraid those days are long gone by now. -- Andrei


 At least for C..
 It sucks not to be able to predict the behavior of sane-looking 
 code without knowing the language standard in all details by 
 heart.

 I'd prefer if D  only did optimizations that are safe and don't 
 change the behavior of the code and thus was less painful to 
 use than C and C++ that need deep understanding of complicated 
 standards to get right ("the last thing D needs is someone like 
 Scott Meyers"?)

 Examples for eliminations that I'd consider safe:
 * if something is checked twice (e.g. due to inlining) it's 
 okay to only keep the first check
 * if a variable is certainly not read before the first 
 assignment, the standard initialization could be optimized 
 away.. same for multiple assignments without reads in between, 
 *but*
  - that would still look strange in a debugger when you'd 
 expect another value
  - don't we have int x = void; to prevent default 
 initialization?
 * turning multiplies into shifts and additions is totally fine 
 if it does not change the behavior, even in corner cases.
 * optimizing out variables

 It's a major PITA to debug problems that only happen in release 
 builds.

 Cheers,
 Daniel


I believe gdc has the full suite of gcc optimiser flags 
available. You don't need to just slap -O3 on and then complain 
about the changes it makes, you can choose with a reasonable 
amount of precision which optimisations you do/don't want done.
Jul 31 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 10:57 AM, John Colvin wrote:

 I believe gdc has the full suite of gcc optimiser flags available. You don't
 need to just slap -O3 on and then complain about the changes it makes, you can
 choose with a reasonable amount of precision which optimisations you do/don't
 want done.


DMC++ has similar switches, but nobody uses them, and they are pretty much 
useless anyway, because:

1. you need to be a compiler guy to know what they mean - very few people know 
what "code hoisting" is

2. it's the combination of optimizations that produces results - they are not 
independent of each other

This is why I didn't include such switches in DMD.
Jul 31 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 10:40 AM, Daniel Gibson wrote:

 It's a major PITA to debug problems that only happen in release builds.


Debugging optimized code was a well known problem even back in the 70's. Nobody 
has solved it, and nobody wants unoptimized code.
Jul 31 2014
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 31.07.2014 23:59, schrieb Walter Bright:

 On 7/31/2014 10:40 AM, Daniel Gibson wrote:

 It's a major PITA to debug problems that only happen in release builds.


 Debugging optimized code was a well known problem even back in the 70's.
 Nobody has solved it, and nobody wants unoptimized code.


Yeah, and because of this I'd like optimizations not to cause different 
behavior if at all possible to keep these kind of bugs as low as possible.

And I agree with your stance on those fine-grained optimization switches 
from your other post. GCC currently has 191 flags the influence 
optimization[1] (+ a version that negates them for most), and I don't 
understand what most of them do, so it would be hard for me to decide 
which optimizations I want and which I don't want.

However, what about an extra flag for "unsafe" optimizations?
I'd like the compiler to do inlining, replacing int multiplications with 
powers of two with shifts and other "safe" optimizations that don't 
change the semantics of my program (see the examples in the post you 
quoted), but I *don't* want it to e.g. remove writes to memory that 
isn't read afterwards or make assumptions based on assertions (that are 
disabled in the current compile mode).

And maybe a warning mode that tells me about "dead"/"superfluous" code 
that would be eliminated in an optimized build so I can check if that 
would break anything for me in that respect without trying to understand 
the asm output would be helpful.

Cheers,
Daniel


[1] according to $ gcc-4.8 --help=optimizer | grep "^  -" | wc -l
Jul 31 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 3:21 PM, Daniel Gibson wrote:

 And I agree with your stance on those fine-grained optimization switches from
 your other post. GCC currently has 191 flags the influence optimization[1]


Just consider this from a testing standpoint. As I mentioned previously, 
optimizations interact with each other to produce emergent behavior. GCC has
191 
factorial different optimizers. Google's calculator puts 191! at infinity,
which 
it might as well be.



 However, what about an extra flag for "unsafe" optimizations?


There's been quite a creep of adding more and more flags. Each one of them is, 
in a way, a failure of design, and we are all too quick to reach for that.



 I *don't*
 want it to e.g. remove writes to memory that isn't read afterwards


That's what volatileStore() is for.



 or make assumptions based on assertions (that are disabled in the current
compile mode).


This is inexorably coming. If you cannot live with it, I suggest writing your 
own version of assert, using the Phobos 'enforce' implementation as a model. 
It'll do what you want.



 And maybe a warning mode that tells me about "dead"/"superfluous" code that
 would be eliminated in an optimized build so I can check if that would break
 anything for me in that respect without trying to understand the asm output
 would be helpful.


If you compile DMD with -D, and then run it with -O --c, it will present you 
with a list of all the data flow optimizations performed on the code. It's very 
useful for debugging the optimizer. Although I think you'll find it 
illuminating, you won't find it very useful - for one thing, a blizzard of info 
is generated.
Jul 31 2014
next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 31, 2014 at 06:12:11PM -0700, Walter Bright via Digitalmars-d wrote:

 On 7/31/2014 3:21 PM, Daniel Gibson wrote:

And I agree with your stance on those fine-grained optimization
switches from your other post. GCC currently has 191 flags the
influence optimization[1]


 
 Just consider this from a testing standpoint. As I mentioned
 previously, optimizations interact with each other to produce emergent
 behavior. GCC has 191 factorial different optimizers. Google's
 calculator puts 191! at infinity, which it might as well be.


[...]

It's actually a number with "only" 360+ digits. :-) There are far larger
numbers between it and infinity, but I get the point, it's impractically
huge, and certainly poorly covered by tests.


T

-- 
"Uhh, I'm still not here." -- KD, while "away" on ICQ.
Jul 31 2014
prev sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 01.08.2014 03:12, schrieb Walter Bright:

 On 7/31/2014 3:21 PM, Daniel Gibson wrote:

 And I agree with your stance on those fine-grained optimization
 switches from
 your other post. GCC currently has 191 flags the influence
 optimization[1]


 Just consider this from a testing standpoint. As I mentioned previously,
 optimizations interact with each other to produce emergent behavior. GCC
 has 191 factorial different optimizers. Google's calculator puts 191! at
 infinity, which it might as well be.


Yep, also a good point.
(Actually it's 187 -f* options, the rest is -O* which can't be combined 
of course and some of them most probably imply many of the -f* switches, 
but it'll still be an unmanageable/untestable amount of possible 
configurations)


 However, what about an extra flag for "unsafe" optimizations?


 There's been quite a creep of adding more and more flags. Each one of
 them is, in a way, a failure of design, and we are all too quick to
 reach for that.


Well, it would be one additional flag that would also imply -O ...
but this should probably be reevaluated when problems with 
overaggressive optimization actually occur.


 I *don't*
 want it to e.g. remove writes to memory that isn't read afterwards


 That's what volatileStore() is for.


Ah, sounds nice, having a standard way to do that is certainly a good 
thing! :-)
I looked at the pull request ( 
https://github.com/D-Programming-Language/druntime/pull/892 ) and I 
wonder if maybe an additional function that's equivalent to memset() and 
overwrites a whole range of memory could be added.
As memset() in C is usually much faster than iterating over the data and 
setting it to $value one by one, I guess this kind of function could 
provide a speedup here as well.


 or make assumptions based on assertions (that are disabled in the
 current compile mode).


 This is inexorably coming. If you cannot live with it, I suggest writing
 your own version of assert, using the Phobos 'enforce' implementation as
 a model. It'll do what you want.


Yeah. But please update the assert() documentation so people can know 
about this.
I also wonder if this might be commonly needed and could be a good 
addition to enforce() in phobos


 And maybe a warning mode that tells me about "dead"/"superfluous" code
 that
 would be eliminated in an optimized build so I can check if that would
 break
 anything for me in that respect without trying to understand the asm
 output
 would be helpful.


 If you compile DMD with -D, and then run it with -O --c, it will present
 you with a list of all the data flow optimizations performed on the
 code. It's very useful for debugging the optimizer. Although I think
 you'll find it illuminating, you won't find it very useful - for one
 thing, a blizzard of info is generated.


Sounds interesting, thanks for the hint :-)
This would probably be useful information that an IDE could (optionally) 
display alongside the actual code - I guess that could make debugging 
issues that only happen in optimized code/release mode much easier (if 
you can narrow your problem down to a few functions).

Cheers,
Daniel
Jul 31 2014
parent "Wyatt" <wyatt.epp gmail.com> writes:
On Friday, 1 August 2014 at 01:41:40 UTC, Daniel Gibson wrote:

 Yep, also a good point.
 (Actually it's 187 -f* options, the rest is -O* which can't be 
 combined of course and some of them most probably imply many of 
 the -f* switches, but it'll still be an unmanageable/untestable 
 amount of possible configurations)


Actually, it's much worse than that. ;)  All of the -O switches 
in GCC have a set of -f switches that they enable or disable.  
What they DON'T tell you up front is that set of switches also 
enables other optimisations. [0]  I encountered a situation a 
couple months ago where this actually mattered (i.e. runtime 
segfault) for one particular C file if you built it with levels 
above -O0, but enabling every single -f switch worked fine (still 
not sure why; solved by switching to Clang and it compiles and 
runs at all -O levels with no warnings).

-Wyatt

[0] 
https://gcc.gnu.org/wiki/FAQ#Is_-O1_.28-O2.2C-O3_or_-Os.29_equivalent_to_individual_-foptimization_options.3F
Aug 01 2014
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Thursday, 31 July 2014 at 22:21:46 UTC, Daniel Gibson wrote:

 Am 31.07.2014 23:59, schrieb Walter Bright:

 On 7/31/2014 10:40 AM, Daniel Gibson wrote:

 It's a major PITA to debug problems that only happen in 
 release builds.


 Debugging optimized code was a well known problem even back in 
 the 70's.
 Nobody has solved it, and nobody wants unoptimized code.


 Yeah, and because of this I'd like optimizations not to cause 
 different behavior if at all possible to keep these kind of 
 bugs as low as possible.

 And I agree with your stance on those fine-grained optimization 
 switches from your other post. GCC currently has 191 flags the 
 influence optimization[1] (+ a version that negates them for 
 most), and I don't understand what most of them do, so it would 
 be hard for me to decide which optimizations I want and which I 
 don't want.

 However, what about an extra flag for "unsafe" optimizations?
 I'd like the compiler to do inlining, replacing int 
 multiplications with powers of two with shifts and other "safe" 
 optimizations that don't change the semantics of my program 
 (see the examples in the post you quoted), but I *don't* want 
 it to e.g. remove writes to memory that isn't read afterwards 
 or make assumptions based on assertions (that are disabled in 
 the current compile mode).

 And maybe a warning mode that tells me about 
 "dead"/"superfluous" code that would be eliminated in an 
 optimized build so I can check if that would break anything for 
 me in that respect without trying to understand the asm output 
 would be helpful.


A compiler is a program that turns code in one programming 
language to equivalent machine code, according to a language 
specification. There are obviously many different equivalent 
machine code programs corresponding to any sufficiently complex 
higher-level program. Classifying them into optimized and 
unoptimized ones is rather arbitrary. The same goes for safe vs. 
unsafe optimizations.

To achieve what you want, the compiler would have to ignore the 
actual language specification and use a different one that is 
tweaked according to your criteria. I don't think this is 
desirable. If the official language specification has parts that 
can lead to the errors you want to avoid, then it's not the 
compiler's fault, and therefore the compiler should not be 
changed to workaround it. Instead, deficiencies in the 
specification should be fixed there.
Aug 01 2014
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 01.08.2014 12:03, schrieb "Marc Schütz" <schuetzm gmx.net>":

 A compiler is a program that turns code in one programming language to
 equivalent machine code, according to a language specification. There
 are obviously many different equivalent machine code programs
 corresponding to any sufficiently complex higher-level program.
 Classifying them into optimized and unoptimized ones is rather
 arbitrary. The same goes for safe vs. unsafe optimizations.

 To achieve what you want, the compiler would have to ignore the actual
 language specification and use a different one that is tweaked according
 to your criteria. I don't think this is desirable. If the official
 language specification has parts that can lead to the errors you want to
 avoid, then it's not the compiler's fault, and therefore the compiler
 should not be changed to workaround it. Instead, deficiencies in the
 specification should be fixed there.


I'd prefer language specifications *not* to include such parts.
C wouldn't be any worse without the "you can eliminate writes to code 
that's not read afterwards" part, for example.

So I wish D could resist adding such "dangerous"/unexpected things to 
the standard and only allow such optimizations in an "unsafe" 
optimization mode.

Cheers,
Daniel
Aug 01 2014
parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Daniel Gibson"  wrote in message news:lrg0k5$1nl1$1 digitalmars.com...


 I'd prefer language specifications *not* to include such parts.
 C wouldn't be any worse without the "you can eliminate writes to code 
 that's not read afterwards" part, for example.


Of course it would, that's why that's in there!!!

int *x = 0; // initialize just to be safe
...
x = ...;

You've just wasted a write.  Your program is now slower than it needs to be.

struct MyStruct s;
mystruct_init(&s);
s->field = something;
s->otherfield = somethingElse();

mystruct_init has to initialize all fields of s (to keep programmer suicide 
rates down) and yet you've just written over some of the values!  The 
compiler can clearly see that you never read from field and otherfield 
between the first write and the second.
Aug 01 2014
prev sibling next sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 31 July 2014 at 15:37:23 UTC, Daniel Gibson wrote:

 Am 31.07.2014 17:26, schrieb Artur Skawina via Digitalmars-d:

 On 07/31/14 15:44, Daniel Gibson via Digitalmars-d wrote:

 And don't forget this (rather old) case: 
 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
 (I really don't get why anyone would want such an 
 optimization: I want an optimizer to use clever inlining, use 
 SSE etc where it makes sense and stuff like that - but not to 
 remove code I wrote.)


 That is actually not a bug, but a perfectly valid 
 optimization. The
 compiler isn't clairvoyant and can not know that some data 
 that you
 wrote, but never read back, matters.


 I don't want the compiler to care about that. When I tell it to 
 write something, I want it to do that, even if it might look 
 like nonsense (if anything, it could create a warning).

 The thing is: I don't want a compiler to remove code I wrote 
 just because it "thinks" it's superfluous.


The idea that the compiler simply lowers your code to 
well-written assembly died decades ago. The job of an optimiser 
is to *not* use your code but instead to compile a program that 
is equivalent to yours but faster/smaller. What is equivalent is 
defined by the language spec.
Jul 31 2014
prev sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Daniel Gibson"  wrote in message news:lrdnri$2fge$1 digitalmars.com...


 One could write a memset_s oneself.. that does a memset, reads the data 
 and writes a char of it or something to a global variable (hoping that the 
 compiler won't optimize that to "just set that variable to 0").


Some compilers will do exactly that optimization.


 The thing is: I don't want a compiler to remove code I wrote just because 
 it "thinks" it's superfluous.
 It could tell me about it as a warning, but it shouldn't just silently do 
 it. If removing code makes my code faster, I can do it myself.


No you don't, no you can't.  You are using an optimizer because writing it 
in the perfectly precise, perfectly fast way makes your code unmaintainable. 
You want the optimizer to delete all those never-read initializations, drop 
all those temporary variables, and turn your multiplies into shifts and 
additions.

If you didn't, you wouldn't be using an optimizer.
Jul 31 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 10:26 AM, Daniel Murphy wrote:

 No you don't, no you can't.  You are using an optimizer because writing it in
 the perfectly precise, perfectly fast way makes your code unmaintainable. You
 want the optimizer to delete all those never-read initializations, drop all
 those temporary variables, and turn your multiplies into shifts and additions.

 If you didn't, you wouldn't be using an optimizer.


Even "unoptimized" code does optimizations.

If you really want unoptimized code, what you write is what you get, the inline 
assembler beckons!
Jul 31 2014
prev sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 31 July 2014 at 15:26:27 UTC, Artur Skawina via 
Digitalmars-d wrote:

 On 07/31/14 15:44, Daniel Gibson via Digitalmars-d wrote:

 And don't forget this (rather old) case: 
 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
 (I really don't get why anyone would want such an 
 optimization: I want an optimizer to use clever inlining, use 
 SSE etc where it makes sense and stuff like that - but not to 
 remove code I wrote.)


 That is actually not a bug, but a perfectly valid optimization. 
 The
 compiler isn't clairvoyant and can not know that some data that 
 you
 wrote, but never read back, matters.

 The solution is to tell the compiler that you really need that 
 newly
 (over-)written data. Eg

    asm {"" : : "m" 
 (*cast(typeof(password[0])[9999999]*)password.ptr); }

 (yes, stdizing compiler barriers would be a good idea)

 artur


Any idea how dead store removal interacts with the modern C(++) 
memory model? Another thread could hold a reference to the memory 
being written to.
Jul 31 2014
parent Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/31/14 19:22, John Colvin via Digitalmars-d wrote:

 On Thursday, 31 July 2014 at 15:26:27 UTC, Artur Skawina via Digitalmars-d
wrote:

 On 07/31/14 15:44, Daniel Gibson via Digitalmars-d wrote:

 And don't forget this (rather old) case:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
 (I really don't get why anyone would want such an optimization: I want an
optimizer to use clever inlining, use SSE etc where it makes sense and stuff
like that - but not to remove code I wrote.)


 That is actually not a bug, but a perfectly valid optimization. The
 compiler isn't clairvoyant and can not know that some data that you
 wrote, but never read back, matters.





 Any idea how dead store removal interacts with the modern C(++) memory model?
Another thread could hold a reference to the memory being written to.


In case of local/stack and TLS objects the compiler can often prove that
there are no other refs (eg because the address is never escaped).

artur
Jul 31 2014
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 31, 2014 at 03:44:35PM +0200, Daniel Gibson via Digitalmars-d wrote:
[...]

 And don't forget this (rather old) case:
 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
 (I really don't get why anyone would want such an optimization: I want
 an optimizer to use clever inlining, use SSE etc where it makes sense
 and stuff like that - but not to remove code I wrote.)


[...]

Modern compilers often have to deal with generated code (that isn't
directly written by the programmer, e.g., expanded from a C++ template
-- or, for that matter, generated by a code generator like lex / yacc).
In this case, you *do* want dead code removal because the code generator
may be written in a way that takes care of the general case, but in your
specific case some of the generated code is redundant.  You don't want
to penalize specific instances of the generic code pattern, after all.


T

-- 
The richest man is not he who has the most, but he who needs the least.
Jul 31 2014
parent "Andrew Godfrey" <X y.com> writes:
On Thursday, 31 July 2014 at 16:37:40 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Thu, Jul 31, 2014 at 03:44:35PM +0200, Daniel Gibson via 
 Digitalmars-d wrote:
 [...]

 And don't forget this (rather old) case:
 https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8537
 (I really don't get why anyone would want such an 
 optimization: I want
 an optimizer to use clever inlining, use SSE etc where it 
 makes sense
 and stuff like that - but not to remove code I wrote.)


 [...]

 Modern compilers often have to deal with generated code (that 
 isn't
 directly written by the programmer, e.g., expanded from a C++ 
 template
 -- or, for that matter, generated by a code generator like lex 
 / yacc).
 In this case, you *do* want dead code removal because the code 
 generator
 may be written in a way that takes care of the general case, 
 but in your
 specific case some of the generated code is redundant.  You 
 don't want
 to penalize specific instances of the generic code pattern, 
 after all.


Both points of view make sense. The problem is that it's hard for 
the compiler to know when the code it elides was generated code 
or explicitly written. (Maybe this is solvable in dmd,  I don't 
know. But it's not a feature I've seen before.)
Jul 31 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/30/2014 4:51 PM, Tobias Müller wrote:

 With relatively 'dumb' compilers, this is not a big problem, but optimizers
 are more and more clever and will take profit of such assumptions if they
 can.


If D wishes to be competitive, it must go down that path.

If you, as a user, do not wish this behavior, then do not use -release.

The documentation for -release says:

"compile release version, which means not generating code for contracts and 
asserts. Array bounds checking is not done for system and trusted functions."

https://dlang.org/dmd-windows.html
Jul 31 2014
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Thursday, 31 July 2014 at 07:47:51 UTC, Walter Bright wrote:

 On 7/30/2014 4:51 PM, Tobias Müller wrote:

 With relatively 'dumb' compilers, this is not a big problem, 
 but optimizers
 are more and more clever and will take profit of such 
 assumptions if they
 can.


 If D wishes to be competitive, it must go down that path.

 If you, as a user, do not wish this behavior, then do not use 
 -release.

 The documentation for -release says:

 "compile release version, which means not generating code for 
 contracts and asserts. Array bounds checking is not done for 
 system and trusted functions."

 https://dlang.org/dmd-windows.html


It's worth noting that ldc (and gdc maybe, can't remember) offer 
some finer control over what does/doesn't get eliminated, e.g. 
-enable-contracts -disable-asserts etc.
Jul 31 2014
prev sibling next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:

 I am not terribly good at writing formal legalese 
 specifications for this. I welcome PR's to improve the 
 specification along these lines, if you find any Aha! Gotcha! 
 issues in it. Of course, implementation errors for this in DMD 
 should be reported on bugzilla.


What is missing is not formal specification but clear guidelines 
"how to use this system in production". Right now it is pretty 
clear that you have implemented something that non-zero amount of 
experienced D developers have no clue how to use without botching 
the application completely. This does indicate that something is 
wrong with the feature even you are perfectly right theoretically.

Currently there is http://dlang.org/contracts.html but neither it 
nor any of referenced materials does explain to me:

- how to distribute binary library packages in presence of 
contracts
- how to organize your application to ensure that contracts can 
be removed in release builds
- are those even applicable to majority of applications

I am less concerned with just assert behavior because there are 
many ways to workaround it to get different semantics. But 
contract system... no clues.
Jul 30 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/30/14, 6:15 PM, Dicebot wrote:

 On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:

 I am not terribly good at writing formal legalese specifications for
 this. I welcome PR's to improve the specification along these lines,
 if you find any Aha! Gotcha! issues in it. Of course, implementation
 errors for this in DMD should be reported on bugzilla.


 What is missing is not formal specification but clear guidelines "how to
 use this system in production". Right now it is pretty clear that you
 have implemented something that non-zero amount of experienced D
 developers have no clue how to use without botching the application
 completely. This does indicate that something is wrong with the feature
 even you are perfectly right theoretically.

 Currently there is http://dlang.org/contracts.html but neither it nor
 any of referenced materials does explain to me:

 - how to distribute binary library packages in presence of contracts
 - how to organize your application to ensure that contracts can be
 removed in release builds
 - are those even applicable to majority of applications

 I am less concerned with just assert behavior because there are many
 ways to workaround it to get different semantics. But contract system...
 no clues.


It would be awesome if you (a) documented formally the current behavior 
and (b) submit bug reports wherever you find egregious faults in it. -- 
Andrei
Jul 30 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 31 July 2014 at 03:52:55 UTC, Andrei Alexandrescu 
wrote:

 It would be awesome if you (a) documented formally the current 
 behavior and (b) submit bug reports wherever you find egregious 
 faults in it. -- Andrei


It is documented at http://dlang.org/contracts.html
As for bug reports - how can I file those if I have no clue how 
system is supposed to be used? I don't think it is necessarily 
bad or buggy, just can't fin the way to make good use of it. Or 
do you expect "please teach me use contracts" bug report? :)
Jul 30 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/30/14, 9:01 PM, Dicebot wrote:

 On Thursday, 31 July 2014 at 03:52:55 UTC, Andrei Alexandrescu wrote:

 It would be awesome if you (a) documented formally the current
 behavior and (b) submit bug reports wherever you find egregious faults
 in it. -- Andrei


 It is documented at http://dlang.org/contracts.html
 As for bug reports - how can I file those if I have no clue how system
 is supposed to be used? I don't think it is necessarily bad or buggy,
 just can't fin the way to make good use of it. Or do you expect "please
 teach me use contracts" bug report? :)


There's this term "learned helplessness". You're a smart guy who can 
slice and dice things. -- Andrei
Jul 30 2014
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 31 July 2014 at 04:24:49 UTC, Andrei Alexandrescu 
wrote:

 There's this term "learned helplessness". You're a smart guy 
 who can slice and dice things. -- Andrei


This is one of those moments when I am not sure if you are 
trolling me yet again or being strangely serious. I am supposed 
to research and propose brand new approach to contract 
programming because I don't understand how existing one should 
work? And I am being told that by one of the authors of existing 
system?

You are taking that "go try yourself" thing way out of proportion.
Jul 30 2014
prev sibling next sibling parent reply "Tofu Ninja" <emmons0 purdue.edu> writes:
On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:


 3. Use of assert to validate input is utterly wrong and will 
 not be supported. Use such constructs at your own risk.


When exactly is it 'ok' to use assert then?

If asserts are not allowed to be used to verify inputs.... then
by extension, they can not be used to verify any derivative of an
input. So by that definition, asserts are only ok to use on any
thing known at compile time... that makes them utterly useless....
Jul 30 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 31, 2014 at 02:05:29AM +0000, Tofu Ninja via Digitalmars-d wrote:

 On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:
 

3. Use of assert to validate input is utterly wrong and will not be
supported. Use such constructs at your own risk.


 
 When exactly is it 'ok' to use assert then?
 
 If asserts are not allowed to be used to verify inputs.... then by
 extension, they can not be used to verify any derivative of an input.


No that doesn't make sense. The idea behind input sanitization is that
user-facing APIs receive arbitrary, unverified input from outside, and
before you use that data, you scrub it. After scrubbing it, it's
perfectly OK to use assert to verify its validity -- because if it's
still invalid, there's a bug in your scrubbing algorithm. The scrubbed
input *is* a derivative of the input, but you can't say that you can't
use assert on it!

(And if you *don't* scrub your data before using it, your design is
flawed and should be fixed.)



 So by that definition, asserts are only ok to use on any thing known
 at compile time... that makes them utterly useless....


Your definition is wrong. ;-) The idea behind asserts is that you're
verifying *program logic*, not checking arbitrary input data. If you
have an algorithm that computes a square root, for example, you'd use an
assert to verify that the square of the result equals the input --
because if not, that means something has gone wrong with your square
root algorithm. But you shouldn't use an assert to verify that the input
to the square root algorithm is a valid numerical string -- because the
user could have typed "abc" instead of a number. Rather, you should
scrub the user input and throw an exception when the input is invalid.

After scrubbing, however, it's perfectly valid to assert that the input
must be a valid number -- because if not, it means the *logic* in your
scrubbing algorithm is flawed (perhaps it missed a corner case).

It's true, however, that this simple idea is not always so simple in
practice. One has to draw a line between "user input" and "internal
state" somewhere, and it's not always obvious where that line falls. For
example, viewed as a whole, the entire software application may be
considered a system that takes input from outside, so "user input" means
things like keystrokes, mouse movements, etc.. But internally, the
system may consist of multiple components, and when data is passed
between components, should they be treated as "internal state" or "user
input"? Should library APIs treat application input as "external input"
and vet it before use, or is it OK to use assert to enforce(!) the
validity of data passed internally between the application's components?
It's not always easy to decide, and sometimes judgment calls have to be
made.


T

-- 
If creativity is stifled by rigid discipline, then it is not true creativity.
Jul 31 2014
parent reply "Tofu Ninja" <emmons0 purdue.edu> writes:
On Thursday, 31 July 2014 at 17:10:27 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Thu, Jul 31, 2014 at 02:05:29AM +0000, Tofu Ninja via 
 Digitalmars-d wrote:

 On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright 
 wrote:
 

3. Use of assert to validate input is utterly wrong and will 
not be
supported. Use such constructs at your own risk.


 
 When exactly is it 'ok' to use assert then?
 
 If asserts are not allowed to be used to verify inputs.... 
 then by
 extension, they can not be used to verify any derivative of an 
 input.


 No that doesn't make sense. The idea behind input sanitization 
 is that
 user-facing APIs receive arbitrary, unverified input from 
 outside, and
 before you use that data, you scrub it. After scrubbing it, it's
 perfectly OK to use assert to verify its validity -- because if 
 it's
 still invalid, there's a bug in your scrubbing algorithm. The 
 scrubbed
 input *is* a derivative of the input, but you can't say that 
 you can't
 use assert on it!

 (And if you *don't* scrub your data before using it, your 
 design is
 flawed and should be fixed.)



 So by that definition, asserts are only ok to use on any thing 
 known
 at compile time... that makes them utterly useless....


 Your definition is wrong. ;-) The idea behind asserts is that 
 you're
 verifying *program logic*, not checking arbitrary input data. 
 If you
 have an algorithm that computes a square root, for example, 
 you'd use an
 assert to verify that the square of the result equals the input 
 --
 because if not, that means something has gone wrong with your 
 square
 root algorithm. But you shouldn't use an assert to verify that 
 the input
 to the square root algorithm is a valid numerical string -- 
 because the
 user could have typed "abc" instead of a number. Rather, you 
 should
 scrub the user input and throw an exception when the input is 
 invalid.

 After scrubbing, however, it's perfectly valid to assert that 
 the input
 must be a valid number -- because if not, it means the *logic* 
 in your
 scrubbing algorithm is flawed (perhaps it missed a corner case).

 It's true, however, that this simple idea is not always so 
 simple in
 practice. One has to draw a line between "user input" and 
 "internal
 state" somewhere, and it's not always obvious where that line 
 falls. For
 example, viewed as a whole, the entire software application may 
 be
 considered a system that takes input from outside, so "user 
 input" means
 things like keystrokes, mouse movements, etc.. But internally, 
 the
 system may consist of multiple components, and when data is 
 passed
 between components, should they be treated as "internal state" 
 or "user
 input"? Should library APIs treat application input as 
 "external input"
 and vet it before use, or is it OK to use assert to enforce(!) 
 the
 validity of data passed internally between the application's 
 components?
 It's not always easy to decide, and sometimes judgment calls 
 have to be
 made.


 T


With that logic(and the proposed optimizations that this whole 
thing is about), weird stuff like this happens...

void foo(int x)
{
     if(x != 0) throw ...;
     assert(x == 0);
}

The if check could be removed because assert will be assumed to 
always be true in release... so x could never not equal 0.... the 
assert just nuked my scrubbing logic...
Jul 31 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Tofu Ninja"  wrote in message news:mhhtxjlrvtqhzztxidbe forum.dlang.org...


 With that logic(and the proposed optimizations that this whole thing is 
 about), weird stuff like this happens...

 void foo(int x)
 {
      if(x != 0) throw ...;
      assert(x == 0);
 }

 The if check could be removed because assert will be assumed to always be 
 true in release... so x could never not equal 0.... the assert just nuked 
 my scrubbing logic...


The if can't be removed - and it's fairly easy to see why.  In the control 
flow path that contains the assert, the compiler is _already_ sure that x == 
0.  The assert adds no new information.

The assumption the compiler can make is "if the program got to here, this 
condition must be true".  The qualification is extremely important.

The corner case is "assert(0)".  It means "if the program got to here, the 
impossible has happened."

So with this:

void foo(int x)
{
    if(x != 0) throw ...;
    assert(0);
}

the compiler doesn't have to bother checking x at all.
Jul 31 2014
parent reply Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 07/31/14 20:14, Daniel Murphy via Digitalmars-d wrote:

 "Tofu Ninja"  wrote in message news:mhhtxjlrvtqhzztxidbe forum.dlang.org...
 

 With that logic(and the proposed optimizations that this whole thing is
about), weird stuff like this happens...

 void foo(int x)
 {
      if(x != 0) throw ...;
      assert(x == 0);
 }

 The if check could be removed because assert will be assumed to always be true
in release... so x could never not equal 0.... the assert just nuked my
scrubbing logic...


 
 The if can't be removed - and it's fairly easy to see why.  In the control
flow path that contains the assert, the compiler is _already_ sure that x == 0.
 The assert adds no new information.


As long as the assert is 100% correct. If you have a hundred+ asserts
and a 1% error rate...
A wrong assert could (under the proposed model) propagate the wrong
assumptions both ways. Silently disabling other checks that would
have otherwise caught the error.

Imagine creating a hotfix for some newly discovered bug, and forgetting
to update an assert expression somewhere. Unless the problem is
triggered while testing a non-release build, you may end up shipping a
broken product, even one with bugs that were not present in the original.
Now imagine that somebody else will handle the next report. He/she will
look at the code, see absolutely no problems with it, all necessary
checks will be there... Figuring out that a) an assert is the cause, and
b) which one it is, will be a very interesting process... 



 The corner case is "assert(0)".  It means "if the program got to here, the
impossible has happened."


It's vaguely defined, overloaded, and not currently treated that way.
Arguably it could mean 'this path won't ever be reached, trust me', but
redefining it now is obviously not possible. (doing this would of course
make assert(0) extremely dangerous)

artur
Jul 31 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Artur Skawina via Digitalmars-d"  wrote in message 
news:mailman.324.1406835164.16021.digitalmars-d puremagic.com...


 The corner case is "assert(0)".  It means "if the program got to here, 
 the impossible


 has happened."

 It's vaguely defined, overloaded, and not currently treated that way.
 Arguably it could mean 'this path won't ever be reached, trust me', but
 redefining it now is obviously not possible. (doing this would of course
 make assert(0) extremely dangerous)


You should probably read the spec on assert.  The assert(0) case is already 
defined that way.
Jul 31 2014
parent Artur Skawina via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 08/01/14 04:33, Daniel Murphy via Digitalmars-d wrote:

 "Artur Skawina via Digitalmars-d"  wrote in message
news:mailman.324.1406835164.16021.digitalmars-d puremagic.com...
 

 The corner case is "assert(0)".  It means "if the program got to here, > the
impossible


 has happened."

 It's vaguely defined, overloaded, and not currently treated that way.
 Arguably it could mean 'this path won't ever be reached, trust me', but
 redefining it now is obviously not possible. (doing this would of course
 make assert(0) extremely dangerous)


 
 You should probably read the spec on assert.  The assert(0) case is already
defined that way.


No, that definition is contradictory ("vaguely defined" was an euphemism).
Requiring either AssertError or halting execution precludes treating assert(0)
as "unreachable code". Removing that requirement would change the meaning of
every existing (reachable) assert(0); all of them would then trigger undefined
behavior...

artur
Jul 31 2014
prev sibling next sibling parent "Andrew Godfrey" <X y.com> writes:
On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:

 2. The compiler can make use of assert expressions to improve 
 optimization, even in -release mode.


For the domain I'm currently working in - a
very large codebase (> 1 MLOC, C/C++) for an application program,
I have to echo what others said, and say I could not use such a 
feature.
I think I can add a reason (though what's been said about the 
'fuzzy middle'
between assertions and input validation, certainly rings true for 
me too).

If my asserts worked this way I would have to stop using them and 
build my own.
The reason is that, while I tend to assert only things that 
should be true,
this codebase is not well factored and so:
a) we tend to write a lot of assertions, and
b) occasionally we learn something from them (i.e. an assertion 
fires,
we go "huh", and our understanding of the codebase improves).

The point is that a priori, we can only guess whether a 
particular assertion
we're considering adding is really "this program is screwed if 
this condition
is true".

I don't lose sleep over this because it is safe to add our kind 
of assertions.
But if adding assertions could affect the optimizer's reasoning, 
then it would NOT be safe to add them, and we'd have to back way 
off. I'd be comfortable using such assertions only for very 
low-level components.

I can see the appeal of allowing the optimizer to do this, but I 
don't understand the idea of making that the default behavior. To 
me that's like array bounds-checking being off by default. And 
speaking of which,
this seems like a useful example:

Surely any program which oversteps the bounds of array, is 
incorrect?
It must have made some logic error (be it forgetting to validate 
inputs,
or some internal reasoning that was erroneous). So we should put 
asserts
on all our array accesses, asserting that they are within bounds!
So... then the optimizer can optimize away all the bounds checks. 
Releae
builds need no checks of any kind. Right? :)
I'm not trying to be as facetious as that sounds, I'm saying that 
your position seems to me to lead logically to the conclusion 
that array bounds-checking
should be off in release.
Jul 30 2014
prev sibling next sibling parent reply simendsjo <simendsjo gmail.com> writes:
On 07/31/2014 12:01 AM, Walter Bright wrote:
(...)

 2. The compiler can make use of assert expressions to improve
 optimization, even in -release mode.


(...)

Does this mean that assertions used for optimization will be left in
-release? There's plenty of times where I've had an old incorrect
assertion in my code after a refactoring - sometimes in seldom used paths.

If the compiler would aggressively optimize my code based on some wrong
assumptions I give it, it would be very useful if that assumption would
stay in the code to trigger an assertion.
Jul 31 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 12:01 AM, simendsjo wrote:

 On 07/31/2014 12:01 AM, Walter Bright wrote:
 (...)

 2. The compiler can make use of assert expressions to improve
 optimization, even in -release mode.


 (...)

 Does this mean that assertions used for optimization will be left in
 -release? There's plenty of times where I've had an old incorrect
 assertion in my code after a refactoring - sometimes in seldom used paths.

 If the compiler would aggressively optimize my code based on some wrong
 assumptions I give it, it would be very useful if that assumption would
 stay in the code to trigger an assertion.


To get this behavior, don't use the -release switch.
Jul 31 2014
prev sibling next sibling parent reply "David Bregman" <drb sfu.ca> writes:
On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:

 I'd like to sum up my position and intent on all this.

 1. I can discern no useful, practical difference between the 
 notions of assume and assert.


People have explained the difference repeatedly, a ridiculous 
number of times now. Could you please take a minute to understand 
it this time instead of flippantly dismissing it again?

assert does a runtime check, assume does not
assume affects code generation/optimization, assert does not
assert is for debugging, assume is not
assume is for optimization, assert is not

In terms of what they practically do, they have *nothing* in 
common, their functions are entirely orthogonal.

Still think there is no practical difference?


 2. The compiler can make use of assert expressions to improve 
 optimization, even in -release mode.


This will introduce a lot of undefined behavior, including making 
 safe code with asserts unsafe. I really think this needs to be 
acknowledged. As far as I can tell from the other thread, it 
still hasn't been.
Jul 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 12:40 AM, David Bregman wrote:

 On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:

 I'd like to sum up my position and intent on all this.

 1. I can discern no useful, practical difference between the notions of assume
 and assert.


 People have explained the difference repeatedly, a ridiculous number of times
 now. Could you please take a minute to understand it this time instead of
 flippantly dismissing it again?

 assert does a runtime check, assume does not
 assume affects code generation/optimization, assert does not
 assert is for debugging, assume is not
 assume is for optimization, assert is not

 In terms of what they practically do, they have *nothing* in common, their
 functions are entirely orthogonal.


They are inextricably entangled. Consider:

    if (x == 0) abort();   // essentially what assert(x) does
    ... at this point, the optimizer knows, beyond doubt, that x!=0 ...
    if (x)  // optimizer can remove this check
       ...

which has the behavior of assume as you listed above, yet it is assert. We can 
pretend assert doesn't affect code, like we can pretend to have massless points 
in physics class, but in reality points have a mass and assert most definitely 
affects code generation, and does in every compiler I've checked, and it
affects 
it in just the way the assume does.



 Still think there is no practical difference?


Yes.



 2. The compiler can make use of assert expressions to improve optimization,
 even in -release mode.


 This will introduce a lot of undefined behavior, including making  safe code
 with asserts unsafe. I really think this needs to be acknowledged. As far as I
 can tell from the other thread, it still hasn't been.


I did acknowledge it for the array bounds case.

Note that your assume() will have the same effect, and worse, there will be no 
option to have the compiler insert a check, because then it would be an
assert() 
and you might as well just use assert().

This is why I see the distinction as being pointless.
Jul 31 2014
next sibling parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
 In terms of what they practically do, they have *nothing* in 
 common, their
 functions are entirely orthogonal.


 They are inextricably entangled. Consider:

    if (x == 0) abort();   // essentially what assert(x) does
    ... at this point, the optimizer knows, beyond doubt, that 
 x!=0 ...
    if (x)  // optimizer can remove this check
       ...


As far as I unterstand, this would be the behaviour without 
-release. With -release the code becomes

if(x)
     ...

and the optimizer cannot remove the (second) check. Or am I 
missing something?
Jul 31 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 1:27 AM, Tobias Pankrath wrote:

 In terms of what they practically do, they have *nothing* in common, their
 functions are entirely orthogonal.


 They are inextricably entangled. Consider:

    if (x == 0) abort();   // essentially what assert(x) does
    ... at this point, the optimizer knows, beyond doubt, that x!=0 ...
    if (x)  // optimizer can remove this check
       ...


 As far as I unterstand, this would be the behaviour without -release. With
 -release the code becomes

 if(x)
      ...

 and the optimizer cannot remove the (second) check. Or am I missing something?


My intention is that the runtime check would be omitted, but the information 
would still be fed to the optimizer. This is not currently implemented.
Jul 31 2014
prev sibling parent reply "David Bregman" <drb sfu.ca> writes:
On Thursday, 31 July 2014 at 08:08:43 UTC, Walter Bright wrote:

 On 7/31/2014 12:40 AM, David Bregman wrote:

 On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright 
 wrote:

 I'd like to sum up my position and intent on all this.

 1. I can discern no useful, practical difference between the 
 notions of assume
 and assert.


 People have explained the difference repeatedly, a ridiculous 
 number of times
 now. Could you please take a minute to understand it this time 
 instead of
 flippantly dismissing it again?

 assert does a runtime check, assume does not
 assume affects code generation/optimization, assert does not
 assert is for debugging, assume is not
 assume is for optimization, assert is not

 In terms of what they practically do, they have *nothing* in 
 common, their
 functions are entirely orthogonal.


 They are inextricably entangled. Consider:

    if (x == 0) abort();   // essentially what assert(x) does
    ... at this point, the optimizer knows, beyond doubt, that 
 x!=0 ...
    if (x)  // optimizer can remove this check
       ...

 which has the behavior of assume as you listed above, yet it is 
 assert. We can pretend assert doesn't affect code, like we can 
 pretend to have massless points in physics class, but in 
 reality points have a mass and assert most definitely affects 
 code generation, and does in every compiler I've checked, and 
 it affects it in just the way the assume does.



 Still think there is no practical difference?


 Yes.


Sigh. Of course you can assume the condition after a runtime 
check has been inserted. You just showed that

assert(x); assume(x);

is semantically equivalent to
assert(x);

as long as the runtime check is not elided. (no -release)

You didn't show that assert and assume are the same, they are not.

The code generated by one will be different than the code 
generated by the other, that is because they are functionally 
different. This is really indisputable..


 2. The compiler can make use of assert expressions to improve 
 optimization,
 even in -release mode.


 This will introduce a lot of undefined behavior, including 
 making  safe code
 with asserts unsafe. I really think this needs to be 
 acknowledged. As far as I
 can tell from the other thread, it still hasn't been.


 I did acknowledge it for the array bounds case.


Ok, thanks! But you still want to assert to become assume in 
release mode? How will you handle the safety issue?


 Note that your assume() will have the same effect, and worse, 
 there will be no option to have the compiler insert a check, 
 because then it would be an assert() and you might as well just 
 use assert().


So what? I did not suggest to use assume() instead of assert() to 
avoid the problem. In fact, that _is_ the problem, _you_ are 
suggesting that assert becomes assume in release mode. assume() 
is not  safe, that is the whole point.
Jul 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 4:28 AM, David Bregman wrote:

 Sigh. Of course you can assume the condition after a runtime check has been
 inserted. You just showed that

 assert(x); assume(x);

 is semantically equivalent to
 assert(x);

 as long as the runtime check is not elided. (no -release)


No. I showed that you cannot have an assert without the assume. That makes them 
equivalent that direction.

For the other direction, adding in a runtime check for an assume is going to be 
expected of an implementation. And, in fact, since the runtime check won't 
change the semantics if the assume is correct, they are equivalent.

I.e. for practical purposes, they are the same thing. You can't have one
without 
the other.



 The code generated by one will be different than the code generated by the
 other, that is because they are functionally different. This is really
 indisputable..


Oh, I dispute it very much!



 But you still want to assert to become assume in release mode? How
 will you handle the safety issue?


I don't know yet.



 So what?


It came up in the thread about assume vs assert. I assumed (!) it mattered to
you.
Jul 31 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 08:58 PM, Walter Bright wrote:

 On 7/31/2014 4:28 AM, David Bregman wrote:

 Sigh. Of course you can assume the condition after a runtime check has
 been
 inserted. You just showed that

 assert(x); assume(x);

 is semantically equivalent to
 assert(x);

 as long as the runtime check is not elided. (no -release)


 No. I showed that you cannot have an assert without the assume.


No you did not. However:

* You showed that an additional 'assume' would not have any effect if 
the check is never elided.

* You showed that the state of knowledge about the program state of the 
optimizer are the same after processing a halting runtime check and 
after processing an 'assume'.

I don't think anybody is contesting that. Now try to zoom your focus out 
a little, and think about _what if_ the assertion and the assumption are 
actually wrong? Why does it make sense to conflate them in this case?


 That makes them equivalent that direction.

 For the other direction, adding in a runtime check for an assume is
 going to be expected of an implementation.


Yes if 'assert' does what 'assert' does now, and if 'assume' does what 
'assert' does now, then 'assert' and 'assume' do the same. I agree with 
that, but the premise is unrelated to this discussion. You are moving 
the goal posts.


 And, in fact, since the
 runtime check won't change the semantics if the assume is correct, they
 are equivalent.
 ...


"If the 'assume'/'assert' are correct" is not a sound assumption to 
make. You are not the compiler, you are the programmer. We are 
discussing _about_ programs, not _within_ programs.


 I.e. for practical purposes, they are the same thing.


All assertions being correct is not a given 'for practical purposes'. 
You are arguing in the context of a theoretical ideal and this context 
alone.
Jul 31 2014
prev sibling parent reply "David Bregman" <drb sfu.ca> writes:
On Thursday, 31 July 2014 at 18:58:11 UTC, Walter Bright wrote:

 On 7/31/2014 4:28 AM, David Bregman wrote:

 Sigh. Of course you can assume the condition after a runtime 
 check has been
 inserted. You just showed that

 assert(x); assume(x);

 is semantically equivalent to
 assert(x);

 as long as the runtime check is not elided. (no -release)


 No. I showed that you cannot have an assert without the assume. 
 That makes them equivalent that direction.


That is only true if assert always generates a runtime check. 
i.e. it is not true for C/C++ assert (and so far, D assert) in 
release mode.


 For the other direction, adding in a runtime check for an 
 assume is going to be expected of an implementation.


No. It is expected that assume does /not/ have a runtime check. 
Assume is used to help the compiler optimize based on trusted 
facts, doing a runtime check could easily defeat the purpose of 
such micro optimizations.


 And, in fact, since the runtime check won't change the 
 semantics if the assume is correct, they are equivalent.


Right, only "if the assume is correct". So they aren't equivalent 
if it isn't correct.

Q.E.D. ?


 But you still want to assert to become assume in release mode? 
 How
 will you handle the safety issue?


 I don't know yet.


I would think the easiest way is to just not inject the 
assumption when inside  safe code, but I don't know anything 
about the compiler internals.

Even for  system code, I'm on the fence about whether asserts 
should affect codegen in release, it doesn't seem like a clear 
tradeoff to make: safety vs some dubious optimization gains. Do 
we really want to go down the same road as C with undefined 
behavior?

I would need to think about it more, but if D adopted that route, 
I would at least feel like I need to be much more careful with 
asserts, so I'm not accidentally making my code more buggy 
instead of less. I think it warrants discussion, anyways.
Jul 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 3:07 PM, David Bregman wrote:

 On Thursday, 31 July 2014 at 18:58:11 UTC, Walter Bright wrote:

 On 7/31/2014 4:28 AM, David Bregman wrote:

 Sigh. Of course you can assume the condition after a runtime check has been
 inserted. You just showed that

 assert(x); assume(x);

 is semantically equivalent to
 assert(x);

 as long as the runtime check is not elided. (no -release)


 No. I showed that you cannot have an assert without the assume. That makes
 them equivalent that direction.


 That is only true if assert always generates a runtime check. i.e. it is not
 true for C/C++ assert (and so far, D assert) in release mode.


 For the other direction, adding in a runtime check for an assume is going to
 be expected of an implementation.


 No. It is expected that assume does /not/ have a runtime check. Assume is used
 to help the compiler optimize based on trusted facts, doing a runtime check
 could easily defeat the purpose of such micro optimizations.


I'm rather astonished you'd take that position. It opens a huge door wide for 
undefined behavior, and no obvious way of verifying that the assume() is
correct.

I'm confident that if D introduced such behavior, the very first comment would 
be "I need it to insert a runtime check on demand."



 And, in fact, since the runtime check won't change the semantics if the assume
 is correct, they are equivalent.


 Right, only "if the assume is correct". So they aren't equivalent if it isn't
 correct.

 Q.E.D. ?


I'm not buying those uncheckable semantics as being workable and practical.



 But you still want to assert to become assume in release mode? How
 will you handle the safety issue?


 I don't know yet.


 I would think the easiest way is to just not inject the assumption when inside
  safe code, but I don't know anything about the compiler internals.

 Even for  system code, I'm on the fence about whether asserts should affect
 codegen in release, it doesn't seem like a clear tradeoff to make: safety vs
 some dubious optimization gains.


So why do you want assume() with no checking whatsoever? Does anybody want
that? 
Why are we even discussing such a misfeature?


 Do we really want to go down the same road as C
 with undefined behavior?


So you don't want assume()? Who does?
Jul 31 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Jul 31, 2014 at 06:19:59PM -0700, Walter Bright via Digitalmars-d wrote:

 On 7/31/2014 3:07 PM, David Bregman wrote:


[...]

I would think the easiest way is to just not inject the assumption
when inside  safe code, but I don't know anything about the compiler
internals.

Even for  system code, I'm on the fence about whether asserts should
affect codegen in release, it doesn't seem like a clear tradeoff to
make: safety vs some dubious optimization gains.


 
 So why do you want assume() with no checking whatsoever? Does anybody
 want that? Why are we even discussing such a misfeature?


[...]

Yikes. That sounds *really* scary. If assume() doesn't insert any
checks, and yet the compiler's optimizer takes it as truth, it leads to
horrible consequences like:

	int add(int x, int y) {
		assume(x + y == x - y);
		return x + y;  // what does this do?!
	}

At least, if assume() inserts checks, blatantly ridiculous things like
the above will quickly and frequently cause runtime aborts, instead of
directing the optimizer to do obviously wrong things that are
untraceable from the actual code. But if we do that, then assume()
starts to sound more and more like assert()...


T

-- 
Bomb technician: If I'm running, try to keep up.
Jul 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 6:37 PM, H. S. Teoh via Digitalmars-d wrote:

 But if we do that, then assume() starts to sound more and more like assert()...


I see that my posts are starting to work :-)
Jul 31 2014
next sibling parent reply "Tofu Ninja" <emmons0 purdue.edu> writes:
On Friday, 1 August 2014 at 04:36:13 UTC, Walter Bright wrote:

 On 7/31/2014 6:37 PM, H. S. Teoh via Digitalmars-d wrote:

 But if we do that, then assume() starts to sound more and more 
 like assert()...


 I see that my posts are starting to work :-)


I don't think we are arguing for the addition of assume, we are
arguing about the redefinition of assert to act like assume in
-release.
Jul 31 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/01/2014 06:42 AM, Tofu Ninja wrote:

 On Friday, 1 August 2014 at 04:36:13 UTC, Walter Bright wrote:

 On 7/31/2014 6:37 PM, H. S. Teoh via Digitalmars-d wrote:

 But if we do that, then assume() starts to sound more and more like
 assert()...


 I see that my posts are starting to work :-)


 I don't think we are arguing for the addition of assume, we are
 arguing about the redefinition of assert to act like assume in
 -release.


Yes, I think what he fails to appreciate is that he is arguing for the 
introduction of 'assume' in -release mode as a stealth operation by 
redefining the conventional meaning of 'assert', and that arguing 
against the introduction of 'assume' because it is redundant with the 
new 'assert' is not relevant.
Aug 01 2014
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/01/2014 06:36 AM, Walter Bright wrote:

 On 7/31/2014 6:37 PM, H. S. Teoh via Digitalmars-d wrote:

 But if we do that, then assume() starts to sound more and more like
 assert()...


 I see that my posts are starting to work :-)


No, you are suffering from confirmation bias.
Aug 01 2014
prev sibling next sibling parent reply "Chris Cain" <zshazz gmail.com> writes:
On Friday, 1 August 2014 at 01:20:04 UTC, Walter Bright wrote:

 I'm rather astonished you'd take that position. It opens a huge 
 door wide for undefined behavior, and no obvious way of 
 verifying that the assume() is correct.

 I'm confident that if D introduced such behavior, the very 
 first comment would be "I need it to insert a runtime check on 
 demand."


If no one else has said it, I'll be the first one to say it now:

If `assume` is implemented, the only sensible thing is to have it 
verify the assumption under most circumstances and remove the 
check (but keep the assumption for the optimizer to use) under 
release code.

Boom, done. Sorry guys, but it's the truth, the idea of `assume` 
is worthless and dangerous if there doesn't exist any way for it 
to actually check the assumption when you're trying to debug your 
code. Walter is absolutely right on that.

Though, I'm gonna say that `assert` doing all the things you're 
suggesting it should do is a bit iffy in ` safe` code. That much 
has also been shown pretty strongly as well. `assert` is 
essentially an ` trusted` concept that can cause bad things to 
happen in ` safe` code, but there's no real way to show the user 
that. Updating the documentation on it probably won't cut it 
alone. Maybe assert should be made illegal in ` safe` code or 
there should be some way to mark it that it's basically trusted 
to be correct (`trusted_assert`? .. or enable ` trusted` blocks 
of code and make `assert` effectively ` system`? The second would 
be a better feature, IMO ... more widely useful, like Rust's 
`unsafe` blocks).

I think this conversation has shown a bit of a weakness in using 
`assert` as currently spec'd in ` safe` code, so we probably 
should come up with some proposals on how to effectively fix that 
(make it obvious that bad `assert`s will break your ` safe` code).
Jul 31 2014
parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Friday, 1 August 2014 at 03:45:55 UTC, Chris Cain wrote:

 If `assume` is implemented, the only sensible thing is to have 
 it verify the assumption under most circumstances and remove 
 the check (but keep the assumption for the optimizer to use) 
 under release code.

 Boom, done. Sorry guys, but it's the truth, the idea of 
 `assume` is worthless and dangerous if there doesn't exist any 
 way for it to actually check the assumption when you're trying 
 to debug your code. Walter is absolutely right on that.


I think you might actually be arguing for our side of the
argument but don't realize it.

Assume is an inherently unsafe and dangerous idea. Some have
pointed out that assume has potential usefulness in a VERY
limited set of cases, but I don't think anyone is arguing that is
not a dangerous thing.

Personally I think any form of assume sounds like a bad idea and
a very dangerous thing, that is why I (and others) are against
the idea of making assert act like assume in -release. It's not
that we are arguing for the addition of assume, it's that we
don't want assert to act like assume in -release. Our reaction to
the idea of making assert act like assume in -release is because
of the fact that assume IS so dangerous.

I don't think anyone is actually arguing for the addition of
assume.
Jul 31 2014
prev sibling parent reply "David Bregman" <drb sfu.ca> writes:
On Friday, 1 August 2014 at 01:20:04 UTC, Walter Bright wrote:

 On 7/31/2014 3:07 PM, David Bregman wrote:

 On Thursday, 31 July 2014 at 18:58:11 UTC, Walter Bright wrote:

 On 7/31/2014 4:28 AM, David Bregman wrote:

 Sigh. Of course you can assume the condition after a runtime 
 check has been
 inserted. You just showed that

 assert(x); assume(x);

 is semantically equivalent to
 assert(x);

 as long as the runtime check is not elided. (no -release)


 No. I showed that you cannot have an assert without the 
 assume. That makes
 them equivalent that direction.


 That is only true if assert always generates a runtime check. 
 i.e. it is not
 true for C/C++ assert (and so far, D assert) in release mode.


 For the other direction, adding in a runtime check for an 
 assume is going to
 be expected of an implementation.


 No. It is expected that assume does /not/ have a runtime 
 check. Assume is used
 to help the compiler optimize based on trusted facts, doing a 
 runtime check
 could easily defeat the purpose of such micro optimizations.


 I'm rather astonished you'd take that position. It opens a huge 
 door wide for undefined behavior, and no obvious way of 
 verifying that the assume() is correct.


It's not a "position". I'm just giving you the definition of 
assume so you can understand the difference from assert.


 I'm confident that if D introduced such behavior, the very 
 first comment would be "I need it to insert a runtime check on 
 demand."



 And, in fact, since the runtime check won't change the 
 semantics if the assume
 is correct, they are equivalent.


 Right, only "if the assume is correct". So they aren't 
 equivalent if it isn't
 correct.

 Q.E.D. ?


 I'm not buying those uncheckable semantics as being workable 
 and practical.


I don't know what this means. Are you saying that you refuse to 
admit the difference between assert and assume because you 
exclude the possibility of the expression being false (buggy)?


 But you still want to assert to become assume in release 
 mode? How
 will you handle the safety issue?


 I don't know yet.


 I would think the easiest way is to just not inject the 
 assumption when inside
  safe code, but I don't know anything about the compiler 
 internals.

 Even for  system code, I'm on the fence about whether asserts 
 should affect
 codegen in release, it doesn't seem like a clear tradeoff to 
 make: safety vs
 some dubious optimization gains.


 So why do you want assume() with no checking whatsoever? Does 
 anybody want that? Why are we even discussing such a misfeature?


Ever since my first reply, I've only been trying to
a) help you to understand the difference between the two, and
b) to highlight the safety issue with assume (and the proposal 
for assert to become assume in release mode).

Whether I want assume or not is beside the point. If you've 
interpreted my posts as some kind of lobbying for the addition of 
assume() to D, that is not the case. If you want my opinion on 
that, I think it would be a good thing to have assume and assert 
separate, for safety and clarity reasons. However the importance 
I would assign to adding assume() is low because it is only 
useful for very specific micro optimizations, and is always a 
tradeoff with its unsafety. In C, I typically use it only a 
couple times in a large code base, compared to assert which I use 
all over the place.


 Do we really want to go down the same road as C
 with undefined behavior?


 So you don't want assume()? Who does?


I do want assume to some extent (see above), but it wasn't my 
point.
Aug 01 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/1/14, 12:40 AM, David Bregman wrote:

 It's not a "position". I'm just giving you the definition of assume so
 you can understand the difference from assert.


After reading your posts I still can't understand what your definition 
of "assume" is. Here's what I found:


 assert:
 is a runtime check of the condition.
 is a debugging/correctness checking feature.
 is used when the expression is believed true, but is not proven so.
 (if it was proven, then there is no purpose in asserting it with a redundant
runtime check that is guaranteed to never activate.)

 assume:
 passes a hint to the optimizer to allow better code generation.
 is used when the expression is proven to be true (by the programmer, like
 trusted).


There are a few corrections needed for "assert", i.e. "is a runtime 
check of the condition in debug mode". The whole "believed true but not 
proven so" is... odd, seeing as assert takes expressions that are 
considered tautological within the design, and sometimes provable 
automatically (e.g. after inlining).

Anyhow, if "assume" is to be taken at face value the its semantics has 
always been what Walter intended for "assert". (Again "proven to be 
true" is an eyebrow raiser because when one thinks of "proof" of 
semantics of programs one thinks of state analysis or progress and 
preservation and such.)

You may dislike what Walter wanted assert to be, but really this has 
been it from the beginning. Back in the day when I joined him I 
questioned the validity of making "assert" a keyword. He explained that 
he wanted it to be magic in the same way he discusses in this thread.


Andrei
Aug 01 2014
next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 01.08.2014 16:58, schrieb Andrei Alexandrescu:

 You may dislike what Walter wanted assert to be, but really this has
 been it from the beginning. Back in the day when I joined him I
 questioned the validity of making "assert" a keyword. He explained that
 he wanted it to be magic in the same way he discusses in this thread.


I'm a bit surprised that back then your reaction was not "well, that's a 
neat idea, but people must know about it, so let's make it explicit in 
the documentation".

This seems to be the main problem here: people assumed that assert() 
behaves like in in C, where it's defined to be a noop when they're 
deactivated, which pretty much forbids using them for optimizations.
So they wanted an assume() that gives the compiler hints about what it 
can assume a variable to be. Of course it would be desirable if in debug 
builds assume() would imply assert() so one can find out if those 
assumptions are not met while testing.
With that (C style) mindset, one would not be surprised if the program 
behaves in an undefined way if an assume() condition is not met at 
runtime - but one is very much surprised about undefined behavior when a 
(deactivated!) assert() condition is not met.

If assert() would have been documented or even advertised as "can be 
used for optimizations by compilers in release mode" from day one, this 
discussion wouldn't have started or at least would have been over very soon.

Cheers,
Daniel
Aug 01 2014
next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Daniel Gibson"  wrote in message news:lrgcei$211u$1 digitalmars.com...


 I'm a bit surprised that back then your reaction was not "well, that's a 
 neat idea, but people must know about it, so let's make it explicit in the 
 documentation".


Haha, I think back then there were much more serious issues with D, like 
abundant segfaults and a development team of ~2.


 If assert() would have been documented or even advertised as "can be used 
 for optimizations by compilers in release mode" from day one, this 
 discussion wouldn't have started or at least would have been over very 
 soon.


I expect that even if it had been documented, people would have completely 
ignored it, and would still be arguing for the exact same positions.
Aug 01 2014
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 01.08.2014 18:47, schrieb Daniel Murphy:

 "Daniel Gibson"  wrote in message news:lrgcei$211u$1 digitalmars.com...


 I'm a bit surprised that back then your reaction was not "well, that's
 a neat idea, but people must know about it, so let's make it explicit
 in the documentation".


 Haha, I think back then there were much more serious issues with D, like
 abundant segfaults and a development team of ~2.


It's not like adding two sentences describing this would take forever.


 If assert() would have been documented or even advertised as "can be
 used for optimizations by compilers in release mode" from day one,
 this discussion wouldn't have started or at least would have been over
 very soon.


 I expect that even if it had been documented, people would have
 completely ignored it, and would still be arguing for the exact same
 positions.


They could be told "look, it's written in the language spec (and has 
been for a long time), deal with it".
Maybe there still would be /some/ discussion, but certainly not as much 
as we're currently seeing - and it could be ignored of a good reason.

Cheers,
Daniel
Aug 01 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 01, 2014 at 06:58:16PM +0200, Daniel Gibson via Digitalmars-d wrote:

 Lines: 29
 
 Am 01.08.2014 18:47, schrieb Daniel Murphy:

"Daniel Gibson"  wrote in message news:lrgcei$211u$1 digitalmars.com...


I'm a bit surprised that back then your reaction was not "well,
that's a neat idea, but people must know about it, so let's make it
explicit in the documentation".


Haha, I think back then there were much more serious issues with D,
like abundant segfaults and a development team of ~2.


 
 It's not like adding two sentences describing this would take forever.


Seriously, this thread has gone on wayyyyy too long with no real work
being done. So I decided to actually do something about it.

	https://github.com/D-Programming-Language/dlang.org/pull/624

There, now the deed is done. Now let the destruction proceed. :-P


T

-- 
Being able to learn is a great learning; being able to unlearn is a greater
learning.
Aug 01 2014
prev sibling parent reply "Sebastiaan Koppe" <mail skoppe.eu> writes:
If assertions are disabled in release builds, and you 
specifically instruct the compiler to build one, are you not 
assuming that the assertions will hold?

Then what is wrong with extending those assumptions to the 
optimizer?

Unless the assertions trigger in debug build, you will not end up 
with bugs in the release.
Aug 01 2014
next sibling parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 8/1/14, 2:19 PM, Sebastiaan Koppe wrote:

 If assertions are disabled in release builds, and you specifically
 instruct the compiler to build one, are you not assuming that the
 assertions will hold?

 Then what is wrong with extending those assumptions to the optimizer?

 Unless the assertions trigger in debug build, you will not end up with
 bugs in the release.


If assertions don't trigger in debug build then assertions don't trigger 
in release.

That's false.
Aug 01 2014
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/01/2014 07:19 PM, Sebastiaan Koppe wrote:

 If assertions are disabled in release builds, and you specifically
 instruct the compiler to build one, are you not assuming that the
 assertions will hold?
 ...


It would often be foolish to simply assume so, and disabling of the 
assertions can be motivated differently (and not all motivations are 
valid; _they don't need to be_.)


 Then what is wrong with extending those assumptions to the optimizer?
 ...


To appreciate this better, simply imagine you are the guy who needs to 
track down the bug (which you don't reproduce locally, there can be many 
reasons for that) but who is not responsible for it.


 Unless the assertions trigger in debug build, you will not end up with
 bugs in the release.


The debug and the release build may be subjected to different input and 
hence traverse different traces of abstract states. It is not valid to 
say that an assertion will never fail just because it hasn't failed yet.
Aug 01 2014
parent reply "eles" <eles215 gzk.dot> writes:
On Friday, 1 August 2014 at 17:43:27 UTC, Timon Gehr wrote:

 On 08/01/2014 07:19 PM, Sebastiaan Koppe wrote:



 The debug and the release build may be subjected to different 
 input and hence traverse different traces of abstract states. 
 It is not valid to say that an assertion will never fail just 
 because it hasn't failed yet.


Yes, but is the same for the C apps. There, you have no assertion 
in the release build, the release build is optimized (I imagine 
very few would use -O0 on it...), then the sefault happens.

Good luck with the debugger and find the bug in the source 
code....

This is why debug builds exist, to reproduce problems and to 
investigate the bugs.
Aug 01 2014
next sibling parent reply "Tofu Ninja" <emmons0 purdue.edu> writes:
On Friday, 1 August 2014 at 20:16:29 UTC, eles wrote:


 Yes, but is the same for the C apps. There, you have no 
 assertion in the release build, the release build is optimized 
 (I imagine very few would use -O0 on it...), then the sefault 
 happens.


In c the assert is just a check, no assume, the Microsoft 
compiler even has its own __assume separate from assert.
Aug 01 2014
parent reply "eles" <eles215 gzk.dot> writes:
On Friday, 1 August 2014 at 20:22:39 UTC, Tofu Ninja wrote:

 On Friday, 1 August 2014 at 20:16:29 UTC, eles wrote:


 Yes, but is the same for the C apps. There, you have no 
 assertion in the release build, the release build is optimized 
 (I imagine very few would use -O0 on it...), then the sefault 
 happens.


 In c the assert is just a check, no assume, the Microsoft 
 compiler even has its own __assume separate from assert.


Because the assert is in the library, not in the language. The 
compiler can make only limited use of it.
Aug 01 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 07:56 AM, eles wrote:

 On Friday, 1 August 2014 at 20:22:39 UTC, Tofu Ninja wrote:

 On Friday, 1 August 2014 at 20:16:29 UTC, eles wrote:


 Yes, but is the same for the C apps. There, you have no assertion in
 the release build, the release build is optimized (I imagine very few
 would use -O0 on it...), then the sefault happens.


 In c the assert is just a check, no assume, the Microsoft compiler
 even has its own __assume separate from assert.


 Because the assert is in the library, not in the language. The compiler
 can make only limited use of it.


It's part of the language standard. The compiler can make as much use of 
it as possible while still conforming to the spec.
Aug 01 2014
parent reply "eles" <eles215 gzk.dot> writes:
On Saturday, 2 August 2014 at 06:01:52 UTC, Timon Gehr wrote:

 On 08/02/2014 07:56 AM, eles wrote:

 On Friday, 1 August 2014 at 20:22:39 UTC, Tofu Ninja wrote:

 On Friday, 1 August 2014 at 20:16:29 UTC, eles wrote:







 It's part of the language standard. The compiler can make as 
 much use of it as possible while still conforming to the spec.


If you put it like that, then you could put *everything* into the 
library. Classes were once defined as a layer over C and they 
could be defined as standard. Still, the concept went into the 
compiler.

What will the optimizer do if you don't link against the standard 
library?
Aug 01 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 08:04 AM, eles wrote:

 On Saturday, 2 August 2014 at 06:01:52 UTC, Timon Gehr wrote:

 On 08/02/2014 07:56 AM, eles wrote:

 On Friday, 1 August 2014 at 20:22:39 UTC, Tofu Ninja wrote:

 On Friday, 1 August 2014 at 20:16:29 UTC, eles wrote:







 It's part of the language standard. The compiler can make as much use
 of it as possible while still conforming to the spec.


 If you put it like that, then you could put *everything* into the
 library. Classes were once defined as a layer over C and they could be
 defined as standard. Still, the concept went into the compiler.

 What will the optimizer do if you don't link against the standard library?


IIRC there are bits of the standard that allow the optimizer to assume 
that you will, but don't take my word for it.
Aug 01 2014
prev sibling next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 01.08.2014 22:16, schrieb eles:

 On Friday, 1 August 2014 at 17:43:27 UTC, Timon Gehr wrote:

 On 08/01/2014 07:19 PM, Sebastiaan Koppe wrote:



 The debug and the release build may be subjected to different input
 and hence traverse different traces of abstract states. It is not
 valid to say that an assertion will never fail just because it hasn't
 failed yet.


 Yes, but is the same for the C apps. There, you have no assertion in the
 release build, the release build is optimized (I imagine very few would
 use -O0 on it...), then the sefault happens.


But there checks are not optimized away because of assert.
assert(x != NULL);
if(x != NULL) { ... }
in C the if check won't be optimized away in NDEBUG builds, in 
equivalent D code it would, because the assert would make the compiler 
assume that x will never be NULL at that point.

Cheers,
Daniel
Aug 01 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 1 August 2014 at 20:30:19 UTC, Daniel Gibson wrote:

 Am 01.08.2014 22:16, schrieb eles:

 On Friday, 1 August 2014 at 17:43:27 UTC, Timon Gehr wrote:

 On 08/01/2014 07:19 PM, Sebastiaan Koppe wrote:



 The debug and the release build may be subjected to different 
 input
 and hence traverse different traces of abstract states. It is 
 not
 valid to say that an assertion will never fail just because 
 it hasn't
 failed yet.


 Yes, but is the same for the C apps. There, you have no 
 assertion in the
 release build, the release build is optimized (I imagine very 
 few would
 use -O0 on it...), then the sefault happens.


 But there checks are not optimized away because of assert.
 assert(x != NULL);
 if(x != NULL) { ... }
 in C the if check won't be optimized away in NDEBUG builds, in 
 equivalent D code it would, because the assert would make the 
 compiler assume that x will never be NULL at that point.


And why is that a problem? By definition, if an assertion fails, 
your code is in an invalid state, and by compiling out 
assertions, you're basically assuming that they all pass, so 
you're code's already screwed if the assertion would have failed 
had it been compiled in. I don't see how having the compiler then 
use that assertion for optimizations really costs you anything. 
Worst case, it just makes already invalid code more invalid. 
You're screwed regardless if the assertion would have failed. And 
if it would have succeeded, then you just got an efficiency boost 
thanks to the assertion.

Thinking about it, I'm actually wondering if I should use 
assertions _more_ so that the compiler might be able to do better 
optimizations in -release. The extra cost in non-release builds 
could be worth that extra boost in -release, and as far as 
correctness goes, it never hurts to have more assertions.

- Jonathan M Davis
Aug 01 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/01/2014 11:50 PM, Jonathan M Davis wrote:

 ... as far as correctness goes, it never hurts to have more assertions.


Pipe dream.
Aug 01 2014
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 2:50 PM, Jonathan M Davis wrote:

 Thinking about it, I'm actually wondering if I should use assertions _more_ so
 that the compiler might be able to do better optimizations in -release.


I think this will be a productive strategy once this gets implemented in
optimizers.

Of course, using them effectively will not be easy unless one is willing to 
understand how data flow analysis works and is willing to examine the generated 
code.
Aug 01 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 01, 2014 at 03:34:00PM -0700, Walter Bright via Digitalmars-d wrote:

 On 8/1/2014 2:50 PM, Jonathan M Davis wrote:

Thinking about it, I'm actually wondering if I should use assertions
_more_ so that the compiler might be able to do better optimizations
in -release.


 
 I think this will be a productive strategy once this gets implemented
 in optimizers.
 
 Of course, using them effectively will not be easy unless one is
 willing to understand how data flow analysis works and is willing to
 examine the generated code.


Actually, I'm thinking of ways of extending this even further, such that
asserts can become the vehicle for user-defined types to declare
high-level optimizations.

For example, a matrix type could declare that certain matrix expressions
are equivalent, thereby allowing the optimizer to substitute one for the
other to simplify matrix expressions where it couldn't before, because
it doesn't know how to infer such equivalences when overloaded operators
may have arbitrary code in their implementation.

I haven't thought about the syntax for this yet, but the idea is
something like this:

	struct Matrix {
		// Implement overloaded operators here
		Matrix opBinary(string op)(Matrix m) { ... }

		Matrix transpose() { ... }

		// (THIS IS TENTATIVE, HYPOTHETICAL SYNTAX) Declare
		// identities the optimizer might use for Matrix
		// expressions
		invariant(Matrix a, Matrix b, Matrix c) {
			// Tell optimizer that Matrix obeys
			// distributivity law
			assert(a*b + a*c == a*(b+c));

			// Tell optimizer about multiplicative identity
			assert(!a.isIdentity || a*b == b);

			// Tell optimizer about idempotence of
			// .transpose
			assert(a.transpose().transpose() == a);
		}
	}

	void main() {
		Matrix x, y, z;
		auto r1 = x*y + x*z; // gets simplified into w = x*(y+z)

		auto I = Matrix.identity();
		assert(I.isIdentity);
		auto r2 = I*x; // gets simplified to r2 = x

		auto r3 = x.transpose;
		auto r4 = r3.transpose; // gets simplified to r4 = x
	}

I think this will be a very powerful feature to further narrow the gap
between built-in types vs. library-defined types.


T

-- 
Never trust an operating system you don't have source for! -- Martin Schulze
Aug 01 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 01:03 AM, H. S. Teoh via Digitalmars-d wrote:

 Actually, I'm thinking of ways of extending this even further, ...


Note that the official stance has been that such things are obviously 
ridiculous if not checked dynamically in non-release mode.
Aug 01 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Aug 02, 2014 at 01:47:42AM +0200, Timon Gehr via Digitalmars-d wrote:

 On 08/02/2014 01:03 AM, H. S. Teoh via Digitalmars-d wrote:

Actually, I'm thinking of ways of extending this even further, ...


 
 Note that the official stance has been that such things are obviously
 ridiculous if not checked dynamically in non-release mode.


Nothing stops the compiler in non-release mode from computing the
expression both ways and comparing them to see if they match.


T

-- 
He who does not appreciate the beauty of language is not worthy to bemoan its
flaws.
Aug 01 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 01:52 AM, H. S. Teoh via Digitalmars-d wrote:

 On Sat, Aug 02, 2014 at 01:47:42AM +0200, Timon Gehr via Digitalmars-d wrote:

 On 08/02/2014 01:03 AM, H. S. Teoh via Digitalmars-d wrote:

 Actually, I'm thinking of ways of extending this even further, ...


 Note that the official stance has been that such things are obviously
 ridiculous if not checked dynamically in non-release mode.


 Nothing stops the compiler in non-release mode from computing the
 expression both ways and comparing them to see if they match.


 T


Side effects. Multiplication of the runtime in non-release builds.

Btw, I don't think one can use == to express those rewrites.
Aug 01 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Aug 02, 2014 at 02:04:22AM +0200, Timon Gehr via Digitalmars-d wrote:

 On 08/02/2014 01:52 AM, H. S. Teoh via Digitalmars-d wrote:

On Sat, Aug 02, 2014 at 01:47:42AM +0200, Timon Gehr via Digitalmars-d wrote:

On 08/02/2014 01:03 AM, H. S. Teoh via Digitalmars-d wrote:

Actually, I'm thinking of ways of extending this even further, ...


Note that the official stance has been that such things are
obviously ridiculous if not checked dynamically in non-release mode.


Nothing stops the compiler in non-release mode from computing the
expression both ways and comparing them to see if they match.


T


 
 Side effects. Multiplication of the runtime in non-release builds.


True. I haven't really thought through how to address all those details
yet. The final design may not quite behave exactly the way I described.
But it's a direction I'd like to go in, because I think there's a lot of
potential here.



 Btw, I don't think one can use == to express those rewrites.


It's just hypothetical syntax.  The point is that it's a hinting system
to provide extra reduction rules to the optimizer that is impractical
for the compiler to automatically deduce.


T

-- 
Claiming that your operating system is the best in the world because
more people use it is like saying McDonalds makes the best food in the
world. -- Carl B. Constantine
Aug 01 2014
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 1 August 2014 at 22:34:02 UTC, Walter Bright wrote:

 On 8/1/2014 2:50 PM, Jonathan M Davis wrote:

 Thinking about it, I'm actually wondering if I should use 
 assertions _more_ so
 that the compiler might be able to do better optimizations in 
 -release.


 I think this will be a productive strategy once this gets 
 implemented in optimizers.

 Of course, using them effectively will not be easy unless one 
 is willing to understand how data flow analysis works and is 
 willing to examine the generated code.


True, but as long as the runtime cost isn't too great for your 
debug builds, having assertions for whatever conditions you think 
your code relies on and would ideally be checked is quite 
beneficial. Any optimizations by the compiler would then be a 
bonus, but the fact that they exist does offset the costs to 
debug builds somewhat (depending on what you're doing) by 
improving the release builds. I would think that any attempt to 
specifically craft assertions to aid the optimizer would be 
something that should generally be left to the cases where you 
really care about getting extra speed out of your program and are 
doing micro-optimizations based on profiling and the generated 
assembly and whatnot.

- Jonathan M Davis
Aug 01 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 4:12 PM, Jonathan M Davis wrote:

 True, but as long as the runtime cost isn't too great for your debug builds,
 having assertions for whatever conditions you think your code relies on and
 would ideally be checked is quite beneficial. Any optimizations by the compiler
 would then be a bonus, but the fact that they exist does offset the costs to
 debug builds somewhat (depending on what you're doing) by improving the release
 builds. I would think that any attempt to specifically craft assertions to aid
 the optimizer would be something that should generally be left to the cases
 where you really care about getting extra speed out of your program and are
 doing micro-optimizations based on profiling and the generated assembly and
 whatnot.


The optimizations thus enabled would be the same as in, say, an if body:

     if (expr) {
        ... take advantage of expr being true ...
     }

and optimizers already do that to an increasing extent.
Aug 01 2014
prev sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 1 August 2014 at 21:50:59 UTC, Jonathan M Davis wrote:

 On Friday, 1 August 2014 at 20:30:19 UTC, Daniel Gibson wrote:

 Am 01.08.2014 22:16, schrieb eles:

 On Friday, 1 August 2014 at 17:43:27 UTC, Timon Gehr wrote:

 On 08/01/2014 07:19 PM, Sebastiaan Koppe wrote:



 The debug and the release build may be subjected to 
 different input
 and hence traverse different traces of abstract states. It 
 is not
 valid to say that an assertion will never fail just because 
 it hasn't
 failed yet.


 Yes, but is the same for the C apps. There, you have no 
 assertion in the
 release build, the release build is optimized (I imagine very 
 few would
 use -O0 on it...), then the sefault happens.


 But there checks are not optimized away because of assert.
 assert(x != NULL);
 if(x != NULL) { ... }
 in C the if check won't be optimized away in NDEBUG builds, in 
 equivalent D code it would, because the assert would make the 
 compiler assume that x will never be NULL at that point.


 And why is that a problem? By definition, if an assertion 
 fails, your code is in an invalid state,


Only in an ideal world. In practice, the condition in the 
assertion could itself be incorrect. It could be a leftover after 
a refactoring, for instance.


 and by compiling out assertions, you're basically assuming that 
 they all pass, so you're code's already screwed if the 
 assertion would have failed had it been compiled in. I don't 
 see how having the compiler then use that assertion for 
 optimizations really costs you anything. Worst case, it just 
 makes already invalid code more invalid. You're screwed 
 regardless if the assertion would have failed. And if it would 
 have succeeded, then you just got an efficiency boost thanks to 
 the assertion.

 Thinking about it, I'm actually wondering if I should use 
 assertions _more_ so that the compiler might be able to do 
 better optimizations in -release. The extra cost in non-release 
 builds could be worth that extra boost in -release, and as far 
 as correctness goes, it never hurts to have more assertions.

 - Jonathan M Davis
Aug 02 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Saturday, 2 August 2014 at 09:46:57 UTC, Marc Schütz wrote:

 On Friday, 1 August 2014 at 21:50:59 UTC, Jonathan M Davis 
 wrote:

 And why is that a problem? By definition, if an assertion 
 fails, your code is in an invalid state,


 Only in an ideal world. In practice, the condition in the 
 assertion could itself be incorrect. It could be a leftover 
 after a refactoring, for instance.


Then it's a bug, and bugs make programs do wrong and invalid 
things. I certainly don't see any reason to not optimize based on 
assertions just in case someone screwed up their assertion any 
more than I see a reason to avoid optimizing based on something 
that's wrong in normal code.

- Jonathan M Davis
Aug 02 2014
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Sunday, 3 August 2014 at 02:27:16 UTC, Jonathan M Davis wrote:

 On Saturday, 2 August 2014 at 09:46:57 UTC, Marc Schütz wrote:

 On Friday, 1 August 2014 at 21:50:59 UTC, Jonathan M Davis 
 wrote:

 And why is that a problem? By definition, if an assertion 
 fails, your code is in an invalid state,


 Only in an ideal world. In practice, the condition in the 
 assertion could itself be incorrect. It could be a leftover 
 after a refactoring, for instance.


 Then it's a bug, and bugs make programs do wrong and invalid 
 things. I certainly don't see any reason to not optimize based 
 on assertions just in case someone screwed up their assertion 
 any more than I see a reason to avoid optimizing based on 
 something that's wrong in normal code.


Yes, it's a bug. The purpose of asserts is to detect these kinds 
of bugs, not to make it harder to detect them, and their effects 
worse.
Aug 03 2014
prev sibling parent Ary Borenszweig <ary esperanto.org.ar> writes:
On 8/1/14, 5:16 PM, eles wrote:

 On Friday, 1 August 2014 at 17:43:27 UTC, Timon Gehr wrote:

 On 08/01/2014 07:19 PM, Sebastiaan Koppe wrote:



 The debug and the release build may be subjected to different input
 and hence traverse different traces of abstract states. It is not
 valid to say that an assertion will never fail just because it hasn't
 failed yet.


 Yes, but is the same for the C apps. There, you have no assertion in the
 release build, the release build is optimized (I imagine very few would
 use -O0 on it...), then the sefault happens.

 Good luck with the debugger and find the bug in the source code....

 This is why debug builds exist, to reproduce problems and to investigate
 the bugs.


The problem is then trying to copy everything C and C++ does and putting 
it in D...
Aug 01 2014
prev sibling parent "eles" <eles215 gzk.dot> writes:
On Friday, 1 August 2014 at 17:19:09 UTC, Sebastiaan Koppe wrote:

 If assertions are disabled in release builds, and you 
 specifically instruct the compiler to build one, are you not 
 assuming that the assertions will hold?

 Then what is wrong with extending those assumptions to the 
 optimizer?

 Unless the assertions trigger in debug build, you will not end 
 up with bugs in the release.


To support: it seems to me that assert() is nothing more than an 
assume() with a check.

Why the check bothers you if the assumption is still true? 
Compiler just like to do overwork in debug builds, let's grant 
that pleasure to it.
Aug 01 2014
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/01/2014 04:58 PM, Andrei Alexandrescu wrote:

 assume:
 passes a hint to the optimizer to allow better code generation.
 is used when the expression is proven to be true (by the programmer,
 like  trusted).


 There are a few corrections needed for "assert", i.e. "is a runtime
 check of the condition in debug mode". The whole "believed true but not
 proven so" is... odd, seeing as assert takes expressions that are
 considered tautological within the design,


Maybe _somebody_ does not know what the entire design actually is, or 
what all its implications are.


 and sometimes provable
 automatically (e.g. after inlining).
 ...


Sometimes.


 Anyhow, if "assume" is to be taken at face value the its semantics has
 always been what Walter intended for "assert".


Even then, such a semantics is non-standard and almost nobody else knew.
Why break 'assert' now, now that it actually behaves as I and many 
others expect (even some of those who argued for the (apparently, even 
inofficially) new and opposite design)?


 (Again "proven to be
 true" is an eyebrow raiser because when one thinks of "proof" of
 semantics of programs


I think he was using a very relaxed notion of proof, and he exemplified 
that by drawing an analogy to  trusted.


 one thinks of state analysis or progress and
 preservation and such.)


Progress and preservation are about soundness of the type system of the 
programming language, not program correctness.  safe may be seen to 
strive for progress and preservation conditional on it holding for 
 trusted functions. Your position has always been that  safe and 
 trusted represent a useful distinction, and this case is analogous (if 
different!).
Aug 01 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/1/14, 9:25 AM, Timon Gehr wrote:

 Even then, such a semantics is non-standard and almost nobody else knew.


This notion of "standard" has been occasionally mentioned in this 
discussion. I agree that D's assert is different from traditional C and 
C++ assert, and I also agree that might surprise some, but I think the 
meaning of D's assert is well within the intent of the larger notion.


 Why break 'assert' now, now that it actually behaves as I and many
 others expect (even some of those who argued for the (apparently, even
 inofficially) new and opposite design)?


I don't see any breakage of "assert", more like realizing more of its 
latent potential. Clearly the documentation could be better, which is 
something we should focus on.

I do agree there's stuff that some may find unexpected, such as:

assert(x > 42);
if (x <= 42) {
   // let me also handle this conservatively
   ...
} else {
   // all good, proceed
   ...
}

The D optimizer might actually deem the entire "then" path unreachable 
in the future, and only compile in the "else" path. Some may find that 
surprising. For my money I never write code like this, and I consider it 
incorrect if I review it. You either assert something, or you check it 
dynamically.

I don't remember having to ding anyone in a code review at Facebook for 
something like that in almost five years of tenure, and we use assert 
all over the place.

Yesterday I changed a bunch of "assert" in hhvm 
(https://github.com/facebook/hhvm) with "BSSERT" having the following 
definition:

#ifdef NDEBUG
#define BSSERT(e) do { if (e) {} else __builtin_unreachable(); } while (0)
#else
#define BSSERT(e) do { assert(e); } while (0)
#endif

That's a >2MLOC project. I was hoping I'd measure a slight improvement 
based on the hints to gcc's optimizer. Tests passed but alas there was 
an 1.4% CPU time increase (which at our scale we consider a large 
performance regression); not sure what's causing it (it does sometimes 
happen that certain gcc optimization end up generating larger code which 
spills the I-Cache more often, something gcc's optimizer is not very 
good at controlling).

Better, clearer language definition and documentation is the real cure 
for this particular matter. Sure enough we need to introduce any future 
optimizations with due care and diligence; that's a given. We also have 
a bunch of issues very important and very urgent to tend to, starting 
with finalizing the new release. Getting into this holier-than-thou 
contest, dinging people for using "exponential" casually, or 
discouraging them to express approval with the language leader is not 
only a waste of time, it's a net negative for everyone involved. It 
makes us look bad among ourselves and also to the larger community.


Andrei
Aug 01 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/01/2014 07:39 PM, Andrei Alexandrescu wrote:

...


I did what I though was right. Do what you like. I'm off.
Aug 01 2014
parent reply "Tofu Ninja" <emmons0 purdue.edu> writes:
On Friday, 1 August 2014 at 17:53:03 UTC, Timon Gehr wrote:

 On 08/01/2014 07:39 PM, Andrei Alexandrescu wrote:

...


 I did what I though was right. Do what you like. I'm off.


I feel you, this whole thread is extremely frustrating.

Having your augment consistently dismissed by "I see no 
difference" or "That is misuse" is frustrating to say the least.
Aug 01 2014
next sibling parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Friday, 1 August 2014 at 19:47:35 UTC, Tofu Ninja wrote:

 Having your augment consistently dismissed by "I see no


*argument not augment :/
Aug 01 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 12:47 PM, Tofu Ninja wrote:

 Having your augment consistently dismissed by "I see no difference" or "That is
 misuse" is frustrating to say the least.


I repeatedly gave detailed rationales for that.
Aug 01 2014
parent reply "David Bregman" <drb sfu.ca> writes:
On Friday, 1 August 2014 at 20:57:29 UTC, Walter Bright wrote:

 On 8/1/2014 12:47 PM, Tofu Ninja wrote:

 Having your augment consistently dismissed by "I see no 
 difference" or "That is
 misuse" is frustrating to say the least.




Indeed.


 I repeatedly gave detailed rationales for that.


Unfortunately these "detailed rationales" consisted mostly of 
attacking straw men, and some other unsound arguments :(

My last reply to you is still unanswered if you want to give it 
another try. I feel like it's getting pretty childish though, so 
I'm also very close to giving up on this topic. So much 
back-and-forth we can't even get past definitions of assert and 
assume, or even agree that there is a difference between the 
two... never mind discussing the real issues here. And there 
definitely are real issues with your proposal to make -release 
transform all asserts into assumes.
Aug 01 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 4:03 PM, David Bregman wrote:

 Unfortunately these "detailed rationales" consisted mostly of attacking straw
 men, and some other unsound arguments :(


Of course, we always believe the other side's arguments are of that nature when 
we find them unconvincing.

To quote my favorite Hamilton Burger line: "irrelevant, immaterial, and 
incompetent!" is a fair thing to say. But saying "Your honor, he did not
present 
a case" is not fair.



 My last reply to you is still unanswered if you want to give it another try.


I didn't because a reply would just be another cut-and-paste of what I've 
already posted. I don't have anything new to say.
Aug 01 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 02:18 AM, Walter Bright wrote:

 On 8/1/2014 4:03 PM, David Bregman wrote:

 Unfortunately these "detailed rationales" consisted mostly of
 attacking straw
 men, and some other unsound arguments :(


 Of course, we always believe the other side's arguments are of that
 nature when we find them unconvincing.


No. No. No.
Aug 01 2014
prev sibling parent reply "David Bregman" <drb sfu.ca> writes:
On Saturday, 2 August 2014 at 00:18:30 UTC, Walter Bright wrote:

 On 8/1/2014 4:03 PM, David Bregman wrote:

 Unfortunately these "detailed rationales" consisted mostly of 
 attacking straw
 men, and some other unsound arguments :(


 Of course, we always believe the other side's arguments are of 
 that nature when we find them unconvincing.


Actually I find most unconvincing arguments originate from 
different axioms/values or different subjective evaluations of 
tradeoffs, rather than unsound logic. Inability to reach 
agreement on purely logical argument such as 
equivalence/non-equivalence of two concepts is pretty rare in 
this kind of setting (tech discussion between people trained in 
logic). Frankly I'm kinda shocked that this hasn't been resolved 
yet after so much back and forth.


 To quote my favorite Hamilton Burger line: "irrelevant, 
 immaterial, and incompetent!" is a fair thing to say. But 
 saying "Your honor, he did not present a case" is not fair.



 My last reply to you is still unanswered if you want to give 
 it another try.


 I didn't because a reply would just be another cut-and-paste of 
 what I've already posted. I don't have anything new to say.


OK, I think I have an idea how to be more convincing (I wish I'd 
thought of this earlier):

is this
http://www.cplusplus.com/reference/cassert/assert/

the same as this?
http://msdn.microsoft.com/en-us/library/1b3fsfxw.aspx

can you see the difference now?
Aug 01 2014
next sibling parent Daniel Gibson <metalcaedes gmail.com> writes:
Am 02.08.2014 04:13, schrieb David Bregman:

 is this
 http://www.cplusplus.com/reference/cassert/assert/

 the same as this?
 http://msdn.microsoft.com/en-us/library/1b3fsfxw.aspx

 can you see the difference now?


Oh, interesting quote from the __assume() link:
"Use __assume in an ASSERT only when the assert is not recoverable. Do 
not use __assume in an assert for which you have subsequent error 
recovery code because the compiler might optimize away the 
error-handling code."

So maybe it's not so unusual to handle an error that is checked with 
assert() after all.

Cheers,
Daniel
Aug 01 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 7:13 PM, David Bregman wrote:

 OK, I think I have an idea how to be more convincing (I wish I'd thought of
this
 earlier):

 is this
 http://www.cplusplus.com/reference/cassert/assert/

 the same as this?
 http://msdn.microsoft.com/en-us/library/1b3fsfxw.aspx

 can you see the difference now?


What I see is Microsoft attempting to bring D's assert semantics into C++. :-)

As I've mentioned before, there is inexorable pressure for this to happen, and 
it will happen.
Aug 01 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 07:35 AM, Walter Bright wrote:

 On 8/1/2014 7:13 PM, David Bregman wrote:

 OK, I think I have an idea how to be more convincing (I wish I'd
 thought of this
 earlier):

 is this
 http://www.cplusplus.com/reference/cassert/assert/

 the same as this?
 http://msdn.microsoft.com/en-us/library/1b3fsfxw.aspx

 can you see the difference now?


 What I see is Microsoft attempting to bring D's assert semantics into
 C++. :-)


Note, regardless of the validity or invalidity of your point:


                 You evaded the question.



 As I've mentioned before, there is inexorable pressure for this to
 happen, and it will happen.


And when it does happen, criminals will rejoice, right?
Aug 01 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 11:08 PM, Timon Gehr wrote:

 You evaded the question.


Those have been posted here before, and I responded in detail to them, more
than 
once.

I have nothing new to say about it.
Aug 01 2014
parent "Tofu Ninja" <emmons0 purdue.edu> writes:
On Saturday, 2 August 2014 at 06:44:48 UTC, Walter Bright wrote:

 On 8/1/2014 11:08 PM, Timon Gehr wrote:

 You evaded the question.


 Those have been posted here before, and I responded in detail 
 to them, more than once.

 I have nothing new to say about it.


Honestly, how can you be so comfortable with adding a feature
that can be used to so easily cause undefined behavior????
Aug 02 2014
prev sibling parent reply "David Bregman" <drb sfu.ca> writes:
On Saturday, 2 August 2014 at 05:35:26 UTC, Walter Bright wrote:

 On 8/1/2014 7:13 PM, David Bregman wrote:

 OK, I think I have an idea how to be more convincing (I wish 
 I'd thought of this
 earlier):

 is this
 http://www.cplusplus.com/reference/cassert/assert/

 the same as this?
 http://msdn.microsoft.com/en-us/library/1b3fsfxw.aspx

 can you see the difference now?


 What I see is Microsoft attempting to bring D's assert 
 semantics into C++. :-)


OK, I'm done. It's clear now that you're just being 
intellectually dishonest in order to "win" what amounts to a 
trivial argument. So much for professionalism.


 As I've mentioned before, there is inexorable pressure for this 
 to happen, and it will happen.


What will happen is you will find out the hard way whether code 
breakage, undefined behavior and security holes are a worthy 
tradeoff for some trivial performance gains which likely can't 
even be measured except in toy benchmarks.
Aug 02 2014
parent reply "Kagamin" <spam here.lot> writes:
On Saturday, 2 August 2014 at 17:36:46 UTC, David Bregman wrote:

 OK, I'm done. It's clear now that you're just being 
 intellectually dishonest in order to "win" what amounts to a 
 trivial argument. So much for professionalism.


Haha, this time it's not as bad as it was in catch syntax 
discussion. I even thought then they are blackmailed or something 
like that.
Aug 03 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 8/3/14, 11:55 AM, Kagamin wrote:

 On Saturday, 2 August 2014 at 17:36:46 UTC, David Bregman wrote:

 OK, I'm done. It's clear now that you're just being intellectually
  dishonest in order to "win" what amounts to a trivial argument. So
  much for professionalism.


 Haha, this time it's not as bad as it was in catch syntax discussion.
 I even thought then they are blackmailed or something like that.


It's really only this kind of stuff that has Walter and myself worried.
We understand spirits can get heated during a debate but the problem
with such comebacks that really hold no punches is they instantly
degrade the level of conversation, invite answers in kind, and are very
difficult to respond to in meaningful ways.

 From what I can tell after many years of having at this, there's a sort
of a heat death of debate in which questions are asked in a definitive,
magisterial manner (bearing an odd implied binding social contract), and
any response except the desired one is instantly dismissed as simply
stupid, intellectually dishonest, or, as it were, coming under duress.

I can totally relate to people who hold a conviction strong enough to
have difficulty acknowledging a contrary belief as even remotely
reasonable, as I've fallen for that many times and I certainly will in
the future. Improving awareness of it only improves the standing of
debate for everyone involved.

For my money, consider Walter's response:


 What I see is Microsoft attempting to bring D's assert semantics into
 C++. :)

 As I've mentioned before, there is inexorable pressure for this to
 happen, and it will happen.


I find it to the point, clear, and funny. Expanded it would go like "I 
see more similarities than differences, and a definite convergence 
dictated by market pressure." I find it highly inappropriate to qualify 
that response as intellectually dishonest even after discounting for a 
variety of factors, and an apology would be in order.


Andrei
Aug 03 2014
next sibling parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Monday, 4 August 2014 at 00:59:10 UTC, Andrei Alexandrescu 
wrote:

 On 8/3/14, 11:55 AM, Kagamin wrote:

 On Saturday, 2 August 2014 at 17:36:46 UTC, David Bregman 
 wrote:

 OK, I'm done. It's clear now that you're just being 
 intellectually
 dishonest in order to "win" what amounts to a trivial 
 argument. So
 much for professionalism.


 Haha, this time it's not as bad as it was in catch syntax 
 discussion.
 I even thought then they are blackmailed or something like 
 that.


 It's really only this kind of stuff that has Walter and myself 
 worried.
 We understand spirits can get heated during a debate but the 
 problem
 with such comebacks that really hold no punches is they 
 instantly
 degrade the level of conversation, invite answers in kind, and 
 are very
 difficult to respond to in meaningful ways.

 From what I can tell after many years of having at this, 
 there's a sort
 of a heat death of debate in which questions are asked in a 
 definitive,
 magisterial manner (bearing an odd implied binding social 
 contract), and
 any response except the desired one is instantly dismissed as 
 simply
 stupid, intellectually dishonest, or, as it were, coming under 
 duress.

 I can totally relate to people who hold a conviction strong 
 enough to
 have difficulty acknowledging a contrary belief as even remotely
 reasonable, as I've fallen for that many times and I certainly 
 will in
 the future. Improving awareness of it only improves the 
 standing of
 debate for everyone involved.

 For my money, consider Walter's response:


 What I see is Microsoft attempting to bring D's assert 
 semantics into
 C++. :)

 As I've mentioned before, there is inexorable pressure for 
 this to
 happen, and it will happen.


 I find it to the point, clear, and funny. Expanded it would go 
 like "I see more similarities than differences, and a definite 
 convergence dictated by market pressure." I find it highly 
 inappropriate to qualify that response as intellectually 
 dishonest even after discounting for a variety of factors, and 
 an apology would be in order.


It's easy to get a very different impression from his post: "I 
don't want to counter your arguments, so I just pick a random 
snippet from your post and make a funny mostly irrelevant 
comment, while ignoring everything else you wrote."

Now, I don't believe that this was Walter's intention, but in the 
end it had the same effect :-( This is very bad, because I feel 
this discussion is important, so it matters for both sides to 
understand where the others are coming from. We can't achieve 
this by evading questions and discussion. And it wasn't just this 
one post; if you look at the discussion, there are entire 
sub-threads where people were obviously just talking past each 
other. But for me it's not easy to see _why_ they were doing it, 
i.e. what exactly the misunderstanding is.
Aug 04 2014
prev sibling next sibling parent "David Bregman" <drb sfu.ca> writes:
On Monday, 4 August 2014 at 00:59:10 UTC, Andrei Alexandrescu
wrote:

 I find it to the point, clear, and funny. Expanded it would go 
 like "I see more similarities than differences, and a definite 
 convergence dictated by market pressure."


I find this to be a non-sequitur. Firstly, making a joke to avoid
answering a direct question is hardly what I'd call "to the
point", regardless of how funny it may be. Secondly, even if I
interpreted it as you did, it still misses the point. The
discussion was about strict logical semantic equivalence of two
concepts, assert and assume. How can "I see more similarities
than differences, and a definite convergence dictated by market
pressure" be a response to a black and white question about
semantic equivalence?


 I find it highly inappropriate to qualify that response as 
 intellectually dishonest even after discounting for a variety 
 of factors, and an apology would be in order.


Sure, that post alone isn't sufficient, but Walter argued much of
the discussion in an unfair manner: dismissive one liners,
fallacious arguments, strawman, even straight up begging the
question. We are talking about several dozens of posts here.
There was more than enough time spent to converge on agreement,
if both sides intended to do so. Anyways I know this is all
completely pointless to say, no one is going to believe it
for one second because it just sounds like poor sportsmanship.
For anyone who actually cares, you just need to read the threads
and judge for yourself. From where I stand though, no apology is
required.
Aug 04 2014
prev sibling next sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Monday, 4 August 2014 at 00:59:10 UTC, Andrei Alexandrescu 
wrote:

 For my money, consider Walter's response:


 What I see is Microsoft attempting to bring D's assert 
 semantics into
 C++. :)

 As I've mentioned before, there is inexorable pressure for 
 this to
 happen, and it will happen.


 I find it to the point, clear, and funny.


Yeah, it was funny because:
( http://msdn.microsoft.com/en-us/library/1b3fsfxw.aspx )

«Caution:
A program must not contain an invalid __assume statement on a 
reachable path. If the compiler can reach an invalid __assume 
statement, the program might cause unpredictable and potentially 
dangerous behavior.»

Please note «if the compiler can reach», not «if a path can be 
executed».


And «inexorable»:
( https://www.google.no/search?q=inexorable )

- "the seemingly inexorable march of new technology"

- (of a person) impossible to persuade; unrelenting.


Of course, being unrelenting goes with the territory of 
single-handedly building a c++ competitor. So it is not a bad 
trait. The bad thing here is not making "compile assert as 
assume" an experimental feature, but making it the default.

People who are autodidact also sometimes find new ways of doing 
things because they are more likely to go explore new paths. But 
I don't think this path is all that new… so I hope Walter, if he 
continues walking down this path, remains unrelenting and keeps 
walking past "assert as assume" until he finds truly new 
territory in the realm of formal methods. That could happen and 
bring great features to D.

There is a significant difference between going down a path to 
see what you can find on the other end and turning the path into 
a goal, a priori.
Aug 05 2014
next sibling parent reply "eles" <eles eles.com> writes:
On Tuesday, 5 August 2014 at 09:42:26 UTC, Ola Fosheim Grøstad 
wrote:

 On Monday, 4 August 2014 at 00:59:10 UTC, Andrei Alexandrescu 
 wrote:

 For my money, consider Walter's response:




I feel a bit confused about the mixture between compiler and 
optimizer. While I agree the compiler does the optimization and 
the two are intrinsically linked, the languages (or the 
instructions) for them seem to me to belong to quite different 
paradigms:

- compiler language is imperative programming
- optimizer language is declarative programming

It is wise to mix them to such degree as to no longer distinguish 
them? For me, assume and the like shall rather go with the 
annotations.
Aug 05 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 5 August 2014 at 10:00:55 UTC, eles wrote:

 It is wise to mix them to such degree as to no longer 
 distinguish them? For me, assume and the like shall rather go 
 with the annotations.


That's one of the reasons I think it is not new territory, since 
letting assert have side effects basically sounds like 
constraints programming/logic programming.

I do think that constraints programming has a place in support 
for generic programming and other things that can be known to 
evaluate at compile time. So I think imperative programming 
languages are going to become hybrids over time.

Also, if you think about the new field "program synthesis", where 
you specify the constraints to generate/fill out boiler code in 
an imperative program, then the distinction becomes blurry. 
Rather than saying sort(x) you just specify that the outcome 
should be sorted in the post condition, but don't care why it 
ended up that way. So the compiler will automatically add sort(x) 
if needed. Sounds like a powerful way to get rid of boring 
programming parts.

Another point, when you think about it, Program Verification and 
Optimization are conceptually closely related.

S = specification // asserts is a weak version of this
P = program
E = executable

ProgramVerification:
Prove( S(x)==P(x) for all x )

Optimization Invariant:
Prove( P(x)==E(x) for all x )
Aug 05 2014
prev sibling parent reply "Andrew Godfrey" <X y.com> writes:
On Tuesday, 5 August 2014 at 09:42:26 UTC, Ola Fosheim Grøstad 
wrote:

 But I don't think this path is all that new… so I hope Walter, 
 if he continues walking down this path, remains unrelenting and 
 keeps walking past "assert as assume" until he finds truly new 
 territory in the realm of formal methods. That could happen and 
 bring great features to D.


This! +1000!

This is what I've been feeling too: Walter is wrong - 
astonishingly wrong in fact - but in a very interesting direction 
that may have something very 'right' on the other side of it. I 
don't know what form it will take, but the example
someone gave, of keeping track of when a range is sorted vs. not 
known to be sorted, to me gives a hint of where this may lead. I 
can't quite imagine that particular example playing out, but in 
general if the compiler keeps track of properties of things then 
it could start making algorithmic-level performance decisions 
that today we always have to make by hand. To me that's 
interesting.
Aug 05 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Tuesday, 5 August 2014 at 16:35:42 UTC, Andrew Godfrey wrote:

 out, but in general if the compiler keeps track of properties 
 of things then it could start making algorithmic-level 
 performance decisions that today we always have to make by 
 hand. To me that's interesting.


I think the most "potent" source for optimization is getting 
expressive formalisms in place for allowing axioms and theorems 
to pass through foreign function calls.

That would allow you to call custom C functions/machine language 
in  a loop with less penalties.
Aug 06 2014
prev sibling parent "Kagamin" <spam here.lot> writes:
On Monday, 4 August 2014 at 00:59:10 UTC, Andrei Alexandrescu 
wrote:

 I can totally relate to people who hold a conviction strong 
 enough to
 have difficulty acknowledging a contrary belief as even remotely
 reasonable


Yes, it's difficult to acknowledge a belief, reason for which 
wasn't provided, but instead "we have nothing to add" reply was 
given.
Aug 05 2014
prev sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 01, 2014 at 10:39:44AM -0700, Andrei Alexandrescu via Digitalmars-d
wrote:

 On 8/1/14, 9:25 AM, Timon Gehr wrote:

Even then, such a semantics is non-standard and almost nobody else
knew.


 
 This notion of "standard" has been occasionally mentioned in this
 discussion. I agree that D's assert is different from traditional C
 and C++ assert, and I also agree that might surprise some, but I think
 the meaning of D's assert is well within the intent of the larger
 notion.
 

Why break 'assert' now, now that it actually behaves as I and many
others expect (even some of those who argued for the (apparently,
even inofficially) new and opposite design)?


 
 I don't see any breakage of "assert", more like realizing more of its
 latent potential. Clearly the documentation could be better, which is
 something we should focus on.


And to help with said focus, here's where y'all may direct your energy:

	https://github.com/D-Programming-Language/dlang.org/pull/624

;-)



 I do agree there's stuff that some may find unexpected, such as:
 
 assert(x > 42);
 if (x <= 42) {
   // let me also handle this conservatively
   ...
 } else {
   // all good, proceed
   ...
 }
 
 The D optimizer might actually deem the entire "then" path unreachable
 in the future, and only compile in the "else" path. Some may find that
 surprising. For my money I never write code like this, and I consider
 it incorrect if I review it. You either assert something, or you check
 it dynamically.


Exactly.


T

-- 
Winners never quit, quitters never win. But those who never quit AND never win
are idiots.
Aug 01 2014
prev sibling next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Walter Bright"  wrote in message news:lrbpvj$mih$1 digitalmars.com...


 5. assert(0); is equivalent to a halt, and the compiler won't remove it.


This is not the same definition the spec gives.  The spec says assert(0) can 
be treated as unreachable, and the compiler is allowed to optimize 
accordingly.

The difference is that in this code:

if (cond)
    assert(0);

With your above definition cond will be evaluated, while with the spec's 
more powerful definition it may be skipped.
Jul 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 1:23 AM, Daniel Murphy wrote:

 "Walter Bright"  wrote in message news:lrbpvj$mih$1 digitalmars.com...


 5. assert(0); is equivalent to a halt, and the compiler won't remove it.


 This is not the same definition the spec gives.  The spec says assert(0) can be
 treated as unreachable, and the compiler is allowed to optimize accordingly.


It says more than that:

"The expression assert(0) is a special case; it signifies that it is
unreachable 
code. Either AssertError is thrown at runtime if it is reachable, or the 
execution is halted (on the x86 processor, a HLT instruction can be used to
halt 
execution). The optimization and code generation phases of compilation may 
assume that it is unreachable code."

   -- http://dlang.org/expression.html#AssertExpression
Jul 31 2014
parent reply "ponce" <contact gam3sfrommars.fr> writes:
On Thursday, 31 July 2014 at 09:13:53 UTC, Walter Bright wrote:

 On 7/31/2014 1:23 AM, Daniel Murphy wrote:

 "Walter Bright"  wrote in message 
 news:lrbpvj$mih$1 digitalmars.com...


 5. assert(0); is equivalent to a halt, and the compiler won't 
 remove it.


 This is not the same definition the spec gives.  The spec says 
 assert(0) can be
 treated as unreachable, and the compiler is allowed to 
 optimize accordingly.


 It says more than that:

 "The expression assert(0) is a special case; it signifies that 
 it is unreachable code. Either AssertError is thrown at runtime 
 if it is reachable, or the execution is halted (on the x86 
 processor, a HLT instruction can be used to halt execution). 
 The optimization and code generation phases of compilation may 
 assume that it is unreachable code."

   -- http://dlang.org/expression.html#AssertExpression


You said "the compiler won't remove it".
http://dlang.org/expression.html#AssertExpression says: "The 
optimization and code generation phases of compilation may assume 
that it is unreachable code."

Who is right?

If I write:

---
switch(expr())
{
case 0: doIt();
case 1: doThat();
default:
   assert(0);
}
---

Will the optimizer be able to remove the default: case?
Because If I use assert(0) it's on purpose and do not want it to 
be elided, ever.
MSVC has __assume(0); for unreachable code, GCC has 
__builtin_unreachable()
Jul 31 2014
next sibling parent reply "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Thursday, 31 July 2014 at 10:24:07 UTC, ponce wrote:

 If I write:

 ---
 switch(expr())
 {
 case 0: doIt();
 case 1: doThat();
 default:
   assert(0);
 }
 ---

 Will the optimizer be able to remove the default: case?


Assuming fall-through (`goto case`), not only the default case. 
The entire switch could be removed, under the condition that the 
compiler can prove that neither `expr()`, `doIt()`, nor 
`doThat()` throws, even if they have side effects. And maybe even 
the entire function, and all functions that call it, depending on 
how exactly the control flow is.
Jul 31 2014
parent "ponce" <contact gam3sfrommars.fr> writes:
On Thursday, 31 July 2014 at 11:01:56 UTC, Marc Schütz wrote:

 On Thursday, 31 July 2014 at 10:24:07 UTC, ponce wrote:

 If I write:

 ---
 switch(expr())
 {
 case 0: doIt();
 case 1: doThat();
 default:
  assert(0);
 }
 ---

 Will the optimizer be able to remove the default: case?


 Assuming fall-through (`goto case`), not only the default case. 
 The entire switch could be removed, under the condition that 
 the compiler can prove that neither `expr()`, `doIt()`, nor 
 `doThat()` throws, even if they have side effects. And maybe 
 even the entire function, and all functions that call it, 
 depending on how exactly the control flow is.


Ok my example was wrong, I meant:

---
switch(expr())
{
case 0: doIt(); break;
case 1: doThat(); break;
default:
      assert(0);
      break;
}
---
Jul 31 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 3:24 AM, ponce wrote:

 On Thursday, 31 July 2014 at 09:13:53 UTC, Walter Bright wrote:

 On 7/31/2014 1:23 AM, Daniel Murphy wrote:

 "Walter Bright"  wrote in message news:lrbpvj$mih$1 digitalmars.com...


 5. assert(0); is equivalent to a halt, and the compiler won't remove it.


 This is not the same definition the spec gives.  The spec says assert(0) can be
 treated as unreachable, and the compiler is allowed to optimize accordingly.


 It says more than that:

 "The expression assert(0) is a special case; it signifies that it is
 unreachable code. Either AssertError is thrown at runtime if it is reachable,
 or the execution is halted (on the x86 processor, a HLT instruction can be
 used to halt execution). The optimization and code generation phases of
 compilation may assume that it is unreachable code."

   -- http://dlang.org/expression.html#AssertExpression


 You said "the compiler won't remove it".


Right, and it doesn't.


 http://dlang.org/expression.html#AssertExpression says: "The optimization and
 code generation phases of compilation may assume that it is unreachable code."

 Who is right?


It means if the control flow does actually get there, a HALT is executed.
Jul 31 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 09:03 PM, Walter Bright wrote:

 On 7/31/2014 3:24 AM, ponce wrote:

 On Thursday, 31 July 2014 at 09:13:53 UTC, Walter Bright wrote:

 On 7/31/2014 1:23 AM, Daniel Murphy wrote:

 "Walter Bright"  wrote in message news:lrbpvj$mih$1 digitalmars.com...


 5. assert(0); is equivalent to a halt, and the compiler won't
 remove it.


 This is not the same definition the spec gives.  The spec says
 assert(0) can be
 treated as unreachable, and the compiler is allowed to optimize
 accordingly.


 It says more than that:

 "The expression assert(0) is a special case; it signifies that it is
 unreachable code. Either AssertError is thrown at runtime if it is
 reachable,
 or the execution is halted (on the x86 processor, a HLT instruction
 can be
 used to halt execution). The optimization and code generation phases of
 compilation may assume that it is unreachable code."

   -- http://dlang.org/expression.html#AssertExpression


 You said "the compiler won't remove it".


 Right, and it doesn't.


 http://dlang.org/expression.html#AssertExpression says: "The
 optimization and
 code generation phases of compilation may assume that it is
 unreachable code."

 Who is right?


 It means if the control flow does actually get there, a HALT is executed.


And assuming control flow does not actually get there?
Jul 31 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 31 July 2014 at 19:11:55 UTC, Timon Gehr wrote:

 On 07/31/2014 09:03 PM, Walter Bright wrote:

 It means if the control flow does actually get there, a HALT 
 is executed.


 And assuming control flow does not actually get there?


Then the HALT instruction is never hit. The compiler would have 
to be able to prove that reaching the HALT instruction was 
impossible in order to remove it (in which case, I would assume 
that it would remove it, but I wouldn't expect that to happen 
very often).

- Jonathan M Davis
Jul 31 2014
next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 31 July 2014 at 19:54:47 UTC, Jonathan M Davis wrote:

 Then the HALT instruction is never hit. The compiler would have 
 to be able to prove that reaching the HALT instruction was 
 impossible in order to remove it […]


Well, no, as the portion of the spec Walter quoted specifically 
states that the compiler can *assume* the assert(0) to be 
unreachable. If you can assume something, you don't have to prove 
it any longer for any sane definition of "assume".

If this is not something we want, we need to fix the spec instead 
of trying to argue around the problem.

David
Jul 31 2014
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 09:54 PM, Jonathan M Davis wrote:

 On Thursday, 31 July 2014 at 19:11:55 UTC, Timon Gehr wrote:

 On 07/31/2014 09:03 PM, Walter Bright wrote:

 It means if the control flow does actually get there, a HALT is
 executed.


 And assuming control flow does not actually get there?


 Then the HALT instruction is never hit.


Indeed. Now note that the compiler is arguing from the same standpoint 
that you were just now.


 The compiler would have to be
 able to prove that reaching the HALT instruction was impossible in order
 to remove it


The compiler is able to prove this immediately. The goal it needs to 
prove ('control flow does not reach assert(0)') is an assumption made by 
Walter and the language specification and readily available.

Note that I'd be happier with the state of affairs if you were right 
about this.
Jul 31 2014
prev sibling next sibling parent "ponce" <contact gamesfrommars.fr> writes:
On Thursday, 31 July 2014 at 19:03:14 UTC, Walter Bright wrote:

 It means if the control flow does actually get there, a HALT is 
 executed.


Fine.
Jul 31 2014
prev sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 31 July 2014 at 19:03:14 UTC, Walter Bright wrote:

 On 7/31/2014 3:24 AM, ponce wrote:

 On Thursday, 31 July 2014 at 09:13:53 UTC, Walter Bright wrote:

 It says more than that:

 "The expression assert(0) is a special case; it signifies 
 that it is
 unreachable code. Either AssertError is thrown at runtime if 
 it is reachable,
 or the execution is halted (on the x86 processor, a HLT 
 instruction can be
 used to halt execution). The optimization and code generation 
 phases of
 compilation may assume that it is unreachable code."

  -- http://dlang.org/expression.html#AssertExpression


 You said "the compiler won't remove it".


 Right, and it doesn't.


This is in direct contradiction to the quoted spec excerpt. If 
the backend can assume that something is unreachable code, why on 
earth should it need to actually emit that code? A small example:

---
void foo(int a) {
    if (a == 42) assert(0);
    // Do something else.
}
---

If the compiler is free to assume that the assert is unreachable, 
please explain to me what stops it from inferring that the branch 
is never taken and transforming the example to the equivalent of:

---
void foo(int a) {
    // Do something else.
}
---

LDC would do this today if we implemented the regarding assuming 
unreachability (we currently emit a halt – actually a ud2 trap on 
x86 – instead).

I've had the questionable pleasure of tracking down a couple of 
related issues in LLVM and the LDC codegen, so please take my 
word for it: Requiring any particular behavior such as halting in 
a case that can be assumed to be unreachable is at odds with how 
the term "unreachable" is used in the wild – at least in projects 
like GCC and LLVM.

Cheers,
David
Jul 31 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 1:33 PM, David Nadlinger wrote:

 I've had the questionable pleasure of tracking down a couple of related issues
 in LLVM and the LDC codegen, so please take my word for it: Requiring any
 particular behavior such as halting in a case that can be assumed to be
 unreachable is at odds with how the term "unreachable" is used in the wild –
at
 least in projects like GCC and LLVM.


For example:

  int foo() {
    while (...) {
        ...
    }
    assert(0);
  }

the compiler needn't issue an error at the end "no return value for foo()" 
because it can assume it never got there.

I'll rewrite that bit in the spec as it is clearly causing confusion.
Jul 31 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 11:25 PM, Walter Bright wrote:

 I'll rewrite that bit in the spec as it is clearly causing confusion.


A wording that avoids all those issues would be something like: 
"'assert(0)' never returns and hence terminates the basic block it 
occurs in."
Jul 31 2014
prev sibling parent reply "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 31 July 2014 at 21:25:25 UTC, Walter Bright wrote:

 On 7/31/2014 1:33 PM, David Nadlinger wrote:

 I've had the questionable pleasure of tracking down a couple 
 of related issues
 in LLVM and the LDC codegen, so please take my word for it: 
 Requiring any
 particular behavior such as halting in a case that can be 
 assumed to be
 unreachable is at odds with how the term "unreachable" is used 
 in the wild – at
 least in projects like GCC and LLVM.


 For example:

  int foo() {
    while (...) {
        ...
    }
    assert(0);
  }

 the compiler needn't issue an error at the end "no return value 
 for foo()" because it can assume it never got there.

 I'll rewrite that bit in the spec as it is clearly causing 
 confusion.


Don't rewrite it because you merely concede that it might be 
confusing. Rewrite it because you admit it's contradictory. If 
you just try to reword the spec without understanding how your 
use of the terminology differs from the established meaning, 
you'll probably come up with something that is confusing to the 
rest of the world just as well.

Perhaps looking at the situation in terms of basic blocks and the 
associated control flow graph will help:

As per your above post, assert(0) has nothing to do with making 
any assumptions on the compiler side. It merely servers as a 
terminator instruction of a BB, making it a leaf in the CFG. This 
seems to be the definition you intend for the spec. Maybe add 
something along the lines of "behaves like a function call that 
never returns" as an explanation to make it easier to understand.

This is not what "unreachable" means. If assert(0) was 
unreachable, then the compiler would be free to assume that no 
CFG edges *into* the BB holding the instruction are ever taken 
(and as a corollary, it could also decide not emit any code for 
it). Thus, the term certainly shouldn't appear anywhere near 
assert(0) in the spec, except to point out the difference.

Cheers,
David
Jul 31 2014
parent "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 31 July 2014 at 22:17:56 UTC, David Nadlinger wrote:

 servers


Gah, "serves". Also, I hope the post didn't come across as 
condescending, as it certainly wasn't intended that way. I just 
figured it would be a good idea to define the terms we are using, 
as we seemed to be continuously talking past each other.

David
Jul 31 2014
prev sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:

 3. Use of assert to validate input is utterly wrong and will 
 not be supported. Use such constructs at your own risk.


...

 6. enforce() is meant to check for input errors (environmental 
 errors are considered input).

 7. using enforce() to check for program bugs is utterly wrong. 
 enforce() is a library creation, the core language does not 
 recognize it.


Could you expand on what you consider input?  For example, if a
function has an "in" contract that validates input parameters, is
the determination that a parameter is invalid a program bug or
simply invalid input?  If you consider this invalid input that
should be checked by enforce(), can you explain why?
Jul 31 2014
next sibling parent reply "ponce" <contact gamesfrommars.fr> writes:
On Thursday, 31 July 2014 at 20:52:30 UTC, Sean Kelly wrote:

 On Wednesday, 30 July 2014 at 22:01:23 UTC, Walter Bright wrote:

 3. Use of assert to validate input is utterly wrong and will 
 not be supported. Use such constructs at your own risk.


 ...

 6. enforce() is meant to check for input errors (environmental 
 errors are considered input).

 7. using enforce() to check for program bugs is utterly wrong. 
 enforce() is a library creation, the core language does not 
 recognize it.


 Could you expand on what you consider input?  For example, if a
 function has an "in" contract that validates input parameters, 
 is
 the determination that a parameter is invalid a program bug or
 simply invalid input?  If you consider this invalid input that
 should be checked by enforce(), can you explain why?


This also puzzles me. There is the point where the two types of 
errors blend to the point of being uncomfortable.

Eg: a program generates files in X format and can also read them 
with a X parser. Its X parser will only ever read output 
generated by itself. Should input errors in X parser be checked 
with assert or exceptions?
Jul 31 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 11:01 PM, ponce wrote:

 Could you expand on what you consider input?  For example, if a
 function has an "in" contract that validates input parameters, is
 the determination that a parameter is invalid a program bug or
 simply invalid input? ...


The assertions in an 'in' contracts are obligations at the call site.

I.e., the code:

void baz(int x){
     assert(x>2);
     // ...
}

is buggy.

The code

void foo(int x)in{ assert(x>2); }body{ assert(x>2); }

is correct.

The code:

void bar(int x){ foo(x); }

is buggy.

The code

void bar(int x){
     enforce(x>2);
     foo(x);
}

is fine.



 This also puzzles me. There is the point where the two types of errors
 blend to the point of being uncomfortable.

 Eg: a program generates files in X format and can also read them with a
 X parser. Its X parser will only ever read output generated by itself.
 Should input errors in X parser be checked with assert or exceptions?


Use 'assert' for checking things which you expect to be true. But don't 
be fooled, besides having some loosely checked code documentation, the 
main reason to write down assertions is that they will occasionally fail 
and tell you something interesting which you didn't know yet about your 
program (as it was nicely formulated somewhere else in this thread.) 
This makes assertions seem a little schizophrenic at times, but after 
all, checking those assertions, which you _expect_, by definition, to be 
no-ops anyway, may be too expensive for you and then they can be 
disabled. I'd check what are the actual performance gains before doing 
this though. You can also control assertions in a more fine-grained way 
by guarding them with version statements.

Furthermore, use 'assert' as well in _in_ contracts, and think about 
them being checked in the context of the caller as an obligation instead 
of as being checked in your own function.

Use 'enforce' for things which you expect might be false sometimes and 
that you may want to handle as an exception in this case.

I wouldn't think about 'enforce' as 'not checking program bugs' too hard 
though. Maybe the bug is in code which does not actually expect the 
exceptional path to be taken.

The difference between

void foo1(int x)in{ assert(x>2); }body{ ... }

and

void foo2(int x){
     enforce(x>2);
}

Is that 'foo2' reliably throws an exception if the contents of x are not 
greater than 2; this is part of it's behaviour and would be part of its 
documentation, while 'foo1' just states in its documentation that it 
will do a certain thing _provided_ x is greater than 2, and that its 
behaviour is left unspecified otherwise.


Or that's what I would say anyway if there wasn't talk about turning 
assertions into assumptions in -release. If nothing changes, always use 
version(assert) to guard your assert statements unless you want their 
failures to be undefined behaviour in -release mode.
Jul 31 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 11:39 PM, Timon Gehr wrote:

 it's behaviour


gah.
Jul 31 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 2:01 PM, ponce wrote:

 This also puzzles me. There is the point where the two types of errors blend to
 the point of being uncomfortable.

 Eg: a program generates files in X format and can also read them with a X
 parser. Its X parser will only ever read output generated by itself. Should
 input errors in X parser be checked with assert or exceptions?


Exceptions. Although there are grey areas, this is not one. Filesystems are 
subject to all kinds of failures, exhaustions, modification by other processes, 
etc., which are not logic bugs in your program.


If you're brave and want to have some fun, fill up your hard disk so it is 
nearly full. Now run your favorite programs that read and write files. Sit back 
and watch the crazy results (far too many programs assume that writes succeed). 
Operating systems also behave erratically in this scenario, hence the 'brave' 
suggestion.
Jul 31 2014
next sibling parent reply Assaf Gordon via Digitalmars-d <digitalmars-d puremagic.com> writes:
Sorry to hijack the thread, but:

On 07/31/2014 09:27 PM, Walter Bright via Digitalmars-d wrote:

 If you're brave and want to have some fun, fill up your hard disk so
 it is nearly full. Now run your favorite programs that read and write
 files. Sit back and watch the crazy results (far too many programs
 assume that writes succeed). Operating systems also behave
 erratically in this scenario, hence the 'brave' suggestion.


If anyone is interested in simulating I/O errors and nearly-full file system on
Linux,
I can suggest the following scripts:
  http://git.savannah.gnu.org/cgit/datamash.git/tree/build-aux/create_corrupted_file_system.sh
  http://git.savannah.gnu.org/cgit/datamash.git/tree/build-aux/create_small_file_system.sh

The scripts create two ext3 images which can be mounted, and simulate I/O
errors.
One simulate file system with corrupted files (so "open" and the first "read"
will succeed, but later "read" will fail with EIO),
and the other simulate a tiny filesystem, so that the first few "writes" will
succeed, but "write" of >40KB will fail with ENOSPC.

The scripts themselves don't require root, but you'll need root to mount the
images.

As Walter said, it's alarming how many programs fail to handle such cases
(though D is pretty solid in that regard).

Hope this helps,
  - Assaf
Aug 01 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 8:18 AM, Assaf Gordon via Digitalmars-d wrote:

 As Walter said, it's alarming how many programs fail to handle such cases
 (though D is pretty solid in that regard).


One of the very cool things about the ranges+algorithms style of programming is 
things like I/O don't get mixed up in code that operates on data.

It means that detecting I/O failures only have to be coded in a relatively 
concentrated and small handful of places rather than sprinkled throughout the
code.

Couple this with the exception model of reporting errors, and we have a fairly 
robust system of not silently overlooking I/O failures.
Aug 01 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 01, 2014 at 12:05:00PM -0700, Walter Bright via Digitalmars-d wrote:

 On 8/1/2014 8:18 AM, Assaf Gordon via Digitalmars-d wrote:

As Walter said, it's alarming how many programs fail to handle such
cases (though D is pretty solid in that regard).


 
 One of the very cool things about the ranges+algorithms style of
 programming is things like I/O don't get mixed up in code that
 operates on data.


[...]

Not to mention, it lets you *really* unittest your code thoroughly,
because it isolates each separate stage of the processing into its own
self-contained unit, with clearly-defined standard interconnects. Your
unittest can therefore easily hook up with the unit's interfaces, and
inject arbitrary inputs to the unit and run arbitrary tests on its
output -- without needing such hackery as redirecting stdout just so you
can confirm correct computation of a numerical result, for example.

In the traditional imperative programming style, you often have code
that has loops within loops within loops, with complex interactions
between each loop body and its surrounding context. It's generally
impossible (or very hard) to extricate the inner loop code from its
tightly-coupled context, which means your unittest will have a hard time
"reaching into" the innards of the nested loops to verify the
correctness of each piece of code.  Often, the result is that rather
than testing each *unit* of the code, you have to do a holistic test by
running the entire machinery of nested loops inside a sandbox and
capturing its output (via stdout redirection, or instrumenting dependent
subsystems, etc.) -- hardly a *unit* test anymore! -- and still, you
have the problem that there are too many possible code paths that these
nested loops may run through, so it would be hard to have any confidence
that you've covered sufficiently many paths in the test. You're almost
certain to miss important boundary cases.


T

-- 
The irony is that Bill Gates claims to be making a stable operating system and
Linus Torvalds claims to be trying to take over the world. -- Anonymous
Aug 01 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 1:02 PM, H. S. Teoh via Digitalmars-d wrote:

 Not to mention, it lets you *really* unittest your code thoroughly,
 because it isolates each separate stage of the processing into its own
 self-contained unit, with clearly-defined standard interconnects. Your
 unittest can therefore easily hook up with the unit's interfaces, and
 inject arbitrary inputs to the unit and run arbitrary tests on its
 output -- without needing such hackery as redirecting stdout just so you
 can confirm correct computation of a numerical result, for example.

 In the traditional imperative programming style, you often have code
 that has loops within loops within loops, with complex interactions
 between each loop body and its surrounding context. It's generally
 impossible (or very hard) to extricate the inner loop code from its
 tightly-coupled context, which means your unittest will have a hard time
 "reaching into" the innards of the nested loops to verify the
 correctness of each piece of code.  Often, the result is that rather
 than testing each *unit* of the code, you have to do a holistic test by
 running the entire machinery of nested loops inside a sandbox and
 capturing its output (via stdout redirection, or instrumenting dependent
 subsystems, etc.) -- hardly a *unit* test anymore! -- and still, you
 have the problem that there are too many possible code paths that these
 nested loops may run through, so it would be hard to have any confidence
 that you've covered sufficiently many paths in the test. You're almost
 certain to miss important boundary cases.


I agree and want to amplify this a bit. When a function accepts ranges as its 
input/output parameters, then the function tends to be a template function.
This 
means that it becomes easy for the unittests to "mock up" ranges and feed them 
to the function to test it.

I've used this to great success with Warp.
Aug 01 2014
prev sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 01, 2014 at 11:18:47AM -0400, Assaf Gordon via Digitalmars-d wrote:

 Sorry to hijack the thread, but:
 
 On 07/31/2014 09:27 PM, Walter Bright via Digitalmars-d wrote:

If you're brave and want to have some fun, fill up your hard disk so
it is nearly full. Now run your favorite programs that read and write
files. Sit back and watch the crazy results (far too many programs
assume that writes succeed). Operating systems also behave
erratically in this scenario, hence the 'brave' suggestion.


 
 If anyone is interested in simulating I/O errors and nearly-full file
 system on Linux, I can suggest the following scripts:
  http://git.savannah.gnu.org/cgit/datamash.git/tree/build-aux/create_corrupted_file_system.sh
  http://git.savannah.gnu.org/cgit/datamash.git/tree/build-aux/create_small_file_system.sh
 
 The scripts create two ext3 images which can be mounted, and simulate
 I/O errors.
 One simulate file system with corrupted files (so "open" and the first
 "read" will succeed, but later "read" will fail with EIO),
 and the other simulate a tiny filesystem, so that the first few
 "writes" will succeed, but "write" of >40KB will fail with ENOSPC.
 
 The scripts themselves don't require root, but you'll need root to
 mount the images.
 
 As Walter said, it's alarming how many programs fail to handle such
 cases (though D is pretty solid in that regard).


[...]

Very nice! Thanks for sharing. I will keep this in mind next time when I
want to stress-test my program. :-)


T

-- 
Век живи - век учись. А дураком помрёшь.
Aug 01 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/31/2014 1:52 PM, Sean Kelly wrote:

 Could you expand on what you consider input?


All state processed by the program that comes from outside the program. That 
would include:

1. user input
2. the file system
3. uninitialized memory
4. interprocess shared memory
5. anything received from system APIs, device drivers, and DLLs that are not 
part of the program
6. resource availability and exhaustion



 For example, if a
 function has an "in" contract that validates input parameters, is
 the determination that a parameter is invalid a program bug or
 simply invalid input?


An "in" contract failure is a program bug. Contracts are ASSERTIONS ABOUT THE 
CORRECTNESS OF THE PROGRAM LOGIC. They are not assertions about the program's
input.


 If you consider this invalid input that
 should be checked by enforce(), can you explain why?


This says it better than I can:

http://en.wikipedia.org/wiki/Design_by_contract
Jul 31 2014
next sibling parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 31 July 2014 at 21:11:17 UTC, Walter Bright wrote:

 On 7/31/2014 1:52 PM, Sean Kelly wrote:

 Could you expand on what you consider input?


 All state processed by the program that comes from outside the 
 program. That would include:

 1. user input
 2. the file system
 3. uninitialized memory
 4. interprocess shared memory
 5. anything received from system APIs, device drivers, and DLLs 
 that are not part of the program
 6. resource availability and exhaustion


So effectively, any factor occurring at runtime.  If I create a
library, it is acceptable to validate function parameters using
assert() because the user of that library knows what the library
expects and should write their code accordingly.  That's fair.
Jul 31 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 07/31/2014 11:29 PM, Sean Kelly wrote:

 On Thursday, 31 July 2014 at 21:11:17 UTC, Walter Bright wrote:

 On 7/31/2014 1:52 PM, Sean Kelly wrote:

 Could you expand on what you consider input?


 All state processed by the program that comes from outside the
 program. That would include:

 1. user input
 2. the file system
 3. uninitialized memory
 4. interprocess shared memory
 5. anything received from system APIs, device drivers, and DLLs that
 are not part of the program
 6. resource availability and exhaustion


 So effectively, any factor occurring at runtime.  If I create a
 library, it is acceptable to validate function parameters using
 assert() because the user of that library knows what the library
 expects and should write their code accordingly.  That's fair.


It is most fair inside the 'in' contract.
Jul 31 2014
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Thursday, 31 July 2014 at 21:29:59 UTC, Sean Kelly wrote:

 On Thursday, 31 July 2014 at 21:11:17 UTC, Walter Bright wrote:

 On 7/31/2014 1:52 PM, Sean Kelly wrote:

 Could you expand on what you consider input?


 All state processed by the program that comes from outside the 
 program. That would include:

 1. user input
 2. the file system
 3. uninitialized memory
 4. interprocess shared memory
 5. anything received from system APIs, device drivers, and 
 DLLs that are not part of the program
 6. resource availability and exhaustion


 So effectively, any factor occurring at runtime.  If I create a
 library, it is acceptable to validate function parameters using
 assert() because the user of that library knows what the library
 expects and should write their code accordingly.  That's fair.


Yes. It basically comes down to whether you would consider 
incorrect input to the function to be an outright program error 
that should _never_ occur (in which case, you use assertions) or 
whether you consider it reasonable for bad input to be given some 
of the time or unreasonable to require that the caller never give 
bad input (in which case, you use exceptions).

Also, if efficiency is of great concern and it's possible for the 
caller to guarantee that the input is valid (which wouldn't be 
the case with something like files, because even if the input was 
validated, it could become invalid afterwords), then using 
assertions might be a better approach. On the other hand, if 
correctness is of greater concern, then it might make more sense 
to use exceptions, because then it makes sure that the programmer 
is never able to give bad input (though if it really makes more 
sense to treat that as program bug - e.g. giving an invalid index 
- then using Errors or assert(0) might make more sense, since 
they'd stay in but be immediately fatal).

All in all, while some cases are very clear-cut as to whether 
assertions or exceptions should be used, many of other cases are 
highly subjective, at which point it generally comes down to 
whether bad input to the function should be considered a 
programming error or whether it makes more sense for the function 
to throw and allow the program to try and recover from the 
problem.

- Jonathan M Davis
Jul 31 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Thursday, 31 July 2014 at 21:29:59 UTC, Sean Kelly wrote:

 So effectively, any factor occurring at runtime.  If I create a
 library, it is acceptable to validate function parameters using
 assert() because the user of that library knows what the library
 expects and should write their code accordingly.  That's fair.


He should, but what if he doesn't and the library is phobos or 
druntime (which are compiled in release mode)? BTW, druntime 
can't use enforce and doesn't verify its input.
Aug 01 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 1 August 2014 at 08:21:28 UTC, Kagamin wrote:

 On Thursday, 31 July 2014 at 21:29:59 UTC, Sean Kelly wrote:

 So effectively, any factor occurring at runtime.  If I create a
 library, it is acceptable to validate function parameters using
 assert() because the user of that library knows what the 
 library
 expects and should write their code accordingly.  That's fair.


 He should, but what if he doesn't and the library is phobos or 
 druntime (which are compiled in release mode)? BTW, druntime 
 can't use enforce and doesn't verify its input.


Druntime uses contracts and asserts in places. Which are of 
course removed because we ship only a "release" build.  Once 
again, the worst naming for a compiler switch ever. What I really 
want is a way to ship release and non-release builds (ie. checked 
and unchecked) and have the proper one chosen at link time based 
on build flags. Basically toss the -defaultlib and -debuglib and 
replace it with -checkedlib and -uncheckedlib.
Aug 01 2014
next sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 1 August 2014 at 14:10:14 UTC, Sean Kelly wrote:

 Druntime uses contracts and asserts in places. Which are of 
 course removed because we ship only a "release" build.  Once 
 again, the worst naming for a compiler switch ever. What I 
 really want is a way to ship release and non-release builds 
 (ie. checked and unchecked) and have the proper one chosen at 
 link time based on build flags. Basically toss the -defaultlib 
 and -debuglib and replace it with -checkedlib and -uncheckedlib.


As a consequence of this thread I think I am going to change the 
way DMD is packaged on Arch Linux to at least use -debuglib :) 
Right we don't provide even that...
Aug 01 2014
parent reply "Kagamin" <spam here.lot> writes:
On Friday, 1 August 2014 at 14:26:35 UTC, Dicebot wrote:

 On Friday, 1 August 2014 at 14:10:14 UTC, Sean Kelly wrote:

 Druntime uses contracts and asserts in places. Which are of 
 course removed because we ship only a "release" build.  Once 
 again, the worst naming for a compiler switch ever. What I 
 really want is a way to ship release and non-release builds 
 (ie. checked and unchecked) and have the proper one chosen at 
 link time based on build flags. Basically toss the -defaultlib 
 and -debuglib and replace it with -checkedlib and 
 -uncheckedlib.


 As a consequence of this thread I think I am going to change 
 the way DMD is packaged on Arch Linux to at least use -debuglib 
 :) Right we don't provide even that...


Wasn't it a common practice? See e.g. 
http://wxwidgets.blogspot.com/2012/08/how-to-use-294-wxmsw-binaries.html
Aug 01 2014
parent "Dicebot" <public dicebot.lv> writes:
On Friday, 1 August 2014 at 15:23:03 UTC, Kagamin wrote:

 On Friday, 1 August 2014 at 14:26:35 UTC, Dicebot wrote:

 On Friday, 1 August 2014 at 14:10:14 UTC, Sean Kelly wrote:

 Druntime uses contracts and asserts in places. Which are of 
 course removed because we ship only a "release" build.  Once 
 again, the worst naming for a compiler switch ever. What I 
 really want is a way to ship release and non-release builds 
 (ie. checked and unchecked) and have the proper one chosen at 
 link time based on build flags. Basically toss the 
 -defaultlib and -debuglib and replace it with -checkedlib and 
 -uncheckedlib.


 As a consequence of this thread I think I am going to change 
 the way DMD is packaged on Arch Linux to at least use 
 -debuglib :) Right we don't provide even that...


 Wasn't it a common practice? See e.g. 
 http://wxwidgets.blogspot.com/2012/08/how-to-use-294-wxmsw-binaries.html


Right now I mostly mimic content of basic .zip distribution and 
it does not provide debug phobos builds.
Aug 01 2014
prev sibling next sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Sean Kelly"  wrote in message news:tngnltzwxprebpbcdkgm forum.dlang.org...


 Druntime uses contracts and asserts in places. Which are of course removed 
 because we ship only a "release" build.  Once again, the worst naming for 
 a compiler switch ever. What I really want is a way to ship release and 
 non-release builds (ie. checked and unchecked) and have the proper one 
 chosen at link time based on build flags. Basically toss the -defaultlib 
 and -debuglib and replace it with -checkedlib and -uncheckedlib.


While shipping a debug version of druntime would be useful, I think the real 
problem is that checking preconditions is disabled by the callee!  If 
preconditions are violated, the error is in the caller, and checking should 
be enabled based on the application's flags.
Aug 01 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 1 August 2014 at 16:52:00 UTC, Daniel Murphy wrote:

 "Sean Kelly"  wrote in message 
 news:tngnltzwxprebpbcdkgm forum.dlang.org...


 Druntime uses contracts and asserts in places. Which are of 
 course removed because we ship only a "release" build.  Once 
 again, the worst naming for a compiler switch ever. What I 
 really want is a way to ship release and non-release builds 
 (ie. checked and unchecked) and have the proper one chosen at 
 link time based on build flags. Basically toss the -defaultlib 
 and -debuglib and replace it with -checkedlib and 
 -uncheckedlib.


 While shipping a debug version of druntime would be useful, I 
 think the real problem is that checking preconditions is 
 disabled by the callee!  If preconditions are violated, the 
 error is in the caller, and checking should be enabled based on 
 the application's flags.


Exactly.  When the user builds with -release set, DMD should link
the unchecked libphobos.  And without this flag it should link
the checked build.  So the user choice of whether to use
contracts extends at least to the standard library.

Also, I don't like the -debuglib option because when a user sets
the debug flag in their build, what they want is to debug their
own code, not the standard library.  While it's sometimes useful
to have debug symbols in the standard library you're linking
against, it's basically never useful to have internal debug code
enabled in the standard library because even if there's a bug in
that library, you can't do anything about it.  And presumably the
library has been tested against that debug code anyway.
Aug 01 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 3:34 PM, Sean Kelly wrote:

 Also, I don't like the -debuglib option because when a user sets
 the debug flag in their build, what they want is to debug their
 own code, not the standard library.  While it's sometimes useful
 to have debug symbols in the standard library you're linking
 against, it's basically never useful to have internal debug code
 enabled in the standard library because even if there's a bug in
 that library, you can't do anything about it.  And presumably the
 library has been tested against that debug code anyway.


It's mainly useful for having a library with symbolic debug info in it, which 
makes it easier to work with in a symbolic debugger.
Aug 01 2014
prev sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 01, 2014 at 10:34:20PM +0000, Sean Kelly via Digitalmars-d wrote:

 On Friday, 1 August 2014 at 16:52:00 UTC, Daniel Murphy wrote:

"Sean Kelly"  wrote in message
news:tngnltzwxprebpbcdkgm forum.dlang.org...


Druntime uses contracts and asserts in places. Which are of course
removed because we ship only a "release" build.  Once again, the worst
naming for a compiler switch ever. What I really want is a way to ship
release and non-release builds (ie. checked and unchecked) and have the
proper one chosen at link time based on build flags. Basically toss the
-defaultlib and -debuglib and replace it with -checkedlib and
-uncheckedlib.


While shipping a debug version of druntime would be useful, I think the
real problem is that checking preconditions is disabled by the callee!  If
preconditions are violated, the error is in the caller, and checking
should be enabled based on the application's flags.


 
 Exactly.  When the user builds with -release set, DMD should link
 the unchecked libphobos.  And without this flag it should link
 the checked build.  So the user choice of whether to use
 contracts extends at least to the standard library.


[...]

I don't like this. This would mean that every D library, whether it's
Phobos or some 3rd party library, must now ship with checked & unchecked
versions, if they want DbC to work.

IMO the correct solution is for the compiler to insert preconditions at
the calling site, rather than the callee. The separate compilation
problem can be solved by having dmd include preconditions in .di files
so that binary-only distributions will still work.


T

-- 
There are two ways to write error-free programs; only the third one works.
Aug 01 2014
next sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 1 August 2014 at 23:13:28 UTC, H. S. Teoh via 
Digitalmars-d wrote:


 IMO the correct solution is for the compiler to insert 
 preconditions at the calling site, rather than the callee.


If we had that, I'd actually start using in blocks. As it is, I 
think that they're useless except when inheritance is involved, 
and they're more verbose than just putting the assertions at the 
top of the function, so I don't bother with the in block.

But certainly, in principle, I agree that whether contracts are 
used would depend on the flags used when compiler the caller, not 
the callee, but the separate compilation model does throw some 
kinks in that. But even if it just worked in cases where the 
source code was available, it would be an improvement.

- Jonathan M Davis
Aug 01 2014
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 01:19 AM, Jonathan M Davis wrote:

 If we had that, I'd actually start using in blocks. As it is, I think
 that they're useless except when inheritance is involved,


And when it is involved, they are broken because the compiler insists on 
removing them on each override by default.
Aug 01 2014
prev sibling parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"Jonathan M Davis"  wrote in message 
news:zbkvnbibbmcfwhjvmhau forum.dlang.org...


 IMO the correct solution is for the compiler to insert preconditions at 
 the calling site, rather than the callee.


 If we had that, I'd actually start using in blocks. As it is, I think that 
 they're useless except when inheritance is involved, and they're more 
 verbose than just putting the assertions at the top of the function, so I 
 don't bother with the in block.


What do you think about 
https://github.com/D-Programming-Language/dmd/pull/3799 ?

Basically, turn obviously invalid failing function calls into compile-time 
errors.  IMO this pushes in-contracts well out of the useless area.
Aug 02 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Daniel Murphy:


 What do you think about 
 https://github.com/D-Programming-Language/dmd/pull/3799 ?

 Basically, turn obviously invalid failing function calls into 
 compile-time errors.  IMO this pushes in-contracts well out of 
 the useless area.


I like compile-time tests, you can see it from several of my last 
posts, but:
1) It's better to be able to express general tests using CTFE, 
instead of covering a limited but growing list of cases. The 
general solution needs only half page of text to be explained in 
a next version of the TDPL, while your proposal will need a 
growing list of cases;
2) For reasons Walter has explained elsewhere, those tests need 
to be explicit, even syntactically-wise (and Walter has a strong 
opinion on this, so it's unlikely you will change his mind. And I 
agree with him on this).

So far I've seen only one proposal to do this (the Enum 
Preconditions), and it too has some limits (caused by compilation 
units, that can be removed turning the Enum Preconditions into 
things handled like templates).

Bye,
bearophile
Aug 02 2014
parent reply "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"bearophile"  wrote in message news:hnbybyssdlwaomescwvg forum.dlang.org...


 I like compile-time tests, you can see it from several of my last posts, 
 but:
 1) It's better to be able to express general tests using CTFE, instead of 
 covering a limited but growing list of cases. The general solution needs 
 only half page of text to be explained in a next version of the TDPL, 
 while your proposal will need a growing list of cases;


This doesn't need a lot of text to explain either: "when possible, the 
compiler will check preconditions at compile time"


 2) For reasons Walter has explained elsewhere, those tests need to be 
 explicit, even syntactically-wise (and Walter has a strong opinion on 
 this, so it's unlikely you will change his mind. And I agree with him on 
 this).


I'm really only interested in Walter's opinion when it's coming from Walter. 
I've responded to Walter's objections and we'll see where that goes.


 So far I've seen only one proposal to do this (the Enum Preconditions), 
 and it too has some limits (caused by compilation units, that can be 
 removed turning the Enum Preconditions into things handled like 
 templates).


I feel like you're confusing this pull request with another enhancement. 
The discussion should be about whether this exact feature is worthwhile, not 
if some other feature would solve some other problems.  Do you really think 
the compiler opportunistically checking preconditions would be a bad thing 
for D?
Aug 02 2014
parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Daniel Murphy:


 This doesn't need a lot of text to explain either: "when 
 possible, the compiler will check preconditions at compile time"


What defines "possible"? A compiler switch that allows to define 
the amount of compile time?



 I feel like you're confusing this pull request with another 
 enhancement. The discussion should be about whether this exact 
 feature is worthwhile, not if some other feature would solve 
 some other problems.


The comparison of the two ideas is useful, because they try to 
solve the same problem.



 Do you really think the compiler opportunistically checking
 preconditions would be a bad thing for D?


I think that adding an undefined, growing, compiled-dependent 
list of static checks that can cause unpredictable compilation 
times is not a good idea. A more general solution has some 
advantages:
- It allows the user to choose what to run at compile-time and 
what at run-time;
- You can defined _exactly_ what it does in a book, unlike the 
growing list;
- Will not grow in features and implementation complexity as 
compilers improve (and I think it's sufficiently simple);
- It's going to be mostly the same in all D compilers;
- It allows to specify and run arbitrary tests in CTFE instead of 
a set of predefined by unknown simple tests. So you can use it 
for SafeIntegers literals, and many other future and 
unpredictable purposes;
- It's explicitly visible in the code, because there is a syntax 
to denote such static testing parts in the code.

Implementing a special-cased solution instead of a more general 
and principled solution could make the implementation of the 
better solution more unlikely :-)

Bye,
bearophile
Aug 02 2014
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
By the way, I will be absent for more than a week. My comments or 
answers in bug reports and threads will have to wait.

Bye,
bearophile
Aug 02 2014
prev sibling parent "Daniel Murphy" <yebbliesnospam gmail.com> writes:
"bearophile"  wrote in message news:nadqefjfnjfzghopeqyf forum.dlang.org...


 What defines "possible"? A compiler switch that allows to define the 
 amount of compile time?


At the moment it's the criteria I listed in the bug report.  Beyond this, it 
will be limited by what constfolding is capable of.  What the exact 
limitations of constfolding are is an evolving part of the language.


 I feel like you're confusing this pull request with another enhancement. 
 The discussion should be about whether this exact feature is worthwhile, 
 not if some other feature would solve some other problems.


 The comparison of the two ideas is useful, because they try to solve the 
 same problem.


I don't think they do.  If you force evaluation of preconditions at compile 
time, you force arguments (or properties of arguments) to be known at 
compile time.  With best-effort checking, you simply get to promote run-time 
errors to compile-time.


 I think that adding an undefined, growing, compiled-dependent list of 
 static checks that can cause unpredictable compilation times is not a good 
 idea. A more general solution has some advantages:
 - It allows the user to choose what to run at compile-time and what at 
 run-time;


Again, these are orthagonal.  I don't care if you define conditions that 
must be met at compile-time, these are about promoting run-time conditions 
to compile time.


 - You can defined _exactly_ what it does in a book, unlike the growing 
 list;


IMO this is only a mild problem.  If they can't be caught at compile time, 
they will still be caught at run-time.


 - Will not grow in features and implementation complexity as compilers 
 improve (and I think it's sufficiently simple);


I don't understand this.  You don't want static checking to improve over 
time?  This is simply a type of static checking.


 - It's going to be mostly the same in all D compilers;


Again, it's a promotion.  It doesn't matter if every frontend doesn't 
implement it, lesser frontends will just mean you catch it at runtime.


 - It allows to specify and run arbitrary tests in CTFE instead of a set of 
 predefined by unknown simple tests. So you can use it for SafeIntegers 
 literals, and many other future and unpredictable purposes;
 - It's explicitly visible in the code, because there is a syntax to denote 
 such static testing parts in the code.


This has nothing to do with promoting run-time checks to compile-time.


 Implementing a special-cased solution instead of a more general and 
 principled solution could make the implementation of the better solution 
 more unlikely :-)


Maybe.  But I think you should be seeing this a stepping stone, not 
competition.

eg all developers like it when the compiler points out guaranteed wrong 
code:

ubyte x = 0xFFF;

but many (including me) find it very annoying when the compiler makes you 
alter code that it thinks _might_ be wrong:

ubyte x = y; // where y is a uint, that you know to be < 256
Aug 02 2014
prev sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
H. S. Teoh:


 IMO the correct solution is for the compiler to insert 
 preconditions at the calling site,


I have seen this suggestion tens of times, but nothing has 
happened. (delegates make the management of contract blame more 
compex).

Bye,
bearophile
Aug 01 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 01, 2014 at 11:24:29PM +0000, bearophile via Digitalmars-d wrote:

 H. S. Teoh:
 

IMO the correct solution is for the compiler to insert preconditions
at the calling site,


 
 I have seen this suggestion tens of times, but nothing has happened.
 (delegates make the management of contract blame more compex).


[...]

I know, I wish I had the time and the know-how to actually implement
this in dmd. I'm tired of the all-talk-and-no-action on the forum. Is
there at least an enhancement request for this though?

The point about delegates is kinda interesting, though. Can they
actually have contracts attached to them? I'd like to think that the
contracts really ought to attach to the *type* of the delegate rather
than the delegate itself. The reason is that I don't see how it would
make sense if you had something like this:

	alias DgType = int delegate(int);

	int f1(int x) in { assert(x < 0); } body { ... }
	int f2(int x) in { assert(x > 0); } body { ... }

	// How does the following make any sense?
	void func(DgType dg, int x) {
		// How are we supposed to know what values of x are
		// acceptable?
		dg(x);
	}

	func(&f1, 1);	// oops
	func(&f2, -1);	// oops

Basically, given a DgType of unknown origin, it's impossible to reliably
tell what arguments you ought to pass to it. So the compiler wouldn't
know which in-contract to insert in func().

OTOH, perhaps one way to work around this, is to have a function with an
in-contract compile into a preamble and a body:

	int func(int x)
	in { assert(x > 5); }
	body {
		return computeResult(x);
	}

would compile to the equivalent of:

	int __func_incontract(int x) {
		assert(x > 5);
		goto __func_body;	// fall through to __func_body
	}
	int __func_body(int x) {
		return computeResult(x);
	}

In non-release mode, calls to func would get translated into calls to
__func_incontract, but in release mode, calls to func get translated
into direct calls to __func_body. In both cases, the compiler always
emits both __func_incontract and __func_body. If the ..._incontract
symbols are never referenced by the final program, the linker discards
them at link time.

This can be applied to both normal functions and delegates. In
non-release mode, the (address of the) delegate would point to its
..._incontract portion, whereas in release mode, the (address of the)
delegate would point directly to its ..._body portion. This way, if the
caller is compiled in release mode, it will skip the contract, but if
called in non-release mode, it will run the contract.


T

-- 
MACINTOSH: Most Applications Crash, If Not, The Operating System Hangs
Aug 01 2014
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 01:46 AM, H. S. Teoh via Digitalmars-d wrote:

 OTOH, perhaps one way to work around this, is to have a function with an
 in-contract compile into a preamble and a body:

 	int func(int x)
 	in { assert(x > 5); }
 	body {
 		return computeResult(x);
 	}

 would compile to the equivalent of:

 	int __func_incontract(int x) {
 		assert(x > 5);
 		goto __func_body;	// fall through to __func_body
 	}
 	int __func_body(int x) {
 		return computeResult(x);
 	}

 In non-release mode, calls to func would get translated into calls to
 __func_incontract,


What if a library function was compiled in release mode?
Aug 01 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Aug 02, 2014 at 01:59:29AM +0200, Timon Gehr via Digitalmars-d wrote:

 On 08/02/2014 01:46 AM, H. S. Teoh via Digitalmars-d wrote:

OTOH, perhaps one way to work around this, is to have a function with
an in-contract compile into a preamble and a body:

	int func(int x)
	in { assert(x > 5); }
	body {
		return computeResult(x);
	}

would compile to the equivalent of:

	int __func_incontract(int x) {
		assert(x > 5);
		goto __func_body;	// fall through to __func_body
	}
	int __func_body(int x) {
		return computeResult(x);
	}

In non-release mode, calls to func would get translated into calls to
__func_incontract,


 
 What if a library function was compiled in release mode?


The compiler always emits the in-contract, so the library would carry
all the in-contracts. If the user code doesn't actually use them, then
the linker just doesn't link them in at link time.


T


-- 
Живёшь только однажды.
Aug 01 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 02:04 AM, H. S. Teoh via Digitalmars-d wrote:

 On Sat, Aug 02, 2014 at 01:59:29AM +0200, Timon Gehr via Digitalmars-d wrote:

 On 08/02/2014 01:46 AM, H. S. Teoh via Digitalmars-d wrote:

 OTOH, perhaps one way to work around this, is to have a function with
 an in-contract compile into a preamble and a body:

 	int func(int x)
 	in { assert(x > 5); }
 	body {
 		return computeResult(x);
 	}

 would compile to the equivalent of:

 	int __func_incontract(int x) {
 		assert(x > 5);
 		goto __func_body;	// fall through to __func_body
 	}
 	int __func_body(int x) {
 		return computeResult(x);
 	}

 In non-release mode, calls to func would get translated into calls to
 __func_incontract,


 What if a library function was compiled in release mode?


 The compiler always emits the in-contract,


I know.


 so the library would carry all the in-contracts.


Indeed.


 If the user code doesn't actually use them, then
 the linker just doesn't link them in at link time.
 ...


I was trying to half-jokingly make a pedantic point about the nature of 
"the equivalent of: [...] assert(x > 5); [...]" in release mode.

Another issue is that if one just emits the above lowering, the 
optimizers might not be able to assume the in contract correct on entry 
of __func__body without further precautions.
Aug 01 2014
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 4:24 PM, bearophile wrote:

 H. S. Teoh:


 IMO the correct solution is for the compiler to insert preconditions at the
 calling site,


 I have seen this suggestion tens of times, but nothing has happened. (delegates
 make the management of contract blame more compex).


There's a bugzilla issue on it, considerable discussion there, and consensus 
that it's the correct solution. But nobody has implemented it yet.
Aug 02 2014
prev sibling parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 1 August 2014 at 14:10:14 UTC, Sean Kelly wrote:

 On Friday, 1 August 2014 at 08:21:28 UTC, Kagamin wrote:

 On Thursday, 31 July 2014 at 21:29:59 UTC, Sean Kelly wrote:

 So effectively, any factor occurring at runtime.  If I create 
 a
 library, it is acceptable to validate function parameters 
 using
 assert() because the user of that library knows what the 
 library
 expects and should write their code accordingly.  That's fair.


 He should, but what if he doesn't and the library is phobos or 
 druntime (which are compiled in release mode)? BTW, druntime 
 can't use enforce and doesn't verify its input.


 Druntime uses contracts and asserts in places. Which are of 
 course removed because we ship only a "release" build.  Once 
 again, the worst naming for a compiler switch ever. What I 
 really want is a way to ship release and non-release builds 
 (ie. checked and unchecked) and have the proper one chosen at 
 link time based on build flags. Basically toss the -defaultlib 
 and -debuglib and replace it with -checkedlib and -uncheckedlib.


Well, to be fair, it's pretty standard practice to remove all 
assertions in release builds in other languages. If anything, 
leaving them in is highly abnormal. Sure, there are reasons to do 
it, but most folks don't. So, while I can see why you don't like 
the switch being called -release, it's behavior matches what most 
people would do for release builds. If anything, I think that 
-debug is the switch that's problematic, because people think 
that it's the opposite of release when it's completely unrelated 
(and useless for most code, because most people aren't going to 
be using debug blocks - especially if they don't understand the 
-debug switch).

But we probably would be better off if none of the switches had 
names like -release or -debug so that folks actually had to 
figure out what they did before using them rather than simply 
assuming that one is for release builds and the other is for 
debug builds.

- Jonathan M Davis
Aug 01 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Fri, Aug 01, 2014 at 09:42:06PM +0000, Jonathan M Davis via Digitalmars-d
wrote:

 On Friday, 1 August 2014 at 14:10:14 UTC, Sean Kelly wrote:


[...]

 But we probably would be better off if none of the switches had names
 like -release or -debug so that folks actually had to figure out what
 they did before using them rather than simply assuming that one is for
 release builds and the other is for debug builds.


[...]

Great. So let's rename all of dmd's command-line options to -a, -b, -c,
-d, -e, -f, ... (in arbitrary order). As long as we document it all, it
will work just perfectly fine, right? After all, it *does* force users
to *really* know what each option does. :-D


T

-- 
Gone Chopin. Bach in a minuet.
Aug 01 2014
parent reply "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Friday, 1 August 2014 at 21:51:10 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Fri, Aug 01, 2014 at 09:42:06PM +0000, Jonathan M Davis via 
 Digitalmars-d wrote:

 On Friday, 1 August 2014 at 14:10:14 UTC, Sean Kelly wrote:


 [...]

 But we probably would be better off if none of the switches 
 had names
 like -release or -debug so that folks actually had to figure 
 out what
 they did before using them rather than simply assuming that 
 one is for
 release builds and the other is for debug builds.


 [...]

 Great. So let's rename all of dmd's command-line options to -a, 
 -b, -c,
 -d, -e, -f, ... (in arbitrary order). As long as we document it 
 all, it
 will work just perfectly fine, right? After all, it *does* 
 force users
 to *really* know what each option does. :-D


LOL. That would be stupid. No, what would make sense would be 
something like --remove-assertions.

But regardless, I'm not advocating that we change the switches. I 
don't think that it's a big enough problem to merit that. But I 
do think that we could have picked better names.

The one that always stumps me though is -debug. It's a bad name, 
because it makes people think debug/release, when all it does is 
enable debug blocks, and it can actually be used in release 
builds, but given that they're debug blocks, -debug makes sense. 
And I can't think of a better name for it. But even if I could 
think of a better name, I think that we're stuck with -debug at 
this point.

- Jonathan M Davis
Aug 01 2014
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 12:58 AM, Jonathan M Davis wrote:

 something like --remove-assertions.


What would that do?
Aug 01 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Sat, Aug 02, 2014 at 01:46:44AM +0200, Timon Gehr via Digitalmars-d wrote:

 On 08/02/2014 12:58 AM, Jonathan M Davis wrote:

something like --remove-assertions.


 
 What would that do?


Automatically edit your source files and delete all assert expressions
for you.

:-D


T

-- 
Why waste time learning, when ignorance is instantaneous? -- Hobbes, from
Calvin & Hobbes
Aug 01 2014
parent Timon Gehr <timon.gehr gmx.ch> writes:
On 08/02/2014 01:51 AM, H. S. Teoh via Digitalmars-d wrote:

 On Sat, Aug 02, 2014 at 01:46:44AM +0200, Timon Gehr via Digitalmars-d wrote:

 On 08/02/2014 12:58 AM, Jonathan M Davis wrote:

 something like --remove-assertions.


 What would that do?


 Automatically edit your source files and delete all assert expressions
 for you.

 :-D


 T


I feared it would!
Aug 01 2014
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 3:58 PM, Jonathan M Davis wrote:

 But even if I
 could think of a better name, I think that we're stuck with -debug at this
point.


One thing that people hate even more than breaking existing code is breaking 
their makefile (or equivalent).

Makefiles typically are cut&pasted from some other project, with the switches 
copied over without review. Worse, the switches sometimes come in from macros, 
and scripts are written to drive the makefiles, and it becomes a wholly 
unmaintainable tangle.

Programmers have gotten a lot better about writing clear, maintainable code,
but 
I can't say the same for the build process. Those are worse than ever.

https://www.youtube.com/watch?v=CHgUN_95UAw
Aug 02 2014
parent Johannes Pfau <nospam example.com> writes:
Am Sat, 02 Aug 2014 12:34:54 -0700
schrieb Walter Bright <newshound2 digitalmars.com>:


 On 8/1/2014 3:58 PM, Jonathan M Davis wrote:

 But even if I
 could think of a better name, I think that we're stuck with -debug
 at this point.


 
 One thing that people hate even more than breaking existing code is
 breaking their makefile (or equivalent).
 
 Makefiles typically are cut&pasted from some other project, with the
 switches copied over without review. Worse, the switches sometimes
 come in from macros, and scripts are written to drive the makefiles,
 and it becomes a wholly unmaintainable tangle.
 
 Programmers have gotten a lot better about writing clear,
 maintainable code, but I can't say the same for the build process.
 Those are worse than ever.
 
 https://www.youtube.com/watch?v=CHgUN_95UAw


Wait - are you arguing that we can't change the name of a compiler flag
because users would have to do a 'search&replace' in their Makefiles,
but breaking existing code (assert/assume) which will be very hard to
track down for users is OK, cause 'that code was broken anyway'?


We could also argue that Makefiles using '-debug' are broken anyway,
cause the name debug doesn't match the behavior of the compiler and
users will likely expect a different behavior.
Aug 02 2014
prev sibling parent "Sean Kelly" <sean invisibleduck.org> writes:
On Friday, 1 August 2014 at 21:42:07 UTC, Jonathan M Davis wrote:

 On Friday, 1 August 2014 at 14:10:14 UTC, Sean Kelly wrote:

 On Friday, 1 August 2014 at 08:21:28 UTC, Kagamin wrote:

 On Thursday, 31 July 2014 at 21:29:59 UTC, Sean Kelly wrote:

 So effectively, any factor occurring at runtime.  If I 
 create a
 library, it is acceptable to validate function parameters 
 using
 assert() because the user of that library knows what the 
 library
 expects and should write their code accordingly.  That's 
 fair.


 He should, but what if he doesn't and the library is phobos 
 or druntime (which are compiled in release mode)? BTW, 
 druntime can't use enforce and doesn't verify its input.


 Druntime uses contracts and asserts in places. Which are of 
 course removed because we ship only a "release" build.  Once 
 again, the worst naming for a compiler switch ever. What I 
 really want is a way to ship release and non-release builds 
 (ie. checked and unchecked) and have the proper one chosen at 
 link time based on build flags. Basically toss the -defaultlib 
 and -debuglib and replace it with -checkedlib and 
 -uncheckedlib.


 Well, to be fair, it's pretty standard practice to remove all 
 assertions in release builds in other languages. If anything, 
 leaving them in is highly abnormal. Sure, there are reasons to 
 do it, but most folks don't. So, while I can see why you don't 
 like the switch being called -release, it's behavior matches 
 what most people would do for release builds.


But is it what they *should* do?  Quoting Matthew Wilson's "The
Nuclear Reactor and the Deep Space Probe"
(http://www.artima.com/cppsource/deepspaceP.html):

"Assertions in C and C++ tend to be included in debug builds
only, and elided in release builds for performance reasons.
However, as the programming community appreciation for contract
programming grows, there is increasing interest in using contract
enforcement in release builds. Indeed, as we'll see in part 4 of
this article, there is a very good argument for leaving contracts
in released software, since the alternative may be worse,
sometimes catastrophically so."

I wish the rest of the series were available online.  I think I
had copies of later versions but can't find them.
Aug 01 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 31 July 2014 at 21:11:17 UTC, Walter Bright wrote:

 On 7/31/2014 1:52 PM, Sean Kelly wrote:

 Could you expand on what you consider input?


 All state processed by the program that comes from outside the 
 program. That would include:

 1. user input
 2. the file system
 3. uninitialized memory
 4. interprocess shared memory
 5. anything received from system APIs, device drivers, and DLLs 
 that are not part of the program
 6. resource availability and exhaustion



 For example, if a
 function has an "in" contract that validates input parameters, 
 is
 the determination that a parameter is invalid a program bug or
 simply invalid input?


 An "in" contract failure is a program bug. Contracts are 
 ASSERTIONS ABOUT THE CORRECTNESS OF THE PROGRAM LOGIC. They are 
 not assertions about the program's input.


 If you consider this invalid input that
 should be checked by enforce(), can you explain why?


 This says it better than I can:

 http://en.wikipedia.org/wiki/Design_by_contract


ok, can this be considered a good summary of using 
assertions/contracts for services where risk of entering 
undefined state is unacceptable?

1) never use `assert` or contracts in actual application code, 
use `enforce` instead
2) never use `enforce` in library code unless it does actual I/O, 
use contracts instead
3) always distribute both release and debug builds of libraries 
and always run tests in both debug and release mode

Does it make sense? Your actual recommendation contradict each 
other but it is best what I was able to combine them into.
Aug 01 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 6:12 AM, Dicebot wrote:

 ok, can this be considered a good summary of using assertions/contracts for
 services where risk of entering undefined state is unacceptable?

 1) never use `assert` or contracts in actual application code, use `enforce`
 instead
 2) never use `enforce` in library code unless it does actual I/O, use contracts
 instead
 3) always distribute both release and debug builds of libraries and always run
 tests in both debug and release mode

 Does it make sense? Your actual recommendation contradict each other but it is
 best what I was able to combine them into.


What makes me hesitate about use of enforce() is its high runtime cost. It's
not 
just the computation, but the call stack above it is affected by enforce()
being 
throwable and allocating via the GC.

Secondly, enforce() is about recoverable errors. Program bugs are simply NOT 
recoverable errors, and I cannot recommend using them for that purpose. I've 
argued for decades with people who insist that they can write code that
recovers 
from unknown programming bugs.
Aug 01 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Friday, 1 August 2014 at 19:10:34 UTC, Walter Bright wrote:

 On 8/1/2014 6:12 AM, Dicebot wrote:

 ok, can this be considered a good summary of using 
 assertions/contracts for
 services where risk of entering undefined state is 
 unacceptable?

 1) never use `assert` or contracts in actual application code, 
 use `enforce`
 instead
 2) never use `enforce` in library code unless it does actual 
 I/O, use contracts
 instead
 3) always distribute both release and debug builds of 
 libraries and always run
 tests in both debug and release mode

 Does it make sense? Your actual recommendation contradict each 
 other but it is
 best what I was able to combine them into.


 What makes me hesitate about use of enforce() is its high 
 runtime cost. It's not just the computation, but the call stack 
 above it is affected by enforce() being throwable and 
 allocating via the GC.


`enforce` does have overload which allows to use pre-allocated 
exception instance. Or, alternatively, we may need a similar 
helper wich does `if (!condition) abort()` with no interaction 
with -release or optimizer.

Whatever is chosen it does seem that contracts can't be used in 
application code because it is too affected by user input and is 
too likely to expose hidden bugs in production.


 Secondly, enforce() is about recoverable errors. Program bugs 
 are simply NOT recoverable errors, and I cannot recommend using 
 them for that purpose. I've argued for decades with people who 
 insist that they can write code that recovers from unknown 
 programming bugs.


1) one can always `enforce` with a pre-allocated Error, will 
druntime handle that as expected?

2) There is certain class of applications where even programming 
bugs can (and must be) considered recoverable. Network services 
that don't yet have full scale high availability infrastructure 
(and thus can't afford downtime of restarting the app) are quite 
likely to only terminate connection fibers when programming bug 
is found - it does not affect serving any other connections. It 
may be fundamentally wrong but is is pragmatical working approach.

These questions are not theoretical - we regularly have 
discussion about how contracts may / should be used at work with 
no clear understanding / agreement so far. I am interested in 
simple set of guidelines that won't turn writing every single 
function in a guessing game (does a contract fit here?)
Aug 01 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 8/1/2014 5:57 PM, Dicebot wrote:

 `enforce` does have overload which allows to use pre-allocated exception
 instance. Or, alternatively, we may need a similar helper wich does `if
 (!condition) abort()` with no interaction with -release or optimizer.


It still throws a *recoverable* exception, which requires the call stack to be 
populated with exception handling frames. Worse, it implies that program bugs 
are recoverable errors, when they are not.



 1) one can always `enforce` with a pre-allocated Error, will druntime handle
 that as expected?


I don't know, but I don't see why not.



 2) There is certain class of applications where even programming bugs can (and
 must be) considered recoverable. Network services that don't yet have full
scale
 high availability infrastructure (and thus can't afford downtime of restarting
 the app) are quite likely to only terminate connection fibers when programming
 bug is found - it does not affect serving any other connections. It may be
 fundamentally wrong but is is pragmatical working approach.

 These questions are not theoretical - we regularly have discussion about how
 contracts may / should be used at work with no clear understanding / agreement
 so far. I am interested in simple set of guidelines that won't turn writing
 every single function in a guessing game (does a contract fit here?)


Putting it another way, consider the rules:

1. A newbie follows the rules because he's told to
2. A master follows the rules because he knows why the rules are good
3. A guru breaks the rules because he knows when the rules don't apply

As to the substance of your question, I can't do it proper justice in a few 
lines. It's an important issue, and it is worthwhile to thoroughly understand 
it, especially for the kind of programming you do and the leading role you have 
in it.

For that I recommend "Object Oriented Software Construction" by Meyers.

http://www.amazon.com/Object-Oriented-Software-Construction-Book-CD-ROM/dp/0136291554/

Don't let the OOP title throw you off. It is a very thorough treatment of 
contract programming, and the rules and their rationales, but in a readable 
manner. You won't be sorry you read it. It's only $34 used, a bargain.
Aug 01 2014
parent "Dicebot" <public dicebot.lv> writes:
On Saturday, 2 August 2014 at 06:01:30 UTC, Walter Bright wrote:

 As to the substance of your question, I can't do it proper 
 justice in a few lines. It's an important issue, and it is 
 worthwhile to thoroughly understand it, especially for the kind 
 of programming you do and the leading role you have in it.

 For that I recommend "Object Oriented Software Construction" by 
 Meyers.

 http://www.amazon.com/Object-Oriented-Software-Construction-Book-CD-ROM/dp/0136291554/

 Don't let the OOP title throw you off. It is a very thorough 
 treatment of contract programming, and the rules and their 
 rationales, but in a readable manner. You won't be sorry you 
 read it. It's only $34 used, a bargain.


Thanks, will read!
Aug 02 2014