www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Suggested Change to Contract Syntax

reply FatalCatharsis <wykennedy gmail.com> writes:
I am very new to D so I apologize if I'm very uninformed. I'm 
learning D by porting a big (awful) c++ project to D so that I 
may take advantage of all the lovely testing and QA features. I 
am currently moving a class over that I have which represents a 2 
float vector and I've run into a problem with contracts. Here is 
a short excerpt:

struct Vector2(T) {
     T x = 0;
     T y = 0;

     Vector2!T opBinary(string op)(const ref Vector2!T rhs)
         if(op == "+" || op == "-" || op == "*" || op == "/")
     in {
         T prevx = this.x;
         T prevy = this.y;
     }
     out {
         static if(isFloatingPoint!T) {
             assert(mixin("approxEqual(this.x, 
prevx"~op~"rhs.x)"));
             assert(mixin("approxEqual(this.y, 
prevy"~op~"rhs.y)"));
         } else {
             assert(mixin("this.x == (prevx"~op~"rhs.x)"));
             assert(mixin("this.y == (prevy"~op~"rhs.y)"));
         }
     }
     body {
         Vector2!T ret;
         mixin("ret.x = this.x"~op~"rhs.x;");
         mixin("ret.y = this.y"~op~"rhs.y;");
         return ret;
     }
}

this example obviously does not compile. What I am attempting to 
do is store the initial value of the x and y before the body is 
run, so that I can use those values in the post condition. I've 
read around and asked a question on stackoverflow, and it seems 
that there is no facility for this.

I am NOT very knowledgable in compilation, but I do get the gist 
of it. I was hoping to suggest a syntax change and get some 
opinions on it. What if contracts were done like this:

struct Vector2(T) {
     T x = 0;
     T y = 0;

     Vector2!T opBinary(string op)(const ref Vector2!T rhs)
         if(op == "+" || op == "-" || op == "*" || op == "/")
     contract {
         T prevx;
         T prevy;

         in {
             prevx = this.x;
             prevy = this.y;
         }
         out {
             static if(isFloatingPoint!T) {
                 assert(mixin("approxEqual(this.x, 
prevx"~op~"rhs.x)"));
                 assert(mixin("approxEqual(this.y, 
prevy"~op~"rhs.y)"));
             } else {
                 assert(mixin("this.x == (prevx"~op~"rhs.x)"));
                 assert(mixin("this.y == (prevy"~op~"rhs.y)"));
             }
         }
     }
     body {
         Vector2!T ret;
         mixin("ret.x = this.x"~op~"rhs.x;");
         mixin("ret.y = this.y"~op~"rhs.y;");
         return ret;
     }
}

In this example, the contract would represent a function that is 
invoked on each invocation of the function. in and out would 
enclose the values prevx and prevy. in would be invoked 
immediately after, followed by the body, and then followed by out.

Syntactically I think this is clear, but I don't know much about 
the technical implementation. Is this feasible?
Mar 10 2016
parent reply Xinok <xinok live.com> writes:
On Thursday, 10 March 2016 at 21:07:09 UTC, FatalCatharsis wrote:

 I am very new to D so I apologize if I'm very uninformed. I'm 
 learning D by porting a big (awful) c++ project to D so that I 
 may take advantage of all the lovely testing and QA features. I 
 am currently moving a class over that I have which represents a 
 2 float vector and I've run into a problem with contracts. Here 
 is a short excerpt:
 ...


The most "elegant" solution I can think of is to move the 
contracts into the body of the function itself and wrap them in 
version(unittest) or version(assert). The pre-contract would be 
placed at the very start of the function and the post-contract 
would be wrapped in a scope(exit) or scope(success).

Regarding your proposal, I don't think it's necessary to 
introduce new syntax; a simple change in semantics would suffice. 
If we simply preserved the body of the pre-contract and made it 
accessible in the post-contract, then your example would work as 
is.

I'm not sure how the compilers translate the contracts into code, 
but it's definitely feasible. Code of the form:

     auto foo()
     in{ ... }
     out(result){ ... }
     body{ ... }

Could simply be rewritten as:

     auto foo()
     {
         // paste pre-contract here

         auto bodyOfFoo()
         {
             // paste body here
         }

         auto result = bodyOfFoo();

         // paste post-contract here

         return result;
     }
Mar 10 2016
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 10 March 2016 at 22:04:24 UTC, Xinok wrote:

 I'm not sure how the compilers translate the contracts into 
 code, but it's definitely feasible. Code of the form:

     auto foo()
     in{ ... }
     out(result){ ... }
     body{ ... }

 Could simply be rewritten as:




Not quite because contracts are inherited. For a child class, 
either the child OR the parent's in contract needs to pass, but 
both the child AND parent's out contracts need to pass.

The result is that the child in contract might run without the 
parent in... but then the parent's out will still be run.

Will it see uninitialized variables? Or partially run stuff as 
the in contract throws a swallowed exception half way through?


Contracts are most interesting in the case of inheritance and 
keeping variables between them isn't going to be easy.
Mar 10 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 10 March 2016 at 22:39:54 UTC, Adam D. Ruppe wrote:

 Contracts are most interesting in the case of inheritance and 
 keeping variables between them isn't going to be easy.


IMHO, at this point, inheritance is the only reason they're worth 
having in the language. Without inheritance, in contracts could 
just as easily be assertions at the top of the function, and 
while out contracts are certainly easier as they are now rather 
than having to put a contract at each return statement or use a 
scope(exit) statement to do the out contracts, you can still do 
out contracts that way, and honestly, I don't think that out 
contracts are worth much anyway, since in almost all cases, unit 
tests do that job already, and it's usually much easier to write 
test that specific input gives specific output than it is to have 
a general test at the end of the function.

But while in most cases, in and out contracts are trivially done 
in the function itself, inheritance is a whole other kettle of 
fish, and having them built into the language solves that problem 
whereas doing it yourself is actually fairly hard and 
error-prone. So, for that reason, and that reason alone, I think 
that having them built into the language is a good thing.

There has been some discussion of getting the compiler to catch 
some stuff based on in contracts, in which case, in contracts 
would be worth a bit more, but as it stands, I pretty much never 
use out contracts (because I think that they're useless), and if 
inheritance isn't involved (which it usually isn't), then I don't 
even bother with an in contract and just put the assertions in 
the body and avoid the extra syntax. If/Once we get more features 
in the compiler which take advantage of contracts, then I'll 
likely change my tune on that one, but for now, IMHO, they're 
really only of value in classes.

- Jonathan M Davis
Mar 10 2016
next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 10 March 2016 at 22:57:41 UTC, Jonathan M Davis 
wrote:

 IMHO, at this point, inheritance is the only reason they're 
 worth
 having in the language. [snip]


I created a simple example to understand your point about 
contracts only really mattering for inheritance, but my example 
is giving assertion errors for the inherited class the same way 
as the base class. What would I need to do for this issue to 
become apparent?

class A
{
	int a;
	
	this(int x)
	{
		a = x;
	}
	
	int foo(int x)
	{
		assert(x != 0);
		scope(exit) assert((this.a - x) != 0);

		return this.a - x;
	}
}

class B : A
{
	this()
	{
		super(4);
	}

}

void main()
{
	import std.stdio : writeln;
	
	auto a = new A(2);
	//writeln(a.foo(0)); //causes assertion failure
	//writeln(a.foo(2)); //causes assertion failure
	
	auto b = new B();
	//writeln(b.foo(0)); //causes assertion failure
	//writeln(b.foo(4)); //causes assertion failure
}
Mar 10 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 10 March 2016 at 23:31:14 UTC, jmh530 wrote:

 On Thursday, 10 March 2016 at 22:57:41 UTC, Jonathan M Davis 
 wrote:

 IMHO, at this point, inheritance is the only reason they're 
 worth
 having in the language. [snip]


 I created a simple example to understand your point about 
 contracts only really mattering for inheritance, but my example 
 is giving assertion errors for the inherited class the same way 
 as the base class. What would I need to do for this issue to 
 become apparent?

 class A
 {
 	int a;
 	
 	this(int x)
 	{
 		a = x;
 	}
 	
 	int foo(int x)
 	{
 		assert(x != 0);
 		scope(exit) assert((this.a - x) != 0);

 		return this.a - x;
 	}
 }

 class B : A
 {
 	this()
 	{
 		super(4);
 	}

 }

 void main()
 {
 	import std.stdio : writeln;
 	
 	auto a = new A(2);
 	//writeln(a.foo(0)); //causes assertion failure
 	//writeln(a.foo(2)); //causes assertion failure
 	
 	auto b = new B();
 	//writeln(b.foo(0)); //causes assertion failure
 	//writeln(b.foo(4)); //causes assertion failure
 }


You're not using in or out contracts here at all, so so of 
course, you're not going to see how in/out contracts work with 
inheritance in this example. To quote 
http://dlang.org/spec/contracts.html:

============
If a function in a derived class overrides a function in its 
super class, then only one of the in contracts of the function 
and its base functions must be satisfied. Overriding functions 
then becomes a process of loosening the in contracts.

A function without an in contract means that any values of the 
function parameters are allowed. This implies that if any 
function in an inheritance hierarchy has no in contract, then in 
contracts on functions overriding it have no useful effect.

Conversely, all of the out contracts need to be satisfied, so 
overriding functions becomes a processes of tightening the out 
contracts.
============

So, it's essentially

assert(baseInContract || derivedInContract);

and

assert(baseOutContract && derivedOutContract);

And here is a totally contrived example:

============
import core.exception;
import std.exception;

class Base
{
     int foo(int i)
     in
     {
         assert(i > 0 && i < 10, "base in failed");
     }
     out(result)
     {
         assert(result % 2 == 0, "base out failed");
     }
     body
     {
         return i;
     }
}

class Derived : Base
{
     override int foo(int i)
     in
     {
         assert(i < 50, "derived in failed");
     }
     out(result)
     {
         // Combined with the Base.foo out contract, this ends up
         // being equivalent to assert(result % 6).
         assert(result % 3 == 0, "derived out failed");
     }
     body
     {
         return i;
     }
}

void main()
{
     Base base = new Base;
     Base derived = new Derived;

     assertNotThrown!AssertError(base.foo(4));
     assertNotThrown!AssertError(base.foo(6));
     assert(collectExceptionMsg!AssertError(base.foo(0)) == "base 
in failed");
     assert(collectExceptionMsg!AssertError(base.foo(12)) == "base 
in failed");
     assert(collectExceptionMsg!AssertError(base.foo(100)) == 
"base in failed");
     assert(collectExceptionMsg!AssertError(base.foo(5)) == "base 
out failed");
     assert(collectExceptionMsg!AssertError(base.foo(9)) == "base 
out failed");

     // Combining the out contracts of the Base and Derived make 
this fail for
     // Derived when it succeeded for Base alone.
     assert(collectExceptionMsg!AssertError(derived.foo(4)) == 
"derived out failed");

     assertNotThrown!AssertError(derived.foo(6)); // same as with 
Base
     assertNotThrown!AssertError(derived.foo(0)); // Derived 
allows <= 0
     assertNotThrown!AssertError(derived.foo(12)); // Derived 
alows >= 10 && < 50

     // Whether it's the Base or Derived that fails here is 
implementation defined,
     // since it fails for both.
     assert(collectExceptionMsg!AssertError(base.foo(100)) == 
"base in failed");

     // This fails the contracts of both Base and Derived
     assert(collectExceptionMsg!AssertError(derived.foo(5)) == 
"base out failed");

     // This passes Derived's contract, but it still doesn't pass 
Base's contract.
     assert(collectExceptionMsg!AssertError(derived.foo(9)) == 
"base out failed");
}
============

In order to get this same behavior without it being built into 
the language, all of the in contracts and out contracts from base 
classes would have to be repeated in the derived classes and be 
||ed or &&ed appropriately. It's feasible, but it's error-prone 
and not particularly maintainable. And considering how easily 
this subject tends to confuse folks and how hard it can be to get 
it right - especially with more complicated contracts - I 
seriously question that it's going to be done right except rarely 
if the programmer doesn't have help like we do by having the 
contracts built into the language in D.

- Jonathan M Davis
Mar 10 2016
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 11 March 2016 at 00:07:45 UTC, Jonathan M Davis wrote:

 You're not using in or out contracts here at all, so so of 
 course, you're not going to see how in/out contracts work with 
 inheritance in this example. To quote 
 http://dlang.org/spec/contracts.html:


Sigh...my point was that I was replacing the in/out contracts 
with what you were saying about asserts and scope(exit). Not that 
I have no idea what in/out contracts are.

I will review what you wrote.
Mar 10 2016
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 11 March 2016 at 01:04:05 UTC, jmh530 wrote:

 On Friday, 11 March 2016 at 00:07:45 UTC, Jonathan M Davis 
 wrote:

 You're not using in or out contracts here at all, so so of 
 course, you're not going to see how in/out contracts work with 
 inheritance in this example. To quote 
 http://dlang.org/spec/contracts.html:


 Sigh...my point was that I was replacing the in/out contracts 
 with what you were saying about asserts and scope(exit). Not 
 that I have no idea what in/out contracts are.


Sure, but if you're not using in/out contracts with a class, 
you're not going to see how they interact with inheritance. To 
mimic what they do, you'd have to duplicate the base class 
contracts in the derived class and make sure that you ||ed the in 
contracts correctly and &&ed the out contracts correctly, which 
isn't very maintainable.

- Jonathan M Davis
Mar 10 2016
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 11 March 2016 at 01:45:36 UTC, Jonathan M Davis wrote:

 Sure, but if you're not using in/out contracts with a class, 
 you're not going to see how they interact with inheritance. To 
 mimic what they do, you'd have to duplicate the base class 
 contracts in the derived class and make sure that you ||ed the 
 in contracts correctly and &&ed the out contracts correctly, 
 which isn't very maintainable.

 - Jonathan M Davis


If I'm understanding you correctly, you could still get the same 
behavior with something like below (though it isn't exactly right 
since Base.foo's in block would technically have a funky rule 
applied to it).

After playing around with your example, I'm finding in/out blocks 
on derived classes to be tricky. I think I'm going to try to 
avoid putting myself in a situation where I would screw something 
up.

import core.exception;
import std.exception;

class Base
{
	void baseCheck(int i)
	{
		assert(i % 2 == 0, "base out failed");
	}

     int foo(int i)
     {
		assert(i > 0 && i < 10, "base in failed");
		scope(exit) baseCheck(i);

         return i;
     }
}

class Derived : Base
{
     override int foo(int i)
	{
		scope(exit) baseCheck(i);
		
         assert(i < 50, "derived in failed");
		scope(exit) assert(i % 3 == 0, "derived out failed");

         return i;
     }
}
Mar 10 2016
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 11 March 2016 at 04:17:51 UTC, jmh530 wrote:

 On Friday, 11 March 2016 at 01:45:36 UTC, Jonathan M Davis 
 wrote:

 Sure, but if you're not using in/out contracts with a class, 
 you're not going to see how they interact with inheritance. To 
 mimic what they do, you'd have to duplicate the base class 
 contracts in the derived class and make sure that you ||ed the 
 in contracts correctly and &&ed the out contracts correctly, 
 which isn't very maintainable.

 - Jonathan M Davis


 If I'm understanding you correctly, you could still get the 
 same behavior with something like below (though it isn't 
 exactly right since Base.foo's in block would technically have 
 a funky rule applied to it).


Yeah, you can do something like that and get it to work, but it 
gets increasingly tricky, the more complicated the contracts are, 
and it's pretty easy to screw it up. Certainly, it's far cleaner 
and less error-prone to have it built into the language like we 
do.


 After playing around with your example, I'm finding in/out 
 blocks on derived classes to be tricky. I think I'm going to 
 try to avoid putting myself in a situation where I would screw 
 something up.


Yeah. Having complicated contracts is probably a bad idea in 
general, but it gets far worse when inheritance is involved. And 
you can give yourself some weird problems even with the built-in 
help that D gives you. For instance, in my example, the derived 
class' in contract was a looser version of the base class' in 
contract, which is usually what you'd be looking to do, but the 
way it works is that the bass class' in contract and the derived 
class' in contract are ||ed. So, technically, you could make it 
so that the two contracts are completely distinct or so that the 
derived class' in contract is tighter, but what you ultimately 
end up with is the two in contracts ||ed, whereas what most folks 
will probably expect at a glance is that the derived class' 
contract will be met. So, doing something like

base class: assert(i < 50);
derived class: assert(i < 10);

or

bass class: assert(i < 50);
derived class: assert(i > 50);

will quickly make it so that you could have a derived class 
function which expects the derived class' in contract to be pass 
when it doesn't, because what's tested is the ||ing or the 
contracts not just the derived class contract.

So, ultimately, you still have to be familiar with how the in and 
out contracts are supposed to work with inheritance to avoid 
shooting yourself in the foot, but having it built in makes it so 
that it's harder to screw it up. It just doesn't fix the whole 
problem for you.

Regardless, I'd be _very_ careful with contracts and inheritance, 
and having complicated contracts with inheritance seems a bit 
suicidal.

- Jonathan M Davis
Mar 10 2016
prev sibling next sibling parent reply FatalCatharsis <wykennedy gmail.com> writes:
On Thursday, 10 March 2016 at 22:57:41 UTC, Jonathan M Davis 
wrote:

 I don't think that out contracts are worth much anyway, since 
 in almost all cases, unit tests do that job already, and it's 
 usually much easier to write test that specific input gives 
 specific output than it is to have a general test at the end of 
 the function.


This is a very good point. I was already using unittests and I 
guess they make 'out' contracts redundant considering that you 
are ensuring the result of a function is correct anyway.


 If/Once we get more features in the compiler which take 
 advantage of contracts, then I'll likely change my tune on that 
 one, but for now, IMHO, they're really only of value in classes.


I'm curious, what kind of features might come out of contracts in 
the future? They seem somewhat helpful in terms of QA and 
organization, but what kind of compiler or performance gains 
could you gain from this system?

I'm going to take your advice and just stick to assertions at the 
beginning of the function and unittests to verify correctness of 
output. I'm already overjoyed with the easiness and integration 
of the unittest system as it is :) .
Mar 10 2016
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Thursday, 10 March 2016 at 23:31:22 UTC, FatalCatharsis wrote:

 On Thursday, 10 March 2016 at 22:57:41 UTC, Jonathan M Davis 
 wrote:

 If/Once we get more features in the compiler which take 
 advantage of contracts, then I'll likely change my tune on 
 that one, but for now, IMHO, they're really only of value in 
 classes.


 I'm curious, what kind of features might come out of contracts 
 in the future? They seem somewhat helpful in terms of QA and 
 organization, but what kind of compiler or performance gains 
 could you gain from this system?


Well, if you had something like

     auto foo(int i)
     in
     {
         assert(i > 10 && i < short.max + 5);
     }
     body
     {
         ...
     }

and the the code called it with a value that's known at compile 
time - e.g. foo(2) - and that value clearly fails the contract, 
then the compiler could treat that as an error. Similarly, it 
would know the valid range of i within the function body, which 
might help with optimizations or with VRP (value range 
propagation - i.e. how it can know that an integral value can fit 
in a smaller integral type and use an implicit cast rather than 
requiring an explicit cast).

Or if you do something with out contracts, e.g.

     int foo(int i)
     out(result)
     {
         assert(result => 0 && result < 100);
     }
     body
     {
         ...
     }

then in theory, the compiler could assume that the result passed 
the out contract and optimize based on that. For instance, with 
the code above, maybe it would then treat the result of foo as 
fitting it a ubyte without a cast, because it would know that as 
long as the contract passed, it would fit. e.g.

     ubyte bar = foo(9);

Now, I suspect that stuff like that tends to be restricted to 
basic examples, particularly since you usually pass variables to 
functions, not literals, and at the moment, I can't think of much 
useful other than VRP for what could be done with out contracts, 
but in theory, the compiler would know more about what the 
function parameters and return value, and in at least some cases, 
it could optimize based on that or allow certain operations that 
might not be done implicitly in the general case.

So, at this point, I think that it's pretty much all theoretical, 
but we might benefit from it at some point. And given that we 
don't know when such improvements might be made or exactly what 
code they'd benefit, it's arguably better to just use in and out 
contracts now just in case we get those improvements later, and 
the code then benefits without you having to change it, but 
personally, I don't think that that's worth the extra syntactic 
mess, particularly since while such improvements have been 
discussed upon occasion, it's not at all clear that we're ever 
going to get anything like them.

- Jonathan M Davis
Mar 10 2016
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2016-03-10 23:57, Jonathan M Davis wrote:


 IMHO, at this point, inheritance is the only reason they're worth having
 in the language. Without inheritance, in contracts could just as easily
 be assertions at the top of the function, and while out contracts are
 certainly easier as they are now rather than having to put a contract at
 each return statement or use a scope(exit) statement to do the out
 contracts, you can still do out contracts that way, and honestly, I
 don't think that out contracts are worth much anyway, since in almost
 all cases, unit tests do that job already, and it's usually much easier
 to write test that specific input gives specific output than it is to
 have a general test at the end of the function.

 But while in most cases, in and out contracts are trivially done in the
 function itself, inheritance is a whole other kettle of fish, and having
 them built into the language solves that problem whereas doing it
 yourself is actually fairly hard and error-prone. So, for that reason,
 and that reason alone, I think that having them built into the language
 is a good thing.

 There has been some discussion of getting the compiler to catch some
 stuff based on in contracts, in which case, in contracts would be worth
 a bit more, but as it stands, I pretty much never use out contracts
 (because I think that they're useless), and if inheritance isn't
 involved (which it usually isn't), then I don't even bother with an in
 contract and just put the assertions in the body and avoid the extra
 syntax. If/Once we get more features in the compiler which take
 advantage of contracts, then I'll likely change my tune on that one, but
 for now, IMHO, they're really only of value in classes.


Another advantage of the contracts would be if the caller would be 
responsible for running the contracts. That is currently not the case. 
Then a library could be shipped with contracts and it's up to the user 
of the library to decide if the contracts should be executed or not.

I don't want to start a discussion which approach is right and which is 
wrong.

-- 
/Jacob Carlborg
Mar 11 2016
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, 11 March 2016 at 10:22:06 UTC, Jacob Carlborg wrote:

 Another advantage of the contracts would be if the caller would 
 be responsible for running the contracts. That is currently not 
 the case. Then a library could be shipped with contracts and 
 it's up to the user of the library to decide if the contracts 
 should be executed or not.


The biggest problem with associating the contract with the source 
rather than the caller is that you only get those assertions when 
that library was built with assertions enabled, whereas in 
contracts are really testing the caller code, so what you really 
want is for them to be enabled or not depending on whether the 
calling code is compiled with assertions enabled or not. So, 
having the in contracts are compiled in based on the calling code 
would be huge IMHO (and certainly make using explicit in 
contracts very valuable), but there are definitely implementation 
issues with pulling that off. Right now, in contracts are 
basically just the beginning part of the function, whereas they 
need to be separate from it for the caller to be able to 
determine whether they're enabled or not, and how to do that in a 
way that works with our calling conventions is not immediately 
obvious.

Whether out contracts or invariants should be compiled in based 
on the caller is far less obvious though, since they're really 
testing the code that they're in, whereas in contracts are really 
testing the caller.

Regardless, if in contracts were improved to be compiled in or 
not based on how the calling code is compiled would definitely 
make it so that I'd use in contracts over just putting the 
assertions at the top of the function.

- Jonathan M Davis
Mar 11 2016