www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - enforce()?

reply bearophile <bearophileHUGS lycos.com> writes:
I have counted about 200 usages of std.contracts.enforce() inside Phobos. Can
you tell me what's the purpose of enforce() in a language that has built-in
Contract Programming?

And what are the purposes of std.contracts.AssumeSorted()? Is it useful for
something?

Bye,
bearophile
(I know this is not the digitalmars.D.learn newsgroup).
Jun 15 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


You need to read TDPL for that :o).


 And what are the purposes of std.contracts.AssumeSorted()? Is it
 useful for something?


AssumeSorted was an experiment. I think it has drawbacks that I don't 
know how to address, so I'll retire it.


Andrei
Jun 15 2010
next sibling parent reply Bernard Helyer <b.helyer gmail.com> writes:
You need to read TDPL for that :o).


Please don't start replying to queries in this fashion. Not everyone
has the wherewithal to get a copy of a book such as TDPL. Especially
seeing as you're the author, this kind of reply just looks like whoring
for the book. I'm not saying that's what it is, just what it can look
like.

I've got TDPL on the way from Amazon, by the way. I just don't want to
see this reply, and wanted to express my distaste.
Jun 16 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bernard Helyer wrote:

 You need to read TDPL for that :o).


 
 Please don't start replying to queries in this fashion. Not everyone
 has the wherewithal to get a copy of a book such as TDPL. Especially
 seeing as you're the author, this kind of reply just looks like whoring
 for the book. I'm not saying that's what it is, just what it can look
 like.
 
 I've got TDPL on the way from Amazon, by the way. I just don't want to
 see this reply, and wanted to express my distaste.


All right, all right.

Basically there's a marked difference between contract checking (which 
verifies the architectural integrity of a program) and error handling 
(which deals with errors that occur in correct programs). Contracts help 
with the former, enforce helps with the latter.

The differences are marked enough that TDPL dedicates a separate chapter 
to each.


Andrei
Jun 16 2010
next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Wed, 16 Jun 2010 00:18:03 -0700, Andrei Alexandrescu wrote:


 Bernard Helyer wrote:

 You need to read TDPL for that :o).


 
 Please don't start replying to queries in this fashion. Not everyone
 has the wherewithal to get a copy of a book such as TDPL. Especially
 seeing as you're the author, this kind of reply just looks like whoring
 for the book. I'm not saying that's what it is, just what it can look
 like.
 
 I've got TDPL on the way from Amazon, by the way. I just don't want to
 see this reply, and wanted to express my distaste.


 
 All right, all right.
 
 Basically there's a marked difference between contract checking (which
 verifies the architectural integrity of a program) and error handling
 (which deals with errors that occur in correct programs). Contracts help
 with the former, enforce helps with the latter.


I think any confusion regarding this may stem from the fact that enforce
() resides in std.contracts.

Personally, I think it's worth moving it to object.d, but maybe it's too 
late for that?  Anyway, I love enforce() -- it's become my standard error 
handling tool.

-Lars
Jun 16 2010
prev sibling parent Walter Bright <newshound1 digitalmars.com> writes:
Andrei Alexandrescu wrote:

 Basically there's a marked difference between contract checking (which 
 verifies the architectural integrity of a program) and error handling 
 (which deals with errors that occur in correct programs). Contracts help 
 with the former, enforce helps with the latter.
 
 The differences are marked enough that TDPL dedicates a separate chapter 
 to each.


Yes, I agree it is extremely important to separate the concepts of contract 
checking from error handling.
Jun 16 2010
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 15 Jun 2010 22:23:15 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


 You need to read TDPL for that :o).


 And what are the purposes of std.contracts.AssumeSorted()? Is it
 useful for something?


 AssumeSorted was an experiment. I think it has drawbacks that I don't  
 know how to address, so I'll retire it.


Hm... what are the drawbacks (besides it not being enforced)?  I thought  
it was a good solution.

-Steve
Jun 16 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 06/16/2010 05:47 AM, Steven Schveighoffer wrote:

 On Tue, 15 Jun 2010 22:23:15 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


 You need to read TDPL for that :o).


 And what are the purposes of std.contracts.AssumeSorted()? Is it
 useful for something?


 AssumeSorted was an experiment. I think it has drawbacks that I don't
 know how to address, so I'll retire it.


 Hm... what are the drawbacks (besides it not being enforced)? I thought
 it was a good solution.


Sorry I took so long (over one month!) to reply to this. I've delayed 
the reply to the point when it could be integrated within the upcoming 
thread about improving std.algorithm.find.

The problem with AssumeSorted is that generally predicates in D (and 
also in most other languages) are not easy to compare. Let's first 
recall AssumSorted's definition. It's just a wrapper:

/**
Passes the type system the information that $(D range) is already
sorted by predicate $(D pred). No checking is performed; debug builds
may insert checks randomly. To insert a check, see $(XREF algorithm,
isSorted).
  */
struct AssumeSorted(Range, alias pred = "a < b")
{
     /// Alias for $(D Range).
     alias Range AssumeSorted;
     /// The passed-in range.
     Range assumeSorted;
     /// The sorting predicate.
     alias pred assumeSortedBy;
}

/// Ditto
AssumeSorted!(Range, pred) assumeSorted(alias pred = "a < b", Range)
(Range r)
{
     AssumeSorted!(Range, pred) result;
     result.assumeSorted = r;
     return result;
}

The recommended way to use the facility is:

int[] a = [ -1, 0, 1, 2, 3, 4, 5 ];
assert(find(assumeSorted(a), 3) == [ 3, 4, 5 ]);

find() uses simple means to detect that its first argument has type 
AssumeSorted and takes advantage of that when searching (specifically by 
doing binary search).

So far so good. The problem ensues when we want to make sure that the 
sorting predicate is in sync with the search predicate. For example, if 
the search predicate is "==" then it's okay to use "<" or ">" as a 
sorting predicate. But searching for "a.zip == b.zip" in a range sorted 
by "a.name < b.name" is not okay.

If predicates were all expressed as strings, probably some string 
manipulation could be done to see whether they are compatible. But as 
things stand, assertSorted has quite limited power.


Andrei
Jul 17 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 17 Jul 2010 16:25:16 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 06/16/2010 05:47 AM, Steven Schveighoffer wrote:

 On Tue, 15 Jun 2010 22:23:15 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


 You need to read TDPL for that :o).


 And what are the purposes of std.contracts.AssumeSorted()? Is it
 useful for something?


 AssumeSorted was an experiment. I think it has drawbacks that I don't
 know how to address, so I'll retire it.


 Hm... what are the drawbacks (besides it not being enforced)? I thought
 it was a good solution.


 Sorry I took so long (over one month!) to reply to this. I've delayed  
 the reply to the point when it could be integrated within the upcoming  
 thread about improving std.algorithm.find.

 The problem with AssumeSorted is that generally predicates in D (and  
 also in most other languages) are not easy to compare. Let's first  
 recall AssumSorted's definition. It's just a wrapper:

 /**
 Passes the type system the information that $(D range) is already
 sorted by predicate $(D pred). No checking is performed; debug builds
 may insert checks randomly. To insert a check, see $(XREF algorithm,
 isSorted).
   */
 struct AssumeSorted(Range, alias pred = "a < b")
 {
      /// Alias for $(D Range).
      alias Range AssumeSorted;
      /// The passed-in range.
      Range assumeSorted;
      /// The sorting predicate.
      alias pred assumeSortedBy;
 }

 /// Ditto
 AssumeSorted!(Range, pred) assumeSorted(alias pred = "a < b", Range)
 (Range r)
 {
      AssumeSorted!(Range, pred) result;
      result.assumeSorted = r;
      return result;
 }

 The recommended way to use the facility is:

 int[] a = [ -1, 0, 1, 2, 3, 4, 5 ];
 assert(find(assumeSorted(a), 3) == [ 3, 4, 5 ]);

 find() uses simple means to detect that its first argument has type  
 AssumeSorted and takes advantage of that when searching (specifically by  
 doing binary search).

 So far so good. The problem ensues when we want to make sure that the  
 sorting predicate is in sync with the search predicate. For example, if  
 the search predicate is "==" then it's okay to use "<" or ">" as a  
 sorting predicate. But searching for "a.zip == b.zip" in a range sorted  
 by "a.name < b.name" is not okay.

 If predicates were all expressed as strings, probably some string  
 manipulation could be done to see whether they are compatible. But as  
 things stand, assertSorted has quite limited power.


Just thinking out loud here, couldn't you use the predicate already in  
AssumeSorted?  I mean, if you're going to pass AssumeSorted into find, you  
don't want to also specify the predicate as then the range just becomes a  
standard range.

There must be some kind of way to use template constraints to kill the  
predicate arg to find when the range is an AssumeSorted struct.  If not,  
there should be.

-Steve
Jul 19 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:

 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If not,
 there should be.


That's a good idea. The find predicate that could be derived from 
AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a).

Thanks, Steve.


Andrei
Jul 19 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 09:36:54 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:

 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If not,
 there should be.


 That's a good idea. The find predicate that could be derived from  
 AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a).

 Thanks, Steve.


You're welcome :)

BTW, you don't need the combo predicate until the very end.  Basically,  
you do a binary search for the first element where pred(a, E) is false  
(where E is the target), and then see if pred(E, a) is also false on that  
element (to test for equality).

-Steve
Jul 19 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 09:27 AM, Steven Schveighoffer wrote:

 On Mon, 19 Jul 2010 09:36:54 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:

 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If not,
 there should be.


 That's a good idea. The find predicate that could be derived from
 AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a).

 Thanks, Steve.


 You're welcome :)

 BTW, you don't need the combo predicate until the very end. Basically,
 you do a binary search for the first element where pred(a, E) is false
 (where E is the target), and then see if pred(E, a) is also false on
 that element (to test for equality).


You mean like this? :o)

http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/algorithm.d?rev=1279#L4703


Andrei
Jul 19 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 11:10:03 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 09:27 AM, Steven Schveighoffer wrote:

 On Mon, 19 Jul 2010 09:36:54 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:

 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If  
 not,
 there should be.


 That's a good idea. The find predicate that could be derived from
 AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a).

 Thanks, Steve.


 You're welcome :)

 BTW, you don't need the combo predicate until the very end. Basically,
 you do a binary search for the first element where pred(a, E) is false
 (where E is the target), and then see if pred(E, a) is also false on
 that element (to test for equality).


 You mean like this? :o)

 http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/algorithm.d?rev=1279#L4703


Yep.  I realized after I wrote this that you probably were already doing  
it :)

Interestingly, I found that when doing the redblacktree that I tried to do  
some optimization on the lookup of an element.  Basically, while I'm going  
down the tree looking for a single element, I'm using the binary predicate  
to move left or right.  However, if I move left (i.e. it's not less), then  
that could be the element I'm looking for!  So I try the opposite of the  
predicate to see if I should return.

But when I allow multiple identical elements (i.e. multiset), I want to  
find the *first* instance of the element, the code is much simpler.  If I  
move to the left child, I store that as the "Best result so far".  Then at  
the end, I simply run the opposite predicate once on the aforementioned  
best result.

The benefit of running the opposite predicate sooner is that if the  
element is higher in the tree, I'll return quicker, but I think it ends up  
being a wash.  I'll probably change it to be the same as the multi style  
tree.

It all comes from the original code which used an int return for the  
comparison, making it just as simple to detect equality as it is to detect  
less-than.

Maybe I'm the first one to make that mistake :)

-Steve
Jul 19 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 10:24 AM, Steven Schveighoffer wrote:

 On Mon, 19 Jul 2010 11:10:03 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 09:27 AM, Steven Schveighoffer wrote:

 On Mon, 19 Jul 2010 09:36:54 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 06:36 AM, Steven Schveighoffer wrote:

 Just thinking out loud here, couldn't you use the predicate already in
 AssumeSorted? I mean, if you're going to pass AssumeSorted into find,
 you don't want to also specify the predicate as then the range just
 becomes a standard range.

 There must be some kind of way to use template constraints to kill the
 predicate arg to find when the range is an AssumeSorted struct. If
 not,
 there should be.


 That's a good idea. The find predicate that could be derived from
 AssumeSorted's predicate pred would be !pred(a, b) && !pred(b, a).

 Thanks, Steve.


 You're welcome :)

 BTW, you don't need the combo predicate until the very end. Basically,
 you do a binary search for the first element where pred(a, E) is false
 (where E is the target), and then see if pred(E, a) is also false on
 that element (to test for equality).


 You mean like this? :o)

 http://www.dsource.org/projects/phobos/browser/trunk/phobos/std/algorithm.d?rev=1279#L4703


 Yep. I realized after I wrote this that you probably were already doing
 it :)


Yah, it's quite the STL classic. STL commonly defines implicitly 
equivalence in terms of !less(a, b) && !less(b, a) but uses only one of 
the two comparisons until the last leg, when it's testing the opposite way.


 Interestingly, I found that when doing the redblacktree that I tried to
 do some optimization on the lookup of an element. Basically, while I'm
 going down the tree looking for a single element, I'm using the binary
 predicate to move left or right. However, if I move left (i.e. it's not
 less), then that could be the element I'm looking for! So I try the
 opposite of the predicate to see if I should return.


Indeed that's 100% what STL's lower_bound and rb_tree.find do.

By the way, I'm still eagerly waiting for your red-black tree 
implementation. I think it would be pure awesomeness if you massaged the 
red/black bit inside one of the pointers. I figured out a way of doing 
that without throwing off the garbage collector:

union
{
     unsigned byte * _gcHelper;
     size_t _bits;
}
bool setRed() { _bits |= 1; }
bool setBlack() { _bits &= ~(cast(size_t) 1); }
bool isRed() { return _bits & 1; }
RBTreeNode * left()
{
     return cast(RBTreeNode *) cast(size_t) (_bits & ~(cast(size_t) 1));
}

The idea is to leave _gcHelper in there as a valid pointer to either a 
RBTreeNode or a pointer to one byte inside the RBTreeNode. That way the 
GC is never confused - it will keep the node.

I think putting that one bit inside the pointer has important consequences.

I also suggest you read up on "left-leaning red-black trees" for a 
recent alternative approach that simplifies the code a fair amount.


Andrei
Jul 19 2010
next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 12:21:36 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:

 By the way, I'm still eagerly waiting for your red-black tree  
 implementation.


Sorry for the delay, I've been very busy at work, and I wanted to slip in  
a couple druntime fixes for array appending.

All that is left really is the unit testing, and making the docs more  
phobos-ish.


 I think it would be pure awesomeness if you massaged the red/black bit  
 inside one of the pointers. I figured out a way of doing that without  
 throwing off the garbage collector:


Yes, that works (BTW, you don't need the union, I hate unions :), just  
substitute _bits for _left everywhere, I think it would even work with a  
moving GC).

But I don't know how important it is to save that extra 4 bytes/node.  A  
redblack node already has 3 pointers in it, the flag puts it to 16 bytes  
instead of overhead instead of 12.  It certainly can be an implementation  
choice.

I can look at left-leaning trees (I've had it on my todo list for  
dcollections too).

-Steve
Jul 19 2010
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 12:23 PM, Steven Schveighoffer wrote:

 On Mon, 19 Jul 2010 12:21:36 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 By the way, I'm still eagerly waiting for your red-black tree
 implementation.


 Sorry for the delay, I've been very busy at work, and I wanted to slip
 in a couple druntime fixes for array appending.

 All that is left really is the unit testing, and making the docs more
 phobos-ish.


 I think it would be pure awesomeness if you massaged the red/black bit
 inside one of the pointers. I figured out a way of doing that without
 throwing off the garbage collector:


 Yes, that works (BTW, you don't need the union, I hate unions :), just
 substitute _bits for _left everywhere, I think it would even work with a
 moving GC).


Walter told me that union is instrumental to keeping the compiler in the 
know about such shenanigans. What does your idea look like? You mean 
keeping a possibly misaligned RBTreeNode pointer and manipulating that? 
I think that's a bit worse than unions because it transforms a sure 
thing into a maybe works thing.


 But I don't know how important it is to save that extra 4 bytes/node. A
 redblack node already has 3 pointers in it, the flag puts it to 16 bytes
 instead of overhead instead of 12. It certainly can be an implementation
 choice.

 I can look at left-leaning trees (I've had it on my todo list for
 dcollections too).


Sounds great. If the payload is one word, on a 32-bit system we'd have 
20 bytes per node. I seem to recall the current GC can allocate 16 bytes 
and then 32 bytes and then 48 bytes, so with the embedded bit we're 
looking at halving the total allocated size. Not too shoddy! Then the 
relative overhead of that extra word is not felt up until a payload of 
20 bytes, at which point again it jumps to 33%.

I wonder what things look like (alignment, granularity) for the 64-bit 
implementation.


Andrei
Jul 19 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 13:47:38 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 12:23 PM, Steven Schveighoffer wrote:

 On Mon, 19 Jul 2010 12:21:36 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 By the way, I'm still eagerly waiting for your red-black tree
 implementation.


 Sorry for the delay, I've been very busy at work, and I wanted to slip
 in a couple druntime fixes for array appending.

 All that is left really is the unit testing, and making the docs more
 phobos-ish.


 I think it would be pure awesomeness if you massaged the red/black bit
 inside one of the pointers. I figured out a way of doing that without
 throwing off the garbage collector:


 Yes, that works (BTW, you don't need the union, I hate unions :), just
 substitute _bits for _left everywhere, I think it would even work with a
 moving GC).


 Walter told me that union is instrumental to keeping the compiler in the  
 know about such shenanigans. What does your idea look like? You mean  
 keeping a possibly misaligned RBTreeNode pointer and manipulating that?  
 I think that's a bit worse than unions because it transforms a sure  
 thing into a maybe works thing.


I don't pretend to know what ominous problems Walter knows about regarding  
the compiler's view, but here is what I'm thinking:

If a pointer points to the beginning of a node, and a node has at least  
one pointer in it (which it must, since it's a tree), then pointing one  
byte into the node means you're still pointing at the same block, making  
sure the GC doesn't collect.

Really, the generated code will be exactly the same as your solution, but  
it's less of a misuse of the type system IMO (believe it or not).  I'd  
rather use casts when you are trying to use something that's typed as one  
thing as something else.  When using unions, I usually expect only one  
member of the union to be valid at any one time.

And wouldn't a union be more egregious with the upcoming mostly-precise  
scanner?


 But I don't know how important it is to save that extra 4 bytes/node. A
 redblack node already has 3 pointers in it, the flag puts it to 16 bytes
 instead of overhead instead of 12. It certainly can be an implementation
 choice.

 I can look at left-leaning trees (I've had it on my todo list for
 dcollections too).


 Sounds great. If the payload is one word, on a 32-bit system we'd have  
 20 bytes per node. I seem to recall the current GC can allocate 16 bytes  
 and then 32 bytes and then 48 bytes, so with the embedded bit we're  
 looking at halving the total allocated size. Not too shoddy!


Not quite :)  There is one byte for padding because (insert gasp-inspiring  
music accent) all struct heap allocations are allocated through newArray  
with a size of 1.  I discovered this when working on the array append  
patch.

So even a 16-byte struct requires a 32-byte block.


 Then the relative overhead of that extra word is not felt up until a  
 payload of 20 bytes, at which point again it jumps to 33%.


Most of this is mitigated if you have a custom allocator that allocates an  
array of nodes at once (what I do in dcollections).  As a simple  
implementation, you could allocate enough nodes to be under a certain  
threshold of wasted space.


 I wonder what things look like (alignment, granularity) for the 64-bit  
 implementation.


They must be 8-byte aligned, and have 3 8-byte pointers, so that means at  
least 24 bytes.  If you store an int, then it will still fit in the  
32-byte block.  I don't know what's planned as the minimum size for 64-bit  
GC.

-Steve
Jul 19 2010
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 01:50 PM, Steven Schveighoffer wrote:

 On Mon, 19 Jul 2010 13:47:38 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 12:23 PM, Steven Schveighoffer wrote:

 On Mon, 19 Jul 2010 12:21:36 -0400, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 By the way, I'm still eagerly waiting for your red-black tree
 implementation.


 Sorry for the delay, I've been very busy at work, and I wanted to slip
 in a couple druntime fixes for array appending.

 All that is left really is the unit testing, and making the docs more
 phobos-ish.


 I think it would be pure awesomeness if you massaged the red/black bit
 inside one of the pointers. I figured out a way of doing that without
 throwing off the garbage collector:


 Yes, that works (BTW, you don't need the union, I hate unions :), just
 substitute _bits for _left everywhere, I think it would even work with a
 moving GC).


 Walter told me that union is instrumental to keeping the compiler in
 the know about such shenanigans. What does your idea look like? You
 mean keeping a possibly misaligned RBTreeNode pointer and manipulating
 that? I think that's a bit worse than unions because it transforms a
 sure thing into a maybe works thing.


 I don't pretend to know what ominous problems Walter knows about
 regarding the compiler's view, but here is what I'm thinking:

 If a pointer points to the beginning of a node, and a node has at least
 one pointer in it (which it must, since it's a tree), then pointing one
 byte into the node means you're still pointing at the same block, making
 sure the GC doesn't collect.

 Really, the generated code will be exactly the same as your solution,
 but it's less of a misuse of the type system IMO (believe it or not).
 I'd rather use casts when you are trying to use something that's typed
 as one thing as something else. When using unions, I usually expect only
 one member of the union to be valid at any one time.

 And wouldn't a union be more egregious with the upcoming mostly-precise
 scanner?


I don't think so (applied to all of the above) for reasons of various 
degree of obviousness, but perhaps it's not worth expanding on such a 
minor issue.

Andrei
Jul 19 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:


 Not quite :)  There is one byte for padding because (insert gasp-inspiring  
 music accent) all struct heap allocations are allocated through newArray  
 with a size of 1.  I discovered this when working on the array append  
 patch.


How much more hidden shit like this do I have to see?
I have filed a bug report:
http://d.puremagic.com/issues/show_bug.cgi?id=4487
Maybe Walter has to fix this before porting dmd to 64 bits.



 Most of this is mitigated if you have a custom allocator that allocates an  
 array of nodes at once


The GC is supposed to not suck that much. If I want to do all manually and use
custom allocators then maybe it's better if I start to switc to C language.
Thank you Steven.
Jul 19 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 15:53:46 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:


 Not quite :)  There is one byte for padding because (insert  
 gasp-inspiring
 music accent) all struct heap allocations are allocated through newArray
 with a size of 1.  I discovered this when working on the array append
 patch.


 How much more hidden shit like this do I have to see?
 I have filed a bug report:
 http://d.puremagic.com/issues/show_bug.cgi?id=4487
 Maybe Walter has to fix this before porting dmd to 64 bits.



 Most of this is mitigated if you have a custom allocator that allocates  
 an
 array of nodes at once


 The GC is supposed to not suck that much. If I want to do all manually  
 and use custom allocators then maybe it's better if I start to switc to  
 C language.
 Thank you Steven.


What's so horrible about it?  It's a corner case.  If you were allocating  
a 20-byte struct, you are wasting 12 bytes per value anyways.  Or what  
about a 36-byte struct?

All I'm saying is the pad byte itself isn't a huge issue, even if it  
didn't exist, there would always be inefficient allocations.  Take the  
redblack tree node for instance.  Get rid of the pad byte, and it's tuned  
for word-size payload.  But go over that, and you're back to pretty much  
wasting 50% of your memory.  If you want to tune your app to be the most  
efficient at memory allocation, you need to study the allocator and learn  
its tricks and nuances.  I think you have some misguided notion that C's  
allocator is perfect, and there's no hidden cost to it.  That's not the  
case.

That being said, it would be good if we could get rid of the 1-byte pad  
for single struct allocations on the heap.  As a bonus, the code will be  
more efficient because it doesn't have to deal with the "array" notion.

-Steve
Jul 19 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 What's so horrible about it?


Using about 210% of the RAM I was planning to use. And not even saying it in a
small print somewhere in the docs.



 It's a corner case.


4-word structs are quite common. It's not a common corner.



 All I'm saying is the pad byte itself isn't a huge issue, even if it  
 didn't exist, there would always be inefficient allocations.


That is too much inefficient.



 If you want to tune your app to be the most  
 efficient at memory allocation, you need to study the allocator and learn  
 its tricks and nuances.


The allocator can have some overhead, but this is offensive.



  I think you have some misguided notion that C's  
 allocator is perfect, and there's no hidden cost to it.  That's not the  
 case.


I have written plenty code in C and its cost is not that high.

I have seen you have changed my bug report, the first I have signed as 'major'
into an 'enhancement'. And you have said:

 DMD functions exactly as designed.


Then I can answer it's a *design* bug. I feel offended.
Jul 19 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 16:24:04 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:

 What's so horrible about it?


 Using about 210% of the RAM I was planning to use. And not even saying  
 it in a small print somewhere in the docs.



 It's a corner case.


 4-word structs are quite common. It's not a common corner.


You mean it *is* a common corner?  I agree, and I think the bug report is  
warranted, we should get it fixed.  But again, you have specifically  
designed your input to thwart the system.  Those examples will always be  
possible to construct, unless you have a perfect memory allocator, which  
probably will be so slow that it's unusable.


 All I'm saying is the pad byte itself isn't a huge issue, even if it
 didn't exist, there would always be inefficient allocations.


 That is too much inefficient.


That is the cost of allocation schemes that use fixed size memory blocks.   
Especially when they grow in powers of 2.  Tune your app for it, and you  
won't have this problem.


 If you want to tune your app to be the most
 efficient at memory allocation, you need to study the allocator and  
 learn
 its tricks and nuances.


 The allocator can have some overhead, but this is offensive.


I guess my answer is, tune your app to the allocator.  If you allocate a  
lot of little items, you will go a long way to allocate by allocating an  
array of them instead and use a free list.


  I think you have some misguided notion that C's
 allocator is perfect, and there's no hidden cost to it.  That's not the
 case.


 I have written plenty code in C and its cost is not that high.


So C does not use pools of fixed-size memory?  All its blocks it hands out  
are exactly the size you ask for?

Hm... let me test...

Yep, C does the same thing.  I wrote a small program to allocate 1,000,000  
blocks of a command-line-supplied size.

Up to 12 bytes per block allocates 17MiB, even if the blocks I request are  
1 byte.  For a single pointer, that's 300% overhead.  How horrid, let's  
lambaste the C allocator developers.  WHERE'S THE FINE PRINT!????

16 bytes per block allocates 24.6MiB.  Wait, shouldn't it be 17, surely  
that holds 16MiB of data?  What's going on here?  How can anyone be asked  
to deal with this shit?  Oh wait, I didn't *tune my app for the allocator*.

Above that, the C allocator does a good job of minimizing overhead, but  
that's for a plastic example of simply calling malloc 1M times in a row.   
And C has less to worry about than D when it comes to memory allocation  
and is far more experienced at it.  But it's not perfect in all  
situations.  It's much easier to tune your app to the allocator than tune  
the allocator to your app.


 I have seen you have changed my bug report, the first I have signed as  
 'major' into an 'enhancement'. And you have said:

 DMD functions exactly as designed.


 Then I can answer it's a *design* bug. I feel offended.


A design bug *is* an enhancement :)

-Steve
Jul 19 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 That is the cost of allocation schemes that use fixed size memory blocks.   
 Especially when they grow in powers of 2.  Tune your app for it, and you  
 won't have this problem.


I did know about the power of 2 allocations for small memory blocks, and I know
it's useful to reduce memory fragmentation. So I have tuned my code for that,
that's why I have several structs 16 bytes long, but now I have to target 15
bytes, that is not a power of 2 :o)



 A design bug *is* an enhancement :)


I see, I didn't know this. Sorry for losing my temper Steven...

Bye,
bearophile
Jul 19 2010
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 17:01:34 -0400, bearophile <bearophileHUGS lycos.com>  
wrote:


 Steven Schveighoffer:

 That is the cost of allocation schemes that use fixed size memory  
 blocks.
 Especially when they grow in powers of 2.  Tune your app for it, and you
 won't have this problem.


 I did know about the power of 2 allocations for small memory blocks, and  
 I know it's useful to reduce memory fragmentation. So I have tuned my  
 code for that, that's why I have several structs 16 bytes long, but now  
 I have to target 15 bytes, that is not a power of 2 :o)


Hm... unfortunately, I think you will end up in the same boat.  Because  
any struct of size 15 is aligned to be on a 16-byte boundary.  From my  
memory, I don't think the array allocation code takes into account if the  
final element coincides with the pad byte, but I may be wrong.  Make sure  
to test this theory before going through and trying to trim bytes off all  
your structs.  I think if you use 12-byte structs, it will fit fine, but  
then of course, you are wasting 25% memory :)

If you can deal with some manual memory management, you may want to  
pre-allocate a large array of the structs and then use a free list to  
"allocate" and "deallocate" them.  This should pack them in as tightly as  
possible with almost no overhead.  Of course, if you depend on the GC to  
free your elements, then it might be more of a burden to change all your  
code to contain manual memory management.

-Steve
Jul 19 2010
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 07/19/2010 03:09 PM, Steven Schveighoffer wrote:

 Take the
 redblack tree node for instance. Get rid of the pad byte, and it's tuned
 for word-size payload. But go over that, and you're back to pretty much
 wasting 50% of your memory.


I think this characterization is a bit inaccurate because it suggests 
that there are gains only for one-word payloads. I think the truth is 
that there are gain for several payloads. Their relative value decreases 
with the size of the payload.

Long story short - the less slack (byte of overhead, bool for red/black 
information), the better (in various quanta and of various values).


Andrei
Jul 19 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 19 Jul 2010 16:41:50 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 07/19/2010 03:09 PM, Steven Schveighoffer wrote:

 Take the
 redblack tree node for instance. Get rid of the pad byte, and it's tuned
 for word-size payload. But go over that, and you're back to pretty much
 wasting 50% of your memory.


 I think this characterization is a bit inaccurate because it suggests  
 that there are gains only for one-word payloads. I think the truth is  
 that there are gain for several payloads. Their relative value decreases  
 with the size of the payload.

 Long story short - the less slack (byte of overhead, bool for red/black  
 information), the better (in various quanta and of various values).


There is a cost though...  which was my point.  Isn't everyone always  
saying around here how cheap memory is these days? ;)

-Steve
Jul 19 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Steven Schveighoffer:

 Isn't everyone always  
 saying around here how cheap memory is these days? ;)


RAM is cheap, but the CPU doesn't used RAM, it mostly uses L1 cache (and a bit
L2/L3 caches too), and they cost a lot :-) The more space your data structure
uses, the less you can fit in the cache. Today cache effects are important for
the code performance.

This is a nice example, shows how reducing the size of the data structure and
changing its arrangement (the original was a normal tree, transversed for each
pixel) can improve the code performance by something like one order of
magnitude for ray-tracing:
http://www.cs.utah.edu/~bes/papers/fastRT/paper-node12.html

Bye,
bearophile
Jul 19 2010
prev sibling parent BLS <windevguy hotmail.de> writes:
On 19/07/2010 18:21, Andrei Alexandrescu wrote:

 I also suggest you read up on "left-leaning red-black trees" for a
 recent alternative approach that simplifies the code a fair amount.


A few month ago (12-15) I have also made that suggestion to Steve. 
Meanwhile I am not that sure that LL RB Trees do offer significant 
complexity reduction... . R. Sedgewick's original implementation in Java 
is not bullet proofed.

Don't get me wrong LL RBTree have a certain appeal but read your self.

--In case that you don't want to use this link :
http://t-t-travails.blogspot.com/2008/04/left-leaning-red-black-trees-are-hard.html
--Here a quote <
Last Monday, I started implementing left-leaning red-black trees, 
expecting to spend perhaps 15 hours on the project. I'm here more than 
60 hours of work later to tell you that left-leaning red-black trees are 
hard to implement, and contrary to Sedgewick's claims, their 
implementation appears to require approximately the same amount of code 
and complexity as standard red-black trees.


Meanwhile I am convinced that Skiplists are more interesting as RBTree 
alternative data-structure. Besidet QT folks are using the skiplist 
algo. for their MAP implementation.

 Andrei, hope you have noticed that Steve's dcollections allow the 
replacement of the underlaying data-structute. ;)
So IMHO let's spend some time in implementing the skiplist data-structure.
Finally > I would like to see std.datastructures. for core 
tree,list,graphs etc structures..
A+.
Bjoern



 Jul 19 2010



prev sibling next sibling parent reply Lutger <lutger.blijdestijn gmail.com>  writes:


bearophile wrote:


 I have counted about 200 usages of std.contracts.enforce() inside Phobos.
 Can you tell me what's the purpose of enforce() in a language that has
 built-in Contract Programming?


I'd think of it this way: enforce() is part of defensive programming, and
contracts are related to software testing. 
 

 And what are the purposes of std.contracts.AssumeSorted()? Is it useful
 for something?
 
 Bye,
 bearophile
 (I know this is not the digitalmars.D.learn newsgroup).





 Jun 15 2010



  parent reply Jonathan M Davis <jmdavisProg gmail.com>  writes:


Lutger wrote:


 bearophile wrote:
 

 I have counted about 200 usages of std.contracts.enforce() inside Phobos.
 Can you tell me what's the purpose of enforce() in a language that has
 built-in Contract Programming?


 
 I'd think of it this way: enforce() is part of defensive programming, and
 contracts are related to software testing.


That's probably a pretty good way of putting it. It's essentially the 
difference between when you use assertions and when you use exceptions. 
Assertions assert that something is _always_ true and that if it isn't, the 
program is wrong, while exceptions are for exceptional circumstances (as 
opposed to _never_) and indicate an error of some kind which is likely 
outside the control of the program - such as something happening with the 
file system, user input, or the amount of available memory.

enforce() appears to effectively be the exception equivalent to assert(). 
You use it when you want an exception thrown rather than when you want to 
kill your program due to an error. Unfortunately, the difference between 
when assertions should be used and when exceptions should be used is one 
that is just subtle enough that it often trips people up, even though in 
theory it should be fairly straightforward.

- Jonathan M Davis



 Jun 16 2010



  parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com>  writes:


Jonathan M Davis wrote:


 Assertions assert that something is _always_ true and that if it 


isn't, the

 program is wrong, while exceptions are for exceptional circumstances


Makes sense.


 You use [enforce] when you want an exception thrown rather than when 


you want to

 kill your program due to an error.


To further confuse the issue, assert throws too:

import std.stdio;
import std.algorithm;

void main()
{
     try {
         assert(false);
     } catch (Throwable) {
         writeln("an assertion failed");
     }
}

The difference is just the exception that is thrown. Throwable seems to 
be most general.

 From what I've read so far, I take enforce as a replacement to what it 
exactly is:

if (condition) {
     throw /* ... */;
}

Since I never use assert for that purpose, I take enforce as a shortcut 
for the above.

Ali



 Jun 16 2010



 next sibling parent  "Simen kjaeraas" <simen.kjaras gmail.com>  writes:


Ali =C3=87ehreli <acehreli yahoo.com> wrote:


 To further confuse the issue, assert throws too:

 import std.stdio;
 import std.algorithm;

 void main()
 {
      try {
          assert(false);
      } catch (Throwable) {
          writeln("an assertion failed");
      }
 }

 The difference is just the exception that is thrown. Throwable seems t=


o  =


 be most general.


Seeing as how Error is supposed to be unrecoverable, and Exception might=

be recoverable, and both inherit from Throwable, one should only very ve=
ry
rarely catch Exception, and by extension, Throwable. One might in fact
argue that Error and Exception should have no common ancestor but Object=
.



  From what I've read so far, I take enforce as a replacement to what i=


t  =


 exactly is:

 if (condition) {
      throw /* ... */;
 }


That is indeed basically what it is.


-- =

Simen



 Jun 16 2010



prev sibling  parent  Jonathan M Davis <jmdavisProg gmail.com>  writes:


Ali Çehreli wrote:


 Jonathan M Davis wrote:
 
  > Assertions assert that something is _always_ true and that if it
 isn't, the
  > program is wrong, while exceptions are for exceptional circumstances
 
 Makes sense.
 
  > You use [enforce] when you want an exception thrown rather than when
 you want to
  > kill your program due to an error.
 
 To further confuse the issue, assert throws too:
 
 import std.stdio;
 import std.algorithm;
 
 void main()
 {
      try {
          assert(false);
      } catch (Throwable) {
          writeln("an assertion failed");
      }
 }
 
 The difference is just the exception that is thrown. Throwable seems to
 be most general.
 
  From what I've read so far, I take enforce as a replacement to what it
 exactly is:
 
 if (condition) {
      throw /* ... */;
 }
 
 Since I never use assert for that purpose, I take enforce as a shortcut
 for the above.
 
 Ali


Well, in a sense, the fact that assertions throw is an implementation detail 
since that's not the case in all languages. The concepts of assertions and 
exceptions are distinctly different.

However, while assertions do throw in D, they throw AssertErrors which are 
Errors and not exceptions, albeit both are Throwable. So, they're still 
different. You _can_ catch Errors, but you probably shouldn't. I believe 
that they're intended for pretty much unrecoverable errors. The fact that 
they're thrown likely makes it easier to exit the program semi-gracefully - 
or at least makes it easier for the generated program to properly indicate 
an error rather than simply dying - but they're still distinctly separate 
from exceptions and shouldn't generally be caught. I suppose that it's kind 
of like the difference between checked and unchecked exceptions in Java. You 
generally only worry about the checked ones.

You are right though in that the fact that Errors are Throwable does muddle 
things somewhat.

- Jonathan M Davis



 Jun 16 2010



prev sibling next sibling parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com>  writes:


bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside Phobos. Can
you tell me what's the purpose of enforce() in a language that has built-in
Contract Programming?


I can see two benefits:

1) enforce throws object.Exception, which is "the root of the exception 
hierarchy"; hence enforce errors can be caught with the same general 
catch(Exception) clause [*].

On the other hand, assert throws a type that is out of the Exception 
hierarchy: core.exception.AssertError

2) As a bonus, the word 'enforce' fits the purpose better than 'assert'

3) (the other 2 :p) The format of the message of the uncaught exceptions 
is a little better (e.g. no   sign before the file name)

Ali

* Note: Actually, Throwable is at the top of the exception hierarchy, 
but I've heard before that the top exception class should be taken to be 
Exception; perhaps for user applications?



 Jun 16 2010



  parent reply Walter Bright <newshound1 digitalmars.com>  writes:


Ali Çehreli wrote:

 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside 
 Phobos. Can you tell me what's the purpose of enforce() in a language 
 that has built-in Contract Programming?


 
 I can see two benefits:


The difference is not based on those 3 points, but on what Andrei wrote here. 
Contracts and error checking are completely distinct activities and should not 
be conflated.



 Jun 16 2010



 next sibling parent reply Ary Borenszweig <ary esperanto.org.ar>  writes:


On 06/16/2010 04:15 PM, Walter Bright wrote:

 Ali Çehreli wrote:

 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


 I can see two benefits:


 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.


Could you please explain them? There are many people here that don't 
understand the difference between these two concepts (including me). So 
maybe we are too dumb, maybe those concepts are not generally known or 
maybe the explanation is not very well clear in the documentation.



 Jun 16 2010



 next sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com>  writes:


On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig <ary esperanto.org.ar>  
wrote:


 On 06/16/2010 04:15 PM, Walter Bright wrote:

 Ali Çehreli wrote:

 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


 I can see two benefits:


 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.


 Could you please explain them? There are many people here that don't  
 understand the difference between these two concepts (including me). So  
 maybe we are too dumb, maybe those concepts are not generally known or  
 maybe the explanation is not very well clear in the documentation.


I think of enforce as a convenient way translating an error in an  
expectation to an exception in a single expression.

For example, take some system call that returns -1 on error, you could do  
this:

if(result < 0)
    throw new Exception("oops!");

or you could do this:

enforce(result >= 0, "oops!");

Think of enforce as "throw if"

And in fact, I think there's an errnoEnforce which throws a standard  
exception with the string error from the system.

I'd say the difference between enforce and assert is exactly what Andrei  
said -- enforce is meant to catch errors that can occur during normal  
operation.  Assert is meant to catch errors that are not expected during  
normal operation.  Assert's more like a sanity check.  Also, assert is  
turned off in release mode, enforce is left on.

-Steve



 Jun 16 2010



 next sibling parent  "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet>  writes:


On Wed, 16 Jun 2010 06:55:21 -0400, Steven Schveighoffer wrote:


 On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig
 <ary esperanto.org.ar> wrote:
 

 On 06/16/2010 04:15 PM, Walter Bright wrote:

 Ali Çehreli wrote:

 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a
 language that has built-in Contract Programming?


 I can see two benefits:


 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.


 Could you please explain them? There are many people here that don't
 understand the difference between these two concepts (including me). So
 maybe we are too dumb, maybe those concepts are not generally known or
 maybe the explanation is not very well clear in the documentation.


 
 I think of enforce as a convenient way translating an error in an
 expectation to an exception in a single expression.
 
 For example, take some system call that returns -1 on error, you could
 do this:
 
 if(result < 0)
     throw new Exception("oops!");
 
 or you could do this:
 
 enforce(result >= 0, "oops!");
 
 Think of enforce as "throw if"


It also adds a file and a line number to the error message, so the 
problem is easier to track down.  Very handy. :)



 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.


That's right, and there's even an enforceEx() which lets you specify 
which exception type to throw:

http://digitalmars.com/d/2.0/phobos/std_contracts.html#enforceEx

-Lars



 Jun 16 2010



prev sibling next sibling parent reply Alex Makhotin <alex bitprox.com>  writes:


Steven Schveighoffer wrote:

  
 Think of enforce as "throw if"
 


So why not concatenating the two and rename it to exactly 'throwif'?
Self descriptive is better than cryptic 'enforce'.


-- 
Alex Makhotin,
the founder of BITPROX,
http://bitprox.com



 Jun 16 2010



 next sibling parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Alex Makhotin wrote:

 Steven Schveighoffer wrote:

  
 Think of enforce as "throw if"


 
 So why not concatenating the two and rename it to exactly 'throwif'?
 Self descriptive is better than cryptic 'enforce'.


Well throwif describes mechanism and enforce describes intent. After all 
assert is not abortif :o).

Andrei



 Jun 16 2010



prev sibling  parent  Bruno Medeiros <brunodomedeiros+spam com.gmail>  writes:


On 16/06/2010 15:45, Andrei Alexandrescu wrote:

 Well throwif describes mechanism and enforce describes intent. After all
 assert is not abortif :o).

 Andrei


Indeed, especially given that other code program may use throwing as 
mechanism, but with a different intent. And you would not be able to 
distinguish it if you called it throwif.


-- 
Bruno Medeiros - Software Engineer



 Jun 17 2010



prev sibling  parent reply Leandro Lucarella <luca llucax.com.ar>  writes:


Steven Schveighoffer, el 16 de junio a las 06:55 me escribiste:

 On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig
 <ary esperanto.org.ar> wrote:
 

On 06/16/2010 04:15 PM, Walter Bright wrote:

Ali Çehreli wrote:

bearophile wrote:

I have counted about 200 usages of std.contracts.enforce() inside
Phobos. Can you tell me what's the purpose of enforce() in a language
that has built-in Contract Programming?


I can see two benefits:


The difference is not based on those 3 points, but on what Andrei wrote
here. Contracts and error checking are completely distinct activities
and should not be conflated.


Could you please explain them? There are many people here that
don't understand the difference between these two concepts
(including me). So maybe we are too dumb, maybe those concepts are
not generally known or maybe the explanation is not very well
clear in the documentation.


 
 I think of enforce as a convenient way translating an error in an
 expectation to an exception in a single expression.
 
 For example, take some system call that returns -1 on error, you
 could do this:
 
 if(result < 0)
    throw new Exception("oops!");
 
 or you could do this:
 
 enforce(result >= 0, "oops!");
 
 Think of enforce as "throw if"


So maybe throw_if() would be a better name =)

Anyway, I think enforce() is poisson, because it make the programmer to
not think about errors at all, just add and enforce() and there you go.
But when you need to be fault tolerant, is very important to know what's
the nature of the error, but thanks to enforce(), almost every error is
a plain Exception, no hierarchy, no extra info, all you can do to get
a little more info about what happened is to parse the exception string,
and that's not really an option.


 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.


That's the only useful case of enforce, because it includes the
*important* information (the actual errno).

There is also enforceEx!(), to use a custom exception, which practically
nobody uses (I counted only 4 uses in phobos).

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Y será el día en que la electricidad deje de ser rayo y sea depilador
femenino.
	-- Ricardo Vaporeso



 Jun 16 2010



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Leandro Lucarella wrote:

 Steven Schveighoffer, el 16 de junio a las 06:55 me escribiste:

 On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig
 <ary esperanto.org.ar> wrote:


 On 06/16/2010 04:15 PM, Walter Bright wrote:

 Ali Çehreli wrote:

 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


 I can see two benefits:


 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.


 Could you please explain them? There are many people here that
 don't understand the difference between these two concepts
 (including me). So maybe we are too dumb, maybe those concepts are
 not generally known or maybe the explanation is not very well
 clear in the documentation.


 I think of enforce as a convenient way translating an error in an
 expectation to an exception in a single expression.

 For example, take some system call that returns -1 on error, you
 could do this:

 if(result < 0)
    throw new Exception("oops!");

 or you could do this:

 enforce(result >= 0, "oops!");

 Think of enforce as "throw if"


 
 So maybe throw_if() would be a better name =)
 
 Anyway, I think enforce() is poisson,


Indeed it is a bit fishy :o).


 because it make the programmer to
 not think about errors at all, just add and enforce() and there you go.
 But when you need to be fault tolerant, is very important to know what's
 the nature of the error, but thanks to enforce(), almost every error is
 a plain Exception, no hierarchy, no extra info, all you can do to get
 a little more info about what happened is to parse the exception string,
 and that's not really an option.


I think there is no real need for exception hierarchies. I occasionally 
dream of eliminating all of the useless exceptions defined left and 
right in Phobos.


 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.


 
 That's the only useful case of enforce, because it includes the
 *important* information (the actual errno).
 
 There is also enforceEx!(), to use a custom exception, which practically
 nobody uses (I counted only 4 uses in phobos).


I'd be hard pressed to find good examples of exception hierarchy use. 
Everybody talks about them but I've seen none.

The fact that the coder doesn't need to think hard to use enforce() 
effectively is a plus, not a minus. An overdesigned enforce that adds 
extra burden to its user would have been a mistake.


Andrei



 Jun 16 2010



 next sibling parent  dsimcha <dsimcha yahoo.com>  writes:


== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article

 Everybody talks about them but I've seen none.
 The fact that the coder doesn't need to think hard to use enforce()
 effectively is a plus, not a minus. An overdesigned enforce that adds
 extra burden to its user would have been a mistake.
 Andrei


IMHO the presence of a simple method of handling errors, even if it's far from
perfect, is a good thing.  If you have to think about a whole exception
hierarchy
every time you hit a possible error condition in your code, you tend to put this
tedious task off until forever, leading to programs that fail for unknown
reasons
because some error condition was never reported.  Well-designed exception
hierarchies are nice, but forcing their use all the time would be making the
perfect the enemy of the good.

Furthermore, I love enforce() because sometimes I want just some subset of
assertions checked in release mode, usually whichever ones can be checked at
negligible performance cost.  I tend to use it a lot as an
assert-even-in-release-mode function.



 Jun 16 2010



prev sibling next sibling parent  Michel Fortin <michel.fortin michelf.com>  writes:


On 2010-06-16 10:53:12 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 Leandro Lucarella wrote:

 Steven Schveighoffer, el 16 de junio a las 06:55 me escribiste:

 On Wed, 16 Jun 2010 05:28:46 -0400, Ary Borenszweig
 <ary esperanto.org.ar> wrote:
 

 On 06/16/2010 04:15 PM, Walter Bright wrote:

 Ali Çehreli wrote:

 bearophile wrote:

 I have counted about 200 usages of std.contracts.enforce() inside
 Phobos. Can you tell me what's the purpose of enforce() in a language
 that has built-in Contract Programming?


 I can see two benefits:


 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.


 Could you please explain them? There are many people here that
 don't understand the difference between these two concepts
 (including me). So maybe we are too dumb, maybe those concepts are
 not generally known or maybe the explanation is not very well
 clear in the documentation.


 I think of enforce as a convenient way translating an error in an
 expectation to an exception in a single expression.
 
 For example, take some system call that returns -1 on error, you
 could do this:
 
 if(result < 0)
    throw new Exception("oops!");
 
 or you could do this:
 
 enforce(result >= 0, "oops!");
 
 Think of enforce as "throw if"


 
 So maybe throw_if() would be a better name =)
 
 Anyway, I think enforce() is poisson,


 
 Indeed it is a bit fishy :o).
 

 because it make the programmer to
 not think about errors at all, just add and enforce() and there you go.
 But when you need to be fault tolerant, is very important to know what's
 the nature of the error, but thanks to enforce(), almost every error is
 a plain Exception, no hierarchy, no extra info, all you can do to get
 a little more info about what happened is to parse the exception string,
 and that's not really an option.


 
 I think there is no real need for exception hierarchies. I occasionally 
 dream of eliminating all of the useless exceptions defined left and 
 right in Phobos.


The need is not really for a hierarchy. The hierarchy serves the need, 
which is to:

1. Be able to programatically check the kind of the error and so your 
program can act appropriately.
2. Propagate additional information related to the error and the 
context it occured.

Displaying a proper error message to a user and offering relevant 
recovery choices often need both. Sometime, a program won't ask the 
user and attempt something by itself as a recovery. In both cases, you 
need to know the kind of error, and may need context information.



 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.


 
 That's the only useful case of enforce, because it includes the
 *important* information (the actual errno).
 
 There is also enforceEx!(), to use a custom exception, which practically
 nobody uses (I counted only 4 uses in phobos).


 
 I'd be hard pressed to find good examples of exception hierarchy use. 
 Everybody talks about them but I've seen none.


The need is not really for a hierarchy. The hierarchy serves the need, 
which is to:

1. Programatically check the kind of the error and so your program can 
act appropriately.
2. Propagate additional information related to the error and the 
context in which it occurred.

Displaying a proper error message to a user and offering relevant 
recovery choices often need both. Sometime, a program won't ask the 
user and attempt something by itself as a recovery. In both cases, you 
need to know the kind of error, and may need context information.

That said, hierarchies are often abused, and aren't universally useful. 
But exceptions should provide the above information in a way or another 
when useful.

Think about a GUI program, if an exception is thrown somewhere during a 
complex operation (say, reading a lot of files), I could catch it as 
some level, create a wrapper exception with the context (file=hello.d 
error=access denied) and rethrow it to unwind until reatching the GUI 
error handler. Or the file function could throw a useful exception from 
the start. In either cases, the code in charge of that operation can 
display a message such as "Creating the archive failed. File 'hello.d' 
could not be read because you do not have read permissions to it." with 
options "Retry as Administrator", "Exclude 'hello.d'" or "Cancel". 
Knowing programatically what has gone wrong is important in many cases.



 The fact that the coder doesn't need to think hard to use enforce() 
 effectively is a plus, not a minus. An overdesigned enforce that adds 
 extra burden to its user would have been a mistake.


That's indeed true. Throwing an Exception with no info is still better 
than not throwing at all, and creating useful exceptions isn't always 
easy, nor economically rewarding. What's important is to make it easy 
to improve the thrown exception when it becomes relevant. For instance

	// first version: throws standard exception
	enforce(1 == 1, "access denied to " ~ filename);

	// refined version: throws custom exception
	enforce(1 == 1, new FileException("access denied to " ~ filename, 
accessDeniedError, filename));


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/



 Jun 16 2010



prev sibling next sibling parent  Leandro Lucarella <luca llucax.com.ar>  writes:


Andrei Alexandrescu, el 16 de junio a las 07:53 me escribiste:

because it make the programmer to
not think about errors at all, just add and enforce() and there you go.
But when you need to be fault tolerant, is very important to know what's
the nature of the error, but thanks to enforce(), almost every error is
a plain Exception, no hierarchy, no extra info, all you can do to get
a little more info about what happened is to parse the exception string,
and that's not really an option.


 
 I think there is no real need for exception hierarchies. I
 occasionally dream of eliminating all of the useless exceptions
 defined left and right in Phobos.


Exception hierarchy is only one way to discriminate error types. Extra
info, is another (like an error code). I agree that a *large* exception
hierarchy hurts more than it helps.


And in fact, I think there's an errnoEnforce which throws a standard
exception with the string error from the system.


That's the only useful case of enforce, because it includes the
*important* information (the actual errno).

There is also enforceEx!(), to use a custom exception, which practically
nobody uses (I counted only 4 uses in phobos).


 
 I'd be hard pressed to find good examples of exception hierarchy
 use. Everybody talks about them but I've seen none.


I think Python has a good one. I find myself discriminating between
ValueError, IndexError, KeyError, OSError and IOError all the time.


 The fact that the coder doesn't need to think hard to use enforce()
 effectively is a plus, not a minus. An overdesigned enforce that
 adds extra burden to its user would have been a mistake.


That is, if you don't care on handling errors and let the program crash
with a backtrace, or add a big try {} catch (Exception) in the main. If
that's not the case, it only produce a false feeling that D (standard
library) is good handling errors when it's not, it's just a binary
"there is an error" - "there is no errors".

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
The average person laughs 13 times a day



 Jun 16 2010



prev sibling  parent reply Jonathan M Davis <jmdavisProg gmail.com>  writes:


Andrei Alexandrescu wrote:


 Anyway, I think enforce() is poisson,


 
 Indeed it is a bit fishy :o).


LOL. My thoughts exactly.

 

 because it make the programmer to
 not think about errors at all, just add and enforce() and there you go.
 But when you need to be fault tolerant, is very important to know what's
 the nature of the error, but thanks to enforce(), almost every error is
 a plain Exception, no hierarchy, no extra info, all you can do to get
 a little more info about what happened is to parse the exception string,
 and that's not really an option.


 
 I think there is no real need for exception hierarchies. I occasionally
 dream of eliminating all of the useless exceptions defined left and
 right in Phobos.
 

 And in fact, I think there's an errnoEnforce which throws a standard
 exception with the string error from the system.


 
 That's the only useful case of enforce, because it includes the
 *important* information (the actual errno).
 
 There is also enforceEx!(), to use a custom exception, which practically
 nobody uses (I counted only 4 uses in phobos).


 
 I'd be hard pressed to find good examples of exception hierarchy use.
 Everybody talks about them but I've seen none.
 
 The fact that the coder doesn't need to think hard to use enforce()
 effectively is a plus, not a minus. An overdesigned enforce that adds
 extra burden to its user would have been a mistake.
 
 
 Andrei


I think that exception hierarchies can be quite useful, but in most cases, I 
haven't seen projects bother with them. I do think that certain types of 
exceptions can be useful as separate types as long as they inherit from the 
base exception type and you therefore don't _have_ to worry about the 
hierarchy.

A good example of a useful exception type IMO is Java's IOException. It 
makes good sense to handle them in a specific way separate from general 
exceptions. You can frequently recover just fine from them, and it allows 
you to handle I/O-related exceptions gracefully while other exceptions might 
be considered fatal. However, those other exceptions - especially those 
which are from more or less unrecoverable errors NullPointerExceptions or 
OutOfMemoryExceptions - don't necessarily gain much from an exception 
hierarchy. So, I think that it really depends on the exception.

I do think, however, that there are certain types of exceptions which can 
benefit from having their own type because it allows you to handle them in a 
specific manner separate from general and/or unrecoverable exceptions.

- Jonathan M Davis



 Jun 16 2010



  parent  Jason Spencer <spencer8 sbcglobal.net>  writes:


I think about it roughly this way (in reverse priority):

Contracts/assertions concern problems in the program(ming) domain.

Exceptions concern problems in the system domain.

Problems in the actual problem domain should be modeled in the design
and have their own abstractions.

These interact a little bit, so I have an excuse to bend my rules
whenever I want :)  For instance, if the system is part of your
problem domain (e.g. embedded code), then exceptions are probably not
the right approach.  That's why I indicate a false idea of priority.

Jason



 Jun 16 2010



prev sibling  parent reply Walter Bright <newshound1 digitalmars.com>  writes:


Ary Borenszweig wrote:

 On 06/16/2010 04:15 PM, Walter Bright wrote:

 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.


 
 Could you please explain them? There are many people here that don't 
 understand the difference between these two concepts (including me). So 
 maybe we are too dumb, maybe those concepts are not generally known or 
 maybe the explanation is not very well clear in the documentation.


It has nothing to do with being dumb, as it is not obvious.

Contracts are for verifying that your program is in a state that it is designed 
to be in. A contract failure is defined as a program bug.

Errors, on the other hand, are things that can go wrong at run time, like your 
disk is full when trying to write a file. These are NOT program bugs.

Another way to look at it is your program should continue to operate correctly 
if all the contracts are removed. This is not true of removing all error 
checking and handling.

Furthermore, errors are something a program can recover from and continue 
operating. Contract failures are ALWAYS fatal. A common newbie (and some
expert) 
misconception is that contract failures can or even must be recovered. This 
comes from a misunderstanding of the basic principles of engineering a safe and 
reliable system.



 Jun 16 2010



 next sibling parent reply Lutger <lutger.blijdestijn gmail.com>  writes:


Walter Bright wrote:


 Ary Borenszweig wrote:

 On 06/16/2010 04:15 PM, Walter Bright wrote:

 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.


 
 Could you please explain them? There are many people here that don't
 understand the difference between these two concepts (including me). So
 maybe we are too dumb, maybe those concepts are not generally known or
 maybe the explanation is not very well clear in the documentation.


 
 It has nothing to do with being dumb, as it is not obvious.
 
 Contracts are for verifying that your program is in a state that it is
 designed to be in. A contract failure is defined as a program bug.
 
 Errors, on the other hand, are things that can go wrong at run time, like your
 disk is full when trying to write a file. These are NOT program bugs.
 
 Another way to look at it is your program should continue to operate correctly
 if all the contracts are removed. This is not true of removing all error
 checking and handling.
 
 Furthermore, errors are something a program can recover from and continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and some
 expert) misconception is that contract failures can or even must be recovered.
 This comes from a misunderstanding of the basic principles of engineering a
 safe and reliable system.


I am not so sure about this last point, usually you want to fail but perhaps
not 
always. This is about what to do after detection of a program bug vs how to 
handle an exceptional condition. 

Suppose for example (actually this is from real life) there is an important 
operation which, as a service, also sends an e-mail notification as part of
that 
operation. It is very bad if the operation fails, but a failed notification is 
not that bad. What to do in case of a bug with the e-mail notification?

1. crash (gracefully), do not complete the operation.
2. log the error for the devs to look into (or crash) *after* the operation is 
complete, let the operation go through without the e-mail notification.

Option 1 is annoying and prevents people from getting work done due to a
'minor' 
bug. Option 2 however probably results in this bug either not getting noticed 
quite early enough or ignored in the face of other issues that always seems to 
have higher priority. Choosing for option 2 can also lead to bugs being 
swallowed silently or mistaken for exceptional conditions, which is more 
dangerous.

I don't mean to split hairs, I bet a lot of software has these kind of cases.



 Jun 16 2010



 next sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com>  writes:


Lutger <lutger.blijdestijn gmail.com> wrote:


 Suppose for example (actually this is from real life) there is an  
 important
 operation which, as a service, also sends an e-mail notification as part  
 of that
 operation. It is very bad if the operation fails, but a failed  
 notification is
 not that bad. What to do in case of a bug with the e-mail notification?

 1. crash (gracefully), do not complete the operation.
 2. log the error for the devs to look into (or crash) *after* the  
 operation is
 complete, let the operation go through without the e-mail notification.

 Option 1 is annoying and prevents people from getting work done due to a  
 'minor'
 bug. Option 2 however probably results in this bug either not getting  
 noticed
 quite early enough or ignored in the face of other issues that always  
 seems to
 have higher priority. Choosing for option 2 can also lead to bugs being
 swallowed silently or mistaken for exceptional conditions, which is more
 dangerous.

 I don't mean to split hairs, I bet a lot of software has these kind of  
 cases.


How did you end up with an email system that is so horribly broken that
it spits Errors instead of Exceptions when things are not quite the way
it wants them to be?

If it cannot send the email, it may throw an Exception. If you try and
pass it a handwritten letter, it should throw an Error.

Basically, throwing an Exception means 'Your attention please, reactor 5
has a cooling problem you might want to look at', whereas an Error means
'Explosion imminent, get the fuck off outta here!'.

-- 
Simen



 Jun 16 2010



 next sibling parent reply Lutger <lutger.blijdestijn gmail.com>  writes:


Simen kjaeraas wrote:


 Lutger <lutger.blijdestijn gmail.com> wrote:
 

 Suppose for example (actually this is from real life) there is an
 important
 operation which, as a service, also sends an e-mail notification as part
 of that
 operation. It is very bad if the operation fails, but a failed
 notification is
 not that bad. What to do in case of a bug with the e-mail notification?

 1. crash (gracefully), do not complete the operation.
 2. log the error for the devs to look into (or crash) *after* the
 operation is
 complete, let the operation go through without the e-mail notification.

 Option 1 is annoying and prevents people from getting work done due to a
 'minor'
 bug. Option 2 however probably results in this bug either not getting
 noticed
 quite early enough or ignored in the face of other issues that always
 seems to
 have higher priority. Choosing for option 2 can also lead to bugs being
 swallowed silently or mistaken for exceptional conditions, which is more
 dangerous.

 I don't mean to split hairs, I bet a lot of software has these kind of
 cases.


 
 How did you end up with an email system that is so horribly broken that
 it spits Errors instead of Exceptions when things are not quite the way
 it wants them to be?


Not Errors, it is not in D and does not distinguish between Errors and 
Exceptions. It was an example, a (design) question. It's very simple:

sendEmail() 
// possibly die here because something relatively unimportant thing is buggy

vs:

try
{
    sendEmail()
}
catch(BadShitThatCanHappen)
{
    RecoverFromBadShitThatCanHappen() // ok, this is good and according to spec
}
catch(Exception ex)
{
    logError() 
    // now crash? assume we know this must be programmer's fault
}


 If it cannot send the email, it may throw an Exception. If you try and
 pass it a handwritten letter, it should throw an Error.
 
 Basically, throwing an Exception means 'Your attention please, reactor 5
 has a cooling problem you might want to look at', whereas an Error means
 'Explosion imminent, get the fuck off outta here!'.
 


No, an Error means the program has a bug. Programs have thousands of bugs, this 
is not related to how critical it is. An Exception can be way more important to 
fix than a bug. WebServerDownException for example, is often not a bug in the 
code that drives websites, but for sure I will contact the sysadmin before even 
thinking of going back to work. The question is how to proceed after the fact.



 Jun 16 2010



  parent reply "Simen kjaeraas" <simen.kjaras gmail.com>  writes:


Lutger <lutger.blijdestijn gmail.com> wrote:


 How did you end up with an email system that is so horribly broken th=




at

 it spits Errors instead of Exceptions when things are not quite the w=




ay

 it wants them to be?


 Not Errors, it is not in D and does not distinguish between Errors and=



 Exceptions. It was an example, a (design) question. It's very simple:


Ah. Then no, handling the exception is perfectly acceptable, and probabl=
y
the right thing to do.


 Basically, throwing an Exception means 'Your attention please, reacto=




r 5

 has a cooling problem you might want to look at', whereas an Error me=




ans

 'Explosion imminent, get the fuck off outta here!'.


 No, an Error means the program has a bug. Programs have thousands of  =



 bugs, this
 is not related to how critical it is. An Exception can be way more  =



 important to
 fix than a bug. WebServerDownException for example, is often not a bug=


  =


 in the
 code that drives websites, but for sure I will contact the sysadmin  =



 before even
 thinking of going back to work. The question is how to proceed after t=


he  =


 fact.


Yes and no. Throwing an error puts the program into an undefined state,
and everything may happen. Because everything happening at once would
strain space-time and cause Bad Things=E2=84=A2 to happen, we would like=
 to limit
the time in which this does occur. Hence, we bail out.

That said, a bug in a rarely-used function may indeed be significantly
less important than getting a server back online. However, I would still=

say an error indicates something is fundamentally wrong.

-- =

Simen



 Jun 17 2010



  parent  Walter Bright <newshound1 digitalmars.com>  writes:


Simen kjaeraas wrote:

 That said, a bug in a rarely-used function may indeed be significantly
 less important than getting a server back online. However, I would still
 say an error indicates something is fundamentally wrong.


The contract failing means you do not know what went wrong. That means there's 
no way the program can determine if it is recoverable or not. For all you know, 
malware may have infected your process and continuing to execute may send your 
credit card database to a thief.



 Jun 17 2010



prev sibling  parent reply Lutger <lutger.blijdestijn gmail.com>  writes:


Simen kjaeraas wrote:

...

 If it cannot send the email, it may throw an Exception. If you try and
 pass it a handwritten letter, it should throw an Error.
 


This is the question: should I segfault on a handwritten letter even if it is 
not such an important letter and could just go on operating?



 Jun 16 2010



  parent reply "Simen kjaeraas" <simen.kjaras gmail.com>  writes:


Lutger <lutger.blijdestijn gmail.com> wrote:


 Simen kjaeraas wrote:

 ...

 If it cannot send the email, it may throw an Exception. If you try and
 pass it a handwritten letter, it should throw an Error.


 This is the question: should I segfault on a handwritten letter even if  
 it is
 not such an important letter and could just go on operating?


Yes. If someone is passing your email system a handwritten letter,
something is so wrong, the program should balk and exit. It's not just
a small mixup, it's an indication something is completely wrong.

As Walter put it, an Error, be it an AssertError or otherwise, means
your program has ventured into uncharted territory, and behavior from
this point on is undefined. "Permissible undefined behavior ranges
 from ignoring the situation completely with unpredictable results, to
having demons fly out of your nose."[1]


-- 
Simen

[1] http://groups.google.com/group/comp.std.c/msg/dfe1ef367547684b?pli=1



 Jun 17 2010



  parent  =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr>  writes:


Simen kjaeraas wrote:

 Lutger <lutger.blijdestijn gmail.com> wrote:
=20

 Simen kjaeraas wrote:

 ...

 If it cannot send the email, it may throw an Exception. If you try an=






d

 pass it a handwritten letter, it should throw an Error.


 This is the question: should I segfault on a handwritten letter even
 if it is
 not such an important letter and could just go on operating?


=20
 Yes. If someone is passing your email system a handwritten letter,
 something is so wrong, the program should balk and exit. It's not just
 a small mixup, it's an indication something is completely wrong.
=20


	Bad example. If someone is passing bad input to your program, it
should signal the mistake and recover. External input must *always*
be checked and wrong inputs must be recovered from gracefully.

	However, if you take (and check) the user input, then put it in a
queue, then take things from the queue for processing, and you get a
handwritten letter out of the queue, *then* it is an error and
cannot be recovered from (because this should have been checked for
before putting the letter in the queue and if this is messed up, you
don't know what else may be messed up nor how bad the situation is).
Which is what you say after:


 As Walter put it, an Error, be it an AssertError or otherwise, means
 your program has ventured into uncharted territory, and behavior from
 this point on is undefined. "Permissible undefined behavior ranges
 from ignoring the situation completely with unpredictable results, to
 having demons fly out of your nose."[1]
=20


		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr



 Jun 17 2010



prev sibling  parent reply Walter Bright <newshound1 digitalmars.com>  writes:


Lutger wrote:

 Walter Bright wrote:

 Furthermore, errors are something a program can recover from and continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and some
 expert) misconception is that contract failures can or even must be recovered.
 This comes from a misunderstanding of the basic principles of engineering a
 safe and reliable system.


 
 I am not so sure about this last point, usually you want to fail but perhaps
not 
 always. This is about what to do after detection of a program bug vs how to 
 handle an exceptional condition. 


First you need to decide if it is a program bug or not. If it is not a program 
bug, it shouldn't be done with contracts.

If it is a program bug, then the only proper thing to do is exit the program. 
The program cannot decide if it is a minor bug or not, nor can it decide if it 
is recoverable. It is, by definition, in an unknown state, and continuing to 
execute may cause anything to happen. (For example, malware may have installed 
itself and that may get executed.)

If you need notifications that the program failed, a separate monitor program 
should be used. This is how people who design safe systems do it. People who 
believe that programs can "recover" from bugs design systems that fail, 
sometimes with terrible consequences.

My articles on the topic:

http://www.drdobbs.com/blog/archives/2009/10/safe_systems_fr.html

http://www.drdobbs.com/blog/archives/2009/11/designing_safe.html



 Jun 16 2010



 next sibling parent reply Lutger <lutger.blijdestijn gmail.com>  writes:


Walter Bright wrote:


 Lutger wrote:

 Walter Bright wrote:

 Furthermore, errors are something a program can recover from and continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and some
 expert) misconception is that contract failures can or even must be
 recovered. This comes from a misunderstanding of the basic principles of
 engineering a safe and reliable system.


 
 I am not so sure about this last point, usually you want to fail but perhaps
 not always. This is about what to do after detection of a program bug vs how
 to handle an exceptional condition.


 
 First you need to decide if it is a program bug or not. If it is not a program
 bug, it shouldn't be done with contracts.
 
 If it is a program bug, then the only proper thing to do is exit the program.
 The program cannot decide if it is a minor bug or not, nor can it decide if it
 is recoverable. It is, by definition, in an unknown state, and continuing to
 execute may cause anything to happen. (For example, malware may have installed
 itself and that may get executed.)


I didn't really get this point from your articles on the subject, but that does 
clarify it for me. The assumption one makes when recovering is that it is
indeed 
possible and safe. Even if it may be likely, it is never reliable to count on 
it.



 Jun 16 2010



  parent  Walter Bright <newshound1 digitalmars.com>  writes:


Lutger wrote:

 The assumption one makes when recovering is that it is indeed 
 possible and safe. Even if it may be likely, it is never reliable to count on 
 it.


Exactly.



 Jun 16 2010



prev sibling  parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail>  writes:


On 17/06/2010 00:27, Walter Bright wrote:

 Lutger wrote:

 Walter Bright wrote:

 Furthermore, errors are something a program can recover from and
 continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and some
 expert) misconception is that contract failures can or even must be
 recovered.
 This comes from a misunderstanding of the basic principles of
 engineering a
 safe and reliable system.


 I am not so sure about this last point, usually you want to fail but
 perhaps not always. This is about what to do after detection of a
 program bug vs how to handle an exceptional condition.


 First you need to decide if it is a program bug or not. If it is not a
 program bug, it shouldn't be done with contracts.


I would go further and state that anything outside the direct control of 
a process (such as network state, disk state, OS state, other processes 
behavior, user interaction, etc.) should be modeled as an error and not 
a contract violation.
Such externals errors may be a "bug" in the system as whole, but they 
are not a bug in the particular process, and thus should not be modeled 
as a contract violation.
In other words, _contract violations should always be situations that 
you can prevent by changing the code of the underlying process_. You 
can't do that for network errors, disk state, etc.. But you can do that 
for stuff like ensuring a variable is never null, an object in your 
program is in some particular state at a particular point in execution, etc.

-- 
Bruno Medeiros - Software Engineer



 Jun 17 2010



 next sibling parent  Walter Bright <newshound1 digitalmars.com>  writes:


Bruno Medeiros wrote:

 I would go further and state that anything outside the direct control of 
 a process (such as network state, disk state, OS state, other processes 
 behavior, user interaction, etc.) should be modeled as an error and not 
 a contract violation.
 Such externals errors may be a "bug" in the system as whole, but they 
 are not a bug in the particular process, and thus should not be modeled 
 as a contract violation.
 In other words, _contract violations should always be situations that 
 you can prevent by changing the code of the underlying process_. You 
 can't do that for network errors, disk state, etc.. But you can do that 
 for stuff like ensuring a variable is never null, an object in your 
 program is in some particular state at a particular point in execution, 
 etc.


That's a reasonable way of looking at it.



 Jun 17 2010



prev sibling  parent  Sean Kelly <sean invisibleduck.org>  writes:


Bruno Medeiros <brunodomedeiros+spam com.gmail> wrote:

 On 17/06/2010 00:27, Walter Bright wrote:

 Lutger wrote:

 Walter Bright wrote:

 Furthermore, errors are something a program can recover from and
 continue
 operating. Contract failures are ALWAYS fatal. A common newbie (and
 some
 expert) misconception is that contract failures can or even must be
 recovered.
 This comes from a misunderstanding of the basic principles of
 engineering a
 safe and reliable system.


 
 I am not so sure about this last point, usually you want to fail but
 perhaps not always. This is about what to do after detection of a
 program bug vs how to handle an exceptional condition.


 
 First you need to decide if it is a program bug or not. If it is not
 a
 program bug, it shouldn't be done with contracts.
 


 
 I would go further and state that anything outside the direct control
 of a process (such as network state, disk state, OS state, other
 processes behavior, user interaction, etc.) should be modeled as an
 error and not a contract violation.
 Such externals errors may be a "bug" in the system as whole, but they
 are not a bug in the particular process, and thus should not be
 modeled as a contract violation.
 In other words, _contract violations should always be situations that
 you can prevent by changing the code of the underlying process_. You
 can't do that for network errors, disk state, etc.. But you can do
 that for stuff like ensuring a variable is never null, an object in
 your program is in some particular state at a particular point in
 execution, etc.


Right. I'd say contracts are to catch logic errors.



 Jun 18 2010



prev sibling  parent  Ary Borenszweig <ary esperanto.org.ar>  writes:


On 06/16/2010 11:44 PM, Walter Bright wrote:

 Ary Borenszweig wrote:

 On 06/16/2010 04:15 PM, Walter Bright wrote:

 The difference is not based on those 3 points, but on what Andrei wrote
 here. Contracts and error checking are completely distinct activities
 and should not be conflated.


 Could you please explain them? There are many people here that don't
 understand the difference between these two concepts (including me).
 So maybe we are too dumb, maybe those concepts are not generally known
 or maybe the explanation is not very well clear in the documentation.


 It has nothing to do with being dumb, as it is not obvious.

 Contracts are for verifying that your program is in a state that it is
 designed to be in. A contract failure is defined as a program bug.

 Errors, on the other hand, are things that can go wrong at run time,
 like your disk is full when trying to write a file. These are NOT
 program bugs.

 Another way to look at it is your program should continue to operate
 correctly if all the contracts are removed. This is not true of removing
 all error checking and handling.

 Furthermore, errors are something a program can recover from and
 continue operating. Contract failures are ALWAYS fatal. A common newbie
 (and some expert) misconception is that contract failures can or even
 must be recovered. This comes from a misunderstanding of the basic
 principles of engineering a safe and reliable system.


Ah, ok, now I understand. Thanks.



 Jun 16 2010



prev sibling  parent reply Michel Fortin <michel.fortin michelf.com>  writes:


On 2010-06-16 05:15:24 -0400, Walter Bright <newshound1 digitalmars.com> said:


 The difference is not based on those 3 points, but on what Andrei wrote 
 here. Contracts and error checking are completely distinct activities 
 and should not be conflated.


True.

Yet, enforce is inside std.contracts. If that isn't conflating the two 
concepts I wonder what it is. :-)

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/



 Jun 16 2010



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright <newshound1 digitalmars.com> 
 said:
 

 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.


 
 True.
 
 Yet, enforce is inside std.contracts. If that isn't conflating the two 
 concepts I wonder what it is. :-)


You're right! I think Lars' suggestion is sensible - we should move 
enforce to object. Better yet we should find a better name for 
std.contracts. Ideas?

Andrei



 Jun 16 2010



  parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet>  writes:


On Wed, 16 Jun 2010 07:31:39 -0700, Andrei Alexandrescu wrote:


 Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com> said:
 

 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.


 
 True.
 
 Yet, enforce is inside std.contracts. If that isn't conflating the two
 concepts I wonder what it is. :-)


 
 You're right! I think Lars' suggestion is sensible - we should move
 enforce to object. Better yet we should find a better name for
 std.contracts. Ideas?
 
 Andrei



A few suggestions (even though I still think it belongs in object.d), in 
no particular order:

std.enforce
std.assumptions
std.constraints
std.checks
std.tests
std.error
std.errcheck

-Lars



 Jun 17 2010



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 06/17/2010 04:10 AM, Lars T. Kyllingstad wrote:

 On Wed, 16 Jun 2010 07:31:39 -0700, Andrei Alexandrescu wrote:


 Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com>  said:


 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.


 True.

 Yet, enforce is inside std.contracts. If that isn't conflating the two
 concepts I wonder what it is. :-)


 You're right! I think Lars' suggestion is sensible - we should move
 enforce to object. Better yet we should find a better name for
 std.contracts. Ideas?

 Andrei



 A few suggestions (even though I still think it belongs in object.d), in
 no particular order:

 std.enforce
 std.assumptions
 std.constraints
 std.checks
 std.tests
 std.error
 std.errcheck

 -Lars


We haven't reached consensus on where to put enforce() and friends. Any 
other ideas? Of the above, I like std.checks.

Better yet, how about defining std.exception that includes a host of 
exception-related functionality (such as defining exceptions that retain 
file and line, perhaps stack traces etc.)?


Andrei



 Jun 27 2010



 next sibling parent  "Simen kjaeraas" <simen.kjaras gmail.com>  writes:


Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:


 Better yet, how about defining std.exception that includes a host of  
 exception-related functionality (such as defining exceptions that retain  
 file and line, perhaps stack traces etc.)?


Sounds good.

-- 
Simen



 Jun 27 2010



prev sibling next sibling parent  Jonathan M Davis <jmdavisprog gmail.com>  writes:


On Sunday 27 June 2010 16:09:02 Andrei Alexandrescu wrote:

 
 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.
 
 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?
 
 
 Andrei


std.exception sounds like a good plan. I'm not overly fond of any of the other 
names, and I'm not sure that I care much one way or the other if we pick one, 
but std.exception with a bunch of exception-related stuff sounds particularly 
useful and could help standardize some of the way exceptions are used in D code.

- Jonathan M Davis



 Jun 27 2010



prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org>  writes:


Andrei Alexandrescu Wrote:

 We haven't reached consensus on where to put enforce() and friends. Any 
 other ideas? Of the above, I like std.checks.
 
 Better yet, how about defining std.exception that includes a host of 
 exception-related functionality (such as defining exceptions that retain 
 file and line, perhaps stack traces etc.)?


The trace functionality already exists in druntime.  As for exceptions, they
may belong there as well if they're ones the runtime should be aware of.



 Jun 28 2010



  parent reply "Rory McGuire" <rmcguire neonova.co.za>  writes:


On Mon, 28 Jun 2010 17:36:15 +0200, Sean Kelly <sean invisibleduck.org>  
wrote:


 Andrei Alexandrescu Wrote:

 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.

 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?


 The trace functionality already exists in druntime.  As for exceptions,  
 they may belong there as well if they're ones the runtime should be  
 aware of.


How does one get a print out of the stack trace then? Is it a setting or  
something?



 Jun 28 2010



  parent reply Sean Kelly <sean invisibleduck.org>  writes:


Rory McGuire Wrote:


 On Mon, 28 Jun 2010 17:36:15 +0200, Sean Kelly <sean invisibleduck.org>  
 wrote:
 

 Andrei Alexandrescu Wrote:

 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.

 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?


 The trace functionality already exists in druntime.  As for exceptions,  
 they may belong there as well if they're ones the runtime should be  
 aware of.


 
 How does one get a print out of the stack trace then? Is it a setting or  
 something?


I should qualify my original statement by saying that it's only implemented for
Linux and OSX so far.  I have some of the declarations in for the Windows
implementation but haven't gotten to it yet.



 Jun 28 2010



 next sibling parent  "Rory McGuire" <rmcguire neonova.co.za>  writes:


On Mon, 28 Jun 2010 18:01:55 +0200, Sean Kelly <sean invisibleduck.org>  
wrote:


 Rory McGuire Wrote:


 On Mon, 28 Jun 2010 17:36:15 +0200, Sean Kelly <sean invisibleduck.org>
 wrote:


 Andrei Alexandrescu Wrote:

 We haven't reached consensus on where to put enforce() and friends.  




 Any

 other ideas? Of the above, I like std.checks.

 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that  




 retain

 file and line, perhaps stack traces etc.)?


 The trace functionality already exists in druntime.  As for  


 exceptions,

 they may belong there as well if they're ones the runtime should be
 aware of.


 How does one get a print out of the stack trace then? Is it a setting or
 something?


 I should qualify my original statement by saying that it's only  
 implemented for Linux and OSX so far.  I have some of the declarations  
 in for the Windows implementation but haven't gotten to it yet.


Is there a way to get the function name/line? I'm using this on ubuntu  
10.04.


void fun() {
	throw new Exception("eeek");
}

void main() {
	fun();
}

Output is:


object.Exception: eeek
----------------
../throw() [0x80493a0]
../throw() [0x804ba44]
../throw() [0x804b9a9]
../throw() [0x804ba81]
../throw() [0x804b9a9]
../throw() [0x804b958]
/lib/tls/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xf764cbd6]
../throw() [0x80492b1]


-Rory



 Jun 28 2010



prev sibling  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Sean Kelly wrote:

 Rory McGuire Wrote:
 

 On Mon, 28 Jun 2010 17:36:15 +0200, Sean Kelly <sean invisibleduck.org>  
 wrote:


 Andrei Alexandrescu Wrote:

 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.

 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?


 The trace functionality already exists in druntime.  As for exceptions,  
 they may belong there as well if they're ones the runtime should be  
 aware of.


 How does one get a print out of the stack trace then? Is it a setting or  
 something?


 
 I should qualify my original statement by saying that it's only implemented
for Linux and OSX so far.  I have some of the declarations in for the Windows
implementation but haven't gotten to it yet.


My stack traces look indecipherable on Ubuntu. They only contain module 
name and memory address.

Andrei



 Jun 28 2010



prev sibling next sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet>  writes:


On Sun, 27 Jun 2010 18:09:02 -0500, Andrei Alexandrescu wrote:


 On 06/17/2010 04:10 AM, Lars T. Kyllingstad wrote:

 On Wed, 16 Jun 2010 07:31:39 -0700, Andrei Alexandrescu wrote:


 Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com>  said:


 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.


 True.

 Yet, enforce is inside std.contracts. If that isn't conflating the
 two concepts I wonder what it is. :-)


 You're right! I think Lars' suggestion is sensible - we should move
 enforce to object. Better yet we should find a better name for
 std.contracts. Ideas?

 Andrei



 A few suggestions (even though I still think it belongs in object.d),
 in no particular order:

 std.enforce
 std.assumptions
 std.constraints
 std.checks
 std.tests
 std.error
 std.errcheck

 -Lars


 
 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.
 
 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?



TDPL mentions several times that enforce() is in std.contracts.  Doesn't 
that preclude moving it or renaming the module?

-Lars



 Jun 28 2010



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Lars T. Kyllingstad wrote:

 On Sun, 27 Jun 2010 18:09:02 -0500, Andrei Alexandrescu wrote:
 

 On 06/17/2010 04:10 AM, Lars T. Kyllingstad wrote:

 On Wed, 16 Jun 2010 07:31:39 -0700, Andrei Alexandrescu wrote:


 Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com>  said:


 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.


 True.

 Yet, enforce is inside std.contracts. If that isn't conflating the
 two concepts I wonder what it is. :-)


 You're right! I think Lars' suggestion is sensible - we should move
 enforce to object. Better yet we should find a better name for
 std.contracts. Ideas?

 Andrei


 A few suggestions (even though I still think it belongs in object.d),
 in no particular order:

 std.enforce
 std.assumptions
 std.constraints
 std.checks
 std.tests
 std.error
 std.errcheck

 -Lars


 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.

 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?


 
 
 TDPL mentions several times that enforce() is in std.contracts.  Doesn't 
 that preclude moving it or renaming the module?


I plan to move it to std.exception in a backward-compatible way (have 
std.conv consist of only one import, then deprecate it).

Andrei



 Jun 28 2010



prev sibling  parent  torhu <no spam.invalid>  writes:


On 28.06.2010 01:09, Andrei Alexandrescu wrote:
[...]

 We haven't reached consensus on where to put enforce() and friends. Any
 other ideas? Of the above, I like std.checks.

 Better yet, how about defining std.exception that includes a host of
 exception-related functionality (such as defining exceptions that retain
 file and line, perhaps stack traces etc.)?


How will std.exception relate to core.exception?  Seems to me having two 
module with that similiar names could easily be confusing.



 Jul 05 2010



prev sibling  parent reply Walter Bright <newshound1 digitalmars.com>  writes:


Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright <newshound1 digitalmars.com> 
 said:
 

 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.


 
 True.
 
 Yet, enforce is inside std.contracts. If that isn't conflating the two 
 concepts I wonder what it is. :-)


I agree completely. enforce must move.



 Jun 16 2010



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Walter Bright wrote:

 Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright 
 <newshound1 digitalmars.com> said:


 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.


 True.

 Yet, enforce is inside std.contracts. If that isn't conflating the two 
 concepts I wonder what it is. :-)


 
 I agree completely. enforce must move.


Where to?

Andrei



 Jun 16 2010



 next sibling parent reply Jonathan M Davis <jmdavisProg gmail.com>  writes:


Andrei Alexandrescu wrote:


 Walter Bright wrote:

 Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright
 <newshound1 digitalmars.com> said:


 The difference is not based on those 3 points, but on what Andrei
 wrote here. Contracts and error checking are completely distinct
 activities and should not be conflated.


 True.

 Yet, enforce is inside std.contracts. If that isn't conflating the two
 concepts I wonder what it is. :-)


 
 I agree completely. enforce must move.


 
 Where to?
 
 Andrei


I would point out that pretty much nothing in std.contracts actually relates 
to contracts. Rather, it relates to error handling. So, it would probably be 
a good idea to simply rename the module - perhaps to std.error.

- Jonathan M Davis



 Jun 16 2010



  parent reply Michel Fortin <michel.fortin michelf.com>  writes:


On 2010-06-16 14:10:17 -0400, Jonathan M Davis <jmdavisProg gmail.com> said:


 I would point out that pretty much nothing in std.contracts actually relates
 to contracts. Rather, it relates to error handling. So, it would probably be
 a good idea to simply rename the module - perhaps to std.error.


I concur: the module is misnamed. The only things not related to error 
handling are assumeUnique and assumeSorted, and I fail to see the link 
with design by contract for either one.


-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/



 Jun 16 2010



  parent reply Michel Fortin <michel.fortin michelf.com>  writes:


On 2010-06-16 14:44:29 -0400, Michel Fortin <michel.fortin michelf.com> said:


 On 2010-06-16 14:10:17 -0400, Jonathan M Davis <jmdavisProg gmail.com> said:
 

 I would point out that pretty much nothing in std.contracts actually relates
 to contracts. Rather, it relates to error handling. So, it would probably be
 a good idea to simply rename the module - perhaps to std.error.


 
 I concur: the module is misnamed. The only things not related to error 
 handling are assumeUnique and assumeSorted, and I fail to see the link 
 with design by contract for either one.


Oh, forgot about "pointsTo" too. What's the link with contracts, or 
error handling?

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/



 Jun 16 2010



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Michel Fortin wrote:

 On 2010-06-16 14:44:29 -0400, Michel Fortin <michel.fortin michelf.com> 
 said:
 

 On 2010-06-16 14:10:17 -0400, Jonathan M Davis <jmdavisProg gmail.com> 
 said:


 I would point out that pretty much nothing in std.contracts actually 
 relates
 to contracts. Rather, it relates to error handling. So, it would 
 probably be
 a good idea to simply rename the module - perhaps to std.error.


 I concur: the module is misnamed. The only things not related to error 
 handling are assumeUnique and assumeSorted, and I fail to see the link 
 with design by contract for either one.


 
 Oh, forgot about "pointsTo" too. What's the link with contracts, or 
 error handling?


Certain functions (notably swap) must make sure that there's no mutual 
aliasing between two objects.

Andrei



 Jun 16 2010



  parent reply Michel Fortin <michel.fortin michelf.com>  writes:


On 2010-06-16 14:59:45 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 Michel Fortin wrote:

 On 2010-06-16 14:44:29 -0400, Michel Fortin <michel.fortin michelf.com> said:
 

 On 2010-06-16 14:10:17 -0400, Jonathan M Davis <jmdavisProg gmail.com> said:
 

 I would point out that pretty much nothing in std.contracts actually relates
 to contracts. Rather, it relates to error handling. So, it would probably be
 a good idea to simply rename the module - perhaps to std.error.


 
 I concur: the module is misnamed. The only things not related to error 
 handling are assumeUnique and assumeSorted, and I fail to see the link 
 with design by contract for either one.


 
 Oh, forgot about "pointsTo" too. What's the link with contracts, or 
 error handling?


 
 Certain functions (notably swap) must make sure that there's no mutual 
 aliasing between two objects.


Ok, so you're using "pointsTo" to check this in a contract? But isn't 
that just a utility function which can be used for contracts as much as 
for everything else? Does it really belong in std.contracts because at 
some place you use it in a contract? I don't think so. But that's 
something for you to decide. And unfortunately I'm not sure where you 
put it.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/



 Jun 16 2010



  parent  Michel Fortin <michel.fortin michelf.com>  writes:


On 2010-06-16 20:45:47 -0400, Michel Fortin <michel.fortin michelf.com> said:


 Ok, so you're using "pointsTo" to check this in a contract? But isn't 
 that just a utility function which can be used for contracts as much as 
 for everything else? Does it really belong in std.contracts because at 
 some place you use it in a contract? I don't think so. But that's 
 something for you to decide. And unfortunately I'm not sure where you 
 put it.


Should have concluded by: "I'm not sure where you *should* put it either."

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/



 Jun 16 2010



prev sibling  parent reply Walter Bright <newshound1 digitalmars.com>  writes:


Andrei Alexandrescu wrote:

 Walter Bright wrote:

 Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright 
 <newshound1 digitalmars.com> said:


 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.


 True.

 Yet, enforce is inside std.contracts. If that isn't conflating the 
 two concepts I wonder what it is. :-)


 I agree completely. enforce must move.


 
 Where to?


Dunno.



 Jun 16 2010



  parent reply Don <nospam nospam.com>  writes:


Walter Bright wrote:

 Andrei Alexandrescu wrote:

 Walter Bright wrote:

 Michel Fortin wrote:

 On 2010-06-16 05:15:24 -0400, Walter Bright 
 <newshound1 digitalmars.com> said:


 The difference is not based on those 3 points, but on what Andrei 
 wrote here. Contracts and error checking are completely distinct 
 activities and should not be conflated.


 True.

 Yet, enforce is inside std.contracts. If that isn't conflating the 
 two concepts I wonder what it is. :-)


 I agree completely. enforce must move.


 Where to?


 
 Dunno.


import std.dunno;
Works for me.



 Jun 16 2010



 next sibling parent  Walter Bright <newshound1 digitalmars.com>  writes:


Don wrote:

 import std.dunno;
 Works for me.


cut & print.



 Jun 16 2010



prev sibling  parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com>  writes:


Don wrote:


 import std.dunno;
 Works for me.


Or std.poisson... :p

Ali



 Jun 16 2010



  parent  biozic <dransic free.fr>  writes:


Le 16/06/10 22:36, Ali Çehreli a écrit :

 Don wrote:


 import std.dunno;
 Works for me.


 Or std.poisson... :p


Better name it std.fishy, because std.poisson could be mistaken for a 
statistical distribution module!



 Jun 16 2010



prev sibling  parent reply bearophile <bearophileHUGS lycos.com>  writes:


Sorry for not answering before, I was quite busy (despite in the meantime I
have written few posts and bug reports).

Thank you to all the people that have answered in this thread, and expecially
Walter that has given the first answer that I have understood in the thread.

In the beginning I didn't like the enforce() but now I can see that it's meant
for a quite different purpose. I was also worried of possible negative
performance impact coming from its so widespread usage in Phobos.


Regarding the exception hierarchy I think I agree with Leandro Lucarella. A
deep and complex exception hierarchy can be negative and overkill, the opposite
extrema, that is having zero specialized exceptions in Phobos, is bad. Using
just single-exception enforce() is bad, there is a small number of well chosen
exception types (organized in a flat or mostly flat list) that are useful to
have.

In my dlibs1 I have defined a flat list of few exceptions that I used:

ArgumentException     
EmptyException        
ExceptionTemplate     
IndexException        
IOException           
KeyException          
MissingMethodException
OverflowException     
RangeException        
UncomparableException 


Inside Phobos2 I have counted about 160 usages of the "body" keyword. I think
contract programming can be used more often inside Phobos2 (and maybe some
usages of enforce() can be turned into contract programming because they are
more similar to program sanity checks).

Bye,
bearophile



 Jun 19 2010



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


Walter and I discussed this and concluded that Phobos should handle its 
parameters as user input. Therefore they need to be scrubbed with hard 
tests, not contracts.

Andrei



 Jun 19 2010



 next sibling parent reply Lutger <lutger.blijdestijn gmail.com>  writes:


Andrei Alexandrescu wrote:


 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


 
 Walter and I discussed this and concluded that Phobos should handle its
 parameters as user input. Therefore they need to be scrubbed with hard
 tests, not contracts.
 
 Andrei


That is sensible. Are private functions (those only called from within Phobos) 
excluded from this rule?



 Jun 19 2010



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 06/19/2010 05:01 PM, Lutger wrote:

 Andrei Alexandrescu wrote:


 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


 Walter and I discussed this and concluded that Phobos should handle its
 parameters as user input. Therefore they need to be scrubbed with hard
 tests, not contracts.

 Andrei


 That is sensible. Are private functions (those only called from within Phobos)
 excluded from this rule?


Yes, precisely. (The actual code does not fully obey this intention; 
there are contracts in places where there shouldn't be.)

Andrei



 Jun 19 2010



prev sibling next sibling parent reply Walter Bright <newshound1 digitalmars.com>  writes:


Andrei Alexandrescu wrote:

 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


 
 Walter and I discussed this and concluded that Phobos should handle its 
 parameters as user input. Therefore they need to be scrubbed with hard 
 tests, not contracts.


I should add that any library that may be used as a dll should have its 
interface API checked with hard tests, not contracts. This is because a dll 
cannot control who connects to it, and therefore must regard anything sent to
it 
as unvalidated user input.



 Jun 19 2010



  parent reply "Vladimir Panteleev" <vladimir thecybershadow.net>  writes:


On Sun, 20 Jun 2010 03:04:31 +0300, Walter Bright  
<newshound1 digitalmars.com> wrote:


 Andrei Alexandrescu wrote:

 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


  Walter and I discussed this and concluded that Phobos should handle  
 its parameters as user input. Therefore they need to be scrubbed with  
 hard tests, not contracts.


 I should add that any library that may be used as a dll should have its  
 interface API checked with hard tests, not contracts. This is because a  
 dll cannot control who connects to it, and therefore must regard  
 anything sent to it as unvalidated user input.


I don't see the logic in this...

Are we talking about validating user input for the sake of security, or  
debugging (catching bugs early)?

If it's for the sake of debugging, debug checks should remain in debug  
builds (that's what contracts are for?). Otherwise, you are stripping the  
programmer of the choice between higher performance or more debug checks.

If it's for the sake of security - parameter validation in DLLs is  
pointless. If you are able to load and call code from inside a DLL, you  
are already able to do everything that the DLL can. DLLs don't have any  
"setuid"-like properties. If we were talking, for example, about syscalls  
for a kernel module (functions called from userland but executed in kernel  
land), then that would be a completely different situation.

Also, I don't think that one rule can apply for everyone. For example, a  
high-performance DLL may specify in its documentation that the function  
parameters are not checked by the DLL and must be valid, otherwise  
undefined behavior will occur. (I believe some Windows APIs do not check  
some parameters and will cause access violations when called with invalid  
parameters.)

-- Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net



 Jun 19 2010



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


Vladimir Panteleev wrote:

 On Sun, 20 Jun 2010 03:04:31 +0300, Walter Bright 
 <newshound1 digitalmars.com> wrote:
 

 Andrei Alexandrescu wrote:

 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


  Walter and I discussed this and concluded that Phobos should handle 
 its parameters as user input. Therefore they need to be scrubbed with 
 hard tests, not contracts.


 I should add that any library that may be used as a dll should have 
 its interface API checked with hard tests, not contracts. This is 
 because a dll cannot control who connects to it, and therefore must 
 regard anything sent to it as unvalidated user input.


 
 I don't see the logic in this...
 
 Are we talking about validating user input for the sake of security, or 
 debugging (catching bugs early)?


An input to a dll is user input, and should be validated (for the sake of 
security, and other reasons). Validating it is not debugging.



 If it's for the sake of debugging, debug checks should remain in debug 
 builds (that's what contracts are for?). Otherwise, you are stripping 
 the programmer of the choice between higher performance or more debug 
 checks.
 
 If it's for the sake of security - parameter validation in DLLs is 
 pointless. If you are able to load and call code from inside a DLL, you 
 are already able to do everything that the DLL can. DLLs don't have any 
 "setuid"-like properties. If we were talking, for example, about 
 syscalls for a kernel module (functions called from userland but 
 executed in kernel land), then that would be a completely different 
 situation.


If you, for example, provide a pluggable interface to your browser app, that's 
done using a dll, and you'd better validate anything you get through that
plugin 
interface!


 Also, I don't think that one rule can apply for everyone. For example, a 
 high-performance DLL may specify in its documentation that the function 
 parameters are not checked by the DLL and must be valid, otherwise 
 undefined behavior will occur. (I believe some Windows APIs do not check 
 some parameters and will cause access violations when called with 
 invalid parameters.)


If you don't validate the input, then you must accept the risk.



 Jun 20 2010



 next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net>  writes:


On Mon, 21 Jun 2010 00:17:28 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:


 An input to a dll is user input, and should be validated (for the sake  
 of security, and other reasons). Validating it is not debugging.


I don't understand why you're saying this. Security checks in DLL  
functions are pointless, for the reasons I already outlined:


  If it's for the sake of security - parameter validation in DLLs is  
 pointless. If you are able to load and call code from inside a DLL, you  
 are already able to do everything that the DLL can. DLLs don't have any  
 "setuid"-like properties. If we were talking, for example, about  
 syscalls for a kernel module (functions called from userland but  
 executed in kernel land), then that would be a completely different  
 situation.


 If you, for example, provide a pluggable interface to your browser app,  
 that's done using a dll, and you'd better validate anything you get  
 through that plugin interface!


Why? When your application loads a DLL, the DLL instantly gets access to  
all of your application's memory, handles, and other resources. It's  
running in the same address space and security context. You need to  
completely trust the DLL - which is why new browsers (Google Chrome and  
experimental Firefox versions) load plugins in separate processes with  
reduced privileges.

-- Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net



 Jun 20 2010



 next sibling parent reply BCS <none anon.com>  writes:


Hello Vladimir,


 On Mon, 21 Jun 2010 00:17:28 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:
 

 An input to a dll is user input, and should be validated (for the
 sake  of security, and other reasons). Validating it is not
 debugging.
 


 I don't understand why you're saying this. Security checks in DLL
 functions are pointless, for the reasons I already outlined:
 


import my.dll;

void fn()
{
    auto data = get.userUncheckedInput();
    my.dll.doSomething(data); // if doSomething dosn't check it's inputs, 
then this can cause a security flaw
}

Yes that's your dll's user's fault but adding the checks solves it even so. 
To boot, it reduces your support cost (as long as people read error message) 
and prevents the user from having to debug starting deep inside your dll.


 If it's for the sake of security - parameter validation in DLLs is
 pointless. If you are able to load and call code from inside a DLL,
 you  are already able to do everything that the DLL can. DLLs don't
 have any  "setuid"-like properties. If we were talking, for example,
 about  syscalls for a kernel module (functions called from userland
 but  executed in kernel land), then that would be a completely
 different  situation.
 


 If you, for example, provide a pluggable interface to your browser
 app,  that's done using a dll, and you'd better validate anything you
 get  through that plugin interface!
 


 Why? When your application loads a DLL, the DLL instantly gets access
 to  all of your application's memory, handles, and other resources.
 It's  running in the same address space and security context. You need
 to  completely trust the DLL - which is why new browsers (Google
 Chrome and  experimental Firefox versions) load plugins in separate
 processes with  reduced privileges.


And you can bet that every byte of data shipped back and forth via IPC is 
validated more than an air traveler at a TSA checkpoint.

As for the case where the dll is local, never attribute to malice that which 
can be adequately explained by stupidity. Unless you have source, you can't 
assume that the data coming out doesn't conation unvalidated user input and 
you should always assume that someone malicious will get ahold of that sooner 
or later.  
 
-- 
... <IXOYE><



 Jun 20 2010



  parent reply "Vladimir Panteleev" <vladimir thecybershadow.net>  writes:


On Mon, 21 Jun 2010 03:02:53 +0300, BCS <none anon.com> wrote:


 import my.dll;

 void fn()
 {
     auto data = get.userUncheckedInput();
     my.dll.doSomething(data); // if doSomething dosn't check it's  
 inputs, then this can cause a security flaw
 }

 Yes that's your dll's user's fault but adding the checks solves it even  
 so. To boot, it reduces your support cost (as long as people read error  
 message) and prevents the user from having to debug starting deep inside  
 your dll.


A well-designed application needs to validate unsafe user input exactly  
once (assuming the process of validation is the same).
DLL interfaces must specify whether the input/output can be considered  
safe or not.
Not doing so results in either security holes or redundant code.


 And you can bet that every byte of data shipped back and forth via IPC  
 is validated more than an air traveler at a TSA checkpoint.


Obviously, no argument here.


 As for the case where the dll is local, never attribute to malice that  
 which can be adequately explained by stupidity. Unless you have source,  
 you can't assume that the data coming out doesn't conation unvalidated  
 user input and you should always assume that someone malicious will get  
 ahold of that sooner or later.


Unless you have the source, you can't assume that simply loading the DLL  
will not create a security hole.

Trusting the DLL but not trusting the data it gives you is a plausible  
case. As I said before, this simply needs to be well-defined, and  
validation doesn't have to happen exactly at the DLL boundary.


-- Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net



 Jun 20 2010



  parent reply BCS <none anon.com>  writes:


Hello Vladimir,


 On Mon, 21 Jun 2010 03:02:53 +0300, BCS <none anon.com> wrote:
 

 import my.dll;
 
 void fn()
 {
 auto data = get.userUncheckedInput();
 my.dll.doSomething(data); // if doSomething dosn't check it's
 inputs, then this can cause a security flaw
 }
 Yes that's your dll's user's fault but adding the checks solves it
 even  so. To boot, it reduces your support cost (as long as people
 read error  message) and prevents the user from having to debug
 starting deep inside  your dll.
 


 A well-designed application needs to validate unsafe user input
 exactly
 once (assuming the process of validation is the same).
 DLL interfaces must specify whether the input/output can be considered
 safe or not.
 Not doing so results in either security holes or redundant code.


If I didn't write the DLL I'm calling, I'll assume it doesn't check stuff. 
If I didn't write the code calling my DLL, I'll assume it doesn't check stuff. 
Why should I assume that the documentation is right or that people will even 
read my documentation? Unless you can show me that this causes a perf problem, 
the benefit just isn't worth the cost.


 And you can bet that every byte of data shipped back and forth via
 IPC  is validated more than an air traveler at a TSA checkpoint.
 


 Obviously, no argument here.
 

 As for the case where the dll is local, never attribute to malice
 that  which can be adequately explained by stupidity. Unless you have
 source,  you can't assume that the data coming out doesn't conation
 unvalidated  user input and you should always assume that someone
 malicious will get  ahold of that sooner or later.
 


 Unless you have the source, you can't assume that simply loading the
 DLL  will not create a security hole.


OK, but just because there might be a risk in loading a DLL is no reason 
to not address other risk that you can deal with.


 
 Trusting the DLL but not trusting the data it gives you is a plausible
 case. As I said before, this simply needs to be well-defined, and
 validation doesn't have to happen exactly at the DLL boundary.


Good point, I might not check it exactly at the call site, but *I* will will 
check it because I will assume that any checks on the other side of a DLL 
interface are flawed, missing, broken or just flat wrong.

-- 
... <IXOYE><



 Jun 20 2010



  parent  "Vladimir Panteleev" <vladimir thecybershadow.net>  writes:


On Mon, 21 Jun 2010 04:53:46 +0300, BCS <none anon.com> wrote:


 If I didn't write the DLL I'm calling, I'll assume it doesn't check  
 stuff. If I didn't write the code calling my DLL, I'll assume it doesn't  
 check stuff. Why should I assume that the documentation is right or that  
 people will even read my documentation? Unless you can show me that this  
 causes a perf problem, the benefit just isn't worth the cost.


If you can't trust the DLL to perform correct user data validation, you  
can't trust it AT ALL! For all you know it can have a buffer overflow  
vulnerability. Re-validating any data you get from it may save you from  
one type of bug, but it doesn't improve security by much overall.

Regarding performance: what is not a "performance problem" in any one  
single place can make a considerable difference when you sum up all the  
redundant checks in your entire codebase.

A practical example from the industry:

For Microsoft partners, Windows is available in a "free" (or retail) build  
and a "checked" build [1].
Since driver code runs in kernel space, drivers can crash the entire  
system anyway - for this reason, there are few checks or kernel mode APIs.  
(That's why sometimes when debugging BSoDs, crashes will happen in a  
completely unrelated kernel module.)
However, if you need to debug your driver, you run it on the checked  
version of Windows, which additionally to having lots of debug checks is  
also built without most compiler optimizations.

   [1]: http://msdn.microsoft.com/en-us/library/ff543450(VS.85).aspx

-- Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net



 Jun 21 2010



prev sibling next sibling parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 06/20/2010 06:18 PM, Vladimir Panteleev wrote:

 On Mon, 21 Jun 2010 00:17:28 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:


 An input to a dll is user input, and should be validated (for the sake
 of security, and other reasons). Validating it is not debugging.


 I don't understand why you're saying this. Security checks in DLL
 functions are pointless, for the reasons I already outlined:


[snip]

I think the matter is simpler than that. Essentially DbC is modular 
integrity checking. If Phobos enforce()s parameters in calls, then it 
considers its own integrity a different matter than the integrity of the 
application it's used with.

If Phobos used contracts to validate parameters, it would directly share 
responsibility for the integrity of the entire application. That way, 
users will not be sure whether the failure is a bug in Phobos or one in 
their own code.


Andrei



 Jun 20 2010



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


Vladimir Panteleev wrote:

 On Mon, 21 Jun 2010 00:17:28 +0300, Walter Bright 
 <newshound2 digitalmars.com> wrote:
 

 An input to a dll is user input, and should be validated (for the sake 
 of security, and other reasons). Validating it is not debugging.


 
 I don't understand why you're saying this. Security checks in DLL 
 functions are pointless, for the reasons I already outlined:


It's true that whenever user code is executed, that code can do anything.
Hello, 
ActiveX. But I still think it's sound practice to treat any data received from 
another program as untrusted, and validate it. Security, like I said, is only 
one reason. Another is to prevent bugs in external code from trashing your
process.



 Jun 20 2010



 next sibling parent  "Vladimir Panteleev" <vladimir thecybershadow.net>  writes:


On Mon, 21 Jun 2010 03:40:48 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Vladimir Panteleev wrote:

 On Mon, 21 Jun 2010 00:17:28 +0300, Walter Bright  
 <newshound2 digitalmars.com> wrote:


 An input to a dll is user input, and should be validated (for the sake  
 of security, and other reasons). Validating it is not debugging.


  I don't understand why you're saying this. Security checks in DLL  
 functions are pointless, for the reasons I already outlined:


 It's true that whenever user code is executed, that code can do  
 anything. Hello, ActiveX. But I still think it's sound practice to treat  
 any data received from another program as untrusted, and validate it.  
 Security, like I said, is only one reason. Another is to prevent bugs in  
 external code from trashing your process.


Yes, but this is a completely different kind of trust (incompetence  
instead of intentional malice) :)

I was simply arguing the technical point of pointlessness of verifying  
data from DLLs specifically for security reasons (buffer overflows, code  
injection etc.).

Other than that, this is the usual performance vs. robustness dilemma  
(though my personal opinion is that an ideal language/platform/etc. should  
allow programmers to take all the responsibility for maximum performance).

-- Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net



 Jun 20 2010



prev sibling  parent reply Leandro Lucarella <luca llucax.com.ar>  writes:


Walter Bright, el 20 de junio a las 17:40 me escribiste:

 Vladimir Panteleev wrote:

On Mon, 21 Jun 2010 00:17:28 +0300, Walter Bright
<newshound2 digitalmars.com> wrote:


An input to a dll is user input, and should be validated (for
the sake of security, and other reasons). Validating it is not
debugging.


I don't understand why you're saying this. Security checks in DLL
functions are pointless, for the reasons I already outlined:


 
 It's true that whenever user code is executed, that code can do
 anything. Hello, ActiveX. But I still think it's sound practice to
 treat any data received from another program as untrusted, and
 validate it. Security, like I said, is only one reason. Another is
 to prevent bugs in external code from trashing your process.


How can you prevent that? If you pass incorrect data to a DLL, then the
bug is *yours*. If the DLL has a bug, it will explode anyways. You are
just trying to catch programs bugs in the DLL, which seems overly
patronizing to me. Why will you assume I'm so dumb that I won't use your
interface correctly?

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Karma police
arrest this girl,
her Hitler hairdo
is making me feel ill
and we have crashed her party.



 Jun 20 2010



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


Leandro Lucarella wrote:

 Why will you assume I'm so dumb that I won't use your
 interface correctly?


Windows has had major legacy compatibility issues because critical third party 
applications misused the APIs.

People *will* misuse your API, and you will get blamed for it. It's unfair, but 
that's how it works.



 Jun 20 2010



  parent reply Leandro Lucarella <luca llucax.com.ar>  writes:


Walter Bright, el 20 de junio a las 19:32 me escribiste:

 Leandro Lucarella wrote:

Why will you assume I'm so dumb that I won't use your
interface correctly?


 
 Windows has had major legacy compatibility issues because critical
 third party applications misused the APIs.
 
 People *will* misuse your API, and you will get blamed for it. It's
 unfair, but that's how it works.


Luckily I haven't used Windows for about 10 years now =)

It's really a shame that D will take the stupidity route.

PS: I don't know how windows work, but if calling the Windows API is
    like going into kernel mode, and you can mess other processes, it
    seems reasonable to do check every API call as if it were user
    input, but if you're confined to your process, is really stupid.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Vaporeso sostenía a rajacincha la teoría del No-Water, la cual le
pertenecía y versaba lo siguiente: "Para darle la otra mejilla al fuego,
éste debe ser apagado con alpargatas apenas húmedas".



 Jun 20 2010



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 06/20/2010 11:08 PM, Leandro Lucarella wrote:

 Walter Bright, el 20 de junio a las 19:32 me escribiste:

 Leandro Lucarella wrote:

 Why will you assume I'm so dumb that I won't use your
 interface correctly?


 Windows has had major legacy compatibility issues because critical
 third party applications misused the APIs.

 People *will* misuse your API, and you will get blamed for it. It's
 unfair, but that's how it works.


 Luckily I haven't used Windows for about 10 years now =)

 It's really a shame that D will take the stupidity route.

 PS: I don't know how windows work, but if calling the Windows API is
      like going into kernel mode, and you can mess other processes, it
      seems reasonable to do check every API call as if it were user
      input, but if you're confined to your process, is really stupid.


Why is it stupid?

Andrei



 Jun 21 2010



 next sibling parent reply Leandro Lucarella <luca llucax.com.ar>  writes:


Andrei Alexandrescu, el 21 de junio a las 08:02 me escribiste:

 On 06/20/2010 11:08 PM, Leandro Lucarella wrote:

Walter Bright, el 20 de junio a las 19:32 me escribiste:

Leandro Lucarella wrote:

Why will you assume I'm so dumb that I won't use your
interface correctly?


Windows has had major legacy compatibility issues because critical
third party applications misused the APIs.

People *will* misuse your API, and you will get blamed for it. It's
unfair, but that's how it works.


Luckily I haven't used Windows for about 10 years now =)

It's really a shame that D will take the stupidity route.

PS: I don't know how windows work, but if calling the Windows API is
     like going into kernel mode, and you can mess other processes, it
     seems reasonable to do check every API call as if it were user
     input, but if you're confined to your process, is really stupid.


 
 Why is it stupid?


Because you're adding unnecessary extra checks, just based on
(Windows?) programmer's stupidity.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
JUGAR COMPULSIVAMENTE ES PERJUDICIAL PARA LA SALUD.
	-- Casino de Mar del Plata



 Jun 21 2010



  parent  "Vladimir Panteleev" <vladimir thecybershadow.net>  writes:


On Mon, 21 Jun 2010 16:30:48 +0300, Leandro Lucarella <luca llucax.com.ar>  
wrote:


 Andrei Alexandrescu, el 21 de junio a las 08:02 me escribiste:

 On 06/20/2010 11:08 PM, Leandro Lucarella wrote:

Walter Bright, el 20 de junio a las 19:32 me escribiste:

Leandro Lucarella wrote:

Why will you assume I'm so dumb that I won't use your
interface correctly?


Windows has had major legacy compatibility issues because critical
third party applications misused the APIs.

People *will* misuse your API, and you will get blamed for it. It's
unfair, but that's how it works.


Luckily I haven't used Windows for about 10 years now =)

It's really a shame that D will take the stupidity route.

PS: I don't know how windows work, but if calling the Windows API is
     like going into kernel mode, and you can mess other processes, it
     seems reasonable to do check every API call as if it were user
     input, but if you're confined to your process, is really stupid.


 Why is it stupid?


 Because you're adding unnecessary extra checks, just based on
 (Windows?) programmer's stupidity.


Walter makes a good point. If someone uses your API in the wrong way and  
relies on undocumented/undefined behavior, you'll end up having to support  
this usage pattern in future implementations of your interface if you want  
businesses and other entities who depend on that product to buy your new  
operating system.

-- Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net



 Jun 21 2010



prev sibling  parent reply Sean Kelly <sean invisibleduck.org>  writes:


Andrei Alexandrescu Wrote:


 On 06/20/2010 06:18 PM, Vladimir Panteleev wrote:

 On Mon, 21 Jun 2010 00:17:28 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:


 An input to a dll is user input, and should be validated (for the sake
 of security, and other reasons). Validating it is not debugging.


 I don't understand why you're saying this. Security checks in DLL
 functions are pointless, for the reasons I already outlined:


 [snip]
 
 I think the matter is simpler than that. Essentially DbC is modular 
 integrity checking. If Phobos enforce()s parameters in calls, then it 
 considers its own integrity a different matter than the integrity of the 
 application it's used with.
 
 If Phobos used contracts to validate parameters, it would directly share 
 responsibility for the integrity of the entire application. That way, 
 users will not be sure whether the failure is a bug in Phobos or one in 
 their own code.


If a unrecoverable failure occurs within the process, does it matter where the
error originated?

I've been thinking about this a bit and am starting to wonder about the benefit
of distinguishing API boundary integrity checking vs. internal integrity
checking.  First, what if a library eats its own dogfood?  If my library
provides a public method to spawn threads and the library itself uses threads
internally then I have two different methods of checking the integrity of my
own library, each possibly throwing different exceptions (enforce throws
Exception while assert throws AssertError).  At the very least, this seems like
it could cause maintenance issues because a logic change deep within the
library may require additional exception handling to deal with what are
intended to be user-facing errors.

In a similar vein, if contracts are used within an API but a different mode of
checking is used at the API boundary, when contracts are enabled in that API
the user is faced with the bizarre issue of receiving AssertErrors from
internal API logic errors but only Exceptions from his own logic errors for API
boundary calls.  When you say that DbC is modular I'd presume that means it's
encapsulated within each distinct subsystem, but it seems like this isn't true
at all.  The alternative is to turn DbC off in the library and either live with
undefined behavior or a hard crash if there's a bug in the library, a
circumstance which is again forced upon the user.

Regarding DbC, I can't say that I've ever worked on a system where lives hung
in the balance (an admittedly extreme example of where DbC is useful), but if I
were sufficiently concerned about process integrity that I had contracts
enabled then I don't think I would trust that a third-party library was
bug-free and therefore didn't need its own contract checking in place.  Once
I've accepted the cost of integrity checking I want it everywhere, not just in
my own code.  It makes for consistent error checking behavior (I'd assume there
is a system in place to trap DbC errors specifically) and provides full-process
integrity checking.

I think the only boundary that really matters is the process boundary.  Any
error within a process has the same effect regardless of whether it's in user
code or library code--corrupted memory, segfaults, etc--so why make a
distinction between code I wrote and code someone else wrote?  By the same
token, if the user chooses to disable contracts then why force them upon him
for some errors but not others?  The user is making the explicit choice to run
his process through a meat-grinder if something unexpected happens, he's
effectively asserting that his code is perfect, so why tell him that "no, it's
actually not."

For me, the more difficult issue is how much integrity checking should be done.
 For example, I created an AVL tree a while back that verified that the tree
was still properly balanced after every API call.  This was great from a code
verification standpoint, but the check completely violated the complexity
guarantees and ultimately checked something that could have been proven to a
reasonable degree of confidence through code reviews and unit testing.  Should
checks like this be enabled automatically when DbC is turned on?  Should there
be different levels of DbC?  In some respects I feel like there's a difference
between in/out contracts and invariants, but even that doesn't seem completely
right.  Thoughts?



 Jun 21 2010



 next sibling parent  bearophile <bearophileHUGS lycos.com>  writes:


Sean Kelly:


First, what if a library eats its own dogfood?  If my library provides a public
method to spawn threads and the library itself uses threads internally then I
have two different methods of checking the integrity of my own library, each
possibly throwing different exceptions (enforce throws Exception while assert
throws AssertError).<


I think that Design by contract, to be useful, needs to be embraced. You need
to trust it and use it everywhere. Now I use DbC quite often in my D code and I
appreciate it. The DbC feature I miss mostly is the "old" (view of the original
input data).



For example, I created an AVL tree a while back that verified that the tree was
still properly balanced after every API call.  This was great from a code
verification standpoint, but the check completely violated the complexity
guarantees and ultimately checked something that could have been proven to a
reasonable degree of confidence through code reviews and unit testing.  Should
checks like this be enabled automatically when DbC is turned on?  Should there
be different levels of DbC?  In some respects I feel like there's a difference
between in/out contracts and invariants, but even that doesn't seem completely
right.  Thoughts?<


Using Design by Contract is not easy, you need to train yourself to use it
well. A problem is that DbC is uncommon, only Eiffel and few other languages
use it seriously, so lot of D users have to learn DbC on D itself.

I face your problem putting inside the contracts code that doesn't change the
complexity of the code it guards (so for example if the code is O(n^2) I don't
add contracts that perform O(n^3) computations).

Then where it's useful I add stronger tests (that can be slower) inside
debug{}. You can see it here too:
http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=109395&header

Inside the invariant there is O(1) test code, and it also contains inside a
debug{} O(n) test code (that is .

In debug mode I want to test the code very well, while in normal nonrelease
mode I can accept less stringent tests that make the code usable.

Unittests and DbC (and integration tests, functional tests, smoke tests, etc)
are both useful, they do different things :-) For example an unittest can tell
me a function is wrong, but a loop invariant can tell me where the bug is and
when it happens inside the function :-)

Bye,
bearophile



 Jun 21 2010



prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


Sean Kelly wrote:

 Regarding DbC, I can't say that I've ever worked on a system where lives hung
 in the balance (an admittedly extreme example of where DbC is useful),


I have, and here's how it's done:

http://www.drdobbs.com/blog/archives/2009/10/safe_systems_fr.html

http://www.drdobbs.com/blog/archives/2009/11/designing_safe.html

I really wish this was more widely known in the software engineering business. 
It's frustrating to see it relearned the hard way, over and over.

And not just the software business, I saw a technical overview of the BP oil 
spill failure, and the rig design violated just about every principle of safe 
system design.



 Jun 22 2010



 next sibling parent reply Sean Kelly <sean invisibleduck.org>  writes:


Walter Bright Wrote:


 Sean Kelly wrote:

 Regarding DbC, I can't say that I've ever worked on a system where lives hung
 in the balance (an admittedly extreme example of where DbC is useful),


 
 I have, and here's how it's done:
 
 http://www.drdobbs.com/blog/archives/2009/10/safe_systems_fr.html
 
 http://www.drdobbs.com/blog/archives/2009/11/designing_safe.html
 
 I really wish this was more widely known in the software engineering business. 
 It's frustrating to see it relearned the hard way, over and over.
 
 And not just the software business, I saw a technical overview of the BP oil 
 spill failure, and the rig design violated just about every principle of safe 
 system design.


A coworker of mine knows a guy who had workers on that rig and told me this
story the other day.  Apparently, there's a system on the drill that when a
failure occurs a cap drops over the hole and shears the drill.  The BP rig was
drilling unusually deep though, and as a result the drill had to be incredibly
hard.  For this and other reasons, the safety system was estimated to have a
70% failure rate.  Furthermore, the rig was known to be on the verge of
failure.  He implored the BP folks to shut it down, but they refused so in
desperation he hired people to fly his team off the rig, fearing for their
safety.  The rig failed a few hours after his team was evacuated.

While I've never worked on systems where lives hang in the balance, I have
worked on systems where 100% uptime is required.  I favor the Erlang approach
where a system is a web of interconnected, redundant processes that terminate
on errors.  I've found this design an extremely hard sell in the internet
server world though.  The design takes more planning and people are in too much
of a hurry.



 Jun 22 2010



  parent reply Sean Kelly <sean invisibleduck.org>  writes:


Sean Kelly Wrote:

 
 While I've never worked on systems where lives hang in the balance, I have
worked on systems where 100% uptime is required.  I favor the Erlang approach
where a system is a web of interconnected, redundant processes that terminate
on errors.  I've found this design an extremely hard sell in the internet
server world though.  The design takes more planning and people are in too much
of a hurry.


I should add that I'm hoping the message passing model in D will help encourage
reliable system design.  With thread isolation it should be pretty easy to move
parts of a program into separate processes as need dictates.



 Jun 22 2010



  parent reply Lutger <lutger.blijdestijn gmail.com>  writes:


Sean Kelly wrote:


 Sean Kelly Wrote:

 
 While I've never worked on systems where lives hang in the balance, I have
 worked on systems where 100% uptime is required.  I favor the Erlang approach
 where a system is a web of interconnected, redundant processes that terminate
 on errors.  I've found this design an extremely hard sell in the internet
 server world though.  The design takes more planning and people are in too
 much of a hurry.


 
 I should add that I'm hoping the message passing model in D will help
 encourage reliable system design.  With thread isolation it should be pretty
 easy to move parts of a program into separate processes as need dictates.


Can we look forward to seeing ipc supported in phobos via the same interface in 
the future? I really like the api you have created.



 Jun 23 2010



  parent reply Sean Kelly <sean invisibleduck.org>  writes:


Lutger <lutger.blijdestijn gmail.com> wrote:

 Sean Kelly wrote:
 

 Sean Kelly Wrote:

 
 While I've never worked on systems where lives hang in the balance,
 I have
 worked on systems where 100% uptime is required.  I favor the Erlang
 approach
 where a system is a web of interconnected, redundant processes that
 terminate
 on errors.  I've found this design an extremely hard sell in the
 internet
 server world though.  The design takes more planning and people are
 in too
 much of a hurry.


 
 I should add that I'm hoping the message passing model in D will help
 encourage reliable system design.  With thread isolation it should be
 pretty
 easy to move parts of a program into separate processes as need
 dictates.


 
 Can we look forward to seeing ipc supported in phobos via the same
 interface in 
 the future? I really like the api you have created.


Yes. The core send/receive API should work just fine for IPC, and it's
definitely on the map.  The greatest obstacle there is probably the need
for a solid serialization/deserialization package in Phobos.



 Jun 24 2010



  parent reply Jacob Carlborg <doob me.com>  writes:


On 2010-06-25 05:17, Sean Kelly wrote:

 Lutger<lutger.blijdestijn gmail.com>  wrote:

 Sean Kelly wrote:


 Sean Kelly Wrote:

 While I've never worked on systems where lives hang in the balance,
 I have
 worked on systems where 100% uptime is required.  I favor the Erlang
 approach
 where a system is a web of interconnected, redundant processes that
 terminate
 on errors.  I've found this design an extremely hard sell in the
 internet
 server world though.  The design takes more planning and people are
 in too
 much of a hurry.


 I should add that I'm hoping the message passing model in D will help
 encourage reliable system design.  With thread isolation it should be
 pretty
 easy to move parts of a program into separate processes as need
 dictates.


 Can we look forward to seeing ipc supported in phobos via the same
 interface in
 the future? I really like the api you have created.


 Yes. The core send/receive API should work just fine for IPC, and it's
 definitely on the map.  The greatest obstacle there is probably the need
 for a solid serialization/deserialization package in Phobos.


I have a serialization library, http://dsource.org/projects/orange/ , 
this is a list of some its features:

* It handles both serializing and deserializing
* It automatically serializes the base classes
* It supports events (before and after (de)serializing)
* It supports non-serialized fields (you can say that some fields in a 
class should not be serialized)
* It's licensed under the Boost license
* It's fairly std/runtime library independent
* You can create new archive types and use them with the existing serializer
* Serializes through base class references
* Serializes third party types

Currently it only works using Tango but the only part of the library 
that is dependent on the std/runtime library is XMLArchive, I'm 
currently working on porting it to Phobos. It also needs testing.

Also issue 2844 and the one, can't find it now, about getMembers is not 
implemented at all (returns an empty array).

-- 
/Jacob Carlborg



 Jun 25 2010



 next sibling parent  Jacob Carlborg <doob me.com>  writes:


On 2010-06-25 14:54, Jacob Carlborg wrote:

 On 2010-06-25 05:17, Sean Kelly wrote:

 Lutger<lutger.blijdestijn gmail.com> wrote:

 Sean Kelly wrote:


 Sean Kelly Wrote:

 While I've never worked on systems where lives hang in the balance,
 I have
 worked on systems where 100% uptime is required. I favor the Erlang
 approach
 where a system is a web of interconnected, redundant processes that
 terminate
 on errors. I've found this design an extremely hard sell in the
 internet
 server world though. The design takes more planning and people are
 in too
 much of a hurry.


 I should add that I'm hoping the message passing model in D will help
 encourage reliable system design. With thread isolation it should be
 pretty
 easy to move parts of a program into separate processes as need
 dictates.


 Can we look forward to seeing ipc supported in phobos via the same
 interface in
 the future? I really like the api you have created.


 Yes. The core send/receive API should work just fine for IPC, and it's
 definitely on the map. The greatest obstacle there is probably the need
 for a solid serialization/deserialization package in Phobos.


 I have a serialization library, http://dsource.org/projects/orange/ ,
 this is a list of some its features:

 * It handles both serializing and deserializing
 * It automatically serializes the base classes
 * It supports events (before and after (de)serializing)
 * It supports non-serialized fields (you can say that some fields in a
 class should not be serialized)
 * It's licensed under the Boost license
 * It's fairly std/runtime library independent
 * You can create new archive types and use them with the existing
 serializer
 * Serializes through base class references
 * Serializes third party types

 Currently it only works using Tango but the only part of the library
 that is dependent on the std/runtime library is XMLArchive, I'm
 currently working on porting it to Phobos. It also needs testing.

 Also issue 2844 and the one, can't find it now, about getMembers is not
 implemented at all (returns an empty array).


... should be fixed.


-- 
/Jacob Carlborg



 Jun 25 2010



prev sibling  parent reply "Robert Jacques" <sandford jhu.edu>  writes:


On Fri, 25 Jun 2010 08:54:54 -0400, Jacob Carlborg <doob me.com> wrote:


 On 2010-06-25 05:17, Sean Kelly wrote:

 Lutger<lutger.blijdestijn gmail.com>  wrote:

 Sean Kelly wrote:


 Sean Kelly Wrote:

 While I've never worked on systems where lives hang in the balance,
 I have
 worked on systems where 100% uptime is required.  I favor the Erlang
 approach
 where a system is a web of interconnected, redundant processes that
 terminate
 on errors.  I've found this design an extremely hard sell in the
 internet
 server world though.  The design takes more planning and people are
 in too
 much of a hurry.


 I should add that I'm hoping the message passing model in D will help
 encourage reliable system design.  With thread isolation it should be
 pretty
 easy to move parts of a program into separate processes as need
 dictates.


 Can we look forward to seeing ipc supported in phobos via the same
 interface in
 the future? I really like the api you have created.


 Yes. The core send/receive API should work just fine for IPC, and it's
 definitely on the map.  The greatest obstacle there is probably the need
 for a solid serialization/deserialization package in Phobos.


 I have a serialization library, http://dsource.org/projects/orange/ ,  
 this is a list of some its features:

 * It handles both serializing and deserializing
 * It automatically serializes the base classes
 * It supports events (before and after (de)serializing)
 * It supports non-serialized fields (you can say that some fields in a  
 class should not be serialized)
 * It's licensed under the Boost license
 * It's fairly std/runtime library independent
 * You can create new archive types and use them with the existing  
 serializer
 * Serializes through base class references
 * Serializes third party types

 Currently it only works using Tango but the only part of the library  
 that is dependent on the std/runtime library is XMLArchive, I'm  
 currently working on porting it to Phobos. It also needs testing.

 Also issue 2844 and the one, can't find it now, about getMembers is not  
 implemented at all (returns an empty array).


I'll volunteer to help test (and to add JSON capabilities) when you're  
ready.



 Jun 25 2010



  parent  Jacob Carlborg <doob me.com>  writes:


On 2010-06-25 17:40, Robert Jacques wrote:

 On Fri, 25 Jun 2010 08:54:54 -0400, Jacob Carlborg <doob me.com> wrote:


 On 2010-06-25 05:17, Sean Kelly wrote:

 Lutger<lutger.blijdestijn gmail.com> wrote:

 Sean Kelly wrote:


 Sean Kelly Wrote:

 While I've never worked on systems where lives hang in the balance,
 I have
 worked on systems where 100% uptime is required. I favor the Erlang
 approach
 where a system is a web of interconnected, redundant processes that
 terminate
 on errors. I've found this design an extremely hard sell in the
 internet
 server world though. The design takes more planning and people are
 in too
 much of a hurry.


 I should add that I'm hoping the message passing model in D will help
 encourage reliable system design. With thread isolation it should be
 pretty
 easy to move parts of a program into separate processes as need
 dictates.


 Can we look forward to seeing ipc supported in phobos via the same
 interface in
 the future? I really like the api you have created.


 Yes. The core send/receive API should work just fine for IPC, and it's
 definitely on the map. The greatest obstacle there is probably the need
 for a solid serialization/deserialization package in Phobos.


 I have a serialization library, http://dsource.org/projects/orange/ ,
 this is a list of some its features:

 * It handles both serializing and deserializing
 * It automatically serializes the base classes
 * It supports events (before and after (de)serializing)
 * It supports non-serialized fields (you can say that some fields in a
 class should not be serialized)
 * It's licensed under the Boost license
 * It's fairly std/runtime library independent
 * You can create new archive types and use them with the existing
 serializer
 * Serializes through base class references
 * Serializes third party types

 Currently it only works using Tango but the only part of the library
 that is dependent on the std/runtime library is XMLArchive, I'm
 currently working on porting it to Phobos. It also needs testing.

 Also issue 2844 and the one, can't find it now, about getMembers is
 not implemented at all (returns an empty array).


 I'll volunteer to help test (and to add JSON capabilities) when you're
 ready.


It's ready to be tested with D1 and Tango. You can also start building a 
JSON archive, if you use D2 this will also make sure there is no other 
Tango dependencies (other than in XMLArchive). Let me no if you need any 
help.

-- 
/Jacob Carlborg



 Jun 25 2010



prev sibling  parent  BCS <none anon.com>  writes:


Hello Walter,


 And not just the software business, I saw a technical overview of the
 BP oil spill failure, and the rig design violated just about every
 principle of safe system design.


link by chance?

-- 
... <IXOYE><



 Jun 22 2010



prev sibling  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 06/21/2010 01:14 PM, Sean Kelly wrote:

 Andrei Alexandrescu Wrote:


 On 06/20/2010 06:18 PM, Vladimir Panteleev wrote:

 On Mon, 21 Jun 2010 00:17:28 +0300, Walter Bright
 <newshound2 digitalmars.com>  wrote:


 An input to a dll is user input, and should be validated (for
 the sake of security, and other reasons). Validating it is not
 debugging.


 I don't understand why you're saying this. Security checks in
 DLL functions are pointless, for the reasons I already outlined:


 [snip]

 I think the matter is simpler than that. Essentially DbC is
 modular integrity checking. If Phobos enforce()s parameters in
 calls, then it considers its own integrity a different matter than
 the integrity of the application it's used with.

 If Phobos used contracts to validate parameters, it would directly
 share responsibility for the integrity of the entire application.
 That way, users will not be sure whether the failure is a bug in
 Phobos or one in their own code.


 If a unrecoverable failure occurs within the process, does it matter
 where the error originated?


It does matter for the postmortem. Black box ftw.


 I've been thinking about this a bit and am starting to wonder about
 the benefit of distinguishing API boundary integrity checking vs.
 internal integrity checking.  First, what if a library eats its own
 dogfood?  If my library provides a public method to spawn threads and
 the library itself uses threads internally then I have two different
 methods of checking the integrity of my own library, each possibly
 throwing different exceptions (enforce throws Exception while assert
 throws AssertError).  At the very least, this seems like it could
 cause maintenance issues because a logic change deep within the
 library may require additional exception handling to deal with what
 are intended to be user-facing errors.


Any complex API will face at some point some tension between reusing and 
duplicating code. Per the classic joke:

"A mathematician was in a habit of making a cup of tea when working late 
at night. His normal method was to get the teapot from the cupboard, 
take the teapot to the sink, add water, heat to boiling, then make the 
cup of tea. Unfortunately, one night when he went to make tea, the 
teapot was already full of water and sitting on the stove. He thought 
about this for several minutes, then emptied the teapot and put it back 
in the cupboard, thereby reducing this to a previously solved problem."

It's often the case that an API throws some water for the sake of 
reusing itself.


 In a similar vein, if contracts are used within an API but a
 different mode of checking is used at the API boundary, when
 contracts are enabled in that API the user is faced with the bizarre
 issue of receiving AssertErrors from internal API logic errors but
 only Exceptions from his own logic errors for API boundary calls.
 When you say that DbC is modular I'd presume that means it's
 encapsulated within each distinct subsystem, but it seems like this
 isn't true at all.  The alternative is to turn DbC off in the library
 and either live with undefined behavior or a hard crash if there's a
 bug in the library, a circumstance which is again forced upon the
 user.


When I say DbC is modular I have in mind the following: "assert() inside 
a well-defined library entity (e.g. Phobos) is supposed to check the 
integrity of that entity, not the integrity of the entity using it."

I think that's reasonable. Integrity is a cross-cutting concern in 
memory-unsafe programs, but not in memory-safe programs. (Whoa, that's 
interesting.)


 Regarding DbC, I can't say that I've ever worked on a system where
 lives hung in the balance (an admittedly extreme example of where DbC
 is useful), but if I were sufficiently concerned about process
 integrity that I had contracts enabled then I don't think I would
 trust that a third-party library was bug-free and therefore didn't
 need its own contract checking in place.  Once I've accepted the cost
 of integrity checking I want it everywhere, not just in my own code.
 It makes for consistent error checking behavior (I'd assume there is
 a system in place to trap DbC errors specifically) and provides
 full-process integrity checking.


There's a large spectrum between "people will die" etc. and "I don't 
give a flying frak". I think an application under construction may well 
choose to use contracts for itself but not for the well-tested (ahem) 
Phobos.


 I think the only boundary that really matters is the process
 boundary.  Any error within a process has the same effect regardless
 of whether it's in user code or library code--corrupted memory,
 segfaults, etc--so why make a distinction between code I wrote and
 code someone else wrote?


For post-mortem and for assigning blame appropriately. If Phobos used 
DbC on user-passed inputs it would essentially share blame for the 
application integrity with all applications.


 By the same token, if the user chooses to
 disable contracts then why force them upon him for some errors but
 not others?  The user is making the explicit choice to run his
 process through a meat-grinder if something unexpected happens, he's
 effectively asserting that his code is perfect, so why tell him that
 "no, it's actually not."


That's a good point, but imho not enough to challenge the status quo. It 
might make sense to have a "unsafe" build for Phobos that assumes 
absolutely all arguments are correct. I wonder how much of an 
improvement it would bring.


 For me, the more difficult issue is how much integrity checking
 should be done.  For example, I created an AVL tree a while back that
 verified that the tree was still properly balanced after every API
 call.  This was great from a code verification standpoint, but the
 check completely violated the complexity guarantees and ultimately
 checked something that could have been proven to a reasonable degree
 of confidence through code reviews and unit testing.  Should checks
 like this be enabled automatically when DbC is turned on?  Should
 there be different levels of DbC?  In some respects I feel like
 there's a difference between in/out contracts and invariants, but
 even that doesn't seem completely right.  Thoughts?


I take no prisoners there: integrity checks MUST NOT affect complexity. 
Complexity is part of the spec, so such checks would automatically 
violate the spec.

At some point in its history, the binary search functions in 
std.algorithm had a enforce(isSorted) check. Should be somewhere buried 
in the svn history. It was my fault, and I didn't realize it until after 
I waited for one day next to a script to complete.


Andrei



 Jun 27 2010



prev sibling next sibling parent reply "Rory McGuire" <rmcguire neonova.co.za>  writes:


On Mon, 21 Jun 2010 06:08:01 +0200, Leandro Lucarella <luca llucax.com.ar>  
wrote:


 Walter Bright, el 20 de junio a las 19:32 me escribiste:

 Leandro Lucarella wrote:

Why will you assume I'm so dumb that I won't use your
interface correctly?


 Windows has had major legacy compatibility issues because critical
 third party applications misused the APIs.

 People *will* misuse your API, and you will get blamed for it. It's
 unfair, but that's how it works.


 Luckily I haven't used Windows for about 10 years now =)

 It's really a shame that D will take the stupidity route.

 PS: I don't know how windows work, but if calling the Windows API is
     like going into kernel mode, and you can mess other processes, it
     seems reasonable to do check every API call as if it were user
     input, but if you're confined to your process, is really stupid.


I think perhaps you mis-understood, it is mostly not stupidity that causes  
people to use
undocumented "features" of an API but rather, it is people being overly  
"clever".



 Jun 21 2010



  parent reply Sean Kelly <sean invisibleduck.org>  writes:


Rory McGuire Wrote:

 
 I think perhaps you mis-understood, it is mostly not stupidity that causes  
 people to use
 undocumented "features" of an API but rather, it is people being overly  
 "clever".


Or sometimes simply desperation.  There are some classes of apps that require
the use of undocumented API calls to operate on Windows--I believe disk
defragmenters are one example.  Microsoft rightly didn't document these calls
because it wasn't prepared to support them long-term, but in doing so they also
prevented users from doing necessary work and effectively forced them into
using API calls that might change unexpectedly.  I think these users accept
this problem and do the necessary verification and updating when new OS
revisions are released however.



 Jun 21 2010



  parent  Don <nospam nospam.com>  writes:


Sean Kelly wrote:

 Rory McGuire Wrote:

 I think perhaps you mis-understood, it is mostly not stupidity that causes  
 people to use
 undocumented "features" of an API but rather, it is people being overly  
 "clever".


 
 Or sometimes simply desperation.  There are some classes of apps that require
the use of undocumented API calls to operate on Windows--I believe disk
defragmenters are one example.  Microsoft rightly didn't document these calls
because it wasn't prepared to support them long-term, but in doing so they also
prevented users from doing necessary work and effectively forced them into
using API calls that might change unexpectedly.  I think these users accept
this problem and do the necessary verification and updating when new OS
revisions are released however.


Remember Windows 3.0? File handling involved undocumented API calls!



 Jun 21 2010



prev sibling  parent reply Adrian Matoga <epi atari8.info>  writes:


Leandro Lucarella pisze:

 Walter Bright, el 20 de junio a las 19:32 me escribiste:

 Leandro Lucarella wrote:

 Why will you assume I'm so dumb that I won't use your
 interface correctly?


 Windows has had major legacy compatibility issues because critical
 third party applications misused the APIs.

 People *will* misuse your API, and you will get blamed for it. It's
 unfair, but that's how it works.


 
 Luckily I haven't used Windows for about 10 years now =)
 
 It's really a shame that D will take the stupidity route.
 
 PS: I don't know how windows work, but if calling the Windows API is
     like going into kernel mode, and you can mess other processes, it
     seems reasonable to do check every API call as if it were user
     input, but if you're confined to your process, is really stupid.
 


It was 15 years ago, at the times of 3.x and 95, when Windows behaved 
like that.

The problem applies not only to Windows, but any API you would imagine.
A common situation is when you need to do your job quickly using only 
some part of a library which otherwise you aren't going to study 
thoroughly, or you want only a proof-of-concept. And if your attempting 
to use something new to you, you do make mistakes, no matter how you are 
convinced yo do not.
If the API is defined not by documentation (which is often a tissue of 
lies, and hardly ever it's unambiguous), but by means of input checking, 
you have benefits in two fields: 1) developers of library had to think 
what they wanted to do, so library probably works, and it's less 
probable that its new versions will break the compatiblity, and 2) users 
of the library will be warned quickly, saving their time.

It's not about messing other processes. It's about saving your time, 
otherwise consumed by effects of common mistakes, misunderstanding the 
documentation, or working in a hurry. And your time costs much more than 
the time of bazillion argument checks.



 Jun 21 2010



  parent  "Vladimir Panteleev" <vladimir thecybershadow.net>  writes:


On Mon, 21 Jun 2010 21:15:12 +0300, Adrian Matoga <epi atari8.info> wrote:


 It was 15 years ago, at the times of 3.x and 95, when Windows behaved  
 like that.


More like 10, Windows Millennium was the last 9x-based Windows operating  
system without strong process isolation.


 The problem applies not only to Windows, but any API you would imagine.
 A common situation is when you need to do your job quickly using only  
 some part of a library which otherwise you aren't going to study  
 thoroughly, or you want only a proof-of-concept. And if your attempting  
 to use something new to you, you do make mistakes, no matter how you are  
 convinced yo do not.
 If the API is defined not by documentation (which is often a tissue of  
 lies, and hardly ever it's unambiguous), but by means of input checking,  
 you have benefits in two fields: 1) developers of library had to think  
 what they wanted to do, so library probably works, and it's less  
 probable that its new versions will break the compatiblity, and 2) users  
 of the library will be warned quickly, saving their time.

 It's not about messing other processes. It's about saving your time,  
 otherwise consumed by effects of common mistakes, misunderstanding the  
 documentation, or working in a hurry. And your time costs much more than  
 the time of bazillion argument checks.


For this particular situation, contracts would be just fine though :)

-- Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net



 Jun 21 2010



prev sibling  parent reply BCS <none anon.com>  writes:


Hello Leandro,


 If the DLL has a bug, it will explode anyways.


A DLL can work just fine (a.k.a. not explode) and still return garbage as 
long as it never depends on not seeing the kind of garbage it's producing. 
Say for instance it's written in D and returns a string with a missing \0 
as a char*/length, the DLL works just fine but a printf blows up.


 You
 are just trying to catch programs bugs in the DLL,


Exactly.


 which seems overly
 patronizing to me.


Um, it doesn't to me.


 Why will you assume I'm so dumb that I won't use your interface correctly?


First because some people are. And second, because it trivially easy to respond 
to support calls that start with "Your DLL is throwing a
YouAreNotUsingThisDLLCorrectlyRTFM 
Exception" <joke/>.

-- 
... <IXOYE><



 Jun 20 2010



  parent reply "Vladimir Panteleev" <vladimir thecybershadow.net>  writes:


On Mon, 21 Jun 2010 07:42:34 +0300, BCS <none anon.com> wrote:


 Why will you assume I'm so dumb that I won't use your interface  
 correctly?


 First because some people are. And second, because it trivially easy to  
 respond to support calls that start with "Your DLL is throwing a  
 YouAreNotUsingThisDLLCorrectlyRTFM Exception" <joke/>.


I think that for such situations you should ship a debug and release  
version of the DLL.
This way you don't sacrifice performance when the user doesn't want to be  
held by the hand.

-- Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net



 Jun 21 2010



  parent reply BCS <none anon.com>  writes:


Hello Vladimir,


 On Mon, 21 Jun 2010 07:42:34 +0300, BCS <none anon.com> wrote:
 

 Why will you assume I'm so dumb that I won't use your interface
 correctly?
 


 First because some people are. And second, because it trivially easy
 to  respond to support calls that start with "Your DLL is throwing a
 YouAreNotUsingThisDLLCorrectlyRTFM Exception" <joke/>.
 


 I think that for such situations you should ship a debug and release
 version of the DLL.
 This way you don't sacrifice performance when the user doesn't want to
 be
 held by the hand.


Until you can show me a perf problem, I don't see any point in doing that. 
(OTOH, deep structure validation, or anything else slower than O(1), is another 
thing all together)

-- 
... <IXOYE><



 Jun 21 2010



  parent reply "Simen kjaeraas" <simen.kjaras gmail.com>  writes:


BCS <none anon.com> wrote:

 I think that for such situations you should ship a debug and release
 version of the DLL.
 This way you don't sacrifice performance when the user doesn't want to
 be
 held by the hand.


 Until you can show me a perf problem, I don't see any point in doing  
 that. (OTOH, deep structure validation, or anything else slower than  
 O(1), is another thing all together)


Also, if you do have two different versions, I'll bet you ready money
someone will program only the release version, because "that's the
version the users will have", "the debug version is slow" or whatever
other inane excuse their minds are capable of coming up with.

-- 
Simen



 Jun 22 2010



 next sibling parent  Sean Kelly <sean invisibleduck.org>  writes:


"Simen kjaeraas" <simen.kjaras gmail.com> wrote:

 BCS <none anon.com> wrote:

 I think that for such situations you should ship a debug and release
 version of the DLL.
 This way you don't sacrifice performance when the user doesn't want
 to
 be
 held by the hand.


 
 Until you can show me a perf problem, I don't see any point in doing 

 that. (OTOH, deep structure validation, or anything else slower
 than  > O(1), is another thing all together)




 
 Also, if you do have two different versions, I'll bet you ready money
 someone will program only the release version, because "that's the
 version the users will have", "the debug version is slow" or whatever
 other inane excuse their minds are capable of coming up with.


What I've done with druntime is build checked and unchecked versions.  I
don't think it makes sense to ship a debug version of a library because
that's for debugging the library, not user code. I'll admit that I like
having debug symbols in place though, just not the debug tests
themselves. It's occasionally nice to not have a trace vanish just
because it passes through library code.



 Jun 22 2010



prev sibling  parent  Lutger <lutger.blijdestijn gmail.com>  writes:


Simen kjaeraas wrote:


 BCS <none anon.com> wrote:

 I think that for such situations you should ship a debug and release
 version of the DLL.
 This way you don't sacrifice performance when the user doesn't want to
 be
 held by the hand.


 Until you can show me a perf problem, I don't see any point in doing
 that. (OTOH, deep structure validation, or anything else slower than
 O(1), is another thing all together)


 
 Also, if you do have two different versions, I'll bet you ready money
 someone will program only the release version, because "that's the
 version the users will have", "the debug version is slow" or whatever
 other inane excuse their minds are capable of coming up with.
 


Naturally, debug is for debugging, not shipping. Instead one could make three 
versions:
- debug  
- release
- unsafe

Or rather let the user compile unsafe themselves if you can distribute the 
source code. 

I am sure most inane users (like me) will choose release.



 Jun 22 2010



prev sibling  parent reply Norbert Nemec <Norbert Nemec-online.de>  writes:


On 20/06/10 22:17, Walter Bright wrote:

 An input to a dll is user input, and should be validated (for the sake
 of security, and other reasons). Validating it is not debugging.


In that case, feel free to compile DLLs with external contract checking 
switched on, but please do not blur the conceptual distinction between 
contracts and exceptions.

You are talking about compiling a libary into a binary DLL that should 
be fit for general usage. In that case, there are good reasons to leave 
the input contract checking active.

In the general case, however, the library user has the control over how 
to compile the library and link to it (just think of inlining). In this 
case, the library user should be allowed to switch off the contract 
checking (at their own risk!)

Conceptually, the ultimate solution would certainly be to place code for 
input contract checking in the *calling* code. After all, this checking 
code serves to debug the calling code, so it should be left to the 
caller to decide whether checking is necessary.

This approach would also allow the compiler to optimize out some checks 
when their correctness can be tested at compile time.

Output contract checks, on the other hand should be compiled inside the 
returning routine.

After all, it is all a matter of trust. A language designer should trust 
the language user to know what he is doing. A library designer should 
trust the library user to act responsibly. After all - if the 
application breaks it is the application designer who has to answer for it.



 Jun 28 2010



 next sibling parent  "Simen kjaeraas" <simen.kjaras gmail.com>  writes:


Norbert Nemec <Norbert nemec-online.de> wrote:


 A library designer should trust the library user to act responsibly.
 After all - if the application breaks it is the application designer
 who has to answer for it.


And if the application designer finds that his design breaks due to
a change in the library, he will blame the library designers. If it
is used in a big, well-known and much-used application, the library
designer might have no choice but to continue supporting broken
code.

-- 
Simen



 Jun 28 2010



prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com>  writes:


Norbert Nemec:

 [...] to place code for input contract checking in the *calling* code. [...]
 Output contract checks, on the other hand should be compiled inside the 
 returning routine.


Is this a positive thing to do? Can this be done? (D must support separate
compilation, but in many situations this is not done/necessary, so maybe in
such situations it can be done). Is Eiffel doing it? if it's a good thing and
it's doable then what kind of changes does it require to the compiler?

Bye,
bearophile



 Jun 28 2010



  parent reply Norbert Nemec <Norbert Nemec-online.de>  writes:


On 28/06/10 12:59, bearophile wrote:

 Norbert Nemec:

 [...] to place code for input contract checking in the *calling* code. [...]
 Output contract checks, on the other hand should be compiled inside the
 returning routine.


 Is this a positive thing to do? Can this be done? (D must support separate
compilation, but in many situations this is not done/necessary, so maybe in
such situations it can be done). Is Eiffel doing it? if it's a good thing and
it's doable then what kind of changes does it require to the compiler?


These are good and pragmatic questions that you ask.

The whole issue only arises when doing separate compilation of a library 
and an application. (I use the term "application" for any code that uses 
the library.)

In an idea world (beware, I am switching of "pragmatic thinking mode" 
for a moment), I would describe the situation would as follows:

Either part can be compiled in "debug" mode or in "release" mode. Debug 
mode in the library means that you want to debug the library code 
itself. Release mode in the library means that you trust the library 
code to be correct and switch off all internal checks.

The interesting situation is when you compile the application in debug 
mode, using a stable library compiled in release mode. In this case, the 
input contracts of the library need to be checked to catch bugs in the 
application. Once the testing phase is finished, you want to compile the 
application and get rid of the contract checks.

In this idealized picture, the input contracts should clearly be checked 
in the application code so that the application developer has control 
over them.

Now, for the real world: Code is hardly never 100% bug free. I agree. 
With this argument, however, the concept of debug and release mode 
becomes pointless as you should never switch off the checks anyway. 
Assuming that you are not quite as paranoid you might still reach a 
point where you trust your code to switch off the checks. How about 
input contracts of a library?

As Simen mentioned in his post, there is the issue of library authors 
trying to avoid blame from application authors and the other way around. 
Ultimately, this is exactly what contracts are for: formalize the 
interface as much as possible and make it machine-checkable. If an 
application breaks, compile it in debug mode and see whether it finds 
the problem. With input contracts checked in the calling code, this 
would automatically switch on all the relevant checks to identify bugs 
in the application. If the application designer still tries to blame the 
library, why not simply supply a library compiled in debug mode? Any 
violation of a contract or an assertion unquestionably signals a bug in 
the code.

Can it be done? Certainly. The contract is part of the interface, so 
rather than compiling it into the DLL, it should be stored with the 
interface definitions and left for the application compiler to insert if 
requested. The code for this should not be any more involved than 
function inlining. Effectively, every function with contracts would be 
turned into a wrapper that first checks the contracts and then calls the 
real function in the DLL. The application compiler could then simply 
decide whether to inline the wrapper or simply call the function without 
checks.

I have no idea how Eiffel does it, but I am quite certain that this 
solution is following the original spirit of DbC of Bertrand Meyer.

Greetings,
Norbert



 Jun 30 2010



  parent reply Sean Kelly <sean invisibleduck.org>  writes:


Norbert Nemec Wrote:


 On 28/06/10 12:59, bearophile wrote:

 Norbert Nemec:

 [...] to place code for input contract checking in the *calling* code. [...]
 Output contract checks, on the other hand should be compiled inside the
 returning routine.


 Is this a positive thing to do? Can this be done? (D must support separate
compilation, but in many situations this is not done/necessary, so maybe in
such situations it can be done). Is Eiffel doing it? if it's a good thing and
it's doable then what kind of changes does it require to the compiler?


 
 These are good and pragmatic questions that you ask.
 
 The whole issue only arises when doing separate compilation of a library 
 and an application. (I use the term "application" for any code that uses 
 the library.)
 
 In an idea world (beware, I am switching of "pragmatic thinking mode" 
 for a moment), I would describe the situation would as follows:
 
 Either part can be compiled in "debug" mode or in "release" mode. Debug 
 mode in the library means that you want to debug the library code 
 itself. Release mode in the library means that you trust the library 
 code to be correct and switch off all internal checks.


I see the choice of "release" for disabling contracts as a huge mistake in
nomenclature.  For libraries, I would ship a checked and unchecked build (with
-release disabled and enabled), but none with -debug or -unittest set.  Those
are for internal testing and the user shouldn't care to turn on debug code in a
library simply because he's debugging his own app.

The idea of compiling the "in" contract into the application code is an
interesting one, but I suspect it could be tricky.  Consider an unchecked build
of the library, a checked build of the app, and now taking the address of a
library function.  Worse, what if a library routine returns the address of
another library routine?  Now the application has a reference to an unchecked
version of the function, even if the involved technical hurdles are surmounted
(multiple entry points or the like).

This seems like a nice idea but it seems too complicated.  I'd rather just ship
checked and unchecked builds of a library and leave it at that.



 Jun 30 2010



  parent reply Norbert Nemec <Norbert Nemec-online.de>  writes:


On 30/06/10 17:45, Sean Kelly wrote:

 Norbert Nemec Wrote:


 On 28/06/10 12:59, bearophile wrote:

 Norbert Nemec:

 [...] to place code for input contract checking in the *calling* code. [...]
 Output contract checks, on the other hand should be compiled inside the
 returning routine.


 Is this a positive thing to do? Can this be done? (D must support separate
compilation, but in many situations this is not done/necessary, so maybe in
such situations it can be done). Is Eiffel doing it? if it's a good thing and
it's doable then what kind of changes does it require to the compiler?


 These are good and pragmatic questions that you ask.

 The whole issue only arises when doing separate compilation of a library
 and an application. (I use the term "application" for any code that uses
 the library.)

 In an idea world (beware, I am switching of "pragmatic thinking mode"
 for a moment), I would describe the situation would as follows:

 Either part can be compiled in "debug" mode or in "release" mode. Debug
 mode in the library means that you want to debug the library code
 itself. Release mode in the library means that you trust the library
 code to be correct and switch off all internal checks.


 I see the choice of "release" for disabling contracts as a huge mistake in
nomenclature.  For libraries, I would ship a checked and unchecked build (with
-release disabled and enabled), but none with -debug or -unittest set.  Those
are for internal testing and the user shouldn't care to turn on debug code in a
library simply because he's debugging his own app.

 The idea of compiling the "in" contract into the application code is an
interesting one, but I suspect it could be tricky.  Consider an unchecked build
of the library, a checked build of the app, and now taking the address of a
library function.  Worse, what if a library routine returns the address of
another library routine?  Now the application has a reference to an unchecked
version of the function, even if the involved technical hurdles are surmounted
(multiple entry points or the like).


That's indeed an interesting aspect: Design by Contract (DbC) and 
function pointers. I am not sure how these concepts would merge properly 
at all.

The contracts are part of the interface, so they should in fact be part 
of the function pointer type! Of course this would quickly become 
ridiculous.

A strongly object oriented language like Eiffel can in principle do 
without function pointers. Instead, one can in most cases use classes 
with virtual functions that offer very similar functionality. A class 
interface comes with all the contracts, so everything is safe and sound.

I really do not know how to deal with function pointers in the clean DbC 
paradigm. If you assign a function with input contracts to a function 
pointer, whoever uses the pointer does not know about the contracts. 
This however, breaks down the strong DbC concept and turns contracts 
into mere run time checks.

Does this mean that D should give up the goal of proper DbC? Simply do 
the pragmatic thing and pick the best pieces from DbC without worrying 
about formal completeness? I guess so...



 Jun 30 2010



  parent reply Jay Byrd <JayByrd rebels.com>  writes:


On Wed, 30 Jun 2010 20:03:07 +0100, Norbert Nemec wrote:


 On 30/06/10 17:45, Sean Kelly wrote:

 Norbert Nemec Wrote:


 On 28/06/10 12:59, bearophile wrote:

 Norbert Nemec:

 [...] to place code for input contract checking in the *calling*
 code. [...] Output contract checks, on the other hand should be
 compiled inside the returning routine.


 Is this a positive thing to do? Can this be done? (D must support
 separate compilation, but in many situations this is not
 done/necessary, so maybe in such situations it can be done). Is
 Eiffel doing it? if it's a good thing and it's doable then what kind
 of changes does it require to the compiler?


 These are good and pragmatic questions that you ask.

 The whole issue only arises when doing separate compilation of a
 library and an application. (I use the term "application" for any code
 that uses the library.)

 In an idea world (beware, I am switching of "pragmatic thinking mode"
 for a moment), I would describe the situation would as follows:

 Either part can be compiled in "debug" mode or in "release" mode.
 Debug mode in the library means that you want to debug the library
 code itself. Release mode in the library means that you trust the
 library code to be correct and switch off all internal checks.


 I see the choice of "release" for disabling contracts as a huge mistake
 in nomenclature.  For libraries, I would ship a checked and unchecked
 build (with -release disabled and enabled), but none with -debug or
 -unittest set.  Those are for internal testing and the user shouldn't
 care to turn on debug code in a library simply because he's debugging
 his own app.

 The idea of compiling the "in" contract into the application code is an
 interesting one, but I suspect it could be tricky.  Consider an
 unchecked build of the library, a checked build of the app, and now
 taking the address of a library function.  Worse, what if a library
 routine returns the address of another library routine?  Now the
 application has a reference to an unchecked version of the function,
 even if the involved technical hurdles are surmounted (multiple entry
 points or the like).


 
 That's indeed an interesting aspect: Design by Contract (DbC) and
 function pointers. I am not sure how these concepts would merge properly
 at all.
 
 The contracts are part of the interface, so they should in fact be part
 of the function pointer type! Of course this would quickly become
 ridiculous.
 
 A strongly object oriented language like Eiffel can in principle do
 without function pointers. Instead, one can in most cases use classes
 with virtual functions that offer very similar functionality. A class
 interface comes with all the contracts, so everything is safe and sound.
 
 I really do not know how to deal with function pointers in the clean DbC
 paradigm. If you assign a function with input contracts to a function
 pointer, whoever uses the pointer does not know about the contracts.
 This however, breaks down the strong DbC concept and turns contracts
 into mere run time checks.
 
 Does this mean that D should give up the goal of proper DbC? Simply do
 the pragmatic thing and pick the best pieces from DbC without worrying
 about formal completeness? I guess so...


This is all very confused, and is reflected in D implementing contracts 
all wrong. Contracts do not belong to function pointers or any other 
dynamic state -- they apply to the invoker, and thus the static type. 
Isn't that obvious? If I have Foo f = getSomeFoo(); result = f.method
(args), the args must satisfy the contract for Foo.method(), regardless 
of what method() in the object returned by getSomeFoo() is willing to 
accept (it must, of course, not require more than Foo.method() does; TDPL 
at least gets that right). And the guarantees on result must be those 
promised by Foo.method(), not some much weaker promise given by method() 
in its base class (and there is no need to check stronger guarantees made 
by method() the actual derived object, since the caller didn't ask for 
them).

The D implementation works much too hard to get the wrong result, both in 
complexities of the compiler and in executing a bunch of irrelevant code 
that is ignored if it fails in preconditions or succeeds in 
postconditions, and after all that it fails to enforce requirements it 
should and to guarantee promises that it should.

-- JB



 Sep 10 2010



 next sibling parent reply bearophile <bearophileHUGS lycos.com>  writes:


Jay Byrd:

 This is all very confused, and is reflected in D implementing contracts all
wrong.


If you know well the ideas of DbC, and you think there are some problems in the
DbC of D2, then I suggest you to not just write what's wrong, why it is wrong
and what bad things such wrong design may cause, and what you suggest to
change, starting from the most important changes.

Maybe Walter will change nothing, but if you write just a little rant where you
say that all is wrong, probably nothing will change, and what you have written
is useless.

Bye,
bearophile



 Sep 11 2010



 next sibling parent  bearophile <bearophileHUGS lycos.com>  writes:


 then I suggest you to not just write what's...


Ignore that 'not', please.



 Sep 11 2010



prev sibling  parent  retard <re tard.com.invalid>  writes:


Sat, 11 Sep 2010 07:16:56 -0400, bearophile wrote:


 Jay Byrd:

 This is all very confused, and is reflected in D implementing contracts
 all wrong.


 
 If you know well the ideas of DbC, and you think there are some problems
 in the DbC of D2, then I suggest you to not just write what's wrong, why
 it is wrong and what bad things such wrong design may cause, and what
 you suggest to change, starting from the most important changes.
 
 Maybe Walter will change nothing, but if you write just a little rant
 where you say that all is wrong, probably nothing will change, and what
 you have written is useless.
 
 Bye,
 bearophile


People are doing it wrong. They shouldn't come and rant here. They should 
write compiler patches instead. All discussion is bad, real code matters.



 Sep 11 2010



prev sibling  parent reply Norbert Nemec <Norbert Nemec-online.de>  writes:


On 11/09/10 08:18, Jay Byrd wrote:

 Contracts do not belong to function pointers or any other
 dynamic state -- they apply to the invoker, and thus the static type.
 Isn't that obvious?


In fact, it is yet one step more complex than that: as the name itself 
suggests, contracts are "between" the caller (static type) and the 
callee (dynamic type). Theoretically, the type system has to ensure that 
both sides are able to fulfill their part of the contract.

The dynamic type must be a subtype of the static type (including 
equality). In subtyping, in-contracts may be weakened, out-contracts may 
be strengthened (in other words, a subtype may require less and promise 
more).

This is all fine, theoretically sound and easy to handle in a clean way 
for object oriented design as it is done in Eiffel.

The complication in D are function pointers and delegates (FP/DG). For a 
clean design, the type of a FP/DG would need to include contract 
information. Contracts are part of the interface and a FP/DG would have 
to include this. Obviously, this would make FP/DG syntax rather awkward. 
Furthermore, FP/DG assignments would need to be type-checked at compile 
time, so contract compatibility would have to be checked at compile time 
as well. This would be completely impossible.

I conclude that within pure OOP, contracts can have strong compile-time 
support. In-contracts should be checked by the caller, out-contracts by 
the callee and both checks could be eliminated if the compiler can 
verify at compile time that they are fulfilled. With FP/DG, this breaks 
down and I believe the best one can do is to implement contracts as 
run-time checks in the callee, just as it is done in D.

The only detail that I would wish for is a more fine-grained tuning of 
DbC contract checks in the compiler and a clearer conceptual separation 
of the concepts of assertions and contracts. However, the former is an 
implementation detail and the latter has been discussed to death before.



 Sep 11 2010



  parent reply bearophile <bearophileHUGS lycos.com>  writes:


Norbert Nemec:

Thank you for your clear explanations of the situation.


 With FP/DG, this breaks 
 down and I believe the best one can do is to implement contracts as 
 run-time checks in the callee, just as it is done in D.


There is also a mixed strategy: to use run-time checks in the callee for FP/DG,
and something better for the other situations (FP/DG are present in D2
programs, but they aren't used everywhere).

Bye,
bearophile



 Sep 12 2010



  parent  Norbert Nemec <Norbert Nemec-online.de>  writes:


On 12/09/10 14:48, bearophile wrote:

 Norbert Nemec:

 Thank you for your clear explanations of the situation.


 With FP/DG, this breaks
 down and I believe the best one can do is to implement contracts as
 run-time checks in the callee, just as it is done in D.


 There is also a mixed strategy: to use run-time checks in the callee for
FP/DG, and something better for the other situations (FP/DG are present in D2
programs, but they aren't used everywhere).


Indeed - this would mean a bare, unchecked interface for each function 
and a wrapper that adds contract checks. If the calling code can verify 
the contract at compile time, it may call the unchecked version. 
Otherwise (like with FP/DG), it will call the checked wrapper.

In fact, this could be understood like a kind of type-casting: A 
function interface includes the contract as part of the type 
information. If the contract is "casted" away (by assigning the function 
to a FP that does not include a contract) the the FP points to the 
run-time checked version of the routine.

This concept may even open the road towards FP/DG that include contract 
information without getting in the way of lightweight, contract-free 
FP/DGs as we have them now.

Greetings,
Norbert



 Sep 12 2010



prev sibling  parent  "Danny Wilson" <danny decube.net>  writes:


Op Mon, 28 Jun 2010 10:33:24 +0200 schreef Norbert Nemec  
<Norbert nemec-online.de>:


 Conceptually, the ultimate solution would certainly be to place code for  
 input contract checking in the *calling* code. After all, this checking  
 code serves to debug the calling code, so it should be left to the  
 caller to decide whether checking is necessary.


I like this idea.


 This approach would also allow the compiler to optimize out some checks  
 when their correctness can be tested at compile time.

 Output contract checks, on the other hand should be compiled inside the  
 returning routine.

 After all, it is all a matter of trust. A language designer should trust  
 the language user to know what he is doing. A library designer should  
 trust the library user to act responsibly. After all - if the  
 application breaks it is the application designer who has to answer for  
 it.




Alot. :-)



 Jun 28 2010



prev sibling  parent reply Norbert Nemec <Norbert Nemec-online.de>  writes:


On 19/06/10 22:46, Andrei Alexandrescu wrote:

 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


 Walter and I discussed this and concluded that Phobos should handle its
 parameters as user input. Therefore they need to be scrubbed with hard
 tests, not contracts.


IMHO, this is plain wrong!

By this kind of decision, you are putting the library user under 
tutelage. The D language is explicitly designed to allow the user to 
take off the safety belt, but to do so at their own responsibility. I 
typically get very annoyed out of principle if I explicitly and 
consciously switch *off* a safety feature and someone decides that they 
do not trust that I am mature enough for that decision and leave part of 
the safety checks in place.

The interface between parts of a program is exactly what contracts are 
designed for. Calling a function with incorrect arguments is a bug in 
the calling code. It should be caught at the interface by a contract 
violation. If the library user trusts their program enough to switch off 
contract checking, the library designer should not worry about 
double-checking for incorrect function calls.

Contracts are part of the public interface, so whatever is specified 
there is automatically documented for the world to see. Replacing 
contracts by "enforce" statements inside the library functions means 
that you have to document in prose what kind of input arguments are allowed.

There are exceptional cases when exceptions may be thrown for incorrect 
arguments: typically, whenever testing for input correctness would be 
too costly, e.g. for badly conditioned matrices in linear algebra code. 
Here, the problem typically shows up during the calculation when it is 
too late to issue a contract violation.

Otherwise, exceptions in library code should only happen in well-defined 
cases for run-time conditions (like I/O errors).

Sorry about by blunt words, but I feel that behind this issue there 
still is a rather fundamental misunderstanding of the concepts of 
contract programming.



 Jun 28 2010



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 06/28/2010 03:15 AM, Norbert Nemec wrote:

 On 19/06/10 22:46, Andrei Alexandrescu wrote:

 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


 Walter and I discussed this and concluded that Phobos should handle its
 parameters as user input. Therefore they need to be scrubbed with hard
 tests, not contracts.


 IMHO, this is plain wrong!

 By this kind of decision, you are putting the library user under
 tutelage. The D language is explicitly designed to allow the user to
 take off the safety belt, but to do so at their own responsibility.


C APIs also check their arguments.

Andrei



 Jun 28 2010



 next sibling parent reply bearophile <bearophileHUGS lycos.com>  writes:


Andrei Alexandrescu:

 C APIs also check their arguments.


Try again, C doesn't have DbC :-) Norbert Nemec says some good things.

Bye,
bearophile



 Jun 28 2010



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


bearophile wrote:

 Andrei Alexandrescu:

 C APIs also check their arguments.


 
 Try again, C doesn't have DbC :-)


What I meant to say was that even the standard library of a language 
famous for its to-the-metal performance still checks parameters 
compulsively whenever it can. Search e.g. this page:

http://google.com/codesearch/p?hl=en#XAzRy8oK4zA/libc/stdio/fseek.c&q=fseek&sa=N&cd=2&ct=rc

for "whence".


 Norbert Nemec says some good things.


I think it's an area where reasonable people may disagree.


Andrei



 Jun 28 2010



  parent  Norbert Nemec <Norbert Nemec-online.de>  writes:


On 28/06/10 19:30, Andrei Alexandrescu wrote:

 bearophile wrote:

 Andrei Alexandrescu:

 C APIs also check their arguments.


 Try again, C doesn't have DbC :-)


 What I meant to say was that even the standard library of a language
 famous for its to-the-metal performance still checks parameters
 compulsively whenever it can. [...]


Indeed, checking input arguments is essential. DbC simply means 
formalizing what has been done in any good libary for ages.

My only intention was to make clear that checking input arguments is 
exactly what contracts are designed for. If the Phobos designers are 
worried that their input checks might be deactivated prematurely, we 
should fix the policy for (de/)activating the input contract checks 
rather than avoiding the use of input contracts they way they are 
intended to be used.

"enforce" simply is not the right tool for this purpose: When a contract 
or and assertion violation is found, it is clear that there is a bug in 
the code. Furthermore, it is even clear which portion of the code is 
responsible for the bug. An assertion violation inside a library is a 
bug in this library. This bug may simply be a missing input contract, 
but this still is a bug in the library.

An "enforce" violation in the library may be anything. The user has to 
dig into the library code to find out whether it is a library bug or a 
piece of incorrect input.

Raising a well-defined exception gives enough information about the 
problem. An anonymous "enforce" on the other hand is a quick-and-dirty 
solution that does not help the developer very much to identify the real 
problem.



 Norbert Nemec says some good things.


 I think it's an area where reasonable people may disagree.


Thanks bearophile, thanks Andrei as well -- I really appreciate this 
open exchange of ideas. Feel free to shoot back as directly as I 
attacked... :-)



 Jun 30 2010



prev sibling next sibling parent reply Michel Fortin <michel.fortin michelf.com>  writes:


On 2010-06-28 07:17:53 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:


 On 06/28/2010 03:15 AM, Norbert Nemec wrote:

 On 19/06/10 22:46, Andrei Alexandrescu wrote:

 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


 
 Walter and I discussed this and concluded that Phobos should handle its
 parameters as user input. Therefore they need to be scrubbed with hard
 tests, not contracts.


 
 IMHO, this is plain wrong!
 
 By this kind of decision, you are putting the library user under
 tutelage. The D language is explicitly designed to allow the user to
 take off the safety belt, but to do so at their own responsibility.


 
 C APIs also check their arguments.


With C you don't have the option to turn the checks on or off. It's 
generally better to have them when you don't need them than not have 
them when you need them. With D, you can turn them on or off on a whim.

If the 'in' contract was enforced at the call site instead of inside 
the function, it'd be up to the one using a function to decide whether 
to check contracts or not. That's not an option in C, but it could work 
like that in D...

I agree though that with the way contracts are currently implemented 
this doesn't work very well. You have to recompile the library with 
contracts on, which in turn forces all the internal contracts inside 
the library to be evaluated. All this because you're trying to validate 
inputs you give to that library? Doesn't make sense. In that context I 
agree that checking explicitly at the library boundaries might be a 
more viable option (like in C).

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/



 Jun 28 2010



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Michel Fortin wrote:

 On 2010-06-28 07:17:53 -0400, Andrei Alexandrescu 
 <SeeWebsiteForEmail erdani.org> said:
 

 On 06/28/2010 03:15 AM, Norbert Nemec wrote:

 On 19/06/10 22:46, Andrei Alexandrescu wrote:

 On 06/19/2010 03:55 PM, bearophile wrote:

 Inside Phobos2 I have counted about 160 usages of the "body" keyword.
 I think contract programming can be used more often inside Phobos2
 (and maybe some usages of enforce() can be turned into contract
 programming because they are more similar to program sanity checks).


 Walter and I discussed this and concluded that Phobos should handle its
 parameters as user input. Therefore they need to be scrubbed with hard
 tests, not contracts.


 IMHO, this is plain wrong!

 By this kind of decision, you are putting the library user under
 tutelage. The D language is explicitly designed to allow the user to
 take off the safety belt, but to do so at their own responsibility.


 C APIs also check their arguments.


 
 With C you don't have the option to turn the checks on or off.


#define NDEBUG


Andrei



 Jun 28 2010



prev sibling  parent reply Sean Kelly <sean invisibleduck.org>  writes:


Andrei Alexandrescu Wrote:

 
 C APIs also check their arguments.


Not the standard C library, as far as I know.  Of course, it's also gotten a
lot of flak for this.



 Jun 28 2010



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


Sean Kelly wrote:

 Andrei Alexandrescu Wrote:

 C APIs also check their arguments.


 
 Not the standard C library, as far as I know.  Of course, it's also gotten a
lot of flak for this.


Nonono. They check whenever they can. Oftentimes they're unable to check.

Example: fseek checks its whence parameter (mentioned in my previous 
post) but itoa cannot check that the target is a valid memory buffer of 
the appropriate length.


Andrei



 Jun 28 2010