• Martin Nowak (6/6) Nov 17 2014 Walter is about to fix an old bug [1] so that invariants are now called
• Steven Schveighoffer (5/9) Nov 17 2014 I'd say no. There are better ways to test for valid data in the .init
• Walter Bright (4/12) Nov 17 2014 Not only that, the runtime check would occur every time an object is cre...
• Steven Schveighoffer (4/20) Nov 17 2014 It's already not worth the expense for Objects, which Object.invariant
• Steven Schveighoffer (3/4) Nov 17 2014 which *calls* Object.invariant for every function call
• Martin Nowak (4/8) Nov 17 2014 If it's init data it is know at compile time right?
• deadalnix (5/11) Nov 17 2014 We don't have default constructor. So we need to be able to rely
• Martin Nowak (2/13) Nov 17 2014 We do have them for classes.
• Rainer Schuetze (11/17) Nov 17 2014 I remember having an invariant on a tree structure checking consistency
• Daniel Murphy (3/13) Nov 19 2014 I think the 'correct' solution is to check the invariants before any of ...
• Steven Schveighoffer (9/26) Nov 19 2014 This may sound feasible, but it rules out some GC algorithms, as the GC
• Rainer Schuetze (9/36) Nov 19 2014 That would be quite within reach for classes with the current GC if the
• Jacob Carlborg (6/9) Nov 19 2014 Tango for D1 had that. A new method, "dispose", on Object was added.
• Walter Bright (3/13) Dec 20 2014 Invariants should be checking the state of the object that it owns, not ...
• Steven Schveighoffer (6/25) Dec 20 2014 Wouldn't a tree own its nodes? I find the idea of a tree checking its
• Walter Bright (10/15) Dec 20 2014 I've replied to this repeatedly. Think of a symbol table tree, in which ...
• Steven Schveighoffer (32/48) Dec 21 2014 But doesn't the AST allocate it's nodes, with a reference to the symbol?...
• Steven Schveighoffer (3/3) Dec 21 2014 On 12/21/14 8:49 AM, Steven Schveighoffer wrote:
• Steven Schveighoffer (32/48) Dec 21 2014 But doesn't the AST allocate it's nodes, with a reference to the symbol?...
• Daniel Murphy (24/26) Dec 21 2014 What? No.
• Walter Bright (3/8) Dec 21 2014 It all depends on how invariant is defined. It's defined as an invariant...
• Daniel Murphy (4/6) Dec 22 2014 Whether or not it owns the data it references is application specific.

Martin Nowak <code+news.digitalmars dawg.eu> writes:

Walter is about to fix an old bug [1] so that invariants are now called 
before destruction and for non-default construction.

A remaining question is whether invariants should also be called for 
default construction [2].

[1]: https://github.com/D-Programming-Language/dmd/pull/4136
[2]: https://issues.dlang.org/show_bug.cgi?id=519#c11

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 11/17/14 3:02 PM, Martin Nowak wrote:
 Walter is about to fix an old bug [1] so that invariants are now called
 before destruction and for non-default construction.

This sounds good.

 A remaining question is whether invariants should also be called for
 default construction [2].

I'd say no. There are better ways to test for valid data in the .init 
version of the object/struct that don't involve a runtime check.

-Steve

Walter Bright <newshound2 digitalmars.com> writes:

On 11/17/2014 12:32 PM, Steven Schveighoffer wrote:
 On 11/17/14 3:02 PM, Martin Nowak wrote:
 Walter is about to fix an old bug [1] so that invariants are now called
 before destruction and for non-default construction.

 This sounds good.

 A remaining question is whether invariants should also be called for
 default construction [2].

 I'd say no. There are better ways to test for valid data in the .init version
of
 the object/struct that don't involve a runtime check.

Not only that, the runtime check would occur every time an object is created, 
yet the .init will always be the same. Doing this check would, I fear, cause 
people to disable invariants as not worth the expense.

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 11/17/14 3:51 PM, Walter Bright wrote:
 On 11/17/2014 12:32 PM, Steven Schveighoffer wrote:
 On 11/17/14 3:02 PM, Martin Nowak wrote:
 Walter is about to fix an old bug [1] so that invariants are now called
 before destruction and for non-default construction.

 This sounds good.

 A remaining question is whether invariants should also be called for
 default construction [2].

 I'd say no. There are better ways to test for valid data in the .init
 version of
 the object/struct that don't involve a runtime check.

 Not only that, the runtime check would occur every time an object is
 created, yet the .init will always be the same. Doing this check would,
 I fear, cause people to disable invariants as not worth the expense.

It's already not worth the expense for Objects, which Object.invariant 
for every function call, even if you don't define one.

-Steve

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 11/17/14 4:09 PM, Steven Schveighoffer wrote:
 ... which Object.invariant for every function call ...

which *calls* Object.invariant for every function call

-Steve

"Martin Nowak" <code dawg.eu> writes:

On Monday, 17 November 2014 at 20:51:36 UTC, Walter Bright wrote:
 Not only that, the runtime check would occur every time an 
 object is created, yet the .init will always be the same. Doing 
 this check would, I fear, cause people to disable invariants as 
 not worth the expense.

If it's init data it is know at compile time right?
Couldn't we try to run the invariant for the init state once in 
CTFE?

"deadalnix" <deadalnix gmail.com> writes:

On Monday, 17 November 2014 at 20:02:36 UTC, Martin Nowak wrote:
 Walter is about to fix an old bug [1] so that invariants are 
 now called before destruction and for non-default construction.

 A remaining question is whether invariants should also be 
 called for default construction [2].

 [1]: https://github.com/D-Programming-Language/dmd/pull/4136
 [2]: https://issues.dlang.org/show_bug.cgi?id=519#c11

We don't have default constructor. So we need to be able to rely
on a struct being in a invalid state at default construction.

Solution obviously being to have default constructor and to run
the invariant after default construction.

"Martin Nowak" <code dawg.eu> writes:

On Tuesday, 18 November 2014 at 00:51:03 UTC, deadalnix wrote:
 On Monday, 17 November 2014 at 20:02:36 UTC, Martin Nowak wrote:
 Walter is about to fix an old bug [1] so that invariants are 
 now called before destruction and for non-default construction.

 A remaining question is whether invariants should also be 
 called for default construction [2].

 [1]: https://github.com/D-Programming-Language/dmd/pull/4136
 [2]: https://issues.dlang.org/show_bug.cgi?id=519#c11

 Solution obviously being to have default constructor and to run
 the invariant after default construction.

We do have them for classes.

Rainer Schuetze <r.sagitario gmx.de> writes:

On 17.11.2014 21:02, Martin Nowak wrote:
 Walter is about to fix an old bug [1] so that invariants are now called
 before destruction and for non-default construction.

 A remaining question is whether invariants should also be called for
 default construction [2].

 [1]: https://github.com/D-Programming-Language/dmd/pull/4136
 [2]: https://issues.dlang.org/show_bug.cgi?id=519#c11

I remember having an invariant on a tree structure checking consistency 
by verifying the children and parent references. This crashed when 
adding a destructor. With the proposed change it will always crash.

The problem is that the destructors of the tree nodes are called in 
arbitrary order when they are collected by the GC. Class instances are 
also made invalid after calling the destructor (the vtbl is zeroed).

I wonder if

- such invariants are invalid,
- the GC should bypass the invariant when calling the destructor
- or we should never call the invariant with the destructor?

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

"Rainer Schuetze"  wrote in message news:m4eu6v$trq$1 digitalmars.com...

 I remember having an invariant on a tree structure checking consistency by 
 verifying the children and parent references. This crashed when adding a 
 destructor. With the proposed change it will always crash.

 The problem is that the destructors of the tree nodes are called in 
 arbitrary order when they are collected by the GC. Class instances are 
 also made invalid after calling the destructor (the vtbl is zeroed).

 I wonder if

 - such invariants are invalid,
 - the GC should bypass the invariant when calling the destructor
 - or we should never call the invariant with the destructor?

I think the 'correct' solution is to check the invariants before any of the 
parts are destroyed. 

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 11/19/14 3:46 AM, Daniel Murphy wrote:
 "Rainer Schuetze"  wrote in message news:m4eu6v$trq$1 digitalmars.com...

 I remember having an invariant on a tree structure checking
 consistency by verifying the children and parent references. This
 crashed when adding a destructor. With the proposed change it will
 always crash.

 The problem is that the destructors of the tree nodes are called in
 arbitrary order when they are collected by the GC. Class instances are
 also made invalid after calling the destructor (the vtbl is zeroed).

 I wonder if

 - such invariants are invalid,
 - the GC should bypass the invariant when calling the destructor
 - or we should never call the invariant with the destructor?

 I think the 'correct' solution is to check the invariants before any of
 the parts are destroyed.

This may sound feasible, but it rules out some GC algorithms, as the GC 
may free things and destroy them in pieces, and not worry about the 
reachability. I don't think we should provide any guarantees about 
destructors being able to access GC data.

We really *REALLY* need a mechanism to determine whether destruction is 
happening in a GC or not. In synchronous destruction (i.e. destroy or 
scoped), you can run the invariant. In GC, you cannot.

-Steve

Rainer Schuetze <r.sagitario gmx.de> writes:

On 19.11.2014 21:08, Steven Schveighoffer wrote:
 On 11/19/14 3:46 AM, Daniel Murphy wrote:
 "Rainer Schuetze"  wrote in message news:m4eu6v$trq$1 digitalmars.com...

 I remember having an invariant on a tree structure checking
 consistency by verifying the children and parent references. This
 crashed when adding a destructor. With the proposed change it will
 always crash.

 The problem is that the destructors of the tree nodes are called in
 arbitrary order when they are collected by the GC. Class instances are
 also made invalid after calling the destructor (the vtbl is zeroed).

 I wonder if

 - such invariants are invalid,
 - the GC should bypass the invariant when calling the destructor
 - or we should never call the invariant with the destructor?

 I think the 'correct' solution is to check the invariants before any of
 the parts are destroyed.

 This may sound feasible, but it rules out some GC algorithms, as the GC
 may free things and destroy them in pieces, and not worry about the
 reachability. I don't think we should provide any guarantees about
 destructors being able to access GC data.

 We really *REALLY* need a mechanism to determine whether destruction is
 happening in a GC or not. In synchronous destruction (i.e. destroy or
 scoped), you can run the invariant. In GC, you cannot.

That would be quite within reach for classes with the current GC if the 
compiler would not generate the invariant into the destructor at all, 
but leave it up to to _d_delclass to call it. The GC uses rt_finalize2 
to destroy collected objects and skip the invariant.

A similar mechanism is possible for structs, but
- the invariant needs to be added to the type info
- the compiler needs to generate code to call both the invariant and the 
destructor, or delegate this to a callback in the runtime

Jacob Carlborg <doob me.com> writes:

On 2014-11-19 21:08, Steven Schveighoffer wrote:

 We really *REALLY* need a mechanism to determine whether destruction is
 happening in a GC or not. In synchronous destruction (i.e. destroy or
 scoped), you can run the invariant. In GC, you cannot.

Tango for D1 had that. A new method, "dispose", on Object was added. 
This was called when an object was deleted via "delete" or when it was 
declared as "scope" and went out of scope.

-- 
/Jacob Carlborg

Walter Bright <newshound2 digitalmars.com> writes:

On 11/17/2014 11:58 PM, Rainer Schuetze wrote:
 I remember having an invariant on a tree structure checking consistency by
 verifying the children and parent references. This crashed when adding a
 destructor. With the proposed change it will always crash.

 The problem is that the destructors of the tree nodes are called in arbitrary
 order when they are collected by the GC. Class instances are also made invalid
 after calling the destructor (the vtbl is zeroed).

 I wonder if

 - such invariants are invalid,
 - the GC should bypass the invariant when calling the destructor
 - or we should never call the invariant with the destructor?

Invariants should be checking the state of the object that it owns, not other 
objects. I would consider such an invariant invalid.

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 12/20/14 7:16 PM, Walter Bright wrote:
 On 11/17/2014 11:58 PM, Rainer Schuetze wrote:
 I remember having an invariant on a tree structure checking
 consistency by
 verifying the children and parent references. This crashed when adding a
 destructor. With the proposed change it will always crash.

 The problem is that the destructors of the tree nodes are called in
 arbitrary
 order when they are collected by the GC. Class instances are also made
 invalid
 after calling the destructor (the vtbl is zeroed).

 I wonder if

 - such invariants are invalid,
 - the GC should bypass the invariant when calling the destructor
 - or we should never call the invariant with the destructor?

 Invariants should be checking the state of the object that it owns, not
 other objects. I would consider such an invariant invalid.

Wouldn't a tree own its nodes? I find the idea of a tree checking its 
nodes to ensure it's properly sorted (or maybe properly balanced) cannot 
possibly be done without actually looking at its nodes.

How do you propose one would check that invariant?

-Steve

Walter Bright <newshound2 digitalmars.com> writes:

On 12/20/2014 7:11 PM, Steven Schveighoffer wrote:
 Wouldn't a tree own its nodes?

I've replied to this repeatedly. Think of a symbol table tree, in which symbols 
are looked up. References to found symbols are then inserted into the AST.

Building a language design that REQUIRES ownership of all references in an 
object would be cripplingly limited.


 I find the idea of a tree checking its nodes to
 ensure it's properly sorted (or maybe properly balanced) cannot possibly be
done
 without actually looking at its nodes.

 How do you propose one would check that invariant?

Not using invariant() to do it. The existence of invariant() with language 
support does not mean that there aren't other ways to do it, or that
invariant() 
must be universally applicable to everything.

For an analogy, constructors don't solve every creation issue - sometimes a 
factory() method is more appropriate.

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 12/20/14 10:24 PM, Walter Bright wrote:
 On 12/20/2014 7:11 PM, Steven Schveighoffer wrote:
 Wouldn't a tree own its nodes?

 I've replied to this repeatedly. Think of a symbol table tree, in which
 symbols are looked up. References to found symbols are then inserted
 into the AST.

But doesn't the AST allocate it's nodes, with a reference to the symbol? 
I'm ignorant of how AST is designed, but when, for instance, you insert 
an element into a RedBlackTree, the RedBlackTree owns the node, but not 
the reference to the element (if it's a reference-style element).

 Building a language design that REQUIRES ownership of all references in
 an object would be cripplingly limited.

This strawman is not what I or anyone else said.

What we are saying is, do not build a language which assumes NO 
ownership of any references.

 I find the idea of a tree checking its nodes to
 ensure it's properly sorted (or maybe properly balanced) cannot
 possibly be done
 without actually looking at its nodes.

 How do you propose one would check that invariant?

 Not using invariant() to do it. The existence of invariant() with
 language support does not mean that there aren't other ways to do it, or
 that invariant() must be universally applicable to everything.

This is kind of silly. What you are proposing is to do this:

void anyfunc()
in
{
    assert(theRealInvariant());
}
out
{
    assert(theRealInvariant());
}
body
{
    ...
}

Basically, you have to re-implement invariant manually on every single 
function, except on the destructor.

Why couldn't you limit invariants to not being called in the destructor, 
but you could call it from the destructor if you wanted? Especially 
since that's the way it was. In fact, we can do that today.

Alternatively, you could pass a parameter to invariant that indicates 
whether it's in the destructor or not.

What you are saying with this change is invariant is strictly for value 
types only.

-Steve

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 12/21/14 8:49 AM, Steven Schveighoffer wrote:

Sorry for the double post, my mailer had issues...

-Steve

Steven Schveighoffer <schveiguy yahoo.com> writes:

On 12/20/14 10:24 PM, Walter Bright wrote:
 On 12/20/2014 7:11 PM, Steven Schveighoffer wrote:
 Wouldn't a tree own its nodes?

 I've replied to this repeatedly. Think of a symbol table tree, in which
 symbols are looked up. References to found symbols are then inserted
 into the AST.

But doesn't the AST allocate it's nodes, with a reference to the symbol? 
I'm ignorant of how AST is designed, but when, for instance, you insert 
an element into a RedBlackTree, the RedBlackTree owns the node, but not 
the reference to the element (if it's a reference-style element).

 Building a language design that REQUIRES ownership of all references in
 an object would be cripplingly limited.

This strawman is not what I or anyone else said.

What we are saying is, do not build a language which assumes NO 
ownership of any references.

 I find the idea of a tree checking its nodes to
 ensure it's properly sorted (or maybe properly balanced) cannot
 possibly be done
 without actually looking at its nodes.

 How do you propose one would check that invariant?

 Not using invariant() to do it. The existence of invariant() with
 language support does not mean that there aren't other ways to do it, or
 that invariant() must be universally applicable to everything.

This is kind of silly. What you are proposing is to do this:

void anyfunc()
in
{
    assert(theRealInvariant());
}
out
{
    assert(theRealInvariant());
}
body
{
    ...
}

Basically, you have to re-implement invariant manually on every single 
function, except on the destructor.

Why couldn't you limit invariants to not being called in the destructor, 
but you could call it from the destructor if you wanted? Especially 
since that's the way it was. In fact, we can do that today.

Alternatively, you could pass a parameter to invariant that indicates 
whether it's in the destructor or not.

What you are saying with this change is invariant is strictly for value 
types only.

-Steve

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

"Walter Bright"  wrote in message news:m753hk$pt2$1 digitalmars.com...

 Invariants should be checking the state of the object that it owns, not 
 other objects. I would consider such an invariant invalid.

What?  No.

This is a perfectly valid use of invariants:

class A
{
    B b;
    invariant()
    {
        assert(!b || b.getA() is this);
    }
}

class B
{
    A a;
    A getA() { return a; }
}

ie checking that the two objects reference each other.  This will fail if B 
is destroyed before A's invariant is run.

I don't see why we'd want anything other than
1. Determine which memory objects are unreferenced
2. Run their invariants
3. Run their destructors

Running the invariants while you're part-way through destroying the object 
graph is just insane. 

Walter Bright <newshound2 digitalmars.com> writes:

On 12/21/2014 11:18 PM, Daniel Murphy wrote:
 "Walter Bright"  wrote in message news:m753hk$pt2$1 digitalmars.com...

 Invariants should be checking the state of the object that it owns, not other
 objects. I would consider such an invariant invalid.

 What?  No.

 This is a perfectly valid use of invariants:

It all depends on how invariant is defined. It's defined as an invariant on
what 
it owns, not whatever is referenced by the object.

"Daniel Murphy" <yebbliesnospam gmail.com> writes:

"Walter Bright"  wrote in message news:m78i71$1c2h$1 digitalmars.com...

 It all depends on how invariant is defined. It's defined as an invariant 
 on what it owns, not whatever is referenced by the object.

Whether or not it owns the data it references is application specific. 
Where are you saying the correct place to put a check like my example, to 
ensure that an owned object correctly references its parent?