www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Struct with default ctor (Was: [dmd-beta] dmd 2.064 beta take 2)

reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 5/17/13, Walter Bright <walter digitalmars.com> wrote:

 I oppose this. D has a lot of nice features because of the .init property.
 Default constructors wreck that.


Would they? I'm thinking the process would be:

struct S
{
    int x;
    int y = void;

    this()  // hypothetical
    {
        // x would already be initialized to int.init here
        assert(x == int.init);

        // y is left uninitialized here
    }
}

Maybe that's already clear. But why is .init actually such a big
problem? If it becomes arbitrarily expensive to call .init of a
struct, well it's because it has to be - if the user really provided
an expensive default ctor. But it's entirely the user's
responsibility. So then .init can even throw, but throwing exceptions
isn't a big deal. Is there some other problem?

A custom default ctor in a struct is one of the most asked for
features. Just yesterday we spent several hours explaining to a C++
user why a default ctor doesn't work, and what .init is for. The whole
conversation could have been avoided if D had support for custom
default ctors for structs. This topic comes up very often in IRC and
the forums.
May 17 2013
parent reply "deadalnix" <deadalnix gmail.com> writes:
On 5/17/13, Walter Bright <walter digitalmars.com> wrote:

 I oppose this. D has a lot of nice features because of the 
 .init property.
 Default constructors wreck that.


So much great feature like :
  - null all over the place.
  - Having NUllable and NonNullable in phobos. BTW, NonNullable is 
unable to ensure that it if effectively non nullable.
  - Having to check for .init state all over the place (which have 
a runtime cost, in addition to be error prone). RefCounted 
implementation is emblematic of that.
May 18 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 12:08 AM, deadalnix wrote:

 On 5/17/13, Walter Bright <walter digitalmars.com> wrote:

 I oppose this. D has a lot of nice features because of the .init property.
 Default constructors wreck that.


 So much great feature like :
   - null all over the place.
   - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to
 ensure that it if effectively non nullable.
   - Having to check for .init state all over the place (which have a runtime
 cost, in addition to be error prone). RefCounted implementation is emblematic
of
 that.


What default would you use for non-null pointers?
May 18 2013
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, May 18, 2013 00:14:30 Walter Bright wrote:

 What default would you use for non-null pointers?


It would basically have to  disable init, because you can't have a default for 
non-nullable pointers. It just wouldn't make sense. Pretty much the whole 
point of them is to guarantee that they've been initialized with a valid 
value.

- Jonathan M Davis
May 18 2013
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 18 May 2013 at 07:14:29 UTC, Walter Bright wrote:

 What default would you use for non-null pointers?


Compile time error as a default sound like a nice option. 
Probably too late now, but that is where most language are going 
now, and having done quite a lot of java myself, I can guarantee 
you that it make sense.
May 18 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 12:30 AM, deadalnix wrote:

 On Saturday, 18 May 2013 at 07:14:29 UTC, Walter Bright wrote:

 What default would you use for non-null pointers?


 Compile time error as a default sound like a nice option. Probably too late
now,
 but that is where most language are going now, and having done quite a lot of
 java myself, I can guarantee you that it make sense.


D has that:

     disable this();

which is not the same thing as allowing default constructors. See:

http://d.puremagic.com/issues/show_bug.cgi?id=10102
May 18 2013
next sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 18 May 2013 at 18:41:49 UTC, Walter Bright wrote:

 On 5/18/2013 12:30 AM, deadalnix wrote:

 On Saturday, 18 May 2013 at 07:14:29 UTC, Walter Bright wrote:

 What default would you use for non-null pointers?


 Compile time error as a default sound like a nice option. 
 Probably too late now,
 but that is where most language are going now, and having done 
 quite a lot of
 java myself, I can guarantee you that it make sense.


 D has that:

     disable this();

 which is not the same thing as allowing default constructors. 
 See:

 http://d.puremagic.com/issues/show_bug.cgi?id=10102


struct S
{
    int a;
     disable this();
    this(int) { a = 1; }
    ~this() { assert(a !is 0); }
}

enum E : S { A = S.init }

union U
{
    S s;
    this(this) { assert (s.a !is 0); }
    ~this() { assert (s.a !is 0); }
}

void foo(out S s, out E e, out U u) { }

void main()
{
    S[] arr;
    arr.length = 5; // compiles
    E[] e;
    e.length = 5; // compiles
    S[1] x = (S[1]).init; // compiles
    U[] u;
    u.length = 5; //compiles
    foo(arr[0], e[0], u[0]); // compiles
}

 disable this(); commitments are cheap.
May 18 2013
parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 1:05 PM, Maxim Fomin wrote:

  disable this(); commitments are cheap.


http://d.puremagic.com/issues/show_bug.cgi?id=10115
May 18 2013
prev sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 18 May 2013 at 18:41:49 UTC, Walter Bright wrote:

 On 5/18/2013 12:30 AM, deadalnix wrote:

 On Saturday, 18 May 2013 at 07:14:29 UTC, Walter Bright wrote:

 What default would you use for non-null pointers?


 Compile time error as a default sound like a nice option. 
 Probably too late now,
 but that is where most language are going now, and having done 
 quite a lot of
 java myself, I can guarantee you that it make sense.


 D has that:

     disable this();

 which is not the same thing as allowing default constructors. 
 See:

 http://d.puremagic.com/issues/show_bug.cgi?id=10102


I played with that. It ensure nothing, convey intent at best.
May 18 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 1:12 PM, deadalnix wrote:

 I played with that. It ensure nothing, convey intent at best.


Please file bug reports for any holes in it.
May 18 2013
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Saturday, 18 May 2013 at 20:19:11 UTC, Walter Bright wrote:

 On 5/18/2013 1:12 PM, deadalnix wrote:

 I played with that. It ensure nothing, convey intent at best.


 Please file bug reports for any holes in it.


Many are, but I think that isn't the point we are discussing here.

Removing all holes in  disable this will require the same 
sacrifices at the ends than default constructor would. For 
isntance, what should happen in this case :

S[] ss;
ss.length = 42;

if S has  disable this ?
May 18 2013
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 1:22 PM, deadalnix wrote:

 Many are, but I think that isn't the point we are discussing here.

 Removing all holes in  disable this will require the same sacrifices at the
ends
 than default constructor would. For isntance, what should happen in this case :

 S[] ss;
 ss.length = 42;

 if S has  disable this ?


Already reported:

http://d.puremagic.com/issues/show_bug.cgi?id=10115
May 18 2013
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/18/2013 1:39 PM, Walter Bright wrote:

 Already reported:

 http://d.puremagic.com/issues/show_bug.cgi?id=10115


And Kenji has already posted a fix!

What can I say, other than Awesome!
May 18 2013
parent "Simen Kjaeraas" <simen.kjaras gmail.com> writes:
On Sun, 19 May 2013 05:17:57 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 5/18/2013 1:39 PM, Walter Bright wrote:

 Already reported:

 http://d.puremagic.com/issues/show_bug.cgi?id=10115


 And Kenji has already posted a fix!

 What can I say, other than Awesome!


Great! One more:

http://d.puremagic.com/issues/show_bug.cgi?id=1528

-- 
Simen
May 20 2013
prev sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Saturday, 18 May 2013 at 20:39:29 UTC, Walter Bright wrote:

 On 5/18/2013 1:22 PM, deadalnix wrote:

 Many are, but I think that isn't the point we are discussing 
 here.

 Removing all holes in  disable this will require the same 
 sacrifices at the ends
 than default constructor would. For isntance, what should 
 happen in this case :

 S[] ss;
 ss.length = 42;

 if S has  disable this ?


 Already reported:

 http://d.puremagic.com/issues/show_bug.cgi?id=10115


New case, will report it:

struct S
{
    int a;
     disable this();
    this(int) { a = 1; }
    ~this() { assert(a !is 0); }
    alias a this;
    int opCall() { return a; }
}

void main()
{
    switch (S.init())
    {
       case 0:
          assert(0); //oops
       default:
    }
}

By the way, here is another bug.

I think there is disagreement about  disable reliability and 
usefulness and similar issues ( safe reliability too) due to 
different attitude to  the problem:
- As a language designer I care about whether some feature is 
claimed to solve some problem - and that all, I put it on a slide 
as lang advantage;
- As a programmer who writes medium importance code I care 
whether the feature stops me from making bugs unintentionally. If 
it does, than I consider that the feature works.
- As a programmer who writes critical code I care whether feature 
prevents from problem, even made deliberately, and if it doesn't, 
than the feature isn't reliable. It doesn't mean that it is 
totally useless, but it does mean that its reliability 
commitments are cheap.

Since in system language there is plenty of ways to deliberately 
pass invalid data to the place where some validity assumptions 
were made,  disable is a broken feature.
May 19 2013
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/19/2013 12:13 PM, Maxim Fomin wrote:

 Since in system language there is plenty of ways to deliberately pass invalid
 data to the place where some validity assumptions were made,  disable is a
 broken feature.


Please report all such holes to bugzilla.
May 19 2013
prev sibling parent reply Kenji Hara <k.hara.pg gmail.com> writes:
Unfortunately this is currently not a bug.
T.init provides "default initialized" object image, and it *does not*
provide "default constructed" object. The difference is important.

That is already documented in lanugage reference.
http://dlang.org/property#init


 Note: .init produces a default initialized object, not default


constructed. That means using .init is sometimes incorrect.

 1. If T is a nested struct, the context pointer in T.init is null.
 2. If T is a struct which has  disable this();, T.init might return a


logically incorrect object.

Kenji Hara

2013/5/20 Maxim Fomin <maxim maxim-fomin.ru>


 On Saturday, 18 May 2013 at 20:39:29 UTC, Walter Bright wrote:


 On 5/18/2013 1:22 PM, deadalnix wrote:


 Many are, but I think that isn't the point we are discussing here.

 Removing all holes in  disable this will require the same sacrifices at
 the ends
 than default constructor would. For isntance, what should happen in this
 case :

 S[] ss;
 ss.length = 42;

 if S has  disable this ?


 Already reported:

 http://d.puremagic.com/issues/**show_bug.cgi?id=10115<http://d.puremagic.com/issues/show_bug.cgi?id=10115>


 New case, will report it:


 struct S
 {
    int a;
     disable this();
    this(int) { a = 1; }
    ~this() { assert(a !is 0); }
    alias a this;
    int opCall() { return a; }
 }

 void main()
 {
    switch (S.init())
    {
       case 0:
          assert(0); //oops
       default:
    }
 }

 By the way, here is another bug.

 I think there is disagreement about  disable reliability and usefulness
 and similar issues ( safe reliability too) due to different attitude to
  the problem:
 - As a language designer I care about whether some feature is claimed to
 solve some problem - and that all, I put it on a slide as lang advantage;
 - As a programmer who writes medium importance code I care whether the
 feature stops me from making bugs unintentionally. If it does, than I
 consider that the feature works.
 - As a programmer who writes critical code I care whether feature prevents
 from problem, even made deliberately, and if it doesn't, than the feature
 isn't reliable. It doesn't mean that it is totally useless, but it does
 mean that its reliability commitments are cheap.

 Since in system language there is plenty of ways to deliberately pass
 invalid data to the place where some validity assumptions were made,
  disable is a broken feature.
May 19 2013
next sibling parent reply "Maxim Fomin" <maxim maxim-fomin.ru> writes:
On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote:

 Unfortunately this is currently not a bug.
 T.init provides "default initialized" object image, and it 
 *does not*
 provide "default constructed" object. The difference is 
 important.

 That is already documented in lanugage reference.
 http://dlang.org/property#init


 Note: .init produces a default initialized object, not default


 constructed. That means using .init is sometimes incorrect.

 1. If T is a nested struct, the context pointer in T.init is 
 null.
 2. If T is a struct which has  disable this();, T.init might 
 return a


 logically incorrect object.

 Kenji Hara


I think this should be fixed otherwise  disable this() is 
compromised. What is rationale behind allowing .init?
May 19 2013
parent reply Kenji Hara <k.hara.pg gmail.com> writes:
I know at least two cases which T.init is commonly used.

1. Inside predicate template for type T.

template isSomething(T) {
    enum isSomething = is(typeof({
        //T t1;    // not good if T is nested struct, or has  disable this()
        //T t2 = void; auto x = t2;  // not good if T is non-mutable type
        T t = T.init;   // avoid default construct check
        ...use t...
    }));
}

2. Some library utilities that treats object state directly, e.g.
std.conv.emplace

Kenji Hara

2013/5/20 Maxim Fomin <maxim maxim-fomin.ru>


 On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote:


 Unfortunately this is currently not a bug.
 T.init provides "default initialized" object image, and it *does not*
 provide "default constructed" object. The difference is important.

 That is already documented in lanugage reference.
 http://dlang.org/property#init

  Note: .init produces a default initialized object, not default


 constructed. That means using .init is sometimes incorrect.


 1. If T is a nested struct, the context pointer in T.init is null.
 2. If T is a struct which has  disable this();, T.init might return a


 logically incorrect object.

 Kenji Hara


 I think this should be fixed otherwise  disable this() is compromised.
 What is rationale behind allowing .init?





 May 19 2013



  parent  "Maxim Fomin" <maxim maxim-fomin.ru>  writes:


On Monday, 20 May 2013 at 06:10:22 UTC, Kenji Hara wrote:

 I know at least two cases which T.init is commonly used.

 1. Inside predicate template for type T.

 template isSomething(T) {
     enum isSomething = is(typeof({
         //T t1;    // not good if T is nested struct, or has 
  disable this()
         //T t2 = void; auto x = t2;  // not good if T is 
 non-mutable type
         T t = T.init;   // avoid default construct check
         ...use t...
     }));
 }

 2. Some library utilities that treats object state directly, 
 e.g.
 std.conv.emplace

 Kenji Hara


I see. But unfortunately this undermines  disable and defeats 
arguments for using it.  disable is another feature (like ref and 
 safe) which cannot be fixed be design.



 May 19 2013



prev sibling  parent  "Bottled Gin" <gin bottled.com>  writes:


On Monday, 20 May 2013 at 00:55:14 UTC, Kenji Hara wrote:

 Unfortunately this is currently not a bug.
 T.init provides "default initialized" object image, and it 
 *does not*
 provide "default constructed" object. The difference is 
 important.

 That is already documented in lanugage reference.
 http://dlang.org/property#init


 Note: .init produces a default initialized object, not default


 constructed. That means using .init is sometimes incorrect.

 1. If T is a nested struct, the context pointer in T.init is 
 null.
 2. If T is a struct which has  disable this();, T.init might 
 return a


 logically incorrect object.

 Kenji Hara


In that case, kindly let me understand why it is not possible to 
allow explicit default constructor for structs given that:

1. The constructor gets called at run time.
2. Is not considered for evaluating S.init (compile time).



 May 20 2013



prev sibling  parent reply "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Sat, 18 May 2013 09:14:30 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 5/18/2013 12:08 AM, deadalnix wrote:

 On 5/17/13, Walter Bright <walter digitalmars.com> wrote:

 I oppose this. D has a lot of nice features because of the .init  
 property.
 Default constructors wreck that.


 So much great feature like :
   - null all over the place.
   - Having NUllable and NonNullable in phobos. BTW, NonNullable is  
 unable to
 ensure that it if effectively non nullable.
   - Having to check for .init state all over the place (which have a  
 runtime
 cost, in addition to be error prone). RefCounted implementation is  
 emblematic of
 that.


 What default would you use for non-null pointers?


Damnit, I thought we'd gotten through to you. non-null pointers have no
default, and it is a compile-time error not to initialize them.

-- 
Simen



 May 18 2013



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/18/13 7:13 AM, Simen Kjaeraas wrote:

 On Sat, 18 May 2013 09:14:30 +0200, Walter Bright
 <newshound2 digitalmars.com> wrote:


 On 5/18/2013 12:08 AM, deadalnix wrote:

 On 5/17/13, Walter Bright <walter digitalmars.com> wrote:

 I oppose this. D has a lot of nice features because of the .init
 property.
 Default constructors wreck that.


 So much great feature like :
 - null all over the place.
 - Having NUllable and NonNullable in phobos. BTW, NonNullable is
 unable to
 ensure that it if effectively non nullable.
 - Having to check for .init state all over the place (which have a
 runtime
 cost, in addition to be error prone). RefCounted implementation is
 emblematic of
 that.


 What default would you use for non-null pointers?


 Damnit, I thought we'd gotten through to you. non-null pointers have no
 default, and it is a compile-time error not to initialize them.


This is what I call the "low potential energy mental block". Very hard 
to get out of.

Andrei



 May 18 2013



 next sibling parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Saturday, 18 May 2013 at 14:22:44 UTC, Andrei Alexandrescu 
wrote:

 This is what I call the "low potential energy mental block". 
 Very hard to get out of.


Can you explain this please ?



 May 18 2013



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/18/13 10:30 AM, deadalnix wrote:

 On Saturday, 18 May 2013 at 14:22:44 UTC, Andrei Alexandrescu wrote:

 This is what I call the "low potential energy mental block". Very hard
 to get out of.


 Can you explain this please ?


Happens to me a lot, and am fighting it for dear life. Occurs when a person:

1. Doesn't understand something

2. Doesn't understand (s)he doesn't understand that something

3. Consequently forms a belief there is no lack of understanding

4. Thus has little motivation to change opinion in the matter and see 
things differently, since there's no need for that (i.e. enters a low 
potential energy that's difficult to get out of)

5. Whenever discussing the matter, amasses further misunderstood pieces 
of evidence due to confirmation bias, therefore furthering the depth of 
the potential hole

Such phenomena are widespread and used most visibly in sitcoms in the 
following classic pattern: two people discuss a matter in which they 
make different assumptions about a context (e.g. one thinks it's about a 
person, the other thinks it's about a bottle). During the discussion, 
both use pieces and tidbits of ambiguous language from the other to 
further consolidate their belief the topic is indeed what they thought 
it would be. The watcher, knowing the context of both characters, 
derives enjoyment figuring how the same conversation may be viewed in 
completely different ways.

Consider the following classics that Walter, myself, or sometimes both 
have been going through:

- virtual machines are a crock
- non-null pointers are meh
- generic is better than OOP
- can't/shouldn't check for arithmetic overflow

Some past ones that we fixed:

- string lambdas are just fine
- can't have many integral types without unsafe casts
- can't fix a < b < c to stay C-compatible and meaningful


Andrei



 May 18 2013



prev sibling  parent reply Andrej Mitrovic <andrej.mitrovich gmail.com>  writes:


On 5/18/13, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 This is what I call the "low potential energy mental block". Very hard
 to get out of.


Well in any case, I'd like to gather all intel on why it's bad to
allow a default ctor so we can put it in the documentation as a
rationale. It will avoid endless discussions.



 May 18 2013



  parent  "Maxim Fomin" <maxim maxim-fomin.ru>  writes:


On Saturday, 18 May 2013 at 14:43:30 UTC, Andrej Mitrovic wrote:

 On 5/18/13, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> 
 wrote:

 This is what I call the "low potential energy mental block". 
 Very hard
 to get out of.


 Well in any case, I'd like to gather all intel on why it's bad 
 to
 allow a default ctor so we can put it in the documentation as a
 rationale. It will avoid endless discussions.


I think when you collect arguments why it is bad to have a 
default ctor, you will found how easily they are beaten by 
contraarguments. Endless discussions comes from community 
disagree with Walter, not because of endless flow of people 
asking same question, getting same answer and quiting discussion 
repeteadly.



 May 18 2013



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2013 4:13 AM, Simen Kjaeraas wrote:

 On Sat, 18 May 2013 09:14:30 +0200, Walter Bright <newshound2 digitalmars.com>
 wrote:


 On 5/18/2013 12:08 AM, deadalnix wrote:

 On 5/17/13, Walter Bright <walter digitalmars.com> wrote:

 I oppose this. D has a lot of nice features because of the .init property.
 Default constructors wreck that.


 So much great feature like :
   - null all over the place.
   - Having NUllable and NonNullable in phobos. BTW, NonNullable is unable to
 ensure that it if effectively non nullable.
   - Having to check for .init state all over the place (which have a runtime
 cost, in addition to be error prone). RefCounted implementation is emblematic
of
 that.


 What default would you use for non-null pointers?


 Damnit, I thought we'd gotten through to you. non-null pointers have no
 default, and it is a compile-time error not to initialize them.


See my reply to deadalnix.



 May 18 2013



  parent reply "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Sat, 18 May 2013 20:42:32 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 5/18/2013 4:13 AM, Simen Kjaeraas wrote:


 On Sat, 18 May 2013 09:14:30 +0200, Walter Bright  
 <newshound2 digitalmars.com>
 wrote:


 What default would you use for non-null pointers?


 Damnit, I thought we'd gotten through to you. non-null pointers have no
 default, and it is a compile-time error not to initialize them.


 See my reply to deadalnix.


On Sat, 18 May 2013 20:41:48 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:


 D has that:

      disable this();

 which is not the same thing as allowing default constructors. See:

 http://d.puremagic.com/issues/show_bug.cgi?id=10102


Oh, absolutely. I'm not at all claiming they are (Nor do I think others  
claim
that). It's just it was so hard to make you see the value of non-nullable
pointers, and I feared you'd regressed.

 disable this is awesome, really. And you're right that it's even better  
than
simple non-nullable pointers. Lastly, it's great that it's getting fixes.  
It's
been one of my favorite non-working features. :p

In a way, I fear that we'll end up like C++, with bare pointers/references
being considered experts-only and 'special use', and everyone will use  
smart
pointers instead.

-- 
Simen



 May 18 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Saturday, 18 May 2013 at 20:59:41 UTC, Simen Kjaeraas wrote:

  disable this is awesome, really. And you're right that it's 
 even better than
 simple non-nullable pointers. Lastly, it's great that it's 
 getting fixes. It's
 been one of my favorite non-working features. :p


The subject here is default constructor, not really nullable 
pointer.  disable this() cost as much as default constructor, but 
provide less. I don't see any objective reason to limit default 
constructor as disabled but not enabled.

I'd even argue that it actually cost more as it introduce yet 
another special case.


 In a way, I fear that we'll end up like C++, with bare 
 pointers/references
 being considered experts-only and 'special use', and everyone 
 will use smart
 pointers instead.


No what will happen it that we will have null all over the place 
with missing check, and no stack trace when it fails, because 
NullPointerError have been decided to be bad.

regular references/pointer are used way more than in C++ because 
you don't need to do the manual memory management that goes with 
it in D.

We will have more bugs and slower programs due to null checks all 
over the place, and unexpressed assumption about what can be null 
and what cannot.



 May 18 2013



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2013 5:04 PM, deadalnix wrote:

 On Saturday, 18 May 2013 at 20:59:41 UTC, Simen Kjaeraas wrote:

  disable this is awesome, really. And you're right that it's even better than
 simple non-nullable pointers. Lastly, it's great that it's getting fixes. It's
 been one of my favorite non-working features. :p


 The subject here is default constructor, not really nullable pointer.  disable
 this() cost as much as default constructor, but provide less. I don't see any
 objective reason to limit default constructor as disabled but not enabled.

 I'd even argue that it actually cost more as it introduce yet another special
case.


     this() { assert(0); }

makes for a runtime check, not a compile time one. Compile time checks are more 
efficient in both programmer time and run time.

Note that C++11 8.4.3 has  disable as well, in the form:

    S() = delete;


 In a way, I fear that we'll end up like C++, with bare pointers/references
 being considered experts-only and 'special use', and everyone will use smart
 pointers instead.


 No what will happen it that we will have null all over the place with missing
 check, and no stack trace when it fails, because NullPointerError have been
 decided to be bad.

 regular references/pointer are used way more than in C++ because you don't need
 to do the manual memory management that goes with it in D.

 We will have more bugs and slower programs due to null checks all over the
 place, and unexpressed assumption about what can be null and what cannot.


With NotNull, the runtime check only happens upon assignment and initialization 
from a pointer type. Dereferencing, copying, etc., of NotNull entails zero 
runtime overhead and no checks.



 May 18 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 00:14:26 UTC, Walter Bright wrote:

     this() { assert(0); }

 makes for a runtime check, not a compile time one. Compile time 
 checks are more efficient in both programmer time and run time.

 Note that C++11 8.4.3 has  disable as well, in the form:

    S() = delete;


D has  disable. If default constructor are allowed, default 
constructor can be disabled, as any other declared symbol.


 With NotNull, the runtime check only happens upon assignment 
 and initialization from a pointer type. Dereferencing, copying, 
 etc., of NotNull entails zero runtime overhead and no checks.


People go for the shortest path. You end up having nullable 
everywhere. Experience have been a countless amount of time in 
bazillion programming languages. But it is probably too late to 
fix this in D anyway.



 May 18 2013



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2013 5:23 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 00:14:26 UTC, Walter Bright wrote:

     this() { assert(0); }

 makes for a runtime check, not a compile time one. Compile time checks are
 more efficient in both programmer time and run time.

 Note that C++11 8.4.3 has  disable as well, in the form:

    S() = delete;


 D has  disable. If default constructor are allowed, default constructor can be
 disabled, as any other declared symbol.


I understand that. But the rationale you gave for having a default constructor 
was to be able to disable default construction.



 With NotNull, the runtime check only happens upon assignment and
 initialization from a pointer type. Dereferencing, copying, etc., of NotNull
 entails zero runtime overhead and no checks.


 People go for the shortest path. You end up having nullable everywhere.
 Experience have been a countless amount of time in bazillion programming
 languages. But it is probably too late to fix this in D anyway.


I know that some languages have a special syntax for non-null pointers. I 
disliked this solution for D because non-nullable pointers are just one
instance 
of creating a type with a limited set of values. Why not go for a general 
solution? Why not create a mechanism where a type can be defined that can only 
consist of prime numbers, for example? Why not ranged integer types?



 May 18 2013



 next sibling parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 05/19/2013 03:20 AM, Walter Bright wrote:

 On 5/18/2013 5:23 PM, deadalnix wrote:

 ...
 People go for the shortest path. You end up having nullable everywhere.
 Experience have been a countless amount of time in bazillion programming
 languages. But it is probably too late to fix this in D anyway.


 I know that some languages have a special syntax for non-null pointers.


The saner ones have special syntax for _nullable_ types.


 I disliked this solution for D because non-nullable pointers are just
 one instance of creating a type with a limited set of values.


I think this is the wrong direction of approaching the problem.

D-style nullable references are an example of adding a value to a type 
whose typing judgements are invalid for that value:

class C{ void foo(){ ... }

void main(){
     typeof(null) a;
     a.foo(); // compile-time error
     C c = a;
     c.foo(); // segmentation fault
}

null is not a C, as it does not provide a C.foo method.


 Why not go for a general solution?


You may want a type that has more than one special value. Why have 
language support for exactly one special value? In any case, why have 
unsound type checking for the sake of special values?


 Why not create a mechanism where a type can be defined that can only consist
of prime numbers, for example?


There are programming languages that allow this.


 Why not ranged integer types?





 May 18 2013



prev sibling  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote:

 I understand that. But the rationale you gave for having a 
 default constructor was to be able to disable default 
 construction.


RAII or construction based on template parameters.


 I know that some languages have a special syntax for non-null 
 pointers. I disliked this solution for D because non-nullable 
 pointers are just one instance of creating a type with a 
 limited set of values. Why not go for a general solution? Why 
 not create a mechanism where a type can be defined that can 
 only consist of prime numbers, for example? Why not ranged 
 integer types?


Actually most languages does the reverse. They have non nullable 
type, and provide Nullable/Option/Maybe/whatever to allow for 
nullable type. It allow to ensure that null don't pops up in 
unexpected places, and that null must be handled when it is an 
option.

The current solution is to rely on faith, and I remember someone 
talking about that at DConf recently.

Now that what other languages does is cleared, let's do some 
consideration on null.

A pointer point on something. For instance, an int* point on an 
integer. null doesn't point on a integer. Non nullable pointer 
aren't a restricted set of values, as, by definition, null isn't 
a value that point to an int. That doesn't stand either.

D already have thing like Nullable in the standard lib. 
Introducing Maybe is also pretty easy. Adding NonNullable in 
addition to Nullable sound like something is not quite right.

The benefit we get from null, ie having a default initialization 
for everything, is moot if  disable this() is introduced, so it 
is questionable at this point how useful it really is except as 
make up over current implementation deficiencies.

Finally it has to be added that null cause holes in  safe in ways 
that are difficult to impossible to solve.



 May 18 2013



 next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com>  writes:


On Sunday, May 19, 2013 05:54:46 deadalnix wrote:

 Actually most languages does the reverse. They have non nullable
 type, and provide Nullable/Option/Maybe/whatever to allow for
 nullable type. It allow to ensure that null don't pops up in
 unexpected places, and that null must be handled when it is an
 option.


Most languages? I'm not aware of even _one_ C-based language that doesn't have 
null as the default for pointer/references types. They may have non-nullable 
reference types in addition to the nullable ones (though that's still rare in 
my experience), but they were all added later. The closest that there is is 
C++'s references, which aren't rebindable and in many ways act more like 
aliases than pointers.

Functional languages do less with nullable, but given D's ancestory, it's not 
at all surprising that nullable is the default, and it would be very bizarre 
if it weren't.

- Jonathan M Davis



 May 18 2013



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2013 9:06 PM, Jonathan M Davis wrote:

 The closest that there is is
 C++'s references, which aren't rebindable and in many ways act more like
 aliases than pointers.


You can trivially create null references in C++:

int* p = NULL;
int& r = *p;



 May 18 2013



  parent reply Jonathan M Davis <jmdavisProg gmx.com>  writes:


On Saturday, May 18, 2013 21:30:57 Walter Bright wrote:

 On 5/18/2013 9:06 PM, Jonathan M Davis wrote:

 The closest that there is is
 C++'s references, which aren't rebindable and in many ways act more like
 aliases than pointers.


 
 You can trivially create null references in C++:
 
 int* p = NULL;
 int& r = *p;


Yes, but they're designed with the idea that they're non-nullable. You can't 
assign NULL to them or check if they're NULL. It's just that it's possible to 
make them NULL by the trick that you just showed. Really, it's an example of 
how C++ references are more like aliases than pointers (much as they're 
pointers underneath the hood). All around, they're a bad example of a non-
nullable pointer even though that's kind of what they're supposed to be in 
principle.

- Jonathan M Davis



 May 18 2013



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2013 9:42 PM, Jonathan M Davis wrote:

 On Saturday, May 18, 2013 21:30:57 Walter Bright wrote:

 On 5/18/2013 9:06 PM, Jonathan M Davis wrote:

 The closest that there is is
 C++'s references, which aren't rebindable and in many ways act more like
 aliases than pointers.


 You can trivially create null references in C++:

 int* p = NULL;
 int& r = *p;


 Yes, but they're designed with the idea that they're non-nullable. You can't
 assign NULL to them or check if they're NULL. It's just that it's possible to
 make them NULL by the trick that you just showed.


I don't even think it's a trick, as it can easily happen unintentionally.


 Really, it's an example of
 how C++ references are more like aliases than pointers (much as they're
 pointers underneath the hood). All around, they're a bad example of a non-
 nullable pointer even though that's kind of what they're supposed to be in
 principle.

 - Jonathan M Davis





 May 18 2013



  parent  Jonathan M Davis <jmdavisProg gmx.com>  writes:


On Saturday, May 18, 2013 22:04:08 Walter Bright wrote:

 On 5/18/2013 9:42 PM, Jonathan M Davis wrote:

 On Saturday, May 18, 2013 21:30:57 Walter Bright wrote:

 On 5/18/2013 9:06 PM, Jonathan M Davis wrote:

 The closest that there is is
 C++'s references, which aren't rebindable and in many ways act more like
 aliases than pointers.


 
 You can trivially create null references in C++:
 
 int* p = NULL;
 int& r = *p;


 
 Yes, but they're designed with the idea that they're non-nullable. You
 can't assign NULL to them or check if they're NULL. It's just that it's
 possible to make them NULL by the trick that you just showed.


 
 I don't even think it's a trick, as it can easily happen unintentionally.


Yes, but it's not something that would be done intentionally, and it's 
something that surprises most people. I expect that the vast majority of C++ 
programmers would think that it's impossible before it was explained to them. 
C++ references are usually sold as being non-nullable, and this is arguably a 
hole in their design.

- Jonathan M Davis



 May 18 2013



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/18/2013 8:54 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote:

 I understand that. But the rationale you gave for having a default constructor
 was to be able to disable default construction.


 RAII or construction based on template parameters.


I know what default constructors are used for in C++. That wasn't what I asked, 
though. I asked for compelling rationale.



 I know that some languages have a special syntax for non-null pointers. I
 disliked this solution for D because non-nullable pointers are just one
 instance of creating a type with a limited set of values. Why not go for a
 general solution? Why not create a mechanism where a type can be defined that
 can only consist of prime numbers, for example? Why not ranged integer types?


 Actually most languages does the reverse. They have non nullable type, and
 provide Nullable/Option/Maybe/whatever to allow for nullable type. It allow to
 ensure that null don't pops up in unexpected places, and that null must be
 handled when it is an option.


Can you list some of those languages?



 The current solution is to rely on faith, and I remember someone talking about
 that at DConf recently.


Rely on what faith?



 Now that what other languages does is cleared, let's do some consideration on
null.

 A pointer point on something. For instance, an int* point on an integer. null
 doesn't point on a integer. Non nullable pointer aren't a restricted set of
 values, as, by definition, null isn't a value that point to an int. That
doesn't
 stand either.


By definition? Pointer semantics are what we choose it to mean.



 D already have thing like Nullable in the standard lib. Introducing Maybe is
 also pretty easy. Adding NonNullable in addition to Nullable sound like
 something is not quite right.


Nullable is something different - it exists to give a 'null' value to things 
that don't have a null representation, like an int.



object reference.

There's no non-null wrapper in Java, either.


 The benefit we get from null, ie having a default initialization for
everything,
 is moot if  disable this() is introduced, so it is questionable at this point
 how useful it really is except as make up over current implementation
deficiencies.


It's not fundamentally different from other schemes to prevent any default 
initialization.



 Finally it has to be added that null cause holes in  safe in ways that are
 difficult to impossible to solve.


I presume you are talking about objects bigger than 64k. It is a problem, and 
we'll have to deal with it, but a rare one.



 May 18 2013



 next sibling parent reply "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Sun, 19 May 2013 06:57:16 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:


 The current solution is to rely on faith, and I remember someone  
 talking about
 that at DConf recently.


 Rely on what faith?


void foo(int* p) {} // p must never be null
void foo(NotNull!(int*) p) {}

One of these is tested at compile time, *and* includes valuable
documentation in the signature. The other is either less performant
or buggy.



 Now that what other languages does is cleared, let's do some  
 consideration on null.

 A pointer point on something. For instance, an int* point on an  
 integer. null
 doesn't point on a integer. Non nullable pointer aren't a restricted  
 set of
 values, as, by definition, null isn't a value that point to an int.  
 That doesn't
 stand either.


 By definition? Pointer semantics are what we choose it to mean.


Of course. But which definition is saner:

   "T* is either a valid pointer to a T, or a value that blows up when used
   in certain ways (but not others)."

or

   "T* is a valid pointer to T."

Of course, the latter also requires something like Maybe!T:

   "Maybe!T is either a valid pointer to a T, or a value on which no
   operations may be performed. In order to gain access to the T, both
   cases have to be handled."

-- 
Simen



 May 19 2013



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 5:02 AM, Simen Kjaeraas wrote:

 By definition? Pointer semantics are what we choose it to mean.


 Of course. But which definition is saner:


For many types, it is extremely useful to have some sort of "invalid" value for 
it. null fills that role nicely for pointers, just as nan does for floating 
point types, and 0xFF does for UTF-8.

There's not anything insane about it. The Nullable type constructor even exists 
in order to provide such an invalid state for types (like int) which normally
do 
not have one.

Yes, I do understand there's a role for pointers which cannot hold the invalid 
value.



 May 19 2013



 next sibling parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 18:23:22 UTC, Walter Bright wrote:

 On 5/19/2013 5:02 AM, Simen Kjaeraas wrote:

 By definition? Pointer semantics are what we choose it to 
 mean.


 Of course. But which definition is saner:


 For many types, it is extremely useful to have some sort of 
 "invalid" value for it. null fills that role nicely for 
 pointers, just as nan does for floating point types, and 0xFF 
 does for UTF-8.


I don't wanted to bring that up because I thought it would 
confuse people, but yes, 0xFF for char is the exact same problem 
and I argue in the same direction : require explicit 
initialization.


 There's not anything insane about it. The Nullable type 
 constructor even exists in order to provide such an invalid 
 state for types (like int) which normally do not have one.


If something can be null, you MUST do something to handle 
specifically the null case. D completely fail to ensure that.

void buzz(Foo f) {
     f.foo(); // Rely in faith. It is invalid and way easier to 
write than the valid code, which is THE recipe for it to spread.
}



 May 19 2013



  parent reply "Minas Mina" <minas_mina1990 hotmail.co.uk>  writes:


On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:

 void buzz(Foo f) {
     f.foo(); // Rely in faith. It is invalid and way easier to 
 write than the valid code, which is THE recipe for it to spread.
 }


Shouldn't this throw a NullPointerSomething?



 May 19 2013



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 12:31 PM, Minas Mina wrote:

 On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:

 void buzz(Foo f) {
     f.foo(); // Rely in faith. It is invalid and way easier to write than the
 valid code, which is THE recipe for it to spread.
 }


 Shouldn't this throw a NullPointerSomething?


It throws a seg fault at runtime. It *is* checked for by the hardware.



 May 19 2013



 next sibling parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 05/19/2013 09:42 PM, Walter Bright wrote:

 On 5/19/2013 12:31 PM, Minas Mina wrote:

 On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:

 void buzz(Foo f) {
     f.foo(); // Rely in faith. It is invalid and way easier to write
 than the
 valid code, which is THE recipe for it to spread.
 }


 Shouldn't this throw a NullPointerSomething?


 It throws a seg fault at runtime. It *is* checked for by the hardware.


Yes, but this code looks like it calls method 'foo', which is probably 
its intention. Hence it is buggy.

D's current answer is the following:

void buzz(Foo f)in{assert(!!f);}body{
     f.foo();
}



 May 19 2013



prev sibling  parent reply "Maxim Fomin" <maxim maxim-fomin.ru>  writes:


On Sunday, 19 May 2013 at 19:42:59 UTC, Walter Bright wrote:

 On 5/19/2013 12:31 PM, Minas Mina wrote:

 On Sunday, 19 May 2013 at 18:30:09 UTC, deadalnix wrote:

 void buzz(Foo f) {
    f.foo(); // Rely in faith. It is invalid and way easier to 
 write than the
 valid code, which is THE recipe for it to spread.
 }


 Shouldn't this throw a NullPointerSomething?


 It throws a seg fault at runtime. It *is* checked for by the 
 hardware.


I think there is difference between catching exception and saving
data which you have typed for some period and letting harware
"check" the exception for you meanwile loosing your work.



 May 19 2013



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 1:03 PM, Maxim Fomin wrote:

 I think there is difference between catching exception and saving
 data which you have typed for some period and letting harware
 "check" the exception for you meanwile loosing your work.


You can catch seg faults. It's easier on Windows, but it's doable on Linux.



 May 19 2013



  parent reply "John Colvin" <john.loughran.colvin gmail.com>  writes:


On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote:

 On 5/19/2013 1:03 PM, Maxim Fomin wrote:

 I think there is difference between catching exception and 
 saving
 data which you have typed for some period and letting harware
 "check" the exception for you meanwile loosing your work.


 You can catch seg faults. It's easier on Windows, but it's 
 doable on Linux.


What's the rational for not doing this by default in D? Wouldn't 
a MemoryAccessError or similar be better than crashing out with 
SIGSEGV ? I have no idea about the consequences of this (other 
than tempting people to catch a segfault when they shouldn't, 
which is pretty much always).



 May 19 2013



 next sibling parent  "deadalnix" <deadalnix gmail.com>  writes:


On Monday, 20 May 2013 at 00:23:59 UTC, John Colvin wrote:

 On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote:

 On 5/19/2013 1:03 PM, Maxim Fomin wrote:

 I think there is difference between catching exception and 
 saving
 data which you have typed for some period and letting harware
 "check" the exception for you meanwile loosing your work.


 You can catch seg faults. It's easier on Windows, but it's 
 doable on Linux.


 What's the rational for not doing this by default in D? 
 Wouldn't a MemoryAccessError or similar be better than crashing 
 out with SIGSEGV ? I have no idea about the consequences of 
 this (other than tempting people to catch a segfault when they 
 shouldn't, which is pretty much always).


https://github.com/D-Programming-Language/druntime/blob/master/src/etc/linux/memoryerror.d

Still you can have weird effect when the code above throw in a 
middle of a C routine or something. As C don't know about D 
exception, it is definitively something to be aware of.



 May 19 2013



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 5:23 PM, John Colvin wrote:

 On Sunday, 19 May 2013 at 20:45:39 UTC, Walter Bright wrote:

 On 5/19/2013 1:03 PM, Maxim Fomin wrote:

 I think there is difference between catching exception and saving
 data which you have typed for some period and letting harware
 "check" the exception for you meanwile loosing your work.


 You can catch seg faults. It's easier on Windows, but it's doable on Linux.


 What's the rational for not doing this by default in D? Wouldn't a
 MemoryAccessError or similar be better than crashing out with SIGSEGV ?


Writing a seg fault handler under Linux has a large number of weird constraints.


 I have
 no idea about the consequences of this (other than tempting people to catch a
 segfault when they shouldn't, which is pretty much always).


At some point, all the scaffolding and workarounds to try to prevent
programmers 
from having to deal with the underlying reality of how the system works is not 
appropriate for a systems programming language.



 May 19 2013



prev sibling  parent reply "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Sun, 19 May 2013 20:23:21 +0200, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 5/19/2013 5:02 AM, Simen Kjaeraas wrote:

 By definition? Pointer semantics are what we choose it to mean.


 Of course. But which definition is saner:


 For many types, it is extremely useful to have some sort of "invalid"  
 value for it. null fills that role nicely for pointers, just as nan does  
 for floating point types, and 0xFF does for UTF-8.

 There's not anything insane about it. The Nullable type constructor even  
 exists in order to provide such an invalid state for types (like int)  
 which normally do not have one.

 Yes, I do understand there's a role for pointers which cannot hold the  
 invalid value.


I contend that not only is there a role for them, but that most pointers
should never be null. Here's two questions that convinced me:

1. How many functions that take a pointer or class reference make sense
    to call with null in that pointer/reference?

2. If pointers were non-nullable by default, how often would you need to
    reach for the nullable one?

I argued in another post that nullable by default is analogous to using a
string instead of an int - any number representable in an int is
representable in a string, *and* the string can represent error states.
But if you only want valid ints, there's no reason to use a string.

-- 
Simen



 May 20 2013



  parent reply "estew" <estewh gmail.com>  writes:


I'm surprised people still have problems with null pointers. I 
for one am glad D has null by default makes life easy coming from 
C++ and Java.

I may have missed something but what happens with the following 
code if I could not have a null pointer?

int*[] pntrs = new int*[10];

Would I need to write something like?

Null!(int*)[] pntrs = new Null!(int*)[10];

Personally, I'd rather have null by default as I find it less 
noisy and I don't need it spelled out in the code, it is implied.


 No it sound like initalizing something to null, then 
 initialize it properly, assume all over the place that it is 
 initialized to something else, and in some rare code path it 
 blows up.




OK, so the D gurus kindly introduce for us NotNull!T, Maybe!T, 
Option!T and SegFault!T (just for me). Now I want to access a 
pointer, write code using it etc. But I need to manually track at 
development time whether it is NotNull!T, Null!T, Maybe!T, 
Option!T or whatever. I cannot just have a pointer anymore, 
knowing it's initialised to null. Now I realise it needs to 
change from NotNull!T to Maybe!T...great yet more refactoring. Ok 
refactoring done (yay sed!) but you know what, now I need to find 
every access to that pointer and check for null. More error prone 
than this:

If you are in doubt (i.e. most multi-threaded apps) then check if 
null, with the added comfort that D has initialised all pointers 
to NULL for you. If still in doubt, don't use pointers.

If you want non-null pointers (please no) then it is all or 
nothing. Allowing some pointers null and others not, via 
Nullable!T or NotNull!T, immediately adds another layer of 
complexity.

I don't want to hear: D pointers cannot be null...well ok, they 
can sometimes, it depends, you'll have to read the code. But 
don't worry, D is very easy to read...

My 1 cent. Disregard if I have totally misunderstood the thread, 
possible as it is very late! :-)

Cheers,
Stewart



 May 20 2013



  parent reply "Dicebot" <m.strashun gmail.com>  writes:


On Monday, 20 May 2013 at 14:49:32 UTC, estew wrote:

 Now I want to access a pointer, write code using it etc. But I 
 need to manually track at development time whether it is 
 NotNull!T, Null!T, Maybe!T, Option!T or whatever. I cannot just 
 have a pointer anymore, knowing it's initialised to null.


Yes and this is awesome. This is correctness enforced by type 
system. Because if you _don't_ track this, you have a possible 
error in your code. Only difference between nullable and 
non-nullable pointers by default is that latter _force_ you to 
write correct code. Which is good.



 May 20 2013



  parent reply "estew" <estewh gmail.com>  writes:


On Monday, 20 May 2013 at 14:57:17 UTC, Dicebot wrote:

 On Monday, 20 May 2013 at 14:49:32 UTC, estew wrote:

 Now I want to access a pointer, write code using it etc. But I 
 need to manually track at development time whether it is 
 NotNull!T, Null!T, Maybe!T, Option!T or whatever. I cannot 
 just have a pointer anymore, knowing it's initialised to null.


 Yes and this is awesome. This is correctness enforced by type 
 system. Because if you _don't_ track this, you have a possible 
 error in your code. Only difference between nullable and 
 non-nullable pointers by default is that latter _force_ you to 
 write correct code. Which is good.


True I grant you that, it was late when I posted :-) I've 
actually come around a bit on this after sleeping on it and 
rereading some of the posts. I am starting to like NotNull!T idea 
but I'm a bit hesitant still with Maybe!T, Option!T.

I cannot remember the last time our team had NULL pointer issues. 
We have 102 devs on four integrated products. Big enough to not 
know context your code might be used in nor the implementation 
details of all libraries. The only pointer troubles we see are 
forget to init to NULL or reset to NULL after freeing resources. 
All devs know raw pointers are initialised to NULL. We are C/C++.

So non-null pointers wouldn't make much difference to us here, 
although it may make the code more readable which is always a 
good thing. D needs nullable pointers though, of some form.

But I'm not convinced it would cost us less to have NotNull!T and 
Nullable!T. I feel it is cheaper to mindlessly write "if(a is 
null) {}" when using pointers than to worry at design time what 
the behaviour of a pointer should be.

Design time is the second most expensive developer time for us. 
The most expensive dev. time is changing a design that turned out 
to be incorrect, or is now outdated for whatever reason. Moving 
pointer behaviour to be a design time issue rather than "knowing 
it could be NULL so check it" could increase the probability of 
redesign bugs creeping in.

Still, I am loving the discussion in this thread it's very 
interesting from both sides.

Stewart



 May 20 2013



  parent  "deadalnix" <deadalnix gmail.com>  writes:


On Tuesday, 21 May 2013 at 01:34:29 UTC, estew wrote:

 But I'm not convinced it would cost us less to have NotNull!T 
 and Nullable!T. I feel it is cheaper to mindlessly write "if(a 
 is null) {}" when using pointers than to worry at design time 
 what the behaviour of a pointer should be.


As a matter of fact, most reference are never null, or are 
assumed never to be null.

For instance, in DMD2.062, e2ir.c line 869 it is assumed that 
irs->sclosure can't be null, when in fact it can and that lead to 
an ICE (and that isn't the first one, which kind of mitigate the 
strength of the arguement that this rarely happens and is easy to 
fix).


 Design time is the second most expensive developer time for us. 
 The most expensive dev. time is changing a design that turned 
 out to be incorrect, or is now outdated for whatever reason. 
 Moving pointer behaviour to be a design time issue rather than 
 "knowing it could be NULL so check it" could increase the 
 probability of redesign bugs creeping in.


You may not know if a reference will be nullable or not when you 
write you code at first. With current model, you start writing 
code as if it can't be null, and then later, when you see you in 
fact need null, you now can have surprise breakage anywhere.

With a Nullable, you'll have code breakage that force you to 
handle the null case. This enforce correctness instead of relying 
on faith.



 May 24 2013



prev sibling next sibling parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 05/19/2013 06:57 AM, Walter Bright wrote:

 D already have thing like Nullable in the standard lib. Introducing
 Maybe is
 also pretty easy. Adding NonNullable in addition to Nullable sound like
 something is not quite right.


 Nullable is something different - it exists to give a 'null' value to
 things that don't have a null representation, like an int.


Type Object ought not to have a "null representation" either.




 non-nullable object reference.

 There's no non-null wrapper in Java, either.


This limits those languages' static type safety as much as D's.



 May 19 2013



prev sibling  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 04:57:15 UTC, Walter Bright wrote:

 On 5/18/2013 8:54 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote:

 I understand that. But the rationale you gave for having a 
 default constructor
 was to be able to disable default construction.


 RAII or construction based on template parameters.


 I know what default constructors are used for in C++. That 
 wasn't what I asked, though. I asked for compelling rationale.


I have bunch of code that goes like :

auto oldVar = var;
scope(exit) var = oldVar;

This is begging for a RAII solution where I pass var as template 
parameter but would require default constructor. This is an 
actual problem I have right now as all save/restore are harder 
and harder to keep in sync for no reason and generate a lot of 
boilerplate.

This is a problem I have right now that default constructor would 
solve, and this isn't the first time I hit that need.



 May 19 2013



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 10:41 AM, deadalnix wrote:

 I have bunch of code that goes like :

 auto oldVar = var;
 scope(exit) var = oldVar;

 This is begging for a RAII solution where I pass var as template parameter but
 would require default constructor. This is an actual problem I have right now
as
 all save/restore are harder and harder to keep in sync for no reason and
 generate a lot of boilerplate.

 This is a problem I have right now that default constructor would solve, and
 this isn't the first time I hit that need.


oldVar isn't being default constructed in your example, nor can I see why you'd 
need a default constructor in order to use RAII for save/restore.



 May 19 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 18:27:08 UTC, Walter Bright wrote:

 oldVar isn't being default constructed in your example, nor can 
 I see why you'd need a default constructor in order to use RAII 
 for save/restore.


I need to save the value at construction and restore at 
destruction. I don't need any runtime parameter at construction.



 May 19 2013



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 11:37 AM, deadalnix wrote:

 On Sunday, 19 May 2013 at 18:27:08 UTC, Walter Bright wrote:

 oldVar isn't being default constructed in your example, nor can I see why
 you'd need a default constructor in order to use RAII for save/restore.


 I need to save the value at construction and restore at destruction. I don't
 need any runtime parameter at construction.


The saved value is initialized with the value to be saved. This is not default 
construction.



 May 19 2013



prev sibling next sibling parent  Jacob Carlborg <doob me.com>  writes:


On 2013-05-19 19:41, deadalnix wrote:


 I have bunch of code that goes like :

 auto oldVar = var;
 scope(exit) var = oldVar;

 This is begging for a RAII solution where I pass var as template
 parameter but would require default constructor. This is an actual
 problem I have right now as all save/restore are harder and harder to
 keep in sync for no reason and generate a lot of boilerplate.

 This is a problem I have right now that default constructor would solve,
 and this isn't the first time I hit that need.


You can do something like this:

void restore (alias value, alias dg) ()
{
     auto tmp = value;

     scope (exit)
         value = tmp;

     dg();
}

int a;

void foo () { a = 4 };

void main ()
{
     a = 3;
     restore!(a, {
         foo();
     });
}

The syntax isn't that pretty but it should work. I wish D had better 
syntax for this, something like:

restore(a) {
     foo();
}

-- 
/Jacob Carlborg



 May 19 2013



prev sibling  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 1:41 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 04:57:15 UTC, Walter Bright wrote:

 On 5/18/2013 8:54 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 01:20:31 UTC, Walter Bright wrote:

 I understand that. But the rationale you gave for having a default
 constructor
 was to be able to disable default construction.


 RAII or construction based on template parameters.


 I know what default constructors are used for in C++. That wasn't what
 I asked, though. I asked for compelling rationale.


 I have bunch of code that goes like :

 auto oldVar = var;
 scope(exit) var = oldVar;

 This is begging for a RAII solution where I pass var as template
 parameter but would require default constructor.


No need for a default constructor. You pass the current value as a 
constructor parameter.

Andrei



 May 19 2013



prev sibling next sibling parent reply Jonathan M Davis <jmdavisProg gmx.com>  writes:


On Sunday, May 19, 2013 02:22:43 Simen Kjaeraas wrote:

 Or... possibly, the current holes in  disable are fixed, and NonNull!T
 becomes the default, because we tell people to always use them, rather
 than flail our arms and behave like idiots. ("regular pointers are
 broken, use NonNull!T" is a pretty good argument if it's true)


I've never understood why so many people feel that nullable pointers are a 
problem. Clearly, many people do, but personally, I've rarely had problems 
with them, and there are plenty of cases where not being to make a pointer 
null would really suck (which is why we're forced to have 
std.typecons.Nullable for non-reference types). I'm not arguing against having 
non-nullable pointers, but I'd probably almost never use them myself, as I 
really don't think that they'd be buying me much. In my experince, problems 
with null pointers are extremely rare and easily caught.

-  Jonathan M Davis



 May 18 2013



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 05/19/2013 02:32 AM, Jonathan M Davis wrote:

 On Sunday, May 19, 2013 02:22:43 Simen Kjaeraas wrote:

 Or... possibly, the current holes in  disable are fixed, and NonNull!T
 becomes the default, because we tell people to always use them, rather
 than flail our arms and behave like idiots. ("regular pointers are
 broken, use NonNull!T" is a pretty good argument if it's true)


 I've never understood why so many people feel that nullable pointers are a
 problem. Clearly, many people do, but personally, I've rarely had problems
 with them, and there are plenty of cases where not being to make a pointer
 null would really suck (which is why we're forced to have
 std.typecons.Nullable for non-reference types). I'm not arguing against having
 non-nullable pointers, but I'd probably almost never use them myself, as I
 really don't think that they'd be buying me much. In my experince, problems
 with null pointers are extremely rare and easily caught.

 -  Jonathan M Davis


I don't write a lot of buggy code for myself either. You are arguing for 
leaving out cheap code documentation. Also, in a sane system you'd be 
forced to use types without a null value in many cases, as potentially 
null references ought not be dereferenced. i.e. potentially null would 
mean they _really_ can be null, because otherwise an actual reference to 
a value type could be used.



 May 18 2013



prev sibling  parent reply "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Sun, 19 May 2013 02:32:49 +0200, Jonathan M Davis <jmdavisProg gmx.com>  
wrote:


 On Sunday, May 19, 2013 02:22:43 Simen Kjaeraas wrote:

 Or... possibly, the current holes in  disable are fixed, and NonNull!T
 becomes the default, because we tell people to always use them, rather
 than flail our arms and behave like idiots. ("regular pointers are
 broken, use NonNull!T" is a pretty good argument if it's true)


 I've never understood why so many people feel that nullable pointers are  
 a
 problem. Clearly, many people do, but personally, I've rarely had  
 problems
 with them, and there are plenty of cases where not being to make a  
 pointer
 null would really suck (which is why we're forced to have
 std.typecons.Nullable for non-reference types). I'm not arguing against  
 having
 non-nullable pointers, but I'd probably almost never use them myself, as  
 I
 really don't think that they'd be buying me much. In my experince,  
 problems
 with null pointers are extremely rare and easily caught.


My experience is the complete opposite - I think maybe 20% of bugs at my  
job
are caused by null references. But as you say, they are very easily fixed.

That said, two things to consider:

How many of the functions you write actually need to accept nullable
pointers/references?

If non-nullable was the default, how often would you explicitly ask for a
nullable pointer/reference?

-- 
Simen



 May 19 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 12:06:01 UTC, Simen Kjaeraas wrote:

 My experience is the complete opposite - I think maybe 20% of 
 bugs at my job
 are caused by null references. But as you say, they are very 
 easily fixed.


Sometime they are super freaking hard. I have a horror story 
debugging Apache Cayenne : http://cayenne.apache.org/ because it 
was throwing NPE on a race condition that would never show up 
once the code is instrumented.

A reference was null for a short moment and then set to 
something. Due to concurrency, in extreme cases it could be seen 
as null where it was assumed everywhere to be set.


 That said, two things to consider:

 How many of the functions you write actually need to accept 
 nullable
 pointers/references?

 If non-nullable was the default, how often would you explicitly 
 ask for a
 nullable pointer/reference?


Not that much and I'd rather be warned when it is the case.



 May 19 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 8:55 AM, deadalnix wrote:

 On Sunday, 19 May 2013 at 12:06:01 UTC, Simen Kjaeraas wrote:

 My experience is the complete opposite - I think maybe 20% of bugs at
 my job
 are caused by null references. But as you say, they are very easily
 fixed.


 Sometime they are super freaking hard. I have a horror story debugging
 Apache Cayenne : http://cayenne.apache.org/ because it was throwing NPE
 on a race condition that would never show up once the code is instrumented.

 A reference was null for a short moment and then set to something. Due
 to concurrency, in extreme cases it could be seen as null where it was
 assumed everywhere to be set.


Sounds like a race problem unrelated to null. With non-null objects the 
race would have manifested itself in a different way, perhaps even more 
pernicious.

Anyhow, this discussion should have finality. We could go on forever 
arguing the usefulness or lack thereof of non-nullable references. They 
didn't catch up in some languages and did in some. My personal opinion 
is "nice to have, but not greatly compelling".

It's reasonable to assume no major language changes will accommodate 
non-null references, so the next best thing would be to focus on a 
library solution. As Walter said, a library solution has the perk of 
allowing other interesting restricted types.


Andrei



 May 19 2013



 next sibling parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote:

 Sounds like a race problem unrelated to null. With non-null 
 objects the race would have manifested itself in a different 
 way, perhaps even more pernicious.


It is both a race condition and a null problem. And having non 
nullable type would have been a compile time error instead of 
days of debugging.



 May 19 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 9:11 AM, deadalnix wrote:

 On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote:

 Sounds like a race problem unrelated to null. With non-null objects
 the race would have manifested itself in a different way, perhaps even
 more pernicious.


 It is both a race condition and a null problem.


No, it's just a race condition.


 And having non nullable
 type would have been a compile time error instead of days of debugging.


No, the race condition would have stayed.


Andrei



 May 19 2013



 next sibling parent reply "Idan Arye" <GenericNPC gmail.com>  writes:


On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:

 On 5/19/13 9:11 AM, deadalnix wrote:

 On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu 
 wrote:

 Sounds like a race problem unrelated to null. With non-null 
 objects
 the race would have manifested itself in a different way, 
 perhaps even
 more pernicious.


 It is both a race condition and a null problem.


 No, it's just a race condition.


 And having non nullable
 type would have been a compile time error instead of days of 
 debugging.


 No, the race condition would have stayed.


 Andrei


I believe this claim requires an explanation:

It's a good practice to initialize references(and all other types 
of variables) as soon as possible - and if possible, right away 
in the declaration. If that reference started as null, it's safe 
to assume it was not possible to initialized it at declaration, 
so it was intentionally initialized with null(if there was no 
initialization Java would scream at you).

Now, let's assume that reference was non-nullable. It is safe to 
assume that this change would not remove the obstacle that 
prevented that reference from being initialized right away in the 
declaration - so you still need to initialize it to something 
else - let's call that something `Nil`. Nil is an object that 
tells you that the reference has not yet been initialized.

So, in the original bug the reference "could be seen
as null where it was assumed everywhere to be set.". But now we 
don't have null - so that piece of code that thought the 
reference is null would now think that it is... what? The 
initialization value? No! Because we didn't switch from 
initializing the reference with null to initializing it with the 
later initialization value - we couldn't do it. Instead, we had 
to use Nil. So now, the reference 'could be seen as Nil where it 
was assumed everywhere to be set'...

Now, if you are lucky and your Nil is the better-kind-of-null 
that is used in dynamic object oriented languages, you'll get a 
nil exception - which is just as good as null exception. But if 
you are not so lucky, and you had to declare Nil as a blank 
object of the type of that reference, you are going to have 
logical bug, which is far worse than exceptions...



 May 19 2013



 next sibling parent reply "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Sun, 19 May 2013 19:12:15 +0200, Idan Arye <GenericNPC gmail.com> wrote:


 It's a good practice to initialize references(and all other types of  
 variables) as soon as possible - and if possible, right away in the  
 declaration. If that reference started as null, it's safe to assume it  
 was not possible to initialized it at declaration, so it was  
 intentionally initialized with null(if there was no initialization Java  
 would scream at you).

 Now, let's assume that reference was non-nullable. It is safe to assume  
 that this change would not remove the obstacle that prevented that  
 reference from being initialized right away in the declaration - so you  
 still need to initialize it to something else - let's call that  
 something `Nil`. Nil is an object that tells you that the reference has  
 not yet been initialized.


Uhm, no. Nononono. No. This is a complete and utter fallacy. What you
have just done is define Nullable!(NonNullable!T). I should not have to
explain too closely why this is a bad thing and should not be done.

-- 
Simen



 May 19 2013



  parent reply "Idan Arye" <GenericNPC gmail.com>  writes:


On Sunday, 19 May 2013 at 17:46:13 UTC, Simen Kjaeraas wrote:

 On Sun, 19 May 2013 19:12:15 +0200, Idan Arye 
 <GenericNPC gmail.com> wrote:


 It's a good practice to initialize references(and all other 
 types of variables) as soon as possible - and if possible, 
 right away in the declaration. If that reference started as 
 null, it's safe to assume it was not possible to initialized 
 it at declaration, so it was intentionally initialized with 
 null(if there was no initialization Java would scream at you).

 Now, let's assume that reference was non-nullable. It is safe 
 to assume that this change would not remove the obstacle that 
 prevented that reference from being initialized right away in 
 the declaration - so you still need to initialize it to 
 something else - let's call that something `Nil`. Nil is an 
 object that tells you that the reference has not yet been 
 initialized.


 Uhm, no. Nononono. No. This is a complete and utter fallacy. 
 What you
 have just done is define Nullable!(NonNullable!T). I should not 
 have to
 explain too closely why this is a bad thing and should not be 
 done.


These are the assumptions I'm working with:
  - We can't use a nullable reference
  - We can't initialize the reference upon declaration to it's 
real value.

The first assumption is required because we want to describe how 
the bug scenario deadalnix brought up would look like if 
references were non-nullable. The second assumption is required 
because if we could initialize the reference upon declaration to 
it's real value, we should have just done it in the first place 
and avoid the whole race hazard.

Now, I'm not saying the solution I presented is good - I'm trying 
to show that given those two assumptions, we are forced to use 
this bad solution.



 May 19 2013



 next sibling parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 18:05:03 UTC, Idan Arye wrote:

 These are the assumptions I'm working with:
  - We can't use a nullable reference
  - We can't initialize the reference upon declaration to it's 
 real value.


If you can't initialize the value, you got to assume when you use 
it that it may not have been initialized and handle that case. 
You need either an Option (where you have to be explicit about 
what you do when the thing is null) or a Maybe (where null is 
ignored and Maybe "contaminate" every result depending on a maybe 
value).


 The first assumption is required because we want to describe 
 how the bug scenario deadalnix brought up would look like if 
 references were non-nullable. The second assumption is required 
 because if we could initialize the reference upon declaration 
 to it's real value, we should have just done it in the first 
 place and avoid the whole race hazard.


But that is the whole point ! The damn thing should have been 
initialized in the first place to avoid the bug. And this should 
have been caught at compile time with any sane type system.

And this is the exact problem with nullable by default : plenty 
of stuff ends up be null is some weird situation that almost 
never occurs when they are assumed not to be and the program 
crashes. NullPointerException now return 4 millions result on 
google, which is probably around once per java developers.


 Now, I'm not saying the solution I presented is good - I'm 
 trying to show that given those two assumptions, we are forced 
 to use this bad solution.


This solution is complex, do not make any sense in a strongly 
typed language and don't even solve the presented case.



 May 19 2013



 next sibling parent reply "Idan Arye" <GenericNPC gmail.com>  writes:


On Sunday, 19 May 2013 at 18:22:16 UTC, deadalnix wrote:

 On Sunday, 19 May 2013 at 18:05:03 UTC, Idan Arye wrote:

 These are the assumptions I'm working with:
 - We can't use a nullable reference
 - We can't initialize the reference upon declaration to it's 
 real value.


 If you can't initialize the value, you got to assume when you 
 use it that it may not have been initialized and handle that 
 case. You need either an Option (where you have to be explicit 
 about what you do when the thing is null) or a Maybe (where 
 null is ignored and Maybe "contaminate" every result depending 
 on a maybe value).


I don't see how Option and Maybe would have helped your bug. The 
problem was that somewhere in the code the reference was 
perceived as null while in fact it wasn't - so now it will be 
perceived as `None`, and you will have the same problem.


 The first assumption is required because we want to describe 
 how the bug scenario deadalnix brought up would look like if 
 references were non-nullable. The second assumption is 
 required because if we could initialize the reference upon 
 declaration to it's real value, we should have just done it in 
 the first place and avoid the whole race hazard.


 But that is the whole point ! The damn thing should have been 
 initialized in the first place to avoid the bug. And this 
 should have been caught at compile time with any sane type 
 system.

 And this is the exact problem with nullable by default : plenty 
 of stuff ends up be null is some weird situation that almost 
 never occurs when they are assumed not to be and the program 
 crashes. NullPointerException now return 4 millions result on 
 google, which is probably around once per java developers.


This is not a problem with "nullable by default" - it is a 
problem with implicit default values. null(or Nil, or None) are 
the only sane default values for reference types - I think you 
would agree that having to construct a new blank object as 
default value for every reference variable would be far worse 
than null...


 Now, I'm not saying the solution I presented is good - I'm 
 trying to show that given those two assumptions, we are forced 
 to use this bad solution.


 This solution is complex, do not make any sense in a strongly 
 typed language and don't even solve the presented case.


It does not solve the bug - it is something you HAVE to do given 
the assumptions. If the reference is not nullable, and you can't 
set it to it's real value until later in the code, then you have 
to initialize it to some temporary value.



 May 19 2013



  parent  "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Sun, 19 May 2013 21:02:11 +0200, Idan Arye <GenericNPC gmail.com> wrote:


 I don't see how Option and Maybe would have helped your bug. The problem  
 was that somewhere in the code the reference was perceived as null while  
 in fact it wasn't


What does that even mean?



 - so now it will be perceived as `None`, and you will have the same  
 problem.


Except that now the code would be forced to handle the None case. In a way,
having nullable by default is like having a stringly typed system:

function foo( s ) {
    return (s + 4) * 2; // Works great when s == "16", falls dead on its
                        // back when s == "goobers".
}

function bar( int i ) {
     return (i + 4) * 2;
}


These functions look very different. That's because they are. One of them
only takes valid parameters, the other takes any old garbage and barfs
when the wrong garbage is given to it.

Of course, if you have a string, and you want to call bar, you need to
convert the string to an int. So you end up with this:

function baz( string s ) {
     return s.parseInt(
         i => bar(i),
         { alert("error"); });
}

Notice how the parseInt function takes two delegates? One of these
(the first) is only called when the string is valid. The other is only
called if the string is invalid. That way, we can be sure that the
failure case is handled.

Exactly the same would be the case for non-nullable pointers - if you
want to convert a nullable pointer to non-nullable, you *have* to
handle the failure case. No two ways about it.

Now, the same example with class references:

int foo(A a) {
    return a.qux(); // Works great when a == new A(), falls dead on its
                    // back when a == null.
}

int bar(NonNull!A a) {
    return a.qux();
}

See how one of these does not blow up in your face (unless you do
something stupid like create a special Nil value that will do exactly
that)? Now, for baz:

int baz(A a) {
     return a.match(
       (NonNull!A a) => bar(a),
       (None) => -1
       );
}



 And this is the exact problem with nullable by default : plenty of  
 stuff ends up be null is some weird situation that almost never occurs  
 when they are assumed not to be and the program crashes.  
 NullPointerException now return 4 millions result on google, which is  
 probably around once per java developers.


 This is not a problem with "nullable by default" - it is a problem with  
 implicit default values. null(or Nil, or None) are the only sane default  
 values for reference types - I think you would agree that having to  
 construct a new blank object as default value for every reference  
 variable would be far worse than null...


If you absolutely cannot initialize the pointer to something sensible,
then use a nullable pointer. But if non-nullable pointers are not
available, or are harder to use than nullable pointer, then people will
use nullable pointers even where non-nullable would have been a much
more fitting choice.




 It does not solve the bug - it is something you HAVE to do given the  
 assumptions. If the reference is not nullable, and you can't set it to  
 it's real value until later in the code, then you have to initialize it  
 to some temporary value.


I don't know enough about the bug to say such things for sure, but I will
say this: If deadalnix solved the bug, he is likely in a much better
position to say anything about what would solve the problem than the rest
of us.

-- 
Simen



 May 20 2013



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 11:22 AM, deadalnix wrote:

 The damn thing should have been initialized in the
 first place to avoid the bug.


Sounds like you have the double-checked locking bug. Using a different value to 
initialize it won't fix it.

If you have a global value accessible from multiple threads, you must use 
synchronization. There is no way around that. If you use some other global
state 
to check for initialization in order to avoid synchronization, you have the 
double checked locking bug. Yes, you do.



 May 19 2013



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 2:02 PM, Walter Bright wrote:

 If you have a global value accessible from multiple threads, you must use
 synchronization. There is no way around that. If you use some other global
state
 to check for initialization in order to avoid synchronization, you have the
 double checked locking bug. Yes, you do.



BTW, a few years ago, I presented my clever solution to the double checked 
locking bug to Scott Meyers. I was very proud of it. Scott handed me my  ss (in 
a nice way), but I got my comeuppance.

It reminded me of the hours I spent in high school determined to show that I 
could trisect an angle with a compass and a straightedge. There was always some 
tiny flaw :-)



 May 19 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 5:11 PM, Walter Bright wrote:

 It reminded me of the hours I spent in high school determined to show
 that I could trisect an angle with a compass and a straightedge. There
 was always some tiny flaw :-)


Yah, sounds familiar. Did you prove the parallel postulate, too?

Andrei



 May 19 2013



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 3:30 PM, Andrei Alexandrescu wrote:

 On 5/19/13 5:11 PM, Walter Bright wrote:

 It reminded me of the hours I spent in high school determined to show
 that I could trisect an angle with a compass and a straightedge. There
 was always some tiny flaw :-)


 Yah, sounds familiar. Did you prove the parallel postulate, too?


No. That one was intuitively obvious!



 May 19 2013



prev sibling  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 21:02:57 UTC, Walter Bright wrote:

 On 5/19/2013 11:22 AM, deadalnix wrote:

 The damn thing should have been initialized in the
 first place to avoid the bug.


 Sounds like you have the double-checked locking bug. Using a 
 different value to initialize it won't fix it.


No it sound like initalizing something to null, then initialize 
it properly, assume all over the place that it is initialized to 
something else, and in some rare code path it blows up.

The fact that this occurs in a multithreaded environment made it 
super hard to debug, but the whole thing was properly 
synchronized.

Don't assume that I do not understand what the problem with 
double check locking is : 
http://d.puremagic.com/issues/show_bug.cgi?id=6607



 May 19 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 5:56 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 21:02:57 UTC, Walter Bright wrote:

 On 5/19/2013 11:22 AM, deadalnix wrote:

 The damn thing should have been initialized in the
 first place to avoid the bug.


 Sounds like you have the double-checked locking bug. Using a different
 value to initialize it won't fix it.


 No it sound like initalizing something to null, then initialize it
 properly, assume all over the place that it is initialized to something
 else, and in some rare code path it blows up.

 The fact that this occurs in a multithreaded environment made it super
 hard to debug, but the whole thing was properly synchronized.


How was there a bug if everything was properly synchronized? You either 
describe the matter with sufficient detail, or acknowledge the 
destruction of your anecdote. This is going nowhere.

Andrei



 May 19 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:

 How was there a bug if everything was properly synchronized? 
 You either describe the matter with sufficient detail, or 
 acknowledge the destruction of your anecdote. This is going 
 nowhere.


I explained over and over. A field is initialized to null, while 
the object lock is owned, and later to its value, while it is 
locked. In the meantime, another thread access the object, owning 
the lock, assuming the field is always initialized.

The exact same problem arise quite often in the single threaded 
world : a reference is set to null, the dev try to be clever when 
initializing it, in a rare case it isn't, and everything blows up 
when it occurs.

It is simply easier to reproduce when things are single threaded, 
and are often quickly debugged in that case.

As explained, the multithreaded environment makes it super hard 
to debug, not the primary cause of the issue. The simply 
consistent in moving the initialization where it was set to null 
in the first place.

It is an instance of the very classic something may be null and 
code use it assuming it is never null.



 May 19 2013



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 4:06 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:

 How was there a bug if everything was properly synchronized? You either
 describe the matter with sufficient detail, or acknowledge the destruction of
 your anecdote. This is going nowhere.


 I explained over and over. A field is initialized to null, while the object
lock
 is owned, and later to its value, while it is locked. In the meantime, another
 thread access the object, owning the lock, assuming the field is always
 initialized.


so, you have:
==========================
Thread A     Thread B

lock
    p = null
unlock
              lock
                 *p = ...
              unlock
lock
    p = new ...
unlock
==========================
Although you are using locks, it still isn't properly synchronized. Changing
the 
p=null to p=someothernonnullvalue will not fix it.



 May 19 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 23:29:53 UTC, Walter Bright wrote:

 On 5/19/2013 4:06 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu 
 wrote:

 How was there a bug if everything was properly synchronized? 
 You either
 describe the matter with sufficient detail, or acknowledge 
 the destruction of
 your anecdote. This is going nowhere.


 I explained over and over. A field is initialized to null, 
 while the object lock
 is owned, and later to its value, while it is locked. In the 
 meantime, another
 thread access the object, owning the lock, assuming the field 
 is always
 initialized.


 so, you have:
 ==========================
 Thread A     Thread B

 lock
    p = null


Here p = null is implicit, this is part of the fun. The 
initialisation is still properly synchronized.


 unlock
              lock
                 *p = ...


It was in java, so more somethign like p.foo(); But yes.


              unlock
 lock
    p = new ...
 unlock
 ==========================
 Although you are using locks, it still isn't properly 
 synchronized. Changing the p=null to p=someothernonnullvalue 
 will not fix it.


No race condition exists in that program. The error lie in 
improper initialization of p in the first place, which should 
never has been null. The example looks dumb as this, you have to 
imagine the pattern hidden in thousands of LOC.

The code is bugguy, but you'll find no undefined threading 
effect? What happen is perfectly defined here and no thread 
access shared data without owning the lock.



 May 19 2013



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 5:28 PM, deadalnix wrote:

 The error lie in improper
 initialization of p in the first place, which should never has been null. The
 example looks dumb as this, you have to imagine the pattern hidden in thousands
 of LOC.


I would find a design that declared a variable in one place, then initialized
it 
in another, while releasing the lock in between as a bad design pattern to
begin 
with. What other default initialized types could be there? What about an int 
default initialized to 0, yet code in another thread expects it to be some
other 
value? I suspect there'd be a lot more bugs in it than just null pointer 
initializations.

It might be time to engineer a new pattern so you don't have to inspect 
thousands of LOC to manually verify correctness.



 May 19 2013



  parent  "deadalnix" <deadalnix gmail.com>  writes:


On Monday, 20 May 2013 at 01:08:16 UTC, Walter Bright wrote:

 On 5/19/2013 5:28 PM, deadalnix wrote:

 The error lie in improper
 initialization of p in the first place, which should never has 
 been null. The
 example looks dumb as this, you have to imagine the pattern 
 hidden in thousands
 of LOC.


 I would find a design that declared a variable in one place, 
 then initialized it in another, while releasing the lock in 
 between as a bad design pattern to begin with.


I cannot agree more. This is what made tracking the cause of the 
bug super hard.


 What other default initialized types could be there? What about 
 an int default initialized to 0, yet code in another thread 
 expects it to be some other value? I suspect there'd be a lot 
 more bugs in it than just null pointer initializations.

 It might be time to engineer a new pattern so you don't have to 
 inspect thousands of LOC to manually verify correctness.


I didn't programed Apache Cayenne, int he first place. But I had 
to patch it anyway.



 May 19 2013



prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 7:06 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:

 How was there a bug if everything was properly synchronized? You
 either describe the matter with sufficient detail, or acknowledge the
 destruction of your anecdote. This is going nowhere.


 I explained over and over. A field is initialized to null, while the
 object lock is owned, and later to its value, while it is locked. In the
 meantime, another thread access the object, owning the lock, assuming
 the field is always initialized.


How does another thread thread accesses the object "owning the lock" 
when the assignment occurs under lock?

How would non-null fix this? Would the object have type Maybe?


Andrei



 May 19 2013



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 05/20/2013 01:39 AM, Andrei Alexandrescu wrote:

 On 5/19/13 7:06 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu wrote:

 How was there a bug if everything was properly synchronized? You
 either describe the matter with sufficient detail, or acknowledge the
 destruction of your anecdote. This is going nowhere.


 I explained over and over. A field is initialized to null, while the
 object lock is owned, and later to its value, while it is locked. In the
 meantime, another thread access the object, owning the lock, assuming
 the field is always initialized.


 How does another thread thread accesses the object "owning the lock"
 when the assignment occurs under lock?


lock{ initialize to null. }
lock{ in the meantime assume correctly initialized. }
lock{ initialize correctly. }

This is nothing new. I think he has been pretty clear about what the 
issue is from the beginning.


 How would non-null fix this? Would the object have type Maybe?


This is one possibility. In this case, the type system would have 
prevented the null dereference.
In the other case, the type system would have caught the invalid 
initialization.



 May 19 2013



prev sibling  parent  "Idan Arye" <GenericNPC gmail.com>  writes:


On Sunday, 19 May 2013 at 23:07:00 UTC, deadalnix wrote:

 On Sunday, 19 May 2013 at 22:32:58 UTC, Andrei Alexandrescu 
 wrote:

 How was there a bug if everything was properly synchronized? 
 You either describe the matter with sufficient detail, or 
 acknowledge the destruction of your anecdote. This is going 
 nowhere.


 I explained over and over. A field is initialized to null, 
 while the object lock is owned, and later to its value, while 
 it is locked. In the meantime, another thread access the 
 object, owning the lock, assuming the field is always 
 initialized.

 The exact same problem arise quite often in the single threaded 
 world : a reference is set to null, the dev try to be clever 
 when initializing it, in a rare case it isn't, and everything 
 blows up when it occurs.

 It is simply easier to reproduce when things are single 
 threaded, and are often quickly debugged in that case.

 As explained, the multithreaded environment makes it super hard 
 to debug, not the primary cause of the issue. The simply 
 consistent in moving the initialization where it was set to 
 null in the first place.

 It is an instance of the very classic something may be null and 
 code use it assuming it is never null.


So this is not a problem of nullableness - rather this is a 
problem of mutability.



 May 19 2013



prev sibling  parent  "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Sun, 19 May 2013 20:05:02 +0200, Idan Arye <GenericNPC gmail.com> wrote:


 These are the assumptions I'm working with:
   - We can't use a nullable reference
   - We can't initialize the reference upon declaration to it's real  
 value.


Indeed, if that's the case, then what you're doing is fairly sensible. But


-- 
Simen



 May 20 2013



prev sibling  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 1:12 PM, Idan Arye wrote:

 On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:

 On 5/19/13 9:11 AM, deadalnix wrote:

 On Sunday, 19 May 2013 at 13:08:53 UTC, Andrei Alexandrescu wrote:

 Sounds like a race problem unrelated to null. With non-null objects
 the race would have manifested itself in a different way, perhaps even
 more pernicious.


 It is both a race condition and a null problem.


 No, it's just a race condition.


 And having non nullable
 type would have been a compile time error instead of days of debugging.


 No, the race condition would have stayed.


 Andrei


 I believe this claim requires an explanation:

 It's a good practice to initialize references(and all other types of
 variables) as soon as possible - and if possible, right away in the
 declaration. If that reference started as null, it's safe to assume it
 was not possible to initialized it at declaration, so it was
 intentionally initialized with null(if there was no initialization Java
 would scream at you).

 Now, let's assume that reference was non-nullable. It is safe to assume
 that this change would not remove the obstacle that prevented that
 reference from being initialized right away in the declaration - so you
 still need to initialize it to something else - let's call that
 something `Nil`. Nil is an object that tells you that the reference has
 not yet been initialized.

 So, in the original bug the reference "could be seen
 as null where it was assumed everywhere to be set.". But now we don't
 have null - so that piece of code that thought the reference is null
 would now think that it is... what? The initialization value? No!
 Because we didn't switch from initializing the reference with null to
 initializing it with the later initialization value - we couldn't do it.
 Instead, we had to use Nil. So now, the reference 'could be seen as Nil
 where it was assumed everywhere to be set'...

 Now, if you are lucky and your Nil is the better-kind-of-null that is
 used in dynamic object oriented languages, you'll get a nil exception -
 which is just as good as null exception. But if you are not so lucky,
 and you had to declare Nil as a blank object of the type of that
 reference, you are going to have logical bug, which is far worse than
 exceptions...


So the race would have manifested it just the same, except under the 
name of Nil instead of null.

Andrei



 May 19 2013



prev sibling  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:

 On 5/19/13 9:11 AM, deadalnix wrote:

 It is both a race condition and a null problem.


 No, it's just a race condition.


 And having non nullable
 type would have been a compile time error instead of days of 
 debugging.


 No, the race condition would have stayed.


That is ridiculous. non nullable would have made the bug non 
existent, and even without race condition the problem would 
exists. a reference is null, it container shared, then set to 
something else. You can put barriers all over the place to make 
that sequentially consistent that it wouldn't change anything and 
the bug would still arise.

You also never provided any convincing solution to the safety 
hole. We can't even add check only on some edges cases as D also 
have values types. The only solution we are left with that is 
really safe is to null check every dereference or give up on 
 safe.

I encourage you to look at this : 
http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare

Most new languages removed nullable by default, or limited its 
uses (scala for instance, allow for null for limited scope).

I once again want to get attention on the fact that GC change 
everything in regard to reference, and that the C++ situation is 
a bad example.

Idan Arye > Nil proposal make no sens in a statically typed 
language. And you'll find no better kind of null. We have all 
tools we need in D to work around null in library.



 May 19 2013



 next sibling parent reply "Idan Arye" <GenericNPC gmail.com>  writes:


On Sunday, 19 May 2013 at 17:35:43 UTC, deadalnix wrote:

 Idan Arye > Nil proposal make no sens in a statically typed 
 language. And you'll find no better kind of null. We have all 
 tools we need in D to work around null in library.


That's not the point. The point is that if you couldn't 
initialize this reference to null, you had to initialize it to 
something else, so you would still get your race condition.



 May 19 2013



  parent  "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 17:51:36 UTC, Idan Arye wrote:

 On Sunday, 19 May 2013 at 17:35:43 UTC, deadalnix wrote:

 Idan Arye > Nil proposal make no sens in a statically typed 
 language. And you'll find no better kind of null. We have all 
 tools we need in D to work around null in library.


 That's not the point. The point is that if you couldn't 
 initialize this reference to null, you had to initialize it to 
 something else, so you would still get your race condition.


Note that it had to be initialized, and my patch to Cayenne was 
simply to reorder that initialization.

The patch itself was ~5lines, and would have impossible to do 
with a nonnullable (except with your Nil stuff, which once again 
don't make any sens in a strongly type language)



 May 19 2013



prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 10:35 AM, deadalnix wrote:

 On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:

 No, the race condition would have stayed.


 That is ridiculous. non nullable would have made the bug non existent, and even
 without race condition the problem would exists. a reference is null, it
 container shared, then set to something else. You can put barriers all over the
 place to make that sequentially consistent that it wouldn't change anything and
 the bug would still arise.


I agree with Andrei that eliminating null is not going to make a race condition 
into a thread safe one. You've got a serious bug that you may succeed in hiding 
by eliminating the null, but it's still there. From your description, I suspect 
the code is suffering from the double-checked locking bug (which can appear in 
very subtle forms).



 You also never provided any convincing solution to the safety hole. We can't
 even add check only on some edges cases as D also have values types. The only
 solution we are left with that is really safe is to null check every
dereference
 or give up on  safe.


Please don't make us guess what exactly you mean by this.



 May 19 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote:

 You also never provided any convincing solution to the safety 
 hole. We can't
 even add check only on some edges cases as D also have values 
 types. The only
 solution we are left with that is really safe is to null check 
 every dereference
 or give up on  safe.


 Please don't make us guess what exactly you mean by this.


This isn't new and I discussed that again and again.

When you dereference null, you hit the first plage, which is 
protected on most systems. But if you access an element with 
sufficient offset you bypass all protections provided by the type 
system and you are back in unsafe world.

And no, putting nullcheck on access of field of sufficient offset 
(as propose dby Andrei) isn't enough because we have value types. 
Consider :

S[BIG_NUMBER]* a;
auto s = &(*a[SLIGHTLY_BELLOW_CHECK_OFFSET]);
s.fieldAccess; // May not have enough offset to trigget null 
check, but still can be usnafe

See bug reports :
http://d.puremagic.com/issues/show_bug.cgi?id=3677
http://d.puremagic.com/issues/show_bug.cgi?id=5176



 May 19 2013



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 3:10 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote:

 You also never provided any convincing solution to the safety hole.
 We can't
 even add check only on some edges cases as D also have values types.
 The only
 solution we are left with that is really safe is to null check every
 dereference
 or give up on  safe.


 Please don't make us guess what exactly you mean by this.


 This isn't new and I discussed that again and again.

 When you dereference null, you hit the first plage, which is protected
 on most systems. But if you access an element with sufficient offset you
 bypass all protections provided by the type system and you are back in
 unsafe world.


Oh, the good old "object of sufficient size". We know how to fix that.


 And no, putting nullcheck on access of field of sufficient offset (as
 propose dby Andrei) isn't enough because we have value types. Consider :

 S[BIG_NUMBER]* a;
 auto s = &(*a[SLIGHTLY_BELLOW_CHECK_OFFSET]);
 s.fieldAccess; // May not have enough offset to trigget null check, but
 still can be usnafe

 See bug reports :
 http://d.puremagic.com/issues/show_bug.cgi?id=3677
 http://d.puremagic.com/issues/show_bug.cgi?id=5176


All of the above are variations on the "sufficiently large object" theme.

Andrei



 May 19 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 19:15:47 UTC, Andrei Alexandrescu wrote:

 Oh, the good old "object of sufficient size". We know how to 
 fix that.


 And no, putting nullcheck on access of field of sufficient 
 offset (as
 propose dby Andrei) isn't enough because we have value types. 
 Consider :

 S[BIG_NUMBER]* a;
 auto s = &(*a[SLIGHTLY_BELLOW_CHECK_OFFSET]);
 s.fieldAccess; // May not have enough offset to trigget null 
 check, but
 still can be usnafe

 See bug reports :
 http://d.puremagic.com/issues/show_bug.cgi?id=3677
 http://d.puremagic.com/issues/show_bug.cgi?id=5176


 All of the above are variations on the "sufficiently large 
 object" theme.

 Andrei


The code above never access a field with a sufficient offset to 
trigger "sufficiently large runtime check". Obviously, in the 
presented code the bug is trivial, but if the dereferences occurs 
across several functions, this is doomed to fail.

The solutions are : prevent any conglomerate of value type to be 
bigger than 4kb (the protection on OSX is 4kb) or put a null 
check on every dereference in  safe code.



 May 19 2013



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 3:41 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 19:15:47 UTC, Andrei Alexandrescu wrote:

 Oh, the good old "object of sufficient size". We know how to fix that.


 And no, putting nullcheck on access of field of sufficient offset (as
 propose dby Andrei) isn't enough because we have value types. Consider :

 S[BIG_NUMBER]* a;
 auto s = &(*a[SLIGHTLY_BELLOW_CHECK_OFFSET]);
 s.fieldAccess; // May not have enough offset to trigget null check, but
 still can be usnafe

 See bug reports :
 http://d.puremagic.com/issues/show_bug.cgi?id=3677
 http://d.puremagic.com/issues/show_bug.cgi?id=5176


 All of the above are variations on the "sufficiently large object" theme.

 Andrei


 The code above never access a field with a sufficient offset to trigger
 "sufficiently large runtime check".


It does, when the pointer to the large static array is dereferenced.

Andrei



 May 19 2013



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 12:10 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote:

 You also never provided any convincing solution to the safety hole. We can't
 even add check only on some edges cases as D also have values types. The only
 solution we are left with that is really safe is to null check every
dereference
 or give up on  safe.


 Please don't make us guess what exactly you mean by this.


 This isn't new and I discussed that again and again.

 When you dereference null, you hit the first plage, which is protected on most
 systems. But if you access an element with sufficient offset you bypass all
 protections provided by the type system and you are back in unsafe world.



And we've replied to this before. But when you say "give up on  safe", that 
implies a far more serious issue, so I want to make sure what you're talking
about.

I agree that we need to deal with the issue. But on a practical note, if we 
solve 99% of the  safe issues, and fail at 1%, that doesn't mean there is no 
value to  safe and we should give up on it.



 May 19 2013



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 4:05 PM, Walter Bright wrote:

 On 5/19/2013 12:10 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 18:46:31 UTC, Walter Bright wrote:

 You also never provided any convincing solution to the safety hole.
 We can't
 even add check only on some edges cases as D also have values types.
 The only
 solution we are left with that is really safe is to null check every
 dereference
 or give up on  safe.


 Please don't make us guess what exactly you mean by this.


 This isn't new and I discussed that again and again.

 When you dereference null, you hit the first plage, which is protected
 on most
 systems. But if you access an element with sufficient offset you
 bypass all
 protections provided by the type system and you are back in unsafe world.



 And we've replied to this before. But when you say "give up on  safe",
 that implies a far more serious issue, so I want to make sure what
 you're talking about.

 I agree that we need to deal with the issue. But on a practical note, if
 we solve 99% of the  safe issues, and fail at 1%, that doesn't mean
 there is no value to  safe and we should give up on it.


Almost safe == almost pregnant.  safe must be 100% safe.

Andrei



 May 19 2013



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 1:35 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:

 On 5/19/13 9:11 AM, deadalnix wrote:

 It is both a race condition and a null problem.


 No, it's just a race condition.


 And having non nullable
 type would have been a compile time error instead of days of debugging.


 No, the race condition would have stayed.


 That is ridiculous.  non nullable would have made the bug non existent,
 and even without race condition the problem would exists. a reference is
 null, it container shared, then set to something else. You can put
 barriers all over the place to make that sequentially consistent that it
 wouldn't change anything and the bug would still arise.


No, your argument is ridiculous. You make a yarn with precious little 
detail that describes for everything everyone knows a textbook race 
condition, essentially ask that you are taking by your word that 
non-null would miraculously solve it, and, to add insult to injury, and 
when we don't buy it, you put the burden of proof on us. This is quite a 
trick, my hat is off to you.


 You also never provided any convincing solution to the safety hole.


What's the safety hole? Objects of large static size?


 We
 can't even add check only on some edges cases as D also have values
 types. The only solution we are left with that is really safe is to null
 check every dereference or give up on  safe.


How about using NonNull. We won't change the language at this point to 
make non-nullable references by default. Even you acknowledged that 
that's not practical. So now you contradict your own affirmation. What 
exactly do you sustain, and what are you asking for?


 I encourage you to look at this :
 http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare


I read it. I don't buy it. Yeah, it's a point, but it's largely 
exaggerated for dramatic purposes.


 Most new languages removed nullable by default, or limited its uses
 (scala for instance, allow for null for limited scope).


So what do you realistically think we should do, seeing that we're 
aiming at stability?


 I once again want to get attention on the fact that GC change everything
 in regard to reference, and that the C++ situation is a bad example.


I don't understand this.


Andrei



 May 19 2013



 next sibling parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote:

 No, your argument is ridiculous. You make a yarn with precious 
 little detail that describes for everything everyone knows a 
 textbook race condition, essentially ask that you are taking by 
 your word that non-null would miraculously solve it, and, to 
 add insult to injury, and when we don't buy it, you put the 
 burden of proof on us. This is quite a trick, my hat is off to 
 you.


I described a very usual null bug : something is set to null, 
then to a specific value. It is assumed not to be null. In a 
specific case it is null and everything explode.

The concurrent context here made it especially hard to debug, but 
isn't the cause of the bug.

Additionally, if you don't have enough information to understand 
what I'm saying, you are perfectly allowed to ask for additional 
details This isn't a shame.


 You also never provided any convincing solution to the safety 
 hole.


 What's the safety hole? Objects of large static size?


Limiting object size isn't going to cut it. Or must be super 
restrictive (the protection is 4kb on some systems).


 We
 can't even add check only on some edges cases as D also have 
 values
 types. The only solution we are left with that is really safe 
 is to null
 check every dereference or give up on  safe.


 How about using NonNull. We won't change the language at this 
 point to make non-nullable references by default. Even you 
 acknowledged that that's not practical. So now you contradict 
 your own affirmation. What exactly do you sustain, and what are 
 you asking for?


1/ NonNull do not work.
2/ It isn't because it is too late to solve a problem that it 
magically isn't a problem anymore.


 Most new languages removed nullable by default, or limited its 
 uses
 (scala for instance, allow for null for limited scope).


 So what do you realistically think we should do, seeing that 
 we're aiming at stability?


Acknowledge it was a mistake and move on. Use the analysis that 
need to be done to track down  disable this issue to warn about 
uninitialized null stuffs.


 I once again want to get attention on the fact that GC change 
 everything
 in regard to reference, and that the C++ situation is a bad 
 example.


 I don't understand this.


I C or C++ you are doomed to manage reference as you need to for 
memory management purpose. In garbage collected languages, you 
ends up with way more unmanaged references, because the GC take 
care of them. Doing so you multiply the surface area where null 
bug can strike.



 May 19 2013



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 3:36 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote:

 No, your argument is ridiculous. You make a yarn with precious little
 detail that describes for everything everyone knows a textbook race
 condition, essentially ask that you are taking by your word that
 non-null would miraculously solve it, and, to add insult to injury,
 and when we don't buy it, you put the burden of proof on us. This is
 quite a trick, my hat is off to you.


 I described a very usual null bug : something is set to null, then to a
 specific value. It is assumed not to be null. In a specific case it is
 null and everything explode.

 The concurrent context here made it especially hard to debug, but isn't
 the cause of the bug.

 Additionally, if you don't have enough information to understand what
 I'm saying, you are perfectly allowed to ask for additional details This
 isn't a shame.


Your argument has been destroyed so no need to ask details about it. 
Replace "null" with "invalid state" and it's the same race in any 
system. Let's move on.


 What's the safety hole? Objects of large static size?


 Limiting object size isn't going to cut it. Or must be super restrictive
 (the protection is 4kb on some systems).


Well you got to do what you got to do. Field accesses for objects larger 
than 4KB would have to be checked in  safe code.


 We
 can't even add check only on some edges cases as D also have values
 types. The only solution we are left with that is really safe is to null
 check every dereference or give up on  safe.


 How about using NonNull. We won't change the language at this point to
 make non-nullable references by default. Even you acknowledged that
 that's not practical. So now you contradict your own affirmation. What
 exactly do you sustain, and what are you asking for?


 1/ NonNull do not work.


You made the argument that although it does work, people will not use it 
because it's not the default. That's not quite "does not work". This 
also ruins your point because if people don't find it worth writing 
NonNull!T instead of T, it means non-null doesn't buy them anything 
worthwhile.


 2/ It isn't because it is too late to solve a problem that it magically
 isn't a problem anymore.


You are blowing it out of proportion. Null references are hardly even on 
the radar of the bug classes I'm encountering in the style of 
programming of the three groups I worked in at Facebook, and also my 
previous employers. People I meet at conferences and consulting gigs 
never mention null references as a real problem, although I very often 
ask about problems. I find it difficult to agree with you just to be nice.


 Most new languages removed nullable by default, or limited its uses
 (scala for instance, allow for null for limited scope).


 So what do you realistically think we should do, seeing that we're
 aiming at stability?


 Acknowledge it was a mistake and move on.


I'd give it more thought if we designed D from scratch. I think it's 
safe to move on. At any rate, I'd love if we got NonNull working nicely 
so we accumulate more experience with it.


Andrei



 May 19 2013



 next sibling parent reply "Mr. Anonymous" <mailnew4ster gmail.com>  writes:


On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:

 Well you got to do what you got to do. Field accesses for 
 objects larger than 4KB would have to be checked in  safe code.


Isn't the solution as easy as doing:
OR PTR:[address], 0
the same way it's done for the stack?

The offset it known at compile time in most cases, so the command 
would be required only if both:
* The object is larger than target OS' guard page size.
* The position is greater than target OS' guard page size, OR is 
unknown at compile time.



 May 19 2013



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 1:08 PM, Mr. Anonymous wrote:

 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:

 Well you got to do what you got to do. Field accesses for objects larger than
 4KB would have to be checked in  safe code.


 Isn't the solution as easy as doing:
 OR PTR:[address], 0
 the same way it's done for the stack?

 The offset it known at compile time in most cases, so the command would be
 required only if both:
 * The object is larger than target OS' guard page size.
 * The position is greater than target OS' guard page size, OR is unknown at
 compile time.


Not a bad idea.



 May 19 2013



prev sibling next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com>  writes:


On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:

 You are blowing it out of proportion. Null references are 
 hardly even on the radar of the bug classes I'm encountering in 
 the style of programming of the three groups I worked in at 
 Facebook, and also my previous employers. People I meet at 
 conferences and consulting gigs never mention null references 
 as a real problem, although I very often ask about problems. I 
 find it difficult to agree with you just to be nice.


Just because people don't mention them as a problem doesn't mean 
it isn't a problem.

For what it's worth, null pointers are a real problem in the code 
I work on (games). I don't know exactly what you work on, but I 
find that they are more of a problem in highly stateful, 
interactive applications. Things like generic libraries, utility 
programs, compilers, etc. probably won't see the same problems 
because they aren't very stateful or interactive.

In my experience, null pointers are easy to fix, but the risk of 
them causes people to litter their code with if (ptr) tests, 
often with poor handling of the failure case, which can cause 
subtle bugs (no crash, but unintended code path).

Just my 2c.



 May 19 2013



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 4:30 PM, Peter Alexander wrote:

 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:

 You are blowing it out of proportion. Null references are hardly even
 on the radar of the bug classes I'm encountering in the style of
 programming of the three groups I worked in at Facebook, and also my
 previous employers. People I meet at conferences and consulting gigs
 never mention null references as a real problem, although I very often
 ask about problems. I find it difficult to agree with you just to be
 nice.


 Just because people don't mention them as a problem doesn't mean it
 isn't a problem.

 For what it's worth, null pointers are a real problem in the code I work
 on (games). I don't know exactly what you work on, but I find that they
 are more of a problem in highly stateful, interactive applications.
 Things like generic libraries, utility programs, compilers, etc.
 probably won't see the same problems because they aren't very stateful
 or interactive.

 In my experience, null pointers are easy to fix, but the risk of them
 causes people to litter their code with if (ptr) tests, often with poor
 handling of the failure case, which can cause subtle bugs (no crash, but
 unintended code path).

 Just my 2c.


OK, this is sensible. One question - would you be willing to type 
symbols as NullType!T instead of T to avoid these issues?

Thanks,

Andrei



 May 19 2013



 next sibling parent  "w0rp" <devw0rp gmail.com>  writes:


On Sunday, 19 May 2013 at 21:36:17 UTC, Andrei Alexandrescu wrote:

 On 5/19/13 4:30 PM, Peter Alexander wrot
 OK, this is sensible. One question - would you be willing to 
 type symbols as NullType!T instead of T to avoid these issues?

 Thanks,

 Andrei


Trying to come up with some once-and-for-all safe way to deal 
with null in languages which allow null references by default is 
something I wonder about now and then. This is due to my desire 
to adopt nice patterns for making my time spent working with a 
language safer and easier.

Option/Maybe I think only really works when the language has 
references as non-null by default. Otherwise, you're writing 
verbose code for Option/Maybe and lies the rest of the time. (I'm 
looking at Scala.) For D, I decided that my way to do it, to be 
applied almost all of the time, is to write contracts.

void func(Klass value) in {
     assert(value !is null);
} body {
}

This stops null from being passed through too many functions, so 
that alleviates a lot of problems. I think contract programming 
is a good solution for this, and it applies more generally to 
other kinds of invalid values. This can include values other than 
the default values.



 May 19 2013



prev sibling next sibling parent  "Peter Alexander" <peter.alexander.au gmail.com>  writes:


On Sunday, 19 May 2013 at 21:36:17 UTC, Andrei Alexandrescu wrote:

 OK, this is sensible. One question - would you be willing to 
 type symbols as NullType!T instead of T to avoid these issues?


Good question. Probably not. I think it's one of those things 
were the awkwardness of it not being the default would lead to 
lack of use (in the same way usage of pure suffers from not being 
default).



 May 20 2013



prev sibling  parent reply Byron Heads <byron.heads gmail.com>  writes:


On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:


 On 5/19/13 4:30 PM, Peter Alexander wrote:

 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:

 You are blowing it out of proportion. Null references are hardly even
 on the radar of the bug classes I'm encountering in the style of
 programming of the three groups I worked in at Facebook, and also my
 previous employers. People I meet at conferences and consulting gigs
 never mention null references as a real problem, although I very often
 ask about problems. I find it difficult to agree with you just to be
 nice.


 Just because people don't mention them as a problem doesn't mean it
 isn't a problem.

 For what it's worth, null pointers are a real problem in the code I
 work on (games). I don't know exactly what you work on, but I find that
 they are more of a problem in highly stateful, interactive
 applications. Things like generic libraries, utility programs,
 compilers, etc. probably won't see the same problems because they
 aren't very stateful or interactive.

 In my experience, null pointers are easy to fix, but the risk of them
 causes people to litter their code with if (ptr) tests, often with poor
 handling of the failure case, which can cause subtle bugs (no crash,
 but unintended code path).

 Just my 2c.


 
 OK, this is sensible. One question - would you be willing to type
 symbols as NullType!T instead of T to avoid these issues?
 
 Thanks,
 
 Andrei


More boiler plate code for functions that take pointers.

void foo(T)(T t)
	if(isPointer!T)
{
    static if(isNullable!T)
	if(!t) throw ....
}

May also introduce then need to check for objects in the init state 
(default state)



outside of NonNull in stdlib, an attribute maybe a possible answer.

void bar( nonnull SomeClass o)
{
	o = foo(); // if foo returns  nonnull, then check is not needed, 
else needs a check added during assignment
}

there are a few compile time checks that can be done to prove o is not 
null, if not the compiler adds runtime checks.

But really, checking for valid input is part of programming, might be 
better to have lint integration that can help find these types of problems



 May 20 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/20/13 11:19 AM, Byron Heads wrote:

 On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:

 OK, this is sensible. One question - would you be willing to type
 symbols as NullType!T instead of T to avoid these issues?

 Thanks,

 Andrei


 More boiler plate code for functions that take pointers.

 void foo(T)(T t)
 	if(isPointer!T)
 {
      static if(isNullable!T)
 	if(!t) throw ....
 }


But this goes both ways. Regardless of the default, you'd sometimes need 
to distinguish between cases. You either hurt one half of your cases or 
the other half. In fact you are now showing the case that assumes 
non-nullable being the default.


 May also introduce then need to check for objects in the init state
 (default state)



 outside of NonNull in stdlib, an attribute maybe a possible answer.

 void bar( nonnull SomeClass o)
 {
 	o = foo(); // if foo returns  nonnull, then check is not needed,
 else needs a check added during assignment
 }

 there are a few compile time checks that can be done to prove o is not
 null, if not the compiler adds runtime checks.


I think a parameterized type is a better match for non-null because it 
attaches to the type, not the symbol.


 But really, checking for valid input is part of programming, might be
 better to have lint integration that can help find these types of problems


Agreed.


Andrei



 May 20 2013



  parent reply Byron Heads <byron.heads gmail.com>  writes:


On Mon, 20 May 2013 11:43:35 -0400, Andrei Alexandrescu wrote:


 On 5/20/13 11:19 AM, Byron Heads wrote:

 On Sun, 19 May 2013 17:36:17 -0400, Andrei Alexandrescu wrote:

 OK, this is sensible. One question - would you be willing to type
 symbols as NullType!T instead of T to avoid these issues?

 Thanks,

 Andrei


 More boiler plate code for functions that take pointers.

 void foo(T)(T t)
 	if(isPointer!T)
 {
      static if(isNullable!T)
 	if(!t) throw ....
 }


 
 But this goes both ways. Regardless of the default, you'd sometimes need
 to distinguish between cases. You either hurt one half of your cases or
 the other half. In fact you are now showing the case that assumes
 non-nullable being the default.
 

 May also introduce then need to check for objects in the init state
 (default state)



 outside of NonNull in stdlib, an attribute maybe a possible answer.

 void bar( nonnull SomeClass o)
 {
 	o = foo(); // if foo returns  nonnull, then check is not needed,
 else needs a check added during assignment }

 there are a few compile time checks that can be done to prove o is not
 null, if not the compiler adds runtime checks.


 
 I think a parameterized type is a better match for non-null because it
 attaches to the type, not the symbol.
 

 But really, checking for valid input is part of programming, might be
 better to have lint integration that can help find these types of
 problems


 
 Agreed.
 
 
 Andrei




What about dealing with externs you want to protect?

extern(C) void foo( nonnull int* x);

other then that I think the library solution is fine

 nonnull Bar* b <=> NotNull!(NotNull!Bar) b



 May 20 2013



  parent  "Simen Kjaeraas" <simen.kjaras gmail.com>  writes:


On Mon, 20 May 2013 18:05:18 +0200, Byron Heads <byron.heads gmail.com>  
wrote:


 What about dealing with externs you want to protect?

 extern(C) void foo( nonnull int* x);


There is nothing stopping you from declaring that with this signature:

extern(C) void foo(NonNull!(int*) x);

-- 
Simen



 May 20 2013



prev sibling  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 20:30:28 UTC, Peter Alexander wrote:

 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu 
 wrote:

 You are blowing it out of proportion. Null references are 
 hardly even on the radar of the bug classes I'm encountering 
 in the style of programming of the three groups I worked in at 
 Facebook, and also my previous employers. People I meet at 
 conferences and consulting gigs never mention null references 
 as a real problem, although I very often ask about problems. I 
 find it difficult to agree with you just to be nice.


 Just because people don't mention them as a problem doesn't 
 mean it isn't a problem.

 For what it's worth, null pointers are a real problem in the 
 code I work on (games). I don't know exactly what you work on, 
 but I find that they are more of a problem in highly stateful, 
 interactive applications. Things like generic libraries, 
 utility programs, compilers, etc. probably won't see the same 
 problems because they aren't very stateful or interactive.

 In my experience, null pointers are easy to fix, but the risk 
 of them causes people to litter their code with if (ptr) tests, 
 often with poor handling of the failure case, which can cause 
 subtle bugs (no crash, but unintended code path).

 Just my 2c.


Exactly !

Most of time null issue are easy to solve and is just the kind of 
crap that slow you down, and once in a while this ends up being 
an horrible mess.

It is also true that rare path are often badly tested (and 
sometime you have no clue if the thing can be null or not, so you 
don't even know how to test it). Same argument Walter like to 
make about very rare failure cases apply here.



 May 19 2013



  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 3:04 PM, deadalnix wrote:

 Same argument Walter like to make about very rare failure cases apply here.



1. rare as in programmers rarely create such a bug

2. rare as in being rare for an existing bug to show itself

I was referring to (1), while you are referring to (2).



 May 19 2013



  parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Monday, 20 May 2013 at 00:09:23 UTC, Walter Bright wrote:

 On 5/19/2013 3:04 PM, deadalnix wrote:

 Same argument Walter like to make about very rare failure 
 cases apply here.



 1. rare as in programmers rarely create such a bug

 2. rare as in being rare for an existing bug to show itself

 I was referring to (1), while you are referring to (2).


When you talk about UNIX utilities not handling properly a full 
filesystem for instance, you are referring to 1.



 May 19 2013



  parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 05/20/2013 02:33 AM, deadalnix wrote:

 On Monday, 20 May 2013 at 00:09:23 UTC, Walter Bright wrote:

 On 5/19/2013 3:04 PM, deadalnix wrote:

 Same argument Walter like to make about very rare failure cases apply
 here.



 1. rare as in programmers rarely create such a bug

 2. rare as in being rare for an existing bug to show itself

 I was referring to (1), while you are referring to (2).


 When you talk about UNIX utilities not handling properly a full
 filesystem for instance, you are referring to 1.


You mean 2.



 May 19 2013



prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com>  writes:


On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:

 On 5/19/13 3:36 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu 
 wrote:

 No, your argument is ridiculous. You make a yarn with 
 precious little
 detail that describes for everything everyone knows a 
 textbook race
 condition, essentially ask that you are taking by your word 
 that
 non-null would miraculously solve it, and, to add insult to 
 injury,
 and when we don't buy it, you put the burden of proof on us. 
 This is
 quite a trick, my hat is off to you.


 I described a very usual null bug : something is set to null, 
 then to a
 specific value. It is assumed not to be null. In a specific 
 case it is
 null and everything explode.

 The concurrent context here made it especially hard to debug, 
 but isn't
 the cause of the bug.

 Additionally, if you don't have enough information to 
 understand what
 I'm saying, you are perfectly allowed to ask for additional 
 details This
 isn't a shame.


 Your argument has been destroyed so no need to ask details 
 about it. Replace "null" with "invalid state" and it's the same 
 race in any system. Let's move on.


"I don't want t to understand because I know I'm right. The fact 
you solved that issue and I didn't is irrelevant, I know better."


 You are blowing it out of proportion. Null references are 
 hardly even on the radar of the bug classes I'm encountering in 
 the style of programming of the three groups I worked in at 
 Facebook, and also my previous employers. People I meet at 
 conferences and consulting gigs never mention null references 
 as a real problem, although I very often ask about problems. I 
 find it difficult to agree with you just to be nice.


Hiphop type annotations are non null by default. Just saying.



 May 19 2013



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 5:52 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 20:03:24 UTC, Andrei Alexandrescu wrote:

 On 5/19/13 3:36 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 19:10:28 UTC, Andrei Alexandrescu wrote:

 No, your argument is ridiculous. You make a yarn with precious little
 detail that describes for everything everyone knows a textbook race
 condition, essentially ask that you are taking by your word that
 non-null would miraculously solve it, and, to add insult to injury,
 and when we don't buy it, you put the burden of proof on us. This is
 quite a trick, my hat is off to you.


 I described a very usual null bug : something is set to null, then to a
 specific value. It is assumed not to be null. In a specific case it is
 null and everything explode.

 The concurrent context here made it especially hard to debug, but isn't
 the cause of the bug.

 Additionally, if you don't have enough information to understand what
 I'm saying, you are perfectly allowed to ask for additional details This
 isn't a shame.


 Your argument has been destroyed so no need to ask details about it.
 Replace "null" with "invalid state" and it's the same race in any
 system. Let's move on.


 "I don't want t to understand because I know I'm right. The fact you
 solved that issue and I didn't is irrelevant, I know better."


Nobody knows what the issue is. It's all unstated assumptions leading to 
vague claims. You have been challenged to explain it so as to count as 
an anecdote in favor of non-null pointers - which it may as well be! -, 
and failed to raise to it.

Andrei



 May 19 2013



prev sibling  parent reply "Steven Schveighoffer" <schveiguy yahoo.com>  writes:


On Sun, 19 May 2013 16:03:24 -0400, Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org> wrote:


 On 5/19/13 3:36 PM, deadalnix wrote:

 I described a very usual null bug : something is set to null, then to a
 specific value. It is assumed not to be null. In a specific case it is
 null and everything explode.

 The concurrent context here made it especially hard to debug, but isn't
 the cause of the bug.

 Additionally, if you don't have enough information to understand what
 I'm saying, you are perfectly allowed to ask for additional details This
 isn't a shame.


 Your argument has been destroyed so no need to ask details about it.  
 Replace "null" with "invalid state" and it's the same race in any  
 system. Let's move on.


I just wanted to chime in with this understanding of the bug that I am  
reading from deadalnix's descriptions:

SomeObj obj;
shareTheObj(&obj); // goes to other threads
obj = new SomeObj; // initialize obj

This is likely simpler than the actual problem, but I think this is the  
gist of it.

A Non-Nullable solution WOULD solve the race:

SomeObj obj; // Compiler: nope, can't do that, must initialize it.

Now, with an "invalid state" not available like null, is the developer  
more likely to go through the trouble of building an invalid state for  
obj, in order to keep the racy behavior?  No.  He's just going to move the  
initialization:

SomeObj obj = new SomeObj;
shareTheObj(&obj);

In essence the point of the anecdote is that a Non-Nullable reference  
would have PROMOTED avoiding the race condition -- it would have been  
harder to keep the racy behavior.

I'm not saying that I think we need NN references as a compiler-supported  
type, or that it needs to be the default, or that NN references ALWAYS  
solve race conditions.  I'm just pointing out what I see is an obvious  
misinterpretation of the underlying story.

-Steve



 May 19 2013



  parent  Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 8:46 PM, Steven Schveighoffer wrote:

 I just wanted to chime in with this understanding of the bug that I am
 reading from deadalnix's descriptions:

 SomeObj obj;
 shareTheObj(&obj); // goes to other threads
 obj = new SomeObj; // initialize obj

 This is likely simpler than the actual problem, but I think this is the
 gist of it.

 A Non-Nullable solution WOULD solve the race:

 SomeObj obj; // Compiler: nope, can't do that, must initialize it.

 Now, with an "invalid state" not available like null, is the developer
 more likely to go through the trouble of building an invalid state for
 obj, in order to keep the racy behavior? No. He's just going to move the
 initialization:

 SomeObj obj = new SomeObj;
 shareTheObj(&obj);

 In essence the point of the anecdote is that a Non-Nullable reference
 would have PROMOTED avoiding the race condition -- it would have been
 harder to keep the racy behavior.


One can only assume the entire point was to delay initialization of the 
object to its first use/sharing point, so the changed pattern simply 
initializes the object eagerly, thus missing the benefits of the 
pattern. Otherwise that's an instance of the old adage "initialize 
variables at the point of definition". That, granted, non-null 
references do help with.


Andrei



 May 20 2013



prev sibling  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 12:36 PM, deadalnix wrote:

 I C or C++ you are doomed to manage reference as you need to for memory
 management purpose. In garbage collected languages, you ends up with way more
 unmanaged references, because the GC take care of them. Doing so you multiply
 the surface area where null bug can strike.


I still don't see the connection.

I've had many more null pointer bugs in non-gc code than in gc code, but I 
attribute that to my being a better programmer in gc code because I did that 
later and was more aware of issues.



 May 19 2013



prev sibling next sibling parent  Timon Gehr <timon.gehr gmx.ch>  writes:


On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote:

 On 5/19/13 1:35 PM, deadalnix wrote:

 On Sunday, 19 May 2013 at 13:13:07 UTC, Andrei Alexandrescu wrote:

 On 5/19/13 9:11 AM, deadalnix wrote:

 It is both a race condition and a null problem.


 No, it's just a race condition.


 And having non nullable
 type would have been a compile time error instead of days of debugging.


 No, the race condition would have stayed.


 That is ridiculous.  non nullable would have made the bug non existent,
 and even without race condition the problem would exists. a reference is
 null, it container shared, then set to something else. You can put
 barriers all over the place to make that sequentially consistent that it
 wouldn't change anything and the bug would still arise.


 No, your argument is ridiculous. You make a yarn with precious little
 detail that describes for everything everyone knows a textbook race
 condition, essentially ask that you are taking by your word that
 non-null would miraculously solve it, and, to add insult to injury, and
 when we don't buy it, you put the burden of proof on us. This is quite a
 trick, my hat is off to you.
 ...


It is easy to buy that the buggy code would have been rejected by a 
stronger type system whereas the fixed code would have passed type 
checking. Especially given that it manifested itself as a NPE, which 
would not even exist.



 May 19 2013



prev sibling  parent reply Timon Gehr <timon.gehr gmx.ch>  writes:


On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote:

 I encourage you to look at this :
 http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare


 I read it. I don't buy it. Yeah, it's a point, but it's largely
 exaggerated for dramatic purposes.


(It is a talk.)



 May 19 2013



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 5/19/13 3:58 PM, Timon Gehr wrote:

 On 05/19/2013 09:10 PM, Andrei Alexandrescu wrote:

 I encourage you to look at this :
 http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare


 I read it. I don't buy it. Yeah, it's a point, but it's largely
 exaggerated for dramatic purposes.


 (It is a talk.)


(I read a transcript.)

Andrei



 May 19 2013



  parent  Walter Bright <newshound2 digitalmars.com>  writes:


On 5/19/2013 1:06 PM, Andrei Alexandrescu wrote:

 On 5/19/13 3:58 PM, Timon Gehr wrote:

 (It is a talk.)


 (I read a transcript.)


I wish more talks had transcripts available. I can read an hour talk 
presentation in 5 minutes.



 May 19 2013



prev sibling  parent  Jonathan M Davis <jmdavisProg gmx.com>  writes:


On Sunday, May 19, 2013 09:08:53 Andrei Alexandrescu wrote:

 Anyhow, this discussion should have finality. We could go on forever
 arguing the usefulness or lack thereof of non-nullable references. They
 didn't catch up in some languages and did in some. My personal opinion
 is "nice to have, but not greatly compelling".
 
 It's reasonable to assume no major language changes will accommodate
 non-null references, so the next best thing would be to focus on a
 library solution. As Walter said, a library solution has the perk of
 allowing other interesting restricted types.


Wow, this thread really expanded since I looked at it last night. Yeah, I 
thought that it was clear that NonNullable or NotNull or whatever we want to 
call it was going to go in std.typecons and that that was going to be our 
solution to non-nullable references. This is one of those discussions that I 
should probably just stay out of, since it never seems to go anywhere useful. 
We've already decided on a solution. We just haven't gotten it into the 
standard library yet.

- Jonathan M Davis



 May 19 2013