www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Spec#, nullables and more

reply bearophile <bearophileHUGS lycos.com> writes:

new language. But both its specs and its implementation are unfinished still.
Designing it is a long & hard task.


Language, Methodology, and Tools to Write Bug-Free Programs" di K. Rustan M.
Leino and Peter Muller (2009):
http://research.microsoft.com/en-us/um/people/leino/papers/krml189.pdf

Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
http://www.rise4fun.com/SpecSharp



slow, and it may be a bit painful too. So it's meant only for code that needs
to be reliable, not for quick scripts.


- Non-nullable types;
- Statically enforced Contract Programming syntax and semantics;
- A way to list what attributes are modified in a method (similar to my  outer).
- Checked exceptions (like Java ones).

It looks like a short list, but that list is both quite hard to implement, and



compiler that verifies the code).


interested in it. I presume the "Microsoft" tag drags it down a little. And
probably not being really free doesn't help (maybe there is a partially open
source version now). (And maybe people are not so interested in a language for
low-bug-count programs).


language with low level features too (it has not GC-managed structs and
pointers and some other things).


the way you code, and requires more brain from the programmer too. Reading the

(better is not always better), you need to keep in mind some complex rules, and



to me.

Several of the ideas I have suggested here or I have put in the D Bugzilla are



enforced Contract Programming. Its DbC uses a static verifier that I don't
think will ever be added to DMD (despite eventually someone may try to write

ideas are useful to fix D design too.


(beside the normal simple tests done by a static type system), and I think they
are an additive change for D2, so they have a chance to be implementable for D3
too.

-----------------------------

Three quotations from the little tutorial (krml189.pdf):


Also, regardless of the compiler mode used, both inflections ? and ! are useful
in the implementations of generic classes: if T is a type parameter constrained
to be a reference type, then the naked name T stands for the actual type
parameter (which might be a possibly-null type or a non-null type), T? stands
for the possibly-null version of the type, and T! stands for the non-null
version of the type.

-----------

Regardless of old, an in-parameter mentioned in a method contract always refers
to the value of the parameter on entry (in other words, the fact that the
language allows in-parameters to be used as local variables inside the method
body has no effect on the meaning of the contract), and an out parameter always
refers to the value of the parameter on exit; ref parameters, which are treated
as copy-in copy-out, are sensitive to the use of old.

-----------

Non-null types express a special kind of invariant that needs to be established
by each
constructor. The virtual machine initially sets all fields of a new object to
zero-equivalent
values, in particular, fields of reference types to null. So before the new
object has been
fully initialized, it would be unjustified to assume that non-null fields
actually contain
non-null values.

Until the initialization of an object is completed, we say that the object is
delayed, meaning
that it is in a raw state where we can rely neither on the non-nullness of its
fields nor
on its object invariants. Moreover, field values of delayed objects are
themselves allowed
to be delayed. By default, the this object is delayed inside constructors.

A delayed object is in general not allowed to escape from its constructor.
However, sometimes
it is useful to call methods on delayed objects or to pass a delayed object as
an
argument to a method call. This is permitted if the callee method or its
parameter is
marked with the attribute [Delayed]. The consequence of this choice is that the
method
is then not allowed to assume fields of non-null types to have non-null values,
let alone
assume the object invariant to hold.(a)

An alternative is to mark the constructor with the attribute [NotDelayed]. This
requires
that all non-null fields of the class be initialized before an explicit call of
the superclass
(aka base class) constructor, base. A constructor can make a base call to a
[NotDelayed]
superclass constructor only if it itself is marked as [NotDelayed]. A
consequence of this
design is that after the base call, the this object is fully initialized and no
longer delayed.
Therefore, it can be passed to methods without the [Delayed] attribute.

Fähndrich and Xia [7] describe the details of delayed objects. Examples for
delayed objects
and explicit base calls can be found on the tutorial web page.

a) Any reference-valued parameter of a method, not just the receiver, can be
marked with
[Delayed]. However, there is a bug in the current version of the program
verifier that
makes the verification of methods with more than one [Delayed] parameter
unsound.


The article also suggests the usage of a shorter form to cast to nullable or
not nullable:
cast( )someRef
cast(?)someRef


My reference issue:
http://d.puremagic.com/issues/show_bug.cgi?id=4571

Bye,
bearophile
Nov 04 2010
next sibling parent reply Gary Whatmore <no spam.sp> writes:
bearophile Wrote:


 Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
 http://www.rise4fun.com/SpecSharp


D can do that too. We had those interactive versions in the newsrgoup. We saw
no value in them.



 - Non-nullable types;


The day D implements non-nullable types is the day I burn my copy of TDPL and
stop using D. Why keep you pushin this crap. I don't want to hear about it. It
doesn't improve my productivity at all.


 - Statically enforced Contract Programming syntax and semantics;


Too difficult to implement. Not worth it.


 - A way to list what attributes are modified in a method (similar to my
 outer).


The compiler should do this itself.


 - Checked exceptions (like Java ones).


Oh god.. what were they smoking?



 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=4571


Walter, please close this as wontfix. We don't need those. These extra runtime
checks will slow down my code. I know myself when my pointer is null.

 - G.W.
Nov 05 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Gary Whatmore:


 I know myself when my pointer is null.


You are unusual then:
http://lambda-the-ultimate.org/node/3186

Bye,
bearophile
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Gary Whatmore:
 

 I know myself when my pointer is null.


 
 You are unusual then:
 http://lambda-the-ultimate.org/node/3186


The $10 billion mistake was C's conversion of arrays to pointers when passing
to 
a function.

http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

Sadly, there's an ongoing failure to recognize this, as it is never addressed
in 
any of the revisions to the C or C++ standards, and is missed by the supposedly 
"safe C" alternatives.
Nov 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:


 The $10 billion mistake was C's conversion of arrays to pointers when passing
to 
 a function.
 
 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html
 
 Sadly, there's an ongoing failure to recognize this, as it is never addressed
in 
 any of the revisions to the C or C++ standards,


I agree, that's a very bad problem, probably worse than null-related bugs. But

to solve other problems. This thread was about a different problem




 and is missed by the supposedly "safe C" alternatives.


This is probably wrong. I don't know many C alternatives, but the well known
Cyclone language uses fat pointers (and other things) to solve that C problem.

Bye,
bearophile
Nov 05 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Walter Bright:
 

 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.
 
 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html
 
 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,


 
 I agree, that's a very bad problem, probably worse than null-related bugs.


It's infinitely worse. Null pointers do not result in memory corruption, buffer 
overflows, and security breaches.



 and is missed by the supposedly "safe C" alternatives.


 
 This is probably wrong. I don't know many C alternatives, but the well known
 Cyclone language uses fat pointers (and other things) to solve that C
 problem.



The Cyclone user manual says you have to rewrite a parameter as:

     void foo(int * numelts(4) arr);

to avoid the bugs with:

     void foo(int arr[]);

I think that latter broken syntax is still supported by Cyclone, but with the 
inadequate manual http://cyclone.thelanguage.org/wiki/User%20Manual it's hard
to 
tell.

Oh, and you have to redeclare the C:

     int sum(int num, int *p);

as:

     int sum(tag_t<`n> num,
         int * notnull  numelts(valueof(`n)) p);

No wonder Cyclone failed.
Nov 05 2010
next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Walter Bright schrieb:

 bearophile wrote:

 Walter Bright:


 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.

 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,


 I agree, that's a very bad problem, probably worse than null-related 
 bugs.


 
 It's infinitely worse. Null pointers do not result in memory corruption, 
 buffer overflows, and security breaches.
 


Not entirely true: Null Pointer dereferences *have* been used for security
breaches, see for 
example: http://lwn.net/Articles/342330/
The problem is that one can mmap() to 0/NULL so it can be dereferenced without
causing a crash.

Of course this is also a problem of the OS, it shouldn't allow mmap()ing to
NULL in the first place 
(it's now forbidden by default on Linux and FreeBSD afaik) - but some software
(dosemu, wine) 
doesn't work without it.

Cheers,
- Daniel
Nov 05 2010
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Daniel Gibson wrote:

 Walter Bright schrieb:

 It's infinitely worse. Null pointers do not result in memory 
 corruption, buffer overflows, and security breaches.


 
 Not entirely true: Null Pointer dereferences *have* been used for 
 security breaches, see for example: http://lwn.net/Articles/342330/
 The problem is that one can mmap() to 0/NULL so it can be dereferenced 
 without causing a crash.
 
 Of course this is also a problem of the OS, it shouldn't allow mmap()ing 
 to NULL in the first place (it's now forbidden by default on Linux and 
 FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it.


I'm surprised. 20 years ago, OS design articles I've seen all said that the 
first thing to do was render the bottom 64Kb of address space inaccessible in 
order to catch null pointer dereferences. People were pretty fed up with
Intel's 
decision to put the interrupt table in the first page of addresses, so any null 
pointers promptly trashed the operating system.

(Intel should have put the BIOS boot rom at location 0.)
Nov 05 2010
prev sibling next sibling parent reply Leandro Lucarella <luca llucax.com.ar> writes:
Daniel Gibson, el  5 de noviembre a las 19:52 me escribiste:

 Walter Bright schrieb:

bearophile wrote:

Walter Bright:


The $10 billion mistake was C's conversion of arrays to pointers when
passing to a function.

http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

Sadly, there's an ongoing failure to recognize this, as it is never
addressed in any of the revisions to the C or C++ standards,


I agree, that's a very bad problem, probably worse than
null-related bugs.


It's infinitely worse. Null pointers do not result in memory
corruption, buffer overflows, and security breaches.


 
 Not entirely true: Null Pointer dereferences *have* been used for
 security breaches, see for example: http://lwn.net/Articles/342330/
 The problem is that one can mmap() to 0/NULL so it can be dereferenced without
causing a crash.
 
 Of course this is also a problem of the OS, it shouldn't allow
 mmap()ing to NULL in the first place (it's now forbidden by default
 on Linux and FreeBSD afaik) - but some software (dosemu, wine)
 doesn't work without it.


And then, you can corrupt memory with something like:

struct S {
	int[1_000_000_000] data;
	int far_data;
}

S* s = null;
s.far_data = 5;

If you are unlucky enough to end up in a valid address. That might not
be a practical example, of course, but theoretically null pointer could
lead to memory corruption.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
<mazzi> gmail is down?
 <Luca> waiting for mail.google.com....
<mazzi> ya vendí todas mis acciones de google en los mercados asiaticos
 <Luca> se viene la ecatombe
<mazzi> mal
<mazzi> es como que te corten el porno en una tarde al pedo en tu casa
Nov 05 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/5/10 4:12 PM, Leandro Lucarella wrote:

 Daniel Gibson, el  5 de noviembre a las 19:52 me escribiste:

 Walter Bright schrieb:

 bearophile wrote:

 Walter Bright:


 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.

 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,


 I agree, that's a very bad problem, probably worse than
 null-related bugs.


 It's infinitely worse. Null pointers do not result in memory
 corruption, buffer overflows, and security breaches.


 Not entirely true: Null Pointer dereferences *have* been used for
 security breaches, see for example: http://lwn.net/Articles/342330/
 The problem is that one can mmap() to 0/NULL so it can be dereferenced without
causing a crash.

 Of course this is also a problem of the OS, it shouldn't allow
 mmap()ing to NULL in the first place (it's now forbidden by default
 on Linux and FreeBSD afaik) - but some software (dosemu, wine)
 doesn't work without it.


 And then, you can corrupt memory with something like:

 struct S {
 	int[1_000_000_000] data;
 	int far_data;
 }

 S* s = null;
 s.far_data = 5;

 If you are unlucky enough to end up in a valid address. That might not
 be a practical example, of course, but theoretically null pointer could
 lead to memory corruption.


The language may limit the static size of object. That's what Java does 
- it limits the size of any class to 64KB, and then every VM 
implementation guarantees that the first 64KB are made verboten one way 
or another.


Andrei
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Andrei Alexandrescu wrote:

 The language may limit the static size of object. That's what Java does 
 - it limits the size of any class to 64KB, and then every VM 
 implementation guarantees that the first 64KB are made verboten one way 
 or another.


I've meant to do that in D, but haven't gotten around to it.
Nov 05 2010
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-05 19:27:03 -0400, Walter Bright <newshound2 digitalmars.com> said:


 Andrei Alexandrescu wrote:

 The language may limit the static size of object. That's what Java does 
 - it limits the size of any class to 64KB, and then every VM 
 implementation guarantees that the first 64KB are made verboten one way 
 or another.


 
 I've meant to do that in D, but haven't gotten around to it.


On 32-bit OS X, that limit is 4 KB.

And what happens if I dereference a null pointer to a static array of 
65k elements and I try to read the last one?

Disallowing objects longer than 64 KB can help, but it's not a complete 
solution.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Nov 05 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Michel Fortin wrote:

 On 2010-11-05 19:27:03 -0400, Walter Bright <newshound2 digitalmars.com> 
 said:
 

 Andrei Alexandrescu wrote:

 The language may limit the static size of object. That's what Java 
 does - it limits the size of any class to 64KB, and then every VM 
 implementation guarantees that the first 64KB are made verboten one 
 way or another.


 I've meant to do that in D, but haven't gotten around to it.


 
 On 32-bit OS X, that limit is 4 KB.


That's good to know.



 And what happens if I dereference a null pointer to a static array of 
 65k elements and I try to read the last one?


Array index out of bounds.



 Disallowing objects longer than 64 KB can help, but it's not a complete 
 solution.


It's pretty darn close.
Nov 05 2010
parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-05 20:04:11 -0400, Walter Bright <newshound2 digitalmars.com> said:


 Michel Fortin wrote:

 On 32-bit OS X, that limit is 4 KB.


 
 That's good to know.


Well, you should already know. I posted this on the Phobos mailing list 
in August and you posted a reply. :-)



 And what happens if I dereference a null pointer to a static array of 
 65k elements and I try to read the last one?


 
 Array index out of bounds.


There's nothing out of the array's bounds in this case. Here's what I meant:

	byte[66000]* arrayPtr = null;
	byte b = (*arrayPtr)[66000-1];

I'm in the array's bounds here, the problem is that I'm dereferencing a 
null pointer but the program will actually only read 65999 bytes 
further, outside of the 64 KB "safe" zone.

Should we limit static arrays to 64 KB too?

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Michel Fortin wrote:

 On 2010-11-05 20:04:11 -0400, Walter Bright <newshound2 digitalmars.com> 
 said:
 

 Michel Fortin wrote:

 On 32-bit OS X, that limit is 4 KB.


 That's good to know.


 
 Well, you should already know. I posted this on the Phobos mailing list 
 in August and you posted a reply. :-)


Yeah, well, my brain is full. In order to learn new facts, I must discard an 
equivalent number of existing ones. I've had to discard everything I ever 
learned about chemistry, for example.




 And what happens if I dereference a null pointer to a static array of 
 65k elements and I try to read the last one?


 Array index out of bounds.


 
 There's nothing out of the array's bounds in this case. Here's what I 
 meant:
 
     byte[66000]* arrayPtr = null;
     byte b = (*arrayPtr)[66000-1];
 
 I'm in the array's bounds here, the problem is that I'm dereferencing a 
 null pointer but the program will actually only read 65999 bytes 
 further, outside of the 64 KB "safe" zone.
 
 Should we limit static arrays to 64 KB too?


That's why pointer arithmetic (which is what this is) is disallowed in safe
mode.
Nov 05 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:


 Yeah, well, my brain is full. In order to learn new facts, I must discard an 
 equivalent number of existing ones. I've had to discard everything I ever 
 learned about chemistry, for example.


As a human brain gets older, its ability to quickly retain new information
decreases. When you are 18 years old you are able to learn lot of stuff the
first time you hear it, while when you are 50 years old you need to listen to
the same information some times to learn (unless you are very trained to learn
a specific kind of information: a chess master is able to quickly memorize all
the moves of a game even at old age).

But what you have said is partially wrong. The mammal brain doesn't have a
fixed space for information, the more you learn the more space you have to
learn, because while neurons are in finite number and their number decreases
with age, new and synapses can be build every day, and higher level ways to
store information in the semantic network may be invented by a mind.

Book titles about what I have said on request :-)

Keep learning,
bye,
bearophile
Nov 05 2010
parent reply Jacob Carlborg <doob me.com> writes:
On 2010-11-06 05:46, bearophile wrote:

 Walter Bright:


 Yeah, well, my brain is full. In order to learn new facts, I must discard an
 equivalent number of existing ones. I've had to discard everything I ever
 learned about chemistry, for example.


 As a human brain gets older, its ability to quickly retain new information
decreases. When you are 18 years old you are able to learn lot of stuff the
first time you hear it, while when you are 50 years old you need to listen to
the same information some times to learn (unless you are very trained to learn
a specific kind of information: a chess master is able to quickly memorize all
the moves of a game even at old age).

 But what you have said is partially wrong. The mammal brain doesn't have a
fixed space for information, the more you learn the more space you have to
learn, because while neurons are in finite number and their number decreases
with age, new and synapses can be build every day, and higher level ways to
store information in the semantic network may be invented by a mind.

 Book titles about what I have said on request :-)

 Keep learning,
 bye,
 bearophile


Even though he may actually not forget a particular fact it will be 
harder and harder to find the fact because of other things "blocking the 
way". Eventaully it will seem you have forgot the fact.

-- 
/Jacob Carlborg
Nov 06 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Jacob Carlborg:


 Even though he may actually not forget a particular fact it will be 
 harder and harder to find the fact because of other things "blocking the 
 way". Eventaully it will seem you have forgot the fact.


It's not a block of the way, it's the networks that accumulate noise or just
slowly get used for other purposes. So it's not a matter of space, as Walter
has said, but mostly a matter of time: if you find some time to refresh old
memories you are able to keep them on the long term.

Bye,
bearophile
Nov 06 2010
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-05 23:27:28 -0400, Walter Bright <newshound2 digitalmars.com> said:


 Michel Fortin wrote:

 There's nothing out of the array's bounds in this case. Here's what I meant:
 
     byte[66000]* arrayPtr = null;
     byte b = (*arrayPtr)[66000-1];
 
 I'm in the array's bounds here, the problem is that I'm dereferencing a 
 null pointer but the program will actually only read 65999 bytes 
 further, outside of the 64 KB "safe" zone.
 
 Should we limit static arrays to 64 KB too?


 
 That's why pointer arithmetic (which is what this is) is disallowed in 
 safe mode.


Really? I'm doing two things in "(*arrayPtr)[66000-1]" in the above code:

1. Dereferencing a pointer to a the static array;
2. Accessing the last element within the bounds of the static array.

Which of these is disallowed in safe mode?

I think you should reread this example more attentively. It's no 
different than if you had put the static array as a member in a struct.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Nov 05 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Michel Fortin wrote:

 On 2010-11-05 23:27:28 -0400, Walter Bright <newshound2 digitalmars.com> 
 said:
 

 Michel Fortin wrote:

 There's nothing out of the array's bounds in this case. Here's what I 
 meant:

     byte[66000]* arrayPtr = null;
     byte b = (*arrayPtr)[66000-1];

 I'm in the array's bounds here, the problem is that I'm dereferencing 
 a null pointer but the program will actually only read 65999 bytes 
 further, outside of the 64 KB "safe" zone.

 Should we limit static arrays to 64 KB too?


 That's why pointer arithmetic (which is what this is) is disallowed in 
 safe mode.


 
 Really? I'm doing two things in "(*arrayPtr)[66000-1]" in the above code:
 
 1. Dereferencing a pointer to a the static array;
 2. Accessing the last element within the bounds of the static array.
 
 Which of these is disallowed in safe mode?
 
 I think you should reread this example more attentively. It's no 
 different than if you had put the static array as a member in a struct.
 


Looks like you're right.
Nov 06 2010
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/5/10 6:54 PM, Michel Fortin wrote:

 On 2010-11-05 19:27:03 -0400, Walter Bright <newshound2 digitalmars.com>
 said:


 Andrei Alexandrescu wrote:

 The language may limit the static size of object. That's what Java
 does - it limits the size of any class to 64KB, and then every VM
 implementation guarantees that the first 64KB are made verboten one
 way or another.


 I've meant to do that in D, but haven't gotten around to it.


 On 32-bit OS X, that limit is 4 KB.

 And what happens if I dereference a null pointer to a static array of
 65k elements and I try to read the last one?

 Disallowing objects longer than 64 KB can help, but it's not a complete
 solution.


It is - it all depends on choosing the right limit for all OSs.

Andrei
Nov 05 2010
prev sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/5/10 6:27 PM, Walter Bright wrote:

 Andrei Alexandrescu wrote:

 The language may limit the static size of object. That's what Java
 does - it limits the size of any class to 64KB, and then every VM
 implementation guarantees that the first 64KB are made verboten one
 way or another.


 I've meant to do that in D, but haven't gotten around to it.


Probably we should bugzillize it.

http://d.puremagic.com/issues/show_bug.cgi?id=5176


Andrei
Nov 05 2010
prev sibling parent reply Jesse Phillips <jessekphillips+D gmail.com> writes:
Leandro Lucarella Wrote:


 And then, you can corrupt memory with something like:
 
 struct S {
 	int[1_000_000_000] data;
 	int far_data;
 }
 
 S* s = null;
 s.far_data = 5;
 
 If you are unlucky enough to end up in a valid address. That might not
 be a practical example, of course, but theoretically null pointer could
 lead to memory corruption.


Interesting, I get this:

test.d(13): Error: index 1000000000 overflow for static array

I'm not exactly sure what you are trying to demonstrate with it. Are you
filling up the stack such that the OS tries to assign to a proper data field?
How is it still not trying to access location 0 causing an Access Violation?
Nov 05 2010
parent Leandro Lucarella <luca llucax.com.ar> writes:
Jesse Phillips, el  5 de noviembre a las 18:38 me escribiste:

 Leandro Lucarella Wrote:
 

 And then, you can corrupt memory with something like:
 
 struct S {
 	int[1_000_000_000] data;
 	int far_data;
 }
 
 S* s = null;
 s.far_data = 5;
 
 If you are unlucky enough to end up in a valid address. That might not
 be a practical example, of course, but theoretically null pointer could
 lead to memory corruption.


 
 Interesting, I get this:
 
 test.d(13): Error: index 1000000000 overflow for static array
 
 I'm not exactly sure what you are trying to demonstrate with it. Are
 you filling up the stack such that the OS tries to assign to a proper
 data field? How is it still not trying to access location 0 causing an
 Access Violation?


s.far_data = 5 is the same as *(s + S.far_data.sizeof) = 5, where
S.far_data.sizeof is 4_000_000_000 (i.e. 4 GB, I put a couple of extra
0 than I should :).

The point is, you're writing to a big address, which might be mapped to
your process, resulting in corrupted memory.

I think the error you are receiving is a compiler limitation, not
a language feature, but I might be wrong...

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Pa' ella cociné, pa' ella lavé, pa' ella soñe
Paella completa, $2,50
Pero, la luz mala me tira, y yo? yo soy ligero pa'l trote
La luz buena, está en el monte, allá voy, al horizonte
Nov 05 2010
prev sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 05/11/2010 18:52, Daniel Gibson wrote:

 Walter Bright schrieb:

 bearophile wrote:

 Walter Bright:


 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.

 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,


 I agree, that's a very bad problem, probably worse than null-related
 bugs.


 It's infinitely worse. Null pointers do not result in memory
 corruption, buffer overflows, and security breaches.


 Not entirely true: Null Pointer dereferences *have* been used for
 security breaches, see for example: http://lwn.net/Articles/342330/
 The problem is that one can mmap() to 0/NULL so it can be dereferenced
 without causing a crash.

 Of course this is also a problem of the OS, it shouldn't allow mmap()ing
 to NULL in the first place (it's now forbidden by default on Linux and
 FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it.

 Cheers,
 - Daniel


I think Walter's point remains true: null pointers bugs are an order of 
magnitude less important, if not downright insignificant, with regards 
to security breaches.

I mean, from my understanding of that article, a NPE bug on its own is 
not enough to allow an exploit, but other bugs/exploits need to be be 
present. (in that particular case, a straight-flush of them it seems). 
On the other hand, buffer overflows bugs nearly always make possible an 
exploit, correct?

-- 
Bruno Medeiros - Software Engineer
Nov 25 2010
next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/25/2010 10:28 AM, Bruno Medeiros wrote:

 I think Walter's point remains true: null pointers bugs are an order of
 magnitude less important, if not downright insignificant, with regards
 to security breaches.

 I mean, from my understanding of that article, a NPE bug on its own is
 not enough to allow an exploit, but other bugs/exploits need to be be
 present. (in that particular case, a straight-flush of them it seems).
 On the other hand, buffer overflows bugs nearly always make possible an
 exploit, correct?


 From a language designer's perspective, I think programmer association 
of semantic meaning to null (along with null's weak typing) is a more 
serious problem than npe vulnerabilities.
Nov 25 2010
parent spir <denis.spir gmail.com> writes:
On Thu, 25 Nov 2010 10:54:39 -0600
Ellery Newcomer <ellery-newcomer utulsa.edu> wrote:


 From a language designer's perspective, I think programmer association=20
 of semantic meaning to null (along with null's weak typing) is a more=20
 serious problem than npe vulnerabilities.


True! An unfortunately rarely pointed issue and somewhat nebulous to expose.
I had the idea of an UNDEF value on the implementation side that could not =
be used/assigned explicitely by the programmer. It would just allow idioms =
like
	if (x) {}	// if x is not undefined...
by beeing under the hood logically equal to false, but any operation, even =
toString (I mean writeTo ;-) would raise an UndefinedSymbol exception. Sinc=
e it's not assignable, a programmer cannot load it with app semantics (eg: =
author name unknown, element not found, invalid color code... whatever). A =
trial to get the best of both worlds. Just remains the problem of false/UND=
EF not beeing distinct in logical expressions.


Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 25 2010
prev sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
On Thu, Nov 25, 2010 at 5:28 PM, Bruno Medeiros
<brunodomedeiros+spam com.gmail> wrote:

 On 05/11/2010 18:52, Daniel Gibson wrote:

 Walter Bright schrieb:

 bearophile wrote:

 Walter Bright:


 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.

 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html

 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,


 I agree, that's a very bad problem, probably worse than null-related
 bugs.


 It's infinitely worse. Null pointers do not result in memory
 corruption, buffer overflows, and security breaches.


 Not entirely true: Null Pointer dereferences *have* been used for
 security breaches, see for example: http://lwn.net/Articles/342330/
 The problem is that one can mmap() to 0/NULL so it can be dereferenced
 without causing a crash.

 Of course this is also a problem of the OS, it shouldn't allow mmap()ing
 to NULL in the first place (it's now forbidden by default on Linux and
 FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it.

 Cheers,
 - Daniel


 I think Walter's point remains true: null pointers bugs are an order of
 magnitude less important, if not downright insignificant, with regards to
 security breaches.


No, that wasn't his point - he thought it was *impossible* to exploit null
pointers ("Null pointers do not result in memory corruption, buffer overflows,
and security breaches.") and I merely pointed out that this is not correct.
I didn't say anything about significance for average applications :-)


 I mean, from my understanding of that article, a NPE bug on its own is not
 enough to allow an exploit, but other bugs/exploits need to be be present.


Well it could be used by a non-privileged user to get root privileges.
If you only have "friendly" non-privileged users you need an exploit
to make them
execute the kernel exploit, of course.

But I agree that this kind of bug is not as relevant as others (for most users)
 - you won't have it in regular programs but only in kernels I guess.
(Of course it could work in regular programs as well, but you won't get more
privileges then you had before. Also I may be completely wrong on this and
maybe there is some way to gain something by using this kind of exploit
on regular programs.)


 (in that particular case, a straight-flush of them it seems). On the other
 hand, buffer overflows bugs nearly always make possible an exploit, correct?


I never said that buffer overflow bugs are better ;-)

Cheers,
- Daniel
Nov 25 2010
parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 25/11/2010 16:55, Daniel Gibson wrote:

 On Thu, Nov 25, 2010 at 5:28 PM, Bruno Medeiros
 <brunodomedeiros+spam com.gmail>  wrote:

 On 05/11/2010 18:52, Daniel Gibson wrote:

 Walter Bright schrieb:

 It's infinitely worse. Null pointers do not result in memory
 corruption, buffer overflows, and security breaches.


 Not entirely true: Null Pointer dereferences *have* been used for
 security breaches, see for example: http://lwn.net/Articles/342330/
 The problem is that one can mmap() to 0/NULL so it can be dereferenced
 without causing a crash.

 Of course this is also a problem of the OS, it shouldn't allow mmap()ing
 to NULL in the first place (it's now forbidden by default on Linux and
 FreeBSD afaik) - but some software (dosemu, wine) doesn't work without it.

 Cheers,
 - Daniel


 I think Walter's point remains true: null pointers bugs are an order of
 magnitude less important, if not downright insignificant, with regards to
 security breaches.


 No, that wasn't his point - he thought it was *impossible* to exploit null
 pointers ("Null pointers do not result in memory corruption, buffer overflows,
 and security breaches.") and I merely pointed out that this is not correct.
 I didn't say anything about significance for average applications :-)


Yes, Walter's statement that it is impossible for a null pointer to 
cause a security vulnerability is (likely) incorrect.
But his point at large, considering the discussion that preceded the 
comment, was that null pointers are utterly insignificant with regards 
to security vulnerabilities.

And I agree with that, and because of that I'm suprised and curious to 
understand why Hoare mentioned (in the abstract on the link posted 
originally), that null pointers have caused "innumerable vulnerabilities.



 I mean, from my understanding of that article, a NPE bug on its own is not
 enough to allow an exploit, but other bugs/exploits need to be be present.


 Well it could be used by a non-privileged user to get root privileges.
 If you only have "friendly" non-privileged users you need an exploit
 to make them
 execute the kernel exploit, of course.

 But I agree that this kind of bug is not as relevant as others (for most users)
   - you won't have it in regular programs but only in kernels I guess.
 (Of course it could work in regular programs as well, but you won't get more
 privileges then you had before. Also I may be completely wrong on this and
 maybe there is some way to gain something by using this kind of exploit
 on regular programs.)


By "exploit", I didn't mean to necessarily imply privilege escalation. I 
meant arbitrary code execution, with or without privilege escalation. (I 
don't know if this usage of the term is common in the security 
community, maybe not)

So, going back, is it correct to say that an NPE bug on its own is not 
enough to allow arbitrary code execution, but that other vulnerabilities 
are necessary?


-- 
Bruno Medeiros - Software Engineer
Nov 26 2010
next sibling parent reply Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 17:28, Bruno Medeiros wrote:

 And I agree with that, and because of that I'm suprised and curious to
 understand why Hoare mentioned (in the abstract on the link posted
 originally), that null pointers have caused "innumerable vulnerabilities.


Hum, cool, I just found out that this link:
http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare
has the actual presentation on it, so I'm gonna take a look.

-- 
Bruno Medeiros - Software Engineer
Nov 26 2010
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 17:54, Bruno Medeiros wrote:

 On 26/11/2010 17:28, Bruno Medeiros wrote:

 And I agree with that, and because of that I'm suprised and curious to
 understand why Hoare mentioned (in the abstract on the link posted
 originally), that null pointers have caused "innumerable vulnerabilities.


 Hum, cool, I just found out that this link:
 http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare

 has the actual presentation on it, so I'm gonna take a look.


I've seen the presentation, but he doesn't explain how a null pointer 
access would have caused a vulnerability. I'm going to assume that in 
all likelihood this applied to older computer architectures and/or OSes 
that didn't handle null pointer access that gracefully (1965 is way 
back...). But not so much to modern ones. Or that the vulnerability 
wasn't an actual arbitrary code execution, but some other system failure 
caused by the program crashing.

In any case this side-topic was just a minor curiosity, it's not really 
relevant for D.


But on his talk as a whole, the general point he made was interesting, 
he expressed the desire for languages to have more safety and checking, 
preferably on compile-time, if possible, and if not, on runtime at least 
(rather than have the program corrupt data, or execute crap). He 
mentioned that the big argument against this at that time was 
performance penalties, but that even so a lot of the people/companies 
were happy with the checks that were introduced (like array bounds 
checking), even if initially it didn't seem like a good idea.


-- 
Bruno Medeiros - Software Engineer
Nov 29 2010
prev sibling next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Bruno Medeiros schrieb:

 On 25/11/2010 16:55, Daniel Gibson wrote:

 On Thu, Nov 25, 2010 at 5:28 PM, Bruno Medeiros
 <brunodomedeiros+spam com.gmail>  wrote:

 On 05/11/2010 18:52, Daniel Gibson wrote:

 Walter Bright schrieb:

 It's infinitely worse. Null pointers do not result in memory
 corruption, buffer overflows, and security breaches.


 Not entirely true: Null Pointer dereferences *have* been used for
 security breaches, see for example: http://lwn.net/Articles/342330/
 The problem is that one can mmap() to 0/NULL so it can be dereferenced
 without causing a crash.

 Of course this is also a problem of the OS, it shouldn't allow 
 mmap()ing
 to NULL in the first place (it's now forbidden by default on Linux and
 FreeBSD afaik) - but some software (dosemu, wine) doesn't work 
 without it.

 Cheers,
 - Daniel


 I think Walter's point remains true: null pointers bugs are an order of
 magnitude less important, if not downright insignificant, with 
 regards to
 security breaches.


 No, that wasn't his point - he thought it was *impossible* to exploit 
 null
 pointers ("Null pointers do not result in memory corruption, buffer 
 overflows,
 and security breaches.") and I merely pointed out that this is not 
 correct.
 I didn't say anything about significance for average applications :-)


 
 Yes, Walter's statement that it is impossible for a null pointer to 
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the 
 comment, was that null pointers are utterly insignificant with regards 
 to security vulnerabilities.
 
 And I agree with that, and because of that I'm suprised and curious to 
 understand why Hoare mentioned (in the abstract on the link posted 
 originally), that null pointers have caused "innumerable vulnerabilities.
 


I don't know for sure, but I guess there have been multiple vulnerabilities in 
kernel code (of different operating systems) caused by null pointers.
So it may not affect most programmers (that don't do kernel stuff), but there 
are still a lot of (?) vulnerabilities caused by that. And those are bad, 
because you mess within the kernel, so you can do *anything*.


 

 I mean, from my understanding of that article, a NPE bug on its own 
 is not
 enough to allow an exploit, but other bugs/exploits need to be be 
 present.


 Well it could be used by a non-privileged user to get root privileges.
 If you only have "friendly" non-privileged users you need an exploit
 to make them
 execute the kernel exploit, of course.

 But I agree that this kind of bug is not as relevant as others (for 
 most users)
   - you won't have it in regular programs but only in kernels I guess.
 (Of course it could work in regular programs as well, but you won't 
 get more
 privileges then you had before. Also I may be completely wrong on this 
 and
 maybe there is some way to gain something by using this kind of exploit
 on regular programs.)


 
 By "exploit", I didn't mean to necessarily imply privilege escalation. I 
 meant arbitrary code execution, with or without privilege escalation. (I 
 don't know if this usage of the term is common in the security 
 community, maybe not)


Yes. I guess (haven't dug too far into this) that NP dereferences are mostly 
used to exploit the kernel, because you mmap() to NULL in the unprivileged 
userland program and this mmap() affects kernel code.
I don't know if it's possible to have mmap() affect userland programs other
than 
the one that called it.
If it isn't possible you'd have to make the programm mmap() stuff to NULL which 
means you've already exploited it, so no need to use some NP dereference hack.


 
 So, going back, is it correct to say that an NPE bug on its own is not 
 enough to allow arbitrary code execution, but that other vulnerabilities 
 are necessary?
 


I don't think it's correct: You may have a "bad" user on your system (e.g. 
pseudo-public server like in universities) who executes code that exploits the 
NPE bug and gains root/kernel privileges that way.
You only need another vulnerability if there are no "bad users" on your system 
=> the only way to execute bad code is via an exploit (e.g. in your webbrowser 
or one of its plugins etc)


But, to make it clear: I'm no expert on this, I just read a few reports on that 
null pointer related kernel exploit.

Cheers,
- Daniel
Nov 26 2010
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 26/11/2010 19:20, Daniel Gibson wrote:

 So, going back, is it correct to say that an NPE bug on its own is not
 enough to allow arbitrary code execution, but that other
 vulnerabilities are necessary?


 I don't think it's correct: You may have a "bad" user on your system
 (e.g. pseudo-public server like in universities) who executes code that
 exploits the NPE bug and gains root/kernel privileges that way.
 You only need another vulnerability if there are no "bad users" on your
 system => the only way to execute bad code is via an exploit (e.g. in
 your webbrowser or one of its plugins etc)


You're still thinking in terms of compromising a machine. I didn't mean 
specifically that, forget users and hosts, I was thinking of a more 
general and simpler case which relates to a process only: can the input 
to some program P cause arbitrary code execution, when it shouldn't? 
Like, can an image file cause my image viewer to run arbitrary code, can 
a zip file to the same for my zip extractor, etc.. (the same if the 
program is a server process and the input comes across the network)


-- 
Bruno Medeiros - Software Engineer
Nov 29 2010
prev sibling parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 11/26/2010 10:28, Bruno Medeiros wrote:

 Yes, Walter's statement that it is impossible for a null pointer to
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the
 comment, was that null pointers are utterly insignificant with regards
 to security vulnerabilities.


I really hate this way of thinking.  Security vulnerabilities are binary
- either they exist or they don't.  Every security vulnerability seems
minor until it is exploited.

Yes, some security vulnerabilities are more likely to be exploited than
others.  But instead of rationalizing about how significant each
individual security vulnerability is, isn't it better to just fix all of
them?

(I know, I'm a hopeless idealist.)


-- 
Rainer Deyke - rainerd eldwood.com
Nov 26 2010
next sibling parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Rainer Deyke schrieb:

 On 11/26/2010 10:28, Bruno Medeiros wrote:

 Yes, Walter's statement that it is impossible for a null pointer to
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the
 comment, was that null pointers are utterly insignificant with regards
 to security vulnerabilities.


 
 I really hate this way of thinking.  Security vulnerabilities are binary
 - either they exist or they don't.  Every security vulnerability seems
 minor until it is exploited.
 
 Yes, some security vulnerabilities are more likely to be exploited than
 others.  But instead of rationalizing about how significant each
 individual security vulnerability is, isn't it better to just fix all of
 them?
 
 (I know, I'm a hopeless idealist.)
 


Of course you should fix all of them, but the (sensible) language level support 
to prevent them may vary.
e.g. buffer overflows are a huge problem, so D has builtin index checking for 
arrays and such.
Null Pointer dereference exploits are - as far as I know - only relevant in 
kernel code and if mmaping to NULL is possible. So it's probably not a common 
problem in D code and people who write code that may be affected can take care 
themselves.

Cheers,
- Daniel
Nov 26 2010
parent %u <webnews digitalmars.com> writes:
Daniel Gibson Wrote:


 Rainer Deyke schrieb:

 On 11/26/2010 10:28, Bruno Medeiros wrote:

 Yes, Walter's statement that it is impossible for a null pointer to
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the
 comment, was that null pointers are utterly insignificant with regards
 to security vulnerabilities.


 
 I really hate this way of thinking.  Security vulnerabilities are binary
 - either they exist or they don't.  Every security vulnerability seems
 minor until it is exploited.
 
 Yes, some security vulnerabilities are more likely to be exploited than
 others.  But instead of rationalizing about how significant each
 individual security vulnerability is, isn't it better to just fix all of
 them?
 
 (I know, I'm a hopeless idealist.)
 


 
 Of course you should fix all of them, but the (sensible) language level
support 
 to prevent them may vary.
 e.g. buffer overflows are a huge problem, so D has builtin index checking for 
 arrays and such.
 Null Pointer dereference exploits are - as far as I know - only relevant in 
 kernel code and if mmaping to NULL is possible. So it's probably not a common 
 problem in D code and people who write code that may be affected can take care 
 themselves.


Even if eliminating unnecessary NPEs or segfaults doesn't prevent many
significant security problems, it can improve the user experience in tremendous
ways.

I'm an embedded / qt / android software engineer working on Linux. My
(graphical) tools experience random segfaults in the range of tens of times per
a single work day. Reloading the same tools over and over again and restoring
the lost data may actually waste almost 10% of my daily work time. Imagine if
the industry lost 10% of its income because of sloppy tools written in too low
level languages, would that make the feature significant enough?

Instead of using automatic GUI builders and such, the tools we use are often
hand written C or C++. Why these happen? We forget to initialize or fail to
track the lifetime properly.
Nov 26 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 27/11/2010 03:29, Rainer Deyke wrote:

 On 11/26/2010 10:28, Bruno Medeiros wrote:

 Yes, Walter's statement that it is impossible for a null pointer to
 cause a security vulnerability is (likely) incorrect.
 But his point at large, considering the discussion that preceded the
 comment, was that null pointers are utterly insignificant with regards
 to security vulnerabilities.


 I really hate this way of thinking.  Security vulnerabilities are binary
 - either they exist or they don't.  Every security vulnerability seems
 minor until it is exploited.

 Yes, some security vulnerabilities are more likely to be exploited than
 others.  But instead of rationalizing about how significant each
 individual security vulnerability is, isn't it better to just fix all of
 them?

 (I know, I'm a hopeless idealist.)


You missed the point. The point wasn't that a vulnerability caused by a 
null pointer access was less serious or significant than a vulnerabitiy 
caused by a buffer overrun. Once a vulnerability exists, it should be 
fixed regardless, yes.
The point was that if you have a null pointer access *bug*, that bug is 
incredibly less likely to create a *vulnerability* than a buffer overrun 
*bug*. Note that "creating a vulnerability" means "making it *possible* 
to exploit the program", it does not mean "someone actually exploiting 
the vulnerability".

-- 
Bruno Medeiros - Software Engineer
Nov 29 2010
prev sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/05/2010 01:39 PM, Walter Bright wrote:

 It's infinitely worse. Null pointers do not result in memory corruption,
 buffer overflows, and security breaches.


Not true. Null pointer dereference exploits are difficult, but very real.
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Ellery Newcomer wrote:

 On 11/05/2010 01:39 PM, Walter Bright wrote:

 It's infinitely worse. Null pointers do not result in memory corruption,
 buffer overflows, and security breaches.


 Not true. Null pointer dereference exploits are difficult, but very real.



Is that like worrying that an airliner will crash into your house and explode?
Nov 05 2010
parent Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/05/2010 02:24 PM, Walter Bright wrote:

 Ellery Newcomer wrote:

 On 11/05/2010 01:39 PM, Walter Bright wrote:

 It's infinitely worse. Null pointers do not result in memory corruption,
 buffer overflows, and security breaches.


 Not true. Null pointer dereference exploits are difficult, but very real.



 Is that like worrying that an airliner will crash into your house and
 explode?


Don't know. Has anyone ever crashed into my house with an airliner and 
lived to tell about it?
Nov 05 2010
prev sibling parent retard <re tard.com.invalid> writes:
Fri, 05 Nov 2010 13:40:59 -0400, bearophile wrote:


 Walter Bright:
 

 The $10 billion mistake was C's conversion of arrays to pointers when
 passing to a function.
 
 http://www.drdobbs.com/blog/archives/2009/12/cs_biggest_mist.html
 
 Sadly, there's an ongoing failure to recognize this, as it is never
 addressed in any of the revisions to the C or C++ standards,


 
 I agree, that's a very bad problem, probably worse than null-related

 now go on and try to solve other problems. This thread was about a

 and D too.


One first needs to study these more modern languages before he/she can 
form any kind of relevant opinions.


 and is missed by the supposedly "safe C" alternatives.


 
 This is probably wrong. I don't know many C alternatives


The primary competition of D is all "safe/better" C clones, and naturally 
C and C++. D is trying to beat C and C++. To me it seems like the authors 
fail to see how and why more modern languages succeed.
Nov 05 2010
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Fri, 05 Nov 2010 14:43:48 +0300, Gary Whatmore <no spam.sp> wrote:


 bearophile Wrote:


 Plus a nice Microsoft site that allows you to try it in interactive  
 way, this is very good:
 http://www.rise4fun.com/SpecSharp


 D can do that too. We had those interactive versions in the newsrgoup.  
 We saw no value in them.



 - Non-nullable types;


 The day D implements non-nullable types is the day I burn my copy of  
 TDPL and stop using D. Why keep you pushin this crap. I don't want to  
 hear about it. It doesn't improve my productivity at all.


Is anyone FORCING you to use non-nullable references?
What's the big deal?
Nov 05 2010
prev sibling next sibling parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 12:43 PM, Gary Whatmore wrote:

 bearophile Wrote:

 - A way to list what attributes are modified in a method (similar to my
 outer).


 The compiler should do this itself.


Doesn't make sense.


 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=4571


 Walter, please close this as wontfix. We don't need those. These extra runtime
checks will slow down my code. I know myself when my pointer is null.

   - G.W.


How, exactly, do you know when your references are null? Without 
runtime checks, of course.
Nov 05 2010
parent reply Gary Whatmore <no spam.sp> writes:
Pelle Månsson Wrote:


 On 11/05/2010 12:43 PM, Gary Whatmore wrote:

 bearophile Wrote:

 - A way to list what attributes are modified in a method (similar to my
 outer).


 The compiler should do this itself.


 
 Doesn't make sense.
 

 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=4571


 Walter, please close this as wontfix. We don't need those. These extra runtime
checks will slow down my code. I know myself when my pointer is null.

   - G.W.


 
 How, exactly, do you know when your references are null? Without 
 runtime checks, of course.


Good code design makes the null pointer exceptions go away. I carefully ponder
what code goes where. Simple as that. That language just introduces a boatload
of unnecessary cruft in forms of runtime null checks. I don't need to know the
exact temporal location of nulls, it's enough if the code takes care of
handling it at run time.
Nov 05 2010
next sibling parent =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 02:48 PM, Gary Whatmore wrote:

 Pelle Månsson Wrote:


 On 11/05/2010 12:43 PM, Gary Whatmore wrote:

 bearophile Wrote:

 - A way to list what attributes are modified in a method (similar to my
 outer).


 The compiler should do this itself.


 Doesn't make sense.


 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=4571


 Walter, please close this as wontfix. We don't need those. These extra runtime
checks will slow down my code. I know myself when my pointer is null.

    - G.W.


 How, exactly, do you know when your references are null? Without
 runtime checks, of course.


 Good code design makes the null pointer exceptions go away. I carefully ponder
what code goes where. Simple as that. That language just introduces a boatload
of unnecessary cruft in forms of runtime null checks. I don't need to know the
exact temporal location of nulls, it's enough if the code takes care of
handling it at run time.


Say you write a library, with a class and a function. Something like this:

class C {
     /* stuff */
}

void foo(C[] cs) {
     foreach (c; cs) {
         // do stuff with c
     }
}

How do you handle null, in this case?
Nov 05 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Fri, 05 Nov 2010 09:48:50 -0400
Gary Whatmore <no spam.sp> wrote:


 Pelle M=C3=A5nsson Wrote:

=20
 How, exactly, do you know when your references are null? Without=20
 runtime checks, of course.


=20
 Good code design makes the null pointer exceptions go away. I carefully p=


onder what code goes where. Simple as that. That language just introduces a=
 boatload of unnecessary cruft in forms of runtime null checks. I don't nee=
d to know the exact temporal location of nulls, it's enough if the code tak=
es care of handling it at run time.

Good code design makes type errors go away. Just have a clear model of the =
app, carefully match your types with true notions of the domain. Simple as =
that. Who gets type errors, anyway? Do you? I don't.
It's just unnecessary load over the compiler dev team, isn't it? Let's get =
rid of that cruft!

Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 05 2010
prev sibling parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Gary Whatmore wrote:

 Pelle M=EF=BF=BDnsson Wrote:
=20

 On 11/05/2010 12:43 PM, Gary Whatmore wrote:

 bearophile Wrote:

 - A way to list what attributes are modified in a method (similar to=








 my  outer).

 The compiler should do this itself.


 Doesn't make sense.


 My reference issue:
 http://d.puremagic.com/issues/show_bug.cgi?id=3D4571


 Walter, please close this as wontfix. We don't need those. These extr=






a runtime checks will slow down my code. I know myself when my pointer is=
 null.

   - G.W.


 How, exactly, do you know when your references are null? Without=20
 runtime checks, of course.


=20
 Good code design makes the null pointer exceptions go away. I carefully=


 ponder what code goes where. Simple as that. That language just introduc=
es a boatload of unnecessary cruft in forms of runtime null checks. I don=
't need to know the exact temporal location of nulls, it's enough if the =
code takes care of handling it at run time.

	Who said anything about runtime checks? Non-nullable types are
enforced at compile-time anyway...

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Nov 05 2010
prev sibling next sibling parent Michel Fortin <michel.fortin michelf.com> writes:
On 2010-11-05 09:51:07 -0400, Andrei Alexandrescu 
<SeeWebsiteForEmail erdani.org> said:



 design team. They were added in a haste and now the designers regret 
 that.


But did they refer to nullable pointers or the ability to make regular 
value types (integers, etc.) nullable? These are actually two different 
things.
<http://msdn.microsoft.com/en-us/library/1t3y8s4s(VS.80).aspx>

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Nov 05 2010
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Gary Whatmore wrote:

 bearophile Wrote:
 

 Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
 http://www.rise4fun.com/SpecSharp


 
 D can do that too. We had those interactive versions in the newsrgoup. We saw
no value in them.


Here it is:

http://codepad.org/

It's been around a while, and nobody cares. Such a capability is marketing 
puffery, and not useful.
Nov 05 2010
next sibling parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Walter Bright Wrote:


 Gary Whatmore wrote:

 bearophile Wrote:
 

 Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
 http://www.rise4fun.com/SpecSharp


 
 D can do that too. We had those interactive versions in the newsrgoup. We saw
no value in them.


 
 Here it is:
 
 http://codepad.org/
 
 It's been around a while, and nobody cares. Such a capability is marketing 
 puffery, and not useful.


Sadly that site uses some Unknown version of D1.

http://www.ideone.com is using dmd 2.042 though. I don't these sites are very
good what you want to learn a new language, but the are nice to demonstrate
code snippets or test something out when you do know the language.
Nov 05 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:


 Here it is:
 
 http://codepad.org/
 
 It's been around a while, and nobody cares. Such a capability is marketing 
 puffery, and not useful.


There are two sites that allow to compile D code online: codepad (D1) and
ideone (D2).

Even if ideone site is just "marketing puffery" for D, recent discussions on
this newsgroup have shown that Andrei is willing to do something for marketing
purposes. And this kind of ideone marketing is not going to worsen the API of
Phobos, it's just a different way to use D compiler. So it doesn't harm.

If you take a look at the Go site you see a way to try code:
http://golang.org/

Or even LLVM compiler:
http://llvm.org/demo/

I have recently discussed with ideone devs to suggest a new feature, and it
seems their site is having enough visitors, so it seems some people care about
the site.

I have personally found ideone and codepad sites useful, because they allow you
to try snippets of code in languages you don't know yet. So you can try a new
language in a very fast and painless way, without committing to install a
compiler/interpreter to your PC. Today web and being online are very important
things, and ideone/codepad allow even compiled languages to be tried online by
young people used to javascript and browsers. I have used ideone to try small
programs in Scala, Haskell, and other languages. I have used ideone to compile
small snippets of D2 code in places where I don't have a D2 compiler available.

If you go take a visit to the IRC #D channel you see an tool based on codepad
that allows to try snippets of code from the IRC channel itself. A tool often
used by feep and others.

So you are wrong, ideone/codepad are used by people, are good marketing, allow
people to try D before installing the compiler and they don't harm D in any way.

What I have suggested in another post is not to put a online demo window in the
D site, but to just add a *link* to ideone and codepad in the D home page, so
people may try D online.

Bye,
bearophile
Nov 05 2010
next sibling parent reply David Gileadi <gileadis NSPMgmail.com> writes:
On 11/5/10 10:36 AM, bearophile wrote:

 What I have suggested in another post is not to put a online demo window in
the D site, but to just add a *link* to ideone and codepad in the D home page,
so people may try D online.


When I started the redesign of the D site, one of my first ideas was to 
add links for all code snippets to a copy of the code on codepad (I 
didn't know about ideone), so that people could see them in action. 
Later I decided that this might not have as much value as I thought, 
since many of the snippets depended on command-line arguments and I 
didn't see a way to provide them.

Nonetheless I still think that providing these kind of links would have 
value to newcomers wanting to learn and experiment with the language.
Nov 05 2010
parent bearophile <bearophileHUGS lycos.com> writes:
David Gileadi:


 Later I decided that this might not have as much value as I thought, 
 since many of the snippets depended on command-line arguments and I 
 didn't see a way to provide them.


ideone allows you to give command line arguments/input too to D programs (but
you can't change compilation switches yet, I have suggested that too to the
ideone authors).

Bye,
bearophile
Nov 05 2010
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 If you take a look at the Go site you see a way to try code:
 http://golang.org/


Yes, I know about that site and the accolades heaped on Go for having it. 
Meanwhile, codepad has had it for years, and comeaucomputing has had it for at 
least 20 years now (for Comeau C++), and nobody cared.

I agree they do no harm, but once you're past the gee-whiz factor, they aren't 
of much use.
Nov 05 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:


 I agree they do no harm, but once you're past the gee-whiz factor, they aren't 
 of much use.


Yet, I will keep using ideone almost daily :-)

Bye,
bearophile
Nov 05 2010
prev sibling parent retard <re tard.com.invalid> writes:
Fri, 05 Nov 2010 13:36:18 -0400, bearophile wrote:

 Walter:
 

 http://codepad.org/
 
 It's been around a while, and nobody cares. Such a capability is
 marketing puffery, and not useful.





 If you go take a visit to the IRC #D channel you see an tool based on
 codepad that allows to try snippets of code from the IRC channel itself.
 A tool often used by feep and others.
 
 So you are wrong, ideone/codepad are used by people, are good marketing,
 allow people to try D before installing the compiler and they don't harm
 D in any way.


It feels like these introvert academic profs and compiler developers with 
huge beards are actually getting more pragmatic than our BDFL. They 
regularly visit their IRC channels and are fluently using these online 
tools. They even adopted open source version control tools and public 
repositories before Walter did. Just take a look at Scala's trac.. This 
shouting from the "pragmatic" ivory tower is getting out of touch with 
reality.
Nov 05 2010
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Gary Whatmore wrote:

 - Checked exceptions (like Java ones).


 
 Oh god.. what were they smoking?


Checked exceptions are one of those ideas that look great on paper but are an 
utter failure in practice. As Bruce Eckel pointed out, they are *worse* than 
useless and *cause* bugs to be inserted into the code.
Nov 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:


 Checked exceptions are one of those ideas that look great on paper but are an 
 utter failure in practice. As Bruce Eckel pointed out, they are *worse* than 
 useless and *cause* bugs to be inserted into the code.


(Just to avoid possible misunderstandings: I have never suggested to add
checked exceptions to D).

I agree that checked exceptions are a pain in a general purpose language. But

language, where the user is supposed to endure some pain in the hope to produce
statically verified (and less buggy) programs. So while checked exceptions are


justified. I don't know if they are right (maybe they are wrong, surely not

well though out language). But you need to keep in account the quite special

idea.

Bye,
bearophile
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Walter Bright:
 

 Checked exceptions are one of those ideas that look great on paper but are
 an utter failure in practice. As Bruce Eckel pointed out, they are *worse*
 than useless and *cause* bugs to be inserted into the code.


 
 (Just to avoid possible misunderstandings: I have never suggested to add
 checked exceptions to D).



the latest in comp sci thought when it adds checked exceptions.



 I agree that checked exceptions are a pain in a general purpose language. But

 language, where the user is supposed to endure some pain in the hope to
 produce statically verified (and less buggy) programs. So while checked
 exceptions are probably a bad idea for a handy general purpose language, the

 exceptions are justified. I don't know if they are right (maybe they are

 looks like a really well though out language). But you need to keep in




I think you misunderstand why checked exceptions are such a bad idea. It's not 
just that they are inconvenient and annoying. They decrease security by
*hiding* 
bugs. That is the opposite of what you'd want in a high security language.

http://www.mindview.net/Etc/Discussions/CheckedExceptions



and the wrong thing easy.
Nov 05 2010
next sibling parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Walter Bright Wrote:


 I think you misunderstand why checked exceptions are such a bad idea. It's not 
 just that they are inconvenient and annoying. They decrease security by
*hiding* 
 bugs. That is the opposite of what you'd want in a high security language.


Not only is swallowing the exception a problem, but exceptions aren't added
liberally in the code since every exception thrown/renamed effects all calling
code. I'm still not in the habit of asserting assumptions.
Nov 05 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 05/11/2010 18:52, Walter Bright wrote:

 I think you misunderstand why checked exceptions are such a bad idea.
 It's not just that they are inconvenient and annoying. They decrease
 security by *hiding* bugs. That is the opposite of what you'd want in a
 high security language.

 http://www.mindview.net/Etc/Discussions/CheckedExceptions


Just to clarify: Checked Exceptions are not a source of bugs per se. 
What is a source of bugs is catch-hiding an exception temporarily and 
then forgetting to change the code later (that's the case Bruce mentions 
in the article).
But with discipline you can avoid this: just don't catch-hide 
exceptions. One never catch-hides exceptions by mistake, it is always 
conscious (unlike other bugs like off-by-ones, logic errors, etc.).
For example in Java I *always* wrap exceptions I don't care about in a 
RuntimeException, (using the adapter Bruce presented in that article, 
actually).
Is it annoying and/or unnecessary? Well, I'm not making a statement 
about that, just that it will only actually cause bugs if you are lazy.

-- 
Bruno Medeiros - Software Engineer
Nov 25 2010
prev sibling next sibling parent reply Ellery Newcomer <ellery-newcomer utulsa.edu> writes:
On 11/04/2010 03:44 PM, bearophile wrote:


new language. But both its specs and its implementation are unfinished still.
Designing it is a long&  hard task.


Language, Methodology, and Tools to Write Bug-Free Programs" di K. Rustan M.
Leino and Peter Muller (2009):
 http://research.microsoft.com/en-us/um/people/leino/papers/krml189.pdf

 Plus a nice Microsoft site that allows you to try it in interactive way, this
is very good:
 http://www.rise4fun.com/SpecSharp


hey, cool


the fact that ms was using it to write an OS kernel
Nov 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Ellery Newcomer:


 hey, cool
 

 the fact that ms was using it to write an OS kernel


It contains a ton of new computer science ideas :-) So it's interesting
regardless its applications. (If you don't keep yourself updated with new
computer science ideas, you can't keep being a productive programmer for many
years).

Bye,
bearophile
Nov 05 2010
parent reply Gary Whatmore <no spam.sp> writes:
bearophile Wrote:


 Ellery Newcomer:
 

 hey, cool
 

 the fact that ms was using it to write an OS kernel


 
 It contains a ton of new computer science ideas :-) So it's interesting
regardless its applications. (If you don't keep yourself updated with new
computer science ideas, you can't keep being a productive programmer for many
years).


Are you saying that Walter Bright or anyone else here isn't as productive as
you are because we haven't read about the latest language research done between
1980 and 2010? Keeping yourself updated with new ideas also means pragmatic
real world books, not just ivory tower research papers that usually aren't
freely accessible without an expensive subscription:

http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882/
http://www.amazon.com/Domain-Specific-Languages-Addison-Wesley-Signature-Martin/dp/0321712943/
http://www.amazon.com/D-Programming-Language-Andrei-Alexandrescu/dp/0321635361/
http://www.amazon.com/Lean-Architecture-Agile-Software-Development/dp/0470684208/
http://www.amazon.com/Mythical-Man-Month-Software-Engineering-Anniversary/dp/0201835959/
http://www.amazon.com/Pragmatic-Programmer-Journeyman-Master/dp/020161622X/
http://www.amazon.com/Programming-Massively-Parallel-Processors-Hands/dp/0123814723/
http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/dp/0201485672/
http://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530/
http://www.amazon.com/Version-Control-Git-Collaborative-Development/dp/0596520123/
http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052/

Do I need to say more?
Nov 05 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Gary Whatmore:


 Are you saying that Walter Bright or anyone else here isn't as productive as
you are because we haven't read about the latest language research done between
1980 and 2010?


I have found that I need to play with some "new" (*) CS ideas new and then, if
I want to keep my mind flexible and open, and keep my programming skills sharp
in the long term.

(*) I am aware that most of those "new" ideas are up to 30-40 years old. But
often a new implementation too is an interesting thing.

Bye,
bearophile
Nov 05 2010
prev sibling parent retard <re tard.com.invalid> writes:
Fri, 05 Nov 2010 09:53:50 -0400, Gary Whatmore wrote:


 bearophile Wrote:
 

 Ellery Newcomer:
 

 hey, cool
 

 least the fact that ms was using it to write an OS kernel


 
 It contains a ton of new computer science ideas :-) So it's interesting
 regardless its applications. (If you don't keep yourself updated with
 new computer science ideas, you can't keep being a productive
 programmer for many years).


 
 Are you saying that Walter Bright or anyone else here isn't as
 productive as you are because we haven't read about the latest language
 research done between 1980 and 2010? Keeping yourself updated with new
 ideas also means pragmatic real world books, not just ivory tower
 research papers that usually aren't freely accessible without an
 expensive subscription:
 
 http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/


dp/0132350882/

 http://www.amazon.com/Domain-Specific-Languages-Addison-Wesley-


Signature-Martin/dp/0321712943/

 http://www.amazon.com/D-Programming-Language-Andrei-Alexandrescu/


dp/0321635361/

 http://www.amazon.com/Lean-Architecture-Agile-Software-Development/


dp/0470684208/

 http://www.amazon.com/Mythical-Man-Month-Software-Engineering-


Anniversary/dp/0201835959/

 http://www.amazon.com/Pragmatic-Programmer-Journeyman-Master/


dp/020161622X/

 http://www.amazon.com/Programming-Massively-Parallel-Processors-Hands/


dp/0123814723/

 http://www.amazon.com/Refactoring-Improving-Design-Existing-Code/


dp/0201485672/

 http://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530/
 http://www.amazon.com/Version-Control-Git-Collaborative-Development/


dp/0596520123/

 http://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/


dp/0131177052/

 
 Do I need to say more?


Those all books deal with tools and non-academic best practices. Probably 
useful information, but it's not necessarily built on previous (written) 
knowledge about the domain. The more academic papers discuss why some 
algoritm works, how to measure it analytically, how language constructs 
are built from smaller pieces etc. You should at least read few of those 
to get a better view of the literature.

Programming language research has indeed found new techniques and results 
since the idea of OOP came some 30 to 40 years ago. Nothing suggests that 
OOP is the be-all and end-all of programming language design. See the 
links at http://harmful.cat-v.org/software/OO_programming/

Even D is slowly adopting real lambdas, algebraic data types, tuples, and 
all kinds of functional features.
Nov 05 2010
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
bearophile Wrote:



 - Non-nullable types;


It's hard to tell, whether they fix anything. When you cast nullable to
non-nullable, you get your runtime exception as usual, if you if out access to
nullable (e.g. in delayed method), you get your runtime exception again or
rather logical bug.
Nov 05 2010
parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 02:39 PM, Kagamin wrote:

 bearophile Wrote:



 - Non-nullable types;


 It's hard to tell, whether they fix anything. When you cast nullable to
non-nullable, you get your runtime exception as usual, if you if out access to
nullable (e.g. in delayed method), you get your runtime exception again or
rather logical bug.


Getting the error early is actually a lot better than getting the error 
late.
Nov 05 2010
next sibling parent reply Gary Whatmore <no spam.sp> writes:
Pelle Månsson Wrote:


 On 11/05/2010 02:39 PM, Kagamin wrote:

 bearophile Wrote:



 - Non-nullable types;


 It's hard to tell, whether they fix anything. When you cast nullable to
non-nullable, you get your runtime exception as usual, if you if out access to
nullable (e.g. in delayed method), you get your runtime exception again or
rather logical bug.


 
 Getting the error early is actually a lot better than getting the error 
 late.


Getting the error early means that less code compiles and that makes the rapid
development fail and turns it into a waterfall misery. It's important to make
your tests run quickly in the background. One reason I prefer Python is that it
let's me run even (semantically) buggy code, because syntactical correctness is
enough. It really improves productivity.
Nov 05 2010
parent =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 02:56 PM, Gary Whatmore wrote:

 Pelle Månsson Wrote:


 On 11/05/2010 02:39 PM, Kagamin wrote:

 bearophile Wrote:



 - Non-nullable types;


 It's hard to tell, whether they fix anything. When you cast nullable to
non-nullable, you get your runtime exception as usual, if you if out access to
nullable (e.g. in delayed method), you get your runtime exception again or
rather logical bug.


 Getting the error early is actually a lot better than getting the error
 late.


 Getting the error early means that less code compiles and that makes the rapid
development fail and turns it into a waterfall misery. It's important to make
your tests run quickly in the background. One reason I prefer Python is that it
let's me run even (semantically) buggy code, because syntactical correctness is
enough. It really improves productivity.


Yes, let's turn off compiler errors entirely!
Nov 05 2010
prev sibling parent reply Kagamin <spam here.lot> writes:
Pelle Månsson Wrote:


 Getting the error early is actually a lot better than getting the error 
 late.


OK, but it doesn't reduce the number of bugs. You had an error with nullables
and you still has error with non-nullables.
Nov 05 2010
parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 03:04 PM, Kagamin wrote:

 Pelle Månsson Wrote:


 Getting the error early is actually a lot better than getting the error
 late.


 OK, but it doesn't reduce the number of bugs. You had an error with nullables
and you still has error with non-nullables.


But in the non-nullable version you actually know where the bug is, 
namely where you assign the null to the thing that shouldn't be null. 
The segfault can come from any unrelated part of the program whereto 
your null has slipped, at any later point in time.
Nov 05 2010
parent reply Don <nospam nospam.com> writes:
Pelle Månsson wrote:

 On 11/05/2010 03:04 PM, Kagamin wrote:

 Pelle Månsson Wrote:


 Getting the error early is actually a lot better than getting the error
 late.


 OK, but it doesn't reduce the number of bugs. You had an error with 
 nullables and you still has error with non-nullables.


 
 But in the non-nullable version you actually know where the bug is, 
 namely where you assign the null to the thing that shouldn't be null. 
 The segfault can come from any unrelated part of the program whereto 
 your null has slipped, at any later point in time.


I've always found it very easy to work out where a null came from. What 
I would hope from a non-nullable type is to eliminate rare code paths 
where a null can occur, which might not show up in testing.
Nov 05 2010
next sibling parent dennis luehring <dl.soluz gmx.net> writes:
Am 05.11.2010 15:27, schrieb Don:

 Pelle Månsson wrote:

  On 11/05/2010 03:04 PM, Kagamin wrote:

  Pelle Månsson Wrote:


  Getting the error early is actually a lot better than getting the error
  late.


  OK, but it doesn't reduce the number of bugs. You had an error with
  nullables and you still has error with non-nullables.


  But in the non-nullable version you actually know where the bug is,
  namely where you assign the null to the thing that shouldn't be null.
  The segfault can come from any unrelated part of the program whereto
  your null has slipped, at any later point in time.


 I've always found it very easy to work out where a null came from. What
 I would hope from a non-nullable type is to eliminate rare code paths
 where a null can occur, which might not show up in testing.


or better -> "how many UNTESTED null-able situations out there?" i think 
an enormous amount of code that just explodes when an null runs through it


is see code like if( x != null ){ if( y != null ){ if( blub != null ... 
checks all day long, and if i don't see code like this does not mean 
that it is safer then...
Nov 05 2010
prev sibling parent Roman Ivanov <isroman.DEL ETE.km.ru> writes:
On 11/5/2010 10:27 AM, Don wrote:

 Pelle MÃ¥nsson wrote:

 On 11/05/2010 03:04 PM, Kagamin wrote:

 Pelle MÃ¥nsson Wrote:


 Getting the error early is actually a lot better than getting the error
 late.


 OK, but it doesn't reduce the number of bugs. You had an error with
 nullables and you still has error with non-nullables.


 But in the non-nullable version you actually know where the bug is,
 namely where you assign the null to the thing that shouldn't be null.
 The segfault can come from any unrelated part of the program whereto
 your null has slipped, at any later point in time.


 
 I've always found it very easy to work out where a null came from. 


I hadn't. It really depends on what kind of code you're dealing with.
I've seen null-related bugs that took many, many hours to trace, because
the code was using several third-party libraries.

For example, library A can return a null in some obscure case without
documenting that behavior. Library B can interpret null as a meaningful
value and try to do something. You code sits in between, does some logic
and passes information from A to B.

Stuff like that is _very_ difficult to debug, and having non-nullable
types would make it much, much easier.


 What
 I would hope from a non-nullable type is to eliminate rare code paths
 where a null can occur, which might not show up in testing.
Nov 05 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Denis Koroskin:


 Is anyone FORCING you to use non-nullable references?
 What's the big deal?


If non-nullables are introduced in D3, then Phobos will start to use them. So
probably you can't avoid using some of them.

Bye,
bearophile
Nov 05 2010
parent reply =?ISO-8859-1?Q?Pelle_M=E5nsson?= <pelle.mansson gmail.com> writes:
On 11/05/2010 08:30 PM, bearophile wrote:

 Denis Koroskin:


 Is anyone FORCING you to use non-nullable references?
 What's the big deal?


 If non-nullables are introduced in D3, then Phobos will start to use them. So
probably you can't avoid using some of them.

 Bye,
 bearophile


If we're still following the 'calls to phobos considered external 
input'-thing, the nulls still have to be checked. So, no loss there, 
performance wise.
Nov 05 2010
parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Pelle M=C3=A5nsson wrote:

 On 11/05/2010 08:30 PM, bearophile wrote:

 Denis Koroskin:


 Is anyone FORCING you to use non-nullable references?
 What's the big deal?


 If non-nullables are introduced in D3, then Phobos will start to use
 them. So probably you can't avoid using some of them.

 Bye,
 bearophile


=20
 If we're still following the 'calls to phobos considered external
 input'-thing, the nulls still have to be checked. So, no loss there,
 performance wise.


	Actually, you have a *gain* performance-wise since Phobos can drop
the checks and use a non-nullable type, which means that there will
be no check if the compiler can determine that you are passing a
non-null pointer statically...

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Nov 06 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Kagamin:


 It's hard to tell, whether they fix anything. When you cast nullable to
non-nullable, you get your runtime exception as usual, if you if out access to
nullable (e.g. in delayed method), you get your runtime exception again or
rather logical bug.


I think you are missing about half of the ideas tied to this nonnullable


Bye,
bearophile
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 I think you are missing about half of the ideas tied to this nonnullable






overflow."

Oh well!
Nov 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:



arithmetic
 overflow."
 
 Oh well!





function/program doesn't overflow.

Well, there is a way to know, you can use that "useless" site:
http://www.rise4fun.com/SpecSharp

And try this snippet:

class Example {
  int x;

  void Inc(int y)
    requires y > 0;
    ensures old(x) < x;
  {
    checked { x += y; }
  }
}



error the things it doesn't understand).
This also shows you why an online way to test programs is useful.

Anyway, the topic of this whole tread is about non-nullable types in D, copying

shown).

Walter, instead of poking and teasing me as a ten year old does, why we don't
start talking about serious things? Like, for example if we want nonnull
references/pointers in D3. If you aren't interested in this feature we can stop
wasting my time. It's not a necessary feature, I will keep using D2 even
without it. But it may be something nice to have.

Bye,
bearophile
Nov 05 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Walter, instead of poking and teasing me as a ten year old does, why we don't
 start talking about serious things? Like, for example if we want nonnull
 references/pointers in D3. If you aren't interested in this feature we can
 stop wasting my time. It's not a necessary feature, I will keep using D2 even
 without it. But it may be something nice to have.



I've explained my position and rationale on null many, many times here.

tl;dr: null serves the same role as the NaN does for floating point types. It 
means an invalid value (not an uninitialized value). Invalid values are highly 
useful as bug detectors (usually mistaken for being a bug). I wish there were a 
NaN value we could use for integers. Without a null, one is reduced to having
to 
craft a default value that throws exceptions when accessed, ending up in the 
same place.

To eliminate null pointers is the same as shooting the canary in your coal mine 
because its twitter annoys you.
Nov 05 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:


 I've explained my position and rationale on null many, many times here.


You are right. And sorry for losing my temper a little :-)
Thank you for your answer.

Bye,
bearophile
Nov 05 2010
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:

 To eliminate null pointers is the same as shooting the canary in your  
 coal mine because its twitter annoys you.


I'm tired of pointing out that NO ONE is talking about eliminating null  
pointers, but rather extending an existing type system to support  
non-nulls. Your hate towards non-nullables comes from misunderstanding of  
the concept.
Nov 05 2010
next sibling parent Jimmy Cao <jcao219 gmail.com> writes:
On Fri, Nov 5, 2010 at 4:31 PM, Denis Koroskin <2korden gmail.com> wrote:


 On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright <
 newshound2 digitalmars.com> wrote:


 To eliminate null pointers is the same as shooting the canary in your coal
 mine because its twitter annoys you.


 I'm tired of pointing out that NO ONE is talking about eliminating null
 pointers, but rather extending an existing type system to support non-nulls.
 Your hate towards non-nullables comes from misunderstanding of the concept.



I've been thinking of Vala while reading this thread and its ideas.
In Vala one puts a question-mark after a type to mark it as nullable, as all
types are non-nullable by default.
For instance, instance of T? can be of type T or null.
The idea is that D could have it, but the other way around.
Could there be support for explicitly specifying that an instance of T can
only be T, never null?
Nov 05 2010
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/5/10 4:31 PM, Denis Koroskin wrote:

 On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:

 To eliminate null pointers is the same as shooting the canary in your
 coal mine because its twitter annoys you.


 I'm tired of pointing out that NO ONE is talking about eliminating null
 pointers, but rather extending an existing type system to support
 non-nulls. Your hate towards non-nullables comes from misunderstanding
 of the concept.


I agree it's a big misunderstanding, in fact a couple of layered ones.

Andrei
Nov 05 2010
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Denis Koroskin:


 I'm tired of pointing out that NO ONE is talking about eliminating null  
 pointers, but rather extending an existing type system to support  
 non-nulls. Your hate towards non-nullables comes from misunderstanding of  
 the concept.


The nullables/nonnullables topic is not basic stuff, but it's not a too much
complex thing. In my bug report I have explained about adding both the ? and  
suffix to reference/pointer types (? is for the nullables), so Walter knows and
understands what you are saying.

A temporary experimental fork of DMD, with nullable types + null path analysis

arrays of nonnull references, with a kind of loop variant that in many cases
avoids testing all items again) may be created to try these ideas
experimentally in D (despite I think it's not a feature for D2, it's for D3).
This may show how much bad or good that extension is. But it's a lot of work.

Bye,
bearophile
Nov 05 2010
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Denis Koroskin wrote:

 On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright 
 <newshound2 digitalmars.com> wrote:

 To eliminate null pointers is the same as shooting the canary in your 
 coal mine because its twitter annoys you.


 
 I'm tired of pointing out that NO ONE is talking about eliminating null 
 pointers, but rather extending an existing type system to support 
 non-nulls. Your hate towards non-nullables comes from misunderstanding 
 of the concept.


Consider non-nullable type T:

   T[] a = new T[4];
   ... time goes by ...
   T[1] = foo;
   T[3] = bar;
   ... more time goes by ...
   bar(T[2]);

In other words, I create an array that I mean to fill in later, because I don't 
have meaningful data for it in advance. What do I use to default initialize it 
with non-nullable data? And once I do that, should bar(T[2]) be an error? How 
would I detect the error?

In general, for a non-nullable type, how would I mark an instance as not having 
meaningful data?

For example, an int is a non-nullable type. But there's no int value that means 
"no meaningful value", and this can hide an awful lot of bugs.

I'm not sure at all that non-nullable types do more than make easy to find bugs 
much, much harder to find.
Nov 05 2010
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
A non-nullable type is basically just:

struct NotNull(T) {
   T _payload;
   alias _payload this;
   invariant() {
       assert(_payload !is null);
   }
}

If we could disable the default constructor, this approach might just work.
Sure,
the checks here would be at runtime, but you'd have them without having to
manually write the assert each time.

(Though, having to write NotNull! each time isn't that much of a gain over the
asserts, but maybe it would help.)
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Adam D. Ruppe wrote:

 A non-nullable type is basically just:
 
 struct NotNull(T) {
    T _payload;
    alias _payload this;
    invariant() {
        assert(_payload !is null);
    }
 }
 
 If we could disable the default constructor, this approach might just work.
Sure,
 the checks here would be at runtime, but you'd have them without having to
 manually write the assert each time.


All that does is reinvent the null pointer seg fault. The hardware does this
for 
you for free.

A null pointer is what's known as a "white hole", all attempts to access the 
object are rejected.
Nov 05 2010
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
 All that does is reinvent the null pointer seg fault.
 The hardware does this for you for free.


The difference is that the hardware tells you when you use it,
whereas the assert tells you when you try to save it.

Let me give you an example from my real world code. I have an xml dom class
that I
use in my websites. An element looks like this:

class Element {
   Element[] children;
/* // I didn't write this invariant originally, but always should be there!
   invariant() {
        foreach(child; children)
             assert(child !is null);
   }
*/
   string toString() {
        string ret = "<blah>";
        foreach(child; children)
            ret ~= child.toString();
        return ret ~ "</blah>";
   }
}


Now, let's use it somewhere for a long time. That children list grows and
shrinks
as I manipulate the document.

Then, when I'm all done, I say:

writeln(myElement.toString());


And the hardware only now throws its null pointer segfault, all the way at the
very end of the program!

But that's not where the real problem was. Somewhere, I did children ~=
someFunction(); and someFunction returned null for
whatever reason (I don't remember the specifics anymore, but it took almost a
full
day to track down!)


I spent most the day tracking this down and the real problem was around line 200
out of the 3500 line program, but the hardware didn't complain until line 3490!



It wasn't until I added the invariant and in/out contracts to all the functions
asserting about null that the problem's true cause became apparent.
Nov 05 2010
next sibling parent Roman Ivanov <isroman.DEL ETE.km.ru> writes:
On 11/5/2010 8:51 PM, Adam D. Ruppe wrote:

 All that does is reinvent the null pointer seg fault.
 The hardware does this for you for free.


 
 The difference is that the hardware tells you when you use it,
 whereas the assert tells you when you try to save it.
 
 Let me give you an example from my real world code. I have an xml dom class
that I
 use in my websites. An element looks like this:
 
 class Element {
    Element[] children;
 /* // I didn't write this invariant originally, but always should be there!
    invariant() {
         foreach(child; children)
              assert(child !is null);
    }
 */
    string toString() {
         string ret = "<blah>";
         foreach(child; children)
             ret ~= child.toString();
         return ret ~ "</blah>";
    }
 }
 
 
 Now, let's use it somewhere for a long time. That children list grows and
shrinks
 as I manipulate the document.
 
 Then, when I'm all done, I say:
 
 writeln(myElement.toString());
 
 
 And the hardware only now throws its null pointer segfault, all the way at the
 very end of the program!
 
 But that's not where the real problem was. Somewhere, I did children ~=
 someFunction(); and someFunction returned null for
 whatever reason (I don't remember the specifics anymore, but it took almost a
full
 day to track down!)
 
 
 I spent most the day tracking this down and the real problem was around line
200
 out of the 3500 line program, but the hardware didn't complain until line 3490!
 
 
 
 It wasn't until I added the invariant and in/out contracts to all the functions
 asserting about null that the problem's true cause became apparent.


Yep, that's exactly the kind of stuff that makes me wish for non-nulls.
I've been in the same situation many times.

And that's not even the worst case scenario. In the worst case scenario
the inadvertent null gets propagated to a 3d party library that uses it
in some way that doesn't result in program termination, but changes the
logic.
Nov 05 2010
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Adam D. Ruppe wrote:

 It wasn't until I added the invariant and in/out contracts to all the functions
 asserting about null that the problem's true cause became apparent.


Couldn't this happen to you with any datum that has an unexpected value in it?

Suppose, for example, you are appending the numbers 1..5 to the array, and 
somehow appended a 17. Many moons later, something crashes because the 17 was 
out of range.
Nov 06 2010
next sibling parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 11/6/2010 02:42, Walter Bright wrote:

 Adam D. Ruppe wrote:

 It wasn't until I added the invariant and in/out contracts to all the
 functions
 asserting about null that the problem's true cause became apparent.


 
 Couldn't this happen to you with any datum that has an unexpected value
 in it?
 
 Suppose, for example, you are appending the numbers 1..5 to the array,
 and somehow appended a 17. Many moons later, something crashes because
 the 17 was out of range.


That's an argument for limited-range data types, not against
non-nullable types.


-- 
Rainer Deyke - rainerd eldwood.com
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Rainer Deyke wrote:

 On 11/6/2010 02:42, Walter Bright wrote:

 Adam D. Ruppe wrote:

 It wasn't until I added the invariant and in/out contracts to all the
 functions
 asserting about null that the problem's true cause became apparent.


 Couldn't this happen to you with any datum that has an unexpected value
 in it?

 Suppose, for example, you are appending the numbers 1..5 to the array,
 and somehow appended a 17. Many moons later, something crashes because
 the 17 was out of range.


 
 That's an argument for limited-range data types, not against
 non-nullable types.


I see it more as an argument for any restricted subset of a data type. It could 
be that you want just odd numbers in the array, and an even one crept in.
Nov 06 2010
parent reply Roman Ivanov <isroman.DEL ETE.km.ru> writes:
On 11/6/2010 2:14 PM, Walter Bright wrote:

 Rainer Deyke wrote:

 On 11/6/2010 02:42, Walter Bright wrote:

 Adam D. Ruppe wrote:

 It wasn't until I added the invariant and in/out contracts to all the
 functions
 asserting about null that the problem's true cause became apparent.


 Couldn't this happen to you with any datum that has an unexpected value
 in it?

 Suppose, for example, you are appending the numbers 1..5 to the array,
 and somehow appended a 17. Many moons later, something crashes because
 the 17 was out of range.


 That's an argument for limited-range data types, not against
 non-nullable types.


 
 I see it more as an argument for any restricted subset of a data type.
 It could be that you want just odd numbers in the array, and an even one
 crept in.


Not quite. I don't want to make an argument in favor of general subsets
of data types. (IMO it can be made, but that's a topic of a different
discussion.) I just want to point out that nulls are a special case,
because they allow anyone to bypass (effortlessly, often unknowingly!)
the type checks enforced by class system, which is probably the most
important part of object-oriented programming.

The problem you outlined above can be solved by creating a class that
represents an odd number. But without unnulable types the representation
will never be precise, because any variables of that type could (in
addition to all the odd numbers) also be null.
Nov 06 2010
parent tls <do notha.ev> writes:
Roman Ivanov Wrote:


 On 11/6/2010 2:14 PM, Walter Bright wrote:

 Rainer Deyke wrote:

 On 11/6/2010 02:42, Walter Bright wrote:

 Adam D. Ruppe wrote:

 It wasn't until I added the invariant and in/out contracts to all the
 functions
 asserting about null that the problem's true cause became apparent.


 Couldn't this happen to you with any datum that has an unexpected value
 in it?

 Suppose, for example, you are appending the numbers 1..5 to the array,
 and somehow appended a 17. Many moons later, something crashes because
 the 17 was out of range.


 That's an argument for limited-range data types, not against
 non-nullable types.


 
 I see it more as an argument for any restricted subset of a data type.
 It could be that you want just odd numbers in the array, and an even one
 crept in.


 
 Not quite. I don't want to make an argument in favor of general subsets
 of data types. (IMO it can be made, but that's a topic of a different
 discussion.) I just want to point out that nulls are a special case,
 because they allow anyone to bypass (effortlessly, often unknowingly!)
 the type checks enforced by class system, which is probably the most
 important part of object-oriented programming.


The silent break when nulls is on purpose right? To make coding easy and fun. I
not want test null everywhere. School taught Java and remember nonnull always
makes stupid tests and exceptions :-(
Nov 06 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:


 Suppose, for example, you are appending the numbers 1..5 to the array, and 
 somehow appended a 17. Many moons later, something crashes because the 17 was 
 out of range.


This bug doesn't happen in Ada (and Delphi), because you define a ranged
integer type like this:
subtype Small is Integer range 1 .. 5;

To see an example of this, look at this "useless" site where I have put a small
Ada program that you may run yourself, with some stdin data:
http://ideone.com/shF5j

I try to put inside the "inputx" variable the following numbers:
1
2
1
4
17

But at runtime it gives:
raised CONSTRAINT_ERROR : prog.adb:9 range check failed

Bye,
bearophile
Nov 06 2010
next sibling parent retard <re tard.com.invalid> writes:
Sat, 06 Nov 2010 05:04:59 -0400, bearophile wrote:


 To see an example of this, look at this "useless" site


I have ambivalent feelings.. you're beginning to find the joys of irony, 
but losing the joys of D :) It's done awful things to me.
Nov 06 2010
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Suppose, for example, you are appending the numbers 1..5 to the array, and 
 somehow appended a 17. Many moons later, something crashes because the 17 was 
 out of range.


 
 This bug doesn't happen in Ada (and Delphi), because you define a ranged
integer type like this:
 subtype Small is Integer range 1 .. 5;


Since you're the second to misunderstand me in the same way, I obviously wasn't 
very clear at all. Allow me to restate:

Suppose, for example, you are appending prime numbers to the array, and
somehow appended a 25. Many moons later, something crashes because the 25 was
not prime.
Nov 06 2010
next sibling parent reply Adam Burton <adz21c gmail.com> writes:
Walter Bright wrote:


 bearophile wrote:

 Suppose, for example, you are appending the numbers 1..5 to the array,
 and somehow appended a 17. Many moons later, something crashes because
 the 17 was out of range.


 
 This bug doesn't happen in Ada (and Delphi), because you define a ranged
 integer type like this: subtype Small is Integer range 1 .. 5;


 
 Since you're the second to misunderstand me in the same way, I obviously
 wasn't very clear at all. Allow me to restate:
 
 Suppose, for example, you are appending prime numbers to the array, and
 somehow appended a 25. Many moons later, something crashes because the 25
 was not prime.


I wouldn't consider that as the same thing. null represents the lack of a 
value where as 25 is the wrong value. Based on that argument the application 
should fail immediately on accessing the item with 25 (not many moons later) 
in the same manner it does nulls, but it doesn't because 25 is the wrong 
value where as null is a lack of value.

As with the array allocation example earlier you initialise the array to 
nulls to represent the lack of value till your application eventually gets 
values to assign to the array (which may still be wrong values). As shown by 
my alternative example non-nulls allow you to define that a 
variable/parameter wants a value and does not work when it receives nothing. 
However in the case of the array because all the information is not there at 
the point of creation it is valid for the array items to represent nothing 
till you have something to put in them.
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Adam Burton wrote:

 I wouldn't consider that as the same thing. null represents the lack of a 
 value where as 25 is the wrong value. Based on that argument the application 
 should fail immediately on accessing the item with 25 (not many moons later) 
 in the same manner it does nulls, but it doesn't because 25 is the wrong 
 value where as null is a lack of value.
 
 As with the array allocation example earlier you initialise the array to 
 nulls to represent the lack of value till your application eventually gets 
 values to assign to the array (which may still be wrong values). As shown by 
 my alternative example non-nulls allow you to define that a 
 variable/parameter wants a value and does not work when it receives nothing. 
 However in the case of the array because all the information is not there at 
 the point of creation it is valid for the array items to represent nothing 
 till you have something to put in them.



I am having a real hard time explaining this. It is conceptually *the same 
thing*, which is having an enforced subset of the values of a type.
Nov 06 2010
next sibling parent reply Gary Whatmore <no spam.sp> writes:
Walter Bright Wrote:


 Adam Burton wrote:

 I wouldn't consider that as the same thing. null represents the lack of a 
 value where as 25 is the wrong value. Based on that argument the application 
 should fail immediately on accessing the item with 25 (not many moons later) 
 in the same manner it does nulls, but it doesn't because 25 is the wrong 
 value where as null is a lack of value.
 
 As with the array allocation example earlier you initialise the array to 
 nulls to represent the lack of value till your application eventually gets 
 values to assign to the array (which may still be wrong values). As shown by 
 my alternative example non-nulls allow you to define that a 
 variable/parameter wants a value and does not work when it receives nothing. 
 However in the case of the array because all the information is not there at 
 the point of creation it is valid for the array items to represent nothing 
 till you have something to put in them.


 
 
 I am having a real hard time explaining this. It is conceptually *the same 
 thing*, which is having an enforced subset of the values of a type.


I'm seeing it. The other arguments for non-null types also fall short because
non-nulls don't solve basic problems like arrays, basic collections in the
library (custom fill policy). Has any mainstream language adopted non-null
types? No they haven't because the idea is broken.
Nov 06 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Gary Whatmore:


 I'm seeing it. The other arguments for non-null types also fall short because
non-nulls don't solve basic problems like arrays, basic collections in the
library (custom fill policy).


Arrays and collections are a subset of the nonull problem I am discussing




 Has any mainstream language adopted non-null types?


Ada has them:
http://en.wikibooks.org/wiki/Ada_Programming/Types/access#Null_exclusions

Most functional languages are based on nonullables.

The GNU version of C has a limited form of them:
http://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html#index-g_t_0040code_007bnonnull_007d-function-attribute-2458

The problem is faced by Eiffel too:
http://docs.eiffel.com/book/papers/void-safety-how-eiffel-removes-null-pointer-dereferencing



 No they haven't because the idea is broken.


I don't think so.

Bye,
bearophile
Nov 06 2010
parent Gary Whatmore <no spam.sp> writes:
bearophile Wrote:


 Gary Whatmore:
 

 I'm seeing it. The other arguments for non-null types also fall short because
non-nulls don't solve basic problems like arrays, basic collections in the
library (custom fill policy).


 
 Arrays and collections are a subset of the nonull problem I am discussing

 
 

 Has any mainstream language adopted non-null types?


 
 Ada has them:
 http://en.wikibooks.org/wiki/Ada_Programming/Types/access#Null_exclusions
 
 Most functional languages are based on nonullables.
 
 The GNU version of C has a limited form of them:
 http://gcc.gnu.org/onlinedocs/gcc/Function-Attributes.html#index-g_t_0040code_007bnonnull_007d-function-attribute-2458
 
 The problem is faced by Eiffel too:
 http://docs.eiffel.com/book/papers/void-safety-how-eiffel-removes-null-pointer-dereferencing


Can't force me to read. I've made my choice of using D already. Nothing will
change that. 


 No they haven't because the idea is broken.


 
 I don't think so.


Yes they are.
Nov 06 2010
prev sibling next sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Gary Whatmore <no spam.sp> wrote:


 non-nulls don't solve basic problems like arrays, basic collections in
 the library (custom fill policy).


As I've posted elsewhere, arrays (and other collections) will have to
work slightly differently for non-nullable elements:

1 .length is read-only (or read/decrease only)
2 Length can be increased by concatenation only
3 Can only be created with a nonzero length with a runtime check of a
   nullable-element array/collection, or as a literal containing only
   non-nullable elements.

And yes, there may be cases where this is not good enough, and you need
to have nullable elements. That does not mean non-nullable types are
borked by default, you just use nullable elements, and accept that you

allow you to create your collection using nullable elements, and casting
it to non-nullable afterwards.



 Can't force me to read. I've made my choice of using D already. Nothing  
 will change that.


He's not trying to make you change language, but to inform you of how
other languages have solved the problem.

-- 
Simen
Nov 06 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Simen kjaeraas:


 3 Can only be created with a nonzero length with a runtime check of a
    nullable-element array/collection, or as a literal containing only
    non-nullable elements.
 ...

 allow you to create your collection using nullable elements, and casting
 it to non-nullable afterwards.



is done on the same variable. So you don't need to do:

class T {...}
T[5] tempArray; // nulls allowed, the same as T?[5]
// creates items of tempArray
T [5] array = cast( )tempArray;


That's not too much bad. But it requires two different variables.




class T {...}
T [5] arr; // nulls not allowed
// creation phase of array, nulls are allowed inside arr
// but you can't read arr contents
arr.done(); // imaginary syntax
// Now arr is not immutable, you may append nonnulls to
// it, replace it items with nonnulls, etc.


To allow this, the compiler has to see the type of arr as different before and
after the call to the "done" function/method/attribute.

This is not different from allowing the usage of assert() to change the state

typestate):

void foo(T  x) {...}
class T {...}
T t = something(); // nullable, the same as T?
assert(t !is null); // using D syntax
foo(t); // no cast to   required here because after
        // the assert the type system knows t is not null


Something similar happens in the branch of this if:

void foo(T  x) {...}
class T {...}
T t = something(); // nullable, the same as T?
if (t !is null) {
  foo(t); // no cast to   required here
}

Bye,
bearophile
Nov 06 2010
prev sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 06 Nov 2010 23:37:09 +0300, Gary Whatmore <no spam.sp> wrote:


 Walter Bright Wrote:


 Adam Burton wrote:

 I wouldn't consider that as the same thing. null represents the lack  


 of a

 value where as 25 is the wrong value. Based on that argument the  


 application

 should fail immediately on accessing the item with 25 (not many moons  


 later)

 in the same manner it does nulls, but it doesn't because 25 is the  


 wrong

 value where as null is a lack of value.

 As with the array allocation example earlier you initialise the array  


 to

 nulls to represent the lack of value till your application eventually  


 gets

 values to assign to the array (which may still be wrong values). As  


 shown by

 my alternative example non-nulls allow you to define that a
 variable/parameter wants a value and does not work when it receives  


 nothing.

 However in the case of the array because all the information is not  


 there at

 the point of creation it is valid for the array items to represent  


 nothing

 till you have something to put in them.



 I am having a real hard time explaining this. It is conceptually *the  
 same
 thing*, which is having an enforced subset of the values of a type.


 I'm seeing it. The other arguments for non-null types also fall short  
 because non-nulls don't solve basic problems like arrays, basic  
 collections in the library (custom fill policy). Has any mainstream  
 language adopted non-null types? No they haven't because the idea is  
 broken.


That's not because the concept is broken, it's because mainstream  
languages were developed before it became clear that non-nullables are a  
must. Although the concept itself is an old one, it's wasn't very popular  
until recently.


to reference types, but found no way to do so without breaking existing  
code.

See also  
http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare
Nov 07 2010
prev sibling next sibling parent reply Adam Burton <adz21c gmail.com> writes:
Walter Bright wrote:


 Adam Burton wrote:

 I wouldn't consider that as the same thing. null represents the lack of a
 value where as 25 is the wrong value. Based on that argument the
 application should fail immediately on accessing the item with 25 (not
 many moons later) in the same manner it does nulls, but it doesn't
 because 25 is the wrong value where as null is a lack of value.
 
 As with the array allocation example earlier you initialise the array to
 nulls to represent the lack of value till your application eventually
 gets values to assign to the array (which may still be wrong values). As
 shown by my alternative example non-nulls allow you to define that a
 variable/parameter wants a value and does not work when it receives
 nothing. However in the case of the array because all the information is
 not there at the point of creation it is valid for the array items to
 represent nothing till you have something to put in them.


 
 
 I am having a real hard time explaining this. It is conceptually *the same
 thing*, which is having an enforced subset of the values of a type.


I *think* I undertand what you mean, however I also think I failed to 
explain myself correctly. After reading bearophiles post I think he put it 
in much better terms of null being a common case that is handled in common 
way. The value 25 requires far more context for the compiler to identify it 
as a bad value and determine how the developer should handle it. null can 
still cause logic errors in the same way 25 can (which could lead to the 
application outputting bad data or crashing) but there is one particular 
case that always results in the application behaving in an unexpected manner 
(attempting to use a null reference on a class type) that makes it different 
and the compiler can help by enforcing checks. Without non-nullables 
enforcing that check would be tiresome as it is going to happen all over the 
place, with non-nullables you actually get the oppurtunity to reduce the 
amount of checks while also helping the developer check when they should be.

The non-nullable propsal doesn't eliminate NPE but it allows you to narrow 
the locations of the error. I know most of my code doesn't use nulls (you 
might say I avoid them like the plague) but sometimes code calls for the use 
of nulls, so if I can reduce the amount of code where a null can go then I 
have much less code to debug.
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Adam Burton wrote:

 I *think* I undertand what you mean, however I also think I failed to 
 explain myself correctly. After reading bearophiles post I think he put it 
 in much better terms of null being a common case that is handled in common 
 way.


I don't see that non-null is such a special case that it would benefit from a 
special case syntax. For example:

     NonNull!T p;

v.s.

     nonnull T p;

or worse:

      nonnull T p;



 The value 25 requires far more context for the compiler to identify it 
 as a bad value and determine how the developer should handle it.


The compiler wouldn't do it. It would be a user supplied filter for a user 
defined subtype.



 The non-nullable propsal doesn't eliminate NPE but it allows you to narrow 
 the locations of the error. I know most of my code doesn't use nulls (you 
 might say I avoid them like the plague) but sometimes code calls for the use 
 of nulls, so if I can reduce the amount of code where a null can go then I 
 have much less code to debug.


The issue of null is only a small part of the possible space of subtyping
checks 
one can find desirable in a program.
Nov 06 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:


 I don't see that non-null is such a special case that it would benefit from a 
 special case syntax.


Well, nonnull are a special cases because:
- There is a good enough way to solve this problem. This is not true in the
general case.
- null exceptions are very common bugs, among the most common ones in Java and




 For example:
 
      NonNull!T p;
 
 v.s.
 
      nonnull T p;
 
 or worse:
 
       nonnull T p;


The syntax I have proposed is:
T  p;



 The compiler wouldn't do it. It would be a user supplied filter for a user 
 defined subtype.
 ...
 The issue of null is only a small part of the possible space of subtyping
checks 
 one can find desirable in a program.


Are you using enough OOP in your programs? In OOP programs you manage object
references all the time, so finding a null where you expect an object is common
enough.

The idea of "a user supplied filter for a user defined subtype" looks nice, but
in practice has problems because:
- It faces a general problem, while the most common problem in OOP code is much
more restricted (presence of nulls).
- It doesn't allow a short light syntax like the   and ? suffixes as I have
proposed.
- And third, and most importantly, the compiler can't perform the second part
of enforcement I was talking about. If we limit ourselves to just nulls, you
may use many ways to tell the compiler that a specific reference can't be null:

If (x == null) {
  // in this branch you do something
} else {
  // in this branch the compiler assumes x is a nonnull type!
}

Or you may add an explicit assert:

// some code...
assert(x != null);
// here the compiler seex x as a nonnullable type
// more code...


Or even a cast, that performs a run-time test:

(cast( )x).someMethod();

Where cast( )x is syntax sugar for cast(typepof(x) )x.

I don't know if you may use those three ways if you try to generalize the
enforcement of a generic predicate attached to a type (there is a name for such
types, we aren't inventing anything in this thread).

Bye,
bearophile
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 I don't see that non-null is such a special case that it would benefit from a 
 special case syntax.


 
 Well, nonnull are a special cases because:
 - There is a good enough way to solve this problem. This is not true in the
general case.
 - null exceptions are very common bugs, among the most common ones in Java and




Any type having an unexpected value in it is a very common bug. They often go 
unnoticed, though, because they don't generate a seg fault. The bug is still 
there, though.

So no, I don't agree it is a special case.
Nov 06 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:


 Any type having an unexpected value in it is a very common bug. They often go 
 unnoticed, though, because they don't generate a seg fault. The bug is still 
 there, though.
 
 So no, I don't agree it is a special case.


Then to D3, beside nonnull types, we may also add ranged integers (as integer
subtypes or subtypes of other ranges). With both those things you have probably
covered a good percentage of what you call "unxpected value" bugs.

Bye,
bearophile
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Walter:
 

 Any type having an unexpected value in it is a very common bug. They often
 go unnoticed, though, because they don't generate a seg fault. The bug is
 still there, though.
 
 So no, I don't agree it is a special case.


 
 Then to D3, beside nonnull types, we may also add ranged integers (as integer
 subtypes or subtypes of other ranges). With both those things you have
 probably covered a good percentage of what you call "unxpected value" bugs.



Adding ranged integer types increases the coverage from 1% to 2% of the cases. 
(Pulling random numbers out of the ether, but still, I think the point is
valid.)
Nov 06 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:


 Adding ranged integer types increases the coverage from 1% to 2% of the cases. 
 (Pulling random numbers out of the ether, but still, I think the point is
valid.)


I don't know if that point is valid. SPARK language is able to have a testable
low bug count, and it only has nonnulls and ranged integers. But it also uses
other unrelated features, like a very precise and defined semantics, design by

outer variables are able to see, plus it has few other safety features (like
not allowing recursion, restricting pointer usage a lot, and so on). So despite
SPARK doesn't seem to need more than those two kinds of value constraints
(nonnulls and ranged integers) I don't know how many bugs those two features
alone avoid and how many are left to be caught to the other safety features. My
hypothesis is that those two often suffice. But indeed I can't be sure. In an
ideal world I'd like to add them to D and then use D for few months and see how
much less bugs I put in my code (I keep an updated list of all my bugs).

Bye,
bearophile
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Walter Bright:
 

 Adding ranged integer types increases the coverage from 1% to 2% of the
 cases. (Pulling random numbers out of the ether, but still, I think the
 point is valid.)


 
 I don't know if that point is valid. SPARK language is able to have a
 testable low bug count, and it only has nonnulls and ranged integers. But it
 also uses other unrelated features, like a very precise and defined

 need to list what outer variables are able to see, plus it has few other
 safety features (like not allowing recursion, restricting pointer usage a
 lot, and so on). So despite SPARK doesn't seem to need more than those two
 kinds of value constraints (nonnulls and ranged integers) I don't know how
 many bugs those two features alone avoid and how many are left to be caught
 to the other safety features. My hypothesis is that those two often suffice.
 But indeed I can't be sure. In an ideal world I'd like to add them to D and
 then use D for few months and see how much less bugs I put in my code (I keep
 an updated list of all my bugs).


Very, very few (if any) dmd issues on bugzilla would have been caught by ranged 
integers or non-null pointers (despite there being several seg fault bugs).

The vast majority of the problems were the result of an incomplete
understanding 
of how things should be done, rather than a coding error. (A coding error being 
things as simple as a typo or forgetting to initialize something, aka "stupid 
mistakes".)

I think that's in no small part due to my programming for a very long time, and 
I've developed all kinds of strategies for avoiding the "stupid mistake" kinds 
of coding errors.

As Andrei suggested out to me, and I agree, the notions of non-null types, 
ranged integers, and the like's primary advantage is not in reducing the bug 
count, but in enhancing the self-documenting nature of the code. It makes the 
code easier to reason about, and easier to modify.
Nov 06 2010
parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Walter Bright wrote:

=20
 Very, very few (if any) dmd issues on bugzilla would have been caught b=


y

 ranged integers or non-null pointers (despite there being several seg
 fault bugs).
=20
 The vast majority of the problems were the result of an incomplete
 understanding of how things should be done, rather than a coding error.=



 (A coding error being things as simple as a typo or forgetting to
 initialize something, aka "stupid mistakes".)
=20
 I think that's in no small part due to my programming for a very long
 time, and I've developed all kinds of strategies for avoiding the
 "stupid mistake" kinds of coding errors.
=20
 As Andrei suggested out to me, and I agree, the notions of non-null
 types, ranged integers, and the like's primary advantage is not in
 reducing the bug count, but in enhancing the self-documenting nature of=



 the code. It makes the code easier to reason about, and easier to modif=


y.

	Do you consider yourself an average programmer? I wouldn't. You say
it yourself: you have a lot of experience at avoiding "stupid
mistakes" and the same probably goes for Andrei as well. The vast
majority of programmers out there are not at that level (in fact, I
would venture a guess that the average level of the people who
participate in this newsgroup is way above the global average). As
someone who has some experience in managing beginner programmers,
I'm pretty sure that non-null pointers and ranged integers would
reduce the bug count *for the average programmer*.

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Nov 07 2010
prev sibling next sibling parent Adam Burton <adz21c gmail.com> writes:
Walter Bright wrote:


 Adam Burton wrote:

 I *think* I undertand what you mean, however I also think I failed to
 explain myself correctly. After reading bearophiles post I think he put
 it in much better terms of null being a common case that is handled in
 common way.


 
 I don't see that non-null is such a special case that it would benefit
 from a special case syntax.


Unfortunately I do seeing it as special. I can see why you don't and if D 
doesn't get non-nulls it is not like I am gonna run away crying but in my 
opinion it is different to values like 25.

 For example:
 
      NonNull!T p;
 
 v.s.
 
      nonnull T p;
 
 or worse:
 
       nonnull T p;
 


If someone is clever enough to figure out a way to do it without changing 
the compiler then that works for me too. It's not the idea of a compiler 
change I'd personally want, it's the feature. However as far as I can see 
the best way to implement it is a compiler change.

 

 The value 25 requires far more context for the compiler to identify it
 as a bad value and determine how the developer should handle it.


 
 The compiler wouldn't do it. It would be a user supplied filter for a user
 defined subtype.
 


Agreed. However null is language defined and forced on all user defined 
class types. Most of the time I don't want/need it and having it entails 
more work in one way or another. Sometimes I do want/need it but that's a 
much smaller case than when I don't so I'd personally like to say when I 
want it.

 

 The non-nullable propsal doesn't eliminate NPE but it allows you to
 narrow the locations of the error. I know most of my code doesn't use
 nulls (you might say I avoid them like the plague) but sometimes code
 calls for the use of nulls, so if I can reduce the amount of code where a
 null can go then I have much less code to debug.


 
 The issue of null is only a small part of the possible space of subtyping
 checks one can find desirable in a program.


Agreed. If a resolution can be identified for the root problem then that is 
even better. While we can't I don't see why this common check can't be 
*almost* eliminated.
Nov 06 2010
prev sibling next sibling parent Rainer Deyke <rainerd eldwood.com> writes:
On 11/6/2010 15:00, Walter Bright wrote:

 I don't see that non-null is such a special case that it would benefit
 from a special case syntax.


I'm less concerned about syntax and more about semantics.  I already use
nn_ptr<T> (short for non-null pointer) in C++.  It works in C++ because
types in C++ are not assumed to have a default constructor.


-- 
Rainer Deyke - rainerd eldwood.com
Nov 06 2010
prev sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Walter Bright <newshound2 digitalmars.com> wrote:


 I don't see that non-null is such a special case that it would benefit  
 from a special case syntax. For example:


There are some things that set non-null apart:

1. Few operations change the value of a pointer/class reference:
    assignment
    pointer arithmetic
    ref passing
Of these, the last two can be disallowed in most cases. For e.g. an
int, operator overloading adds bunches of more cases, and for structs/
classes where fields factor into the calculation, every single public
member function and field would need to be wrapped. Simply put, null
pointers are low-hanging fruit.

2. There is a single invalid value
This is a simplification, of course, as you could argue that a reference
to an A is not a valid reference to a B, and pointers may point to
arbitrary areas of memory, where nothing of interest may be found.
Again, low-hanging fruit.



 The value 25 requires far more context for the compiler to identify it  
 as a bad value and determine how the developer should handle it.


 The compiler wouldn't do it. It would be a user supplied filter for a  
 user defined subtype.


Definitely a possibility, also for non-nullable. However, for such a
system to be foolproof (and that's what we want), we need a way to
disable the default constructor for structs.

There is also the problem of arrays, as I expounded on in a different
post. Without language support, this will be a hole in the system.

-- 
Simen
Nov 06 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 06/11/2010 19:57, Walter Bright wrote:

 Adam Burton wrote:

 I wouldn't consider that as the same thing. null represents the lack
 of a value where as 25 is the wrong value. Based on that argument the
 application should fail immediately on accessing the item with 25 (not
 many moons later) in the same manner it does nulls, but it doesn't
 because 25 is the wrong value where as null is a lack of value.

 As with the array allocation example earlier you initialise the array
 to nulls to represent the lack of value till your application
 eventually gets values to assign to the array (which may still be
 wrong values). As shown by my alternative example non-nulls allow you
 to define that a variable/parameter wants a value and does not work
 when it receives nothing. However in the case of the array because all
 the information is not there at the point of creation it is valid for
 the array items to represent nothing till you have something to put in
 them.



 I am having a real hard time explaining this. It is conceptually *the
 same thing*, which is having an enforced subset of the values of a type.


Indeed, it is the same thing: to enforce a subset of the values of a 
type (and these are contracts, generally speaking).

So, Adam, if you specify as a contract that a certain variable/value 
should have never 25 on it, and one time it is accessed and it does have 
25 there, then there is a bug, and you should consider your application 
to have failed immediately (even if in practice it likely won't, well, 
not immediately at least).

The only difference in these cases is that wanting to assert something 
to be nonnull (and have the compiler check that) is *much* more common 
than any particular restriction on numeric values. But it is not a 
difference in nature, or concept.


-- 
Bruno Medeiros - Software Engineer
Nov 26 2010
prev sibling next sibling parent reply =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Walter Bright wrote:

 bearophile wrote:

 Suppose, for example, you are appending the numbers 1..5 to the
 array, and somehow appended a 17. Many moons later, something crashes=







 because the 17 was out of range.


 This bug doesn't happen in Ada (and Delphi), because you define a
 ranged integer type like this:
 subtype Small is Integer range 1 .. 5;


=20
 Since you're the second to misunderstand me in the same way, I obviousl=


y

 wasn't very clear at all. Allow me to restate:
=20
 Suppose, for example, you are appending prime numbers to the array, and=



 somehow appended a 25. Many moons later, something crashes because the
 25 was
 not prime.


	I thought D was supposed to be a pragmatic language? All this means
is that we need a compromise between what would be ideal (being able
to represent arbitrary conditions and have them statically enforced
at compile time) and what can be realistically achieved. Your
argument seems to be: "since there are cases which we won't be able
to handle, let's not do anything at all, not even the cases which we
could handle".

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
Jérôme M. Berger wrote:

 	I thought D was supposed to be a pragmatic language? All this means
 is that we need a compromise between what would be ideal (being able
 to represent arbitrary conditions and have them statically enforced
 at compile time) and what can be realistically achieved. Your
 argument seems to be: "since there are cases which we won't be able
 to handle, let's not do anything at all, not even the cases which we
 could handle".



The idea is to find a way to solve this in the general case, not the specific 
case of non-null pointers. Then we've got dynamite in our hands, rather than 
warmed over toast.
Nov 06 2010
next sibling parent reply Roman Ivanov <isroman.DEL ETE.km.ru> writes:
On 11/6/2010 4:02 PM, Walter Bright wrote:

 Jérôme M. Berger wrote:

     I thought D was supposed to be a pragmatic language? All this means
 is that we need a compromise between what would be ideal (being able
 to represent arbitrary conditions and have them statically enforced
 at compile time) and what can be realistically achieved. Your
 argument seems to be: "since there are cases which we won't be able
 to handle, let's not do anything at all, not even the cases which we
 could handle".


 
 
 The idea is to find a way to solve this in the general case, not the
 specific case of non-null pointers. Then we've got dynamite in our
 hands, rather than warmed over toast.


But it's not a specific case if you look at it from OOP perspective!

The problem of restricting types was solved in OOP. Every time you
inherit a class, you make it more specific. This allows you to restrict
types all you want, and the compiler enforces your restrictions.

For example, you can create a class that represents an even number and
accepts only even numbers in its constructor (trowing exception
otherwise). You can then extend it and create a class that only accepts
every second even number, and so on. Since D is a statically typed
language, attempts to assign arbitrary class to a variable intended for
Even objects is caught at compile time (instead of waiting until someone
references a missing method or attribute at run-time).

Null is the only wrench in the gears here.
Nov 06 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Roman Ivanov wrote:

 But it's not a specific case if you look at it from OOP perspective!
 
 The problem of restricting types was solved in OOP. Every time you
 inherit a class, you make it more specific. This allows you to restrict
 types all you want, and the compiler enforces your restrictions.
 
 For example, you can create a class that represents an even number and
 accepts only even numbers in its constructor (trowing exception
 otherwise). You can then extend it and create a class that only accepts
 every second even number, and so on. Since D is a statically typed
 language, attempts to assign arbitrary class to a variable intended for
 Even objects is caught at compile time (instead of waiting until someone
 references a missing method or attribute at run-time).
 
 Null is the only wrench in the gears here.



Your insight that this is related to OOP is right on target. It's polymorphism, 
but a different kind of polymorphism than class OOP tries to solve.
Nov 06 2010
prev sibling parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
On 06/11/2010 20:02, Walter Bright wrote:

 Jérôme M. Berger wrote:

 I thought D was supposed to be a pragmatic language? All this means
 is that we need a compromise between what would be ideal (being able
 to represent arbitrary conditions and have them statically enforced
 at compile time) and what can be realistically achieved. Your
 argument seems to be: "since there are cases which we won't be able
 to handle, let's not do anything at all, not even the cases which we
 could handle".



 The idea is to find a way to solve this in the general case, not the
 specific case of non-null pointers. Then we've got dynamite in our
 hands, rather than warmed over toast.


Indeed. I agree that the use-case for non-null has some value, and we 
should try to look for a general solution, instead of adding specific 
syntax for it.
Not only because D seems very close to be able to do this with library 
code only, but also because solving this in the general way might allow 
plenty of other interesting type specifications to be defined. For 
example, with those changes to struct behavior, I wonder how close we 
could be to directly specify a type such as Java's wildcards (the <? 
extends Foo> thingy), that would work with arrays or other containers.

-- 
Bruno Medeiros - Software Engineer
Nov 26 2010
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Walter:


 Suppose, for example, you are appending prime numbers to the array, and
 somehow appended a 25. Many moons later, something crashes because the 25 was
 not prime.


There are two main ways to avoid this bug:
- Put some test in the code, such test may be an explicit test at the point
where you add items to the array (and test that this test works in the unittest
of this function), or add a precondition (Contract) that tests the array
contents are calling time of the function that inserts the data. Generally in D
I now use both unittests and DbC.
- Wrap your data inside a struct (that contains an alias this) that has a
invariant that makes sure your wrapped numbers are prime (there are type
systems that allow you to attach a run-time function like isPrime() to the type
system, to allow you to define a type that allows only prime numbers).



 I see it more as an argument for any restricted subset of a data type. It could
 be that you want just odd numbers in the array, and an even one crept in.


If I understand what you are meaning, you are trying to say that:
nonnulls aren't a general solution, because in general what are "correct" and
"wrong" items for an array is a generic predicate, like isPrime(), so having a
built-in feature to enforce isNotNull() on a type is useless because it's just
a special case.

In theory this is true. But it's not the best way to design a language.
Experience shows that while your items may need to satisfy a generic predicate,
there are many many many situations where the predicate you want to enforce is
isNotNull(Sometype) (or isIntegerInsideBound(x,low,hi)). If such enforcement is
so common in programs (and if you look at Java code you may see several if ==
null tests), then the wise thing to do is to put in the language handy
shortcuts that allow the programmer this specific case. This means adding the
?/  type annotations I have proposed.

As you say, in some situations those ?/  type annotations are useless because
you must assure your integer numbers inside the array are prime, but practice
shows this is a less common case in a Object Priented language, and even if
it's not common, you can't define a simple syntax to enforce it.

You may create a syntax to attach generic predicates to a subset type, using a
kind of invariant, similar to this (this is syntax sugar for a struct with
alias this + struct invariant):

typedef int PrimeInt invariant() { return isPrime(this); }

This may be useful, but it's not as handy and simple as the isNull()

there are several ways to use it as nonnullable.

If (x == null) {
  // in this branch you do something
} else {
  // in this branch the compiler assumes x is a nonnull type!
}

Or you may add an explicit assert:

// some code...
assert(x != null);
// here the compiler seex x as a nonnullable type
// more code...


Or even a cast, that performs a run-time test:

(cast( )x).someMethod();

Where cast( )x is syntax sugar for cast(typepof(x) )x.

Bye,
bearophile
Nov 06 2010
prev sibling next sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Walter Bright <newshound2 digitalmars.com> wrote:


 Adam D. Ruppe wrote:

 It wasn't until I added the invariant and in/out contracts to all the  
 functions
 asserting about null that the problem's true cause became apparent.


 Couldn't this happen to you with any datum that has an unexpected value  
 in it?


Yes. And by augmenting the type system (such as with non-nullable
pointers or bounded integers), such a mistake could be caught early,
likely at compile time, because the bounded type or non-null pointer does
not allow assignment from arbitrary pointers or values.



 Suppose, for example, you are appending the numbers 1..5 to the array,  
 and somehow appended a 17. Many moons later, something crashes because  
 the 17 was out of range.


Bounded!(int, 1, 5)[] myArr;
myArr ~= 1; // Compile-time error: int is not implicitly castable to  
Bounded!(int, 1, 5)
myArr ~= Bounded!(int, 1, 5)( 1 ); // Works perfectly


-- 
Simen
Nov 06 2010
next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Simen kjaeraas:


 Bounded!(int, 1, 5)[] myArr;
 myArr ~= 1; // Compile-time error: int is not implicitly castable to  
 Bounded!(int, 1, 5)
 myArr ~= Bounded!(int, 1, 5)( 1 ); // Works perfectly


I can't say this is a nice syntax :-)
(Isn't the implicit conversion + bound test better?)

Bye,
bearophile
Nov 06 2010
parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
bearophile <bearophileHUGS lycos.com> wrote:


 Simen kjaeraas:


 Bounded!(int, 1, 5)[] myArr;
 myArr ~= 1; // Compile-time error: int is not implicitly castable to
 Bounded!(int, 1, 5)
 myArr ~= Bounded!(int, 1, 5)( 1 ); // Works perfectly


 I can't say this is a nice syntax :-)
 (Isn't the implicit conversion + bound test better?)


Not sure. This way is more explicit, and errors will be caught at
compile-time.

-- 
Simen
Nov 06 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Simen kjaeraas:


 Not sure. This way is more explicit, and errors will be caught at
 compile-time.


I see. But if a syntax is ugly and too much heavy, people (rightly) don't use
it... (This is why bounded values are good as builtins).

Bye,
bearophile
Nov 06 2010
parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
bearophile <bearophileHUGS lycos.com> wrote:


 Simen kjaeraas:


 Not sure. This way is more explicit, and errors will be caught at
 compile-time.


 I see. But if a syntax is ugly and too much heavy, people (rightly)  
 don't use it... (This is why bounded values are good as builtins).


Of course. Now, aliases help a bit here, turning that into something
like:

alias Bounded!(int, 1, 5) myInt;

myInt[] myArr;
myArr ~= myInt( 1 );


I believe using an alias would be a good idea in most such cases, as
one's bound to write Bounded!(int,1,6) in a long program, and
wonder why it doesn't compile.

-- 
Simen
Nov 06 2010
next sibling parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Simen kjaeraas <simen.kjaras gmail.com> wrote:


 bearophile <bearophileHUGS lycos.com> wrote:


 Simen kjaeraas:


 Not sure. This way is more explicit, and errors will be caught at
 compile-time.


 I see. But if a syntax is ugly and too much heavy, people (rightly)  
 don't use it... (This is why bounded values are good as builtins).


 Of course. Now, aliases help a bit here, turning that into something
 like:

 alias Bounded!(int, 1, 5) myInt;

 myInt[] myArr;
 myArr ~= myInt( 1 );


 I believe using an alias would be a good idea in most such cases, as
 one's bound to write Bounded!(int,1,6) in a long program, and
 wonder why it doesn't compile.


Worth adding too: myInt(648492) is of course not catchable at compile
time, while such would be easy with builtin bounded integers.


-- 
Simen
Nov 06 2010
prev sibling parent so <so so.do> writes:
Hah people bringing up the argument "bad syntax", when they got nothing to  
say!
Like they type "Matrix!(double, 5, 5)" every time they use a matrix
no you never do that, you just:
alias Matrix!(double, 5, 5) mat5; // sweet isn't it? Remember! it is a  
type!

On Sat, 06 Nov 2010 12:27:34 +0200, Simen kjaeraas  
<simen.kjaras gmail.com> wrote:


 bearophile <bearophileHUGS lycos.com> wrote:


 Simen kjaeraas:


 Not sure. This way is more explicit, and errors will be caught at
 compile-time.


 I see. But if a syntax is ugly and too much heavy, people (rightly)  
 don't use it... (This is why bounded values are good as builtins).


 Of course. Now, aliases help a bit here, turning that into something
 like:

 alias Bounded!(int, 1, 5) myInt;

 myInt[] myArr;
 myArr ~= myInt( 1 );


 I believe using an alias would be a good idea in most such cases, as
 one's bound to write Bounded!(int,1,6) in a long program, and
 wonder why it doesn't compile.



-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 07 2010
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Simen kjaeraas wrote:

 Bounded!(int, 1, 5)[] myArr;
 myArr ~= 1; // Compile-time error: int is not implicitly castable to 
 Bounded!(int, 1, 5)
 myArr ~= Bounded!(int, 1, 5)( 1 ); // Works perfectly


Yes, that's it.
Nov 06 2010
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
Walter said:

 Couldn't this happen to you with any datum that has an unexpected value in it?


Yes, indeed. And being able to disable struct default constructors would (I
believe anyway) let us fix that in the library too, by forcing initialization
upon
declaration.

Right now, we could write a struct that does runtime checks to ensure its
payload
is always in range, with the exception of one line:

Bounded!(10, 100) myInt; // can't do the check

In this specific case, the bounded template could change the default
init value for its payload, arbitrarily picking some value in range. But you
can't
do that with a NotNull struct - there is no default value that's usable, except
for null, which we're aiming to avoid!

If the default constructor were  disable-able, these structs could say:

Bounded!(10, 100) myInt; // compile error

Bounded!(10, 100) myInt = 10; // calls Bounded.this(10), passing the compile
time
check, and allowing us to do the runtime check - it automatically runs the
struct's invariant!

The very same thing allows a not null struct - it forces explicit
initialization,
which gives the invariant a chance to run, thus catching the error instantly at
runtime.


While compile time checks are probably more ideal, most of this works in the
language today, so I think it's a pretty conservative change that enables a few
simple library solutions to this whole range of situations.
Nov 06 2010
prev sibling next sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Walter Bright <newshound2 digitalmars.com> wrote:


 Adam D. Ruppe wrote:

 A non-nullable type is basically just:
  struct NotNull(T) {
    T _payload;
    alias _payload this;
    invariant() {
        assert(_payload !is null);
    }
 }
  If we could disable the default constructor, this approach might just  
 work. Sure,
 the checks here would be at runtime, but you'd have them without having  
 to
 manually write the assert each time.


 All that does is reinvent the null pointer seg fault. The hardware does  
 this for you for free.

 A null pointer is what's known as a "white hole", all attempts to access  
 the object are rejected.


I believe you managed to miss the important part, "If we could disable the
default constructor". The invariant here is not about using the NotNull!T,
but for when you assign to it. In other words:

struct NotNull( T ) if ( is( T == class ) || isPointer!T ) {
     private T _payload;
     alias _payload this;
     this( T data ) {
         assert( data !is null );
         _payload = data;
     }
      disable this( );
      disable void opAssign( T );
}

NotNull!T notNull( T )( T ptr ) {
     return NotNull!T( ptr );
}

It might not cover all bases, but the idea is it is impossible to create a
NotNull!T from a null T, and impossible to assign a null T to one.

-- 
Simen
Nov 05 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
Simen kjaeraas wrote:

 I believe you managed to miss the important part, "If we could disable the
 default constructor".


Yes, I believe you are correct.
Nov 06 2010
prev sibling parent reply FeepingCreature <default_357-line yahoo.de> writes:
Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware does this
for 
 you for free.


Walter, I know you're a Windows programmer but this cannot be the first time
somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS UNDER LINUX.

Access violations are not a cross-platform substitute for exceptions.
Nov 06 2010
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
FeepingCreature wrote:

 Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware does
 this for you for free.


 
 Walter, I know you're a Windows programmer but this cannot be the first time
 somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS UNDER LINUX.
 
 Access violations are not a cross-platform substitute for exceptions.


Why would you want to recover from a seg fault?

(asserts in D are not recoverable exceptions)
Nov 06 2010
parent reply FeepingCreature <default_357-line yahoo.de> writes:
Walter Bright Wrote:


 FeepingCreature wrote:

 Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware does
 this for you for free.


 
 Walter, I know you're a Windows programmer but this cannot be the first time
 somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS UNDER LINUX.
 
 Access violations are not a cross-platform substitute for exceptions.


 
 Why would you want to recover from a seg fault?
 
 (asserts in D are not recoverable exceptions)


You cannot do a lot of things from a signal handler. You can't sjlj, which
means sjlj exception handling is out (as are, in fact, most methods of
exception handling). This means stack traces are out unless you have special
handling for segfaults that decodes the stack and prints the error pos. That in
turn means you need to have a debugger attached to get stacktraces, which can
be annoying especially in long-running programs where the crash is often the
first indication you have of a problem.

Furthermore, scope guards will not be run. You can't run the GC from a signal
handler because it requires pthreads which is not safe for signal handlers.
Garbage collected classes will not be freed. Static class destructors cannot be
run. Module destructures can not be safely executed.

Can I stop?
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
FeepingCreature wrote:

 Walter Bright Wrote:
 

 FeepingCreature wrote:

 Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware does
  this for you for free.


 Walter, I know you're a Windows programmer but this cannot be the first
 time somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS
 UNDER LINUX.
 
 Access violations are not a cross-platform substitute for exceptions.


 Why would you want to recover from a seg fault?
 
 (asserts in D are not recoverable exceptions)


 
 You cannot do a lot of things from a signal handler. You can't sjlj, which
 means sjlj exception handling is out (as are, in fact, most methods of
 exception handling). This means stack traces are out unless you have special
 handling for segfaults that decodes the stack and prints the error pos. That
 in turn means you need to have a debugger attached to get stacktraces, which
 can be annoying especially in long-running programs where the crash is often
 the first indication you have of a problem.
 
 Furthermore, scope guards will not be run. You can't run the GC from a signal
 handler because it requires pthreads which is not safe for signal handlers.
 Garbage collected classes will not be freed. Static class destructors cannot
 be run. Module destructures can not be safely executed.


Sure. That's why assert errors are not recoverable.
Nov 06 2010
next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Sun, 07 Nov 2010 00:58:57 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:


 FeepingCreature wrote:

 Walter Bright Wrote:


 FeepingCreature wrote:

 Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware  
 does
  this for you for free.


 Walter, I know you're a Windows programmer but this cannot be the  
 first
 time somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS
 UNDER LINUX.
  Access violations are not a cross-platform substitute for exceptions.


 Why would you want to recover from a seg fault?
  (asserts in D are not recoverable exceptions)


  You cannot do a lot of things from a signal handler. You can't sjlj,  
 which
 means sjlj exception handling is out (as are, in fact, most methods of
 exception handling). This means stack traces are out unless you have  
 special
 handling for segfaults that decodes the stack and prints the error pos.  
 That
 in turn means you need to have a debugger attached to get stacktraces,  
 which
 can be annoying especially in long-running programs where the crash is  
 often
 the first indication you have of a problem.
  Furthermore, scope guards will not be run. You can't run the GC from a  
 signal
 handler because it requires pthreads which is not safe for signal  
 handlers.
 Garbage collected classes will not be freed. Static class destructors  
 cannot
 be run. Module destructures can not be safely executed.


 Sure. That's why assert errors are not recoverable.


And that's why modern languages are designed to *prevent* segfaults from  
happening using non-nullable types.
D has nothing to prevent them, and that's why everyone here is pushing  
this concept so hard.

D has its power in compile-time facilities. Non-nullables will only make  
this power stronger.
Nov 07 2010
prev sibling parent FeepingCreature <default_357-line yahoo.de> writes:
Walter Bright Wrote:


 FeepingCreature wrote:

 Walter Bright Wrote:
 

 FeepingCreature wrote:

 Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware does
  this for you for free.


 Walter, I know you're a Windows programmer but this cannot be the first
 time somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS
 UNDER LINUX.
 
 Access violations are not a cross-platform substitute for exceptions.


 Why would you want to recover from a seg fault?
 
 (asserts in D are not recoverable exceptions)


 
 You cannot do a lot of things from a signal handler. You can't sjlj, which
 means sjlj exception handling is out (as are, in fact, most methods of
 exception handling). This means stack traces are out unless you have special
 handling for segfaults that decodes the stack and prints the error pos. That
 in turn means you need to have a debugger attached to get stacktraces, which
 can be annoying especially in long-running programs where the crash is often
 the first indication you have of a problem.
 
 Furthermore, scope guards will not be run. You can't run the GC from a signal
 handler because it requires pthreads which is not safe for signal handlers.
 Garbage collected classes will not be freed. Static class destructors cannot
 be run. Module destructures can not be safely executed.


 
 Sure. That's why assert errors are not recoverable.
 


I don't see how that addresses what I said in the slightest.
Nov 07 2010
prev sibling parent reply =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
FeepingCreature wrote:

 Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware doe=




s this for=20

 you for free.


=20
 Walter, I know you're a Windows programmer but this cannot be the first=


 time somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS UND=
ER LINUX.

=20
 Access violations are not a cross-platform substitute for exceptions.


	I really, really hope that you can't recover from seg faults on
Windows (or MacOS for that matter). Segmentation faults should be
non recoverable: once the program has started accessing memory that
it shouldn't, there is no way to guarantee that the program is in a
coherent state. Walter has stated this several times and I agree
with him 100% on this.

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Nov 06 2010
parent retard <re tard.com.invalid> writes:
Sat, 06 Nov 2010 21:52:22 +0100, Jérôme M. Berger wrote:


 FeepingCreature wrote:

 Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware
 does this for you for free.


 
 Walter, I know you're a Windows programmer but this cannot be the first
 time somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS
 UNDER LINUX.
 
 Access violations are not a cross-platform substitute for exceptions.


 
 	I really, really hope that you can't recover from seg faults on
 Windows (or MacOS for that matter). Segmentation faults should be non
 recoverable: once the program has started accessing memory that it
 shouldn't, there is no way to guarantee that the program is in a
 coherent state. Walter has stated this several times and I agree with
 him 100% on this.


I also agree with him 100% on this.

The problem non-nullable types try to solve is they reduce the number of 
possible segfaults in the first place. I don't care how hard the program 
crashes. I just don't want my client to experience that ever. Segfaults 
are terribly common in C/C++ code. I experience those every week. My 
initial motivation to study/use D was born from these crashes. I can't 
believe you're not using every possible known mechanism to prevent them.
Nov 06 2010
prev sibling next sibling parent =?UTF-8?B?UGVsbGUgTcOlbnNzb24=?= <pelle.mansson gmail.com> writes:
On 11/06/2010 12:41 AM, Walter Bright wrote:

 Denis Koroskin wrote:

 On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:

 To eliminate null pointers is the same as shooting the canary in your
 coal mine because its twitter annoys you.


 I'm tired of pointing out that NO ONE is talking about eliminating
 null pointers, but rather extending an existing type system to support
 non-nulls. Your hate towards non-nullables comes from misunderstanding
 of the concept.


 Consider non-nullable type T:

 T[] a = new T[4];
 ... time goes by ...
 T[1] = foo;
 T[3] = bar;
 ... more time goes by ...
 bar(T[2]);

 In other words, I create an array that I mean to fill in later, because
 I don't have meaningful data for it in advance. What do I use to default
 initialize it with non-nullable data? And once I do that, should
 bar(T[2]) be an error? How would I detect the error?

 In general, for a non-nullable type, how would I mark an instance as not
 having meaningful data?

 For example, an int is a non-nullable type. But there's no int value
 that means "no meaningful value", and this can hide an awful lot of bugs.

 I'm not sure at all that non-nullable types do more than make easy to
 find bugs much, much harder to find.


I tried to find a good analogy but failed, so I'll just say that in the 
case you presented you would obviously not use a non-nullable type. As, 
you know, you wanted nulls in the array.

We're using signalling nans to get rid of the null-ish thing for floats, 
as well.
Nov 05 2010
prev sibling next sibling parent Adam Burton <adz21c gmail.com> writes:
Walter Bright wrote:


 Denis Koroskin wrote:

 On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright
 <newshound2 digitalmars.com> wrote:

 To eliminate null pointers is the same as shooting the canary in your
 coal mine because its twitter annoys you.


 
 I'm tired of pointing out that NO ONE is talking about eliminating null
 pointers, but rather extending an existing type system to support
 non-nulls. Your hate towards non-nullables comes from misunderstanding
 of the concept.


 
 Consider non-nullable type T:
 
    T[] a = new T[4];
    ... time goes by ...
    T[1] = foo;
    T[3] = bar;
    ... more time goes by ...
    bar(T[2]);
 
 In other words, I create an array that I mean to fill in later, because I
 don't have meaningful data for it in advance. What do I use to default
 initialize it with non-nullable data? And once I do that, should bar(T[2])
 be an error? How would I detect the error?
 
 In general, for a non-nullable type, how would I mark an instance as not
 having meaningful data?
 
 For example, an int is a non-nullable type. But there's no int value that
 means "no meaningful value", and this can hide an awful lot of bugs.
 
 I'm not sure at all that non-nullable types do more than make easy to find
 bugs much, much harder to find.


My understanding on what everyone is requesting is below.

class T
{
    private string x;
    public this(string xin) { x = xin; }
    public void doSomething() { writeln(x); }
}

//////Currently////////

function doSomething(T myT)
in
{
    Assert(myT is not null);	// NPE during debugging on a[3]
}
{
   myT.doSomething();	// "1", "2", "null", NPE during release
}

T[] a = new T[4];
a[0] = new T("1");
a[1] = new T("2");
a[2] = new T(null);

foreach(i; a)
   doSomething(i);

////// With NonNullables. Assume toNonNullable() is override to allow 
nullables to assign to nonNullables ////

function doSomething(T myT)	// the in is now redundant, function 
signature says no nulls allowed
{
   myT.doSomething();
}

T?[] a = new T?[4];	// Nullable array items
a[0] = new T("1");
a[1] = new T("2");
a[3] = new T(null);	// Compile error, null assignment to non-nullable

foreach(i; a)
   doSomething(i);	// Compile error potential null assignment to non-
nullable, please check for null (e.g. if statement) or override

foreach(i; a)
   doSomething(toNonNullable(i));	// "1", "2", NPE (toNonNullable checks for 
null and NPE if it is null. However we fail here when attempting to send a 
null into code that does not allow nulls instead of later.

In theory the altered doSomething is saying "don't give me meaningless 
data". Where as the array says "at this point I don't know enough so I'll 
take nulls in the mean time". However what we have done is shrink the amount 
of code a null can live in offering more guarantuees about doSomething.

Am I going along the right lines with that?
Nov 05 2010
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Friday, November 05, 2010 16:41:25 Walter Bright wrote:

 Denis Koroskin wrote:

 On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright
 
 <newshound2 digitalmars.com> wrote:

 To eliminate null pointers is the same as shooting the canary in your
 coal mine because its twitter annoys you.


 
 I'm tired of pointing out that NO ONE is talking about eliminating null
 pointers, but rather extending an existing type system to support
 non-nulls. Your hate towards non-nullables comes from misunderstanding
 of the concept.


 
 Consider non-nullable type T:
 
    T[] a = new T[4];
    ... time goes by ...
    T[1] = foo;
    T[3] = bar;
    ... more time goes by ...
    bar(T[2]);
 
 In other words, I create an array that I mean to fill in later, because I
 don't have meaningful data for it in advance. What do I use to default
 initialize it with non-nullable data? And once I do that, should bar(T[2])
 be an error? How would I detect the error?
 
 In general, for a non-nullable type, how would I mark an instance as not
 having meaningful data?
 
 For example, an int is a non-nullable type. But there's no int value that
 means "no meaningful value", and this can hide an awful lot of bugs.
 
 I'm not sure at all that non-nullable types do more than make easy to find
 bugs much, much harder to find.


I thought that the point of non-nullable types were that they _always_ had to 
have meaningful data. I certainly see no point in a type which just uses a
value 
other than null to indicate that it doesn't have meaningful data. That's just 
another type of null.

Now, I think that your example is a perfect example of one of the reasons why 
you  do _not_ want to have to use non-nullable types. There are cases where you 
_know_ that a variable should never be null, and having that guarantee can be 
very useful. However, there are plenty of other cases where you _need_ to be 
able to have a variable be null, even if it isn't null normally. So, while I 
wouldn't necessarily be opposed to adding support for non-nullable references 
and/or pointers to D, I would very much be opposed to making _all_ pointers or 
references be non-nullable. For that matter, I'd be opposed to it even becoming 
the default.

So, I can understand wanting non-nullable pointers and references, and I'm not 
really opposed to them being supported, but I wouldn't want to have to use them 
or have to bend over backwards to get properly nullable pointers and references.

- Jonathan M Davis
Nov 05 2010
prev sibling next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 06 Nov 2010 02:41:25 +0300, Walter Bright  
<newshound2 digitalmars.com> wrote:


 Denis Koroskin wrote:

 On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright  
 <newshound2 digitalmars.com> wrote:

 To eliminate null pointers is the same as shooting the canary in your  
 coal mine because its twitter annoys you.


  I'm tired of pointing out that NO ONE is talking about eliminating  
 null pointers, but rather extending an existing type system to support  
 non-nulls. Your hate towards non-nullables comes from misunderstanding  
 of the concept.


 Consider non-nullable type T:

    T[] a = new T[4];
    ... time goes by ...
    T[1] = foo;
    T[3] = bar;
    ... more time goes by ...
    bar(T[2]);

 In other words, I create an array that I mean to fill in later, because  
 I don't have meaningful data for it in advance. What do I use to default  
 initialize it with non-nullable data? And once I do that, should  
 bar(T[2]) be an error? How would I detect the error?


How is T non-nullable? By the time you call bar(T[2]) T[2] is not  
initialized yet, i.e. null. As such, T[]
This is clearly not a use-case for non-nullables. In your use case this  
array contains un-initialized values for some time. Non-nullables means  
that there is no such state as unitialized. As such, you use T?[], not T[].

If you need an array of elements that are never null, use the following:

T[] a = [new T1(), new T2(), new T3()];

or the following:

T?[] nullables = ...;
// setup a

T[] nonNullables = toNonnull(nullables); // or cast(T[])nullables;

similar to the way you create mutable objects and then cast to immutable.


 In general, for a non-nullable type, how would I mark an instance as not  
 having meaningful data?


Err... what? By definition, a non-nullable is an object that can NEVER  
hold meaningless data. If T must contain meaningless data, use T?, not T.


 For example, an int is a non-nullable type. But there's no int value  
 that means "no meaningful value", and this can hide an awful lot of bugs.


Use int?, which is a special type that has an additional "not initialized  
yet" value. This comes at a cost of a larger size though - there is simply  
no other way to do it.


 I'm not sure at all that non-nullable types do more than make easy to  
 find bugs much, much harder to find.


You are simply wrong. You clearly misunderstand the concept.
Nov 05 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Fri, 05 Nov 2010 16:41:25 -0700
Walter Bright <newshound2 digitalmars.com> wrote:


 Consider non-nullable type T:
=20
    T[] a =3D new T[4];
    ... time goes by ...
    T[1] =3D foo;
    T[3] =3D bar;
    ... more time goes by ...
    bar(T[2]);
=20
 In other words, I create an array that I mean to fill in later, because I=


 don't=20

 have meaningful data for it in advance. What do I use to default initiali=


ze it=20

 with non-nullable data? And once I do that, should bar(T[2]) be an error?=


 How=20

 would I detect the error?
=20
 In general, for a non-nullable type, how would I mark an instance as not =


having=20

 meaningful data?
=20
 For example, an int is a non-nullable type. But there's no int value that=


 means=20

 "no meaningful value", and this can hide an awful lot of bugs.
=20
 I'm not sure at all that non-nullable types do more than make easy to fin=


d bugs=20

 much, much harder to find.


Maybe the whole point is rather to catch variables left undefined (with a m=
eaningful value) before use, isn't it?

I was first astonished by D's .init feature. Thought it's a bad idea, as it=
 lets pass through unitialised vars; and worse, except for floats & pointer=
s (maybe more), with a perfectly valid value for the type.
Also, when reading "int i;", there is no way to tell whether the programmer=
 (including myself some time later) actually intends to initialise I with 0=
. So, I always write "int i=3D0;".

I understand there must be something in the var's memory cell, and rather f=
ill it with zeroes or NaN than let it undefined. This aspect of the problem=
 may mean that non-nullable (or more generally never undefined?) types cann=
ot be a solution for languages that need to keep low-level control and effi=
ciency like D.
On a language that can tolerate to be higher-level (from the above pov), wh=
ere for instance all values can be referenced or even wrapped in a struct, =
then possibly there would be a solution. I would try to introduce a bottom =
type called eg UNDEF, which single value is thus valid for all types. This =
would be the unique .init. All operations would fail when passed UNDEF. (Ex=
cept for a func or method expression() intended for programmer feedback.) T=
hen, I guess it would be possible to do it so that each time a programmer t=
ries to use an undefined value, whatever its type, they get "UndefError: at=
tempt to use undefined variable...".

Is this the actual point of non-nullables? After all, we use null only to m=
ean "currently undefined -- don't use me", don't we? Because there must be =
something in memory, and we don't want to let there random bits. If I not t=
otally wrong, then I currently rather agree that non-nullables are not the =
proper tool for D -- but maybe there is no solution at all.
(Maybe I'm completely aside the actual question; but I wanted to ask, so th=
at at least I know ;-)


Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 05 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Sat, 6 Nov 2010 05:01:26 +0100
spir <denis.spir gmail.com> wrote:

Just wanted to add 2 notes.


 In general, for a non-nullable type, how would I mark an instance as no=




t having=20

 meaningful data?
=20
 For example, an int is a non-nullable type. But there's no int value th=




at means=20

 "no meaningful value", and this can hide an awful lot of bugs.
=20
 I'm not sure at all that non-nullable types do more than make easy to f=




ind bugs=20

 much, much harder to find.


=20
 Maybe the whole point is rather to catch variables left undefined (with a=


 meaningful value) before use, isn't it?

=20
 I was first astonished by D's .init feature. Thought it's a bad idea, as =


it lets pass through unitialised vars; and worse, except for floats & point=
ers (maybe more), with a perfectly valid value for the type.

 Also, when reading "int i;", there is no way to tell whether the programm=


er (including myself some time later) actually intends to initialise I with=
 0. So, I always write "int i=3D0;".

=20
 I understand there must be something in the var's memory cell, and rather=


 fill it with zeroes or NaN than let it undefined. This aspect of the probl=
em may mean that non-nullable (or more generally never undefined?) types ca=
nnot be a solution for languages that need to keep low-level control and ef=
ficiency like D.

In that sense, floats are a non-nullable type. That is indeed a very partic=
uliar case: there is a value we can put there in memory, which is strictly =
speaking valid for the type, but invalid for any operation.


 On a language that can tolerate to be higher-level (from the above pov), =


where for instance all values can be referenced or even wrapped in a struct=
, then possibly there would be a solution. I would try to introduce a botto=
m type called eg UNDEF, which single value is thus valid for all types. Thi=
s would be the unique .init. All operations would fail when passed UNDEF. (=
Except for a func or method expression() intended for programmer feedback.)=
 Then, I guess it would be possible to do it so that each time a programmer=
 tries to use an undefined value, whatever its type, they get "UndefError: =
attempt to use undefined variable...".

=20
 Is this the actual point of non-nullables? After all, we use null only to=


 mean "currently undefined -- don't use me", don't we? Because there must b=
e something in memory, and we don't want to let there random bits. If I not=
 totally wrong, then I currently rather agree that non-nullables are not th=
e proper tool for D -- but maybe there is no solution at all.

For ints of all sorts, should we have a kind of NaN? -2.e(N-1) (so that as =
a side effect we get back symmetry). But then there is testing overhead for=
 all arithmetic operations. Or introduce this only optionally, for a non-un=
def type variant intended for super-safe software.


 (Maybe I'm completely aside the actual question; but I wanted to ask, so =


that at least I know ;-)

=20
=20
 Denis
 -- -- -- -- -- -- --
 vit esse estrany =E2=98=A3
=20
 spir.wikidot.com
=20





-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 05 2010
prev sibling next sibling parent spir <denis.spir gmail.com> writes:
On Fri, 05 Nov 2010 16:41:25 -0700
Walter Bright <newshound2 digitalmars.com> wrote:



 In general, for a non-nullable type, how would I mark an instance as not =


having=20

 meaningful data?
 For example, an int is a non-nullable type. But there's no int value that=


 means=20

 "no meaningful value", and this can hide an awful lot of bugs.


In D, a variable can be declared and not explicitely initialised. The probl=
em is then, I guess: what bit pattern in memory should the language use to =
mean "I'm invalid, don't use me"? (-128 for signed bytes?). this is for run=
time checking. NaN, in my opinion, precisely plays this role for floats.
See also (*)

Now, what about static checking? Is it at all possible for a compiler to st=
atically catch every attempt to use a variable of undefined value (whatever=
 its type)? Should symbol tables have a flag? Or how else do that?
Also, how to check all possible control flows pathes? Could there be lingui=
stic means helping in that? I mean, syntactic constraints that would make i=
t easy for the compiler (and the human reader as well!) to obviously know w=
hether a variable is initialised or not at a given point in code? Esp. at f=
unc boundaries (parameters of calls, return values).


Denis


(*) Dynamic languages do not suffer of this issue the same way because one =
cannot introduce a variable without giving it a value; which is good. But t=
hen, programmers still need some placeholder meaning "undef", esp for struc=
tured type fields. For this, languages provide null/nil/none; this conventi=
on is possible because _variables_ are untyped. In this sense, null can pla=
y in those languages the role of element of a bottom type meaning undef I e=
voked in another post.

But a new problem arises due to the fact that those languages let one freel=
y play with null and pass it around like any other value (or more generally=
 assignable element). As a consequence, programmers use it like a meaningfu=
l element loaded with app semantics. Thus, in a sense, highjacking it for t=
heir own use, and breaking the intended purpose of the feature. I guess nul=
l should not be assignable, for instance; but instead the language should p=
rovide a way to introduce yet undefined vars, that would automatically be s=
et to null by the language. (=3D=3D> more or less the role of NaN in D for =
floats.)

When programmers need a special element to carry app semantics (eg "no auth=
or known" in a book database), they should define it especially -- not be a=
ble use null/nil/none instead. I think languages could provide a convenienc=
e type for those thingies (I call them "Marks"); the point is similar to so=
me uses of enums: elements that have a sense but no value properly speaking.

-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 05 2010
prev sibling next sibling parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 11/5/2010 17:41, Walter Bright wrote:

 In other words, I create an array that I mean to fill in later, because
 I don't have meaningful data for it in advance.


That's a faulty idiom.  A data structure that exists but contains no
valid data is a bug waiting to happen - no, it /is/ a bug, even if it
does not yet manifest as incorrect observable behavior.  (Or at best,
it's an unsafe optimization technique that should be wrapped up in an
encapsulating function.)


-- 
Rainer Deyke - rainerd eldwood.com
Nov 05 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Rainer Deyke Wrote:


 That's a faulty idiom.  A data structure that exists but contains no
 valid data is a bug waiting to happen - no, it /is/ a bug, even if it
 does not yet manifest as incorrect observable behavior.  (Or at best,
 it's an unsafe optimization technique that should be wrapped up in an
 encapsulating function.)


Regardless what that is, the presence of nonnull reference types in the
language doesn't forbid you to use that idiom used by Walter, because on
default in D references and pointers will keep being nullable, so the old code
keeps working.

Bye,
bearophile
Nov 05 2010
prev sibling next sibling parent reply spir <denis.spir gmail.com> writes:
On Fri, 05 Nov 2010 23:13:44 -0600
Rainer Deyke <rainerd eldwood.com> wrote:


 On 11/5/2010 17:41, Walter Bright wrote:

 In other words, I create an array that I mean to fill in later, because
 I don't have meaningful data for it in advance.


=20
 That's a faulty idiom.  A data structure that exists but contains no
 valid data is a bug waiting to happen - no, it /is/ a bug, even if it
 does not yet manifest as incorrect observable behavior.  (Or at best,
 it's an unsafe optimization technique that should be wrapped up in an
 encapsulating function.)


You may be right as for local variables. But think at elements of structure=
d data. It constantly happens that one needs to define fields that have no =
meaningful value at startup, maybe even never will on some instances. (Even=
 more in static languages, indeed.) What placeholder can we feed the struct=
 with? Isn't this precisely the hole .init is intended to fill?
What do you think?

Denis
-- -- -- -- -- -- --
vit esse estrany =E2=98=A3

spir.wikidot.com
Nov 06 2010
parent reply Rainer Deyke <rainerd eldwood.com> writes:
On 11/6/2010 01:12, spir wrote:

 On Fri, 05 Nov 2010 23:13:44 -0600 Rainer Deyke <rainerd eldwood.com>
 wrote:

 That's a faulty idiom.  A data structure that exists but contains
 no valid data is a bug waiting to happen - no, it /is/ a bug, even
 if it does not yet manifest as incorrect observable behavior.  (Or
 at best, it's an unsafe optimization technique that should be
 wrapped up in an encapsulating function.)


 
 You may be right as for local variables. But think at elements of
 structured data. It constantly happens that one needs to define
 fields that have no meaningful value at startup, maybe even never
 will on some instances.


It doesn't happen in dynamic languages.  It doesn't happen in pure
functional languages, since these languages provide no way to alter a
data structure after it has been created.  In my experience, it happens
very rarely in C++.

If it happens "constantly" in D, then that's a flaw in the language.


-- 
Rainer Deyke - rainerd eldwood.com
Nov 06 2010
parent retard <re tard.com.invalid> writes:
Sat, 06 Nov 2010 02:48:02 -0600, Rainer Deyke wrote:


 On 11/6/2010 01:12, spir wrote:

 On Fri, 05 Nov 2010 23:13:44 -0600 Rainer Deyke <rainerd eldwood.com>
 wrote:

 That's a faulty idiom.  A data structure that exists but contains no
 valid data is a bug waiting to happen - no, it /is/ a bug, even if it
 does not yet manifest as incorrect observable behavior.  (Or at best,
 it's an unsafe optimization technique that should be wrapped up in an
 encapsulating function.)


 
 You may be right as for local variables. But think at elements of
 structured data. It constantly happens that one needs to define fields
 that have no meaningful value at startup, maybe even never will on some
 instances.


 
 It doesn't happen in dynamic languages.  It doesn't happen in pure
 functional languages, since these languages provide no way to alter a
 data structure after it has been created.  In my experience, it happens
 very rarely in C++.
 
 If it happens "constantly" in D, then that's a flaw in the language.


Scala doesn't have non-nullable references, but the shallow immutability 
(val) of variables makes many sloppy initialization sequences impossible. 
Usually the code ends up being of much higher quality when using vals as 
much as possible.
Nov 06 2010
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
Rainer Deyke wrote:

 On 11/5/2010 17:41, Walter Bright wrote:

 In other words, I create an array that I mean to fill in later, because
 I don't have meaningful data for it in advance.


 
 That's a faulty idiom.  A data structure that exists but contains no
 valid data is a bug waiting to happen - no, it /is/ a bug, even if it
 does not yet manifest as incorrect observable behavior.  (Or at best,
 it's an unsafe optimization technique that should be wrapped up in an
 encapsulating function.)


An example would be the bucket array for a hash table. It starts out initially 
empty, and values get added to it. I have a hard time agreeing that such a 
ubiquitous and useful data structure is a bad idiom.
Nov 06 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Walter:


 An example would be the bucket array for a hash table. It starts out initially 
 empty, and values get added to it. I have a hard time agreeing that such a 
 ubiquitous and useful data structure is a bad idiom.


I agree, there are several situations where nulls are important and very
useful. I have never suggested to remove them.

Bye,
bearophile
Nov 06 2010
prev sibling next sibling parent Leandro Lucarella <luca llucax.com.ar> writes:
Walter Bright, el  6 de noviembre a las 01:47 me escribiste:

 Rainer Deyke wrote:

On 11/5/2010 17:41, Walter Bright wrote:

In other words, I create an array that I mean to fill in later, because
I don't have meaningful data for it in advance.


That's a faulty idiom.  A data structure that exists but contains no
valid data is a bug waiting to happen - no, it /is/ a bug, even if it
does not yet manifest as incorrect observable behavior.  (Or at best,
it's an unsafe optimization technique that should be wrapped up in an
encapsulating function.)


 
 An example would be the bucket array for a hash table. It starts out
 initially empty, and values get added to it. I have a hard time
 agreeing that such a ubiquitous and useful data structure is a bad
 idiom.


In that example, null is *valid* data. Invalid data is when it has no
meaning to your algorithm and in you example null has a very important
meaning.

-- 
Leandro Lucarella (AKA luca)                     http://llucax.com.ar/
----------------------------------------------------------------------
GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
----------------------------------------------------------------------
Un camión lleno de amigos, míos.
Cada uno dando vueltas, en su cabeza.
Mientras yo, sufro la picadura de mi propia abeja.
Nov 06 2010
prev sibling parent Rainer Deyke <rainerd eldwood.com> writes:
On 11/6/2010 02:47, Walter Bright wrote:

 Rainer Deyke wrote:

 On 11/5/2010 17:41, Walter Bright wrote:

 In other words, I create an array that I mean to fill in later, because
 I don't have meaningful data for it in advance.


 That's a faulty idiom.  A data structure that exists but contains no
 valid data is a bug waiting to happen - no, it /is/ a bug, even if it
 does not yet manifest as incorrect observable behavior.  (Or at best,
 it's an unsafe optimization technique that should be wrapped up in an
 encapsulating function.)


 
 An example would be the bucket array for a hash table. It starts out
 initially empty, and values get added to it. I have a hard time agreeing
 that such a ubiquitous and useful data structure is a bad idiom.


Empty is a valid value for a hash table, so that's a completely
different situation.  Obviously the bucket array would not use a
non-nullable type, and less obviously the bucket array should be
explicitly initialized to nulls at creation time.


-- 
Rainer Deyke - rainerd eldwood.com
Nov 06 2010
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 11/6/10 12:13 AM, Rainer Deyke wrote:

 On 11/5/2010 17:41, Walter Bright wrote:

 In other words, I create an array that I mean to fill in later, because
 I don't have meaningful data for it in advance.


 That's a faulty idiom.  A data structure that exists but contains no
 valid data is a bug waiting to happen - no, it /is/ a bug, even if it
 does not yet manifest as incorrect observable behavior.  (Or at best,
 it's an unsafe optimization technique that should be wrapped up in an
 encapsulating function.)


To find an array that always has initialized data, look no further than 
std::vector. There is no way to grow an std::vector without filling it 
with data under user's control. The only place where std::vector assumes 
a default is the resize function:

void vector<T>::resize(size_type newSize, T filler = T());

If that default went away, the user would always be required to provide 
a filler when growing the vector.


Andrei
Nov 06 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:


 To find an array that always has initialized data, look no further than 
 std::vector. There is no way to grow an std::vector without filling it 
 with data under user's control. The only place where std::vector assumes 
 a default is the resize function:
 
 void vector<T>::resize(size_type newSize, T filler = T());
 
 If that default went away, the user would always be required to provide 
 a filler when growing the vector.


In D the array append is not an efficient operation, and in general it's not
handy to limit to just that the ways to create an array of nonnullables. In

divide the array building phase from the phase where you may read the array
items).

Bye,
bearophile
Nov 06 2010
prev sibling next sibling parent =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Walter Bright wrote:

 Consider non-nullable type T:
=20
   T[] a =3D new T[4];
   ... time goes by ...
   T[1] =3D foo;
   T[3] =3D bar;
   ... more time goes by ...
   bar(T[2]);
=20
 In other words, I create an array that I mean to fill in later, because=



 I don't have meaningful data for it in advance. What do I use to defaul=


t

 initialize it with non-nullable data? And once I do that, should
 bar(T[2]) be an error? How would I detect the error?
=20


<sarcasm>
Consider immutable type immutable T:

  immutable T] a =3D new immutable T[4];
  ... time goes by ...
  T[1] =3D foo;
  T[3] =3D bar;

In other words I create an array that I mean to fill in later,
because I don't have meaningful data for it in advance. How do I do
that with immutable types?
</sarcasm>

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Nov 06 2010
prev sibling parent reply "Simen kjaeraas" <simen.kjaras gmail.com> writes:
Walter Bright <newshound2 digitalmars.com> wrote:


 Consider non-nullable type T:

    T[] a = new T[4];


As others have pointed out, this would be impossible for a proper
non-nullable type. The only ways to create an array of non-nullable
elements would be a literal, concatenation with non-nullable elements,
or casting an existing array of nullable elements.

You do not have write access to the length of an array of non-nullable
elements, so that you cannot increase the length (thus adding null
elements). You can decrease the length by slicing.

This way, it should be impossible to have elements of our array that are
null.

-- 
Simen
Nov 06 2010
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Simen kjaeraas:


 As others have pointed out, this would be impossible for a proper
 non-nullable type. The only ways to create an array of non-nullable
 elements would be a literal, concatenation with non-nullable elements,
 or casting an existing array of nullable elements.
 
 You do not have write access to the length of an array of non-nullable
 elements, so that you cannot increase the length (thus adding null
 elements). You can decrease the length by slicing.
 
 This way, it should be impossible to have elements of our array that are
 null.


That's right in theory, but in practice if you want more efficiency, some other

http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=121140

In practice the creation of the array is in two phases, in a phase the array
can't be read, and its items can't be used, but they are allowed to be null.
Then you use something to tell the compiler that the array is committed. Then
you are free to use your array of nonnulls. At that moment the array is mutable
still, but you may only assign its elements with nonnulls (using a runtime cast
or just the right item of nonnull type), or you may append an nonnull item to
the end of the array.

Bye,
bearophile
Nov 06 2010
prev sibling parent reply retard <re tard.com.invalid> writes:
Sat, 06 Nov 2010 10:20:24 +0100, Simen kjaeraas wrote:


 Walter Bright <newshound2 digitalmars.com> wrote:
 

 Consider non-nullable type T:

    T[] a = new T[4];


 
 As others have pointed out, this would be impossible for a proper
 non-nullable type. The only ways to create an array of non-nullable
 elements would be a literal, concatenation with non-nullable elements,
 or casting an existing array of nullable elements.


That's bs.. the functional way to doing this is to wrap all elements in a 
Maybe monad. It makes the "null check" explicit.
Nov 06 2010
parent reply Lutger <lutger.blijdestijn gmail.com> writes:
retard wrote:


 Sat, 06 Nov 2010 10:20:24 +0100, Simen kjaeraas wrote:
 

 Walter Bright <newshound2 digitalmars.com> wrote:
 

 Consider non-nullable type T:

    T[] a = new T[4];


 
 As others have pointed out, this would be impossible for a proper
 non-nullable type. The only ways to create an array of non-nullable
 elements would be a literal, concatenation with non-nullable elements,
 or casting an existing array of nullable elements.


 
 That's bs.. the functional way to doing this is to wrap all elements in a
 Maybe monad. It makes the "null check" explicit.


Isn't a list of Maybe T a functional way to express the nullable side 
effect, rather than express non-nullable types? After all, it is typed as 
Maybe T, not T. There is a code path for nil in the monadic case right, but 
not for nullable types. Or do I completely miss the point?
Nov 06 2010
parent reply retard <re tard.com.invalid> writes:
Sat, 06 Nov 2010 13:18:34 +0100, Lutger wrote:


 retard wrote:
 

 Sat, 06 Nov 2010 10:20:24 +0100, Simen kjaeraas wrote:
 

 Walter Bright <newshound2 digitalmars.com> wrote:
 

 Consider non-nullable type T:

    T[] a = new T[4];


 
 As others have pointed out, this would be impossible for a proper
 non-nullable type. The only ways to create an array of non-nullable
 elements would be a literal, concatenation with non-nullable elements,
 or casting an existing array of nullable elements.


 
 That's bs.. the functional way to doing this is to wrap all elements in
 a Maybe monad. It makes the "null check" explicit.


 
 Isn't a list of Maybe T a functional way to express the nullable side
 effect, rather than express non-nullable types? After all, it is typed
 as Maybe T, not T. There is a code path for nil in the monadic case
 right, but not for nullable types. Or do I completely miss the point?


Right, the type 'T' expresses the basic case where you don't have any 
null/nil values - the non-nullable case. Everything must be initialized 
with a proper value (lazy initialization is possible, though). The type 
'Maybe T' adds the special null case. In D when a function receives a 
nullable argument, you must do:

class Car { void start() {} }

void start_the_car(Car c) {
  if (c != null)
    c.start();
  else
    throw new Error("The car wasn't initialized!");
}

In a functional language:

start_the_car c = case c of
  Just car -> start car
  Nothing -> error "not initialized"

----

With non-nullable types:

class Car { void start() {} }


  c.start();
}

In a functional language:

start_the_car = start
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
retard wrote:

 In a functional language:
 
 start_the_car c = case c of
   Just car -> start car
   Nothing -> error "not initialized"


And the null pointer exception is reinvented!
Nov 06 2010
next sibling parent retard <re tard.com.invalid> writes:
Sat, 06 Nov 2010 11:24:01 -0700, Walter Bright wrote:


 retard wrote:

 In a functional language:
 
 start_the_car c = case c of
   Just car -> start car
   Nothing -> error "not initialized"


 
 And the null pointer exception is reinvented!


What was the point of my post again? To be an inspiration for stupid 
remarks?
Nov 06 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:


 retard wrote:

 In a functional language:
 
 start_the_car c = case c of
   Just car -> start car
   Nothing -> error "not initialized"


 
 And the null pointer exception is reinvented!


That "case" statement (is a pattern matching) forces you to manage the null
case everywhere you use a Nothing-able type like that, otherwise your program
doesn't compile. In a language like D the compiler doesn't remind you to manage
the null case, so you may forget it. And experience shows that indeed in many
people around the world forget to test it and important programs stop running.
When you are running a program, it's not nice to see it stop in the middle of
its work.

This request of explicit management of nulls is indeed the second half of the
nonnull proposal I have explained in my enhancement request. There are several
ways to face this in D, and so far only a person in this thread has touched
this second part of the proposal.

Bye,
bearophile
Nov 06 2010
parent reply tls <do notha.ev> writes:
bearophile Wrote:


 Walter:
 

 retard wrote:

 In a functional language:
 
 start_the_car c = case c of
   Just car -> start car
   Nothing -> error "not initialized"


 
 And the null pointer exception is reinvented!


 
 That "case" statement (is a pattern matching) forces you to manage the null
case everywhere you use a Nothing-able type like that, otherwise your program
doesn't compile. In a language like D the compiler doesn't remind you to manage
the null case, so you may forget it. And experience shows that indeed in many
people around the world forget to test it and important programs stop running.
When you are running a program, it's not nice to see it stop in the middle of
its work.


Walter make this point earlier. That it imrpoves development time when real
hardware segfault happens. These other nulls causes runtime or compile time
unnecessary checks just like testing prime numbers. Thus not there is need for
nonnull pointers. It only complicate the way you think algorithms and make
programming hard. I never seen segfault in practice. Have you? I download
latest version always it works. And cheap in copy markets.


 
 This request of explicit management of nulls is indeed the second half of the
nonnull proposal I have explained in my enhancement request. There are several
ways to face this in D, and so far only a person in this thread has touched
this second part of the proposal.


Walter touched? He gives through out analysis and I think its good the way is
now.
Nov 06 2010
parent "Simen kjaeraas" <simen.kjaras gmail.com> writes:
tls <do notha.ev> wrote:


 I never seen segfault in practice. Have you?


Have you ever touched a computer? I see segfaults about daily. In my own
code, not that often.


-- 
Simen
Nov 06 2010
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
Denis Koroskin wrote:

 On Fri, 05 Nov 2010 23:44:58 +0300, Walter Bright 
 <newshound2 digitalmars.com> wrote:

 To eliminate null pointers is the same as shooting the canary in your 
 coal mine because its twitter annoys you.


 
 I'm tired of pointing out that NO ONE is talking about eliminating null 
 pointers, but rather extending an existing type system to support 
 non-nulls. Your hate towards non-nullables comes from misunderstanding 
 of the concept.


I've decided that I should stick my neck out and write up a blog post about 
this. So stay tuned, armed with roses or bricks as apropos.
Nov 05 2010
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Anyway, the topic of this whole tread is about non-nullable types in D,







 have shown).
 
 Walter, instead of poking and teasing me as a ten year old does, why we don't
 start talking about serious things?


Given the store you've set by integer overflow detection, sorry, I couldn't
Nov 05 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:






From that tutorial is looks well though-out, in general (but it's not complete

(ignoring the fact it enforces those contracts statically), because this may
give good insights about how to improve D DbC.

Bye,
bearophile
Nov 05 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 From that tutorial is looks well though-out, in general (but it's not

 contract (ignoring the fact it enforces those contracts statically), because
 this may give good insights about how to improve D DbC.



DbC 
design is almost a direct copy.
Nov 05 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Walter Bright:



DbC 
 design is almost a direct copy.



have written about thirty little/snippets programs in it, and from what I've

cases of an OOP language (all that stuff about aggregate objects, ownership,
mutable/consistent/committed state for objects, peer groups, and all the
relative details. I didn't remember all those things in Eiffel. And those
things aren't useless) (but also a bit less flexible, because it's all designed
to be statically verifiable).

Bye,
bearophile
Nov 05 2010
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Thank you for this post Walter, because here you actually discuss with me/us
about the topic (regardless what the final decisions will be).

Non-null types aren't a fully tidy and simple idea, they have some complexities
and special situations. And probably some problems need to be improved along
the way (as the refinements to pure functions done in D).


designers have given lot of thought in that design, so ignoring their work is



and maybe a Windows too).


to read that part of the document to see how they have faced the relative
problems. You may even write tiny programs in the interactive site, to see how

questions.



D, or we may not like them, or we may even decide to try something different
just to be different and explore a different solution with different
trade-offs. This is normal. But after seeing the amount of design work done on
that language, it's wise to refuse those decisions only after we know and
understand them, and we know why we refuse them.

The article talks about the relation between arrays and nonnullables from page
32, the section "4.1 Arrays of Non-Null Elements".


T?[]! (T?[]  in D syntax, I have replaced the ! with a trailing  , because in D
the bang has enough purposes already).

An array like T?[]  means it's a nonnullable array of nullable reference types.

This quotation is one of the things it says about arrays:


Unlike fields of non-null types, whose initialization in the constructors of
the class can be assured by syntactic definite-assignment rules, arrays of
non-null elements are initialized by arbitrary code that follows the new
allocation of the arrays. Until that initialization is completed, one cannot
rely on the type of the array to accurately reflect the non-nullness of the

marker, in the form of a method NonNullType.AssertInitialized, which is used to
indicate a program point where the initialization code has completed. The type
checker will not give the array its declared non-null type until that point.<



Later it says, a quotation:


Thus, before these arrays can be used as having type string[], the code must
call AssertInitialized. At that call site, the program verifier checks that
every array element is non-null. (At run time, AssertInitialized performs a
dynamic check that the array elements are not null. The time needed to do so is
proportional to the length of the array, but that is no worse than the time
required to initialize the array in the first place.)<


Even later, a quotation:


If a program tries to use the array element before the array has been given its
declared type, the compiler will complain. For example, if the assignment to
series[2] in Fig. 13 is replaced by series[2] = series[1], the following type
error results:


Fig13.ssc(13,17): Cannot store delayed value into non(or incompatibly)delayed
location
despite the fact that the right-hand side of the assignment actually does have
a non-null value at that time.
Also, if the code does not include a call to AssertInitialized for an array of
non-null elements, the type checker complains:
Fig13.ssc(10,14): Variable ’series’, a nonnull element array, may not have been
initialized. Did you forget to call NonNullType.AssertInitialized()?
Perhaps confusingly, the source location mentioned in the error message points
to where the array is declared, but this does not mean that AssertInitialized
has to be called there.<


(This design maybe works, but it doesn't look wonderful. I am able to think
about better ideas, like using a kind of loop variant to prove to the compiler
that an array has a monotonically increasing number of nonnull items. But while

maybe not doable in D).


Walter:



 Consider non-nullable type T:
 
    T[] a = new T[4];
    ... time goes by ...
    T[1] = foo;
    T[3] = bar;
    ... more time goes by ...
    bar(T[2]);
 
 In other words, I create an array that I mean to fill in later, because I don't
 have meaningful data for it in advance. What do I use to default initialize it
 with non-nullable data? And once I do that, should bar(T[2]) be an error? How
 would I detect the error?


I think that if you want to fill items of your array later, you may use a
growable array (so there are never empty items in it), or you just use an array
of nullable references. Nonnull isn't meant ot replace all usage cases. You use
it only when you need it. If you have different needs, then you use nullable
references.

Keep in mind that in this discussion has not come up another problem with

objects. I have shown it up a bit in my bug report:
http://d.puremagic.com/issues/show_bug.cgi?id=4571

This is an example in D-like syntax:


class Foo {}

class A {
    Foo  name;
    this(Foo  s) {
        this.name = s;
        this.m();
    }

    void m() { /*...*/ }
}

class B : A {
    Foo  path;
    this(Foo  p, Foo  s) {
        super(s);
        this.path = p;
    }

    override void m() {
        // here this.path is null despite it's a non-null
        assert(this.path !is null);
    }
}

void main() {
    new B(new Foo, new Foo);
}


I have adapted that example from this paper, it discusses about partially
uninitialized objects too:
http://research.microsoft.com/pubs/67461/non-null.pdf

A comment about that program from the paper:


The problem with the code is that during the base call to A's constructor, the
virtual method B.m may be invoked. At this time, field path of the object under
construction has not yet been initialized. Thus, accesses of this.path in
method B.m may yield a possibly-null value, even though the field has been
declared as being non-null.<


Bye,
bearophile
Nov 05 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
J. M. Berger: 


 <sarcasm>
 Consider immutable type immutable T:
 
   immutable T] a = new immutable T[4];
   ... time goes by ...
   T[1] = foo;
   T[3] = bar;
 
 In other words I create an array that I mean to fill in later,
 because I don't have meaningful data for it in advance. How do I do
 that with immutable types?
 </sarcasm>


Despite your answer was sarcastic, it's a very interesting note.


nonnulls, using NonNullType.AssertInitialized():
http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=121140

What's interesting in your sarcastic note is that you have shown that building
a collection of immutables and a collection of nonnulls in some situations
share the same basic problem, that is how to build something that has
constraints regarding its changes or its not being present yet.

Two bug reports about the topic:
http://d.puremagic.com/issues/show_bug.cgi?id=5147
http://d.puremagic.com/issues/show_bug.cgi?id=5081

In my opinion currently in D there aren't very good ways to build collections
of immutables.

The Clojure solves a related problem (it's a performance problem too), using
transients:
http://clojure.org/transients


NonNullType.AssertInitialized() to mark inside a function the point where you
state an array of nonnulls is done, the transient had ended. Even if
syntactically they are very different, on a semantic level they are doing the
same thing, they are both ways to tell apart the building phase from the
finished phase. Surely there are many different ways to tell the compiler a way
to tell apart such two phases.

A solution for the creation of immutable collections in D may be used to build
collections of nonulls too. They aren't the same problem, but they share enough
that the same solution (with little changes) may be used for both.

Bye,
bearophile
Nov 06 2010
parent Don <nospam nospam.com> writes:
bearophile wrote:

 J. M. Berger: 
 

 <sarcasm>
 Consider immutable type immutable T:

   immutable T] a = new immutable T[4];
   ... time goes by ...
   T[1] = foo;
   T[3] = bar;

 In other words I create an array that I mean to fill in later,
 because I don't have meaningful data for it in advance. How do I do
 that with immutable types?
 </sarcasm>


 
 Despite your answer was sarcastic, it's a very interesting note.
 

nonnulls, using NonNullType.AssertInitialized():
 http://www.digitalmars.com/webnews/newsgroups.php?art_group=digitalmars.D&article_id=121140
 
 What's interesting in your sarcastic note is that you have shown that building
a collection of immutables and a collection of nonnulls in some situations
share the same basic problem, that is how to build something that has
constraints regarding its changes or its not being present yet.
 
 Two bug reports about the topic:
 http://d.puremagic.com/issues/show_bug.cgi?id=5147
 http://d.puremagic.com/issues/show_bug.cgi?id=5081
 
 In my opinion currently in D there aren't very good ways to build collections
of immutables.
 
 The Clojure solves a related problem (it's a performance problem too), using
transients:
 http://clojure.org/transients
 

NonNullType.AssertInitialized() to mark inside a function the point where you
state an array of nonnulls is done, the transient had ended. Even if
syntactically they are very different, on a semantic level they are doing the
same thing, they are both ways to tell apart the building phase from the
finished phase. Surely there are many different ways to tell the compiler a way
to tell apart such two phases.
 
 A solution for the creation of immutable collections in D may be used to build
collections of nonulls too. They aren't the same problem, but they share enough
that the same solution (with little changes) may be used for both.
 
 Bye,
 bearophile


As was pointed out in a recent post, the return value of a pure function 
is guaranteed to be unique, so could be allowed to implictly cast to 
immutable. I'm planning a patch for that soon, to see how well it works 
in practice.

But I don't think that would work for non-nulls. I don't think the two 
situations have a great deal in common.
Nov 06 2010
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Don:


 bearophile wrote:

 ...
 http://d.puremagic.com/issues/show_bug.cgi?id=5081





 As was pointed out in a recent post, the return value of a pure function 
 is guaranteed to be unique, so could be allowed to implictly cast to 
 immutable. I'm planning a patch for that soon, to see how well it works 
 in practice.


Right, that's the bug 5081 :-) It works with strong pure. It's a simple but
very nice idea that avoids some casts.



 But I don't think that would work for non-nulls. I don't think the two 
 situations have a great deal in common.


I see, they may have different solutions too.

Bye and thank you,
bearophile
Nov 06 2010
parent reply Christopher Bergqvist <chris digitalpoetry.se> writes:
Does D have anything comparable to C++ references =E0 la "void
nullCheckLessFunction(const std::string& notNullStr) {...}" or does it only
have the equivalent of "void nullCheckingRequired(const std::string*
mightByNullStr) {...}"?
Nov 06 2010
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 06 Nov 2010 14:06:20 +0300, Christopher Bergqvist  =

<chris digitalpoetry.se> wrote:


 Does D have anything comparable to C++ references =C3=A0 la "void
 nullCheckLessFunction(const std::string& notNullStr) {...}" or does it=


  =


 only
 have the equivalent of "void nullCheckingRequired(const std::string*
 mightByNullStr) {...}"?


void nullCheckLessFunction(ref const(string) notNullStr) { .. }
Nov 06 2010
parent Christopher Bergqvist <chris digitalpoetry.se> writes:
On Sat, Nov 6, 2010 at 12:23 PM, Denis Koroskin <2korden gmail.com> wrote:


 On Sat, 06 Nov 2010 14:06:20 +0300, Christopher Bergqvist <
 chris digitalpoetry.se> wrote:

  Does D have anything comparable to C++ references =E0 la "void

 nullCheckLessFunction(const std::string& notNullStr) {...}" or does it
 only
 have the equivalent of "void nullCheckingRequired(const std::string*
 mightByNullStr) {...}"?


 void nullCheckLessFunction(ref const(string) notNullStr) { .. }


I made two comparison snippets between D & C++.

http://ideone.com/VPzz6 (D)
http://ideone.com/HzFRB (C++)

I feel like C++ is one small step ahead of D in this respect. It's not
possible to trust that C++ references are non-null, but at least they serve
as concise documentation of the expected contents and tend to make
surrounding code perform the null-check up front before dereferencing from
pointer to C++ reference.
Nov 06 2010
prev sibling next sibling parent reply Gary Whatmore <no spam.sp> writes:
FeepingCreature Wrote:


 Walter Bright Wrote:

 All that does is reinvent the null pointer seg fault. The hardware does this
for 
 you for free.


 
 Walter, I know you're a Windows programmer but this cannot be the first time
somebody has told you this - YOU CANNOT RECOVER FROM SEG FAULTS UNDER LINUX.
 
 Access violations are not a cross-platform substitute for exceptions.


You're missing the point. The reason for seg faults is to terminate the
application as quickly as possible. The developer then fires up the debugger
and fixes the app. Seg faults should never happen in production code. You only
release when all seg faults are fixed. The builtin unit tests in D can
guarantee this with 100% test coverage.
Nov 06 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Gary Whatmore:


 You're missing the point. The reason for seg faults is to terminate the
application as quickly as possible. The developer then fires up the debugger
and fixes the app. Seg faults should never happen in production code. You only
release when all seg faults are fixed. The builtin unit tests in D can
guarantee this with 100% test coverage.


Unittests help remove and avoid some bugs, but experience shows they don't
solve/avoid all problems. Null exceptions do happen in production code. 100%
coverage of unittests can't avoid all possible bugs, because it's easy to miss
some code paths in unittests. Take a look at bugzilla to see many basic bugs in
Phobos despite the usage of unittests.

Bye,
bearophile
Nov 06 2010
parent reply Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Gary Whatmore:
 

 You're missing the point. The reason for seg faults is to terminate the
 application as quickly as possible. The developer then fires up the
 debugger and fixes the app. Seg faults should never happen in production
 code. You only release when all seg faults are fixed. The builtin unit
 tests in D can guarantee this with 100% test coverage.


 
 Unittests help remove and avoid some bugs, but experience shows they don't
 solve/avoid all problems. Null exceptions do happen in production code. 100%
 coverage of unittests can't avoid all possible bugs, because it's easy to
 miss some code paths in unittests. Take a look at bugzilla to see many basic
 bugs in Phobos despite the usage of unittests.


Do a:

    make -cov

in the phobos directory, which will run the coverage analyzer on all the unit 
tests in the library. You'll find that the coverage is nowhere near 100%.

Granted, even 100% coverage is no guarantee of no seg faults, but in practice
it 
is very effective.

And finally, null exceptions are not a bug. They are the messenger that there
is 
a bug in the code. Until one knows why a null pointer was dereferenced, one 
cannot conclude that the existence of the null pointer was the bug.
Nov 06 2010
parent bearophile <bearophileHUGS lycos.com> writes:
Walter:


 Granted, even 100% coverage is no guarantee of no seg faults, but in practice
it 
 is very effective.


Unittests are widely used in Java, still null exceptions are not uncommon in
Java.



 And finally, null exceptions are not a bug. They are the messenger that there
is 
 a bug in the code. Until one knows why a null pointer was dereferenced, one 
 cannot conclude that the existence of the null pointer was the bug.


I see, null exceptions are not a bug. But for the semantics of a part of a
program it may be a bug to put a null inside a collection or inside a variable,
or it may be a bug to pass a null to a certain function. Those are true bugs. A
notnull type allows you to spot the bug, because the type system screams (if
you have asked those things to be nonnull)

Your may write your function like this:
void foo(T  x) {...}

it means T can't be null. As you say a segfault inside foo() is not a bug, the
real bug is where that x was initialized, collected or created. The advantage
of a nonnull type is that you will receive an error when you try to put a null
inside the variable that will later be passed to foo(), this means where the
true bug is. That foo() is also better than this:

void foo(T x)
  in { assert(x !is null); }
  body {...}

Because:
- You need to put a whole precondition there, while in the other case you just
need to add a  . Less typing, etc.
- The test is done (or not done) at run-time. This slows down the function a
little. In release mode you don't have the slowdown, but you lose the safety.
- The D contract system is not enforced at compile-time. This means that the
true spot where the bug is, where the variable that later will become x is
created, is not spotted. So your program will keep working until you call foo()
and then a runtime exception or segfault will be generated. Instead if you are
using a static type system, it means you are using that   suffix, the type
system statically spots the possible problem for you (it somehow forces you to
put a nonnull inside the variable that will become x. At worst this will
require a runtime test. But this test is done only once, if you later you call
another function bar() with he same nonnull argument, it too will not need
another runtime test).

Bye,
bearophile
Nov 06 2010
prev sibling next sibling parent reply Gary Whatmore <no spam.sp> writes:
bearophile Wrote:


 Gary Whatmore:
 

 You're missing the point. The reason for seg faults is to terminate the
application as quickly as possible. The developer then fires up the debugger
and fixes the app. Seg faults should never happen in production code. You only
release when all seg faults are fixed. The builtin unit tests in D can
guarantee this with 100% test coverage.


 
 Unittests help remove and avoid some bugs, but experience shows they don't
solve/avoid all problems. Null exceptions do happen in production code. 100%
coverage of unittests can't avoid all possible bugs, because it's easy to miss
some code paths in unittests. Take a look at bugzilla to see many basic bugs in
Phobos despite the usage of unittests.


Doesn't it mean that instead of complicating the language we write more unit
tests? Surely the stream of new bugs ends at some point. Then we are production
ready. Just look how look it took to stabilize C++. 30+ years is acceptable for
D. C++ didn't have test driven development knowledge, we have.
Nov 06 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
Gary Whatmore:


 Doesn't it mean that instead of complicating the language we write more unit
tests?


This is true, it's a design decision. And we are indeed trying to decide if
this feature is worth it.

A sufficiently large amount of unittests is indeed able to remove and avoid a
large enough percentage of bugs. And you are are right in saying that a too
much complex language (with too many features) is not a good thing. So I can't
answer you, but you too can't answer yourself.

So it's also a matter of what the most handy solution is. If nonnull types are
able to avoid the need of a sufficiently large amount of unittesting, then the
feature is good to have, because it saves you time. If the amount of unittests
it allows you to avoid is too much small, then it's not worth to have it.

Nonull reference types are an extension of the static type system. Unittests
are known to be a partial replacement for (static) type systems. Dynamic
languages as Python use unitesting to perform some of the tests done at compile
time by the D static type system.

Python programmers argue that static strong type systems are not better than
strong dynamic typing because the kind of bugs caught by the simple static type
systems as the Java one are easily and quickly avoided with simple & quick to
write unittests. And then you need to add other unittests anyway, that test the
logic of the code, that simple static type systems aren't able to test and
enforce.

New and much more flexible static type systems are possible, like the SPARK and
ATS ones, but they require a kind of programming that asks lot of brain from
the programmer, so they are mostly for special purposes only. Languages like


Bye,
bearophile
Nov 06 2010
parent bearophile <bearophileHUGS lycos.com> writes:
 New and much more flexible static type systems are possible, like the SPARK
and ATS ones, but they require a kind of programming that asks lot of brain
from the programmer, so they are mostly for special purposes only. 


Sorry, unfinished post. I was saying:


usable for general purpose programs despite having a very flexible and powerful
static type system.

Bye,
bearophile
Nov 06 2010
prev sibling next sibling parent reply steveh <steveh57 useshotmai.l> writes:
bearophile Wrote:


 Walter:
 

 I don't see that non-null is such a special case that it would benefit from a 
 special case syntax.


 
 Well, nonnull are a special cases because:
 - There is a good enough way to solve this problem. This is not true in the
general case.
 - null exceptions are very common bugs, among the most common ones in Java and



citation needed. I've made software with 100% line and path coverage. No
segfaults happened ever. Spent enormous time optimizing the code and its
quality. In quality applications NPE/segfaults simply don't exist.
Nov 06 2010
parent reply bearophile <bearophileHUGS lycos.com> writes:
steveh:


 citation needed. I've made software with 100% line and path coverage. No
segfaults happened ever. Spent enormous time optimizing the code and its
quality. In quality applications NPE/segfaults simply don't exist.


You are right, some citation is needed. I have seen or read many papers that
talk about facing the null problem (I can list them if you want), but hard data
is always not easy to find.

And by the way, I believe you. Bugs that manifest themselves through a null
exception are probably the most common in Java code, but they aren't the harder
to fix. Concurrency bugs are far harder to fix (or even to find) than null
pointer exceptions. This thread is not about ways to avoid concurrency bugs (D
uses message sending to avoid some concurrency bugs).

Bye,
bearophile
Nov 06 2010
parent Walter Bright <newshound2 digitalmars.com> writes:
bearophile wrote:

 Concurrency bugs are far harder to fix (or even to find) than
 null pointer exceptions. This thread is not about ways to avoid concurrency
 bugs


The hardest problems are concurrency issues, followed by memory corruption.
Nov 07 2010
prev sibling parent reply Kagamin <spam here.lot> writes:
FeepingCreature Wrote:


 This means stack traces are out unless you have special handling for segfaults
that decodes the stack and prints the error pos. That in turn means you need to
have a debugger attached to get stacktraces, which can be annoying especially
in long-running programs where the crash is often the first indication you have
of a problem.
 


Doesn't it write coredump, which can be investigated later?
Nov 07 2010
parent Daniel Gibson <metalcaedes gmail.com> writes:
Kagamin schrieb:

 FeepingCreature Wrote:
 

 This means stack traces are out unless you have special handling for segfaults
that decodes the stack and prints the error pos. That in turn means you need to
have a debugger attached to get stacktraces, which can be annoying especially
in long-running programs where the crash is often the first indication you have
of a problem.


 Doesn't it write coredump, which can be investigated later?


depends on your system, linux doesn't do core dumps by default, you have to
allow it with
ulimit -c <somevalue>
Nov 07 2010