www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Pluggable type sytems

reply bearophile <bearophileHUGS lycos.com> writes:
I think pluggable type systems will become more common in the following years
(see also the optional annotations of Python3 that are designed for that too).
This is more or less related:

http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/

(but nonnullability is so basic that it's better inside the language, and not
left out to a plug-in type system).

Bye,
bearophile
Jan 18 2009
next sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2009-01-18 21:29:02 -0500, bearophile <bearophileHUGS lycos.com> said:


 I think pluggable type systems will become more common in the following 
 years (see also the optional annotations of Python3 that are designed 
 for that too). This is more or less related:
 
 http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/


Nice post.


 (but nonnullability is so basic that it's better inside the language, 
 and not left out to a plug-in type system).


I agree for non-nullability.

In fact, I'd even argue that non-nullability should be the default for 
pointers and class references, because it is either safer (if the 
compiler doesn't do the null check for you) or less troublesome to use 
(if the compiler forces you to check for null everytime). Another 
reason being consistency: value-types can't be null by default and so 
should be class references and pointers. And making pointers 
non-nullable by default can only be done at the language level, so 
obviously it's better in the language than as a user-defined type 
modifier.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Jan 19 2009
next sibling parent "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 19 Jan 2009 16:26:47 +0300, Michel Fortin <michel.fortin michelf.com>
wrote:


 On 2009-01-18 21:29:02 -0500, bearophile <bearophileHUGS lycos.com> said:


 I think pluggable type systems will become more common in the following  
 years (see also the optional annotations of Python3 that are designed  
 for that too). This is more or less related:
  http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/


 Nice post.


 (but nonnullability is so basic that it's better inside the language,  
 and not left out to a plug-in type system).


 I agree for non-nullability.

 In fact, I'd even argue that non-nullability should be the default for  
 pointers and class references, because it is either safer (if the  
 compiler doesn't do the null check for you) or less troublesome to use  
 (if the compiler forces you to check for null everytime). Another reason  
 being consistency: value-types can't be null by default and so should be  
 class references and pointers. And making pointers non-nullable by  
 default can only be done at the language level, so obviously it's better  
 in the language than as a user-defined type modifier.


Agree.
Jan 20 2009
prev sibling parent reply Bartosz Milewski <bartosz relisoft.com> writes:
I also believe that non-null should be the default. Most references in a
typical program are assumed to be non-null. 

Like for every default, there should be an escape mechanism from it. However,
adding "nullable" to the type system might be controversial, especially because
it requires special treatment of constructors and polymorphism (hence
additional type modifiers, raw and poly-null, and I haven't even mentioned
wildcard annotations).

So some time ago I came up with a more modest proposal. The compiler should
catch nullable references at their source. It should enforce what I call
"declaration is initialization" (DII). For instance, the following is correct:

Foo foo = new Foo;

and the following is not:

Foo foo; // error
...
foo = new Foo;

Obviously this rule is too strict, so there must be a way to override it.
Here's the simple syntax:

Foo foo = null; // not an error!

Notice that the programmer explicitly states that one should be careful when
using foo, because it might be null. 

Since this solution doesn't involve the type system, the nullability doesn't
propagate across function calls. But it might be a tradeoff worth taking, if
only for the simplicity.

Constructors still need some special treatment. Every class member variable
must be initialized in the constructor. Again, the escape is to _explicitly_
initialize it to null. This might seem redundant, since the variable is already
default initialized to null, but the compiler can easily eliminate such
redundancies. 

I also found out that Java's editor, Eclipse, enforces DII--a very useful
feature that paid off for me in the first 100 lines of code I wrote after a
long period of inactivity in Java ;-) 


 Michel Fortin Wrote:



 In fact, I'd even argue that non-nullability should be the default for 
 pointers and class references, because it is either safer (if the 
 compiler doesn't do the null check for you) or less troublesome to use 
 (if the compiler forces you to check for null everytime). Another 
 reason being consistency: value-types can't be null by default and so 
 should be class references and pointers. And making pointers 
 non-nullable by default can only be done at the language level, so 
 obviously it's better in the language than as a user-defined type 
 modifier.
Feb 08 2009
next sibling parent reply Robert Fraser <fraserofthenight gmail.com> writes:
Bartosz Milewski wrote:

 I also found out that Java's editor, Eclipse, enforces DII--a very useful
feature that paid off for me in the first 100 lines of code I wrote after a
long period of inactivity in Java ;-) 


The Java spec does, actually. Eclipse JDT just reports compiler errors 
as you type.
Feb 08 2009
parent Bartosz Milewski <bartosz relisoft.com> writes:
Shows you what a beginner in Java I am. Again and again I keep reinventing the
wheel. That settles it--we have to have it in D!


Robert Fraser Wrote:


 Bartosz Milewski wrote:

 I also found out that Java's editor, Eclipse, enforces DII--a very useful
feature that paid off for me in the first 100 lines of code I wrote after a
long period of inactivity in Java ;-) 


 
 The Java spec does, actually. Eclipse JDT just reports compiler errors 
 as you type.
Feb 08 2009
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Bartosz Milewski:

 I also believe that non-null should be the default.


That' probably right.
And you can add ? to the things that can be null. Better to let the type system
deal with this.

Bye,
bearophile
Feb 08 2009
prev sibling parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 09 Feb 2009 01:19:55 +0300, Bartosz Milewski <bartosz relisoft.com>
wrote:


 I also believe that non-null should be the default. Most references in a  
 typical program are assumed to be non-null.


Oh, that's true!


 Like for every default, there should be an escape mechanism from it.  
 However, adding "nullable" to the type system might be controversial,  
 especially because it requires special treatment of constructors and  
 polymorphism (hence additional type modifiers, raw and poly-null, and I  
 haven't even mentioned wildcard annotations).


Common opinion is that this is worth the trouble these days. More and more
people discuss, write articles on non-nullable types and incorporate them into

default is his greatest mistake that is too late to be fixed.


 So some time ago I came up with a more modest proposal. The compiler  
 should catch nullable references at their source. It should enforce what  
 I call "declaration is initialization" (DII). For instance, the  
 following is correct:

 Foo foo = new Foo;

 and the following is not:

 Foo foo; // error
 ...
 foo = new Foo;

 Obviously this rule is too strict, so there must be a way to override  
 it. Here's the simple syntax:

 Foo foo = null; // not an error!


And what about value types? Why special treatment for reference types only? I
don't like rules like that in a language:

Foo foo; // error
int i; // okay?

If not, and this is an error, too, then this defeats the whole purpose of
having T.init in D, because it will never be used per your proposal:

int i = 0; // initialized explicitly
char c = char.init; // would you use that?


 Notice that the programmer explicitly states that one should be careful  
 when using foo, because it might be null.

 Since this solution doesn't involve the type system, the nullability  
 doesn't propagate across function calls. But it might be a tradeoff  
 worth taking, if only for the simplicity.


Naah.. The whole purpose of having non-nullable types is that given a pointer
you are sure it is not null:

string toString(Object o)
{
   // is it null or not? Should I check for null or not? What if it is null?
   // Should I throw? What kind of exception?
   // But was going to make it nothrow! So what, no checks?
   // Just fail with access violation? Is this the best D can do?

   return o.toString();
}

As an end user, you neglect and avoid null checks in your code ("I'm sure I
won't pass null here"). And then you run your program half an hour, it
segfaults with access violation and you don't have any clue where did null
dereference take place. I have came across this situation so many times that I
hate it.

As a library designer, you end up having null checks everywhere in your code
which is some kind of a paranoia because in 99% of cases the pointer will be
not null. But why put so many special case and checks if it is *disallowed* to
run code with null pointer in first place? So people decided - let's enforce
that at compile time! That's the whole point of the feature - disallow invalid
use cases at compile time. You get cleaner code (no unnecessary null-checks and
throws) and small performance improvement as a bonus.

I would like to write code as this as be sure that it will never fail:

string toString(Object o) // non-nullability is enforced at compile time
{
    return o.toString();
}

This is D style - do as much as possible at compile time - and I believe
non-nullable types fit D type system very well.


 Constructors still need some special treatment. Every class member  
 variable must be initialized in the constructor. Again, the escape is to  
 _explicitly_ initialize it to null. This might seem redundant, since the  
 variable is already default initialized to null, but the compiler can  
 easily eliminate such redundancies.

 I also found out that Java's editor, Eclipse, enforces DII--a very  
 useful feature that paid off for me in the first 100 lines of code I  
 wrote after a long period of inactivity in Java ;-)


 Michel Fortin Wrote:



 In fact, I'd even argue that non-nullability should be the default for
 pointers and class references, because it is either safer (if the
 compiler doesn't do the null check for you) or less troublesome to use
 (if the compiler forces you to check for null everytime). Another
 reason being consistency: value-types can't be null by default and so
 should be class references and pointers. And making pointers
 non-nullable by default can only be done at the language level, so
 obviously it's better in the language than as a user-defined type
 modifier.
Feb 08 2009
parent reply Bartosz Milewski <bartosz relisoft.com> writes:
I just assume that any extension of the already complex D type system will be
met with a lot of resistance. I remember the all-out wars about const and
immutable (a.k.a invariant). Even those extensions are still half-assed: the
construction of immutable objects and const polymorphism issues remain. 

My impression is that theoreticians and very advanced programmers love
elaborate type systems. Nine-to-five programmers, which are in the majority,
prefer simplicity even if the cost is reliability. Just look at the comments to
my blog post. 

I know we have to do something about null references in D, but I'm still on the
fence about how we should accomplish that.

Denis Koroskin Wrote:


 On Mon, 09 Feb 2009 01:19:55 +0300, Bartosz Milewski <bartosz relisoft.com>
wrote:
 

 I also believe that non-null should be the default. Most references in a  
 typical program are assumed to be non-null.


 
 Oh, that's true!
 

 Like for every default, there should be an escape mechanism from it.  
 However, adding "nullable" to the type system might be controversial,  
 especially because it requires special treatment of constructors and  
 polymorphism (hence additional type modifiers, raw and poly-null, and I  
 haven't even mentioned wildcard annotations).


 
 Common opinion is that this is worth the trouble these days. More and more
people discuss, write articles on non-nullable types and incorporate them into

default is his greatest mistake that is too late to be fixed.
 

 So some time ago I came up with a more modest proposal. The compiler  
 should catch nullable references at their source. It should enforce what  
 I call "declaration is initialization" (DII). For instance, the  
 following is correct:

 Foo foo = new Foo;

 and the following is not:

 Foo foo; // error
 ...
 foo = new Foo;

 Obviously this rule is too strict, so there must be a way to override  
 it. Here's the simple syntax:

 Foo foo = null; // not an error!


 
 And what about value types? Why special treatment for reference types only? I
don't like rules like that in a language:
 
 Foo foo; // error
 int i; // okay?
 
 If not, and this is an error, too, then this defeats the whole purpose of
having T.init in D, because it will never be used per your proposal:
 
 int i = 0; // initialized explicitly
 char c = char.init; // would you use that?
 

 Notice that the programmer explicitly states that one should be careful  
 when using foo, because it might be null.

 Since this solution doesn't involve the type system, the nullability  
 doesn't propagate across function calls. But it might be a tradeoff  
 worth taking, if only for the simplicity.


 
 Naah.. The whole purpose of having non-nullable types is that given a pointer
you are sure it is not null:
 
 string toString(Object o)
 {
    // is it null or not? Should I check for null or not? What if it is null?
    // Should I throw? What kind of exception?
    // But was going to make it nothrow! So what, no checks?
    // Just fail with access violation? Is this the best D can do?
 
    return o.toString();
 }
 
 As an end user, you neglect and avoid null checks in your code ("I'm sure I
won't pass null here"). And then you run your program half an hour, it
segfaults with access violation and you don't have any clue where did null
dereference take place. I have came across this situation so many times that I
hate it.
 
 As a library designer, you end up having null checks everywhere in your code
which is some kind of a paranoia because in 99% of cases the pointer will be
not null. But why put so many special case and checks if it is *disallowed* to
run code with null pointer in first place? So people decided - let's enforce
that at compile time! That's the whole point of the feature - disallow invalid
use cases at compile time. You get cleaner code (no unnecessary null-checks and
throws) and small performance improvement as a bonus.
 
 I would like to write code as this as be sure that it will never fail:
 
 string toString(Object o) // non-nullability is enforced at compile time
 {
     return o.toString();
 }
 
 This is D style - do as much as possible at compile time - and I believe
non-nullable types fit D type system very well.
 

 Constructors still need some special treatment. Every class member  
 variable must be initialized in the constructor. Again, the escape is to  
 _explicitly_ initialize it to null. This might seem redundant, since the  
 variable is already default initialized to null, but the compiler can  
 easily eliminate such redundancies.

 I also found out that Java's editor, Eclipse, enforces DII--a very  
 useful feature that paid off for me in the first 100 lines of code I  
 wrote after a long period of inactivity in Java ;-)


 Michel Fortin Wrote:



 In fact, I'd even argue that non-nullability should be the default for
 pointers and class references, because it is either safer (if the
 compiler doesn't do the null check for you) or less troublesome to use
 (if the compiler forces you to check for null everytime). Another
 reason being consistency: value-types can't be null by default and so
 should be class references and pointers. And making pointers
 non-nullable by default can only be done at the language level, so
 obviously it's better in the language than as a user-defined type
 modifier.
Feb 08 2009
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Mon, 09 Feb 2009 04:09:26 +0300, Bartosz Milewski <bartosz relisoft.com>
wrote:


 I just assume that any extension of the already complex D type system  
 will be met with a lot of resistance. I remember the all-out wars about  
 const and immutable (a.k.a invariant). Even those extensions are still  
 half-assed: the construction of immutable objects and const polymorphism  
 issues remain.

 My impression is that theoreticians and very advanced programmers love  
 elaborate type systems. Nine-to-five programmers, which are in the  
 majority, prefer simplicity even if the cost is reliability. Just look  
 at the comments to my blog post.

 I know we have to do something about null references in D, but I'm still  
 on the fence about how we should accomplish that.


So, let's ask the community: Would you like to see nullable types in D?

http://www.micropoll.com/akira/mpview/539369-138652
(please, don't abuse by voting multiple time)

Explain your reasoning in newsgroups. Thank you.
Feb 08 2009
parent "Danny Wilson" <bluezenix gmail.com> writes:
Op Mon, 09 Feb 2009 02:25:55 +0100 schreef Denis Koroskin  
<2korden gmail.com>:


 On Mon, 09 Feb 2009 04:09:26 +0300, Bartosz Milewski  
 <bartosz relisoft.com> wrote:


 ...

 My impression is that theoreticians and very advanced programmers love  
 elaborate type systems. Nine-to-five programmers, which are in the  
 majority, prefer simplicity even if the cost is reliability. Just look  
 at the comments to my blog post.

 I know we have to do something about null references in D, but I'm  
 still on the fence about how we should accomplish that.


 So, let's ask the community: Would you like to see nullable types in D?

 http://www.micropoll.com/akira/mpview/539369-138652
 (please, don't abuse by voting multiple time)

 Explain your reasoning in newsgroups. Thank you.


I voted yes. There have been far too many NullPointerExceptions in various  
languages, for the 9-5 programmer to understand in 15 minutes what  
non-nullable types are :-)
Feb 09 2009
prev sibling next sibling parent bmeck <bmeck stedwards.edu> writes:
 In fact, I'd even argue that non-nullability should be the default for 
 pointers and class references


Not against it for class references, but utterly against it for pointers as
that is one of main points of using a pointer to me. Using structs, classes,
primitive variables, and arrays I would agree the majority of the time do not
need null checks as we do not want nulls, however non-nullability of such
things can lead to confusion as default values of these (especially classes)
does not always have a well defined expectation. We could add something to the
compiler to check if it is initialized but what if we do get a null reference
for some reason (calling convention to heterogeneous code). If i want a
reference to a class and get a pointer from a library as such i would end up
getting the pointer and checking for null before making it non-nullable by a
cast both of which also take time.
Jan 20 2009
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Michel Fortin:

 I agree for non-nullability.


More discussions about this topic:
http://lambda-the-ultimate.org/node/1478

A very C-like language with such features (it's a language different from
Cyclone):
http://www.splint.org/manual/manual.html#null

Bye,
bearophile
Jan 21 2009
prev sibling parent "Nick Sabalausky" <a a.a> writes:
"bearophile" <bearophileHUGS lycos.com> wrote in message 
news:gl0ohe$1gda$1 digitalmars.com...

I think pluggable type systems will become more common in the following 
years (see also the optional annotations of Python3 that are designed for 
that too). This is more or less related:

 http://bartoszmilewski.wordpress.com/2009/01/18/java-pluggable-types/

 (but nonnullability is so basic that it's better inside the language, and 
 not left out to a plug-in type system).

 Bye,
 bearophile


Interesting article. Sounds a lot like D2 but with that flow-analysis 
nice-ity, and  PolyNull (which has been proposed here...often). Although I 
wonder...maybe  PolyNull should be the default (wherever applicable) instead 
of  Nullable or  NonNull? (Or maybe I'm overlooking some obvious problem 
with that.)

I see pluggable type sytems as potentially being a specific form of a more 
general thing I've been observing an increasing need for/trend towards: 
customizable languages.

Ex: A language that doesn't say "Semicolons mark end-of-statement" or 
"Newline marks end-of-statement and lines can be split with a backslash", 
but provides both as mutually-exclusive programmer-selectable options. Or a 
choice of C-style curly-braces vs Python-style indentation. Etc. (Side note: 
This stuff could really cut down on religious language debates. Not 
eliminate, of course, but reduce.)

Also, a tool could then be used to translate code between the various 
options, maybe even integrated into the IDE, so that people working on 
multi-person projects could each set up their own environments and use their 
own "customized language" even while working on the same line of code. The 
only project-wide standard would be what settings are used for checked-in 
code, just like tab/space settings and EOL markers (but ideally performed 
automatically (and reliably) by the check-in/check-out process). Eventually 
this might even be generalizable to the point where most languages can be 
reduced to nothing more than a collection of language settings. Which I 
think is the direction we appear to be headed anyway, since it seems like 
everyone and their grandmother is designing their own language (or two) 
these days, and many of these languages are more-or-less "just like language 
X, but with A, B and C changed to be more like language Y."

Obviously this is all much easier said than done, but it's something I've 
been thinking a lot about lately, and I think may be worth exploring.
Jan 22 2009