www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Opt-in non-null class references?

reply SimonN <eiderdaus gmail.com> writes:
Hi,

Andrei said in 2014 that not-null-references should be the 
priority of 2014's language design, with consideration to make 
not-null the default. In case the code breakage is too high, this 
can be an opt-in compiler flag.

Discussion here: 
https://forum.dlang.org/post/lcq2il$2emp$1 digitalmars.com

Everybody in the 2014 thread was hyped, but has anything ever 
happened in the language? In November 2017, the D forum discussed 

since?

In D, to prevent immense breakage, non-nullable class references 
need to be opt-in. I would love to see them and don't mind 
adapting my 25,000-line D-using project during a weekend.

Are there any counter-arguments to why non-nullable 
references/pointers haven't made it into D yet? Feel free to 
attack my answers below.

* * *

Argument: If A denotes non-null reference to class A, it can't 
have an init value.
Answer: Both A?.init and A.init shall be null, then use code-flow 
analysis.

This would match D's immutable: In a class constructor, you may 
assign the value 5 to a field of type immutable(int) that has 
init value 0. The compiler is happy as long as it can prove that 
we never write a second time during this constructor, and that we 
never read before the first assignment.

Likewise, it should be legal to assign from A to another A 
expression such as new A(), and the compiler is happy as long as 
the reference is assigned eventually, and if the reference is 
never read before assignment. (I haven't contributed to the 
compiler, I can't testify it's that easy.)

To allow hacks, it should remain legal to cast A? (nullable 
reference) to A (non-nullable). This should pass compilation 
(because casting takes all responsibility from the compiler) and 
then segfault at runtime, like any null dereference today.

* * *

Argument: I shall express non-null with contracts.
Answer: That's indeed the best solution without any language 
change. But it's bloaty and doesn't check anything at 
compile-time.

     class A { }
     void f1(A a) in { assert(a); } do { f2(a); }
     void f2(A a) in { assert(a); } do { f3(a); }
     void f3(A a) in { assert(a); } do { ...; }
     void g(A a) { if (a) ...; else ...; }

Sturdy D code must look like this today. Some functions handle 
the nulls, others request non-null refs from their callers. The 
function signature should express this, and a contract is part of 
the signature.

But several maintenance problems arise from non-null via contract.

First issue: We now rely on unit-testing to ensure our types are 
correct. You would do that in dynamic languages where the type 
system can't give you meaningful diagonstic errors otherwise. I'd 
rather not fall back to this in D. It's easy to forget such 
tests, coverage analysis doesn't help here.

Second issue: Introducing new fields requires updating all 
methods that uses the fields. This isn't necessarily only the 
methods in the class. If you have this code:

     class B {
         A a1;
         void f1() in { assert(a1); } do { ... }
         void f2() in { assert(a1); } do { ... }
     }

When you introduce more fields, you must update every method. 
This is bug-prone; we have final-switch (a full-blown language 
feature) just to solve similar issues:

     class B {
         A a1;
         A a2;
         void f1() in { assert(a1); assert(a2); } do { ... }
         void f2() in { assert(a1); /+ forgot +/ } do { ... }
     }

Third issue: Most references in a program aren't null. Especially 
class references that are fields of another class are often 
initialized in the constructor once, and never re-set. This is 
the predominant use of references. In D, the default, implicit 
case should do the Right Thing; it's fine when nonstandard 
features (allowing null) are explicit.

Assuming that A means non-null A, I would love this instead:

     class A { }
     void f1(A a) { f2(a); }
     void f2(A a) { f3(a); }
     void f3(A a) { ...; }
     void g(A? a) { if (a) ...; else ...; }
Or:
     void g(A  nullable a) { if (a) ...; else ...; }

Code-flow analysis can already statically check that we 
initialize immutable values only once. Likewise, it should check 
that we only pass A? to f1 after we have tested it for non-null, 
and that we only call methods on A? after checking for its 
non-null-ness (and the type of `a' inside the `if' block should 
probably still be A?, not A.)

* * *

Argument: null refs aren't a problem, they're memory-safe.
Answer: Memory-safety is not the concern here. Readability of 
code is, and preventing at compiletime what safely explodes at 
runtime.

* * *

Argument: Roll your own non-null type as a wrapper around D's 
nullable class reference.
Answer: That will look ugly, is an abstraction inversion, and 
checks at runtime only.

     class A { }

     struct NotNull(T)
         if (is(T == class))
     {
         T payload;
          disable this();
         this(T t) {
             assert(t !is null);
             payload = t;
         }
         alias payload this;
     }

     NotNull!A a = NotNull!A(new A());

The non-nullable type is type with simpler behavior, I can call 
all methods without segfault. The nullable type is the more 
complex type, I can either call methods on it or must check first 
for non-nullness. My NotNull implements a simple type in terms of 
a more complex type. Such abstraction inversion is dubious design.

And this solution would only assert at runtime again, not at 
compile time.

Microsoft's C++ Guideline Support Library has not_null<T>. That 
attacks the right problem, but becomes boilerplate when it 
appears everywhere in your codebase.

* * *

Argument: If A is going to denote non-null-A, then this will 
break huge amounts of code.
Answer: Like  safe, any such massive break must be opt-in.

The biggest downside of opt-in is that few projects will use it, 
and the feature will be buggy for a long time.

For example, associative arrays in opt-in  safe code together 
with overriding opEquals with  safe-nothrow-... annotations, all 
this can subtly fail if you mix it in complicated ways. 
Sometimes, you resort to ripping out the good annotations in your 
projects to please the compiler instead of dustmiting your 
project.

* * *

Argument: It's not worth it.

I firmly believe it's worth it, but I accept that others deem 
other things more important.

I merely happen to love OOP and use D classes almost everywhere, 
thus I have references everywhere, and methods everywhere that 
accept references as parameters.

-- Simon

I'll be happy to discuss this in person at DConf 2018. :-)
Feb 28 2018
next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, February 28, 2018 13:43:37 SimonN via Digitalmars-d wrote:

 Answer: Both A?.init and A.init shall be null, then use code-flow
 analysis.


I expect that pretty much anything you propose that requires code flow
analysis is DOA. Walter is almost always against features that require it,
because it's so hard to get right, and the places that D does use it tend to
have problems (e.g. it's actually quite trivial to use a const or immutable
member variable before it's initialized). In fact, IIRC, in the most recent
discussion on having the compiler give an error when it can detect that a
null pointer or reference is being dereferenced, Walter was arguing against
precisely because code-flow analysis is so hard to get right, and encoding
it in the spec is particularly bad (which would be required for anything
involving errors). If non-nullable references were added to D, I expect that
they would have to be like structs marked with

 disable this();

with all of the cons that go with that.


 Argument: It's not worth it.


I'm very much in that camp. I've never understood why some folks have so
many problems with null pointers. Personally, about the worst that I
normally have to deal with is forgetting to initialize class reference or
pointer, and that blows up quite quickly such that it's fixed quite quickly.
And that's in C++, D, Java, or any other language that I've used. Null
pointers/references are simply not something that I've ever seen much of a
problem with even when I use pointers heavily.

And as idiomatic D code tends to use classes rarely, it's that much less
useful for D than it would be for many other languages. I know that some
folks think that null is a huge problem, and some folks use classes or
pointers much more than idiomatic D typically does, but I definitely don't
think that adding a new type of pointer or reference to the language to try
to deal with null pointers/references is worth the extra complication. It's
a huge complication for what I personally believe is a small problem, though
obviously, not everyone agrees on that point. I have no idea what Walter and
Andrei's current stances on such an idea are other than the fact that Walter
is very much against using code-flow analysis for something like verifying
that a pointer or reference has been initialized before it's dereferenced.

- Jonathan M Davis
Feb 28 2018
next sibling parent reply SimonN <eiderdaus gmail.com> writes:
On Wednesday, 28 February 2018 at 14:05:19 UTC, Jonathan M Davis 
wrote:

 I expect that pretty much anything you propose that requires 
 code flow analysis is DOA.
 Walter was arguing against precisely because code-flow analysis 
 is so hard to get right,


Thanks, that's an important judgement. I've read the 3 threads 
that I found around this issue, but didn't notice this sentiment 
before that code-flow analysis is so problematic.

Yeah, non-null class fields hinge on code-flow analysis. And I'll 
accept that pushing non-null refs won't lead to anything if the 
necessary code-flow analysis is too tricky for the benefit.


 I've never understood why some folks have so many problems with 
 null pointers.


My gripe is that the necessarily-nullable class reference doesn't 
express the intent.
Either a codebase must rely on silent conventions or every 
function with asserts.


 and that blows up quite quickly such that it's fixed quite 
 quickly.


Yeah, I admit that most null crashes surface adequately quickly 
even when you have to run the program first.

It's merely sad to see D, with all its powerful static 
inspection, rely on runtime tests for nulls while other languages 

it's the most natural thing in the world.

-- Simon
Feb 28 2018
parent reply aliak <something something.com> writes:
On Wednesday, 28 February 2018 at 15:25:40 UTC, SimonN wrote:

 My gripe is that the necessarily-nullable class reference 
 doesn't express the intent.
 Either a codebase must rely on silent conventions or every 
 function with asserts.


I've put up a little experiment that you may be interested in for 
this part of the problem at least, also a little bit of compile 
time help because D treats dot on pointers and non pointers the 
same, it makes for some nice syntax:

class C {
     int i = 3;
}

auto a = no!C;

if (auto c = a.unwrap) {
     writeln("not here: ", c.i);
}
a.dispatch.i; // not ideal, but is at least safe no op.

a = some(new C());

if (auto c = a.unwrap) {
     writeln("here: ", c.i);
}

If you're interested: https://github.com/aliak00/optional

Syntax allowing:
Optional!C = null;

is not there yet, but I think is doable because typeof(null) is 
distinguishable. So with an optional type this can be well 
defined as "none" internally as well

Not ideal I know, but maybe a start to see where it can go?


 It's merely sad to see D, with all its powerful static 
 inspection, rely on runtime tests for nulls while other 

 compile-time, as if it's the most natural thing in the world.


Yeah, I whole heartedly agree with this. Maybe is an exposure 
thing? I remember seeing a talk (by I think the kickstarter 
people) that said after migrating to swift (from objective-c) 
their crash reports because dropped significantly (my brain for 
some reason remembers this to be zero, but I'm not sure and I 
can't figure out which talk this was).

Cheers,
- Ali
Mar 01 2018
parent reply SimonN <eiderdaus gmail.com> writes:
On Thursday, 1 March 2018 at 16:37:02 UTC, aliak wrote:

 I've put up a little experiment
 If you're interested: https://github.com/aliak00/optional


Nice! Optional is like std's Nullable with extra bells and 
whistles to make it as painless as you can offer it. I've read 
all of Optional's source, skimming the dispatching introspection 
magic, and it's a concise wrapper that is as transparent as it 
can get.

I can envision using this Optional type whenever I want nullable 
class reference, and use normal D class references as 
non-nullable references together with a codebase-wide rule that 
assigning null to normal D references is always a bug.

Even the generated compiler error messages are good: Optional!C x 
= some(new C()); int y = x; // Error: Cannot implicitly convert 
from Optional!C to C.


 a.dispatch.i; // not ideal, but is at least safe no op.


Hah, I've toyed with some examples, and this stuck out as verbose 
because everything else was really smooth. Optional!int 
dispatches wonderfully without extra syntax, whereas class C {}; 
Optional!C needs the extra dispatch before calling methods.

-- Simon
Mar 02 2018
parent reply aliak <something something.com> writes:
On Friday, 2 March 2018 at 19:47:23 UTC, SimonN wrote:

 I can envision using this Optional type whenever I want 
 nullable class reference, and use normal D class references as 
 non-nullable references together with a codebase-wide rule that 
 assigning null to normal D references is always a bug.


Ay, maybe you can even have a type that enforces non nullable 
references?

(not tested):
struct NonNull(T) if (isPointer!T || is(T == class))
{
   T value;
   alias value this;
   this(Args...)(Args args) { this.value = new T(args); } // 
always force creation
}



 Hah, I've toyed with some examples, and this stuck out as 
 verbose because everything else was really smooth. Optional!int 
 dispatches wonderfully without extra syntax, whereas class C 
 {}; Optional!C needs the extra dispatch before calling methods.


Yeah, it's unfortunate. If D had custom operators then it 
could've been  a?.f or whatever :(

So I had to either have a custom function that starts a dispatch 
chain or implement opDispatch directly inside Optional. The 
problem with the latter is that means if type T has any of the 
functions that are part of Optional's interface, they are 
basically un-callable unless unwrapped first. I found this 
unacceptable.

I'm going to experiment with moving Optional's methods out as 
free functions when I get the chance. Then the dynamics between 
UFCS/dispatch become a bit weird. And I'm not sure how to work 
range functionality in as free functions, but let's see.

If you know of other ways though I'm all ears :)

Cheers
Mar 03 2018
next sibling parent reply arturg <var.spool.mail700 gmail.com> writes:
On Saturday, 3 March 2018 at 18:28:42 UTC, aliak wrote:

 On Friday, 2 March 2018 at 19:47:23 UTC, SimonN wrote:

 If you know of other ways though I'm all ears :)

 Cheers


maybe not exactly what you want, but here are some templates i 
wrote a while ago which basically are a more flexible form of the 
?. operator.

https://run.dlang.io/gist/598c59506699a0f81c3abbbaf7d5feee?compiler=dmd
Mar 03 2018
parent aliak <something something.com> writes:
On Saturday, 3 March 2018 at 19:20:26 UTC, arturg wrote:

 On Saturday, 3 March 2018 at 18:28:42 UTC, aliak wrote:

 On Friday, 2 March 2018 at 19:47:23 UTC, SimonN wrote:

 If you know of other ways though I'm all ears :)

 Cheers


 maybe not exactly what you want, but here are some templates i 
 wrote a while ago which basically are a more flexible form of 
 the ?. operator.

 https://run.dlang.io/gist/598c59506699a0f81c3abbbaf7d5feee?compiler=dmd


Aye, nice, I might steal that when stuff :D I also want to 
implement something like 
https://dlang.org/library/std/variant/visit.html

so Optional!T a;
a.visit!(
   (T value) => value here,
   () => no value here
);
Mar 04 2018
prev sibling parent reply SimonN <eiderdaus gmail.com> writes:
On Saturday, 3 March 2018 at 18:28:42 UTC, aliak wrote:

 struct NonNull(T) if (isPointer!T || is(T == class))
 {
   T value;
   alias value this;
   this(Args...)(Args args) { this.value = new T(args); } // 
 always force creation
 }


The pitfall here is that all structs must be 
default-constructible, and then all of the fields have the static 
init value:

     class A { this(int) { } }
     void main()
     {
         NonNull!A a;
         assert (a.value is null);
     }

...compiles and passes.

Wrapping class references in structs is costly, we lose a lot of 
automatic conversions across subclasses, const/immutable, ... For 
the over 90 % of references where null is a bug (sign of 
forgotten assignment), it will be convenient to keep D's 
references. A library solution must be super-convenient to be 
worthwhile here. For weeks, I've put off re-discussing non-null 
refs directly in D because a language change is a last resort.


 I'm going to experiment with moving Optional's methods out
 If you know of other ways though I'm all ears :)


No ideas for now, but I'll certainly follow this development 
closely, and will be happy to test things!

-- Simon
Mar 03 2018
parent reply aliak <something something.com> writes:
On Saturday, 3 March 2018 at 20:52:25 UTC, SimonN wrote:


 The pitfall here is that all structs must be 
 default-constructible, and then all of the fields have the 
 static init value:

     class A { this(int) { } }
     void main()
     {
         NonNull!A a;
         assert (a.value is null);
     }

 ...compiles and passes.


Maybe this part can be fixed with:

struct NonNull(T) if (isPointer!T || is(T == class))
{
     static if (isPointer!T)
     	alias U = PointerTarget!T;
     else
         alias U = T;

     T value = new U();

     alias value this;

      disable this(typeof(null)); // no no

     this(Args...)(Args args) {
         this.value = new U(args);
     }

      disable opAssign(V)(V v) {} // no no no

     void opAssign(V)(NonNull!V other) { // only this allowed.
         this.value = other.value;
     }
  }

Tested a bit this time and added a few extra checks. (thinking 
out loud: Maybe the constraint should only be is(T == class) 
actually and we can keep this to reference types...)

The above will also fail right now if T is a class with a non 
default init. Maybe there's a way to fix that as well though.



 Wrapping class references in structs is costly, we lose a lot 
 of automatic conversions across subclasses, const/immutable,


Ok this part would require some testing and playing around, can't 
really think of anything off the top of my head.

Can you provide an example case here? It might be workable with 
some (hurts my head to think about right now) combinations of 
auto ref/inout on initializers or opAssigns or something.


 No ideas for now, but I'll certainly follow this development 
 closely, and will be happy to test things!


Glad to hear!
Mar 04 2018
parent reply SimonN <eiderdaus gmail.com> writes:
On Sunday, 4 March 2018 at 12:03:32 UTC, aliak wrote:

 Maybe this part can be fixed with:
 struct NonNull(T) if (isPointer!T || is(T == class))
 {
      disable opAssign(V)(V v) {} // no no no
     void opAssign(V)(NonNull!V other) { // only this allowed.


Interesting approach -- I'll keep it on the backburner for 
testing. Converting a class-heavy codebase to this would be 
daunting, but I don't want to rule it out yet.


 Wrapping class references in structs is costly, we lose a lot


 Can you provide an example case here?


Automatic subtype detection will fail, and, thus, e.g., covariant 
return types.

To make it clearer that this isn't a bug in Optional, but rather 
an artifact of D's subtype detection, I will give examples with 
std.typecons.Rebindable instead of with Optional. For a class A, 
Rebindable!(const(A)) is designed as a reference to const(A) that 
can be rebound to point to a different const(A). I believe 
Rebindable!A aliases itself away and thus Rebindable!A will not 
be discussed, only Rebindable!(const(A)).

     import std.typecons;
     class Base { }
     class Derived : Base { }
     static assert(is(const(Derived) : const(Base))); // OK
     static assert(is(Rebindable!(const(Derived)) : 
Rebindable!(const(Base)))); // fails

Covariant return types will also fail. The following example 
doesn't compile, but would if we stripped all `rebindable' and 
all `Rebindable!':

     import std.typecons;
     class Base {
         Rebindable!(const(Base)) create()
         {
             return rebindable(new Base());
         }
     }
     class Derived : Base {
         override Rebindable!(const(Derived)) create()
         {
             return rebindable(new Derived());
         }
     }

Speculation: There may be problems once we try to wrap class 
references in two different structs. We might want Optional or 
NonNull, and then another wrapper (e.g., Rebindable) for some 
other benefit. When these wrappers are written with 
if-constraints to take only raw D class references, we may pick 
only one wrapper. But I haven't investigated this yet.

-- Simon
Mar 04 2018
parent reply aliak <something something.com> writes:
On Sunday, 4 March 2018 at 13:26:21 UTC, SimonN wrote:

 On Sunday, 4 March 2018 at 12:03:32 UTC, aliak wrote:

 Maybe this part can be fixed with:
 struct NonNull(T) if (isPointer!T || is(T == class))
 {
      disable opAssign(V)(V v) {} // no no no
     void opAssign(V)(NonNull!V other) { // only this allowed.


 Interesting approach -- I'll keep it on the backburner for 
 testing. Converting a class-heavy codebase to this would be 
 daunting, but I don't want to rule it out yet.


Heh yeah, it would be indeed!


 Automatic subtype detection will fail, and, thus, e.g., 
 covariant return types.

 To make it clearer that this isn't a bug in Optional, but 
 rather an artifact of D's subtype detection, I will give 
 examples with std.typecons.Rebindable instead of with Optional. 
 For a class A, Rebindable!(const(A)) is designed as a reference 
 to const(A) that can be rebound to point to a different 
 const(A). I believe Rebindable!A aliases itself away and thus 
 Rebindable!A will not be discussed, only Rebindable!(const(A)).

     import std.typecons;
     class Base { }
     class Derived : Base { }
     static assert(is(const(Derived) : const(Base))); // OK
     static assert(is(Rebindable!(const(Derived)) : 
 Rebindable!(const(Base)))); // fails

 Covariant return types will also fail. The following example 
 doesn't compile, but would if we stripped all `rebindable' and 
 all `Rebindable!':

     import std.typecons;
     class Base {
         Rebindable!(const(Base)) create()
         {
             return rebindable(new Base());
         }
     }
     class Derived : Base {
         override Rebindable!(const(Derived)) create()
         {
             return rebindable(new Derived());
         }
     }



Ah, thanks, that cleared things up!

Does this maybe boil down to if two templates should be covariant 
on their parameter types? Considering that static if allows you 
to do whatever you want inside based on static inference, in 
theory Rebindable!(const Derived) can be completely different 
than Rebindable(const Base). So are covariant return types even 
possible I wonder?

If D had implicit conversions then types can specifically say 
that they explicitly allow behavior like this where it made sense 
though.


 Speculation: There may be problems once we try to wrap class 
 references in two different structs. We might want Optional or 
 NonNull, and then another wrapper (e.g., Rebindable) for some 
 other benefit. When these wrappers are written with 
 if-constraints to take only raw D class references, we may pick 
 only one wrapper. But I haven't investigated this yet.

 -- Simon


Aye, I guess Optional and NonNull would have to know about each 
other to make the most out of them together. I've actually been 
tempted to put in a NonOptional type inside the optional package. 
But maybe NonNull would make more sense since we may just want 
this behavior for reference types. and NonOptional!int is kinda 
... well wtf is that :p

Cheers
- Ali
Mar 11 2018
parent SimonN <eiderdaus gmail.com> writes:
On Sunday, 11 March 2018 at 09:28:39 UTC, aliak wrote:

 Does this maybe boil down to if two templates should be 
 covariant on their parameter types?


I'm not sure if this is always good. I haven't thought about it 
deeply, but I assume that some templated functions should be 
contravariant, and that there is no catch-all rule.


 inference, in theory Rebindable!(const Derived) can be 
 completely different than Rebindable(const Base). So are 
 covariant return types even possible I wonder?


I don't think covariant return types will work here with the 
2018-03 language spec. We're returning structs 
(Rebindable!Something) and covariant return types is specifically 
for classes. :-)

It's one of the reasons why I shun wrapping types unless the 
benefit is really big.


 If D had implicit conversions then types can specifically say 
 that they explicitly allow behavior like this where it made 
 sense though.


Yeah -- it shifts another burden of implementation to library 
type authors, I'm not sure if this path should be followed, 
especially since implicit constructor calls are shunned in D. (I 
have no opinion whether C++'s implicit constructor calls are 
handy enough for the obscured program flow.)

* * *

Here's another example where wrapped class references fail: You 
can override opEquals in the class, and the struct's opEquals 
need not necessarily call that.

     import std.stdio;
     import std.typecons;

     class C {
         int x;
         override bool opEquals(Object rhsObj)
         {
             const(C) rhs = cast(const(C)) rhsObj;
             return this.x == rhs.x;
         }
     }

     void main()
     {
         C a = new C();
         C b = new C();
         assert (a == b); // OK: Even though a !is b, we overrode 
opEquals
                          // and compare a.x == b.x, which is true 
(0 == 0).

         Rebindable!(const(C)) ca = a;
         Rebindable!(const(C)) cb = b;
         assert (ca == cb); // Fails! struct Rebindable doesn't 
forward its
                            // opEquals to the class's opEquals!
     }

If this is by design, then it's very surprising. I've reported 
this as a bug to get a judgement from the Phobos authors: 
https://issues.dlang.org/show_bug.cgi?id=18615

-- Simon
Mar 14 2018
prev sibling next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Wednesday, 28 February 2018 at 14:05:19 UTC, Jonathan M Davis 
wrote:

 On Wednesday, February 28, 2018 13:43:37 SimonN via 
 Digitalmars-d wrote:

 [...]


 I expect that pretty much anything you propose that requires 
 code flow analysis is DOA. Walter is almost always against 
 features that require it, because it's so hard to get right, 
 and the places that D does use it tend to have problems (e.g. 
 it's actually quite trivial to use a const or immutable member 
 variable before it's initialized). In fact, IIRC, in the most 
 recent discussion on having the compiler give an error when it 
 can detect that a null pointer or reference is being 
 dereferenced, Walter was arguing against precisely because 
 code-flow analysis is so hard to get right, and encoding it in 
 the spec is particularly bad (which would be required for 
 anything involving errors). If non-nullable references were 
 added to D, I expect that they would have to be like structs 
 marked with

 [...]


I don't understand the problems with null either - my program 
segfaults, I look at the core dump and initialise whatever it is 
that was T.init. And this in the rare case I actually use a 
pointer or a class instance to begin with.

I also declare nearly every variable with `const var = <expr>;`, 
so I guess that makes it even more unlikely for me.

Atila
Feb 28 2018
prev sibling next sibling parent reply Kagamin <spam here.lot> writes:
On Wednesday, 28 February 2018 at 14:05:19 UTC, Jonathan M Davis 
wrote:

 Walter is almost always against features that require it, 
 because it's so hard to get right


Doesn't difficulty depend on what exactly to get right? It's not 
a spherical problem in vacuum.
Feb 28 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Wednesday, February 28, 2018 19:43:07 Kagamin via Digitalmars-d wrote:

 On Wednesday, 28 February 2018 at 14:05:19 UTC, Jonathan M Davis

 wrote:

 Walter is almost always against features that require it,
 because it's so hard to get right


 Doesn't difficulty depend on what exactly to get right? It's not
 a spherical problem in vacuum.


Feel free to discuss any code-flow analysis issues with Walter, but he's
consistently been against using it for much of anything whenever I've seen
him discuss it. Not even stuff like VRP covers multiple lines of code,
because doing so would require code-flow analysis. And he's specifically
said that he's against using it for detecting when a null pointer is
dereferenced. IIRC, he's stated that dmd's optimizer uses code-flow analysis
for some stuff, but for anything that involves putting it in the frontend
where the behavior would have to be encoded in the spec, he's been against
it.

- Jonathan M Davis
Feb 28 2018
next sibling parent Kagamin <spam here.lot> writes:
On Wednesday, 28 February 2018 at 23:58:44 UTC, Jonathan M Davis 
wrote:

 Feel free to discuss any code-flow analysis issues with Walter, 
 but he's consistently been against using it for much of 
 anything whenever I've seen him discuss it.


I'd say massive breakage is what precludes it. Well, I like the 
way default initialization works in D.


 Not even stuff like VRP covers multiple lines of code, because 
 doing so would require code-flow analysis.


It feels like handling VRP for linear code can be simple enough, 
tracking VRP across branching is more of an overkill.
Mar 01 2018
prev sibling next sibling parent reply aliak <something something.com> writes:
On Wednesday, 28 February 2018 at 23:58:44 UTC, Jonathan M Davis 
wrote:

 On Wednesday, February 28, 2018 19:43:07 Kagamin via 
 Digitalmars-d wrote:

 On Wednesday, 28 February 2018 at 14:05:19 UTC, Jonathan M 
 Davis

 wrote:

 [...]


 Doesn't difficulty depend on what exactly to get right? It's 
 not a spherical problem in vacuum.


 Feel free to discuss any code-flow analysis issues with Walter, 
 but he's consistently been against using it for much of 
 anything whenever I've seen him discuss it. Not even stuff like 
 VRP covers multiple lines of code, because doing so would 
 require code-flow analysis. And he's specifically said that 
 he's against using it for detecting when a null pointer is 
 dereferenced. IIRC, he's stated that dmd's optimizer uses 
 code-flow analysis for some stuff, but for anything that 
 involves putting it in the frontend where the behavior would 
 have to be encoded in the spec, he's been against it.

 - Jonathan M Davis


Isn't DIP1000 [1] basically code flow analysis and implemented 
(kinda?) in dmd with -dip1000 now?

[1] https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md
Mar 01 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, March 01, 2018 14:59:38 aliak via Digitalmars-d wrote:

 On Wednesday, 28 February 2018 at 23:58:44 UTC, Jonathan M Davis

 wrote:

 On Wednesday, February 28, 2018 19:43:07 Kagamin via

 Digitalmars-d wrote:

 On Wednesday, 28 February 2018 at 14:05:19 UTC, Jonathan M
 Davis

 wrote:

 [...]


 Doesn't difficulty depend on what exactly to get right? It's
 not a spherical problem in vacuum.


 Feel free to discuss any code-flow analysis issues with Walter,
 but he's consistently been against using it for much of
 anything whenever I've seen him discuss it. Not even stuff like
 VRP covers multiple lines of code, because doing so would
 require code-flow analysis. And he's specifically said that
 he's against using it for detecting when a null pointer is
 dereferenced. IIRC, he's stated that dmd's optimizer uses
 code-flow analysis for some stuff, but for anything that
 involves putting it in the frontend where the behavior would
 have to be encoded in the spec, he's been against it.

 - Jonathan M Davis


 Isn't DIP1000 [1] basically code flow analysis and implemented
 (kinda?) in dmd with -dip1000 now?

 [1] https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


DIP 1000 operates based on the type. As I understand it, it looks at the
fact that something is scope or not and then determines whether a particular
operation is valid or not based on whether the operation could result in a
reference to the data escaping. An operation is then valid or not regardless
of what other lines in the code are doing. As such, if I understand
correctly, code-flow analysis isn't really necessary, just like code-flow
analysis isn't necessary to determine what's valid or not when const is
involved.

- Jonathan M Davis
Mar 01 2018
parent aliak <something something.com> writes:
On Thursday, 1 March 2018 at 19:10:29 UTC, Jonathan M Davis wrote:

 DIP 1000 operates based on the type. As I understand it, it 
 looks at the fact that something is scope or not and then 
 determines whether a particular operation is valid or not based 
 on whether the operation could result in a reference to the 
 data escaping. An operation is then valid or not regardless of 
 what other lines in the code are doing. As such, if I 
 understand correctly, code-flow analysis isn't really 
 necessary, just like code-flow analysis isn't necessary to 
 determine what's valid or not when const is involved.

 - Jonathan M Davis


Ah right, makes sense yep. Thanky!
Mar 01 2018
prev sibling parent Jacob Carlborg <doob me.com> writes:
On Wednesday, 28 February 2018 at 23:58:44 UTC, Jonathan M Davis 
wrote:

 he's stated that dmd's optimizer uses code-flow analysis for 
 some stuff, but for anything that involves putting it in the 
 frontend where the behavior would have to be encoded in the 
 spec, he's been against it.


The compiler can actually detect simple cases of null 
dereferences if optimizations are turned on, "-O", due to the 
optimizer using flow analysis.

--
/Jacob Carlborg
Mar 01 2018
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Wednesday, 28 February 2018 at 14:05:19 UTC, Jonathan M Davis 
wrote:

 I expect that pretty much anything you propose that requires 
 code flow analysis is DOA. Walter is almost always against 
 features that require it, because it's so hard to get right, 
 and the places that D does use it tend to have problems (e.g. 
 it's actually quite trivial to use a const or immutable member 
 variable before it's initialized).


Honestly, this is not that hard. It's very hard in DMD because it 
doesn't go through an SSA like form at any point. It's rather 
disappointing to see the language spec being decided upon based 
on design decision made in a compiler many years ago.
Mar 02 2018
parent reply Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 2 March 2018 at 09:59:53 UTC, deadalnix wrote:

 Honestly, this is not that hard. It's very hard in DMD because 
 it doesn't go through an SSA like form at any point. It's 
 rather disappointing to see the language spec being decided 
 upon based on design decision made in a compiler many years ago.


And how easy is transformation into SSA ?
that's not all loads and stores ?
Mar 02 2018
parent Paolo Invernizzi <paolo.invernizzi gmail.com> writes:
On Friday, 2 March 2018 at 10:41:05 UTC, Stefan Koch wrote:

 On Friday, 2 March 2018 at 09:59:53 UTC, deadalnix wrote:

 Honestly, this is not that hard. It's very hard in DMD because 
 it doesn't go through an SSA like form at any point. It's 
 rather disappointing to see the language spec being decided 
 upon based on design decision made in a compiler many years 
 ago.


 And how easy is transformation into SSA ?
 that's not all loads and stores ?


It's the single assignment the key...
Mar 02 2018
prev sibling next sibling parent reply Mike Franklin <slavo5150 yahoo.com> writes:
On Wednesday, 28 February 2018 at 13:43:37 UTC, SimonN wrote:

 Hi,

 Andrei said in 2014 that not-null-references should be the 
 priority of 2014's language design, with consideration to make 
 not-null the default. In case the code breakage is too high, 
 this can be an opt-in compiler flag.


You might be interested in this little experiment:  
https://github.com/dlang/dmd/pull/7375

Mike
Feb 28 2018
parent SimonN <eiderdaus gmail.com> writes:
On Wednesday, 28 February 2018 at 15:29:17 UTC, Mike Franklin 
wrote:

 You might be interested in this little experiment:  
 https://github.com/dlang/dmd/pull/7375


Indeed, this looks extremely useful, at the very least in a 
linter. I probably rely on ints getting initialized to 0 
throughout the program, and only rarely make that explicit.

With null references, the problem is not forgetting the 
initialization; it's expressing the intent of the variable. 
Usually, I want the non-null, but sometimes, I want a nullable 
reference and would like to require the using code to test the 
reference for null. Merely verifying for initialization doesn't 
help here; it may well be intended that null will be assigned 
later to the reference.

-- Simon
Feb 28 2018
prev sibling parent Chris M. <chrismohrfeld comcast.net> writes:
On Wednesday, 28 February 2018 at 13:43:37 UTC, SimonN wrote:

 Hi,

 Andrei said in 2014 that not-null-references should be the 
 priority of 2014's language design, with consideration to make 
 not-null the default. In case the code breakage is too high, 
 this can be an opt-in compiler flag.

 [...]


I've slowly come around to supporting this idea. I'd rather avoid 
segfaults in the first place and avoid extra effort checking for 
null if possible.

It also sets clearer expectations for a user. For example, D now:

Class func(T param); // user always needs to worry about if the 
return value is null or not, there may be that edge case where it 
is null

D with non-nullable references:

Class func(T param); // user knows that the return value will not 
be null, no need to check
Nullable!Class func(T param); // user knows they need to check 
for null and handle it.

That's my two cents anyways
Feb 28 2018