www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Blogpost about the T.init problem

reply FeepingCreature <feepingcreature gmail.com> writes:
I've written up a short blogpost about the T.init issue.

It is not very enthusiastic.

https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

Related links:

https://github.com/dlang/phobos/pull/6594 problem with T.init and 
toString

https://github.com/dlang/phobos/pull/6619 Nullable can't work 
with types where T.init violates invariants

https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
disable invariant on struct ~this
Jul 10 2018
next sibling parent reply Dukc <ajieskola gmail.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 I've written up a short blogpost about the T.init issue.


I believe that whoever wrote that spec meant that the invariant 
WOULD not need to hold if MyDomainData.init WAS called, but that 
MyDomainData.init must not be called if this is the case.

It definitely needs clarification if I understood it's intent 
right.
Jul 10 2018
parent Dukc <ajieskola gmail.com> writes:
On Tuesday, 10 July 2018 at 15:10:21 UTC, Dukc wrote:

 It definitely needs clarification if I understood it's intent 
 right.


https://github.com/dlang/dlang.org/pull/2418
Jul 11 2018
prev sibling next sibling parent Meta <jared771 gmail.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

 Related links:

 https://github.com/dlang/phobos/pull/6594 problem with T.init 
 and toString

 https://github.com/dlang/phobos/pull/6619 Nullable can't work 
 with types where T.init violates invariants

 https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
 disable invariant on struct ~this


D's contract programming features have been around since D1 and 
haven't really been updated to work with newer D2 features (this 
is no excuse IMO; these issues should have been addressed long 
ago).

My opinion is that it's a vicious circle; D's contract 
programming features are underutilized, thus nobody cares enough 
to put effort into ironing out the bugs, thus contract 
programming in D is buggy and interacts poorly with other 
language features, thus D's contract programming features are 
underutilized.

IMO, if you don't have the knowledge, desire, or time to fix 
them, the next best thing to do is write articles like these 
bringing some attention to the various defects of contract 
programming in D, and/or write DIPs to propose ways to fix it.
Jul 10 2018
prev sibling next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

 Related links:

 https://github.com/dlang/phobos/pull/6594 problem with T.init 
 and toString

 https://github.com/dlang/phobos/pull/6619 Nullable can't work 
 with types where T.init violates invariants

 https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
 disable invariant on struct ~this


First of all I must point that I would very much like to have 
seen a code actually producing an error in that article. Contrary 
to what is hinted just taking the struct and putting using it 
with Nullable or format() caused no error for me and worked as 
expected. Taking .init explicitely was the only thing that 
actually caused an error. I'm not saying you didn't experience 
these issues, but if you want to demonstrate a problem then 
please demonstrate it.

That said, I may be missing something obvious but what prevents 
you from overloading the init field?

     struct MyDomainData {
         string username;
          disable this(); // don't make a MyDomainData() by 
accident!
         this(string username)
         in(!username.empty) // only non-empty usernames please!
         do { this.username = username; }
         // let's formalise the restriction.
         invariant { assert(!username.empty); }
         string toString() { ... }

         static  property MyDomainData init() {
             return MyDomainData("uninitialized");
         }

         ...
     }

     auto test = MyDomainData.init;  // There, no error

Of course that value means nothing but .init isn't meant to 
actually mean something anyway, it's just a valid value and 
that's what that init is proposing, so it shouldn't cause any 
more bugs than empty .init in a normal case.
Jul 10 2018
parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Tuesday, 10 July 2018 at 21:08:32 UTC, Cym13 wrote:

 First of all I must point that I would very much like to have 
 seen a code actually producing an error in that article. 
 Contrary to what is hinted just taking the struct and putting 
 using it with Nullable or format() caused no error for me and 
 worked as expected.


To reproduce the format issue, try to print the struct with 
writefln!"%s"(MyDomainType()).

To reproduce the Nullable issue, you need to slightly modify the 
struct. In Phobos, Nullable will (due to an abundance of caution) 
refuse to initialize the struct if the default constructor is 
disabled; also you need a destructor. However, for this it is 
enough to use any type that has a destructor, so that an implicit 
struct destructor is generated. For verbosity, I'll write it out:


struct MyDomainData {
     string username;

     this(string username)  safe
     in(!username.empty) // only non-empty usernames please!
     do { this.username = username; }

     // let's formalise the restriction.
     invariant { assert(!username.empty); }

     string toString() { return null; }

     ~this()  safe { }
}

Then just stick it in a Nullable. No explicit .init needed.


 That said, I may be missing something obvious but what prevents 
 you from overloading the init field?

     struct MyDomainData {
         string username;
          disable this(); // don't make a MyDomainData() by 
 accident!
         this(string username)
         in(!username.empty) // only non-empty usernames please!
         do { this.username = username; }
         // let's formalise the restriction.
         invariant { assert(!username.empty); }
         string toString() { ... }

         static  property MyDomainData init() {
             return MyDomainData("uninitialized");
         }

         ...
     }

     auto test = MyDomainData.init;  // There, no error

 Of course that value means nothing but .init isn't meant to 
 actually mean something anyway, it's just a valid value and 
 that's what that init is proposing, so it shouldn't cause any 
 more bugs than empty .init in a normal case.


That would work, it's just a really horrible hack and I hate it. 
We're constructing a fictitious domain value that passes our 
invariants while having zero correspondence to the real world, 
*just to pass our invariants*. It's an obvious sign of a language 
issue.
Jul 11 2018
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature wrote:

 Then just stick it in a Nullable. No explicit .init needed.


To clarify this point some more, since on reflection it's 
ambiguous: you might well say that "well yeah, the default 
constructor returns an invalid value, no shit it breaks." The 
semantics of Nullable are weird here though - Nullable!S 
constructs an S while pretending to not contain an S. The deeper 
problem is that there is straight up *no way* to implement 
Nullable correctly in a way that lets it handle types with 
 disabled this(); without using pointers there's no way to bypass 
S's destructor, and any implementation of Nullable that uses 
T.init explicitly dies when D tries to destruct it.


 That would work, it's just a really horrible hack and I hate 
 it. We're constructing a fictitious domain value that passes 
 our invariants while having zero correspondence to the real 
 world, *just to pass our invariants*. It's an obvious sign of a 
 language issue.


Furthermore, note that this limits the amount of invariants we 
can define. For instance, we can't define an invariant that says 
that a value has to be registered in a central table somewhere - 
since our fictional value obviously won't be. Better to do bool 
isInitialized = false, and that's already crappy.
Jul 11 2018
next sibling parent reply FeepingCreature <feepingcreature gmail.com> writes:
Destructors are not called for fields embedded in unions.

On the one hand this is a horrible, horrible hack. On the other, 
whee!
Jul 11 2018
parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Wednesday, 11 July 2018 at 10:43:40 UTC, FeepingCreature wrote:

 Destructors are not called for fields embedded in unions.

 On the one hand this is a horrible, horrible hack. On the 
 other, whee!


Not only a horrible hack, but a bug, and something you really 
can't rely on working in the future. Especially since this 
behavior is only observed with named unions, not with anonymous 
ones.

I'll also mention section 14.19 2 on the Structs, Unions page 
(https://dlang.org/spec/struct.html):

2. Unions may not have fields that have destructors.

Whether that means it's undefined behavior or the compiler should 
statically disallow it is up for debate, I guess.

--
   Simen
Jul 12 2018
parent FeepingCreature <feepingcreature gmail.com> writes:
On Thursday, 12 July 2018 at 08:54:17 UTC, Simen Kjærås wrote:

 Whether that means it's undefined behavior or the compiler 
 should statically disallow it is up for debate, I guess.

 --
   Simen


Honestly, half the reason I'm using it so enthusiastically is 
that I want to emphasize that this is an important usecase that 
can't be neglected, so that if unions are ever fixed they give us 
some other way to manually handle struct destruction, like 
 managed or  nodestroy.
Jul 12 2018
prev sibling parent reply Meta <jared771 gmail.com> writes:
On Wednesday, 11 July 2018 at 07:35:24 UTC, FeepingCreature wrote:

 On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature 
 wrote:

 Then just stick it in a Nullable. No explicit .init needed.


 To clarify this point some more, since on reflection it's 
 ambiguous: you might well say that "well yeah, the default 
 constructor returns an invalid value, no shit it breaks." The 
 semantics of Nullable are weird here though - Nullable!S 
 constructs an S while pretending to not contain an S. The 
 deeper problem is that there is straight up *no way* to 
 implement Nullable correctly in a way that lets it handle types 
 with  disabled this();


I hate to say I told you so, but... 
https://github.com/dlang/phobos/pull/5855#issuecomment-345783238

Just joking, of course =)

Nullable has needed to be completely overhauled for a long time 
because it was only really designed with POD types in mind.
Jul 11 2018
parent FeepingCreature <feepingcreature gmail.de> writes:
On Wednesday, 11 July 2018 at 20:10:17 UTC, Meta wrote:

 I hate to say I told you so, but... 
 https://github.com/dlang/phobos/pull/5855#issuecomment-345783238

 Just joking, of course =)

 Nullable has needed to be completely overhauled for a long time 
 because it was only really designed with POD types in mind.


Good news is the union hack seems to be working actually ( 
https://github.com/dlang/phobos/pull/6619 ). See the Turducken 
post in General ( 
https://forum.dlang.org/thread/ekbxqxhnttihkoszzvxl forum.dlang.org ); I didn't
use the full Turducken in the PR for Phobos because I don't think Nullable is
specced to be able to handle reassigning const data anyway, so the union assign
in moveEmplace seems sufficient.
Jul 11 2018
prev sibling next sibling parent reply Cym13 <cpicard openmailbox.org> writes:
On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature wrote:

 That would work, it's just a really horrible hack and I hate it.


Bastiaan's solution to simply change the default value slipped my 
mind but is really cleaner and in the same line of thought.


 We're constructing a fictitious domain value that passes our 
 invariants while having zero correspondence to the real world, 
 *just to pass our invariants*. It's an obvious sign of a 
 language issue.


I'm not sure I understand, that's what T.init is: a fictitious 
domain value that just happens to be the default value. It 
doesn't have to have any meaning and shouldn't be used that way. 
It's just a value until it has a value. If it happens to be 
conveniently a useful value, all right, but that's not its first 
goal IIUC.

To present things the other way: you are defining constraints on 
a type while also defining the default value of that type as not 
meeting these contraints. No matter how you look at it the 
default value of a type should be a valid value. How is that not 
an issue with your own code? Just change the default so that it 
is within the constraints.

Furthermore, while changing the default field value directly is 
less of a hack the solution to redefine init() entirely actually 
allows you to do things like making sure the struct is registered 
in a table somewhere. So I think you do have the option to meet 
your invariants.
Jul 12 2018
parent FeepingCreature <feepingcreature gmail.com> writes:
On Friday, 13 July 2018 at 01:26:28 UTC, Cym13 wrote:

 I'm not sure I understand, that's what T.init is: a fictitious 
 domain value that just happens to be the default value. It 
 doesn't have to have any meaning and shouldn't be used that 
 way. It's just a value until it has a value. If it happens to 
 be conveniently a useful value, all right, but that's not its 
 first goal IIUC.


T.init is a fictitious but valid instance of the type domain, but 
it isn't necessarily a valid instance of the *data* domain that 
the value represents; that's why we can  disable this() and 
invariant in the first place, to impose additional restrictions 
that are not modeled by the typesystem.


 To present things the other way: you are defining constraints 
 on a type while also defining the default value of that type as 
 not meeting these contraints. No matter how you look at it the 
 default value of a type should be a valid value. How is that 
 not an issue with your own code? Just change the default so 
 that it is within the constraints.


It's impossible to have a fixed value that is a valid value of a 
type that's semantically embedded in a dynamic data structure.

Null will never be a valid class instance.


 Furthermore, while changing the default field value directly is 
 less of a hack the solution to redefine init() entirely 
 actually allows you to do things like making sure the struct is 
 registered in a table somewhere.


At which point, not content to ruin our type with invalid "valid" 
data, we've ruined the rest of our runtime state as well!


 So I think you do have the option to meet your invariants.


Sure, at the cost of making them worthless.
Jul 13 2018
prev sibling next sibling parent reply Nick Treleaven <nick geany.org> writes:
On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature wrote:

 To reproduce the format issue, try to print the struct with 
 writefln!"%s"(MyDomainType()).


I implemented the compile time format string checking by 
evaluating `format(fmtStr, Args.init)` at compile time and seeing 
if an exception was thrown. So the above problem could be worked 
around by guarding the invariant test with a check that CTFE is 
not active.
Jul 23 2018
parent FeepingCreature <feepingcreature gmail.com> writes:
On Monday, 23 July 2018 at 14:26:17 UTC, Nick Treleaven wrote:

 On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature 
 wrote:

 To reproduce the format issue, try to print the struct with 
 writefln!"%s"(MyDomainType()).


 I implemented the compile time format string checking by 
 evaluating `format(fmtStr, Args.init)` at compile time and 
 seeing if an exception was thrown. So the above problem could 
 be worked around by guarding the invariant test with a check 
 that CTFE is not active.


Note that I've gotten a fix for this merged that doesn't break 
format use in ctfe: the compile time format string check simply 
writes into a NoOpSink, and the formattedWriter detects the 
NoOpSink and skips the toString.

So that's good now.
Jul 27 2018
prev sibling parent Jim Balter <Jim Balter.name> writes:
On Wednesday, 11 July 2018 at 07:30:59 UTC, FeepingCreature wrote:

 On Tuesday, 10 July 2018 at 21:08:32 UTC, Cym13 wrote:

 First of all I must point that I would very much like to have 
 seen a code actually producing an error in that article. 
 Contrary to what is hinted just taking the struct and putting 
 using it with Nullable or format() caused no error for me and 
 worked as expected.


 To reproduce the format issue, try to print the struct with 
 writefln!"%s"(MyDomainType()).

 To reproduce the Nullable issue, you need to slightly modify 
 the struct. In Phobos, Nullable will (due to an abundance of 
 caution) refuse to initialize the struct if the default 
 constructor is disabled; also you need a destructor. However, 
 for this it is enough to use any type that has a destructor, so 
 that an implicit struct destructor is generated. For verbosity, 
 I'll write it out:


 struct MyDomainData {
     string username;

     this(string username)  safe
     in(!username.empty) // only non-empty usernames please!
     do { this.username = username; }

     // let's formalise the restriction.
     invariant { assert(!username.empty); }

     string toString() { return null; }

     ~this()  safe { }
 }

 Then just stick it in a Nullable. No explicit .init needed.


 That said, I may be missing something obvious but what 
 prevents you from overloading the init field?

     struct MyDomainData {
         string username;
          disable this(); // don't make a MyDomainData() by 
 accident!
         this(string username)
         in(!username.empty) // only non-empty usernames please!
         do { this.username = username; }
         // let's formalise the restriction.
         invariant { assert(!username.empty); }
         string toString() { ... }

         static  property MyDomainData init() {
             return MyDomainData("uninitialized");
         }

         ...
     }

     auto test = MyDomainData.init;  // There, no error

 Of course that value means nothing but .init isn't meant to 
 actually mean something anyway, it's just a valid value and 
 that's what that init is proposing, so it shouldn't cause any 
 more bugs than empty .init in a normal case.


 That would work, it's just a really horrible hack and I hate 
 it. We're constructing a fictitious domain value that passes 
 our invariants while having zero correspondence to the real 
 world, *just to pass our invariants*. It's an obvious sign of a 
 language issue.


If so, then the only language solution is to remove either 
invariants or .init, because as long as .init can be called but 
cannot be made to conform to your invariant, then your design is 
beyond the scope of the language and you're in a pickle.

But the fact is that it's not a language issue and there are 
several ways in user code to guarantee that .init satisfies the 
invariant. In fact that very statement suggests a solution that 
Timothy Cour suggested earlier:

      invariant {
       if(this is typeof(this).init) return;
       assert(!username.empty);
     }
Jul 29 2018
prev sibling next sibling parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43



Have you tried giving your invariants a valid  initial value? 
Change

   string username;


into

   string username = "noname";


-Bastiaan
Jul 10 2018
prev sibling next sibling parent reply Ali <fakeemail example.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43


Somehow, this is the type of problem, i thought point 1 in the 
vision document is aimed to solve
https://wiki.dlang.org/Vision/2018H1

"1. Lock down the language definition: D is a powerful language 
but its definition is not precise enough. A number of subtleties 
can only be assessed only by running the compiler, not by 
perusing the specification. This semester we are pushing for a 
better, more precise, and more complete specification of the D 
language."

ensuring that the language features are coherent together
Jul 10 2018
parent reply =?UTF-8?B?THXDrXM=?= Marques <luis luismarques.eu> writes:
On Wednesday, 11 July 2018 at 03:00:48 UTC, Ali wrote:

 Somehow, this is the type of problem, i thought point 1 in the 
 vision document is aimed to solve
 https://wiki.dlang.org/Vision/2018H1

 "1. Lock down the language definition: D is a powerful language 
 but its definition is not precise enough. A number of 
 subtleties can only be assessed only by running the compiler, 
 not by perusing the specification. This semester we are pushing 
 for a better, more precise, and more complete specification of 
 the D language."

 ensuring that the language features are coherent together


I think that point isn't supposed to be so ambitious. I think it 
only refers to the fact that the documentation isn't particularly 
formal/precise/complete regarding some language details. Making 
the language features more coherent and compatible among 
themselves is a whole other goal, independent of that one, IMO.
Jul 12 2018
parent Timothee Cour <thelastmammoth gmail.com> writes:
the following seems like a easy enough workaround:
just add

` if(this is typeof(this).init) return;` at 1st line of your invariant:

```d
import std.typecons;
import std.range;

struct MyDomainData {
     string username;

     this(string username)  safe
     in(!username.empty)
     do { this.username = username; }

     invariant {
      if(this is typeof(this).init) return;
      assert(!username.empty);
    }

     string toString() { return null; }

     ~this()  safe { }
}

void main(){
  Nullable!MyDomainData b;
}
```
On Thu, Jul 12, 2018 at 9:00 AM Luís Marques via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Wednesday, 11 July 2018 at 03:00:48 UTC, Ali wrote:

 Somehow, this is the type of problem, i thought point 1 in the
 vision document is aimed to solve
 https://wiki.dlang.org/Vision/2018H1

 "1. Lock down the language definition: D is a powerful language
 but its definition is not precise enough. A number of
 subtleties can only be assessed only by running the compiler,
 not by perusing the specification. This semester we are pushing
 for a better, more precise, and more complete specification of
 the D language."

 ensuring that the language features are coherent together


 I think that point isn't supposed to be so ambitious. I think it
 only refers to the fact that the documentation isn't particularly
 formal/precise/complete regarding some language details. Making
 the language features more coherent and compatible among
 themselves is a whole other goal, independent of that one, IMO.
Jul 12 2018
prev sibling next sibling parent reply Greatsam4sure <greatsam4sure gmail.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

 Related links:

 https://github.com/dlang/phobos/pull/6594 problem with T.init 
 and toString

 https://github.com/dlang/phobos/pull/6619 Nullable can't work 
 with types where T.init violates invariants

 https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
 disable invariant on struct ~this


Sincerely speaking D language does not merit all these criticism. 
The magnitude of criticism on D language does not really make 
sense to me. I am yet to see a language so user friendly as D 
with such power and strength.I trust one day the world will see 
and know that D is a language build for programmers for great 
productivity and not just another money making machine

This article justified your great love for D.  Thanks for your 
great love for D
Jul 11 2018
parent Jim Balter <Jim Balter.name> writes:
On Wednesday, 11 July 2018 at 16:54:18 UTC, Greatsam4sure wrote:

 On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 [...]


 Sincerely speaking D language does not merit all these 
 criticism. The magnitude of criticism on D language does not 
 really make sense to me. I am yet to see a language so user 
 friendly as D with such power and strength.I trust one day the 
 world will see and know that D is a language build for 
 programmers for great productivity and not just another money 
 making machine


Yeah, D is great so let's not even have a bug database, eh?
Jul 29 2018
prev sibling parent reply Greatsam4sure <greatsam4sure gmail.com> writes:
On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 I've written up a short blogpost about the T.init issue.

 It is not very enthusiastic.

 https://medium.com/ feepingcreature/d-structs-dont-work-for-domain-data-c09332349f43

 Related links:

 https://github.com/dlang/phobos/pull/6594 problem with T.init 
 and toString

 https://github.com/dlang/phobos/pull/6619 Nullable can't work 
 with types where T.init violates invariants

 https://github.com/dlang/dmd/pull/8462 A somewhat sketchy PR to 
 disable invariant on struct ~this


Every language is plague with one bug or the order.  For those 
will great love for the language they lend a helping hand to 
fixed the bug. I expect you to help also in whatsoever capacity 
you can.

I am just learning D but I am thoroughly satisfy with the 
language.  For me it is truly joy.
Jul 11 2018
parent Jim Balter <Jim Balter.name> writes:
On Wednesday, 11 July 2018 at 16:58:53 UTC, Greatsam4sure wrote:

 On Tuesday, 10 July 2018 at 13:41:56 UTC, FeepingCreature wrote:

 [...]


 Every language is plague with one bug or the order.  For those 
 will great love for the language they lend a helping hand to 
 fixed the bug. I expect you to help also in whatsoever capacity 
 you can.


What do you think pull requests are?


 I am just learning D but I am thoroughly satisfy with the 
 language.  For me it is truly joy.


Goody for you, but how does that help?
Jul 29 2018