www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Optional type - how to correctly reset a wrapped immutable T

reply aliak <something something.com> writes:
Hi, I have this optional type I'm working on and I've run in to a 
little snag when it comes to wrapping an immutable. Basically 
what I want is for an Optional!(immutable T) to still be settable 
to "some" value or "no" value because the Optional wrapper itself 
is mutable.

Optional!(immutable int) a = some(3);
a = none;

I used to do this via a dynamic array and opAssign would recreate 
the array

struct Optional(T) {
   T[] bag;
   opAssign(T t) {
     bag = [t]
   }
}

But then there were problems with inout, namely:

Error: variable `Optional!(inout(A)).Optional.bag` only 
parameters or stack based variables can be inout

So I changed it to a stack variable (prefer this anyway, it's not 
only because of the inout error) but now I'm unsure if I'm 
violating the type system. Basically I'm now storing T as 
Unqual!T, but what I'm looking for is a Rebindable implementation 
that's for value types as well. Is this possible?

Now I do this:

struct Optional(T) {
   Unqual!T value;
   opAssign(T t) {
     value = cast(Unqual!T)(t);
   }
}

I put up a PR if anyone wants to see the code. Any pointers, tips 
would be highly appreciated:

https://github.com/aliak00/optional/pull/13/files#diff-cb543fea6a0b5eeb07b6aac9f068e262

Cheers
- Ali
Mar 25 2018
next sibling parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Sunday, 25 March 2018 at 21:26:57 UTC, aliak wrote:

 struct Optional(T) {
   Unqual!T value;
   opAssign(T t) {
     value = cast(Unqual!T)(t);
   }
 }


Consider this case:

Optional!(immutable int) a = some(3);
immutable int* p = &a.value;
a = some(5);

Clearly the above code shouldn't compile - you can't overwrite 
the value, as it would break immutability. If you want to support 
both immutability and reassignment, you will need to use 
redirection - either an array as you did, or a pointer.

As for the problems you've had with inout, I wrote this template 
a few years back:

template SubstituteInout(FromType, ToType) {
     static if (is(ToType == inout(SubType), SubType)) {
         alias SubstituteInout = CopyTypeQualifiers!(FromType, 
SubType);
     } else static if (is(ToType == SubType*, SubType)) {
         alias SubstituteInout = SubstituteInout!(FromType, 
SubType)*;
     } else static if (is(ToType == SubType[], SubType)) {
         alias SubstituteInout = SubstituteInout!(FromType, 
SubType)[];
     } else static if (is(ToType == SubType[n], SubType, size_t 
n)) {
         alias SubstituteInout = SubstituteInout!(FromType, 
SubType)[n];
     } else static if (is(ToType == SubType[KeyType], SubType, 
KeyType)) {
         alias SubstituteInout = SubstituteInout!(FromType, 
SubType)[SubstituteInout!(FromType, KeyType)];
     } else {
         alias SubstituteInout = ToType;
     }
}

unittest {
     static assert(is(SubstituteInout!(const(string), int) == 
int));
     static assert(is(SubstituteInout!(const(string), 
inout(int)[]) == const(int)[]));
     static assert(is(SubstituteInout!(const(string), inout(int)) 
== const(int)));
     static assert(is(SubstituteInout!(const(string), 
inout(int)*[][3][int]) == const(int)*[][3][int]));
     static assert(is(SubstituteInout!(const(string), 
inout(int)[inout(string)]) == const(int)[const(string)]));
}

I really should get around to making a PR for it...

--
   Simen
Mar 25 2018
next sibling parent Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Sunday, 25 March 2018 at 23:00:11 UTC, Simen Kjærås wrote:

 On Sunday, 25 March 2018 at 21:26:57 UTC, aliak wrote:

 struct Optional(T) {
   Unqual!T value;
   opAssign(T t) {
     value = cast(Unqual!T)(t);
   }
 }


 Consider this case:

 Optional!(immutable int) a = some(3);
 immutable int* p = &a.value;
 a = some(5);


Of course, if Optional offers no way to get a reference to the 
wrapped value (say, if a.value is an  property function that 
doesn't return ref T), then using Unqual internally is safe*.

Someone with knowledge of the internal layout of Optional might 
use tricks like *cast(immutable(int)*)&a, but in that case 
they're breaking the type system anyway, and one simply cannot 
negotiate with terrorists.

*actually, this may not be 100% true, in cases where T.opAssign 
does weird things. Consider:

struct Foo {
     int* p;

     void opAssign(Foo rhs) {
         p = rhs.p;
         (*p)++;
     }
}

unittest {
     immutable a = Foo(new int(3));
     assert(*a.p == 3); // Passes
     Optional!(immutable(Foo)) b;
     b = a;
     assert(*a.p == 3); // Fails
}

There actually is a workaround for this, using destroy() and 
move() instead of assignment. I'm unsure if there are other 
corner cases to consider.


--
   Simen
Mar 26 2018
prev sibling parent aliak <something something.com> writes:
On Sunday, 25 March 2018 at 23:00:11 UTC, Simen Kjærås wrote:

 On Sunday, 25 March 2018 at 21:26:57 UTC, aliak wrote:

 struct Optional(T) {
   Unqual!T value;
   opAssign(T t) {
     value = cast(Unqual!T)(t);
   }
 }


 Consider this case:

 Optional!(immutable int) a = some(3);
 immutable int* p = &a.value;
 a = some(5);


Ahh, bugger.


 As for the problems you've had with inout, I wrote this 
 template a few years back:


This template seems to substitute type qualifiers from the 
FromType to any inout qualifiers in the ToType yes? I'm not sure 
how I'd use this actually. I would like to support the inout 
usecase when a user does

Optional!(inout int) a = 3; // for e.g.

I guess for that template I'd need to have some extra information 
that tells me what to substitute inout with yeah?


 Of course, if Optional offers no way to get a reference to the 
 wrapped value (say, if a.value is an  property function that 
 doesn't return ref T), then using Unqual internally is safe*.


This thought crossed my mind as well, value should be private 
anyway, and there's an unwrap function that provides a pointer to 
the value if it exists, so I wouldn't know how to get around that 
(return a dynamic array with one or no element that contains a 
copy and only if it's immutable, else return a pointer? erm ... 
O_o ) and then I was thinking compiler optimizations may throw in 
wrench in that if I did find a way around.


 unittest {
     immutable a = Foo(new int(3));
     assert(*a.p == 3); // Passes
     Optional!(immutable(Foo)) b;
     b = a;
     assert(*a.p == 3); // Fails
 }

 There actually is a workaround for this, using destroy() and 
 move() instead of assignment. I'm unsure if there are other 
 corner cases to consider.


I'm curious what the move/destroy workaround would be?

Cheers,
- Ali
Mar 26 2018
prev sibling next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 25 March 2018 at 21:26:57 UTC, aliak wrote:

 Hi, I have this optional type I'm working on and I've run in to 
 a little snag when it comes to wrapping an immutable. Basically 
 what I want is for an Optional!(immutable T) to still be 
 settable to "some" value or "no" value because the Optional 
 wrapper itself is mutable.

 Optional!(immutable int) a = some(3);
 a = none;

 I used to do this via a dynamic array and opAssign would 
 recreate the array

 struct Optional(T) {
   T[] bag;
   opAssign(T t) {
     bag = [t]
   }
 }

 But then there were problems with inout, namely:

 Error: variable `Optional!(inout(A)).Optional.bag` only 
 parameters or stack based variables can be inout

 So I changed it to a stack variable (prefer this anyway, it's 
 not only because of the inout error) but now I'm unsure if I'm 
 violating the type system. Basically I'm now storing T as 
 Unqual!T, but what I'm looking for is a Rebindable 
 implementation that's for value types as well. Is this possible?

 Now I do this:

 struct Optional(T) {
   Unqual!T value;
   opAssign(T t) {
     value = cast(Unqual!T)(t);
   }
 }

 I put up a PR if anyone wants to see the code. Any pointers, 
 tips would be highly appreciated:

 https://github.com/aliak00/optional/pull/13/files#diff-cb543fea6a0b5eeb07b6aac9f068e262

 Cheers
 - Ali


Have a look at Rebindable: 
https://dlang.org/phobos/std_typecons.html#rebindable
Mar 26 2018
parent reply Simen =?UTF-8?B?S2rDpnLDpXM=?= <simen.kjaras gmail.com> writes:
On Monday, 26 March 2018 at 09:46:57 UTC, Nicholas Wilson wrote:

 Have a look at Rebindable: 
 https://dlang.org/phobos/std_typecons.html#rebindable


Allow me to quote from aliak's post:


 what I'm looking for is a Rebindable implementation that's for 
 value types


As can be surmised from the above line, aliak has looked at 
Rebindable, and it doesn't work with value types.

--
   Simen
Mar 26 2018
next sibling parent reply Seb <seb wilzba.ch> writes:
On Monday, 26 March 2018 at 10:13:08 UTC, Simen Kjærås wrote:

 On Monday, 26 March 2018 at 09:46:57 UTC, Nicholas Wilson wrote:

 Have a look at Rebindable: 
 https://dlang.org/phobos/std_typecons.html#rebindable


 Allow me to quote from aliak's post:


 what I'm looking for is a Rebindable implementation that's for 
 value types


 As can be surmised from the above line, aliak has looked at 
 Rebindable, and it doesn't work with value types.

 --
   Simen


Though Rebindable could be improved to work with value types:

https://github.com/dlang/phobos/pull/6136
Mar 26 2018
parent aliak <something something.com> writes:
On Monday, 26 March 2018 at 11:19:31 UTC, Seb wrote:

 On Monday, 26 March 2018 at 10:13:08 UTC, Simen Kjærås wrote:

 On Monday, 26 March 2018 at 09:46:57 UTC, Nicholas Wilson 
 wrote:

 Have a look at Rebindable: 
 https://dlang.org/phobos/std_typecons.html#rebindable


 Allow me to quote from aliak's post:


 what I'm looking for is a Rebindable implementation that's 
 for value types


 As can be surmised from the above line, aliak has looked at 
 Rebindable, and it doesn't work with value types.

 --
   Simen


 Though Rebindable could be improved to work with value types:

 https://github.com/dlang/phobos/pull/6136


And of course Seb knows about a related PR :p Will take a look to 
see what kind of magic is in there. Thanks !
Mar 26 2018
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Monday, March 26, 2018 10:13:08 Simen Kjærås via Digitalmars-d-learn 
wrote:

 On Monday, 26 March 2018 at 09:46:57 UTC, Nicholas Wilson wrote:

 Have a look at Rebindable:
 https://dlang.org/phobos/std_typecons.html#rebindable


 Allow me to quote from aliak's post:

 what I'm looking for is a Rebindable implementation that's for
 value types


 As can be surmised from the above line, aliak has looked at
 Rebindable, and it doesn't work with value types.


It doesn't make sense for it to work with value types. Its entire purpose is
to be able to do the equivalent of const(T)* and immutable(T)* with classes.

If I correctly follow what the OP is trying to do, it's in violation of the
type system. You can't cast away const or immutable like that and mutate the
object. Honestly, what Rebindable does is pretty questionable as far as the
type system goes, but it does what it does by forcing pointer semantics on a
class reference, so the point is arguable. However, I think that the reality
of the matter is that Rebindable technically breaks the type system. It just
happens to do so in a way that always works, and there is no other way to do
what it does. But trying to do something like

struct Foo(T)
{
    const T _member;
}

where you cast away const and mutate the member is definitely in violation
of the type system and risks serious bugs depending on the optimizations
that the compiler chooses to employ - even more so if immutable is used.

- Jonathan M Davis
Mar 26 2018
parent SimonN <eiderdaus gmail.com> writes:
On Monday, 26 March 2018 at 14:17:03 UTC, Jonathan M Davis wrote:

 Rebindable does is pretty questionable as far as the type 
 system goes, but it does what it does by forcing pointer 
 semantics on a class reference, so the point is arguable.


Yeah, I've always assumed that Rebindable cannot be implemented 
without internally breaking the type system, then exposing a safe 
interface.

But this sparked my interest, I've dug out the Rebindable code:

     private mixin template RebindableCommon(T, U, alias This)
         if (is(T == class) || is(T == interface) || 
isAssociativeArray!T)
     {
         private union
         {
             T original; // e.g., immutable(A) for classs A
             U stripped; // the unqualified type, e.g., A
         }
         // ...
     }

Does Rebindable-using code, oblivious of the hacks inside 
Rebindable, remain 100 % safe even with aggressive compiler 
optimizations? For class A, inside Rebindable!(immutable A), 
there is a union of (immutable A) and A. I suspect that the D 
compiler is allowed to treat this reference to (immutable A) as 
immutable itself. Have there never been bugs here when, later, 
stripped = another.stripped;?

-- Simon
Mar 27 2018
prev sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Sunday, 25 March 2018 at 21:26:57 UTC, aliak wrote:

 Hi, I have this optional type I'm working on and I've run in to 
 a little snag when it comes to wrapping an immutable. Basically 
 what I want is for an Optional!(immutable T) to still be 
 settable to "some" value or "no" value because the Optional 
 wrapper itself is mutable.


[snip]

You might look at the source for std.typecons.Nullable. They use 
an inout constructor.
Mar 26 2018
parent reply aliak <something something.com> writes:
On Monday, 26 March 2018 at 21:17:10 UTC, jmh530 wrote:

 On Sunday, 25 March 2018 at 21:26:57 UTC, aliak wrote:

 Hi, I have this optional type I'm working on and I've run in 
 to a little snag when it comes to wrapping an immutable. 
 Basically what I want is for an Optional!(immutable T) to 
 still be settable to "some" value or "no" value because the 
 Optional wrapper itself is mutable.


 [snip]

 You might look at the source for std.typecons.Nullable. They 
 use an inout constructor.


Unfortunately

Nullable!(inout int) b = 3;

produces that compiler error. This seems more a case on how T is 
stored than about construction in particular. Was interesting to 
see a use case for hasElaborateAssign though!

By the by,  how come inout has to be stack based and 
const/immutable/mutable doesn't? Isn't inout just one of those 
depending on context?

Cheers,
- Ali
Mar 26 2018
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 27 March 2018 at 06:26:57 UTC, aliak wrote:

 [snip]

 By the by,  how come inout has to be stack based and 
 const/immutable/mutable doesn't? Isn't inout just one of those 
 depending on context?


Example?
Mar 27 2018
parent reply aliak <something something.com> writes:
On Tuesday, 27 March 2018 at 11:57:28 UTC, jmh530 wrote:

 On Tuesday, 27 March 2018 at 06:26:57 UTC, aliak wrote:

 [snip]

 By the by,  how come inout has to be stack based and 
 const/immutable/mutable doesn't? Isn't inout just one of those 
 depending on context?


 Example?


Hmm, now that I'm explicitly trying to produce it, I feel I maybe 
using inout incorrectly?

struct Optional(T) {
     T[] bag;
     this(T t) {
         bag = [t];
     }
}

struct S {
     Optional!(inout(int)) f() inout
     {
         return Optional!(inout(int))(3);
     }
}

void main()
{
     auto a = S().f;
}

Above gives: Error: variable 
`onlineapp.Optional!(inout(int)).Optional.bag` only parameters or 
stack based variables can be inout

Change inout to const e.g. and it's all good.
Mar 27 2018
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 27 March 2018 at 13:02:50 UTC, aliak wrote:

 Hmm, now that I'm explicitly trying to produce it, I feel I 
 maybe using inout incorrectly?

 struct Optional(T) {
     T[] bag;
     this(T t) {
         bag = [t];
     }
 }

 struct S {
     Optional!(inout(int)) f() inout
     {
         return Optional!(inout(int))(3);
     }
 }

 void main()
 {
     auto a = S().f;
 }

 Above gives: Error: variable 
 `onlineapp.Optional!(inout(int)).Optional.bag` only parameters 
 or stack based variables can be inout

 Change inout to const e.g. and it's all good.


You may refer to the section on struct constructors. inout on a 
constructor serves as both a const and an immutable constructor. 
This would allow you to create mutable/const/immutable objects of 
whatever struct. The payload could have some different setting.

https://dlang.org/spec/struct.html#struct-constructor

How about:

struct Optional(T) {
     T[] bag;
     this(T t) {
         bag = [t];
     }
}

Optional!T optional(T)(T x)
{
     return Optional!T(x);
}

void main()
{
     int x = 3;
     const(int) xx = 3;
     immutable(int) xxx = 3;

     auto y = optional(x);
     auto yy = optional(xx);
     auto yyy = optional(xxx);
}
Mar 27 2018
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 27 March 2018 at 13:51:20 UTC, jmh530 wrote:

 

 How about:
 [snip]


I can kind of like this more, but after re-reading your original 
post I'm not sure it really resolves your issue:

struct Optional(T) {
     import std.traits : isMutable;
     T[] bag;

     this(T t) inout {
         bag = [t];
     }

     void opAssign(T rhs)
     {
         static if (isMutable!T)
             bag[0] = rhs;
         else
             bag = [rhs];
     }
}

Optional!T optional(T)(T x)
{
     return Optional!T(x);
}

void main()
{
     int x = 3;
     const(int) xx = 3;
     immutable(int) xxx = 3;
     immutable(int) xxxx = 4;

     auto y = optional(x);
     auto yy = optional(xx);
     auto yyy = optional(xxx);
     yyy = xxxx;
}
Mar 27 2018
parent reply SimonN <eiderdaus gmail.com> writes:
On Tuesday, 27 March 2018 at 15:28:40 UTC, jmh530 wrote:

         static if (isMutable!T)
             bag[0] = rhs;
         else
             bag = [rhs];


I like this idea. I'd even take it a step futher:

When T is a pointer or class reference, then we can put the 
reference on the stack (instead of into the array) and handle 
assignments like Rebindable handles assignments -- provided that 
Rebindable really is 100 % safe to the outside, see my concerns 
from 2 posts above. In this case (static if), we won't even 
declare the array T[] bag, and instead implement as T value, bool 
isPresent.

When T is a mutable value type, it goes on the stack, too. Again 
no array.

When T is a const/immutable/inout value type, we declare the 
array as before and rebind on assignment with bag = [rhs], as you 
proposed here.

-- Simon
Mar 27 2018
parent aliak <something something.com> writes:
On Tuesday, 27 March 2018 at 15:37:11 UTC, SimonN wrote:

 On Tuesday, 27 March 2018 at 15:28:40 UTC, jmh530 wrote:

         static if (isMutable!T)
             bag[0] = rhs;
         else
             bag = [rhs];


 I like this idea. I'd even take it a step futher:

 When T is a pointer or class reference, then we can put the 
 reference on the stack (instead of into the array) and handle 
 assignments like Rebindable handles assignments -- provided 
 that Rebindable really is 100 % safe to the outside, see my 
 concerns from 2 posts above. In this case (static if), we won't 
 even declare the array T[] bag, and instead implement as T 
 value, bool isPresent.

 When T is a mutable value type, it goes on the stack, too. 
 Again no array.

 When T is a const/immutable/inout value type, we declare the 
 array as before and rebind on assignment with bag = [rhs], as 
 you proposed here.

 -- Simon


I think this all sounds good. I found the "magic" as well in 
Seb's Rebindable PR link for value types so might be able to have 
stack storage for const value types as well [1]. Not sure how 
safe it is.

Inout poses a problem in all cases though and I'm unsure how to 
solve it without stripping the qualifiers completely from the 
payload. Basically:

struct S(T) {
     T t;
}

auto make(T)(T t) {
     return S!(T)(t);
}

class C {
     int i;
     auto f() inout {
         make(i);
     }
}

Will not compile.

I can make T an Unqual!T, but then we are open to type breakage 
in cases where users should have access to member t.


[1] 
https://github.com/dlang/phobos/pull/6136/files#diff-4e008aedb3026d4a84f58323e53bf017R2223
Mar 29 2018