www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - eating inout dogfood

reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
Recently, inout has become significantly useful in dmd.  The latest  
incarnation of the compiler in git has fixed all the previous issues with  
inout (special thanks to Kenji Hara for creating the related pull  
requests!)

Since I was a main designer of the inout system, I figured I should try  
and eat my own dogfood in terms of figuring out how well inout works for  
my main D side project -- dcollections.

So I went about modifying all of dcollections to be inout and const  
compliant.

The result was quite pleasant.  By applying inout and const to all the  
appropriate members, dcollections quickly became const aware, and I didn't  
have to add a single overload to do it!  There are a few issues with inout  
that I realize really do still need to be addressed.

For instance, we really do need a way to apply tail-const/tail-inout to  
custom structures.  Currently, any inout functions that return ranges or  
cursors return a fully-inout range or cursor.  This means, getting a range  
on a const container returns a range which cannot be iterated (you can  
still get the front and rear elements, call save, and other const-aware  
members).

So how do we fix this? One might think "why not just return an iterator  
with a tail-inout pointer?"  However, that doesn't work.  You cannot have  
a field that is inout, because inout is a temporary condition, only  
applicable to stack data.  For more details on this, see  
http://d.puremagic.com/issues/show_bug.cgi?id=6770

Not only that, but the issue is really the implicit casting.  Assuming the  
field-inout restriction didn't exist, we might define a simple cursor like  
this (for purposes of demonstration, I'm going to use a range over  
contiguous memory, ignoring the fact that slices already provide this):

struct cursor(T)
{
    T* node;
    void popFront() {node++;}
    ... // usual suspects front, empty, etc
}

This seems like cursor!(inout(V)) might work (V is the element type of the  
container) as the return type for inout functions.  However, one of the  
major requirements of inout is that it correctly cast back to the  
constancy of the container.

So this means cursor!(inout(V)) must cast to cursor!(const(V)).  However,  
as we all know, templates instantiated with different types where the  
types implicitly cast, do not allow implicit casting of the template.   
This is for good reason:

1. the representation might be different.  For example, int casts  
implicitly to long, but some struct S!int might not implicitly cast to  
S!long because S!long likely has a larger footprint.
2. the template might actually be completely different based on the  
parameters.  For example, you can use a static if to change the layout or  
functions depending on if the type is const or not.

I think this problem needs solving.  It would greatly improve the D story  
as a language where one can use superpowered generic programming to solve  
many problems that other languages need language modifications to solve.   
I have some ideas, but I wanted to let other people bring any ideas they  
might have first, as Walter has not been receptive to my attempts at  
solving this problem in the past.


============================
Another interesting "problem" I had while doing inout is I had many  
functions like this:

cursor elemAt(V v)
{
     cursor it;
     it.position = _hash.find(v);
     it._empty = it.position is _hash.end;
     return it;
}

But what I discovered (quite rapidly) is just applying inout to this  
function doesn't work.  When one is making a function that returns an  
inout aggregate, constructing that aggregate is almost required to be in  
the return expression.  So the above becomes:

inout(cursor) elemAt(const(V) v) inout
{
     auto pos = _hash.find(v);
     return inout(cursor)(pos, pos is _hash.end);
}

Although I think the latter is technically cleaner, it does bring up an  
interesting issue with regards to inout.  In some cases, applying inout is  
a no-brainer.  You just slap inout on the parameters and return values,  
and things just *work*.  However, in many cases, a redesign of the  
function is necessary.  This is something to keep in mind when promoting  
inout as a wonderful tool to bring const-awareness to existing code.


=============================
In any case, the next version of dcollections will support const to a  
certain degree (with the exception of iterable const ranges) when the next  
version of the compiler comes out.

Here is the commit which adds inout/const to dcollections:  
http://www.dsource.org/projects/dcollections/changeset/114

Be sure to use the latest version of dmd from git to try it out (if you're  
so inclined).

-Steve
Oct 13 2011
next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Thu, 13 Oct 2011 09:27:55 -0400, Steven Schveighoffer wrote:


 Recently, inout has become significantly useful in dmd.  The latest
 incarnation of the compiler in git has fixed all the previous issues
 with inout (special thanks to Kenji Hara for creating the related pull
 requests!)
 
 Since I was a main designer of the inout system, I figured I should try
 and eat my own dogfood in terms of figuring out how well inout works for
 my main D side project -- dcollections.


The new std.path will also benefit greatly from using inout.  It contains 
a lot of functions which simply return a slice of an input array without 
modifying its contents.  Currently, this const-ness is not enforced, but 
inout will change that.

-Lars
Oct 13 2011
prev sibling parent reply Kagamin <spam here.lot> writes:
Steven Schveighoffer Wrote:


 struct cursor(T)
 {
     T* node;
     void popFront() {node++;}
     ... // usual suspects front, empty, etc
 }
 
 This seems like cursor!(inout(V)) might work (V is the element type of the  
 container) as the return type for inout functions.  However, one of the  
 major requirements of inout is that it correctly cast back to the  
 constancy of the container.


try to return inout(cursor!(Unqual!(V)))
Oct 14 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 14 Oct 2011 04:43:15 -0400, Kagamin <spam here.lot> wrote:


 Steven Schveighoffer Wrote:


 struct cursor(T)
 {
     T* node;
     void popFront() {node++;}
     ... // usual suspects front, empty, etc
 }

 This seems like cursor!(inout(V)) might work (V is the element type of  
 the
 container) as the return type for inout functions.  However, one of the
 major requirements of inout is that it correctly cast back to the
 constancy of the container.


 try to return inout(cursor!(Unqual!(V)))


This won't solve the problem.  I need inout to be on V, not on the  
cursor.  Essentially, I need cursor to be tail-inout.

BTW, this assumes V is not const.  I'm not exactly sure how well the  
containers work when V is not mutable.  I'd expect they would fail  
spectacularly :)

-Steve
Oct 14 2011
next sibling parent reply Kagamin <spam here.lot> writes:
Steven Schveighoffer Wrote:


 This won't solve the problem.  I need inout to be on V, not on the  
 cursor.  Essentially, I need cursor to be tail-inout.


Since const is transitive, constness of cursor eventually propagates to V.
Oct 14 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 14 Oct 2011 09:02:04 -0400, Kagamin <spam here.lot> wrote:


 Steven Schveighoffer Wrote:


 This won't solve the problem.  I need inout to be on V, not on the
 cursor.  Essentially, I need cursor to be tail-inout.


 Since const is transitive, constness of cursor eventually propagates to  
 V.


I think you misunderstand the problem.  I need constness to be applied to  
the V, but *NOT* to the cursor.  If constness is applied to the cursor,  
it's not iterable (i.e. popFront does not work).

-Steve
Oct 14 2011
parent Kagamin <spam here.lot> writes:
Steven Schveighoffer Wrote:


 On Fri, 14 Oct 2011 09:02:04 -0400, Kagamin <spam here.lot> wrote:
 

 Steven Schveighoffer Wrote:


 This won't solve the problem.  I need inout to be on V, not on the
 cursor.  Essentially, I need cursor to be tail-inout.


 Since const is transitive, constness of cursor eventually propagates to  
 V.


 
 I think you misunderstand the problem.  I need constness to be applied to  
 the V, but *NOT* to the cursor.  If constness is applied to the cursor,  
 it's not iterable (i.e. popFront does not work).


the cursor can provide a method, say, dup, which would return it's mutable
copy, properly templated on const.

foreach(v; dic.values) // doesn't compile?
foreach(v; dic.values.dup()) // should work
Oct 16 2011
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 10/14/2011 4:29 AM, Steven Schveighoffer wrote:

 On Fri, 14 Oct 2011 04:43:15 -0400, Kagamin <spam here.lot> wrote:


 Steven Schveighoffer Wrote:


 struct cursor(T)
 {
 T* node;
 void popFront() {node++;}
 ... // usual suspects front, empty, etc
 }

 This seems like cursor!(inout(V)) might work (V is the element type of the
 container) as the return type for inout functions. However, one of the
 major requirements of inout is that it correctly cast back to the
 constancy of the container.


 try to return inout(cursor!(Unqual!(V)))


 This won't solve the problem. I need inout to be on V, not on the cursor.


return cursor!(Unqual!(inout(V));

?
Oct 14 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 14 Oct 2011 14:17:09 -0400, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 10/14/2011 4:29 AM, Steven Schveighoffer wrote:

 On Fri, 14 Oct 2011 04:43:15 -0400, Kagamin <spam here.lot> wrote:


 Steven Schveighoffer Wrote:


 struct cursor(T)
 {
 T* node;
 void popFront() {node++;}
 ... // usual suspects front, empty, etc
 }

 This seems like cursor!(inout(V)) might work (V is the element type  
 of the
 container) as the return type for inout functions. However, one of the
 major requirements of inout is that it correctly cast back to the
 constancy of the container.


 try to return inout(cursor!(Unqual!(V)))


 This won't solve the problem. I need inout to be on V, not on the  
 cursor.


 return cursor!(Unqual!(inout(V));


I don't think the compiler will auto-convert someTemplate!(inout(V)) to  
e.g. someTemplate!(const(V)).  Does that work?  I know that for instance,  
I can't do this:

struct S(T)
{
    T * x;
}

S!(int) ptr;
S!(const(int)) cptr = ptr;

So I wouldn't expect the compiler to do the above translation either...

-Steve
Oct 14 2011
next sibling parent kenji hara <k.hara.pg gmail.com> writes:
2011/10/15 Steven Schveighoffer <schveiguy yahoo.com>:

 I don't think the compiler will auto-convert someTemplate!(inout(V)) to e=


.g.

 someTemplate!(const(V)). =A0Does that work? =A0I know that for instance, =


I can't

 do this:

 struct S(T)
 {
 =A0 T * x;
 }

 S!(int) ptr;
 S!(const(int)) cptr =3D ptr;

 So I wouldn't expect the compiler to do the above translation either...


IMO, for the purpose we need covariant/contravariant template type
parameter like Scala.

Kenji Hara
Oct 14 2011
prev sibling parent reply Michel Fortin <michel.fortin michelf.com> writes:
On 2011-10-14 20:05:09 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 I don't think the compiler will auto-convert someTemplate!(inout(V)) to 
  e.g. someTemplate!(const(V)).  Does that work?  I know that for 
 instance,  I can't do this:
 
 struct S(T)
 {
     T * x;
 }
 
 S!(int) ptr;
 S!(const(int)) cptr = ptr;
 
 So I wouldn't expect the compiler to do the above translation either...


Perhaps you can add this to the struct:

	void this(U)(in S!U other) if (__traits(compiles, this.x = other.x))
	{
		this.x = other.x;
	}

If that works, maybe a similar approach could be used to solve the 
problem with inout: if you can construct the requested type from the 
provided one it get converted automatically at the call site.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Oct 14 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Fri, 14 Oct 2011 20:21:59 -0400, Michel Fortin  
<michel.fortin michelf.com> wrote:


 On 2011-10-14 20:05:09 +0000, "Steven Schveighoffer"  
 <schveiguy yahoo.com> said:


 I don't think the compiler will auto-convert someTemplate!(inout(V)) to  
  e.g. someTemplate!(const(V)).  Does that work?  I know that for  
 instance,  I can't do this:
  struct S(T)
 {
     T * x;
 }
  S!(int) ptr;
 S!(const(int)) cptr = ptr;
  So I wouldn't expect the compiler to do the above translation either...


 Perhaps you can add this to the struct:

 	void this(U)(in S!U other) if (__traits(compiles, this.x = other.x))
 	{
 		this.x = other.x;
 	}

 If that works, maybe a similar approach could be used to solve the  
 problem with inout: if you can construct the requested type from the  
 provided one it get converted automatically at the call site.


This looks promising.  However, I seem to recall D specifically  
disallowing implicit conversion using a constructor...

Is this going to fly with Walter?

-Steve
Oct 17 2011
parent Michel Fortin <michel.fortin michelf.com> writes:
On 2011-10-17 12:26:17 +0000, "Steven Schveighoffer" 
<schveiguy yahoo.com> said:


 On Fri, 14 Oct 2011 20:21:59 -0400, Michel Fortin  
 <michel.fortin michelf.com> wrote:
 

 Perhaps you can add this to the struct:
 
 	void this(U)(in S!U other) if (__traits(compiles, this.x = other.x))
 	{
 		this.x = other.x;
 	}
 
 If that works, maybe a similar approach could be used to solve the  
 problem with inout: if you can construct the requested type from the  
 provided one it get converted automatically at the call site.
 


 
 This looks promising.  However, I seem to recall D specifically  
 disallowing implicit conversion using a constructor...
 
 Is this going to fly with Walter?


For some reason I couldn't make it work with the constraint, but this 
works fine with the current compiler:

	struct S(T)
	{
		this(U)(in S!U other)
		{
			this.x = other.x;
		}

		T * x;
	}

	void main()
	{
		S!(int) ptr;
		S!(const(int)) cptr = ptr;
	}

Whether it's intended or not I don't know. Ask Walter.

-- 
Michel Fortin
michel.fortin michelf.com
http://michelf.com/
Oct 17 2011