www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Reference counting

reply "Regan Heath" <regan netwin.co.nz> writes:
I got little or no comment on this idea/code I posted to another (rather  
large) thread here, I figured I'd post it again and see what people  
thought.

After reading this thread (the one I mention above) and also this one  
Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

It appears you need a type that:

1 - is stack based.
2 - has deterministic destruction.
3 - has opAssign.

It seems (from recent posts) that Walter plans to put auto classes on the  
stack, so they'd then fulfil requirements 1 and 2.

But what about 3? To me, opAssign isn't generally a good idea for classes  
(assigning a value to a reference seems illogical) but would be a good for  
structs (they're value types, assignment assigns a value).

Can you do reference counting without opAssign?

The idea for D (or so I've read) is that opAssign isn't required, instead  
you use a copy constructor. Now, I've little experience with ref counting,  
or with writing an implementation of one, but, does this work?

If not, why not? What's missing, what else is required? Overload of '.'  
perhaps? The ability to pass a RefPtr as the object being counted?

import std.c.windows.windows;
import std.random;
import std.thread;
import std.stdio;

class RefPtr
{
	RefPtr parent;
	Object resource = null;
	int refs = 0;

	this(Object res)
	{
		resource = res;
		writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for  
resource=(",resource,")");
		increment();		
	}

	this(RefPtr rhs)
	{
		parent = rhs;
		parent.increment();
		writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for  
resource=(",parent.resource,")");
	}
	
	~this()
	{
		int r;
		
		if ((r = decrement()) == 0)
		{
			writefln("ThreadID=(",Thread.getThis().id,") release last ref  
Ref=(",r,")");
			if (parent) parent = null;
			else if (resource)
			{
				writefln("ThreadID=(",Thread.getThis().id,") delete  
resource=(",resource,")");
				delete resource;
				resource = null;
			}
		}
		writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
	}
protected:
	int increment()
	{
		int ret;
		
		if (parent) ret = parent.increment();
		else {
			synchronized(this)
			{
				ret = ++refs;
				writefln("ThreadID=(",Thread.getThis().id,") increment to  
Ref=(",refs,")");
			}
		}
		
		return ret;
	}
	int decrement()
	{
		int ret;
		
		if (parent) ret = parent.decrement();
		else {
			synchronized(this)
			{
				ret = --refs;
				writefln("ThreadID=(",Thread.getThis().id,") decrement to  
Ref=(",refs,")");
			}
		}
		
		return ret;
	}
}

class Resource
{
	char[] name = "11 was a racehorse";
	char[] toString() { return name; }
}

RefPtr pbob;

static this()
{
	pbob = new RefPtr(new Resource());
}

static ~this()
{
	delete pbob;
}

void main()
{
	Thread[] threads;
	int i;
	
	writefln("ThreadID=(",Thread.getThis().id,") is the main thread");
	
	for(i = 0; i < 10; i++)
	{
		threads ~= new Thread(&thread_function,null);
		threads[$-1].start();
	}
	
	while(true)
	{
		i = 0;
		foreach(Thread t; threads)
		{
			if (t.getState() == Thread.TS.TERMINATED) i++;
		}
		if (i == 10) break;
		Sleep(100);
	}
	
	writefln("Main exiting");
}

int thread_function(void* isnull)
{
	auto RefPtr p = new RefPtr(pbob);
	Sleep(1000+rand()%1000);	
	return 0;
}

Regan
Jun 26 2005
next sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
Reagan, let's take a look on more simple case - implementation
of 'auto' pointer (this is simplifed version)

  struct Auto(X)
  {

    // constructor.
    this( X _ptr = 0 ) { ptr = _ptr; owner = true; }

    // the destructor deletes ptr only if we are the owner.
    ~this() { if( owner && ptr ) delete ptr; }

    // the assignment operator.
    void opAssign( X r )
    {
        if( owner && ptr ) delete ptr;
        ptr = r;
        owner = true;
    }

    // the assignment operator. transfer
    // ownership. (not full implementation, sic)
    void opAssign( Auto r )
    {
        if( owner && ptr ) delete ptr;
        ptr = r.release();
        owner = true;
    }

    X value() { return  ptr; }

    // release returns the ptr value and releases ownership
    // if we were previously the owner.

    X release()
    {
      owner = false;
      return value();
    }
    private X ptr;
    private bool owner;
  }

-------------------------

{
    Auto!(Object) autop = new Object(); // first instance
    autop = new Object(); // first instance will be deleted;
                                           // and autop contains second 
instance.
} // second instance will be deleted here
  // even in case of exception.

This is basic auto_ptr used in C++
and it simplfies life a lot.
opAssign and dtors are crucial here.
And such Auto(X) must be a struct.
Classes do not work here.

Andrew.


"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss0boj0r23k2f5 nrage.netwin.co.nz...

I got little or no comment on this idea/code I posted to another (rather 
large) thread here, I figured I'd post it again and see what people 
thought.

 After reading this thread (the one I mention above) and also this one 
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on the 
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for classes 
 (assigning a value to a reference seems illogical) but would be a good for 
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

 The idea for D (or so I've read) is that opAssign isn't required, instead 
 you use a copy constructor. Now, I've little experience with ref counting, 
 or with writing an implementation of one, but, does this work?

 If not, why not? What's missing, what else is required? Overload of '.' 
 perhaps? The ability to pass a RefPtr as the object being counted?

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class RefPtr
 {
 RefPtr parent;
 Object resource = null;
 int refs = 0;

 this(Object res)
 {
 resource = res;
 writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for 
 resource=(",resource,")");
 increment(); }

 this(RefPtr rhs)
 {
 parent = rhs;
 parent.increment();
 writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for 
 resource=(",parent.resource,")");
 }

 ~this()
 {
 int r;

 if ((r = decrement()) == 0)
 {
 writefln("ThreadID=(",Thread.getThis().id,") release last ref 
 Ref=(",r,")");
 if (parent) parent = null;
 else if (resource)
 {
 writefln("ThreadID=(",Thread.getThis().id,") delete 
 resource=(",resource,")");
 delete resource;
 resource = null;
 }
 }
 writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
 }
 protected:
 int increment()
 {
 int ret;

 if (parent) ret = parent.increment();
 else {
 synchronized(this)
 {
 ret = ++refs;
 writefln("ThreadID=(",Thread.getThis().id,") increment to 
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 int decrement()
 {
 int ret;

 if (parent) ret = parent.decrement();
 else {
 synchronized(this)
 {
 ret = --refs;
 writefln("ThreadID=(",Thread.getThis().id,") decrement to 
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 RefPtr pbob;

 static this()
 {
 pbob = new RefPtr(new Resource());
 }

 static ~this()
 {
 delete pbob;
 }

 void main()
 {
 Thread[] threads;
 int i;

 writefln("ThreadID=(",Thread.getThis().id,") is the main thread");

 for(i = 0; i < 10; i++)
 {
 threads ~= new Thread(&thread_function,null);
 threads[$-1].start();
 }

 while(true)
 {
 i = 0;
 foreach(Thread t; threads)
 {
 if (t.getState() == Thread.TS.TERMINATED) i++;
 }
 if (i == 10) break;
 Sleep(100);
 }

 writefln("Main exiting");
 }

 int thread_function(void* isnull)
 {
 auto RefPtr p = new RefPtr(pbob);
 Sleep(1000+rand()%1000); return 0;
 }

 Regan
Jun 26 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
Looking at the code given I believe my code sample can be modified to  
achieve it. opAssign isn't required, you simply use a copy constructor (as  
I have done). Why can't it be a class? Specifically why can't it be an  
'auto' class reference (which will exist on the stack in future)?

Regan

p.s. It's "Regan" not "Reagan" ;)

On Sun, 26 Jun 2005 21:39:48 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:

 Reagan, let's take a look on more simple case - implementation
 of 'auto' pointer (this is simplifed version)

   struct Auto(X)
   {

     // constructor.
     this( X _ptr = 0 ) { ptr = _ptr; owner = true; }

     // the destructor deletes ptr only if we are the owner.
     ~this() { if( owner && ptr ) delete ptr; }

     // the assignment operator.
     void opAssign( X r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r;
         owner = true;
     }

     // the assignment operator. transfer
     // ownership. (not full implementation, sic)
     void opAssign( Auto r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r.release();
         owner = true;
     }

     X value() { return  ptr; }

     // release returns the ptr value and releases ownership
     // if we were previously the owner.

     X release()
     {
       owner = false;
       return value();
     }
     private X ptr;
     private bool owner;
   }

 -------------------------

 {
     Auto!(Object) autop = new Object(); // first instance
     autop = new Object(); // first instance will be deleted;
                                            // and autop contains second
 instance.
 } // second instance will be deleted here
   // even in case of exception.

 This is basic auto_ptr used in C++
 and it simplfies life a lot.
 opAssign and dtors are crucial here.
 And such Auto(X) must be a struct.
 Classes do not work here.

 Andrew.


 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...

 I got little or no comment on this idea/code I posted to another (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on  
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for  
 classes
 (assigning a value to a reference seems illogical) but would be a good  
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

 The idea for D (or so I've read) is that opAssign isn't required,  
 instead
 you use a copy constructor. Now, I've little experience with ref  
 counting,
 or with writing an implementation of one, but, does this work?

 If not, why not? What's missing, what else is required? Overload of '.'
 perhaps? The ability to pass a RefPtr as the object being counted?

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class RefPtr
 {
 RefPtr parent;
 Object resource = null;
 int refs = 0;

 this(Object res)
 {
 resource = res;
 writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for
 resource=(",resource,")");
 increment(); }

 this(RefPtr rhs)
 {
 parent = rhs;
 parent.increment();
 writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for
 resource=(",parent.resource,")");
 }

 ~this()
 {
 int r;

 if ((r = decrement()) == 0)
 {
 writefln("ThreadID=(",Thread.getThis().id,") release last ref
 Ref=(",r,")");
 if (parent) parent = null;
 else if (resource)
 {
 writefln("ThreadID=(",Thread.getThis().id,") delete
 resource=(",resource,")");
 delete resource;
 resource = null;
 }
 }
 writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
 }
 protected:
 int increment()
 {
 int ret;

 if (parent) ret = parent.increment();
 else {
 synchronized(this)
 {
 ret = ++refs;
 writefln("ThreadID=(",Thread.getThis().id,") increment to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 int decrement()
 {
 int ret;

 if (parent) ret = parent.decrement();
 else {
 synchronized(this)
 {
 ret = --refs;
 writefln("ThreadID=(",Thread.getThis().id,") decrement to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 RefPtr pbob;

 static this()
 {
 pbob = new RefPtr(new Resource());
 }

 static ~this()
 {
 delete pbob;
 }

 void main()
 {
 Thread[] threads;
 int i;

 writefln("ThreadID=(",Thread.getThis().id,") is the main thread");

 for(i = 0; i < 10; i++)
 {
 threads ~= new Thread(&thread_function,null);
 threads[$-1].start();
 }

 while(true)
 {
 i = 0;
 foreach(Thread t; threads)
 {
 if (t.getState() == Thread.TS.TERMINATED) i++;
 }
 if (i == 10) break;
 Sleep(100);
 }

 writefln("Main exiting");
 }

 int thread_function(void* isnull)
 {
 auto RefPtr p = new RefPtr(pbob);
 Sleep(1000+rand()%1000); return 0;
 }

 Regan
Jun 26 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
AutoPtr in D...

import std.c.windows.windows;
import std.random;
import std.thread;
import std.stdio;

class AutoPtr
{
	Object resource = null;
	bool owner;

	this(Object rhs)
	{
		resource = rhs;
		owner = true;
		writefln("%x constructed for %x",this,resource);
	}

	this(AutoPtr rhs)
	{		
		resource = rhs.release();
		owner = true;
		writefln("%x obtaining %x from %x",this,resource,rhs);
	}
	
	~this()
	{
		if (resource && owner)
		{
			writefln("%x deleting %x",this,resource);
			delete resource;
			resource = null;
		}		
	}
protected:
	Object value()
	{
		return resource;
	}
	
	Object release()
	{
		writefln("%x releasing %x",this,resource);
		owner = false;
		return value();
	}
}

class Resource
{
	char[] name = "11 was a racehorse";
	char[] toString() { return name; }
}

void main()
{
	Resource Bob = new Resource();
	auto AutoPtr p = new AutoPtr(Bob);
	auto AutoPtr q = new AutoPtr(p);
	writefln("end of main");
}

On Mon, 27 Jun 2005 16:52:09 +1200, Regan Heath <regan netwin.co.nz> wrote:

 Looking at the code given I believe my code sample can be modified to  
 achieve it. opAssign isn't required, you simply use a copy constructor  
 (as I have done). Why can't it be a class? Specifically why can't it be  
 an 'auto' class reference (which will exist on the stack in future)?

 Regan

 p.s. It's "Regan" not "Reagan" ;)

 On Sun, 26 Jun 2005 21:39:48 -0700, Andrew Fedoniouk  
 <news terrainformatica.com> wrote:

 Reagan, let's take a look on more simple case - implementation
 of 'auto' pointer (this is simplifed version)

   struct Auto(X)
   {

     // constructor.
     this( X _ptr = 0 ) { ptr = _ptr; owner = true; }

     // the destructor deletes ptr only if we are the owner.
     ~this() { if( owner && ptr ) delete ptr; }

     // the assignment operator.
     void opAssign( X r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r;
         owner = true;
     }

     // the assignment operator. transfer
     // ownership. (not full implementation, sic)
     void opAssign( Auto r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r.release();
         owner = true;
     }

     X value() { return  ptr; }

     // release returns the ptr value and releases ownership
     // if we were previously the owner.

     X release()
     {
       owner = false;
       return value();
     }
     private X ptr;
     private bool owner;
   }

 -------------------------

 {
     Auto!(Object) autop = new Object(); // first instance
     autop = new Object(); // first instance will be deleted;
                                            // and autop contains second
 instance.
 } // second instance will be deleted here
   // even in case of exception.

 This is basic auto_ptr used in C++
 and it simplfies life a lot.
 opAssign and dtors are crucial here.
 And such Auto(X) must be a struct.
 Classes do not work here.

 Andrew.


 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...

 I got little or no comment on this idea/code I posted to another  
 (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on  
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for  
 classes
 (assigning a value to a reference seems illogical) but would be a good  
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

 The idea for D (or so I've read) is that opAssign isn't required,  
 instead
 you use a copy constructor. Now, I've little experience with ref  
 counting,
 or with writing an implementation of one, but, does this work?

 If not, why not? What's missing, what else is required? Overload of '.'
 perhaps? The ability to pass a RefPtr as the object being counted?

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class RefPtr
 {
 RefPtr parent;
 Object resource = null;
 int refs = 0;

 this(Object res)
 {
 resource = res;
 writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for
 resource=(",resource,")");
 increment(); }

 this(RefPtr rhs)
 {
 parent = rhs;
 parent.increment();
 writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for
 resource=(",parent.resource,")");
 }

 ~this()
 {
 int r;

 if ((r = decrement()) == 0)
 {
 writefln("ThreadID=(",Thread.getThis().id,") release last ref
 Ref=(",r,")");
 if (parent) parent = null;
 else if (resource)
 {
 writefln("ThreadID=(",Thread.getThis().id,") delete
 resource=(",resource,")");
 delete resource;
 resource = null;
 }
 }
 writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
 }
 protected:
 int increment()
 {
 int ret;

 if (parent) ret = parent.increment();
 else {
 synchronized(this)
 {
 ret = ++refs;
 writefln("ThreadID=(",Thread.getThis().id,") increment to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 int decrement()
 {
 int ret;

 if (parent) ret = parent.decrement();
 else {
 synchronized(this)
 {
 ret = --refs;
 writefln("ThreadID=(",Thread.getThis().id,") decrement to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 RefPtr pbob;

 static this()
 {
 pbob = new RefPtr(new Resource());
 }

 static ~this()
 {
 delete pbob;
 }

 void main()
 {
 Thread[] threads;
 int i;

 writefln("ThreadID=(",Thread.getThis().id,") is the main thread");

 for(i = 0; i < 10; i++)
 {
 threads ~= new Thread(&thread_function,null);
 threads[$-1].start();
 }

 while(true)
 {
 i = 0;
 foreach(Thread t; threads)
 {
 if (t.getState() == Thread.TS.TERMINATED) i++;
 }
 if (i == 10) break;
 Sleep(100);
 }

 writefln("Main exiting");
 }

 int thread_function(void* isnull)
 {
 auto RefPtr p = new RefPtr(pbob);
 Sleep(1000+rand()%1000); return 0;
 }

 Regan
Jun 26 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss0is0r223k2f5 nrage.netwin.co.nz...

 AutoPtr in D...

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class AutoPtr
 {
 Object resource = null;
 bool owner;

 this(Object rhs)
 {
 resource = rhs;
 owner = true;
 writefln("%x constructed for %x",this,resource);
 }

 this(AutoPtr rhs)
 { resource = rhs.release();
 owner = true;
 writefln("%x obtaining %x from %x",this,resource,rhs);
 }

 ~this()
 {
 if (resource && owner)
 {
 writefln("%x deleting %x",this,resource);
 delete resource;
 resource = null;
 } }
 protected:
 Object value()
 {
 return resource;
 }

 Object release()
 {
 writefln("%x releasing %x",this,resource);
 owner = false;
 return value();
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 void main()
 {
 Resource Bob = new Resource();
 auto AutoPtr p = new AutoPtr(Bob);
 auto AutoPtr q = new AutoPtr(p);
 writefln("end of main");
 }



void main()
{
   Resource Bob = new Resource();
   Resource Mary = new Resource();

   auto AutoPtr p = new AutoPtr(Bob);
   auto AutoPtr q;

   p = Mary; // What will happen here?

   q = p; // And here?

}

In D, for classes,

      q = p

means assignment of reference and not value
this is why classes do not work here.

Having opAssign implemented allows
to implement control of assigning values -
to catch moment when *variable*
is getting new value.

And there are many useful implications of this feature.
The Auto is just one example.



 On Mon, 27 Jun 2005 16:52:09 +1200, Regan Heath <regan netwin.co.nz> 
 wrote:

 Looking at the code given I believe my code sample can be modified to 
 achieve it. opAssign isn't required, you simply use a copy constructor 
 (as I have done). Why can't it be a class? Specifically why can't it be 
 an 'auto' class reference (which will exist on the stack in future)?

 Regan

 p.s. It's "Regan" not "Reagan" ;)




Ooops, terribly sorry :)))



 On Sun, 26 Jun 2005 21:39:48 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:

 Reagan, let's take a look on more simple case - implementation
 of 'auto' pointer (this is simplifed version)

   struct Auto(X)
   {

     // constructor.
     this( X _ptr = 0 ) { ptr = _ptr; owner = true; }

     // the destructor deletes ptr only if we are the owner.
     ~this() { if( owner && ptr ) delete ptr; }

     // the assignment operator.
     void opAssign( X r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r;
         owner = true;
     }

     // the assignment operator. transfer
     // ownership. (not full implementation, sic)
     void opAssign( Auto r )
     {
         if( owner && ptr ) delete ptr;
         ptr = r.release();
         owner = true;
     }

     X value() { return  ptr; }

     // release returns the ptr value and releases ownership
     // if we were previously the owner.

     X release()
     {
       owner = false;
       return value();
     }
     private X ptr;
     private bool owner;
   }

 -------------------------

 {
     Auto!(Object) autop = new Object(); // first instance
     autop = new Object(); // first instance will be deleted;
                                            // and autop contains second
 instance.
 } // second instance will be deleted here
   // even in case of exception.

 This is basic auto_ptr used in C++
 and it simplfies life a lot.
 opAssign and dtors are crucial here.
 And such Auto(X) must be a struct.
 Classes do not work here.

 Andrew.


 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...

 I got little or no comment on this idea/code I posted to another 
 (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on 
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for 
 classes
 (assigning a value to a reference seems illogical) but would be a good 
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?

 The idea for D (or so I've read) is that opAssign isn't required, 
 instead
 you use a copy constructor. Now, I've little experience with ref 
 counting,
 or with writing an implementation of one, but, does this work?

 If not, why not? What's missing, what else is required? Overload of '.'
 perhaps? The ability to pass a RefPtr as the object being counted?

 import std.c.windows.windows;
 import std.random;
 import std.thread;
 import std.stdio;

 class RefPtr
 {
 RefPtr parent;
 Object resource = null;
 int refs = 0;

 this(Object res)
 {
 resource = res;
 writefln("ThreadID=(",Thread.getThis().id,") Initial RefPtr for
 resource=(",resource,")");
 increment(); }

 this(RefPtr rhs)
 {
 parent = rhs;
 parent.increment();
 writefln("ThreadID=(",Thread.getThis().id,") Ref=(",parent.refs,") for
 resource=(",parent.resource,")");
 }

 ~this()
 {
 int r;

 if ((r = decrement()) == 0)
 {
 writefln("ThreadID=(",Thread.getThis().id,") release last ref
 Ref=(",r,")");
 if (parent) parent = null;
 else if (resource)
 {
 writefln("ThreadID=(",Thread.getThis().id,") delete
 resource=(",resource,")");
 delete resource;
 resource = null;
 }
 }
 writefln("ThreadID=(",Thread.getThis().id,") release Ref=(",r,")");
 }
 protected:
 int increment()
 {
 int ret;

 if (parent) ret = parent.increment();
 else {
 synchronized(this)
 {
 ret = ++refs;
 writefln("ThreadID=(",Thread.getThis().id,") increment to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 int decrement()
 {
 int ret;

 if (parent) ret = parent.decrement();
 else {
 synchronized(this)
 {
 ret = --refs;
 writefln("ThreadID=(",Thread.getThis().id,") decrement to
 Ref=(",refs,")");
 }
 }

 return ret;
 }
 }

 class Resource
 {
 char[] name = "11 was a racehorse";
 char[] toString() { return name; }
 }

 RefPtr pbob;

 static this()
 {
 pbob = new RefPtr(new Resource());
 }

 static ~this()
 {
 delete pbob;
 }

 void main()
 {
 Thread[] threads;
 int i;

 writefln("ThreadID=(",Thread.getThis().id,") is the main thread");

 for(i = 0; i < 10; i++)
 {
 threads ~= new Thread(&thread_function,null);
 threads[$-1].start();
 }

 while(true)
 {
 i = 0;
 foreach(Thread t; threads)
 {
 if (t.getState() == Thread.TS.TERMINATED) i++;
 }
 if (i == 10) break;
 Sleep(100);
 }

 writefln("Main exiting");
 }

 int thread_function(void* isnull)
 {
 auto RefPtr p = new RefPtr(pbob);
 Sleep(1000+rand()%1000); return 0;
 }

 Regan
Jun 26 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Sun, 26 Jun 2005 22:40:03 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 void main()
 {
    Resource Bob = new Resource();
    Resource Mary = new Resource();

    auto AutoPtr p = new AutoPtr(Bob);
    auto AutoPtr q;

    p = Mary; // What will happen here?


ERROR: cannot implicitly convert expression (Mary) of type  
autoptr.Resource to autoptr.AutoPtr

(note "autoptr" was the name of the source file in my case)


    q = p; // And here?


The reference 'q' will now equal the reference 'p'.


 }

 In D, for classes,

       q = p

 means assignment of reference and not value


Correct.


 this is why classes do not work here.


I disagree. They *work* fine. If you meant to create a new AutoPtr and  
transfer ownership you should have used the copy constructor, eg.

   q = new AutoPtr(p);

What you did instead was make a 2nd reference to the same AutoPtr.

This is part of D's design goals (as I read them on the web site) classes  
shall not have opAssign instead you shall use a copy constructor.

I can however see/understand the desire to make "q = p" an error, in order  
to prevent incorrect usage (as above). Have you a suggestion as to how to  
do that? (one that does not involve adding opAssign to classes)

The way forwards, as I see it, is to solve the question above. Asking for  
opAssign for classes will likely get us nowhere.

Of course the other option is to use a struct, the thread you posted  
earlier:
   http://www.digitalmars.com/d/archives/7988.html

tends to suggest to me that struct will not be modified to solve this  
issue.


 Having opAssign implemented allows
 to implement control of assigning values -
 to catch moment when *variable*
 is getting new value.


Notice you say "value" above. You do not assign values to reference, you  
assign references to references, thus opAssign has no place in classes. (I  
believe we've already agreed to this point elsewhere).

Regan
Jun 27 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss0r4ry023k2f5 nrage.netwin.co.nz...

 On Sun, 26 Jun 2005 22:40:03 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 void main()
 {
    Resource Bob = new Resource();
    Resource Mary = new Resource();

    auto AutoPtr p = new AutoPtr(Bob);
    auto AutoPtr q;

    p = Mary; // What will happen here?


 ERROR: cannot implicitly convert expression (Mary) of type 
 autoptr.Resource to autoptr.AutoPtr

 (note "autoptr" was the name of the source file in my case)


    q = p; // And here?


 The reference 'q' will now equal the reference 'p'.


 }

 In D, for classes,

       q = p

 means assignment of reference and not value


 Correct.


 this is why classes do not work here.


 I disagree. They *work* fine. If you meant to create a new AutoPtr and 
 transfer ownership you should have used the copy constructor, eg.

   q = new AutoPtr(p);


Intention for
q = p here is:
1) to free object q is holding.
2) to transfer ownership from p to q.

If p and q will be deleted then only owner
will delete owned object - not both of them.


 What you did instead was make a 2nd reference to the same AutoPtr.

 This is part of D's design goals (as I read them on the web site) classes 
 shall not have opAssign instead you shall use a copy constructor.

 I can however see/understand the desire to make "q = p" an error, in order 
 to prevent incorrect usage (as above). Have you a suggestion as to how to 
 do that? (one that does not involve adding opAssign to classes)

 The way forwards, as I see it, is to solve the question above. Asking for 
 opAssign for classes will likely get us nowhere.

 Of course the other option is to use a struct, the thread you posted 
 earlier:
   http://www.digitalmars.com/d/archives/7988.html

 tends to suggest to me that struct will not be modified to solve this 
 issue.


 Having opAssign implemented allows
 to implement control of assigning values -
 to catch moment when *variable*
 is getting new value.


 Notice you say "value" above. You do not assign values to reference, you 
 assign references to references, thus opAssign has no place in classes. (I 
 believe we've already agreed to this point elsewhere).


Yes. This is why I am using struct in my example.
For classes '=' is copying references.
For sructures it is copying values.
The problem is that for structures '='
silently overrides values without any control
from your side.
Jun 27 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 27 Jun 2005 08:55:50 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0r4ry023k2f5 nrage.netwin.co.nz..

 On Sun, 26 Jun 2005 22:40:03 -0700, Andrew Fedoniouk
 <news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 void main()
 {
    Resource Bob = new Resource();
    Resource Mary = new Resource();

    auto AutoPtr p = new AutoPtr(Bob);
    auto AutoPtr q;

    p = Mary; // What will happen here?


 ERROR: cannot implicitly convert expression (Mary) of type
 autoptr.Resource to autoptr.AutoPtr

 (note "autoptr" was the name of the source file in my case)


    q = p; // And here?


 The reference 'q' will now equal the reference 'p'.


 }

 In D, for classes,

       q = p

 means assignment of reference and not value


 Correct.


 this is why classes do not work here.


 I disagree. They *work* fine. If you meant to create a new AutoPtr and
 transfer ownership you should have used the copy constructor, eg.

   q = new AutoPtr(p);


 Intention for
 q = p here is:
 1) to free object q is holding.
 2) to transfer ownership from p to q.

 If p and q will be deleted then only owner
 will delete owned object - not both of them.


I know what the intention is. My code does that, provided you do not write  
"q = p" but instead write:
   q = new AutoPtr(p);

You could also write "p = null;" to signal the GC may collect (tho if it's  
auto it probably doesn't?)

Stop thinking in C++ and start thinking in D. I repeat. D does not have  
opAssign, where you want opAssign use a copy constructor.

The _only_ problem I can see with the code I posted is that you cannot  
prevent someone accidently writing "q = p" when they meant to write "q =  
new AutoPtr(p);". I'm not sure what the solution is, but it is the way  
forward with this concept IMO.


 Notice you say "value" above. You do not assign values to reference, you
 assign references to references, thus opAssign has no place in classes.  
 (I
 believe we've already agreed to this point elsewhere).


 Yes. This is why I am using struct in my example.
 For classes '=' is copying references.
 For sructures it is copying values.
 The problem is that for structures '='
 silently overrides values without any control
 from your side.


Structs also do not have ctor/dtor.
Classes are the only option for AutoPtr IMO.

Regan
Jun 27 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss1p7dkp23k2f5 nrage.netwin.co.nz...

 On Mon, 27 Jun 2005 08:55:50 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0r4ry023k2f5 nrage.netwin.co.nz..

 On Sun, 26 Jun 2005 22:40:03 -0700, Andrew Fedoniouk
 <news terrainformatica.com> wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 void main()
 {
    Resource Bob = new Resource();
    Resource Mary = new Resource();

    auto AutoPtr p = new AutoPtr(Bob);
    auto AutoPtr q;

    p = Mary; // What will happen here?


 ERROR: cannot implicitly convert expression (Mary) of type
 autoptr.Resource to autoptr.AutoPtr

 (note "autoptr" was the name of the source file in my case)


    q = p; // And here?


 The reference 'q' will now equal the reference 'p'.


 }

 In D, for classes,

       q = p

 means assignment of reference and not value


 Correct.


 this is why classes do not work here.


 I disagree. They *work* fine. If you meant to create a new AutoPtr and
 transfer ownership you should have used the copy constructor, eg.

   q = new AutoPtr(p);


 Intention for
 q = p here is:
 1) to free object q is holding.
 2) to transfer ownership from p to q.

 If p and q will be deleted then only owner
 will delete owned object - not both of them.


 I know what the intention is. My code does that, provided you do not write 
 "q = p" but instead write:
   q = new AutoPtr(p);

 You could also write "p = null;" to signal the GC may collect (tho if it's 
 auto it probably doesn't?)

 Stop thinking in C++ and start thinking in D. I repeat. D does not have 
 opAssign, where you want opAssign use a copy constructor.


If I have C++ why do I need degradation?

And about yours q = new AutoPtr(p);

It does not call destructor of q on assignment
so what is the point of this AutoPtr(p).
'Auto' means that it manages pointer for
me.


 The _only_ problem I can see with the code I posted is that you cannot 
 prevent someone accidently writing "q = p" when they meant to write "q = 
 new AutoPtr(p);". I'm not sure what the solution is, but it is the way 
 forward with this concept IMO.


 Notice you say "value" above. You do not assign values to reference, you
 assign references to references, thus opAssign has no place in classes. 
 (I
 believe we've already agreed to this point elsewhere).


 Yes. This is why I am using struct in my example.
 For classes '=' is copying references.
 For sructures it is copying values.
 The problem is that for structures '='
 silently overrides values without any control
 from your side.


 Structs also do not have ctor/dtor.
 Classes are the only option for AutoPtr IMO.


Without control of assignment and ctor/dtor
it is impossible to implement the whole class of
smart pointers and such thing as COW string.

We agreed that opAssign in classes makes no
sense. So only structs left.

About real COW strings and motivation:
http://www.wxwidgets.org/manuals/2.6.0/wx_wxstringoverview.html#wxstringrefcount

It is impossible to reproduce in D such class
and any other class based on pointer management
paradigm. I mean to define string class which
will behave as a string type and not just a set
of C style functions dealing with
struct slice { char* ptr; size_t length; }.

Which is fine if we agree that D is a next
letter after C and not after C++.

Again, it depends on author motivation.

As far as I can see there are three features
that C++ has and D doesn't have:
struct(opAssign/ctor/dtor) and const.

If they will be there then D may be considered
as a successor of C++. With all possible
implications.

IMHO.

Andrew.
Jun 27 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Mon, 27 Jun 2005 21:31:18 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:

 And about yours q = new AutoPtr(p);
 It does not call destructor of q on assignment


True, In D the destructor is only called when the object is collected, or  
you call "delete".


 so what is the point of this AutoPtr(p).
 'Auto' means that it manages pointer for
 me.


Well, it seems to me that one solution would be for an 'auto' class to be  
deleted the moment it is no longer referenced. (rather than waiting till  
the end of the scope). Not sure if that is possible, but it makes sense.

So, that just leaves the problem of preventing a normal reference  
assignment, got any ideas?


 Notice you say "value" above. You do not assign values to reference,  
 you
 assign references to references, thus opAssign has no place in  
 classes.
 (I
 believe we've already agreed to this point elsewhere).


 Yes. This is why I am using struct in my example.
 For classes '=' is copying references.
 For sructures it is copying values.
 The problem is that for structures '='
 silently overrides values without any control
 from your side.


 Structs also do not have ctor/dtor.
 Classes are the only option for AutoPtr IMO.


 Without control of assignment and ctor/dtor
 it is impossible to implement the whole class of
 smart pointers and such thing as COW string.


Perhaps, but that does not mean we require opAssign. There may be another  
way.


 We agreed that opAssign in classes makes no
 sense. So only structs left.


We could still use classes without using opAssign. This is where I have  
concentrated my efforts since reading the thread you found from back in  
2002 where they decide on using 'auto'.


 About real COW strings and motivation:
 http://www.wxwidgets.org/manuals/2.6.0/wx_wxstringoverview.html#wxstringrefcount

 It is impossible to reproduce in D such class
 and any other class based on pointer management
 paradigm.


It's currently impossible to reproduce classes that work in 'exactly' the  
same way. The D solution does not have to be exactly the same, provided it  
still achieve the goals.


 I mean to define string class which
 will behave as a string type and not just a set
 of C style functions dealing with
 struct slice { char* ptr; size_t length; }.


What would this string class do that char[] cannot?

If your only motivation is to provide a way of passing 'const' string data  
then I think enforcing readonly for "const" and "in" is much more  
productive and would also work for other types in D.


 Again, it depends on author motivation.

 As far as I can see there are three features
 that C++ has and D doesn't have:
 struct(opAssign/ctor/dtor) and const.

 If they will be there then D may be considered
 as a successor of C++. With all possible
 implications.


You have to remember that D is different to C++ and will therefore solve  
the same problems in different ways. I'm looking for a D solution. You  
seem to be asking for D to become C++. I agree D cannot solve these  
problems, yet, but the best way to fix that is to think of a way it could  
solve them *without* changing some of the fundamental design decisions  
that make D what it is.

IMO.

Regan
Jun 27 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss2fh7pw23k2f5 nrage.netwin.co.nz...

 On Mon, 27 Jun 2005 21:31:18 -0700, Andrew Fedoniouk 
 <news terrainformatica.com> wrote:

 And about yours q = new AutoPtr(p);
 It does not call destructor of q on assignment


 True, In D the destructor is only called when the object is collected, or 
 you call "delete".


 so what is the point of this AutoPtr(p).
 'Auto' means that it manages pointer for
 me.


 Well, it seems to me that one solution would be for an 'auto' class to be 
 deleted the moment it is no longer referenced. (rather than waiting till 
 the end of the scope). Not sure if that is possible, but it makes sense.

 So, that just leaves the problem of preventing a normal reference 
 assignment, got any ideas?


 Notice you say "value" above. You do not assign values to reference, 
 you
 assign references to references, thus opAssign has no place in 
 classes.
 (I
 believe we've already agreed to this point elsewhere).


 Yes. This is why I am using struct in my example.
 For classes '=' is copying references.
 For sructures it is copying values.
 The problem is that for structures '='
 silently overrides values without any control
 from your side.


 Structs also do not have ctor/dtor.
 Classes are the only option for AutoPtr IMO.


 Without control of assignment and ctor/dtor
 it is impossible to implement the whole class of
 smart pointers and such thing as COW string.


 Perhaps, but that does not mean we require opAssign. There may be another 
 way.


 We agreed that opAssign in classes makes no
 sense. So only structs left.


 We could still use classes without using opAssign. This is where I have 
 concentrated my efforts since reading the thread you found from back in 
 2002 where they decide on using 'auto'.


 About real COW strings and motivation:
 http://www.wxwidgets.org/manuals/2.6.0/wx_wxstringoverview.html#wxstringrefcount

 It is impossible to reproduce in D such class
 and any other class based on pointer management
 paradigm.


 It's currently impossible to reproduce classes that work in 'exactly' the 
 same way. The D solution does not have to be exactly the same, provided it 
 still achieve the goals.


 I mean to define string class which
 will behave as a string type and not just a set
 of C style functions dealing with
 struct slice { char* ptr; size_t length; }.


 What would this string class do that char[] cannot?

 If your only motivation is to provide a way of passing 'const' string data 
 then I think enforcing readonly for "const" and "in" is much more 
 productive and would also work for other types in D.


 Again, it depends on author motivation.

 As far as I can see there are three features
 that C++ has and D doesn't have:
 struct(opAssign/ctor/dtor) and const.

 If they will be there then D may be considered
 as a successor of C++. With all possible
 implications.


 You have to remember that D is different to C++ and will therefore solve 
 the same problems in different ways. I'm looking for a D solution. You 
 seem to be asking for D to become C++. I agree D cannot solve these 
 problems, yet, but the best way to fix that is to think of a way it could 
 solve them *without* changing some of the fundamental design decisions 
 that make D what it is.


Sigh... Round came to the end. The cycle is closed.

I've started screaming about opAssign after I realized
that const arrays/pointers are persons non grata in D for
some ideological reasons. So idea was to have features in language
allowing to implement const types by myself. That is why
opAssign and co.

Again, currently there are no solutions in D
for strings and smart pointers. I mean real and natural solutions.

--------------------
And about string and char[]
char[] is a slice - fragment of character sequence and not a string.
String variable represent/manage storage of textual data.
In contrary char[] is just a poitner bundled with length.
It is not more than char* - it does not have a concept
of ownership - who and when can or cannot modify a string.

BTW: char* and char[] are different in principle:
in char* length of string is part of data (zero terminated).
and in char[] it is part of particular variable. This
is also why char[] is not a string.

char[] can be used as a string field in some structures only
if it *always* points to some *unique* memory location -
owns it. So you can *safely* manipulate owning text.

By itself char[] is convenient way to define and pass slices
to functions - temporary (short living) structures.
But here usability is limited by lack of const too. Sigh...

string class is a COW manager of character data. Using it you
don't need to invent COW algorithms each time when you are dealing
with text and to account what source this text came from - it significantly
reduces complexity of code, increases robustness of system in general
and uses memory in most optimal way.

Andrew.
Jun 28 2005
next sibling parent reply Brad Beveridge <brad somewhere.net> writes:
<SNIP>

 
 Sigh... Round came to the end. The cycle is closed.
 
 I've started screaming about opAssign after I realized
 that const arrays/pointers are persons non grata in D for
 some ideological reasons. So idea was to have features in language
 allowing to implement const types by myself. That is why
 opAssign and co.
 


<snip>


 Andrew. 
 
 


This topic, and many like them, have been kicked around for quite a 
while now.  I would like to hear what Walter has to say on all this. 
Walter - do you have particular reasons why opAssign, or "const" 
can't/won't/shouldn't be implemented in D?
To me it looks like Regan and Andrew (and many others) both want the 
same thing in the end - Regan is trying to do it within the confines of 
the current language (and lets face it, not having an easy time of it, 
which should tell us something), and Andrew has concluded that it is 
impossible without changes to D.

So, what do you say?

Are Andrew and Regan trying to do something that isn't required in D? 
Or is D lacking important functionality?

Cheers
Brad
Jun 28 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Tue, 28 Jun 2005 11:20:36 -0700, Brad Beveridge <brad somewhere.net>  
wrote:

 <SNIP>

  Sigh... Round came to the end. The cycle is closed.
  I've started screaming about opAssign after I realized
 that const arrays/pointers are persons non grata in D for
 some ideological reasons. So idea was to have features in language
 allowing to implement const types by myself. That is why
 opAssign and co.


 <snip>


 Andrew.


 This topic, and many like them, have been kicked around for quite a  
 while now.  I would like to hear what Walter has to say on all this.  
 Walter - do you have particular reasons why opAssign, or "const"  
 can't/won't/shouldn't be implemented in D?
 To me it looks like Regan and Andrew (and many others) both want the  
 same thing in the end - Regan is trying to do it within the confines of  
 the current language (and lets face it, not having an easy time of it,  
 which should tell us something), and Andrew has concluded that it is  
 impossible without changes to D.

 So, what do you say?

 Are Andrew and Regan trying to do something that isn't required in D? Or  
 is D lacking important functionality?


Good post.

Regan
Jun 28 2005
prev sibling parent =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= <afb algonet.se> writes:
Brad Beveridge wrote:


 So, what do you say?
 
 Are Andrew and Regan trying to do something that isn't required in D?


Maybe isn't required if there was some other way to protect the strings.


 Or is D lacking important functionality?


Yes, as most have pointed out by now: D is sorely lacking a "readonly" !


This makes everyone look for workarounds like unsafe character arrays
(just like in old C) or string wrapper classes (just like in Java)...

Instead of having a "readonly char[]" parameter ?
(old name "const" changed, for political reasons)


My 2 öre,
--anders
Jun 29 2005
prev sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Tue, 28 Jun 2005 11:08:49 -0700, Andrew Fedoniouk  
<news terrainformatica.com> wrote:

 You have to remember that D is different to C++ and will therefore solve
 the same problems in different ways. I'm looking for a D solution. You
 seem to be asking for D to become C++. I agree D cannot solve these
 problems, yet, but the best way to fix that is to think of a way it  
 could
 solve them *without* changing some of the fundamental design decisions
 that make D what it is.


 Sigh... Round came to the end. The cycle is closed.

 I've started screaming about opAssign after I realized
 that const arrays/pointers are persons non grata in D for
 some ideological reasons. So idea was to have features in language
 allowing to implement const types by myself. That is why
 opAssign and co.

 Again, currently there are no solutions in D
 for strings and smart pointers. I mean real and natural solutions.


I agree.


 --------------------
 And about string and char[]
 char[] is a slice - fragment of character sequence and not a string.
 String variable represent/manage storage of textual data.


I still don't see the difference. A char[] represents textual data, a char  
manages storage of textual data (it has a length you can set, it handles  
concatenation, with const/in protection it would handle ownership)


 In contrary char[] is just a poitner bundled with length.
 It is not more than char* - it does not have a concept
 of ownership - who and when can or cannot modify a string.


It would if we had readonly protection for 'const' and 'in' thought, right?


 BTW: char* and char[] are different in principle:
 in char* length of string is part of data (zero terminated).


I know, I have a C background.


 and in char[] it is part of particular variable. This
 is also why char[] is not a string.


So char* is a string but char[] isn't? That statement seems odd to me.


 char[] can be used as a string field in some structures only
 if it *always* points to some *unique* memory location -
 owns it. So you can *safely* manipulate owning text.


You are supposed to use COW. I agree without some sort of const/in  
protection you're left guessing whether you own it. With const/in  
protection you *know* whether you own a string, if it's const or passed as  
'in' you dont, otherwise you do, right?


 By itself char[] is convenient way to define and pass slices
 to functions - temporary (short living) structures.
 But here usability is limited by lack of const too. Sigh...


Sure, we need const/in protection. We both agree there.


 string class is a COW manager of character data.


Aha! So this is the definition you're using. I believe with const/in  
protection char[] can be used with COW safely and with surety (no  
guessing).


 Using it you
 don't need to invent COW algorithms each time when you are dealing
 with text and to account what source this text came from - it  
 significantly
 reduces complexity of code, increases robustness of system in general
 and uses memory in most optimal way.


Agreed. We need something to help us use COW without guessing or adding  
complicated code to handle it.

In short, if char[] had const/in protection do you agree that it would be  
the 'string class' you require? If not, what other factors are missing?  
And would those factors be handled if we could design and use an AutoPtr  
to a char[]?

Regan.
Jun 28 2005
prev sibling next sibling parent reply David Medlock <noone nowhere.com> writes:
Regan Heath wrote:


 I got little or no comment on this idea/code I posted to another 
 (rather  large) thread here, I figured I'd post it again and see what 
 people  thought.
 
 After reading this thread (the one I mention above) and also this one  
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
 
 It appears you need a type that:
 
 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.
 
 It seems (from recent posts) that Walter plans to put auto classes on 
 the  stack, so they'd then fulfil requirements 1 and 2.
 
 But what about 3? To me, opAssign isn't generally a good idea for 
 classes  (assigning a value to a reference seems illogical) but would be 
 a good for  structs (they're value types, assignment assigns a value).
 
 Can you do reference counting without opAssign?
 


<snip>

 Regan


I thought the stack based types were to be useful for speed gains, not 
managing memory.

Why do you need reference counting?  Why not just let the gc deallocator 
handle it?  Are you running into a problem running out of memory? 
Deallocation should not be such a central concept, otherwise we might as 
well ditch the gc.

In your threads example, you probably would use a manager to watch the 
threads, not rely on the runtime to tell you when to free one.

-DavidM
Jun 29 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com>  
wrote:

 Regan Heath wrote:


 I got little or no comment on this idea/code I posted to another  
 (rather  large) thread here, I figured I'd post it again and see what  
 people  thought.
  After reading this thread (the one I mention above) and also this one   
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.
  It seems (from recent posts) that Walter plans to put auto classes on  
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for  
 classes  (assigning a value to a reference seems illogical) but would  
 be a good for  structs (they're value types, assignment assigns a  
 value).
  Can you do reference counting without opAssign?


 <snip>

 Regan


 I thought the stack based types were to be useful for speed gains, not  
 managing memory.


Correct. We need/want the reference counter or auto pointer to be as fast  
as possible because they're typically instantiated and destroyed with  
abandon. Andrew posted a good example of this, where his graphics library  
needs to obtain ownership of a graphics context on every call to the  
'paint' method, it needs to get it, use it and release it, this function  
is called very often.


 Why do you need reference counting?


I don't, yet. Andrew does, now.


 Why not just let the gc deallocator handle it?


Because the problem we're trying to solve is 'ownership' and 'efficiency  
of memory'. In other words knowing when you need to copy a char[] not just  
guessing. At the moment people are likely copying to be safe, meaning a  
bunch of memory allocations you dont actually need.

IMO we can solve these issues by enforcing readonly for  
'const/readonly/whatever' and 'in parameters', eg.

//this class returns readonly access to a string it owns
class Foo {
   char[] ownedByFoo;
   readonly char[] getIt() {
     return ownedByFoo[0..5];
   }
}

void mutate(char[] arr) {
   arr[0] = 'b'; //err 'arr' is readonly
}

void mutate2(inout char[] arr) {
   arr[0] = 'b';
}

Foo f = new Foo();
char[] p = f.getIt(); //'readonly' not required**
char[] s = p.dup; //dup does not copy readonly flag
p[0] = 'a'; //error p is readonly
s[0] = 'a'; //ok
mutate(p);  //ok, function def is actually in error
mutate(s);  //ok
mutate2(p); //error p is readonly
mutate2(s); //ok

** on variable decls (it's _not_ a seperate type, it's the _same_ type  
with a readonly bit flagged).

All types could have a readonly bit. Ideally only during compile, i.e. the  
compiler does a pass where it verifies no variable it has previously  
flagged as readonly (does the flagging during it's normal existing passes)  
has been passed as a mutable variable, or modified with inbuilt operator  
etc.


 Are you running into a problem running out of memory?


No. It's all about ownership and not having to guess when COW should be  
actioned. A reference counted string knows it's the only reference thus  
does not need to dup. An autopointer knows whether it's the owner and if  
not does a dup. Currently char[] has no such knowledge.


 Deallocation should not be such a central concept, otherwise we might as  
 well ditch the gc.


Of course not, however for certain types you need to action something as  
they die, refcnt, autoptr and classes with resources to release. 'auto'  
seems to handle these cases.

A remaining problem is that refcnt and autoptr need to fast, so we want  
them on the stack. Walter has mentioned 'auto' classes will eventually be  
on the stack, yay.

The last issue is that it need to be illegal to assign an RefCnt to  
another RefCnt without increasing the reference count, i.e.

RefCnt p = new RefCnt(object);
RefCnt s = p; //should either be illegal, or should increment refcnt.

This is currently impossible in D.


 In your threads example, you probably would use a manager to watch the  
 threads, not rely on the runtime to tell you when to free one.


A manager? you mean a whole other thread just to watch the first 10  
threads?

I've since decided it's just simpler to store the whole lot of them in an  
array, at least then I know the references cant be null (which was the  
problem with initial code - not handling null case).

You can add null handling, and assume null means terminated and the  
initial code functions correctly.


All in all, if you read the entire thread, you'll come to Brad's post  
which sums the situation up nicely. We need some direction from Walter,  
either:

  - You dont need those in D (and reasoning)
  - Heres how we do it in D  (example)
  - I see, I will think about soln
  - Ok you're right I will add x,y and z

Regan
Jun 29 2005
parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss5jwkvj23k2f5 nrage.netwin.co.nz...

 On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com> 
 wrote:

 Regan Heath wrote:


 I got little or no comment on this idea/code I posted to another 
 (rather  large) thread here, I figured I'd post it again and see what 
 people  thought.
  After reading this thread (the one I mention above) and also this one 
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.
  It seems (from recent posts) that Walter plans to put auto classes on 
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for 
 classes  (assigning a value to a reference seems illogical) but would 
 be a good for  structs (they're value types, assignment assigns a 
 value).
  Can you do reference counting without opAssign?


 <snip>

 Regan


 I thought the stack based types were to be useful for speed gains, not 
 managing memory.


 Correct. We need/want the reference counter or auto pointer to be as fast 
 as possible because they're typically instantiated and destroyed with 
 abandon. Andrew posted a good example of this, where his graphics library 
 needs to obtain ownership of a graphics context on every call to the 
 'paint' method, it needs to get it, use it and release it, this function 
 is called very often.


 Why do you need reference counting?


 I don't, yet. Andrew does, now.


Reference counting is one of most famous design patterns.
RC as a memory management has its own limitations though ( e.g. cyclic 
refs ).
Its power in those areas where GC is weak. And vice versa.
So be able to use both will benefit the whole system twice.

Good example of RC is in implementations of COW strings and shared_ptr
in C++.


 Why not just let the gc deallocator handle it?


 Because the problem we're trying to solve is 'ownership' and 'efficiency 
 of memory'. In other words knowing when you need to copy a char[] not just 
 guessing. At the moment people are likely copying to be safe, meaning a 
 bunch of memory allocations you dont actually need.

 IMO we can solve these issues by enforcing readonly for 
 'const/readonly/whatever' and 'in parameters', eg.


Pretty much yes.


 //this class returns readonly access to a string it owns
 class Foo {
   char[] ownedByFoo;
   readonly char[] getIt() {
     return ownedByFoo[0..5];
   }
 }

 void mutate(char[] arr) {
   arr[0] = 'b'; //err 'arr' is readonly
 }

 void mutate2(inout char[] arr) {
   arr[0] = 'b';
 }

 Foo f = new Foo();
 char[] p = f.getIt(); //'readonly' not required**
 char[] s = p.dup; //dup does not copy readonly flag
 p[0] = 'a'; //error p is readonly
 s[0] = 'a'; //ok
 mutate(p);  //ok, function def is actually in error
 mutate(s);  //ok
 mutate2(p); //error p is readonly
 mutate2(s); //ok

 ** on variable decls (it's _not_ a seperate type, it's the _same_ type 
 with a readonly bit flagged).

 All types could have a readonly bit. Ideally only during compile, i.e. the 
 compiler does a pass where it verifies no variable it has previously 
 flagged as readonly (does the flagging during it's normal existing passes) 
 has been passed as a mutable variable, or modified with inbuilt operator 
 etc.


 Are you running into a problem running out of memory?


 No. It's all about ownership and not having to guess when COW should be 
 actioned. A reference counted string knows it's the only reference thus 
 does not need to dup. An autopointer knows whether it's the owner and if 
 not does a dup. Currently char[] has no such knowledge.


 Deallocation should not be such a central concept, otherwise we might as 
 well ditch the gc.


 Of course not, however for certain types you need to action something as 
 they die, refcnt, autoptr and classes with resources to release. 'auto' 
 seems to handle these cases.

 A remaining problem is that refcnt and autoptr need to fast, so we want 
 them on the stack. Walter has mentioned 'auto' classes will eventually be 
 on the stack, yay.


Exactly!


 The last issue is that it need to be illegal to assign an RefCnt to 
 another RefCnt without increasing the reference count, i.e.

 RefCnt p = new RefCnt(object);
 RefCnt s = p; //should either be illegal, or should increment refcnt.

 This is currently impossible in D.


Sigh....


 In your threads example, you probably would use a manager to watch the 
 threads, not rely on the runtime to tell you when to free one.


 A manager? you mean a whole other thread just to watch the first 10 
 threads?

 I've since decided it's just simpler to store the whole lot of them in an 
 array, at least then I know the references cant be null (which was the 
 problem with initial code - not handling null case).

 You can add null handling, and assume null means terminated and the 
 initial code functions correctly.


 All in all, if you read the entire thread, you'll come to Brad's post 
 which sums the situation up nicely. We need some direction from Walter, 
 either:

  - You dont need those in D (and reasoning)
  - Heres how we do it in D  (example)
  - I see, I will think about soln
  - Ok you're right I will add x,y and z

 Regan
Jun 29 2005
parent reply David Medlock <noone nowhere.com> writes:
Andrew Fedoniouk wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message 
 news:opss5jwkvj23k2f5 nrage.netwin.co.nz...
 

On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com> 
wrote:


Regan Heath wrote:



Regan


I thought the stack based types were to be useful for speed gains, not 
managing memory.


Correct. We need/want the reference counter or auto pointer to be as fast 
as possible because they're typically instantiated and destroyed with 
abandon. Andrew posted a good example of this, where his graphics library 
needs to obtain ownership of a graphics context on every call to the 
'paint' method, it needs to get it, use it and release it, this function 
is called very often.




So why not simply grab the reference once per frame(as in frames per 
second) and draw all the needed objects?  Dont pass the context to the 
objects, pass the objects to the context owner.

I've written code of this type (managing texture memory in OpenGL for 
one) and its much easier managed on a per-frame basis than per object.

(Sorry if I am not understanding this correctly, couldnt find the 
example you mention)

Why do you need reference counting?


I don't, yet. Andrew does, now.


 
 
 Reference counting is one of most famous design patterns.
 RC as a memory management has its own limitations though ( e.g. cyclic 
 refs ).
 Its power in those areas where GC is weak. And vice versa.
 So be able to use both will benefit the whole system twice.
 
 Good example of RC is in implementations of COW strings and shared_ptr
 in C++.
 
 


Those are required in C++ because there is no garbage collector.  In C++ 
Copy on write is a programming discipline/design.  Its still up to the 
programmer though, const just helps to keep the programmers honest.

With (non-transactional)concurrent programming, there is no way to use 
reference counting without locks at each assignment/deref.

Consider:
A resource we call A has 2 references to it, owned by a variable in both 
Thread T and U.

Thread T calls deref, but is blocked before it can check the reference 
count of A.

Thread U calls deref as well, sees the reference count is zero, and 
deallocates the reference.

Thread T awakens, sees the reference count is  zero, calls 
destructor....OOPS.

Not to mention modern architectures are very cache-sensitive. When you 
call destructors throughout your code, you 'thrash the cache' jumping 
around in memory a lot more than a gc calling the same destructors 
several times in sequence.

I reject the assertion that managing references at each object is 
superior to an external collector(of any type, not simply memory). All 
those AddRef/Decref/Ref checks are not free.

Managed resources should be managed by the code, not the runtime.  I'm 
sure some will disagree with some(or all) of this, but I have yet to see 
evidence to the contrary.

-DavidM
Jun 30 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 09:29:19 -0400, David Medlock <noone nowhere.com>  
wrote:

 Andrew Fedoniouk wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message  
 news:opss5jwkvj23k2f5 nrage.netwin.co.nz...


 On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com>  
 wrote:


 Regan Heath wrote:



 Regan


 I thought the stack based types were to be useful for speed gains,  
 not managing memory.


 Correct. We need/want the reference counter or auto pointer to be as  
 fast as possible because they're typically instantiated and destroyed  
 with abandon. Andrew posted a good example of this, where his graphics  
 library needs to obtain ownership of a graphics context on every call  
 to the 'paint' method, it needs to get it, use it and release it, this  
 function is called very often.




 So why not simply grab the reference once per frame(as in frames per  
 second) and draw all the needed objects?  Dont pass the context to the  
 objects, pass the objects to the context owner.

 I've written code of this type (managing texture memory in OpenGL for  
 one) and its much easier managed on a per-frame basis than per object.

 (Sorry if I am not understanding this correctly, couldnt find the  
 example you mention)


I'll let Andrew pick this up if he wants.


 Why do you need reference counting?


 I don't, yet. Andrew does, now.


   Reference counting is one of most famous design patterns.
 RC as a memory management has its own limitations though ( e.g. cyclic  
 refs ).
 Its power in those areas where GC is weak. And vice versa.
 So be able to use both will benefit the whole system twice.
  Good example of RC is in implementations of COW strings and shared_ptr
 in C++.


 Those are required in C++ because there is no garbage collector.  In C++  
 Copy on write is a programming discipline/design.  Its still up to the  
 programmer though, const just helps to keep the programmers honest.


Yes, "const" helps make it bug tolerant.


 With (non-transactional)concurrent programming, there is no way to use  
 reference counting without locks at each assignment/deref.

 Consider:
 A resource we call A has 2 references to it, owned by a variable in both  
 Thread T and U.

 Thread T calls deref, but is blocked before it can check the reference  
 count of A.

 Thread U calls deref as well, sees the reference count is zero, and  
 deallocates the reference.

 Thread T awakens, sees the reference count is  zero, calls  
 destructor....OOPS.


Yes, the process needs to be sync'ed/locked.


 Not to mention modern architectures are very cache-sensitive. When you  
 call destructors throughout your code, you 'thrash the cache' jumping  
 around in memory a lot more than a gc calling the same destructors  
 several times in sequence.

 I reject the assertion that managing references at each object is  
 superior to an external collector(of any type, not simply memory). All  
 those AddRef/Decref/Ref checks are not free.


Oh I agree. But I'm not suggesting you'd want to use this for every  
object. Just certain objects. Being able to reply on the GC is great for  
the bulk of tasks, however certain objects, those that require explicit  
deterministic destruction are the ones you'd want to reference count.


 Managed resources should be managed by the code, not the runtime.  I'm  
 sure some will disagree with some(or all) of this, but I have yet to see  
 evidence to the contrary.


Reference counting, like GC, is not a panacea, it's not the perfect  
solution to all memory management problems. What it is, is a useful tool  
for certain occasions, a tool which can be bug tolerant and easy to use  
(if only it were possible to write an implementation which was these  
things in D).

Regan
Jun 30 2005
prev sibling parent reply "Andrew Fedoniouk" <news terrainformatica.com> writes:
"David Medlock" <noone nowhere.com> wrote in message 
news:da0s3i$1309$1 digitaldaemon.com...

 Andrew Fedoniouk wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message 
 news:opss5jwkvj23k2f5 nrage.netwin.co.nz...


On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com> 
wrote:


Regan Heath wrote:



Regan


I thought the stack based types were to be useful for speed gains, not 
managing memory.


Correct. We need/want the reference counter or auto pointer to be as fast 
as possible because they're typically instantiated and destroyed with 
abandon. Andrew posted a good example of this, where his graphics library 
needs to obtain ownership of a graphics context on every call to the 
'paint' method, it needs to get it, use it and release it, this function 
is called very often.




 So why not simply grab the reference once per frame(as in frames per 
 second) and draw all the needed objects?  Dont pass the context to the 
 objects, pass the objects to the context owner.

 I've written code of this type (managing texture memory in OpenGL for one) 
 and its much easier managed on a per-frame basis than per object.

 (Sorry if I am not understanding this correctly, couldnt find the example 
 you mention)


window::draw()
{
   auto Graphics g = new Graphics( this );
   draw_content(...);
}

Graphics is contains single memeber HDC (on win32).
destructor contains : ReleaseDC(...).

Heap allocation of Graphics here is far from optimal.



Why do you need reference counting?


I don't, yet. Andrew does, now.



 Reference counting is one of most famous design patterns.
 RC as a memory management has its own limitations though ( e.g. cyclic 
 refs ).
 Its power in those areas where GC is weak. And vice versa.
 So be able to use both will benefit the whole system twice.

 Good example of RC is in implementations of COW strings and shared_ptr
 in C++.


 Those are required in C++ because there is no garbage collector.  In C++ 
 Copy on write is a programming discipline/design.  Its still up to the 
 programmer though, const just helps to keep the programmers honest.

 With (non-transactional)concurrent programming, there is no way to use 
 reference counting without locks at each assignment/deref.

 Consider:
 A resource we call A has 2 references to it, owned by a variable in both 
 Thread T and U.

 Thread T calls deref, but is blocked before it can check the reference 
 count of A.

 Thread U calls deref as well, sees the reference count is zero, and 
 deallocates the reference.

 Thread T awakens, sees the reference count is  zero, calls 
 destructor....OOPS.

 Not to mention modern architectures are very cache-sensitive. When you 
 call destructors throughout your code, you 'thrash the cache' jumping 
 around in memory a lot more than a gc calling the same destructors several 
 times in sequence.

 I reject the assertion that managing references at each object is superior 
 to an external collector(of any type, not simply memory). All those 
 AddRef/Decref/Ref checks are not free.


Absolutely true.

Refcounting as a sole method of managing memory is bad.

The point is simple:  Nor GC nor RC are silver bullets. Together
they can be considered as a suboptimal solution.
To be short - I would like to have a choice in each particular case.

The thing is - in D there is no physical way to implement ligthweight
RC.


 Managed resources should be managed by the code, not the runtime.  I'm 
 sure some will disagree with some(or all) of this, but I have yet to see 
 evidence to the contrary.


I agree.


 -DavidM
Jul 01 2005
parent "Ben Hinkle" <bhinkle mathworks.com> writes:
"Andrew Fedoniouk" <news terrainformatica.com> wrote in message 
news:da40ce$1olo$1 digitaldaemon.com...

 "David Medlock" <noone nowhere.com> wrote in message 
 news:da0s3i$1309$1 digitaldaemon.com...

 Andrew Fedoniouk wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message 
 news:opss5jwkvj23k2f5 nrage.netwin.co.nz...


On Wed, 29 Jun 2005 09:41:01 -0400, David Medlock <noone nowhere.com> 
wrote:


Regan Heath wrote:



Regan


I thought the stack based types were to be useful for speed gains, not 
managing memory.


Correct. We need/want the reference counter or auto pointer to be as 
fast as possible because they're typically instantiated and destroyed 
with abandon. Andrew posted a good example of this, where his graphics 
library needs to obtain ownership of a graphics context on every call to 
the 'paint' method, it needs to get it, use it and release it, this 
function is called very often.




 So why not simply grab the reference once per frame(as in frames per 
 second) and draw all the needed objects?  Dont pass the context to the 
 objects, pass the objects to the context owner.

 I've written code of this type (managing texture memory in OpenGL for 
 one) and its much easier managed on a per-frame basis than per object.

 (Sorry if I am not understanding this correctly, couldnt find the example 
 you mention)


 window::draw()
 {
   auto Graphics g = new Graphics( this );
   draw_content(...);
 }

 Graphics is contains single memeber HDC (on win32).
 destructor contains : ReleaseDC(...).

 Heap allocation of Graphics here is far from optimal.


As another datapoint MinWin uses a free list to manage Graphics instances. 
The draw() method grabs a Graphics from the free list and returns it after 
finishing drawing. It's the only class in MinWin with custom memory 
management (so far).
Jul 01 2005
prev sibling next sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Regan Heath wrote:

 I got little or no comment on this idea/code I posted to another 
 (rather  large) thread here, I figured I'd post it again and see what 
 people  thought.
 
 After reading this thread (the one I mention above) and also this one  
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
 
 It appears you need a type that:
 
 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.


You can give a struct or class a method called opAssign if you want. 
Whether

     qwert = yuiop;

will act as syntactic sugar for it is another matter.


 It seems (from recent posts) that Walter plans to put auto classes on 
 the  stack, so they'd then fulfil requirements 1 and 2.
 
 But what about 3? To me, opAssign isn't generally a good idea for 
 classes  (assigning a value to a reference seems illogical) but would be 
 a good for  structs (they're value types, assignment assigns a value).
 
 Can you do reference counting without opAssign?


<snip>

Yes.

Of the claims by various people that we "need" overloading of the 
assignment operator, I'm yet to see one that's true.  AFAIC if you want 
something to the effect of custom assignment behaviour, you can create 
your own method with whatever name that does it.  Can anyone supply an 
example, let alone a real-world one, that doesn't work unless the 
aforementioned syntactic sugar is available?

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 Regan Heath wrote:

 I got little or no comment on this idea/code I posted to another  
 (rather  large) thread here, I figured I'd post it again and see what  
 people  thought.
  After reading this thread (the one I mention above) and also this one   
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.


 You can give a struct or class a method called opAssign if you want.  
 Whether

      qwert = yuiop;

 will act as syntactic sugar for it is another matter.


True, but not the problem.. read on :)


 It seems (from recent posts) that Walter plans to put auto classes on  
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for  
 classes  (assigning a value to a reference seems illogical) but would  
 be a good for  structs (they're value types, assignment assigns a  
 value).
  Can you do reference counting without opAssign?


 <snip>

 Yes.


Then please enlighten me. I've tried, it 'almost' works but it suffers the  
fatal flaw shown below.


 Of the claims by various people that we "need" overloading of the  
 assignment operator, I'm yet to see one that's true. AFAIC if you want  
 something to the effect of custom assignment behaviour, you can create  
 your own method with whatever name that does it.  Can anyone supply an  
 example, let alone a real-world one, that doesn't work unless the  
 aforementioned syntactic sugar is available?


The problem, as I see it, is that it's not just syntactic sugar. You  
cannot prevent reference assignment in cases where the idea/technique  
requires it. As in these 2 cases. (reference counting, auto pointer)

I've tried to write a reference counting implementation and also an auto  
pointer implementation using the recommended practices found on the D  
website. Both work, both use a "copy constructor" instead of opAssign (as  
suggested by the D spec/website). The problem is this:

auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p owns  
object. (or "pwns" for those of you..)
auto AutoPtr s;

s = new AutoPtr(p); //no problem, the copy constructor transfers ownership.
s = p; //problem, s now refers to p, both 'own' "object".

The same problem occurs with the reference counting implementation. s  
becomes a 2nd reference to object but the reference count does not  
increase.

We can 'ask' people not to use a straight reference assignment but we  
cannot enforce it.

Logically I agree, opAssign makes no sense for reference types, you assign  
references to reference types *not* values. opAssign would effectively  
change a reference assignment into a value assignment in a non obvious  
way. I agree with all this reasoning, however, the problem remains.

Regan
Jun 30 2005
next sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
Oh, before you get the impression I think we *need* opAssign... I don't.

It all stems from a desire to protect data and/or be totally sure about  
when to COW and when not to COW. Guessing, which is where we are now is  
inefficient and seems like voodoo magic as opposed to computer science.

We might want to return a slice to some private data, eg.

class Foo {
   private char[] data;
   public char[] get() { return data[0..4]; }
}

Foo f = new Foo();
char[] p = f.get();
p[1] = 'd'; //oops they just nuked it.

So, we're left saying:

class Foo {
   private char[] data;
   public char[] get() { return data[0..4].dup; }
}

which works, but in cases where they didn't write to it, or also thought  
"I should dup this before I write to it" (COW) we have a wasted an  
allocation or 2.

The best soln I've read here / thought of is to enforce some sort of  
"readonly" (be it the oft maligned "const" or some other keyword) the  
result would be a *compile* time enforcement of "readonly", eg.

class Foo {
   private char[] data;
   public readonly char[] get() { return data[0..4]; }
}

Foo f = new Foo();
char[] p = f.get();
p[1] = 'd'; //error 'p' is readonly

(note, "readonly" is not required to be applied to the declaration of p.  
This is an important difference between this idea and 'const' from  
C++ "readonly char[]" is _not_ a seperate type to "char[]" it's simply  
achar[] with a readonly flagged to true during compile)

Further, 'in' parameters also require readonly enforcement, eg.

Foo f = new Foo();
mutate(f.get());

void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
}

if a function wants to modify a parameter, that parameter should be 'out'  
or 'inout', meaning, if we re-write mutate:

Foo f = new Foo();
mutate(f.get()); //error, f.get() is readonly

void mutate(inout char[] p) {
   p[0] = 'd'; //no error
}

It's compile time checking, all the compiler need do is flag variables as  
being readonly, and pass that flag on during assignment (as in the case of  
"char[] p = f.get();" above). It's not the "const hell" some see in C++.

With these abilities I believe the requirement for Auto pointers and  
reference counting is lessened (perhaps removed entirely)

Regan

p.s. This has got to be the 10th time I've posted this idea, I'm sorry if  
I sound like a broken record but I've yet to hear any opposition to it,  
 from anyone, Walter included.

On Thu, 30 Jun 2005 22:58:44 +1200, Regan Heath <regan netwin.co.nz> wrote:


 On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
 wrote:

 Regan Heath wrote:

 I got little or no comment on this idea/code I posted to another  
 (rather  large) thread here, I figured I'd post it again and see what  
 people  thought.
  After reading this thread (the one I mention above) and also this  
 one  Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.


 You can give a struct or class a method called opAssign if you want.  
 Whether

      qwert = yuiop;

 will act as syntactic sugar for it is another matter.


 True, but not the problem.. read on :)


 It seems (from recent posts) that Walter plans to put auto classes on  
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for  
 classes  (assigning a value to a reference seems illogical) but would  
 be a good for  structs (they're value types, assignment assigns a  
 value).
  Can you do reference counting without opAssign?


 <snip>

 Yes.


 Then please enlighten me. I've tried, it 'almost' works but it suffers  
 the fatal flaw shown below.


 Of the claims by various people that we "need" overloading of the  
 assignment operator, I'm yet to see one that's true. AFAIC if you want  
 something to the effect of custom assignment behaviour, you can create  
 your own method with whatever name that does it.  Can anyone supply an  
 example, let alone a real-world one, that doesn't work unless the  
 aforementioned syntactic sugar is available?


 The problem, as I see it, is that it's not just syntactic sugar. You  
 cannot prevent reference assignment in cases where the idea/technique  
 requires it. As in these 2 cases. (reference counting, auto pointer)

 I've tried to write a reference counting implementation and also an auto  
 pointer implementation using the recommended practices found on the D  
 website. Both work, both use a "copy constructor" instead of opAssign  
 (as suggested by the D spec/website). The problem is this:

 auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p  
 owns object. (or "pwns" for those of you..)
 auto AutoPtr s;

 s = new AutoPtr(p); //no problem, the copy constructor transfers  
 ownership.
 s = p; //problem, s now refers to p, both 'own' "object".

 The same problem occurs with the reference counting implementation. s  
 becomes a 2nd reference to object but the reference count does not  
 increase.

 We can 'ask' people not to use a straight reference assignment but we  
 cannot enforce it.

 Logically I agree, opAssign makes no sense for reference types, you  
 assign references to reference types *not* values. opAssign would  
 effectively change a reference assignment into a value assignment in a  
 non obvious way. I agree with all this reasoning, however, the problem  
 remains.

 Regan
Jun 30 2005
next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 The best soln I've read here / thought of is to enforce some sort of 
 "readonly" (be it the oft maligned "const" or some other keyword) the 
 result would be a *compile* time enforcement of "readonly", eg.

 class Foo {
   private char[] data;
   public readonly char[] get() { return data[0..4]; }
 }

 Foo f = new Foo();
 char[] p = f.get();
 p[1] = 'd'; //error 'p' is readonly

 (note, "readonly" is not required to be applied to the declaration of p. 
 This is an important difference between this idea and 'const' from  C++ 
 "readonly char[]" is _not_ a seperate type to "char[]" it's simply 
 achar[] with a readonly flagged to true during compile)


Hmm. What about
 char[] p = f.get();
 char[] q = p;
 q[1] = 'd'; //is 'q' readonly?
or
 char[] p = test ? some_string : f.get();
 p[1] = 'd'; //is 'p' readonly if some_string isn't?
or
 void foo(char[] x){ x[1] = 'd'; }
 char[] p = f.get();
 foo(p);
I think it would need to be carried around as a different type.


 Further, 'in' parameters also require readonly enforcement, eg.

 Foo f = new Foo();
 mutate(f.get());

 void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
 }

 if a function wants to modify a parameter, that parameter should be 'out' 
 or 'inout', meaning, if we re-write mutate:

 Foo f = new Foo();
 mutate(f.get()); //error, f.get() is readonly

 void mutate(inout char[] p) {
   p[0] = 'd'; //no error
 }


I could see explicit 'in' array parameters are readonly. I don't think that 
would cause much pain. It wouldn't have any effect on non-array parameters. 
Making implicit 'in' array parameters readonly would be too much IMHO. It is 
non-trivial making an 'in' parameter into 'inout' since 'inout' requires an 
lvalue while 'in' simply requires an rvalue.


 It's compile time checking, all the compiler need do is flag variables as 
 being readonly, and pass that flag on during assignment (as in the case of 
 "char[] p = f.get();" above). It's not the "const hell" some see in C++.

 With these abilities I believe the requirement for Auto pointers and 
 reference counting is lessened (perhaps removed entirely)

 Regan

 p.s. This has got to be the 10th time I've posted this idea, I'm sorry if 
 I sound like a broken record but I've yet to hear any opposition to it, 
 from anyone, Walter included.
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 07:58:33 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:

 Hmm. What about
  char[] p = f.get();
  char[] q = p;
  q[1] = 'd'; //is 'q' readonly?


Yes. (as I said) The compile time readonly flag is passed on during  
assignment.


 or
  char[] p = test ? some_string : f.get();
  p[1] = 'd'; //is 'p' readonly if some_string isn't?


Ahh, this is the first example (anyones given me) where we require runtime  
checking also. The other half of my idea is that we have runtime flags  
also. They'd be in the same class as array bounds checking, i.e. disabled  
by -release.


 or
  void foo(char[] x){ x[1] = 'd'; }


error 'in' parameters are readonly.


  char[] p = f.get();
  foo(p);



 I think it would need to be carried around as a different type.


Most cases can be handled at compile time without that.

The case above that cannot, why does it need to be a seperate type? I dont  
think it does, it's the same type with a readonly bit set to true (rather  
than false). This bit can be checked at runtime, and disabled by -release.

One goal IMO is to remove the need for us to tell the compiler it's  
readonly when it can clearly tell all by itself (in many cases) and verify  
for itself in others (runtime readonly flag).

So we don't need to use 'const' or 'readonly' everywhere eg.

readonly char[] p = f.get();
char[] s;
p = s; //error incompatible types



 Further, 'in' parameters also require readonly enforcement, eg.

 Foo f = new Foo();
 mutate(f.get());

 void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
 }

 if a function wants to modify a parameter, that parameter should be  
 'out'
 or 'inout', meaning, if we re-write mutate:

 Foo f = new Foo();
 mutate(f.get()); //error, f.get() is readonly

 void mutate(inout char[] p) {
   p[0] = 'd'; //no error
 }


 I could see explicit 'in' array parameters are readonly. I don't think  
 that would cause much pain. It wouldn't have any effect on non-array  
 parameters.
 Making implicit 'in' array parameters readonly would be too much IMHO.


It has to be implicit to work. It's not bug tolerant otherwise.


 It is non-trivial making an 'in' parameter into 'inout' since 'inout'  
 requires an lvalue while 'in' simply requires an rvalue.


I can see how this would be painful for value types, but why is it painful  
for references (can I have an example?).

Regan
Jun 30 2005
next sibling parent reply Brad Beveridge <brad somewhere.net> writes:
Regan Heath wrote:

 On Thu, 30 Jun 2005 07:58:33 -0400, Ben Hinkle <ben.hinkle gmail.com>  
 wrote:
 

 Hmm. What about
  char[] p = f.get();
  char[] q = p;
  q[1] = 'd'; //is 'q' readonly?


 
 
 Yes. (as I said) The compile time readonly flag is passed on during  
 assignment.


IMHO, the answer here should be "no".  I would (I think) prefer that the 
line
char[] p = f.get();

flags an error if f.get() happens to return an immutable char[].  Ie, 
you cannot explicitly assign from an immutable char[] to a char[].  That 
leaves the problem solvable at compile time.  The correct line would be

immutable char[] p = f.get();

Now, for the sake of simplicity I do not think that the compiler should 
do run-time checking, nor should it prevent you from casting 
immutability away.  IMHO all we want/need is a way to prevent accidental 
mistakes & be more descriptive in the way a variable is used.  Yes, 
casting away const is bad - but any language with pointers can 
eventually do it.

Brad
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 08:32:19 -0700, Brad Beveridge <brad somewhere.net>  
wrote:

 Regan Heath wrote:

 On Thu, 30 Jun 2005 07:58:33 -0400, Ben Hinkle <ben.hinkle gmail.com>   
 wrote:


 Hmm. What about
  char[] p = f.get();
  char[] q = p;
  q[1] = 'd'; //is 'q' readonly?


   Yes. (as I said) The compile time readonly flag is passed on during   
 assignment.


 IMHO, the answer here should be "no".  I would (I think) prefer that the  
 line char[] p = f.get();

 flags an error if f.get() happens to return an immutable char[].  Ie,  
 you cannot explicitly assign from an immutable char[] to a char[].


Ahh, but that is half my point, they are not distinct types, nor do they  
need to be.


 That leaves the problem solvable at compile time.  The correct line  
 would be


It is still solvable at compile time. Every char[] has a compile time flag  
called readonly, which is flagged when 'in' or 'readonly' is used, and  
passed on every assignment/slice, and can be checked in cases like the one  
above.


 immutable char[] p = f.get();


I want to avoid this at all costs, otherwise we get:

char[] s;
..s is a temp var used up here..
immutable char[] p = f.get();
s = p[0..2]; //error, p is immuable

in other words, we cannot reuse the variable and we get all these really  
annoying errors which then lead us to cast the immutable away (as is done  
with C++).


 Now, for the sake of simplicity I do not think that the compiler should  
 do run-time checking, nor should it prevent you from casting  
 immutability away.  IMHO all we want/need is a way to prevent accidental  
 mistakes & be more descriptive in the way a variable is used.  Yes,  
 casting away const is bad - but any language with pointers can  
 eventually do it.


If it were implemented how I'm suggesting there would be no need to cast  
it away. If you did you would actually be blatantly and intentionally  
doing something wrong.

Regan
Jun 30 2005
parent reply Brad Beveridge <brad somewhere.net> writes:
I didn't fully understand what you were describing before - I think I do 
now.  My initial thought is that your concept is superior, but may take 
more compiler effort.  I also don't like the idea of extra runtime 
overhead.

Once again though, we need Walter to put forth _something_ about what he 
is thinking before we hash out the details.
Jun 30 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 14:41:57 -0700, Brad Beveridge <brad somewhere.net>  
wrote:

 I didn't fully understand what you were describing before - I think I do  
 now.


Yay :)


 My initial thought is that your concept is superior, but may take more  
 compiler effort.


Indeed, but that's its job IMO ;) More effort once, less effort x times  
where x is the number of people writing in D for the forseeable future.


 I also don't like the idea of extra runtime overhead.


Neither, but I think most of it can be done at compile time, the little  
runtime checking that remains can be disabled with a compile time flag  
after developement and before release.


 Once again though, we need Walter to put forth _something_ about what he  
 is thinking before we hash out the details.


Yep, he holds the deciding vote (the only vote really).

Regan
Jun 30 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 or
  void foo(char[] x){ x[1] = 'd'; }


 error 'in' parameters are readonly.


  char[] p = f.get();
  foo(p);




OK. And I assume the readonly flag gets cleared if passed to an inout or out 
parameter?


 I think it would need to be carried around as a different type.


 Most cases can be handled at compile time without that.


Is there a compile-time mechanism to check for this flag? As a code 
maintainer how can one tell if the flag is set or not?


 The case above that cannot, why does it need to be a seperate type? I dont 
 think it does, it's the same type with a readonly bit set to true (rather 
 than false). This bit can be checked at runtime, and disabled by -release.


I'm not sure how a compile-time bit can be checked at run-time. Where is the 
bit stored during runtime?


 One goal IMO is to remove the need for us to tell the compiler it's 
 readonly when it can clearly tell all by itself (in many cases) and verify 
 for itself in others (runtime readonly flag).


I like the idea that the compiler (and/or dlint) can warn (or possibly error 
if the situation is completely unambiguous) when it thinks a readonly value 
is being modified. I've suggested before that dlint include flow analysis to 
warn about possible COW violations and other stuff like that. Runtime checks 
should be avoided, though.


 So we don't need to use 'const' or 'readonly' everywhere eg.

 readonly char[] p = f.get();
 char[] s;
 p = s; //error incompatible types



 Further, 'in' parameters also require readonly enforcement, eg.

 Foo f = new Foo();
 mutate(f.get());

 void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
 }

 if a function wants to modify a parameter, that parameter should be 
 'out'
 or 'inout', meaning, if we re-write mutate:

 Foo f = new Foo();
 mutate(f.get()); //error, f.get() is readonly

 void mutate(inout char[] p) {
   p[0] = 'd'; //no error
 }


 I could see explicit 'in' array parameters are readonly. I don't think 
 that would cause much pain. It wouldn't have any effect on non-array 
 parameters.
 Making implicit 'in' array parameters readonly would be too much IMHO.


 It has to be implicit to work. It's not bug tolerant otherwise.


This is slightly OT but the phrase "bug tolerant" to me means the language 
tolerates bugs - which sounds like "bugs are ok". Of course I know this 
isn't what you mean by "bug tolerant" (I hope) but it's what I think of 
every time I read it.


 It is non-trivial making an 'in' parameter into 'inout' since 'inout' 
 requires an lvalue while 'in' simply requires an rvalue.


 I can see how this would be painful for value types, but why is it painful 
 for references (can I have an example?).


With
 void foo(inout char[] x) { ... }
 char[] bar() { ... }
it is not legal to write
 foo(bar());
Instead one has to write
 char[] temp = bar();
 foo(temp);
The output of bar is not an lvalue.
Jun 30 2005
next sibling parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
 One goal IMO is to remove the need for us to tell the compiler it's 
 readonly when it can clearly tell all by itself (in many cases) and 
 verify for itself in others (runtime readonly flag).


 I like the idea that the compiler (and/or dlint) can warn (or possibly 
 error if the situation is completely unambiguous) when it thinks a 
 readonly value is being modified. I've suggested before that dlint include 
 flow analysis to warn about possible COW violations and other stuff like 
 that. Runtime checks should be avoided, though.


For example I just hacked up a change to dlint with the following code in 
AssignExp::semantic()
        IdentifierExp* idexp = dynamic_cast<IdentifierExp*>(ae->e1);
        if (idexp != NULL) {
            Dsymbol* dsym = sc->search(idexp->ident,NULL);
            if (dsym != NULL) {
                Declaration* d = dsym->isDeclaration();
                if (d != NULL && d->isParameter() && d->isIn())
                    error("Possible COW violation assigning through input 
parameter");
            }
        }
that will error (I will make that a warning next) if you perform indexed 
assignment on an input parameter. It's very basic and doesn't catch anything 
sneaky at all like
 void foo(char[] s) {
   char[] y = s;
   y[1] = 'd';
 }
but it will catch
 void foo(char[] s) {
   s[1] = 'd';
 }
Jun 30 2005
parent reply Brad Beveridge <brad somewhere.net> writes:
Ben Hinkle wrote:

One goal IMO is to remove the need for us to tell the compiler it's 
readonly when it can clearly tell all by itself (in many cases) and 
verify for itself in others (runtime readonly flag).


I like the idea that the compiler (and/or dlint) can warn (or possibly 
error if the situation is completely unambiguous) when it thinks a 
readonly value is being modified. I've suggested before that dlint include 
flow analysis to warn about possible COW violations and other stuff like 
that. Runtime checks should be avoided, though.


 
 
 For example I just hacked up a change to dlint with the following code in 
 AssignExp::semantic()
         IdentifierExp* idexp = dynamic_cast<IdentifierExp*>(ae->e1);
         if (idexp != NULL) {
             Dsymbol* dsym = sc->search(idexp->ident,NULL);
             if (dsym != NULL) {
                 Declaration* d = dsym->isDeclaration();
                 if (d != NULL && d->isParameter() && d->isIn())
                     error("Possible COW violation assigning through input 
 parameter");
             }
         }
 that will error (I will make that a warning next) if you perform indexed 
 assignment on an input parameter. It's very basic and doesn't catch anything 
 sneaky at all like
  void foo(char[] s) {
    char[] y = s;
    y[1] = 'd';
  }
 but it will catch
  void foo(char[] s) {
    s[1] = 'd';
  }
 
 


But how do you _know_ that is wrong?  I don't see how the compiler/dlint 
can tell the difference between a correct case of changing the input 
array, and a bug.  Which means that for the times that you actually want 
to index into an array, you get dlint grumbling to you for no reason.

Brad
Jun 30 2005
parent "Ben Hinkle" <bhinkle mathworks.com> writes:
"Brad Beveridge" <brad somewhere.net> wrote in message 
news:da1elm$1lp7$1 digitaldaemon.com...

 Ben Hinkle wrote:

One goal IMO is to remove the need for us to tell the compiler it's 
readonly when it can clearly tell all by itself (in many cases) and 
verify for itself in others (runtime readonly flag).


I like the idea that the compiler (and/or dlint) can warn (or possibly 
error if the situation is completely unambiguous) when it thinks a 
readonly value is being modified. I've suggested before that dlint 
include flow analysis to warn about possible COW violations and other 
stuff like that. Runtime checks should be avoided, though.



 For example I just hacked up a change to dlint with the following code in 
 AssignExp::semantic()
         IdentifierExp* idexp = dynamic_cast<IdentifierExp*>(ae->e1);
         if (idexp != NULL) {
             Dsymbol* dsym = sc->search(idexp->ident,NULL);
             if (dsym != NULL) {
                 Declaration* d = dsym->isDeclaration();
                 if (d != NULL && d->isParameter() && d->isIn())
                     error("Possible COW violation assigning through input 
 parameter");
             }
         }
 that will error (I will make that a warning next) if you perform indexed 
 assignment on an input parameter. It's very basic and doesn't catch 
 anything sneaky at all like
  void foo(char[] s) {
    char[] y = s;
    y[1] = 'd';
  }
 but it will catch
  void foo(char[] s) {
    s[1] = 'd';
  }


 But how do you _know_ that is wrong?  I don't see how the compiler/dlint 
 can tell the difference between a correct case of changing the input 
 array, and a bug.  Which means that for the times that you actually want 
 to index into an array, you get dlint grumbling to you for no reason.


Hence it should be a warning. Also note only indexed assignment through an 
'in' parameter will warn. Arbitrary indexing is just fine and inout indexed 
assignment is fine so it won't grumble for most of the cases. Finally, in 
dlint there is a mechanism to tell dlint to squelch any warning on that line 
by putting (dlint) in a comment on the line. So something like
  void foo(char[] s) {
    s[1] = 'd'; //(dlint)
  }
will shut up dlint if you know the assignment is fine.
Jun 30 2005
prev sibling parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 12:54:18 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:

 or
  void foo(char[] x){ x[1] = 'd'; }


 error 'in' parameters are readonly.


  char[] p = f.get();
  foo(p);




 OK. And I assume the readonly flag gets cleared if passed to an inout or  
 out parameter?


As 'out' yes (it's initialised as false), as 'inout' no that would be an  
error (passing readonly as mutable).


 I think it would need to be carried around as a different type.


 Most cases can be handled at compile time without that.


 Is there a compile-time mechanism to check for this flag?As a code
 maintainer how can one tell if the flag is set or not?


That might be useful. I'm not sure whether it's required, after all you'll  
get errors from the compiler if you violate it (that's telling you right  
there).


 The case above that cannot, why does it need to be a seperate type? I  
 dont
 think it does, it's the same type with a readonly bit set to true  
 (rather
 than false). This bit can be checked at runtime, and disabled by  
 -release.


 I'm not sure how a compile-time bit can be checked at run-time. Where is  
 the bit stored during runtime?


If a runtime bit is required, it doesn't have to be the *same* bit as used  
during compile. That is entirely up to the implementors of the compiler.  
It does have to be set based on the compile time bit however. The runtime  
bit would be part of every single object. (yes, I hear you gasp, that's a  
lot of extra bits, which is why it's a design time feature and disabled by  
a flag or -release or whatever)


 One goal IMO is to remove the need for us to tell the compiler it's
 readonly when it can clearly tell all by itself (in many cases) and  
 verify
 for itself in others (runtime readonly flag).


 I like the idea that the compiler (and/or dlint) can warn (or possibly  
 error
 if the situation is completely unambiguous) when it thinks a readonly  
 value
 is being modified. I've suggested before that dlint include flow  
 analysis to
 warn about possible COW violations and other stuff like that. Runtime  
 checks
 should be avoided, though.


I agree, but, if it's DBC system and disabled with -release and/or a  
seperate compiler flag then you can 'avoid' them. I see them as the same  
sort of thing as array bounds checking (tho they differ because readonly  
is permanant data for every object rather than a check that can be done  
when required).


 It has to be implicit to work. It's not bug tolerant otherwise.


 This is slightly OT but the phrase "bug tolerant" to me means the  
 language tolerates bugs - which sounds like "bugs are ok". Of course I  
 know this
 isn't what you mean by "bug tolerant" (I hope) but it's what I think of
 every time I read it.


Ok, time to change my phrase. (I meant "bug tolerant" as in programming in  
the language was bug tolerant, not the language itself was bug tolerant)  
How about: "bug preventative"


 It is non-trivial making an 'in' parameter into 'inout' since 'inout'
 requires an lvalue while 'in' simply requires an rvalue.


 I can see how this would be painful for value types, but why is it  
 painful
 for references (can I have an example?).


 With
  void foo(inout char[] x) { ... }
  char[] bar() { ... }
 it is not legal to write
  foo(bar());


True.


 Instead one has to write
  char[] temp = bar();
  foo(temp);
 The output of bar is not an lvalue.


Ok, I dont really think that is a problem, because:

A. If you have a function:

void foo(char[] x) { x[0] = 's'; }

and someone is passing a rvalue into it, what is the point? I mean,  
they're not going to see the modification on the outside, in which case I  
assume you're using the change internally for something. If so, IMO the  
function should be written:

void foo(char[] x) {
   char[] y = x.dup;
}

In order to be safe and to hold to it's design contract of only reading  
it's inputs.

B. If you have the function:

void foo(char[] x) { x[0] = 's'; }

and the intention is to see the changes on the outside then it needs to be  
rewritten to:

void foo(inout char[] x) { x[0] = 's'; }

and you can't pass an lvalue to that, nor would you want to.

Regan
Jun 30 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 01 Jul 2005 09:47:50 +1200, Regan Heath <regan netwin.co.nz> wrote:

 On Thu, 30 Jun 2005 12:54:18 -0400, Ben Hinkle <ben.hinkle gmail.com>  
 wrote:

 or
  void foo(char[] x){ x[1] = 'd'; }


 error 'in' parameters are readonly.


  char[] p = f.get();
  foo(p);




 OK. And I assume the readonly flag gets cleared if passed to an inout  
 or out parameter?


 As 'out' yes (it's initialised as false), as 'inout' no that would be an  
 error (passing readonly as mutable).


 I think it would need to be carried around as a different type.


 Most cases can be handled at compile time without that.


 Is there a compile-time mechanism to check for this flag?As a code
 maintainer how can one tell if the flag is set or not?


 That might be useful. I'm not sure whether it's required, after all  
 you'll get errors from the compiler if you violate it (that's telling  
 you right there).


 The case above that cannot, why does it need to be a seperate type? I  
 dont
 think it does, it's the same type with a readonly bit set to true  
 (rather
 than false). This bit can be checked at runtime, and disabled by  
 -release.


 I'm not sure how a compile-time bit can be checked at run-time. Where  
 is the bit stored during runtime?


 If a runtime bit is required, it doesn't have to be the *same* bit as  
 used during compile. That is entirely up to the implementors of the  
 compiler. It does have to be set based on the compile time bit however.  
 The runtime bit would be part of every single object. (yes, I hear you  
 gasp, that's a lot of extra bits, which is why it's a design time  
 feature and disabled by a flag or -release or whatever)


 One goal IMO is to remove the need for us to tell the compiler it's
 readonly when it can clearly tell all by itself (in many cases) and  
 verify
 for itself in others (runtime readonly flag).


 I like the idea that the compiler (and/or dlint) can warn (or possibly  
 error
 if the situation is completely unambiguous) when it thinks a readonly  
 value
 is being modified. I've suggested before that dlint include flow  
 analysis to
 warn about possible COW violations and other stuff like that. Runtime  
 checks
 should be avoided, though.


 I agree, but, if it's DBC system and disabled with -release and/or a  
 seperate compiler flag then you can 'avoid' them. I see them as the same  
 sort of thing as array bounds checking (tho they differ because readonly  
 is permanant data for every object rather than a check that can be done  
 when required).


 It has to be implicit to work. It's not bug tolerant otherwise.


 This is slightly OT but the phrase "bug tolerant" to me means the  
 language tolerates bugs - which sounds like "bugs are ok". Of course I  
 know this
 isn't what you mean by "bug tolerant" (I hope) but it's what I think of
 every time I read it.


 Ok, time to change my phrase. (I meant "bug tolerant" as in programming  
 in the language was bug tolerant, not the language itself was bug  
 tolerant) How about: "bug preventative"


 It is non-trivial making an 'in' parameter into 'inout' since 'inout'
 requires an lvalue while 'in' simply requires an rvalue.


 I can see how this would be painful for value types, but why is it  
 painful
 for references (can I have an example?).


 With
  void foo(inout char[] x) { ... }
  char[] bar() { ... }
 it is not legal to write
  foo(bar());


 True.


 Instead one has to write
  char[] temp = bar();
  foo(temp);
 The output of bar is not an lvalue.


 Ok, I dont really think that is a problem, because:

 A. If you have a function:

 void foo(char[] x) { x[0] = 's'; }

 and someone is passing a rvalue into it, what is the point? I mean,  
 they're not going to see the modification on the outside, in which case  
 I assume you're using the change internally for something. If so, IMO  
 the function should be written:

 void foo(char[] x) {
    char[] y = x.dup;
 }

 In order to be safe and to hold to it's design contract of only reading  
 it's inputs.

 B. If you have the function:

 void foo(char[] x) { x[0] = 's'; }

 and the intention is to see the changes on the outside then it needs to  
 be rewritten to:

 void foo(inout char[] x) { x[0] = 's'; }

 and you can't pass an lvalue to that, nor would you want to.


Ooops, meant rvalue here.

Regan
Jun 30 2005
prev sibling parent Kramer <Kramer_member pathlink.com> writes:
Regan (and everyone) please keep posting.  Everytime you post an example, it
just further shows what a useful and necessary feature this is.  So please, keep
the posts coming!

-Kramer

P.S. - Let me just say this.  I enjoy reading these examples and posts because
other people who are smarter and better programmers explain these things in much
clearer ways than I ever could.  Having said that, without even knowing if this
feature existed or not in D, I went looking for const/readonly so I could use
it.  It seems like such a logical thing to have to make really robust software.
It helps the programmer better state what they want their code to do; the
compiler should help.  I'm not a compiler writer and don't know how difficult it
is, but it just seems right to have it.  I would feel better just knowing it's
coming even if we didn't get it for several releases.  But there's been no word
on it.  I think that's the frustrating part...

In article <opss6km00b23k2f5 nrage.netwin.co.nz>, Regan Heath says...

Oh, before you get the impression I think we *need* opAssign... I don't.

It all stems from a desire to protect data and/or be totally sure about  
when to COW and when not to COW. Guessing, which is where we are now is  
inefficient and seems like voodoo magic as opposed to computer science.

We might want to return a slice to some private data, eg.

class Foo {
   private char[] data;
   public char[] get() { return data[0..4]; }
}

Foo f = new Foo();
char[] p = f.get();
p[1] = 'd'; //oops they just nuked it.

So, we're left saying:

class Foo {
   private char[] data;
   public char[] get() { return data[0..4].dup; }
}

which works, but in cases where they didn't write to it, or also thought  
"I should dup this before I write to it" (COW) we have a wasted an  
allocation or 2.

The best soln I've read here / thought of is to enforce some sort of  
"readonly" (be it the oft maligned "const" or some other keyword) the  
result would be a *compile* time enforcement of "readonly", eg.

class Foo {
   private char[] data;
   public readonly char[] get() { return data[0..4]; }
}

Foo f = new Foo();
char[] p = f.get();
p[1] = 'd'; //error 'p' is readonly

(note, "readonly" is not required to be applied to the declaration of p.  
This is an important difference between this idea and 'const' from  
C++ "readonly char[]" is _not_ a seperate type to "char[]" it's simply  
achar[] with a readonly flagged to true during compile)

Further, 'in' parameters also require readonly enforcement, eg.

Foo f = new Foo();
mutate(f.get());

void mutate(char[] p) {
   p[0] = 'd'; //error p is in/readonly
}

if a function wants to modify a parameter, that parameter should be 'out'  
or 'inout', meaning, if we re-write mutate:

Foo f = new Foo();
mutate(f.get()); //error, f.get() is readonly

void mutate(inout char[] p) {
   p[0] = 'd'; //no error
}

It's compile time checking, all the compiler need do is flag variables as  
being readonly, and pass that flag on during assignment (as in the case of  
"char[] p = f.get();" above). It's not the "const hell" some see in C++.

With these abilities I believe the requirement for Auto pointers and  
reference counting is lessened (perhaps removed entirely)

Regan

p.s. This has got to be the 10th time I've posted this idea, I'm sorry if  
I sound like a broken record but I've yet to hear any opposition to it,  
 from anyone, Walter included.

On Thu, 30 Jun 2005 22:58:44 +1200, Regan Heath <regan netwin.co.nz> wrote:


 On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
 wrote:

 Regan Heath wrote:

 I got little or no comment on this idea/code I posted to another  
 (rather  large) thread here, I figured I'd post it again and see what  
 people  thought.
  After reading this thread (the one I mention above) and also this  
 one  Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html
  It appears you need a type that:
  1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.


 You can give a struct or class a method called opAssign if you want.  
 Whether

      qwert = yuiop;

 will act as syntactic sugar for it is another matter.


 True, but not the problem.. read on :)


 It seems (from recent posts) that Walter plans to put auto classes on  
 the  stack, so they'd then fulfil requirements 1 and 2.
  But what about 3? To me, opAssign isn't generally a good idea for  
 classes  (assigning a value to a reference seems illogical) but would  
 be a good for  structs (they're value types, assignment assigns a  
 value).
  Can you do reference counting without opAssign?


 <snip>

 Yes.


 Then please enlighten me. I've tried, it 'almost' works but it suffers  
 the fatal flaw shown below.


 Of the claims by various people that we "need" overloading of the  
 assignment operator, I'm yet to see one that's true. AFAIC if you want  
 something to the effect of custom assignment behaviour, you can create  
 your own method with whatever name that does it.  Can anyone supply an  
 example, let alone a real-world one, that doesn't work unless the  
 aforementioned syntactic sugar is available?


 The problem, as I see it, is that it's not just syntactic sugar. You  
 cannot prevent reference assignment in cases where the idea/technique  
 requires it. As in these 2 cases. (reference counting, auto pointer)

 I've tried to write a reference counting implementation and also an auto  
 pointer implementation using the recommended practices found on the D  
 website. Both work, both use a "copy constructor" instead of opAssign  
 (as suggested by the D spec/website). The problem is this:

 auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p  
 owns object. (or "pwns" for those of you..)
 auto AutoPtr s;

 s = new AutoPtr(p); //no problem, the copy constructor transfers  
 ownership.
 s = p; //problem, s now refers to p, both 'own' "object".

 The same problem occurs with the reference counting implementation. s  
 becomes a 2nd reference to object but the reference count does not  
 increase.

 We can 'ask' people not to use a straight reference assignment but we  
 cannot enforce it.

 Logically I agree, opAssign makes no sense for reference types, you  
 assign references to reference types *not* values. opAssign would  
 effectively change a reference assignment into a value assignment in a  
 non obvious way. I agree with all this reasoning, however, the problem  
 remains.

 Regan
Jun 30 2005
prev sibling parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Regan Heath wrote:

 On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon 
 <smjg_1998 yahoo.com>  wrote:


<snip>

 Of the claims by various people that we "need" overloading of the 
 assignment operator, I'm yet to see one that's true. AFAIC if you 
 want something to the effect of custom assignment behaviour, you can 
 create your own method with whatever name that does it.  Can anyone 
 supply an example, let alone a real-world one, that doesn't work 
 unless the aforementioned syntactic sugar is available?


 
 The problem, as I see it, is that it's not just syntactic sugar. You 
 cannot prevent reference assignment in cases where the idea/technique 
 requires it. As in these 2 cases. (reference counting, auto pointer)


I guess that if you use either of these, you're supposed to know what 
you're doing and that you have to be careful.  Remember that because of 
the way D is designed, most applications will be using the built-in GC 
rather than this.  As such, a facility for disabling the assignment 
operator for select types would probably be seen as an unnecessary 
complexity.


 I've tried to write a reference counting implementation and also an 
 auto pointer implementation using the recommended practices found on 
 the D website. Both work, both use a "copy constructor" instead of 
 opAssign (as suggested by the D spec/website). The problem is this:
 
 auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p 
 owns  object. (or "pwns" for those of you..)
 auto AutoPtr s;
 
 s = new AutoPtr(p); //no problem, the copy constructor transfers ownership.
 s = p; //problem, s now refers to p, both 'own' "object".


http://www.digitalmars.com/d/attribute.html#auto
"Assignment to an auto, other than initialization, is not allowed."

I just had a go at it with GDC 0.10.  It appears from the destruction 
order that if an auto variable is initialised to another, then the new 
reference becomes the owning one.  However, there seems to be a bug 
whereby it accepts assignments after initialisation; however, doing this 
doesn't seem to have any effect on what gets destructed when.

Here's the program I just tried:

----------
import std.stream;
import std.stdio;

class Thing {
     int value;
     this(int v) { value = v; }
     ~this() {
         writefln("destructed %d", value);
     }
}

void main() {
     auto Thing o = new Thing(1);
     auto Thing p = new Thing(2);
	
     char[] line = std.stream.stdin.readLine();
     auto Thing q = line[0] == 'y' ? o : new Thing(3);
		
     if (line[1] == 'y') o = q;
     writefln(q.value);
     writefln(o.value);
}
----------


 The same problem occurs with the reference counting implementation. s 
 becomes a 2nd reference to object but the reference count does not 
 increase.
 
 We can 'ask' people not to use a straight reference assignment but we  
 cannot enforce it.


<snip>

True.  But I guess what we really need is a clear spec of what happens 
if one auto is assigned to another.  If only by some means (not sure how 
easy or not it would be to implement) the auto stayed with the original 
owner, then such reference assignment could be OK (and indeed more 
efficient) as long as it isn't going to outlive the original.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 15:20:46 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 Regan Heath wrote:

 On Thu, 30 Jun 2005 11:19:04 +0100, Stewart Gordon  
 <smjg_1998 yahoo.com>  wrote:


 <snip>

 Of the claims by various people that we "need" overloading of the  
 assignment operator, I'm yet to see one that's true. AFAIC if you want  
 something to the effect of custom assignment behaviour, you can create  
 your own method with whatever name that does it.  Can anyone supply an  
 example, let alone a real-world one, that doesn't work unless the  
 aforementioned syntactic sugar is available?


  The problem, as I see it, is that it's not just syntactic sugar. You  
 cannot prevent reference assignment in cases where the idea/technique  
 requires it. As in these 2 cases. (reference counting, auto pointer)


 I guess that if you use either of these, you're supposed to know what  
 you're doing and that you have to be careful.  Remember that because of  
 the way D is designed, most applications will be using the built-in GC  
 rather than this.  As such, a facility for disabling the assignment  
 operator for select types would probably be seen as an unnecessary  
 complexity.


Just because you're being clever doesn't mean you can't use all the help  
you can get. Being able to write an idiot proof reference counting  
implementation would be an enormous boon for everybody. I don't care how  
it's done (to a degree), I'm not sure disabling assignment is the solution.


 I've tried to write a reference counting implementation and also an  
 auto pointer implementation using the recommended practices found on  
 the D website. Both work, both use a "copy constructor" instead of  
 opAssign (as suggested by the D spec/website). The problem is this:
  auto AutoPtr p = new AutoPtr(object); //create ptr passing object, p  
 owns  object. (or "pwns" for those of you..)
 auto AutoPtr s;
  s = new AutoPtr(p); //no problem, the copy constructor transfers  
 ownership.
 s = p; //problem, s now refers to p, both 'own' "object".


 http://www.digitalmars.com/d/attribute.html#auto
 "Assignment to an auto, other than initialization, is not allowed."

 I just had a go at it with GDC 0.10.  It appears from the destruction  
 order that if an auto variable is initialised to another, then the new  
 reference becomes the owning one.  However, there seems to be a bug  
 whereby it accepts assignments after initialisation; however, doing this  
 doesn't seem to have any effect on what gets destructed when.


Indeed, it's broken then. I wasn't aware of the rule:
   "Assignment to an auto, other than initialization, is not allowed."

in fact, I suggested it after trying to write the implementation in  
response to Andrew pointing out what I just showed you. Funny that.


 Here's the program I just tried:

 ----------
 import std.stream;
 import std.stdio;

 class Thing {
      int value;
      this(int v) { value = v; }
      ~this() {
          writefln("destructed %d", value);
      }
 }

 void main() {
      auto Thing o = new Thing(1);
      auto Thing p = new Thing(2);
 	
      char[] line = std.stream.stdin.readLine();
      auto Thing q = line[0] == 'y' ? o : new Thing(3);
 		
      if (line[1] == 'y') o = q;
      writefln(q.value);
      writefln(o.value);
 }
 ----------


 The same problem occurs with the reference counting implementation. s  
 becomes a 2nd reference to object but the reference count does not  
 increase.
  We can 'ask' people not to use a straight reference assignment but we   
 cannot enforce it.


 <snip>

 True.  But I guess what we really need is a clear spec of what happens  
 if one auto is assigned to another.


That is step 1.


 If only by some means (not sure how easy or not it would be to  
 implement) the auto stayed with the original owner, then such reference  
 assignment could be OK (and indeed more efficient) as long as it isn't  
 going to outlive the original.


I think having 2 references to a reference counter without incrementing  
the counter is still illegal and should be prevented. Same goes for an  
auto pointer really.

Preventing the assignment of an 'auto' and preventing assigning an auto to  
another reference would solve the problem, from what I can see.

Regan
Jun 30 2005
parent reply Stewart Gordon <smjg_1998 yahoo.com> writes:
Regan Heath wrote:

 On Thu, 30 Jun 2005 15:20:46 +0100, Stewart Gordon 
 <smjg_1998 yahoo.com>  wrote:


<snip>

 I guess that if you use either of these, you're supposed to know what  
 you're doing and that you have to be careful.  Remember that because 
 of the way D is designed, most applications will be using the 
 built-in GC  rather than this.  As such, a facility for disabling the 
 assignment operator for select types would probably be seen as an 
 unnecessary complexity.


 
 Just because you're being clever doesn't mean you can't use all the 
 help you can get.


True.  But there's certainly a limit to the extent to which language 
features to protect you during with uncommon D&D tasks are worth it.


 Being able to write an idiot proof reference 
 counting implementation would be an enormous boon for everybody.


Everybody?  Even those of us who are perfectly happy with D's own GC?


 I don't care how it's done (to a degree), I'm not sure disabling 
 assignment is the solution.


That makes a change.


 If only by some means (not sure how easy or not it would be to 
 implement) the auto stayed with the original owner, then such 
 reference assignment could be OK (and indeed more efficient) as long 
 as it isn't going to outlive the original.


 
 I think having 2 references to a reference counter without incrementing 
 the counter is still illegal and should be prevented. Same goes for an 
 auto pointer really.


I refer you back to this:

http://www.digitalmars.com/d/garbage.html

"Reference counting is a common solution to solve explicit memory 
allocation problems. The code to implement the increment and decrement 
operations whenever assignments are made is one source of slowdown. 
Hiding it behind smart pointer classes doesn't help the speed."

So you're saying it should be forbidden to partly get around this issue 
by circumventing the reference count even when you know it's safe?


 Preventing the assignment of an 'auto' and preventing assigning an auto 
 to another reference would solve the problem, from what I can see.


Up to a point.  But autos can be passed into functions, which generally 
don't know that they're dealing with auto objects and so can arbitrarily 
create more references to them.  But arguably this is a good thing, as 
such ability is useful for Windows GDI stuff and the like.

Stewart.

-- 
My e-mail is valid but not my primary mailbox.  Please keep replies on 
the 'group where everyone may benefit.
Jul 01 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Fri, 01 Jul 2005 11:23:44 +0100, Stewart Gordon <smjg_1998 yahoo.com>  
wrote:

 Being able to write an idiot proof reference counting implementation  
 would be an enormous boon for everybody.


 Everybody?  Even those of us who are perfectly happy with D's own GC?


Reference counting is not an "alternative" to the GC, it's an "additional"  
tool. I can imagine using the GC for 80-90% of my memory management and  
reference counting for those remaining situations where it is beneficial.


 I don't care how it's done (to a degree), I'm not sure disabling  
 assignment is the solution.


 That makes a change.


?


 If only by some means (not sure how easy or not it would be to  
 implement) the auto stayed with the original owner, then such  
 reference assignment could be OK (and indeed more efficient) as long  
 as it isn't going to outlive the original.


  I think having 2 references to a reference counter without  
 incrementing the counter is still illegal and should be prevented. Same  
 goes for an auto pointer really.


 I refer you back to this:

 http://www.digitalmars.com/d/garbage.html

 "Reference counting is a common solution to solve explicit memory  
 allocation problems. The code to implement the increment and decrement  
 operations whenever assignments are made is one source of slowdown.  
 Hiding it behind smart pointer classes doesn't help the speed."


True.


 So you're saying it should be forbidden to partly get around this issue  
 by circumventing the reference count even when you know it's safe?


As long as the new reference does not outlive the original it's probably  
safe, but that sort of thing is not the general or common usage case.  
Meaning, ideally, it shouldn't be easy to do by accident. Currently it is:

AutoPtr p = new AutoPtr(object);
AutoPtr s;
s = p; //oops


 Preventing the assignment of an 'auto' and preventing assigning an auto  
 to another reference would solve the problem, from what I can see.


 Up to a point.  But autos can be passed into functions, which generally  
 don't know that they're dealing with auto objects and so can arbitrarily  
 create more references to them.  But arguably this is a good thing, as  
 such ability is useful for Windows GDI stuff and the like.


But again, not a general or comman usage case. So ideally, shouldn't be  
easy to do accidentally. It sounds like it is currently.

Regan
Jul 01 2005
prev sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss0boj0r23k2f5 nrage.netwin.co.nz...

I got little or no comment on this idea/code I posted to another (rather 
large) thread here, I figured I'd post it again and see what people 
thought.

 After reading this thread (the one I mention above) and also this one 
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on the 
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for classes 
 (assigning a value to a reference seems illogical) but would be a good for 
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?


See the section in the Memory Management doc about reference counting. It 
says the add/release must be explicit. Perhaps Walter could expound on that 
design decision somewhere to address people's concerns. Personally I have no 
beef with making non-trivial memory mangement explicit since it should be 
rarely needed. And in those cases having it explicit will alert code 
maintainers to what is going on. Also it avoids having to add special "give 
me a reference without manipulating the count for performance" issue. On the 
other hand by making it explicit one has to advertise the fact it uses fancy 
memory management so that users know to increment and decrement the count. 
Currently D errs on the side of simplicity and making code explicit - which 
is a fairly common tactic in D in general.
Jun 30 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 07:38:01 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...

 I got little or no comment on this idea/code I posted to another (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on  
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for  
 classes
 (assigning a value to a reference seems illogical) but would be a good  
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?


 See the section in the Memory Management doc about reference counting. It
 says the add/release must be explicit.


Indeed. This strikes me as being rather bug intolerant i.e. open for  
mistakes and bugs to creep in.
I am fine with it being this explicit:

RefCnt r = new RefCnt(object); //explicit
RefCnt s;
s = new RefCent(r); //explicit

However the problem is actually this:

s = r; //needs to be disallowed or to inc refcnt.


 Perhaps Walter could expound on that design decision somewhere to  
 address people's concerns.


Yes please.


 Personally I have no beef with making non-trivial memory mangement  
 explicit since it should be
 rarely needed. And in those cases having it explicit will alert code
 maintainers to what is going on.


I agree.


 Also it avoids having to add special "give me a reference without  
 manipulating the count forperformance" issue.


Ah, but that is the problem.


 On the other hand by making it explicit one has to advertise the fact it  
 uses fancy memory management so that users know to increment and  
 decrement the count.


So, you're saying it should be done:

RefCnt r = new RefCnt(object);
r.inc(); //perhaps optional?;

RefCnt s = r;
s.inc();

the room for error here is rather larger than I'd like.


 Currently D errs on the side of simplicity and making code explicit -  
 which is a fairly common tactic in D in general.


I am a big believer in simple code and making odd things explicit. What I  
want in this case is something simple, that is bug tolerant. I don't mind  
if it can be abused/broken with intent, it just needs to be accident(bug)  
tolerant.

Regan
Jun 30 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opss6mi5iy23k2f5 nrage.netwin.co.nz...

 On Thu, 30 Jun 2005 07:38:01 -0400, Ben Hinkle <ben.hinkle gmail.com> 
 wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...

 I got little or no comment on this idea/code I posted to another (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on 
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for 
 classes
 (assigning a value to a reference seems illogical) but would be a good 
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?


 See the section in the Memory Management doc about reference counting. It
 says the add/release must be explicit.


 Indeed. This strikes me as being rather bug intolerant i.e. open for 
 mistakes and bugs to creep in.
 I am fine with it being this explicit:

 RefCnt r = new RefCnt(object); //explicit
 RefCnt s;
 s = new RefCent(r); //explicit

 However the problem is actually this:

 s = r; //needs to be disallowed or to inc refcnt.


People will want to be able to get a reference without changing the ref 
count and = does that. There are advantages and disadvantages to using =. To 
me the main advantage is that = does the right thing and matches the user's 
expectations of what = does and the main disadvantage is that people who 
don't know they need to care about ref counting will just use = and get into 
trouble. To me the advantages outweigh the disadvantages.


 Perhaps Walter could expound on that design decision somewhere to 
 address people's concerns.


 Yes please.


 Personally I have no beef with making non-trivial memory mangement 
 explicit since it should be
 rarely needed. And in those cases having it explicit will alert code
 maintainers to what is going on.


 I agree.


 Also it avoids having to add special "give me a reference without 
 manipulating the count forperformance" issue.


 Ah, but that is the problem.


I'm not sure what you mean. I don't think it's unusual to have a 
ref-counting API that has a way to get a reference without changing the 
count.


 On the other hand by making it explicit one has to advertise the fact it 
 uses fancy memory management so that users know to increment and 
 decrement the count.


 So, you're saying it should be done:

 RefCnt r = new RefCnt(object);
 r.inc(); //perhaps optional?;

 RefCnt s = r;
 s.inc();

 the room for error here is rather larger than I'd like.


Actually I wouldn't even bother having a separate RefCnt class. The ref 
counting can be mixed into the class of object, much like it is for COM 
objects. So for example
  Foo f = new Foo(); // creates with refcount of 1
  Foo g = f.incRef; // inc ref count and return this
  g.decRef; // dec ref count


 Currently D errs on the side of simplicity and making code explicit - 
 which is a fairly common tactic in D in general.


 I am a big believer in simple code and making odd things explicit. What I 
 want in this case is something simple, that is bug tolerant. I don't mind 
 if it can be abused/broken with intent, it just needs to be accident(bug) 
 tolerant.


I agree but I think in this case there are a number of factors that make the 
decision non-trivial. Sometimes it's best to keep everything simple and 
allow people to shoot themselves in the foot instead of making things 
complicated to prevent them from shooting themselves in the foot. The 
weights one gives to all the factors can tip the design in one way or 
another. The current situation is a reasonable set of compromises IMHO. 
Adding opAssign would have an impact beyond ref-counting so it's a pretty 
big step to take.
Jun 30 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 30 Jun 2005 08:37:56 -0400, Ben Hinkle <ben.hinkle gmail.com>  
wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss6mi5iy23k2f5 nrage.netwin.co.nz...

 On Thu, 30 Jun 2005 07:38:01 -0400, Ben Hinkle <ben.hinkle gmail.com>
 wrote:

 "Regan Heath" <regan netwin.co.nz> wrote in message
 news:opss0boj0r23k2f5 nrage.netwin.co.nz...

 I got little or no comment on this idea/code I posted to another  
 (rather
 large) thread here, I figured I'd post it again and see what people
 thought.

 After reading this thread (the one I mention above) and also this one
 Andrew dug up:
   http://www.digitalmars.com/d/archives/7988.html

 It appears you need a type that:

 1 - is stack based.
 2 - has deterministic destruction.
 3 - has opAssign.

 It seems (from recent posts) that Walter plans to put auto classes on
 the
 stack, so they'd then fulfil requirements 1 and 2.

 But what about 3? To me, opAssign isn't generally a good idea for
 classes
 (assigning a value to a reference seems illogical) but would be a good
 for
 structs (they're value types, assignment assigns a value).

 Can you do reference counting without opAssign?


 See the section in the Memory Management doc about reference counting.  
 It
 says the add/release must be explicit.


 Indeed. This strikes me as being rather bug intolerant i.e. open for
 mistakes and bugs to creep in.
 I am fine with it being this explicit:

 RefCnt r = new RefCnt(object); //explicit
 RefCnt s;
 s = new RefCent(r); //explicit

 However the problem is actually this:

 s = r; //needs to be disallowed or to inc refcnt.


 People will want to be able to get a reference
 without changing the ref
 count and = does that. There are advantages and disadvantages to using  
 =. To
 me the main advantage is that = does the right thing and matches the  
 user's
 expectations of what = does and the main disadvantage is that people who
 don't know they need to care about ref counting will just use = and get  
 into
 trouble. To me the advantages outweigh the disadvantages.


Do you mean get a reference to the "reference counting object", or a  
reference to the "object being counted"?

= gives the former, a reference to the "reference counting object".

Getting a reference the the "reference counting object" without  
incrementing the reference count is not dangerous, provided the new  
reference does not outlive the original (as the object could be free'd  
then). I can see how you might want one in order to log something or  
whatever, but the general case you want to increment the count. that  
suggests to me that the generally expected behaviour is to increment the  
count, and the specialist behaviour is not to, in which case the obvious  
thing = should do the general case.


 Perhaps Walter could expound on that design decision somewhere to
 address people's concerns.


 Yes please.


 Personally I have no beef with making non-trivial memory mangement
 explicit since it should be
 rarely needed. And in those cases having it explicit will alert code
 maintainers to what is going on.


 I agree.


 Also it avoids having to add special "give me a reference without
 manipulating the count forperformance" issue.


 Ah, but that is the problem.


 I'm not sure what you mean. I don't think it's unusual to have a
 ref-counting API that has a way to get a reference without changing the
 count.


See above.


 On the other hand by making it explicit one has to advertise the fact  
 it
 uses fancy memory management so that users know to increment and
 decrement the count.


 So, you're saying it should be done:

 RefCnt r = new RefCnt(object);
 r.inc(); //perhaps optional?;

 RefCnt s = r;
 s.inc();

 the room for error here is rather larger than I'd like.


 Actually I wouldn't even bother having a separate RefCnt class. The ref
 counting can be mixed into the class of object, much like it is for COM
 objects. So for example
   Foo f = new Foo(); // creates with refcount of 1
   Foo g = f.incRef; // inc ref count and return this
   g.decRef; // dec ref count


The point of a seperate object is that it does this leg work for you.  
Meaning, you cannot get it wrong, meaning it's more "bug preventative",  
robust, and in the end simply easier to use.


 Currently D errs on the side of simplicity and making code explicit -
 which is a fairly common tactic in D in general.


 I am a big believer in simple code and making odd things explicit. What  
 I
 want in this case is something simple, that is bug tolerant. I don't  
 mind
 if it can be abused/broken with intent, it just needs to be  
 accident(bug)
 tolerant.


 I agree but I think in this case there are a number of factors that make  
 the decision non-trivial. Sometimes it's best to keep everything simple  
 and
 allow people to shoot themselves in the foot instead of making things
 complicated to prevent them from shooting themselves in the foot. The
 weights one gives to all the factors can tip the design in one way or
 another. The current situation is a reasonable set of compromises IMHO.
 Adding opAssign would have an impact beyond ref-counting so it's a pretty
 big step to take.


I'm not asking for opAssign!! I'm asking for a solution to the problem,  
everyone assumes the only solution is opAssign (which might be because it  
is, I don't know).

I'm saying, here is the problem, find me a solution (because I've  
exhausted my abilities trying). I'm starting to understand Andrew's  
frustration. Everyone here seems incapable of exploring the landscape for  
a solution, they'd all much rather sit in their dirt huts and bandage  
their bleeding foot. (just ignore me here, this is frustration talking..)

Regan
Jun 30 2005