www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - scope class members -> in-situ

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I think this has been discussed in this group already.

An object storing another object needs two allocations:

class A { ... }
class B {
    A a;
    this() {
       a = new A;
    }
}

auto b = new B; // two allocations

I'm thinking of using "scope" in this situation to imply in-situ storage:

class B {
    scope A a;
    this() {
       a = new A;
    }
}

Now the A member actually lies inside of B - no more indirection. That 
means the constructor needs special scrutiny, in particular a cannot be 
null because that wouldn't make much sense.

What do you think?


Andrei
Oct 02 2009
next sibling parent reply Bill Baxter <wbaxter gmail.com> writes:
On Fri, Oct 2, 2009 at 8:33 AM, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org> wrote:

 I think this has been discussed in this group already.

 An object storing another object needs two allocations:

 class A { ... }
 class B {
 =A0 A a;
 =A0 this() {
 =A0 =A0 =A0a =3D new A;
 =A0 }
 }

 auto b =3D new B; // two allocations

 I'm thinking of using "scope" in this situation to imply in-situ storage:

 class B {
 =A0 scope A a;
 =A0 this() {
 =A0 =A0 =A0a =3D new A;
 =A0 }
 }

 Now the A member actually lies inside of B - no more indirection. That me=


ans

 the constructor needs special scrutiny, in particular a cannot be null
 because that wouldn't make much sense.

 What do you think?


I think it would be nice, but there are enough issues that I'm not
convinced that saving a few allocations is worth it.

Mainly what happens if you do   someB.a =3D someOtherA  later on?

The answer to that might be different than how you treat  someB.a =3D new A=
().

The latter could be converted into a placement new.   But the former
has to still point to the original someOtherA to maintain proper
reference semantics.

You can sidestep these issues by saying that a scope A in a class is
not rebindable.  But that no doubt cuts out some useful cases.

Another option would be to make "scope A" reserve space for both a
pointer to an A and the memory for it.  Then the someB.a is just a
regular A reference that can be rebound like any other, or made null
if desired.  It would just leave orphaned memory sitting there if
rebound to point to some other A.   Or it could use placement
construction if you assign a new A to it.   (but not if you say
someB.a =3D new DerivedFromA()).


--bb
Oct 02 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Bill Baxter wrote:

 On Fri, Oct 2, 2009 at 8:33 AM, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org> wrote:

 I think this has been discussed in this group already.

 An object storing another object needs two allocations:

 class A { ... }
 class B {
   A a;
   this() {
      a = new A;
   }
 }

 auto b = new B; // two allocations

 I'm thinking of using "scope" in this situation to imply in-situ storage:

 class B {
   scope A a;
   this() {
      a = new A;
   }
 }

 Now the A member actually lies inside of B - no more indirection. That means
 the constructor needs special scrutiny, in particular a cannot be null
 because that wouldn't make much sense.

 What do you think?


 
 I think it would be nice, but there are enough issues that I'm not
 convinced that saving a few allocations is worth it.
 
 Mainly what happens if you do   someB.a = someOtherA  later on?
 
 The answer to that might be different than how you treat  someB.a = new A().
 
 The latter could be converted into a placement new.   But the former
 has to still point to the original someOtherA to maintain proper
 reference semantics.
 
 You can sidestep these issues by saying that a scope A in a class is
 not rebindable.  But that no doubt cuts out some useful cases.


Yah, the idea was to make it not rebindable. Initially I even thought of 
using "final" instead of "scope".


 Another option would be to make "scope A" reserve space for both a
 pointer to an A and the memory for it.  Then the someB.a is just a
 regular A reference that can be rebound like any other, or made null
 if desired.  It would just leave orphaned memory sitting there if
 rebound to point to some other A.   Or it could use placement
 construction if you assign a new A to it.   (but not if you say
 someB.a = new DerivedFromA()).


Having a reference plus in-situ storage also crossed my mind, but I 
think it would serve corner cases that could best be served by either 
using two allocations, or having the user herself define one reference + 
one scope object.


Andrei
Oct 02 2009
prev sibling parent bearophile <bearophileHUGS lycos.com> writes:
Bill Baxter:

 Or it could use placement
 construction if you assign a new A to it.   (but not if you say
 someB.a = new DerivedFromA()).


I hope Andrei's book will explain how (and where) to use placement construction
for classes in D.

Bye,
bearophile
Oct 02 2009
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:


 class B {
     scope A a;
     this() {
        a = new A;
     }
 }
 
 Now the A member actually lies inside of B - no more indirection. That 
 means the constructor needs special scrutiny, in particular a cannot be 
 null because that wouldn't make much sense.
 What do you think?


I have recently discussed about this. One idea is to keep the compiler simpler,
keeping such scoped class as possible similar to normal class. So you keep the
indirection. So B keeps inside itself the memory needed to keep an A plus a
reference to A itself. This wastes a little memory for the reference and keeps
the indirection, but allows to keep the semantics of A almost unchanged,
because you can always do reassign "a" to another instance of A (an instance
that can allocated on the heap too).

In such situation the main and maybe only thing you have to keep care of is the
deallocation of "a", that has to not happen (well, you can call its destructor,
but you can't actually free its memory), because even if the GC has declared it
as dead (because of a "delete" or because "a" has being reassigned to something
else), it can't be deallocated until the instance of B is deallocated.

There are other ways to implement this, but I think they require more
special-casing and more changes to the frontend and probably a little more
complexity.

Bye,
bearophile
Oct 02 2009
prev sibling next sibling parent reply Daniel Keep <daniel.keep.lists gmail.com> writes:
Andrei Alexandrescu wrote:

 I think this has been discussed in this group already.


*Think*?!

http://www.digitalmars.com/d/archives/digitalmars/D/scope_inline_optimizations_scoped_attributes_95025.html

http://www.digitalmars.com/d/archives/digitalmars/D/D2_s_feature_set_91823.html

http://www.digitalmars.com/d/archives/digitalmars/D/important_proposal_scope_keyword_for_class_members_85524.html

http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_write_a_proper_class_destructor_6113.html#N6120

http://www.digitalmars.com/d/archives/digitalmars/D/38329.html

And I'm certain I've missed quite a few.



I know history is cyclical, but this is getting ridiculous.
Oct 02 2009
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Daniel Keep wrote:

 
 Andrei Alexandrescu wrote:

 I think this has been discussed in this group already.


 
 *Think*?!
 
 http://www.digitalmars.com/d/archives/digitalmars/D/scope_inline_optimizations_scoped_attributes_95025.html
 
 http://www.digitalmars.com/d/archives/digitalmars/D/D2_s_feature_set_91823.html
 
 http://www.digitalmars.com/d/archives/digitalmars/D/important_proposal_scope_keyword_for_class_members_85524.html
 
 http://www.digitalmars.com/d/archives/digitalmars/D/learn/How_to_write_a_proper_class_destructor_6113.html#N6120
 
 http://www.digitalmars.com/d/archives/digitalmars/D/38329.html
 
 And I'm certain I've missed quite a few.
 
 
 
 I know history is cyclical, but this is getting ridiculous.


Well "think" does qualify doesn't it :o).

Andrei
Oct 02 2009
prev sibling next sibling parent reply Tom S <h3r3tic remove.mat.uni.torun.pl> writes:
Andrei Alexandrescu wrote:

 I think this has been discussed in this group already.
 
 An object storing another object needs two allocations:
 
 class A { ... }
 class B {
    A a;
    this() {
       a = new A;
    }
 }
 
 auto b = new B; // two allocations
 
 I'm thinking of using "scope" in this situation to imply in-situ storage:
 
 class B {
    scope A a;
    this() {
       a = new A;
    }
 }
 
 Now the A member actually lies inside of B - no more indirection. That 
 means the constructor needs special scrutiny, in particular a cannot be 
 null because that wouldn't make much sense.
 
 What do you think?


I think it should be done in userland, not built-in. Here's a 
proof-of-concept implementation:

----
import std.stdio;


string scopeInstance(string type, string name) {
	return `
	private byte[__traits(classInstanceSize, `~type~`)] 
_scopeInstance__`~name~`;

	static this() {
		const int off = _scopeInstance__`~name~`.offsetof;
		const int len = __traits(classInstanceSize, `~type~`);
		typeof(this).classinfo.init[off .. off + len] = `~type~`.classinfo.init;
	}

	`~type~` `~name~`() {
		return cast(`~type~`)_scopeInstance__`~name~`.ptr;
	}

	void `~name~`(`~type~` f) {
		_scopeInstance__`~name~`[] = 
(cast(byte*)f)[0.._scopeInstance__`~name~`.sizeof];
	}`;
}

class Foo {
	int x;
	float y;
	string zomg = "zomg";

	this(int x, float y) {
		writeln("making a Foo");
		this.x = x;
		this.y = y;
	}

	~this() {
		writeln("death-tracting a Foo");
	}
}

class Bar {
	string x;

	this (string x) {
		writeln("making a Bar");
		this.x = x;
	}
}

class Baz {
	mixin(scopeInstance("Foo", "foo"));
	mixin(scopeInstance("Bar", "bar"));

	this() {
		writeln("making a Baz");
		foo.__ctor(1, 3.14f);
		bar.__ctor("ohai");
	}

	~this() {
		writeln("death-tracting a Baz");
		foo.__dtor();
	}
}


void main() {
	scope b = new Baz;
	writeln(b.foo.x, " ", b.foo.y, " ", b.foo.zomg);
	writeln(b.bar.x);
	writeln("Foo size: ", __traits(classInstanceSize, Foo));
	writeln("Bar size: ", __traits(classInstanceSize, Bar));
	writeln("Baz size: ", __traits(classInstanceSize, Baz));
}
----

Result:
----
making a Baz
making a Foo
making a Bar
1 3.14 zomg
ohai
Foo size: 24
Bar size: 16
Baz size: 48
death-tracting a Baz
death-tracting a Foo
----

Now for a brief summary:
* I couldn't get hold of the initializer of a class at compile-time. 
I've tried using symbol.mangleof ~ "6__initZ", but DMD told me it could 
not be an initializer for a static array. This forced me to initialize 
the 'inline' / 'scope' class instances in a static ctor. I'm not sure if 
this is legal - can ClassInfo ever wind up to live in ROM? Perhaps 
__traits(classInitializer, _) could be added.
* The long proposed __ident extension would allow turning the string 
mixin into a regular template mixin.
* __ctor should be standardized (or is it?)

I'm not very good at D2, so perhaps there are better ways to implement 
it by now. Here's a similar proposal in pseudo-D2-code: 
http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digitalmars.D&artnum=85684


-- 
Tomasz Stachowiak
http://h3.team0xf.com/
h3/h3r3tic on #D freenode
Oct 02 2009
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Tom S wrote:

 I think it should be done in userland, not built-in. Here's a 
 proof-of-concept implementation:


[awesome code snipped]

I am struck with awe. Thanks.

Andrei
Oct 02 2009
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
Tom S wrote:

 I think it should be done in userland, not built-in. Here's a 
 proof-of-concept implementation:


[awesome code snipped]

I am struck with awe. Thanks.

Andrei
Oct 02 2009
prev sibling parent reply Christian Kamm <kamm-incasoftware removethis.de> writes:
Tom S wrote:

 I think it should be done in userland, not built-in. Here's a
 proof-of-concept implementation:


I agree and also had a go at it a few months back:
http://www.digitalmars.com/d/archives/digitalmars/D/scope_as_template_struct_82104.html

What about alignment issues though? I think we need to force that byte array 
with the class data to have the same alignment restrictions as the class 
data.
Oct 02 2009
parent reply "Denis Koroskin" <2korden gmail.com> writes:
On Sat, 03 Oct 2009 10:52:09 +0400, Christian Kamm
<kamm-incasoftware removethis.de> wrote:


 Tom S wrote:

 I think it should be done in userland, not built-in. Here's a
 proof-of-concept implementation:


 I agree and also had a go at it a few months back:
 http://www.digitalmars.com/d/archives/digitalmars/D/scope_as_template_struct_82104.html

 What about alignment issues though? I think we need to force that byte  
 array
 with the class data to have the same alignment restrictions as the class
 data.


Yes, it is possible, but their use is dangerous without non-nullable  
*value* types, because user is not forced to initialize it in ctor.

I believe compiler should complain unless user explicitly assigns a value  
to such variable:

class Foo
{
       Scope!(Bar) bar;

       this()
       {
           //bar.construct(args); - a no-go, use of an unassigned variable.
           // How can compiler distinguish between bar.constract(args),
           // which is allowed to be called on non-constructed object, and
           // bar.doSomething(), which is not?
           // Besides, it hides possibly existing Bar.construct method.

           // bar = new Bar(args); - desirable syntax, but impossible w/o  
compiler help

           bar = Scope!Bar(args); // explicitly initialized
           bar.doSomething();     // okay
       }
}

I think we should either drop "scope" feature altogether in favor of  
library solution, or extend it to scope classes.
The former is not possible unless there is a way to implement scope(exit),  
scope(success) and scope(failure) in library, which I am not aware of.

That's why I prefer built-in scope class members ATM.

Be it implemented in a compiler, should it be rebindable or not? Andrei  
mentioned that it's probably not, but then it creates an inconsistency  
with local scope variable (those are rebindable).

Another difference is that currently the following code is allowed:

Foo createFoo() {
     return new FooDerivative();
}

scope Foo foo = createFoo(); // allocates on heap anyway, which might be  
confusing

Shouldn't compiler complain in case scope variable is heap-allocated?

Shall we unify the two concepts (local and class member scope variables)?  
If yes, then how?
Oct 03 2009
parent Yigal Chripun <yigal100 gmail.com> writes:
On 03/10/2009 11:59, Denis Koroskin wrote:

 On Sat, 03 Oct 2009 10:52:09 +0400, Christian Kamm
 <kamm-incasoftware removethis.de> wrote:


 Tom S wrote:

 I think it should be done in userland, not built-in. Here's a
 proof-of-concept implementation:


 I agree and also had a go at it a few months back:
 http://www.digitalmars.com/d/archives/digitalmars/D/scope_as_template_struct_82104.html


 What about alignment issues though? I think we need to force that byte
 array
 with the class data to have the same alignment restrictions as the class
 data.


 Yes, it is possible, but their use is dangerous without non-nullable
 *value* types, because user is not forced to initialize it in ctor.

 I believe compiler should complain unless user explicitly assigns a
 value to such variable:

 class Foo
 {
 Scope!(Bar) bar;

 this()
 {
 //bar.construct(args); - a no-go, use of an unassigned variable.
 // How can compiler distinguish between bar.constract(args),
 // which is allowed to be called on non-constructed object, and
 // bar.doSomething(), which is not?
 // Besides, it hides possibly existing Bar.construct method.

 // bar = new Bar(args); - desirable syntax, but impossible w/o compiler
 help

 bar = Scope!Bar(args); // explicitly initialized
 bar.doSomething(); // okay
 }
 }

 I think we should either drop "scope" feature altogether in favor of
 library solution, or extend it to scope classes.
 The former is not possible unless there is a way to implement
 scope(exit), scope(success) and scope(failure) in library, which I am
 not aware of.

 That's why I prefer built-in scope class members ATM.

 Be it implemented in a compiler, should it be rebindable or not? Andrei
 mentioned that it's probably not, but then it creates an inconsistency
 with local scope variable (those are rebindable).

 Another difference is that currently the following code is allowed:

 Foo createFoo() {
 return new FooDerivative();
 }

 scope Foo foo = createFoo(); // allocates on heap anyway, which might be
 confusing

 Shouldn't compiler complain in case scope variable is heap-allocated?

 Shall we unify the two concepts (local and class member scope
 variables)? If yes, then how?


I think scope should be completely removed from the language. it seems 
to me like the register keyword in C.
this aspect of memory management would be better off as an optimization 
by the compiler when applicable instead of user specified.

consider:
class B {...}
class A {
   // desirable syntax:
   // const B obj;
   NonRebindable!B obj; // assume language support instead of template

   this (args) { obj = new B(args); }
}

since obj is non-rebindable the compiler can optimize this without 
requiring the user to specify "scope".

local variables in a function would have similar semantics:

void foo() {
   NonRebindable!B = new B;
   ..
}

final classes and immutables can also scoped.

unless the user wants to explicitly handle memory with C malloc, I think 
the best thing would be to separate lifetime management from memory 
management. the programmer defines the lifetime of the data while the 
language/compiler/runtime handle all other aspects like choosing stack 
vs. heap, allocating and de-allocating (GC)
Oct 03 2009
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:


I'm thinking of using "scope" in this situation to imply in-situ storage:<


It may be good for the compiler to show a warning (that explains exactly why)
where the compiler can't scope a class marked as scope (both in a function or
in a method).

Bye,
bearophile
Oct 02 2009
prev sibling parent BCS <none anon.com> writes:
Hello Andrei,


 I think this has been discussed in this group already.
 


[...]

 That
 means the constructor needs special scrutiny, in particular a cannot
 be null because that wouldn't make much sense.


I think the same device used for the base class constructor could be used. 
What syntax it would use could be an issue but...
Oct 03 2009