www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - How to get nogc to work with manual memory allocation

reply "Bienlein" <jeti789 web.de> writes:
Hello,

I was having a look at the new nogc annotation and therefore 
wrote some code that creates an instance on the heap bypassing 
the GC (code adapted from http://dpaste.dzfl.pl/2377217c7870). 
Problem is that calls to call the class' constructor, destructor 
and others can't be called anymore once nogc is used. So the 
question is how to get manual allocation and deallocation done 
with using nogc. Here is the experimental code:

 nogc
T nogcNew(T, Args...) (Args args)
{
     import std.conv : emplace;
     import core.stdc.stdlib : malloc;

     // get class size of class object in bytes
     auto size = __traits(classInstanceSize, T);

     // allocate memory for the object
     auto memory = malloc(size)[0..size];
     if(!memory)
     {
         import core.exception : onOutOfMemoryError;
         onOutOfMemoryError();
     }

     // call T's constructor and emplace instance on
     // newly allocated memory
     return emplace!(T, Args)(memory, args);
}

 nogc
void nogcDel(T)(T obj)
{
     import core.stdc.stdlib : free;

     // calls obj's destructor
     destroy(obj);

     // free memory occupied by object
     free(cast(void*)obj);
}

 nogc
void main()
{
	TestClass test = nogcNew!TestClass();
	test.x = 123;
         nogcDel(test);
	test.x = 456;   // no protection violation ?!

	// remove  nogc to run this
	TestClass t = new TestClass();
	t.x = 789;
	delete t;
	t.x = 678;	// protection violation as expected
}

I have omitted the code for the TestClass class to save space. 
Problem is that the compiler outputs this:

Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' cannot 
call non- nogc function 'core.exception.onOutOfMemoryError'	
Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' cannot 
call non- nogc function 'std.conv.emplace!(TestClass, ).emplace'
Error:  nogc function 'main.nogcDel!(TestClass).nogcDel' cannot 
call non- nogc function 'object.destroy!(TestClass).destroy'

Is there a way to get around this? Then the test.x = 456; did not 
cause a protection violation although the instance was 
deallocated before calling nogcDel. Something with the 
deallocation in nogcDel seems not to work. Some hint appreciated 
on this. When calling delete t the protection violation happens 
on the next line as expected.

Thanks a lot, Bienlein
Aug 24 2014
next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Bienlein:


 Is there a way to get around this?


Perhaps there are ways, but note that  nogc is meant mostly for 
stack-allocation.
In general when in D you have to write lot of hairy code to do 
something, it means that in most cases you shouldn't do that 
something.



 When calling delete t the protection violation


Don't use "delete", it's deprecated (despite it's not yet 
deprecated because D designers love to keep some important things 
in such Limbo for several years).

Bye,
bearophile
Aug 24 2014
next sibling parent reply "Bienlein" <jeti789 web.de> writes:
On Sunday, 24 August 2014 at 08:48:03 UTC, bearophile wrote:


 Perhaps there are ways, but note that  nogc is meant mostly for 
 stack-allocation.


Ah, I missed that. Thanks for telling me. I changed nogcDel now 
to null out the deallocated object:

void nogcDel(T)(ref T obj)
{
     import core.stdc.stdlib : free;

     // calls obj's destructor
     destroy(obj);

     // free memory occupied by object
     free(cast(void*)obj);

     obj = null;
}

And now I also get the dearly missed protection violation ;-).
Aug 24 2014
parent "bearophile" <bearophileHUGS lycos.com> writes:
Bienlein:


  nogc is meant mostly for stack-allocation.


 Ah, I missed that. Thanks for telling me.


 nogc is also for allocation from the C heap, despite this is 
less common :-)

Bye,
bearophile
Aug 24 2014
prev sibling parent "eles" <eles215 gzk.dot> writes:
On Sunday, 24 August 2014 at 08:48:03 UTC, bearophile wrote:

 Bienlein:



 things in such Limbo for several years).


decades
Aug 24 2014
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-08-24 10:03, Bienlein wrote:


 I have omitted the code for the TestClass class to save space. Problem
 is that the compiler outputs this:

 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' cannot call
 non- nogc function 'core.exception.onOutOfMemoryError'
 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' cannot call
 non- nogc function 'std.conv.emplace!(TestClass, ).emplace'
 Error:  nogc function 'main.nogcDel!(TestClass).nogcDel' cannot call
 non- nogc function 'object.destroy!(TestClass).destroy'

 Is there a way to get around this?


 nogc is a very new attribute. The runtime and standard library have not 
been properly annotated with this attribute yet.

As a workaround you could try copy implementation of these functions to 
you're own code and annotate them as appropriate.

-- 
/Jacob Carlborg
Aug 24 2014
next sibling parent reply "Kagamin" <spam here.lot> writes:
On Sunday, 24 August 2014 at 09:29:53 UTC, Jacob Carlborg wrote:

 On 2014-08-24 10:03, Bienlein wrote:


 I have omitted the code for the TestClass class to save space. 
 Problem
 is that the compiler outputs this:

 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' 
 cannot call
 non- nogc function 'core.exception.onOutOfMemoryError'
 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' 
 cannot call
 non- nogc function 'std.conv.emplace!(TestClass, ).emplace'
 Error:  nogc function 'main.nogcDel!(TestClass).nogcDel' 
 cannot call
 non- nogc function 'object.destroy!(TestClass).destroy'

 Is there a way to get around this?


  nogc is a very new attribute. The runtime and standard library 
 have not been properly annotated with this attribute yet.

 As a workaround you could try copy implementation of these 
 functions to you're own code and annotate them as appropriate.


Shouldn't emplace and destroy infer their attributes instead of 
strictly annotating them as nogc.
Aug 24 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 2014-08-24 14:18, Kagamin wrote:


 Shouldn't emplace and destroy infer their attributes instead of strictly
 annotating them as nogc.


If they are templates, I guess they should. I don't know how good the 
compiler is at inferring attributes. I also haven't looked at the source 
code for these functions so I don't know if there's anything stopping 
them from begin  nogc.

-- 
/Jacob Carlborg
Aug 24 2014
parent reply "Kagamin" <spam here.lot> writes:
On Sunday, 24 August 2014 at 13:27:01 UTC, Jacob Carlborg wrote:

 On 2014-08-24 14:18, Kagamin wrote:


 Shouldn't emplace and destroy infer their attributes instead 
 of strictly
 annotating them as nogc.


 If they are templates, I guess they should. I don't know how 
 good the compiler is at inferring attributes. I also haven't 
 looked at the source code for these functions so I don't know 
 if there's anything stopping them from begin  nogc.


emplace calls constructor, and constructor can't be realistically 
required to be nogc. It depends on the constructor. Similar for 
destroy.
Sep 04 2014
parent Jacob Carlborg <doob me.com> writes:
On 04/09/14 22:30, Kagamin wrote:


 emplace calls constructor, and constructor can't be realistically
 required to be nogc. It depends on the constructor. Similar for destroy.


But if the constructor is  nogc or if there's a default constructor.

-- 
/Jacob Carlborg
Sep 04 2014
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Sunday, 24 August 2014 at 09:29:53 UTC, Jacob Carlborg wrote:

 On 2014-08-24 10:03, Bienlein wrote:


 I have omitted the code for the TestClass class to save space. 
 Problem
 is that the compiler outputs this:

 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' 
 cannot call
 non- nogc function 'core.exception.onOutOfMemoryError'
 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' 
 cannot call
 non- nogc function 'std.conv.emplace!(TestClass, ).emplace'
 Error:  nogc function 'main.nogcDel!(TestClass).nogcDel' 
 cannot call
 non- nogc function 'object.destroy!(TestClass).destroy'

 Is there a way to get around this?


  nogc is a very new attribute. The runtime and standard library 
 have not been properly annotated with this attribute yet.

 As a workaround you could try copy implementation of these 
 functions to you're own code and annotate them as appropriate.


The real issue here is actually the *language*. Exceptions and 
Errors are GC allocated, so if you try to "check" your no-GC 
allocation, you'll use the GC...

A possible workaround is to preemptively static allocate the 
Error. However, this can cause issues if you chain Errors, or 
re-enter your function while throwing.
Sep 04 2014
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Friday, 5 September 2014 at 06:43:56 UTC, monarch_dodra wrote:

 On Sunday, 24 August 2014 at 09:29:53 UTC, Jacob Carlborg wrote:

 On 2014-08-24 10:03, Bienlein wrote:


 I have omitted the code for the TestClass class to save 
 space. Problem
 is that the compiler outputs this:

 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' 
 cannot call
 non- nogc function 'core.exception.onOutOfMemoryError'
 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' 
 cannot call
 non- nogc function 'std.conv.emplace!(TestClass, ).emplace'
 Error:  nogc function 'main.nogcDel!(TestClass).nogcDel' 
 cannot call
 non- nogc function 'object.destroy!(TestClass).destroy'

 Is there a way to get around this?


  nogc is a very new attribute. The runtime and standard 
 library have not been properly annotated with this attribute 
 yet.

 As a workaround you could try copy implementation of these 
 functions to you're own code and annotate them as appropriate.


 The real issue here is actually the *language*. Exceptions and 
 Errors are GC allocated, so if you try to "check" your no-GC 
 allocation, you'll use the GC...

 A possible workaround is to preemptively static allocate the 
 Error. However, this can cause issues if you chain Errors, or 
 re-enter your function while throwing.


It was also discussed to exempt `new Error` (and assert) from the 
 nogc restriction.
Sep 05 2014
prev sibling parent "Kiith-Sa" <kiithsacmp gmail.com> writes:
On Sunday, 24 August 2014 at 08:03:11 UTC, Bienlein wrote:

 Hello,

 I was having a look at the new nogc annotation and therefore 
 wrote some code that creates an instance on the heap bypassing 
 the GC (code adapted from http://dpaste.dzfl.pl/2377217c7870). 
 Problem is that calls to call the class' constructor, 
 destructor and others can't be called anymore once nogc is 
 used. So the question is how to get manual allocation and 
 deallocation done with using nogc. Here is the experimental 
 code:

  nogc
 T nogcNew(T, Args...) (Args args)
 {
     import std.conv : emplace;
     import core.stdc.stdlib : malloc;

     // get class size of class object in bytes
     auto size = __traits(classInstanceSize, T);

     // allocate memory for the object
     auto memory = malloc(size)[0..size];
     if(!memory)
     {
         import core.exception : onOutOfMemoryError;
         onOutOfMemoryError();
     }

     // call T's constructor and emplace instance on
     // newly allocated memory
     return emplace!(T, Args)(memory, args);
 }

  nogc
 void nogcDel(T)(T obj)
 {
     import core.stdc.stdlib : free;

     // calls obj's destructor
     destroy(obj);

     // free memory occupied by object
     free(cast(void*)obj);
 }

  nogc
 void main()
 {
 	TestClass test = nogcNew!TestClass();
 	test.x = 123;
         nogcDel(test);
 	test.x = 456;   // no protection violation ?!

 	// remove  nogc to run this
 	TestClass t = new TestClass();
 	t.x = 789;
 	delete t;
 	t.x = 678;	// protection violation as expected
 }

 I have omitted the code for the TestClass class to save space. 
 Problem is that the compiler outputs this:

 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' 
 cannot call non- nogc function 
 'core.exception.onOutOfMemoryError'	
 Error:  nogc function 'main.nogcNew!(TestClass, ).nogcNew' 
 cannot call non- nogc function 'std.conv.emplace!(TestClass, 
 ).emplace'
 Error:  nogc function 'main.nogcDel!(TestClass).nogcDel' cannot 
 call non- nogc function 'object.destroy!(TestClass).destroy'

 Is there a way to get around this? Then the test.x = 456; did 
 not cause a protection violation although the instance was 
 deallocated before calling nogcDel. Something with the 
 deallocation in nogcDel seems not to work. Some hint 
 appreciated on this. When calling delete t the protection 
 violation happens on the next line as expected.

 Thanks a lot, Bienlein



 nogc is *no GC*, it has nothing to do with where you allocate 
from, no matter how some would choose to interpret it (I use it 
to mean 'no heap' pretty often in my own projects, though, can be 
useful).

You can mark it  nogc by wrapping the non- nogc stuff in a 
function and casting it, but you *absolutely* must be sure GC is 
not used in any called functions.

You could possibly do that by requiring T's constructor to be 
 nogc, not sure if the other functions can use GC or not.


Also, your function seems to be written only for classes,
if this is intended, you should have an

if(is(T == class))

template constraint there.
Sep 04 2014