www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Allocation failures

reply =?UTF-8?B?SmFyZMOtaw==?= <idontgive any.com> writes:
When I was interested in D some time ago, I believe GC was 
aborting the application on allocation failures. Is that still 
the case today? I am looking into using D for my new application, 
but I need some guarantees that I can at least save some critical 
data, when such thing happens, perhaps freeing some stuff that is 
not really needed for the saving part (e.g. I could destroy GUI, 
free all saved unmodified documents and after that proceed to 
save unsaved changes to temporary file of some kind (even 
unbuffered as i may not have enough memory for a buffer). This is 
essential for me, that I am able to do it, that there is a change 
to handle such situation. The application will run on a system 
with over-commit disabled, so that (C) malloc can correctly 
report failures. Is it currently possible? Will it be possible in 
a near future?

I could use malloc directly for such data and check its return 
values, but it won't save me when there would be enough memory 
and while after that GC will fail on something else and crash my 
application.

I was also thinking about using rust for it, but for this very 
same reason I can't (rust aborts() almost everywhere on 
allocation failures).
Feb 14 2016
next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
Since you want such fine grained control, you probably don't want to use 
the GC to allocate with.

Take a look at [0].
That should give you the fine grained control you desire.

Most importantly make, makeArray, expandArray, shrinkArray and dispose.
You would need to be a bit careful with some language features like new 
to make sure you don't use them in favor of this. Keep in mind you won't 
be using the GC to deallocate with unless you add it to it (addRoot).

[0] http://dlang.org/phobos/std_experimental_allocator.html
Feb 14 2016
prev sibling parent reply cym13 <cpicard openmailbox.org> writes:
On Sunday, 14 February 2016 at 12:10:34 UTC, Jardík wrote:
 When I was interested in D some time ago, I believe GC was 
 aborting the application on allocation failures. Is that still 
 the case today? I am looking into using D for my new 
 application, but I need some guarantees that I can at least 
 save some critical data, when such thing happens, perhaps 
 freeing some stuff that is not really needed for the saving 
 part (e.g. I could destroy GUI, free all saved unmodified 
 documents and after that proceed to save unsaved changes to 
 temporary file of some kind (even unbuffered as i may not have 
 enough memory for a buffer). This is essential for me, that I 
 am able to do it, that there is a change to handle such 
 situation. The application will run on a system with 
 over-commit disabled, so that (C) malloc can correctly report 
 failures. Is it currently possible? Will it be possible in a 
 near future?

 [...]
Allocation failures throw errors such as onOutOfMemoryError which are meant to terminate the program (see the distinction between errors and exceptions in D) but I guess if you really need to you could catch it and save your data before exiting.
Feb 14 2016
parent reply =?UTF-8?B?SmFyZMOtaw==?= <idonthaveany idonthaveany.com> writes:
But if I couldn't use GC and do all allocations and deallocations 
manually, I wouldn't even be able to use exceptions and there 
would also no longer be much reason to write it in D. I did some 
searching and came into discussion and there found out that in 
case of an error thrown, D just forcibly unwinds the stack 
without running destructors or scope guards. Is that true? Can I 
even catch that Error?
Feb 16 2016
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 16 February 2016 at 17:15:17 UTC, Jardík wrote:
 But if I couldn't use GC and do all allocations and 
 deallocations manually, I wouldn't even be able to use 
 exceptions and there would also no longer be much reason to 
 write it in D.
You can use exceptions without the GC and D offers a lot of other things too.
 I did some searching and came into discussion and there found 
 out that in case of an error thrown, D just forcibly unwinds 
 the stack without running destructors or scope guards. Is that 
 true?
No, not generally.
 Can I even catch that Error?
Yes.
Feb 16 2016
parent reply Chris Wright <dhasenan gmail.com> writes:
On Tue, 16 Feb 2016 17:35:02 +0000, Adam D. Ruppe wrote:

 On Tuesday, 16 February 2016 at 17:15:17 UTC, Jardík wrote:
 But if I couldn't use GC and do all allocations and deallocations
 manually, I wouldn't even be able to use exceptions and there would
 also no longer be much reason to write it in D.
You can use exceptions without the GC and D offers a lot of other things too.
Specifically using http://dpldocs.info/search/search?searchTerm=emplace to create an exception object in manually allocated memory.
Feb 16 2016
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Wednesday, 17 February 2016 at 00:45:35 UTC, Chris Wright 
wrote:
 http://dpldocs.info/search/search?searchTerm=emplace to create 
 an exception object in manually allocated memory.
Aye, this overload: http://dpldocs.info/experimental-docs/std.conv.emplace.3.html though the example there is awful, wtf is it even trying to show? But anyway, you can also throw a static object. That's actually what outOfMemoryError does <http://dpldocs.info/experimental-docs/core.exception.onOutOfMemoryError.html> - it throws a statically allocated object.
Feb 16 2016
next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Wed, 17 Feb 2016 01:09:20 +0000, Adam D. Ruppe wrote:

 On Wednesday, 17 February 2016 at 00:45:35 UTC, Chris Wright wrote:
 http://dpldocs.info/search/search?searchTerm=emplace to create an
 exception object in manually allocated memory.
Aye, this overload: http://dpldocs.info/experimental-docs/std.conv.emplace.3.html though the example there is awful, wtf is it even trying to show? But anyway, you can also throw a static object. That's actually what outOfMemoryError does <http://dpldocs.info/experimental-docs/
core.exception.onOutOfMemoryError.html>
 - it throws a statically allocated object.
Sure, but that's pretty annoying. You have to ensure that you will never need to throw two of the same exception at the same time -- or you allocate several statically and go through hoops to ensure you're using one that's not currently in use. It's a lot easier to malloc and free exceptions. Since they're presumably only cropping up in exceptional circumstances, there's a lot less worry about making many small allocations.
Feb 17 2016
prev sibling parent Marc =?UTF-8?B?U2Now7x0eg==?= <schuetzm gmx.net> writes:
On Wednesday, 17 February 2016 at 01:09:20 UTC, Adam D. Ruppe 
wrote:
 On Wednesday, 17 February 2016 at 00:45:35 UTC, Chris Wright 
 wrote:
 http://dpldocs.info/search/search?searchTerm=emplace to create 
 an exception object in manually allocated memory.
Aye, this overload: http://dpldocs.info/experimental-docs/std.conv.emplace.3.html though the example there is awful, wtf is it even trying to show?
I guess it is primarily supposed to be a unittest to verify that emplace works with interfaces...
Feb 17 2016
prev sibling next sibling parent reply cym13 <cpicard openmailbox.org> writes:
On Tuesday, 16 February 2016 at 17:15:17 UTC, Jardík wrote:
 But if I couldn't use GC and do all allocations and 
 deallocations manually, I wouldn't even be able to use 
 exceptions and there would also no longer be much reason to 
 write it in D. I did some searching and came into discussion 
 and there found out that in case of an error thrown, D just 
 forcibly unwinds the stack without running destructors or scope 
 guards. Is that true? Can I even catch that Error?
Such errors are static errors, they aren't allocated on the stack, a 128 bytes buffer is shared accross threads to keep them.
Feb 16 2016
parent reply cym13 <cpicard openmailbox.org> writes:
On Wednesday, 17 February 2016 at 02:23:34 UTC, cym13 wrote:
 Such errors are static errors, they aren't allocated on the 
 stack, a 128 bytes buffer is shared accross threads to keep 
 them.
Sorry, of course I meant they *are* allocated on the stack.
Feb 16 2016
parent Chris Wright <dhasenan gmail.com> writes:
On Wed, 17 Feb 2016 02:24:51 +0000, cym13 wrote:

 On Wednesday, 17 February 2016 at 02:23:34 UTC, cym13 wrote:
 Such errors are static errors, they aren't allocated on the stack, a
 128 bytes buffer is shared accross threads to keep them.
Sorry, of course I meant they *are* allocated on the stack.
In the context of exceptions, you can't allocate them on the stack at the point at which you throw them. As soon as you throw them, the stack frame is invalidated (to run destructors for stack-scoped items, possibly other things), so your exception object is at least moderately likely to be corrupted. You *could* allocate them on the stack in, say, main(), but then you'd have to pass a reference to them down the stack somehow. Kind of awkward.
Feb 17 2016
prev sibling parent reply thedeemon <dlang thedeemon.com> writes:
On Tuesday, 16 February 2016 at 17:15:17 UTC, Jardík wrote:
 But if I couldn't use GC and do all allocations and 
 deallocations manually, I wouldn't even be able to use 
 exceptions and there would also no longer be much reason to 
 write it in D.
I'd say if you're going to grow GC heap so big it doesn't fit in memory it's a bad idea in the first place, since each GC cycle will take a lot of time, like, a lot. It might also leak. In a 32-bit process it will leak almost for sure. So keep GC heap for small litter (including exceptions) and use other allocators for large pieces of data. This way you may get best of what D offers without long-GC-pause or out-of-memory-termination pains.
Feb 16 2016
parent reply =?UTF-8?B?SmFyZMOtaw==?= <idonthaveany idonthaveany.com> writes:
On Wednesday, 17 February 2016 at 06:52:21 UTC, thedeemon wrote:
 So keep GC heap for small litter (including exceptions) and use 
 other allocators for large pieces of data. This way you may get 
 best of what D offers without long-GC-pause or 
 out-of-memory-termination pains.
I will probably do that. Since others said I can actually catch that out of memory error and that the destructors will run (and thus free some of the resources) I see no problem in writing it in D. I am happy it doesn't force abort()s as some other languages (rust) or libraries (glib). Thank you everyone for answering my questions and consider this solved (where can I mark it as such?).
Feb 17 2016
parent Mike Parker <aldacron gmail.com> writes:
On Wednesday, 17 February 2016 at 14:01:17 UTC, Jardík wrote:

 consider this solved (where can I mark it as such?).
The forum is a web fronted for a news server. No such feature.
Feb 17 2016