www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Why is sformat and formattedWrite (appender) allocating GC mem here?

reply cc <cc nevernet.com> writes:
And what, if anything, can I do to avoid it?

	string[] arr = ["abcd", "efgh", "ijkl"];
	char[4096] buf;
	char[] res;
	writeln(GC.stats);
	res = sformat(buf, "%s", arr);
	assert(res.ptr == buf.ptr);
	writeln(res);
	writeln(GC.stats);
	res = sformat(buf, "%s", arr);
	assert(res.ptr == buf.ptr);
	writeln(res);
	writeln(GC.stats);

	// Results:
	Stats(64, 1048512, 80)
	["abcd", "efgh", "ijkl"]
	Stats(272, 1048304, 272)
	["abcd", "efgh", "ijkl"]
	Stats(464, 1048112, 464)

I get similar behavior trying to use formattedWrite with an 
appender:

	string[] arr = ["abcd", "efgh", "ijkl"];
	auto formatBuffer = appender!(char[])();
	formatBuffer.reserve(4096);
	formatBuffer.clear();
	writeln(GC.stats);
	formatBuffer.formattedWrite!"%s"(arr);
	writeln(formatBuffer.data);
	writeln(GC.stats);
	formatBuffer.clear();
	formatBuffer.formattedWrite!"%s"(arr);
	writeln(formatBuffer.data);
	writeln(GC.stats);

	// Results:
	Stats(12288, 5230592, 4208)
	["abcd", "efgh", "ijkl"]
	Stats(12432, 5230448, 4352)
	["abcd", "efgh", "ijkl"]
	Stats(12576, 5230304, 4496)

Same thing if I use a format string like "%(%s,%)" rather than 
just "%s".  I'm guessing it's allocating a string first to write 
the contents of the array and then inserting that string into the 
buffer I supplied.  Is there no way to have it skip this step and 
just write each element (plus the joining punctuation, etc) 
directly into the buffer?
Aug 31 2019
next sibling parent Mike Parker <aldacron gmail.com> writes:
On Saturday, 31 August 2019 at 12:07:56 UTC, cc wrote:
 And what, if anything, can I do to avoid it?
import core.stdc.stdio : printf; There are no nogc versions of the Phobos write* and format functions.
Aug 31 2019
prev sibling parent reply ag0aep6g <anonymous example.com> writes:
On 31.08.19 14:07, cc wrote:
 I'm guessing it's allocating a string first to write the contents 
 of the array and then inserting that string into the buffer I supplied.  
 Is there no way to have it skip this step and just write each element 
 (plus the joining punctuation, etc) directly into the buffer?
`formatElement` does something like that. It writes the string into a temporary buffer while looking for invalid Unicode. When it finds some, the temporary is discarded and the whole string is formatted differently. When the string is a-ok, the data is copied over to the actual destination. I'm not sure if that's the best approach, though. The temporary buffer and the string copy are costly. There is also a closure being allocated for no reason in `sformat` itself. The compiler isn't smart enough to see that it's not really needed. I've made a pull request to get rid of those allocations: https://github.com/dlang/phobos/pull/7163
Aug 31 2019
next sibling parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Saturday, 31 August 2019 at 21:12:32 UTC, ag0aep6g wrote:
 I've made a pull request to get rid of those allocations:
 https://github.com/dlang/phobos/pull/7163
Merged.
Aug 31 2019
prev sibling next sibling parent cc <cc nevernet.com> writes:
On Saturday, 31 August 2019 at 21:12:32 UTC, ag0aep6g wrote:
 I've made a pull request to get rid of those allocations:
 https://github.com/dlang/phobos/pull/7163
Thanks for the responses, very cool seeing these updates happen so fluidly.
Sep 01 2019
prev sibling parent reply James Blachly <james.blachly gmail.com> writes:
On 8/31/19 5:12 PM, ag0aep6g wrote:
 I've made a pull request to get rid of those allocations:
 https://github.com/dlang/phobos/pull/7163
Wonderful! For my own learning, why was a unittest to ensure no GC added to sformat instead of a nogc annotation?
Sep 03 2019
parent ag0aep6g <anonymous example.com> writes:
On 03.09.19 16:03, James Blachly wrote:
 For my own learning, why was a unittest to ensure no GC added to sformat 
 instead of a  nogc annotation?
`sformat` uses the GC less now, but it's not nogc. It can still throw GC-allocated exceptions, e.g. when the arguments don't match the format string.
Sep 03 2019