www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - toString, to!(char[]) & co

reply Fawzi Mohamed <fmohamed mac.com> writes:
Taking a hint from bearophile I repost here some of my thoughts about 
converting to string.

If you look at the allocations needed for a toString, one sees the 
allocation of a lot of small strings, alloacation for the concatenation 
of a couple of the small strings, then again for the larger blocks 
until you get the final string.
To me it reeks of bad design, especially if there is an easy way to 
avoid most problems, a way that can avoid almost all allocations.

I find it much better to base everything on something like
	void writeOut(T,S...)(void delegate char[]sink, T object,S formatting)
that outputs to a sink, so that you can have output without memory
allocation.
(well my version is little more complex because I want to accept also
other stuff not just a sink 
http://github.com/fawzi/blip/blob/master/blip/io/BasicIO.d
  )

If you want a single string you can easily write helpers like
	T[] collectAppender(T)(void delegate(void delegate(T[]))
appender,char[] buf=null)
or similar ( 
http://github.com/fawzi/blip/blob/master/blip/container/GrowableArray.d
  ) that convert sink based stuff to a single string.

I think that that design is very efficient and clean, well I have to
admit that I *hate* toString and methods that convert stuff to string,
because they are unnecessarily inefficient, and can be trivially 
obtained from the more efficient ones.

Real serialization is a different beast (at least what I want
from serialization), there the "how" should be in the target serializer,
the serialization function should just give enough information to the
serializer, so that it might serialize how it pleases to him (in
particular meaningful labels should be given for serialization to
json,xml,...), at least that is what I do in my serialization library.

Fawzi
Mar 10 2010
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 03/10/2010 04:16 PM, Fawzi Mohamed wrote:

 If you look at the allocations needed for a toString, one sees the
 allocation of a lot of small strings, alloacation for the concatenation
 of a couple of the small strings, then again for the larger blocks until
 you get the final string.
 To me it reeks of bad design, especially if there is an easy way to
 avoid most problems, a way that can avoid almost all allocations.

 I find it much better to base everything on something like
 void writeOut(T,S...)(void delegate char[]sink, T object,S formatting)
 that outputs to a sink, so that you can have output without memory
 allocation.
 (well my version is little more complex because I want to accept also
 other stuff not just a sink
 http://github.com/fawzi/blip/blob/master/blip/io/BasicIO.d
 )


I agree. For the longest time I wanted to find the time to output and 
format stuff to arbitrary output streams (including string appenders as 
a particular case), but couldn't find the time.

Andrei
Mar 10 2010
parent Fawzi Mohamed <fawzi gmx.ch> writes:
On 10-mar-10, at 23:23, Andrei Alexandrescu wrote:


 On 03/10/2010 04:16 PM, Fawzi Mohamed wrote:

 If you look at the allocations needed for a toString, one sees the
 allocation of a lot of small strings, alloacation for the  
 concatenation
 of a couple of the small strings, then again for the larger blocks  
 until
 you get the final string.
 To me it reeks of bad design, especially if there is an easy way to
 avoid most problems, a way that can avoid almost all allocations.

 I find it much better to base everything on something like
 void writeOut(T,S...)(void delegate char[]sink, T object,S  
 formatting)
 that outputs to a sink, so that you can have output without memory
 allocation.
 (well my version is little more complex because I want to accept also
 other stuff not just a sink
 http://github.com/fawzi/blip/blob/master/blip/io/BasicIO.d
 )


 I agree. For the longest time I wanted to find the time to output  
 and format stuff to arbitrary output streams (including string  
 appenders as a particular case), but couldn't find the time.


good, well time is always too short :)

Fawzi
Mar 10 2010
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Fawzi Mohamed:


 If you look at the allocations needed for a toString, one sees the 
 allocation of a lot of small strings, alloacation for the concatenation 
 of a couple of the small strings, then again for the larger blocks 
 until you get the final string.


A possible idea is to replace the current object.toString:
string toString(); 

With something vaguely like (keeping it simple):
string toString(void delegate(string) sink=null);

If called with no arguments the toString has to output a string, as now.

If called with a sink that is not null, the toString can choose to use it for
the output and return an empty string, or ignore it and return a string.

The caller can give a sink or not. But even if gives a sink it has to test if
the return value is an nonempty string, because the toString is free to ignore
the given sink.

[Extra: if toString both returns a nonempty string and gives something to the
sink, then the caller can use a nice silver coin to choose among the two.]

Bye,
bearophile
Mar 10 2010
parent Fawzi Mohamed <fawzi gmx.ch> writes:
On 10-mar-10, at 23:52, bearophile wrote:


 Fawzi Mohamed:


 If you look at the allocations needed for a toString, one sees the
 allocation of a lot of small strings, alloacation for the  
 concatenation
 of a couple of the small strings, then again for the larger blocks
 until you get the final string.


 A possible idea is to replace the current object.toString:
 string toString();

 With something vaguely like (keeping it simple):
 string toString(void delegate(string) sink=null);

 If called with no arguments the toString has to output a string, as  
 now.

 If called with a sink that is not null, the toString can choose to  
 use it for the output and return an empty string, or ignore it and  
 return a string.

 The caller can give a sink or not. But even if gives a sink it has  
 to test if the return value is an nonempty string, because the  
 toString is free to ignore the given sink.

 [Extra: if toString both returns a nonempty string and gives  
 something to the sink, then the caller can use a nice silver coin to  
 choose among the two.]


this makes the implementation of toString more complicated, I have
	void desc(void delegate(string))
and actually normally I mixin serialization and then mixin a template  
(printOut) that implements desc using Json serialization, and toString  
by collecting desc into a GrowableArray...
Mar 10 2010