www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Efficient outputting of to-string conversions

reply Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Looks like std.conv.to always allocates behind the scenes. It's a shame as the
returned string is immediately processed and discarded in my XML writer. Are
there plans to include a custom output variant, e.g. to!string(7, outputRange)?

-- 
Tomek
Feb 07 2011
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday 07 February 2011 13:10:09 Tomek Sowi=C5=84ski wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9
Feb 07 2011
next sibling parent reply Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Jonathan M Davis napisa=B3:


 On Monday 07 February 2011 13:10:09 Tomek Sowi=F1ski wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a shame=




 as

 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


=20
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9


I know about Steven's proposal but it applies only to user types not primit=
ives. Either way std.conv.to would need a buffered output range as integers=
 are written from the right. Any chance for an abstraction analogous to buf=
fered input ranges discussed recently?

--=20
Tomek
Feb 07 2011
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/7/11 5:38 PM, Tomek Sowi�ski wrote:

 Jonathan M Davis napisa�:


 On Monday 07 February 2011 13:10:09 Tomek Sowi�ski wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9


 I know about Steven's proposal but it applies only to user types not
primitives. Either way std.conv.to would need a buffered output range as
integers are written from the right. Any chance for an abstraction analogous to
buffered input ranges discussed recently?


Generally I found it more difficult to define a solid output buffer 
abstraction. This is a great motivating example though.

To my surprise, an API of the same form seems to be what the doctor 
prescribed. Here's a semi-formal definition:

A buffered output range R is defined as such:

R.front returns the currently uncommitted buffer of type T[]

R.moreFront(n) makes n more elements available for writing

R.commitFront(n) writes the first n elements in front()

R.flushFront() writes the buffer currently held in front() and makes 
another buffer available (initially empty).


Andrei
Feb 07 2011
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/7/11 7:25 PM, Andrei Alexandrescu wrote:

 On 2/7/11 5:38 PM, Tomek Sowi�ski wrote:

 Jonathan M Davis napisa�:


 On Monday 07 February 2011 13:10:09 Tomek Sowi�ski wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a
 shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9


 I know about Steven's proposal but it applies only to user types not
 primitives. Either way std.conv.to would need a buffered output range
 as integers are written from the right. Any chance for an abstraction
 analogous to buffered input ranges discussed recently?


 Generally I found it more difficult to define a solid output buffer
 abstraction. This is a great motivating example though.

 To my surprise, an API of the same form seems to be what the doctor
 prescribed.


I meant "of the same form as buffered input ranges". The primitives are 
similar: expose a buffer, allow it to grow, and discard it progressively.

Andrei
Feb 07 2011
prev sibling parent Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Andrei Alexandrescu napisa=B3:


 I know about Steven's proposal but it applies only to user types not pr=




imitives. Either way std.conv.to would need a buffered output range as inte=
gers are written from the right. Any chance for an abstraction analogous to=
 buffered input ranges discussed recently? =20

=20
 Generally I found it more difficult to define a solid output buffer=20
 abstraction. This is a great motivating example though.
=20
 To my surprise, an API of the same form seems to be what the doctor=20
 prescribed. Here's a semi-formal definition:
=20
 A buffered output range R is defined as such:
=20
 R.front returns the currently uncommitted buffer of type T[]
=20
 R.moreFront(n) makes n more elements available for writing
=20
 R.commitFront(n) writes the first n elements in front()
=20
 R.flushFront() writes the buffer currently held in front() and makes=20
 another buffer available (initially empty).


I was thinking along the same lines. There's one missing:

R.skipFront(n) skips the first n elements without outputting

Why? Look at integral conversions in std.conv.to. It first calculates maxim=
um string size, then writes numbers to the char array back to front, then r=
eturns result[$ - ndigits .. $] where ndigits is how long the string turned=
 out.

Returning to Steven's DIP, I think writeTo should take the above rather tha=
n void delegate(char[]). With the latter you still have to allocate the pie=
ces. Our buffered output range is friends with polymorphism too. If you set=
 T=3Dchar, its API is devoid of generics. Such interface can be placed in o=
bject.d with an official blessing.

--=20
Tomek
Feb 08 2011
prev sibling next sibling parent reply spir <denis.spir gmail.com> writes:
On 02/07/2011 11:38 PM, Tomek Sowiński wrote:

 Jonathan M Davis napisał:


 On Monday 07 February 2011 13:10:09 Tomek Sowiński wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9


 I know about Steven's proposal but it applies only to user types not
primitives. Either way std.conv.to would need a buffered output range as
integers are written from the right. Any chance for an abstraction analogous to
buffered input ranges discussed recently?


I haven't read a solution like the proposal for writeTo would not apply to 
primitives. Is it really so? (If only for language consistency, I would prefere 
this big shift to apply to all types.)

-- 
_________________
vita es estrany
spir.wikidot.com
Feb 08 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 08 Feb 2011 10:55:23 -0500, spir <denis.spir gmail.com> wrote:


 On 02/07/2011 11:38 PM, Tomek Sowiński wrote:

 Jonathan M Davis napisał:


 On Monday 07 February 2011 13:10:09 Tomek Sowiński wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a  
 shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9


 I know about Steven's proposal but it applies only to user types not  
 primitives. Either way std.conv.to would need a buffered output range  
 as integers are written from the right. Any chance for an abstraction  
 analogous to buffered input ranges discussed recently?


 I haven't read a solution like the proposal for writeTo would not apply  
 to primitives. Is it really so? (If only for language consistency, I  
 would prefere this big shift to apply to all types.)


writeTo does not need to apply to primitives because primitives can be  
written to streams without first converting to char[].  For instance, one  
does not write:

writeln(to!string(5))

The whole point of writeTo is to allow easy output of custom data types  
that the standard library does not know how to output.  It was not meant  
to extend to primitives.

As far as Tomek's request, I would expect  
std.format.formattedWrite(buffer, "%s", 7) to work, or  
std.format.formatValue (with associated FormatSpec, which I don't feel  
like looking up the usage of)

-Steve
Feb 08 2011
parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 08 Feb 2011 11:09:44 -0500, Steven Schveighoffer  
<schveiguy yahoo.com> wrote:


 On Tue, 08 Feb 2011 10:55:23 -0500, spir <denis.spir gmail.com> wrote:


 On 02/07/2011 11:38 PM, Tomek Sowiński wrote:

 Jonathan M Davis napisał:


 On Monday 07 February 2011 13:10:09 Tomek Sowiński wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a  
 shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9


 I know about Steven's proposal but it applies only to user types not  
 primitives. Either way std.conv.to would need a buffered output range  
 as integers are written from the right. Any chance for an abstraction  
 analogous to buffered input ranges discussed recently?


 I haven't read a solution like the proposal for writeTo would not apply  
 to primitives. Is it really so? (If only for language consistency, I  
 would prefere this big shift to apply to all types.)


 writeTo does not need to apply to primitives because primitives can be  
 written to streams without first converting to char[].  For instance,  
 one does not write:

 writeln(to!string(5))

 The whole point of writeTo is to allow easy output of custom data types  
 that the standard library does not know how to output.  It was not meant  
 to extend to primitives.

 As far as Tomek's request, I would expect  
 std.format.formattedWrite(buffer, "%s", 7) to work, or  
 std.format.formatValue (with associated FormatSpec, which I don't feel  
 like looking up the usage of)


BTW, I think we probably should have a formatValue override like this

void formatValue(Char, Writer, T)(Writer w, T t)
{
    FormatSpec!Char fs; // use default options
    formatValue(w, t, fs);
}

Having to construct a FormatSpec when I just want the default seems like  
overkill.  Esp. since FormatSpec arg is always ref.

-Steve
Feb 08 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Tuesday 08 February 2011 07:55:23 spir wrote:

 On 02/07/2011 11:38 PM, Tomek Sowi=C5=84ski wrote:

 Jonathan M Davis napisa=C5=82:

 On Monday 07 February 2011 13:10:09 Tomek Sowi=C5=84ski wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a sha=








me

 as the returned string is immediately processed and discarded in my
 XML writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


=20
 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9


=20
 I know about Steven's proposal but it applies only to user types not
 primitives. Either way std.conv.to would need a buffered output range as
 integers are written from the right. Any chance for an abstraction
 analogous to buffered input ranges discussed recently?


=20
 I haven't read a solution like the proposal for writeTo would not apply to
 primitives. Is it really so? (If only for language consistency, I would
 prefere this big shift to apply to all types.)


Umm. _How_ would it apply to primitives. Does _toString_ apply to primitive=
s?=20
No. toString and writeTo apply to user-defined types that they're defined o=
n. That=20
doesn't mean that we can't or shouldn't find a buffered solution for dealin=
g with=20
primitives and I/O similar to writeTo, but primitives can't have writeTo de=
fined=20
on them any more than they can have toString defined on them.

=2D Jonathan M Davis
Feb 08 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/7/11 4:43 PM, Jonathan M Davis wrote:

 On Monday 07 February 2011 13:10:09 Tomek Sowiński wrote:

 Looks like std.conv.to always allocates behind the scenes. It's a shame as
 the returned string is immediately processed and discarded in my XML
 writer. Are there plans to include a custom output variant, e.g.
 to!string(7, outputRange)?


 http://prowiki.org/wiki4d/wiki.cgi?LanguageDevel/DIPs/DIP9


Yah, we should hop on that. The DIP is concerned mainly with toString, 
not std.conv. For the latter, Tomek's idea of passing an output range as 
an optional second parameter seems appropriate. Please file as an 
enhancement to bugzilla. If anyone has time to work on this, please do. 
If not, I'll work on it as my schedule allows.

Andrei
Feb 07 2011
parent Tomek =?ISO-8859-2?B?U293afFza2k=?= <just ask.me> writes:
Andrei Alexandrescu napisa=B3:


 For the latter, Tomek's idea of passing an output range as=20
 an optional second parameter seems appropriate. Please file as an=20
 enhancement to bugzilla. If anyone has time to work on this, please do.=20
 If not, I'll work on it as my schedule allows.


http://d.puremagic.com/issues/show_bug.cgi?id=3D5548

--=20
Tomek
Feb 08 2011