www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - One way to deal with transient ranges & char[] buffers

reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
Recently, I discovered an interesting idiom for dealing with transient
ranges, esp. w.r.t. strings / char[]. Many places in D require string,
but sometimes what you have is char[] which can't be converted to string
except by .idup. But you don't want to .idup in generic code, because if
the input's already a string, then it's wasteful duplication. The
solution is to do this:

	void func(S)(S input) if (isSomeString!S) {
		string x = to!string(input);
		... // use at will
	}

The nice thing about this is that if input is already a string,
to!string does nothing, and if it's char[] or const(char)[], to!()
handles calling .idup for you so you don't have to pollute generic code
with it.

When writing a generic function that takes a range of some string, and
you're not sure if the range is transient or not, the same trick helps
ensure that you don't run into transience-related problems:

	auto func(R)(R range) if (isSomeString!(ElementType!S)) {
		struct WrapperRange {
			...
			auto front() {
				// This ensures we don't run into
				// transience related problems, and that
				// the range we return will *not* be
				// transient.
				return range.front.to!string();
			}
		}
		return WrapperRange(...);
	}


T

-- 
Freedom: (n.) Man's self-given right to be enslaved by his own depravity.
Aug 01 2013
next sibling parent reply "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Friday, 2 August 2013 at 05:35:28 UTC, H. S. Teoh wrote:

 Recently, I discovered an interesting idiom for dealing with 
 transient
 ranges, esp. w.r.t. strings / char[]. Many places in D require 
 string,
 but sometimes what you have is char[] which can't be converted 
 to string
 except by .idup. But you don't want to .idup in generic code, 
 because if
 the input's already a string, then it's wasteful duplication.


Places in D that require `string` either do so because they need 
the immutable guarantee or they do so out of error (e.g. should 
have used a string of const characters instead). The latter can 
of course be worked around, but the only *solution* involves 
fixing the upstream code, so I'll assume we're discussing the 
former case.

We don't have any generic mechanism for deep copying ranges. The 
`save` primitive is often implemented by means of copying, but 
conceptually is doing something very different, so it cannot be 
applied here. So, I don't see how your idea translates to ranges 
in general (not completely sure if it was intended to).

Thus, let's tackle the case of arrays/slices in particular, of 
which strings are the most common example. There is a precedent 
in D to push to the decision to copy an array upwards in the 
code. When the operations at hand require the immutable 
guarantee, state it in the interface of the code, such as by 
asking for `string` on a function's parameter. That's why so many 
functions take `string` when they need the immutable guarantee, 
as opposed to `const(char)[]` or a template parameter, followed 
by a GC copy operation. This way, copies are not only minimized, 
but centralized more in user code where they are more visible, 
and the method of making the copy - remember, not all client code 
is fine with rampant GC use - is also pushed up. Also, copies are 
one thing, but what if the caller had a string but in a different 
encoding? Not only does an allocation have to be made, but 
decoding and encoding is also necessary; the details of how to 
handle this are also pushed up, with the same benefits. It's a 
pretty mainstream idiom and is often reiterated by members of the 
community, such as in Ali's talk at dconf.

Your proposed solution only shares one benefit with the solution 
described above - that if the direct caller had a `string` 
already (or a range of `string`s), nothing has to be done. It 
forfeits all the other benefits for convenience. It also has 
problems with template bloat, which can be fixed but at a 
syntactical cost.

Overall I think it reduces the genericity of algorithms by trying 
to handle input types it doesn't actually support, which can be a 
big problem for performance-critical code.
Aug 02 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 2 August 2013 at 07:50:28 UTC, Jakob Ovrum wrote:

 Places in D that require `string` either do so because they 
 need the immutable guarantee or they do so out of error (e.g. 
 should have used a string of const characters instead). The 
 latter can of course be worked around, but the only *solution* 
 involves fixing the upstream code, so I'll assume we're 
 discussing the former case.


One of the problems is often the return type. For example, readln 
will return a string, because it is simply more convenient in end 
user code. The "standard" string format in D is "string", so 
that's what it returns.

However, if you call readln!(char[]), then you'll the *exact* 
same string, but of a non-immutable type.

This means that you *can* get the best out of both worlds, at no 
extra run-time cost. I think more functions should do this.

Without doing this, you face the eternal problem: Should I return 
a "string", to give my end user more guarantees, when in fact my 
char array is perfectly mutable, or should I return a char[], 
forcing my end user to make an idup(or an unsafe cast) if he 
actually needed a string?

It's a tough problem to tackle.
Aug 02 2013
parent reply "Tobias Pankrath" <tobias pankrath.net> writes:
On Friday, 2 August 2013 at 08:46:18 UTC, monarch_dodra wrote:

 Without doing this, you face the eternal problem: Should I 
 return a "string", to give my end user more guarantees, when in 
 fact my char array is perfectly mutable, or should I return a 
 char[], forcing my end user to make an idup(or an unsafe cast) 
 if he actually needed a string?

 It's a tough problem to tackle.


The solution has been posted to this newsgroup already in form of 
unique/unaliased types. Thats THE selling point for them.

If you have an array of char and it could be char[] as well as 
string, than let the type system infer the right one for you.
Aug 02 2013
parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 2 August 2013 at 11:19:05 UTC, Tobias Pankrath wrote:

 On Friday, 2 August 2013 at 08:46:18 UTC, monarch_dodra wrote:

 Without doing this, you face the eternal problem: Should I 
 return a "string", to give my end user more guarantees, when 
 in fact my char array is perfectly mutable, or should I return 
 a char[], forcing my end user to make an idup(or an unsafe 
 cast) if he actually needed a string?

 It's a tough problem to tackle.


 The solution has been posted to this newsgroup already in form 
 of unique/unaliased types. Thats THE selling point for them.

 If you have an array of char and it could be char[] as well as 
 string, than let the type system infer the right one for you.


Interesting. Thanks for the tip. I'll look these up.
Aug 02 2013
parent "Tobias Pankrath" <tobias pankrath.net> writes:
On Friday, 2 August 2013 at 11:27:48 UTC, monarch_dodra wrote:

 On Friday, 2 August 2013 at 11:19:05 UTC, Tobias Pankrath wrote:

 On Friday, 2 August 2013 at 08:46:18 UTC, monarch_dodra wrote:

 Without doing this, you face the eternal problem: Should I 
 return a "string", to give my end user more guarantees, when 
 in fact my char array is perfectly mutable, or should I 
 return a char[], forcing my end user to make an idup(or an 
 unsafe cast) if he actually needed a string?

 It's a tough problem to tackle.


 The solution has been posted to this newsgroup already in form 
 of unique/unaliased types. Thats THE selling point for them.

 If you have an array of char and it could be char[] as well as 
 string, than let the type system infer the right one for you.


 Interesting. Thanks for the tip. I'll look these up.


http://www.digitalmars.com/d/archives/digitalmars/D/Immutable_and_unique_in_C_180572.html
Aug 02 2013
prev sibling parent reply "monarch_dodra" <monarchdodra gmail.com> writes:
On Friday, 2 August 2013 at 05:35:28 UTC, H. S. Teoh wrote:

 	void func(S)(S input) if (isSomeString!S) {
 		string x = to!string(input);
 		... // use at will
 	}


+1. I saw this used recently, and I find it very clever.
Aug 02 2013
parent "Jakob Ovrum" <jakobovrum gmail.com> writes:
On Friday, 2 August 2013 at 08:37:50 UTC, monarch_dodra wrote:

 On Friday, 2 August 2013 at 05:35:28 UTC, H. S. Teoh wrote:

 	void func(S)(S input) if (isSomeString!S) {
 		string x = to!string(input);
 		... // use at will
 	}


 +1. I saw this used recently, and I find it very clever.


It's really not. It takes the decision to make a GC-allocated 
copy away from the client code in return for some minor 
convenience.
Aug 02 2013