www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - best way to convert string array of floats to a float array

reply "Stephen Jones" <siwenjo gmail.com> writes:
When it comes to reading float data from a text file into a float 
array is there a better option than reading the data into a 
string array and then stepping though the array and converting 
the values into a float array:

string[] textData=read text data;
float[] f;

foreach(string s; textData){
     f~=to!(float)(s);
}
Jul 23 2012
parent reply "Brad Anderson" <eco gnuk.net> writes:
On Tuesday, 24 July 2012 at 02:03:40 UTC, Stephen Jones wrote:

 When it comes to reading float data from a text file into a 
 float array is there a better option than reading the data into 
 a string array and then stepping though the array and 
 converting the values into a float array:

 string[] textData=read text data;
 float[] f;

 foreach(string s; textData){
     f~=to!(float)(s);
 }


auto flist = textDataRange.map!(a => a.to!float).array();
Jul 23 2012
parent reply "Stephen Jones" <siwenjo gmail.com> writes:
On Tuesday, 24 July 2012 at 02:05:18 UTC, Brad Anderson wrote:

 On Tuesday, 24 July 2012 at 02:03:40 UTC, Stephen Jones wrote:

 When it comes to reading float data from a text file into a 
 float array is there a better option than reading the data 
 into a string array and then stepping though the array and 
 converting the values into a float array:

 string[] textData=read text data;
 float[] f;

 foreach(string s; textData){
    f~=to!(float)(s);
 }


 auto flist = textDataRange.map!(a => a.to!float).array();


Thanks Brad. This converts the string into an auto type, but I 
need to dump the data in the auto into a float array:

auto f=map!(to!(float))(split(chompPrefix(s, "lno:")));

float[] dat;
dat~=f;

The above won't compile complaining that "Cannot append type 
Result to type float[]"
Jul 24 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, July 25, 2012 04:06:57 Stephen Jones wrote:

 On Tuesday, 24 July 2012 at 02:05:18 UTC, Brad Anderson wrote:

 On Tuesday, 24 July 2012 at 02:03:40 UTC, Stephen Jones wrote:

 When it comes to reading float data from a text file into a
 float array is there a better option than reading the data
 into a string array and then stepping though the array and
 converting the values into a float array:
 
 string[] textData=read text data;
 float[] f;
 
 foreach(string s; textData){
 
    f~=to!(float)(s);
 
 }


 
 auto flist = textDataRange.map!(a => a.to!float).array();


 
 Thanks Brad. This converts the string into an auto type, but I
 need to dump the data in the auto into a float array:
 
 auto f=map!(to!(float))(split(chompPrefix(s, "lno:")));
 
 float[] dat;
 dat~=f;
 
 The above won't compile complaining that "Cannot append type
 Result to type float[]"


map returns a lazy range. If you want an array, pass the result to 
std.array.array.

- Jonathan M Davis
Jul 24 2012
parent reply "Stephen Jones" <siwenjo gmail.com> writes:
On Wednesday, 25 July 2012 at 03:23:52 UTC, Jonathan M Davis 
wrote:

 On Wednesday, July 25, 2012 04:06:57 Stephen Jones wrote:

 On Tuesday, 24 July 2012 at 02:05:18 UTC, Brad Anderson wrote:

 On Tuesday, 24 July 2012 at 02:03:40 UTC, Stephen Jones 
 wrote:

 When it comes to reading float data from a text file into a
 float array is there a better option than reading the data
 into a string array and then stepping though the array and
 converting the values into a float array:
 
 string[] textData=read text data;
 float[] f;
 
 foreach(string s; textData){
 
    f~=to!(float)(s);
 
 }


 
 auto flist = textDataRange.map!(a => a.to!float).array();


 
 Thanks Brad. This converts the string into an auto type, but I
 need to dump the data in the auto into a float array:
 
 auto f=map!(to!(float))(split(chompPrefix(s, "lno:")));
 
 float[] dat;
 dat~=f;
 
 The above won't compile complaining that "Cannot append type
 Result to type float[]"


 map returns a lazy range. If you want an array, pass the result 
 to
 std.array.array.

 - Jonathan M Davis


Thanks Jonathan. Very cool language this:

float[] dat;
dat~=array(map!(to!(float))(split(chompPrefix(s, "lno:"))));

The data is read from a string that contains floats, parsed by 
the split into an array, converted into an array of auto via the 
mapping with the to!float function, and then cast into the float 
array dat.

Longhand that data would be read into the program as a string, 
converted into an array of strings then converted into an array 
of floats and the initial string and the array of strings would 
both accumulate on the GC waiting list. Does the code above weed 
out some of the redundant data thereby reducing calls upon the GC?
Jul 24 2012
parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Wednesday, July 25, 2012 06:18:13 Stephen Jones wrote:

 Thanks Jonathan. Very cool language this:
 
 float[] dat;
 dat~=array(map!(to!(float))(split(chompPrefix(s, "lno:"))));
 
 The data is read from a string that contains floats, parsed by
 the split into an array, converted into an array of auto via the
 mapping with the to!float function, and then cast into the float
 array dat.
 
 Longhand that data would be read into the program as a string,
 converted into an array of strings then converted into an array
 of floats and the initial string and the array of strings would
 both accumulate on the GC waiting list. Does the code above weed
 out some of the redundant data thereby reducing calls upon the GC?


I'm not quite sure what you mean. Are you asking whether it tries to avoid 
unnecessary copies? It's pretty straightforward in what it does, but there 
_is_ some copying going on here.

chompPrefix slices s, so there's no copying.

split _does_ copy, since it returns a new array. Use std.algorithm.splitter if 
you want to avoid that copy.

to!float is going to create a float on the stack, so there's no heap alloction.

map is just a wrapper struct which sits on the stack. So, there's no copying 
going on there.

array is going to copy the data from map and allocate a new array, which is 
then appended to dat, which may or may not cause dat to be reallocated, 
depending on whether there are any slices after it or whatnot. In this 
particular case, it wouldn't have to, beacause dat is null, but you might as 
well assign it directly given the code that you give. If you're appending it 
to an array with actual data, then it depends on the state of that array as to 
whether reallocation is necessary. If you _really_ want to avoid reallocation 
in that case, then you'd do something more like

auto result = map!(to!float)(splitter(chompPrefix(s, "lno:")));
auto dat.reserve(walkLength(result));
foreach(e; result)
    dat ~= e;

though that would mean iterating over map twice, since map won't have length 
in this case (since splitter doesn't), which could actually result in worse 
performance, depending. But it does make it so that the only heap allocation 
that you might get is when calling reserve, and if dat happens to have a large 
enough capacity already, then there should be zero heap allocations here.

In most cases though, the single line is probably fine (especially if you 
change it to use splitter instead of split). I'd only do the second if I were 
really trying to eek all the performance out of it that I could.

- Jonathan M Davis
Jul 24 2012