www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - std.parallelism taskPool.map example throws exception

reply "Jay Norwood" <jayn prismnet.com> writes:
I tried to create a working example from the std.parallelism 
taskPool.map code, and it throws with empty strings with length 1 
being passed to to!double.  Anyone have a working example?  I'm 
building on Windows with 2.067.1 dmd.

import std.parallelism;
import std.algorithm;
import std.stdio;
import std.conv;
import std.math;
import std.range;
import std.file;

void main()
{

	auto fn = "numberList.txt";
	auto f = File(fn,"w");
	scope(exit) std.file.remove(fn);

	foreach (i ; iota(10.0,2_000.0)){
		f.writefln("%g",i+0.5);
	}

	f.close();

	auto lineRange = File(fn).byLine();
	auto dupedLines = std.algorithm.map!"a.idup"(lineRange);
	auto nums = taskPool.map!(to!double)(dupedLines);
	auto logs = taskPool.map!log10(nums);

	double sum = 0;
	foreach(elem; logs)
	{
		sum += elem;
	}
	writeln("sum=",sum);

}
Aug 07 2015
parent reply "Jay Norwood" <jayn prismnet.com> writes:
This appears to work ... at least, no exception:

	auto sm = File(fn).byLine(KeepTerminator.no)
		.map!"a.chomp"()
		.map!"a.idup"()
		.map!(to!double)
	    .map!"a.log10"()
		.sum();

	writeln("sum=",sm);
Aug 07 2015
parent reply "Jay Norwood" <jayn prismnet.com> writes:
This also works.

	auto sm = File(fn).byLineCopy()
		.map!"a.chomp"()
		.map!(to!double)
	    .map!"a.log10"()
		.sum();

	writeln("sum=",sm);
Aug 07 2015
parent reply "Jay Norwood" <jayn prismnet.com> writes:
and, finally, this works using the taskPool.map, as in the 
std.parallelism example.  So, the trick appears to be that the 
call to chomp is needed.

	auto lineRange = File(fn).byLineCopy();
	auto chomped = std.algorithm.map!"a.chomp"(lineRange);
	auto nums = taskPool.map!(to!double)(chomped);
	auto logs = taskPool.map!log10(nums);

	double sum = 0;
	foreach(elem; logs)
	{
		sum += elem;
	}
	writeln("sum=",sum);
Aug 07 2015
parent "Jay Norwood" <jayn prismnet.com> writes:
Unfortunately, this is not a very good example for 
std.parallelism, since the measured times are better using the 
std.algorithm.map calls. I know from past experience that 
std.parallelism routines can work well when the work is spread 
out correctly, so this example could be improved.

This is parallel
D:\visd\map\map\Release>map
sum=1.17335e+07
time msecs:1242

Non-parallel
D:\visd\map\map\Release>map
sum=1.17335e+07
time msecs:970

I think this example

import std.parallelism;
import std.algorithm;
import std.stdio;
import std.conv;
import std.math;
import std.range;
import std.file;
import std.datetime;

void main()
{

	auto fn = "numberList.txt";
	auto f = File(fn,"w");
	scope(exit) std.file.remove(fn);

	foreach (i ; iota(10.0,2_000_000.0)){
		f.writefln("%g",i+0.5);
	}

	f.close();
	std.datetime.StopWatch sw;
	sw.start();

	auto lineRange = File(fn).byLineCopy();
	auto chomped = std.algorithm.map!"a.chomp"(lineRange);
	auto nums = std.algorithm.map!(to!double)(chomped);
	auto logs = std.algorithm.map!log10(nums);

	double sum = 0;
	foreach(elem; logs)
	{
		sum += elem;
	}

	long tm = sw.peek().msecs;
	writeln("sum=",sum);
	writeln("time msecs:", tm);

}
Aug 07 2015