digitalmars.D - Bug? taskPool.map() with bufSize and writeln() gets stuck
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (33/33) Feb 10 2012 Sorry for the double-post; I have asked the same question on D.learn
- MattCodr (8/42) Feb 10 2012 For what I see, maybe that's because it's a "non-random access
- Martin Nowak (7/41) Feb 11 2012 s =
- =?UTF-8?B?QWxpIMOHZWhyZWxp?= (33/82) Feb 11 2012 deadlock
- Martin Nowak (10/95) Feb 11 2012 rn
Sorry for the double-post; I have asked the same question on D.learn
earlier but I think this is more of a question to this forum.
Tested on Ubuntu 11.10 64-bit dmd.
The following program gets stuck during the writeln() call.
- Note that the foo() call alone works fine.
- Also note that the program works fine when there is no writeln() call
nor foo() call. All elements get processed in that case and the results
are ignored.
Am I using taskPool.map incorrectly or is this a bug? Can you help
identify where the problem may be? How is writeln() using the range
differently than foo() to cause this behavior?
import std.stdio;
import std.parallelism;
import core.thread;
int func(int i)
{
writeln("processing ", i);
return i;
}
void main()
{
auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
writeln(results); // <-- Gets stuck HERE
foo(results); // this works fine
}
void foo(R)(R range)
{
for ( ; !range.empty; range.popFront()) {
writeln(range.front);
}
}
Thank you,
Ali
Feb 10 2012
"MattCodr" <matheus_nab hotmail.com>
On Saturday, 11 February 2012 at 01:31:29 UTC, Ali Çehreli wrote:
Sorry for the double-post; I have asked the same question on
D.learn earlier but I think this is more of a question to this
forum.
Tested on Ubuntu 11.10 64-bit dmd.
The following program gets stuck during the writeln() call.
- Note that the foo() call alone works fine.
- Also note that the program works fine when there is no
writeln() call nor foo() call. All elements get processed in
that case and the results are ignored.
Am I using taskPool.map incorrectly or is this a bug? Can you
help identify where the problem may be? How is writeln() using
the range differently than foo() to cause this behavior?
import std.stdio;
import std.parallelism;
import core.thread;
int func(int i)
{
writeln("processing ", i);
return i;
}
void main()
{
auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2);
writeln(results); // <-- Gets stuck HERE
foo(results); // this works fine
}
void foo(R)(R range)
{
for ( ; !range.empty; range.popFront()) {
writeln(range.front);
}
}
Thank you,
Ali
For what I see, maybe that's because it's a "non-random access
ranges". So to access "results", you need access from it's
aggregate...
try this inside your main:
foreach(int r; results)
writeln(r);
// foo(results); // this works fin
Feb 10 2012
On Sat, 11 Feb 2012 02:31:29 +0100, Ali =C3=87ehreli <acehreli yahoo.com=wrote:Sorry for the double-post; I have asked the same question on D.learn =earlier but I think this is more of a question to this forum. Tested on Ubuntu 11.10 64-bit dmd. The following program gets stuck during the writeln() call. - Note that the foo() call alone works fine. - Also note that the program works fine when there is no writeln() cal=l =nor foo() call. All elements get processed in that case and the result=s =are ignored. Am I using taskPool.map incorrectly or is this a bug? Can you help =identify where the problem may be? How is writeln() using the range =differently than foo() to cause this behavior? import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); return i; } void main() { auto results =3D taskPool.map!func([1,2,3,4,5,6,7,8], 2); writeln(results); // <-- Gets stuck HERE foo(results); // this works fine } void foo(R)(R range) { for ( ; !range.empty; range.popFront()) { writeln(range.front); } } Thank you, AliYeah, you have a deadlock in there, it's somewhat hidden though. The issue is that writeln will take lock on stdout once. This will deadl= ock with the lazy processing of the map range.
Feb 11 2012
On 02/11/2012 12:56 AM, Martin Nowak wrote:On Sat, 11 Feb 2012 02:31:29 +0100, Ali Çehreli <acehreli yahoo.com>wrote:deadlockSorry for the double-post; I have asked the same question on D.learn earlier but I think this is more of a question to this forum. Tested on Ubuntu 11.10 64-bit dmd. The following program gets stuck during the writeln() call. - Note that the foo() call alone works fine. - Also note that the program works fine when there is no writeln() call nor foo() call. All elements get processed in that case and the results are ignored. Am I using taskPool.map incorrectly or is this a bug? Can you help identify where the problem may be? How is writeln() using the range differently than foo() to cause this behavior? import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); return i; } void main() { auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); writeln(results); // <-- Gets stuck HERE foo(results); // this works fine } void foo(R)(R range) { for ( ; !range.empty; range.popFront()) { writeln(range.front); } } Thank you, AliYeah, you have a deadlock in there, it's somewhat hidden though. The issue is that writeln will take lock on stdout once. This willwith the lazy processing of the map range.Thank you. I was trying to visualize the semi-lazy nature of taskPool.map. Now I get what I want when the writeln() call in main() is changed to be on stderr: import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); Thread.sleep(dur!"seconds"(1)); return i; } void main() { auto results = taskPool.map!func([1,2,3,4,5,6,7,8], 2); stderr.writeln(results); // <-- now on stderr } Now the output hints at how taskPool.map is semi-lazy: processing 1 processing 2 [processing 3 1, 2processing 4 , 3processing 5 , 4processing 6 , 5, 6processing 7 processing 8 , 7, 8] Good. :) Ali
Feb 11 2012
On Sat, 11 Feb 2012 17:18:21 +0100, Ali =C3=87ehreli <acehreli yahoo.com=wrote:On 02/11/2012 12:56 AM, Martin Nowak wrote: > On Sat, 11 Feb 2012 02:31:29 +0100, Ali =C3=87ehreli <acehreli yaho=o.com> =wrote: > >> Sorry for the double-post; I have asked the same question on D.lea=rn>> earlier but I think this is more of a question to this forum. >> >> Tested on Ubuntu 11.10 64-bit dmd. >> >> The following program gets stuck during the writeln() call. >> >> - Note that the foo() call alone works fine. >> >> - Also note that the program works fine when there is no writeln()=>> call nor foo() call. All elements get processed in that case and t=he>> results are ignored. >> >> Am I using taskPool.map incorrectly or is this a bug? Can you help=>> identify where the problem may be? How is writeln() using the rang=e>> differently than foo() to cause this behavior? >> >> import std.stdio; >> import std.parallelism; >> import core.thread; >> >> int func(int i) >> { >> writeln("processing ", i); >> return i; >> } >> >> void main() >> { >> auto results =3D taskPool.map!func([1,2,3,4,5,6,7,8], 2); >> >> writeln(results); // <-- Gets stuck HERE >> >> foo(results); // this works fine >> } >> >> void foo(R)(R range) >> { >> for ( ; !range.empty; range.popFront()) { >> writeln(range.front); >> } >> } >> >> Thank you, >> Ali > > Yeah, you have a deadlock in there, it's somewhat hidden though. > The issue is that writeln will take lock on stdout once. This will ==deadlock > with the lazy processing of the map range. Thank you. I was trying to visualize the semi-lazy nature of taskPool.map. Now I ==get what I want when the writeln() call in main() is changed to be on ==stderr: import std.stdio; import std.parallelism; import core.thread; int func(int i) { writeln("processing ", i); Thread.sleep(dur!"seconds"(1)); return i; } void main() { auto results =3D taskPool.map!func([1,2,3,4,5,6,7,8], 2); stderr.writeln(results); // <-- now on stderr } Now the output hints at how taskPool.map is semi-lazy: processing 1 processing 2 [processing 3 1, 2processing 4 , 3processing 5 , 4processing 6 , 5, 6processing 7 processing 8 , 7, 8] Good. :) AliGoing with foreach(e; results) will work too and you can write to the = right output.
Feb 11 2012