digitalmars.D.learn - Fold in Parallelism
- Vino (20/20) Dec 17 2017 HI All,
- =?UTF-8?Q?Ali_=c3=87ehreli?= (4/8) Dec 17 2017 fold has only been added to std.algorithm. Opened an enhancement request...
- Vino (6/18) Dec 18 2017 Hi Ali,
- =?UTF-8?Q?Ali_=c3=87ehreli?= (43/67) Dec 18 2017 The implementation is almost a copy+paste from std.algorithm.fold. You
- Russel Winder (14/79) Dec 18 2017 Ali,
- Vino (10/20) Dec 19 2017 Hi Ali,
- =?UTF-8?Q?Ali_=c3=87ehreli?= (5/6) Dec 20 2017 The change was a little more complicated than my naive adaptation from
- Russel Winder (10/21) Dec 21 2017 Thanks for doing this. Having consistency is a good thing.
- Vino (6/12) Dec 21 2017 Hi Ali,
HI All,
As per the document form std.parallelism it states that we can
use taskPool.reduce so can we use the same for fold
(taskPool.fold) as basically both are same with slight variation
on seed values, if possible can can we define the same in the
below lines
Tried the below but getting an error
auto SdFiles = Array!ulong(dirEntries("C:\\TEMP\\BACKUP",
SpanMode.depth).map!(a => a.size).taskPool.fold!((a,b) => a + b)
(x))[];
Error : Srvnscleaner.d(89): Error: function
std.parallelism.taskPool () is not callable using argument types
(MapResult!(__lambda2, DirIterator))
auto SdFiles = Array!ulong(dirEntries("C:\\TEMP\\BACKUP",
SpanMode.depth).map!(a => a.size).fold!((a,b) => a + b)
(x)).taskPool[];
Error : Size.d(89): Error: function std.parallelism.taskPool ()
is not callable using argument types (Array!ulong)
From,
Vino.B
Dec 17 2017
On 12/17/2017 08:11 AM, Vino wrote:As per the document form std.parallelism it states that we can use taskPool.reduce so can we use the same for fold (taskPool.fold) as basically both are same with slight variation on seed values, if possible can can we define the same in the below linesfold has only been added to std.algorithm. Opened an enhancement request: https://issues.dlang.org/show_bug.cgi?id=18096 Ali
Dec 17 2017
On Sunday, 17 December 2017 at 20:00:53 UTC, Ali Çehreli wrote:On 12/17/2017 08:11 AM, Vino wrote:Hi Ali, Thank you very much, may we know if possible as to when this would be added. From, Vino.BAs per the document form std.parallelism it states that wecan usetaskPool.reduce so can we use the same for fold(taskPool.fold) asbasically both are same with slight variation on seed values,ifpossible can can we define the same in the below linesfold has only been added to std.algorithm. Opened an enhancement request: https://issues.dlang.org/show_bug.cgi?id=18096 Ali
Dec 18 2017
On 12/18/2017 02:18 AM, Vino wrote:On Sunday, 17 December 2017 at 20:00:53 UTC, Ali Çehreli wrote:The implementation is almost a copy+paste from std.algorithm.fold. You can simply copy the following fold template to your project and start using it: import std.parallelism; import std.stdio; int adder(int result, int element) { return result + element; } template fold(fun...) if (fun.length >= 1) { import std.parallelism : TaskPool; auto fold(R, S...)(TaskPool t, R r, S seed) { static if (S.length < 2) { return t.reduce!fun(seed, r); } else { import std.typecons : tuple; return t.reduce!fun(tuple(seed), r); } } } unittest { import std.range; const N = 10_000; const expected = N * (N - 1) / 2; foreach (numThreads; 1 .. 100) { auto t = new TaskPool(numThreads); const result = t.fold!adder(N.iota); assert(result == expected); t.finish(); } } void main() { } If you want to provide an explicit seed value, good luck making sense of how it affects the final result. :) (The documentation of TaskPool.reduce explains it but I find it too involved to understand.) AliOn 12/17/2017 08:11 AM, Vino wrote:Hi Ali, Thank you very much, may we know if possible as to when this would be added. From, Vino.BAs per the document form std.parallelism it states that wecan usetaskPool.reduce so can we use the same for fold(taskPool.fold) asbasically both are same with slight variation on seed values,ifpossible can can we define the same in the below linesfold has only been added to std.algorithm. Opened an enhancement request: https://issues.dlang.org/show_bug.cgi?id=18096 Ali
Dec 18 2017
Ali, Shouldn't this be a pull request for std.parallelism to be extended?=20 If the function is in std.algorithm, then people should not have to write it for themselves in std.parallelism. On Mon, 2017-12-18 at 11:01 -0800, Ali =C3=87ehreli via Digitalmars-d-learn wrote:=20[=E2=80=A6]--=20 Russel. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.ukHi Ali, =20 Thank you very much, may we know if possible as to when this would be=20 added. =20 From, Vino.B=20 The implementation is almost a copy+paste from std.algorithm.fold. You=20 can simply copy the following fold template to your project and start=20 using it: =20 import std.parallelism; import std.stdio; =20 int adder(int result, int element) { return result + element; } =20 template fold(fun...) if (fun.length >=3D 1) { import std.parallelism : TaskPool; auto fold(R, S...)(TaskPool t, R r, S seed) { static if (S.length < 2) { return t.reduce!fun(seed, r); } else { import std.typecons : tuple; return t.reduce!fun(tuple(seed), r); } } } =20 unittest { import std.range; =20 const N =3D 10_000; const expected =3D N * (N - 1) / 2; =20 foreach (numThreads; 1 .. 100) { auto t =3D new TaskPool(numThreads); const result =3D t.fold!adder(N.iota); assert(result =3D=3D expected); t.finish(); } } =20 void main() { } =20 If you want to provide an explicit seed value, good luck making sense of=20 how it affects the final result. :) (The documentation of=20 TaskPool.reduce explains it but I find it too involved to understand.) =20 Ali
Dec 18 2017
On Monday, 18 December 2017 at 20:53:28 UTC, Russel Winder wrote:Ali, Shouldn't this be a pull request for std.parallelism to be extended? If the function is in std.algorithm, then people should not have to write it for themselves in std.parallelism. On Mon, 2017-12-18 at 11:01 -0800, Ali Çehreli via Digitalmars-d-learn wrote:Hi Ali, Sorry, I would like to echo the statement from Russel, as an user(newbie) I personally do not want to fiddle around the libraries even though it is a simple code copy+paste, as this might cause some serious issue in case if anything went wrong, so i would like to leave it to expert's like you to help us. If possible can we added this to the next release (78). From, Vino.B.[...][…][...]
Dec 19 2017
On 12/19/2017 02:32 AM, Vino wrote:even though it is a simple code copy+pasteThe change was a little more complicated than my naive adaptation from std.algorithm.fold. Here is the pull request: https://github.com/dlang/phobos/pull/5951 Ali
Dec 20 2017
Russel Winder <russel winder.org.uk>
On Wed, 2017-12-20 at 22:31 -0800, Ali =C3=87ehreli via Digitalmars-d-learn
wrote:
On 12/19/2017 02:32 AM, Vino wrote:
=20
> even though it is a simple code copy+paste
=20
The change was a little more complicated than my naive adaptation
from=20
std.algorithm.fold. Here is the pull request:
=20
https://github.com/dlang/phobos/pull/5951
=20
Ali
Thanks for doing this. Having consistency is a good thing.
--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Dr Russel Winder t: +44 20 7585 2200
41 Buckmaster Road m: +44 7770 465 077
London SW11 1EN, UK w: www.russel.org.uk
Dec 21 2017
On Thursday, 21 December 2017 at 06:31:52 UTC, Ali Çehreli wrote:On 12/19/2017 02:32 AM, Vino wrote:Hi Ali, Thank you very much, the pull request is in open state, so can you please let me know when can we can test the same. From, Vino.Beven though it is a simple code copy+pasteThe change was a little more complicated than my naive adaptation from std.algorithm.fold. Here is the pull request: https://github.com/dlang/phobos/pull/5951 Ali
Dec 21 2017
On Friday, 22 December 2017 at 00:12:45 UTC, Vino wrote:On Thursday, 21 December 2017 at 06:31:52 UTC, Ali Çehreli wrote:It will take a couple of days for this pull request to reach a ready form and to be approved. The best way to help it to move forward is to review it yourself or at least vote for it on GitHub ;-) Once merged it will appear on dmd-nightly on the next day, but I'm not sure why you depend on his pull request? Can't you just use the code directly?On 12/19/2017 02:32 AM, Vino wrote:Hi Ali, Thank you very much, the pull request is in open state, so can you please let me know when can we can test the same. From, Vino.Beven though it is a simple code copy+pasteThe change was a little more complicated than my naive adaptation from std.algorithm.fold. Here is the pull request: https://github.com/dlang/phobos/pull/5951 Ali
Dec 21 2017
On Friday, 22 December 2017 at 00:18:40 UTC, Seb wrote:On Friday, 22 December 2017 at 00:12:45 UTC, Vino wrote:Hi Seb, I am fine waiting for a couple of days for this pull request to reach to teh ready form and approved. Sure will vote on GItHub. From, Vino.BOn Thursday, 21 December 2017 at 06:31:52 UTC, Ali Çehreli wrote:It will take a couple of days for this pull request to reach a ready form and to be approved. The best way to help it to move forward is to review it yourself or at least vote for it on GitHub ;-) Once merged it will appear on dmd-nightly on the next day, but I'm not sure why you depend on his pull request? Can't you just use the code directly?[...]Hi Ali, Thank you very much, the pull request is in open state, so can you please let me know when can we can test the same. From, Vino.B
Dec 21 2017