www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - MIR vs. Numpy

reply Tobias Schmidt <tobias.ts.schmidt fau.de> writes:
Dear all,

to compare MIR and Numpy in the HPC context, we implemented a 
multigrid solver in Python using Numpy and in D using Mir and 
perforemd some benchmarks with them.

You can find our code and results here:
https://github.com/typohnebild/numpy-vs-mir

Feedback is very welcome. Please feel free to open issues, pull 
requests or simply post your thoughts below.

Kind regards,
Tobias
Nov 18 2020
next sibling parent reply Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Wednesday, 18 November 2020 at 10:05:06 UTC, Tobias Schmidt 
wrote:

 Dear all,

 to compare MIR and Numpy in the HPC context, we implemented a 
 multigrid solver in Python using Numpy and in D using Mir and 
 perforemd some benchmarks with them.

 You can find our code and results here:
 https://github.com/typohnebild/numpy-vs-mir


Nice numbers. I’m not a Python guy but I was under the impression 
that Numpy actually is written in C, so that when you benchmark 
Numpy you’re mostly benchmarking C, not Python. Therefore I had 
expected the Numpy performance to be much closer to D’s. An 
important factor I think, which I’m not sure you have discussed 
(didn’t look too closely), is the compiler backend that was used 
to compile D and Numpy. Then again, as a user one is mostly 
interested in the out-of-the-box performance, which this seems to 
be a good measure of.

— Bastiaan.
Nov 18 2020
parent John Colvin <john.loughran.colvin gmail.com> writes:
On Wednesday, 18 November 2020 at 13:01:42 UTC, Bastiaan Veelo 
wrote:

 On Wednesday, 18 November 2020 at 10:05:06 UTC, Tobias Schmidt 
 wrote:

 Dear all,

 to compare MIR and Numpy in the HPC context, we implemented a 
 multigrid solver in Python using Numpy and in D using Mir and 
 perforemd some benchmarks with them.

 You can find our code and results here:
 https://github.com/typohnebild/numpy-vs-mir


 Nice numbers. I’m not a Python guy but I was under the 
 impression that Numpy actually is written in C, so that when 
 you benchmark Numpy you’re mostly benchmarking C, not Python. 
 Therefore I had expected the Numpy performance to be much 
 closer to D’s. An important factor I think, which I’m not sure 
 you have discussed (didn’t look too closely), is the compiler 
 backend that was used to compile D and Numpy. Then again, as a 
 user one is mostly interested in the out-of-the-box 
 performance, which this seems to be a good measure of.

 — Bastiaan.


A lot of numpy is in C, C++, fortran, asm etc....

But when you chain a bunch of things together, you are going via 
python. The language boundary (and python being slow) means that 
internal iteration in native code is a requirement for 
performance, which leads to eager allocation for composability 
via python, which then hurts performance. Numpy makes a very good 
effort, but is always constrained by this. Clever schemes with 
laziness where operations in python are actually just composing 
operations for execution later/on-demand can work as an 
alternative, but a) that's hard and b) even if you can completely 
avoid calling back in to python during iteration you would still 
need JIT to really unlock the performance.

Julia fixes this by having all/most in one language which is JIT'd

D can do the same with templates AOT, like C++/Eigen does but 
more flexible and less terrifying code. That's (one part of) what 
mir provides.
Nov 18 2020
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 18 November 2020 at 10:05:06 UTC, Tobias Schmidt 
wrote:

 Dear all,

 to compare MIR and Numpy in the HPC context, we implemented a 
 multigrid solver in Python using Numpy and in D using Mir and 
 perforemd some benchmarks with them.

 You can find our code and results here:
 https://github.com/typohnebild/numpy-vs-mir

 Feedback is very welcome. Please feel free to open issues, pull 
 requests or simply post your thoughts below.

 Kind regards,
 Tobias


Very nice write up.

It's been a while since I've used numba, so I was a little 
confused on the numba 1 and numba 8 runs.

It also looks like you are compiling on ldc with -mcpu=native 
--boundscheck=off. Why not -O as well?
Nov 18 2020
next sibling parent reply 9il <ilyayaroshenko gmail.com> writes:
On Wednesday, 18 November 2020 at 13:14:37 UTC, jmh530 wrote:

 On Wednesday, 18 November 2020 at 10:05:06 UTC, Tobias Schmidt 
 wrote:

 It also looks like you are compiling on ldc with -mcpu=native 
 --boundscheck=off. Why not -O as well?


-O is added by DUB
Nov 18 2020
next sibling parent Max Haughton <maxhaton gmail.com> writes:
On Wednesday, 18 November 2020 at 15:20:19 UTC, 9il wrote:

 On Wednesday, 18 November 2020 at 13:14:37 UTC, jmh530 wrote:

 On Wednesday, 18 November 2020 at 10:05:06 UTC, Tobias Schmidt 
 wrote:

 It also looks like you are compiling on ldc with -mcpu=native 
 --boundscheck=off. Why not -O as well?


 -O is added by DUB


Just -O? LDC is quite impressive with lto and 
cross-module-inlining turned on
Nov 18 2020
prev sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 18 November 2020 at 15:20:19 UTC, 9il wrote:

 [snip]

 -O is added by DUB


Ah, the -release-nobounds
Nov 18 2020
prev sibling parent Tobias Schmidt <tobias.ts.schmidt fau.de> writes:
Thanks for all of your feedback!

On Wednesday, 18 November 2020 at 13:14:37 UTC, jmh530 wrote:

 It's been a while since I've used numba, so I was a little 
 confused on the numba 1 and numba 8 runs.


The number was meant as the number of used threads in our runs. 
The prefix 'numba' is indicating if numba was used (numba) or not 
(nonumba).
We have added a section to clarify this. Thanks for the hint.
Nov 20 2020
prev sibling parent 9il <ilyayaroshenko gmail.com> writes:
On Wednesday, 18 November 2020 at 10:05:06 UTC, Tobias Schmidt 
wrote:

 Dear all,

 to compare MIR and Numpy in the HPC context, we implemented a 
 multigrid solver in Python using Numpy and in D using Mir and 
 perforemd some benchmarks with them.

 You can find our code and results here:
 https://github.com/typohnebild/numpy-vs-mir

 Feedback is very welcome. Please feel free to open issues, pull 
 requests or simply post your thoughts below.

 Kind regards,
 Tobias


Thank you a lot! It is a huge benefit for Mir and D to have so 
quality benchmarks.

Python's sweep_3D access memory only once for one element 
computation, while old D's sweep_slice access it 7 times.

A PR [1] for new version of sweep_slice was added, I expect it 
will be at least twice faster. The new sweep_slice uses a more 
D'sh approach and single memory access to the computation element.

[1] https://github.com/typohnebild/numpy-vs-mir/pull/1

Cheers,
Ilya
Nov 18 2020