www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Any Idea how Fast is the dlang dataframe library ?

reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
Dear,

Today I seen a powerful python library written in Rust.
homepage: https://pypi.org/project/polars/
benchmark: https://pypi.org/project/polars/

I remember that they are one year or two they had a sponsored D 
project on this topic.

So the project, is it …
i) still in development ?
ii) table ?
iii) better than polars and others ?


Thanks


Best regards
Jan 06 2022
parent reply bioinfornatics <bioinfornatics fedoraproject.org> writes:
On Thursday, 6 January 2022 at 23:05:22 UTC, bioinfornatics wrote:

 Dear,

 Today I seen a powerful python library written in Rust.
 homepage: https://pypi.org/project/polars/
 benchmark: https://pypi.org/project/polars/

 I remember that they are one year or two they had a sponsored D 
 project on this topic.

 So the project, is it …
 i) still in development ?
 ii) table ?
 iii) better than polars and others ?


 Thanks


 Best regards


*stable*
Jan 06 2022
parent reply Tejas <notrealemail gmail.com> writes:
On Thursday, 6 January 2022 at 23:06:20 UTC, bioinfornatics wrote:

 On Thursday, 6 January 2022 at 23:05:22 UTC, bioinfornatics 
 wrote:

 Dear,

 Today I seen a powerful python library written in Rust.
 homepage: https://pypi.org/project/polars/
 benchmark: https://pypi.org/project/polars/

 I remember that they are one year or two they had a sponsored 
 D project on this topic.

 So the project, is it …
 i) still in development ?
 ii) table ?
 iii) better than polars and others ?


 Thanks


 Best regards


 *stable*


I believe you're talking about 
[magpie](https://code.dlang.org/packages/magpie)

The project has been declared completed, but the author moved on 
and nobody has touched it since then, I think

Don't know about performance, most likely inferior than anything 
that's being developed and released _today_, but still, it's best 
if you benchmark it yourself
Jan 06 2022
parent reply jmh530 <john.michael.hall gmail.com> writes:
On Friday, 7 January 2022 at 04:04:44 UTC, Tejas wrote:

 [snip]

 I believe you're talking about 
 [magpie](https://code.dlang.org/packages/magpie)

 The project has been declared completed, but the author moved 
 on and nobody has touched it since then, I think

 Don't know about performance, most likely inferior than 
 anything that's being developed and released _today_, but 
 still, it's best if you benchmark it yourself


I had provided some feedback to the author when he was working on 
it and it does some interesting things. I feel like it really 
needs someone who cares about doing analysis in D with Dataframes 
to take over and continue making improvements. An evaluation of 
the performance capabilities would help identify what is good 
about magpie and what would need to get improved.

Ideally, some of this work would get incorporated into mir. Mir 
has some basic time series functionality and you can make 
homogenous Dataframes (although there might be some functionality 
that is missing compared to magpie). However, mir does not really 
have an easy way to handle heterogenous types in the columns. 
Magpie does and it is important for many practical uses of 
Dataframes.
Jan 07 2022
parent reply Sergey <kornburn yandex.ru> writes:
On Friday, 7 January 2022 at 15:40:26 UTC, jmh530 wrote:

 On Friday, 7 January 2022 at 04:04:44 UTC, Tejas wrote:

 [snip]

 I believe you're talking about 
 [magpie](https://code.dlang.org/packages/magpie)




Just tried to use it - and it is not seems like usable library in 
its current status.
Jan 20 2022
parent jmh530 <john.michael.hall gmail.com> writes:
On Thursday, 20 January 2022 at 19:39:09 UTC, Sergey wrote:

 On Friday, 7 January 2022 at 15:40:26 UTC, jmh530 wrote:

 On Friday, 7 January 2022 at 04:04:44 UTC, Tejas wrote:

 [snip]

 I believe you're talking about 
 [magpie](https://code.dlang.org/packages/magpie)




 Just tried to use it - and it is not seems like usable library 
 in its current status.


If there are any specific problems, you should file issues. Even 
if the author is no longer able to work on it, it might be 
something that people look at if they build a new dataframe 
library.
Jan 20 2022