digitalmars.D - Machine learning framework in D
- Tim (11/11) May 24 2021 Hi all,
- Jon Degenhardt (5/12) May 24 2021 I haven’t tried it, but you might want to take a look at the
- Tim (3/16) May 24 2021 That looks really interesting, thanks! Not sure yet whether a NN
- jmh530 (3/11) May 24 2021 Grain is a deep learning framework for D.
Hi all, I'm needing to do a bit of ML soon for a project I'm involved in and I can't find any ML libs for D. Most of the libs available are for python (sci-kit learn, TensorFlow, etc.). I originally looked at using TensorFlow's C API but it doesn't include the actual ML stuff, just the tensor things. Could anybody help me out here? Any recommendations? I was thinking of using RPC to talk to TensorFlow in python, could there be any bad sides to this? I haven't managed to get RPC to talk to python form D in the past Thanks in advance
May 24 2021
On Tuesday, 25 May 2021 at 00:20:26 UTC, Tim wrote:Hi all, I'm needing to do a bit of ML soon for a project I'm involved in and I can't find any ML libs for D. … Any recommendations? I was thinking of using RPC to talk to TensorFlow in python, could there be any bad sides to this? I haven't managed to get RPC to talk to python form D in the pastI haven’t tried it, but you might want to take a look at the Netflix Vector Flow library, which is written in D. https://github.com/Netflix/vectorflow —Jon
May 24 2021
Tim <tim.oliver tutanota.com> On Tuesday, 25 May 2021 at 00:44:15 UTC, Jon Degenhardt wrote:On Tuesday, 25 May 2021 at 00:20:26 UTC, Tim wrote:That looks really interesting, thanks! Not sure yet whether a NN would be what I need, but I'll definitely look into this moreHi all, I'm needing to do a bit of ML soon for a project I'm involved in and I can't find any ML libs for D. … Any recommendations? I was thinking of using RPC to talk to TensorFlow in python, could there be any bad sides to this? I haven't managed to get RPC to talk to python form D in the pastI haven’t tried it, but you might want to take a look at the Netflix Vector Flow library, which is written in D. https://github.com/Netflix/vectorflow —Jon
May 24 2021
On Tuesday, 25 May 2021 at 00:20:26 UTC, Tim wrote:Hi all, I'm needing to do a bit of ML soon for a project I'm involved in and I can't find any ML libs for D. Most of the libs available are for python (sci-kit learn, TensorFlow, etc.). I originally looked at using TensorFlow's C API but it doesn't include the actual ML stuff, just the tensor things. Could anybody help me out here? [snip]Grain is a deep learning framework for D. https://github.com/ShigekiKarita/grain
May 24 2021
On Tuesday, 25 May 2021 at 01:04:19 UTC, jmh530 wrote:On Tuesday, 25 May 2021 at 00:20:26 UTC, Tim wrote:Also, there is grain2[1], and tfd: tensorflow for D[2]. But seriously, all these ML libraries in Dlang do not have many needed features you can easily find in Python ML libraries. If we step back: when those big companies like Google (TensorFlow etc.), Facebook (PyTorch etc.), Amazon (MxNet etc.), M$ (CNTK etc.) have already invested billions of dollars in either building these libraries directly or sponsoring them, which have been used by millions of users worldwide, why we want to build & use yet another one just in D? Few D users also means you have high probability to hit bugs in some very basic stuff, e.g. bug in grain's Adam optimizer was only fixed 24 days ago [3]. D is already late for the ML game, maybe the best thing to do is concentrate the limited resource of the D communality on wrapping other mature libraries, e.g. tfd. Given the current situation, on the practical side I'm using D to pre-processing data, and feed them into Python's ML libraries. To communicate between D & Python: -- grpc-dlang cannot talk to Python [4], as you have noted, and this issue has been around for half year now. (As a workaround, I added a Python pyd client talk to D server [8]) -- apache-thrift has build failure [5], which I just send a PR [6] to fix it today. (again, I'm not sure how many real users these 2 libraries have, they are broken at some very basic stuff, not sure how other people can use them) -- pyd: this is what I'm using right now. There are some rough edges [7], but at least I can get the basic stuff working finally, i.e. D & Python can talk. Anyway, it's not easy, good luck! [1]: https://github.com/ShigekiKarita/grain2 [2]: https://github.com/ShigekiKarita/tfd [3]: https://github.com/ShigekiKarita/grain/pull/15 [4]: https://github.com/huntlabs/grpc-dlang/issues/15 [5]: https://github.com/D-Programming-Deimos/openssl/issues/63 [6]: https://github.com/apache/thrift/pull/2397 [7]: https://github.com/ariovistus/pyd/issues [8]: https://github.com/huntlabs/grpc-dlang/pull/19Hi all, I'm needing to do a bit of ML soon for a project I'm involved in and I can't find any ML libs for D. Most of the libs available are for python (sci-kit learn, TensorFlow, etc.). I originally looked at using TensorFlow's C API but it doesn't include the actual ML stuff, just the tensor things. Could anybody help me out here? [snip]Grain is a deep learning framework for D. https://github.com/ShigekiKarita/grain
May 25 2021
On Tuesday, 25 May 2021 at 23:37:24 UTC, mw wrote:links: 1: https://github.com/ShigekiKarita/grain2 2: https://github.com/ShigekiKarita/tfd 3: https://github.com/ShigekiKarita/grain/pull/15 4: https://github.com/huntlabs/grpc-dlang/issues/15 5: https://github.com/D-Programming-Deimos/openssl/issues/63 6: https://github.com/apache/thrift/pull/2397 7: https://github.com/ariovistus/pyd/issues 8: https://github.com/huntlabs/grpc-dlang/pull/19Grain is a deep learning framework for D. https://github.com/ShigekiKarita/grainAlso, there is grain2[1], and tfd: tensorflow for D[2]. But seriously, all these ML libraries in Dlang do not have many needed features you can easily find in Python ML libraries. If we step back: when those big companies like Google (TensorFlow etc.), Facebook (PyTorch etc.), Amazon (MxNet etc.), M$ (CNTK etc.) have already invested billions of dollars in either building these libraries directly or sponsoring them, which have been used by millions of users worldwide, why we want to build & use yet another one just in D? Few D users also means you have high probability to hit bugs in some very basic stuff, e.g. bug in grain's Adam optimizer was only fixed 24 days ago [3]. D is already late for the ML game, maybe the best thing to do is concentrate the limited resource of the D communality on wrapping other mature libraries, e.g. tfd. Given the current situation, on the practical side I'm using D to pre-processing data, and feed them into Python's ML libraries. To communicate between D & Python: -- grpc-dlang cannot talk to Python [4], as you have noted, and this issue has been around for half year now. (As a workaround, I added a Python pyd client talk to D server [8]) -- apache-thrift has build failure [5], which I just send a PR [6] to fix it today. (again, I'm not sure how many real users these 2 libraries have, they are broken at some very basic stuff, not sure how other people can use them) -- pyd: this is what I'm using right now. There are some rough edges [7], but at least I can get the basic stuff working finally, i.e. D & Python can talk. Anyway, it's not easy, good luck!
May 25 2021