www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - ML and contributing to mir

reply Marc <mailnawman gmail.com> writes:
Hi everyone,

I'm interested in tensors, scientific computing and ML. After 
trying to write a tensor library for c++, I came to the 
realisation that I'm starting to hate the language the more code 
i write. D seems like an obvious choice for my interest, a fast 
compiled language but the gc is putitng me off as well. I would 
be however very interested in contributing to mir maybe even 
writting examples for its use cases. I would like to write a 
classical machine learning library (decision trees, linear 
models...) in D. Are there currently any effort in the community 
towards this goal? I would appreciate any help or suggestions.
Aug 06
next sibling parent reply Bradley Chatha <sealabjaster gmail.com> writes:
On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 Hi everyone,

 I'm interested in tensors, scientific computing and ML. After 
 trying to write a tensor library for c++, I came to the 
 realisation that I'm starting to hate the language the more 
 code i write. D seems like an obvious choice for my interest, a 
 fast compiled language but the gc is putitng me off as well. I 
 would be however very interested in contributing to mir maybe 
 even writting examples for its use cases. I would like to write 
 a classical machine learning library (decision trees, linear 
 models...) in D. Are there currently any effort in the 
 community towards this goal? I would appreciate any help or 
 suggestions.


This isn't really my area of interest or expertise, but you might 
find some historical interest from VectorFlow - a neural network 
library Netflix developed (and I feel it stopped existing after 
like a month, would love if someone has an interesting story to 
tell about it): https://github.com/Netflix/vectorflow
Aug 06
parent Marc <mailnawman gmail.com> writes:
On Wednesday, 6 August 2025 at 18:43:21 UTC, Bradley Chatha wrote:

 This isn't really my area of interest or expertise, but you 
 might find some historical interest from VectorFlow - a neural 
 network library Netflix developed (and I feel it stopped 
 existing after like a month, would love if someone has an 
 interesting story to tell about it): 
 https://github.com/Netflix/vectorflow


This is a deep learning library so different but it’s nice 
there’s one written in D.
Aug 06
prev sibling next sibling parent reply monkyyy <crazymonkyyy gmail.com> writes:
On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 the gc is putitng me off as well


gc only runs when you allocate; use the same big arrays c++ would 
use and youll be fine
Aug 06
parent reply Marc <mailnawman gmail.com> writes:
On Wednesday, 6 August 2025 at 19:15:51 UTC, monkyyy wrote:

 On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 the gc is putitng me off as well


 gc only runs when you allocate; use the same big arrays c++ 
 would use and youll be fine


Can I allocate and deallocate without triggering the gc? I don’t 
know how mir uses the gc in the low level but I’m interested in 
having reference counter pointers, they are essentials for some 
operations on tensors without copying the data every time.
Aug 06
parent monkyyy <crazymonkyyy gmail.com> writes:
On Wednesday, 6 August 2025 at 20:08:04 UTC, Marc wrote:

 On Wednesday, 6 August 2025 at 19:15:51 UTC, monkyyy wrote:

 On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 the gc is putitng me off as well


 gc only runs when you allocate; use the same big arrays c++ 
 would use and youll be fine


 Can I allocate and deallocate without triggering the gc? I 
 don’t know how mir uses the gc in the low level but I’m 
 interested in having reference counter pointers, they are 
 essentials for some operations on tensors without copying the 
 data every time.


You can straight up disable it, by compiler flag or at runtime

but more realistically just put stuff on the stack or global 
scope 99% of the time, or if you must do something dynamic make 
sure its allot all at once.

Dont know anything about mir, but I see no reason to think they 
would be unable to be used reasonably; it isnt some functional 
languge with gc's enable linked lists everywhere, you dont have 
to linked list of bools and have your 1 bit be stored by 180ish 
bits and be a cache miss every iteration like a slow language. 
You can put arrays on the stack, you can call malloc, you can 
cast void*'s.

Its a O(c) overhead to have a gc you dont misuse, it will make 
the exe bigger then it should be from the run time.
Aug 06
prev sibling next sibling parent reply Lance Bachmeier <no spam.net> writes:
On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:


 but the gc is putitng me off as well


Then don't use it?

I do lots of statistical work with D and honestly I have found 
little to gain from avoiding the GC. It'll all depends on what 
you're doing, of course, but look at std.typecons.RefCounted if 
you want to avoid D's GC. I use it because I often call into C 
libraries that allocate memory. Just don't expect it to improve 
performance. YMMV.
Aug 06
parent Marc <mailnawman gmail.com> writes:
On Wednesday, 6 August 2025 at 19:37:19 UTC, Lance Bachmeier 
wrote:

 On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:


 but the gc is putitng me off as well


 Then don't use it?

 I do lots of statistical work with D and honestly I have found 
 little to gain from avoiding the GC. It'll all depends on what 
 you're doing, of course, but look at std.typecons.RefCounted if 
 you want to avoid D's GC. I use it because I often call into C 
 libraries that allocate memory. Just don't expect it to improve 
 performance. YMMV.


Thanks, this might be exactly what I’m looking for, referenced 
counting.
Aug 06
prev sibling next sibling parent reply Serg Gini <kornburn yandex.ru> writes:
On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 Hi everyone,

 I'm interested in tensors, scientific computing and ML. After 
 trying to write a tensor library for c++, I came to the 
 realisation that I'm starting to hate the language the more 
 code i write. D seems like an obvious choice for my interest, a 
 fast compiled language but the gc is putitng me off as well. I 
 would be however very interested in contributing to mir maybe 
 even writting examples for its use cases. I would like to write 
 a classical machine learning library (decision trees, linear 
 models...) in D. Are there currently any effort in the 
 community towards this goal? I would appreciate any help or 
 suggestions.


there are some things to do in this area
most of the work is stalled now

but if you are interested join discord and we can discuss 
something there
Aug 06
parent reply Marc <mailnawman gmail.com> writes:
On Wednesday, 6 August 2025 at 19:40:17 UTC, Serg Gini wrote:

 there are some things to do in this area
 most of the work is stalled now

 but if you are interested join discord and we can discuss 
 something there


Thanks for your reply. Sure, I’m interested and I’m on discord. 
What channel or server is it? If you don’t mind sharing here so I 
can join.
Aug 06
parent Serg Gini <kornburn yandex.ru> writes:
On Wednesday, 6 August 2025 at 20:15:47 UTC, Marc wrote:

 On Wednesday, 6 August 2025 at 19:40:17 UTC, Serg Gini wrote:

 there are some things to do in this area
 most of the work is stalled now

 but if you are interested join discord and we can discuss 
 something there


 Thanks for your reply. Sure, I’m interested and I’m on discord. 
 What channel or server is it? If you don’t mind sharing here so 
 I can join.


Sure
https://discord.gg/bMZk9Q4
Aug 06
prev sibling next sibling parent Ferhat =?UTF-8?B?S3VydHVsbXXFnw==?= <aferust gmail.com> writes:
On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 Hi everyone,

 I'm interested in tensors, scientific computing and ML. After 
 trying to write a tensor library for c++, I came to the 
 realisation that I'm starting to hate the language the more 
 code i write. D seems like an obvious choice for my interest, a 
 fast compiled language but the gc is putitng me off as well. I 
 would be however very interested in contributing to mir maybe 
 even writting examples for its use cases. I would like to write 
 a classical machine learning library (decision trees, linear 
 models...) in D. Are there currently any effort in the 
 community towards this goal? I would appreciate any help or 
 suggestions.


mir ndslices supports ref counted slices as well.
http://mir-algorithm.libmir.org/mir_ndslice_allocation.html
Aug 06
prev sibling next sibling parent Tim Moldazhan <tim.moldazhan gmail.com> writes:
On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 Hi everyone,

 I'm interested in tensors, scientific computing and ML. After 
 trying to write a tensor library for c++, I came to the 
 realisation that I'm starting to hate the language the more 
 code i write. D seems like an obvious choice for my interest, a 
 fast compiled language but the gc is putitng me off as well. I 
 would be however very interested in contributing to mir maybe 
 even writting examples for its use cases. I would like to write 
 a classical machine learning library (decision trees, linear 
 models...) in D. Are there currently any effort in the 
 community towards this goal? I would appreciate any help or 
 suggestions.


Perhaps the package bindbc-onnxruntime 1.2.0 will help you. Its 
official github repository is 
https://github.com/lempiji/bindbc-onnxruntime
Aug 09
prev sibling parent reply Ki <rill.ki yahoo.com> writes:
On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 Hi everyone,

 I'm interested in tensors, scientific computing and ML. After 
 trying to write a tensor library for c++, I came to the 
 realisation that I'm starting to hate the language the more 
 code i write. D seems like an obvious choice for my interest, a 
 fast compiled language but the gc is putitng me off as well. I 
 would be however very interested in contributing to mir maybe 
 even writting examples for its use cases. I would like to write 
 a classical machine learning library (decision trees, linear 
 models...) in D. Are there currently any effort in the 
 community towards this goal? I would appreciate any help or 
 suggestions.


Here some contributions over the years:
* [Grain](https://github.com/ShigekiKarita/grain/tree/master) - 
Tensor & NN library, implements tensors on GPU with CUDA. 
Abandoned.
* [Golem](https://github.com/lempiji/golem/tree/master) - the 
same as above, but on CPU only.

Under the hood:
* [Mir ndslice](https://github.com/libmir/mir-algorithm)
Aug 14
next sibling parent Ki <rill.ki yahoo.com> writes:
On Friday, 15 August 2025 at 02:37:26 UTC, Ki wrote:

 On Wednesday, 6 August 2025 at 17:54:27 UTC, Marc wrote:

 Hi everyone,

 I'm interested in tensors, scientific computing and ML. After 
 trying to write a tensor library for c++, I came to the 
 realisation that I'm starting to hate the language the more 
 code i write. D seems like an obvious choice for my interest,




I too have the same idea with 
[Densor](https://github.com/rillki/densor), but I'm slow on 
implementation.

I thought of the following structure:
* Densor - Tensors with GPU/CPU.
* SomeNN - NN and DL library that depends on Densor.
* SomeXX... - other libraries that use Densor and SomeNN as 
dependencies.
Aug 14
prev sibling parent reply Marc <mailnawman gmail.com> writes:
On Friday, 15 August 2025 at 02:37:26 UTC, Ki wrote:

 Here some contributions over the years:
 * [Grain](https://github.com/ShigekiKarita/grain/tree/master) - 
 Tensor & NN library, implements tensors on GPU with CUDA. 
 Abandoned.
 * [Golem](https://github.com/lempiji/golem/tree/master) - the 
 same as above, but on CPU only.

 Under the hood:
 * [Mir ndslice](https://github.com/libmir/mir-algorithm)


Thanks for your reply.
Grain looks promising, Its sad that is has been abandonned.
What would be the licence if I'm interested in forking it?
Aug 16
parent Sergey <kornburn yandex.ru> writes:
On Saturday, 16 August 2025 at 20:46:57 UTC, Marc wrote:

 On Friday, 15 August 2025 at 02:37:26 UTC, Ki wrote:

 Here some contributions over the years:
 * [Grain](https://github.com/ShigekiKarita/grain/tree/master) 
 - Tensor & NN library, implements tensors on GPU with CUDA. 
 Abandoned.
 * [Golem](https://github.com/lempiji/golem/tree/master) - the 
 same as above, but on CPU only.

 Under the hood:
 * [Mir ndslice](https://github.com/libmir/mir-algorithm)


 Thanks for your reply.
 Grain looks promising, Its sad that is has been abandonned.
 What would be the licence if I'm interested in forking it?


Boost Software License - Version 1.0
Aug 16