www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Some crazy ideas from a high level perspective

reply _d0s_ <oggs gmx.at> writes:
As i'm a student and i should be busy with writing some papers 
... i'm spending waaaaaaay to much time derping around having 
crazy ideas. since they are forgotten quickly most of the time, i 
thought i share them with you.

this time: what D is missing for me

i'm coming from the field of computer vision and computer 
graphics. algorithms used in that field often utilize some heavy 
maths. like solving some heavy least squares equation systems, or 
graph algorithms.
because of that Matlab is often our tool of choice for quickly 
testing algorithms. After everything is running we translate 
things to c++.

Idea1: a general interface to describe n-dimensional matrices
i think the D syntax has the power to manipulate matrixes in a 
very expressive way. a great addition to the standard library 
would be to define a basic interface to work with matrices.

mat2d = DenseMatrix!float(5,5); // create a 5x5 float matrix
mat2d.zeros(); // initialize with zeros
mat2d[0..$, 2:3] = 1; // slice the matrix and fill the slice with 
ones

Specialized matrix types would then define the underlying memory 
layout and capabilities of a matrix. Like a matrix could be dense 
or sparse ( stored in the yale format or whatever other format ). 
depending on the needed operations the programmer would choose a 
fitting matrix implementation.

This would not only be useful to solve math problems. But also to 
define for example adjacency matrices for graphs, or to store 
images.

Idea2: Matrix Solvers
Have some base algorithms to solve linear and quadtratic systems 
on top of those matrices. (LU, QR, ...)

Idea3: Visualization
Imho D is lacking a good UI toolkit. I again like here the 
simplicity of matlab that lets me just pop up a window, and show 
a simple diagram or picture. Often its not necessary to create a 
sophisticated UI, but having a visual representation of data 
often helps with debugging and understanding of algorithms.

Some modern applications like Avast, Steam or Spotify create 
their user interfaces based on web technologies. Recently i've 
seen that somebody threw together a demo with libcef, which lets 
you use the chrome embedded framework, to basically have chrome 
in a little window.
http://dblog.aldacron.net/2015/01/derelictcef-binding-to-chromium-embedded-framework/
http://imgur.com/pYlP3dE

maybe that would be suitable to throw together a simple image 
viewer / diagram viewer. Based on some opensource web chart 
library ( like d3.js or whatever )



until now i didnt have yet the time and/or skill to realize those 
still very rough ideas. i hope you can give me some suggestions 
...
... on what does already exist maybe in a similar form
... on what you would love to see to be realized of those ideas
... on why my ideas are garbage :'D destory pls

cheers :)
Mar 22 2016
next sibling parent Xinok <xinok live.com> writes:
On Tuesday, 22 March 2016 at 14:08:55 UTC, _d0s_ wrote:

 Idea1: a general interface to describe n-dimensional matrices


Perhaps this is what you're looking for?

https://dlang.org/phobos/std_experimental_ndslice.html
Mar 22 2016
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 22 March 2016 at 14:08:55 UTC, _d0s_ wrote:

 Idea2: Matrix Solvers
 Have some base algorithms to solve linear and quadtratic 
 systems on top of those matrices. (LU, QR, ...)


You would be interested in
https://github.com/DlangScience

They have cblas for matrix multiplication and scid.linalg has an 
interface to LAPACK (the most common way to do LU/QR/etc). I 
don't have experience with these, but I think they were created 
before std.experimental.ndslice, so I'm not sure how well they 
inter-operate. I believe there is work being done on this, but 
I'm sure John Colvin et al would appreciate any help you would be 
able to provide.

With respect to your point about about sparse vs. dense matrix 
types and other special layouts, it would probably be possible to 
build some of that functionality on top of ndslice. I'd prefer 
there to be a focus of getting blas/LAPACK working seamlessly 
with ndslice before that.
Mar 22 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 22 March 2016 at 14:42:18 UTC, jmh530 wrote:

 On Tuesday, 22 March 2016 at 14:08:55 UTC, _d0s_ wrote:

 Idea2: Matrix Solvers
 Have some base algorithms to solve linear and quadtratic 
 systems on top of those matrices. (LU, QR, ...)


 You would be interested in
 https://github.com/DlangScience

 They have cblas for matrix multiplication and scid.linalg has 
 an interface to LAPACK (the most common way to do LU/QR/etc). I 
 don't have experience with these, but I think they were created 
 before std.experimental.ndslice, so I'm not sure how well they 
 inter-operate. I believe there is work being done on this, but 
 I'm sure John Colvin et al would appreciate any help you would 
 be able to provide.

 With respect to your point about about sparse vs. dense matrix 
 types and other special layouts, it would probably be possible 
 to build some of that functionality on top of ndslice. I'd 
 prefer there to be a focus of getting blas/LAPACK working 
 seamlessly with ndslice before that.


Mir v0.15.0-beta2 contains sparse matrixes/tensors in DOK and 
CSR/CSR formats. Also some COO ranges are supported too. Current 
state is half of future API.

https://github.com/DlangScience/mir/blob/sparse/source/mir/ndslice/sparse.d

Dub: http://code.dlang.org/packages/mir
Apr 16 2016
prev sibling next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Tuesday, 22 March 2016 at 14:08:55 UTC, _d0s_ wrote:

 Idea3: Visualization
 Imho D is lacking a good UI toolkit. I again like here the 
 simplicity of matlab that lets me just pop up a window, and 
 show a simple diagram or picture. Often its not necessary to 
 create a sophisticated UI, but having a visual representation 
 of data often helps with debugging and understanding of 
 algorithms.


If all you need are plotting tools, these would be useful
https://code.dlang.org/packages/ggplotd
https://code.dlang.org/packages/plotd

More complicated than that, I don't have familiarity with, but 
there are some other libraries worth looking at on 
https://code.dlang.org/
Mar 22 2016
prev sibling next sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
Idea 1:

Sigh I do wish the author of gl3n had given permission for relicense for 
Phobos. Would do what you want.

Idea 3:

My goal is get windowing/image library into Phobos. Now that is not a UI 
toolkit but you can atleast get a window up and show an image like a graph.
Kinda everything you want.

About web based UI's, just ew. They've been shown to not work time and 
time again. Nothing better than a native UI.

If you want to help me with windowing/image library, I need 
https://github.com/D-Programming-Language/phobos/pull/2845 to support 
blending.
Otherwise blah work being 
https://github.com/rikkimax/alphaPhobos/tree/master/source/std/e
perimental/bindings 
severely bigger.

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
Mar 22 2016
parent reply Edwin van Leeuwen <edder tkwsping.nl> writes:
On Tuesday, 22 March 2016 at 15:19:10 UTC, rikki cattermole wrote:

 Idea 1:

 Sigh I do wish the author of gl3n had given permission for 
 relicense for Phobos. Would do what you want.

 Idea 3:

 My goal is get windowing/image library into Phobos. Now that is 
 not a UI toolkit but you can atleast get a window up and show 
 an image like a graph.
 Kinda everything you want.


Any tips on how easy it will be to plot a cairo surface to a 
window? ggplotd currently relies on gtkd for that, but would be 
great if there is a D based solution.

I'm especially interested in plotting image of the below type in 
a window:

CAIRO_FORMAT_ARGB32[1]
each pixel is a 32-bit quantity, with alpha in the upper 8 bits, 
then red, then green, then blue. The 32-bit quantities are stored 
native-endian. Pre-multiplied alpha is used. (That is, 50% 
transparent red is 0x80800000, not 0x80ff0000.)

[1] cairographics.org/manual-1.2.0/cairo-Image-Surfaces.html
Mar 22 2016
parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 22 March 2016 at 15:33:04 UTC, Edwin van Leeuwen 
wrote:

 On Tuesday, 22 March 2016 at 15:19:10 UTC, rikki cattermole 
 wrote:

 Idea 1:

 Sigh I do wish the author of gl3n had given permission for 
 relicense for Phobos. Would do what you want.

 Idea 3:

 My goal is get windowing/image library into Phobos. Now that 
 is not a UI toolkit but you can atleast get a window up and 
 show an image like a graph.
 Kinda everything you want.


 Any tips on how easy it will be to plot a cairo surface to a 
 window ?


Very easy.

- wrap a context in a Canvas with the methods for drawing.
- BLIT the cairo surface.

It also works with simple bitmaps, so image proceduraly rendered.
Actually I've started to work on this at the end of 2015.

(https://github.com/BBasile/kheops/blob/master/src/kheops/bitmap.d#L396)

While it was planned to make a native D UI, using it to plot 2D 
data would require very few work. There's also a OO wrapper 
called cairoD or something like that which is for sure very easy 
to use to render bitmap proceduraly.

The positive point with cairo is that it has a "png surfaces" so 
images can be saved in a single call (cairo_surface_write_to_png).
Mar 22 2016
parent reply Edwin van Leeuwen <edder tkwsping.nl> writes:
On Tuesday, 22 March 2016 at 15:50:15 UTC, Basile B. wrote:

 On Tuesday, 22 March 2016 at 15:33:04 UTC, Edwin van Leeuwen 
 wrote:

 On Tuesday, 22 March 2016 at 15:19:10 UTC, rikki cattermole 
 wrote:

 Idea 1:

 Sigh I do wish the author of gl3n had given permission for 
 relicense for Phobos. Would do what you want.

 Idea 3:

 My goal is get windowing/image library into Phobos. Now that 
 is not a UI toolkit but you can atleast get a window up and 
 show an image like a graph.
 Kinda everything you want.


 Any tips on how easy it will be to plot a cairo surface to a 
 window ?


 Very easy.

 - wrap a context in a Canvas with the methods for drawing.
 - BLIT the cairo surface.


Thanks!


 It also works with simple bitmaps, so image proceduraly 
 rendered.
 Actually I've started to work on this at the end of 2015.

 (https://github.com/BBasile/kheops/blob/master/src/kheops/bitmap.d#L396)


That looks interesting. Would you say kheops is stable enough to 
be relied upon  by other projects? Any plans to make it into a 
dub package?

Since kheops is vector based it would be even nicer to be able to 
convert a cairo svg/pdf surface to kheops.


 While it was planned to make a native D UI, using it to plot 2D 
 data would require very few work. There's also a OO wrapper 
 called cairoD or something like that which is for sure very 
 easy to use to render bitmap proceduraly.


cairoD is indeed what ggplotd uses to produce the plot. And 
saving svg/png/pdf to disk is indeed very easy with cairo.
Mar 22 2016
parent Basile B. <b2.temp gmx.com> writes:
On Tuesday, 22 March 2016 at 16:01:43 UTC, Edwin van Leeuwen 
wrote:

 That looks interesting. Would you say kheops is stable enough 
 to be relied upon  by other projects? Any plans to make it into 
 a dub package?


No. This would require a bit of work (currently it's linux only).
Go with cairoD since it's already used for that.
Mar 22 2016
prev sibling next sibling parent reply _d0s_ <oggs gmx.at> writes:
thanks for all the opinions!


 Perhaps this is what you're looking for?
 https://dlang.org/phobos/std_experimental_ndslice.html


that would be a good starting point for a matrix library


 You would be interested in
 https://github.com/DlangScience



 They have cblas for matrix multiplication and scid.linalg has 
 an interface to LAPACK (the > most common way to do LU/QR/etc). 
 I don't have experience with these, but I think they were 
 created before std.experimental.ndslice, so I'm not sure how 
 well they inter-operate. I > believe there is work being done 
 on this, but I'm sure John Colvin et al would appreciate > any 
 help you would be able to provide.


thx, havent found that before :)
i haven't used lapack/blas before, i will have a look at it. i'm 
using mostly Eigen for my projects.


 If all you need are plotting tools, these would be useful
 https://code.dlang.org/packages/ggplotd
 https://code.dlang.org/packages/plotd


that would be somewhat close already to matlab like plotting


 Sigh I do wish the author of gl3n had given permission for 
 relicense for Phobos. Would do what you want.


i have used gl3n before, it's not really what i'm looking for. 
gl3n afaik only provides up to 4x4 dense matrices, with the focus 
on being a GLM (http://glm.g-truc.net/0.9.7/index.html) 
replacement for the use with Opengl. correct me if i'm wrong.


Cairo and Kheops look interesting. I have used Openvg before. 
Some time ago i've created a little Openvg D wrapper for shiva vg 
( https://code.dlang.org/packages/dopenvg ) . From a diagram or 
image viewer i'd also expect some level of interactivity. That's 
why i thought a web app with CEF would be suitable

do you have experience with creating web apps for desktop, or 
reasons why i'd definitively want to use a native framework for 
such a project? .. or for UI in general
Mar 22 2016
next sibling parent reply cym13 <cpicard openmailbox.org> writes:
On Tuesday, 22 March 2016 at 18:18:50 UTC, _d0s_ wrote:

 thanks for all the opinions!
 ...


A quite good presentation of ndslice [1]

If you want quick and dirty visualization and find plotting 
libraries too plotting-oriented you may want to play a bit with 
simpledisplay [2], it is without pretention but very easy to use 
and it's proved to come in handy quite often.

[1]: http://jackstouffer.com/blog/nd_slice.html
[2]: 
https://github.com/adamdruppe/arsd/blob/master/simpledisplay.d
Mar 22 2016
parent Adam D. Ruppe <destructionator gmail.com> writes:
On Tuesday, 22 March 2016 at 20:52:26 UTC, cym13 wrote:

 [2]: 
 https://github.com/adamdruppe/arsd/blob/master/simpledisplay.d


simpledisplay has some examples in its docs:

http://dpldocs.info/experimental-docs/simpledisplay.html

I haven't actually written plotting examples yet, but it is the 
same basic operations as Pong... drawing lines and circles.
Mar 23 2016
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 23/03/16 7:18 AM, _d0s_ wrote:

 thanks for all the opinions!


 Perhaps this is what you're looking for?
 https://dlang.org/phobos/std_experimental_ndslice.html


 that would be a good starting point for a matrix library


 You would be interested in
 https://github.com/DlangScience



 They have cblas for matrix multiplication and scid.linalg has an
 interface to LAPACK (the > most common way to do LU/QR/etc). I don't
 have experience with these, but I think they were created before
 std.experimental.ndslice, so I'm not sure how well they inter-operate.
 I > believe there is work being done on this, but I'm sure John Colvin
 et al would appreciate > any help you would be able to provide.


 thx, havent found that before :)
 i haven't used lapack/blas before, i will have a look at it. i'm using
 mostly Eigen for my projects.


 If all you need are plotting tools, these would be useful
 https://code.dlang.org/packages/ggplotd
 https://code.dlang.org/packages/plotd


 that would be somewhat close already to matlab like plotting


 Sigh I do wish the author of gl3n had given permission for relicense
 for Phobos. Would do what you want.


 i have used gl3n before, it's not really what i'm looking for. gl3n
 afaik only provides up to 4x4 dense matrices, with the focus on being a
 GLM (http://glm.g-truc.net/0.9.7/index.html) replacement for the use
 with Opengl. correct me if i'm wrong.


Its old code, it would need to be updated anyway. But yes more GLM then 
a generic solution.


 Cairo and Kheops look interesting. I have used Openvg before. Some time
 ago i've created a little Openvg D wrapper for shiva vg (
 https://code.dlang.org/packages/dopenvg ) . From a diagram or image
 viewer i'd also expect some level of interactivity. That's why i thought
 a web app with CEF would be suitable

 do you have experience with creating web apps for desktop, or reasons
 why i'd definitively want to use a native framework for such a project?
 .. or for UI in general


I don't have experience past a little with WebOS. But my understanding 
is simple. Companies like Facebook and Apple have all tried it on mobile 
and all failed. It just comes down to most programs with UI's should be 
sitting around 0% cpu most of the time. They just don't need the 
resources. Things like web based UI's will always go well above that.

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
Mar 22 2016
prev sibling next sibling parent reply _d0s_ <oggs gmx.at> writes:
just found this ...
http://forum.dlang.org/thread/nilhvnqbsgqhxdshpqfl forum.dlang.org
maybe that will be what i'm looking for :)

as for the web ui's ...
the performance is of course not perfect as in a native app. but 
i think apps like atom.io, visual studio code or discord show 
that it is perfectly possible to create apps with really good 
look, feel not too bad performance :)
Mar 24 2016
parent sigod <sigod.mail gmail.com> writes:
On Thursday, 24 March 2016 at 09:54:37 UTC, _d0s_ wrote:

 just found this ...
 http://forum.dlang.org/thread/nilhvnqbsgqhxdshpqfl forum.dlang.org
 maybe that will be what i'm looking for :)

 as for the web ui's ...
 the performance is of course not perfect as in a native app. 
 but i think apps like atom.io, visual studio code or discord 
 show that it is perfectly possible to create apps with really 
 good look, feel not too bad performance :)


https://www.youtube.com/watch?v=k56wra39lwA
Mar 24 2016
prev sibling parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 22 March 2016 at 14:08:55 UTC, _d0s_ wrote:

 As i'm a student and i should be busy with writing some papers 
 ... i'm spending waaaaaaay to much time derping around having 
 crazy ideas. since they are forgotten quickly most of the time, 
 i thought i share them with you.

 [...]


Generic matrix-matrix multiplication is available in Mir version 
0.16.0-beta2
http://docs.mir.dlang.io/latest/mir_glas_gemm.html
Aug 22 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 23 August 2016 at 06:05:32 UTC, Ilya Yaroshenko wrote:

 On Tuesday, 22 March 2016 at 14:08:55 UTC, _d0s_ wrote:

 As i'm a student and i should be busy with writing some papers 
 ... i'm spending waaaaaaay to much time derping around having 
 crazy ideas. since they are forgotten quickly most of the 
 time, i thought i share them with you.

 [...]


 Generic matrix-matrix multiplication is available in Mir 
 version 0.16.0-beta2
 http://docs.mir.dlang.io/latest/mir_glas_gemm.html


It should be compiled with recent LDC beta, and -mcpu=native flag.
Aug 22 2016