www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - stc.experimental.ndslice -> sci.ndslice

reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
We plan to add a set of numeric packages and this would be real 
pain if they would be one-by-one moved from experimental to 
stable std. So sci.* should be considered as experimental during 
few years.


https://github.com/dlang/phobos/pull/4207
Apr 16 2016
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be real 
 pain if they would be one-by-one moved from experimental to 
 stable std. So sci.* should be considered as experimental 
 during few years.


 https://github.com/dlang/phobos/pull/4207


I'm not sure this is a good idea.

1. std.experimental is self documenting, you know right away that 
the modules are not stable
2. std, core, and etc are all reserved for Phobos, adding in 
others is an unnecessary breaking change

I really don't see why ndslice and other numerical libraries 
should have special treatment.
Apr 17 2016
parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 17 April 2016 at 07:02:25 UTC, Jack Stouffer wrote:

 On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be 
 real pain if they would be one-by-one moved from experimental 
 to stable std. So sci.* should be considered as experimental 
 during few years.


 https://github.com/dlang/phobos/pull/4207


 I'm not sure this is a good idea.

 1. std.experimental is self documenting, you know right away 
 that the modules are not stable
 2. std, core, and etc are all reserved for Phobos, adding in 
 others is an unnecessary breaking change


1. The same would be for all sci
2. requires only few changes in configs


 I really don't see why ndslice and other numerical libraries 
 should have special treatment.


Because synchronization between different numeric parts. sci part 
of phobos would be _very_ integrated each other. So it would be 
headache to move any package from ndslice from experimental 
because all other packages and Mir would require changes. Note, 
that we also have other compiler that should be supported without 
breaking code.

So the reason is introduce the small breaking change now instead 
of dozens breaking changes in the future.
Apr 17 2016
prev sibling next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be real 
 pain if they would be one-by-one moved from experimental to 
 stable std. So sci.* should be considered as experimental 
 during few years.


 https://github.com/dlang/phobos/pull/4207


As I wrote on the PR --

Introducing a new top-level package is a HUGE DEAL. For one 
thing, it will break build tools such as rdmd, which know which 
top-level packages to exclude from linking (as they'll be in 
phobos.lib). For another, it might conflict with users' packages.

We need a very strong reason to do this.

--

Also, what precedent does this set? If we start throwing 
image-processing code into Phobos, does that mean we should 
introduce another top-level package? Where do you draw the line?
Apr 17 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 17 April 2016 at 07:11:17 UTC, Vladimir Panteleev 
wrote:

 On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be 
 real pain if they would be one-by-one moved from experimental 
 to stable std. So sci.* should be considered as experimental 
 during few years.


 https://github.com/dlang/phobos/pull/4207


 As I wrote on the PR --

 Introducing a new top-level package is a HUGE DEAL. For one 
 thing, it will break build tools such as rdmd, which know which 
 top-level packages to exclude from linking (as they'll be in 
 phobos.lib).For another, it might conflict with users' packages.


sci name was reserved for future use: 
http://code.dlang.org/packages/sci


 We need a very strong reason to do this.

 --

 Also, what precedent does this set? If we start throwing 
 image-processing code into Phobos, does that mean we should 
 introduce another top-level package? Where do you draw the line?


The goal is to include only good consistent basic functionality 
for data/numeric/math/sci users:

1. Tensors - ndslice
2. Sparse tensors - future ndslice.sparse
3. FFT - fast & multidimensional & vectorized & pure D
4. BLAS - fast & multidimensional & vectorized & pure D
Apr 17 2016
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Sunday, 17 April 2016 at 07:25:55 UTC, Ilya Yaroshenko wrote:

 sci name was reserved for future use: 
 http://code.dlang.org/packages/sci


What does a Dub package have to do with D module system packages?


 The goal is to include only good consistent basic functionality 
 for data/numeric/math/sci users:

 1. Tensors - ndslice
 2. Sparse tensors - future ndslice.sparse
 3. FFT - fast & multidimensional & vectorized & pure D
 4. BLAS - fast & multidimensional & vectorized & pure D


I don't understand, what's wrong with std.sci or etc.sci?
Apr 17 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 17 April 2016 at 07:30:38 UTC, Vladimir Panteleev 
wrote:

 On Sunday, 17 April 2016 at 07:25:55 UTC, Ilya Yaroshenko wrote:

 sci name was reserved for future use: 
 http://code.dlang.org/packages/sci


 What does a Dub package have to do with D module system 
 packages?


 The goal is to include only good consistent basic 
 functionality for data/numeric/math/sci users:

 1. Tensors - ndslice
 2. Sparse tensors - future ndslice.sparse
 3. FFT - fast & multidimensional & vectorized & pure D
 4. BLAS - fast & multidimensional & vectorized & pure D


 I don't understand, what's wrong with std.sci or etc.sci?


I am ok with std.sci for example. I just want to exclude dances 
with transaction between stable <->`experimental`. So, if all 
`std.sci` would be marked as experimental during few years this 
would what i want.
Apr 17 2016
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 17 April 2016 at 07:35:19 UTC, Ilya Yaroshenko wrote:

 On Sunday, 17 April 2016 at 07:30:38 UTC, Vladimir Panteleev 
 wrote:

 On Sunday, 17 April 2016 at 07:25:55 UTC, Ilya Yaroshenko 
 wrote:

 sci name was reserved for future use: 
 http://code.dlang.org/packages/sci


 What does a Dub package have to do with D module system 
 packages?


 The goal is to include only good consistent basic 
 functionality for data/numeric/math/sci users:

 1. Tensors - ndslice
 2. Sparse tensors - future ndslice.sparse
 3. FFT - fast & multidimensional & vectorized & pure D
 4. BLAS - fast & multidimensional & vectorized & pure D


 I don't understand, what's wrong with std.sci or etc.sci?


 I am ok with std.sci for example. I just want to exclude dances 
 with transaction between stable <->`experimental`. So, if all 
 `std.sci` would be marked as experimental during few years this 
 would what i want.


I can only agree with Ilya. It's currently a huge pain to develop 
something with mir or submit a bug fix, as Ilya has to manually 
diff and then send it Phobos.
I think the same should be said about other packages that are in 
experimental, but for now mir is the only one with active 
development.
If there would be a language solution to this problem - e.g.
1) that it is possible to overwrite modules from the standard 
library
2) it is possible to exclude experimental from the standard 
library
3) your ideas..

Then we would be even happier :)
It's really a huge deal to us, because it currently wastes a lot 
of development time.

Another idea would be to go with std.experimental.sci.*, if that 
has a higher consensus.
Apr 17 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 17 April 2016 at 08:49:53 UTC, Seb wrote:

 On Sunday, 17 April 2016 at 07:35:19 UTC, Ilya Yaroshenko wrote:

 On Sunday, 17 April 2016 at 07:30:38 UTC, Vladimir Panteleev

 I don't understand, what's wrong with std.sci or etc.sci?


 I am ok with std.sci for example. I just want to exclude 
 dances with transaction between stable <->`experimental`. So, 
 if all `std.sci` would be marked as experimental during few 
 years this would what i want.


 Another idea would be to go with std.experimental.sci.*, if 
 that has a higher consensus.


Just with std.sci.* without `experimental`. Otherwise this would 
be the same problem.
Apr 17 2016
next sibling parent reply QAston <qaston gmail.com> writes:
On Sunday, 17 April 2016 at 09:09:52 UTC, Ilya Yaroshenko wrote:

 Just with std.sci.* without `experimental`. Otherwise this 
 would be the same problem.


What's wrong with std.experimental.sci.*? experimental is where 
experimental modules/packages should go.
Apr 17 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Sunday, 17 April 2016 at 09:32:25 UTC, QAston wrote:

 On Sunday, 17 April 2016 at 09:09:52 UTC, Ilya Yaroshenko wrote:

 Just with std.sci.* without `experimental`. Otherwise this 
 would be the same problem.


 What's wrong with std.experimental.sci.*? experimental is where 
 experimental modules/packages should go.


This would be a real pain for a user if he supports few compiler 
versions.


static if(__VERSION__ < 2072)
{
	import std.experimental.sci.ndslice;
}
else
{
	import std.sci.ndslice;
}

static if(__VERSION__ < 2074)
{
	import std.experimental.sci.fft;
}
else
{
	import std.sci.fft;
}


static if(__VERSION__ < 2075)
{
	import std.experimental.sci.blas;
}
else
{
	import std.sci.blas;
}

static if(__VERSION__ < 2076)
{
	import std.experimental.sci.lapack;
}
else
{
	import std.sci.lapack;
}
Apr 17 2016
next sibling parent reply ag0aep6g <anonymous example.com> writes:
On 17.04.2016 11:44, Ilya Yaroshenko wrote:

 This would be a real pain for a user if he supports few compiler versions.


 static if(__VERSION__ < 2072)
 {
      import std.experimental.sci.ndslice;
 }
 else
 {
      import std.sci.ndslice;
 }


[...]

Breakage is what you sign up for when using experimental modules.

However, when std.experimental.foo moves over to std.foo, it's probably 
possible to keep the experimental name as an alias for the new one. That 
way you just import std.experimental.foo until a reasonable amount of 
past versions have std.foo. That's how it went with std.typetuple -> 
std.meta.

ndslice should move out of experimental when it's ready, not sooner. If 
this hinders the creation of other std modules, then push for ndslice to 
get ready.
Apr 17 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/17/16 6:04 AM, ag0aep6g wrote:

 On 17.04.2016 11:44, Ilya Yaroshenko wrote:

 This would be a real pain for a user if he supports few compiler
 versions.


 static if(__VERSION__ < 2072)
 {
      import std.experimental.sci.ndslice;
 }
 else
 {
      import std.sci.ndslice;
 }


 [...]

 Breakage is what you sign up for when using experimental modules.

 However, when std.experimental.foo moves over to std.foo, it's probably
 possible to keep the experimental name as an alias for the new one. That
 way you just import std.experimental.foo until a reasonable amount of
 past versions have std.foo. That's how it went with std.typetuple ->
 std.meta.


Yes. In cases of moving code from std.experimental to main package (I 
don't think this has happened yet), we can leave a public re-import in 
std.experiemental for several releases before removing.

And we should remove it. It should not be standard to allow import 
std.experimental when it has been moved to std.

-Steve
Apr 18 2016
prev sibling parent QAston <qaston gmail.com> writes:
On Sunday, 17 April 2016 at 09:44:30 UTC, Ilya Yaroshenko wrote:

 This would be a real pain for a user if he supports few 
 compiler versions.


This approach is wrong because you assume there won't be changes 
to how module works while in experimental and during transition 
from experimental to stable. There will be changes, so supporting 
older compiler versions like that is just asking for bugs. Module 
name change is there so that you won't be using broken api by 
mistake.

Instead you should do the following:
-develop experimental module in std.experimental.sci.*
-do breaking changes to those modules during several release 
cycles
-when module reaches stability move the module to std.sci.*; and 
create a polyfill package for it inside dub registry, like 
https://code.dlang.org/packages/experimental_allocator , so that 
older compilers can use the latest std.sci.* module instead of 
broken std.experimental.sci.* shipped with old compiler.
Apr 17 2016
prev sibling parent reply Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Sunday, 17 April 2016 at 09:09:52 UTC, Ilya Yaroshenko wrote:

 On Sunday, 17 April 2016 at 08:49:53 UTC, Seb wrote:

 On Sunday, 17 April 2016 at 07:35:19 UTC, Ilya Yaroshenko 
 wrote:

 On Sunday, 17 April 2016 at 07:30:38 UTC, Vladimir Panteleev

 I don't understand, what's wrong with std.sci or etc.sci?


 I am ok with std.sci for example. I just want to exclude 
 dances with transaction between stable <->`experimental`. So, 
 if all `std.sci` would be marked as experimental during few 
 years this would what i want.


 Another idea would be to go with std.experimental.sci.*, if 
 that has a higher consensus.


 Just with std.sci.* without `experimental`. Otherwise this 
 would be the same problem.


Why ... ?  Just keep everything in std.experimental.sci until all 
the details are worked out, and only transition to std.sci in one 
go once it's all done.
Apr 22 2016
parent reply Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Friday, 22 April 2016 at 22:11:27 UTC, Joseph Rushton Wakeling 
wrote:

 On Sunday, 17 April 2016 at 09:09:52 UTC, Ilya Yaroshenko wrote:

 On Sunday, 17 April 2016 at 08:49:53 UTC, Seb wrote:

 On Sunday, 17 April 2016 at 07:35:19 UTC, Ilya Yaroshenko 
 wrote:

[...]


 Another idea would be to go with std.experimental.sci.*, if 
 that has a higher consensus.


 Just with std.sci.* without `experimental`. Otherwise this 
 would be the same problem.


 Why ... ?  Just keep everything in std.experimental.sci until 
 all the details are worked out, and only transition to std.sci 
 in one go once it's all done.


3-5 years... And then switch to D3.
Apr 23 2016
parent Joseph Rushton Wakeling <joseph.wakeling webdrake.net> writes:
On Saturday, 23 April 2016 at 09:57:20 UTC, Ilya Yaroshenko wrote:

 On Friday, 22 April 2016 at 22:11:27 UTC, Joseph Rushton 
 Wakeling wrote:

 On Sunday, 17 April 2016 at 09:09:52 UTC, Ilya Yaroshenko 
 wrote:

 On Sunday, 17 April 2016 at 08:49:53 UTC, Seb wrote:

 On Sunday, 17 April 2016 at 07:35:19 UTC, Ilya Yaroshenko 
 wrote:

[...]


 Another idea would be to go with std.experimental.sci.*, if 
 that has a higher consensus.


 Just with std.sci.* without `experimental`. Otherwise this 
 would be the same problem.


 Why ... ?  Just keep everything in std.experimental.sci until 
 all the details are worked out, and only transition to std.sci 
 in one go once it's all done.


 3-5 years... And then switch to D3.


I don't really see the analogy.

If we go your sci.* route, or std.sci.*, then the module naming 
suggests stability, and things require explicit documentation as 
experimental -- and the downstream user has to deal with the 
mental burden of unpicking what is not actually stabilized and 
reliable.

If we go with std.experimental.sci.*, then the module naming 
explicitly documents that the package as a whole is _not_ 
stabilized.  You can document modules that _are_ considered as 
stable and reliable, and in doing so you're making a clearer 
promise to the downstream user.

You could also move stable std.experimental.sci modules to 
std.sci while providing deprecated public imports in the old 
std.experimental.sci module, and those could be maintained for 
the entire lifetime of std.experimental.sci (i.e. until the 
entire package is stabilized).  So again, downstream users would 
only have to pay the full price of a breaking change once.
Apr 23 2016
prev sibling next sibling parent Dicebot <public dicebot.lv> writes:
On 04/17/2016 09:10 AM, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be real pain if
 they would be one-by-one moved from experimental to stable std. So sci.*
 should be considered as experimental during few years.
 
 
 https://github.com/dlang/phobos/pull/4207


This is a good example of why using a dedicated dub category is
completely superior to std.experimental
Apr 17 2016
prev sibling next sibling parent reply Dejan Lekic <dejan.lekic gmail.com> writes:
On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be real 
 pain if they would be one-by-one moved from experimental to 
 stable std. So sci.* should be considered as experimental 
 during few years.


 https://github.com/dlang/phobos/pull/4207


As I have said many times on both IRC and NG, I would prefer to 
have `stdx` instead of ridiculously long (4x) `std.experimental`. 
Then you would have a nice package named `stdx.sci`.

PS. this is not my "invention" - Java community has `javax` for 
similar (but different, as nothing in javax is experimental) 
purpose!
Apr 18 2016
next sibling parent jmh530 <john.michael.hall gmail.com> writes:
On Monday, 18 April 2016 at 16:13:54 UTC, Dejan Lekic wrote:

 As I have said many times on both IRC and NG, I would prefer to 
 have `stdx` instead of ridiculously long (4x) 
 `std.experimental`. Then you would have a nice package named 
 `stdx.sci`.


Fantastic idea (that I had not heard before).
Apr 18 2016
prev sibling next sibling parent Jack Stouffer <jack jackstouffer.com> writes:
On Monday, 18 April 2016 at 16:13:54 UTC, Dejan Lekic wrote:

 As I have said many times on both IRC and NG, I would prefer to 
 have `stdx` instead of ridiculously long (4x) 
 `std.experimental`. Then you would have a nice package named 
 `stdx.sci`.


How often are you typing in import statements that their length 
becomes a problem?
Apr 18 2016
prev sibling next sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 4/18/16 12:13 PM, Dejan Lekic wrote:

 On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be real pain
 if they would be one-by-one moved from experimental to stable std. So
 sci.* should be considered as experimental during few years.


 https://github.com/dlang/phobos/pull/4207


 As I have said many times on both IRC and NG, I would prefer to have
 `stdx` instead of ridiculously long (4x) `std.experimental`. Then you
 would have a nice package named `stdx.sci`.

 PS. this is not my "invention" - Java community has `javax` for similar
 (but different, as nothing in javax is experimental) purpose!


javax, and its history and vagueness, is exactly why we have 
std.experimental. std.exp was also proposed, but rejected because it's 
not obvious what it means.

See, for instance, javax.swing, which was supposed to be promoted to 
java package, but produced a huge backlash for people who were going to 
have to modify their imports.

The idea is to spell out "expect breakage here, this module is 
experimental". If you don't know what this means, then that's too bad. 
But you can't complain that we didn't warn you :)

-Steve
Apr 18 2016
prev sibling parent Dicebot <public dicebot.lv> writes:
On 04/18/2016 07:13 PM, Dejan Lekic wrote:

 On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be real pain
 if they would be one-by-one moved from experimental to stable std. So
 sci.* should be considered as experimental during few years.


 https://github.com/dlang/phobos/pull/4207


 
 As I have said many times on both IRC and NG, I would prefer to have
 `stdx` instead of ridiculously long (4x) `std.experimental`. Then you
 would have a nice package named `stdx.sci`.
 
 PS. this is not my "invention" - Java community has `javax` for similar
 (but different, as nothing in javax is experimental) purpose!


Java is commonly mentioned as example of experimental package concept
that didn't work at all.
Apr 22 2016
prev sibling parent reply deadalnix <deadalnix gmail.com> writes:
On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be real 
 pain if they would be one-by-one moved from experimental to 
 stable std. So sci.* should be considered as experimental 
 during few years.


 https://github.com/dlang/phobos/pull/4207


sci as in ?

If it is as in science, please just name this science. We have 
terabytes of hard drive, we can do with 4 extra characters.
Apr 18 2016
next sibling parent Seb <seb wilzba.ch> writes:
On Tuesday, 19 April 2016 at 01:13:01 UTC, deadalnix wrote:

 On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be 
 real pain if they would be one-by-one moved from experimental 
 to stable std. So sci.* should be considered as experimental 
 during few years.


 https://github.com/dlang/phobos/pull/4207


 sci as in ?

 If it is as in science, please just name this science. We have 
 terabytes of hard drive, we can do with 4 extra characters.


It's the same reason why std is not called "standard" - 
programmers are pretty lazy and a top level module name does 
appear quite often.
Apr 18 2016
prev sibling parent Ilya Yaroshenko <ilyayaroshenko gmail.com> writes:
On Tuesday, 19 April 2016 at 01:13:01 UTC, deadalnix wrote:

 On Sunday, 17 April 2016 at 06:10:34 UTC, Ilya Yaroshenko wrote:

 We plan to add a set of numeric packages and this would be 
 real pain if they would be one-by-one moved from experimental 
 to stable std. So sci.* should be considered as experimental 
 during few years.


 https://github.com/dlang/phobos/pull/4207


 sci as in ?

 If it is as in science, please just name this science. We have 
 terabytes of hard drive, we can do with 4 extra characters.


I decided to do not push packages to Phobos because it is not 
flexible. When Mir would contain enough good packages it may be 
packed with compiler.
Apr 19 2016