www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Voting for std.experimental.checkedint

reply Robert burner Schadek <rburners gmail.com> writes:
This is the voting thread to decide if the proposed addition to 
Phobos, std.experimental.checkedint, should be accepted.

To vote, please respond to this post. You have three options:

* Yes
* Yes with a single condition
* No

If you vote "yes" you can still mention something you'd like 
improved, but please be explicit if that problem is a non starter 
for you and you are choosing option two. If you vote no, please 
state why, though you aren't required to.

Some things to consider when making a vote:

* Is this functionality needed in Phobos?
* The API is practically permanent once the module is accepted. 
Some minor changes can be made, but a full redesign is no longer 
an option.

The voting will end 2017-01-31

The PR can be found here:
https://github.com/dlang/phobos/pull/4613

The dub package can be found here:
http://code.dlang.org/packages/checkedint_andralex

The review thread can be found here:
http://forum.dlang.org/post/mnounbaobgphbmanfaks forum.dlang.org
Jan 13 2017
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Is the doc available somewhere in a readable form ?
Jan 13 2017
parent reply Robert burner Schadek <rburners gmail.com> writes:
On Friday, 13 January 2017 at 12:49:53 UTC, deadalnix wrote:

 Is the doc available somewhere in a readable form ?


CyberShadow/DAutoTest build the docs, you can find the link at 
the end of the PR under checks
Jan 13 2017
parent Bastiaan Veelo <Bastiaan Veelo.net> writes:
On Friday, 13 January 2017 at 13:25:10 UTC, Robert burner Schadek 
wrote:

 On Friday, 13 January 2017 at 12:49:53 UTC, deadalnix wrote:

 Is the doc available somewhere in a readable form ?


 CyberShadow/DAutoTest build the docs, you can find the link at 
 the end of the PR under checks


Readers trying to find that link on their phone should switch to 
the desktop version (link at the very bottom of the page.)

However volatile, at the moment the docs are here: 
http://dtest.thecybershadow.net/artifact/website-f99d0fe6d09e288faf22f3eb515fc56e3c892179-48800882159648c96641690c7485b586/web/phobos-prerelease/std_experimental_checkedint.html

Bastiaan.
Jan 16 2017
prev sibling next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 13 January 2017 at 12:39:38 UTC, Robert burner Schadek 
wrote:

 ...


Overall, the code looks good and the design looks solid. However, 
I have no personal use for such a module, so I can't really 
comment on it's design with any authority.

Abstain.
Jan 14 2017
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Saturday, January 14, 2017 20:54:11 Jack Stouffer via Digitalmars-d 
wrote:

 On Friday, 13 January 2017 at 12:39:38 UTC, Robert burner Schadek

 wrote:

 ...


 Overall, the code looks good and the design looks solid. However,
 I have no personal use for such a module, so I can't really
 comment on it's design with any authority.

 Abstain.


That's pretty much the boat I'm in, though I've never looked at it in depth.
It's one of those things that a few folks seem to think is vital, but I have
zero use for it. It's trying to solve a problem that I simply don't have.

- Jonathan M Davis
Jan 16 2017
parent reply Atila Neves <atila.neves gmail.com> writes:
On Monday, 16 January 2017 at 19:51:38 UTC, Jonathan M Davis 
wrote:

 On Saturday, January 14, 2017 20:54:11 Jack Stouffer via 
 Digitalmars-d wrote:

 On Friday, 13 January 2017 at 12:39:38 UTC, Robert burner 
 Schadek

 wrote:

 ...


 Overall, the code looks good and the design looks solid. 
 However, I have no personal use for such a module, so I can't 
 really comment on it's design with any authority.

 Abstain.


 That's pretty much the boat I'm in, though I've never looked at 
 it in depth. It's one of those things that a few folks seem to 
 think is vital, but I have zero use for it. It's trying to 
 solve a problem that I simply don't have.

 - Jonathan M Davis


Same here.

Atila
Jan 17 2017
parent Guillaume Piolat <first.last gmail.com> writes:
On Tuesday, 17 January 2017 at 11:53:16 UTC, Atila Neves wrote:

 Same here.

 Atila


No interest either. Have zilch problems with integers.
Jan 17 2017
prev sibling next sibling parent Thorsten Sommer <vektoren gmail.com> writes:
Yes
Jan 16 2017
prev sibling next sibling parent Chris Wright <dhasenan gmail.com> writes:
On Fri, 13 Jan 2017 12:39:38 +0000, Robert burner Schadek wrote:


 This is the voting thread to decide if the proposed addition to Phobos,
 std.experimental.checkedint, should be accepted.
 
 To vote, please respond to this post. You have three options:
 
 * Yes * Yes with a single condition * No


Yes.

Most of the time in my code, integer overflow is a bug. I want to defend 
myself against bugs. My new code will use checkedint by default (with 
some convenience aliases, and with regular integers as an option in the 
public interface).

There are some minor documentation changes I would like; I have submitted 
a PR.
Jan 17 2017
prev sibling next sibling parent deadalnix <deadalnix gmail.com> writes:
Alright some feedback.

It is rather disappointing that Warn and Abort only write to 
stderr. Being able to specify the sink would be great. i may want 
to log the issue or something.

There is option to throw on error.

Checked!(Checked!(int, ProperCompare), WithNaN) is rather 
inelegent. Why not Checked!(int, ProperCompare, WithNaN) ?

get() should not be inout. It returns a value type. const is fine.

Otherwise, the overall design looks pretty solid. Congrats to you 
guys. Idealy, I'd like to see these things polished, but I'm 
rather pleased to see where this is going.

I'd say yes, modulo the above.
Jan 17 2017
prev sibling next sibling parent Meta <jared771 gmail.com> writes:
On Friday, 13 January 2017 at 12:39:38 UTC, Robert burner Schadek 
wrote:

 This is the voting thread to decide if the proposed addition to 
 Phobos, std.experimental.checkedint, should be accepted.

 To vote, please respond to this post. You have three options:

 * Yes
 * Yes with a single condition
 * No

 If you vote "yes" you can still mention something you'd like 
 improved, but please be explicit if that problem is a non 
 starter for you and you are choosing option two. If you vote 
 no, please state why, though you aren't required to.

 Some things to consider when making a vote:

 * Is this functionality needed in Phobos?
 * The API is practically permanent once the module is accepted. 
 Some minor changes can be made, but a full redesign is no 
 longer an option.

 The voting will end 2017-01-31

 The PR can be found here:
 https://github.com/dlang/phobos/pull/4613

 The dub package can be found here:
 http://code.dlang.org/packages/checkedint_andralex

 The review thread can be found here:
 http://forum.dlang.org/post/mnounbaobgphbmanfaks forum.dlang.org


Yes, with the comment that this would probably be better as a Dub 
package, at least for the time being. If std.experimental didn't 
exist I would say no outright.
Jan 17 2017
prev sibling parent reply Robert burner Schadek <rburners gmail.com> writes:
checkedint got voted in. With 2 Yes and 2 yes with remarks.

I will set the autotester to merge.

Thank you  andralex for the hard work.
Feb 24 2017
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 2/24/17 4:20 PM, Robert burner Schadek wrote:

 checkedint got voted in. With 2 Yes and 2 yes with remarks.


Remarkably unpopular vote we have here.
If I read it right it implies that
nobody cares for checked integers.


 I will set the autotester to merge.

 Thank you  andralex for the hard work.


---
Dmitry Olshansky
Feb 24 2017
next sibling parent Jack Stouffer <jack jackstouffer.com> writes:
On Friday, 24 February 2017 at 20:37:28 UTC, Dmitry Olshansky 
wrote:

 If I read it right it implies that nobody cares for checked 
 integers.


I guess you can say I don't personally care about them because I 
have no personal use case for them. But, as I said in my remarks, 
I understand why we should have them. Time will tell from users 
if this solution is workable.
Feb 24 2017
prev sibling next sibling parent Ola Fosheim Grostad <ola.fosheim.grostad gmail.com> writes:
On Friday, 24 February 2017 at 20:37:28 UTC, Dmitry Olshansky 
wrote:

 On 2/24/17 4:20 PM, Robert burner Schadek wrote:

 checkedint got voted in. With 2 Yes and 2 yes with remarks.


 Remarkably unpopular vote we have here.
 If I read it right it implies that
 nobody cares for checked integers.


It is more useful as a compiler switch, a type won't help when 
you call into third party libraries.
Feb 24 2017
prev sibling parent reply rumbu <rumbu rumbu.ro> writes:
On Friday, 24 February 2017 at 20:37:28 UTC, Dmitry Olshansky 
wrote:

 On 2/24/17 4:20 PM, Robert burner Schadek wrote:

 checkedint got voted in. With 2 Yes and 2 yes with remarks.


 Remarkably unpopular vote we have here.
 If I read it right it implies that
 nobody cares for checked integers.


A lot of bloat code for something extremely basic.

Newbie asks: How do I check for integer overflow in D?
Response: 
http://dtest.thecybershadow.net/artifact/website-f99d0fe6d09e288faf22f3eb515fc56e3c892179-48800882159648c96641690c7485b586/web/phobos-prerelease/std_experimental_checkedint.html

* newbie runs scared.

My 2 cents.
Feb 25 2017
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 02/25/2017 10:17 AM, rumbu wrote:

 A lot of bloat code for something extremely basic.


If you can do it with less code, I'm all ears. Thanks! -- Andrei
Feb 25 2017
next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Saturday, 25 February 2017 at 15:21:10 UTC, Andrei 
Alexandrescu wrote:

 On 02/25/2017 10:17 AM, rumbu wrote:

 A lot of bloat code for something extremely basic.


 If you can do it with less code, I'm all ears. Thanks! -- Andrei


Perhaps a simpler example for the most basic use case could be 
added near the top. In the heat of solving a problem, 
encountering two pages of theory and explanation for something 
the usage of which should be simple might be discouraging.

Basically, something like:

writeln((checked(5) + 7).get); // 12
writeln((checked(10) * 1000 * 1000 * 1000).get); // Overflow on 
binary operator
Feb 25 2017
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/25/17 11:00 AM, Vladimir Panteleev wrote:

 On Saturday, 25 February 2017 at 15:21:10 UTC, Andrei Alexandrescu wrote:

 On 02/25/2017 10:17 AM, rumbu wrote:

 A lot of bloat code for something extremely basic.


 If you can do it with less code, I'm all ears. Thanks! -- Andrei


 Perhaps a simpler example for the most basic use case could be added
 near the top. In the heat of solving a problem, encountering two pages
 of theory and explanation for something the usage of which should be
 simple might be discouraging.

 Basically, something like:

 writeln((checked(5) + 7).get); // 12
 writeln((checked(10) * 1000 * 1000 * 1000).get); // Overflow on binary
 operator


OK, let's do this:

https://github.com/dlang/phobos/pull/5192
https://github.com/dlang/phobos/pull/5195


Thanks,

Andrei
Feb 25 2017
prev sibling parent reply rumbu <rumbu rumbu.ro> writes:
On Saturday, 25 February 2017 at 15:21:10 UTC, Andrei 
Alexandrescu wrote:

 On 02/25/2017 10:17 AM, rumbu wrote:

 A lot of bloat code for something extremely basic.


 If you can do it with less code, I'm all ears. Thanks! -- Andrei


This was not about coding skills, was about usability. The module 
contains too many options and failure scenarios instead of a 
simple default behavior.

Considering that in most languages with integrated overflow 
checking, the default behavior is throwing some kind of exception 

highlighted at the top of the documentation.
Feb 26 2017
parent reply Seb <seb wilzba.ch> writes:
On Sunday, 26 February 2017 at 09:41:46 UTC, rumbu wrote:

 On Saturday, 25 February 2017 at 15:21:10 UTC, Andrei 
 Alexandrescu wrote:

 On 02/25/2017 10:17 AM, rumbu wrote:

 A lot of bloat code for something extremely basic.


 If you can do it with less code, I'm all ears. Thanks! -- 
 Andrei


 This was not about coding skills, was about usability. The 
 module contains too many options and failure scenarios instead 
 of a simple default behavior.

 Considering that in most languages with integrated overflow 
 checking, the default behavior is throwing some kind of 



If you want a module with a lot less features, the low-level 
core.checkedint might be interesting for you:

http://dlang.org/phobos/core_checkedint.html


 this one must be at least [be] highlighted at the top of the 
 documentation.



It is now: 
http://dlang.org/phobos-prerelease/std_experimental_checkedint.html

If this is still unclear, please submit a PR to improve the docs! 
;-)
Feb 26 2017
next sibling parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Sunday, 26 February 2017 at 09:53:42 UTC, Seb wrote:

 On Sunday, 26 February 2017 at 09:41:46 UTC, rumbu wrote:

 [...]


 If you want a module with a lot less features, the low-level 
 core.checkedint might be interesting for you:

 http://dlang.org/phobos/core_checkedint.html


 [...]



 It is now: 
 http://dlang.org/phobos-prerelease/std_experimental_checkedint.html

 If this is still unclear, please submit a PR to improve the 
 docs! ;-)


The runnable examples fail at compilation with
/d947/f268.d(1): Error: module checkedint is in file 
'std/experimental/checkedint.d' which cannot be read
Feb 26 2017
parent Seb <seb wilzba.ch> writes:
On Sunday, 26 February 2017 at 10:34:07 UTC, Patrick Schluter 
wrote:

 On Sunday, 26 February 2017 at 09:53:42 UTC, Seb wrote:

 On Sunday, 26 February 2017 at 09:41:46 UTC, rumbu wrote:

 [...]


 If you want a module with a lot less features, the low-level 
 core.checkedint might be interesting for you:

 http://dlang.org/phobos/core_checkedint.html


 [...]



 It is now: 
 http://dlang.org/phobos-prerelease/std_experimental_checkedint.html

 If this is still unclear, please submit a PR to improve the 
 docs! ;-)


 The runnable examples fail at compilation with
 /d947/f268.d(1): Error: module checkedint is in file 
 'std/experimental/checkedint.d' which cannot be read


Yes that's expected as DPaste doesn't support dmd-nightly builds 
:/
There's not much I can update this - the maintainer of DPaste 
hasn't been very active recently.
So if you want to do sth. about it, there are two ways:

1) Ping him friendly -> https://github.com/nazriel
2) Write your DPaste replacement (could be based on the 
dlang-tour [1])

[1] https://github.com/stonemaster/dlang-tour/issues/501
Feb 26 2017
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 2/26/17 4:53 AM, Seb wrote:

 On Sunday, 26 February 2017 at 09:41:46 UTC, rumbu wrote:

 On Saturday, 25 February 2017 at 15:21:10 UTC, Andrei Alexandrescu wrote:

 On 02/25/2017 10:17 AM, rumbu wrote:

 A lot of bloat code for something extremely basic.


 If you can do it with less code, I'm all ears. Thanks! -- Andrei


 This was not about coding skills, was about usability. The module
 contains too many options and failure scenarios instead of a simple
 default behavior.

 Considering that in most languages with integrated overflow checking,

 Pascal, Rust, Swift)


 If you want a module with a lot less features, the low-level
 core.checkedint might be interesting for you:

 http://dlang.org/phobos/core_checkedint.html


Thanks for making this point. I agree with the sentiment "you mean I 
need a 1 KLOC library just to check a handful of operations?" This 
paradox is very interesting and worth looking into.

(BTW the number of lines as dscanner --sloc counts is 1261.)

Indeed, the routines in core.checkedint are everything needed (in 
addition to some inline code for comparisons) if the purpose is to check 
operations individually. However, if the intent is to check for errors 
systematically for certain values or program fragments, that doesn't 
scale; before long, the code becomes a bloatfest. Not to mention the 
difficulty in making sure that all operations of interest have been, in 
fact, covered.

So the next logical step is to attempt encapsulation of these checks in 
a type. Here is where one way or another the code bulk must increase, 
and the key question here is how much ability to customize you get per 
unit of code increase.

One issue with checked integers in general, and as a standard (i.e. 
highly reusable) library in particular, is that they are quite project 
specific: what to do upon violation, and which operations to verify and 
which to let run at full speed. As soon as a library does something even 
slightly different from what's necessary, the usability and efficiency 
margins are so narrow, you need to throw the library away and write your 
own. This is very opposite from, say, writing a sorting algorithm 
wherein the API design is very narrow and the difficulty is in the 
algorithm itself. So if you want to write a highly reusable checkedint 
library, you must put ability to customize front, left, and center.

I've started work on an article on DbI, and did a little research on 
other libraries. I found these:

* Mozilla's CheckedInt: 
https://hg.mozilla.org/mozilla-central/file/tip/mfbt/CheckedInt.h, 
clocking at only 791 LoC (no docs and unittests). Though compact and 
ingenious, it makes two design decisions that I think are problematic: 
(a) it stores a "valid" bit (which costs an actual word) together with 
the integral value 
(https://hg.mozilla.org/mozilla-central/file/tip/mfbt/CheckedInt.h#l503), 
which leads to an inefficient layout and also puts all enforcement onus 
on the user; and (b) it separates overflow checks from the actual 
operations, which leads to bulky and inefficient overflow checks (see 
e.g. 
https://hg.mozilla.org/mozilla-central/file/tip/mfbt/CheckedInt.h#l256 
for addition).

* https://safeint.codeplex.com by Microsoft - a behemoth of a library 
clocking at 7055 LoC including comments. Speed is an explicit goal. It 
makes a number of design decisions that might not work for everyone, for 
example:

- accepts (somewhat obliquely) implicit conversion back to the 
representation type, which is kind of defeating the purpose

- taking the address decays to a pointer to unchecked integral (what?)

- has a rigid error policy (either assert or throw)

- the checks and the error handling policies are awkwardly controlled 
via command line instead of template parameters

- binary operators don't work against two SafeInts

- signed/unsigned comparisons are not checked (this is a consequence of 
the implicit decay)

* https://github.com/robertramey/safe_numerics, meant as an addition to 
Boost. That's also a large lbrary (4969 lines with light comments, going 
up to over 10K lines with unittests, and requiring 6 other Boost 
libraries: MPL, Integer, Config, Concept Checking, Tribool, and 
Enable_if). The author also wrote a recent article (Overload Feb 2017) 
that describes the library: 
http://www.rrsd.com/software_development/safe_numerics/Overload137.pdf. 
The article does a great job at motivating such libraries. The facility 
allows good error policy customization, and allows to some extent 
customizing the checks being done (only for promotions). It also has a 
mode that is at least theoretically interesting - it expands the result 
of operations whenever possible to preserve correctness, and refuses to 
compile code that might overflow. I speculate that that feature is of 
very limited use; in just a couple of steps everything goes to 64 bits, 
and we're done. The implementation has the usual genuflections one would 
expect, for example:

    template<class T, class U>
     using calculate_max_t =
         typename boost::mpl::if_c<
             // clause 1 - if both operands have the same sign
             std::numeric_limits<T>::is_signed
             == std::numeric_limits<U>::is_signed,
             // use that sign
             typename boost::mpl::if_c<
                 std::numeric_limits<T>::is_signed,
                 std::intmax_t,
                 std::uintmax_t
             >::type,
         // clause 2 - otherwise if the rank of the unsigned type exceeds
         // the rank of the of the maximum signed type
         typename boost::mpl::if_c<
             (rank< select_unsigned<T, U>>::value
             > rank< std::intmax_t >::value),
             // use unsigned type
             std::uintmax_t,
         // clause 3 - otherwise if the type of the signed integer type can
         // represent all the values of the unsigned type
         typename boost::mpl::if_c<
             std::numeric_limits< std::intmax_t >::digits >=
             std::numeric_limits< select_unsigned<T, U> >::digits,
             // use signed type
             std::intmax_t,
         // clause 4 - otherwise use unsigned version of the signed type
             std::uintmax_t
         >::type >::type >::type;

* https://code.dlang.org/packages/checkedint by Thomas Stuart Bockman. I 
might be biased but it compares favorably against all of the above. A 
major goal of std.experimental.checkedint was to allow more 
customization in a smaller package.


Andrei
Feb 26 2017
parent Walter Bright <newshound2 digitalmars.com> writes:
On 2/26/2017 1:15 PM, Andrei Alexandrescu wrote:

 Indeed, the routines in core.checkedint are everything needed (in addition to
 some inline code for comparisons) if the purpose is to check operations
 individually.


The purpose of core.checkedint is to provide the smallest possible building 
block for doing checked integers. This is to encapsulate it so it:

1. can be made portable

2. can be recognized by the compiler with the potential for using the knowledge 
of the semantics of it to generate /better/faster/reliable/more correct/ code

3. is a clue to the reader of the code what the point of the odd looking 
expressions is

As John Regehr pointed out in a series of articles,

   http://blog.regehr.org/archives/1139

most people do ad-hoc checking which turns out to be very fragile in the face
of 
compiler optimizations and handling of undefined behavior.
Feb 26 2017