www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Phobos uses opSlice and opDollar without checking for it

reply qznc <qznc web.de> writes:
Today I learned [0] that opDollar must be explicitly implemented 
and might not be available by some ranges. Likewise slicing. If 
you think it further, there are lots of functions in Phobos (I'm 
currently looking into std.algorithm.searching) which use more 
features than they check capabilities for. Ranges which satisfy 
isRandomAccessRange do not automatically satisfy hasSlicing.

Can we provide opSlice and opDollar implicitly for random access 
ranges?

Alternatively, we should comb through Phobos to fix the 
capability checks and write unittests with minimalistic ranges.

I already filed issue 16073 [1], but then realized it does not go 
far enough.

Also, std.range.primitives should have a predicate for opDollar 
similar to hasSlicing.

[0] https://github.com/dlang/phobos/pull/4362#discussion_r64576690
[1] https://issues.dlang.org/show_bug.cgi?id=16073
May 25 2016
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, May 25, 2016 at 03:05:33PM +0000, qznc via Digitalmars-d wrote:
 Today I learned [0] that opDollar must be explicitly implemented and
 might not be available by some ranges. Likewise slicing. If you think
 it further, there are lots of functions in Phobos (I'm currently
 looking into std.algorithm.searching) which use more features than
 they check capabilities for. Ranges which satisfy isRandomAccessRange
 do not automatically satisfy hasSlicing.
These are bugs. Please file them and fix them.
 Can we provide opSlice and opDollar implicitly for random access
 ranges?
Perhaps... though this would be a breaking change for existing random access ranges that don't implement opSlice/opDollar.
 Alternatively, we should comb through Phobos to fix the capability
 checks and write unittests with minimalistic ranges.
 
 I already filed issue 16073 [1], but then realized it does not go far
 enough.
 
 Also, std.range.primitives should have a predicate for opDollar
 similar to hasSlicing.
It's hard to imagine having much use for a range that has slicing but not opDollar. For simplicity's sake, I'd suggest keeping the two together. Since we already have hasSlicing, I'd say put the check for opDollar in hasSlicing. T -- People tell me I'm stubborn, but I refuse to accept it!
May 25 2016
parent qznc <qznc web.de> writes:
On Wednesday, 25 May 2016 at 16:21:07 UTC, H. S. Teoh wrote:
 On Wed, May 25, 2016 at 03:05:33PM +0000, qznc via
 Also, std.range.primitives should have a predicate for 
 opDollar similar to hasSlicing.
It's hard to imagine having much use for a range that has slicing but not opDollar. For simplicity's sake, I'd suggest keeping the two together. Since we already have hasSlicing, I'd say put the check for opDollar in hasSlicing.
That would be a breaking change, although breakage is unlikely.
May 25 2016
prev sibling next sibling parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, May 25, 2016 at 03:05:33PM +0000, qznc via Digitalmars-d wrote:
[...]
 Can we provide opSlice and opDollar implicitly for random access ranges?
[...] P.S. If you're talking about using UFCS to provide fallback implementations of opSlice / opDollar, I don't think that's possible because operator overloading doesn't work with UFCS. Besides, how would you even implement opDollar if the range in question is infinite? It's perfectly legit to take a slice [n .. $] of an infinite range, since $ doesn't need to be an actual value. But you couldn't implement such a thing outside the range implementation itself. T -- It always amuses me that Windows has a Safe Mode during bootup. Does that mean that Windows is normally unsafe?
May 25 2016
prev sibling parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wednesday, May 25, 2016 15:05:33 qznc via Digitalmars-d wrote:
 Today I learned [0] that opDollar must be explicitly implemented
 and might not be available by some ranges. Likewise slicing. If
 you think it further, there are lots of functions in Phobos (I'm
 currently looking into std.algorithm.searching) which use more
 features than they check capabilities for. Ranges which satisfy
 isRandomAccessRange do not automatically satisfy hasSlicing.

 Can we provide opSlice and opDollar implicitly for random access
 ranges?

 Alternatively, we should comb through Phobos to fix the
 capability checks and write unittests with minimalistic ranges.

 I already filed issue 16073 [1], but then realized it does not go
 far enough.

 Also, std.range.primitives should have a predicate for opDollar
 similar to hasSlicing.

 [0] https://github.com/dlang/phobos/pull/4362#discussion_r64576690
 [1] https://issues.dlang.org/show_bug.cgi?id=16073
hasSlicing and isRandomAccessRange do require that opDollar work correctly if they're there, but it would have been way too large a breaking change to make them require opDollar, and it's kind of silly to make them implement opDollar when the compiler could generate for them from length, which is why https://issues.dlang.org/show_bug.cgi?id=7177 exists. Unfortunately, it's never been implemented (IIRC due to issues related to AAs having length but not being ranges; the implementation likely needs to check for the range-based functions in addition to length). And as long as 7177 hasn't been implemented, hasSlicind and isRandomAccessRange can't require $, and until they require opDollar, Phobos should never use $ on ranges unless they're guaranteed to be arrays or guaranteed to be ranges that implemented opDollar, and that's really true of _any_ D code at this point, not just Phobos. IMHO, we really should implement some form of 7177 so that we can actually make it possible to use $ with ranges, but until that happens, we're out of luck. - Jonathan M Davis
May 25 2016
parent reply Era Scarecrow <rtcvb32 yahoo.com> writes:
On Wednesday, 25 May 2016 at 21:43:19 UTC, Jonathan M Davis wrote:
 Unfortunately, it's never been implemented (IIRC due to issues 
 related to AAs having length but not being ranges; the 
 implementation likely needs to check for the range-based 
 functions in addition to length). And as long as 7177 hasn't 
 been implemented, hasSlicing and isRandomAccessRange can't 
 require $, and until they require opDollar, Phobos should never 
 use $ on ranges unless they're guaranteed to be arrays or 
 guaranteed to be ranges that implemented opDollar, and that's 
 really true of _any_ D code at this point, not just Phobos.
Hmmm if an algorithm can make use of opDollar, we should be able to test for it. Maybe extend the tests that include it say hasFullSlicing? Unless something explicitly needs opDollar then it probably shouldn't be included. Someone's already mentioned infinite ranges/arrays, and AA not using $ would cause major breakages. Are there any major patterns that we can't implement without opDollar? Or are there any that would perform better with it present?
May 25 2016
parent reply Jonathan M Davis via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wednesday, May 25, 2016 22:08:31 Era Scarecrow via Digitalmars-d wrote:
 On Wednesday, 25 May 2016 at 21:43:19 UTC, Jonathan M Davis wrote:
 Unfortunately, it's never been implemented (IIRC due to issues
 related to AAs having length but not being ranges; the
 implementation likely needs to check for the range-based
 functions in addition to length). And as long as 7177 hasn't
 been implemented, hasSlicing and isRandomAccessRange can't
 require $, and until they require opDollar, Phobos should never
 use $ on ranges unless they're guaranteed to be arrays or
 guaranteed to be ranges that implemented opDollar, and that's
 really true of _any_ D code at this point, not just Phobos.
Hmmm if an algorithm can make use of opDollar, we should be able to test for it. Maybe extend the tests that include it say hasFullSlicing? Unless something explicitly needs opDollar then it probably shouldn't be included. Someone's already mentioned infinite ranges/arrays, and AA not using $ would cause major breakages.
Infinite ranges will need to implement opDollar if they want to work with opDollar, but infinite ranges typically aren't sliceable anyway. However, hasSlicing does test that if they use opDollar correctly if they support it. I think that it's overkill to add something like hasFullSlicing to have a versions of hasSlicing that requires opDollar. It's downright silly really that we have slicing without it, but I really don't think that it's worth adding yet another range trait to check for it. At this point, generic code should just never use opDollar unless it specifically tests for it, and usually, there's really no reason to do that rather than just using length at this point, much as it's far more aesthetically pleasing to use $. But ranges really should support $ just like arrays do, so the current state of things is definitely annoying.
   Are there any major patterns that we can't implement without
 opDollar? Or are there any that would perform better with it
 present?
The only reason you would _need_ opDollar is for slicing infinite ranges to the end, and most infinite ranges aren't sliceable. Otherwise, using opDollar and using explicit length should be identical. And if they aren't there's either something funny about how slicing was implemented and/or the optimizer isn't doing its job. - Jonathan M Davis
May 25 2016
parent qznc <qznc web.de> writes:
On Thursday, 26 May 2016 at 06:07:36 UTC, Jonathan M Davis wrote:
 At this point, generic code should just never use opDollar 
 unless it specifically tests for it, and usually, there's 
 really no reason to do that rather than just using length at 
 this point, much as it's far more aesthetically pleasing to use 
 $. But ranges really should support $ just like arrays do, so 
 the current state of things is definitely annoying.
The most common uses are [$..$] and [x..$]. I think those should simply be fixed to take(0) and drop(x) from the range primitives. They are as aesthetically pleasing, generic for any ranges, and efficient for arrays.
May 26 2016