www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Why splitter() doesn't work?

reply "Mehrdad" <wfunction hotmail.com> writes:
Why doesn't code like this work?

chain(repeat("a").take(5), repeat("b").take(5)).splitter("b");
Oct 14 2012
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 14 October 2012 at 09:50:20 UTC, Mehrdad wrote:

 Why doesn't code like this work?

 chain(repeat("a").take(5), repeat("b").take(5)).splitter("b");


There's two versions of splitter, the version that splits a range 
using an element, and a version that splits a range using another 
range.

The chain you're using creates a range of strings, and you are 
splitting it using a string, so it appears you are going for the 
first version.

The signature for that version is:

auto splitter(Range, Separator)(Range r, Separator s)
if (is(typeof(ElementType!Range.init == Separator.init))
         && (hasSlicing!Range || isNarrowString!Range))

chain doesn't support slicing (and isn't a narrow string), so you 
can't split it using this function.
Oct 14 2012
parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 19:26:37 UTC, Peter Alexander wrote:

 On Sunday, 14 October 2012 at 09:50:20 UTC, Mehrdad wrote:

 Why doesn't code like this work?

 chain(repeat("a").take(5), repeat("b").take(5)).splitter("b");


 There's two versions of splitter, the version that splits a 
 range using an element, and a version that splits a range using 
 another range.

 The chain you're using creates a range of strings, and you are 
 splitting it using a string, so it appears you are going for 
 the first version.

 The signature for that version is:

 auto splitter(Range, Separator)(Range r, Separator s)
 if (is(typeof(ElementType!Range.init == Separator.init))
         && (hasSlicing!Range || isNarrowString!Range))

 chain doesn't support slicing (and isn't a narrow string), so 
 you can't split it using this function.


Yup, but why does it need slicing?
Oct 14 2012
parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Sunday, 14 October 2012 at 20:03:52 UTC, Mehrdad wrote:

 On Sunday, 14 October 2012 at 19:26:37 UTC, Peter Alexander 
 wrote:

 On Sunday, 14 October 2012 at 09:50:20 UTC, Mehrdad wrote:

 Why doesn't code like this work?

 chain(repeat("a").take(5), repeat("b").take(5)).splitter("b");


 chain doesn't support slicing (and isn't a narrow string), so 
 you can't split it using this function.


 Yup, but why does it need slicing?


Well, for example:

splitter("hello world", ' ')

gives

["hello", "world"]

The splits are slices of the original range.

splitter could allocate new ranges, but if you want that 
behaviour then it's better to specify it manually:

chain(repeat("a").take(5), 
repeat("b").take(5)).array().splitter("b");

That works, giving:

[["a", "a", "a", "a", "a"], [], [], [], [], []]

I wonder if a better design for splitter would automatically 
allocate an array when the input range doesn't support slicing?
Oct 14 2012
next sibling parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 20:09:44 UTC, Peter Alexander wrote:

 Well, for example:

 splitter("hello world", ' ')

 gives

 ["hello", "world"]

 The splits are slices of the original range.

 splitter could allocate new ranges, but if you want that 
 behaviour then it's better to specify it manually:

 chain(repeat("a").take(5), 
 repeat("b").take(5)).array().splitter("b");

 That works, giving:

 [["a", "a", "a", "a", "a"], [], [], [], [], []]

 I wonder if a better design for splitter would automatically 
 allocate an array when the input range doesn't support slicing?



Oh I didn't realize that, it's because it wants to slice the 
original range...

Yeah, I think that might not be such a bad idea. It should be 
possible to slice infinite ranges too, after all.
Oct 14 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, October 14, 2012 22:13:28 Mehrdad wrote:


 Yeah, I think that might not be such a bad idea. It should be
 possible to slice infinite ranges too, after all.


There's actually some discussion on making it so that you can't. The main 
problem is that a slice can't be the same type as the original for an infinite 
range, which tends to screw with things. But if infinite ranges end up losing 
the ability to be sliced, then other, similar functionality would have to be 
added, because you _do_ need to be able to take chunks of an infinite range 
like that.

- Jonathan M Davis
Oct 14 2012
parent reply "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 22:58:27 UTC, Jonathan M Davis 
wrote:

 On Sunday, October 14, 2012 22:13:28 Mehrdad wrote:


 Yeah, I think that might not be such a bad idea. It should be
 possible to slice infinite ranges too, after all.


 There's actually some discussion on making it so that you 
 can't. The main problem is that a slice can't be the same type 
 as the original for an infinite range, which tends to screw 
 with things. But if infinite ranges end up losing the ability 
 to be sliced, then other, similar functionality would have to 
 be added, because you _do_ need to be able to take chunks of an 
 infinite range like that.

 - Jonathan M Davis




How are you supposed to split a range that doesn't support 
slicing though?

You can't just call array() because it might be too big to be 
reasonable for fitting it into memory...
Oct 14 2012
next sibling parent "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 23:11:32 UTC, Mehrdad wrote:

 How are you supposed to split a range that doesn't support 
 slicing though?

 You can't just call array() because it might be too big to be 
 reasonable for fitting it into memory...




(Canonical example: splitting a stream by newlines)
Oct 14 2012
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Monday, October 15, 2012 01:11:28 Mehrdad wrote:

 How are you supposed to split a range that doesn't support
 slicing though?
 
 You can't just call array() because it might be too big to be
 reasonable for fitting it into memory...


If you're splitting on an element, slicing isn't necessary. It's only 
splitting on a range that requires slicing. However, because splitter is lazy, 
I would think that it would be perfectly possible to define it such that it 
didn't need slicing. It would still need a forward range (for the same reason 
that find requires forward ranges if it's not looking for a single element), 
but I would think that you could define it with just forward ranges. Worst 
case, the result would have to be a range over Take!Range rather than over 
slices of Range, and that's basically what infinite ranges have to do when 
slicing anyway. So, unless I'm missing something, I think that splitter is 
currently being overly restrictive in its implementation.

- Jonathan M Davis
Oct 14 2012
prev sibling parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, October 14, 2012 22:09:43 Peter Alexander wrote:

 I wonder if a better design for splitter would automatically
 allocate an array when the input range doesn't support slicing?


We generally try and avoid any kind of allocation like that in std.algorithm. 
We leave it up to the programmer to do that. Otherwise, you'd end up with all 
kinds of invisible performance hits. And it's doubly bad for anyone trying to 
avoid using the GC.

- Jonathan M Davis
Oct 14 2012
parent "Mehrdad" <wfunction hotmail.com> writes:
On Sunday, 14 October 2012 at 23:01:02 UTC, Jonathan M Davis 
wrote:

 On Sunday, October 14, 2012 22:09:43 Peter Alexander wrote:

 I wonder if a better design for splitter would automatically
 allocate an array when the input range doesn't support slicing?


 We generally try and avoid any kind of allocation like that in 
 std.algorithm. We leave it up to the programmer to do that. 
 Otherwise, you'd end up with all kinds of invisible performance 
 hits. And it's doubly bad for anyone trying to avoid using the 
 GC.

 - Jonathan M Davis



Well, you don't need to automatically allocate anything.

As long as you require the input to be a forward range, you can 
simply iterate over it over and over again every time you want to 
return front(). AFAICT no need to store anything anywhere.
Oct 14 2012