www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - how to skip the next (n) item & continue from (n+1) with a range ?

reply mw <mingwu gmail.com> writes:
Hi,

I need this logic:

```
auto range = File(fn).byLine();
foreach (line; range) {
   if (comeCond(line)) {
      // skip the next n line
      // and continue the foreach loop from the (n+1) line
   } else {
      regularProcess(line);
   }
}
```

Is it possible to do this in a foreach loop?

If not, how can I achieve that, esp. when reading a file 
line-by-line?
do I have to read the whole file into memory, and split by line 
and using regular `for` loop?
Jun 22 2020
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/22/20 3:53 PM, mw wrote:

 Hi,
 
 I need this logic:
 
 ```
 auto range = File(fn).byLine();
 foreach (line; range) {
    if (comeCond(line)) {
       // skip the next n line
       // and continue the foreach loop from the (n+1) line
    } else {
       regularProcess(line);
    }
 }
 ```
 
 Is it possible to do this in a foreach loop?


I wouldn't recommend it, instead do a while loop:

auto range = File(fn).byLine;

while(!range.empty)
{
    auto line = range.front;
    if(someCond(line)) {
         range.popFrontN(n);
    } else {
         regularProcess(line);
         range.popFront;
    }
}

-Steve
Jun 22 2020
parent reply mw <mingwu gmail.com> writes:
On Monday, 22 June 2020 at 20:00:50 UTC, Steven Schveighoffer 
wrote:

 I wouldn't recommend it, instead do a while loop:

 auto range = File(fn).byLine;

 while(!range.empty)
 {
    auto line = range.front;
    if(someCond(line)) {
         range.popFrontN(n);
    } else {
         regularProcess(line);
         range.popFront;
    }
 }


Thanks.

so `front` is peek, and `popFront` is the pop action whose return 
type is `void`, why we need two *separate* calls instead of just 
let `popFront` return T (or do we have another function for 
this)? i.e if the user forget `popFront`, it will prone to 
infinite loop bug?
Jun 22 2020
next sibling parent reply mw <mingwu gmail.com> writes:
On Monday, 22 June 2020 at 20:46:30 UTC, mw wrote:

 On Monday, 22 June 2020 at 20:00:50 UTC, Steven Schveighoffer 
 wrote:

 I wouldn't recommend it, instead do a while loop:

 auto range = File(fn).byLine;

 while(!range.empty)
 {
    auto line = range.front;
    if(someCond(line)) {
         range.popFrontN(n);




I'm asking this, because here it need to be range.popFrontN(n+1);

i.e. bug-prone

can be fixed by:
    auto line = range.front;
    range.popFront;  // pop immediately



    } else {
         regularProcess(line);
         range.popFront;
    }
 }


 Thanks.

 so `front` is peek, and `popFront` is the pop action whose 
 return type is `void`, why we need two *separate* calls instead 
 of just let `popFront` return T (or do we have another function 
 for this)? i.e if the user forget `popFront`, it will prone to 
 infinite loop bug?


so my question.
Jun 22 2020
next sibling parent reply mw <mingwu gmail.com> writes:
On Monday, 22 June 2020 at 20:49:55 UTC, mw wrote:

 On Monday, 22 June 2020 at 20:46:30 UTC, mw wrote:

 On Monday, 22 June 2020 at 20:00:50 UTC, Steven Schveighoffer 
 wrote:

 I wouldn't recommend it, instead do a while loop:

 auto range = File(fn).byLine;

 while(!range.empty)
 {
    auto line = range.front;
    if(someCond(line)) {
         range.popFrontN(n);




 I'm asking this, because here it need to be


either


 range.popFrontN(n+1);

 i.e. bug-prone


or


 can be fixed by:
    auto line = range.front;
    range.popFront;  // pop immediately


of course.
Jun 22 2020
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Mon, Jun 22, 2020 at 08:51:49PM +0000, mw via Digitalmars-d-learn wrote:
[...]

    auto line = range.front;
    range.popFront;  // pop immediately




[...]

This is dangerous, because it assumes .front is not invalidated by
.popFront.  It will not work, for example, with byLine because .front
returns a buffer which is reused by .popFront (a so-called "transient
range").

In the realm of defensive programming, it is better to make less
assumptions (don't assume .front remains valid after .popFront) than add
implicit assumptions (range is non-transient) that may break in subtle
ways that are not immediately obvious.


T

-- 
Why ask rhetorical questions? -- JC
Jun 22 2020
parent mw <mingwu gmail.com> writes:
On Monday, 22 June 2020 at 21:22:10 UTC, H. S. Teoh wrote:

 On Mon, Jun 22, 2020 at 08:51:49PM +0000, mw via 
 Digitalmars-d-learn wrote: [...]

    auto line = range.front;
    range.popFront;  // pop immediately




 [...]

 This is dangerous, because it assumes .front is not invalidated 
 by .popFront.  It will not work, for example, with byLine 
 because .front returns a buffer which is reused by .popFront (a 
 so-called "transient range").


This is valid reason, but as I replied in the other post: it 
depends on the actual underlying data structure and usage 
scenario, so:

"why not provide a popAndReturnFront(), and let the user decide 
based on his/her own actual usage?"
Jun 22 2020
prev sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 6/22/20 4:49 PM, mw wrote:

 On Monday, 22 June 2020 at 20:46:30 UTC, mw wrote:

 On Monday, 22 June 2020 at 20:00:50 UTC, Steven Schveighoffer wrote:

 I wouldn't recommend it, instead do a while loop:

 auto range = File(fn).byLine;

 while(!range.empty)
 {
    auto line = range.front;
    if(someCond(line)) {
         range.popFrontN(n);




 
 I'm asking this, because here it need to be range.popFrontN(n+1);


`n` actually isn't defined, you defined it in a comment in your original 
code ;) I just threw it in there. Of course, make sure it works how you 
are expecting, I don't know what your code is doing.


 
 i.e. bug-prone
 
 can be fixed by:
     auto line = range.front;
     range.popFront;  // pop immediately


This is a bad idea, once you popFront, the original front is possibly 
invalid (and technically is the case for byLine).


 
 

    } else {
         regularProcess(line);
         range.popFront;
    }
 }


 Thanks.

 so `front` is peek, and `popFront` is the pop action whose return type 
 is `void`, why we need two *separate* calls instead of just let 
 `popFront` return T (or do we have another function for this)? i.e if 
 the user forget `popFront`, it will prone to infinite loop bug?


 
 so my question.


There is no requirement to actually construct members. e.g. popFrontN 
calls popFront N times, but does not actually invoke front at all.

Separating the concerns is for correctness and performance.

-Steve
Jun 22 2020
parent mw <mingwu gmail.com> writes:
On Monday, 22 June 2020 at 21:27:12 UTC, Steven Schveighoffer 
wrote:

     auto line = range.front;
     range.popFront;  // pop immediately


 This is a bad idea, once you popFront, the original front is 
 possibly invalid (and technically is the case for byLine).


In this case, it's caused by underlying structure may reuse the 
`byLine` buffer, but I'm asking a more general question about 
range interface: why not provide an extra popAndReturnFront(), 
and the user to choose in the actual usage scenario.
Jun 22 2020
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 6/22/20 1:46 PM, mw wrote:


 so `front` is peek, and `popFront` is the pop action whose return type
 is `void`, why we need two *separate* calls instead of just let
 `popFront` return T


Others have other explanations for this but my understanding is about 
exception safety: If it changed internal state and returned the front 
object, you would no be able to make a function like popFront() strongly 
exception safe. (There are ample documentation for this topic in C++ 
circles.)

Another reason is cohesion: We want functions to have as little 
responsibility as possible (ideally single).


 (or do we have another function for this)?


There are many useful functions in std.range:

   https://dlang.org/phobos/std_range.html

The "take" and "drop" functions may be useful.

Ali
Jun 22 2020
parent reply mw <mingwu gmail.com> writes:
On Monday, 22 June 2020 at 20:58:58 UTC, Ali Çehreli wrote:


 Others have other explanations for this but my understanding is 
 about exception safety: If it changed internal state and 
 returned the front object, you would no be able to make a 
 function like popFront() strongly exception safe. (There are 
 ample documentation for this topic in C++ circles.)


That's one consideration. But, it will be more interesting in 
knowing (based on actual usage):

(a) how many bugs % are due to exception un-safe
(b) how many bugs % are due to front / popFront separation?

And which is more bug-prone for a typical programmer? my gut 
feeling is (b), at least we just saw one in this thread.


And

-- loop thru a static structure content like a simple array, why 
we need to worry about exception safe?

-- loop thru dynamically generated content, esp. network or more 
external complex structure may worth considering exception 
safety. But even there, do people always need to call 
!range.empty() check first? when it's not empty, how much more 
exception un-safety that popAndReturnFront() can introduce than 
the combination of `front(); ...; popFront();"?

And why not provide a popAndReturnFront(), and let the user 
decide based on his/her own actual usage?


 Another reason is cohesion: We want functions to have as little 
 responsibility as possible (ideally single).


Again we have to weight which way is more bug-prone, any actual 
statistics on the above (a) v.s (b)?



 (or do we have another function for this)?


 There are many useful functions in std.range:

   https://dlang.org/phobos/std_range.html

 The "take" and "drop" functions may be useful.


Use these functions inside a while(!range.empty()) {...} can only 
introduce more code complexity.
Jun 22 2020
parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 6/22/20 2:37 PM, mw wrote:


 On Monday, 22 June 2020 at 20:58:58 UTC, Ali =C3=87ehreli wrote:


 Others have other explanations for this but my understanding is about=





 exception safety: If it changed internal state and returned the front=





 object, you would no be able to make a function like popFront()
 strongly exception safe. (There are ample documentation for this topi=




c

 in C++ circles.)


 That's one consideration. But, it will be more interesting in knowing
 (based on actual usage):

 (a) how many bugs % are due to exception un-safe


I am not claiming that strong exception safety is the reason for Phobos=20
design. However, knowing what I know, I would use the same primitive=20
operations. It's the same with e.g. C++ as well: !=3D end(), operator*,=20
operator++. And operator++ does not return the current object either.

Even if zero bugs are due to exception un-safe, a library designer would =

not oversee that knowledge. It is impossible to make an interface=20
strongly exception safe but the reverse is always possible.


 (b) how many bugs % are due to front / popFront separation?


I made the mistake of forgetting to call popFront() perhaps 10 times and =

got stuck in an infinite loop and quickly hit a segmentation fault and=20
that was it.


 And which is more bug-prone for a typical programmer? my gut feeling i=


s

 (b), at least we just saw one in this thread.


 And

 -- loop thru a static structure content like a simple array, why we ne=


ed

 to worry about exception safe?


*If* strong exception guarantee is needed, it doesn't matter whether=20
it's a simple array or not.


 -- loop thru dynamically generated content, esp. network or more
 external complex structure may worth considering exception safety. But=



 even there, do people always need to call !range.empty() check first?
 when it's not empty, how much more exception un-safety that
 popAndReturnFront() can introduce than the combination of `front(); ..=


=2E;

 popFront();"?


It has been demonstrated on a Stack type that conflating top() and pop() =

cannot be made strongly exception safe. It's the same with front() and=20
popFront().


 And why not provide a popAndReturnFront(), and let the user decide bas=


ed

 on his/her own actual usage?


Because if the primitives were empty() and popAndReturnFront(), then it=20
wouldn't be made strongly exception safe. With the current primitives of =

empty(), front(), and popFront(), it's possible to implement=20
popAndReturnFront(). I like the following one that I wrote in three=20
minutes. :)

import std.stdio;

auto popAndReturnFront(R)(ref R range) {
   import std.range : empty, front, popFront, ElementType;
   import std.typecons : Nullable, nullable;

   if (range.empty) {
     return Nullable!(ElementType!R)();
   }

   scope (success) range.popFront();
   return nullable(range.front);
}

void main() {
   auto range =3D [ 1, 2, 3 ];
   while (true) {
     auto front =3D range.popAndReturnFront;
     if (!front.get()) {
       break;
     }
     writeln(front);
   }
}


 Another reason is cohesion: We want functions to have as little
 responsibility as possible (ideally single).


 Again we have to weight which way is more bug-prone, any actual
 statistics on the above (a) v.s (b)?


I am not aware of any bugs related to the separation of front() and=20
popFront().

Ali
Jun 22 2020
prev sibling parent reply kinke <noone nowhere.com> writes:
If you are referring to the next line, not the next n lines, 
that's a simple `continue;` statement.
Jun 22 2020
parent kinke <noone nowhere.com> writes:
On Monday, 22 June 2020 at 20:02:22 UTC, kinke wrote:

 If you are referring to the next line, not the next n lines, 
 that's a simple `continue;` statement.


[Please discard, that'd obviously be skipping the *current* line.]
Jun 22 2020