www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - writeln wipes contents of variables ?

reply W.J. <invalid email.address> writes:
Hi everybody!

I'm new to D and trying to wrap my head around ranges.

For a start I'm trying to take a input string and transform it, 
group it, etc.
After each step I inspect the result.
It works fine until step 3 where all of a sudden, after 
write(ln)ing the result, "step3" holds an array of empty arrays.
What gives? Program and output are attached below.

My question is why are "step3"'s contents before and after the 
writeln (1) different?
If I remove this line at (1) the foreach loops print the contents 
as I'd expect and the following writeln (2) prints the contents 
as expected, too.
However the writeln (3) prints empty arrays again.
I don't understand this behavior and I'd really like to know 
what's going on.

Your help and time is much appreciated!

I'm using an unmodified local build of dmd, druntime, and phobos 
from github updated and recompiled today.

[code]
import std.range;
import std.stdio;
import std.algorithm;

void main() {
   string test = " \r  node1\n    nodea\r\n    key:val\n\n  
node2\n    nodea\n";
   writeln(test);
   writeln("---");

   auto step1 = test.replace("\r\n", "\n")
                 .replace("  ", "\t")
                 .splitter!"a == 10 || a == 13"()
                 ;
   writeln("Step 1:", step1);
   writeln("---");
   writeln("  type: ", typeid(step1));
   writeln("---");

   import std.typecons: Tuple;
   string[][] step2;
   foreach(a; step1) {
     auto e = findSplitAfter(a, "\t").array;
     step2 ~= e;
   }
   writeln("Step 2:", step2);
   writeln("---");
   writeln("  type: ", typeid(step2));
   writeln("---");

   auto step3 = step2.chunkBy!((a,b) => a[0]==b[0]).array;
   writeln("Step 3:", step3); // (1)
   writeln("---");
   writeln("  type: ", typeid(step3));
   writeln("-+-");

   foreach(x; step3) {
     writeln("x:",typeid(x),x);
     foreach(y; x)
       writeln("  y:",typeid(y),y);
   }
   writeln("--",step3); // (2)
   writeln("--",step3); // (3)
   writeln("the end");
}
[/code]

Output:
[code]
   node1
     nodea
     key:val

   node2
     nodea

---
Step 1:[" ", "\tnode1", "\t\tnodea", "\t\tkey:val", "", 
"\tnode2", "\t\tnodea", ""]
---
   type: std.algorithm.iteration.SplitterResult!(unaryFun, 
string).SplitterResult
---
Step 2:[["", " "], ["\t", "node1"], ["\t", "\tnodea"], ["\t", 
"\tkey:val"], ["", ""], ["\t", "node2"], ["\t", "\tnodea"], ["", 
""]]
---
   type: immutable(char)[][][]
---
Step 3:[[["", " "]], [["\t", "node1"], ["\t", "\tnodea"], ["\t", 
"\tkey:val"]], [["", ""]], [["\t", "node2"], ["\t", "\tnodea"]], 
[["", ""]]]
---
   type: app.main.ChunkByImpl!(__lambda1, 
string[][]).ChunkByImpl.Group[]
-+-
x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
--[[], [], [], [], []]
the end
[/code]
Jan 21 2016
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 22/01/16 2:11 AM, W.J. wrote:

 Hi everybody!

 I'm new to D and trying to wrap my head around ranges.

 For a start I'm trying to take a input string and transform it, group
 it, etc.
 After each step I inspect the result.
 It works fine until step 3 where all of a sudden, after write(ln)ing the
 result, "step3" holds an array of empty arrays.
 What gives? Program and output are attached below.

 My question is why are "step3"'s contents before and after the writeln
 (1) different?
 If I remove this line at (1) the foreach loops print the contents as I'd
 expect and the following writeln (2) prints the contents as expected, too.
 However the writeln (3) prints empty arrays again.
 I don't understand this behavior and I'd really like to know what's
 going on.

 Your help and time is much appreciated!

 I'm using an unmodified local build of dmd, druntime, and phobos from
 github updated and recompiled today.

 [code]
 import std.range;
 import std.stdio;
 import std.algorithm;

 void main() {
    string test = " \r  node1\n    nodea\r\n    key:val\n\n node2\n
 nodea\n";
    writeln(test);
    writeln("---");

    auto step1 = test.replace("\r\n", "\n")
                  .replace("  ", "\t")
                  .splitter!"a == 10 || a == 13"()
                  ;
    writeln("Step 1:", step1);
    writeln("---");
    writeln("  type: ", typeid(step1));
    writeln("---");

    import std.typecons: Tuple;
    string[][] step2;
    foreach(a; step1) {
      auto e = findSplitAfter(a, "\t").array;
      step2 ~= e;
    }
    writeln("Step 2:", step2);
    writeln("---");
    writeln("  type: ", typeid(step2));
    writeln("---");

    auto step3 = step2.chunkBy!((a,b) => a[0]==b[0]).array;
    writeln("Step 3:", step3); // (1)
    writeln("---");
    writeln("  type: ", typeid(step3));
    writeln("-+-");

    foreach(x; step3) {
      writeln("x:",typeid(x),x);
      foreach(y; x)
        writeln("  y:",typeid(y),y);
    }
    writeln("--",step3); // (2)
    writeln("--",step3); // (3)
    writeln("the end");
 }
 [/code]

 Output:
 [code]
    node1
      nodea
      key:val

    node2
      nodea

 ---
 Step 1:[" ", "\tnode1", "\t\tnodea", "\t\tkey:val", "", "\tnode2",
 "\t\tnodea", ""]
 ---
    type: std.algorithm.iteration.SplitterResult!(unaryFun,
 string).SplitterResult
 ---
 Step 2:[["", " "], ["\t", "node1"], ["\t", "\tnodea"], ["\t",
 "\tkey:val"], ["", ""], ["\t", "node2"], ["\t", "\tnodea"], ["", ""]]
 ---
    type: immutable(char)[][][]
 ---
 Step 3:[[["", " "]], [["\t", "node1"], ["\t", "\tnodea"], ["\t",
 "\tkey:val"]], [["", ""]], [["\t", "node2"], ["\t", "\tnodea"]], [["",
 ""]]]
 ---
    type: app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
 -+-
 x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
 x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
 x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
 x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
 x:app.main.ChunkByImpl!(__lambda1, string[][]).ChunkByImpl.Group[]
 --[[], [], [], [], []]
 the end
 [/code]


Ok so input ranges.
An input range is a little bit like an iterator (if you know what that is).

When an input range has been read fully, it is empty aka no longer has 
any values associated with it.

writeln, reads an input range fully (since you can't ask for what is 
next without removing the current item) and outputs each entry.

So yes, writeln will remove all entries from an input range.
Note however it will not do this for arrays since you can read anywhere 
within them.
Jan 21 2016
parent reply W.J. <invalid email.address> writes:
On Thursday, 21 January 2016 at 13:15:46 UTC, Rikki Cattermole 
wrote:

 Ok so input ranges.
 An input range is a little bit like an iterator (if you know 
 what that is).

 When an input range has been read fully, it is empty aka no 
 longer has any values associated with it.

 writeln, reads an input range fully (since you can't ask for 
 what is next without removing the current item) and outputs 
 each entry.

 So yes, writeln will remove all entries from an input range.
 Note however it will not do this for arrays since you can read 
 anywhere within them.



Thanks for your reply.

So writeln consumes the values in an InputRange. That leads me to 
believe that if I feed an InputRange to foreach, it will consume 
the values, too.
Did I get that right ?

If that's the case, why I can iterate and write the values just 
fine using nested foreach loops *and* writeln them afterwards ?
I would expect the foreach loops leaves "step3" in the same state 
as writeln does.

So I'm still a little confused as to how foreach goes about it. I 
thought foreach would be iterating an InputRange the same way 
writeln does.

By attaching ".array" to the result of I was under the impression 
the result was actually converted to an array.
That seems not to be the case. What I probably got was something 
like an array of arrays of groups - which are apparently 
InputRanges ?

So, yes, writeln... InputRange... makes perfect sense except for 
the foreach case.

Thanks again for the help :)
Jan 21 2016
next sibling parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 22/01/16 3:07 AM, W.J. wrote:

 On Thursday, 21 January 2016 at 13:15:46 UTC, Rikki Cattermole wrote:

 Ok so input ranges.
 An input range is a little bit like an iterator (if you know what that
 is).

 When an input range has been read fully, it is empty aka no longer has
 any values associated with it.

 writeln, reads an input range fully (since you can't ask for what is
 next without removing the current item) and outputs each entry.

 So yes, writeln will remove all entries from an input range.
 Note however it will not do this for arrays since you can read
 anywhere within them.



 Thanks for your reply.

 So writeln consumes the values in an InputRange. That leads me to
 believe that if I feed an InputRange to foreach, it will consume the
 values, too.
 Did I get that right ?


Correct.


 If that's the case, why I can iterate and write the values just fine
 using nested foreach loops *and* writeln them afterwards ?
 I would expect the foreach loops leaves "step3" in the same state as
 writeln does.

 So I'm still a little confused as to how foreach goes about it. I
 thought foreach would be iterating an InputRange the same way writeln does.

 By attaching ".array" to the result of I was under the impression the
 result was actually converted to an array.
 That seems not to be the case. What I probably got was something like an
 array of arrays of groups - which are apparently InputRanges ?

 So, yes, writeln... InputRange... makes perfect sense except for the
 foreach case.

 Thanks again for the help :)
Jan 21 2016
parent reply W.J. <invalid email.address> writes:
On Thursday, 21 January 2016 at 14:36:36 UTC, Rikki Cattermole 
wrote:

 On 22/01/16 3:07 AM, W.J. wrote:

 On Thursday, 21 January 2016 at 13:15:46 UTC, Rikki Cattermole 
 wrote:

 [...]



 Thanks for your reply.

 So writeln consumes the values in an InputRange. That leads me 
 to
 believe that if I feed an InputRange to foreach, it will 
 consume the
 values, too.
 Did I get that right ?


 Correct.


 If that's the case, why I can iterate and write the values 
 just fine
 using nested foreach loops *and* writeln them afterwards ?
 I would expect the foreach loops leaves "step3" in the same 
 state as
 writeln does.

 So I'm still a little confused as to how foreach goes about 
 it. I
 thought foreach would be iterating an InputRange the same way 
 writeln does.

 By attaching ".array" to the result of I was under the 
 impression the
 result was actually converted to an array.
 That seems not to be the case. What I probably got was 
 something like an
 array of arrays of groups - which are apparently InputRanges ?

 So, yes, writeln... InputRange... makes perfect sense except 
 for the
 foreach case.

 Thanks again for the help :)




After playing around some more and apart from requiring much more 
practice I think this topic can be marked as 'solved'. However, I 
can't seem to find the edit button for my initial post.

Anyways, thanks for your contributions :) Very much appreciated!
Jan 21 2016
parent reply Rikki Cattermole <alphaglosined gmail.com> writes:
On 22/01/16 5:45 AM, W.J. wrote:

 On Thursday, 21 January 2016 at 14:36:36 UTC, Rikki Cattermole wrote:

 On 22/01/16 3:07 AM, W.J. wrote:

 On Thursday, 21 January 2016 at 13:15:46 UTC, Rikki Cattermole wrote:

 [...]



 Thanks for your reply.

 So writeln consumes the values in an InputRange. That leads me to
 believe that if I feed an InputRange to foreach, it will consume the
 values, too.
 Did I get that right ?


 Correct.


 If that's the case, why I can iterate and write the values just fine
 using nested foreach loops *and* writeln them afterwards ?
 I would expect the foreach loops leaves "step3" in the same state as
 writeln does.

 So I'm still a little confused as to how foreach goes about it. I
 thought foreach would be iterating an InputRange the same way writeln
 does.

 By attaching ".array" to the result of I was under the impression the
 result was actually converted to an array.
 That seems not to be the case. What I probably got was something like an
 array of arrays of groups - which are apparently InputRanges ?

 So, yes, writeln... InputRange... makes perfect sense except for the
 foreach case.

 Thanks again for the help :)




 After playing around some more and apart from requiring much more
 practice I think this topic can be marked as 'solved'. However, I can't
 seem to find the edit button for my initial post.

 Anyways, thanks for your contributions :) Very much appreciated!


Contrary to what it is called in the links, this is a NewsGroup/Mailing 
list and not a forum. So no editing ability :)
Jan 21 2016
parent W.J. <invalid email.address> writes:
On Friday, 22 January 2016 at 01:47:19 UTC, Rikki Cattermole 
wrote:

 Contrary to what it is called in the links, this is a 
 NewsGroup/Mailing list and not a forum. So no editing ability :)



I'll keep that in mind. Thanks :)
Jan 22 2016
prev sibling parent reply Chris Wright <dhasenan gmail.com> writes:
On Thu, 21 Jan 2016 14:07:16 +0000, W.J. wrote:

 So writeln consumes the values in an InputRange. That leads me to
 believe that if I feed an InputRange to foreach, it will consume the
 values, too.
 Did I get that right ?


In general, yes.

Some ranges have value semantics and can be saved simply by assigning 
them to a new variable, or passing them to a function.
Jan 21 2016
parent reply W.J. <invalid email.address> writes:
On Thursday, 21 January 2016 at 21:59:10 UTC, Chris Wright wrote:

 On Thu, 21 Jan 2016 14:07:16 +0000, W.J. wrote:

 So writeln consumes the values in an InputRange. That leads me 
 to
 believe that if I feed an InputRange to foreach, it will 
 consume the
 values, too.
 Did I get that right ?


 In general, yes.


Thanks for your reply.


 Some ranges have value semantics and can be saved simply by 
 assigning them to a new variable, or passing them to a function.


How can I identify those ranges, or, how can I tell if any 
particular range has value semantics ? I didn't read any of this 
in the manual - not that I could remember anyways.

Thanks for your help!
Jan 21 2016
parent reply anonymous <anonymous example.com> writes:
On 22.01.2016 01:49, W.J. wrote:

 How can I identify those ranges, or, how can I tell if any particular
 range has value semantics ? I didn't read any of this in the manual -
 not that I could remember anyways.


Generally you shouldn't. If you care about it either way, use .save or 
std.range.refRange.

If you don't want some range r to be consumed by some operation, pass 
r.save instead of plain r. If you want r to be consumed, pass 
refRange(&r). Only if you don't care if r is consumed or not, should you 
pass simply r.

If you know for a fact that copying r is the same as r.save, then you 
can just pass (and copy) r, of course. We know it's that way with 
dynamic arrays, because of their nature as pointer+length structures. 
But there's nothing wrong with calling .save on an array anyway.

Also, when a function takes a range via a ref parameter, then you don't 
need refRange, of course. The ref parameter ensures that no copy is made 
and that the original range is affected by the function.
Jan 21 2016
parent W.J. <invalid email.address> writes:
On Friday, 22 January 2016 at 01:49:58 UTC, anonymous wrote:

 On 22.01.2016 01:49, W.J. wrote:

 How can I identify those ranges, or, how can I tell if any 
 particular
 range has value semantics ? I didn't read any of this in the 
 manual -
 not that I could remember anyways.


 Generally you shouldn't. If you care about it either way, use 
 .save or std.range.refRange.

 If you don't want some range r to be consumed by some 
 operation, pass r.save instead of plain r. If you want r to be 
 consumed, pass refRange(&r). Only if you don't care if r is 
 consumed or not, should you pass simply r.

 If you know for a fact that copying r is the same as r.save, 
 then you can just pass (and copy) r, of course. We know it's 
 that way with dynamic arrays, because of their nature as 
 pointer+length structures. But there's nothing wrong with 
 calling .save on an array anyway.

 Also, when a function takes a range via a ref parameter, then 
 you don't need refRange, of course. The ref parameter ensures 
 that no copy is made and that the original range is affected by 
 the function.


This is even better than trying to figure out whether value 
semantics are supported or not.

So, .safe returns a copy of the range - I suppose a copy of the 
current state - and refRange always consumes the values in the 
range.

Thanks a lot for your reply! Very much appreciated.
Jan 22 2016