www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - File.byLine for either Windows / Unix newlines

reply Dennis <dkorpel gmail.com> writes:
I'm on Windows and I recently got confused by how Phobos 
functions handle newlines.

```
void main() {
     import std.stdio;
     import std.path : buildPath, tempDir;

     auto path = buildPath(tempDir(), "test.txt");
     auto file = new File(path, "w");

     file.write("hello there!\n");   //actually writes hello 
there!\r\n
     file.write('\n');               //actually writes \r\n
     file.rawWrite("\n");            //actually writes \n
     //file.rawWrite('\n');          //doesn't compile
     file.close();

     //byLine uses \n as default terminator, so a trailing \r is 
kept
     writeln(File(path).byLine.front~"huh?");   //prints "huh?o 
there!"
}
```

- Why do these functions behave like that?
- What is the easiest way to iterate over a file by line that 
works with both \r\n and \n line endings? I prefer not to write a 
new method for this.
Dec 11 2017
parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 12/11/17 10:40 AM, Dennis wrote:
 I'm on Windows and I recently got confused by how Phobos functions 
 handle newlines.
 
 ```
 void main() {
      import std.stdio;
      import std.path : buildPath, tempDir;
 
      auto path = buildPath(tempDir(), "test.txt");
      auto file = new File(path, "w");
 
      file.write("hello there!\n");   //actually writes hello there!\r\n
      file.write('\n');               //actually writes \r\n
      file.rawWrite("\n");            //actually writes \n
      //file.rawWrite('\n');          //doesn't compile
      file.close();
 
      //byLine uses \n as default terminator, so a trailing \r is kept
      writeln(File(path).byLine.front~"huh?");   //prints "huh?o there!"
 }
 ```
 
 - Why do these functions behave like that?
 - What is the easiest way to iterate over a file by line that works with 
 both \r\n and \n line endings? I prefer not to write a new method for this.
Please note that D uses FILE * as its underlying implementation. This means that whatever the C library does, D's library does. In the case of Windows, there is this lovely concept that line endings should be \r\n, not just \n. So the C library helpfully inserts \r whenever it sees a \n. This is known as "text mode" as opposed to "binary mode". rawWrite temporarily sets the mode to binary mode, then restores the original mode. This is why the \n is not translated there. Here's the fun part: the default open mode is "rb". See the docs here: https://dlang.org/phobos/std_stdio.html#.File.this So you are actually opening the file in binary mode, whereas you were writing it in text mode. You can fix this in one of 3 ways: 1. Open your file for writing with "wb". This will not store the \r by default. 2. Open your file for reading with "r" (e.g. File(path, "r")). This will replace the \r\n with just \n in the underlying C library. 3. Stop using Windows ;) -Steve
Dec 11 2017
parent Dennis <dkorpel gmail.com> writes:
Thanks for your reply, that clears it up.

On Monday, 11 December 2017 at 21:13:11 UTC, Steven Schveighoffer 
wrote:
 3. Stop using Windows ;)
Haha, if only the rest of the userbase would follow.
Dec 11 2017