www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - reading from file

reply Namal <sotis22 mail.ru> writes:
Hello, comming from C++, I find it hard to remember and 
understand how reading from file should be done in D. Especially 
since I am not very good in functional programming. So I have a 
file which looks like this:

1,2,3,4
5,6,7,8
9,11,11,12

and so on

How could I read it row by row and create an array accordingly 
without reading the comma?
Dec 13 2016
next sibling parent bluecat <1bluecat7 gmail.com> writes:
On Tuesday, 13 December 2016 at 15:42:16 UTC, Namal wrote:

 Hello, comming from C++, I find it hard to remember and 
 understand how reading from file should be done in D. 
 Especially since I am not very good in functional programming. 
 So I have a file which looks like this:

 1,2,3,4
 5,6,7,8
 9,11,11,12

 and so on

 How could I read it row by row and create an array accordingly 
 without reading the comma?


import std.stdio;
import std.string:    strip;
import std.algorithm: splitter, each;
import std.array:     array;

void main() {
   //prepare variables
   File file = File("new.txt", "r");
   string[] arr;

   //read file
   while(!file.eof) {
     file
       .readln
       .strip
       .splitter(",")
       .array
       .each!(n => arr ~= n);
   }
   arr.writeln;
}

//here is my attempt. copy, paste, run the program to see if it 
is what you want.
//feel free to ask any questions about my code, it isn't perfect 
but it works.
Dec 13 2016
prev sibling parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 14/12/2016 4:42 AM, Namal wrote:

 Hello, comming from C++, I find it hard to remember and understand how
 reading from file should be done in D. Especially since I am not very
 good in functional programming. So I have a file which looks like this:

 1,2,3,4
 5,6,7,8
 9,11,11,12

 and so on

 How could I read it row by row and create an array accordingly without
 reading the comma?


import std.stdio : File;
import std.algorithm : splitter, count;
import std.conv : to;

foreach(line; File("myfile.csv", "r").byLine) {
	int[] linetemp;
	linetemp.length = line.count(",") + 1;

	size_t i;
	foreach(num; line.splitter(",")) {
		linetemp[i] = to!int(num);
		i++;
	}

	// ... = linetemp.dup;
}

This could be done a lot simpler especially with std.csv help for your 
case. But this should work with only one allocation per line.
Dec 13 2016
parent reply Namal <sotis22 mail.ru> writes:
Sorry if I wasn't clear. The array should be two demensional and 
each line in text line should be a row in that 2x2 array.
Dec 13 2016
parent reply Namal <sotis22 mail.ru> writes:
On Tuesday, 13 December 2016 at 16:57:40 UTC, Namal wrote:

 Sorry if I wasn't clear. The array should be two demensional 
 and each line in text line should be a row in that 2x2 array.


Also, it should be saved as an integer.
Dec 13 2016
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 12/13/2016 08:59 AM, Namal wrote:

 On Tuesday, 13 December 2016 at 16:57:40 UTC, Namal wrote:

 Sorry if I wasn't clear. The array should be two demensional and each
 line in text line should be a row in that 2x2 array.


 Also, it should be saved as an integer.


Here is another one:

import std.stdio;
import std.algorithm;
import std.array;
import std.conv;

void main() {
     auto a = File("deneme.txt")
              .byLine
              .map!(line => line.splitter(',').map!(to!int).array)
              .array;

     writeln(a);
}

Ali
Dec 13 2016
prev sibling parent reply Ali <something something.com> writes:
On Tuesday, 13 December 2016 at 16:59:17 UTC, Namal wrote:

 On Tuesday, 13 December 2016 at 16:57:40 UTC, Namal wrote:

 Sorry if I wasn't clear. The array should be two demensional 
 and each line in text line should be a row in that 2x2 array.


 Also, it should be saved as an integer.


And extending Ali's solution you can actually get the data in to 
a two dimentional array at compile time and have it in static 
memory with a small adjustment:

static immutable matrix = import("data.txt")
     .split("\n")
     .map!(a => a.split(",").map!(to!int).array)
     .array;

void main() {
     writeln(matrix);
}
Dec 13 2016
parent reply KaattuPoochi <visionofarun gmail.com> writes:
On Tuesday, 13 December 2016 at 21:13:26 UTC, Ali wrote:

 And extending Ali's solution you can actually get the data in
 to a two dimentional array at compile time and have it in 
 static memory with a small adjustment:

 static immutable matrix = import("data.txt")
     .split("\n")
     .map!(a => a.split(",").map!(to!int).array)
     .array;

 void main() {
     writeln(matrix);
 }


1. For any non-trivial matrices (with 500 lines) runs DMD 2.072.1 
out of memory (2GB). Not sure if this is a known bug. Works fine 
with LDC 1.0.0.
2. The EOL on the last line results in an empty row in the end. 
Is there a way to overcome this?
Dec 15 2016
next sibling parent =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 12/15/2016 10:47 PM, KaattuPoochi wrote:

 On Tuesday, 13 December 2016 at 21:13:26 UTC, Ali wrote:

 And extending Ali's solution you can actually get the data in
 to a two dimentional array at compile time and have it in static
 memory with a small adjustment:

 static immutable matrix = import("data.txt")
     .split("\n")
     .map!(a => a.split(",").map!(to!int).array)
     .array;

 void main() {
     writeln(matrix);
 }


 1. For any non-trivial matrices (with 500 lines) runs DMD 2.072.1 out of
 memory (2GB). Not sure if this is a known bug. Works fine with LDC 1.0.0.


Compile time features are awesome but currenty very inefficient. :)


 2. The EOL on the last line results in an empty row in the end. Is there
 a way to overcome this?


If appropriate, you can filter out all empty lines. Added .map!strip and 
.filter:

import std.stdio;
import std.algorithm;
import std.array;
import std.conv;
import std.string;

void main() {
     auto a = File("deneme.txt")
              .byLine
              .map!strip
              .filter!(line => !line.empty)
              .map!(line => line.splitter(',').map!(to!int).array)
              .array;

     writeln(a);
}

Ali
Dec 15 2016
prev sibling next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Friday, 16 December 2016 at 06:47:15 UTC, KaattuPoochi wrote:

 On Tuesday, 13 December 2016 at 21:13:26 UTC, Ali wrote:

 And extending Ali's solution you can actually get the data in
 to a two dimentional array at compile time and have it in 
 static memory with a small adjustment:

 static immutable matrix = import("data.txt")
     .split("\n")
     .map!(a => a.split(",").map!(to!int).array)
     .array;

 void main() {
     writeln(matrix);
 }


 1. For any non-trivial matrices (with 500 lines) runs DMD 
 2.072.1 out of memory (2GB). Not sure if this is a known bug. 
 Works fine with LDC 1.0.0.
 2. The EOL on the last line results in an empty row in the end. 
 Is there a way to overcome this?


Most likely you are using a 64bit ldc, and a 32bit dmd.
Since I am pretty sure the ldc guys have no CTFE patches.

I am working on fixing that problem.
Dec 16 2016
prev sibling parent reply =?UTF-8?Q?Ali_=c3=87ehreli?= <acehreli yahoo.com> writes:
On 12/15/2016 10:47 PM, KaattuPoochi wrote:

 On Tuesday, 13 December 2016 at 21:13:26 UTC, Ali wrote:

 And extending Ali's solution you can actually get the data in
 to a two dimentional array at compile time and have it in static
 memory with a small adjustment:

 static immutable matrix = import("data.txt")
     .split("\n")
     .map!(a => a.split(",").map!(to!int).array)
     .array;

 void main() {
     writeln(matrix);
 }


 1. For any non-trivial matrices (with 500 lines) runs DMD 2.072.1 out of
 memory (2GB). Not sure if this is a known bug. Works fine with LDC 1.0.0.


Just to note, there are 501 arrays generated at compile. You can reduce 
that number to 1 if you know the width and the height of the matrix. The 
following solution reads everything into a single array and then uses it 
as an int[N][]:

import std.algorithm;
import std.conv;
import std.array;
import std.stdio;

static immutable buffer = import("deneme.txt")
                           .splitter
                           .map!(a => a.splitter(',').map!(to!int))
                           .joiner
                           .array;

// Assume these are known or are calculated at compile time:
enum width = 4;
enum height = 3;

immutable int[width][] matrix;

shared static this() {
     matrix = (cast(immutable(int[width])*)buffer.ptr)[0..height];

     // Make sure we did not copy into matrix
     assert(cast(void*)matrix.ptr == cast(void*)buffer.ptr);
}

void main() {
     writeln(matrix);
}


 2. The EOL on the last line results in an empty row in the end. Is there
 a way to overcome this?


As an added bonus, the solution above does not suffer from empty lines.

Ali
Dec 16 2016
parent KaattuPoochi <visionofarun gmail.com> writes:
On Friday, 16 December 2016 at 08:03:22 UTC, Ali Çehreli wrote:

 shared static this() {
     matrix = 
 (cast(immutable(int[width])*)buffer.ptr)[0..height];

     // Make sure we did not copy into matrix
     assert(cast(void*)matrix.ptr == cast(void*)buffer.ptr);
 }


Thanks Ali, this is neat!!
Dec 30 2016