• manfred toppoint.de (14/14) Jan 29 2004 Hello,
• Walter (3/17) Jan 29 2004 Try std.file.read().
• C (7/21) Jan 29 2004 Hmm thats interesting.
• C (18/45) Jan 29 2004 This C++ takes 3 seconds to read line by line.
• Matthew (5/56) Jan 29 2004 line
• Walter (4/6) Jan 30 2004 line
• Damon Gray (9/44) Jan 29 2004 I believe it is readLine() that is may be making this slow. readLine
• C (6/50) Jan 29 2004 Ok cool I look forward to it. I can't believe we don't have a phobos
• Patrick Down (3/30) Jan 29 2004 I don't have the source in front of me right now but I believe it's the
• Matthew (16/53) Jan 29 2004 You weren't kidding:
• Walter (5/7) Jan 30 2004 This
• Matthew (3/10) Jan 30 2004 No, but I'll certainly volunteer reviewing services
• Andy Friesen (8/20) Jan 30 2004 Here's an attempt:
• Manfred Nowak (14/16) Jan 31 2004 [...]
• Andy Friesen (5/28) Jan 31 2004 hm. Works for me.
• Manfred Nowak (6/7) Feb 01 2004 Shame on me. I started the wrong program.
• Manfred Hansen (17/40) Feb 01 2004 How are you install stream.d and compile this?
• C (31/51) Jan 31 2004 Quick note ( 859 )
• larry cowan (2/58) Jan 31 2004
• C (10/81) Jan 31 2004 Doh! Im a jackass :S.
• Andy Friesen (9/21) Jan 31 2004 Since handle is a pointer, and not a reference, it doesn't matter. I
• C (16/41) Feb 01 2004 Hmm ok. You've done alot with D templates, anyway to convinvce you to w...
• Matthew (8/64) Jan 31 2004 ,

manfred toppoint.de writes:

Hello,

i have write a test programm that takes 65 seconds.
File tralala has 213753 lines.   

import std.stream;
int main() {
File fd = new File("tralala");
char[][] werte;
while (!fd.eof()) {
fd.readLine();
}
return 0;
}

Give it a better way to read a File line by line? 
The same perl programm takes 11 seconds.

"Walter" <walter digitalmars.com> writes:

<manfred toppoint.de> wrote in message
news:bvb54a$1qk5$1 digitaldaemon.com...
 Hello,

 i have write a test programm that takes 65 seconds.
 File tralala has 213753 lines.

 import std.stream;
 int main() {
 File fd = new File("tralala");
 char[][] werte;
 while (!fd.eof()) {
 fd.readLine();
 }
 return 0;
 }

 Give it a better way to read a File line by line?
 The same perl programm takes 11 seconds.

Try std.file.read().

"C" <dont respond.com> writes:

Hmm thats interesting.

std.file.read took about 1 second on a 250K line file , the while loop took
about 12 seconds for me , which is a very long time.

is it the readLine call thats making this so slow ?

C

<manfred toppoint.de> wrote in message
news:bvb54a$1qk5$1 digitaldaemon.com...
 Hello,

 i have write a test programm that takes 65 seconds.
 File tralala has 213753 lines.

 import std.stream;
 int main() {
 File fd = new File("tralala");
 char[][] werte;
 while (!fd.eof()) {
 fd.readLine();
 }
 return 0;
 }

 Give it a better way to read a File line by line?
 The same perl programm takes 11 seconds.

"C" <dont respond.com> writes:

This C++ takes 3 seconds to read line by line.

#include <fstream>
#include <string>

using namespace std;

int main () {

 ifstream in("tralala");
 string line;
 while ( !in.eof ()  ) {

  std::getline(in,line);

 }
 return 1;
}

Ill try to use Mattys libs to find the bottleneck , as reading line by line
is a common operation.

C

"C" <dont respond.com> wrote in message
news:bvbolu$2qpf$1 digitaldaemon.com...
 Hmm thats interesting.

 std.file.read took about 1 second on a 250K line file , the while loop

took
 about 12 seconds for me , which is a very long time.

 is it the readLine call thats making this so slow ?

 C

 <manfred toppoint.de> wrote in message
 news:bvb54a$1qk5$1 digitaldaemon.com...
 Hello,

 i have write a test programm that takes 65 seconds.
 File tralala has 213753 lines.

 import std.stream;
 int main() {
 File fd = new File("tralala");
 char[][] werte;
 while (!fd.eof()) {
 fd.readLine();
 }
 return 0;
 }

 Give it a better way to read a File line by line?
 The same perl programm takes 11 seconds.

"Matthew" <matthew.hat stlsoft.dot.org> writes:

"C" <dont respond.com> wrote in message
news:bvbpjm$2sjj$1 digitaldaemon.com...
 This C++ takes 3 seconds to read line by line.

 #include <fstream>
 #include <string>

 using namespace std;

 int main () {

  ifstream in("tralala");
  string line;
  while ( !in.eof ()  ) {

   std::getline(in,line);

  }
  return 1;
 }

 Ill try to use Mattys libs to find the bottleneck , as reading line by

line
 is a common operation.

Cool. Then you can "suggest" to Walter that we include it in Phobos. I've
had no luck in that so far. :)

 C

 "C" <dont respond.com> wrote in message
 news:bvbolu$2qpf$1 digitaldaemon.com...
 Hmm thats interesting.

 std.file.read took about 1 second on a 250K line file , the while loop

 took
 about 12 seconds for me , which is a very long time.

 is it the readLine call thats making this so slow ?

 C

 <manfred toppoint.de> wrote in message
 news:bvb54a$1qk5$1 digitaldaemon.com...
 Hello,

 i have write a test programm that takes 65 seconds.
 File tralala has 213753 lines.

 import std.stream;
 int main() {
 File fd = new File("tralala");
 char[][] werte;
 while (!fd.eof()) {
 fd.readLine();
 }
 return 0;
 }

 Give it a better way to read a File line by line?
 The same perl programm takes 11 seconds.

"Walter" <walter digitalmars.com> writes:

"C" <dont respond.com> wrote in message
news:bvbpjm$2sjj$1 digitaldaemon.com...
 Ill try to use Mattys libs to find the bottleneck , as reading line by

line
 is a common operation.

I tend to use std.file.read() followed by std.string.splitlines().

Damon Gray <dontbotherasking go.away.mr.bad.spammer.net> writes:

I believe it is readLine() that is may be making this slow. readLine 
seems to read a line one byte at a time. See the code for readLine in 
stream.d.

That is why I've been spending some free time lately creating a 
BufferedStream class that actually buffers read()s and uses that to get 
line oriented input. I'll post it when I've had a chance to get away 
from work enough to finish it. :)

-Damon-


C wrote:
 Hmm thats interesting.
 
 std.file.read took about 1 second on a 250K line file , the while loop took
 about 12 seconds for me , which is a very long time.
 
 is it the readLine call thats making this so slow ?
 
 C
 
 <manfred toppoint.de> wrote in message
 news:bvb54a$1qk5$1 digitaldaemon.com...
 
Hello,

i have write a test programm that takes 65 seconds.
File tralala has 213753 lines.

import std.stream;
int main() {
File fd = new File("tralala");
char[][] werte;
while (!fd.eof()) {
fd.readLine();
}
return 0;
}

Give it a better way to read a File line by line?
The same perl programm takes 11 seconds.

 
 
 

"C" <dont respond.com> writes:

Ok cool I look forward to it.   I can't believe we don't have a phobos
library group yet.  So frustrurating...

C

"Damon Gray" <dontbotherasking go.away.mr.bad.spammer.net> wrote in message
news:40196AB3.5040107 go.away.mr.bad.spammer.net...
 I believe it is readLine() that is may be making this slow. readLine
 seems to read a line one byte at a time. See the code for readLine in
 stream.d.

 That is why I've been spending some free time lately creating a
 BufferedStream class that actually buffers read()s and uses that to get
 line oriented input. I'll post it when I've had a chance to get away
 from work enough to finish it. :)

 -Damon-


 C wrote:
 Hmm thats interesting.

 std.file.read took about 1 second on a 250K line file , the while loop


took
 about 12 seconds for me , which is a very long time.

 is it the readLine call thats making this so slow ?

 C

 <manfred toppoint.de> wrote in message
 news:bvb54a$1qk5$1 digitaldaemon.com...

Hello,

i have write a test programm that takes 65 seconds.
File tralala has 213753 lines.

import std.stream;
int main() {
File fd = new File("tralala");
char[][] werte;
while (!fd.eof()) {
fd.readLine();
}
return 0;
}

Give it a better way to read a File line by line?
The same perl programm takes 11 seconds.

Patrick Down <Patrick_member pathlink.com> writes:

I don't have the source in front of me right now but I believe it's the 
eof().  It is implemented in an atrocious fashion if I remember correctly.


In article <bvbolu$2qpf$1 digitaldaemon.com>, C says...
Hmm thats interesting.

std.file.read took about 1 second on a 250K line file , the while loop took
about 12 seconds for me , which is a very long time.

is it the readLine call thats making this so slow ?

C

<manfred toppoint.de> wrote in message
news:bvb54a$1qk5$1 digitaldaemon.com...
 Hello,

 i have write a test programm that takes 65 seconds.
 File tralala has 213753 lines.

 import std.stream;
 int main() {
 File fd = new File("tralala");
 char[][] werte;
 while (!fd.eof()) {
 fd.readLine();
 }
 return 0;
 }

 Give it a better way to read a File line by line?
 The same perl programm takes 11 seconds.

"Matthew" <matthew.hat stlsoft.dot.org> writes:

 I don't have the source in front of me right now but I believe it's the
 eof().  It is implemented in an atrocious fashion if I remember correctly.


You weren't kidding:

    bit eof() { return position() == size(); }

It w*nks. Unbelievable stuff!

    ulong position() { return seek(0, SeekPos.Current); }

    // returns size of stream
    ulong size()
    {
        ulong pos = position(), result = seek(0, SeekPos.End);
        position(pos);
        return result;
    }

That's three kernel calls to check whether we've hit EOF!

My confidence in stream.d has just fallen to about 5%.

Walter, did anyone do a review on this before accepting it into Phobos? This
needs a serious rewrite, if not a redesign.


 In article <bvbolu$2qpf$1 digitaldaemon.com>, C says...
Hmm thats interesting.

std.file.read took about 1 second on a 250K line file , the while loop


took
about 12 seconds for me , which is a very long time.

is it the readLine call thats making this so slow ?

C

<manfred toppoint.de> wrote in message
news:bvb54a$1qk5$1 digitaldaemon.com...
 Hello,

 i have write a test programm that takes 65 seconds.
 File tralala has 213753 lines.

 import std.stream;
 int main() {
 File fd = new File("tralala");
 char[][] werte;
 while (!fd.eof()) {
 fd.readLine();
 }
 return 0;
 }

 Give it a better way to read a File line by line?
 The same perl programm takes 11 seconds.

"Walter" <walter digitalmars.com> writes:

"Matthew" <matthew.hat stlsoft.dot.org> wrote in message
news:bvc0vu$82o$2 digitaldaemon.com...
 Walter, did anyone do a review on this before accepting it into Phobos?

This
 needs a serious rewrite, if not a redesign.

No. It was very early on, done just to get things off the ground. It could
use a rewrite. Anyone up for it?

"Matthew" <matthew.hat stlsoft.dot.org> writes:

No, but I'll certainly volunteer reviewing services

"Walter" <walter digitalmars.com> wrote in message
news:bvfhj7$31e4$1 digitaldaemon.com...
 "Matthew" <matthew.hat stlsoft.dot.org> wrote in message
 news:bvc0vu$82o$2 digitaldaemon.com...
 Walter, did anyone do a review on this before accepting it into Phobos?

 This
 needs a serious rewrite, if not a redesign.

 No. It was very early on, done just to get things off the ground. It could
 use a rewrite. Anyone up for it?

Andy Friesen <andy ikagames.com> writes:

Walter wrote:
 "Matthew" <matthew.hat stlsoft.dot.org> wrote in message
 news:bvc0vu$82o$2 digitaldaemon.com...
 
Walter, did anyone do a review on this before accepting it into Phobos?

 
 This
 
needs a serious rewrite, if not a redesign.

 
 No. It was very early on, done just to get things off the ground. It could
 use a rewrite. Anyone up for it?
 

Here's an attempt:

http://ikagames.com/andy/d/stream.d

It's basically a bunch of classes that each do some small thing, and 
accepts a parent class as a template argument.  The stream aliases at 
the very bottom mash them into a cohesive class.

If nothing else, it'll get some ideas on the table. :)

  -- andy

Manfred Nowak <svv1999 hotmail.com> writes:

Andy Friesen wrote:

 Here's an attempt:
 http://ikagames.com/andy/d/stream.d

[...]

Tested it like this:

import stream;

void main(){
  ReadableFile src=new ReadableFile("");
  while(!src.eof()){
    printf(".");
    Console.writeLine= src.readLine;
  }
  Console.writeLine="Hello world\n";
}


But it did not throw an error.

So long.

Andy Friesen <andy ikagames.com> writes:

Manfred Nowak wrote:

 Andy Friesen wrote:
 
Here's an attempt:
http://ikagames.com/andy/d/stream.d

 
 [...]
 
 Tested it like this:
 
 import stream;
 
 void main(){
   ReadableFile src=new ReadableFile("");
   while(!src.eof()){
     printf(".");
     Console.writeLine= src.readLine;
   }
   Console.writeLine="Hello world\n";
 }
 
 
 But it did not throw an error.
 

hm.  Works for me.

If it's not throwing an error, then fopen("") is returning a non-null 
value when "" is passed as a filename.  This is puzzling.

  -- andy

Manfred Nowak <svv1999 hotmail.com> writes:

Andy Friesen wrote:

 This is puzzling.

Shame on me. I started the wrong program.

The compiled `stream.exe' throws the exception. Accidently I started 
`test.exe', which only showed, `Hello world'.

Sorry for the inconvenience.

So long.

Manfred Hansen <manfred toppoint.de> writes:

How are you install stream.d and compile this?
I have tried following:

dmd stream_tes1.d stream.d

i get the following errormessage from the linker:
hansen hansen-lx:~/d$ dmd stream_tes1.d stream1.d
gcc stream_tes1.o stream1.o -o stream_tes1 -lphobos -lpthread -lm
stream_tes1.o(.gnu.linkonce.t_Dmain+0x12): In function `_Dmain':
: undefined reference to
`_Class_7stream111__anonymous116ReadableFileStream_C7stream111__anonymous59BinaryReader_C7stream111__anonymous15Seeker_C6Object6Seeker12BinaryReader18ReadableFileStream'
collect2: ld returned 1 exit status
--- errorlevel 256

I have change the first line from stream.d to:
module stream;

The file's stream_tes1.d and stream.d are in the same directory. 

dmd -c stream_tes1.d stream.d run without any error.

Manfred

Manfred Nowak wrote:

 Andy Friesen wrote:
 
 Here's an attempt:
 http://ikagames.com/andy/d/stream.d

 [...]
 
 Tested it like this:
 
 import stream;
 
 void main(){
   ReadableFile src=new ReadableFile("");
   while(!src.eof()){
     printf(".");
     Console.writeLine= src.readLine;
   }
   Console.writeLine="Hello world\n";
 }
 
 
 But it did not throw an error.
 
 So long.

"C" <dont respond.com> writes:

Quick note ( 859 )
        return handle != null;

should read !== null ( i know u know this ).

I like this template stacking , 'bolt-ins' you call them ?  Looks like fun ,
Ill play around with them :D.  Is Modern C++ the only book that does things
like this ?  Another source of documentation , examples , tutorial would be
cool too.

Also some unittests and examples ?  I tested the streams on the 250K file
with


void main ()
{
 try
 {
  ReadableFile x = new ReadableFile("tralala");

  while ( x.eof() )
  {
   Console.writeLine(x.readLine() );

  }
 }

 catch (IOError x)
 {
  Console.writeLine( x.toString() ) ;
 }
}

and I get no printing to stdout , it ends within .5 second , am I using it
wrong ?

Thanks,
C
"Andy Friesen" <andy ikagames.com> wrote in message
news:bvfk4b$3tl$1 digitaldaemon.com...
 Walter wrote:
 "Matthew" <matthew.hat stlsoft.dot.org> wrote in message
 news:bvc0vu$82o$2 digitaldaemon.com...

Walter, did anyone do a review on this before accepting it into Phobos?

 This

needs a serious rewrite, if not a redesign.

 No. It was very early on, done just to get things off the ground. It


could
 use a rewrite. Anyone up for it?

 Here's an attempt:

 http://ikagames.com/andy/d/stream.d

 It's basically a bunch of classes that each do some small thing, and
 accepts a parent class as a template argument.  The stream aliases at
 the very bottom mash them into a cohesive class.

 If nothing else, it'll get some ideas on the table. :)

   -- andy

larry cowan <larry_member pathlink.com> writes:

In article <bvh0i9$2cqa$1 digitaldaemon.com>, C says...
Quick note ( 859 )
        return handle != null;

should read !== null ( i know u know this ).

I like this template stacking , 'bolt-ins' you call them ?  Looks like fun ,
Ill play around with them :D.  Is Modern C++ the only book that does things
like this ?  Another source of documentation , examples , tutorial would be
cool too.

Also some unittests and examples ?  I tested the streams on the 250K file
with


void main ()
{
 try
 {
  ReadableFile x = new ReadableFile("tralala");

  while ( x.eof() )

---------------------- while ( !x.eof() ) -------------------------- ?
  {
   Console.writeLine(x.readLine() );

  }
 }

 catch (IOError x)
 {
  Console.writeLine( x.toString() ) ;
 }
}

and I get no printing to stdout , it ends within .5 second , am I using it
wrong ?

Thanks,
C
"Andy Friesen" <andy ikagames.com> wrote in message
news:bvfk4b$3tl$1 digitaldaemon.com...
 Walter wrote:
 "Matthew" <matthew.hat stlsoft.dot.org> wrote in message
 news:bvc0vu$82o$2 digitaldaemon.com...

Walter, did anyone do a review on this before accepting it into Phobos?

 This

needs a serious rewrite, if not a redesign.

 No. It was very early on, done just to get things off the ground. It


could
 use a rewrite. Anyone up for it?

 Here's an attempt:

 http://ikagames.com/andy/d/stream.d

 It's basically a bunch of classes that each do some small thing, and
 accepts a parent class as a template argument.  The stream aliases at
 the very bottom mash them into a cohesive class.

 If nothing else, it'll get some ideas on the table. :)

   -- andy

"C" <dont respond.com> writes:

Doh!  Im a jackass :S.

Took 3 seconds ( aproximate ), considerably faster !

C

"larry cowan" <larry_member pathlink.com> wrote in message
news:bvh1cj$2e8i$1 digitaldaemon.com...
 In article <bvh0i9$2cqa$1 digitaldaemon.com>, C says...
Quick note ( 859 )
        return handle != null;

should read !== null ( i know u know this ).

I like this template stacking , 'bolt-ins' you call them ?  Looks like


fun ,
Ill play around with them :D.  Is Modern C++ the only book that does


things
like this ?  Another source of documentation , examples , tutorial would


be
cool too.

Also some unittests and examples ?  I tested the streams on the 250K file
with


void main ()
{
 try
 {
  ReadableFile x = new ReadableFile("tralala");

  while ( x.eof() )

 ---------------------- while ( !x.eof() ) -------------------------- ?
  {
   Console.writeLine(x.readLine() );

  }
 }

 catch (IOError x)
 {
  Console.writeLine( x.toString() ) ;
 }
}

and I get no printing to stdout , it ends within .5 second , am I using


it
wrong ?

Thanks,
C
"Andy Friesen" <andy ikagames.com> wrote in message
news:bvfk4b$3tl$1 digitaldaemon.com...
 Walter wrote:
 "Matthew" <matthew.hat stlsoft.dot.org> wrote in message
 news:bvc0vu$82o$2 digitaldaemon.com...

Walter, did anyone do a review on this before accepting it into





Phobos?
 This

needs a serious rewrite, if not a redesign.

 No. It was very early on, done just to get things off the ground. It


could
 use a rewrite. Anyone up for it?

 Here's an attempt:

 http://ikagames.com/andy/d/stream.d

 It's basically a bunch of classes that each do some small thing, and
 accepts a parent class as a template argument.  The stream aliases at
 the very bottom mash them into a cohesive class.

 If nothing else, it'll get some ideas on the table. :)

   -- andy

Andy Friesen <andy ikagames.com> writes:

C wrote:
 Quick note ( 859 )
         return handle != null;
 
 should read !== null ( i know u know this ).

Since handle is a pointer, and not a reference, it doesn't matter.  I 
suppose it's a good idea, though, as it's better self-documentation if 
it consistently behaves like a reference.

 
 I like this template stacking , 'bolt-ins' you call them ?  Looks like fun ,
 Ill play around with them :D.  Is Modern C++ the only book that does things
 like this ?  Another source of documentation , examples , tutorial would be
 cool too.

Modern C++ goes way, way further than this.  Most compilers tremble at 
the mere mention of Loki. (a library made by the same author, for 
demonstrating the techniques implemented in the book)

 
 Also some unittests and examples ?
 

Good idea.

  -- andy

"C" <dont respond.com> writes:

 Since handle is a pointer, and not a reference, it doesn't matter.

Doh , your right of course.

 Modern C++ goes way, way further than this.  Most compilers tremble at
 the mere mention of Loki. (a library made by the same author, for
 demonstrating the techniques implemented in the book)

Hmm ok.  You've done alot with D templates, anyway to convinvce you to write
a tutorial on C++ and D templates ? I almost never write generic code in C++
but would like to in D.  A tutorial would help alot of people, myself
included.

Also im a little confused on the use of interface , in C++ this would be a
pure virtual abstract class ?  How does interface work with templates , are
there some advantages ?  And looking over the code its not clear to me why
you instantiate with Object , ill keep reading it.

Thanks,
C

"Andy Friesen" <andy ikagames.com> wrote in message
news:bvh82p$2p2c$1 digitaldaemon.com...
 C wrote:
 Quick note ( 859 )
         return handle != null;

 should read !== null ( i know u know this ).

 Since handle is a pointer, and not a reference, it doesn't matter.  I
 suppose it's a good idea, though, as it's better self-documentation if
 it consistently behaves like a reference.

 I like this template stacking , 'bolt-ins' you call them ?  Looks like


fun ,
 Ill play around with them :D.  Is Modern C++ the only book that does


things
 like this ?  Another source of documentation , examples , tutorial would


be
 cool too.

 Modern C++ goes way, way further than this.  Most compilers tremble at
 the mere mention of Loki. (a library made by the same author, for
 demonstrating the techniques implemented in the book)

 Also some unittests and examples ?

 Good idea.

   -- andy

"Matthew" <matthew.hat stlsoft.dot.org> writes:

"C" <dont respond.com> wrote in message
news:bvh0i9$2cqa$1 digitaldaemon.com...
 Quick note ( 859 )
         return handle != null;

 should read !== null ( i know u know this ).

 I like this template stacking , 'bolt-ins' you call them ?  Looks like fun

,
 Ill play around with them :D.  Is Modern C++ the only book that does

things
 like this ?  Another source of documentation , examples , tutorial would

be
 cool too.

ATL uses this a lot, although they don't name the technique, so ATL
Internals is also a good text.

 Also some unittests and examples ?  I tested the streams on the 250K file
 with


 void main ()
 {
  try
  {
   ReadableFile x = new ReadableFile("tralala");

   while ( x.eof() )
   {
    Console.writeLine(x.readLine() );

   }
  }

  catch (IOError x)
  {
   Console.writeLine( x.toString() ) ;
  }
 }

 and I get no printing to stdout , it ends within .5 second , am I using it
 wrong ?

 Thanks,
 C
 "Andy Friesen" <andy ikagames.com> wrote in message
 news:bvfk4b$3tl$1 digitaldaemon.com...
 Walter wrote:
 "Matthew" <matthew.hat stlsoft.dot.org> wrote in message
 news:bvc0vu$82o$2 digitaldaemon.com...

Walter, did anyone do a review on this before accepting it into




Phobos?
 This

needs a serious rewrite, if not a redesign.

 No. It was very early on, done just to get things off the ground. It


 could
 use a rewrite. Anyone up for it?

 Here's an attempt:

 http://ikagames.com/andy/d/stream.d

 It's basically a bunch of classes that each do some small thing, and
 accepts a parent class as a template argument.  The stream aliases at
 the very bottom mash them into a cohesive class.

 If nothing else, it'll get some ideas on the table. :)

   -- andy