www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D2 and stream

reply Mafi <mafi example.org> writes:
Hi all,
what are the directions of D2's streams. I heared std.stream is going to 
get deprecated in D2. I think it's good as long as 
BufferedFile.readLine() returns char[] instead of string. Is D2 going to 
get a Java-like stream model? What's wrong with the old one except that 
it's a bit out of date?
Mafi
Aug 08 2010
next sibling parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Mafi (mafi example.org)'s article

 Hi all,
 what are the directions of D2's streams. I heared std.stream is going to
 get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going to
 get a Java-like stream model? What's wrong with the old one except that
 it's a bit out of date?
 Mafi


Two things off the top of my head:

1.  std.stream doesn't correctly support files > 2GB on 32-bit Linux, and
possibly
on other OS's as well.

2.  It doesn't have a range interface.  I can't tell you how useful something
like
this is to me:

filter!"a.length > 0"(
    map!strip(
        File(myFilename).byLine()
    )
);

That said, please don't use Java as a reference for how to design a stream API.
It just seems horribly verbose when trying to do something simple.
Aug 08 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:


 == Quote from Mafi (mafi example.org)'s article

 Hi all,
 what are the directions of D2's streams. I heared std.stream is going to
 get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going to
 get a Java-like stream model? What's wrong with the old one except that
 it's a bit out of date?
 Mafi


 Two things off the top of my head:

 1.  std.stream doesn't correctly support files > 2GB on 32-bit Linux,  
 and possibly
 on other OS's as well.


Doesn't that have to do with using C's FILE * interface?

-Steve
Aug 09 2010
next sibling parent reply "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Mon, 09 Aug 2010 08:11:54 -0400, Steven Schveighoffer wrote:


 On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:
 

 == Quote from Mafi (mafi example.org)'s article

 Hi all,
 what are the directions of D2's streams. I heared std.stream is going
 to get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going
 to get a Java-like stream model? What's wrong with the old one except
 that it's a bit out of date?
 Mafi


 Two things off the top of my head:

 1.  std.stream doesn't correctly support files > 2GB on 32-bit Linux,
 and possibly
 on other OS's as well.


 
 Doesn't that have to do with using C's FILE * interface?


FILE* isn't used anywhere in std.stream.  You're either thinking of 
std.stdio.File or std.cstream.CFile.

-Lars
Aug 09 2010
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 09 Aug 2010 09:01:33 -0400, Lars T. Kyllingstad  
<public kyllingen.nospamnet> wrote:


 On Mon, 09 Aug 2010 08:11:54 -0400, Steven Schveighoffer wrote:


 On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:


 == Quote from Mafi (mafi example.org)'s article

 Hi all,
 what are the directions of D2's streams. I heared std.stream is going
 to get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going
 to get a Java-like stream model? What's wrong with the old one except
 that it's a bit out of date?
 Mafi


 Two things off the top of my head:

 1.  std.stream doesn't correctly support files > 2GB on 32-bit Linux,
 and possibly
 on other OS's as well.


 Doesn't that have to do with using C's FILE * interface?


 FILE* isn't used anywhere in std.stream.  You're either thinking of
 std.stdio.File or std.cstream.CFile.


Ah, ok.  But does that mean that the problem isn't going away by  
deprecating std.stream?  I thought FILE * doesn't support large files.

If that's the case, then assertion 1 from David is incorrect, it's not a  
good reason to get rid of std.stream.

If that's not the case, then I'm wrong :)

-Steve
Aug 09 2010
parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Mon, 09 Aug 2010 09:11:19 -0400, Steven Schveighoffer wrote:


 On Mon, 09 Aug 2010 09:01:33 -0400, Lars T. Kyllingstad
 <public kyllingen.nospamnet> wrote:
 

 On Mon, 09 Aug 2010 08:11:54 -0400, Steven Schveighoffer wrote:


 On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:


 == Quote from Mafi (mafi example.org)'s article

 Hi all,
 what are the directions of D2's streams. I heared std.stream is
 going to get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2
 going to get a Java-like stream model? What's wrong with the old one
 except that it's a bit out of date?
 Mafi


 Two things off the top of my head:

 1.  std.stream doesn't correctly support files > 2GB on 32-bit Linux,
 and possibly
 on other OS's as well.


 Doesn't that have to do with using C's FILE * interface?


 FILE* isn't used anywhere in std.stream.  You're either thinking of
 std.stdio.File or std.cstream.CFile.


 
 Ah, ok.  But does that mean that the problem isn't going away by
 deprecating std.stream?  I thought FILE * doesn't support large files.
 
 If that's the case, then assertion 1 from David is incorrect, it's not a
 good reason to get rid of std.stream.
 
 If that's not the case, then I'm wrong :)



A bit of googling shows you are right.  We really should get rid of the 
FILE* dependency...

-Lars
Aug 09 2010
prev sibling parent dsimcha <dsimcha yahoo.com> writes:
== Quote from Steven Schveighoffer (schveiguy yahoo.com)'s article

 On Sun, 08 Aug 2010 09:35:21 -0400, dsimcha <dsimcha yahoo.com> wrote:

 == Quote from Mafi (mafi example.org)'s article

 Hi all,
 what are the directions of D2's streams. I heared std.stream is going to
 get deprecated in D2. I think it's good as long as
 BufferedFile.readLine() returns char[] instead of string. Is D2 going to
 get a Java-like stream model? What's wrong with the old one except that
 it's a bit out of date?
 Mafi


 Two things off the top of my head:

 1.  std.stream doesn't correctly support files > 2GB on 32-bit Linux,
 and possibly
 on other OS's as well.


 Doesn't that have to do with using C's FILE * interface?
 -Steve


No, std.stdio uses this.  std.stream uses OS-native APIs at least in some
places.
Aug 09 2010
prev sibling next sibling parent Adrian Matoga <epi atari8.info> writes:
On 2010-08-08 14:11, Mafi wrote:

 Hi all,
 what are the directions of D2's streams. I heared std.stream is going to 
 get deprecated in D2. I think it's good as long as 
 BufferedFile.readLine() returns char[] instead of string. Is D2 going to 
 get a Java-like stream model? What's wrong with the old one except that 
 it's a bit out of date?
 Mafi


AIUI from the discussion under the topic "Network I/O and streaming in 
D2" the whole I/O will go through std.stdio.File, which looks great for 
a closed set of possible streams, but I'm afraid that it won't be 
sufficiently extensible for some cases.

Like many people, I found Java stream model bloated.
However, in my opinion, Stream makes it possible to extend I/O subsystem 
at library level with custom streams, while still having common 
(standard!) interface to them, and a possibility to use a fair amount of 
already implemented operations like reading/writing various primitive 
types and arrays, at very small cost of implementing only a constructor 
and read/write primitives.

In my modest experience, I found std.stream.Stream a good solution in 
some cases: once the application could have a common interface for both 
secure and raw network stream (the first being a custom stream, which 
wrapped OpenSSL SSL structures, and the second being SocketStream from 
std.socketstream). Another case is what I'm currently developing - a 
library for accessing 8-bit Atari disk images. There are a few different 
filesystems possible inside such images, and for each one there will
be a different translation between image contents and file operations,
but the file operations should be hidden behind a common interface. 
Again, Stream seemed the best choice for me - firstly, it comes from 
standard library; secondly, it requires only readBlock and writeBlock to 
be implemented, and the rest is already done.
I also wondered if things like DirIterator and DirEntry could also be 
interfaces (I wrote my own interfaces for this purpose), but these could 
be of much less usage.

To sum up, I think it's worth saving Stream as a common interface for 
I/O, but it possibly needs some improvements in details. For example, 
instead of having its own File class, conflicting with std.stdio.File, 
it could simply wrap std.stdio.File into, let's say, FileStream or sth. 
Another thing would be making the set of methods resembling that of 
std.stdio.File. It would be nice to use std.stdio.File when you don't 
need extensibility, but have an option to easily move to descendants of 
std.stream.Stream without changing every single call.

What are your opinions?

Adrian
Aug 08 2010
prev sibling parent Kagamin <spam here.lot> writes:
Adrian Matoga Wrote:


 What are your opinions?


If you only want readBlock and writeBlock, you can use delegates instead of
streams.
Aug 09 2010