www.digitalmars.com         C & C++   DMDScript  

D - More problems with std.stream

reply C <dont respond.com> writes:
Using File 's readLine method , I read in a line , that has nothing but =
a =

number and I get trash preceding each string.  Its the same if i write t=
o =

stdout or write to a file ( see attached )

This is bordering on ridicoulous ... I hate having to re-compile phobos =

everytime a new version comes out just to take out this worse than usele=
ss =

stream package.  Why not adopt Andy's version untill some can be written=
 =

?? Something ... anything ... please!!!

C



-- =

D Newsgroup.
Mar 28 2004
parent reply Vathix <vathix dprogramming.com> writes:
C wrote:

 Using File 's readLine method , I read in a line , that has nothing but 
 a number and I get trash preceding each string.  Its the same if i write 
 to stdout or write to a file ( see attached )
 
 This is bordering on ridicoulous ... I hate having to re-compile phobos 
 everytime a new version comes out just to take out this worse than 
 useless stream package.  Why not adopt Andy's version untill some can be 
 written ?? Something ... anything ... please!!!
 
 C


Stream.write() puts the string length (binary) before the string, which 
doesn't look good in the console. This is good for files, so you can use 
a corresponding Stream.read() to easily reconstruct the same length 
string. What you want is Stream.writeString() or Stream.writeLine().


-- 
Christopher E. Miller
Mar 28 2004
parent reply C <dont respond.com> writes:
Dont I feel like an ass :/.  That works well thank you.

On Sun, 28 Mar 2004 17:11:36 -0500, Vathix <vathix dprogramming.com> wro=
te:


 C wrote:

 Using File 's readLine method , I read in a line , that has nothing b=




ut =


 a number and I get trash preceding each string.  Its the same if i =





 write to stdout or write to a file ( see attached )

 This is bordering on ridicoulous ... I hate having to re-compile phob=




os =


 everytime a new version comes out just to take out this worse than =





 useless stream package.  Why not adopt Andy's version untill some can=




 =


 be written ?? Something ... anything ... please!!!

 C


 Stream.write() puts the string length (binary) before the string, whic=


h =


 doesn't look good in the console. This is good for files, so you can u=


se =


 a corresponding Stream.read() to easily reconstruct the same length =



 string. What you want is Stream.writeString() or Stream.writeLine().




-- =

D Newsgroup.
Mar 28 2004
parent reply "C. Sauls" <ibisbasenji yahoo.com> writes:
Don't feel bad, the current Stream implementation /does/ need some work, 
and in my opinion needs to get split into multiple modules (ie 
std.stream.console for stdin/out/err, std.stream.file for FileStream, 
etc, with a std.stream.stream defining the Stream interfact and 
StdStream base class... just as a suggested design.  ;))

-C. Sauls
-Invironz

C wrote:

 Dont I feel like an ass :/.  That works well thank you.
Mar 28 2004
next sibling parent reply C <dont respond.com> writes:
I agree , needs work.  So far no-one has stepped up , you want to give i=
t =

ago :) ?  Your uniquely qualified ;).

C

On Sun, 28 Mar 2004 16:51:52 -0600, C. Sauls <ibisbasenji yahoo.com> wro=
te:


 Don't feel bad, the current Stream implementation /does/ need some wor=


k, =


 and in my opinion needs to get split into multiple modules (ie =



 std.stream.console for stdin/out/err, std.stream.file for FileStream, =



 etc, with a std.stream.stream defining the Stream interfact and =



 StdStream base class... just as a suggested design.  ;))

 -C. Sauls
 -Invironz

 C wrote:

 Dont I feel like an ass :/.  That works well thank you.






-- =

D Newsgroup.
Mar 28 2004
parent reply "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
If you're interested, I have a high performance IO package that could use
some practical feedback. It's part of the Dsc (D servlet container) project,
but operates quite happily in a standalone configuration. Coincidently, it's
partitioned pretty much as described below <g>

- Kris


"C" <dont respond.com> wrote in message news:opr5l0f6imehmtou localhost...
I agree , needs work.  So far no-one has stepped up , you want to give it
ago :) ?  Your uniquely qualified ;).

C

On Sun, 28 Mar 2004 16:51:52 -0600, C. Sauls <ibisbasenji yahoo.com> wrote:


 Don't feel bad, the current Stream implementation /does/ need some work,
 and in my opinion needs to get split into multiple modules (ie
 std.stream.console for stdin/out/err, std.stream.file for FileStream,
 etc, with a std.stream.stream defining the Stream interfact and
 StdStream base class... just as a suggested design.  ;))

 -C. Sauls
 -Invironz

 C wrote:

 Dont I feel like an ass :/.  That works well thank you.






--
D Newsgroup.
Mar 28 2004
next sibling parent reply C <dont respond.com> writes:
Sweet , I was going to try the DSC out, but I have no idea what a servle=
t =

container is  :D.  Is there a download for the project ?

C



On Sun, 28 Mar 2004 20:38:50 -0800, Kris =

<someidiot earthlink.dot.dot.dot.net> wrote:


 If you're interested, I have a high performance IO package that could =


use

 some practical feedback. It's part of the Dsc (D servlet container) =



 project,
 but operates quite happily in a standalone configuration. Coincidently=


, =


 it's
 partitioned pretty much as described below <g>

 - Kris


 "C" <dont respond.com> wrote in message =



 news:opr5l0f6imehmtou localhost...
 I agree , needs work.  So far no-one has stepped up , you want to give=


 it

 ago :) ?  Your uniquely qualified ;).

 C

 On Sun, 28 Mar 2004 16:51:52 -0600, C. Sauls <ibisbasenji yahoo.com> =



 wrote:


 Don't feel bad, the current Stream implementation /does/ need some wo=




rk,

 and in my opinion needs to get split into multiple modules (ie
 std.stream.console for stdin/out/err, std.stream.file for FileStream,=





 etc, with a std.stream.stream defining the Stream interfact and
 StdStream base class... just as a suggested design.  ;))

 -C. Sauls
 -Invironz

 C wrote:

 Dont I feel like an ass :/.  That works well thank you.






 --
 D Newsgroup.




-- =

D Newsgroup.
Mar 29 2004
parent "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Not yet, though it will be available at dsource.org after the alpha round
has completed. If you're willing to provide some feedback I'll email you a
copy (though your stated email address is unlikely to be a real one :-)

- Kris


"C" <dont respond.com> wrote in message news:opr5l6lebhehmtou localhost...
Sweet , I was going to try the DSC out, but I have no idea what a servlet
container is  :D.  Is there a download for the project ?

C



On Sun, 28 Mar 2004 20:38:50 -0800, Kris
<someidiot earthlink.dot.dot.dot.net> wrote:


 If you're interested, I have a high performance IO package that could use
 some practical feedback. It's part of the Dsc (D servlet container)
 project,
 but operates quite happily in a standalone configuration. Coincidently,
 it's
 partitioned pretty much as described below <g>

 - Kris


 "C" <dont respond.com> wrote in message
 news:opr5l0f6imehmtou localhost...
 I agree , needs work.  So far no-one has stepped up , you want to give it
 ago :) ?  Your uniquely qualified ;).

 C

 On Sun, 28 Mar 2004 16:51:52 -0600, C. Sauls <ibisbasenji yahoo.com>
 wrote:


 Don't feel bad, the current Stream implementation /does/ need some work,
 and in my opinion needs to get split into multiple modules (ie
 std.stream.console for stdin/out/err, std.stream.file for FileStream,
 etc, with a std.stream.stream defining the Stream interfact and
 StdStream base class... just as a suggested design.  ;))

 -C. Sauls
 -Invironz

 C wrote:

 Dont I feel like an ass :/.  That works well thank you.






 --
 D Newsgroup.




--
D Newsgroup.
Mar 28 2004
prev sibling parent reply "C. Sauls" <ibisbasenji yahoo.com> writes:
I'm interested.  I was rather seriously thinking of just writing my own, 
and it would be nice not to have to (at least for some things).

-C. Sauls
-Invironz
-ibisbasenji yahoo.com (Yeah I'm bold.)

Kris wrote:

 If you're interested, I have a high performance IO package that could use
 some practical feedback. It's part of the Dsc (D servlet container) project,
 but operates quite happily in a standalone configuration. Coincidently, it's
 partitioned pretty much as described below <g>
 
 - Kris
 
 
 "C" <dont respond.com> wrote in message news:opr5l0f6imehmtou localhost...
 I agree , needs work.  So far no-one has stepped up , you want to give it
 ago :) ?  Your uniquely qualified ;).
 
 C
 
 On Sun, 28 Mar 2004 16:51:52 -0600, C. Sauls <ibisbasenji yahoo.com> wrote:
 
 

Don't feel bad, the current Stream implementation /does/ need some work,
and in my opinion needs to get split into multiple modules (ie
std.stream.console for stdin/out/err, std.stream.file for FileStream,
etc, with a std.stream.stream defining the Stream interfact and
StdStream base class... just as a suggested design.  ;))

-C. Sauls
-Invironz

C wrote:


Dont I feel like an ass :/.  That works well thank you.




 
 
 
 
 --
 D Newsgroup.
Mar 29 2004
parent reply "C. Sauls" <ibisbasenji yahoo.com> writes:
Just got the email, downloaded it, and gave it a quick read through. 
The design is interesting (in the sense of not quite what I'm used to) 
but rather elegant in its own way IMHO.  Do try to get scanf working 
soon, as once our file format is finalized I plan to do away with most 
of the readline/writeline code and replace it with scanf/printf so I can 
compress the data a bit by stacking several values into a line.

So, from looking at it and not yet having a chance to actually try using 
it, I believe I'd create my main files using something like:

import dsc.io.FileStyle,
        dsc.io.FileConduit;

FileConduit logfile, dbfile, chkptfile;
logfile   = new FileConduit(db.name ~ ".log", FileStyle.WriteAppending);
dbfile    = new FileConduit(db.name ~ ".neo", FileStyle.ReadExisting  );
chkptfile = new FileConduit(db.name ~ ".chk", FileStyle.WriteTruncate );

-C. Sauls
-Invironz
Mar 29 2004
parent reply "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Scanf integration:  Will do ...
Example below:  Yep, that's how the file stuff works.

I'm not familiar with neo ... what does it do?

- Kris

BTW, I just sent you an alpha-4 release.


"C. Sauls" <ibisbasenji yahoo.com> wrote in message
news:c4a80l$qc0$1 digitaldaemon.com...

 Just got the email, downloaded it, and gave it a quick read through.
 The design is interesting (in the sense of not quite what I'm used to)
 but rather elegant in its own way IMHO.  Do try to get scanf working
 soon, as once our file format is finalized I plan to do away with most
 of the readline/writeline code and replace it with scanf/printf so I can
 compress the data a bit by stacking several values into a line.

 So, from looking at it and not yet having a chance to actually try using
 it, I believe I'd create my main files using something like:

 import dsc.io.FileStyle,
         dsc.io.FileConduit;

 FileConduit logfile, dbfile, chkptfile;
 logfile   = new FileConduit(db.name ~ ".log", FileStyle.WriteAppending);
 dbfile    = new FileConduit(db.name ~ ".neo", FileStyle.ReadExisting  );
 chkptfile = new FileConduit(db.name ~ ".chk", FileStyle.WriteTruncate );

 -C. Sauls
 -Invironz
Mar 29 2004
parent reply "C. Sauls" <ibisbasenji yahoo.com> writes:
Cool.  Sockets too... double cool.  Neo = scripted virtual environment 
server, actually an update and re-write of a server designed in the 
mid-80's.  Its partly because I think a better version could be written 
to take advantage of D, and partly because it'd look good in my 
portfolio.  ;)  Plus later on if someone asks something like how well D 
handles things like sockets and threading, then I can point at it as an 
example.

-C. Sauls
-Invironz

Kris wrote:

 Scanf integration:  Will do ...
 Example below:  Yep, that's how the file stuff works.
 
 I'm not familiar with neo ... what does it do?
 
 - Kris
 
 BTW, I just sent you an alpha-4 release.
Mar 29 2004
parent "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
That sounds pretty nifty! Is there any doc/info on it?

BTW, the Dsc.io socket support is built upon Christopher Miller's socket.d.
He even went to the trouble of adding SocketSet (wraps the wacky select()
function). Dsc.io uses the latter to support timeout on a socket read, with
no spinlock. SocketSet also supports accept() listeners: this is a great way
to go for building scalable network-servers.

- Kris

"C. Sauls" <ibisbasenji yahoo.com> wrote in message
news:c4acmv$1366$1 digitaldaemon.com...

 Cool.  Sockets too... double cool.  Neo = scripted virtual environment
 server, actually an update and re-write of a server designed in the
 mid-80's.  Its partly because I think a better version could be written
 to take advantage of D, and partly because it'd look good in my
 portfolio.  ;)  Plus later on if someone asks something like how well D
 handles things like sockets and threading, then I can point at it as an
 example.

 -C. Sauls
 -Invironz

 Kris wrote:

 Scanf integration:  Will do ...
 Example below:  Yep, that's how the file stuff works.

 I'm not familiar with neo ... what does it do?

 - Kris

 BTW, I just sent you an alpha-4 release.
Mar 29 2004
prev sibling parent reply Ben Hinkle <bhinkle4 juno.com> writes:
On Sun, 28 Mar 2004 16:51:52 -0600, "C. Sauls" <ibisbasenji yahoo.com>
wrote:


Don't feel bad, the current Stream implementation /does/ need some work, 


Is there a thread or page somewhere listing what should be done
to std.stream? Is it just efficiency (ie: add BufferedStream)?
Are people thinking of something like Java's stream package
with interfaces for reading/writing etc?
I like std.stream's simplicity - there is no heavy machinery
or APIs to wade through.


and in my opinion needs to get split into multiple modules (ie 
std.stream.console for stdin/out/err, std.stream.file for FileStream, 
etc, with a std.stream.stream defining the Stream interfact and 
StdStream base class... just as a suggested design.  ;))


if the std.stream API gets alot bigger then sure. Otherwise 
I'd vote to leave it the way it is. In general have D coders
been putting one "public class" per module? I guess I do but
that is just because I'm used to Java.


-C. Sauls
-Invironz

C wrote:

 Dont I feel like an ass :/.  That works well thank you.
Mar 29 2004
next sibling parent reply C <dont respond.com> writes:
 Is there a thread or page somewhere listing what should be done
 to std.stream?


Theres the phobos wiki page : http://www.prowiki.org/wiki4d/wiki.cgi?Pho=
bos

I think the .eof thing is still much too slow, and File doesn't create t=
he =

file with FileMode.Out if the file doesnt already exist.  The former =

probably hard to fix without a re-write the latter trivial i imagine.


 In general have D coders
 been putting one "public class" per module? I guess I do but
 that is just because I'm used to Java.


I usually do , Ive never used Java though.

C



On Mon, 29 Mar 2004 08:53:32 -0500, Ben Hinkle <bhinkle4 juno.com> wrote=
:


 On Sun, 28 Mar 2004 16:51:52 -0600, "C. Sauls" <ibisbasenji yahoo.com>=



 wrote:


 Don't feel bad, the current Stream implementation /does/ need some wo=




rk,

 Is there a thread or page somewhere listing what should be done
 to std.stream? Is it just efficiency (ie: add BufferedStream)?
 Are people thinking of something like Java's stream package
 with interfaces for reading/writing etc?
 I like std.stream's simplicity - there is no heavy machinery
 or APIs to wade through.


 and in my opinion needs to get split into multiple modules (ie
 std.stream.console for stdin/out/err, std.stream.file for FileStream,=





 etc, with a std.stream.stream defining the Stream interfact and
 StdStream base class... just as a suggested design.  ;))


 if the std.stream API gets alot bigger then sure. Otherwise
 I'd vote to leave it the way it is. In general have D coders
 been putting one "public class" per module? I guess I do but
 that is just because I'm used to Java.


 -C. Sauls
 -Invironz

 C wrote:

 Dont I feel like an ass :/.  That works well thank you.








-- =

D Newsgroup.
Mar 29 2004
parent reply "Ben Hinkle" <bhinkle4 juno.com> writes:
 I think the .eof thing is still much too slow, and File doesn't create the
 file with FileMode.Out if the file doesnt already exist.  The former
 probably hard to fix without a re-write the latter trivial i imagine.


Glancing at the code it looks like eof could also be easy to fix. File
could override the method to do what it wants. I don't see why it needs
a re-write. Is the user pushed data back it would be easy to check
without calling the os. Am I missing something?

-Ben
Mar 29 2004
parent C <dont respond.com> writes:
 Am I missing something?


Doubtful, im talking out my arse as usual.  Do you think you could suppl=
y =

a patch or something ?

C

On Mon, 29 Mar 2004 15:04:18 -0500, Ben Hinkle <bhinkle4 juno.com> wrote=
:


 I think the .eof thing is still much too slow, and File doesn't creat=




e =


 the
 file with FileMode.Out if the file doesnt already exist.  The former
 probably hard to fix without a re-write the latter trivial i imagine.=





 Glancing at the code it looks like eof could also be easy to fix. File=



 could override the method to do what it wants. I don't see why it need=


s

 a re-write. Is the user pushed data back it would be easy to check
 without calling the os. Am I missing something?

 -Ben




-- =

D Newsgroup.
Mar 29 2004
prev sibling next sibling parent reply Brad Anderson <brad dsource.dot.org> writes:
Ben Hinkle wrote:

 Is there a thread or page somewhere listing what should be done
 to std.stream? Is it just efficiency (ie: add BufferedStream)?
 Are people thinking of something like Java's stream package
 with interfaces for reading/writing etc?
 I like std.stream's simplicity - there is no heavy machinery
 or APIs to wade through.


If you can get your hands on Kris Bell's IO lib (alpha), you'll see a 
bunch of work done on it.  I have only briefly looked at it, but there 
is promise.  He's overloaded << and >>, but you also have put() and 
get() methods to use if you don't like the C++ style.  There are a few 
different tokenizers, readers, writers, buffers, and it looks to be 
pretty efficient.

Discussion can happen in the DSC forum:

http://www.dsource.org/forums/viewforum.php?f=5

I only wish I had a bit more time to spend on it, kicking the tires.

The e-mail cautious people out there can send a personal message to Kris 
through the DSource forums application.

BA
Mar 29 2004
parent "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
Good point Brad (re: the dsource.org forums).

Regarding this IO thing, I personally feel there will always be at least
three different camps:

1) those who prefer to get close to the metal
2) those who prefer the comfort of various abstraction layers
3) those who happily switch between either, depending on the task at hand.

Dsc.io is fully buffered and has a vague Java flavour to it (because it's
built for a D servlet container), so that puts it closer to the second camp
than the first. In my opinion, the std.stream package is closer to the first
camp than the second due to, if nothing else, its current lack of buffered
IO. Another way to differentiate is the API depth ... std.stream is
decidedly anorexic, whereas Dsc.io is just skinny. Both are equally
appropriate in their own way, so this is fertile ground for the
fundamentalists to fight over <g>

In terms of performance, there are times when buffered IO will run rings
around the unbuffered variety; other times, buffering just gets in the way.
I personally rather like the abstraction layers (surprise!) because one can
easily plug in alternate ways of performing IO, such as native
memory-mapping. Of course, one can always write their own wrappers to do
whatever they like so, as always, it's horses-for-courses and
your-mileage-may-vary.

Slainte,

- Kris


"Brad Anderson" <brad dsource.dot.org> wrote in message
news:c49m22$2sec$1 digitaldaemon.com...


 Discussion can happen in the DSC forum:

 http://www.dsource.org/forums/viewforum.php?f=5
Mar 29 2004
prev sibling parent reply "C. Sauls" <ibisbasenji yahoo.com> writes:
One class per module?  Hmm.. my neo.database.d has like five... so nah. 
  But consider this, my main project (neo) is a daemon and uses the heck 
out of files, but doesn't touch stdin, stdout, stderr at all.  I don't 
need memory or slice streams, I don't need the console stream instances. 
  If the stream classes were split, then I could write neo to import 
just the File stream and overload it (as I already do) to add my two 
writeInt and writeDouble methods (that just wrap calls to 
this.writeLine(toString(arg)) anyhow) and the matching readInt/readDouble.

-C. Sauls
-Invironz

Ben Hinkle wrote:

 if the std.stream API gets alot bigger then sure. Otherwise 
 I'd vote to leave it the way it is. In general have D coders
 been putting one "public class" per module? I guess I do but
 that is just because I'm used to Java.
Mar 29 2004
parent "Kris" <someidiot earthlink.dot.dot.dot.net> writes:
I posted an earlier question asking whether the linker strips out unused
classes and/or modules, for exactly this reason.

For instance, dsc.io.Tokenizer contains a secondary class that imports
std.regexp. If the linker removes that class when unreferenced by an
application then I'm happy; otherwise I'll move that secondary class to a
separate module.

- Kris

"C. Sauls" <ibisbasenji yahoo.com> wrote in message
news:c4a4bf$jsc$1 digitaldaemon.com...

 One class per module?  Hmm.. my neo.database.d has like five... so nah.
   But consider this, my main project (neo) is a daemon and uses the heck
 out of files, but doesn't touch stdin, stdout, stderr at all.  I don't
 need memory or slice streams, I don't need the console stream instances.
   If the stream classes were split, then I could write neo to import
 just the File stream and overload it (as I already do) to add my two
 writeInt and writeDouble methods (that just wrap calls to
 this.writeLine(toString(arg)) anyhow) and the matching readInt/readDouble.

 -C. Sauls
 -Invironz

 Ben Hinkle wrote:

 if the std.stream API gets alot bigger then sure. Otherwise
 I'd vote to leave it the way it is. In general have D coders
 been putting one "public class" per module? I guess I do but
 that is just because I'm used to Java.
Mar 29 2004