www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Stream, Socket == SocketStream...

reply "Regan Heath" <regan netwin.co.nz> writes:
Ok,

I want to create a class/wrapper that lets me call Stream methods i.e.  
readLine, writeLine, etc. But, also lets me call Socket methods i.e.  
'connect' eg.

version(build) {pragma(link, wsock32);}

import std.socket;
import std.stream;
import std.stdio;

class SocketStream : Stream
{
	this(Socket s)
	{
		_socket = s;
		readable = true;
		writeable = true;
		seekable = false;
		isopen = _socket.isAlive();
	}
	
	override size_t readBlock(void* result, size_t len)
	{
		void[] tmp = result[0..len];
		return socket.receive(tmp);
	}
	
	override size_t writeBlock(void* result, size_t len)
	{
		void[] tmp = result[0..len];
		return socket.send(tmp);
	}
	
	override ulong seek(long offset, SeekPos whence)
	{
		assert(0);
	}
private:
	Socket _socket;
}

int main(char[][] args)
{
	BufferedStream s = new BufferedStream(new SocketStream(new TcpSocket()));
	
	//this is not possible
	s.connect(new InternetAddress("www.digitalmars.com",80));

	//these work.
	s.writeLine("GET / HTTP/1.0");
	s.writeLine("");
	writefln("%s",s.readLine());
}

Before anyone points it out, yes, I realise I can put the hostname and  
port in the constructor for TcpSocket. I don't want to do that.

Before anyone (Kris) directs me to another library which contains an  
existing SocketStream like class, thanks, but I'm also interested in how  
you solve the general problem.

It seems to me, that multiple inheritance would let me do this, so, what's  
the D solution?

My first thought is that if we took the Stream class methods, placed them  
in templates TInputStream and TOutputStream, then I could mix them into my  
class.

Meaning, I could derive my SocketStream above from "TcpSocket" and mixin  
the TInputStream and TOutputStream methods.

But! doesn't that mean it's not technically a "Stream" and as such so I  
wouldn't be able to use it with BufferedStream? If so, ideally I'd want a  
"Stream" interface to derive from also.

That would mean the Stream class would have to go in favour of 3  
interfaces (InputStream, OutputStream, Stream) and 2 templates  
(TInputStream, TOutputStream).

Thoughts? Ideas?

Regan
Jun 08 2005
next sibling parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
"Regan Heath" <regan netwin.co.nz> wrote in message 
news:opsr2zs8mt23k2f5 nrage.netwin.co.nz...

 Ok,

 I want to create a class/wrapper that lets me call Stream methods i.e. 
 readLine, writeLine, etc. But, also lets me call Socket methods i.e. 
 'connect' eg.

 version(build) {pragma(link, wsock32);}

 import std.socket;
 import std.stream;
 import std.stdio;

 class SocketStream : Stream
 {
 this(Socket s)
 {
 _socket = s;
 readable = true;
 writeable = true;
 seekable = false;
 isopen = _socket.isAlive();
 }

 override size_t readBlock(void* result, size_t len)
 {
 void[] tmp = result[0..len];
 return socket.receive(tmp);
 }

 override size_t writeBlock(void* result, size_t len)
 {
 void[] tmp = result[0..len];
 return socket.send(tmp);
 }

 override ulong seek(long offset, SeekPos whence)
 {
 assert(0);
 }
 private:
 Socket _socket;
 }

 int main(char[][] args)
 {
 BufferedStream s = new BufferedStream(new SocketStream(new TcpSocket()));

 //this is not possible
 s.connect(new InternetAddress("www.digitalmars.com",80));

 //these work.
 s.writeLine("GET / HTTP/1.0");
 s.writeLine("");
 writefln("%s",s.readLine());
 }

 Before anyone points it out, yes, I realise I can put the hostname and 
 port in the constructor for TcpSocket. I don't want to do that.

 Before anyone (Kris) directs me to another library which contains an 
 existing SocketStream like class, thanks, but I'm also interested in how 
 you solve the general problem.

 It seems to me, that multiple inheritance would let me do this, so, what's 
 the D solution?

 My first thought is that if we took the Stream class methods, placed them 
 in templates TInputStream and TOutputStream, then I could mix them into my 
 class.


I tried factoring out TInputStream and TOutputStream a while ago but the 
private unget buffer couldn't go into the mixin so I didn't pursue it very 
far (you can't mix in private members). We could just make the unget buffer 
public and live with it.


 Meaning, I could derive my SocketStream above from "TcpSocket" and mixin 
 the TInputStream and TOutputStream methods.

 But! doesn't that mean it's not technically a "Stream" and as such so I 
 wouldn't be able to use it with BufferedStream? If so, ideally I'd want a 
 "Stream" interface to derive from also.

 That would mean the Stream class would have to go in favour of 3 
 interfaces (InputStream, OutputStream, Stream) and 2 templates 
 (TInputStream, TOutputStream).

 Thoughts? Ideas?

 Regan


Stream should be pretty much the same as the union of InputStream and 
OutputStream with the missing functions being the seeking functions and 
read/writeBlock. You've given a good argument for making BufferedStream wrap 
an abbreviated interface that just supplies the readBlock, writeBlock and 
seek members (and maybe one or two other things I've forgotten the buffering 
needs). Maybe call this interface ... something like BasicStream or 
SimpleStream. It would look something like
interface BasicStream {
  size_t readBlock(void* buffer, size_t size);
  size_t writeBlock(void* buffer, size_t size);
  ulong seek(long offset, SeekPos rel);
}
class Stream : InputStream, OutputStream, BasicStream { ... }
class BufferedStream : Stream {
  BasicStream s;
  this(BasicStream s) {this.s = s; ... }
}
That way your class wouldn't subclass Stream but would implement some or all 
of the interfaces.
Jun 09 2005
parent reply "Kris" <fu bar.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote ...

 Stream should be pretty much the same as the union of InputStream and
 OutputStream with the missing functions being the seeking functions and
 read/writeBlock. You've given a good argument for making BufferedStream


wrap

 an abbreviated interface that just supplies the readBlock, writeBlock and
 seek members (and maybe one or two other things I've forgotten the


buffering

 needs). Maybe call this interface ... something like BasicStream or
 SimpleStream. It would look something like
 interface BasicStream {
   size_t readBlock(void* buffer, size_t size);
   size_t writeBlock(void* buffer, size_t size);
   ulong seek(long offset, SeekPos rel);
 }
 class Stream : InputStream, OutputStream, BasicStream { ... }
 class BufferedStream : Stream {
   BasicStream s;
   this(BasicStream s) {this.s = s; ... }
 }
 That way your class wouldn't subclass Stream but would implement some or


all

 of the interfaces.



BasicStream should probably not have a seek() method, since not all streams
are seekable (socket, for example). This kind of issue, and the limitations
implied via Decorators, is why mango.io took the route it did. It doesn't
exhibit these problems, is more flexible, and is simpler to use ;)
Jun 09 2005
parent reply "Ben Hinkle" <bhinkle mathworks.com> writes:
"Kris" <fu bar.com> wrote in message news:d8a2n1$22pn$1 digitaldaemon.com...

 "Ben Hinkle" <ben.hinkle gmail.com> wrote ...

 Stream should be pretty much the same as the union of InputStream and
 OutputStream with the missing functions being the seeking functions and
 read/writeBlock. You've given a good argument for making BufferedStream


 wrap

 an abbreviated interface that just supplies the readBlock, writeBlock and
 seek members (and maybe one or two other things I've forgotten the


 buffering

 needs). Maybe call this interface ... something like BasicStream or
 SimpleStream. It would look something like
 interface BasicStream {
   size_t readBlock(void* buffer, size_t size);
   size_t writeBlock(void* buffer, size_t size);
   ulong seek(long offset, SeekPos rel);
 }
 class Stream : InputStream, OutputStream, BasicStream { ... }
 class BufferedStream : Stream {
   BasicStream s;
   this(BasicStream s) {this.s = s; ... }
 }
 That way your class wouldn't subclass Stream but would implement some or


 all

 of the interfaces.



 BasicStream should probably not have a seek() method, since not all 
 streams
 are seekable (socket, for example). This kind of issue, and the 
 limitations
 implied via Decorators, is why mango.io took the route it did. It doesn't
 exhibit these problems, is more flexible, and is simpler to use ;)


If the goal is to make an interface that can be the backing stream for a 
BufferedStream then seek() is needed. Streams throw if they aren't 
seekable - as they throw if they aren't readable or writeable (or 
writable...)
Anyway, some interface for the backing stream for BufferedStream, 
EndianStream and SliceStream will probably be fine given the template 
mixins. The name BasicStream doesn't sound quite right so any suggestions 
are welcome. Glancing at those wrapper streams the interface will probably 
have to inherit both InputStream and OutputStream and add one or two seek 
members.
Jun 09 2005
parent reply "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 9 Jun 2005 17:15:02 -0400, Ben Hinkle <bhinkle mathworks.com>  
wrote:

 "Kris" <fu bar.com> wrote in message  
 news:d8a2n1$22pn$1 digitaldaemon.com...

 "Ben Hinkle" <ben.hinkle gmail.com> wrote ...

 Stream should be pretty much the same as the union of InputStream and
 OutputStream with the missing functions being the seeking functions and
 read/writeBlock. You've given a good argument for making BufferedStream


 wrap

 an abbreviated interface that just supplies the readBlock, writeBlock  
 and
 seek members (and maybe one or two other things I've forgotten the


 buffering

 needs). Maybe call this interface ... something like BasicStream or
 SimpleStream. It would look something like
 interface BasicStream {
   size_t readBlock(void* buffer, size_t size);
   size_t writeBlock(void* buffer, size_t size);
   ulong seek(long offset, SeekPos rel);
 }
 class Stream : InputStream, OutputStream, BasicStream { ... }
 class BufferedStream : Stream {
   BasicStream s;
   this(BasicStream s) {this.s = s; ... }
 }
 That way your class wouldn't subclass Stream but would implement some  
 or


 all

 of the interfaces.



 BasicStream should probably not have a seek() method, since not all
 streams
 are seekable (socket, for example). This kind of issue, and the
 limitations
 implied via Decorators, is why mango.io took the route it did. It  
 doesn't
 exhibit these problems, is more flexible, and is simpler to use ;)


 If the goal is to make an interface that can be the backing stream for a
 BufferedStream then seek() is needed.


I dont think that is the 'goal'. The goal is to have a logical set of  
classes/templates/interfaces that define how streams are to be handled in  
D. Some streams aren't seekable. BufferedStream can only operate on a  
stream which is seekable? that doesn't seem right. What if I want to  
buffer my input, but never plan to seek it?

It seems to me we have 3 properties Readability, Writability and  
Seekability, a stream may have 1, 2 or all 3 of them. So we need 3  
seperate interfaces, 3 seperate templates to mixin, and some way of saying  
"I require a stream which can read and seek" or "write and seek" or "read  
and write" or "read and write and seek".


 Streams throw if they aren't
 seekable - as they throw if they aren't readable or writeable (or
 writable...)


Wouldn't it be better to know at compile time that it's not seekable. As  
in:

void foo(ISeekable a) {}
SocketStream s = new SocketStream();
foo(s); //compile time error.

We can _also_ throw if the seekable nature of a stream changes.


 Anyway, some interface for the backing stream for BufferedStream,
 EndianStream and SliceStream will probably be fine given the template
 mixins. The name BasicStream doesn't sound quite right so any suggestions
 are welcome. Glancing at those wrapper streams the interface will  
 probably
 have to inherit both InputStream and OutputStream and add one or two seek
 members.


It's a stream which can read, write, and seek. SeekStream might not be too  
bad a name.

Regan
Jun 09 2005
parent reply "Ben Hinkle" <ben.hinkle gmail.com> writes:
 BasicStream should probably not have a seek() method, since not all
 streams
 are seekable (socket, for example). This kind of issue, and the
 limitations
 implied via Decorators, is why mango.io took the route it did. It 
 doesn't
 exhibit these problems, is more flexible, and is simpler to use ;)


 If the goal is to make an interface that can be the backing stream for a
 BufferedStream then seek() is needed.


 I dont think that is the 'goal'. The goal is to have a logical set of 
 classes/templates/interfaces that define how streams are to be handled in 
 D. Some streams aren't seekable. BufferedStream can only operate on a 
 stream which is seekable? that doesn't seem right. What if I want to 
 buffer my input, but never plan to seek it?


BufferedStream works fine with non-seekable streams. Kris's design is to 
make seekable a compile-time decision. std.stream makes seekable a run-time 
decision. It's handy to make seekable a run-time decision since that allows 
better integration with std.c.stdio and the OS concept of files since pipes 
are considered files.


 It seems to me we have 3 properties Readability, Writability and 
 Seekability, a stream may have 1, 2 or all 3 of them. So we need 3 
 seperate interfaces, 3 seperate templates to mixin, and some way of saying 
 "I require a stream which can read and seek" or "write and seek" or "read 
 and write" or "read and write and seek".


Yup that's a possiblity. The question is how many interfaces does one need? 
If we had Read/Write/Seek and the combinations of those then that's seven 
total interfaces. I propose three of those cover enough surface area: Read, 
Write and Read+Write+Seek. If one wants one of the other interface then the 
run-time decisions ala Stream are available. The Read interface is 
InputStream; Write is OutputStream and Read+Write+Seek is <name TDB>.


 Streams throw if they aren't
 seekable - as they throw if they aren't readable or writeable (or
 writable...)


 Wouldn't it be better to know at compile time that it's not seekable. As 
 in:

 void foo(ISeekable a) {}
 SocketStream s = new SocketStream();
 foo(s); //compile time error.

 We can _also_ throw if the seekable nature of a stream changes.


The use cases of a function that takes ISeekable is very small (ie I can't 
think of any).


 Anyway, some interface for the backing stream for BufferedStream,
 EndianStream and SliceStream will probably be fine given the template
 mixins. The name BasicStream doesn't sound quite right so any suggestions
 are welcome. Glancing at those wrapper streams the interface will 
 probably
 have to inherit both InputStream and OutputStream and add one or two seek
 members.


 It's a stream which can read, write, and seek. SeekStream might not be too 
 bad a name.


To me SeekStream would just contain the seek methods and wouldn't cover the 
rest of streaming. So off the top of my head something more inclusive would 
be better.
Jun 09 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 9 Jun 2005 19:21:44 -0400, Ben Hinkle <ben.hinkle gmail.com> wrote:

 BufferedStream works fine with non-seekable streams.


Ok.


 Kris's design is to
 make seekable a compile-time decision.


Yep.


 std.stream makes seekable a run-time decision.


A runtime decision *only*.


 It's handy to make seekable a run-time decision since that allows better  
 integration with std.c.stdio and the OS concept of files since pipes are  
 considered files.


It's handier to do both (compile+runtime)


 It seems to me we have 3 properties Readability, Writability and
 Seekability, a stream may have 1, 2 or all 3 of them. So we need 3
 seperate interfaces, 3 seperate templates to mixin, and some way of  
 saying "I require a stream which can read and seek" or "write and seek"  
 or "read and write" or "read and write and seek".


 Yup that's a possiblity. The question is how many interfaces does one  
 need? If we had Read/Write/Seek and the combinations of those then  
 that's seven
 total interfaces.


True. But what's wrong with that?

--basic
InputStream
OutputStream
SeekStream

--combinations
InputSeekStream
OutputSeekStream
InputOutputStream

--full
InputOutputSeekStream (or simply "Stream" for brevity)

It does get kinda long winded, but, you'd use the combinations rarely.

Q: Why include the word 'Stream' at all?
Q: What is a stream anyway?

It seems to me a "Stream" is anything you can read bytes from *or*  
anything you can write bytes to, my definition has never included  
seekability. So, IMO, a "Stream" should not be seekable at all. Nor would  
it include any sort of unget or other advanced idea.

Of course, we want seekable things, we want unget, we want buffering, we'd  
probably never directly use something that didn't have one of those, but,  
my feeling is that they should be factored in outside of the basic  
"Stream" interface, i.e. in BufferedStream perhaps.

I think perhaps I should have a go at designing something myself, at least  
then I'll have something concrete to bring to the table. To be honest I am  
relatively new to the whole Stream concept, being from a C (not C++)  
background.


 Streams throw if they aren't
 seekable - as they throw if they aren't readable or writeable (or
 writable...)


 Wouldn't it be better to know at compile time that it's not seekable. As
 in:

 void foo(ISeekable a) {}
 SocketStream s = new SocketStream();
 foo(s); //compile time error.

 We can _also_ throw if the seekable nature of a stream changes.


 The use cases of a function that takes ISeekable is very small (ie I  
 can't think of any).


True, I was assuming it would also read or write, the example would be  
better asking for the combination of seek + read or write.

I guess the only point I'm making is that it would be nice to be able to  
select/check seekability at compile time as well as runtime (which we  
currently have).

Regan
Jun 09 2005
prev sibling parent "Kris" <fu bar.com> writes:
"Ben Hinkle" <ben.hinkle gmail.com> wrote...

 BufferedStream works fine with non-seekable streams. Kris's design is to
 make seekable a compile-time decision. std.stream makes seekable a


run-time

 decision. It's handy to make seekable a run-time decision since that


allows

 better integration with std.c.stdio and the OS concept of files since


pipes

 are considered files.


Correction: the developer has a choice with mango.io ~ use compile-time or
runtime checking ~ one is not forced down a single path. This is not true of
phobos.io, but it's really not that important <g>
Jun 09 2005
prev sibling next sibling parent reply Carlos Santander <csantander619 gmail.com> writes:
Regan Heath escribió:

 Ok,
 
 I want to create a class/wrapper that lets me call Stream methods i.e.  
 readLine, writeLine, etc. But, also lets me call Socket methods i.e.  
 'connect' eg.
 


How about std.socketstream?

-- 
Carlos Santander Bernal
Jun 09 2005
parent "Regan Heath" <regan netwin.co.nz> writes:
On Thu, 09 Jun 2005 15:20:08 -0500, Carlos Santander  
<csantander619 gmail.com> wrote:

 Regan Heath escribió:

 Ok,
  I want to create a class/wrapper that lets me call Stream methods  
 i.e.  readLine, writeLine, etc. But, also lets me call Socket methods  
 i.e.  'connect' eg.


 How about std.socketstream?


It's almost exactly the same as the code I provided and has the same  
problems (which aren't really huge problems, it's just not quite what I'd  
prefer).

Regan
Jun 09 2005
prev sibling parent "Regan Heath" <regan netwin.co.nz> writes:
Being dissatisfied with the current stream implementation (no offence  
intended Ben you do some great work) I've had a play with various stream  
ideas.

I keep coming back to seperating input and output, it makes the most sense  
to me, but it's not my preffered choice for usability i.e. it's nice to  
have a single object/class and both read and write to/from it. I can't  
seem to achieve this, how I'd like, without some sort of multiple  
inheritance.

What are peoples favourite stream libraries (I've poked my nose into the  
Java one and the mango one, what else should I look at for inspiration).

I found a good solution to the whole seekability problem. You define a  
seekable interface, eg.

interface ISeekable
{
   size_t seek(size_t offset, SeekPos from);
   ..etc..
}

then you can use "static if" to check for seekability (rather than having  
a boolean flag in the class. The same applies to readability and  
writability. This way you can have compile time checking (I believe).

Regan
Jun 20 2005