digitalmars.D - Stream mixins?
- Janice Caron (17/17) Nov 10 2007 Hi, Walter, I just had a good idea. (Well, I think it's a good idea!)
- Kris (5/22) Nov 10 2007 This is a fair example of why Tango does things differently. The I/O str...
- Janice Caron (42/43) Nov 11 2007 Thank you, but since I don't use Tango that's not much help. Allow me
- BCS (11/35) Nov 11 2007 It is defined, but something like this: "Write takes the argument an con...
- Janice Caron (35/38) Nov 11 2007 There are plenty of examples of those!
- Kris (20/28) Nov 11 2007 Indeed. You perhaps don't care about this, Janice, but others may since ...
- BCS (22/51) Nov 11 2007 No, in your case the write() functions are useless. What you seem to wan...
- Janice Caron (30/46) Nov 11 2007 No, a stream of data. A property of hash functions is that you can
- BCS (42/96) Nov 11 2007 that is in no way contrdictory with what I sugested. The lowlevel stuff ...
- Sean Kelly (5/10) Nov 11 2007 Not much help for Phobos I suppose, but I did this in an example stream
Hi, Walter, I just had a good idea. (Well, I think it's a good idea!)
Please could you separate out the member functions of
std.stream.Stream into three mixins: ReadMixin, WriteMixin and
SeekMixin?
ReadMixin to include every read function except readBlock()
WriteMixin to include every write function except writeBlock()
SeekMixin to include every seek function.
Then you could implement Stream by mixing in the mixins.
More importantly, I could write
class MyOutputStream : OutputStream
{
this() { writeable = true; }
uint writeBlock(const const void* buffer, uint size) { /*...*/ }
mixin WriteMixin;
}
instead of having to derive from Stream (I don't want an input stream)
or implement writefln()
Nov 10 2007
This is a fair example of why Tango does things differently. The I/O stream
are discrete, they have interfaces for cases where inheritance won't
suffice, and Seekable is a trivial interface.
"Janice Caron" <caron800 googlemail.com> wrote in message
news:mailman.31.1194728178.2338.digitalmars-d puremagic.com...
Hi, Walter, I just had a good idea. (Well, I think it's a good idea!)
Please could you separate out the member functions of
std.stream.Stream into three mixins: ReadMixin, WriteMixin and
SeekMixin?
ReadMixin to include every read function except readBlock()
WriteMixin to include every write function except writeBlock()
SeekMixin to include every seek function.
Then you could implement Stream by mixing in the mixins.
More importantly, I could write
class MyOutputStream : OutputStream
{
this() { writeable = true; }
uint writeBlock(const const void* buffer, uint size) { /*...*/ }
mixin WriteMixin;
}
instead of having to derive from Stream (I don't want an input stream)
or implement writefln()
Nov 10 2007
On 11/10/07, Kris <foo bar.com> wrote:This is a fair example of why Tango does things differently.Thank you, but since I don't use Tango that's not much help. Allow me to bump my request. Walter, please could you separate out the member functions of std.stream.Stream into three mixins: ReadMixin, WriteMixin and SeekMixin, so I can write class MyOutputStream : OutputStream { uint writeBlock(const const void* buffer, uint size) { /*...*/ } mixin WriteMixin; } ? Without that, the interface OutputStream is impractical to implement - I'd have to implement printf() and writef(); I'd have to implement wunget() and opApply(); ... and not make any mistakes! It's too daunting, and a mixin would do the trick nicely. Also - a question. There are functions in the InputStream and OutputStream interfaces whose definitions are implementation dependent. For example, InputStream.readStringW(). The documentation says "The file format is implementation-specific and should not be used except as opposite actions to write". Now - forgive me for being confused here, but shouldn't an interface define what each function must do? Isn't that the whole point of an interface? I mean if I published: interface ConfusingInterface { int f(void[] a); } but then documented that the content of the input array a is implementation-specific, then I have an interface that doesn't mean anything! Moreover, the phrase "except as opposite actions to write" make no sense, because InputStream has no write functions. The only way I can make sense of this instruction is if I assume that every InputStream must have a /corresponding/ OutputStream. But if that's true - why not just do away with the separate interfaces and just have Stream? So, it seems to me that (1) All abstract functions whose purpose is not completely defined should be removed from InputStream and OutputStream, and (2) We need those mixins available. Disclaimer. My goal here is make std.stream better. /Please/ nobody turn this into another Tango-versus-Phobos thread. I'm so bored with those.
Nov 11 2007
Reply to Janice,Also - a question. There are functions in the InputStream and OutputStream interfaces whose definitions are implementation dependent. For example, InputStream.readStringW(). The documentation says "The file format is implementation-specific and should not be used except as opposite actions to write". Now - forgive me for being confused here, but shouldn't an interface define what each function must do? Isn't that the whole point of an interface? I mean if I published:It is defined, but something like this: "Write takes the argument an converts it to bytes that are pushed into a stream, read takes a stream at a location that write started at and read in some bytes and returns the data write was given" what is undefined is only the format of the data.interface ConfusingInterface { int f(void[] a); } but then documented that the content of the input array a is implementation-specific, then I have an interface that doesn't mean anything!If it is defined where void[]a should come from it can be useful. Defining the format requires either looking stuff down to future change or adding all sorts of holes for extension.Moreover, the phrase "except as opposite actions to write" make no sense, because InputStream has no write functions. The only way I can make sense of this instruction is if I assume that every InputStream must have a /corresponding/ OutputStream. But if that's true - why not just do away with the separate interfaces and just have Stream?What would you then do for an object that has no write abilities, or no read abilities? To be a bit silly, what good is a InputStrem without an OutputStream somewhere?
Nov 11 2007
On 11/11/07, BCS <ao pathlink.com> wrote:What would you then do for an object that has no write abilities, or no read abilities? To be a bit silly, what good is a InputStrem without an OutputStream somewhere?There are plenty of examples of those! /dev/random is an input stream with no output stream /dev/null is an output stream with no input stream What I was actually planning to do was implement a hash function (e.g. SHA-256) as an output stream. You write your bytes to the stream, then call a function to get the hash of those bytes. By definition this would /have/ to be an output-only stream. That's the whole POINT of a hash function - it's /one way/! But I don't see why, in order to do that, I should have to implement: abstract void write(char[] s); abstract void write(const(wchar)[] s); Writes a string, together with its length. The format is implementation-specific and should only be used in conjunction with read. Throw WriteException on error. If the format is implementation specific, then does that mean that I, as the creator of HashOutputStream, get to define the implementation? If so, how does a function like wstring f(InputStream is) { wstring s = is.readStringW(); /* do something with s */ return s; } know what to expect? Of course, the silly thing is that it's /easy/ for me to make a Stream (as opposed to an OutputSteam). That's because Stream is a class, not an interface. All I have to do is subclass Stream and override the three abstract functions - job done! And if I make readBlock() through a ReadException, and seek() throw a SeekException, then I do actually /have/ what I want. That, basically, is problem solved. Except ... it seems overkill because I don't need the read half. And it makes me question the purpose of the OutputStream interface. Why have it, if it so damned hard to use?
Nov 11 2007
"Kris" <foo bar.com> "Janice Caron" <caron800 googlemail.com> [snip]What I was actually planning to do was implement a hash function (e.g. SHA-256) as an output stream. You write your bytes to the stream, then call a function to get the hash of those bytes. By definition this would /have/ to be an output-only stream. That's the whole POINT of a hash function - it's /one way/!Indeed. You perhaps don't care about this, Janice, but others may since they won't have to write it themselves: Tango has a broad suite of digest classes, including MD2, MD4, MD5, SHA0, SHA01, SHA1, SHA256, and Tiger (contributed by NG regulars). You can use those independently, or combine with a DigestStream to slurp the digest as it flow by. For example: And similarly for input, if you need that also:But I don't see why, in order to do that, I should have to implement: abstract void write(char[] s); abstract void write(const(wchar)[] s);Yeah, I agree.
Nov 11 2007
Reply to Janice,On 11/11/07, BCS <ao pathlink.com> wrote:No, in your case the write() functions are useless. What you seem to want is to pass a chunk of data. For this you should use the lower level functions. However, you may have a point in that the write functions may cause issues for some cases. Maybe there should be a WriteStreamLL that just has the non implementation specific stuff. and then have WriteStream extend it with the high level stuff. I think it all derives from a design choice of "don't say you will do more than you have to". As an example, if the format was specified, what big-vs.-little endian? Do you pick one and let the other side suffer the overhead? As soon as you start nailing things down, where do you stop? The only logical choices I see are no spec and full spec. At the base level, full spec doesn't work to well. You still can get all the benefits of nailing down the format with, as you pointed out, a mixin. You can even have several (big, endian, little endian, pick-one-at-write-time-endian) I think the point in the end is that if you need to be consistent about how you implement the stuff. If you ave a write only stream that never returns the stuff then impalement it however you want(*). If you have a reader and a writer, make sure they match and expect the user to use them in pairs. * come to think of it. as long as you don't need to match your hash with someone else using different code then the formatting is irrelevant.What would you then do for an object that has no write abilities, or no read abilities? To be a bit silly, what good is a InputStrem without an OutputStream somewhere?There are plenty of examples of those! /dev/random is an input stream with no output stream /dev/null is an output stream with no input stream What I was actually planning to do was implement a hash function (e.g. SHA-256) as an output stream. You write your bytes to the stream, then call a function to get the hash of those bytes. By definition this would /have/ to be an output-only stream. That's the whole POINT of a hash function - it's /one way/! But I don't see why, in order to do that, I should have to implement: abstract void write(char[] s); abstract void write(const(wchar)[] s); Writes a string, together with its length. The format is implementation-specific and should only be used in conjunction with read. Throw WriteException on error. If the format is implementation specific, then does that mean that I, as the creator of HashOutputStream, get to define the implementation?
Nov 11 2007
On 11/11/07, BCS <ao pathlink.com> wrote:What you seem to want is to pass a chunk of data. For this you should use the lower level functions.No, a stream of data. A property of hash functions is that you can pass the data all in one chunk, or you can pass it in lots of little chunks, but either way you'll get the same answer. If you can accept the data a byte at a time, then it should be trivial to turn it into a stream. This is, in fact, the case, with Tango. I'm happy to agree they got that right.However, you may have a point in that the write functions may cause issues for some cases. Maybe there should be a WriteStreamLL that just has the non implementation specific stuff. and then have WriteStream extend it with the high level stuff.I may have been misunderstood. I have no problem with a /class/ doing high level (or low level) stuff. Putting it all into a class makes thing easier for the implementor, because all you have to do is derive from the class. My complaint was that these functions exist in an /interface/. This means that implementers aren't being told exactly what they need to implement, and functions which accept such interfaces won't know what to expect. If it's in an interface, we really need to nail down exactly what the functions are supposed to do.I think it all derives from a design choice of "don't say you will do more than you have to". As an example, if the format was specified, what big-vs.-little endian? Do you pick one and let the other side suffer the overhead? As soon as you start nailing things down, where do you stop? The only logical choices I see are no spec and full spec. At the base level, full spec doesn't work to well.Again, none of this is a problem for classes. The base class could, if desired, implement writeBigEndian() and writeLittleEndian(), and then they'd be available for all subclasses. But the minute you put stuff like that into an interface, with no handy class to derive from, you just make w-a-y too much work (and potential for errors) for the implementor.You still can get all the benefits of nailing down the format with, as you pointed out, a mixin.Yes. That's what I was asking for. Although, even in that case, it would still be advisable to change the docs for those interfaces from "this is implementation specific" to "here is exactly what it must do".come to think of it. as long as you don't need to match your hash with someone else using different code then the formatting is irrelevant.Exactly, yes. But the same argument works in general - not just for hash functions. For /any/ kind of output-only stream, you should not be expected to have hand-code your own printf()!
Nov 11 2007
Reply to Janice,On 11/11/07, BCS <ao pathlink.com> wrote:that is in no way contrdictory with what I sugested. The lowlevel stuff just says "here is a lump of bytes to write" the high level stuff says "here is an array of uints to incode and write"What you seem to want is to pass a chunk of data. For this you should use the lower level functions.No, a stream of data. A property of hash functions is that you can pass the data all in one chunk, or you can pass it in lots of little chunks, but either way you'll get the same answer. If you can accept the data a byte at a time, then it should be trivial to turn it into a stream.IMHO, they are nailed down exactly: The write functions will take the given data and convert it to a bit pattern that can be sent through a stream and then, with a corresponding call to a read, converted back to the original data. This /is/ exact provided that you consider the "data on the wire" to be below the level of abstraction you are working at. And I think that assumption is valid in most cases.However, you may have a point in that the write functions may cause issues for some cases. Maybe there should be a WriteStreamLL that just has the non implementation specific stuff. and then have WriteStream extend it with the high level stuff.I may have been misunderstood. I have no problem with a /class/ doing high level (or low level) stuff. Putting it all into a class makes thing easier for the implementor, because all you have to do is derive from the class. My complaint was that these functions exist in an /interface/. This means that implementers aren't being told exactly what they need to implement, and functions which accept such interfaces won't know what to expect. If it's in an interface, we really need to nail down exactly what the functions are supposed to do.if you have a writeBigEndian() and writeLittleEndian() what do you do when you need to pass something to a function that shouldn't known what enden is being used? void WriteWhateverEndean(OutputStream str) { str. // what here?????? } really what is needs is something like multiple inheritance but without the problems.I think it all derives from a design choice of "don't say you will do more than you have to". As an example, if the format was specified, what big-vs.-little endian? Do you pick one and let the other side suffer the overhead? As soon as you start nailing things down, where do you stop? The only logical choices I see are no spec and full spec. At the base level, full spec doesn't work to well.Again, none of this is a problem for classes. The base class could, if desired, implement writeBigEndian() and writeLittleEndian(), and then they'd be available for all subclasses.But the minute you put stuff like that into an interface, with no handy class to derive from, you just make w-a-y too much work (and potential for errors) for the implementor.interface WithLotsOfStuff {...} template LotsOfStuff() {...} class Use : WithLotsOfStuff { mixin LotsOfStuff!(); } That's not much work. If they want something different than one of the defaults, then they need to do the work anywayOk. Big or little endian? when sending an array should the length be 16, 32, 64 or 128 bits? when sending floats, should they be sent in IEEE format (Note some systems don't use IEEE) What about real, 80bit or 128bit floats? My point is not that this should not be defined, my point is that the INTERFACE is the wrong place to define it. The correct place is the code (in a mixin) that implements it. The questions mentioned above only scratch the surface of the issue. If you nail them down in the interface than immediately someone is out in the cold where what you picked doesn't work. The correct solution IMHO, as glossed over in the code example above, is to define this in the implementing code and then have the code using the interface ignore the differences. This does require that the user be carful to keep things paired up properly but that is no more of an issue with my solution than it is with yours.You still can get all the benefits of nailing down the format with, as you pointed out, a mixin.Yes. That's what I was asking for. Although, even in that case, it would still be advisable to change the docs for those interfaces from "this is implementation specific" to "here is exactly what it must do".you should not be expected to have hand-code your own printf()!No objection there :)
Nov 11 2007
Janice Caron wrote:Hi, Walter, I just had a good idea. (Well, I think it's a good idea!) Please could you separate out the member functions of std.stream.Stream into three mixins: ReadMixin, WriteMixin and SeekMixin?Not much help for Phobos I suppose, but I did this in an example stream implementation about three years ago: http://www.invisibleduck.org/~sean/code/io/stream.d Sean
Nov 11 2007