digitalmars.D.learn - std.socket classes
- Jonathan Marler (20/20) Apr 09 2017 Does anyone know why Socket and Address in phobos were created as
- rikki cattermole (2/2) Apr 09 2017 Don't think too hard, times have changed since std.socket was written.
- Jonathan Marler (9/13) Apr 09 2017 What an odd response... You don't think I should ask questions
- rikki cattermole (16/26) Apr 09 2017 Oh sorry, I had a brain derp and thought at the end there you had that
- Jonathan Marler (13/51) Apr 09 2017 Ah ok. That response was surprising to me coming from you (based
- rikki cattermole (13/56) Apr 09 2017 I personally disagree with the designs used in libasync[0] and
- Adam D. Ruppe (7/9) Apr 09 2017 It is probably just the historical evolution, but I find it
- Jonathan Marler (54/63) Apr 10 2017 An interesting benefit. However, I don't think this is the ideal
- Adam D. Ruppe (16/24) Apr 10 2017 If I was doing it myself, I'd probably do an interface / final
- Jonathan Marler (11/35) Apr 10 2017 Ah, it seems someone already ran into this accept problem, hence
- David Nadlinger (9/13) Apr 10 2017 Just to definitively answer the question: The design has been
Does anyone know why Socket and Address in phobos were created as classes instead of structs? My guess is that making Socket a class prevents socket handle leaks because you can clean up the handle in the destructor when the memory gets freed if no one closes it. Is this the reason it is a class and are there any other reasons? As for Address, I can't think of a reason why this one is a class. It doesn't have to free any underlying OS resources, it's just a chunk of memory that can be passed to and from the socket API. Using sockaddr in C/C++ is more flexible because it allows the application to decide where the memory lives (which will almost always be on the stack). It feels like whoever made Address a class probably wasn't familiar with sockaddr. Is that the case or are there reasons why it was made a class? If I was implementing sockaddr in D, I would have chosen to use addressFamily as a sort of "makeshift Vptr", which is really how it is used in C (even though C doesn't support classes). Using this technique, I believe you could expose pretty much the same API without the overhead of wrapping it inside a D object. Does anyone know if this solution was considered?
Apr 09 2017
Don't think too hard, times have changed since std.socket was written. It certainly isn't designed for high performance hence e.g. libasync.
Apr 09 2017
On Sunday, 9 April 2017 at 14:49:14 UTC, rikki cattermole wrote:Don't think too hard, times have changed since std.socket was written. It certainly isn't designed for high performance hence e.g. libasync.What an odd response... You don't think I should ask questions about why decisions were made? If I took that approach how would I learn? And if you discourage other people from asking questions by telling them they are "thinking too hard" what kind of effect does that have on the community? As for "high performance", my questions have less to do with performance than they do with an API that makes sense and doesn't feel "kludgy".
Apr 09 2017
On 09/04/2017 3:56 PM, Jonathan Marler wrote:On Sunday, 9 April 2017 at 14:49:14 UTC, rikki cattermole wrote:Oh sorry, I had a brain derp and thought at the end there you had that you thought about it and it didn't make sense. Hence the slightly weirder response. What I meant is that, for common use cases it works well enough and it does use reasonably sound API even if a bit cludgy. When asking about classes, one of the big things is the vtable. They are slow (compared to final classes and structs). This is the main reason people want to switch over to structs instead of classes. However if you take a look at the more performance aware libraries like libasync you will see classes used extensively still. Here is my suggestion, its a little harder to toy with ideas without real code to show for it. All our more existing API's are mostly class based for high performance sockets, timers ext. So, would you like to have a go and explore this area since we are playing it a bit too safe for your liking?Don't think too hard, times have changed since std.socket was written. It certainly isn't designed for high performance hence e.g. libasync.What an odd response... You don't think I should ask questions about why decisions were made? If I took that approach how would I learn? And if you discourage other people from asking questions by telling them they are "thinking too hard" what kind of effect does that have on the community? As for "high performance", my questions have less to do with performance than they do with an API that makes sense and doesn't feel "kludgy".
Apr 09 2017
On Sunday, 9 April 2017 at 15:04:29 UTC, rikki cattermole wrote:On 09/04/2017 3:56 PM, Jonathan Marler wrote:Ah ok. That response was surprising to me coming from you (based on what I've read from you in the past) but I see it was a misunderstanding.On Sunday, 9 April 2017 at 14:49:14 UTC, rikki cattermole wrote:Oh sorry, I had a brain derp and thought at the end there you had that you thought about it and it didn't make sense. Hence the slightly weirder response.Don't think too hard, times have changed since std.socket was written. It certainly isn't designed for high performance hence e.g. libasync.What an odd response... You don't think I should ask questions about why decisions were made? If I took that approach how would I learn? And if you discourage other people from asking questions by telling them they are "thinking too hard" what kind of effect does that have on the community? As for "high performance", my questions have less to do with performance than they do with an API that makes sense and doesn't feel "kludgy".What I meant is that, for common use cases it works well enough and it does use reasonably sound API even if a bit cludgy. When asking about classes, one of the big things is the vtable. They are slow (compared to final classes and structs). This is the main reason people want to switch over to structs instead of classes. However if you take a look at the more performance aware libraries like libasync you will see classes used extensively still. Here is my suggestion, its a little harder to toy with ideas without real code to show for it. All our more existing API's are mostly class based for high performance sockets, timers ext. So, would you like to have a go and explore this area since we are playing it a bit too safe for your liking?What I've found myself having to do is use the lower level platform specific APIs that use socket_t and sockaddr, but then I get platform dependencies and can't access alot of the library because it requires the higher level Socket and Address objects. I would be willing to explore this area, but before I do work in an area I research what's already been done. Hence why I'm asking the questions about why it was done this way in the first place. For all I know there are very good reasons it was done this way that I just don't know about.
Apr 09 2017
On 09/04/2017 4:19 PM, Jonathan Marler wrote:On Sunday, 9 April 2017 at 15:04:29 UTC, rikki cattermole wrote:I personally disagree with the designs used in libasync[0] and vibe-core(vibe.d's replacement/new core implementation). Very class heavy and not enough emphasis on the event loop. But as you can see with my lib SPEW[2] even I don't get away from classes (in fact I go all in). Keep in mind only a handful of people really try to toy with this area. So you may very well come up with a completely different strategy that'll blow our minds without too much work. Existing implementations all have there trade offs, you simply may choose a completely different set and that is exciting. [0] https://github.com/etcimon/libasync [1] https://github.com/vibe-d/vibe-core [2] https://github.com/Devisualization/spewOn 09/04/2017 3:56 PM, Jonathan Marler wrote:Ah ok. That response was surprising to me coming from you (based on what I've read from you in the past) but I see it was a misunderstanding.On Sunday, 9 April 2017 at 14:49:14 UTC, rikki cattermole wrote:Oh sorry, I had a brain derp and thought at the end there you had that you thought about it and it didn't make sense. Hence the slightly weirder response.Don't think too hard, times have changed since std.socket was written. It certainly isn't designed for high performance hence e.g. libasync.What an odd response... You don't think I should ask questions about why decisions were made? If I took that approach how would I learn? And if you discourage other people from asking questions by telling them they are "thinking too hard" what kind of effect does that have on the community? As for "high performance", my questions have less to do with performance than they do with an API that makes sense and doesn't feel "kludgy".What I meant is that, for common use cases it works well enough and it does use reasonably sound API even if a bit cludgy. When asking about classes, one of the big things is the vtable. They are slow (compared to final classes and structs). This is the main reason people want to switch over to structs instead of classes. However if you take a look at the more performance aware libraries like libasync you will see classes used extensively still. Here is my suggestion, its a little harder to toy with ideas without real code to show for it. All our more existing API's are mostly class based for high performance sockets, timers ext. So, would you like to have a go and explore this area since we are playing it a bit too safe for your liking?What I've found myself having to do is use the lower level platform specific APIs that use socket_t and sockaddr, but then I get platform dependencies and can't access alot of the library because it requires the higher level Socket and Address objects. I would be willing to explore this area, but before I do work in an area I research what's already been done. Hence why I'm asking the questions about why it was done this way in the first place. For all I know there are very good reasons it was done this way that I just don't know about.
Apr 09 2017
On Sunday, 9 April 2017 at 14:47:39 UTC, Jonathan Marler wrote:Does anyone know why Socket and Address in phobos were created as classes instead of structs?It is probably just the historical evolution, but I find it pretty handy: I did a subclass of Socket to do SSL, then as is nice with classes, I can pass that to other code using the Socket interface and have it work. So I'd be sad if they changed it now too much, the class is legitimately useful here and actually not very expensive`.
Apr 09 2017
On Monday, 10 April 2017 at 04:32:20 UTC, Adam D. Ruppe wrote:On Sunday, 9 April 2017 at 14:47:39 UTC, Jonathan Marler wrote:An interesting benefit. However, I don't think this is the ideal way to support such a use case. I think it would have been better if there was a shared stream/socket-like interface that you could override to use raw sockets or SSL. I'll explain why. My first thought is that since the interface you are using (the Socket class) wasn't really designed to be overridden, you probably had to do some interesting hacks to make it work. For example, when you accept a new socket, you probably had to delete the Socket object you got and create a new SSLSocket object passing the handle from one to the other, and make sure that the original Socket object didn't close it. I'm guessing that to prevent this close you probably set the socket handle on the accepted Socket object to null/invalid? Or maybe you just called the raw accept function and created a new object. But if the library is the one calling accept, then you would obviously have to override the accept function and do something that I would call "hacky". My other thought is that by separating both the virtual interface and the raw socket functions, you have provided both a low-level and high-level API that each application can choose to use. The tradeoff being "control" vs "extensibility". The high-level being more extensible (can be overriden to support things like SSL), and the low-level being less abstracted and therefore provides more control or access to the underlying implementation. This low-level access is more useful for code that needs to use socket-specific features. I will say that one disadvantage with this approach is that by separating both the virtual interface and the direct socket interface, you open up the door for library writers to make the mistake of using the wrong level of the API. If a library used the lower-level API and you wanted to override it with say, an SSL implementation, then you are out of luck unless you update the library to use the higher level interface. Of course this is more of a "practical real world" disadvantage that "in theory" can be prevented with good libraries. ----------------------- DISCLAIMER ----------------------- I would like to say something to anyone who wants to contribute to this thread. These comments are meant to discuss the pros/cons of the std.socket design and DO NOT serve as justification for changing phobos. Such a change would require much more discussion. The problem I've seen is that people will immediately halt a conversation by jumping to the end and arguing that such ideas will never be implemented because the benefit to risk ratio is too low. Benefit to risk ratio is very good and necessary discussion to have, however it's not good to stop a conversation early by jumping to this stage before people have even had a chance to discuss the merits of design and ideas on their own. Any discussion on the ideas/design with your thoughts/feedback/experience are welcome. If you want to discuss whether ideas/changes should be carried out, I would hold off those comments since it derails good discussion. Thanks.Does anyone know why Socket and Address in phobos were created as classes instead of structs?It is probably just the historical evolution, but I find it pretty handy: I did a subclass of Socket to do SSL, then as is nice with classes, I can pass that to other code using the Socket interface and have it work. So I'd be sad if they changed it now too much, the class is legitimately useful here and actually not very expensive`.
Apr 10 2017
On Monday, 10 April 2017 at 16:18:20 UTC, Jonathan Marler wrote:An interesting benefit. However, I don't think this is the ideal way to support such a use case.If I was doing it myself, I'd probably do an interface / final class split too (which also opens up a wee bit of additional easy optimization), but the Socket class isn't *bad* for this.My first thought is that since the interface you are using (the Socket class) wasn't really designed to be overridden, you probably had to do some interesting hacks to make it work.No, the code is very straight-forward, regular class method overrides with appropriate forwards to the base class implementation where needed.For example, when you accept a new socket, you probably had to delete the Socket object you got and create a new SSLSocket object passing the handle from one to the other, and make sureI didn't implement the server, but if I did, it would be another simple case of override SslSocket accept() { return new SslSocket(....); } or better yet, `override SslSocket accepting() { ...}`, since there IS a method specifically designed for this: http://dpldocs.info/experimental-docs/std.socket.Socket.accepting.html That'd work fine too.
Apr 10 2017
On Monday, 10 April 2017 at 18:57:13 UTC, Adam D. Ruppe wrote:On Monday, 10 April 2017 at 16:18:20 UTC, Jonathan Marler wrote:Yeah I agree, not perfect but not *bad*.An interesting benefit. However, I don't think this is the ideal way to support such a use case.If I was doing it myself, I'd probably do an interface / final class split too (which also opens up a wee bit of additional easy optimization), but the Socket class isn't *bad* for this.Ah, it seems someone already ran into this accept problem, hence why the new "accepting" function was added. Funny timing that this was added near the time I asked the question. Since my last post I've been looking through std.socket and I see quite a bit of inefficiency especially when it comes to GC memory. I think moving forward the best solution for me is to use my version of std.socket. It would also be great if Walter's new ref-counted exceptions proposal gets implemented soon because then I could make it nogc. Anyway, thanks for the feedback.My first thought is that since the interface you are using (the Socket class) wasn't really designed to be overridden, you probably had to do some interesting hacks to make it work.No, the code is very straight-forward, regular class method overrides with appropriate forwards to the base class implementation where needed.For example, when you accept a new socket, you probably had to delete the Socket object you got and create a new SSLSocket object passing the handle from one to the other, and make sureI didn't implement the server, but if I did, it would be another simple case of override SslSocket accept() { return new SslSocket(....); } or better yet, `override SslSocket accepting() { ...}`, since there IS a method specifically designed for this: http://dpldocs.info/experimental-docs/std.socket.Socket.accepting.html That'd work fine too.
Apr 10 2017
On Sunday, 9 April 2017 at 14:47:39 UTC, Jonathan Marler wrote:My guess is that making Socket a class prevents socket handle leaks because you can clean up the handle in the destructor when the memory gets freed if no one closes it. Is this the reason it is a class and are there any other reasons?Just to definitively answer the question: The design has been done way before even D 1.0 – actually, 0.86, I just checked the Git history – and has stayed more or less unchanged since. Several people, including me, have patched up various issues since, but the clunky design stayed unchanged. Including it in D2 proper was of course a mistake – it certainly wouldn't even come close to making it into Phobos today. — David
Apr 10 2017