www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - std.socket tutorials? examples?

reply Chris Katko <ckatko gmail.com> writes:
I've been Google'ing and there's like... nothing out there.

One of the top results for "std.socket dlang examples"... is for 
TANGO. That's how old it is.
Oct 04 2018
next sibling parent reply Andrea Fontana <nospam example.com> writes:
On Thursday, 4 October 2018 at 08:32:13 UTC, Chris Katko wrote:

 I've been Google'ing and there's like... nothing out there.

 One of the top results for "std.socket dlang examples"... is 
 for TANGO. That's how old it is.


Socket paradigm is quite standard across languages.

Anyway you can find a couple of example here:
https://github.com/dlang/dmd/blob/master/samples/listener.d
https://github.com/dlang/dmd/blob/master/samples/htmlget.d

Andrea
Oct 04 2018
parent reply Chris Katko <ckatko gmail.com> writes:
On Thursday, 4 October 2018 at 08:52:28 UTC, Andrea Fontana wrote:

 On Thursday, 4 October 2018 at 08:32:13 UTC, Chris Katko wrote:

 I've been Google'ing and there's like... nothing out there.

 One of the top results for "std.socket dlang examples"... is 
 for TANGO. That's how old it is.


 Socket paradigm is quite standard across languages.

 Anyway you can find a couple of example here:
 https://github.com/dlang/dmd/blob/master/samples/listener.d
 https://github.com/dlang/dmd/blob/master/samples/htmlget.d

 Andrea


I was hoping someone would have a walk-through or something. 
Those examples are 110 lines each!

Usually, with D, there's plenty of useful 

socket example over, how am I supposed to know if I'm doing it 
"the right/proper/best way" in D? That kind of thing.
Oct 04 2018
parent reply bauss <jj_1337 live.dk> writes:
On Thursday, 4 October 2018 at 09:54:40 UTC, Chris Katko wrote:

 On Thursday, 4 October 2018 at 08:52:28 UTC, Andrea Fontana 
 wrote:

 On Thursday, 4 October 2018 at 08:32:13 UTC, Chris Katko wrote:

 I've been Google'ing and there's like... nothing out there.

 One of the top results for "std.socket dlang examples"... is 
 for TANGO. That's how old it is.


 Socket paradigm is quite standard across languages.

 Anyway you can find a couple of example here:
 https://github.com/dlang/dmd/blob/master/samples/listener.d
 https://github.com/dlang/dmd/blob/master/samples/htmlget.d

 Andrea


 I was hoping someone would have a walk-through or something. 
 Those examples are 110 lines each!

 Usually, with D, there's plenty of useful 

 socket example over, how am I supposed to know if I'm doing it 
 "the right/proper/best way" in D? That kind of thing.


Not exactly a tutorial or a walkthrough, but I'll try to explain 
basic usage of std.socket using an asynchronous tcp server as 
example.

Usually sockets are pretty low-level though, so not a lot of D 
magic will be present in the example.

First of all D implements classes for the two most common 
protocols UDP and TCP, both called UdpSocket and TcpSocket for 
obvious reasons.

We'll focus on TcpSocket in these examples, but using UdpSocket 
is not much different if you at the very least understand UDP and 
how it works.

The first thing you want to do for the server is creating a new 
instance of the TcpSocket class.

```
auto server = new TcpSocket;
```

Then to make it a non-blocking socket you simply set "blocking" 
to false.

This will call the low-level OS functions that creates 
non-blocking sockets on Windows it would be: ioctlsocket().

https://docs.microsoft.com/en-us/windows/desktop/api/winsock/nf-winsock-ioctlsocket

For Posix it would be fcntl() wih first "F_GETFL" and then 
"F_SETFL" where the flags are updated with "O_NONBLOCK"

http://man7.org/linux/man-pages/man2/fcntl.2.html

```
server.blocking = false;
```

The next thing we do is binding the socket.

In D you can use the "InternetAddress" class to construct a valid 
address for the socket.


```
server.bind(new InternetAddress(ip, port));
```

"ip" and "port" can be changed to the ip and port of your server.

Ex:

```
const ip = "127.0.0.1";
const port = 9988;
```

Once the socket has been bound then we can simply listen for 
connections.

It can be done with the listen function.

```
server.listen(100); // The backlog is set to 100, can be whatever 
you prefer.
```

As you can see so far there has been no real D magic, because 
it's pretty low-level.

Now to actual listen for the connections etc. it'll be a little 
more complex, because we're going to use the "SocketSet" class.

The major difference between something like implementing sockets 
in ex. C++ and D is that in D you don't have to implement your 
socket logic per platform, because phobos already creates that 
logic for you. Of course that's not taking something like boost 
into account which gives same usability in C++.

The next thing is to actually accept the sockets and since we're 
using non-blocking sockets then we don't really need to make a 
separate thread for accepting them, neither do the sockets 
actually need a thread for themselves.

First we need some collection that can hold all current sockets 
accepted (An array will be fine for now.)

```
Socket[] clients;
```

Then we'll create two instances of the "SocketSet" class.

That's because we need a socket set for the server socket and one 
for all the connected sockets.

```
auto serverSet = new SocketSet;
auto clientSet = new SocketSet;
```

A simple infinite while loop will be okay.

```
while (true)
{
     ...
}
```

Now let's dive into our loop, because this is where the "magic" 
of the socket handling actually happens.

At the beginning of the loop we'll want to reset the socket sets.

This can be done using the "reset" function.

This would most definitely be handled differently in a more 
performance critical server, but you'd also use multiple threads 
that each holds a set of sockets etc. we'll not do such things 
for the sake of simplicity.

```
serverSet.reset();
clientSet.reset();
```
First we want to add the server socket to "serverSet".

```
serverSet.add(server);
```

Next we'll loop through our client array and add each socket to 
the client set.

```
if (clients)
{
   foreach (client; clients)
   {
     clientSet.add(client);
   }
}
```

At first we want to check if there are any new sockets connected 
that we can accept.

By calling "Socket.select()" we can get different socket states 
based on a socket set.

```
auto serverResult = Socket.select(serverSet, null, null);
```

If the result from "Socket.select()" is below 1 then there are no 
new sockets to accept.

If the result is 0 then it timed out, if it's -1 then it was 
interrupted.

We want to check for that before we can call "accept()"

```
if (serverResult > 0)
{
   auto client = server.accept();

   ...
}
```

Once we have called "accept()" and received a socket then we can 
add that to our client array.


```
if (client)
{
   clients ~= client;
}
```

The next thing is to simply read data from the sockets.

Again we need to check the set's result using "Socket.select()" 
to make sure there actually are sockets that have data we can 
read from.

```
auto clientResult = Socket.select(clientSet, null, null);

if (clientSet < 1)
{
   continue;
}

```

If the result is above 0 then we have sockets that can be read 
from.

The next thing is simply to read the data they have available.

Now we want to loop through each client and check if they're in 
the set.

They will be removed from the set if they don't have data.

```
foreach (client; clients)
{
   if (!clientSet.isSet(client))
   {
     continue;
   }

   ...
}
```

You simply call the "receive()" function with a buffer and it'll 
fill the buffer with the data currently available.

If the function returns 0 then it has disconnected.

In that case you can remove it from the client array. Normally an 
associative array would be more suitable, because we can give 
each socket an identifier.

Else you can check if the function returned "Socket.ERROR" which 
generally means disconnect in one way or another, but it could be 
more specific.

If the value is above 0 then it'll be the amount of bytes that 
were available in the socket.

```
auto buffer = new ubyte[1024];

auto received = client.receive(buffer);

if (recv == 0 || recv == Socket.ERROR)
{
   continue; // Most likely disconnected ...
}

buffer = buffer[0 .. $]; // Slice the buffer to the actual size 
of the received data.
```

Important: The data received from "receive()" may not be the 
whole buffer when using non-blocking sockets; make sure that you 
have all the data before you start processing it and/or slices it.

Now the next thing to do is simply using the buffer with the data.

Disclaimer: This is not a 100% ideal way of writing socket 
applications, but it should be sufficient enough to get by for a 
start and to at least understand sockets in general.
Oct 04 2018
next sibling parent reply bauss <jj_1337 live.dk> writes:
On Thursday, 4 October 2018 at 13:07:30 UTC, bauss wrote:

 buffer = buffer[0 .. $]; // Slice the buffer to the actual size 
 of the received data.
 ```


Typo...

Was supposed to be "received" and not "$"...


 buffer = buffer[0 .. received]; // Slice the buffer to the 
 actual size
Oct 04 2018
parent bauss <jj_1337 live.dk> writes:
On Thursday, 4 October 2018 at 13:09:17 UTC, bauss wrote:

 On Thursday, 4 October 2018 at 13:07:30 UTC, bauss wrote:

 buffer = buffer[0 .. $]; // Slice the buffer to the actual 
 size of the received data.
 ```


 Typo...

 Was supposed to be "received" and not "$"...


 buffer = buffer[0 .. received]; // Slice the buffer to the 
 actual size




Another typo is "recv" should be "received".

That's what happens when you copy-paste from old code.
Oct 04 2018
prev sibling parent reply Jan Allersma <janallersma gmail.com> writes:
On Thursday, 4 October 2018 at 13:07:30 UTC, bauss wrote:

 Not exactly a tutorial or a walkthrough, but I'll try to 
 explain basic usage of std.socket using an asynchronous tcp 
 server as example.


So I tried to code the example as described. After the program is 
at

```d
auto clientResult = Socket.select(clientSet, null, null);
```

The program gets stuck and doesn't do anything. I use `telnet` in 
Bash as client to connect to the server. What am I doing wrong?

For completeness, this is the full code made from the example:

```d
import std.stdio, std.socket;

     void startServer() {
         auto server = new TcpSocket;
         server.blocking = false;
         server.bind(new InternetAddress("localhost", 1234));
         server.listen(100);

         Socket[] clients;
         auto serverSet = new SocketSet;
         auto clientSet = new SocketSet;

         while (true) {
             serverSet.reset();
             clientSet.reset();
             serverSet.add(server);

             if (clients) {
                 foreach (client; clients) {
                     clientSet.add(client);
                 }
             }


             auto serverResult = Socket.select(serverSet, null, 
null);
             if (serverResult > 0) {
                 auto client = server.accept();
                 if (client) {
                     clients ~= client;
                 }

                 writeln("Gets stuck");
                 auto clientResult = Socket.select(clientSet, 
null, null);
                 writeln("Nevermind...");

     // Keep it commented out as it gives compilation errors
     /*if (clientSet < 1)
     {
       continue;
     }*/


                 foreach (_client; clients) {
                     if (!clientSet.isSet(_client)) {
                         continue;
                     }

                     auto buffer = new ubyte[1024];

                     auto received = client.receive(buffer);

                     if (received == 0 || received == 
Socket.ERROR) {
                         continue; // Most likely disconnected ...
                     }

                     buffer = buffer[0 .. received]; // Slice the 
buffer to the actual size of the received data.
                 }
             }
         }
     }
```
Apr 29 2023
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Saturday, 29 April 2023 at 10:56:46 UTC, Jan Allersma wrote:

 auto clientResult = Socket.select(clientSet, null, null);



There's probably nothing in clientSet, so it is waiting for 
nothing.... you almost always want to have just one call to 
select in the program, not two, the whole point is to combine 
checks.

I wrote a thing you might want to read too:

http://dpldocs.info/this-week-in-d/Blog.Posted_2019_11_11.html#sockets-tutorial
Apr 29 2023
parent reply Cecil Ward <cecil cecilward.com> writes:
On Saturday, 29 April 2023 at 11:26:20 UTC, Adam D Ruppe wrote:

 On Saturday, 29 April 2023 at 10:56:46 UTC, Jan Allersma wrote:

 auto clientResult = Socket.select(clientSet, null, null);



 There's probably nothing in clientSet, so it is waiting for 
 nothing.... you almost always want to have just one call to 
 select in the program, not two, the whole point is to combine 
 checks.

 I wrote a thing you might want to read too:

 http://dpldocs.info/this-week-in-d/Blog.Posted_2019_11_11.html#sockets-tutorial


How do we wait for an ‘or’ of multiple asynchronous events in 
this kind of code? In WinNT iirc there is a very nice o/s 
function that can wait on various kinds of asynch i/o, waiting on 
operations of different types if I’ve understood it correctly. 
The kind of thing I might want to do is wait on an or of IP 
packets arriving, send-completion of IP packets, timers 
completing, IPC messages coming in, key presses or event mouse 
events and the order in which these events arrive is of course 
not predictable. I might want a key press event to break out of 
something or to control an application. Everyone wants timer 
events for timeouts.
Apr 30 2023
parent reply Adam D Ruppe <destructionator gmail.com> writes:
On Sunday, 30 April 2023 at 22:10:31 UTC, Cecil Ward wrote:

 How do we wait for an ‘or’ of multiple asynchronous events in 
 this kind of code?


You can set a timeout value for Socket.select, but Phobos isn't 
going to help you with anything other than sockets and timeouts 
(despite the fact the underlying operating systems can, in fact, 
do it).

There's a few other libs that can help with this, including one 
I'm aiming to release some time in May, or vibe.d has its own 
ways of doing it, among others. You can also import 
core.sys.stuff and call the OS functions without a middle man.

But the D stdlib is quite underpowered when it comes to these 
things. Socket.select is ok but just the basics.
Apr 30 2023
parent Cecil Ward <cecil cecilward.com> writes:
On Sunday, 30 April 2023 at 22:37:48 UTC, Adam D Ruppe wrote:

 On Sunday, 30 April 2023 at 22:10:31 UTC, Cecil Ward wrote:

 How do we wait for an ‘or’ of multiple asynchronous events in 
 this kind of code?


 You can set a timeout value for Socket.select, but Phobos isn't 
 going to help you with anything other than sockets and timeouts 
 (despite the fact the underlying operating systems can, in 
 fact, do it).

 There's a few other libs that can help with this, including one 
 I'm aiming to release some time in May, or vibe.d has its own 
 ways of doing it, among others. You can also import 
 core.sys.stuff and call the OS functions without a middle man.

 But the D stdlib is quite underpowered when it comes to these 
 things. Socket.select is ok but just the basics.


Many thanks, Adam.
Apr 30 2023
prev sibling parent Adam D. Ruppe <destructionator gmail.com> writes:
On Thursday, 4 October 2018 at 08:32:13 UTC, Chris Katko wrote:

 I've been Google'ing and there's like... nothing out there.


My book has a few examples
https://www.packtpub.com/application-development/d-cookbook


of course, buying it for just std.socket (which is just like one 
page out of the 300) is a bit silly, but the code examples are 
also here

http://arsdnet.net/dcode/book/chapter_02/03/


The code examples have little to no explanation (that's what the 
book is for lol) but std.socket is pretty simple so you can 
probably figure it out - like the others said, it is a very, very 
thin wrapper over the same basic BSD socket library every other 
language uses.

The client example is like 15 lines too so you can use it easily, 
and the server is only like 30 so that isn't too bad either.

The library docs http://dpldocs.info/std.socket can be used to 
look up the specific functions used.
Oct 04 2018