www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Strange socket error

reply Bob Cowdery <bob bobcowdery.plus.com> writes:
Hi all,

This is a long shot but I'm out of ideas. I ported an app from Windows
to Linux and after many issues it is working but I'm left with a strange
problem. The app basically reads data streams from a USB device,
processes them and outputs real-time graphical data to a browser. There
is also some control input from the browser. The interface to the
browser is web sockets for which I have written a D web-socket server.
Depending on how much of my application I allow to run I get a stream of
these errors.

Error accepting: std.socket.SocketAcceptException: Unable to accept
socket connection: Resource temporarily unavailable
----------------
./DcSdr() [0x80aa2ed]
./DcSdr() [0x806f52b]
./DcSdr() [0x804f752]
./DcSdr() [0x809b422]
./DcSdr() [0x80ae77e]
/lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
/lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]

This is the web-socket accept. I seem to fall straight through the
accept call even when I am not making any connection attempt. When I do
connect the stream stops and I get one of these outputs each time I send
data from the browser. I should not even be near the accept at that
point as the connection is made.

The app appears to be unaffected and works as expected. The odd think is
if I shut down part of the app these errors stop. I can't tie it down to
any specific thing but I suspect threading as the number of threads is
reduced by stopping parts of the app. The error also seems to indicate
threads are involved. I did not get this problem on Windows.

Any ideas would be much appreciated.

Thanks
bob
Dec 21 2010
next sibling parent reply =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Bob Cowdery wrote:

 Hi all,
=20
 This is a long shot but I'm out of ideas. I ported an app from Windows
 to Linux and after many issues it is working but I'm left with a strang=


e

 problem. The app basically reads data streams from a USB device,
 processes them and outputs real-time graphical data to a browser. There=



 is also some control input from the browser. The interface to the
 browser is web sockets for which I have written a D web-socket server.
 Depending on how much of my application I allow to run I get a stream o=


f

 these errors.
=20
 Error accepting: std.socket.SocketAcceptException: Unable to accept
 socket connection: Resource temporarily unavailable
 ----------------
 ./DcSdr() [0x80aa2ed]
 ./DcSdr() [0x806f52b]
 ./DcSdr() [0x804f752]
 ./DcSdr() [0x809b422]
 ./DcSdr() [0x80ae77e]
 /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
 /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]
=20
 This is the web-socket accept. I seem to fall straight through the
 accept call even when I am not making any connection attempt. When I do=



 connect the stream stops and I get one of these outputs each time I sen=


d

 data from the browser. I should not even be near the accept at that
 point as the connection is made.
=20
 The app appears to be unaffected and works as expected. The odd think i=


s

 if I shut down part of the app these errors stop. I can't tie it down t=


o

 any specific thing but I suspect threading as the number of threads is
 reduced by stopping parts of the app. The error also seems to indicate
 threads are involved. I did not get this problem on Windows.
=20
 Any ideas would be much appreciated.
=20


	Do you have several threads trying to accept connections on the
same socket simultaneously? If that is the case, the first thread
would block on the socket until your client connects and the other
threads will fail in the call to "accept".

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Dec 21 2010
parent reply Bob Cowdery <bob bobcowdery.plus.com> writes:
On 21/12/2010 22:22, "Jérôme M. Berger" wrote:

 Bob Cowdery wrote:

 Hi all,

 This is a long shot but I'm out of ideas. I ported an app from Windows
 to Linux and after many issues it is working but I'm left with a strange
 problem. The app basically reads data streams from a USB device,
 processes them and outputs real-time graphical data to a browser. There
 is also some control input from the browser. The interface to the
 browser is web sockets for which I have written a D web-socket server.
 Depending on how much of my application I allow to run I get a stream of
 these errors.

 Error accepting: std.socket.SocketAcceptException: Unable to accept
 socket connection: Resource temporarily unavailable
 ----------------
 ./DcSdr() [0x80aa2ed]
 ./DcSdr() [0x806f52b]
 ./DcSdr() [0x804f752]
 ./DcSdr() [0x809b422]
 ./DcSdr() [0x80ae77e]
 /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
 /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]

 This is the web-socket accept. I seem to fall straight through the
 accept call even when I am not making any connection attempt. When I do
 connect the stream stops and I get one of these outputs each time I send
 data from the browser. I should not even be near the accept at that
 point as the connection is made.

 The app appears to be unaffected and works as expected. The odd think is
 if I shut down part of the app these errors stop. I can't tie it down to
 any specific thing but I suspect threading as the number of threads is
 reduced by stopping parts of the app. The error also seems to indicate
 threads are involved. I did not get this problem on Windows.

 Any ideas would be much appreciated.


 	Do you have several threads trying to accept connections on the
 same socket simultaneously? If that is the case, the first thread
 would block on the socket until your client connects and the other
 threads will fail in the call to "accept".

 		Jerome


The listner is a single thread that creates an instance of my web socket
class for each connection. I'm not trying to support lots of users, most
of the time just one. The listner code is almost the same as the sockets
example. If you or anyone else can spare a few moments the code can be
browsed at http://code.google.com/p/dcsdr/source/browse
(branches/linux/DcSdr/WsServer/listener.d). The trunk is the Windows
version. I'd be very happy if it's something I'm doing wrong.

bob
Dec 22 2010
parent reply =?UTF-8?B?IkrDqXLDtG1lIE0uIEJlcmdlciI=?= <jeberger free.fr> writes:
Bob Cowdery wrote:

 The listner is a single thread that creates an instance of my web socke=


t

 class for each connection. I'm not trying to support lots of users, mos=


t

 of the time just one. The listner code is almost the same as the socket=


s

 example. If you or anyone else can spare a few moments the code can be
 browsed at http://code.google.com/p/dcsdr/source/browse
 (branches/linux/DcSdr/WsServer/listener.d). The trunk is the Windows
 version. I'd be very happy if it's something I'm doing wrong.
=20


	I looked at your code and I don't see what could cause this issue.
Sorry.

	I did however find a bug: when a client connection is closed, you
remove the corresponding socket from reads but you don't remove the
WebSocket from wsset...

		Jerome
--=20
mailto:jeberger free.fr
http://jeberger.free.fr
Jabber: jeberger jabber.fr
Dec 22 2010
parent Bob Cowdery <bob bobcowdery.plus.com> writes:
On 22/12/2010 10:04, "Jérôme M. Berger" wrote:

 Bob Cowdery wrote:

 The listner is a single thread that creates an instance of my web socket
 class for each connection. I'm not trying to support lots of users, most
 of the time just one. The listner code is almost the same as the sockets
 example. If you or anyone else can spare a few moments the code can be
 browsed at http://code.google.com/p/dcsdr/source/browse
 (branches/linux/DcSdr/WsServer/listener.d). The trunk is the Windows
 version. I'd be very happy if it's something I'm doing wrong.


 	I looked at your code and I don't see what could cause this issue.
 Sorry.

 	I did however find a bug: when a client connection is closed, you
 remove the corresponding socket from reads but you don't remove the
 WebSocket from wsset...

 		Jerome


Thanks for looking and pointing out the bug. That could be causing some
odd connection issues I see when I have to restart things. I didn't
expect it to be a simple coding fault as it goes away when I stop
unrelated threads. I guess my next step is to try and reproduce with the
minimum of code.

bob
Dec 22 2010
prev sibling parent reply Heywood Floyd <soul8o8 gmail.com> writes:
Hi Bob!


My guess: You're listener is set to be non-blocking. That means that when you
call listener.accept() it will return immediately with an
SocketAcceptException, if there's no connection. And you're basically calling
listener.accept() over and over again in an infinite loop.

The following example shows it:

import std.concurrency, std.stdio, std.conv, std.socket;
void main()
{
	ushort port = 9999;
	Socket listener = new TcpSocket;
	assert(listener.isAlive);
	listener.blocking = false;
	listener.bind(new InternetAddress(port));
	listener.listen(10);
	writeln("Listening on port: ", port);
	Socket sn;
	while(true){
		try
		{
			writeln ("Accepting");
			sn = listener.accept();
			writeln("Connection from ", sn.remoteAddress().toString(), " established" );
			assert(sn.isAlive);
			assert(listener.isAlive);
			break;
		}
		catch(Exception e)
		{
			writeln("Error accepting: ", e.toString() );
			if(sn)
			sn.close();
		}
	}
	writeln("end");
}

Running the example will print
    Error accepting: std.socket.SocketAcceptException: Unable to accept socket
connection: Resource temporarily unavailable
for ever. (Tested on Mac.) When a user connects to 127.0.0.1:9999 (by for
instance opening the URL in a browser) socket.accept _does_ return a connection
and the program prints "end".

I don't know why the program doesn't do this on Windows. As far as I can tell
the endless exceptions _is_ the correct behaviour, right?

Anyway, if you comment out the the line
    listener.blocking = false;
in 'listener.d', does it work as intended then? In the example above this will
cause the listener.accept()-call to actually wait until it gets a connection,
and thus not spew out all the exceptions.

BR
/heywood




Bob Cowdery Wrote:


 Hi all,
 
 This is a long shot but I'm out of ideas. I ported an app from Windows
 to Linux and after many issues it is working but I'm left with a strange
 problem. The app basically reads data streams from a USB device,
 processes them and outputs real-time graphical data to a browser. There
 is also some control input from the browser. The interface to the
 browser is web sockets for which I have written a D web-socket server.
 Depending on how much of my application I allow to run I get a stream of
 these errors.
 
 Error accepting: std.socket.SocketAcceptException: Unable to accept
 socket connection: Resource temporarily unavailable
 ----------------
 ./DcSdr() [0x80aa2ed]
 ./DcSdr() [0x806f52b]
 ./DcSdr() [0x804f752]
 ./DcSdr() [0x809b422]
 ./DcSdr() [0x80ae77e]
 /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
 /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]
 
 This is the web-socket accept. I seem to fall straight through the
 accept call even when I am not making any connection attempt. When I do
 connect the stream stops and I get one of these outputs each time I send
 data from the browser. I should not even be near the accept at that
 point as the connection is made.
 
 The app appears to be unaffected and works as expected. The odd think is
 if I shut down part of the app these errors stop. I can't tie it down to
 any specific thing but I suspect threading as the number of threads is
 reduced by stopping parts of the app. The error also seems to indicate
 threads are involved. I did not get this problem on Windows.
 
 Any ideas would be much appreciated.
 
 Thanks
 bob
Dec 22 2010
parent reply Bob Cowdery <bob bobcowdery.plus.com> writes:
Hi Heywood

Thankyou for your time. Yes I agree making the call blocking does stop
the exceptions churning. Unfortunately the application stops accepting
data now because after the first incoming transfer from the web socket
client it sees data on the listening socket and promptly blocks on it
and never comes off until I make another connection. I think this is
expected behaviour. I could spawn a thread for each connection which I
would normally do but shouldn't need to as it's really only a few users
and I believe that would only sidestep the real problem.

There are lots of things here that don't make sense.

1. It works for Windows and it should work for Linux (and Mac) unless
Windows is broken and Linux is behaving as it should.
2. I should only see data on the listen socket when I make a new
connection. What I see is data on the listen socket when I send data on
the connected socket and with non-blocking even when I don't make a
connection or send any data at all.
2. The really confusing thing is if I stop my USB and DSP threads it
stops behaving like this and sees data on the correct sockets. In other
words it works as it does on Windows so I have to assume this is the
correct behaviour.
3. I've played around with trying to figure out exactly what it is in
the other threads that causes the behaviour. I got as far as finding
that only the DSP thead is required so that rules out misbehaving USB.
Very oddly there is a loop that is creating some Json data (not very
efficiently) as it's doing a lot of string concatenation. If I comment
out this loop or reduce its iterations the exception slow down to the
point where I just get 2 and then they stop. This points to something
horrible going on.

I can only hope I've done something stupid that I just happen to be
getting away with on Windows. If it's a bug in the compiler or libraries
I think I'm stuffed as I wouldn't know where to start.

Regards
bob

On 23/12/2010 00:20, Heywood Floyd wrote:

 Hi Bob!


 My guess: You're listener is set to be non-blocking. That means that when you
call listener.accept() it will return immediately with an
SocketAcceptException, if there's no connection. And you're basically calling
listener.accept() over and over again in an infinite loop.

 The following example shows it:

 import std.concurrency, std.stdio, std.conv, std.socket;
 void main()
 {
 	ushort port = 9999;
 	Socket listener = new TcpSocket;
 	assert(listener.isAlive);
 	listener.blocking = false;
 	listener.bind(new InternetAddress(port));
 	listener.listen(10);
 	writeln("Listening on port: ", port);
 	Socket sn;
 	while(true){
 		try
 		{
 			writeln ("Accepting");
 			sn = listener.accept();
 			writeln("Connection from ", sn.remoteAddress().toString(), " established" );
 			assert(sn.isAlive);
 			assert(listener.isAlive);
 			break;
 		}
 		catch(Exception e)
 		{
 			writeln("Error accepting: ", e.toString() );
 			if(sn)
 			sn.close();
 		}
 	}
 	writeln("end");
 }

 Running the example will print
     Error accepting: std.socket.SocketAcceptException: Unable to accept socket
connection: Resource temporarily unavailable
 for ever. (Tested on Mac.) When a user connects to 127.0.0.1:9999 (by for
instance opening the URL in a browser) socket.accept _does_ return a connection
and the program prints "end".

 I don't know why the program doesn't do this on Windows. As far as I can tell
the endless exceptions _is_ the correct behaviour, right?

 Anyway, if you comment out the the line
     listener.blocking = false;
 in 'listener.d', does it work as intended then? In the example above this will
cause the listener.accept()-call to actually wait until it gets a connection,
and thus not spew out all the exceptions.

 BR
 /heywood




 Bob Cowdery Wrote:


 Hi all,

 This is a long shot but I'm out of ideas. I ported an app from Windows
 to Linux and after many issues it is working but I'm left with a strange
 problem. The app basically reads data streams from a USB device,
 processes them and outputs real-time graphical data to a browser. There
 is also some control input from the browser. The interface to the
 browser is web sockets for which I have written a D web-socket server.
 Depending on how much of my application I allow to run I get a stream of
 these errors.

 Error accepting: std.socket.SocketAcceptException: Unable to accept
 socket connection: Resource temporarily unavailable
 ----------------
 ./DcSdr() [0x80aa2ed]
 ./DcSdr() [0x806f52b]
 ./DcSdr() [0x804f752]
 ./DcSdr() [0x809b422]
 ./DcSdr() [0x80ae77e]
 /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
 /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]

 This is the web-socket accept. I seem to fall straight through the
 accept call even when I am not making any connection attempt. When I do
 connect the stream stops and I get one of these outputs each time I send
 data from the browser. I should not even be near the accept at that
 point as the connection is made.

 The app appears to be unaffected and works as expected. The odd think is
 if I shut down part of the app these errors stop. I can't tie it down to
 any specific thing but I suspect threading as the number of threads is
 reduced by stopping parts of the app. The error also seems to indicate
 threads are involved. I did not get this problem on Windows.

 Any ideas would be much appreciated.

 Thanks
 bob
Dec 23 2010
parent reply Heywood Floyd <soul8o8 gmail.com> writes:
Hi!


I see.

I think my previous answer was a bit naive—I didn't appreciate the full scope
of the problem. Sorry for that, but you know, internet is fast, snap snap : )

Ok, for now I'm afraid I don't have any more to add. (An isolated example would
of course help greatly!)

All I can say is, in my experience, the D sockets library _does_ behave
differently on different platforms. Had a server that just died seemingly
random on osx, while working fine on Linux. Turned out to be bug in D sockets.
So there you go. (The bug is still there btw... *cough*)

Hm, also, I realize I'm not sure I quite understand the structure of your app,
and where in this structure the bug happends. Is this bug happening when your
html/5-client tries to connect to the server? Or are the threads using sockets
to communicate between eachother?


BR
/heywood



Bob Cowdery Wrote:


 Hi Heywood
 
 Thankyou for your time. Yes I agree making the call blocking does stop
 the exceptions churning. Unfortunately the application stops accepting
 data now because after the first incoming transfer from the web socket
 client it sees data on the listening socket and promptly blocks on it
 and never comes off until I make another connection. I think this is
 expected behaviour. I could spawn a thread for each connection which I
 would normally do but shouldn't need to as it's really only a few users
 and I believe that would only sidestep the real problem.
 
 There are lots of things here that don't make sense.
 
 1. It works for Windows and it should work for Linux (and Mac) unless
 Windows is broken and Linux is behaving as it should.
 2. I should only see data on the listen socket when I make a new
 connection. What I see is data on the listen socket when I send data on
 the connected socket and with non-blocking even when I don't make a
 connection or send any data at all.
 2. The really confusing thing is if I stop my USB and DSP threads it
 stops behaving like this and sees data on the correct sockets. In other
 words it works as it does on Windows so I have to assume this is the
 correct behaviour.
 3. I've played around with trying to figure out exactly what it is in
 the other threads that causes the behaviour. I got as far as finding
 that only the DSP thead is required so that rules out misbehaving USB.
 Very oddly there is a loop that is creating some Json data (not very
 efficiently) as it's doing a lot of string concatenation. If I comment
 out this loop or reduce its iterations the exception slow down to the
 point where I just get 2 and then they stop. This points to something
 horrible going on.
 
 I can only hope I've done something stupid that I just happen to be
 getting away with on Windows. If it's a bug in the compiler or libraries
 I think I'm stuffed as I wouldn't know where to start.
 
 Regards
 bob
 
 On 23/12/2010 00:20, Heywood Floyd wrote:

 Hi Bob!


 My guess: You're listener is set to be non-blocking. That means that when you
call listener.accept() it will return immediately with an
SocketAcceptException, if there's no connection. And you're basically calling
listener.accept() over and over again in an infinite loop.

 The following example shows it:

 import std.concurrency, std.stdio, std.conv, std.socket;
 void main()
 {
 	ushort port = 9999;
 	Socket listener = new TcpSocket;
 	assert(listener.isAlive);
 	listener.blocking = false;
 	listener.bind(new InternetAddress(port));
 	listener.listen(10);
 	writeln("Listening on port: ", port);
 	Socket sn;
 	while(true){
 		try
 		{
 			writeln ("Accepting");
 			sn = listener.accept();
 			writeln("Connection from ", sn.remoteAddress().toString(), " established" );
 			assert(sn.isAlive);
 			assert(listener.isAlive);
 			break;
 		}
 		catch(Exception e)
 		{
 			writeln("Error accepting: ", e.toString() );
 			if(sn)
 			sn.close();
 		}
 	}
 	writeln("end");
 }

 Running the example will print
     Error accepting: std.socket.SocketAcceptException: Unable to accept socket
connection: Resource temporarily unavailable
 for ever. (Tested on Mac.) When a user connects to 127.0.0.1:9999 (by for
instance opening the URL in a browser) socket.accept _does_ return a connection
and the program prints "end".

 I don't know why the program doesn't do this on Windows. As far as I can tell
the endless exceptions _is_ the correct behaviour, right?

 Anyway, if you comment out the the line
     listener.blocking = false;
 in 'listener.d', does it work as intended then? In the example above this will
cause the listener.accept()-call to actually wait until it gets a connection,
and thus not spew out all the exceptions.

 BR
 /heywood




 Bob Cowdery Wrote:


 Hi all,

 This is a long shot but I'm out of ideas. I ported an app from Windows
 to Linux and after many issues it is working but I'm left with a strange
 problem. The app basically reads data streams from a USB device,
 processes them and outputs real-time graphical data to a browser. There
 is also some control input from the browser. The interface to the
 browser is web sockets for which I have written a D web-socket server.
 Depending on how much of my application I allow to run I get a stream of
 these errors.

 Error accepting: std.socket.SocketAcceptException: Unable to accept
 socket connection: Resource temporarily unavailable
 ----------------
 ./DcSdr() [0x80aa2ed]
 ./DcSdr() [0x806f52b]
 ./DcSdr() [0x804f752]
 ./DcSdr() [0x809b422]
 ./DcSdr() [0x80ae77e]
 /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
 /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]

 This is the web-socket accept. I seem to fall straight through the
 accept call even when I am not making any connection attempt. When I do
 connect the stream stops and I get one of these outputs each time I send
 data from the browser. I should not even be near the accept at that
 point as the connection is made.

 The app appears to be unaffected and works as expected. The odd think is
 if I shut down part of the app these errors stop. I can't tie it down to
 any specific thing but I suspect threading as the number of threads is
 reduced by stopping parts of the app. The error also seems to indicate
 threads are involved. I did not get this problem on Windows.

 Any ideas would be much appreciated.

 Thanks
 bob
Dec 23 2010
next sibling parent Bob Cowdery <bob bobcowdery.plus.com> writes:
Hi
All communication is appreciated. I often find just trying to explain a
problem can lead to fixing it. Sorry this is a bit rambling.

I know D is not done yet and there will be bugs. I don't know yet if its
me or D.

The program structure is quite simple.
1. I have 3 pieces of C code. An interface to libusb. A library of
signal processing functions. An embedded Mongoose HTTP server.
2. There is a main module which has the outer loop and it starts 4
threads. One for the module that talks to the USB hardware. One for the
module that talks to the signal processing library. One for the HTTP
server and one the Web Sockets server which is also the back end of the GUI.
3. These threads all register their TID's with a registry module and
then each has a receive loop. Messages are then passed between threads.
The main threads do start some internal threads as some have more than
one task to perform.

There is no connection at all between threads except the messages that
are passed. It is pure message passing concurrency which is one of the
major reasons I went for D.

On the web side first connection is from the browser to the HTTP server
which simply provides the static content, html, css and js. Once the
browser kicks itself into life it make a web socket connection to my web
socket server and all chit chat goes on over this connection. I am
streaming realtime graphics data to the browser and receiving data from
it when I scroll digits, click buttons etc.

The only bug that is visible in the whole app occurs in the web socket
server as discussed. The thing I was playing with that seems to effect
it was firstly i stopped the USB and sig processing threads from being
created and commented out a few things so the sockets side still
functioned. This seemed to cure the problem completly. I put back the
sig processing thread and got the full force of the problem. In the sig
processing thread it constructs the grapic data to send to the browser
as series of points to plot. Even if I don't actually send the data the
problem still occurs. However, by not constructing the last bit which is
a Json string it reduces the problem considerably.

Yes. I need to produce a smaller example that demonstrates this.

Regards
bob

On 23/12/2010 17:06, Heywood Floyd wrote:

 Hi!


 I see.

 I think my previous answer was a bit naive—I didn't appreciate the full scope
of the problem. Sorry for that, but you know, internet is fast, snap snap : )

 Ok, for now I'm afraid I don't have any more to add. (An isolated example
would of course help greatly!)

 All I can say is, in my experience, the D sockets library _does_ behave
differently on different platforms. Had a server that just died seemingly
random on osx, while working fine on Linux. Turned out to be bug in D sockets.
So there you go. (The bug is still there btw... *cough*)

 Hm, also, I realize I'm not sure I quite understand the structure of your app,
and where in this structure the bug happends. Is this bug happening when your
html/5-client tries to connect to the server? Or are the threads using sockets
to communicate between eachother?


 BR
 /heywood



 Bob Cowdery Wrote:


 Hi Heywood

 Thankyou for your time. Yes I agree making the call blocking does stop
 the exceptions churning. Unfortunately the application stops accepting
 data now because after the first incoming transfer from the web socket
 client it sees data on the listening socket and promptly blocks on it
 and never comes off until I make another connection. I think this is
 expected behaviour. I could spawn a thread for each connection which I
 would normally do but shouldn't need to as it's really only a few users
 and I believe that would only sidestep the real problem.

 There are lots of things here that don't make sense.

 1. It works for Windows and it should work for Linux (and Mac) unless
 Windows is broken and Linux is behaving as it should.
 2. I should only see data on the listen socket when I make a new
 connection. What I see is data on the listen socket when I send data on
 the connected socket and with non-blocking even when I don't make a
 connection or send any data at all.
 2. The really confusing thing is if I stop my USB and DSP threads it
 stops behaving like this and sees data on the correct sockets. In other
 words it works as it does on Windows so I have to assume this is the
 correct behaviour.
 3. I've played around with trying to figure out exactly what it is in
 the other threads that causes the behaviour. I got as far as finding
 that only the DSP thead is required so that rules out misbehaving USB.
 Very oddly there is a loop that is creating some Json data (not very
 efficiently) as it's doing a lot of string concatenation. If I comment
 out this loop or reduce its iterations the exception slow down to the
 point where I just get 2 and then they stop. This points to something
 horrible going on.

 I can only hope I've done something stupid that I just happen to be
 getting away with on Windows. If it's a bug in the compiler or libraries
 I think I'm stuffed as I wouldn't know where to start.

 Regards
 bob

 On 23/12/2010 00:20, Heywood Floyd wrote:

 Hi Bob!


 My guess: You're listener is set to be non-blocking. That means that when you
call listener.accept() it will return immediately with an
SocketAcceptException, if there's no connection. And you're basically calling
listener.accept() over and over again in an infinite loop.

 The following example shows it:

 import std.concurrency, std.stdio, std.conv, std.socket;
 void main()
 {
 	ushort port = 9999;
 	Socket listener = new TcpSocket;
 	assert(listener.isAlive);
 	listener.blocking = false;
 	listener.bind(new InternetAddress(port));
 	listener.listen(10);
 	writeln("Listening on port: ", port);
 	Socket sn;
 	while(true){
 		try
 		{
 			writeln ("Accepting");
 			sn = listener.accept();
 			writeln("Connection from ", sn.remoteAddress().toString(), " established" );
 			assert(sn.isAlive);
 			assert(listener.isAlive);
 			break;
 		}
 		catch(Exception e)
 		{
 			writeln("Error accepting: ", e.toString() );
 			if(sn)
 			sn.close();
 		}
 	}
 	writeln("end");
 }

 Running the example will print
     Error accepting: std.socket.SocketAcceptException: Unable to accept socket
connection: Resource temporarily unavailable
 for ever. (Tested on Mac.) When a user connects to 127.0.0.1:9999 (by for
instance opening the URL in a browser) socket.accept _does_ return a connection
and the program prints "end".

 I don't know why the program doesn't do this on Windows. As far as I can tell
the endless exceptions _is_ the correct behaviour, right?

 Anyway, if you comment out the the line
     listener.blocking = false;
 in 'listener.d', does it work as intended then? In the example above this will
cause the listener.accept()-call to actually wait until it gets a connection,
and thus not spew out all the exceptions.

 BR
 /heywood




 Bob Cowdery Wrote:


 Hi all,

 This is a long shot but I'm out of ideas. I ported an app from Windows
 to Linux and after many issues it is working but I'm left with a strange
 problem. The app basically reads data streams from a USB device,
 processes them and outputs real-time graphical data to a browser. There
 is also some control input from the browser. The interface to the
 browser is web sockets for which I have written a D web-socket server.
 Depending on how much of my application I allow to run I get a stream of
 these errors.

 Error accepting: std.socket.SocketAcceptException: Unable to accept
 socket connection: Resource temporarily unavailable
 ----------------
 ./DcSdr() [0x80aa2ed]
 ./DcSdr() [0x806f52b]
 ./DcSdr() [0x804f752]
 ./DcSdr() [0x809b422]
 ./DcSdr() [0x80ae77e]
 /lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0x48496e]
 /lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xc5fa4e]

 This is the web-socket accept. I seem to fall straight through the
 accept call even when I am not making any connection attempt. When I do
 connect the stream stops and I get one of these outputs each time I send
 data from the browser. I should not even be near the accept at that
 point as the connection is made.

 The app appears to be unaffected and works as expected. The odd think is
 if I shut down part of the app these errors stop. I can't tie it down to
 any specific thing but I suspect threading as the number of threads is
 reduced by stopping parts of the app. The error also seems to indicate
 threads are involved. I did not get this problem on Windows.

 Any ideas would be much appreciated.

 Thanks
 bob
Dec 23 2010
prev sibling parent Bob Cowdery <bob bobcowdery.plus.com> writes:
Perseverance pays off in the end. Having worked out how the socket class
was supposed to work it seemed that the select call was returning when
it was not supposed to and giving a false indication in the socket set
of who spoke. When I looked at the return code of select() it was -1
(interrupted). Depending on how much of my other code was enabled the
frequency of these interruptions changed which is why I couldn't home in
on anything in particular. I don't know why the call is getting
interrupted. I don't use signals anywhere but I suspect the D library is
and possibly something to do with the receive function.

I can avoid the problem simply by sitting on the select until i get a >0
return code which means someone really did speak. I guess this does not
happen on Windows because Windows does not use signals. It's only a work
round. I would like to know why it's getting interrupted from someone
who knows the language and library implementation.

Regards
Bob

On 23/12/2010 18:28, Bob Cowdery wrote:

 Hi
 All communication is appreciated. I often find just trying to explain a
 problem can lead to fixing it. Sorry this is a bit rambling.

 I know D is not done yet and there will be bugs. I don't know yet if its
 me or D.

 The program structure is quite simple.
 1. I have 3 pieces of C code. An interface to libusb. A library of
 signal processing functions. An embedded Mongoose HTTP server.
 2. There is a main module which has the outer loop and it starts 4
 threads. One for the module that talks to the USB hardware. One for the
 module that talks to the signal processing library. One for the HTTP
 server and one the Web Sockets server which is also the back end of the GUI.
 3. These threads all register their TID's with a registry module and
 then each has a receive loop. Messages are then passed between threads.
 The main threads do start some internal threads as some have more than
 one task to perform.

 There is no connection at all between threads except the messages that
 are passed. It is pure message passing concurrency which is one of the
 major reasons I went for D.

 On the web side first connection is from the browser to the HTTP server
 which simply provides the static content, html, css and js. Once the
 browser kicks itself into life it make a web socket connection to my web
 socket server and all chit chat goes on over this connection. I am
 streaming realtime graphics data to the browser and receiving data from
 it when I scroll digits, click buttons etc.

 The only bug that is visible in the whole app occurs in the web socket
 server as discussed. The thing I was playing with that seems to effect
 it was firstly i stopped the USB and sig processing threads from being
 created and commented out a few things so the sockets side still
 functioned. This seemed to cure the problem completly. I put back the
 sig processing thread and got the full force of the problem. In the sig
 processing thread it constructs the grapic data to send to the browser
 as series of points to plot. Even if I don't actually send the data the
 problem still occurs. However, by not constructing the last bit which is
 a Json string it reduces the problem considerably.

 Yes. I need to produce a smaller example that demonstrates this.

 Regards
 bob
Dec 24 2010