www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - network help needed

reply "nobody_" <spam spam.spam> writes:
Which way is the best way to get data(1000Hz 8channel double) over a network 
(directly connected).
The data should have an low as possible delay, or a known delay.

I'm really not into networking (as apparent in this post :), but understand 
it needs the std.socket(stream).
Oct 23 2006
parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
It sounds like you've got an audio stream you want to send.  And it 
sounds like really low quality data too?

I'm afraid I'm not really an audio person, Hz, etc.. but I think 
uncompressed you're talking about 1kHz 8-bit stereo sound.  Or you're 
not talking about sound and you're just using that terminology to 
confuse me.

1kHz 8-bit stereo means 2000 bytes per second.  That's about 2kb/s which 
is not really too bad even over dial-up.  Compress it, and you might 
find even better savings... depending.

Anyway, for code, it's really not that hard.  Imagine a switchboard. 
You know, those old-timey things.  With the operator, and a bunch of 
plugs.  One of those.

Now, a "socket" could be thought of as a plug.  A "connection" is the 
wire in between.  Operators are probably routers or something.  Doesn't 
matter.

There are two parts to it.  Clients, and servers.

For clients: first, you want to create a socket:

Socket sock = new TcpSocket(AddressFamily.INET);

Now, to make a connection, you need to have somewhere to connect to.  A 
phone number, if you will.  This is an IP address.  You also need a 
"port".  That's like who you're calling for.  And yes, you need one.

Address addr = new InternetAddress("10.0.0.1", 9432);

After this, you connect.  Happy days:

sock.connect(addr);

 From then on, you can read and write data using send and receive. 
However, note that these functions may return:

   - an error code.
   - a number of bytes less than the number you wanted to send.
   - the number of bytes you wanted to send.

So, you make a buffer, and slowly send all of it.

For servers, it's not much different.  But this time, you take your 
socket, as created above... and "bind" to an address.

Binding to an address basically means you're taking/getting a phone 
number and port.  So, like this:

sock.bind(new InternetAddress("10.0.0.1", 9432));

Conveniently, this port number should be the same as the one the client 
uses.

Then you have to start "listening":

sock.listen(32);

The parameter is how many people can wait in line.

After that, it gets a lot more complicated, and you might do select() or 
threads, etc.  But, somewhere you accept a new connection, like so:

Socket client = sock.accept();

Once you have the new socket, this one can have data sent or read from 
it.  Again, just use send() and receive().  They work the same.

Hope that helps some.  But your question is fairly vague.

-[Unknown]



 Which way is the best way to get data(1000Hz 8channel double) over a network 
 (directly connected).
 The data should have an low as possible delay, or a known delay.
 
 I'm really not into networking (as apparent in this post :), but understand 
 it needs the std.socket(stream).
Oct 24 2006
parent reply "nobody_" <spam spam.spam> writes:
Thanks for your reply!
Yes, I just used Hz to confuse you :D, its not sound, but eyetracker data.
This eyetracker has 8(maybe only 4 needed) analog outputs, each generating a 
thousand doubles per second.
A second computer needs to get this data with the lowest lag possible, or 
with a known lag.

I'm terribly sorry for the vagueness of my previous post.
Is tcp the best protocol for this?
Oct 24 2006
parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
Normally you want to use TCP because it provides a lot of groundwork for 
you.

UDP, on the other hand, is sort of like shooting spitballs.  They 
usually make it but it's not as reliable.

What's a double?  Is that two bytes?  What's going to matter for lag is 
how much data you're sending and the connection between the two 
computers.  Are they in the same network, or across the internet?

-[Unknown]



 Thanks for your reply!
 Yes, I just used Hz to confuse you :D, its not sound, but eyetracker data.
 This eyetracker has 8(maybe only 4 needed) analog outputs, each generating a 
 thousand doubles per second.
 A second computer needs to get this data with the lowest lag possible, or 
 with a known lag.
 
 I'm terribly sorry for the vagueness of my previous post.
 Is tcp the best protocol for this?
Oct 24 2006
parent reply "nobody_" <spam spam.spam> writes:
 Normally you want to use TCP because it provides a lot of groundwork for 
 you.

 UDP, on the other hand, is sort of like shooting spitballs.  They usually 
 make it but it's not as reliable.

 What's a double?  Is that two bytes?  What's going to matter for lag is 
 how much data you're sending and the connection between the two computers. 
 Are they in the same network, or across the internet?

 -[Unknown]


It will be:
1000*4=4000 bytes/sec (double=64 bit)
As they are directly connected using network cards, this shouldn't be a 
problem.
Is it possible to send data at a rate of 1000samples/sec over a tcp 
connection? (4 byte chunks :)
Oct 24 2006
parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
Ah, sorry, still holding on to that audio terminology.  Of course, 
doubles as in double precision floating point numbers... heh.

So, 4kB/s is nothing to worry about.  Your typical 56k modem can do 7 
(56 / 8) kilobytes per second usually, so if you've got even 10 meg 
cable between the two boxes you'll have no problems with lag.

All you do is send the data when you feel like sending it.  If you want 
to limit the rate the data is sent at, send it at only that rate (being 
4 bytes per millisecond.)

The operating system does some buffering and etc., but so long as you're 
sending and the client is receiving, packets will get sent between.  If 
you do experience any problems with these buffers, you could pad the 
data or something... but I'd be surprised if you did.

If you're getting this from the eyetracker at 4 bytes per millisecond, 
just end it as you get it.  That should be fine.  Or, buffer it and send 
it every 10 milliseconds (being 40 bytes/10 milliseconds) - shouldn't 
really matter for an intranet connection.

-[Unknown]



 Normally you want to use TCP because it provides a lot of groundwork for 
 you.

 UDP, on the other hand, is sort of like shooting spitballs.  They usually 
 make it but it's not as reliable.

 What's a double?  Is that two bytes?  What's going to matter for lag is 
 how much data you're sending and the connection between the two computers. 
 Are they in the same network, or across the internet?

 -[Unknown]


 
 It will be:
 1000*4=4000 bytes/sec (double=64 bit)
 As they are directly connected using network cards, this shouldn't be a 
 problem.
 Is it possible to send data at a rate of 1000samples/sec over a tcp 
 connection? (4 byte chunks :)
Oct 24 2006
parent reply "nobody_" <spam spam.spam> writes:
 The operating system does some buffering and etc., but so long as you're 
 sending and the client is receiving, packets will get sent between.  If 
 you do experience any problems with these buffers, you could pad the data 
 or something... but I'd be surprised if you did.


Just to surprise you, I read:
http://tangentsoft.net/wskfaq/articles/effective-tcp.html
And it looks like I should do some length-prefixing.


 If you're getting this from the eyetracker at 4 bytes per millisecond, 
 just end it as you get it.  That should be fine.  Or, buffer it and send 
 it every 10 milliseconds (being 40 bytes/10 milliseconds) - shouldn't 
 really matter for an intranet connection.

 -[Unknown]


Hmm, I forgot about the other 5 channels in my last post :/
So that will be 6*4 bytes per millisecond, still not a problem I think ;)

The computer recieving the data also sends start/stop signals to my 
computer.
Should I use a different port for these signals, or just use the one port 
for both way communication (won't it interfere eg. give higher latencies?)
Should the sockettype be stream?

I'm sorry for all the questions, this whole area of programming is new to 
me. I'm more into single theaded games programming :)
Oct 25 2006
parent reply "Unknown W. Brackets" <unknown simplemachines.org> writes:
That is only discussing how to determine how much data you should read. 
  If you expect the data at a constant rate, and you're consuming it, 
this shouldn't be a problem.

That is, if each packet is going to be one double, or 10 doubles, you 
know the length.  Prefixing things with the length will change nothing. 
  It's like putting a large sign on your car that says "CAR" just in 
case you should forget what it is.

My concern was that data might sit in the operating system's socket 
buffer, but this probably won't be a worry.  Normally, packets are much 
bigger than 4 bytes, though... but it depends on network card and router 
settings, iirc.

You should be able to use one port for two-way communication.  I 
honestly have never used the "socketstream" and don't know of its quality.

However, what you would probably want to do is make the socket 
non-blocking and use Socket.select() to check if it is ready for reading 
or for writing.  This way, your server side can determine if it needs to 
read a stop/start command or should send data.

For stop/start commands, you may want to use a fixed size, length 
prefixing, or delimiting.  As the page you linked to states, you do want 
to know how much to read; it makes it faster and cleaner.

As an example, HTTP/1.0 only supported Content-Length (so you had to 
know the entity size before hand) and was one connection per entity. 
HTTP/1.1 was a huge improvement in performance, because it uses a single 
connection for multiple entities, allows for pipelining (asking for 
multiple entities before receiving any), and uses length prefixing for 
the entity-body if negotiated.

All that said, your program sounds like it will be dealing with so few 
bytes as to not be an issue anyway.... even at 24kB/s.

Hope that helps.

-[Unknown]



 The operating system does some buffering and etc., but so long as you're 
 sending and the client is receiving, packets will get sent between.  If 
 you do experience any problems with these buffers, you could pad the data 
 or something... but I'd be surprised if you did.


 Just to surprise you, I read:
 http://tangentsoft.net/wskfaq/articles/effective-tcp.html
 And it looks like I should do some length-prefixing.
 

 If you're getting this from the eyetracker at 4 bytes per millisecond, 
 just end it as you get it.  That should be fine.  Or, buffer it and send 
 it every 10 milliseconds (being 40 bytes/10 milliseconds) - shouldn't 
 really matter for an intranet connection.

 -[Unknown]


 
 Hmm, I forgot about the other 5 channels in my last post :/
 So that will be 6*4 bytes per millisecond, still not a problem I think ;)
 
 The computer recieving the data also sends start/stop signals to my 
 computer.
 Should I use a different port for these signals, or just use the one port 
 for both way communication (won't it interfere eg. give higher latencies?)
 Should the sockettype be stream?
 
 I'm sorry for all the questions, this whole area of programming is new to 
 me. I'm more into single theaded games programming :)
Oct 25 2006
parent reply "nobody_" <spam spam.spam> writes:
 That is only discussing how to determine how much data you should read.
  If you expect the data at a constant rate, and you're consuming it, this 
 shouldn't be a problem.


I thought that by defining the size it won't buffer anymore.


 That is, if each packet is going to be one double, or 10 doubles, you know 
 the length.  Prefixing things with the length will change nothing. It's 
 like putting a large sign on your car that says "CAR" just in case you 
 should forget what it is.

 My concern was that data might sit in the operating system's socket 
 buffer, but this probably won't be a worry.  Normally, packets are much 
 bigger than 4 bytes, though... but it depends on network card and router 
 settings, iirc.


Yes this was my concern too :(


 You should be able to use one port for two-way communication.  I honestly 
 have never used the "socketstream" and don't know of its quality.

 However, what you would probably want to do is make the socket 
 non-blocking and use Socket.select() to check if it is ready for reading 
 or for writing.  This way, your server side can determine if it needs to 
 read a stop/start command or should send data.

 For stop/start commands, you may want to use a fixed size, length 
 prefixing, or delimiting.  As the page you linked to states, you do want 
 to know how much to read; it makes it faster and cleaner.

 As an example, HTTP/1.0 only supported Content-Length (so you had to know 
 the entity size before hand) and was one connection per entity. HTTP/1.1 
 was a huge improvement in performance, because it uses a single connection 
 for multiple entities, allows for pipelining (asking for multiple entities 
 before receiving any), and uses length prefixing for the entity-body if 
 negotiated.

 All that said, your program sounds like it will be dealing with so few 
 bytes as to not be an issue anyway.... even at 24kB/s.

 Hope that helps.


Thanks, I will try and implement it now. I'll be back when its finished (or 
broken :D)
Now I need to read into multithreading... another totally new thing for me.
(I don't even know how one thead gets his data from another)
Greets
Oct 25 2006
parent reply Unknown W. Brackets <unknown simplemachines.org> writes:
No, the operating system doesn't peek in and notice sizes or delimiters.  I
mean,
think of the endianness issues!

If you run into buffering, try padding.  I would suggest doing a simple test
program, connect the two machines to each other... and on one, send 4 bytes
every
millisecond.

On the other, constantly receive from the socket, and see how many bytes you get
back each time.

Multithreading isn't too bad, actually.  Just avoid the use of globals... which
are, as it happens, how threads communicate.  Basically, if you have a pointer,
any thread can access the memory it points to (given a pointer to the same
memory.)  This is how they can communicate.

In other words, they all share one heap.

They can also access the memory at the same time, which can cause problems.  I
think there are good tutorials available on multithreading, which probably
explain
it and the troubles involved better than I can.

-[Unknown]
Oct 25 2006
parent "nobody_" <spam spam.spam> writes:
Thanks for everything,
Now I just need to code it and test everything out a bit.
I'll report my findings ;)
Oct 26 2006