www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.socket

reply Steve Teale <steve.teale britseyeview.com> writes:
Who is the resident expert? Sean maybe?

How do I get the receive buffer size up into the megabytes range for use 
between the D client and localhost. setOption seems to only get me about 
262k.

Steve
Nov 02 2011
next sibling parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
Steve Teale wrote:

 Who is the resident expert? Sean maybe?
 
 How do I get the receive buffer size up into the megabytes range for use 
 between the D client and localhost. setOption seems to only get me about 
 262k.


How do you do it?

From the documentation I would expect that


auto sizeInBytes = 10 * 2^^20;
mySocket.setOption(SocketOptionLevel.SOCKET, SocketOption.RCVBUF, sizeInBytes);

should set it to 10 MiB;
I'm unsure about the socket option level. There is also
SocketOptionLevel.IP.

Jens
Nov 02 2011
parent reply Steve Teale <steve.teale britseyeview.com> writes:
On Wed, 02 Nov 2011 17:17:07 +0100, Jens Mueller wrote:

 should set it to 10 MiB;
 I'm unsure about the socket option level. There is also
 SocketOptionLevel.IP.
 


Yes Jens, that's what I thought, and that approach does indeed change the 
buffer size, but not by anything close to what you ask for. I've tried 
both SOCKET and IP.

Steve
Nov 02 2011
next sibling parent reply "Regan Heath" <regan netmail.co.nz> writes:
On Wed, 02 Nov 2011 16:35:25 -0000, Steve Teale  
<steve.teale britseyeview.com> wrote:


 On Wed, 02 Nov 2011 17:17:07 +0100, Jens Mueller wrote:

 should set it to 10 MiB;
 I'm unsure about the socket option level. There is also
 SocketOptionLevel.IP.


 Yes Jens, that's what I thought, and that approach does indeed change the
 buffer size, but not by anything close to what you ask for. I've tried
 both SOCKET and IP.


What operating system are you on?

This may be of interest (even if you're not on windows, as it references  
some RFCs etc which may have some bearing on how things work on your OS):
http://msdn.microsoft.com/en-us/library/ms819736.aspx

Also, this thread mentions fpathconf() which I've not heard of before  
(probably because it's more of a UNIX thing?):
http://stackoverflow.com/questions/565804/max-value-for-so-rcvbuf-in-windows

.. why do you want/need such a large TCPIP recv buffer?

Regan

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/
Nov 02 2011
parent reply Steve Teale <steve.teale britseyeview.com> writes:
On Wed, 02 Nov 2011 16:45:30 +0000, Regan Heath wrote:


 Yes Jens, that's what I thought, and that approach does indeed change
 the buffer size, but not by anything close to what you ask for. I've
 tried both SOCKET and IP.


 
 What operating system are you on?
 
 This may be of interest (even if you're not on windows, as it references
 some RFCs etc which may have some bearing on how things work on your
 OS): http://msdn.microsoft.com/en-us/library/ms819736.aspx
 
 Also, this thread mentions fpathconf() which I've not heard of before
 (probably because it's more of a UNIX thing?):
 http://stackoverflow.com/questions/565804/max-value-for-so-rcvbuf-in-


windows

 
 .. why do you want/need such a large TCPIP recv buffer?


I'm using Ubuntu i386. I just thought that since MySQL talks blithely 
about packet sizes of 16mb I would investigate how my protocol interface 
performs with large buffers and large BLOBS.

I believe Ubuntu imposes the buffer size limitation and you have to mess 
with kernel configuration files to change that. But I don't want to go 
there and wondered if I was missing some simple point.

Steve
Nov 02 2011
parent reply Sean Kelly <sean invisibleduck.org> writes:
On Nov 2, 2011, at 9:59 AM, Steve Teale wrote:


 On Wed, 02 Nov 2011 16:45:30 +0000, Regan Heath wrote:
=20

 .. why do you want/need such a large TCPIP recv buffer?


=20
 I'm using Ubuntu i386. I just thought that since MySQL talks blithely=20=



 about packet sizes of 16mb I would investigate how my protocol =


interface=20

 performs with large buffers and large BLOBS.


I suspect that MySQL's "packet size" has no relation to TCP packet size. =
 Their max_packet_size is probably actually a max_request_size?  Even if =
you had a huge TCP buffer, the MTU used by the network (typically 1500 =
bytes) would break a MySQL "packet" into multiple TCP packets.=
Nov 02 2011
parent reply Steve Teale <steve.teale britseyeview.com> writes:
On Wed, 02 Nov 2011 16:17:12 -0700, Sean Kelly wrote:


 On Nov 2, 2011, at 9:59 AM, Steve Teale wrote:
 

 On Wed, 02 Nov 2011 16:45:30 +0000, Regan Heath wrote:
 

 .. why do you want/need such a large TCPIP recv buffer?




 
 I suspect that MySQL's "packet size" has no relation to TCP packet size.
  Their max_packet_size is probably actually a max_request_size?  Even if
 you had a huge TCP buffer, the MTU used by the network (typically 1500
 bytes) would break a MySQL "packet" into multiple TCP packets.


Yes Sean, I'm sure from what I've seem that that is the case. They're 
logical packets, and I shall stop worrying about it.
Nov 02 2011
parent Jens Mueller <jens.k.mueller gmx.de> writes:
Steve Teale wrote:

 On Wed, 02 Nov 2011 16:17:12 -0700, Sean Kelly wrote:
 

 On Nov 2, 2011, at 9:59 AM, Steve Teale wrote:
 

 On Wed, 02 Nov 2011 16:45:30 +0000, Regan Heath wrote:
 

 .. why do you want/need such a large TCPIP recv buffer?




 
 I suspect that MySQL's "packet size" has no relation to TCP packet size.
  Their max_packet_size is probably actually a max_request_size?  Even if
 you had a huge TCP buffer, the MTU used by the network (typically 1500
 bytes) would break a MySQL "packet" into multiple TCP packets.


 
 Yes Sean, I'm sure from what I've seem that that is the case. They're 
 logical packets, and I shall stop worrying about it.


Shouldn't we anyway write a bug report for std.socket? Yesterday I tried
doing it by hand using setsockopt but even that didn't not work.

Jens
Nov 03 2011
prev sibling parent Jens Mueller <jens.k.mueller gmx.de> writes:
Steve Teale wrote:

 On Wed, 02 Nov 2011 17:17:07 +0100, Jens Mueller wrote:

 should set it to 10 MiB;
 I'm unsure about the socket option level. There is also
 SocketOptionLevel.IP.
 


 Yes Jens, that's what I thought, and that approach does indeed change the 
 buffer size, but not by anything close to what you ask for. I've tried 
 both SOCKET and IP.


I'm looking into std.socket.setOption at the moment.
There is

(&value)[0 .. 1]

where value is an int32_t.
Then setsockopt is called using this void[] array. I wonder what this
code (&value)[0 .. 1] does and I don't how the argument to setsockopt
needs to be (need to consult the manual). Maybe it's an endianess issue.
In sum, besides the above code setOption is a direct wrapper around
setsockopt.

Jens
Nov 02 2011
prev sibling parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
Steve Teale wrote:

 Who is the resident expert? Sean maybe?
 
 How do I get the receive buffer size up into the megabytes range for use 
 between the D client and localhost. setOption seems to only get me about 
 262k.
 
 Steve


I found what's causing this behavior (see man 7 socket (search for
SO_RCVBUF)).
The maximum value is specified in /proc/sys/net/core/rmem_max. On my
system it's 131071.

Jens
Nov 03 2011
parent Steve Teale <steve.teale britseyeview.com> writes:
On Thu, 03 Nov 2011 10:24:19 +0100, Jens Mueller wrote:


 I found what's causing this behavior (see man 7 socket (search for
 SO_RCVBUF)).
 The maximum value is specified in /proc/sys/net/core/rmem_max. On my
 system it's 131071.
 
 Jens


Yup, I got there too, so I've dropped the idea.

Thanks for helping.

Steve
Nov 03 2011