Ideas required for usage of sockets API for efficient netw..

Hi all,

I am evaluating various methods of using sockets API for a network
program to make it efficient. Your ideas are most welcome. (reference:
rstevens book)

For the purpose of our discussion lets take reference the following
definition of the programs:

1) The server handles multiple connections, the connections can be
many in no. (~20-50)
2) The server reads data from clients and writes data to clients.
3) The clients too read data from server and write data to it. This
and the point 2 imply there is a protocol between the server and
clients. The protocol mandates exchange of data in form of buffers
where each buffer has a fixed size header(X bytes) followed by
variable length payload(N bytes).
4) Both servers/clients are high bandwidth applications.
5) There exists some latency while processing data in servers and
clients (Z units of time)
6) The performance metric is low CPU consumption for Y no. of messages
in the system.


In my opinion such programs must use :

1) epoll API instead of select.

2) non blocking sockets for read/write. Non blocking for write
necessitates use of some efficient buffering mechanism of saving
partial writes. We must avoid memcopy here. Please suggest some
algorithms here.

3) Reduce the no. of reads of messages by using scatter read API.
However it is complicated as the data is always processed as header
+payload. How a read vector has to be setup I am unable to understand.

4) Reduce the no. of writes of messages by using gather write API. The
implementation becomes difficult if it is combined with non blocking
sockets and one has to handle partial writes.

Lets have a discussion on each of the above 4 points. Your algorithms
and ideas and any new points are welcome.

On Jun 6, 1:49 pm, saurabhth <saurab... DeleteThis @gmail.com> wrote:
> Hi all,
>
> I am evaluating various methods of using sockets API for a network
> program to make it efficient. Your ideas are most welcome. (reference:
> rstevens book)
>
> For the purpose of our discussion lets take reference the following
> definition of the programs:
>
> 1) The server handles multiple connections, the connections can be
> many in no. (~20-50)
> 2) The server reads data from clients and writes data to clients.
> 3) The clients too read data from server and write data to it. This
> and the point 2 imply there is a protocol between the server and
> clients. The protocol mandates exchange of data in form of buffers
> where each buffer has a fixed size header(X bytes) followed by
> variable length payload(N bytes).
> 4) Both servers/clients are high bandwidth applications.
> 5) There exists some latency while processing data in servers and
> clients (Z units of time)
> 6) The performance metric is low CPU consumption for Y no. of messages
> in the system.
>
> In my opinion such programs must use :
>
> 1) epoll API instead of select.
>
> 2) non blocking sockets for read/write. Non blocking for write
> necessitates use of some efficient buffering mechanism of saving
> partial writes. We must avoid memcopy here. Please suggest some
> algorithms here.
>
> 3) Reduce the no. of reads of messages by using scatter read API.
> However it is complicated as the data is always processed as header
> +payload. How a read vector has to be setup I am unable to understand.
>
> 4) Reduce the no. of writes of messages by using gather write API. The
> implementation becomes difficult if it is combined with non blocking
> sockets and one has to handle partial writes.
>
> Lets have a discussion on each of the above 4 points. Your algorithms
> and ideas and any new points are welcome.

One thing I forgot to add: The client/server programs use TCP/IP