Re: Simulating smaller MTU? ie sending small packets.



Also sprach Ed W:

> Hi, for various reasons I'm writing a little stress test app which tries
> to simulate the effects of varying sized TCP packets on the overall
> transfer speed.

This is probably a moot venture. The smaller the packets are, the lower
the overall throughput is going to be. This is due to the fact that TCP
packet have to be acknowledged. If no piggy-bagging is used (which is
the case for a pure-receiver side), then an additional 40 bytes (IP +
TCP minimum header size) need to be sent out on each acknowledgement.

The number of ACKs sent depend on the window size. Normally a receiver
tries to minimize ACKs and window size update messages (Clark).

> So I have written a little app which acts as a server, waits for a
> connection and then spews data in fixed sized chunks of your choice. I
> also turn off nagle, turn on autoflush, and as far as I can tell ask for
> the data to go out immediately
>
> What I observe (using an ethernet dump) is that once the receiver is not
> keeping up with the speed the sender is spewing packets, the *sender*
> (which in this case is linux 2.6.12) is starting to coallesce the packets
>
> So for example if I ask it to send 1000 byte packets I can see from the
> network trace that it starts to send lots of MTU sized packets instead
> (larger).

But it's probably going to send these larger packets at a lower rate.
Did you also check the ACK packets from the receiver? The Nagle
algorithm tells the sender never to send small packets. Turning it off
means sending them immediately as long as the receiver's side can keep
up. Now, if the receiver is congested, I would assume that the sender
still buffers small packets and once an ACK packet arrives it sends out
as many data as there is space in the receiving window.

> This is not what I was expecting at all, in fact I had no idea that
> there was some clever process in linux to coallesce small network
> packets? Am I tripping over some perl buffering instead? Any thoughts
> on where to look?

No, you're tripping over a sane implementation of the TCP stack. TCP by
nature is slow and has some overhead which is reduced by various means,
most notably the Nagle (sender) and Clark (receiver) algorithms.
Furthermore, in order to avoid clogging the subnet between sender and
receiver, congestion control is carried out (see TCP slow start
algorithm).

> Note, that it's not a mis-measurement problem at the receiving side. A
> Network trace is showing me that the packets are coming out at MTU sized
> (in general, but with a smattering of packets the size I requested).
>
> If I slow down the sending rate, or speedup the receiver then the
> packets go through at the correct size...

In order to do your measurements, you should probably adjust parameters
on the receiving side. If you want smaller packets, try to set the
window size (TCP_WINDOW_CLAMP, I think). TCP_MAX_SEG also needs to be
set there as the MSS is announced by the receiver during the
three-way-handshake when the connection is established.

Tassilo
--
use bigint;
$n=71423350343770280161397026330337371139054411854220053437565440;
$m=-8,;;$_=$n&(0xff)<<$m,,$_>>=$m,,print+chr,,while(($m+=8)<=200);
.



Relevant Pages

  • Re: Simulating smaller MTU? ie sending small packets.
    ... This is due to the fact that TCP ... If you want smaller packets, ... >> set there as the MSS is announced by the receiver during the ... Yes, per connection. ...
    (comp.lang.perl.misc)
  • Packet timing to send key bits
    ... I was wondering if perhaps you could use the transmission time between ... need to monitor many packets and do some post processing to derive ... connection time necessary between sender and receiver would be ...
    (sci.crypt)
  • Re: .Net TCP Socket Problem With Missing Chunks From Large Packets
    ... >> An acquaintance asked me to assist him with some TCP socket issues. ... However, when large packets are sent over longer>> distances, the data arrives with portions missing. ... That means that if you send n bytes, the> receiver might receive anywhere from 1 to n bytes when reading data out of> the socket. ...
    (microsoft.public.dotnet.framework.remoting)
  • Re: Deaf CAsyncSocket on Windows Service.
    ... To read them out I need a buffer to ... Short packets at the sender are coalesced into longer packets. ... THe receiver says "I have this much buffer space" and the sender is free to send ... You get an OnReceive notification if there is input data to be read. ...
    (microsoft.public.vc.mfc)
  • Re: HEADS UP: SACK committed to HEAD
    ... TCP may experience poor performance when multiple packets are lost ... An aggressive sender could ... then retransmit only the missing data segments. ...
    (freebsd-current)