Re: Lost data on socket - Can we start over politely?

From: Thomas Kratz (ThomasKratz_at_REMOVEwebCAPS.de)
Date: 03/31/04 

Date: Wed, 31 Mar 2004 11:26:33 +0200

Vorxion wrote:

> In article <4069a132.0@juno.wiesbaden.netsurf.de>, Thomas Kratz wrote:
>
>>Instead please try this server ( some quickly reduced code from a bigger
>>server ) and mini client. If this works for you, you could extend it.
>>I don't loose any data with this setup. And I don't even bother to set
>>unbuffered mode (tested on Win32).
>
>
> It does indeed work. This is the best sign I've seen in a week, since I've
> been afraid it's simply not possible to do this, although I figured it was
> a goof on my end.
>
> I note that if I decrease the input chunks from 1024 down to 1, and that's
> the -only- change I make to yours, it will eventually exhibit exactly what
> I'm experiencing in my own server, which is a premature end of data--and
> in your case, a disconnect of the client due to how you coded the error
> checking--after about 30k.
>
> Given that, I strongly suspect that my problem is taking expensive
> time to sniff the first 15-20 bytes of each packet individually to
> find the application level packet lengths and a first internal packet
> separator, even though once I have the length I'm trying to read the
> rest of the packet at the full listed size of the packet. I'm talking
> application-level packets here, not TCP packets.
>
> In short, I think my code is simply lagging behind, and when it lags far
> enough, the rest of the data vanishes. I'm working on fixing that bit, and
> I'll also have to prioritize it so that when data is present, it stops
> working on processing its data internally and goes immediately back to
> reading from the socket. And I'm simply going to have it scarf up as much
> data as is present and basically parse and process when there is no
> actual communication going on. That should (hopefully) elminiate the
> problem.

That was my guess to, but I couldn't confirm it :-). The problem could be,
that the TCP buffers of the machine the server runs on, are filled because
you are draining them too slow, but the client is sending data anyway. I
didn't use IO::Select on the client with a can_write(), which should take
care of that ( provided you have control over the client code :-)

>
>>If you look at the server code, you'll get an idea what is meant by "a
>>small but complete sample to reproduce the problem". The base
>>functionality you need is in there. And if I were able to understand, what
>>your code tries to do, I would be able to shorten it to that level too.
>
>
> Well, mine is multiplexing--it's meant to take in more than one connection
> at a time. Yours was written with a single connection in mind, although it
> demonstrates perfectly that buffering shouldn't be an issue, even at 1k
> block sizes. The multiplexing is what's complicated mine so much. :)

No. Have you tried it? It handles exactly SOMAXCONN connections.
The only cause for using the $max_buf variable is preventing one client
from flooding the server with data and neglecting the other clients.
If the client data dripples in slowly, one could also use a $max_time
value (with Time:HiRes) for the maximum time the server processes one
socket at a time.

The foreach loop looks for all readable sockets and handles them. The only
thing left out is handling a specific timeout for connected client sockets
without incoming data (could be easily done via a lookup hash with the
stringified socket values as keys and the time value of the last action on
the socket as values, shouldn't be more than a few lines).

Thomas 

-- 
open STDIN,"<&DATA";$=+=14;$%=50;while($_=(seek( #J~.> a>n~>>e~.......>r.
STDIN,$:*$=+$,+$%,0),getc)){/\./&&last;/\w| /&&( #.u.t.^..oP..r.>h>a~.e..
print,$_=$~);/~/&&++$:;/\^/&&--$:;/>/&&++$,;/</  #.>s^~h<t< ..~. ...c.^..
&&--$,;$:%=4;$,%=23;$~=$_;++$i==1?++$,:_;}__END__#....>>e>r^..>l^...>k^..

Relevant Pages

  • [REVS] Backdoor Spotcom Analysis
    ... Spotcom is a backdoor client application that allows a hacker to control ... The server IP address is hard-coded in ... msrsvp.exe accepts a couple of command line arguments. ... the packet payload. ...
    (Securiteam)
  • Re: Socket switch delay
    ... both at the client and at the server (and why ... would you set the send buffer size to zero on a non-overlapped ... One glaring error is your client does ... So when you use a single socket, ...
    (microsoft.public.win32.programmer.networks)
  • Re: Socket weirdness
    ... client) before you will notice a shutdown receive at server. ... Then eventually a packet comes from the peer, and that will contain data, so the server responds RST: ... way back across the network. ...
    (microsoft.public.dotnet.framework)
  • Re: Locking on async calls
    ... you must synchronize the entire SendMessage routine as an atomic ... operation to prevent mixed messages from being transmitted to the server. ... You are correct that read and write on the socket do not interfere with each ... If you want to handle multiple client connections from one server object ...
    (microsoft.public.dotnet.general)
  • Re: Design issue with WinSock/GetQueuedCompletionStatus
    ... delegate that to a shutdown routine called after all worker threads ... The application I've created is a server accepting connections on a few ... different TCP/IP ports and then lets the client run different commands. ... a TCP/IP socket can be closed for 2 different reasons: ...
    (microsoft.public.win32.programmer.networks)