Re: sending very large packets over the network
- From: Azazello <tylerca@xxxxxxxxxxxx>
- Date: Thu, 02 Aug 2007 15:27:42 -0000
On Aug 1, 8:50 pm, Gary Herron <gher...@xxxxxxxxxxxxxxxxxx> wrote:
Walker Lindley wrote:
OK, I'm back with another networking question. I'm trying to seend
large amounts of information over TCP (the length of data being given
to send() is on the order of 16000 characters in length).
Unfortunately on the receiving end, the packets appear to be
truncated. So I wrote some code that continuously tries to send bigger
and bigger packets until it fails and noticed that it never fails at
the same length. I'm not even sure these two things are related, but
is there some undocumented (or documented and I missed it) maximum
size for data you can pass to send()?
For ethernet connections the size is often about 1500. But the size
depends on the underlying protocol, and even if you know the underlying
protocol along the full route, you can't rely on packets that are
received being the same as those that are sent.
TCP/IP is a *stream* connection. What you are guaranteed is this: All
the bytes that are sent from one end will be received eventually on the
other end, in the proper order. (Or failing that, you will receive an
error notification.) No guarantee is made about the packet sizes on
either end, and you can't even rely on the packets being the same in
number or length on the two ends.
Your send code can try sending a packet of any size, but it must be
prepared to examine the number of bytes actually sent, and retry with
the remainder in a loop until all bytes are sent. Similarly your
receiving code must loop around the receive accepting whatever sized
packets makes it through the connection.
In many TCP/IP connections, it seems that the packets received are
one-for-one with the packets sent, but relying on this *IS AN ERROR*
that will bite you. It fails to across the internet (sometimes) and
when (at least some) wireless cards are involved.
You may be better off using a package that knows all this and handles it
properly. Modules asyncore and asynchat are one possibility.
Gary Herron
the sample code is as follows
#server
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("", 2000))
s.listen(0)
sock, addrinfo = s.accept()
for i in range(2 ** 16):
length = int(sock.recv(16))
print "excpecting data of length:", length
data = sock.recv(length)
print "received data of length:", len(data)
s.close()
sock.close()
#client
import socket
import time
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("", 2001))
s.connect(("localhost", 2000))
for i in range(2 ** 16):
packet = "h" * i
s.send("%16d" % len(packet))
print "attempting to send data of length:", len(packet)
tmp = s.send(packet)
print "actually sent data of length:", tmp
time.sleep(.001)
s.close()
i just put them in different files and ran them from the command line.
Any help or suggestions would be greatly appreciated. Thanks.
-Walker
--
This e-mail is licensed under the Creative Commons
Attribution-NoDerivs 2.5 License. To view a copy of this license,
visithttp://creativecommons.org/licenses/by-nd/2.5/or send a letter
to Creative Commons, 543 Howard Street, 5th Floor, San Francisco,
California, 94105, USA.- Hide quoted text -
- Show quoted text -
I'm not an expert on networking so take my advice with a grain of
salt!
My guess is that you're encountering race-like conditions between the
send and recv. commmands because you're running your communication
command 2^16 times, hoping that send and recv. are synchronous on
every step. This is especially true when you're running other
commands in your data sending loops! Prints take quite a lot of time.
Also, the send command in C (i'm not sure if this is true in python)
does not guarantee that all of your data gets out.
I'm assuming that you're somewhat new to networking so I would
recommend shying away from running an Asynchronous polling server.
Although it is probably a better way to do this.
My suggestions:
Write a while loop in the server with some end of stream checking,
timeouts, et cetera. This will give you a bit of flexibility and
feedback when things go to pot. Use the sendall() command in your
client because it throws an exception if data isn't properly sent. and
will eliminate a unsightly for loop.
Client
socket.sendall(data)
Something akin to this old piece of test code
datastream = ''
## A timeout exception is thrown if the receiver is waiting on the
line for .25 seconds
socket.settimeout(.25)
while 1:
try:
buf = socket.recv(1024) ## If data on line put in buffer
except socket.timeout: ## Data buffer timeout, breaks
while loop
break
if buf == '': ## If an empty buffer, jump to top of loop (beginning
of file check)
continue
datastream = datastream + str(buf)
if not len(buf) == 1024: ## If the buffer is not full break
signaling end of file.
break
Hope this helps! And like i said i'm pretty new to this so be
careful! Foundations of Network Programming by John Goerzen is a
great book if you want some good information on the topic.
Tyler
.
- References:
- Re: sending very large packets over the network
- From: Gary Herron
- Re: sending very large packets over the network
- Prev by Date: RE: Bug in Time module, or in my understanding?
- Next by Date: Memory Leak with Tkinter Canvas (Python 2.5 Win32)
- Previous by thread: Re: sending very large packets over the network
- Next by thread: frequency analysis of a DB column
- Index(es):
Relevant Pages
|
