Re: Question about using urllib2 to load a url



On Apr 2, 2:52 am, "ken" <ken.carl...@xxxxxxxxx> wrote:
Hi,

i have the following code to load a url.
My question is what if I try to load an invalide url
("http://www.heise.de/";), will I get an IOException? or it will wait
forever?


Depends on why the URL is invalid. If the URL refers to a non-
existent domain, a DNS request will result in error and you will get
an "urllib2.URLError: <urlopen error (-2, 'Name or service not
known')>". If the name resolves but the host is not reachable, the
connect code will timeout (eventually) and result in an
"urllib2.URLError: <urlopen error (113, 'No route to host')>". If the
host exists but does not have a web server running, you will get an
"urllib2.URLError: <urlopen error (111, 'Connection refused')>". If a
webserver is running but the requested page does not exist, you will
get an "urllib2.HTTPError: HTTP Error 404: Not Found".

The URL you gave above does not meet any of these conditions, so
results in a valid handle to read from.

If, at any time, an error response fails to reach your machine, the
code will have to wait for a timeout. It should not have to wait
forever.

Thanks for any help.

opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
urllib2.install_opener(opener)

txheaders = {'User-agent': 'Mozilla/5.0 (X11; U; Linux i686; en-
US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3'}

try:
req = Request(url, txdata, txheaders)
handle = urlopen(req)
except IOError, e:
print e
print 'Failed to open %s' % url
return 0;

--
Kushal

.



Relevant Pages

  • Re: threads & sockets
    ... > better under load rather than worse. ... > yourself back on 250 queues perhaps dozens of times a second. ... > 350mS timeout, you do so only about three times a second. ... 250 fds in it, do you mean that when I call select, then kernel puts all ...
    (comp.unix.programmer)
  • dc0: Watchdog Timeout
    ... I am getting this message any time there is "load" on the system. ... Samba lost the connection right when the timeout came up. ... Motherboard is an Asus A7V600-X. ...
    (comp.unix.bsd.freebsd.misc)
  • Re: Assistance needed: Cant timeout calling MSXML2.XMLHTTP control
    ... the timeout to 20 which is more than enough time for the page to load. ... This tels me that it's got to be something I'm missing on my system - ...
    (microsoft.public.scripting.vbscript)
  • Re: em0: watchdog timeout -- resetting (6.1-STABLE)
    ... timeout happens, so I tried that. ... It happens a lot when my machine is under load. ... Always when there's a lot of CPU and disk ... Also sometimes my USB keyboard would become unresponsive at about the ...
    (freebsd-stable)
  • Re: Remoting is hanging from Custom to IIS Host swap
    ... if your custom host previously used a Tcp channel, the timeout will ... after the same amount of calls and/or at the same load level?) ... > Any calls after a certain amount of time will just eventually timeout. ...
    (microsoft.public.dotnet.framework.remoting)