Re: Any python scripts to do parallel downloading?

On Jan 31, 5:23 pm, "Frank Potter" <could....@xxxxxxxxx> wrote:
I want to find a multithreaded downloading lib in python,
can someone recommend one for me, please?
Thanks~

Why do you want to use threads for that? Twisted is the
obvious solution for your problem, but you may use any
asynchronous framework, as for instance the good ol
Tkinter:

"""
Example of asynchronous programming with Tkinter. Download 10 times
the same URL.
"""

import sys, urllib, itertools, Tkinter

URL = 'http://docs.python.org/dev/lib/module-urllib.html'

class Downloader(object):
chunk = 1024

def __init__(self, urls, frame):
self.urls = urls
self.downloads = [self.download(i) for i in range(len(urls))]
self.tkvars = []
self.tklabels = []
for url in urls:
var = Tkinter.StringVar(frame)
lbl = Tkinter.Label(frame, textvar=var)
lbl.pack()
self.tkvars.append(var)
self.tklabels.append(lbl)
frame.pack()

def download(self, i):
src = urllib.urlopen(self.urls[i])
size = int(src.info()['Content-Length'])
for block in itertools.count():
chunk = src.read(self.chunk)
if not chunk: break
percent = block * self.chunk * 100/size
msg = '%s: downloaded %2d%% of %s K' % (
self.urls[i], percent, size/1024)
self.tkvars[i].set(msg)
yield None
self.tkvars[i].set('Downloaded %s' % self.urls[i])

if __name__ == '__main__':
root = Tkinter.Tk()
frame = Tkinter.Frame(root)
downloader = Downloader([URL] * 10, frame)
def next(cycle):
try:
cycle.next().next()
except StopIteration:
pass
root.after(50, next, cycle)
root.after(0, next, itertools.cycle(downloader.downloads))
root.mainloop()


Michele Simionato

.

Relevant Pages

  • Re: Any python scripts to do parallel downloading?
    ... Michele Simionato wrote: ... asynchronous framework, as for instance the good ol ... def download: ... chunk = src.read ...
    (comp.lang.python)
  • Command config, quitting, binary, Timer
    ... I am still ignorant about Tkinter. ... def binary_conv: ... and calls a function for each tick. ... I think that maybe it can be added to the threading standard module. ...
    (comp.lang.python)
  • Re: Mindboggling Scope Issue
    ... def inner(): ... variable references in the function definition are only ... > def cancel(): ... whether you've seen much other Tkinter code, ...
    (comp.lang.python)
  • Re: getting n items at a time from a generator
    ... from itertools import * ... def group: ... iters = tee ... Kugutsumen> list version since I need to buffer that chunk in memory at ...
    (comp.lang.python)
  • Tkinter.Canvas thread safety problem?
    ... I have started a project using Tkinter. ... having the delay there, it hangs. ... print "thread loop finished, sleeping ..." ... def setStop: ...
    (comp.lang.python)