Re: Do I have to use threads?

aditya shukla wrote:
Hello people,

I have 5 directories corresponding 5 different urls .I want to download images from those urls and place them in the respective directories.I have to extract the contents and download them simultaneously.I can extract the contents and do then one by one. My questions is for doing it simultaneously do I have to use threads?

Please point me in the right direction.


Thanks

Aditya

You've been given some bad advice here.

First -- threads are lighter-weight than processes, so threads are probably *more* efficient. However, with only five thread/processes, the difference is probably not noticeable. (If the prejudice against threads comes from concerns over the GIL -- that also is a misplaced concern in this instance. Since you only have network connection, you will receive only one packet at a time, so only one thread will be active at a time. If the extraction process uses a significant enough amount of CPU time so that the extractions are all running at the same time *AND* if you are running on a machine with separate CPU/cores *AND* you would like the extractions to be running truly in parallel on those separate cores, *THEN*, and only then, will processes be more efficient than threads.)

Second, running 5 wgets is equivalent to 5 processes not 5 threads.

And third -- you don't have to use either threads *or* processes. There is another possibility which is much more light-weight: asynchronous I/O, available through the low level select module, or more usefully via the higher-level asyncore module. (Although the learning curve might trip you up, and some people find the programming model for asyncore hard to fathom, I find it more intuitive in this case than threads/processes.)

In fact, the asyncore manual page has a ~20 line class which implements a web page retrieval. You could replace that example's single call to http_client with five calls, one for each of your ULRs. Then when you enter the last line (that is the asyncore.loop() call) the five will be downloading simultaneously.

See http://docs.python.org/library/asyncore.html

Gary Herron

.

Relevant Pages

  • Re: VB6 - Do I Have It? Do I Need it?
    ... The file you downloaded will extract another single file and place it in the selected location. ... One of my flight simulator add-on programs crashed repeatedly and the software developer suggested I need to reinstall VB6. ... Follow the devoloper's advice, it hurts nothing reinstalling these, it's a small download and installation is usually a breeze. ...
    (microsoft.public.windowsxp.help_and_support)
  • Re: Office 2007 ADM files
    ... To extract the ADM templates see if these steps help. ... download file and note the folder/path location. ... Create a new folder on your PC and note that ... Office 2007 Beta 2 Online Test Drive, Downloadable beta, ...
    (microsoft.public.office.misc)
  • Re: Problem unzipping files on external HD
    ... Remember I had downloaded one of the files (that wouldn't unzip on the ... and this copy of copy of good file fails to extract. ... the folder on the desktop, ... >> I download three zip files, ...
    (microsoft.public.windowsxp.general)
  • Re: uh oh
    ... >This will download the zipped reg file. ... Extract All, and I extracted it to the same folder. ... I opened up the new unzipped folder and double clicked ...
    (microsoft.public.windowsxp.accessibility)