Re: Is it better to use threads or fork in the following case

grocery_stocker wrote:
Let's say there is a new zip file with updated information every 30
minutes on a remote website. Now, I wanna connect to this website
every 30 minutes, download the file, extract the information, and then
have the program search the file search for certain items.

Would it be better to use threads to break this up? I have one thread
download the data and then have another to actually process the data .
Or would it be better to use fork?


I concur with Diez that I don't think threading/forking will bring significant advantages for this particular case.

That said, if you are thinking from a responsiveness perspective, I would definitely say threading.

If you ask from a performance perspective I would need to know what OS you are running (that is is if forking is even supported) and if you have multiple CPU's and if you are actually planning on spawning that sub-process on a (possibly) a different CPU as the parent process.

So the workflow would be something like this:
Downloading block
If block has enough data to process, spawn a new process (using multiprocessing module) and let it write the result back to x (requiring lock and release).

Things to keep in mind, is the overhead of:
- Multiple interpreters running on the multiple CPU's
- IPC
- Locking/Releasing
Still less then if you would have no threading at all?

About forking, this usually means that the child process starts out as an exact copy of the parent process and runs ideally mostly independent of the parent meaning that the best case would be that the child process can run fine without the presents of the parent process, is this really what you want to do?

--
MPH
http://blog.dcuktec.com
'If consumed, best digested with added seasoning to own preference.'
.

Relevant Pages

  • Re: List of Hotfixes to fix DX10/SLI/GPU/Memory in Gaming
    ... 'KB933590 - Fixes issues when running multiple monitors when playing ... Request Download: ... Download the AMD 64 Multiple Core Optimizer (Enhances performance while ... nHancer - The Advanced Control Panel and Profile Editor for ...
    (microsoft.public.windows.vista.games)
  • WinFlash Educator 10.10.07
    ... Educator offers powerful multimedia support while retaining its ... Choose from standard, fill-in or multiple ... foreign language study, speeding both creation and study phases. ... dictionaries are available for free download. ...
    (comp.software.shareware.announce)
  • Re: Which folders are shortcuts?
    ... "For general shortcuts, I would agree with what you say, but the OP is ... >> for what appears to be multiple entries.. ... >> computer illiterates download stuff and don't know where its gone, ... >> to be in multiple folders, ...
    (microsoft.public.windowsxp.basics)
  • Re: Writing a bare bones Download manager
    ... or can i tune tune wget so that it can download a file streaming man ... i mean to download using multiple connections for each section (say 4 ...
    (comp.programming)
  • Re: Multiple file download using WebClient
    ... For what reason would you prefer raw sockets? ... download methods, but anyways... ... Add threading in and your all set. ... >> The same limitation seems to be put on IE, if I'm using that as Client. ...
    (microsoft.public.dotnet.languages.csharp)
Loading