Re: Writing a bare bones Download manager

Phlip wrote:
parser wrote:
What i mean to say is that "is there alternative for GNU/Linux"?
or can i tune tune wget so that it can download a file streaming man
parts if it all at once ( using multiple threads )?

Then one of us doesn't understand TCP/IP very well.

Correct. However, its not the one of you suspect.

How can running multiple threads make your wire work harder?

Because 1) the "wire" does not demand a single sequential
communications thread, 2) requests to "get" something usually result
in much more downloaded than uploaded and 3) On the internet you can
target multiple machines as servers at once.

[...] Given one wire, your packets must still
travel one at a time, in single file, regardless how many sockets they go
to.

But you don't have to wait for responses in single file, or send the
packets to only one thread at a time. This is how Firefox and Opera
are able to download so many pages at once, with far less of a speed
hit than you would expect.

For example, I hate written my own wget-like application which can grab
some 1600 IMDb web pages in a minute or two (and that's intentionally
slowed down so the IMDb people don't think I am DOSing them.) From
other link aggregator sites I can pull down their target URLs even
faster (so long as they are all different servers.)

--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/

.

Relevant Pages

  • Re: Writing a bare bones Download manager
    ... or can i tune tune wget so that it can download a file streaming man ... parts if it all at once (using multiple threads)? ... target multiple machines as servers at once. ...
    (comp.programming)
  • Re: downloading a single file using multiple threads
    ... i'm downloading a single file using multiple threads... ... you can't *download* a file in chunks, unless you implement a client/server protocol for this. ... The question is - what makes you think you need multiple threads to *download* a file? ... The bottleneck will always be the network, so if you download the file in one chunk, you'll get the maximum throughput, introducing multiple threads will actually slowdown the whole process. ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Download problems! Help Please
    ... > I can not download a single file from the internet. ... Netscape-Style Plug-ins Do Not Work After Upgrading Internet Explorer ... Remove the plugin with Regedit.exe ...
    (microsoft.public.windows.inetexplorer.ie6.browser)
  • Re: Any downlaod tool that can download a file concurrently?
    ... That split a file to several parts, and download it concurrently. ... I don't understand why you want multiple threads. ... or by server response time. ... Please use the corrected version of the address below for replies. ...
    (comp.unix.bsd.freebsd.misc)
  • Re: Fastcode MM - Nexus Validation
    ... > but running the third benchmark from the top is enough. ... Third one from top is 'Block downsize test' and the resulting file ... if I want to run all tests do I need to download every ... single file to download --please. ...
    (borland.public.delphi.language.basm)
Loading