Re: STRANGE QUESTION : search engine query and web site dowload

From: igthibau (igthibau_at_wanadoo.fr)
Date: 10/26/03 

Date: Sun, 26 Oct 2003 22:36:16 +0000

Hmmmmm not sure I understood all that, but it looks as though I am embarking
on network programming...
on which I have not a clue.
any book / reference I might find useful ?

thanks
G.

Moonlit wrote:

> Hi,
>
>
> "igthibau" <igthibau@wanadoo.fr> wrote in message
> news:bnfug9$ihe$1@news-reader3.wanadoo.fr...
>> Hello everyone,
>> well, the title says pretty much all.
>> First off though I'd like to say I have no idea what ng I should post
>> this to. I chose this one because I am more likely to understand
>> c-based answers than other language's.
>> So, my problem is simple : to submit a query to a search engine, one goes
> to
>> , say www.google.com, enters keywords and a long list appears. Then one
>> clicks onto the appropriate hyperlink to go to the chosen site.
>> Question : how does one do that using scripts / fortran / C / html /
>> whatnot. As in : using command line interfacing to as to enable, say,
> loops
>> or non-graphical interactivity?
>> The idea is to be able to, within a program, call up a search engine, run
> it
>> using keywords and then download the first X sites.
>
> Well would you should do is create a connection to the server (port 80)
> using a socket. Then you would send information to it and read from it.
> The protocol used is described in the http rfc (google for it). You could
> then create a query and scan the pages for the results. Of course you have
> to look at the pages to see how the results are returned.
>
> There is also a library available somewhere that let you build for
> instance a browser, It has al kinds of http function which might save you
> a lot of time.
>
> I actually wrote an app that talks to our local SMS http server in the
> company to send sms to mobile phones. I just intercepted all trafic from
> my webbrowser to the site and back to see how they communicated, then I
> wrote a class that implemented a very small subset of the http protocol..
> Such an approach might work for you too.
>
>
>>
>> There, I said it. I have no idea where to start from let alone what
> langage
>> to use and would hugely appreciate any help whatsoever !!!
>>
>> thanks to all.
>> G.
>
> Regards, Ron AF Greve.

Relevant Pages

  • Re: STRANGE QUESTION : search engine query and web site dowload
    ... protocol used is described in the http rfc. ... I actually wrote an app that talks to our local SMS http server in the ... class that implemented a very small subset of the http protocol.. ...
    (comp.lang.cpp)
  • Re: Submit button stops working when I update my website
    ... had the right http protocol, and tried a few variations, but nothing. ... "Publisher web publication forms 101": ... what is the difference between HTTP and FTP uploading ...
    (microsoft.public.publisher.webdesign)
  • Re: webHttpRequest doesnt like my netgear router
    ... HTTP protocol standard). ... I cannot snoop packets on this HTTP ... connection because it forces an SSL-encrypted connection. ... It would be interesting to see an actual HTTP message to understand what causes the protocol violation. ...
    (microsoft.public.dotnet.framework)
  • Re: sending personal information
    ... www.oehelp.com/ielnk.aspx There is also an HTTP for privacy. ... That should have the same values as the HTTP protocol. ... > cookies where blocked, ...
    (microsoft.public.windows.inetexplorer.ie6_outlookexpress)
  • Re: connection to server via HTTP & GET/POST method in vc++
    ... This is my first application on socket programming in vc++. ... that he's trying to learn how to do network programming. ... Then he can learn about HTTP from the relevant RFC. ...
    (comp.lang.c)