Re: Extract data using Curl Unix Command & Perl Script from Webpage
From: Bob Walton (invalid-email_at_rochester.rr.com)
Date: 03/07/04
- Next message: Tad McClellan: "Re: Extract data using Curl Unix Command & Perl Script from Webpage"
- Previous message: Ben Morrow: "Re: SOAP::Lite in a Safe ?"
- In reply to: Fiaz Idris: "Extract data using Curl Unix Command & Perl Script from Webpage"
- Next in thread: Tad McClellan: "Re: Extract data using Curl Unix Command & Perl Script from Webpage"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Sun, 07 Mar 2004 20:01:33 GMT
Fiaz Idris wrote:
> I have used curl and perl script to extract data from sequence
> of webpages before.
>
> But, in the following case I couldn't find a way to do it.
>
> So, if someone can guide me a better way or add any comments
> on top of my own to do it would be appreciated.
>
> HOW I EXPECT IT TO BE DONE
> --------------------------
>
> The webpage is the following:
>
> http://www.chennaionline.com/msuniversity/submit.asp?code=BA
>
> and I have to extract the Registration numbers from 2225683 to
> 2225867.
>
> You might want to try out a single number for e.g. 2225683 to see
> the results it returns.
>
> I normally will group all the webpage source of each of the
> registration
> numbers in a single file using something like
>
> $results = qx{curl -s
> http://www.chennaionline.com/msuniversity/result.asp?RegistraitonNumber=$regno};
Accuracy counts------------------------------------------------^^
>
> redirected to a file and then use regular expressions to extract the
> Registration No., Name, College and the marks & results of each
> subject
> for each student.
>
> WHAT I EXPECT FROM YOU
> ----------------------
>
> I can't find a correct way to locate the URL which will return the
> results
> of each Registration Number as it seems to be using JavaScript or
> something.
>
> How can I do it in this case?
The HTML page generating the request indicates it is using the POST
method. Perhaps the CGI script which accepts the request checks to
verify that the POST method was used? In the case of the POST method,
the arguments are not supplied as part of the URL.
>
> If there is a complete alternative to do it. Please guide me.
use LWP::UserAgent;
would be the Perlish way of doing it. See:
perldoc lwpcook
for a tutorial.
>
> I have used the same technique in some other pages and it works like a
> wonder.
Did their forms use the POST method?
-- Bob Walton Email: http://bwalton.com/cgi-bin/emailbob.pl
- Next message: Tad McClellan: "Re: Extract data using Curl Unix Command & Perl Script from Webpage"
- Previous message: Ben Morrow: "Re: SOAP::Lite in a Safe ?"
- In reply to: Fiaz Idris: "Extract data using Curl Unix Command & Perl Script from Webpage"
- Next in thread: Tad McClellan: "Re: Extract data using Curl Unix Command & Perl Script from Webpage"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
