problem/help with using LWP
From: Richard Bell (rbell01824_at_earthlink.net)
Date: 03/28/04
- Next message: Fred Ma: "Re: Choosing Perl/Python for my particular niche"
- Previous message: Garry Heaton: "Quality control for CPAN?"
- Next in thread: Bob Walton: "Re: problem/help with using LWP"
- Reply: Bob Walton: "Re: problem/help with using LWP"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Date: Sat, 27 Mar 2004 23:43:29 GMT
I'm trying to use LWP to scrape some data from a web site fronting a
database of some sort (don't really know/care what). To use the site
from a browser you enter (at some point you get a session cookie, not
really sure when) and proceed to navigate through many pages (they are
all frames with the top part having various navigation widgets and the
bottom part having both visible and hidden forms - the hidden forms
are used to hold data being entered through the process). As each
page is navigated, the site sends various cookies (many) and refreshes
the hidden form. When all is said and done a form is POSTed to
retrieve the data (data is actually a .csv file, but that doesn't
matter). The navigation to the final POST always creates the same
hidden form data. I've used wsp and template.pl to figure out what is
happening and build a perl app that works, at least until the session
cookie expires!! That's the rub.
How do I find out which of the MANY cookies is the ONE that I must
refresh?
How do I refresh it?
Thanks for any help.
R
- Next message: Fred Ma: "Re: Choosing Perl/Python for my particular niche"
- Previous message: Garry Heaton: "Quality control for CPAN?"
- Next in thread: Bob Walton: "Re: problem/help with using LWP"
- Reply: Bob Walton: "Re: problem/help with using LWP"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Relevant Pages
|
