Re: parsing website with a "searching...." page
- From: Erwin Moller <since_humans_read_this_I_am_spammed_too_much@xxxxxxxxxxxxxxxx>
- Date: Fri, 06 Apr 2007 16:55:07 +0200
Aaron wrote:
I'm trying to parse a table on a webpage to pull down some data I
need. The page is based off of information entered into a form. when
you submit the data from the form it displays a "Searching..." page
then, refreshes and displays the table I want. I have code that grabs
data from the page using cURL but when I look at the data it contains
the "Searching..." page and not the table that I want. below is the
code i have so far....Thanks in advance for any help.
<?php
$url="http://www.website.com";;
$post_data = array();
$post_data['postvar1'] = "val1";
$post_data['postvar2'] = "val2";
$o="";
foreach($post_data as $k=>$v)
{
$o.= "$k=".utf8_encode($v)."&";
}
$post_data=substr($o,0,-1);
$ch= curl_init();
curl_setopt($ch, CURLOPT_POST,1);
curl_setopt($ch, CURLOPT_HEADER,0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
$result = curl_exec($ch);
curl_close($ch);
$result=explode("\n",$result);
?>
Hi,
The page is probably using JavaScript to give that effect.
Inspect the content of $result to see if this is the case.
Look for divs that are not visible and made visible when the page loads (end
of script, or an onLoad event).
The data you want might very well be inside the $result.
If not, give more information WHAT the $result contained.
Regards,
Erwin Moller
.
- Follow-Ups:
- Re: parsing website with a "searching...." page
- From: Aaron
- Re: parsing website with a "searching...." page
- References:
- parsing website with a "searching...." page
- From: Aaron
- parsing website with a "searching...." page
- Prev by Date: |OT| Re: Windows to Unix text doc parsing problem
- Next by Date: Re: Handling GET parameters in new Zend Framework 0.9.1
- Previous by thread: parsing website with a "searching...." page
- Next by thread: Re: parsing website with a "searching...." page
- Index(es):
Relevant Pages
|
