Re: Inconsistent result from urllib.urlopen


Laszlo Nagy wrote:
Any ideas why I don't get the same result from the python script as I
do from a web browser? This problem seems to be a recent
development. The scripts I wrote like this worked fine for a while
and then stopped working within the past couple of weeks.

Maybe it has to do something with your user agent string. The server
side can decide to return a different content when your user agent is
not 'mozilla', 'internet explorer' or 'opera' etc.

Do you want to know how to change your user agent string? Google for
it.... :-)

Laszlo

Thanks. That is the fix I needed. I added

urllib.URLopener.version = 'Mozilla/5.0 (Windows; U; Windows NT 5.1;
en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)'

as the second line of code and now it is actually getting content, not
just an error message. It's not the exact same format as you get from
saving the page from the web browser, but all the links and image
addresses are in place.

.

Relevant Pages

  • Re: Inconsistent result from urllib.urlopen
    ... The scripts I wrote like this worked fine for a while ... and then stopped working within the past couple of weeks. ... Maybe it has to do something with your user agent string. ... The server side can decide to return a different content when your user agent is not 'mozilla', ...
    (comp.lang.python)
  • Re: Check if .NET Framework installed on web browser.
    ... machine resources such as the registry. ... more than just Windows can ... > Microsoft Internet Explorer sends a user agent string contains .NET ... > framework installed on a web browser? ...
    (microsoft.public.dotnet.framework.aspnet)
  • Using scripts in Media Player 8
    ... When using Windows Media Player embedded in a web browser (installed ... WMP is version 8.x), scripts do not necessarily always run. ... URL scripts through a Windows Media Encoder to client browsers. ...
    (microsoft.public.windowsmedia.player.web)
  • Re: How to use Perl for API testing
    ... For web browser automation, use Mozilla web browsers, such as Mozilla ... LWP::Simple; ## (Simple User Agent) ... HTML::TreeBuilder; ## parser ...
    (perl.beginners)
  • Re: Inconsistent result from urllib.urlopen
    ... do from a web browser? ... The scripts I wrote like this worked fine for a while ... and then stopped working within the past couple of weeks. ... Gabriel Genellina ...
    (comp.lang.python)