download pages using a cookies.txt file



Hi,

I would like to download a page that requires authentication with
cookies. I've already figured out how to do it with wget (blog post
here: http://bit.ly/pp25LP). In short:
(1) extract cookies from Firefox's cookies.sqlite and store them in a text file
(2) wget --cookies=on --load-cookies=cookies.txt
--keep-session-cookies "http://...";

My question is: how to do it from Python? If possible, I want to avoid
making external calls to wget.

Thanks,

Laszlo
.



Relevant Pages

  • Re: wget to fully simulate browsers
    ... I tried to use wget to download a web page ... I tried use use-agent option to simulate as IE or firefox, ... Do those browsers prompt you for a username & password? ... cookies either so if you're logged in, ...
    (comp.os.linux.misc)
  • Re: download pages using a cookies.txt file
    ... I've already figured out how to do it with wget (blog post ... extract cookies from Firefox's cookies.sqlite and store them in a text file ... how to do it from Python? ...
    (comp.lang.python)
  • Re: download pages using a cookies.txt file
    ... I would like to download a page that requires authentication with ... I've already figured out how to do it with wget (blog post ... extract cookies from Firefox's cookies.sqlite and store them in a text file ...
    (comp.lang.python)
  • Re: [Full-disclosure] [Webappsec] Paper: Weaning the Web off of Session Cookies
    ... Always good to try and raise the bar, but the world has voted cookies ... Great writeup of the state of the union for Web-based authentication methods. ... your paper is primarily an argument for fixing HTTP ... "If We Wean the Web Off of Session Cookies, This Is Some of What We'd ...
    (Full-Disclosure)
  • Re: [Webappsec] Paper: Weaning the Web off of Session Cookies
    ... Always good to try and raise the bar, but the world has voted cookies ... Great writeup of the state of the union for Web-based authentication methods. ... your paper is primarily an argument for fixing HTTP ... "If We Wean the Web Off of Session Cookies, This Is Some of What We'd ...
    (Bugtraq)