Re: Web Scraping Login Required

In article <1188476317.240151.144430@xxxxxxxxxxxxxxxxxxxxxxxxxxxx>,
<schlenk@xxxxxxxxxxxxxxxx> wrote:

Mike Labedz wrote:
I'm trying to scrape my play list off of Yahoo Music and store it.

I've read http://wiki.tcl.tk/2915 and understand the retrieval/parsing
side of a web page. What I'm missing is whether this is possible for
a site that requires a login? If so does anyone have any examples or
recommended packages to look at.

Thank you in advance
Usually this uses http authentication. http://wiki.tcl.tk/6087 is
relevant here, but you might need some cookie support too and if the
site uses vast amounts of javascript to handle login (ajax style) you
might be left with a mess like scripting IE or firefox to do the
download.
.
.
.
On yet another hand, some sites whose implementation involves
a mass of JavaScript still have branches that require no more
than, for example, basic HTTP authentication. It's thought-
demanding reverse engineering, at that point, and, while easy
enough for those with experience, a challenge to explain
briefly.
.

Relevant Pages

  • RE: Secure Login Form
    ... I developped an extranet/internet website for my former companys, ... Subject: Secure Login Form ... javascript, etc.) but I am concerned about the security provided. ...
    (Security-Basics)
  • XMLHTTP.Open(...) causes "Permissions Denied" error
    ... I have an ASPX web app I'm developing at home. ... Login redirects to Main.aspx as it's supposed to. ... The client javascript fires off and works great. ...
    (microsoft.public.scripting.jscript)
  • RE: password cracking a web form, tried hydra and brutus
    ... The problem is you're trying to use HTTP authentication, ... requests just as would happen if you were to sequentially try various login ... > I came across Hydra and Brutus. ...
    (Pen-Test)
  • Re: php ajax redirect
    ... > I have a Login page that checks a Db for a Un and Pw using ajax. ... I just want to redirect out to a new page and keep my _session ... but new at AJAX. ... If you're getting the output with an javascript request, sending redirect headers won't help you changing the current page. ...
    (comp.lang.php)
  • Re: Frames and response.redirect
    ... javascript for the command would be so I doubt what i put would work). ... > I have a login page and when the program accepts the login ... > the page is redirected to a page with 3 frames. ... > redirect to the Login page. ...
    (microsoft.public.dotnet.framework.aspnet)