urllib2/cookies - surely there's a better way ?

From: Richard Shea (richardshea_at_fastmail.fm)
Date: 04/30/04

  • Next message: Nikolai Kirsebom: "Automated installation framework" 
    Date: 29 Apr 2004 23:48:36 -0700

    Hi - I'm writing a script which fetches a page from a web server and
    takes note of any set-cookies which are served in the headers so that
    when I next request a page I can send those cookies back to the
    server. This is so that the usage analysis software on the server
    (based on cookies) will take account of the scripts activities.

    Now the thing is I'm halfway through doing this but I'm thinking there
    must be a more refined mechanism than the one I'm using (see below).

    I'm not really asking if there's a way to smarten up the rather clunky
    splits (although if necessary that would be welcome) I'm more asking
    is there not a more refined interface to the whole area of cookies.

    Strangely enought the doco says that f.info() "return the
    meta-information of the page, as a dictionary-like object" - well as
    far as I can see it's a string and the f.info().headers is a list. I
    don't usually find errors in the doco so this makes me wonder if
    there's something I'm doing fundamentally wrong ?

    Anyway any ideas would be welcome. Here goes with the work in progress
    ...
     

    import urllib2
    from string import split
    from string import upper

    req_headers = {
            'User-Agent': 'Mozilla/4.0 (compatible; MSIE 5.5; Windows
    NT)',
            'Referer':''
    }

    TARGETURL = 'http://www.somedomain.com/a/b/'

    req = urllib2.Request(TARGETURL, None, req_headers)
    f = urllib2.urlopen(req)
    lstHeaders=f.info().headers

    for h in lstHeaders:

            lstHeaderContents = split(h,":",1)
            if upper(lstHeaderContents[0]) == "SET-COOKIE":
                    print lstHeaderContents[1]
                    #get the keyword value pair to the left of the first
    ';'
                    lstYetAnother = split(lstHeaderContents[1],";",1)
                    #put the keyword value pair into a list
                    lstOneMore = split(lstYetAnother[0],"=",1)
                    print "Keyword=" + lstOneMore[0] + ". Value = " +
    lstOneMore[1] + "."

    That's the script - thanks for readind this far.

    regards

    richard.

  • Next message: Nikolai Kirsebom: "Automated installation framework"

    Relevant Pages

    • Re: Javascript code
      ... On serverside you cannot debug clientside script. ... This string on server can be changed dinamically for accept ... RegisterClientScriptBlock MSDN remark: ...
      (microsoft.public.dotnet.framework.aspnet)
    • Re: How to retreive the URL of the chosen (but non-exsisting) page after redirect?
      ... cunning attacker can form a query string which will print a string to ... cookies live on the client. ... Or the web server might not ...
      (comp.lang.javascript)
    • Re: Dynamically switch Javascript
      ... Basically, within an HTML page, I have a tag that looks as follows: ... A call gets made to a server, and it returns a string. ... I have tried using "eval" and also constructing a script node dynamically ...
      (comp.lang.javascript)
    • Re: Parsing NSLOOKUP for Error
      ... defaults to the name server configured on the server or workstation. ... I want to capture the following output after the script runs ... sData = Trim) ... returns 0 if the string is not found, ...
      (microsoft.public.scripting.vbscript)
    • Re: LWP UserAgent form post problem
      ... Cookies may not be a factor but LWP ... I put those in my code and I got back a response from the server, ... when I use them in the script... ...
      (comp.lang.perl.modules)