Re: FORM data in cgi



Mike Meyer wrote:
"jponiato" <jponiato@xxxxxxxxxxxxxxxx> writes:


Greetings.
An HTML form submits it's data to a python cgi script on my server. This script accepts this POST data, and uses urllib.urlopen() to call a different cgi script (on an external server), passing this same data. I'm using cgi.FieldStorage() to create a mapping of the FORM data, then using urllib.urlencode() to turn it back into form data for urlopen().
Question - is there a more efficient way to do this?


Yes. But the question you should be asking is "Is there an easier way
to do it that's worth doing?"

Unless you're passing around files, or really huge forms, the amount
of time you spend doing decoding and encodinng the form data will be
pretty trivial. Unless you're really pressed for cycles, why bother
fixing it? And if you're really pressed for cycles, you should start
by instrumenting things to make sure that you're optimizing something
that will do you some good.

Anyway, the general idea is to skip cgi.FieldStorage, and parse the
request yourself. You'll have to deal with the headers. But you can
just grab the post data with a read(). I'm not sure you can use urllib
to send pre-encoded POST data; you'll have to check that yourself. If
not, you'll have to do the HTTP request processing by yourself. That's
not hard, though.

<mike

In point of fact the OP (whose post wasn't threaded with your reply in my newsreader) is actually describing a requirement for an HTTP proxy!


As you describe it, since the data stream in the request body will be exactly the same in both cases (though the HTTP request line and headers will differ) is possible to handle the request by having the CGI script a direct connection to the intended destination server, parsing the incoming headers, generating the outgoing ones, and then just relaying the request body. The response will then have to be sent back to the client, possibly requiring some parsing on the way, so the processing time might come out a wash.

Of course it would be even easier to set up direct proxying through the web server if it's something accommodating like Apache ...

regards
 Steve
--
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/

.



Relevant Pages

  • POST from a CGI
    ... I'm receiving POST data to a CGI, which I'd like to forward to another ... CGI using urllib2. ... getting 400 errors when I try it (server thinks request is malformed). ... I've checked the headers and the body data and they seem normal and I ...
    (comp.lang.python)
  • Re: XMLHTTP setRequestHeader method fails in IE6?
    ... method, url, async parameter to open the HTTP request, then call ... setRequestHeader to set the headerfor the opened request. ... isn't the setting of the request headers supposed to be ...
    (comp.lang.javascript)
  • Re: XMLHTTP setRequestHeader method fails in IE6?
    ... method, url, async parameter to open the HTTP request, then call ... setRequestHeader to set the headerfor the opened request. ... isn't the setting of the request headers supposed to be ... I am pretty stumped with the 'setRequestHeader' method. ...
    (comp.lang.javascript)
  • Re: HTTP - basic authentication example.
    ... or *never* knowing the realm..) ... This is called authentication and is implemented ... requests a web page it sends a request to the server. ... consists of headers with certain information about the request. ...
    (comp.lang.python)
  • [NT] 04WebServer Multiple Vulnerabilities (CSS, Log File Injection, AUX DoS)
    ... 04WebServer is a HTTP server developed by Soft3304 for Windows platforms. ... Characters into Log File ... filtering on the request URL before writing it into the log file. ... following HTTP request, when submitted to a vulnerable 04WebServer, will ...
    (Securiteam)