Re: minimal httpd response

From: Michael Wojcik (mwojcik_at_newsguy.com)
Date: 02/25/04


Date: 25 Feb 2004 22:09:14 GMT


In article , "Martijn" <subscription_remove_101@hot_remove_mail.com> writes:

[Please preserve proper attributions when following-up.]

> > >> The point is, when I use direct telnet, everything works fine, but
> > >> when I use links, it chokes and tells me the following:
> > >
> > >> Alert!: Unexpected network read error; connection aborted.

"connection aborted" is probably lynx's way of stating that its recv()
call, or equivalent, returned -1 with errno set to ECONNABORT. That
means the peer (or some intermediary, such as a firewall) sent a TCP
RST (Reset), abortively closing the connection.

> > > I built a toy httpd server once and alot of browsers got PO'd if I
> > > closesocket the connection after sending the headers. They wanted the
> > > socket to linger. Perhaps that is your culprit?
> >
> > I should qualify that some more, linger is an option you set on the
> > socket, you should still closesocket of course. Forgive me it was a long
> > time ago...
>
> Hmmm, I honestly don't know where to look for this one.

I wouldn't bother; if you aren't familiar with the SO_LINGER socket
option, you're probably not using it. The default behavior for all
the BSD sockets-based TCP application APIs is to do a normal close,
which lingers (the connection teardown is negotiated by the peers,
with exchanges of FIN and ACK packets).

Never use SO_LINGER, by the way. (When you know enough about TCP to
know when to violate this rule, you may know enough to violate it
safely and correctly.)

> Check the man pages
> for some of the commands I use (in C), but haven't found anything. Any
> pointers?

Well, from your original post, I can make a couple of guesses.

  Looking up proxy.xxxxxxx.nl:3128
  proxy.xxxxxxx.nl:3128
  Making HTTP connection to proxy.xxxxxxx.nl:3128

lynx is going through an HTTP proxy.

Your telnet command and output:

   $ telnet localhost 12000
   Trying 127.0.0.1...
   Connected to localhost (127.0.0.1).

There's no proxy involved there; you're connecting directly to your
server. The lynx and telnet cases are completely incommensurate.

The responsibilities of an HTTP/1.1 proxy, and of an HTTP/1.1 server
when receiving a request from a proxy, are complex. Read RFC 2616.

I *strongly* recommend that you get your server working with a real
user agent (browser), without a proxy, first, and then try to deal
with the proxy if you must.

The "minimal set of required headers" is given in the appropriate
RFC: 2616 for HTTP/1.1, 1945 for HTTP/1.0. In HTTP/1.1 the required
headers for a request (as generated by a user agent - proxies and
gateways have to add "Via") are:

   Host (RFC 2616 14.23)
   User-Agent (* RFC 2616 14.43)

(*The User-Agent header is only required for fully-compliant HTTP/1.1
user agents. Conditionally-compliant user agents can skip it, though
it's nice to put it in.)

The headers required for a response (as generated by an origin server)
in HTTP/1.1 are more complicated. For example, if a server implements
a cache, and it's returning a response generated from the cache, it
has to include an Age header. If a server is providing a content-body,
it has to indicate the end of that content body in one of three ways:
by terminating the connection, in which case it has to include a
Connection header; with a Content-length header; or with the chunked
transfer-encoding, in which case it has to include a Transfer-encoding
header. And so on.

HTTP/1.0 is another ball of wax entirely. I do more work with
HTTP/1.1, so I'm not so familiar with HTTP/1.0's requirements and
don't feel like pouring over them. Read RFC 1945 if you want to use
HTTP/1.0. For a toy HTTP server - not an HTTPD server, by the way;
httpd is the name of some HTTP servers - HTTP/1.0 is definitely
easier, since it has far fewer compliance requirements. The same
applies to some special HTTP servers, eg ones embedded in appliances,
which don't have to deal with many clients or provide a lot of fancy
features. Real, general-purpose HTTP servers should be HTTP/1.1-
compliant.

Getting back to your telnet example: you send a request which
includes two Accept headers, an Accept-Encoding header, an
Accept-Language header - none of them necessary for this test. (Does
your server even do anything with them?) Then you send User-Agent;
that's fine, though in this case it's a lie. (I'd use "User-Agent:
telnet", if I bothered to send anything.) Then you send a Via header
which is completely bogus - you're connected directly to your server.
 Ditto the X-Forwarded-For header, though since that's not a
registered header it may not matter; the server can treat it in any
way it wants. The Host header you send is probably OK, inasmuch as
the port is correct; I'm assuming that "www.xxxxxxx.nl" is the
canonical hostname of your local host. (The Host header must be set
to the hostname as it is given in the URL from which the request is
generated. In this case you're generating the request in your head
from some URL only you know, so I can't be sure.) The rest of the
headers are OK.

The response your server sends:

   HTTP/1.0 200 OK
   Content-Length: 8
   Content-Type: text/html

   test

Content-length looks OK, though I'm not sure there are actually 8
bytes of content-body there. (Impossible to tell, since you don't
show us a real trace.) In any case, your server closed the
connection after sending the content-body; this is legit for
HTTP/1.0. (It's OK for HTTP/1.1 also, except that you'd have to put
a "Connection: close" header in the response.)

Content-Type is a lie. That's not (well-formed) HTML. It's well-
formed plain text, so I'd use text/plain. However, that's unlikely
to bother any real user agent.

If your response's content-body is not in fact 8 bytes long - that's
8 bytes *after* the blank line separating it from the headers - that
might bother some user agents. (I don't know how picky lynx is.) If
your response is ill-formed - in particular, if each line of the header
isn't terminated with CR LF, and if the separator between header and
content-body isn't a CR LF on a line by itself - that also might be a
problem.

I suspect proxying is the real issue here, though. Either disable
lynx's proxy and connect directly, or use another user agent. The
libwww distribution from CERN includes a very simple command-line one
called "www", for example.

Oh, and comp.programming probably isn't the best group for HTTP
questions. comp.infosystems.www.servers.misc is probably best;
failing that, comp.protocols.tcp-ip sees more than its share of
HTTP-related questions.

-- 
Michael Wojcik                  michael.wojcik@microfocus.com
Duck: No secret what's worth a hoot ought to be kept quiet.
Pogo: Secrets is usually perty doggone fascinatin'.
Duck: Egg-zackly ... it's completely illogical to keep a secret secret.
Pogo: An' unfair.					-- Walt Kelly


Relevant Pages

  • Re: How come this doesnt work?
    ... Specify the correct Content-Type for it ... Use the "Connection: Keep-Alive" header to be able to do multiple HTTP-Requests on the same connection. ... Supply a correct Content-Length header, so the server knows where the next request starts. ...
    (comp.lang.ruby)
  • Re: What is this code waiting for?
    ... and the web server is sat waiting for another request. ... the server times out the connection - and your read method returns ... open by default for further requests. ... If you send that header then the server ...
    (comp.lang.java.help)
  • Re: linux socket programming and HTTP Protocol Problem
    ... Why are you sending an *empty* content length header? ... Both web browsers connect to the server but both fail ... closing the connection before they've even *sent* their query. ...
    (comp.os.linux.development.apps)
  • Custom HTTP header
    ... I want a simple http header like the one below. ... Server: Mongrel 1.1.5 ... Connection: close ...
    (comp.lang.ruby)
  • IIS5 Dropping HTTP/1.0 HTML Connection
    ... without the keep-alive header given to it from the client end. ... We need a server side fix, the GET command will return the header and then ... drops the connection, just like if it was a HEAD command sent to it. ...
    (microsoft.public.inetserver.iis)