Re: NIO and accepts()

From: Douwe (douwe_at_parkserver.net)
Date: 12/22/03 

Date: 22 Dec 2003 09:25:57 -0800

> >> The classic blocking socket scheme does not scale well and this is
> >> why writing a powerful server in Java wasn't reasonable. I thought
> >> that NIO had been written to solve this problem.
> >
> > Dont know exactly what you mean with scaling
>
> Quoting webopedia : "A popular buzzword that refers to how well a
> hardware or software system can adapt to increased demands."
>
> This has something to do with the asymptotic behaviour of functions as
> well (response time = f(nb flients) ). Typically a system which responds
> in o(n^2) where n is the number of clients isn't scalable while one that
> responds in o(n) is scalable.
>
> In a nutshell the idea is that an increasing number of clients will slow
> down the server but not overwhelm it.
>
> For instance, with less than 50 clients a 1-thread-per-client
> server and a iocp server give almost the same results, with about 2000
> clients the 1-thread-per-client is overwhelmed (it does not respond
> anymore) whereas the iocp server still works.

Thanks for your fine definition ... :)

> > question is why you then created multiple threads ... if their is only
> > one queue that is dispatching the enlisted information one by one to
> > the different Threads you could better implement a single Thread (IMO
> > this is just overkill)
>
> The fact is that worker threads take more time to complete than the
> dispatcher thread to cycle because for example they have to parse a HTTP
> request when it's sent. And it's automatically done by the operating
> system under windows (IOCP server model).
>
> This method has been tried and tested and in multiprocessor environments
> it has been proven to yield a significant performance gain.
>
>
> It is my goal to compare different io strategies and if you're right, the
> benchmarks should show it.

Could be that I´ve misunderstood you ... I thought you had built an
queue-thread that dispatches its actions to different worker threads
one by one waiting for each seperate worker thread (and so generating
a sequential program with multiple threads) ... I thought so because
you wrote

>>>> I've written servers in which only one thread at a time handles a
client.

> >> Then Java lacks an asynchronous accept() method.
> >
> > Would an asynchronous accept help to speed up the initialisation
> > process ?? If you can answer thiw with no then an asynchronous accept
> > doesn´t bring much.
>
> I bet it will since the accept() operation is time consuming but not cpu
> consuming it will allow the system to do something in the meantime.
>
> As I explained in another post, upon accpetance the operating system
> has to allocate a new socket descriptor (which involves thread
> synchronization with the socket subsystem) and perhaps send SYN/ACK
> packets which leaves the cpu with many cyles to spare.
>
> Again, it is my goal to see whether or not this would lead to a gain in
> performance.

Asuming the accept is slow caused by the reasons you described above.
If the new socket has to be synchronized with the sub-system the
Thread handling this will do it calls to the socket subsystem ... go
into a WAITING state ... [subsystem sends SYN and waits for ACK and
maybe does other stuff].... WAKING up again (being signaled by the
subsystem) .. and return from the accept method. As far as I can see
two threads are involved here were the "outer" thread is a Java Thread
and the inner Thread is a system thread (owned by the JVM). The outer
Thread is going into a WAITING state and will not use any CPU cycles.
The inner Thread will (in most cases) go into a WAITING state as well
as soon as it has sent the SYN to the IO-Device/Networkcard and it
will wake up as soon as the IO-Device has new data. The second
(System-)Thread therefor doesn´t consume much time either while being
asleep. This is the situation if the non-asynchronized accept() is
used.

In the asynchronized way their is not much different ... the Selector
could be seen as the first thread ... the second thread stays more or
less the same ... but now the first thread can be notified by multiple
events from different Threads. I even could imaging that after being
notified by one Thread the Selector waits for a few milliseconds in
the hope more threads will notify it (but therefor I would have to
look into the implementation).

In both situations you have the system-thread that does the actual
work, this can not be changed/improved. The first situation is using
an older implementation for IO then the second one but it does not
waste CPU cycles. I can´t tell you which of the two implementations
will be faster that is just trial and error.

> > That indeed doesn´t solve the accept issue ... as far as I can see you
> > don't need to solve the slow accept() initilzing .. all you need to
> > solve is that the slow accept is not interfering with the other
> > clients that are being handled.
>
> ... and with other clients being accepted.

Not sure what you mean but I could be that if two clients are being
accepted at the same moment the Socket implementation will handle the
accepts sequentially ... but this is IMO OS dependent and has nothing
to do with the Java Socket API and neither with the accept being
asynchronized or not.

> > But using a single thread you can not solve this problem.
>
> It is not my goal to use only one thread but an arbitrary number of
> threads that I can change at will to take advantage of multiprocessor
> architectures.

Relevant Pages

  • Re: C sockets
    ... :if no then select is only thing to have server socket program on one ... :pc and other clients on many pcs to have communication with server? ... serving N clients requires at least times the amount of memory ...
    (comp.lang.c)
  • Re: Asynchronous Sockets and the I/O Completion Port Model
    ... > I'm looking to build a TCP based service that will listen for connections ... > It will handle XML messages that are sent by connected clients, ... and point out that the .NET Socket class when used on NT-based ...
    (microsoft.public.dotnet.languages.csharp)
  • Problem with a Socket server Opening/Accepting Many sockets and the GC running.
    ... listening on port 11000 and ready to take a new connection. ... I created the Listener socket, ... THen all is well and my clients can connect for about another 3900 ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: NIO and accepts()
    ... The classic blocking socket scheme does not scale well and this is why ... I thought that NIO ... Even though the maximum number of clients is higher, ... On a 4 CPU machine, I'd typically want to have 8 threads processing IO ...
    (comp.lang.java.programmer)
  • Critical Section limits?
    ... A project I'm working on at present involves a thread pool which services ... clients (IOCP server). ... It's all completely written from scratch and ...
    (borland.public.delphi.nativeapi)
Loading