Architecture ideas for batch processing?

I'm working on an architecture for a fairly large scale system
in Java. User interactions will be through JSP and servlets,
but there is also a big batch processing component with multiple
jobs that can run for minutes to hours, and some of them running
in parallel.

I'm looking for architecture suggestions for the batch job
portion.

Specifically, does anyone have opinions on running jobs as
separate threads in a single virtual machine, vs running
multiple independent JVMs?

A single JVM has the advantage of less system overhead and
easier inter-thread communication - which may or may not turn
out to be useful. But it means that if one job crashes it can
bring the others down with it. It also means that if one job
goes into a loop and has to be terminated, the operator will
have no choice but to kill everything.

Multiple independent jobs looks more robust, but Java's penchant
for swallowing memory may make this significantly less
efficient.

Anyone have experience or opinions on this?

Any other good ideas for someone doing batch processing in Java?

Anyone know of open source or commercial toolkits of special
interest for batch processing?

Thanks.

Alan


.

Relevant Pages

  • Re: Architecture ideas for batch processing?
    ... > I'm working on an architecture for a fairly large scale system ... > jobs that can run for minutes to hours, ... > Any other good ideas for someone doing batch processing in Java? ...
    (comp.lang.java.programmer)
  • Re: Architecture ideas for batch processing?
    ... > I'm working on an architecture for a fairly large scale system ... > jobs that can run for minutes to hours, ... > Any other good ideas for someone doing batch processing in Java? ...
    (comp.lang.java.programmer)
  • Re: new to c# but not to programming
    ... people fail to understand that working as an employee is only a ... "Mark Broadbent" wrote in message ... >> Don't know what it's like in America but in Britain, the jobs market ... >>> that the industry is looking for ...more namely C++, Java, an Msc ...
    (microsoft.public.dotnet.languages.csharp)
  • Java Training For Programmer
    ... So I learned whatever was needed for particular jobs, like Lotus Notes, ... Now, I've started to learn Java several times in the past, but since I ... However, since I'm thinking specifically of broadening my employability, I ... So what I'm looking for, after all that buildup, is an online Java course ...
    (comp.lang.java.help)
  • Re: What is Lisp used for?
    ... My main languages are Perl, Java, and ColdFusion. ... I have been trying to learn Lisp for several years now and have ... What kinds of jobs do you use Lisp for? ...
    (comp.lang.lisp)