Re: Architecture ideas for batch processing?

Alan Meyer wrote:
> I'm working on an architecture for a fairly large scale system
> in Java. User interactions will be through JSP and servlets,
> but there is also a big batch processing component with multiple
> jobs that can run for minutes to hours, and some of them running
> in parallel.
>
> I'm looking for architecture suggestions for the batch job
> portion.
>
> Specifically, does anyone have opinions on running jobs as
> separate threads in a single virtual machine, vs running
> multiple independent JVMs?
>
> A single JVM has the advantage of less system overhead and
> easier inter-thread communication - which may or may not turn
> out to be useful. But it means that if one job crashes it can
> bring the others down with it. It also means that if one job
> goes into a loop and has to be terminated, the operator will
> have no choice but to kill everything.
>
> Multiple independent jobs looks more robust, but Java's penchant
> for swallowing memory may make this significantly less
> efficient.
>
> Anyone have experience or opinions on this?
>
> Any other good ideas for someone doing batch processing in Java?
>
> Anyone know of open source or commercial toolkits of special
> interest for batch processing?

Multi-JVM. If you worry about crash, multi-process is the only solution
- even .NET's Application Domain cannot ensure 100% isolation inside a
process.

memory problem could be solved later, but a single-process solution has
failed since the beginning.

However, most developers of java application servers wouldn't agree
with me :-)

.

Relevant Pages

  • Re: Architecture ideas for batch processing?
    ... > I'm working on an architecture for a fairly large scale system ... > jobs that can run for minutes to hours, ... > Any other good ideas for someone doing batch processing in Java? ...
    (comp.lang.java.programmer)
  • Architecture ideas for batch processing?
    ... I'm working on an architecture for a fairly large scale system ... does anyone have opinions on running jobs as ... Any other good ideas for someone doing batch processing in Java? ...
    (comp.lang.java.programmer)
  • Re: new to c# but not to programming
    ... people fail to understand that working as an employee is only a ... "Mark Broadbent" wrote in message ... >> Don't know what it's like in America but in Britain, the jobs market ... >>> that the industry is looking for ...more namely C++, Java, an Msc ...
    (microsoft.public.dotnet.languages.csharp)
  • Java Training For Programmer
    ... So I learned whatever was needed for particular jobs, like Lotus Notes, ... Now, I've started to learn Java several times in the past, but since I ... However, since I'm thinking specifically of broadening my employability, I ... So what I'm looking for, after all that buildup, is an online Java course ...
    (comp.lang.java.help)
  • Re: What is Lisp used for?
    ... My main languages are Perl, Java, and ColdFusion. ... I have been trying to learn Lisp for several years now and have ... What kinds of jobs do you use Lisp for? ...
    (comp.lang.lisp)