Re: Using JMS to transfer large volumes of data

From: Lew <noone@xxxxxxxxxxxxx>
Date: Mon, 13 Oct 2008 23:15:25 -0400

marlow.andrew@xxxxxxxxxxxxxx wrote:

Has anyone used JMS to transfer very large volumes of data? I am
thinking about several gigs per day, every day. Phew. Whilst it is
possible to write all this data to a logfile and then FTP it, the
events of interest are occuring in several servers at the same time.
FTPing all these server logs is very time consuming. I have in mind
that the data could be trickled over using several queues. Any
thoughts? I wonder if anyone has done this sort of thing before..

I am aware of production systems in use today that use IBM WebSphere and their MQ implementation of JMS to transfer up to gigabytes per transaction, tens of millions per day. Use of multiple servers in that scenario, along with a whole shi--, er, cartload of additional infrastructure, is /de rigueur/ in such a scenario.

I suspect that this is not the only such installation.

I know there are alternatives, as you scale up to this kind of distributed architecture, and that JMS is not the only option. Regardless of the software layer involved, the physical infrastructure and the balanced deployment of multiple disparate components will dominate your considerations.

We speak not yet of redundant systems, uptime, failover, clustering and other technology matters, much less of the human-resource considerations of obtaining the services of competent operations personnel and the concomitant costs.

--
Lew
.

Follow-Ups:
- Re: Using JMS to transfer large volumes of data
  - From: marlow . andrew

References:
- Using JMS to transfer large volumes of data
  - From: marlow . andrew

Prev by Date: Re: NetBeans debugger: suspend and see current stack trace how?
Next by Date: alternative to for loops
Previous by thread: Using JMS to transfer large volumes of data
Next by thread: Re: Using JMS to transfer large volumes of data
Index(es):
- Date
- Thread

Relevant Pages

Re: Thank you for your respsones
... domain or forest admin. ... scenario will apply to all the branches. ... My only concern about where to PUT file servers and DHCP servers and other ... Microsoft Certified Trainer ...
(microsoft.public.windows.server.active_directory)
Thank you for your respsones
... (to be honest i didnt expected so quick response:):):)) ... The same scenario will apply to all the branches. ... Single Domain forest with emtpy root domain ... My only concern about where to PUT file servers and DHCP servers and other Application servers. ...
(microsoft.public.windows.server.active_directory)
Re: XFS: inode with st_mode == 0
... >> So apart from the general well known instability problems that will ... Scenario 1: Mailservers: ... Resolution to the mailserver problem: ... Mentioned errors seen on a total of 17 servers. ...
(Linux-Kernel)
Re: fault tolerant setup SBS2003
... Can I ask a dumb question why you have to have a persistent VPN rather than a wired connection? ... Another issue is that the two servers are in the same building but on different locations in the building without a possibility to have them connected via a LAN. ... Is there any builtin functionality in SBS which can facilitate the connections between the two servers and keep authentication and file servers in-synch with the WIndows built in replication for both AD and Files services. ... Exchange and SQL fault tolerancy is out of scope in this scenarion. ...
(microsoft.public.windows.server.sbs)