Re: Question about 'remote objects'

"Frank Millman" <frank@xxxxxxxxxxxx> writes:

Hi all

I am writing a multi-user business/accounting application. It is getting
rather complex and I am looking at how to, not exactly simplify it, but find
a way to manage the complexity.

I have realised that it is logically made up of a number of services -
database service with connection to database
workflow engine for business processes
services manager to handle automated services, such as web services
client manager to service logins from client workstations
possibly others

I have made a start by splitting some of these off into separate modules and
running them in their own threads.

I am concerned about scalability if they are all running on the same
machine, so I am looking into how to enable these services to run on
separate servers if required.

Have you finished the application already?

At my job we're still serving just over 1M+ web requests (a month),
processing 15k+ uploads, and searching through over 5M+ database records
a day. We're still running on 3 boxes. You can get a lot out of your
machines before you have to think about the complex task of
scaling/distributing.


My first thought was to look into Pyro. It seems quite nice. One concern I
had was that it creates a separate thread for each object made available by
the server. My database server creates separate objects for each instance of
a row read in from the database, and with multiple users running multiple
applications, with each one opening multiple tables, this could run into
hundreds, so I was not sure if that would work.

It probably will work.

Pyro is a very nice framework and one that I've built a few applications
on. It has a lot of flexible design patterns available. Just look in
the examples included with the distribution.


Then I read that the multiprocessing module allows processes to be spread
across multiple servers. The documentation is not as clear as Pyro's, but it
looks as if it could do what I want. I assume it would use processes rather
than threads to make multiple objects available, but I don't know if there
is a practical limit.

There is a theoretical limit to all of the resources on a machine.
Threads don't live outside of that limit. They just have a speedier
start-up time and are able to communicate with one another in a single
process. It doesn't sound like that will buy you a whole lot in your
application.

You can spawn as many processes as you need.


Then I thought that, instead of the database server exposing each object
remotely, I could create one 'proxy' object on the server through which all
clients would communicate, and it in turn would communicate with each
instance locally.

That felt more managable, but then I thought - why bother with remote
objects at all? Why not just run a SocketServer on the database server, and
design a mini-protocol to allow clients to make requests and receive
results. This is a technology I am already comfortable with, as this is how
I handle client workstation logins. If I did go this route, I could apply
the same principle to all the services.

Because unless you wrote your own database or are using some arcane
relic, it should already have its own configurable socket interface?


I don't have the experience to make an informed decision at this point, so I
thought I would see if there is any consensus on the best way to go from
here.

Finish building the application.

Do the benchmarks. Profile. Optimize.

Find the clear boundaries of each component.

Build an API along those boundaries.

Add a network layer in front of the boundaries. Pyro is a good choice,
twisted is also good. Roll your own if you think you can do better or
it would fit your projects' needs.

Is there any particular benefit in using remote objects as opposed to
writing a SocketServer?

Abstraction. Pyro is just an abstraction over an RPC mechanism.
Nothing special about it. Twisted has libraries to do the same thing.
Writing your own socket-level code can be messy if you don't do it
right.


Any advice will be much appreciated.

Thanks

Frank Millman

Best of luck.
.

Relevant Pages

  • Re: Help with first VB application - Data Entry form
    ... I assumed a desktop / winform client application ... time' stamp from the database machine - control machine ... ... problem solved - web server is control system. ...
    (microsoft.public.dotnet.languages.vb)
  • Re: Help with first VB application - Data Entry form
    ... I assumed a desktop / winform client application ... time' stamp from the database machine - control machine ... ... problem solved - web server is control system. ...
    (microsoft.public.dotnet.languages.vb)
  • Re: Remobjects v KBM
    ... >> client query components) follow from that. ... Then, connections can be created to say SQL Server, Oracle, Interbase and ... can then be created from the abstract dataset definition in 'customers' to ... implicitly - this makes your code not be database connection specific). ...
    (borland.public.delphi.thirdpartytools.general)
  • Re: Help with first VB application - Data Entry form
    ... stamp from the database machine - control machine ... ... unnecessary data to the client ... ... and when building a database independent UI / Client - Server application, ... JavaScript, for example) and thus, will get the time from the web server, ...
    (microsoft.public.dotnet.languages.vb)
  • Re: Opinions needed about the best "Middleware suite" kbmMW vs. RODA
    ... kbmMW supports cross db in such way that all you need to do in your application is to set one property to switch to ... What one have to concentrate about is minimizing the amount of data moved from the app server to the client. ... C/S setup's usually have a quite active chatter going on between the client and the database, ...
    (borland.public.delphi.thirdpartytools.general)