Re: Database vs Data Structure?

On 2008-04-18 05:37, erikcw wrote:
Hi,

I'm working on a web application where each user will be creating
several "projects" in there account, each with 1,000-50,000 objects.
Each object will consist of a unique name, an id, and some meta data.

The number of objects will grow and shrink as the user works with
their project.

I'm trying to decided whether to store the objects in the database
(each object gets it's own row) or to use some sort of data-structure
(maybe nested dictionaries or a custom class) and store the pickled
data-structure in a single row in the database (then unpickle the data
and query in memory).

A few requirements:
-Fast/scalable (web app)
-able to query objects based on name and id.
-will play nicely with versioning (undo/redo)

Any input on the best way to go?

Relational databases offer the best scalability and reliability,
so I'd go for those as backend.

If your data is mostly read-only and usually only touches
a small part of your database (e.g. the data for a day or two),
then you can increase query performance by keeping that part in
an in-memory database (e.g. use sqlite or Oracle TimesTen)
and only update the copy in case something changes in the
backend.

Regards,
--
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source (#1, Apr 18 2008)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
.

Relevant Pages

  • Re: Split database, modified backend, imported and having record p
    ... query joins together multiple tables. ... DriverID in tblDrivers for every value of TransDriverID in tblTransports, ... I modified the backend and added the four fields... ... frontend into "actual" database. ...
    (microsoft.public.access.externaldata)
  • Re: DBMS and lisp, etc.
    ... Naively implemented with SQL, again for 10 ... (1 query for the initial orders, 1 query for each order for its ... soon as you upgrade to the SQL database. ... (eq (order-customer orderA) ...
    (comp.lang.lisp)
  • Re: OT: SQL
    ... query processing. ... FROM Employees e, Employees m, Management mgt ... Manager and Employee Salaries. ... The scheme used does not model database files in general, ...
    (sci.logic)
  • Re: Split database, modified backend, imported and having record p
    ... You say there are 17612 rows in the table, and only 16548 in the query. ... DriverID in tblDrivers for every value of TransDriverID in tblTransports, ... I modified the backend and added the four fields... ... Are you not allowed to modify a table on split database? ...
    (microsoft.public.access.externaldata)
  • Re: access 2003
    ... I removed the parameters from the form query source. ... boxes from the form header, events, code, etc and ran the form query source ... forms queries and the SQL because syntax of the SQL will change randomly. ... the Access 97 database, I wouldn't have thought any expressions would be ...
    (microsoft.public.access.conversion)