Re: Server Advice Wanted.

Eric Grange wrote:

> > I need some advice for 1) hardware 2xComputer with a 8TB SCSI
> > Raid 5 (4TB for work and 4TB Mirroring) One for a day to day work
> > and one for mirroring in case something goes realy wrong with the
> > first to give me a couple of days to respond (travel to
> > customer 500~700KM from my office).
>
> RAID 0+1 will be more efficient (stripping and mirroring), and safer
> in case something goes wrong. It's more expensive on the
> harddisk front though, which is why you see it less than RAID 5.

Yes I can understand the issues between raid 5 and raid 1 and in the
specific installation I want raid 1 for maximum confidense.

> > 2) Tape Streamers capable to back up at least 4TB once or twice a
> > week and take incremental backups with the ability to work with
> > the database server.
>
> Not much issue here with Oracle. If customers allows it, you could
> consider backing up via network to a company specialized in remote
> storage. You would gain a guarantee in case the library burns or
> other physical damage happens to the server room (and avoid the too
> often encountered "nobody cared about the streamer or checked if they
> could still restore from backups because it takes day to do so, so we
> ended up with useless backups")

I already have a second machine for mirroring the first completly as
a network backup solution. I need a solution in which the backup will
not be on a computer for security reasons. They will not accept a
network backup service provider. Even the photographer is not alowed to
take books on the studio to work with he has to build a studio on the
customer's site.

> > 3) Database server capable serving 20~30MB per second on 50
> > concarent users
>
> 20-30 MB looks a lot, you should be able to slash that significantly,
> after all, you never really have to send more pixels than the user can
> see, which is around 1 megapixel at 1280x800, ie. 3 MB uncompressed in
> 24bit colours, with compression, you should easily be able to keep
> those 3 MB under 100 kB, ie. 100 kB/sec something at 1 full-screen
> per second, and for a typical scrolling that replaces 10% of screen
> space, you're at 10 images/sec. Even at 1600x1200 200kB/sec should be
> enough. (and even if you want to transfer everything, you can always
> send what's on the screen first, and stream the rest in background).

Yes indeed is a lot I expect an average of 15 MB per second which I
might be able to drop it even lower If they will give me the time or
accept my recomendations on the issues, but better design for worst
case senario.

> Full resolution images are only useful for exports or printing, which
> should be infrequent if I understood your requirements correctly.

This library is very strange. It consist of old books (at least 150
years old) for the most part with drawings and pictures of varius
theams like the books with paintings. Although a fast first view of
a number of pages might be accepted a high resolution page will
almost always be requested when the user wants to stady a page.

> > I am thinging to use Oracle for this customer any other
> > recomendations with technical links will be appreciated.
>
> In terms of CPU load just about any database should be able to handle
> it, but given the amount of data, Oracle is definetely recommended
> here as it is one of the few really DBs proven with multi-TB database.
> The real performance limitation will probably come from network
> transfers, make sure you have plenty and tht you provide compressed
> data.

I know that is why I have a minimum of gigabit network.

> I would actually recommend something we have for scanned documents
> here, which is to keep multiple images of the same page at different
> resolutions and send lower-res versions quickly, and only after a
> small delay, the high-res version (people often browse back and forth
> a few pages, if you don't have such a low-res mechanism, browsing
> will be slow and put a disproportionnate strain on network & server).
> Depending on how many lower-res versions you have, if you cache them
> in DB tables or generate them statically, this can add 50% to the
> amount of data (well worth it in the end-user convenience department).

I am exploring a number of alternatives for this kind of solution
focusing on dynamic image manipulation mostly to avoid using an other
1.5TB of disk space but I haven't rouled out the static low resolution
images yet. I am running some tests at the moment to help me dicide but
I think that I'll implement both solutions with some kind of usage
statistcs and caching in order to maximize both speed and disk space.


Thank you for your comments appresiated.

Regards
Yannis.
.

Relevant Pages

  • Re: will it work?
    ... automated backups to either internal hd (like raid ... mirroring) or usb external hd. ... If you're planning to do backups of the client computers' hard ...
    (freebsd-questions)
  • Re: RAID 0 or have apps on seperate drive
    ... RAID 0 and I do weekly backups to an external so that should be okay. ... the head would just proceed linearly across the disk. ... I'm wondering if the speed increase from using 2 drives in a RAID 0 ...
    (alt.comp.hardware.pc-homebuilt)
  • Re: RAID 0 or have apps on seperate drive
    ... RAID 0 and I do weekly backups to an external so that should be okay. ... always had this fear that the drives will dump during a backup! ... versus failures while the disk is sitting there. ...
    (alt.comp.hardware.pc-homebuilt)
  • Re: suggestions on network storage
    ... Raid is thoroughly unnecessary. ... We do weekly full backups, ... You do understand that this is a SBS group, ... So, if you've got this non-standard SBS solution running, the server ...
    (microsoft.public.windows.server.sbs)
  • Re: Check list for basic security measures
    ... - unauthorized use of computing resources ... what is the overall monetary loss if you fail to protect something ... what traffic is needed from outside to your network and from your network ... Then backups - to protect your data from sudden loss or corruption. ...
    (comp.security.misc)