Re: Server Advice Wanted.
- From: "Brett Watters" <bwatters@xxxxxxxxxxxxxxx>
- Date: Thu, 8 Sep 2005 11:33:34 -0700
Yannis,
> My experience with CDs is very different than what you
> describe. In most case I expect a maximum of 3~4 years
> per disk on a mildle used disk on the other a hard drive
> will give me around 8~10 years of life before the troubles
> begin. I do not have any experience with CD/DVD jukeboxes
> (no scratchs better handling etc) so I cant comment on that
> really.
I was talking storage time here. If you take the backup
and put it on a self. DVDs say 50-100 years. Tape says
10 years.
In terms of actual usage... hard drives and tapes obviously
fail more often than DVDs since a) the DVD is read-only,
b) it there isn't physical contact, and c) there are no moving
parts. Again, I'm only talking about the physical media.
> My concern is not how hard it is for me to do it
> it is how hard the customer will perceive it is.
> I am dealing with people that know almost nothing
> about computers and they are not very willing to
> learn so my software needs to be as simple as possible
> and the use of external programs or procedures that
> require knowledge of the OS at zero level if possible.
Your system is likely to run into the hundreds of thousands
of dollars. Some level of knowledge is going to be required.
I don't know of too many 3TB systems with 50 workstations
which don't require some maintanance.
> I can have the photographers do the writting on
> DVDs as they scan the images they have required
> knowledge this is not my problem.
Ok.
> This process will probably require me to go at
> the customer at least once a year. This is no
> problem since my contruct has a number of visits
> on the customer per year for maintance reasons
> but if this becomes once per month it would be
> prooven problematic for me.
If someone else is going the scanning, you should
just have the company save up several months
worth of books and have the someone else send
them the DVD. Just insert it, go to a panel, and tell
it to 'Load' disk 681. It could just scan it, read the
info into the DB, and continue.
> That is correct but at the same time I need to
> minimize the wait time as far as possible.
> I am considering some kind of caching mechanism
> in order to avoid hitting the DVDs all the time.
I've already suggested the caching system, which
is about the only purpose of the HDs - other than
the DB. The issue is how much caching is needed
and cost effective. Whether you use 250MB,
500MB, or 1TB, for caching depends upon
usage. You won't know this until you start
trying it. However, at some point, 5%, 10%,
or 20%, you'll likely need see much cost
effectiveness in adding more space. If they
have electronic check-out systems, you
might get an idea of what percentage of
books are checked out frequently.
BTW. If you have workstations, you could
also consider caching individual pages locally.
Since you have to move them across from the
network anyway.
However, even if you put the entire 3.2TB
on disk, having the jukeboxes is still a far
better system than a tape backup.
a) The information is static.
b) Weekly backups of 3.2TB systems is
unnecessary and time consuming.
c) Restoring a 3.2TB backup is painful.
d) The DVD jukeboxes mean that the
backup is always available in a form
which can be accessed.
e) The DVDs make an easy system
to store backups off-site.
f) DVDs are a cheeper backup media
than tape -- considering they last 50-100
years vs. tapes which are rated for 10.
> I know I haven't any usage statistics yet but from
> what it has been described I need a backup once a week
> and I have a 4 tape sets so a lost of one weeks work
> is the worst case senario and only if both systems will
> fail at the same time and this is going to happen only
> from external sources (Physical disasters, electrical
> porblems etc) in this case I think that a lost of a weeks
> work is at least acceptable.
Why once per week? If the books aren't changing,
the only information changing is the DB. You shouldn't
need to be backing up this amount of information.
> That is why I need raid 0 and Raid 1 raid 0 is the
> ability to combine multiple disks as one which meens
> that if a disk fails then part of the data will be
> lost and replacing a 250GB disk is easier than replacing
> a 4TB disk isn't it? Raid 1 is the ability to mirror
> a disk in a second disk sector by sector having a
> second copy of the disk when one of the disks fails
> but sectors etc then the mirrored is used automaticaly
> by the controler until the difective disk is changed.
Ok. Based on experience with large RAID systems....
First, large RAID systems decrease the chance that
the entire disk system will fail, but increase the chance
that one of the disks will fail. i.e. if you have 50
disks and 5 need to fail to take down the system,
this isn't likely. However, it is 50 times more likely
that one of the disks will fail. With 50 disks, in
hard access work, one of them will be failing
yearly.
Second, when a disk fails, in a massive array,
it takes time to repair the system.
Third, no raid system protects against logical
failures on the part of the OS, DB, software, etc.
It you get crossed FAT entries, the FAT system gets
fragmented, someone deletes a file, someone
overwrites a block, etc. the mirroring and
checksums don't protect the bad data.
Forth, restoring backups, defrags, check-disk,
etc. still requires the system to be down -- and
depend no upon the HD system, but solely
the amount of data to backup restore.
> keep in mind that this is a very simplefied explenation.
Understood.
> The company doesn't <want> two copies of the
> data but they require a maximum down time of
> 2 hours this is impossible for me to support
> only the trip to the customers site is around
> 5 hours so I need the second computer and
> backup although this is decreases my earnings
> and increases the customers cost I need this
> installation to work with out any problems
> if possible because this is a key customer.
You can still have two machines using DVD
jukeboxes. In fact, your backup machine can
have less HD space since it's cache doesn't
need to be as large. It only has to last until
you fix the main one. Duplicating out a 3.2TB
disk system (plus tape backup) is massively
expensive.
> In order to reach the same level of security
> as the one I have in mind with hard disks
> I will probably need 16 jukeboxes 8 per machine
> and a set of backups. Considering that I have to
> actually code part of RAID 1 in my program along
Not correct.
Large RAID system increase the chance that
something will go wrong. They just decrease
the chance that that something will take down the
entire system.
DVDs are more reliable than HDs. The chance
that a HD will fail when it is in constant use
24/7, writing to it, reading from it over and over,
is far greater than a DVD failing. There is no
need to RAID/mirror the DVD itself since
it can't fail like a hard disk -- since once burned
it won't get corrupted, have a head crash on
it, etc. With HD, just reading them risks
corrupting the information on them. With
DVDs this is never an issue.
The only thing you need to worry about is
the drive systems. In which case, mirroring
the entire system helps this. It also decreases
access time and usage cycles on each
jukebox.
> with the fact that I need to support writting
> DVDs and a few other extras that are not needed
> in a raid chain of disks this has some headen costs
> that I need to care for.
You only need to write to the DVD once. It
then gets verified. The cost of equipment
machines with DVD writers is hardly a factor
in the equation when you are already looking
at 250K+ worth of hardware. It also neatly
separates the tasks of the application, allows
it to be done off-line, and places it in a format
which can be used by other systems.
> You have a number of valid comments in hear most
> of them very well thought and I really appreciate
> your time and knowledge the only thing I need is
> some hardware to play with to see how things work
> for my self do you have any recomendations that I
> could follow?
You are about to purchase $250K worth of hardware.
That gives you some options. I assume that you are
going with name brand hardware -- IBM, Dell,
Compaq, etc. If you contact them and let them know
you need 50 workstations, two high-end storage
servers, two high-end SQL database servers,
a giga-bit network, 8 DVD jukeboxes, tape
backups, etc. I think you'll find sales reps to be
*very* helpful.
You can ask them about other clients with similiar
systems. You can ask to see individual bits of
hardware in action. You can ask for technical
specs, programming information, etc. You can
also pick their brains (and those of clients)
about how they see such systems working,
if there are other options out there, etc.
You aren't the only one who has a 3.2TB
data warehouse, or 100,000 scanned images.
Many government offices and other corporations
will have similiar systems.
> Thank you for your time and effort appreciated.
No problem.
Thanks,
Brett
.
- References:
- Server Advice Wanted.
- From: Yannis
- Re: Server Advice Wanted.
- From: Brett Watters
- Re: Server Advice Wanted.
- From: Yannis
- Re: Server Advice Wanted.
- From: Brett Watters
- Re: Server Advice Wanted.
- From: Yannis
- Server Advice Wanted.
- Prev by Date: EDS Cat Herder commercial from Superbowl XXXIV
- Next by Date: Re: Fastcode Donations - Help
- Previous by thread: Re: Server Advice Wanted.
- Next by thread: A very biased review
- Index(es):
Relevant Pages
|
