Re: IBM in talks to buy Sun

On Fri, 20 Mar 2009, Larry K. Wollensham wrote:

Lew wrote:
John B. Matthews wrote:
Larry K. Wollensham wrote:
That's in the form of separate 1TB disks without RAID, though. I expect* 14TB of RAID will top $10,000. That's still peanuts compared to a big company's IT budget, and still less than adding one vehicle to their motor pool.

Are you pricing high-reliability, 10K+ RPM SAS or SATA drives with large RAM buffers in a rack-mount format?

A 14 TB SATA system is roughly $21.8K plus maintenance. Call it $1,500 per TB, not including further costs.

Roughly twelve grand (plus maintenance) for 12 TB, and that's only 7200 RPM SATA. Call it $1,000 per TB, not including further costs.

One thing big-iron shops avoid is consumer-grade hardware.

That might change when they do the math.

To compensate for slower speeds, double the number of file servers behind a load-balancer, and ultimately the number of disks. This works as long as you don't need to serve *single* files *really* fast.

To compensate for lower reliability (but consumer grade hardware is getting better), assume a doubled disk replacement rate in the RAIDs.

Overall, that means four times the disks. If they're $200/TB each the above doublings produce $800 in place of the $1000-1500 you cite for the non-consumer-grade hardware.

Parallelism in various forms (multiprocessors, load-balanced clusters, RAID, and so forth) make consumer grade hardware able to "add up" to be equivalent to higher-grade hardware. Sometimes still with lower price tags.

If anyone still isn't persuaded by Larry's argument, they should consider Google. And i don't mean go and do a search. Google run their data centres in exactly this way, using vast amounts of cheap commodity hardware. It's an approach that doesn't work at the small scale, where a single better machine plus operating costs may well work out cheaper than two or more cheaper, less reliable ones, but at the large scale, it works out very well indeed. They've even written a paper about it:

http://labs.google.com/papers/disk_failures.pdf

It'd be interesting to compare that hard data to comparable data for high-end drives - except oh wait, there isn't any, because nobody's ever used them in that kind of volume.

The one large-scale study that does make an attempt, amongst other things, to compare cheap and expensive disks find that there isn't a significant difference (assuming that you accept 10K FC disks as representatives of expensive disks, and 7200 rpm SATA disks as cheap):

http://www.cs.cmu.edu/~bianca/fast07.pdf

tom

--
In my view, this is no different than a parent introducing his child to
Shakespeare (except that the iambic pentameter is replaced by a framework
of profanity, misogyny, substance abuse, violence, retaliation, crime
and infidelity). -- Dad Gone Mad, on rap
.

Relevant Pages

  • Re: Again: LILO 22.5.9 still gives Error: duplicated volume ID
    ... The 2.4 kernels had serious problems with hardware RAID1 support; ... I've not seen SCSI raid before (hardware). ... but also the two underlying disks. ...
    (comp.os.linux.setup)
  • Re: T2000 SAS RAID controller fun and games
    ... We had created the hardware raid volumes before installing solaris and ... 2/ moved a pre-installed disks over to the new hardware in bay0 ... 4/ Verified we could see the disk/volume by selecting the raid device ...
    (comp.unix.solaris)
  • Re: IBM in talks to buy Sun
    ... To compensate for slower speeds, double the number of file servers behind a load-balancer, and ultimately the number of disks. ... To compensate for lower reliability (but consumer grade hardware is getting better), assume a doubled disk replacement rate in the RAIDs. ... Parallelism in various forms (multiprocessors, load-balanced clusters, RAID, and so forth) make consumer grade hardware able to "add up" to be equivalent to higher-grade hardware. ... Google run their data centres in exactly this way, using vast amounts of cheap commodity hardware. ...
    (comp.lang.java.programmer)
  • Re: IBM in talks to buy Sun
    ... that means four times the disks. ... If they're $200/TB each the above doublings produce $800 in place of the $1000-1500 you cite for the non-consumer-grade hardware. ... Parallelism in various forms (multiprocessors, load-balanced clusters, RAID, and so forth) make consumer grade hardware able to "add up" to be equivalent to higher-grade hardware. ...
    (comp.lang.java.programmer)
  • Re: ad10: WARNING - READ_DMA UDMA ICRC error (retrying request) LBA=11441599
    ... >>that if I send hardware I want it back when you're done, ... I have two which reliably fail if you put TWO disks on them in a gmirror ... BROKEN by FreeBSD and likely to cause people trouble - including irrevocable ... the machine as the errors cause irrevocable data corruption. ...
    (freebsd-stable)