Re: [SAGE] SANs, Oracle, and RAID

[Doug Hughes <doug@eng.auburn.edu>]

On Fri, 7 May 2004, John Borwick wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hey, everybody.  I hope you're doing well.
>
> Our DBAs are implementing a huge new product (Banner, from Sungard SCT),
> and we're just getting to their disk requirements.
>
> Their ideal is Oracle's "twenty-two disk solution" with separate
> partitions for each control file, etc.  We, of course, are not going
> down that road.
>
> The DBAs would be satisfied with the "seven-disk compromise", which
> means they want 7 spindles for maximum I/O.  They do not want any RAID
> 5.  I think they would want 5-7 separate, small RAID 1 arrays.
>
> We just got in a big IBM Fast-T SAN with ca. 4.5 T of raw disk.  Only
> part of the SAN can go towards this project.  Management (and us) would
> greatly prefer RAID 5 and a minimal number of separate arrays.  The main
> issues are performance cost, percentage of raw space available, and a
> coherent argument to justify our decision.
>
> So, here's what I haven't found anywhere else:
>
> * Should RAID levels be considered differently in a SAN environment than
> ~  in a local-disk environment?  That is, does the cost of a RAID 5 write
> get mitigated by the SAN cache?
to a very large extent, yes. benchmark it yourself. Most vendors will
give you hardware for evaluation. I find iozone and bonnie++ to be
excellent tools.

>
> * Can we get away with one huge RAID 5 array instead of 5-7 small RAID 1
> arrays (as far as performance goes)?
>
quite possibly. 2GB/sec is a lot of bandwidth on a single channel. Many
vendors will give you multi-channel capability.

> * How are SANs optimized for databases, in general?
The SAN is just a transport fabric like SCSI: a big pipe (with extra
tidbits like device-device connectivity for transfers).  dual 2GB
cards are available even now. you'll want at least a 64 bit slot
with 64mhz performance for one of those. PCI-X cards are the standard
spec for the HBAs.

More spindles is a good approach. Oracle likes to recommend using
a 1MB interleave factor and stripe and mirror everything to better utilize
all disks and eliminate hotspots (aka SAME - Stripe and Mirror
Everything). Sun calls the technique Wide-Thin striping (white paper
at www.sun.com/blueprints)

Are you optimizing for number of IO operations per second like a TPC
or are you optimizing for sequential throughput? (or something in
between). IOPS wants as many fast spindles as possible with minimal
prefetching.
Sequential throughput doesn't require as many spindles but likes higher
RPM disks and lots of pre-fetching.