RAID stands for “Redundant Array of Inexpensive Disks” ^[1].

The basic idea behind RAID is to combine multiple small, inexpensive disk drives into an array which yields performance exceeding that of one large and expensive drive. This array of drives will appear to the computer as a single logical storage unit or drive.

Some of the concepts used in this chapter are important factors in deciding which level of RAID to use in a particular situation. Parity is a concept used to add redundancy to storage. A parity bit is a binary digit which is added to ensure that the number of bits with value of one in a given set of bits is always even or odd. By using part of the capacity of the RAID for storing parity bits in a clever way, single disk failures can happen without data-loss since the missing bit can be recalculated using the parity bit. I/O transactions are movements of data from or to a RAID device or it's members. Each I/O transaction consists of one or more blocks of data. A single disc can handle a maximum random number of transactions per second, since the mechanism has a seek time before data can be read or written. Depending on the configuration of the RAID, a single I/O transaction to the RAID can trigger multiple I/O transactions to it's members. This affects the performance of the RAID device in terms of maximum number of I/O transactions and data transfer rate. Data transfer rate is the amount of data a single disc or RAID device can handle per second. This value usually varies for read and write actions, random or sequential access etc.

RAID is a method by which information is spread across several disks, using techniques such as disk striping (RAID Level 0) and disk mirroring (RAID level 1) to achieve redundancy, lower latency and/or higher bandwidth for reading and/or writing to disk, and maximize recoverability from hard-disk crashes.

The underlying concept in RAID is that data may be distributed across each drive in the array in a consistent manner. To do this, the data must first be broken into consistently-sized chunks (often 32K or 64K in size, although different sizes can be used). Each chunk is then written to each drive in turn. When the data is to be read, the process is reversed, giving the illusion that multiple drives are actually one large drive. Primary reasons to use RAID include:

There are a number of different ways to configure a RAID subsystem -- some maximize performance, others maximize availability, while others provide a mixture of both. For the LPIC-2 exam the following are relevant:

Level 0.  RAID level 0, often called “striping”, is a performance-oriented striped data mapping technique. This means the data being written to the array is broken down into strips and written across the member disks of the array. This allows high I/O performance at low inherent cost but provides no redundancy. Storage capacity of the array is equal to the sum of the capacity of the member disks.

Level 1.  RAID level 1, or “mirroring”, has been used longer than any other form of RAID. Level 1 provides redundancy by writing identical data to each member disk of the array, leaving a mirrored copy on each disk. Mirroring remains popular due to its simplicity and high level of data availability. Level 1 operates with two or more disks that may use parallel access for high data-transfer rates when reading, but more commonly operate independently to provide high I/O transaction rates. Level 1 provides very good data reliability and improves performance for read-intensive applications but at a relatively high cost. Array capacity is equal to the capacity of the smallest member disk.

Level 4.  RAID level 4 uses parity concentrated on a single disk drive to protect data. It's better suited to transaction I/O rather than large file transfers. Because the dedicated parity disk represents an inherent bottleneck, level 4 is seldom used without accompanying technologies such as write-back caching. Array capacity is equal to the capacity of member disks, minus the capacity of one member disk.

Level 5.  The most common type of RAID is level 5 RAID. By distributing parity across some or all of the member disk drives of an array, RAID level 5 eliminates the write bottleneck inherent in level 4. The only bottleneck is the parity calculation process. With modern CPUs and software RAID, that isn't a very big bottleneck. As with level 4, the result is asymmetrical performance, with reads substantially outperforming writes. Level 5 is often used with write-back caching to reduce the asymmetry. Array capacity is equal to the capacity of member disks, minus capacity of one member disk.

Linear RAID.  Linear RAID is a simple grouping of drives to create a larger virtual drive. In linear RAID, the chunks are allocated sequentially from one member drive, going to the next drive only when the first is completely filled. This grouping provides no performance benefit for single process applications, as it is unlikely that any single I/O operations will be split between member drives. This changes when multiple processes access the same array and access becomes more random. Linear RAID also offers no redundancy, and in fact decreases reliability -- if any one member drive fails, the entire array cannot be used. The capacity is the total of all member disks.

RAID can be implemented either in hardware or in software; both scenarios are explained below.

The hardware-based system manages the RAID subsystem independently from the host and presents to the host only a single disk per RAID array.

A typical hardware RAID device might connect to a SCSI controller and present the RAID arrays as a single SCSI drive. An external RAID system moves all RAID handling intelligence into a controller located in the external disk subsystem.

RAID controllers also come in the form of cards that act like a SCSI controller to the operating system, but handle all of the actual drive communications themselves. In these cases, you plug the drives into the RAID controller just as you would a SCSI controller, but then you add them to the RAID controller's configuration, and the operating system never knows the difference.

Software RAID implements the various RAID levels in the kernel disk (block device) code. It also offers the cheapest possible solution: Expensive disk controller cards or hot-swap chassis are not required, and software RAID works with cheaper IDE disks as well as SCSI disks. With today's fast CPUs, software RAID performance can excel against hardware RAID.

Software RAID allows you to dramatically increase Linux disk IO performance and reliability without buying expensive hardware RAID controllers or enclosures. The MD driver in the Linux kernel is an example of a RAID solution that is completely hardware independent. The performance of a software-based array is dependent on the server CPU performance and load. Also, the implementation and setup of the software RAID can significantly influence performance.

The concept behind Software RAID is simple -- it allows you to combine two or more block devices (usually disk partitions) into a single RAID device. So if you have three empty partitions (for example: hda3, hdb3, and hdc3), using Software RAID, you can combine these partitions and address them as a single RAID device, /dev/md0. /dev/md0 can then be formatted to contain a filesystem and used like any other partition.

Setting up RAID (in software) requires only two files to be set up under Linux: /etc/raidtab and /etc/rc.d/rc.local (or equivalent). The Linux kernel provides a special driver, /dev/md0, to access separate disk partitions as a logical RAID unit. These partitions under RAID do not actually need to be different disks, but in order to eliminate risks it is better to use different disks.

Follow these steps to set up RAID in Linux:

Each of these steps is now described in detail:

Initialize partitions to be used in the RAID setup.  Create partitions using any disk partitioning tool.

Configure the driver.  To configure the driver, you need to create or edit a configuration file. By default this is the file /etc/raidtab, but mkraid can be told to use another file. In the configuration file you can specify the RAID-level to use, which partitions/drives to use, the device-name for the RAID drive and more. In the next section (/etc/raidtab) we will offer an example configuration file.

Initialize RAID drive.  Here's an example of the sequence of commands needed to create and format a RAID drive (mkraid):

Automate RAID activation after reboot.  Run raidstart in one of the startup files (e.g. /etc/rc.d/rc.local or /etc/init.d/rcS) before mounting the filesystem at boot time using the following command: raidstart /dev/md0 Repeat for each RAID device present on the system that you want to automatically be started during system startup.

mount the filesystem on the RAID drive.  Edit /etc/fstab to automatically mount the filesystem on the RAID drive. The entry in /etc/fstab is identical to normal block devices containing file systems.

Manual RAID activation and mounting.  Run raidstart /dev/md0 for each RAID block device you want to start manually. After starting the RAID device you can mount any filesystem present on the device using mount with the appropriate options.

mkraid sets up a set of block devices into a single RAID array. It looks in its configuration file for the md devices mentioned on the command line, and initializes those arrays. mkraid works for all types of RAID arrays (RAID1, RAID4, RAID5, LINEAR and RAID0). Note that initializing RAID devices destroys all of the data on the constituent devices.

mkraid uses the file /etc/raidtab by default as its configuration file.

By default the RAID configuration is stored in a file (i.e. /etc/raidtab). To read a file, it must be in a mounted filesystem. To mount a filesystem, the kernel must be able to recognize the disk. So initially, you could not boot from a RAID array. To solve this problem, you can activate auto detection in your kernel and configure mkraid so it will write persistent superblocks.

The persistent superblock is a small disk area that contains information about the RAID device. It is allocated at the end of each RAID device and helps the kernel to safely detect RAID devices - even if disks were moved between SCSI controllers. When a persistent superblock is found and the current setup does not match the administration found in there, the kernel will either correctly reorder disks, or will refuse to start up the array.

Every member of the array contains a copy of this superblock. Since this superblock carries all information necessary to start up the whole array, the array can be constructed and started if the members of that array are auto detected by the kernel. For auto detection to work this way, all the “member” RAID partitions should be marked type 0xfd (Linux RAID auto detect) e.g. by using fdisk. Your kernel should be configured to enable it to read persistent superblocks.

On a system that uses persistent superblocks the RAID configuration can be constructed even if the filesystem that contains /etc/raidtab was not mounted yet. Be aware: the file /etc/raidtab will not become superfluous - it is needed to be able to change and/or rebuild your configuration.

Various Linux RAID tools use the default configuration file, /etc/raidtab.

/etc/raidtab has multiple sections, one for each md device which is being configured. Each section begins with the raiddev keyword.

Later raiddev entries can use earlier ones, and the parsing code isn't overly bright, so be sure to follow the ordering specified in the raidtab manpage for best results.

Here's a sample configuration file:

       #
       # sample raiddev configuration file
       #
       raiddev /dev/md0
           raid-level              0
           nr-raid-disks           2  # Specified below
           persistent-superblock   0
           chunk-size              8

           # device #1:
           #
           device                  /dev/hda1
           raid-disk               0

           # device #2:
           #
           device                  /dev/hdb1
           raid-disk               1

       # A new section always starts with the 
       # keyword 'raiddev'
 
       raiddev /dev/md1
           raid-level              5
           nr-raid-disks           3  # Specified below
           nr-spare-disks          1  # Specified below
           persistent-superblock   1
           parity-algorithm        left-symmetric

           # Devices to use in the RAID array:
           #
           device                  /dev/sda1
           raid-disk               0
           device                  /dev/sdb1
           raid-disk               1
           device                  /dev/sdc1
           raid-disk               2

           # The spare disk:
           device                  /dev/sdd1
           spare-disk              0                      

The steps taken in using mdadm for RAID configuration, setup and activation are basically the same as with mkraid and raidstart. With mdadm however all actions are being handled by just one utility and mdadm doesn't necessarily need a configuration file. Read the mdadm manpage and mdadm.conf manpages for more information about mdadm, especially about create and assemble modes.