Setting up RAID0 in Ubuntu 12.04 in AWS High I/O

Amazon announced high I/O instances today. This is huge for anyone with a database larger than available memory, as it’s been a complete nightmare dealing with EBS up till now. Now your Cassandra, MongoDB, MySQL, or whatever your using should be able to perform well without requiring keeping your entire dataset in memory.

With each instance you get 2x1TB of disk. In this tutorial I’ll be setting it up as a RAID0 to get a single 2TB disk which should deliver excellent performance.

Before you get started, make sure you’ve got mdadm installed:

apt-get install mdadm

To begin, check fdisk and make sure your 1TB drives are mounted.

root@ip-10-140-128-232:~# fdisk -l

Disk /dev/xvda1: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders, total 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0×00000000

Disk /dev/xvda1 doesn’t contain a valid partition table

Disk /dev/xvdf: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0×00000000

Disk /dev/xvdf doesn’t contain a valid partition table

Disk /dev/xvdg: 1099.5 GB, 1099511627776 bytes
255 heads, 63 sectors/track, 133674 cylinders, total 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0×00000000

Disk /dev/xvdg doesn’t contain a valid partition table

Now you’ll want to format each of the 1TB drives. Here’s what my console looks like, minus some extra help text:

root@ip-10-140-128-232:~# fdisk /dev/xvdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x2aabe5ed.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won’t be recoverable.

Warning: invalid flag 0×0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-2147483647, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-2147483647, default 2147483647):
Using default value 2147483647

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L
…..
-> fd Linux raid auto
…..
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

Do this for both drives.

Now, tell mdadm to build the RAID:

root@ip-10-140-128-232:~# mdadm –create –verbose –auto=yes /dev/md0 –level=0 –raid-devices=2 /dev/xvdf1 /dev/xvdg1

We’re using XFS, so I needed to install the xfs tools for the next part.

root@ip-10-140-128-232:~# apt-get install xfsprogs

Now format your drive. I got some output about log strip unit being too large, but I think it’s OK.

root@ip-10-140-128-232:~# mkfs -t xfs /dev/md0
log stripe unit (524288 bytes) is too large (maximum is 256KiB)
log stripe unit adjusted to 32KiB
meta-data=/dev/md0 isize=256 agcount=32, agsize=16777088 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=536866816, imaxpct=5
= sunit=128 swidth=256 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=262144, version=2
= sectsz=512 sunit=8 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

Mount it and check it out:

root@ip-10-140-128-232:~# mkdir /mnt/bigraid
root@ip-10-140-128-232:~# mount /dev/md0 /mnt/bigraid/
root@ip-10-140-128-232:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 8.0G 878M 6.8G 12% /
udev 30G 12K 30G 1% /dev
tmpfs 12G 192K 12G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 30G 0 30G 0% /run/shm
/dev/md0 2.0T 34M 2.0T 1% /mnt/bigraid

As a final note, a few quick tests:

root@ip-10-140-128-232:~# dd if=/dev/zero of=/mnt/bigraid/somefile bs=512
^C2790615+0 records in
2790615+0 records out
1428794880 bytes (1.4 GB) copied, 11.1162 s, 129 MB/s

root@ip-10-140-128-232:~# dd if=/dev/zero of=/mnt/bigraid/somefile bs=2048
^C2441837+0 records in
2441837+0 records out
5000882176 bytes (5.0 GB) copied, 13.9482 s, 359 MB/s

root@ip-10-140-128-232:~# hdparm -t /mnt/bigraid/

/mnt/bigraid/:
Timing buffered disk reads: read() failed: Is a directory
BLKFLSBUF failed: Inappropriate ioctl for device
root@ip-10-140-128-232:~# hdparm -t /dev/
Display all 196 possibilities? (y or n)
root@ip-10-140-128-232:~# hdparm -t /dev/xvd
xvda1 xvdf xvdf1 xvdg xvdg1
root@ip-10-140-128-232:~# hdparm -t /dev/xvd
xvda1 xvdf xvdf1 xvdg xvdg1
root@ip-10-140-128-232:~# hdparm -t /dev/m
mapper/ md0 mem
root@ip-10-140-128-232:~# hdparm -t /dev/m
mapper/ md0 mem
root@ip-10-140-128-232:~# hdparm -t /dev/md0

/dev/md0:
Timing buffered disk reads: 1188 MB in 3.00 seconds = 395.42 MB/sec

I based my post off the instructions found here. Modified for AWS and a newer version of Ubuntu.