The Fugue

This stuff is documented out there, but I thought I’d give a brief summary and anecdotal evidence that it works.

I wanted to move falcon onto a RAID1 array, including root. Falcon was already using LVM for everything but root, and I wanted root on the RAID as well and so it made sense to just go all the way.

I had a bit of a hairy time, primarily because I wasn’t as prepared as I thought I was. This is in part because I was doing all this without internet, in part because of stupidity and lack of foresight, and in part because some vital things that I thought were in order, were not. But really it’s not that bad if you know what you’re doing. Having learned my lessons I did the same thing for gwythaint without incident in just a few minutes (not counting time spent copying files).

Let’s begin as all good chefs do, with mise en place.

You need a good boot disk with grub and LVM/RAID support. This is harder to find than you might think. Ubuntu’s Live CD doesn’t have LVM support (not sure about RAID). Debian Etch’s installer has RAID and LVM support, but its grub facilities are sorely lacking, and it takes forever to boot. Now would be a very good time to get familiar with the PLD Rescue Disk. This puppy is an absolute gem and no sysadmin should be without it. It supports LVM, RAID, every important filesystem, has a full suite of tools, super grub disk, GRUB4DOS, and more. I can’t sing enough praises for PLD’s rescue disk, so I won’t try. Just get it.

You naturally need to install the lvm2 and mdadm packages.

You need a kernel that supports RAID and LVM. Most stock kernels will, but make sure! Also, regenerate the initrd after installing the lvm2 and mdadm packages. You can do this with dpkg-reconfigure linux-image-`uname -r`. I got bit by an initrd generated by yaird that didn’t like being moved, so you should make sure you have initramfs-tools installed when you regenerate the initrd.

Grub2 reportedly has RAID and LVM support but I didn’t have grub2 and didn’t want to press my luck. So I made a boot partition (not on the RAID or in LVM) and copied over /boot.

Now set up the new RAID and LVM. In my case it was as simple as creating the RAID with one disk (the disk that currently hosted my system was eventually added too) then making a physical volume out of that array (pvcreate), adding it to the virtual group (vgextend), creating a new logical volume for root (lvcreate) along with its filesystem, and moving the already-existing logical volumes over to the new physical volume (lvmove). Then reboot into the rescue disk so you can safely copy over the root filesystem (PLD has rsync which makes this a snap).

PLD doesn’t automatically assemble RAID arrays or activate volume groups, so you need to do this manually. Something like this:

mdadm -A /dev/md0 /dev/sdb2
vgchange -ay

Now for the fun part: getting things to boot. First, you need to modify the new /etc/fstab to reflect the new location of /. That is, /dev/vg1/root (or whatever you called it). Also put in /boot.

Reboot and choose super grub disk at the PLD boot. At first I played with the menus but after a dozen boots (literally, I told you it was hairy) I was an expert at the grub command line and was doing things by hand. Hit c to get the grub cli, then type something like this:

find /grub/menu.lst
root (hd0,0)
setup (hd0)
kernel /vmlinuz-2.6.18-6-k7 root=/dev/mapper/vg1-root ro
initrd /initrd.img-2.6.18-6-k7
boot

Let’s analyze that.
First we find /grub/menu.lst which just tells us what grub calls the boot partition: (hd0,0) in this case. (I had a heck of a time with grub not finding my boot partition and I don’t know why. Maybe fdisk or BIOS issues. I still don’t know. I eventually was able to get a partitioning that worked, hopefully you don’t have any issues.)

Next we tell grub to use that partition as root. The setup line installs grub on the MBR so we won’t need to use the rescue disk in the future.

The kernel and initrd lines are normal grub fare. The important thing here is the root option. You must use the /dev/mapper/vg1-root path, not /dev/vg1/root. At least this is true on Debian Etch stock kernels, but it seems like a good idea in any case. The reason is that the /dev/mapper paths are there from the kernel but the /dev/vg1 symlinks are added with boot scripts.

You should boot right up. After boot, issue mount to verify that things do look like they ought to. Edit /boot/grub/menu.lst and run update-grub, then try booting without the rescue CD. You’re in business!

In summary: read the RAID and LVM howtos, LVM goes on top of RAID, get PLD, make sure your kernel and ramdisks are in order, leave /boot off the RAID/LVM, don’t forget to change /etc/fstab, and use the /dev/mapper/${VG}-${LV} path in the kernel root option.