Swapping boot drives around

Swapping boot drives around

The NZ FreeBSD User Group was given a
box which was destined to become the cvsup and www mirror.  The first thing I did was
swap the two 500MB drives out and put in three 1GB drives.  This articles documents
how I did that.

You may wish to first read about a couple of other methods as supplied by other readers.

Overview

The following steps will be performed:

  1. backup the boot drive
  2. install FreeBSD on a new boot drive
  3. restore the old boot drive over the new boot drive.

This procedure assumes you have enough disk space for the backup.  I’m sure there
must be an easier way to do this.  If you know, please let us know your comments.   This
article also assumes you know how to add a new drive to a box, set up the
primary/secondary stuff (if using IDE) or set device IDs (if using SCSI).  The main
thing this article shows you is how I backed up and restored the boot drive.

The backup

For this exercise, I had three drives.  You could get away with just two.  I
mounted the spare drive in /mnt.

mkdir /mnt
mount /dev/da1s1e /mnt

The first step was to backup each file system on the boot drive.  This is slightly
more complex that I first thought.  If you look at the existing mount points, you’ll
see what I mean:

# mount
/dev/da0s1a on / (local, writes: sync 14 async 215)
/dev/da0s1f on /usr (local, writes: sync 3 async 95)
/dev/da0s1e on /var (local, writes: sync 91 async 237)
procfs on /proc (local)

In this example, you’ll see three mount points we must deal with: /, /usr,
and /var.  Each mount point represents a different file system.  I
decided to backup each separately.  I’m not sure why.  But I did.  I used
the tar command for this.

tar cvlf /mnt/root.tar /
tar cvlf /mnt/var.tar /var
tar cvlf /mnt/usr.tar /usr

Here is a brief explanation of each option:

  • p – same permissions – essential to a successful backup (not used above, but used below)
  • c – creates a new archive
  • v – lists the files as they go into the tarball
  • l – don’t cross mount points (backs up only one file system)
  • f – specifies that the next argument is the output tarball

Then I did a "shutdown -h now" and powered off the box once the
shutdown was complete (see the console messages for this).

Create a new boot drive

I removed the existing boot drive from the box and installed a new blank drive in it’s
place.  Then I went through the normal FreeBSD install process for that drive.  
Actually, it wasn’t a normal install as I need a custom kernel for this box because the
GENERIC kernel won’t work.  See Installing 3.2-release [the
hard way]
for details on that.  Otherwise, the install was pretty much the same
as Installing FreeBSD to replace Windows 95.

Then I
did a "shutdown -h now" and powered off the box once the shutdown was
complete (see the console messages for this).

Restoring the old to the new

The next step was to reinstall the original boot drive but retain the new boot drive.
  You must be careful to avoid conflicts.  In my case, I merely changed the SCSI
ID on the new boot drive from 0 to 2.

I mounted the spare drive:

mount /dev/da1s1e /mnt

I mounted the new boot drive.  You’ll remember that we had three file system to
backup.  So that requires three mount points.

mkdir /newboot
mount /dev/da2s1a /newboot
mount /dev/da2s1f /newboot/usr
mount /dev/da2s1e /newboot/var

Then I started restoring from /mnt to drive to the new boot drive:

tar xvpf /mnt/root.tar -C /newboot
tar xvpf /mnt/usr.tar -C /newboot
tar xvpf /mnt/var.tar -C /newboot

The -C option tells tar to change to the /newboot
directory before untar’ing any files.

Then I did a "shutdown -h now" and powered off the box once the
shutdown was complete (see the console messages for this).

Out with the old and in with the new

After shutting down the machine, I swapped the SCSI IDs of the new and old boot
drives.   The old one became 2 and the new one became 0.  I powered on the
machine, and it booted up straight away.  No problems.

Wait!  All my permissions are gone!

If you find that the permissions on your files are messed.  Perhaps everything is
either root or wheel.  Well, you probably haven’t restored your users from the tar
file.  Try that.

Other methods

People have responded to my query regarding easier ways to do these things.  
Thanks to those that have helped.

Duncan Barclay <dmlb@ragnet.demon.co.uk> supplied this information.  Thanks.

All you need to do to use another drive as a boot drive is the following:

  1. create a small freebsd partition on the target drive.  I have 2MB on my first ide
    drive.
  2. disklabel it, using the -B option to add some boot blocks (boot0 and boot1)
  3. newfs it
  4. mount it e.g. /mnt
  5. then use pax/tar/cpio to copy /boot to /mnt/boot
  6. edit /mnt/boot/loader.conf.local to have
    currdev=disk<n>s1a root_disk_unit=<m>
    where <n> is the BIOS disk number, and <m> is the scsi id number if you are
    booting from a scsi disk.
  7. reboot!

This is the scheme Duncan uses to have FreeBSD booted from his scsi drives in a mixed
ide/scsi setup.

Dean <madscientist@thegrid.net> supplied this.  Cheers.

  1. Shutdown system and add the drive that you want to be your new boot disk
  2. Partition the New Boot Drive the way you want and mount it on /mnt. ie: /mnt,
    /mnt/usr, /mnt/var, etc.
  3. Now, you simply copy the filesystems with tar. The command to do this is:
    tar clf - -C /    . | tar xpvf - -C /mnt
    tar clf - -C /usr . | tar xpvf - -C /mnt/usr
    etc
    using a dash as the filename causes tar to read/write from/to stdin/stdout. Not quite sure
    why you need the dot
    at the end of the first tar command, but that’s what they say in the man page.  I use
    it without the -p option and it
    seems to work fine, but again, the man page suggests you use it.
  4. Power off the machine, swap the scsi id’s and boot up.

No re-install required. You also don’t have to make sure you have enough space lying
around to hold the tarball.

Hope that helps,
Dean

Was this efficient?

I’m sure there are simpler ways of doing this.  For example, I think I could have
done this with a single tarball.  And I think I could have created a boot drive
without actually having to install FreeBSD on the new drive.  If you know of an
easier way, please add your comments.

5 thoughts on “Swapping boot drives around”

  1. (posted by Dan Langille)

    I came across your article while researching disk and controller issues and
    thought I would pass along the method I regularly use to replace disks in
    FBSD, in fact I just got done with a boot disk swap on a 3.4 RELEASE system
    about an hour ago on one of my development machines.

    I use dump and restore to copy the filesystems in one operation.

    The first thing I do is install and prep the disks using the "Dangerously
    dedicated method" described in the FreeBSD Handbook. Methodology doesn’t
    matter as long as you can newfs and mount the disks and access their
    filesystems. If your new disk will be a boot disk you must make sure it is
    bootable using sysinstall’s "partition" utility or via the -B option to
    "disklabel."

    "newfs" your new filesystems and "mount" them up.

    "cd" to the new filesystem, here we’ll build a new root on /slash:

    # /slash

    Now dump the root FS to it:

    dump -0b 512 -f – / | buffer -S 2048K -p 75 | restore -rb 512 -f –

    If you don’t have the "buffer" utility remove "buffer -S 2048K -p 75 | " and
    pipe dump directly to restore.

    In the above dump command I specify a Level 0 full dump with a 512k blocksize
    for speed, your mileage may vary, experimenting with the blocksize is
    necessary to determine an optimal value. I use the "-f -" option to dump to
    stdout and "/" is the filesystem to dump to stdout. I am piping the dump to
    "buffer" to speed up the I/O a tad and that pipes to restore which uses the -r
    option to "rebuild a filesystem", -b to specify my 512k blocksize, and "-f -"
    once again to specify that it is restoring from stdin. I frequently clone
    disks with gigabytes of data on them so maximum speed is desirable and the use
    of the "-S 2048" has buffer spitting out the progress for every two megs
    through the pipe. "-p 75" instructs buffer to start a write when the 75% of
    the internal queue is full. "buffer" is available through the misc FreeBSD
    ports collection. "team" is another nifty buffering utility which can be used.

    Follow the same procedure for each filesystem on the hard disk and when
    finished you will have a perfect copy which is ready to replace the old drive
    about as fast as is possible.

    When I’m finished cloning the disk I set whatever jumpers may be necessary for
    its new position on the SCSI or IDE daisy chain and remove the old drive.

    If replacing an IDE boot drive with a SCSI drive you’ll want to build a new
    kernel specifying "kernel root on dax" or "kernel root on sdx" in the conf
    file where "x" is the unit number of the boot disk and da or sd as the disk
    IS as approriate. Your SCSI controller must be able to be used for booting,
    consult your hardware documentation for more information. When trying to
    diagnose strange system bootstrapping preferences take some time to examine
    your /boot/ directory’s files and read the boot(8) manpage. FreeBSD will boot
    from the first bootable device it can figure out how to boot from and it can
    be quite confusing to initially boot from a kernel on one disk and then have
    your root filesystem mounted from another disk which has your new and
    still _unused_ kernel on it! If you will no longer have IDE drives in the
    system and your SCSI controller hijack’s the bios on its own then you
    shouldn’t even need to modify the kernel, it should just pick up the ball and
    run with it all on its own.

  2. To swap root drives from SCSI to IDE (I was also using Vinum).

    Follow Dean’s Tar example

    vi /mnt/etc/fstab and alter the entries to suit your new drive

    Change drives and reboot

    The boot will stop as the vinum drive (if you have one) cannot be fsck’d

    then

    mount -u /

    /dev/MAKEDEV all

    ^d back out of the shell and it should start fine.

  3. Just a quick note to let you know that the procedure contributed by Dean at
    the bottom of the page on http://www.freebsddiary.org/driveswap.php no
    longer works in FreeBSD 5.X.

    The nice folks have updated the options that tar takes, and the correct
    command is of the form:
    tar –one-file-system -c -f – -C / . | tar xpvf – -C /mnt

    E.g.
    tar –one-file-system -c -f – -C / . | tar xpvf – -C /mnt
    tar –one-file-system -c -f – -C /var . | tar xpvf – -C /mnt/var
    tar –one-file-system -c -f – -C /tmp . | tar xpvf – -C /mnt/tmp
    tar –one-file-system -c -f – -C /usr . | tar xpvf – -C /mnt/usr

    [%sig%]

    1. today i tried Dean’s solution on my 6.1 PRE-RELEASE machine,

      after 3 or 4 hours in hell, there are 2 things you should be noted:

      1. if you mount your new disk at /mnt,
      then /mnt should be excluded from tar,
      otherwise, the /mnt/ directory will be tarred recursively.

      2. use gtar, otherwise the permission of the directories (like /usr, /var!) will be a mess. (drwx———, owned by root)
      it’s really strange. i hope it’s just a temporary bug.

      so, this is what i did

      gtar cf – -C / –one-file-system –exclude /mnt/ . | gtar xpvf – -C /mnt/

      [%sig%]

      Post Edited (11-02-06 09:02)

Leave a Comment

Scroll to Top