Concatenated Disk Configuration

Concatenated Disk Configuration

It seems like today everyone has a collection of multimedia files.
Everything from mp3’s to video clips. I’ve converted most of my audio
CDROM collection to mp3’s so I can have all of my music in one
centralized location, and not have to hunt down the audio CD with
that one song I got stuck in my head. The problem I was faced with is
where to store all these files?

When choosing a mass storage solution, the most important factors to
consider are speed, reliability, and cost. It is very rare to have
all three in favor, normally a fast, reliable mass storage device is
expensive, and to cut back on cost either speed or reliability must be
sacrificed. In designing my system, I ranked the requirements by most
favorable to least favorable. In this situation, cost was the biggest
factor. I needed a lot of storage for a reasonable price. The next
factor, speed, is not quite as important, since most of the usage
would be over a one hundred megabit switched Ethernet, and that would
most likely be the bottleneck. The ability to spread the file
input/output operations out over several disks would be more than
enough speed for this network. Finally, the consideration of
reliability was an easy one to answer. All of the data being put on
this mass storage device was already backed up on CD-R’s. This drive
was primarily here for online live storage for easy access, so if a
drive went bad, I could just replace it, rebuild the filesystem, and
copy back the data from CD-R’s.

To sum it up, I need something that will give me the most amount of
storage space for my money. The cost of large IDE disks are cheap
these days. I found a place that was selling Western Digital 30.7gb
5400 RPM IDE disks for about one-hundred and thirty US dollars. I
bought three of them, giving me approximately ninety gigabytes of
online storage.

Installing the Hardware

I installed the hard drives in a system that already had
one IDE disk
in as the system disk. The ideal solution would be for each IDE disk
to have its own IDE controller and cable, but without fronting more
costs to acquire a dual IDE controller this wouldn’t be a possibility.
So, I jumpered two disks as slaves, and one as master. One went on
the first IDE controller as a slave to the system disk, and the other
two where slave/master on the secondary IDE controller.

Upon reboot, the system BIOS was configured to automatically detect
the disks attached. More importantly, FreeBSD detected them on
reboot:

ad0: 19574MB <WDC WD205BA> [39770/16/63] at ata0-master UDMA33
ad1: 29333MB <WDC WD307AA> [59598/16/63] at ata0-slave  UDMA33
ad2: 29333MB <WDC WD307AA> [59598/16/63] at ata1-master UDMA33
ad3: 29333MB <WDC WD307AA> [59598/16/63] at ata1-slave  UDMA33

At this point, if FreeBSD doesn’t detect the disks, be sure that you
have jumpered them correctly. I have heard numerous reports with
problems using cable select instead of true slave/master configuration.

The next consideration was how to attach them as part of the
filesystem. I did a little research on vinum(8) and FreeBSD’s ccd(4). In
this particular configuration, ccd(4) appeared to be a better choice
mainly because it has fewer parts. Less parts tends to indicate less
chance of breakage. Vinum appears to be a bit of an overkill for my
needs.

Setting up the CCD

CCD allows me to take several identical disks and concatenate them
into one logical filesystem. In order to use ccd, I need a kernel
with ccd support built into it. I added this line to my kernel
configuration file and rebuilt the kernel:

pseudo-device   ccd     4

ccd support can also be loaded as a kernel loadable module
in FreeBSD 4.0 or later.

To set up ccd, first I need to disklabel the disks. Here’s how I
disklabeled them:

disklabel -r -w ad1 auto
disklabel -r -w ad2 auto
disklabel -r -w ad3 auto

This created a disklabel ad1c, ad2c and ad3c that spans the
entire disk.

The next step is to change the disklabel type. To do that
I had to edit the disklabel:

disklabel -e ad1
disklabel -e ad2
disklabel -e ad3

This opened up the current disklabel on each disk respectively in
whatever editor the EDITOR environment variable was set to, in my
case, vi(1). Inside the editor I had a section like this:

8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 60074784        0    unused        0     0     0   # (Cyl.    0 - 59597)

I need to add a new “e” partition for ccd(4) to use. This
can be a copy of the “c” partition. Once I was done, my disklabel
shoud look like this:

8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  c: 60074784        0    unused        0     0     0   # (Cyl.    0 - 59597)
  e: 60074784        0    unused        0     0     0   # (Cyl.    0 - 59597)

Building the Filesystem

Now that I have all of the disks labeled, I needed to build the
ccd. To do that, I used a utility called ccdconfig(8). ccdconfig
takes several arguments, the first argument being the device to
configure, in this case, /dev/ccd0c. The device node for ccd0c may
not exsist yet, so to create it, preform the following commands:

cd /dev
sh MAKEDEV ccd0

The next argument ccdconfig expects is the interleave for
the filesystem. The interleave defines the size of a stripe in disk
blocks, normally five hundred and twelve bytes. So, an interleave of
thirty-two would be sixteen thousand three hundred and eighty-four
bytes.

After the interleave comes the flags for ccdconfig. If you
want to enable drive mirroring, you can specificy a flag here. In
this configuration, I am not mirroring the ccd, so I left it as
zero.

The final arguments to ccdconfig are the devices to place
into the array. Putting it all together I get this command:

ccdconfig ccd0 32 0 /dev/ad1e /dev/ad2e /dev/ad3e

This configures the ccd. I can now newfs(8) the filesystem.

newfs /dev/ccd0c

Making it All Automagic

Finally, if I want to be able to mount the ccd, I need to
configure it first. I write out my current configuration to
/etc/ccd.conf using the following command:

ccdconfig -g > /etc/ccd.conf

When I reboot, the script /etc/rc runs
ccdconfig -C if
/etc/ccd.conf exists. This automatically configures the ccd so
it can be mounted.

If you are booting into single user mode, before you can
mount the ccd, you need to issue the following command to configure
the array:

ccdconfig -C

Then, we need an entry for the ccd in /etc/fstab so it will
be mounted at boot time.

/dev/ccd0c              /media       ufs     rw      2       2

3 thoughts on “Concatenated Disk Configuration”

  1. I know the ccd driver is built into FreeBSD, but why not attempt the same with vinum plexes? I’ve actually had pretty good luck with vinum for multiple applications, and specifically for media servers, you can do a software RAID configuration to speed disk IO up a bit so you don’t get the jitters (but then again, I’d also like to have a fiber backbone so the disks were the least of my worries). Granted vinum may be a bit difficult to the beginner, but it works like a charm and is quite reliable once you get the hang of it. Plus, you can use a mix of drives (SCSI and IDE) if you’re crazy enough to do so,

  2. Hi.

    I’ve been futzing with ccd for the last four days and I’ve finally gotten it working correctly. Part of the problem was an error in this document (and the corresponding section of the handbook):

    The author states that the e: partition may be an exact copy of the line that sets up the c: partition. Then it would turn out to be:

    e: 60074784 0 unused 0 0 0 # (Cyl. 0 – 59597)

    But in reality, the "unused" filesystem type for the e: partition will cause ccdconfig to barf with the error:

    ccdconfig: ioctl (CCDIOCSET): /dev/ccd0c: Inappropriate file type or format

    and this is a FAQ:
    http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/disks.html#INAPPROPRIATE-CCD

    which, to sum up, states that the line above would rightfully look like this:

    e: 60074784 0 4.2BSD 0 0 0 # (Cyl. 0 – 59597)

    And when config’ing the ccd you’ll need to use the 4.2BSD partition entries you’ve created instead of the c parts described in the article:

    ccdconfig ccd0 64 0 /dev/da1e /dev/da2e

    I should point out that you *can* do this by changing the fstype of the c partition from "unused" to "4.2BSD" instead of creating a new partition and setting the fstype of that, but you’ll get scary errors from disklabel and who knows what other kind of trouble (if any) in the future.

    This was my experience, and now I have a working ccd 🙂 The key difference I can see from the article is that I was using SCSI drives, perhaps that explains the async experience.

Leave a Comment

Scroll to Top