vinum – A simple install to improve performance

vinum – A simple install to improve performance

vinum is a great tool.  As
the author says:

The Vinum Volume Manager is a block device driver which implements virtual disk drives.
It isolates disk hardware from the block device interface and maps data in ways which
result in an increase in flexibility, performance and reliability compared to the
traditional slice view of disk storage. Vinum implements the RAID-0, RAID-1 and RAID-5
models, both individually and in combination.

The vinum home page is at http://www.vinumvm.org/.
  There is also lost article by Jake Burkholder
which I used for the basis of this exercise.  You should read
vinum(4) and
vinum(8) and before continuing with this article.  It explains the concepts and
terms used within this article (such as plexes, subdisks, and volumes) and which are
necessary if you wish to implement vinum.

There is also an
article
by Jake Burkholder in FreeBSD’Zine

Why use vinum?

Well, in my case, it was to create a larger volume and to increase performance.  
The box I was working on was destined to become the www and cvsup mirror for New Zealand FreeBSD User Group. But the box
had only 1GB.  So I removed those two 500MB drives and added in three 1GB drives.
  You may wish to read how I did the drive swap
If you wish to learn more about striping and RAID, please read http://www.lemis.com/vinum.html as I will not
cover those topics at all.

The best vinum resources

Remember your size

The first step was to figure out what size my disks were.  My first attempts
involved assuming they were both 1GB drives.  There weren’t.  And this caused
problems.  So here’s how to find out the size.  The drives I’m using are SCSI
drives da1 and da2.

# fdisk da1
******* Working on device /dev/rda1 *******
parameters extracted from in-core disklabel are:
cylinders=496 heads=118 sectors/track=35 (4130 blks/cyl)

parameters to be used for BIOS calculations are:
cylinders=496 heads=118 sectors/track=35 (4130 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
    start 0, size 2051000 (1001 Meg), flag 80 (active)
        beg: cyl 0/ sector 1/ head 0;
        end: cyl 342/ sector 35/ head 117
The data for partition 2 is:
<UNUSED>
The data for partition 3 is:
<UNUSED>
The data for partition 4 is:
<UNUSED>
# fdisk da2
******* Working on device /dev/rda2 *******
parameters extracted from in-core disklabel are:
cylinders=4085 heads=14 sectors/track=36 (504 blks/cyl)

Figures below won't work with BIOS for partitions not in cyl 1
parameters to be used for BIOS calculations are:
cylinders=4085 heads=14 sectors/track=36 (504 blks/cyl)

Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165,(FreeBSD/NetBSD/386BSD)
    start 0, size 2059140 (1005 Meg), flag 80 (active)
        beg: cyl 0/ sector 1/ head 0;
        end: cyl 953/ sector 36/ head 13
The data for partition 2 is:
<UNUSED><unused>
The data for partition 3 is:
<UNUSED><unused>
The data for partition 4 is:<unused>
<UNUSED>

If you look above, you’ll find that da1 is 1001MB and da2 is
1005MB.

Disk partitions

I am assuming your have your disk partitions and slices ready to go.  If you
don’t, please see Formatting Media For Use With FreeBSD
but for your reference, my two disk are "dangerously" dedicated.

Addenda – 26 August 1999

When I originally wrote the section on File System Types, my default editor was vi.
  Then a reader sent me a message telling me telling me that that various UNIX
commands use the EDITOR command.  Thus, you can set your default editor for various
commands.  I thought that was brilliant!  Of course I rushed off to write a new
article about that, only to find I’d already written one and forgotten about it!

If you
want to see how to set your default editor, see changing your
default editor
.

File system types

vinum only works on vinum type file systems.  By default, the
file systems are of type unused. You can change the file system type (fstype) to
"vinum" by using disklabel.

# disklabel -e da1

If you know how to use vi, then go ahead and ignore the following
instructions.   Otherwise this should help a big help:

  1. use the arrow keys to position yourself at the start of where it says unused
    under the fstype column.
  2. press x to delete the existing fstype entry
  3. press i to start insert
  4. type vinum
  5. press escape to end insert mode
  6. press :wq, then enter to save your changes and exit
  7. OR presse :q! to exit without saving.

Here is a properly configured disk:

[root@fred:~] # disklabel da1
# /dev/rda1c:
type: SCSI
disk: da1s1
label: 
flags:
bytes/sector: 512
sectors/track: 35
tracks/cylinder: 171
sectors/cylinder: 5985
cylinders: 342
sectors/unit: 2051000
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # milliseconds
track-to-track seek: 0  # milliseconds
drivedata: 0 

8 partitions:
#        size offset fstype   [fsize bsize bps/cpg]
  c:  2051000      0 unused        0     0  # (Cyl.    0 - 342*)
  e:  2051000      0  vinum                 # (Cyl.    0 - 342*)

Protect your data

The process of creating a vinum volume will erase the data which exists on
the file system.  If you are converting existing disks to use vinum, back up
your data first.  You can restore the data later.

Create the configuration file

WARNING: Grey Lehey has said not to create striped plexes with a power of 2 stripe size.
DO NOT USE THE 256k VALUES I USED.

As mentioned in Jake’s article, I used /etc/vinum.conf as my configuration
file.  Here is what it contains:

drive d1 device /dev/da1e
drive d2 device /dev/da2e

volume test

 plex org striped 256k  <==== do not use a power of 2 like I did
  sd length 1001m drive d1
  sd length 1001m drive d2

See the vinum documentation for more information.  The new volume device I am
creating will be called test.  Also note that I am striping and specifying
the same size (1001MB) for both drives.  This is despite that fact that one drive
actually contains 1005 MB.  See below for more information on the error messages I
encountered when I didn’t specify the correct sizes.

man vinum also contains a "HOW TO SET UP VINUM" section which
gives various options.  Please refer to that when deciding what to put into your
configuration file.

Create the vinum devices

vinum uses it’s own set of devices to implement what it needs to do what it
does best.   These devices are located in /dev/vinum.  To create
these devices, here’s what I did:

# vinum create -f /etc/vinum.conf 
Configuration summary

Drives:         2 (4 configured)
Volumes:        1 (4 configured)
Plexes:         1 (8 configured)
Subdisks:       2 (16 configured)

D d1          State: up  Device /dev/da1e   Avail: 0/1001 MB (0%)
D d2          State: up  Device /dev/da2e   Avail: 4/1005 MB (0%)

V test        State: up  Plexes:       1 Size:       2002 MB

P test.p0   S State: up  Subdisks:     2 Size:       2002 MB

S test.p0.s0  State: up  PO:        0  B Size:       1001 MB
S test.p0.s1  State: up  PO:      256 kB Size:       1001 MB

Creating the new file system

The next step is to create the new file system.  That’s really easy actually.

# newfs -v /dev/vinum/test
newfs: /dev/vinum/test: not a character-special device
/dev/vinum/test:    4100096 sectors in 1001 cylinders of 1 tracks,
4096 sectors
        2002.0MB in 63 cyl groups (16 c/g, 32.00MB/g, 7936 i/g)
super-block backups (for fsck -b #) at:
 32, 65568, 131104, 196640, 262176, 327712, 393248, 458784,
 524320, 589856, 655392, 720928, 786464, 852000, 917536, 983072,
 etc...

The -v option is very interesting:


-v      Specify that the disk does not contain any partitions, and that
        newfs should build a file system on the whole disk.  This option
        is useful for synthetic disks such as vinum.

Creation errors

If you don’t change your fstype to vinum, you will get the following error when you
try to create.

1: drive d1 device /dev/da1e
** 1 Drive d1 has invalid partition type: Inappropriate file 
                                 type or format
2: drive d2 device /dev/da2e
** 2 Drive d2 has invalid partition type: Inappropriate file 
                                 type or format

Always check /var/log/messages after doing a vinum create. 
When I used the following configuration, I encountered my first problem:

sd length 1g drive d1
sd length 1g drive d2

But this configuration resulted in the following:

# vinum create -f /etc/vinum.conf 
   7:   sd length 1g drive d1
** 7 No space for test.p0.s0 on d1: No space left on device
   8:   sd length 1g drive d2
** 8 Unnamed sd is not associated with a plex: Invalid argument
Configuration summary

Drives:         2 (4 configured)
Volumes:        1 (4 configured)
Plexes:         1 (8 configured)
Subdisks:       0 (16 configured)

D d1       State: up   Device /dev/da1e  Avail: 1001/1001 MB (99%)
D d2       State: up   Device /dev/da2e  Avail: 1005/1005 MB (99%)

V test     State: down     Plexes:       1 Size:       1024 MB

P test.p0  S State: faulty   Subdisks:     1 Size:       1024 MB

S test.p0.s0 State: down     PO:        0  B Size:       1024 MB

The log messages for this are:

Aug  9 20:30:45 fred /kernel: vinum: drive d1 is up
Aug  9 20:30:45 fred /kernel: vinum: drive d2 is up
Aug  9 20:30:45 fred /kernel: vinum: plex test.p0 does not have at 
                                              least 2 subdisks
Aug  9 20:30:45 fred last message repeated 2 times
Aug  9 20:30:45 fred /kernel: vinum: test.p0 is faulty

So then I changed the configuration to:

sd length 1001m drive d1
sd length 1005m drive d2

And I tried again:

# vinum create -f /etc/vinum.conf 
Configuration summary

Drives:         2 (4 configured)
Volumes:        2 (4 configured)
Plexes:         2 (8 configured)
Subdisks:       2 (16 configured)

D d1           State: up   Device /dev/da1e Avail: 0/1001 MB (0%)
D d2           State: up   Device /dev/da2e Avail: 0/1005 MB (0%)

V test         State: down     Plexes:       2 Size:      2006 MB

P test.p0    S State: faulty   Subdisks:     1 Size:      1024 MB
P test.p1    S State: down     Subdisks:     2 Size:      2006 MB

S test.p0.s0   State: down     PO:        0  B Size:      1024 MB
S test.p1.s0   State: stale    PO:        0  B Size:      1001 MB
S test.p1.s1   State: stale    PO:      256 kB Size:      1005 MB

Well, that looks much better!  We’re making progress.  At least, that’s what
I thought.  Note the faulty and down entries in the
above.  And if you check the logs, you’ll find.

Aug  9 20:34:28 fred /kernel: vinum: plex test.p0 does not have at
                                             least 2 subdisks
Aug  9 20:34:28 fred /kernel: vinum: test.p1 must have 
                                             equal sized subdisks
Aug  9 20:34:28 fred /kernel: vinum: test.p1 is down

And if you try the newfs, you’ll get this:

# newfs -v /dev/vinum/test
newfs: /dev/vinum/test: Input/output error

So the moral of the story: always check /var/log/messages if encounter an
unusual error.  And please note that after every failed vinum create, you must reset
your vinum configuration.  See the next section for more detail on how to do that.

Vinum reset

One thing to remember is that you sometimes have to remove your existing vinum
configuration.  If you encounter any errors during the create, you should clear out
the vinum devices.  This is because each create is cumulative.  If youi keep
doing creates, you will wind up with more and more devices.  Which is not necesarily
what you want.

Please note that resetting the configuration does not delete your data
but you must do another create with your original configuration.

Here’s how I cleared out the vinum configuration:

# vinum resetconfig
 WARNING!  This command will completely wipe out your vinum 
                                                  configuration.
 All data will be lost.  If you really want to do this, enter 
                                                    the text

 NO FUTURE
 Enter text -> NO FUTURE
 Vinum configuration obliterated

Don’t use resetconfig unless you want to lose everything. Instead, use rm.

Mounting the vinum volume

This too is quite easy.

# mount /dev/vinum/test /mnt

There you go!  Welcome to the vinum world!  And if I typed mount,
here is what I found:

/dev/vinum/test on /mnt (local, writes: sync 2 async 1)

After all that, here is a set of successful logs from the above:

Aug  9 19:47:53 fred /kernel: vinum: drive d1 is up
Aug  9 19:47:53 fred /kernel: vinum: drive d2 is up
Aug  9 19:47:53 fred /kernel: vinum: test.p0.s0 is up
Aug  9 19:47:53 fred /kernel: vinum: test.p0.s1 is up
Aug  9 19:47:53 fred /kernel: vinum: test.p0 is up
Aug  9 19:47:53 fred /kernel: vinum: test is up
Aug  9 19:47:53 fred /kernel: vinum: test.p0 is up

Making it all automagical

This section shows how I made sure the vinum volume was always mounted when
the system started up.

First, I found the vinum options which are recognized by /etc/rc.conf.

# grep vinum /etc/defaults/rc.conf
start_vinum=""  # set to YES to start vinum

Then I added the following to /etc/rc.conf.

start_vinum="YES"                  # set to YES to start vinum
vinum_drives="/dev/da1e /dev/da2e"
             # put in names of disks containing vinum drives
             # to start vinum on only specific disks.

Here’s what I found in dmesg:

# dmesg | grep vinum
vinum: loaded
vinum: reading configuration from /dev/da2e
vinum: updating configuration from /dev/da1e

12 thoughts on “vinum – A simple install to improve performance”

    1. Bob Van Valzah

      Looks like Dan and I recoginzed the same vaccuum at about the same time and wrote Vinum articles. The article I wrote is available as one of the FreeBSD doc project articles:

      http://www.freebsd.org/doc/en_US.ISO8859-1/articles/vinum/article.html

      I have a hard time being concise, so there’s more detail and rationale discussion there. Also, I have a section on recovery procedures. Running through them every now and then helps me sleep better at night 🙂

  1. Well…..here it is, just over two years after this article was posted. Myself, a Net-Enable Systems Administrator(NESA){i.e. w/o the net, I’d be dead meat!}, trying to create a backup server with a stack of 100gig drives. And, what a joy to find a plain concise article to help me out.

    Once I got past my own hardware problems with an additional Promise Ultra66 controller, I was set. {sidebar: I still don’t know what all that was about. ‘disklabel -e’ would fail even when you’d just "q" out of vi. I finally gave up. Switched two drives at a time onto the controller on the motherboard. ‘disklabel -e’ to change the fstype to "vinum". Put the drives back on the Promise. Did a /dev/MAKEDEV for all the drives.{ad4 – ad7} And THEN ‘disklabel -e’ each drive to change the "disk" entry.{which now worked but is apparently unneccessary.}

    Anyway, the only thing that tripped me up after that was the "init" status on my raid5 plex. But, the man page clearly states that raid5s must be initialized, so that was certainly not the article’s fault! :-/

    I don’t know if this post will go out into the echoing emptyness of the timeless internet, but I just wanted the freebsd diary to see activity on this article and to know that I was SO glad to have found it!

    Thanks again to the author and to this site for providing the info. {I’d like to thank the academy and….. oh, ok. I’ll shut up now.}

    ~r

  2. >>>>> "doug" == doug poland <doug.poland@omniresources.com> writes:

    >> Does someone have a cookbook procedure for building and/or
    >> upgrading to a Raid 1 or Raid 5 configuration using VINUM? I want
    >> as much as of the system on RAID as possible, specifically
    >> including /usr and (if possible) root (/).

    doug> This will get you going with vinum:
    doug> http://www.freebsddiary.org/vinum.php

    I have a different recipe that I’ve used successfully on 2 drive
    (raid-1) and 8 drive (raid-5) systems.

    For the following ad? will be a 2 drive raid-1 and da? will be an 8
    drive raid-5.

    1. install the bare minimum on drive da0 (or ad0).
    1a. Use the custom install
    1b. fdisk all drives.
    1c. (ad) disklabel
    ad0s1a 128M /
    ad0s1b 256M swap
    ad0s1g rest of disk /d1
    Disklabel ad1 with anything, we’ll change it in step 2.

    (da) disklabel
    da0s1a 128M /
    da0s1g rest of disk /d1
    disklabel rest with anything, we’ll change it later.

    1d. install minimum from menu.

    2. Boot from your minimum install, edit disklabels
    2a. unmount /d1 and any other partitions you’ve made
    2b. edit your disk 0 partion (disklabel -e) to change the g
    partition to be "vinum" instead of "4.2BSD"
    2c. (ad) "disklabel ad0 >ad0.label"
    "disklabel -R -r ad1 ad0.label"

    (da) "disklabel da0 >da0.label"
    "disklabel -R -r da1 da0.label"
    "disklabel -R -r da2 da0.label"
    edit (disklabel -e) your da2 label. Change the ‘a’
    partition to a ‘b’ partition and change it’s type to
    swap. Then copy it to da3 thru da7 (or whatever)
    using the same method above.
    2d. (ad) you have two swap partitions now … ad0s1b and
    ad1s1b. Add them to fstab and swapon -a

    (da) you have 6 possible swap partitions (da2s1b thru
    da7s1b). Add 4 of them to fstab and swapon -a.
    I don’t know why you can’t add six yet, but your
    raid sever probably shouldn’t swap that much :).

    3. vinum config. Run "vinum create" and add all your disks together
    into whatever RAID(s) you want. Make sure you start them, etc.
    Also… use mkfs. Finally mount your new vinum partition on /mnt.
    (if you’re creating multiple raid partitions, you might mkdir
    /mnt/usr /mnt/var /mnt/u, etc.) Put "vinum_start="YES"" in
    your /etc/rc.conf

    4. Copy /usr and/or /var to the new raid partitions. Edit fstab to
    put the new partitions into the right place. mv old /usr and /var
    to /usr-old and /var-old. Reboot.

    5. Fire up /stand/sysinstall and install everything _but_ bin that you
    want on the server. If everything’s working, you can also remove
    /usr-old and /var-old.

    (bonus steps)

    Your server is now working, but there are a few bonus steps:

    6. (ad) copy / to ad1s1a and make ad1s1a also bootable.

    6. (da) copy / to da1s1a and make da1s1a also bootable. You could
    also have more than one other drive that was bootable. It’s also
    possible to use the other 2 disks that don’t have swap as another
    raid partition for /var (raid-0 or raid-1).

    7. cvsup and make world (it is FreeBSD, not Linux, after all)!

    Here are the two configurations I have:

    — 8 drive SCSI RAID-5 —

    [1:14:2014]root@arbiter:/usr/ports/lang/python-doc-html> vinum dumpconfig
    Drive d0: Device /dev/da0s1g
    Created on at Sat May 5 12:42:19 2001
    Config last updated Mon May 28 17:09:41 2001
    Size: 18232800256 bytes (17388 MB)
    volume usr state up
    plex name usr.p0 state up org raid5 128s vol usr
    sd name usr.p0.s0 drive d0 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 0s
    sd name usr.p0.s1 drive d1 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 128s
    sd name usr.p0.s2 drive d2 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 256s
    sd name usr.p0.s3 drive d3 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 384s
    sd name usr.p0.s4 drive d4 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 512s
    sd name usr.p0.s5 drive d5 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 640s
    sd name usr.p0.s6 drive d6 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 768s
    sd name usr.p0.s7 drive d7 plex usr.p0 state up len 35610624s driveoffset 265s plexoffset 896s

    [1:7:2007]root@arbiter:/usr/ports/lang/python-doc-html> vinum list
    8 drives:
    D d0 State: up Device /dev/da0s1g Avail: 0/17388 MB (0%)
    D d1 State: up Device /dev/da1s1g Avail: 0/17388 MB (0%)
    D d2 State: up Device /dev/da2s1g Avail: 0/17388 MB (0%)
    D d3 State: up Device /dev/da3s1g Avail: 0/17388 MB (0%)
    D d4 State: up Device /dev/da4s1g Avail: 0/17388 MB (0%)
    D d5 State: up Device /dev/da5s1g Avail: 0/17388 MB (0%)
    D d6 State: up Device /dev/da6s1g Avail: 0/17388 MB (0%)
    D d7 State: up Device /dev/da7s1g Avail: 0/17388 MB (0%)

    1 volumes:
    V usr State: up Plexes: 1 Size: 118 GB

    1 plexes:
    P usr.p0 R5 State: up Subdisks: 8 Size: 118 GB

    8 subdisks:
    S usr.p0.s0 State: up PO: 0 B Size: 16 GB
    S usr.p0.s1 State: up PO: 64 kB Size: 16 GB
    S usr.p0.s2 State: up PO: 128 kB Size: 16 GB
    S usr.p0.s3 State: up PO: 192 kB Size: 16 GB
    S usr.p0.s4 State: up PO: 256 kB Size: 16 GB
    S usr.p0.s5 State: up PO: 320 kB Size: 16 GB
    S usr.p0.s6 State: up PO: 384 kB Size: 16 GB
    S usr.p0.s7 State: up PO: 448 kB Size: 16 GB

    [1:8:2008]root@arbiter:/usr/ports/lang/python-doc-html> cat /etc/rc.conf
    inetd_enable="NO"
    sendmail_enable="NO"
    sshd_enable="YES"
    usbd_enable="NO"
    start_vinum="YES"
    nfs_server_enable="YES"
    nfs_server_flags="-u -t -n 8"
    nfs_client_enable="YES"
    ifconfig_rl0="inet blah netmask 255.255.255.252 up"
    defaultrouter="blah"
    hostname="blah.velocet.net"

    [1:9:2009]root@arbiter:/usr/ports/lang/python-doc-html> disklabel da0

    8 partitions:
    # size offset fstype [fsize bsize bps/cpg]
    a: 262144 0 4.2BSD 1024 8192 16 # (Cyl. 0 – 16*)
    c: 35873082 0 unused 0 0 # (Cyl. 0 – 2232*)
    g: 35610938 262144 vinum # (Cyl. 16*- 2232*)

    [1:11:2011]root@arbiter:/usr/ports/lang/python-doc-html> disklabel da2

    8 partitions:
    # size offset fstype [fsize bsize bps/cpg]
    b: 262144 0 swap # (Cyl. 0 – 16*)
    c: 35873082 0 unused 0 0 # (Cyl. 0 – 2232*)
    g: 35610938 262144 vinum # (Cyl. 16*- 2232*)

    [1:12:2012]root@arbiter:/usr/ports/lang/python-doc-html> df -k
    Filesystem 1K-blocks Used Avail Capacity Mounted on
    /dev/da0s1a 127023 112459 4403 96% /
    /dev/vinum/usr 120802676 10283498 100854964 9% /usr

    [1:13:2013]root@arbiter:/usr/ports/lang/python-doc-html> pstat -s
    Device 1K-blocks Used Avail Capacity Type
    /dev/da2s1b 130944 8 130936 0% Interleaved
    /dev/da3s1b 130944 0 130944 0% Interleaved
    /dev/da4s1b 130944 0 130944 0% Interleaved
    /dev/da5s1b 130944 0 130944 0% Interleaved
    Total 523776 8 523768 0%

    — 2 drive IDE RAID-1 —

    [1:13:326]root@sheppard:~> vinum dumpconfig
    Drive d0: Device /dev/ad0s1f
    Created on freenet.velocet.ca at Fri Nov 3 13:52:16 2000
    Config last updated Tue May 29 12:26:36 2001
    Size: 30064460288 bytes (28671 MB)
    volume var state up
    volume usr state up
    volume u state up
    plex name var.p0 state up org concat vol var
    plex name var.p1 state up org concat vol var
    plex name usr.p0 state up org concat vol usr
    plex name usr.p1 state up org concat vol usr
    plex name u.p0 state up org concat vol u
    plex name u.p1 state up org concat vol u
    sd name var.p0.s0 drive d0 plex var.p0 state up len 4194304s driveoffset 265s plexoffset 0s
    sd name var.p1.s0 drive d1 plex var.p1 state up len 4194304s driveoffset 265s plexoffset 0s
    sd name usr.p0.s0 drive d0 plex usr.p0 state up len 8388608s driveoffset 4194569s plexoffset 0s
    sd name usr.p1.s0 drive d1 plex usr.p1 state up len 8388608s driveoffset 4194569s plexoffset 0s
    sd name u.p0.s0 drive d0 plex u.p0 state up len 46136472s driveoffset 12583177s plexoffset 0s
    sd name u.p1.s0 drive d1 plex u.p1 state up len 46136472s driveoffset 12583177s plexoffset 0s

    [1:4:317]root@sheppard:~> vinum list
    2 drives:
    D d0 State: up Device /dev/ad0s1f Avail: 0/28671 MB (0%)
    D d1 State: up Device /dev/ad1s1f Avail: 0/28671 MB (0%)

    3 volumes:
    V var State: up Plexes: 2 Size: 2048 MB
    V usr State: up Plexes: 2 Size: 4096 MB
    V u State: up Plexes: 2 Size: 21 GB

    6 plexes:
    P var.p0 C State: up Subdisks: 1 Size: 2048 MB
    P var.p1 C State: up Subdisks: 1 Size: 2048 MB
    P usr.p0 C State: up Subdisks: 1 Size: 4096 MB
    P usr.p1 C State: up Subdisks: 1 Size: 4096 MB
    P u.p0 C State: up Subdisks: 1 Size: 21 GB
    P u.p1 C State: up Subdisks: 1 Size: 21 GB

    6 subdisks:
    S var.p0.s0 State: up PO: 0 B Size: 2048 MB
    S var.p1.s0 State: up PO: 0 B Size: 2048 MB
    S usr.p0.s0 State: up PO: 0 B Size: 4096 MB
    S usr.p1.s0 State: up PO: 0 B Size: 4096 MB
    S u.p0.s0 State: up PO: 0 B Size: 21 GB
    S u.p1.s0 State: up PO: 0 B Size: 21 GB

    [1:5:318]root@sheppard:~> cat /etc/rc.conf
    # This file now contains just the overrides from /etc/defaults/rc.conf
    # please make all changes to this file.

    # — sysinstall generated deltas — #
    ifconfig_rl0="inet blah netmask 255.255.255.0"
    defaultrouter="199.71.188.30"
    inetd_enable="YES"
    #ifconfig_rl0="DHCP"
    hostname="blah"
    sendmail_enable="YES"
    sshd_enable="YES"
    usbd_enable="NO"
    start_vinum="YES"
    portmap_enable="NO"

    [1:7:320]root@sheppard:~> disklabel ad0

    8 partitions:
    # size offset fstype [fsize bsize bps/cpg]
    a: 262144 0 4.2BSD 1024 8192 16 # (Cyl. 0 – 260*)
    b: 1048576 262144 swap # (Cyl. 260*- 1300*)
    c: 60030369 0 unused 0 0 # (Cyl. 0 – 59553*)
    f: 58719649 1310720 vinum # (Cyl. 1300*- 59553*)

    Filesystem 1K-blocks Used Avail Capacity Mounted on
    /dev/ad0s1a 127023 114415 2447 98% /
    mfs:29 507639 3 467025 0% /tmp
    /dev/vinum/var 2032623 34395 1835619 2% /var
    /dev/vinum/usr 4065262 1523152 2216890 41% /usr
    /dev/vinum/u 22358577 4457713 16112178 22% /u

    [1:10:323]root@sheppard:~> pstat -s
    Device 1K-blocks Used Avail Capacity Type
    /dev/ad0s1b 524160 0 524160 0% Interleaved
    /dev/ad1s1b 524160 0 524160 0% Interleaved
    Total 1048320 0 1048320 0%

    1. Nicolai Schlenzig

      This article/commet was helping me the most. I had problems getting more than 2 drives in my vinum setup, but after I actually created a partition of type freebsd in the visual disklabel and then replaced "freebsd" with "vinum" via "disklabel -e adXs1", it looks all good.

      Maybe I could have done something via disklabel – I don’t know. All I could see was errors when I tried to change the partition type with 3rd and later disks.

      For all disks I have 2 partition lines: 1 saying unused and 1 vinum now, like:

      c: 241248042 0 unused 0 0 # (Cyl. 0 – 15016*)
      e: 241248042 0 vinum # (Cyl. 0 – 15016*)

      Then "vinum -> concat -v /dev/adXs1e /dev/adX..Ns1e" and "newfs -v /dev/vinum/vinum0" left me with a nice raid.

      Thanks! 🙂

  3. I was asked to build a 1TB RAID5 FreeBSD Server for under $5000 and since I had used vinum before in small (2 striped disks) configurations (very painless, thanks to this article!) I decided to use 8 160GB ide disks, 8 acard aec-7720uw scsi bridges on a board with onboard aic7896 scsi.

    Not quite as painless.

    The fdisk part was easy, but while disklabel worked great on da1, da2 and da3 it would not show, or let me change da4s1e
    (same problem on da5/6/7/8)
    For some reason I had to use ‘disklabel -e da4s1’ for those disks.
    (same disks, same controller .. any ideas ?)

    As mentioned in the previous comment, getting the raid5 volume ‘up’ wasnt obvious to me either .. turns out you have to start the raid plex rather than the volume.
    (# vinum start audio.p0 in my case)
    newfs initiall complained about a ‘preposterous size’, because yes, 8 times 160GB in RAID5 is bigger than 1024GB.

  4. This is a fantastic howto. I setup my Stripe with ease.

    Now my only problem is that I have lost my configuration. The drive with the OS has failed.

    I only have a rough idea of my previous vinum configuration, and have no idea on how to go about restoring my stripe. I am asking for any advise on this as I do not want to do attempt restoring the stripe, do something wrong , and lose my data.

    Any help would be appreciated.

    1. Jason Teddy wrote:

      > This is a fantastic howto. I setup my Stripe with ease.

      Thanks.

      > Now my only problem is that I have lost my configuration. The
      > drive with the OS has failed.
      >
      > I only have a rough idea of my previous vinum configuration,
      > and have no idea on how to go about restoring my stripe. I am
      > asking for any advise on this as I do not want to do attempt
      > restoring the stripe, do something wrong , and lose my data.
      >
      > Any help would be appreciated.

      You should seek help in the Support forum.


      The Man Behind The Curtain

Leave a Comment

Scroll to Top