vinum – A simple install to improve performancevinum is a great tool. As the author says:
The Vinum Volume Manager is a block device driver which implements virtual disk drives. It isolates disk hardware from the block device interface and maps data in ways which result in an increase in flexibility, performance and reliability compared to the traditional slice view of disk storage. Vinum implements the RAID-0, RAID-1 and RAID-5 models, both individually and in combination.
The vinum home page is at http://www.vinumvm.org/. There is also lost article by Jake Burkholder which I used for the basis of this exercise. You should read vinum(4) and vinum(8) and before continuing with this article. It explains the concepts and terms used within this article (such as plexes, subdisks, and volumes) and which are necessary if you wish to implement vinum.
Why use vinum?Well, in my case, it was to create a larger volume and to increase performance. The box I was working on was destined to become the www and cvsup mirror for New Zealand FreeBSD User Group. But the box had only 1GB. So I removed those two 500MB drives and added in three 1GB drives. You may wish to read how I did the drive swap. If you wish to learn more about striping and RAID, please read http://www.lemis.com/vinum.html as I will not cover those topics at all.
The best vinum resources
Remember your sizeThe first step was to figure out what size my disks were. My first attempts involved assuming they were both 1GB drives. There weren’t. And this caused problems. So here’s how to find out the size. The drives I’m using are SCSI drives da1 and da2.
# fdisk da1 ******* Working on device /dev/rda1 ******* parameters extracted from in-core disklabel are: cylinders=496 heads=118 sectors/track=35 (4130 blks/cyl) parameters to be used for BIOS calculations are: cylinders=496 heads=118 sectors/track=35 (4130 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 165,(FreeBSD/NetBSD/386BSD) start 0, size 2051000 (1001 Meg), flag 80 (active) beg: cyl 0/ sector 1/ head 0; end: cyl 342/ sector 35/ head 117 The data for partition 2 is: <UNUSED> The data for partition 3 is: <UNUSED> The data for partition 4 is: <UNUSED> # fdisk da2 ******* Working on device /dev/rda2 ******* parameters extracted from in-core disklabel are: cylinders=4085 heads=14 sectors/track=36 (504 blks/cyl) Figures below won't work with BIOS for partitions not in cyl 1 parameters to be used for BIOS calculations are: cylinders=4085 heads=14 sectors/track=36 (504 blks/cyl) Media sector size is 512 Warning: BIOS sector numbering starts with sector 1 Information from DOS bootblock is: The data for partition 1 is: sysid 165,(FreeBSD/NetBSD/386BSD) start 0, size 2059140 (1005 Meg), flag 80 (active) beg: cyl 0/ sector 1/ head 0; end: cyl 953/ sector 36/ head 13 The data for partition 2 is: <UNUSED><unused> The data for partition 3 is: <UNUSED><unused> The data for partition 4 is:<unused> <UNUSED>
If you look above, you’ll find that da1 is 1001MB and da2 is 1005MB.
Disk partitionsI am assuming your have your disk partitions and slices ready to go. If you don’t, please see Formatting Media For Use With FreeBSD but for your reference, my two disk are "dangerously" dedicated.
Addenda – 26 August 1999When I originally wrote the section on File System Types, my default editor was vi. Then a reader sent me a message telling me telling me that that various UNIX commands use the EDITOR command. Thus, you can set your default editor for various commands. I thought that was brilliant! Of course I rushed off to write a new article about that, only to find I’d already written one and forgotten about it!
If you want to see how to set your default editor, see changing your default editor.
File system typesvinum only works on vinum type file systems. By default, the file systems are of type
unused. You can change the file system type (fstype) to "vinum" by using disklabel.
# disklabel -e da1
If you know how to use vi, then go ahead and ignore the following instructions. Otherwise this should help a big help:
- use the arrow keys to position yourself at the start of where it says
unusedunder the fstype column.
- press x to delete the existing fstype entry
- press i to start insert
- type vinum
- press escape to end insert mode
- press :wq, then enter to save your changes and exit
- OR presse :q! to exit without saving.
Here is a properly configured disk:
[root@fred:~] # disklabel da1 # /dev/rda1c: type: SCSI disk: da1s1 label: flags: bytes/sector: 512 sectors/track: 35 tracks/cylinder: 171 sectors/cylinder: 5985 cylinders: 342 sectors/unit: 2051000 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # milliseconds track-to-track seek: 0 # milliseconds drivedata: 0 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 2051000 0 unused 0 0 # (Cyl. 0 - 342*) e: 2051000 0 vinum # (Cyl. 0 - 342*)
Protect your dataThe process of creating a vinum volume will erase the data which exists on the file system. If you are converting existing disks to use vinum, back up your data first. You can restore the data later.
Create the configuration file
WARNING: Grey Lehey has said not to create striped plexes with a power of 2 stripe size. DO NOT USE THE 256k VALUES I USED.
As mentioned in Jake’s article, I used /etc/vinum.conf as my configuration file. Here is what it contains:
drive d1 device /dev/da1e drive d2 device /dev/da2e volume test plex org striped 256k <==== do not use a power of 2 like I did sd length 1001m drive d1 sd length 1001m drive d2
See the vinum documentation for more information. The new volume device I am creating will be called test. Also note that I am striping and specifying the same size (1001MB) for both drives. This is despite that fact that one drive actually contains 1005 MB. See below for more information on the error messages I encountered when I didn’t specify the correct sizes.
man vinum also contains a "HOW TO SET UP VINUM" section which gives various options. Please refer to that when deciding what to put into your configuration file.
Create the vinum devicesvinum uses it’s own set of devices to implement what it needs to do what it does best. These devices are located in /dev/vinum. To create these devices, here’s what I did:
# vinum create -f /etc/vinum.conf Configuration summary Drives: 2 (4 configured) Volumes: 1 (4 configured) Plexes: 1 (8 configured) Subdisks: 2 (16 configured) D d1 State: up Device /dev/da1e Avail: 0/1001 MB (0%) D d2 State: up Device /dev/da2e Avail: 4/1005 MB (0%) V test State: up Plexes: 1 Size: 2002 MB P test.p0 S State: up Subdisks: 2 Size: 2002 MB S test.p0.s0 State: up PO: 0 B Size: 1001 MB S test.p0.s1 State: up PO: 256 kB Size: 1001 MB
Creating the new file systemThe next step is to create the new file system. That’s really easy actually.
# newfs -v /dev/vinum/test newfs: /dev/vinum/test: not a character-special device /dev/vinum/test: 4100096 sectors in 1001 cylinders of 1 tracks, 4096 sectors 2002.0MB in 63 cyl groups (16 c/g, 32.00MB/g, 7936 i/g) super-block backups (for fsck -b #) at: 32, 65568, 131104, 196640, 262176, 327712, 393248, 458784, 524320, 589856, 655392, 720928, 786464, 852000, 917536, 983072, etc...
-v option is very interesting:
-v Specify that the disk does not contain any partitions, and that newfs should build a file system on the whole disk. This option is useful for synthetic disks such as vinum.
Creation errorsIf you don’t change your fstype to vinum, you will get the following error when you try to create.
1: drive d1 device /dev/da1e ** 1 Drive d1 has invalid partition type: Inappropriate file type or format 2: drive d2 device /dev/da2e ** 2 Drive d2 has invalid partition type: Inappropriate file type or format
Always check /var/log/messages after doing a vinum create. When I used the following configuration, I encountered my first problem:
sd length 1g drive d1 sd length 1g drive d2
But this configuration resulted in the following:
# vinum create -f /etc/vinum.conf 7: sd length 1g drive d1 ** 7 No space for test.p0.s0 on d1: No space left on device 8: sd length 1g drive d2 ** 8 Unnamed sd is not associated with a plex: Invalid argument Configuration summary Drives: 2 (4 configured) Volumes: 1 (4 configured) Plexes: 1 (8 configured) Subdisks: 0 (16 configured) D d1 State: up Device /dev/da1e Avail: 1001/1001 MB (99%) D d2 State: up Device /dev/da2e Avail: 1005/1005 MB (99%) V test State: down Plexes: 1 Size: 1024 MB P test.p0 S State: faulty Subdisks: 1 Size: 1024 MB S test.p0.s0 State: down PO: 0 B Size: 1024 MB
The log messages for this are:
Aug 9 20:30:45 fred /kernel: vinum: drive d1 is up Aug 9 20:30:45 fred /kernel: vinum: drive d2 is up Aug 9 20:30:45 fred /kernel: vinum: plex test.p0 does not have at least 2 subdisks Aug 9 20:30:45 fred last message repeated 2 times Aug 9 20:30:45 fred /kernel: vinum: test.p0 is faulty
So then I changed the configuration to:
sd length 1001m drive d1 sd length 1005m drive d2
And I tried again:
# vinum create -f /etc/vinum.conf Configuration summary Drives: 2 (4 configured) Volumes: 2 (4 configured) Plexes: 2 (8 configured) Subdisks: 2 (16 configured) D d1 State: up Device /dev/da1e Avail: 0/1001 MB (0%) D d2 State: up Device /dev/da2e Avail: 0/1005 MB (0%) V test State: down Plexes: 2 Size: 2006 MB P test.p0 S State: faulty Subdisks: 1 Size: 1024 MB P test.p1 S State: down Subdisks: 2 Size: 2006 MB S test.p0.s0 State: down PO: 0 B Size: 1024 MB S test.p1.s0 State: stale PO: 0 B Size: 1001 MB S test.p1.s1 State: stale PO: 256 kB Size: 1005 MB
Well, that looks much better! We’re making progress. At least, that’s what I thought. Note the faulty and down entries in the above. And if you check the logs, you’ll find.
Aug 9 20:34:28 fred /kernel: vinum: plex test.p0 does not have at least 2 subdisks Aug 9 20:34:28 fred /kernel: vinum: test.p1 must have equal sized subdisks Aug 9 20:34:28 fred /kernel: vinum: test.p1 is down
And if you try the newfs, you’ll get this:
# newfs -v /dev/vinum/test newfs: /dev/vinum/test: Input/output error
So the moral of the story: always check /var/log/messages if encounter an unusual error. And please note that after every failed vinum create, you must reset your vinum configuration. See the next section for more detail on how to do that.
Vinum resetOne thing to remember is that you sometimes have to remove your existing vinum configuration. If you encounter any errors during the create, you should clear out the vinum devices. This is because each create is cumulative. If youi keep doing creates, you will wind up with more and more devices. Which is not necesarily what you want.
Please note that resetting the configuration does not delete your data but you must do another create with your original configuration.
Here’s how I cleared out the vinum configuration:
# vinum resetconfig WARNING! This command will completely wipe out your vinum configuration. All data will be lost. If you really want to do this, enter the text NO FUTURE Enter text -> NO FUTURE Vinum configuration obliterated
resetconfig unless you want to lose everything. Instead, use
Mounting the vinum volumeThis too is quite easy.
# mount /dev/vinum/test /mnt
There you go! Welcome to the vinum world! And if I typed mount, here is what I found:
/dev/vinum/test on /mnt (local, writes: sync 2 async 1)
After all that, here is a set of successful logs from the above:
Aug 9 19:47:53 fred /kernel: vinum: drive d1 is up Aug 9 19:47:53 fred /kernel: vinum: drive d2 is up Aug 9 19:47:53 fred /kernel: vinum: test.p0.s0 is up Aug 9 19:47:53 fred /kernel: vinum: test.p0.s1 is up Aug 9 19:47:53 fred /kernel: vinum: test.p0 is up Aug 9 19:47:53 fred /kernel: vinum: test is up Aug 9 19:47:53 fred /kernel: vinum: test.p0 is up
Making it all automagicalThis section shows how I made sure the vinum volume was always mounted when the system started up.
First, I found the vinum options which are recognized by /etc/rc.conf.
# grep vinum /etc/defaults/rc.conf start_vinum="" # set to YES to start vinum
Then I added the following to /etc/rc.conf.
start_vinum="YES" # set to YES to start vinum vinum_drives="/dev/da1e /dev/da2e" # put in names of disks containing vinum drives # to start vinum on only specific disks.
Here’s what I found in dmesg:
# dmesg | grep vinum vinum: loaded vinum: reading configuration from /dev/da2e vinum: updating configuration from /dev/da1e