NFS – sharing file systems across a network

NFS – sharing file systems across a network

This article was originally written in November 1998, but has only just been completed.  Yes, that’s slack.  I know.  But I never had a need for NFS until today.

When you install a port [from the Internet; not from a CD], files are downloaded from the Internet and stored in /usr/ports/distfiles.  If you install the same port on more than one machine, the file is downloaded again.  My goal is to minimize the Internet traffic and to increase the speed with which ports can be installed.  NFS allows a box (the server) to share it’s files with other boxes (the clients).  A given box can act as both and client and a server.  Thus, boxes can share files with other boxes.

You might also want to see the Samba article.

I used an article from FreeBSD’zine as the basis for this exercise.

NOTE: Under FreeBSD 6.x, the rpcbind utility is used in place of the portmap utility. Thus, in FreeBSD 6.x the user is required to replace every instance of portmap with rcpbind in the forthcoming examples.

What will you share?

I think the first step to implementing NFS is to decide what you are going to share.   In my case, I’m going to share /usr/ports/distfiles.  Normally, this is an easy thing to do, but in my situation, it turned out to be much more complex than I originally anticipated.

/etc/exports defines the remote mount point for NFS mount requests (straight from man exports).  It is in this file that you specify what you will share, how you will share it, and who you will share it with.

The following will share my /usr/ports/distfiles directory with the box at 192.168.0.10:

/usr/ports/distfiles 192.168.0.10

For more examples, see man exports and the FreeBSD’Zine article mentioned in the first part of this article.

showmount can be used to display the exports on a given server:

# showmount -e
Exports list on localhost:
/usr/home/www/ 192.168.0.78

But don’t run that until after you have started the NFS server.

Configuring the server

My first step was to configure the server.  This is the machine on which the files will reside.  Other machines (clients) will access the file on this box.   I did a quick check to see what options were available to me:

[root@fred:/etc] # grep nfs defaults/rc.conf

I decided upon the following options and added them to /etc/rc.conf:

nfs_server_enable="YES"       # This host is an NFS server (or NO)
nfs_server_flags="-u -t -n 4" # Flags to nfsd (if enabled).

You can either reboot or start the NFS server manually:

nfsd -u -t -n 4
mountd -r

For FreeBSD 6.x, you need these settings:

nfs_server_enable="YES"
rpcbind_enable="YES"
rpcbind_flags="-r"

And under 6.x, this starts the NFS server:

rpcbind
nfsd -u -t -n 4
mountd -r

Configuring the client

The client is much easier to configure.  I added this to /etc/rc.conf:

nfs_client_enable="YES" # This host is an NFS client (or NO).
nfs_client_flags="-n 4" # Flags to nfsiod (if enabled).

You can then either reboot, or start the client software manually:

nfsiod -n 4

Mounting the remote volume

To mount the remote volume, I issued the following command on a client:

# mount -v mybox:/usr/ports/distfiles /mnt
mybox:/usr/ports/distfiles on /mnt (nfs)

The above indicates a successful mount.  In this example, the server is the host mybox.   But I didn’t have an easy time of it.  See the next section for the problems I encountered.  Note that I didn’t actually use /usr/ports/distfiles and the above is only an example.

The -v flag produces additional information and is entirely optional.

The following entry in /etc/fstab on the client will mount the remote volume each time the box is restarted:

fred:/usr/ports/distfiles  /usr/ports/distfiles  nfs  rw   0   0

Problems I encountered

My first attempt to connect resulted in this message:

# mount fred:/usr/ports/distfiles /mnt
NFS Portmap: RPC: Port mapper failure - RPC: Timed out

I cleared out my firewall rules on the NFS server and tried again.  Note: the NFS server is a test box and the firewall rules were not necessary.  Don’t just clear out your firewall rules unless you are aware of the implications of doing so.

My next attempt resulted in this:

# mount fred:/usr/ports/distfiles /mnt
nfs: can't access /usr/ports/distfiles: Permission denied

I checked my logs and found this:

mountd[42593]: mount request denied from 192.168.0.10 for /nzmirror/ports/distfiles

Ahhh, yes, I remember now.   This box actually has a lot of symlinks on it because it contains three disks.  Look at this:

# ls -ld /usr/ports
lrwxr-xr-x 1 root wheel 16 Dec 24 20:21 /usr/ports -> /nzmirror/ports/

As you can see, the actual physical pathname is something different from what I was supplying in the exports file.  And if you check man exports, you’ll see that symbolic links are not permitted.  The following is from the third paragraph under DESCRIPTION:

The pathnames must not have any symbolic links in them and should not have any "." or ".." components.

Well, that explains that.  So I changed /etc/exports on the server to contain this:

/nzmirror/ports/distfiles 192.168.0.10

Remember that after making changes to /etc/exports, you should hup mountd:

killall -hup mountd

On the client, I tried that mount again:

mount fred:/nzmirror/ports/distfiles /mnt

The next problem I encountered was a permissions issue:

[root@ducky:/mnt] # mkdir temp
mkdir: temp: Permission denied

I tried many things to resolve this.  Eventually it went away.  I have no idea why.  Sorry.

6 May 2000

I’ve just experience a situationwhich may explain the above problem which just "went away".  When I was writing Tranferring websites/users from one box to another I had this in my exports on the server:

/usr/home/www 192.168.0.78

But this was how I was trying to mount it on the client:

# mount -v ducky:/usr/local/www /mnt
nfs: can't access /usr/local/www: Permission denied

The problem is the pathname.  Note that the exports contains "home" but the mount contains "local".  Trying to mount a non-existant export will give you a plain simple error like that.  Be sure to check the basics such as that.

4 October 2000

I was rebooting a NFS client when I spotted this message on the console:

nfs: bad MNT RPC: RPC: Timed out

This message repeated several times.  I checked the NFS server:

# showmount -e
RPC: Timed out
showmount: can't do exports rpc

So I restarted mountd and tried again:

# killall -term mountd
# mountd
# showmount -e
Exports list on localhost:
/usr/home/www/        10.0.0.1
/usr/home/justine     10.0.0.1
/usr/home/eimi        10.0.0.1
/usr/home/chrissy     10.0.0.1

But these messages kept repeating on the client:

nfs: bad MNT RPC: RPC: Timed out

I pressed CONTROL-C on the client’s console.  The client then completed the boot process.  But no NFS volumes were mounted.

So I mounted them manually, one at a time.  I have no idea what caused this error.   If you do, please add your comments.

5 October 2000

I figured it out.  It was a problem on the server.  After another reboot of the client, I found that the volumes were again not mounted.  So I mounted them by hand.  But I found one volume which could not be mounted.  I had forgotten that this client mounted NFS volumes from two different servers.  I went to the other NFS server, I killed mountd and nfsd  and then restarted them.   Then the volumes mounted properly on the client.

18 June 2004

Today I encounted this error for the first time. I was compiling a kernel on my fast box and while I was waiting I set up the slow box. I did this:

$ mount polo:/usr/src /usr/src
polo:/usr/src: RPCPROG_MNT: RPC: Authentication error; why = Client credential too weak

That confused me. I checked /var/log/messageson the NFS server:

mountd[95]: mount request from 10.0.0.20 from unprivileged port

Ahhh! I wasn’t root when I tried to mount! I su’d to root, and all was well.

Making use of a centralised /usr/ports/distfiles collection

My first step was to transfer the contents of /usr/ports/distfiles to the server.  Here is what I did on the box which contained most of my distfiles.   The following copied the distfiles from the client to the server (mybox).

# mount mybox:/usr/ports/distfiles /mnt
# cd /usr/ports/distfiles
# cp * /mnt

Then I changed the mounting situation.  On the client box, I did this:

# cd /usr/ports/
# mv distfiles distfiles.old
# mount mybox:/usr/ports/distfiles /usr/ports/distfiles

The above retains the existing distfiles in case of a problem.  It then mounts the remote directory where it normally resides.

To make this mount permanent, see Mounting the remote volume.

All exports from a single file system must be on the same export entry. For example:

/usr/ports /usr/ports/distfiles -maproot=0 -network 192.168.0.0 -mask 255.255.255.0

nfsd: RPCPROG_NFS: RPC: Program not registered

If you are seeing this on the client:

[root@laptop:/home/dan] # mount polo:/usr/obj /usr/obj
polo:/usr/obj: nfsd: RPCPROG_NFS: RPC: Program not registered

Then you should check the server for a message such as this in /var/log/messages: :

Jun 12 11:20:07 polo portmap[55049]: connect from 192.168.0.34 to getport(nfs): request from unauthorized host

If you find it, then help is at hand. Please read NFS Portmap: RPC: Program not registered for how I solved it.

rpcbind: connect from 192.168.0.34 to getport/addr(nfs): request from unauthorized host

I saw the following error on FreeBSD 6.2

# mount /usr/ports/distfiles ngaio:/usr/ports/distfiles: nfsd: RPCPROG_NFS: RPC: Port mapper failure - RPC: Authentication error

Checking on the server, I saw this error message:

Feb 16 22:36:25 ngaio rpcbind: connect from 10.55.0.18 to getport/addr(nfs): request from unauthorized host

You are missing an entry such as this from /etc/hosts.allow:

rpcbind : 192.168.0.0/255.255.255.0  : allow

That worked for me.

Diskless NFS box

If you want to set up a diskless NFS box, perhaps for a lab or classroom, have a look at this resource:

http://www.iet.unipi.it/~luigi/diskless.html

bad exports list line

If you see this:

bad exports list line /home/dan/distributions/4.9-RELEASE/binary/usr/src

Then you might be using a path that contains a symlink. Don’t do that. Use this path instead:

/usr/home/dan/distributions/4.9-RELEASE/binary/usr/src

NOTE: /home is often a symlink for /usr/home.