NFS – sharing file systems across a network
This article was originally written in November 1998, but has only just been
completed. Yes, that’s slack. I know. But I never had a need for NFS
until today.
When you install a port [from the Internet; not from a CD], files are
downloaded from the Internet and stored in /usr/ports/distfiles. If you
install the same port on more than one machine, the file is downloaded again. My
goal is to minimize the Internet traffic and to increase the speed with which ports can be
installed. NFS allows a box (the server) to share it’s files with other boxes (the
clients). A given box can act as both and client and a server. Thus, boxes can
share files with other boxes.
You might also want to see the Samba article.
I used an article from FreeBSD’zine as the basis for this exercise.
NOTE: Under FreeBSD 6.x, the rpcbind utility is used in place of the portmap
utility. Thus, in FreeBSD 6.x the user is required to replace every instance of portmap
with rcpbind in the forthcoming examples.
What will you share?
I think the first step to implementing NFS is to decide what you are going to share.
In my case, I’m going to share /usr/ports/distfiles. Normally,
this is an easy thing to do, but in my situation, it turned out to be much more complex
than I originally anticipated.
/etc/exports defines the remote mount point
for NFS mount requests (straight from man exports). It is in this file
that you specify what you will share, how you will share it, and who you will share it
with.
The following will share my /usr/ports/distfiles directory with the box at
192.168.0.10:
/usr/ports/distfiles 192.168.0.10
For more examples, see man exports and the FreeBSD’Zine article mentioned in the first
part of this article.
showmount can be used to display the exports on a given server:
# showmount -e
Exports list on localhost:
/usr/home/www/ 192.168.0.78
But don’t run that until after you have started the NFS server.
Configuring the server
My first step was to configure the server. This is the machine on which the
files will reside. Other machines (clients) will access the file on this box.
I did a quick check to see what options were available to me:
[root@fred:/etc] # grep nfs defaults/rc.conf
I decided upon the following options and added them to /etc/rc.conf:
nfs_server_enable="YES" # This host is an NFS server (or NO)
nfs_server_flags="-u -t -n 4" # Flags to nfsd (if enabled).
You can either reboot or start the NFS server manually:
nfsd -u -t -n 4
mountd -r
For FreeBSD 6.x, you need these settings:
nfs_server_enable="YES"
rpcbind_enable="YES"
rpcbind_flags="-r"
And under 6.x, this starts the NFS server:
rpcbind
nfsd -u -t -n 4
mountd -r
Configuring the client
The client is much easier to configure. I added this to /etc/rc.conf
:
nfs_client_enable="YES" # This host is an NFS client (or NO).
nfs_client_flags="-n 4" # Flags to nfsiod (if enabled).
You can then either reboot, or start the client software manually:
nfsiod -n 4
Mounting the remote volume
To mount the remote volume, I issued the following command on a client:
# mount -v mybox:/usr/ports/distfiles /mnt
mybox:/usr/ports/distfiles on /mnt (nfs)
The above indicates a successful mount. In this example, the server is the host mybox.
But I didn’t have an easy time of it. See the next section for the problems I
encountered. Note that I didn’t actually use /usr/ports/distfiles and the above is
only an example.
The -v flag produces additional information and is entirely optional.
The following entry in /etc/fstab on the client will mount the remote volume
each time the box is restarted:
fred:/usr/ports/distfiles /usr/ports/distfiles nfs rw 0 0
Problems I encountered
My first attempt to connect resulted in this message:
# mount fred:/usr/ports/distfiles /mnt
NFS Portmap: RPC: Port mapper failure - RPC: Timed out
I cleared out my firewall rules on the NFS server and tried again. Note: the NFS
server is a test box and the firewall rules were not necessary. Don’t just clear out
your firewall rules unless you are aware of the implications of doing so.
My next attempt resulted in this:
# mount fred:/usr/ports/distfiles /mnt
nfs: can't access /usr/ports/distfiles: Permission denied
I checked my logs and found this:
mountd[42593]: mount request denied from 192.168.0.10 for
/nzmirror/ports/distfiles
Ahhh, yes, I remember now. This box actually has a lot of symlinks on it
because it contains three disks. Look at this:
# ls -ld /usr/ports
lrwxr-xr-x 1 root wheel 16 Dec 24 20:21 /usr/ports -> /nzmirror/ports/
As you can see, the actual physical pathname is something different from what I was
supplying in the exports file. And if you check man exports, you’ll see
that symbolic links are not permitted. The following is from the third paragraph
under DESCRIPTION:
The pathnames must not have any symbolic links in them and
should not have any "." or ".." components.
Well, that explains that. So I changed /etc/exports on the server to
contain this:
/nzmirror/ports/distfiles 192.168.0.10
Remember that after making changes to /etc/exports, you should hup mountd:
killall -hup mountd
On the client, I tried that mount again:
mount fred:/nzmirror/ports/distfiles /mnt
The next problem I encountered was a permissions issue:
[root@ducky:/mnt] # mkdir temp
mkdir: temp: Permission denied
I tried many things to resolve this. Eventually it went away. I have no
idea why. Sorry.
6 May 2000
I’ve just experience a situationwhich may explain the above problem which just "went
away". When I was writing Tranferring
websites/users from one box to another I had this in my exports on the server:
/usr/home/www 192.168.0.78
But this was how I was trying to mount it on the client:
# mount -v ducky:/usr/local/www /mnt
nfs: can't access /usr/local/www: Permission denied
The problem is the pathname. Note that the exports contains "home" but
the mount contains "local". Trying to mount a non-existant export will
give you a plain simple error like that. Be sure to check the basics such as that.
4 October 2000
I was rebooting a NFS client when I spotted this message on the console:
nfs: bad MNT RPC: RPC: Timed out
This message repeated several times. I checked the NFS server:
# showmount -e
RPC: Timed out
showmount: can't do exports rpc
So I restarted mountd and tried again:
# killall -term mountd
# mountd
# showmount -e
Exports list on localhost:
/usr/home/www/ 10.0.0.1
/usr/home/justine 10.0.0.1
/usr/home/eimi 10.0.0.1
/usr/home/chrissy 10.0.0.1
But these messages kept repeating on the client:
nfs: bad MNT RPC: RPC: Timed out
I pressed CONTROL-C on the client’s console. The client then completed the boot
process. But no NFS volumes were mounted.
So I mounted them manually, one at a time. I have no idea what caused this error.
If you do, please add your comments.
5 October 2000
I figured it out. It was a problem on the server. After another reboot
of the client, I found that the volumes were again not mounted. So I mounted them by
hand. But I found one volume which could not be mounted. I had forgotten that
this client mounted NFS volumes from two different servers. I went to the other NFS
server, I killed mountd and nfsd and then restarted them.
Then the volumes mounted properly on the client.
18 June 2004
Today I encounted this error for the first time. I was compiling a
kernel on my fast box and while I was waiting
I set up the slow box. I did this:
$ mount polo:/usr/src /usr/src
polo:/usr/src: RPCPROG_MNT: RPC: Authentication error; why = Client credential too weak
That confused me. I checked /var/log/messages
on the NFS server:
mountd[95]: mount request from 10.0.0.20 from unprivileged port
Ahhh! I wasn’t root when I tried to mount! I su’d to root, and all was well.
Making use of a centralised /usr/ports/distfiles collection
My first step was to transfer the contents of /usr/ports/distfiles to the
server. Here is what I did on the box which contained most of my distfiles.
The following copied the distfiles from the client to the server (mybox).
# mount mybox:/usr/ports/distfiles /mnt
# cd /usr/ports/distfiles
# cp * /mnt
Then I changed the mounting situation. On the client box, I did this:
# cd /usr/ports/
# mv distfiles distfiles.old
# mount mybox:/usr/ports/distfiles /usr/ports/distfiles
The above retains the existing distfiles in case of a problem. It then mounts the
remote directory where it normally resides.
To make this mount permanent, see Mounting the remote volume.
All exports from a single file system must be on the same export entry. For example:
/usr/ports /usr/ports/distfiles -maproot=0 -network 192.168.0.0 -mask 255.255.255.0
nfsd: RPCPROG_NFS: RPC: Program not registered
If you are seeing this on the client:
[root@laptop:/home/dan] # mount polo:/usr/obj /usr/obj
polo:/usr/obj: nfsd: RPCPROG_NFS: RPC: Program not registered
Then you should check the server for a message such as this in /var/log/messages
:
:
Jun 12 11:20:07 polo portmap[55049]: connect from 192.168.0.34 to getport(nfs): request from unauthorized host
If you find it, then help is at hand. Please read NFS Portmap: RPC: Program not registered
for how I solved it.
rpcbind: connect from 192.168.0.34 to getport/addr(nfs): request from unauthorized host
I saw the following error on FreeBSD 6.2
# mount /usr/ports/distfiles
ngaio:/usr/ports/distfiles: nfsd: RPCPROG_NFS: RPC: Port mapper failure - RPC: Authentication error
Checking on the server, I saw this error message:
Feb 16 22:36:25 ngaio rpcbind: connect from 10.55.0.18 to getport/addr(nfs): request from unauthorized host
You are missing an entry such as this from /etc/hosts.allow:
rpcbind : 192.168.0.0/255.255.255.0 : allow
That worked for me.
Diskless NFS box
If you want to set up a diskless NFS box, perhaps for a lab or classroom,
have a look at this resource:
bad exports list line
If you see this:
bad exports list line /home/dan/distributions/4.9-RELEASE/binary/usr/src
Then you might be using a path that contains a symlink. Don’t do that. Use
this path instead:
/usr/home/dan/distributions/4.9-RELEASE/binary/usr/src
NOTE: /home
is often a symlink for /usr/home
.
"NFS Portmap: RPC: Port mapper failure – RPC: Timed out" error will also occur if you have portmap disabled on NFS server. 🙂
Make sure that portmap_enable is set to YES in /etc/rc.conf
Regards,
Roman
(posted by Dan Langille)
Hi, I was just setting up NFS on two FreeBSD 4.2 boxes for the first time and used your write up as a guide. It was very helpful however you had a few problems that where not explained very well.
You say…..
The next problem I encountered was a permissions issue:
[root@ducky:/mnt] # mkdir temp
mkdir: temp: Permission denied
I tried many things to resolve this. Eventually it went away. I have no idea why. Sorry.
which could have been fixed with a few changes to /etc/exports. *should* read something like the following;
/export/path1 -alldirs clientPCName
/path2 -maproot=username clientPCName
this will allow /export/path1 to be read only by the PC ‘clientPCName’ and /path2 will give clientPCName the same permissions as ‘username’ to the /path2 directory tree.
a simple ‘man exports’ explains a number of the options for this file
Hope that makes sense and thanx for putting the page up, it had me sharing files in no time at all :]
A couple good reads for improving NFS perf once you get your mounts working properly:
http://www.netsys.com/sunmgr/1997-09/msg00070.html – This gives a good working overview of tcp vs udp connections and when to use which.
http://www.freebsd.org/cgi/man.cgi?query=mount_nfs&apropos=0&sektion=8&manpath=FreeBSD+5.0-current&format=html – It seems obvious, but this shows all the little tweaks you can apply to your mounts to wind up the performance.
OK, Im getting a Program not registered error when trying to connect to an nfs server.
Now these two machines used to talk together. due to a problem I had to rebuild. Im running FreeBSD 5 on the client and FreeBSD 4 on the server.
i had the RPC time out error as well, but when I flush IPFW it didnt help. its when they actually can ping each other that i get the Program not registerd
see below
[udp] 10.0.0.11:/home/c: RPCPROG_NFS: RPC: Program not registered
1 – If you are looking for help, please post in the Support Forum.
2 – have you read <A HREF="/nfs-portmap.php">NFS Portmap: RPC: Program not registered</A>?
You say that all exports from a single fs must be exported on one line, man 5 exports says the following:
—
Each line in the file (other than comment lines that begin with a #)specifies the mount point(s) and export flags within one local server filesystem for one or more hosts. A host may be specified only once for each local filesystem on the server and there may be only one default entry for each server filesystem that applies to all other hosts. The latter exports the filesystem to the “world” and should be used only when the filesystem contains public information.
—
I see this as allowing multiple exports on a filesystem on one line, but not requiring that all exports for a given file system must be on a single line. I have NFS exports from a single filesystem on multiple lines and it appears to work fine so far.
Furthermore, if this were indeed a requirement, it’d be impossible to have two mutually exclusive exports on a single filesystem. That is, to export /usr/ports to host A and /usr/share to host B where /usr/ is a single filesystem.
Corrections welcome,
mike
Mike Erickson wrote:
>
> You say that all exports from a single fs must be
> exported on one line, man 5 exports says the following:
>
> —
> Each line in the file (other than comment lines that begin
> with a #)specifies the mount point(s) and export flags within
> one local server filesystem for one or more hosts. A host
> may be specified only once for each local filesystem on the
> server and there may be only one default entry for each
> server filesystem that applies to all other hosts. The
> latter exports the filesystem to the “world” and should be
> used only when the filesystem contains public information.
> —
I find the above unnecessarily difficult to comprehend. A fault easily fixed by a few practical examples. This is a fault I find common amongst many many man pages.
> I see this as allowing multiple exports on a filesystem on
> one line, but not requiring that all exports for a given file
> system must be on a single line. I have NFS exports from a
> single filesystem on multiple lines and it appears to work
> fine so far.
Have you been able to do the following on two lines?
/usr/ports /usr/ports/distfiles -maproot=0 -network 192.168.0.0 – mask 255.255.255.0
No, I haven’t been able to do that one two lines.
I agree that the man page is unnecessarily difficult to comprehend. I’d like to try to reword it and submit a PR and patch. Any suggestions on how to express this? I’m not very confident of my own understanding.
I think breaking up this sentence is the key:
> A host
> may be specified only once for each local filesystem on the
> server and there may be only one default entry for each
> server filesystem that applies to all other hosts.
pre-coffee verbose (and probably erroneous) breakdown:
A host may be specified only once for each local filesystem. The way to get around this is to use the -alldirs flag and then selectively mount different points on a server-local filesystem from the client.
I still don’t understand the comment regarding default entries.
mike
i want to know about file system aor file management in freebsd, at least the concept to reference my task in class