Build world on your fast box, install on your slow box

Build world on your fast box, install on your slow box

The build world process is used to upgrade FreeBSD from one version to
another (e.g. 4.2-RELEASE to 4.2-STABLE).  This process involves downloading source
code, compiling it,  installing it, and resolving any configuration file issues,
although not necessarily in that order.  The best place to read about this process is
the FreeBSD Handbook
(just search for NFS).

In my case, I have a fast dual Xeon box,
and a much slower 486 box (see "my first" at the bottom right of this page).  When I first did a build world, it took
57 hours on my 486 with 16MB of RAM.  That’s not something I’m likely to do very
often.  Fortunately, the build world process is divided up nicely into distinct
sections which will me to build world on the Xeon, and then install to the 486.

This process uses NFS, so we’ll assume you have NFS configured
on both boxes. I’ve configured my Xeon box as an NFS server and my 486 as an NFS client.
  Full instructions on how to do these configurations are included in the NFS link

13 March 2000 – I updated the script file and modified this article
accordingly.  Things have changed in the kernel building arena, so I’ve added
KERNCONF to the makefile. For more information on KERNCONF, see
the man page for

I also added three new options to the makefile:

  • kernelbuild
  • kernelinstall
  • cleanlogs

I also changed the format of the log files:

  • <action>.YYYY.MM.DD.HH.MM.SS.log

The value of <action> depends on what you did:




cvsup results


build world


install world


build and install kernel


build kernel


kernel install



You can also delete the logs with make cleanlogs.

A few warnings

You must ensure that you use the same settings for building as for
installing . If you don’t, you could be asking for trouble.

  1. Any -D variables defined in the buildworld must also be supplied in the installworld
    just like the makeworld handbook
    page says.  In short, any options you use on the make buildworld must also be used on
    the make installworld.  There will be tears if you don’t.
  2. Don’t set CPU specific optimization flags and then do the installworld on a other-than
    cpu machine.
  3. Read /usr/src/UPDATING.  It contains the latest information about
    building.  The information on build world can quickly become dated.

Those are the primary things you should consider.  But some recent talk has been
about /etc/make.conf on each box contain similar settings.  Remember to
take into account /etc/defaults/make.conf.   Most people will not have
to worry about this.  If you have no idea what I’m talking about, then you probably
don’t need to be concerned.

The other thing to note is that I am using a custom makefile for this process.  
You don’t have to use this makefile, but I’ve found it quite useful.  See the Resources section for details on this makefile.  As much as is
possible, I will give both sets of commands: the command for my makefile and the command
the makefile actually issues.  I use this makefile because it is easy, convenient,
saves typing, and automatically creates logs for me.  I normally put this makefile in
/root/make but it can reside anywhere.  I merely find that location
convenient so I can easily do this:

[dan@xeon:~] $ su -l
[root@xeon:~] # cd make
[root@xeon:~/make] #

From there, I can issue all the commands which appear in the following
sections…unless otherwise stated.

NOTE: I put this makefile on both boxes, the XEON and the 486.  I configured each
box accordingly:

  • on the XEON box, I put MYKERNEL=XEON
  • on the 486, I put MYKERNEL=DUCKY

If you don’t set them correctly on each box, you might wind up installing the wrong
kernel.  That’s no big deal.  Just follow the instructions in the handbook for
booting from an alternative kernel.  Here’s the short version: press space at the
booting countdown, then enter unload, then enter boot kernel.old (or
whatever kernel you want; type ls for a list).


I’m using my make world script
(originally from Jim Mock), which isn’t necessary, but keeps the commands handy and keeps
nice logs.  I will assume you have already downloaded this script, installed it, and
configured it.

I found out how to do this type of install by reading the FreeBSD
Handbook (available both online and in book form).  In the handbook, I found this section on make world.
  Search the page for a reference to NFS (it’s under Version
2.2.5 and above

NFS exports – on the Xeon (server)

Here are the NFS exports I put on my Xeon box. is the IP
address of my 486.

[root@xeon:~] # more /etc/exports
/usr/src /usr/obj

Note that all exported directories within the same file system must appear on one line.
  By default, /usr/src and /usr/obj are on the same file

NFS imports – on the 486 (client)

My 486 already had a /usr/src and a /usr/obj.
  But I didn’t worry about that.  These commands will not overwrite these files.
  You may have to create these directories before issuing the following mounts.

are the NFS mounts which I issued on the 486.  These will allow the 486 to see the
files on the Xeon.

# mount -v xeon:/usr/src /usr/src
xeon:/usr/src on /usr/src (nfs)
# mount -v xeon:/usr/obj /usr/obj
xeon:/usr/obj on /usr/obj (nfs)

Instead of xeon, you could use the IP address of the server or the name of
the box.  Use whatever works and what you normally use.

The quick and easy guide

This section outlines the steps which are detailed below.  This
intended to be a quick reference for those who are familiar with the steps and just need a
quick reminder of what to do.  Again, this assumes you have installed the makefile in
~/make when you were su’d to root.

Remember to create your kernel
configuration file first and set the name of this file within the Makefile (see MYKERNEL).

  1. su to root
  2. cd ~/make
  3. make update (updates your sources)
  4. make build – does a build world.
  5. make kernelbuild MYKERNEL=DUCKY on the fast box
  6. make kernelinstall MYKERNEL=DUCKY on the slow box
  7. /usr/src/UPDATING recommends a reboot here just to make sure the kernel you
    just built is operational.  If it is not you can still reboot to kernel.old and still
    have your world and kernel in sync.  For what it’s worth, I don’t usually do this
  8. mergemaster -p
  9. make install – installs the results of the above builds
  10. make merge – merges the configuration file changes
  11. reboot

update sources and build world – on the XEON

The build world is always done on the fast box.  This involves
several steps:

# make update
# make world

These two steps update your source code and does a make world (compiles the base
system).  If you were issuing these commands manually, this is what you would type:

# cd /usr/src
# cvsup -g -z -L2 supfile
# make buildworld

where supfile is the name of your cvsup configuration file.

My elapsed time: 2.5 hours.

I suspect it’s my disks which are slowing me down.  I know it takes only about an
hour on a Pentium III (compared to the dual XEON I have) box a client has.  But I
know they have faster disks.

Build the 486 kernel on the XEON

I upgraded the XEON first to 4.2-stable first.  Mostly so I could use
it to build the kernel for the 486.  I didn’t want to do anything on the 486 which I
didn’t have to do.  I copied the 486 kernel configuration file to /usr/src/sys/i386/conf
on the Xeon. 

# make kernelbuild MYKERNEL=DUCKY

This assumes the kernel configuration file for the 486 is named DUCKY and has been
copied to the XEON box (i.e. /usr/src/sys/i386/conf).

These are the commands you would type manually:

# make buildkernel KERNCONF=DUCKY

Now that doesn’t look like much saving on keystrokes, but the script does create a log
for you.

Note that I did not do an install.  The above commands were issued on the Xeon,
not the 486.  The install will be done on the 486.

My elapsed time: 18  minutes.

kernel install on the 486

I’ve already built the 486 kernel on the Xeon.  I copied the
configuration file from the 486 to the XEON via FTP.  Then I moved it into /usr/src/sys/i386/conf/and
build a kernel as normal.  Remember: I had my custom makefile on both boxes!  
This time, I’m issuing the commands on the 486.

# make kernelinstall MYKERNEL=DUCKY

Remember: this directory actually resides on the XEON, because of the NFS mount.  
But it will install the kernel on the 486.

The manual command for this is:

# cd /usr/src
# make installkernel

My elapsed time: 2 minutes.

Optional reboot

This is where you can reboot to make sure you have a working kernel.
  If the box doesn’t reboot, follow the instructions in the handbook for booking an
old kernel.  Basically, you press the space bar during the boot countdown, then type unload,
then boot kernel.old.

install world on the 486

In a previous section, I set up the NFS mounts on the 486.  This
allows me to install the files from the Xeon onto the 486.  This will still take
quite some time (the 486 isn’t very fast, but at least it’s not compiling world). 

# make install

Actually, I had the make world script on the 486,
so all I did, as root, was cd ~/make ; make install.

The manual commands would be:

# cd /usr/src
# make installworld

My elapsed time: 30 minutes.

Since this was a production box, I actually did a nice 15 make install so
that it ran at a lower priority.


The next step was do to the mergemaster
run on the 486.  mergemaster I used my script for that:

# cd ~/make
# make merge

If you’re not using my script, I suggest you follow the normal mergemaster instructions
for upgrading.  But here’s the extract from my makefile:

# mergemaster -w 120 -t

My elapsed time: 10 minutes.

reboot time

After a shutdown -r now on the 486, it came up with FreeBSD

Note: I do remember a problem with /etc/pam.conf.  This
may have been my fault, but I didn’t get the new file during the upgrade.  I may have
screwed up with my merge.  The symptoms turned up when I tried to used SSH.  
Sorry, but I didn’t record the details.  I think it had to do with system keys being
in a different format.

Errors I have known (added on 15 August 2003)

I got this error when I forgot to mount the NFS volumes on the slow box.

>>> Installing everything..
cd /usr/src; make -f Makefile.inc1 install
===> share/info
===> include
creating osreldate.h from
setvar PARAMFILE /usr/src/include/../sys/sys/param.h; . /usr/src/include/../sys/conf/; echo "$COPYRIGHT" > osreldate.h;
echo "#ifdef _KERNEL" >> osreldate.h; echo '#error "osreldate.h must not be used in the kernel, use sys/param.h"' >> osreldate.h; echo "#else"
>> osreldate.h; echo \#'undef __FreeBSD_version' >> osreldate.h; echo \#'define __FreeBSD_version' $RELDATE >> osreldate.h; echo "#endif"
>> osreldate.h
touch: not found
*** Error code 127

Stop in /usr/src/include.
*** Error code 1

Stop in /usr/src.
*** Error code 1

I solved the problem by mounting the NFS volumes and trying again. On the slow box, I actually had a fully
populated /usr/src and /usr/obj from a previous build world.
When I first saw this error, I thought it was
caused by the clocks on the two boxes not being set to the same time. After fixing that issue, the problem
still occurred. It wasn’t until later that I noticed I had not remounted the NFS volumes.


After the upgrade is completed, you may wish to disable NFS on the client
and server.

10 thoughts on “Build world on your fast box, install on your slow box”

  1. How’s everybody doing? I’m a first time poster, so please be easy on me. I’m running nat and I noticed that from my windows box my downloads have been quite a bit slower (only in the 30-40k range) than downloads that previously yielded speeds of several hundred k over my cable connection when I was using a hardware-based router. I’ve checked for collisions between interfaces on the BSD box which are 0. I also changed out hubs, which didn’t help. Any advice is appreciated.


  2. I don’t know if your script does this or not, but following compile flags while installing world is critical. For example, if you used -DNOPROFILE to avoid compiling profiled libs in ‘buildword’ then you must use that flag while installing world or else it will complain that the profiled libs you skipped don’t exist and the install will fail.

    What’s more important is if you have all your flags in /etc/make.conf. Make sure you sync /etc/make.conf on both machines before installing world on to the remote machine!

  3. if the "slow box" is, say, an alpha or a sparc … how can I build the world/kernel on my fast i386 box? I’ve searched for a good doc on this "cross compiling" topic but found nothing really. Perhaps it is so simple that it seems obvious, but it isn’t obvious to me :/

    1. The short answer is : don’t bother. I asked in a developers channel:

      Someone mentioned: make TARGET_ARCH=alpha buildworld

      Same person said: I think cross-compiling from a 32-bit to 64-bit arch produces inferior results, at least with gcc. I believe that person shouldn’t cross-build to alpha.The speed isn’t worth the problems

      Another person said: yes, gcc still has cross compile issues with some platforms, where it generates invalid code.. especially with -O and 32->64 bit…. i386(32) -> alpha(64) is notorious. its similar on i386->ia64. the only thing cross world builds are really useful for are testing syntax and bootstrapping. You do NOT want to release from it.

      make TARGET_ARCH=alpha buildworld; make installworld DESTDIR=/somewhere TARGET_ARCH=alpha

      $ grep TARGET Makefile
      # If TARGET_ARCH=arch (e.g. alpha) is specified you can
      ${MAKE} ${.TARGET}
      ${MAKE} -f Makefile.upgrade -m ${.CURDIR}/share/mk ${.TARGET}

  4. Marco Steinbach

    Hi there,

    at least for 4.8-STABLE one might note, that if you installed the games, the installation of fortune will try to write to the NFS mount /usr/obj, and bring the build to a screeching halt in case it fails. (I’m using the standard make installworld).

    I did not check for other anoyances like this, because I simply added -maproot=root to /etc/exports.

    I hate to do this, so if someone has suggestions on how to installworld without the need to allow write-access, I’d be most thankful. I’ll post this question to the support area, also. Please followup there.

    1. Interesting, I upgraded 4.8 on several machines tonight, all from an NFS mount. No problem with games.

      I wonder what is different with your installation.

  5. I’ve found it easier to install with the slow box as the server. Set the slow box up as an NFS server, exporting all filesystems with -maproot=0. This can even be done booting off the Live CD if done correctly.

    Mount these filesystems on the fast box, under /mnt, and then make installworld DESTDIR=/mnt

    This approach has the benefit that all of the clever installworld stuff is running on the fast machine, and the slow box only has to do the bare minimum. Also, it helps keep compile settings (eg. CFLAGS) consistent between the two.


Leave a Comment

Scroll to Top