Jails under FreeBSD 6
This article shows you how I created a jail for under FreeBSD 6.
This article is based on a previous article.
I have written previously about jails on FreeBSD
4 and on FreeBSD 5 . The goal of this jail is
the creation of a test environment for updating the Bacula ports that I
maintain
Lift and separate
Jails can be used to separate different processes and keep them apart from each
other so they cannot interfere. For example, you could run Apache in a jail and
keep it away from everything else on the machine. Should an exploit be found in Apache
and used to compromise your system, the intruders can only do what the jail allows them
to do. A jail can consist of a full operating system, or a single executable.
From within a jail, they are chroot‘d
and cannot see anything outside of the jail. At the same time, it appears to them
as if they are running on their own machine with their own operating system.
As far as they know, they have their own computer and nobody else is on the system.
Running a virtual system within a jail is a good solution if you want to provide someone
with resources, but don’t want them to have complete control over your system. A jail
can help you deal with issues of security, access, and increase utilization of
existing resources, all at the same time.
Jail documentation
The main document for creating a jail is
man jail.
I followed the instructions listed under Setting up a Jail Directory
Tree. I used those instructions to create the jail. You will need the full
source tree for the system you going to create. I used the
/usr/src/ directory I had from my most recent
build world.
One step from man jail that I did not follow:
- I left sendmail (actually, postfix) running. I just changed it so that
it did not listen on all IP addresses. I added this setting to
/usr/local/etc/postfix/main.cf:
inet_interfaces = $myhostname
That setting allows the jail to run its own mail server.
I put my jail at /home/jails/ducky/. This is
the value I assigned to D in the instructions. After you
have installed the jail, if
you peek inside that directory, you’ll see it looks just like the root directory
of a typical FreeBSD system:
[dan@dfc:/home/jails/ducky/] $ ls
COPYRIGHT etc libexec root usr
bin home mnt sbin var
boot kernel proc sys
dev lib rescue tmp
[dan@dfc:/home/jails/ducky/] $
Terminology: host versus jail
The host environment is the main system and is where you first install FreeBSD on the
computer. It is in the host environment that you create a jail. The Bacula project
will do their testing in the jail. They have access to the jail and only the jail.
They will not have access to the host environment at all.
This concept of host environment and jail environment will be used later in this article.
It is important that you understand what each one is.
In this example, the host environment will be at IP address 192.168.0.100 and the jail
will be at 192.168.0.155.
Modifying other daemons
Most daemons will listen to whatever IP addresses are available to them. After starting your jail,
if you try to ssh to it, you will not get into it. You’ll be in the host environment instead.
To get into the jail environment via ssh, you need to:
- Tell the host environment sshd not to listen to the jail’s IP address
- run sshd in the jail
Host environment syslogd
This entry in /etc/rc.conf tells
syslogd to not listen on any IP address.
syslogd_flags="-ss"
That allows syslogd to run in both the host and the
jail environments.
Host environment inetd
This entry in /etc/rc.conf tells
inetd to listen on a specific IP address.
This address is that of the host environment:
inetd_flags="-wW -C 60 -a 192.168.0.100"
You should note that the first part of the above flags is from
/etc/defaults/rc.conf:
inetd_flags="-wW -C 60" # Optional flags to inetd
Host environment sshd
To alter the host environment sshd so it listens only to host environment IP addresses,
modify /etc/ssh/sshd_config and set the IP address for the Listen directive:
ListenAddress 192.168.0.100
Then restart the main sshd process:
kill -HUP `cat /var/run/sshd.pid`
Use telnet to verify that the host environment is not listening on the jail address:
$ telnet 192.168.0.155 22
Trying 192.168.0.155...
telnet: connect to address 192.168.0.155: Connection refused
telnet: Unable to connect to remote host
If you don’t get a connection, the host environment is not listening. This assumes that you have not yet
started sshd in the jail environment.
Jail environment sshd
To start sshd in the jail environment, add the following line to /etc/rc.conf:
sshd_enable="YES"
To get DNS working, add something like this to /etc/resolv.conf:
search example.org nameserver 10.0.0.67 nameserver 10.0.0.98
Jail environment syslogd
In addition, I also swapped console output to /var/log/messages as shown
in this snipped from /etc/syslog.conf:
#*.err;kern.warning;auth.notice;mail.crit /dev/console
*.err;kern.warning;auth.notice;mail.crit /var/log/messages
Configuring the Jail
Next, you’ll want to read the part of the man page titled Configuring the Jail. Included
there you will to make the following settings within the jail.
I made these changes to the jail directly from the host environment (that is,
I did not start the jail; I modified the files from outside).
These are things I changed, but I can’t point to a man page as to why it’s a good thing
to do this:
adjkerntz– Not sure about this. I
commented out the/etc/crontabentry foradjkerntz
within the jail environment. If you don’t do this, you’ll see this type of notification
from cron via email:
adjkerntz[11643]: sysctl(put_wallclock): Operation not permitted
/etc/ssh– I was actually duplicating an existing physical machine into this environment.
Therefore, copying over the keys from this directory will avoid “WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!”
warning messages. If you are creating a new environment, you don’t have to worry about
this step.
Starting the jail for the first time
From man jail, to start a jail, issue this command:
[root@mtwenty:/home/dan] # jail /home/jails/ducky/ ducky.example.org 192.168.0.155 /bin/sh
#
That prompt (#) indicates you are now in the jail environment. Now you can run the start up processes:
# sh /etc/rc
Loading configuration files.
ducky.example.org
Setting hostname: ducky.example.org.
Generating nsswitch.conf.
Generating host.conf.
Starting syslogd.
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout
Starting local daemons:.
Updating motd.
/etc/rc: WARNING: Setting entropy source to blocking mode.
====================================================
Type a full screenful of random junk to unblock
it and remember to finish with. This will
timeout in 300 seconds, but waiting for
the timeout without typing junk may make the
entropy source deliver predictable output.Just hit
for fast+insecure startup.
====================================================
kern.random.sys.seeded: 1
jalkjlkajdflkajdfl iur opiquv dropuivwaopieuaoijdfl;uiop9^[[12~84718e0r9invpfinadfpisad;ifsda;lsajdfl lk;kasf;kladfs
Generating public/private rsa1 key pair.
Your identification has been saved in /etc/ssh/ssh_host_key.
Your public key has been saved in /etc/ssh/ssh_host_key.pub.
The key fingerprint is:
5c:48:47:4f:e0:c5:a2:ed:71:bc:83:b5:42:3f:95:e4 root@ducky.example.org
Generating public/private dsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_dsa_key.
Your public key has been saved in /etc/ssh/ssh_host_dsa_key.pub.
The key fingerprint is:
4d:bb:af:fa:b1:4b:43:cc:47:b6:78:44:ad:4e:ef:1f root@ducky.example.org
Generating public/private rsa key pair.
Your identification has been saved in /etc/ssh/ssh_host_rsa_key.
Your public key has been saved in /etc/ssh/ssh_host_rsa_key.pub.
The key fingerprint is:
9b:b5:26:98:f8:0d:da:bb:2c:57:75:d1:c4:58:52:c1 root@ducky.example.org
Starting sshd.
Starting cron.
Local package initialization:.Sun Sep 11 17:22:42 EDT 2005
#
For the most part, this looks exactly like a normal startup.
While you have it running, you might want to add a user, set the root password,
etc.
I had some problems with ps:
# ps auwx
ps: bad namelist
This usually indicate a kernel that is not in sync with world. To fix this problem, I
repeated some of the steps under man 8 jail.
make distribution DESTDIR=$D
mount_devfs devfs $D/dev
cd $D
ln -sf dev/null kernel
In hindsight, I think I missed the mount_devfs step. Symptoms
included getting logged in by ssh, but then the screen would freeze. The above steps
fixed that problem.
Starting and stopping the jail automagically
With 6.2 comes /etc/rc.d/jail, a startup script for jails. With a bit of
fishing in /etc/defaults/rc.conf and some trial and error, I figured out
how to use this script. Here are my entries from /etc/rc.conf:
jail_enable="YES" # Defaults for all jails: jail_interface="fxp0" # Interface to create the IP alias on jail_devfs_enable="YES" # mount devfs in the jail jail_procfs_enable="YES" # mount procfs in jail # list of jails on this machine jail_list="ducky" # values for each jail listed above jail_ducky_rootdir="/usr/home/jails/ducky" # Jail's root directory jail_ducky_hostname="ducky.example.org" # Jail's hostname jail_ducky_ip="192.168.0.155" # Jail's IP number
I could have used jail_ducky_interface, jail_ducky_devfs_enable, and jail_ducky_procfs_enable,
but all jail variables, except rootdir, hostname and ip, can be defined without the jail name.
Thus, you can set defaults, and then specify only the exception. A fine FreeBSD tradition.
Additional resources
Just after this article appeared on ONLamp, Anthony Nguyen wrote in with these great
resources:
- FreeBSD Jails by Khairil Yusof, Tom McLaughlin, Radek Kozlowski.
He also mentioned:
- Before you even set up a jail if you have already done a build world, you
can skip a compiling step.
quote
“If you have already built world, you can replace make world with make
installworld on the fourth line to avoid recompiling everything again.”- After you set up your first jail and want more jails, look for the
heading “Reusing jails”. It clones it.- Starting jail automatically, look at heading “Using system rc scripts to
start and stop jails”.
Paul Dekkers also wrote in with:
You know you don’t need jailtools anymore these days to have rc-scripts
and all, don’t you? You can just use the right parameters in
/etc/rc.conf, like:jail_enable="YES" jail_list="milter" jail_milter_rootdir="/data/jail/milter" jail_milter_ip="192.87.110.82" jail_milter_hostname="milter.wind.example.org" jail_milter_devfs_enable="YES" #jail_milter_exec="/different/script"(Which is real example ;-))
Starting and stopping then goes with /etc/rc.d/jail start and I believe
you can add the jail name after that.|BTW, I never really needed the security.jail.set_hostname_allowed because I set the hostname in the rc.conf command|
Jails run well
Jails run virtual machines very well. They look very much like a real system.
You must look pretty close to be able to tell you’re in a jail. My jail allows the
Bacula developers to have a machine of their own. It also allows me to keep their
work totally separate from my own.
A jail can be used to deal with security
issues and to increase the utilization of an existing machine while giving everyone
their own virtual machine. There’s no reason why you couldn’t run many different
jails on the same computer.
Enjoy.
The following patch fixes the adjkerntz issue:
<http://people.freebsd.org/~seanc/patches/#adjkrntz_jail>
And the following patch allows you to set the netmask of jails.
<http://people.freebsd.org/~seanc/patches/#jail_netmask>
Hope they’re useful to someone. -sc
Thank you
—
The Man Behind The Curtain