Configuring IPsec on your XP Professional laptop

Configuring IPsec on your XP Professional laptop

IPsec is a tool you can use to secure your network communications. I use IPsec
to ensure my wireless connections are secured. IPsec offers security and peace
of mind over and above the traditional WEP and MAC-address filtering.

NOTE: although this article was written with wireless communication in mind,
it applies equally well to traditional wired communication as well.

I have already written about configuring my
FreeBSD IPsec gateway and workstations. In this
article I will show how I configured my Windows XP box to use the same gateway.

The main resource I used for this exercise was
FreeBSD Wi-Fi IPsec easy-setup guide (since removed from the Internet).

You might ask why I’m writing about Windows XP on a website about FreeBSD?
My terse answer is because I can. My realistic answer is because it will
help people. It’s something I did, with my FreeBSD gateway. I use XP on a regular
basis. Use the right tool for the job. Sometimes that’s XP. Sometimes it’s FreeBSD.

A recap of the configuration

I will give a brief overview of the gateway configuration. I have a FreeBSD machine
that functions as a dedicated wireless gateway. The primary purpose of this box is
to keep out all the stuff that is not allowed and to ensure that only my machines
are used on this WAP.

This is the content of the gateways’s /etc/ipsec.conf:

spdadd   any -P in  ipsec esp/tunnel/;
spdadd any -P out ipsec esp/tunnel/;

I am using racoon as my
key server. It does the job, but I’m hoping for other alternatives. Sometimes I have
to restart racoon on the gateway in order to get a session established. Here is the relevant
portion of /usr/local/etc/racoon/racoon.conf. This works for my laptop
whether it’s running XP, FreeBSD 4.10, or FreeBSD 5.3.

remote anonymous {
  exchange_mode aggressive,main,base;
  lifetime time 24 hour;
  proposal {
    encryption_algorithm  3des;
    hash_algorithm        sha1;
    authentication_method pre_shared_key;
    dh_group              2;

sainfo anonymous {
  lifetime                 time 12 hour ;
  encryption_algorithm     des, 3des, des_iv64, des_iv32, null_enc, rijndael, blowfish;
  authentication_algorithm hmac_sha1, hmac_md5;
  compression_algorithm    deflate ;

By the way, to install racoon from the ports tree, I did this:

cd /usr/ports/security/racoon
make install clean

Under more recent versions of FreeBSD, you must also add this to /etc/rc.conf:


This command can be used to start racoon manually:

/usr/local/etc/rc.d/ start

Configuring the XP client

The Windows XP client should be similar to that found in Windows 2000, and
hopefully Windows 2003. IPsec configuration is performed through the
Microsoft Management Console (mmc). To start
mmc, perform the following steps:

  1. click on Start
  2. click on Run
  3. type mmc
  4. press ENTER

You should now see something like this:

Adding the Security Policy Management Snap-in

Click on File | Add/Remove Snap-in....

Click on Add... and scroll down to and click on
IP Security Policy Management. You should see this:

Then you need to select the local computer and click on Finish:

If you are running XP, you should also select the IP security Monitor snap-in and click add.
Then click on Close. You should now see this:

Click on OK, and you’ll be back at mmc but with two new
entries under Console Root.

NOTE: if you click on IP Security Policies on Local Computer
you will see this:

Creating the IP Security Policy

We are now going to create the IP Security Policy we will use on this
laptop. Start this process by right clicking on
IP Security Policies on Local Computer and
then selecting Create IP Security Policy. This
will invoke the IP Security Policy Wizard.

Give your policy a name (I called mine My WIFI Security Policy) and a description:

Be sure to uncheck Activate the default response rule.

Also uncheck Edit Properties. Then click Finish.

You now have a security policy, although it has no content.. Now we will
define filter lists for the inbound and outbound traffic.

Create the Outbound filter

Start by right clicking on IP Security Policies on Local Computer
and select Manage IP filter lists and filter actions... and
you should see this:

Click on Add and you will see this:

Click on Add and name your filter list (I called mine
OutboundIPsec). This is what you should see:

Now we need to add define the filter contents. Click on add and you’ll
see the IP Filter Wizard:

Click on Next and select the Traffic Source. You want
My IP address:

Click on Next and select the Traffic Destination. You want
Any IP address:

Protocal type is Any:

Be sure to check the Edit Properties box:

When you get to the Filter properties window, be sure
to uncheck Mirrored. This is important.

Now you should be back at the IP Filter List window.
Your filter should be listed in the Filters section.
Click OK.

After you clicked OK, you should see this:

You have now created your outbound filter list. Now we will create a similar
list for the inbound traffic.

Create the Inbound filter

You have just created the outbound filter. Now repeat the same steps again
but for inbound traffic. The differences will be:

  1. Use Any IP Address for the Source Address
  2. Use My IP address for the Destination Address

Remember to check Edit Properties and to uncheck
Mirrored. After completion, you should see this:

As Timothy Ham cautioned, pay attention to what have now. You should have two entries
in IP Filter Lists. Each list should contain one rule.
You should not have one Filter List, with two rules. Verify that the two filters
are not mirrored. Click on Close and you should be back at the MMC console.

Using the filters

So far we have:

  1. created a security policy
  2. added an outbound filter list
  3. added an inbound filter list

Creating the Outbound Security Rule

Now we will start using the filter lists. Double click on
My WIFI Security Policy and you should see this:

Click on add to start up the Security Rule Wizard:

Click Next, and you will see the Tunnel Endpoint window. Click on
The tunnel endpoint is specified by this IP address
and supply the IP address of your gateway (for me, that’s

Click on Next, and specify the type of traffic that must be encrypted.
I chose Local Area Network (LAN). I’m
sure All network connections would work as well.

Click on Next and select the Authentication Method. We will be using preshared keys.
Enter your key on in the space provided. For testing purposes, I used the word

Click Next, and you’ll be asked to select the Filter List against which this
security rule should apply. Choose OutboundIPSec.

Click on Next. Select the action for this security rule. In this case,
we want Require Security. We do not want any
traffic to pass unless it is IPsec (note: DHCP etc will still
get through without IPsec).

Click on Next, and you will see the last window in the Security Rule Wizard.
Ensure that Edit Properties is off, and click Finish.

Your policy properties should now look like this:

Creating the Inbound Security Rule

You should now repeat the same steps again, but for the Inbound traffic. The differences

  1. The tunnel endpoint should be the IP address of this PC.
  2. Apply this rule to the InboundIPsec filter list

Once you have completed this, your policy properties should look like this:

All done, save the results

Here is what your MMC console should look like now:

You will note that the Policy Assigned column contains No. That means
your policy is not in effect. We will change that soon.

Save your data using File | Save. I named my file

Invoking the rules

So far, we have established a policy, created two filter lists, and added one rule to each filter list.
Now we will invoke that policy to ensure that only IPsec traffic flows between the laptop and the wireless

Unwanted DNS updates

If you start seeing this message in your logs on your DNS server, then I know the fix:

named[111]: denied update from [].40061 for "" IN

In this case, is the internal IP address of my wireless gateway. That gateway
also has the IP address used above as the end point of the tunnel. is the domain name (not really, I changed it for this
article) given to the laptop.

To prevent these DNS updates, turn off connection registration. To do this perform the following

  1. Click on Start
  2. Click on Control Panel
  3. Click on Network and Internet Connections
  4. Click on Network Connections
  5. Right click on your [wireless] connection and select Properties
  6. Under “This Connection uses the following items” select “Internet Protocol (TCP/IP)”
  7. Click on Properties
  8. Click on Advanced
  9. Select the DNS tab

You should not be looking at something like this:

Uncheck Register this connection's addresses in DNS and you should
stop seeing those messages.

Leave a Comment

Scroll to Top