Nov 292009

logcheck – a log file scanner

Every decent system generates logs. They are useful both from a forensic and from a debug point point of view. Some systems generate huge volumes of logs. Scanning those logs manually is both tedious and error-prone. This calls for an automated solution. Enter logcheck. Logcheck will scan your log files and report any entries which do not match a list previously flagged as OK to ignore. The pattern matching is flexible and easily extended.


logcheck has been around at least 10 years. I starting using logcheck in 1999, just about 10 years ago. Since then, logcheck underwent quite a transformation. It once had just a handful of matching files. Now it has over 180 files. logcheck works by ignoring known benign patterns and reports any log file entries that do not match those patterns. You can add to these patterns easily. Logcheck can scan a number of files. The list is kept in /usr/local/etc/logcheck/logcheck.logfiles. I choose to scan these files:
# these files will be checked by logcheck
# This has been tuned towards a default syslog install
NOTE: the comments are not mine. For logcheck to scan all the files on a default FreeBSD system, you will need to make some changes to file permissions, /etc/newsyslog.conf, and /etc/group. See the next section for details.


logcheck runs as the logcheck user:
# grep logcheck /etc/passwd
logcheck:*:915:915:Logcheck system account:/var/db/logcheck:/usr/local/bin/bash
This user is created by the install process. I’m assuming you have the ports tree intact.
cd /usr/ports/security/logcheck
make install clean
If the cd fails, you need to do this first because you probably don’t have a ports tree checked out:
portsnap fetch && portsnap extract
If you do not alter the permission and update some configuration files, you’ll soon get one of these emails:
Subject: Logcheck: 2009-11-20 12:02 exiting due to errors
Message-Id: <>
Date: Fri, 20 Nov 2009 12:02:01 +0000 (GMT)
From: (Logcheck system account)

Warning: If you are seeing this message, your log files may not have been

Could not run logtail or save output

Check temporary directory: /tmp/logcheck.ZOjfJO

Also verify that the logcheck user can read all files referenced in

declare -x HOME="/var/db/logcheck"
declare -x LOGNAME="logcheck"
declare -x MAILTO="root"
declare -x OLDPWD
declare -x PATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin"
declare -x PWD="/var/db/logcheck"
declare -x SHELL="/bin/sh"
declare -x SHLVL="1"
declare -x USER="logcheck"
The email contains the wrong location for the file. It is assuming an installation location which has been changed at configuration/install time and /usr/local/sbin/logcheck has not been refreshed accordingly. I have submitted a patch for that. Check the permissions for the files listed in /usr/local/etc/logcheck/logcheck.logfiles:
# ls -l /var/log/messages /var/log/auth.log /var/log/maillog
-rw-------  1 root  wheel   6564 Nov 28 21:13 /var/log/auth.log
-rw-r-----  1 root  wheel     60 Nov 28 00:00 /var/log/maillog
-rw-r--r--  1 root  wheel  83127 Nov 28 22:00 /var/log/messages
As you can see, the logcheck user will be unable to read auth.log and maillog. We can change that.
# chgrp logcheck /var/log/auth.log /var/log/maillog
# chmod g+r /var/log/auth.log
# ls -l /var/log/messages /var/log/auth.log /var/log/maillog
-rw-r-----  1 root  logcheck   6564 Nov 28 21:13 /var/log/auth.log
-rw-r-----  1 root  logcheck     60 Nov 28 00:00 /var/log/maillog
-rw-r--r--  1 root  wheel     83277 Nov 28 22:05 /var/log/messages
logcheck will now be able to read the files, but as you know, these files are rotated by newsyslog.conf. So let’s see the entries for them:
# egrep "/var/log/auth.log|/var/log/maillog" /etc/newsyslog.conf
/var/log/auth.log                       600  7     100  *     JC
/var/log/maillog                        640  7     *    @T00  JC
The above is before my changes, the following is after:
# egrep "/var/log/auth.log|/var/log/maillog" /etc/newsyslog.conf
/var/log/auth.log       root:logcheck   640  7     100  *     JC
/var/log/maillog        root:logcheck   640  7     *    @T00  JC
Note that you have to add the root:logcheck to both *and* change the mode for auth.log to 640.


Recent versions of logcheck default the outgoing email to the logcheck user. To get these emails sent to myself, I added this entry to /etc/mail/aliases:
logcheck:       dan


logcheck will initally produce notices about things you do not care to see again. They are normal for your system and they do not need to be brought to your attention again. You can train logcheck to ignore these items. You will see both System Events and Security Events emails. For example:
Security Events
Nov 28 16:37:55 dbclone postgres[93778]: [2-1] ERROR:  table "mac" does not exist

System Events
Nov 28 16:28:22 dbclone bacula-dir: Shutting down Bacula service: localhost-dir ...
These items are normal for this system. It is used for Bacula regression testing. For the Security Events, I created /usr/local/etc/logcheck/violations.ignore.d/local-postgres with the following contents:
# grep mac /usr/local/etc/logcheck/violations.ignore.d/local-postgres
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ postgres\[[0-9]+\]: \[[0-9]+\-[0-9]+\] ERROR:  table "mac" does not exist
That preamble seems like a lot. But I grabbed it from logcheck-postgres. The logcheck project recommends that you put your own customizations into files prefixed with local- so they are easily identified. logcheck itself does not care. For the System Event, I added this entry to /usr/local/etc/logcheck/ignore.d.server/local-postgres
bacula-fd: Shutting down Bacula service: localhost-fd
Notice that my System Event exceptions are specified in the ignore.d.server directory. This is because I selected the following option in /usr/local/etc/logcheck/logcheck.conf:
If you are using “workstation”, you would add your file to the ignore.d.workstation directory.


There you go. That should get you started with logcheck. I’ve been using it for 10 years. It’s a great idea. I hope and trust it will save you a great deal of ready. Best wishes.

  2 Responses to “logcheck – a log file scanner”

  1. If you see the following error, install textproc/docbook-to-man

    The clue to this error was a ‘recent’ commit by glarkin:

    Looking at the diffs for that commit gave me the idea to install docbook-to-man manually.

    # make
    ===> Extracting for logcheck-1.2.54_3
    => MD5 Checksum OK for logcheck_1.2.54.tar.gz.
    => SHA256 Checksum OK for logcheck_1.2.54.tar.gz.
    ===> logcheck-1.2.54_3 depends on file: /usr/local/bin/perl5.8.9 – found
    ===> Patching for logcheck-1.2.54_3
    ===> logcheck-1.2.54_3 depends on file: /usr/local/bin/perl5.8.9 – found
    ===> Applying FreeBSD patches for logcheck-1.2.54_3
    ===> logcheck-1.2.54_3 depends on executable: docbook2man – found
    ===> logcheck-1.2.54_3 depends on file: /usr/local/bin/perl5.8.9 – found
    ===> Configuring for logcheck-1.2.54_3
    ===> Building for logcheck-1.2.54_3
    /usr/bin/sed -i.bak -e ‘s!/var/log/syslog!/var/log/messages!’ /usr/ports/securi
    /usr/bin/sed -i.bak -e ‘s!/etc/logcheck!/usr/local/etc/logcheck!’ -e ‘s!/usr/sh
    eck/README.logcheck-database!’ /usr/ports/security/logcheck/work/logcheck-1.2.5
    cd /usr/ports/security/logcheck/work/logcheck-1.2.54/docs && docbook2man -s /us
    r/local/share/docbook2X/xslt/man/docbook.xsl –sgml logcheck.sgml 2> /dev/null
    && /bin/mv Logcheck.8 logcheck.8
    *** Error code 255

    Stop in /usr/ports/security/logcheck.

    The Man Behind The Curtain

    • Actually, the bits between ‘Building for logcheck’ and ‘*** Error’ are only displayed if you remove the @ which appear in the do-build section of the Makefile. I made these changes when trying to debug the problem.

      The Man Behind The Curtain