Nov 292009
logcheck – a log file scanner
Every decent system generates logs. They are useful both from a forensic and from a debug point point of view. Some systems generate huge volumes of logs. Scanning those logs manually is both tedious and error-prone. This calls for an automated solution. Enter logcheck. Logcheck will scan your log files and report any entries which do not match a list previously flagged as OK to ignore. The pattern matching is flexible and easily extended.Background
logcheck has been around at least 10 years. I starting using logcheck in 1999, just about 10 years ago. Since then, logcheck underwent quite a transformation. It once had just a handful of matching files. Now it has over 180 files. logcheck works by ignoring known benign patterns and reports any log file entries that do not match those patterns. You can add to these patterns easily. Logcheck can scan a number of files. The list is kept in /usr/local/etc/logcheck/logcheck.logfiles. I choose to scan these files:# these files will be checked by logcheck # This has been tuned towards a default syslog install /var/log/messages /var/log/auth.log /var/log/maillogNOTE: the comments are not mine. For logcheck to scan all the files on a default FreeBSD system, you will need to make some changes to file permissions, /etc/newsyslog.conf, and /etc/group. See the next section for details.
Permissions
logcheck runs as the logcheck user:# grep logcheck /etc/passwd logcheck:*:915:915:Logcheck system account:/var/db/logcheck:/usr/local/bin/bashThis user is created by the install process. I’m assuming you have the ports tree intact.
cd /usr/ports/security/logcheck make install cleanIf the cd fails, you need to do this first because you probably don’t have a ports tree checked out:
portsnap fetch && portsnap extractIf you do not alter the permission and update some configuration files, you’ll soon get one of these emails:
To: root@ngaio.example.org Subject: Logcheck: ngaio.example.org 2009-11-20 12:02 exiting due to errors Message-Id: <20091120120201.7ACFF17104@ngaio.example.org> Date: Fri, 20 Nov 2009 12:02:01 +0000 (GMT) From: logcheck@ngaio.example.org (Logcheck system account) Warning: If you are seeing this message, your log files may not have been checked! Details: Could not run logtail or save output Check temporary directory: /tmp/logcheck.ZOjfJO Also verify that the logcheck user can read all files referenced in /etc/logcheck/logcheck.logfiles! declare -x HOME="/var/db/logcheck" declare -x LOGNAME="logcheck" declare -x MAILTO="root" declare -x OLDPWD declare -x PATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin" declare -x PWD="/var/db/logcheck" declare -x SHELL="/bin/sh" declare -x SHLVL="1" declare -x USER="logcheck"The email contains the wrong location for the file. It is assuming an installation location which has been changed at configuration/install time and /usr/local/sbin/logcheck has not been refreshed accordingly. I have submitted a patch for that. Check the permissions for the files listed in /usr/local/etc/logcheck/logcheck.logfiles:
# ls -l /var/log/messages /var/log/auth.log /var/log/maillog -rw------- 1 root wheel 6564 Nov 28 21:13 /var/log/auth.log -rw-r----- 1 root wheel 60 Nov 28 00:00 /var/log/maillog -rw-r--r-- 1 root wheel 83127 Nov 28 22:00 /var/log/messagesAs you can see, the logcheck user will be unable to read auth.log and maillog. We can change that.
# chgrp logcheck /var/log/auth.log /var/log/maillog # chmod g+r /var/log/auth.log # ls -l /var/log/messages /var/log/auth.log /var/log/maillog -rw-r----- 1 root logcheck 6564 Nov 28 21:13 /var/log/auth.log -rw-r----- 1 root logcheck 60 Nov 28 00:00 /var/log/maillog -rw-r--r-- 1 root wheel 83277 Nov 28 22:05 /var/log/messageslogcheck will now be able to read the files, but as you know, these files are rotated by newsyslog.conf. So let’s see the entries for them:
# egrep "/var/log/auth.log|/var/log/maillog" /etc/newsyslog.conf /var/log/auth.log 600 7 100 * JC /var/log/maillog 640 7 * @T00 JCThe above is before my changes, the following is after:
# egrep "/var/log/auth.log|/var/log/maillog" /etc/newsyslog.conf /var/log/auth.log root:logcheck 640 7 100 * JC /var/log/maillog root:logcheck 640 7 * @T00 JCNote that you have to add the root:logcheck to both *and* change the mode for auth.log to 640.
logcheck: dan
Customizations
logcheck will initally produce notices about things you do not care to see again. They are normal for your system and they do not need to be brought to your attention again. You can train logcheck to ignore these items. You will see both System Events and Security Events emails. For example:Security Events =-=-=-=-=-=-=-= Nov 28 16:37:55 dbclone postgres[93778]: [2-1] ERROR: table "mac" does not exist System Events =-=-=-=-=-=-= Nov 28 16:28:22 dbclone bacula-dir: Shutting down Bacula service: localhost-dir ...These items are normal for this system. It is used for Bacula regression testing. For the Security Events, I created /usr/local/etc/logcheck/violations.ignore.d/local-postgres with the following contents:
# grep mac /usr/local/etc/logcheck/violations.ignore.d/local-postgres ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ postgres\[[0-9]+\]: \[[0-9]+\-[0-9]+\] ERROR: table "mac" does not existThat preamble seems like a lot. But I grabbed it from logcheck-postgres. The logcheck project recommends that you put your own customizations into files prefixed with local- so they are easily identified. logcheck itself does not care. For the System Event, I added this entry to /usr/local/etc/logcheck/ignore.d.server/local-postgres
bacula-fd: Shutting down Bacula service: localhost-fdNotice that my System Event exceptions are specified in the ignore.d.server directory. This is because I selected the following option in /usr/local/etc/logcheck/logcheck.conf:
REPORTLEVEL="server"If you are using “workstation”, you would add your file to the ignore.d.workstation directory.
If you see the following error, install textproc/docbook-to-man
The clue to this error was a ‘recent’ commit by glarkin: http://www.freshports.org/commit.php?category=security&port=logcheck&files=yes&message_id=200905262025.n4QKPxAO031130@repoman.freebsd.org
Looking at the diffs for that commit gave me the idea to install docbook-to-man manually.
# make
===> Extracting for logcheck-1.2.54_3
=> MD5 Checksum OK for logcheck_1.2.54.tar.gz.
=> SHA256 Checksum OK for logcheck_1.2.54.tar.gz.
===> logcheck-1.2.54_3 depends on file: /usr/local/bin/perl5.8.9 – found
===> Patching for logcheck-1.2.54_3
===> logcheck-1.2.54_3 depends on file: /usr/local/bin/perl5.8.9 – found
===> Applying FreeBSD patches for logcheck-1.2.54_3
===> logcheck-1.2.54_3 depends on executable: docbook2man – found
===> logcheck-1.2.54_3 depends on file: /usr/local/bin/perl5.8.9 – found
===> Configuring for logcheck-1.2.54_3
===> Building for logcheck-1.2.54_3
/usr/bin/sed -i.bak -e ‘s!/var/log/syslog!/var/log/messages!’ /usr/ports/securi
ty/logcheck/work/logcheck-1.2.54/etc/logcheck.logfiles
/usr/bin/sed -i.bak -e ‘s!/etc/logcheck!/usr/local/etc/logcheck!’ -e ‘s!/usr/sh
are/doc/logcheck-database/README.logcheck-database.gz!/usr/local/share/doc/logch
eck/README.logcheck-database!’ /usr/ports/security/logcheck/work/logcheck-1.2.5
4/docs/logcheck.sgml
cd /usr/ports/security/logcheck/work/logcheck-1.2.54/docs && docbook2man -s /us
r/local/share/docbook2X/xslt/man/docbook.xsl –sgml logcheck.sgml 2> /dev/null
&& /bin/mv Logcheck.8 logcheck.8
*** Error code 255
Stop in /usr/ports/security/logcheck.
—
The Man Behind The Curtain
Actually, the bits between ‘Building for logcheck’ and ‘*** Error’ are only displayed if you remove the @ which appear in the do-build section of the Makefile. I made these changes when trying to debug the problem.
—
The Man Behind The Curtain