NetSaint – a network monitor

NetSaint – a network monitor

NetSaint is a network monitor. You can use it to keep close tabs on your routers,
printers, computers, and services. NetSaint can watch/monitor various services (SMTP, POP3, HTTP) to make sure they keep running.
It can ping boxes to make sure they are still up. If a problem occurs, NetSaint can send out email, page people, and escalate
problems if they persist. When the problem is fixed, NetSaint can also send out a recovery notification. And my personal favourite,
you can view the current status through a webpage.

Regular readers will know that in early 2000, I wrote about another network monitor,
Big Brother.

NetSaint uses a feature called plugins to perform the actual service checks. This is a very flexible approach
which allows new plugins to be created for specific situations as and when necessary. If also allows you to create
your own plugins if you so wish. In fact, there is a port for the plugins
(net/netsaint-plugins). This port is a dependency of NetSaint and will be installed
when you install NetSaint.

Installing NetSaint

It seems like everything I install is already available in the ports collection. That makes life simple. Here’s
what you do:

cd /usr/ports/net/netsaint
make install

Don’t forget to do a make clean afterwards. Otherwise all the files created
by the build will sit around clutting up your disk.

The NetSaint port has a fair number of dependencies (including gd, bzip2, etc, about a dozen in total)
so it may take some time to install everything. But that’s the beauty of the ports system. It’s just
start and forget.

But don’t forget it for too long. You will eventually be given a list of things to choose from:

NetSaint plugins configuration options

 Please select desired options:

    [ ] QStat         Game server query support
    [X] FPing         Support for non-flooding fast ping
    [X] NetSNMP       SNMP support
    [ ] Radiusclient  Radius support
    [X] MySQL         MySQL support
    [X] PostgreSQL    PostgreSQL support
    [ ] OpenLDAP      OpenLDAP support

            [  OK  ]       Cancel

You can see the selections I choose. Only you can figure out what makes sense to you.
If none of the above options make any sense, don’t select them. If you’re not running
mysql or postgresql, then don’t
worry about those options.

I answered y to both of the following questions:

You need a group "netsaint".
Would you like me to create it [y]?
Done.
You need a user "netsaint".
Would you like me to create it [y]?

NetSaint will run as a specific user, which is a good thing from a security point of view.
The use of a specific group is also a good security concept. Combining the use of a specific
user and a specific group for NetSaint is a basic and commonly used security precaution.
Everything, including your NetSaint configuration files should be treated on a ‘need to know’ basis.

The web interface

This section may seem out of place, but I think it’s a good idea to get the web interface set up
and at least partially functional before we go very far down the track. It also has the added benefit
of giving you online access to the documentation….

You need to add the following to your apache configuration file (normally, this is
/usr/local/etc/apache/httpd.conf). You should place these directives
in the vhost section for the website you wish to use for NetSaint.


Alias /netsaint /usr/local/share/netsaint
ScriptAlias /cgi-bin/netsaint/ /usr/local/share/netsaint/sbin/

For a complete working example, see samples/netsaint.vhosts.txt.

Take note that the Alias does not contain a trailing /. On this point, I disagree with the
NetSaint documentation where it specifies
a trailing slash. I originally followed the documentation but encountered a “The page cannot be found” message
when I did not specify a trailing slash in my browser. Removing the trailing slash from the Apache configuration
file solved that problem.

To further restrict access to NetSaint, I also added the following to my Apache configuration file for
the vhost in question:

<Directory "/usr/local/share/netsaint/sbin">
       AllowOverride None AuthConfig
       Options FollowSymLinks ExecCGI
       Order allow,deny
       Allow from 192.168.0.99
</Directory>

The AuthConfig will be used later to authenticate people using NetSaint.
The ‘Allow from’ directive will restrict access to the CGI directory to the specified IP address. You can specify
multiple IP addresses or hostnames on that line, separated by commas.

If you are even more worried about access, try adding this entry:

<Directory "/usr/local/share/netsaint">
        Order deny,allow
        Deny from all
        Allow from 192.168.0.99
</Directory>

This will prevent anyone from even getting to /netsaint/ unless they are at that IP.

You should then test your Apache changes:

# /usr/local/sbin/apachectl configtest

You should then HUP Apache to invoke your changes:

# /usr/local/sbin/apachectl graceful

You should then be able to browse to your NetSaint directory using http://www.example.org/netsaint/ (subsitute your own domain).
But if you click on any of the CGI links, you may see an error something like this:


Error: Could not read host and service status information!

That’s because NetSaint isn’t running yet with host/service information. But we will deal with that problem after we complete the configuration.

Seen this error? (added on 27 January 2003)

If you encounter this error:


Error: Could not open CGI configuration file '/usr/local/etc/netsaint/nscgi.cfg' for reading!

If so, then you haven’t done what the next section tells you to do!

Configuration overview

NetSaint configuration files are located at /usr/local/etc/netsaint. The main configuration files are:

netsaint.cfg
The main configuration file for NetSaint. Basically, I didn’t do anything to this file.

hosts.cfg
The hosts file is where you define what you want NetSaint to monitor. It allows you to specify the computers,
collect them into groups for ease of use, and specify who should be notified for a given problem.
It was this file which took up most of my configuration time. The
hosts.cfg can be split up into almost as many sub config files as you
want which is especially useful to keep track of monitoring a large
network.
commands.cfg
The commands are what NetSaint uses to perform various tasks. I modified this file slightly, but nothing
major.

nscgi.cfg
NetSaint provides a web interface and makes heavy use of CGI programs. Access to these CGI programs
is controlled by this file.

After the first install, you need to copy the distributed files to the file names
expected by NetSaint:

cp commands.cfg.dist commands.cfg
cp netsaint.cfg.dist netsaint.cfg
cp nscgi.cfg.dist    nscgi.cfg
cp resource.cfg.dist resource.cfg
chown netsaint *
chmod o-r *

WARNING: I just removed world-readability using the above step. I run Apache as www. So
I used this to allow Apache to read the above (note: this tip may apply to other items as well,
which I may have neglected to include; you’ll soon find out….):

chgrp www *

I purposely have not taken the distributed hosts file (hosts.cfg.dist) as I’m going to provide
a simple example later in the next section.

That chown ensures the files are owned by netsaint rather than root. And the chmod makes sure nobody but
netsaint can read the files. Both of these steps may be considered overkill, but I think it’s a good security
measure. Keep things tidy.

hosts.cfg

The distributed hosts.cfg contains a very rich example. I suggest that you closely examine that
file and use that as the basis for your actual configuration. But first, as a practical example, I suggest you
use the following example and comment out the existing sections. This will get you going and allow you to see
a small example with a minimum of fuss. Start small. Walk first. Then run.

The sample hosts file is provided via this link. I don’t want to display it in this article
because some of the lines are very long. You will need to make modifications to this sample file. Be sure
to get the IP address right. Fill in the name, probably your own name. That name will be used in the next
section for authentication.

nscgi.cfg

nscgi.cfg controls access to the CGI programs. It’s pretty obvious that you don’t want just anyone to see
the system monitor information. The official information for this step is contained in your online documentation
(available via the /netsaint/docs/cgiauth.html link) or
at NetSaint. The short version of the story is
we use .htaccess and a database in order to authenticate users via CGI. For more information about .htaccess files,
try reading Protected Apache directories. In the meantime, I’ll just press ahead
with a simple setup.

The directory you want to protect is /usr/local/share/netsaint/sbin. You can protect it by placing a
.htaccess file in that directory. In that file you would place the following:

AuthName "NetSaint Access"
AuthType Basic
AuthUserFile /usr/local/etc/netsaint/htpasswd.users
require valid-user

You then create a password entry using the steps shown in Protected Apache directories.
Be sure to use the same name as you used in the hosts.cfg file.
You add this entry to /usr/local/etc/netsaint/htpasswd.users:

dan:esaBHd2tW29g2

After doing those changes, Apache will prompt you for a user name and password when you broswe to one of
the CGI directories (e.g. Status Overview).

The next step is to specify what commands are avialble to you. This is done via the nscgi.cfg.
To get you started, just add the following. WARNING: This may be more than you want to allow. Read
the documentation.

authorized_for_all_hosts=dan
authorized_for_all_host_commands=dan
authorized_for_all_services=dan
authorized_for_system_information=dan
authorized_for_system_commands=dan
authorized_for_configuration_information=dan

Now it’s time to start NetSaint and see what happens….

Starting NetSaint

Starting NetSaint is easy:


# /usr/local/etc/rc.d/netsaint.sh start
Starting network monitor: netsaint
PID TT STAT TIME COMMAND
88260 ?? Ss 0:00.03 /usr/local/bin/netsaint -d /usr/local/etc/netsaint/netsaint.cfg

Similarly, you can use the stop and restart options.

NetSaint process may not be running!

If you happen to see the message ‘NetSaint process may not be running!’ when you broswe to /netsaint,
don’t worry. First, verify that NetSaint is indeed running. You should see something like this:


$ ps auwx | grep netsaint
netsaint 89648 0.0 2.1 1876 1288 ?? Ss 7:54PM 0:00.11 /usr/local/bin/netsaint -d /usr/local/etc/netsaint/netsaint.cfg

If you see something like that, then you know that NetSatint is actually running. If you click
on the link ‘Click here for more info.’ (right below ‘NetSaint process may not be running!’), you’ll find that
you need to modify nscgi.cfg and supply a value for netsaint_check_command. That’s
pretty easy. All you need to do is uncomment the line on which that keyword is found and
restart
NetSaint.

OK, you should be running now

You should have NetSaint running now. There’s no reason why it shouldn’t be running. If you are having
trouble, and it’s because of something not covered in this article, please add your solution/problem
to the article comments. In any case, please consider this. Thanks.

Optional configuration

The steps in this section are optional. They may not be applicable to you. I strongly urge you to get
everything else working, then consider these steps.

In this section, I’m going to restrict things. I’m not going to let just anyone read my NetSaint configuration
files. I’m only going to let the netsaint user read them. Let’s just say a little bit of paranoia is a good thing
when it comes to security.

The first step is to chmod o-r the configuration files.

cd /usr/local/etc/netsaint
chmod o-r *

But if you do just that, you’ll find that Apache can’t read htpasswd.users. This will manifest
itself as authentation failures in your Apache logs:


[error] [client 24.102.95.54] user dan not found: /cgi-bin/netsaint/status.cgi

The solution is to make htpasswd.users readable by Apache:

chgrp www htpasswd.users

But then you’ll have trouble with the other configuration files as well. So I found it easiest to just do this:

chgrp www *

That will go some way to restricting access to your configuration files but still allow the
web interface to function.

FreeBSD graphics for NetSaint

NOTE: The graphics rerferred to by in this section are now included in the FreeBSD port. The steps
below are no longer necessary.

If you have a look at the web pages for NetSaint, you’ll notice it’s rather Linux-centric. Fair enough.
It is designed for Linux (it says so right on their homepage. But
RevDigger decided that some better graphics were in order. In the spirit of sharing, he’s allowed me
to distribute these graphics.

The graphics are available at:

These files should be place at /usr/local/share/netsaint/images
There are only two simple changes you need to make. Remember to change both the URL and
the graphic dimensions. I won’t provide a diff. Merely a simple before-and-after. If the
graphics don’t look right, you probably messed up on the dimensions. The filee you want to change are
located at /usr/local/share/netsaint/.

  • side.html


    <img src="/images/sblogo.gif"     width="130" height="125" border="0">
    ---
    <img src="/images/bsd_sblogo.gif" width="110" height="95"  border="0">

  • main.html


    <IMG SRC="/images/logofullsize.gif"     BORDER="0" ALT="NetSaint" width="238" height="227">
    ---
    <IMG SRC="/images/bsd_logofullsize.gif" BORDER="0" ALT="NetSaint" width="268" height="221">

Possible problems

I think NetSaint has the ability to change configuration items from the webpage interface. If so, you
may need to make the configuration files chmod +w with respect to your http process. In my case, I’m running
Apache as www. So I made the NetSaint configuration files (/usr/local/etc/netsaint)
a member of the www group and chmod g+r.

If you happen to see the following message:

It appears as though you do not have permission to view
information for any of the hosts you requested…

Then I suspect your .htaccess user name and your NetSaint user names are not matching up. Or you didn’t
specify any permission for your user name in nscgi.cfg.

If you keep seeing messages like this on your screen:


/home/dan/.mail_aliases: Permission denied
/home/dan/.mail_aliases: Permission denied

Or these in /var/log/messages:


forward /usr/local/var/netsaint/.forward.dev+: Group writable directory

Then I know of a change which will fix the problem (if it hasn’t already made it into NetSaint>.
Modify /usr/local/etc/rc.d/netsaint.sh to add a single line (unset -v HOME) as shown
below:

rm -f $NetsaintCmd
unset -v HOME
$NetsaintBin -d $NetsaintCfg

That should stop the messages.

Thanks

I wish to thank the following people for reviewing this article.

  • Shane Allen
  • Atle Veka

It’s always helpful to have people read what you’ve written before it is released upon the world.

Was that useful

If you found that article useful, then please consider supporting this website.
At the moment, we’re trying to raise funds to pay for the new year of colocation. Thank you.

19 thoughts on “NetSaint – a network monitor”

  1. Brandon D. Valentine

    Those BSD graphics are pretty sharp. Someone really oughta write a patch and add that to the port so that on FreeBSD netsaint installs with those by default. Hmm…checking copious free time schedule… We’ll see.

    -Brandon

    P.S. Isn’t it annoying when you’re typing in a web form and try to hit escape to enter command mode, only to remember Netscape doesn’t support modal editing. =(

    1. I’ll field this one.

      I’ve been running BB for a couple years now and it’s been pretty decent. The overall impression that one gets from BB is that it is a giant hack. It’s all shell scripts and convoluted pipes and greps and stuff. I’m sure the author was an over worked sysadmin who needed to take a day off, so he cobbled together a bunch of hacks, and it worked, so he let his friends use it too.

      Netsaint, however, gives one the impression that it was actually designed. Things are well organized, funtions are in plugins, etc. I have found Netsaint allows me a lot finer control over it behaviour, but then it also required a lot more work to get going properly. Netsaint seems to do fewer weird things.

      By the way, I have the original PSDs of the BSD-Style graphics, if anyone want to play with them…

  2. Has anyone gotton the netsaint.cmd file for external commands working [FreeBSD 4.3_REL]?

    I follow the directions to the T, but when Apache [as nobody] creates the netsaint.cmd, the sticky bit for the group gets ignored, thus the file’s group can’t do squat and NetSaint never processes it.

    I sifted through the newgroups, but found little if nothing on it..

    Thanks

    1. chmod 666 /usr/local/var/netsaint/rw/netsaint.cmd after the netsaint program is running… rm the file if netsaint dies without reason. rm’ng the file cleans up the last crash, and the chmod will allow the web server (user nobody) to write to the cmd file.

      DJM:>

    2. I’m having the same problem too. I can make the cgi’s work onthe command line but only if the command file is prwxrwxrwx. It never works when I try to do anything like disabling a test or functions that involve the nagios.cmd file…

  3. Netsaint comes with lots of tools that monitor remote services, like the HTTP and SMTP checkers. But how do you check remote status properties like free disk, or load average? Well of course, people have thought of this.

    The "Addons: section of the Netsaint download page:

    <A HREF="http://www.netsaint.org/download/">http://www.netsaint.org/download/</A&gt;

    has at least 3 different packages for doing this. I looked at:

    netsaint_statd v2.11
    NetSaint Remote Plugin Executor (NRPE) 1.2.4
    NetSaint Remote Plugin Executor/Perl (NRPEP) 0.2

    For various reasons, I went with netsaint_statd, but if you have had a good experience with any of the others, let me know. netsaint_statd consists of an RPC daemon that runs on the remote machine, and a series of plugins with which the Netsaint server can query it. For security, it implements an allowed-hosts list and will only report a limited set of data.

    After untaring and reading through the brief INSTALL doc, I simply put a copy of the rpc daemon on each of the servers from which I wanted reports. I decided that /usr/local/libexec/ was a decent place for it. The RPC daemon is a Perl script, and you will want to edit it to add your Netsaint server to the list of addresses allowed to query it. Since I seem to be restarting these servers all the time (every 3 – 6 months), I dropped a little init script in /usr/local/etc/rc.d/. It looks like:

    #!/bin/sh
    /usr/local/libexec/netsaint_statd

    Once I had those all running I returned to my Netsaint server and dropped the new plugin in my Netsaint plugins directory. Actually I put them in a sub directory,

    /usr/local/libexec/netsaint/netsaint_statd/

    to keep things tidy. Last thing to do was to let Nesaint know about its new toys.

    First I added the following lines to Netsaint’s commands.cfg:

    # netsaint_statd remote commands
    command[check_rload]=$USER1$/netsaint_statd/check_load.pl $HOSTADDRESS$ $ARG1$ $ARG2$ $ARG3$
    command[check_rprocs]=$USER1$/netsaint_statd/check_procs.pl $HOSTADDRESS$ $ARG1$ $ARG2$ $ARG3$ $ARG4$
    command[check_rusers]=$USER1$/netsaint_statd/check_users.pl $HOSTADDRESS$ $ARG1$ $ARG2$ $ARG3$
    command[check_rdisks]=$USER1$/netsaint_statd/check_all_disks.pl $HOSTADDRESS$ $ARG1$ $ARG2$ $ARG3$ $ARG4$
    command[check_rnamed_proc=$USER1$/netsaint_statd/check_named_proc.pl $HOSTADDRESS$ $ARG1$ $ARG2$ $ARG3$ $ARG4$

    which I mostly just copied from the netsaint_statd README. Lastly, I added services I wanted to monitor to my hosts.cfg and restarted my Netsaint process. That’s it, really.

    Another nice thing is that adding your own plugins looks like it would be easy to hack in.

  4. Here’s what I had to do for checking https:

    hosts.cfg:

    service[dev]=HTTPS;0;24×7;3;2;1;freebsd-admins;120;24×7;1;1;1;;check_https!

    commands.cfg:

    command[check_https]=$USER1$/check_http –ssl $HOSTADDRESS$

    I recommend testing from the command line to ensure your check_http can do this:

    $ /usr/local/libexec/netsaint/check_http –ssl 10.0.0.1
    HTTP ok: HTTP/1.1 200 OK – 1 second response time

    Make sure you have https not running and http running. Then test it again:

    $ /usr/local/libexec/netsaint/check_http –ssl 10.0.0.1
    Connection refused by host

    $ /usr/local/libexec/netsaint/check_http 216.187.106.227
    HTTP ok: HTTP/1.1 200 OK – 1 second response time

    Then you know it’s working and it’s actually checking https and not http.

    You might, and I stree might have to recompile the plugins. My memory is hazy but I don’t think I had to do this in order to get the –ssl option to work:

    cd /usr/local/etc/netsaint/plugins/netsaint-plugins-1.2.9-4
    make clean
    ./configure
    make
    cp check_http /usr/local/libexec/netsaint

    I’ll verify this and update as necessary.

    1. I forget if this is standard or something I did myself, but my commands.cfg has this:

      command[check_ssl]=$USER1$/check_http -H $ARG1$ -S -C $ARG2$

      -H is for the host name, in case it’s a VHost and that is pertinent. -S tells it to use ssl. And -C is the number of days before the cert expires to start warning me. That’s right, it checks the expire date on the cert for you. Too cool.

      So in my hosts.cfg I have something like:

      service[lisa]=SSL;0;24×7;3;2;1;unix-admins;180;24×7;1;1;1;;check_ssl!secure.myhost.com!14

      Works great. In my Netsaint Status Detail, I get, "Certificate will expire on 07/03/2002 23:59."

  5. Dale Offret Jr.

    I have followed your article, by the way it is great, and setup my cfg files and such. I do have netsaint running but thttpd reports and internal error.

    500 Internal Error

    There was an unusual problem serving the requested URL ‘/cgi-bin/netsaint/statusmap.cgi?host=all’.

    thttpd/2.21b 23apr2001

    Anyone who has this working please let me know.

    Thanks,

    Dale Offret Jr.

    1. The only people who might see your message here are those that are reading the article. Why don’t you click on FORUMS and post your problem on the Support forum where many more people will see it.

  6. Hello,

    I read the netsaint documentation and put as many things as possible into MySQL database. To do this, you have to make these steps
    cd /usr/ports/net/netsaint/

    make CONFIGURE_ARGS+=’–with-mysql-xdata –with-mysql-lib=${LOCALBASE}/lib/mysql’ all install

    mysqladmin create netsaint
    cat /usr/ports/net/netsaint/work/netsaint-0.0.7/contrib/database/ | mysql netsaint

    mysql mysql
    —- cut here —–
    insert into user (host,user,password) values (‘localhost’,’netsaint’,password(‘your_password_here’));
    insert into db values (‘%’,’netsaint’,’netsaint’,’Y’,’Y’,’Y’,’Y’,’Y’,’Y’,’Y’,’Y’,’Y’,’Y’);
    flush privileges;
    \q
    —- cut here —–

    cd /usr/local/etc/netsaint/
    edit following files:
    resource.cfg to activate DB * DATA
    nscgi.cfg to activate DB * DATA
    nscgi.cfg to *CHANGE* command for checking netsaint alive (read below)
    netsaint.cfg to comment (#) comment_file

    Why you have to cahnge netsaint_check_command? Well, netsaint uses:
    /usr/local/libexec/netsaint/check_netsaint /usr/local/var/netsaint/status.log 5 ‘/usr/local/bin/netsaint’

    with default. When you activate all possible data with DB, there is no /usr/local/var/netsaint/status.log file, so you will receive error message, that possible netsaint is not running. And then, some command could not be run. So to make netsaint daemon satisfied, I change netsaint_check_command with value:

    /usr/local/libexec/netsaint/check_procs -w 2 -c 2 -C ‘/usr/local/bin/netsaint’

    And everything is OK.

    What are advantages of using DB? Well, there are some:

    * it is very, very easy to modify 2D or 3D status map (just ine update in hostexinfo table)
    * you save disk sapce for some logs (status.log)
    * it’s much quicker to get some info from DB.
    * after update in hostexinfo table there is no need to reload netsaint with new configuration.

    Unfortunatelly in current release (0.0.7_2,1) there no possibilities to move hosts.cfg to database. but maybe in the future… Who knows…

    I hope, that I missed nothing, so you could repeat this steps without any error.

    Please feel fre to post me, if you have any problems

    Greeting
    Krzysztof Stryjek

    1. Hello,

      Some time ago I’ve posted a PR to include Database support in netsaint and nagios ports. This PR was closed (support included) about 2-3 month ago. So now it is much easier to do it.

      Best regards
      Krzysztof Stryjek

Leave a Comment

Scroll to Top