On a Linux machine, gathering and graphing network traffic statistics is another tool in the sysadmin toolbox for determine usage patterns of a server. To accomplish this we will use a tool called Multi Router Traffic Grapher (MRTG) which gathers SNMP data from the server, stores it in a database file, and generates graphs viewable through a CGI program.
This document will describe setting up the system on a Debian Woody machine, but it should be easy to adapt to other distributions.
First we'll need a few software packages:
If you don't already have SNMP on this machine, you'll need it.
• Configure SNMP
OK, first we have to configure SNMP to allow MRTG to read statistics from the snmp daemon.
First backup the snmpd.conf file.
cp /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.origNow, we need to make a few changes to this config file, so open it in your favorite editor. You'll see a section that looks like this near the top:
# sec.name source community com2sec paranoid default public #com2sec readonly default public #com2sec readwrite default privateChange it to look like this:
# sec.name source community #com2sec paranoid default public #com2sec readonly default public #com2sec readwrite default private com2sec readonly localhost publicWe've commented out the "paranoid" line and added a line at the end that allows readonly access to the SNMP data from localhost. This change will NOT allow SNMP access from other hosts on the network, so rest easy that security is preserved.
I am by no means an expert on configuring SNMP, so I will simply provide my snmpd.conf file for those that are trying to get this working on a non-Debian system.
After you change this file, you must tell snmpd to read its new configuration.
/etc/init.d/snmpd reload # - or send snmpd a SIGHUP - ps -ef | grep snmpd | grep -v grep | cut -c10-15 | xargs kill -HUP
Now, you can test out SNMP with the command.
Debian Woody: snmpwalk -v 1 localhost public interface Debian Sarge: snmpwalk -v 1 -c public localhost interfaceIf you receive data back from this command, then MRTG should be able to as well. (If you just see "End of MIB" then SNMP is not configured correctly. Did you remember to reload snmpd?) Also, you can use this command from another Linux host to make sure that you have not opened SNMP access to the world. Just replace the 'localhost' parameter with the IP address of your machine.
• Configure MRTG
Next, we'll configure MRTG for this machine.
First, backup the existing MRGT config file.
mv /etc/mrtg.cfg /etc/mrtg.cfg.orig
Next run the following command:
cd /etc cfgmaker public@127.0.0.1 > mrtg.cfgTake a look at your new mrtg.cfg file. cfgmaker should have created an entry for every network interface in the machine. (By default the 'lo' interface will be commented out and I would leave it that way unless you have a compelling reason to monitor the loopback interface.)
There are a few things in this file we have to edit ourselves, so hop
into it. The first you may want to change is the WorkDir option.
WorkDir: /var/www/mrtgThis specifies the directory in which mrtg writes its database files. Change it if you want.
Underneath this line, we'll add two lines.
LogFormat: rrdtool LibAdd: /usr/lib/perl5/This specifies that we want to use rrdtool to store the database files. The "rrd" stands for round-robin database, meaning that it only lets the database files grow to a certain size before it overwrites the oldest data. This is preferable to the native MRTG file format in which the files just keep growing and growing and growing. The LibAdd options explicitly tells MRTG where your Perl libraries are located. This is the directory containing MRTG_lib.pm among other Perl libraries that MRTG needs.
Next, remove the # mark from the beginning of the following line to activate the
Options option.
Options[_]: growright, bitsThis tells MRTG to put the most current data on the right side of the graph, and to store its data in bits rather than bytes. You can read about more MRTG options here.
Finally, you will probably want to edit the Title[...] and PageTop[...] definitions for each network interface entry in the file. This is the descriptive text that will appear on the top of the graphs displayed by CGI programs. This is an optional change.
You will want to make this file readable by the webserver if you plan to
use a CGI program to view the statistics.
chmod o+r /etc/mrtg.cfg
• Schedule MRTG Data Collection
MRTG must be scheduled to run at regular intervals to collect the statistics. The standard is to collect data every 5 minutes. Debian creates a job for you in /etc/cron.d/mrtg to do this.
At this time, you will want to make sure MRTG is running correctly. Take a look at the /var/log/mrtg/mrtg.log file and make sure there are no errors. If everything is as it should be, MRTG should have added at least one RRD database file in your WorkDir directory. There will be one file for each interface that it is monitoring.
If your Linux distro has not scheduled a job, you must
add a cron job to do this. Here is the line you need to add to the crontab:
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /usr/bin/mrtg /etc/mrtg.cfg --logging /var/log/mrtg/mrtg.logIf you create this cron job for a user other than root, you must make sure that the /etc/mrtg.cfg file is readable by that user, and the WorkDir directory (from the MRTG config file) must be writeable by that user, as well as the logging directory specified in the cron job.
• mrtg-rrd CGI
Now to view the data. We'll need to grab a CGI program to view the data. I suggest mrtg-rrd for this. You can get it here. (Note: This document does not explain how to configure CGI programs to run on your webserver.)
There is one line in mrtg-rrd that you'll have to edit.
# EDIT THIS to reflect all your MRTG config files
BEGIN { @config_files = qw(/home/fadmin/mrtg/cfg/mrtg.cfg); }
Change it to point to your config file in the /etc directory.
# EDIT THIS to reflect all your MRTG config files
BEGIN { @config_files = qw(/etc/mrtg.cfg); }
mrtg-rrd will also want to write to the graph files it generates to the MRTG WorkDir directory. You must give the webserver write access to this directory, or change the mrtg-rrd program to make it write elsewhere.
Once mrtg-rrd is working correctly, it should generate a graph that will
look something like this.
The network traffic data points will start to appear on the right side
of the graph, and if you give MRTG more time to run throughout the day,
the graph will start to fill with data. There should also be one graph
for every interface that MRTG is configured to monitor. The green data
points are the incoming traffic, the blue is the outgoing.
You will probably have a couple of broken images at the bottom of the page. To fix this, you can grab the images from here and put them in an mrtg-icons directory off your webserver's DocumentRoot directory. Then add the following line to your /etc/mrtg.cfg file beneath the LibAdd option.
IconDir: /mrtg-icons
• mrtg-rrd CGI Hacks
The mrtg-rrd CGI is released under the GPL, so you can freely modify it as long as you follow the terms of the license. So, you might want to customize it to follow the look and feel of your site, or add extra data. For example, you can add 95th percentile statistics to the graph as this is a common method for billing based on network traffic.
If you want to add the 95th percentile statistics, you'll need the get95 tool. This tool require
that you grow your RRD database files, but this is easy to do and won't
destroy your existing data. Make sure you follow the instructions for
get95 closely or it won't work. The key command is making rrdtool grow
the RRD files.
rrdtool resize file.rrd 0 GROW 8400You also have to edit get95.pl to change the path for the MRTG config file to point to yours, the /etc/mrtg.cfg file.
Here is a version of mrtd-rrd that I have modified to add the 95th
percentile statistics to the graphs.
mrtg-rrd-get95.cgi
It assumes that you have installed get95.pl in the /usr/local/bin directory.
• Sources
| MRTG | RRDtool | |
| mrtg-rrd | Debian | |
| get95 |
| Copyright (C) 2004 Adam P. Whitney |