Tuesday, February 15, 2011

Monitoring Servers and Clients using Munin in Debian Linux

What is Munin?

"Munin" means "memory".

Munin the tool surveys all your computers and remembers what it saw. It presents all the information in in graphs through a web interface. Its emphasis is on plug and play capabilities. After completing a installation a high number of monitoring plugins will be playing with no more effort. Using Munin you can easily monitor the performance of your computers, networks, SANs, and quite possibly applications as well. It makes it easy to determine "what's different today" when a performance problem crops up. It makes it easy to see how you're doing capacity wise on all limited resources.

It uses the excellent RRDTool  and is written in Perl. Munin has a master/node architecture in which the master connects to all the nodes at regular intervals and asks them for sdata. It then stores the data in RRD files, and (if needed) updates the graphs. One of the main goals has been ease of creating new plugins (graphs).


Download Munin

http://sourceforge.net/projects/munin/

Munin Documentation

http://munin.projects.linpro.no/wiki/Documentation

Munin Plugins

http://munin.projects.linpro.no/wiki/PluginCat

Munin in Debian

Munin is in the Debian archive in two parts
  • munin((munin server) - the part that creates the monitoring graphs
  • munin-node((munin Client) - the munin client program.
Munin Server Configuration in Debian

If you want to install munin server you need to install munin and munin-node packages using the following command

#apt-get install munin munin-node
this will install munin in /etc/munin directory this includes the following files

munin.conf  munin-node.conf  plugin-conf.d  plugins  templates
This will install files for webserver root directory in /var/www/munin directory this includes the following files

definitions.html index.html  localdomain  logo.png  style.css

If you want to configure munin server you need to edit the /etc/munin/munin.conf file.The sample file looks like this

# Example configuration file for Munin, generated by 'make build'

# The next three variables specifies where the location of the RRD
# databases, the HTML output, and the logs, severally.  They all
# must be writable by the user running munin-cron.
dbdir   /var/lib/munin
htmldir /var/www/munin
logdir  /var/log/munin
rundir  /var/run/munin

# Where to look for the HTML templates
tmpldir /etc/munin/templates

# Make graphs show values per minute instead of per second
#graph_period minute

# Drop somejuser@fnord.comm and anotheruser@blibb.comm an email everytime
# something changes (OK -> WARNING, CRITICAL -> OK, etc)
#contact.someuser.command mail -s "Munin notification" somejuser@fnord.comm
#contact.anotheruser.command mail -s "Munin notification" anotheruser@blibb.comm
#
# For those with Nagios, the following might come in handy. In addition,
# the services must be defined in the Nagios server as well.
#contact.nagios.command /usr/sbin/send_nsca -H nagios.host.com -c /etc/send_nsca.cfg

# a simple host tree
[localhost.localdomain]
   address 127.0.0.1
   use_node_name yes


In this above sample config file we need to look maily these files

dbdir   /var/lib/munin
htmldir /var/www/munin
logdir  /var/log/munin
rundir  /var/run/munin


If you want to change any of the paths you need to menction here

Most Important thing is adding clients machines to munin.conf file for this you can see by default localhost.localdomain is added under a simple host tree that looks like this

# a simple host tree
[localhost.localdomain]
   address 127.0.0.1
   use_node_name yes


Now server side configuration ready. Now we are going check clients configuration

Munin Clients Configuration in Debian
If you want to monitor any number of client machines using munin you need to install munin-node package in all your clients machines

Installing munin Client in Debian

If you want to install munin client in your machine you need to enter the following command

#apt-get install munin-node
This is install munin node package and it will create a folder called /etc/munin this contains the following files

munin-node.conf  plugin-conf.d  plugins
munin-node.conf - Client Configuration File

plugin-conf.d - Configuration of plugins for this node

plugins - A directory in which each file is a symlink to a real plugin in /usr/share/munin/plugins

Now you need to configure the munin-node.conf file

same file looks like this

#
# Example config-file for munin-node
#

log_level 4
log_file /var/log/munin/munin-node.log
port 4949
pid_file /var/run/munin/munin-node.pid
background 1
setseid 1

# Which port to bind to;
host *
user root
group root
setsid yes

# Regexps for files to ignore

ignore_file ~$
ignore_file \.bak$
ignore_file %$
ignore_file \.dpkg-(tmp|new|old|dist)$
ignore_file \.rpm(save|new)$

# Set this if the client doesn't report the correct hostname when
# telnetting to localhost, port 4949
#
#host_name localhost.localdomain

host_name   munintest.test.com

# A list of addresses that are allowed to connect.  This must be a
# regular expression, due to brain damage in Net::Server, which
# doesn't understand CIDR-style network notation.  You may repeat
# the allow line as many times as you'd like

allow ^127\.0\.0\.1$

allow ^172\.30\.5\.132$


In the above configuration file there are two important things you need to enter first one is under
#host_name localhost.localdomain you need to add your client machine fully qualified name

#host_name localhost.localdomain
host_name   munintest.test.com


Second one is you need to enter the server ipaddress by defauly you can see 127.0.0.1 in your config file under that you need to add your munin server ipaddress example as follows

allow ^127\.0\.0\.1$
allow ^172\.30\.5\.132$


Now you need to add plugins for for your client machine to monitor the required services for this edit the file located at /etc/munin/plugin-conf.d/munin-node sample file looks like below

# This file is used to configure how the plugins are invoked.
#
# user <user>         # Set the user to run the plugin as.
# group <group>       # Set the group to run the plugin as.
# command <command>   # Run <command> instead of the plugin. %c expands to
#                       what would normally be run.
# env.<variable>      # Sets <variable> in the plugin's environment, see the
#                       individual plugins to find out which variables they
#                       care about.

[apt]
user root

[cps*]
user root


[fw_conntrack]
user root

[hddtemp_smartctl]
user root

[if_*]
user root

[if_err_*]
user nobody


here you need to add the user,group,command,env.<variable> and plugin name

Now look in to plugins/ directory

A directory in which each file is a symlink to a real plugin in /usr/share/munin/plugins.

Any plugin linked in here will be checked for and displayed in the resulting web pages.

Add the plugins you want (e.g. if running exim4 then I'd add exim_mailqueue and exim_mailstats).

You'll need to set user/group rights in the munin-node conf file.

Most plugins can be run from the command line with the autoconf param to check if they can run - e.g.

./exim_mailstats autoconf yes
You can add any plugins you want to monitor and default plugins are located at /etc/munin/plugins directory

Default munin Plugins are as follows

cpu entropy forks if_eth0 iostat  memory mysql_slowqueries  open_files processes df   exim_mailqueue  if_err_eth0  if_eth1  irqstats  mysql_bytes    mysql_threads      open_inodes  swap df_inode  exim_mailstats  if_err_eth1  interrupts  load      mysql_queries  netstat          postfix_mailvolume  vmstat

Now you need to restart your munis service for your client machine

#/etc/init.d/munin-node restart
That's it from client side configuration

Finall one important step you need to do in your munin server config file that is located in your munin server machine /etc/munin/munin.conf

You need to add all your client machines list under this

# a simple host tree
[localhost.localdomain]
   address 127.0.0.1
   use_node_name yes


example:-

# a simple host tree
[localhost.localdomain]
   address 127.0.0.1
   use_node_name yes
[munintest.test.com]        -  This is our test client we have mentioned earlier
    address 172.30.5.129
    use_node_name yes


After entering all the client machine details you need to the following command to take the effect of our new changes effect.

#/usr/share/munin/munin-update  --force-root
Running munin and testing munin in Debian

Munin sets up a cron job via the file /etc/cron.d/munin which will run /usr/bin/munin-cron.

Running this file will poll each of the nodes - and then will create the graphs in /var/www/html which you can then browse under http://ipaddress/munin

Examples can be found here: http://www.linpro.no/projects/munin/example/

Keeping an eye on these graphs will help you to keep your servers running healthily - and can give advance warning of problems to come.

If you have any problems you need to check the log files of munin located at /var/log/munin directory

On server side Important log files are

munin-node.log  - should show the connections that are occuring.
munin-graph.log - should show info on the services being graphed.
munin-html.log  - should show info on the html being generated.


On Client side important log files are
munin-node.log - should show the connections that are occuring

No comments:

Post a Comment