Two years of silence

I have been so focused on vFense, that I have been negligent of this blog. vFense has come a long way from it’s inception a little over 20 months ago. We are still in the Beta, but we are continuously making improvements. As of right now there

Posted in Blog | Leave a comment

HowTo check redis availability and get stats using Redis-py, Zenoss, and Python

My team was assigned to create a redis slave status check to be ran under Zenoss. So while they are creating that check, I decided to google for redis checks written in python that work under Nagios and or Zenoss and none of the checks i found did exactly what I wanted.

So I decided to write my own check, that will grab every piece of data that the redis info() command was able to retrieve. Since this info is all in a python dictionary, I was able to get all the stats that were labeled as type int or as type float, which made my job that much easier.

So 1st I had to install 2 Redis instances on my local Ubuntu server at home. Now that i completed that, I had to make sure the slave was syncing off the master server. I used this link “http://redis.io/topics/replication” to help me configure redis replication.

Now that all that stuff is out of the way, I wrote an easy_peasy python script to connect to redis and grab all of the performance stats. As well as verify if the instance is a master or a slave instance. If it is a slave instance, than it also verifies if it is syncing to the master or not.

The script is using Redis-py that was installed using PIP.

Here is the script I wrote

Continue reading “HowTo check redis availability and get stats using Redis-py, Zenoss, and Python” »

Posted in Python, Redis, Zenoss | Tagged , , | Leave a comment

How To monitor RTSP streaming videos using openRTSP and Zenoss

Since I do currently work for a streaming company, that would imply that we should have some type of monitoring for our RTSP streams ;-). You will 1st need to get the openRTSP command.

  • You will need to download the openRTSP command from http://www.live555.com/openRTSP/
  • Or if you are running Zenoss on top of Debian you can just run a apt-get install livemedia-utils

Once you do that, all you have to do next is to download my script and have a valid server to point at and a path to test… Example below.. Download

or with stats…

So you can trend the different stats that openRTSP provides in Zenoss..

Posted in Python, Zenoss | Tagged , | Leave a comment

How To Monitor and get Bind 9 stats using Zenoss

Hey guys, here is another Python script by me :-) . The reason for this script, is to give you the capabilities to monitor and graph Bind 9 stats. This script will be executed through SNMP.. We will be using the UCD-MIB 1.3.6.1.4.1.2021.8.1

You can download the script here .. Download

Continue reading “How To Monitor and get Bind 9 stats using Zenoss” »

Posted in Python, SNMP, Zenoss | Tagged , , | Leave a comment

How To monitor NodeJS using the Stats plugin that comes with Cluster using Zenoss.

We needed a way to monitor all of our NodeJS instances in Zenoss and not just a simple TCP connection. Since we were already using the Cluster plugin for NodeJS ‘http://learnboost.github.com/cluster/’ and we are using the stats plugin for Cluster ‘http://learnboost.github.com/cluster/docs/stats.html’. We decided to write a quick python script to connect to the stats plugin and parse that data to represent it to Zenoss.

You have 2 options… Run this script by itself using Zenoss to connect to the host or if you are using a socket connection or connecting to localhost only then use SNMP.

Download Here .. Download

  • ( IN SNMP ) Options are modifiable ‘exec CheckClusterLiveStats /usr/local/bin/check_clusterlive_stats.py -t tcp -c localhost:8888
  • Or directly in Zenoss ‘/usr/local/bin/check_clusterlive_stats.py -t tcp -c remotehost:8888′

This script can be used to connect to a TCP socket or to a local file socket. Example output from Zenoss Test Command…

The command above you can run with out snmp. The reason we use snmp because we only allow the stats plugin to run on the localhost interface. So we use snmp to execute the script and in return, it returns the results.

 

Posted in HowTo, NodeJS, Python, Zenoss | Leave a comment

How To Monitor Http Status Codes using Zenoss and Snmp.

We needed a way to monitor a few different type of HTTP Status Codes, specifically… ( 400, 404, 500, 503, 504, 200 ). This list is modifiable within the script. Though I’m thinking of turning this list into an option, since everyone might not one to gather the count just for those mentioned above.

So In order to get this data, you will need to use SNMP and install logtail. To make this script work you will need to have an snmpd exec statement in the snmpd.conf file and a crontab entry for the script to run as often as you like.

This script has been tested, using Nginx and Ruby On Rails.
You can download the script here.. Download

  • On cron, remember this is modifiable ‘* * * * * /usr/local/bin/get_http_codes.py -d “/var/log/nginx” -f “www-access.log” >/tmp/errorcount.txt’
  • In snmpd.conf without quotes ‘exec GetHttpStats /bin/cat /tmp/errorcount.txt

In Zenoss you will have to use the check_snmp nagios command or the inherent snmp check from zenoss. The oid you need to use is the UCDavis OID. For instance I’m using 2 exec statements in snmpd.conf, so the OID I am using for this check is this one ‘.1.3.6.1.4.1.2021.8.1.101.2‘ The command I’m using in zenoss is like this ‘check_snmp -H ${dev/manageIp} -C readonly -P 2c -o .1.3.6.1.4.1.2021.8.1.101.2

The current options for get_http_codes.py is ‘-d‘ which is the directory where the log file is located and ‘-f ‘ the name of the logfile. This script uses logtail so that I can always get the difference from the last time I scanned the log file.

The purpose of this script is so that you can graph or create a threshold of how many of the below Http Error Codes you are getting between every check. For instance we run this check every 30 seconds in Cron and in Zenoss and we divide the results by 30 so we can get how many of the error codes are happening per second.
Example of the output of the script is ‘SNMP OK – “Nginx Codes OK|count200=181 count400=1 count404=0 count500=0 count503=0 count504=0″ |

If you have any feature request or have any questions, please leave a comment. Thank you

Posted in HowTo, Python, Zenoss | Tagged , , , | Leave a comment

It’s been a while..

I have not been doing to many interesting projects as of the past year. This is the reason for my lack of updates. I am now starting to do some interesting stuff, so let the updates roll.

Posted in Blog | 4 Comments

HowTo add Aggregate Data Graphs from existing datapoints in Zenoss

Recently I had to aggregate the amount of 200, 400, 404, 500, 503,  and 504 HTTP 1.1 Codes from all of our Nginx and Ruby On Rails systems. So I decided to write a quick plugin for Zenoss that all you need to do is pass the devices or Device Class you want to aggregate the data from and the DataPoint name.

Also this makes it simple to use the Zenoss Thresholds and create Aggregate Reports based on the datapoints you use with out having to do some fancy python coding… :) Enjoy

 

Example..

Download

 

Posted in Python, Zenoss | Tagged , , | Leave a comment

Monitoring RabbitMQ with Zenoss

RabbitMQ was recently deployed in the company I currently work for. At the last minute ( as always ) they came to me and ask me to please add RabbitMQ monitoring to Zenoss. They said here is the url and now please monitor for a few stats ( using the RabbitMQ Status Plugin ). So I said to myself, I could easily just write a quick shell script to get the 3 stats that they needed and add them into Zenoss. After thinking about it…. In the near future they might ask for more than those 3 stats. So I decided to write a quick python script ( Zenoss Compatible ) to get all the stats from that status page and input them into Zenoss…

Prerequisites

  1. Zenoss 2.5 and above ( I have not tested on 3.+ or <2 .4 )
  2. lxml Python module    “easy_install lxml” as the zenoss user
  3. Zenoss_Template_Manager.py” Optional, Only needed if you do not want to add all the datapoints manually… I DON’T!!!!
  4. RabbitMQ Installed
  5. RabbitMQ Status Plugin from http://www.lshift.net/blog/2009/11/30/introducing-rabbitmq-status-plugin
  6. check_rabbitmq.py

Zenoss Template Manager = Download

CheckRabbitMQ.py = Download

Once you have all the above, we are ready to go..

  1. copy both Python Scripts above into the /opt/zenoss/libexec/ folder  ( If you are using RedHat/CentOS ) and make sure they are executable.

As the Zenoss user run the script ….
/opt/zenoss/libexec/check_rabbitmq.py -u ‘http://rabbitmq-server:55672′ -a ‘mon-user mon-passwd’ |sed -re “s/^OK|/ /g” |sed -re “s/([A-Za-z0-9_.]+*)?=[0-9]+/-p “1,G”/g” |xargs /opt/zenoss/libexec/Zenoss_Template_Manager.py -o “/Devices/Server/Linux/RabbitMQ” -c ‘/opt/zenoss/libexec/check_rabbitmq.py -u “http://rabbitmq-server:55672″ -a “mon-user mon-passwd”‘ –template=RabbitMQ –dsource=RabbitMQStats -V $1

If you need to know how to use the Zenoss_Template_Manager.y script, check here http://www.linuxdynasty.org/howto-add-multiple-datapoints-to-zenoss-using-the-zenoss-api.html

So let me explain what the sed statements above are doing…

  • “sed -re “s/^OK|/ /g”” This sed statement is removing the OK| from the beginning of the line
  • “sed -re “s/([A-Za-z0-9_.]+*)?=[0-9]+/-p “1,G”/g”
    This 1st part of the sed statement is matching any letter,number,underscore, and period, any number of times until it reaches the equal “=”  “[A-Za-z0-9_.]+*)?=
    The  2nd part of this statement is going to match the “=” and any number of integers after it. “[0-9]+
    Now we need to make the substitution…. So we are going to substitute, every match with a -p,  then a space and then the 1st group match in escaped quotes, then a comma and G for GAUGE.

The 2 sed statements above will do that for every match it finds. If you were to add each datapoint by hand, it would look like this….
/opt/zenoss/libexec/Zenoss_Template_Manager.py -o “/Devices/Server/Linux//RabbitMQ” -c ‘/opt/zenoss/libexec/check_rabbitmq.py -u “http://rabbitmq-server:55672″ -a “mon-user mon-passwd”‘ –template=RabbitMQ –dsource=RabbitMQStats -V -p “queue.conversion.event.tracking_msg_unack,G” -p “connections,G” -p “erlang_processes_used,G” -p “erlang_processes_avail,G” -p “file_descriptors_used,G” -p “file_descriptors_avai,G” -p “binary_memory,G” -p “memory_used,G” -p “memory_avail,G”

I hope the above scripts will save someone time and frustration……

connections=26 erlang_processes_used=252 erlang_processes_avail=1048576 file_descriptors_used=1 file_descriptors_avail=1024 pid=3167  binary_memory=0 ets_memory=0  memory_used=27 memory_avail=99
3

 

Posted in Zenoss | Tagged , | Leave a comment

HowTo Send Splunk Alerts To Zenoss, And make them Look Like Splunk

We needed to integrate the Splunk Alerts into Zenoss, because even though Splunk can indeed send out alerts. Splunk does not have any clue about what an “Escalation Process” is. With Zenoss you can create an “Escalation Process”.

I have 2 ways to send events to Zenoss from Splunk..

  1. Write a Script that uses the snmptrap command.
  2. Write a Script that uses the Zenoss zensendevent command.

I decided to go with the Zenoss zensendevent command ( Which is a python script with no external dependencies, which can be copied from the Zenoss Server at $ZENHOME/bin/zensendevent ).

Now it’s time to get the ball rolling..

  1. On the Splunk Server I copied the zensendevent script from the Zenoss Server to Splunk on /opt/splunk/bin/scripts/zensendevent.
  2. I then created a shell script called Splunk2Zenoss.sh. ( This script takes the Saved Splunk Search and passes it over to Zenoss )This script will also be located in /opt/splunk/bin/scripts/Splunk2Zenoss.sh
  3. You will then need to modify the options in the script. (For instance the severity of the alert, the zenoss server, the event mapping, event key, login and passwd )
  4. I then created the saved search in Splunk and make sure to check the Trigger Shell Script option. ( Make sure to put the script name in here )

Continue reading “HowTo Send Splunk Alerts To Zenoss, And make them Look Like Splunk” »

Posted in Zenoss | Tagged , , | 2 Comments