Two years of silence

I have been so focused on vFense, that I have been negligent of this blog. vFense has come a long way from it’s inception a little over 20 months ago. We are still in the Beta, but we are continuously making improvements. As of right now there

Posted in Blog | Leave a comment

HowTo check redis availability and get stats using Redis-py, Zenoss, and Python

My team was assigned to create a redis slave status check to be ran under Zenoss. So while they are creating that check, I decided to google for redis checks written in python that work under Nagios and or Zenoss and none of the checks i found did exactly what I wanted.

So I decided to write my own check, that will grab every piece of data that the redis info() command was able to retrieve. Since this info is all in a python dictionary, I was able to get all the stats that were labeled as type int or as type float, which made my job that much easier.

So 1st I had to install 2 Redis instances on my local Ubuntu server at home. Now that i completed that, I had to make sure the slave was syncing off the master server. I used this link “http://redis.io/topics/replication” to help me configure redis replication.

Now that all that stuff is out of the way, I wrote an easy_peasy python script to connect to redis and grab all of the performance stats. As well as verify if the instance is a master or a slave instance. If it is a slave instance, than it also verifies if it is syncing to the master or not.

The script is using Redis-py that was installed using PIP.

Here is the script I wrote

Continue reading “HowTo check redis availability and get stats using Redis-py, Zenoss, and Python” »

Posted in Python, Redis, Zenoss | Tagged , , | Leave a comment

It’s been a while..

I have not been doing to many interesting projects as of the past year. This is the reason for my lack of updates. I am now starting to do some interesting stuff, so let the updates roll.

Posted in Blog | 4 Comments

HowTo add Aggregate Data Graphs from existing datapoints in Zenoss

Recently I had to aggregate the amount of 200, 400, 404, 500, 503,  and 504 HTTP 1.1 Codes from all of our Nginx and Ruby On Rails systems. So I decided to write a quick plugin for Zenoss that all you need to do is pass the devices or Device Class you want to aggregate the data from and the DataPoint name.

Also this makes it simple to use the Zenoss Thresholds and create Aggregate Reports based on the datapoints you use with out having to do some fancy python coding… :) Enjoy

 

Example..

Download

 

Posted in Python, Zenoss | Tagged , , | Leave a comment

Monitoring RabbitMQ with Zenoss

RabbitMQ was recently deployed in the company I currently work for. At the last minute ( as always ) they came to me and ask me to please add RabbitMQ monitoring to Zenoss. They said here is the url and now please monitor for a few stats ( using the RabbitMQ Status Plugin ). So I said to myself, I could easily just write a quick shell script to get the 3 stats that they needed and add them into Zenoss. After thinking about it…. In the near future they might ask for more than those 3 stats. So I decided to write a quick python script ( Zenoss Compatible ) to get all the stats from that status page and input them into Zenoss…

Prerequisites

  1. Zenoss 2.5 and above ( I have not tested on 3.+ or <2 .4 )
  2. lxml Python module    “easy_install lxml” as the zenoss user
  3. Zenoss_Template_Manager.py” Optional, Only needed if you do not want to add all the datapoints manually… I DON’T!!!!
  4. RabbitMQ Installed
  5. RabbitMQ Status Plugin from http://www.lshift.net/blog/2009/11/30/introducing-rabbitmq-status-plugin
  6. check_rabbitmq.py

Zenoss Template Manager = Download

CheckRabbitMQ.py = Download

Once you have all the above, we are ready to go..

  1. copy both Python Scripts above into the /opt/zenoss/libexec/ folder  ( If you are using RedHat/CentOS ) and make sure they are executable.

As the Zenoss user run the script ….
/opt/zenoss/libexec/check_rabbitmq.py -u ‘http://rabbitmq-server:55672′ -a ‘mon-user mon-passwd’ |sed -re “s/^OK|/ /g” |sed -re “s/([A-Za-z0-9_.]+*)?=[0-9]+/-p “1,G”/g” |xargs /opt/zenoss/libexec/Zenoss_Template_Manager.py -o “/Devices/Server/Linux/RabbitMQ” -c ‘/opt/zenoss/libexec/check_rabbitmq.py -u “http://rabbitmq-server:55672″ -a “mon-user mon-passwd”‘ –template=RabbitMQ –dsource=RabbitMQStats -V $1

If you need to know how to use the Zenoss_Template_Manager.y script, check here http://www.linuxdynasty.org/howto-add-multiple-datapoints-to-zenoss-using-the-zenoss-api.html

So let me explain what the sed statements above are doing…

  • “sed -re “s/^OK|/ /g”” This sed statement is removing the OK| from the beginning of the line
  • “sed -re “s/([A-Za-z0-9_.]+*)?=[0-9]+/-p “1,G”/g”
    This 1st part of the sed statement is matching any letter,number,underscore, and period, any number of times until it reaches the equal “=”  “[A-Za-z0-9_.]+*)?=
    The  2nd part of this statement is going to match the “=” and any number of integers after it. “[0-9]+
    Now we need to make the substitution…. So we are going to substitute, every match with a -p,  then a space and then the 1st group match in escaped quotes, then a comma and G for GAUGE.

The 2 sed statements above will do that for every match it finds. If you were to add each datapoint by hand, it would look like this….
/opt/zenoss/libexec/Zenoss_Template_Manager.py -o “/Devices/Server/Linux//RabbitMQ” -c ‘/opt/zenoss/libexec/check_rabbitmq.py -u “http://rabbitmq-server:55672″ -a “mon-user mon-passwd”‘ –template=RabbitMQ –dsource=RabbitMQStats -V -p “queue.conversion.event.tracking_msg_unack,G” -p “connections,G” -p “erlang_processes_used,G” -p “erlang_processes_avail,G” -p “file_descriptors_used,G” -p “file_descriptors_avai,G” -p “binary_memory,G” -p “memory_used,G” -p “memory_avail,G”

I hope the above scripts will save someone time and frustration……

connections=26 erlang_processes_used=252 erlang_processes_avail=1048576 file_descriptors_used=1 file_descriptors_avail=1024 pid=3167  binary_memory=0 ets_memory=0  memory_used=27 memory_avail=99
3

 

Posted in Zenoss | Tagged , | Leave a comment

HowTo Send Splunk Alerts To Zenoss, And make them Look Like Splunk

We needed to integrate the Splunk Alerts into Zenoss, because even though Splunk can indeed send out alerts. Splunk does not have any clue about what an “Escalation Process” is. With Zenoss you can create an “Escalation Process”.

I have 2 ways to send events to Zenoss from Splunk..

  1. Write a Script that uses the snmptrap command.
  2. Write a Script that uses the Zenoss zensendevent command.

I decided to go with the Zenoss zensendevent command ( Which is a python script with no external dependencies, which can be copied from the Zenoss Server at $ZENHOME/bin/zensendevent ).

Now it’s time to get the ball rolling..

  1. On the Splunk Server I copied the zensendevent script from the Zenoss Server to Splunk on /opt/splunk/bin/scripts/zensendevent.
  2. I then created a shell script called Splunk2Zenoss.sh. ( This script takes the Saved Splunk Search and passes it over to Zenoss )This script will also be located in /opt/splunk/bin/scripts/Splunk2Zenoss.sh
  3. You will then need to modify the options in the script. (For instance the severity of the alert, the zenoss server, the event mapping, event key, login and passwd )
  4. I then created the saved search in Splunk and make sure to check the Trigger Shell Script option. ( Make sure to put the script name in here )

Continue reading “HowTo Send Splunk Alerts To Zenoss, And make them Look Like Splunk” »

Posted in Zenoss | Tagged , , | 2 Comments

Download Section, no longer requires you to be a member

I first made the Download Section for members only, just cause I wanted an idea on who is downloading the software I have written. Now, I no longer care who downloads it, as long as it helps out the people who did download it. That is all that matters. Though the forums section, does require you to be a member. I also recognize, most people are members of enough websites, that becoming a member of another one, can be over bearing.

Tagged , , | 2 Comments

HowTo get 3par disk IO stats into Zenoss

I’ve come to realize, that CIM is the new SNMP, but on steroids. Most new SAN, NAS, Network, and Operating Systems now support CIM/WBEM. To me it is easier to gather statistics and information through CIM, then it is through SNMP. In this article I am going to give you a script that will allow you to query the 3par for Disk IO stats. You will be able to grab Disk IO stats on a per Volume, per Port, or per Disk basis.You will also be able to search for a Volume, Port, or Disk, instead of just dumping all the Volumes, Ports, or Disk.

Before you download this script, you will need to download pywbem from sourceforge and install it. .

get3ParIOstats.py == Download

All the data that you get from the script, must be saved as a COUNTER and not a GAUGE.

Here is an example of searching for statistics by DISK for DISK 2:6:1..

Continue reading “HowTo get 3par disk IO stats into Zenoss” »

Posted in Zenoss | Tagged , , , , , , , | Leave a comment

How to find Alerting Rules using the Zenoss API

Good Afternoon my fellow Zenoss users. here I bring you another Python tool to use with Zenoss. This tool will allow you to list all of your alerting rules, either by Group, User, or Query. If you are like me and have over 100 Alerting Rules. Managing these rules are more then a pain in the butt. Especially when you create a new rule and you know you created it correctly, but it just does not work… Well you might have another conflicting rule. Well what do you do in this situation?? You can go through each rule manually and sooner or later you will find it.

Or you can use my tool to find it for you. You might ask… How will yout tool find the conflicting rule for me??? Well to be honest it will not find the rule for you. What it can do is search all the rules for a certain query. Which can return one or more alerting rules if they are similiar in nature. or you can have the tool, list all of your Alerting rules. Or you can have the script list all of your alerting rules by group or user. The toll will report back to you The Group/User, The Alerting Rules attached to that Group/user, and the Schedules Attached to that Alerting Rules.

Where this tool really helped me, was when the DST changed happened. None of my schedules were working anymore, so the work around is to move all the schedules ahead of time. Using this script I found all my schedules very quickly, with out haveing to go through each of my alerting rules one by one.

Download

Continue reading “How to find Alerting Rules using the Zenoss API” »

Posted in Zenoss | Tagged , , , | Leave a comment

HowTo add Multiple DataPoints to Zenoss using the Zenoss API

I’ve seen a few people in the Zenoss community forums asking for an easier way to add multiple Datapoints into Zenoss, instead of adding 1 by 1. Well, I’ve decided to try and build such a tool today. Now even though the tool is not complete (IN MY EYES). It is already has been helpful to me! As of right now the script can List Templates in a Organizer ( aka /Devices/Server/Linux ) or in a Device ( zenoss1.linuxdynasty ).

It can also create a Template, with a Command DataSource, with Multiple DataPoints. This means you can now automate the creation of Templates and DataSources and DataPoints. So if you are using a tool like Puppet, Cfengine, Bcfg2, or Chef, this process can be automated. I will be adding more features in the next few weeks.

Any question or help about this script, please post them here http://www.linuxdynasty.org/forums/Scripting/scripting/ZenossTemplateManager
UPDATES!!!!!!

  • Update 1.0.6, Fixed an issue where if youdo not pass the -G option for graph it will fail.
  • Update 1.0.5, Now have the ability to add multiple graphs with multiple datapoints
  • Update 1.0.4, Now have the ability to create thresholds, bind templates to class or device, set severities on thresholds and
    datasources, attach datapoints to thresholds.
  • Update 1.0.3, I broke down the templateManager function into 3 smaller functions. You also can now list templates and datasources for either device or organizer, and list datapoints for a device. I also added more verbosity with the -V option.
  • Update 1.0.2, You now do not have to pass a command with the script. So i f you already have a command in the DataSource, it wll not get overwritten.. Thank you Eangel, for telling me about this issue..

Upcoming Features such as….

  • Deleteing Templates, DataSources, and DataPoints
  • Copying Templates
  • Adding Graphs

The tool is called Zenoss_Template_Manager.py. You can download it here..
Download

Continue reading “HowTo add Multiple DataPoints to Zenoss using the Zenoss API” »

Posted in Zenoss | Tagged , , | Leave a comment