if you already have nagios setup to monitor your servers, it's a good idea to use it to monitor your hadoop data nodes as well. you can use nrpe to call your dfs monitoring plugin. I found this blog post about setting up nrpe on CentOS extremely useful. The only trick to get it to work was to also setup rpmforge repositories for yum, so it finds the nrpe packages (nagios-nrpe and nagios-plugins-nrpe).
The next step was to add monitoring script for hadoop dfs, which I found here. Since I was using hadoop 0.20.2, I had to make some changes to the script to correctly parse values out of the dfs report:More...
In a previous post I showed how to setup Hadoop/Hive to use Derby in server mode as the metastore. Many believe MySQL is a better choice for such purpose, so here I'm going to show how we can configure our cluster which we created previously to use a MySQL server as the metastore for Hive.
First we need to install MySQL. In this scenario, I'm going to install MySQL on our Master node, which is named centos1. More...
18. November 2009 23:10
If you are reading this post chances are you are trying to setup a hadoop/hive cluster. I hope these instructions save you some time with your installation.
I assume you are starting from scratch and want to setup a small cluster with minimum OS components (as far as I could minimize it). To start, download the OS setup media, and boot each machine with it. Go through the More...
After realizing that asp.net menu is generating a huge HTML and causes the application pages to be large and slow, I decided to use CSS friendly control adapters. It reduced the page size by 50 KB, but I had some challenges to get it to show the way I wanted it. Here I want to share some of the things I learned along the way:More...
SQL Server Reporting Services ReportViewer control has a toolbar that is not displayed correctly in Google Chrome browser. You will see something like this:
The correct display looks like this:
Following code can be added to the bottom of the page to accomplish this:More...
TreeView server control in asp.net can use AJAX behind the scenes to populate child nodes when user clicks on the expand icon next to each node. If the loading process is fast, it works perfectly. But when child nodes are generated by a slow process (like a slow database query) then user experience is unacceptable. User would click on the expand and nothing would happen for a long time, leaving user wondering if the click really did anything.
In such scenarios, an animated icon that lets user know data is being loaded would help a lot. More...
I was experimenting different themes with my BlogEngine blog and noticed not all of them show the correct color coding of my source code examples. There is a simple fix for that which is to copy the section of style.css file related to "Syntax Highlighter" from a working theme to the style.css of the theme you like. That should do the trick.
Suppose you want to start your SysTray application at installation time and remove it when your application is uninstalled. Here is one way to do this:
Add an Installer Class to your project, and overwrite the Install and Uninstall methods. For Example, assuming your SysTray program is named "My SysTray": More...
If you have setup more than one email account in your Outlook (Work and Home) you may have realized a need for some type of pop-up dialog which will ask you which account you want to use when creating a new email. Outlook lets you define a default account, but obviously that's not good enough. I decided to write a small and simple add-in which will ask me to select the sending account every time I compose a new email. (Even if Outlook already has such a tool, I couldn't find it! and I wanted to gain some Office programming knowledge anyways.)
I used Visual Studio 2008 and created a new Office project of type Outlook Add-in. This will give you the basics you need. Just fill the methods with your code and you are done. More...