I'm proud to announce the release of another open source project from myself and my colleagues at Box -- Rain Gauge. It's puts a nice interface on pt-stalk, and helps manage running it as a daemon on all your mysql servers, so you don't miss the details when your database starts acting up.
What is Rain Gauge? You can think of it like a web-based pt-sift. You run pt-stalk continuously on all your database servers; this will watch your mysql process and when a configurable condition is met, then it triggers a host of collections to record what your disks, processor, and memory were doing at the time, along with mysql internal status, processlist and much much more. It can even kick off GDB backtrace collections and oprofile gathering if desired.
For example, you might trigger a collection when the number of Threads_running exceeds 100 (depending on your server, this may be a sign queries are backing up.) Something like this can happen frequently on a busy production database, but these "stalls" frequently last only a second or two. If you're not watching carefully, you'll miss them.
Why do you care about brief events like these? Because they can be a sign of trouble down the road. Small stalls may cripple your server when traffic increases. Because even these events can affect query performance. So it can impact customer experience, and every part of your backend systems as well. Cascading failures often start at the database level.
It's not necessary to fall victim to these issues. By monitoring your servers in this way, you can begin to stay ahead of some of these issues. But let's lets face it -- most problems are unexpected. When you do suffer database problems you can take the data that is gathered and immediately learn from them, you can find the solutions and fix them before you get affected by them again and again.
Rain Gauge provides a central collection point, a web interface for exploring collected data, and a set of best practices for running pt-stalk and collecting data.
What does it look like?
![]()
That's an example of exploring a sample. You can see all the different files you can select at the top, and some of them -- as in the mysqladmin sample shown -- are parsed to provide you with a neat view of the data within.
Again, like our query performance reporting tool Anemometer, this isn't anything revolutionary; I'm sure quite a few people have built similar in-house tools, and many people already use pt-stalk with the --daemonize option to continuously monitor their servers for trouble.
However, I'm betting many more DBAs out there *don't* do this yet. By releasing this, we hope to help standardize this process. We want to provide tools that we think are essential for supporting a high performance mysql database with lots of servers.
And in the best tradition of open source software, we're standing on the shoulders of giants here. Building Rain Gauge on top of the percona toolkit means this tool will benefit from the constant improvements an input of lots of admins and development that percona puts into their toolkit.
What is Rain Gauge? You can think of it like a web-based pt-sift. You run pt-stalk continuously on all your database servers; this will watch your mysql process and when a configurable condition is met, then it triggers a host of collections to record what your disks, processor, and memory were doing at the time, along with mysql internal status, processlist and much much more. It can even kick off GDB backtrace collections and oprofile gathering if desired.
For example, you might trigger a collection when the number of Threads_running exceeds 100 (depending on your server, this may be a sign queries are backing up.) Something like this can happen frequently on a busy production database, but these "stalls" frequently last only a second or two. If you're not watching carefully, you'll miss them.
Why do you care about brief events like these? Because they can be a sign of trouble down the road. Small stalls may cripple your server when traffic increases. Because even these events can affect query performance. So it can impact customer experience, and every part of your backend systems as well. Cascading failures often start at the database level.
It's not necessary to fall victim to these issues. By monitoring your servers in this way, you can begin to stay ahead of some of these issues. But let's lets face it -- most problems are unexpected. When you do suffer database problems you can take the data that is gathered and immediately learn from them, you can find the solutions and fix them before you get affected by them again and again.
Rain Gauge provides a central collection point, a web interface for exploring collected data, and a set of best practices for running pt-stalk and collecting data.
What does it look like?
That's an example of exploring a sample. You can see all the different files you can select at the top, and some of them -- as in the mysqladmin sample shown -- are parsed to provide you with a neat view of the data within.
Again, like our query performance reporting tool Anemometer, this isn't anything revolutionary; I'm sure quite a few people have built similar in-house tools, and many people already use pt-stalk with the --daemonize option to continuously monitor their servers for trouble.
However, I'm betting many more DBAs out there *don't* do this yet. By releasing this, we hope to help standardize this process. We want to provide tools that we think are essential for supporting a high performance mysql database with lots of servers.
And in the best tradition of open source software, we're standing on the shoulders of giants here. Building Rain Gauge on top of the percona toolkit means this tool will benefit from the constant improvements an input of lots of admins and development that percona puts into their toolkit.
PlanetMySQL Voting: Vote UP / Vote DOWN