MySQL doesn’t often crash, but, if you use MySQL on a production system you should have High Availability.
Maybe you’ve not heard of MHA. Yoshinori Matsunobu only released it in July of this year (2011). I’ve been reviewing it and I think you should too. There are a lot of people chasing after this “holy grail” and most systems are complex and / or hard to recover when they fail. MHA simple and easy to use.
MHA is a fail over tool. It’s designed to fail a master and promote a slave to a new master. (This is simply to say but you should read how it’s done.) It can monitor your master or you can manually fail over. Failing back doesn’t happen. Your expected to fix the failed master and turn it into a slave. The monitoring program (masterha_manager) will stop running after a fail over.
I’m not going to give details on how MHA works. I’ll assume you have a master and some number of slaves already working.
Download the latest version of MHA from the Google Code server. http://code.google.com/p/mysql-master-ha/downloads/list
Every MySQL server need to have a copy of the MHA node installed.
# yum install perl-DBD-MySQL.x86_64 # rpm -i mha4mysql-node-0.52-0.noarch.rpm
You can run the Manager on any server. It doesn’t have to be a MySQL server.
# yum install perl-Config-Tiny.noarch # yum install perl-Parallel-ForkManager.noarch # rpm -i mha4mysql-manager-0.52-0.noarch.rpm
Because I installed on a x64 version (as you should be) I found I had to copy the MHA libraries to the perl lib64 space. You might want to make simlinks instead.
# cp -crp /usr/lib/perl5/vendor_perl/MHA /usr/lib64/perl5/vendor_perl
You only need one configuration file if you are not going to monitor multiple Master/Slave clusters. You can name this configuration file anything you want. Change these setting and make sure the directories exist and are writable.
# vi /etc/MHA.cnf [server default] user=hauser password=P@ssw0rd manager_workdir=/var/log/masterha manager_log=/var/log/masterha/MHA.log remote_workdir=/var/log/masterha [server1] hostname=db1 [server2] hostname=db2
The MHA will need access to each server so you need to add the user to each.
mysql> grant all on *.* to 'hauser'@'192.168.0.%' identified by 'P@ssw0rd';
Now is a good time to make sure all the server names resolve on each system.
MHA also need ssh access to each server. You need to create ssh keys and copy to the mysql servers. Don’t do this if you already have ssh keys. If you have no keys this should work. Just press enter when you are ask for something.
# ssh-keygen -t dsa -f ~/.ssh/id_dsa -N "" # cp ~/.ssh/id_dsa.pub ~/.ssh/authorized_keys # scp -r ~/.ssh db1: # scp -r ~/.ssh db2:
You can check if you have ssl working with the masterha_check_ssh tool. Look for the OK after the connect test.
# masterha_check_ssh --conf=/etc/MHA.cnf
Thu Dec 1 10:10:52 2011 - [info] Starting SSH connection tests..
Thu Dec 1 10:10:53 2011 - [debug]
Thu Dec 1 10:10:52 2011 - [debug] Connecting via SSH from root@db1(192.168.0.11) to root@db2(192.168.2.12)..
Thu Dec 1 10:10:53 2011 - [debug] ok.
MHA need to control replication so you should test it with masterha_check_repl. This will outout a lot of chat. Your looking for the ‘MySQL Replication Health is OK’ at the end. You might also pay attention to any warnings.
# masterha_check_repl --conf=/etc/MHA.cnf
---
Thu Dec 1 10:07:22 2011 - [info] Checking slave configurations..
Thu Dec 1 10:07:22 2011 - [warning] read_only=1 is not set on slave db2(192.168.0.11:3306).
Thu Dec 1 10:07:22 2011 - [warning] relay_log_purge=0 is not set on slave db2(192.168.0.12:3306).
---
MySQL Replication Health is OK.
You now have MHA installed. If you configure it wrong MHA will give some errors that are not helpful and die. Check your settings again.
If you want MHA to monitor the status of your master server you need to run masterha_manager. It does not normally run as a daemon. If masterha_manage is not running your server will not automatically fail over. This is a weakness in MHA. You might want to use something like Pacemaker(Heartbeat) to monitor your systems and MHA to do the fail over. Here is how I make MHA run in the background.
# nohup masterha_manager --conf=/etc/MHA.cnf &;
You can check masterha_manager with masterha_check_status.
# masterha_check_status --conf=/etc/MHA.cnf
Now the magic. If you want to manual fail over you can use masterha_master_switch and tell it the master is alive.
# masterha_master_switch --master_state=alive --conf=/etc/MHA.cnf
If your using another monitoring system like Pacemaker you can have it run this command to automatically fail your master.
# masterha_master_switch --master_state=dead --dead_master_host=db1 --conf=/etc/MHA.cnf
If you find I’ve missed something or it works different on your OS please drop me a comment.
PlanetMySQL Voting: Vote UP / Vote DOWN