When deciding on your backup strategy, one of the key components for Point In Time Recovery (PITR) will be the binary logs. Thankfully, the mysqlbinlog command allows you to easily take binary log backups, including those that would otherwise be encrypted on disk using encrypt_binlog=ON.
When
mysqlbinlogis used with
--raw --read-from-remote-server --stop-never --verify-binlog-checksumthen it will retrieve binary logs from whichever master it is pointed to, and store them locally on disk in the same format as they were written on the master. Here is an example with the extra arguments that would normally be used:
/usr/bin/mysqlbinlog --raw --read-from-remote-server \ --stop-never --connection-server-id=1234 \ --verify-binlog-checksum \ --host=localhost --port=3306 mysql-bin.000024
This would retrieve the localhost binary logs (starting from mysql-bin.000024) reporting as server_id 1234, verify the checksum and then write each of them to disk.
Changes to the mysqlbinlog source code are relatively infrequent, except for when developing for a new major version, so you may be fooled into thinking that the specific version that you use is not so important—a little like the client version. This is something that is more likely to vary when you are taking remote backups.
Here is the result from the 5.7 branch of mysql-server to show the history of commits by year:
$ git blame --line-porcelain client/mysqlbinlog.cc | egrep "^(committer-time|committer-tz)" | cut -f2 -d' ' | while read ct; do read ctz; date --date "Jan 1, 1970 00:00:00 ${ctz} + ${ct} seconds" --utc +%Y; done | sort -n | uniq -c
105 2000
52 2001
52 2002
103 2003
390 2004
113 2005
58 2006
129 2007
595 2008
53 2009
349 2010
151 2011
382 2012
191 2013
308 2014
404 2015
27 2016
42 2017
15 2018
Since the first GA release of 5.7 (October 2015), there haven’t been too many bugs and so if you aren’t using new features then you may think that it is OK to keep using the same version as before:
$ git log --regexp-ignore-case --grep bug --since="2015-10-19" --oneline client/mysqlbinlog.cc 1ffd7965a5e Bug#27558169 BACKPORT TO 5.7 BUG #26826272: REMOVE GCC 8 WARNINGS [noclose] 17c92835bb3 Bug #24674276 MYSQLBINLOG -R --HEXDUMP CRASHES FOR INTVAR, USER_VAR, OR RAND EVENTS 052dbd7b079 BUG#26878022 MYSQLBINLOG: ASSERTION `(OLD_MH->M_KEY == KEY) || (OLD_MH->M_KEY == 0)' FAILED 543129a577c BUG#26878022 MYSQLBINLOG: ASSERTION `(OLD_MH->M_KEY == KEY) || (OLD_MH->M_KEY == 0)' FAILED ba1a99c5cd7 Bug#26825211 BACKPORT FIX FOR #25643811 TO 5.7 1f0f4476b28 Bug#26117735: MYSQLBINLOG READ-FROM-REMOTE-SERVER NOT HONORING REWRITE_DB FILTERING 12508f21b63 Bug #24323288: MAIN.MYSQLBINLOG_DEBUG FAILS WITH A LEAKSANITIZER ERROR e8e5ddbb707 Bug#24609402 MYSQLBINLOG --RAW DOES NOT GET ALL LATEST EVENTS 22eec68941f Bug#23540182:MYSQLBINLOG DOES NOT FREE THE EXISTING CONNECTION BEFORE OPENING NEW REMOTE ONE 567bb732bc0 Bug#22932576 MYSQL5.6 DOES NOT BUILD ON SOLARIS12 efc42d99469 Bug#22932576 MYSQL5.6 DOES NOT BUILD ON SOLARIS12 6772eb52d66 Bug#21697461 MEMORY LEAK IN MYSQLBINLOG
However, this is not always the case and some issues are more obvious than others! To help show this, here are a couple of the issues that you might happen to notice.
Warning: option ‘stop-never-slave-server-id’: unsigned value <xxxxxxxx> adjusted to <yyyyy>
The server_id that is used by a server in a replication topology should always be unique within the topology. One of the easy ways to ensure this is to use a conversion of the external IPv4 address to an integer, such as INET_ATON , which provides you with an unsigned integer.
The introduction of --connection-server-id
(which deprecates
--stop-never-slave-server-id) changes the behaviour here (for the better). Prior to this you may experience warnings where your server_id was cast to the equivalent of an UNSIGNED SMALLINT. This didn’t seem to be a reported bug, just fixed as a by-product of the change.
ERROR: Could not find server version: Master reported unrecognized MySQL version ‘xxx’
When running mysqlbinlog, the version of MySQL is checked so that the event format is set correctly. Here is the code from MySQL 5.7:
switch (*version) {
case '3':
glob_description_event= new Format_description_log_event(1);
break;
case '4':
glob_description_event= new Format_description_log_event(3);
break;
case '5':
/*
The server is soon going to send us its Format_description log
event, unless it is a 5.0 server with 3.23 or 4.0 binlogs.
So we first assume that this is 4.0 (which is enough to read the
Format_desc event if one comes).
*/
glob_description_event= new Format_description_log_event(3);
break;
default:
glob_description_event= NULL;
error("Could not find server version: "
"Master reported unrecognized MySQL version '%s'.", version);
goto err;
}
This section of the code last changed in 2008, but of course there is another vendor that no longer uses a 5-prefixed-version number: MariaDB. With MariaDB, it is impossible to take a backup without using a MariaDB version of the program, as you are told that the version is unrecognised. The MariaDB source code contains a change to this section to resolve the issue when the version was bumped to 10:
83c02f32375b client/mysqlbinlog.cc (Michael Widenius 2012-05-31 22:39:11 +0300 1900) case 5: 83c02f32375b client/mysqlbinlog.cc (Michael Widenius 2012-05-31 22:39:11 +0300 1901) case 10:
Interestingly, MySQL 8.0 gets a little closer to not producing an error (although it still does), but finally sees off those rather old ancestral relatives:
switch (*version) {
case '5':
case '8':
case '9':
/*
The server is soon going to send us its Format_description log
event.
*/
glob_description_event = new Format_description_log_event;
break;
default:
glob_description_event = NULL;
error(
"Could not find server version: "
"Master reported unrecognized MySQL version '%s'.",
version);
goto err;
}
These are somewhat trivial examples. In fact, you are more likely to suffer from more serious differences, perhaps ones that do not become immediately apparent, if you are not matching the mysqlbinlog version to the one provided by the version for the server producing the binary logs.
Sadly, it is not so easy to check the versions as the reported version was seemingly left unloved for quite a while (Ver 3.4), so you should check the binary package versions (e.g. using Percona-Server-client-57-5.7.23 with Percona-Server-server-57-5.7.23). Thankfully, the good news is that MySQL 8.0 fixes it!
So reduce the risk and match your package versions!
The post The Importance of mysqlbinlog –version appeared first on Percona Database Performance Blog.