When the NDB version is upgraded it requires the underlying MySQL server as well is upgraded. Internal table storage format is different in MySQL 5.7 and for new tables created on MySQL 5.6. Row_format, temporal data type (time, datetime etc.) storage are handled differently in 5.7.
You can find the details on how MySQL handles temporal in these versions from the link below. The new format have time value in microseconds resolution, and also changes the way datetime is stored, which is more optimized
Here is a quick glance of how temporal are handled in different versions
5.5 – tables in old format
5.6 – Supports both old and new format tables. New tables created will be in new format. Alter existing tables if you want to bring them into new format
5.7 – All tables must be in new format.
The challenge !
The 5.7 mandates the existing tables in older format (created on 5.5 and before) to be promoted to new format, which requires an alter table before the upgrade or using the mysql_upgrade program right after the binary upgrade. This can be a very lengthy process for cluster with huge tables and sometimes the alter operation fails or stuck with no progress.
Explained below is a very efficient method for upgrading large databases using the ndb backup and restore programs , which is must faster than any other approach like using mysqldump restore, alter table, mysql_upgrade etc.
How we solved it !
- Create a new MySQL Cluster 7.6 and start it with initial. Which will creates the data nodes empty
- Create ndb backup of 7.4 cluster
- Create a schema backup of MySQL cluster 7.4 using mysqldump (no-data option)
- Restore the metadata alone to new MySQL 7.6 cluster from the ndb backup created on step 2
- Restore the schema dump to 7.6 cluster which will just drop and re-create the tables , but in new format. Note that this is just structure but no data
- Restore the MySQL cluster 7.4 backup DATA ONLY on the new MySQL cluster 7.6 using the 7.6 version of NDB restore with --promote-attributes option
- Restoring the 7.4 backup to 7.6 will create all MySQL objects under the newer version 5.7
- Do a rolling restart of all API and data nodes
- That’s it ! All set and you are good to go 😀