In this 101 article, I will show how to drop/remove a column from a table in MySQL.

In this article I will use an example table:

CREATE TABLE tb(
c1 INT PRIMARY KEY,
c2 char(1),
c3 varchar(2)
) ENGINE=InnoDB;

To remove a column we will make use of ALTER TABLE command:

ALTER TABLE tb DROP COLUMN c2;

The command allows you to remove multiple columns at once:

ALTER TABLE tb DROP COLUMN c2, DROP COLUMN c3;

If you are running MySQL 5.6 onwards, you can make this operation online, allowing other sessions to read and write to your table while the operation is been performed:

ALTER TABLE tb DROP COLUMN c2, ALGORITHM=INPLACE, LOCK=NONE;

Reference:

https://dev.mysql.com/doc/refman/5.7/en/alter-table.html
https://dev.mysql.com/doc/refman/5.7/en/innodb-create-index-overview.html

In this post, I will cover a bit of ProxySQL. ProxySQL is a proxy which implements MySQL protocol, allowing it to do things that other proxies cannot do. ProxySQL is gaining a lot of traction nowadays and it’s capable to integrate with a variety of products from MySQL ecosystems, such as replication (master – slave / master – master), Percona XtraDB Cluster and Group Replication.
One of its many functionalities (which IMHO makes it awesome) is the ability to do read/write split seamless to the application. You can start sending reads to your slave without doing a single line of code.

In this article, I will cover a few important points to understand how it works.

Instalation

The instalation is easy, you can do it by downloading the corresponding package for your OS from its official github repo https://github.com/sysown/proxysql/releases

sudo yum install https://github.com/sysown/proxysql/releases/download/v1.4.9/proxysql-1.4.9-1-centos7.x86_64.rpm

Now we just need to start the service

sudo service proxysql start 

Interfaces

ProxySQL splits application interface from the admin interface. It will listen on 2 network ports. Admin will be on 6032 and application will listen on 6033 (reverse of 3306 ).

Layers

Other important part to understand how the proxy works is to understand its layers. I am gonna show you a diagram that can be found on its official documentation:

+-------------------------+
|      1. RUNTIME         |   
+-------------------------+
       /|\          |
        |           |
        |           |
        |          \|/
+-------------------------+
|     2.  MEMORY          |   
+-------------------------+ 
       /|\          |      
        |           |      
        |           |       
        |          \|/      
+-------------------------+  
|     3.   DISK           |  
+-------------------------+  

ProxySQL will always read information from 1.Runtime layer, which is stored in memory.
Every time we connect to the admin port (6032) we are manipulating information from layer 2.Memory. As the name infers, its also stored in memory.
We then have the layer 3.Disk. As the other two layers are stored in memory, we need a layer to persist information across service/server restarts.

What is the benefic of this layout?
It allows us to manipulate different areas and apply the changes at once. We can think about how a transaction works, where we run multiple queries and commit them at once. When we alter something, we will be manipulating the Memory layer, then we will run a command LOAD MYSQL [SERVERS | USERS | QUERY RULES] TO RUNTIME to load this information to runtime and we will save the information to disk layer by issuing SAVE MYSQL [SERVERS | USERS | QUERY RULES] TO DISK.

Hostgroups

ProxySQL group servers in something named hostgroup. In a topology which we have a master and two slaves, we will create a hostgroup(HG) 1 and specify that our master is part of that HG and we will create a HG 2 and specify that both slaves belong to that HG. Hostgroup creation is done at the time we specify servers on mysql_servers table. There is not fixed enumeration, you can create your HG with any ID you want.

Later we will configure user and queries to identify if the query coming in is a read it should be answered by one of the servers from HG 2 (where we configured our slaves). If the query is not a read, then our server on HG 1 will receive it.

Authentication

ProxySQL has functionalities like firewall, in which it has the capability of blocking a query even before it reaches our backend server. To do it it’s required to have the user authentication module also present on the proxy side. So we will be required to create all users that we wish to connect via ProxySQL also create on the proxy side.

Now that you understand a bit of the basic of how ProxySQL works, you can start playing with it.

In this 101 article, I will show how to drop/remove a column from a table in MySQL.

In this article I will use an example table:

CREATE TABLE tb(
c1 INT PRIMARY KEY,
c2 char(1),
c3 varchar(2)
) ENGINE=InnoDB;

To remove a column we will make use of ALTER TABLE command:

ALTER TABLE tb DROP COLUMN c2;

The command allows you to remove multiple columns at once:

ALTER TABLE tb DROP COLUMN c2, DROP COLUMN c3;

If you are running MySQL 5.6 onwards, you can make this operation online, allowing other sessions to read and write to your table while the operation is been performed:

ALTER TABLE tb DROP COLUMN c2, ALGORITHM=INPLACE, LOCK=NONE;

Reference:

https://dev.mysql.com/doc/refman/5.7/en/alter-table.html
https://dev.mysql.com/doc/refman/5.7/en/innodb-create-index-overview.html

In this post, I will cover a bit of ProxySQL. ProxySQL is a proxy which implements MySQL protocol, allowing it to do things that other proxies cannot do. ProxySQL is gaining a lot of traction nowadays and it’s capable to integrate with a variety of products from MySQL ecosystems, such as replication (master – slave / master – master), Percona XtraDB Cluster and Group Replication.
One of its many functionalities (which IMHO makes it awesome) is the ability to do read/write split seamless to the application. You can start sending reads to your slave without doing a single line of code.

In this article, I will cover a few important points to understand how it works.

Instalation

The instalation is easy, you can do it by downloading the corresponding package for your OS from its official github repo https://github.com/sysown/proxysql/releases

sudo yum install https://github.com/sysown/proxysql/releases/download/v1.4.9/proxysql-1.4.9-1-centos7.x86_64.rpm

Now we just need to start the service

sudo service proxysql start 

Interfaces

ProxySQL splits application interface from the admin interface. It will listen on 2 network ports. Admin will be on 6032 and application will listen on 6033 (reverse of 3306 ).

Layers

Other important part to understand how the proxy works is to understand its layers. I am gonna show you a diagram that can be found on its official documentation:

+-------------------------+
|      1. RUNTIME         |   
+-------------------------+
       /|\          |
        |           |
        |           |
        |          \|/
+-------------------------+
|     2.  MEMORY          |   
+-------------------------+ 
       /|\          |      
        |           |      
        |           |       
        |          \|/      
+-------------------------+  
|     3.   DISK           |  
+-------------------------+  

ProxySQL will always read information from 1.Runtime layer, which is stored in memory.
Every time we connect to the admin port (6032) we are manipulating information from layer 2.Memory. As the name infers, its also stored in memory.
We then have the layer 3.Disk. As the other two layers are stored in memory, we need a layer to persist information across service/server restarts.

What is the benefic of this layout?
It allows us to manipulate different areas and apply the changes at once. We can think about how a transaction works, where we run multiple queries and commit them at once. When we alter something, we will be manipulating the Memory layer, then we will run a command LOAD MYSQL [SERVERS | USERS | QUERY RULES] TO RUNTIME to load this information to runtime and we will save the information to disk layer by issuing SAVE MYSQL [SERVERS | USERS | QUERY RULES] TO DISK.

Hostgroups

ProxySQL group servers in something named hostgroup. In a topology which we have a master and two slaves, we will create a hostgroup(HG) 1 and specify that our master is part of that HG and we will create a HG 2 and specify that both slaves belong to that HG. Hostgroup creation is done at the time we specify servers on mysql_servers table. There is not fixed enumeration, you can create your HG with any ID you want.

Later we will configure user and queries to identify if the query coming in is a read it should be answered by one of the servers from HG 2 (where we configured our slaves). If the query is not a read, then our server on HG 1 will receive it.

Authentication

ProxySQL has functionalities like firewall, in which it has the capability of blocking a query even before it reaches our backend server. To do it it’s required to have the user authentication module also present on the proxy side. So we will be required to create all users that we wish to connect via ProxySQL also create on the proxy side.

Now that you understand a bit of the basic of how ProxySQL works, you can start playing with it.

The MySQL error log is an essential part of database server performance monitoring. Whenever something goes wrong or performance degrades, the Error Logs are usually the first place we look to start troubleshooting.

The MySQL Error Log is one of three related log types:

• The Error Log: It contains information about errors that occur while the server is running (as well as server start and stop events).
• The General Query Log: This is a general record of what mysqld is doing (connect, disconnect, queries).
• The Slow Query Log: It consists of “slow” SQL statements as defined in the long_query_time global variable.

You can enable error log monitoring to allow Monyog to keep an eye on your MySQL Error Log, and notify you when something goes awry. Moreover, Monyog combines the General Query, Slow Query and Error logs in a single view for both network and cloud servers. For example, in the case of Amazon RDS, Monyog utilizes the Amazon RDS Application Programming Interface (API).

In the Error Logging in MySQL 8 blog article, we were introduced to MySQL’s new component-based architecture. This new architectural model is more modular because components may only interact with each other through the services they provide. Services provided by components are available to the server as well as to other components.

These changes have added to error logging flexibility. Case in point, log messages may now be output with a particular format and output either to a file or the system log.

The improved flexibility correlates to an equal increase configuration complexity as a log component can be a filter or a sink:

• A filter processes log events, to add, remove, or modify individual event fields, or to delete events entirely.
• A sink is a destination (writer) for log events. Typically, a sink processes log events into log messages that have a particular format and writes these messages to its associated output, such as a file or the system log.

For users of Monyog, these complexities are a non-issue, as Monyog makes error log monitoring a straight-forward process of entering log details on the Settings screen and then viewing output on the Monitors screen.

This blog will describe how to configure your MySQL/MariaDB Error Log settings, view output, and glean useful insights from error messages.

What Goes in the Error Log?

The MySQL Error Log may contain messages of varying severities. It collects messages that require persistence beyond that of a single connection or the server’s runtime, that have an intended audience of the DBA or system administrator, as opposed to the application developer. Though the latter may sometimes find the Error Log contents instructive as well. Examples of Error Log entries include exhaustion of resources on the host machine, certain errors reported by the host operating system, stack traces of crashes, messages about damaged databases and indexes, errors on start-up, failed or refused connections, etc.

The error log contains a record of mysqld startup and shutdown times. It also contains diagnostic messages such as errors, warnings, and notes that occur during server startup and shutdown, and while the server is running. For example, if mysqld comes across a table that needs to be checked or repaired, it writes a message to the error log.

On some operating systems, the error log contains a stack trace if mysqld exits abnormally. The trace can be used to determine where mysqld exited.

Configuring Error Log Settings

Monitoring the error log is crucial as changes to the log are indicative of disastrous outages. Monyog makes the task of monitoring the error log very simple. All one needs to do is configure Monyog, and it will take care of the rest! Monyog will alert you of changes in the error log, and in addition, if there is an entry of type [ERROR] in the log, Monyog will extract the corresponding message and send an email it to designated persons.

The Error Log is disabled in MySQL and MariaDB by default, so we have to enable it before Monyog can read and analyze it. To do that:

1. Navigate to the SERVERS screen and click on the ellipsis […] to open the context menu (#1 in the image below).
2. Choose the Edit Server item from the context menu (#2 in the image below).
3. On the Server Settings dialog (#3 in the image below):
   1. 1. Click on the ADVANCED header to bring up the advanced server settings (#3A in the image below).
      2. Select MySQL Server Log from the tree on the left-hand side of the dialog (#3B in the image below).

To configure error log monitoring:

1. To Click the Enable error log monitoring toggle switch so that the slider moves to the right. The background color will also go from grey to blue:
2. You can either enter the error log FILE PATH manually, or you let Monyog get the path of the error log from the MySQL server for you. Just click on the Fetch button to the right of the FILE PATH text field, and Monyog will locate the file!
3. The READ FILE FROM field sets how Monyog will access the Error Log File. There are 3 ways of accessing the log files:
   1. Select Local path if the Error Log File resides on the same machine that Monyog is running on or on a shared network drive.
   2. Choose Via SFTP if you have configured Monyog to use SSH. You’ll then have to supply your SSH details.
   3. Select RDS/Aurora (Using API) if your server is an RDS/Aurora instance. For file-based logging, you’ll have to fill four additional fields, as follows:
      • DB instance identifier: A unique name to identify your RDS/Aurora instance.
      • Instance region: The region in which your instance is hosted, e.g: “us-east-1”
      • Access key ID: It is a 20 character long key ID which can be created from the AWS Management console. It is used to make programmatic requests to AWS.
      • Secret access key: It is 40 characters long and can be created from the AWS Management console.
4. Click the TEST READING THE FILE button to check if Monyog can access the file specified in the File path. A message will appear in the bottom right corner of the browser window:
5. You can APPLY THE SETTING TO Only this server or All servers with tags same as this server.

Click the SAVE button to save your Server Settings and close the dialog.

Viewing the Error Log File

MySQL Log data is combined on the MySQL Logs screen. Like all monitors, it can show log information for multiple servers. The MySQL Logs screen is accessible via the Monitors icon (on the left-hand toolbar) > MySQL Logs (in the MONITOR GROUP list).

The MySQL Log screen displays multiple servers side-by-side:

This MySQL Log screen shows the local Sakila server:

The MySQL Error Log occupies the first three rows of the MONITORS table:

1. MONyog able to read MySQL error log?: A Yes/No value indicating whether Monyog is able to access the MySQL error log. If No is indicated, go back to the Error Log Settings screen and make sure that Error Log monitoring is enabled and that Monyog can read the file.
2. New entries in error log?: This Yes/No value indicates whether the error log has changed during the selected TIMEFRAME.
3. Recent entries of type [ERROR]: MySQL logs three kinds of Log entries – Notes, Warnings, and Errors. Any log entries recorded with an [ERROR] type are displayed here.

Getting the Most out of the MySQL Log Screen

The MySQL Log Screen does a lot more than display server errors. In addition to alerting you of changes in the error log, Monyog provides useful information about each monitor, as well as historical data. It can even extract the message from [ERROR] entries and send them to you via email.

Monitor Editing

Monitor names are actually clickable links that open the associated EDIT MONITOR screen at the right of the screen. It’s a screen where you can view and modify monitor parameters.

Here’s a description of the form fields. Mandatory fields are marked with an asterisk (*):

• Name*: The name of the monitor.
• Monitor Group*: The monitor group that this monitor belongs to.
• Type of counter: Choices include MySQL, System, or Custom SQL.
• MySQL or System indicate that this Monitor displays MySQL or system-related information respectively, while Custom SQL indicates that this Monitor is based on a Custom SQL Object. Only MySQL is applicable to existing Error Log monitors.
• Enabled?: A toggle switch indicating whether or not the monitor is enabled. Moving the slider to the right enables the monitor while moving it to the left turns the monitor off. The background color will also go from grey to blue when the monitor is enabled.
• Formula: A MySQL server parameter on which the value of this counter is based. None are applicable to the Error Log, so this field should normally be left blank.
• Value: This specifies a JavaScript function that computes the value. For example, this code fetches the last Error Log entry:
• function x()
  {
     val = MONyog.MySQL.ErrorLog.Last_error;
     if(typeof val == “undefined” || val == 0 || val == “”)
         return “(n/a)”;
     else
         return val;        
  }
• Description: A detailed summary of the monitor.
• Advice text: Provides additional information about the monitor, such as how to interpret its output as well as how to correct potential problems.

Setting Alerts

Monyog tells you that a monitor reading could be pointing to a potential issue to investigate by placing a Red or Yellow alert next to it. For example, here is a Critical alert next to a Log Error entry:

Alert settings are configured on the Alerts tab of the EDIT MONITOR screen.

The Email Recipients textbox accepts a comma-delimited list of email addresses to send notifications to when an alert is triggered. Clicking the Advanced link divides recipients into Critical and Warning lists.

You can turn Notifications on and off via the Notifications slider control. Note that you also have to configure SMTP or(and) SNMP for MONyog and Notification Settings for servers for this feature to work.

You can also override the notification settings via the Override notify when stable? and Override notify till stable? fields. Each includes radio buttons for “Yes”, “No”, and “Disable”. If set to “Disable”, this variable will not override the server-level setting.

When notifications are enabled for a monitor, the Notifications enabled icon appears beside the monitor name:

Viewing Historical Trend Values

The term “Trend Analysis” refers to the concept of collecting information in an effort to spot a pattern, or trend, in the data. With respect to database administration, analyzing history reports gives a way to track trends and identify problem areas in your infrastructure. The idea is that, by using historical information to find recurring problems, you may be able to prevent future issues.

Each server column cell has a History trend value icon on the right-hand side. Clicking it displays historical trend data for the selected TIMEFRAME.

Clicking the SHOW ONLY CHANGED VALUES slider toggles between all log entries and only those where the value has changed.

Conclusion

An essential part of database server performance monitoring, the MySQL error log is usually the first place we look to start troubleshooting database issues. In today’s blog, we learned how Monyog facilitates error log monitoring on MySQL and MariaDB servers by combining all of the log output together on one screen. Moreover, Monyog offers additional value by providing features such as Alerts, Notifications, and History Trend Values.

The post Analyze MySQL & MariaDB Error Log Messages using Monyog appeared first on Monyog Blog.

We are all familiar with “.frm” files since the earliest days of MySQL, The community has been continuously requesting for replacement of file-system based metadata for several good reasons, So with MySQL 8.0 “.frm” files are gone for ever, Going forward MySQL stores table metadata in the data dictionary tables which uses InnoDB storage engine. This blog is about MySQL 8.0 data dictionary and how it creates value for MySQL going forward:

How file based metadata management used to work in the past (before MySQL 8.0) ? 

• Every table in MySQL will have corresponding .frm file, This .frm file stores information like column names and data-types in the binary format, In addition to the .frm file, there are .trn, .trg and .par files to support triggers, trigger namespace and partitioning .

What are major bottlenecks faced due to the usage of file based metadata management ? 

• Operationally it always appeared very irrational, Why we need to have an separate mechanism to track the schema information ? Originally this was the idea from Drizzle –  Drizzle made it very clear (almost ) that it should get out of the way and let the storage engines be the storage engines and not try to second guess them or keep track of things behind their back.
• Dictionaries out of synch.– Before MySQL 8.0, the data dictionary is a  “split brain”, where  the “server” and InnoDB have their own separate data dictionary, where some information duplicated. Information that is duplicated in the MySQL server dictionary and the InnoDB dictionary might get out of synch, and we need one common “source of truth”  for dictionary information.
• INFORMATION_SCHEMA is the bottleneck– The main reason behind these performance issues in the INFORMATION_SCHEMA (before MySQL 8.0) implementation is that INFORMATION_SCHEMA tables are implemented as temporary tables that are created on-the-fly during query execution. For a MySQL server having hundreds of databases, each with hundreds of tables within them, the INFORMATION_SCHEMA query would end-up doing lot of I/O reading each individual FRM files from the file system. And it would also end-up using more CPU cycles in effort to open the table and prepare related in-memory data structures. It does attempt to use the MySQL server table cache (the system variable ‘table_definition_cache‘), however in large server instances it’s very rare to have a table cache that is large enough to accommodate all of these tables.
• No atomic DDL– Storing the data dictionary in non-transactional tables and files, means that DDLs are unsafe for replication (they are not transactional, not even atomic). If a compound DDL fails we still need to replicate it and hope that it fails with the same error. This is a best effort approach and there is a lot of logic coded to handle this . It is hard to maintain, slows down progress and bloats the replication codebase. The data dictionary is stored partly in non-transactional tables. These are not safe for replication building resilient HA systems on top of MySQL. For instance, some dictionary tables need to be manipulated using regular DML, which causes problems for GTIDs.
• Crash recovery. Since the DDL statements are not atomic, it is challenging to recover after crashing in the middle of a DDL execution, and is especially problematic for replication.

How things are changed with MySQL 8.0  ? 

MySQL 8.0 introduced a native data dictionary based on InnoDB.  This change has enabled us to get rid of file-based metadata store (FRM files) and also help MySQL to move towards supporting transactional DDL. We have now the metadata of all database tables stored in transactional data dictionary tables, it enables us to design an INFORMATION_SCHEMA table as a database VIEW over the data dictionary tables. This eliminates costs such as the creation of temporary tables for each INFORMATION_SCHEMA query during execution on-the-fly, and also scanning file-system directories to find FRM files. It is also now possible to utilize the full power of the MySQL optimizer to prepare better query execution plans using indexes on data dictionary tables.

The following diagram (Source: MySQL server team blog) explains the difference in design in MySQL 5.7 and 8.0 :

The post MySQL 8.0 Data Dictionary appeared first on MySQL Consulting, Support and Remote DBA Services By MinervaDB.

ProxySQL 1.4.2 introduced native clustering, allowing several ProxySQL instances to communicate with and share configuration updates with each other. In this blog post, I’ll review this new feature and how we can start working with 3 nodes.

Before I continue, let’s review two common methods to installing ProxySQL.

ProxySQL as a centralized server

This is the most common installation, where ProxySQL is between application servers and the database. It is simple, but without any high availability. If ProxySQL goes down you lose all connectivity to the database.

ProxySQL on app instances

Another common setup is to install ProxySQL onto each application server. This is good because the loss of one ProxySQL/App server will not bring down the entire application.

For more information about the previous installation, please visit this link Where Do I Put ProxySQL?

Sometimes our application and databases grow fast. Maybe you need add a loadbalancer, for example, and in that moment you start thinking … “What could I do to configure and maintain all these ProxySQL nodes without mistakes?”

To do that, there are many tools like Ansible, Puppet, and Chef, but you will need write/create/maintain scripts to do those tasks. This is really difficult to administer for one person.

Now, there is a native solution, built into ProxySQL, to create and administer a cluster in an easy way.

At the moment this feature is EXPERIMENTAL and subject to change. Think very carefully before installing it in production, in fact I strongly recommend you wait. However, if you would like to start testing this feature, you need to install ProxySQL 1.4.2, or better.

This clustering feature is really useful if you have installed one ProxySQL per application instance, because all the changes in one of the ProxySQL nodes will be propagated to all the other ProxySQL nodes. You can also configure a “master-slave” style setup with ProxySQL clustering.

There are only 4 tables where you can make changes and propagate the configuration:

• mysql_query_rules
• mysql_servers
• mysql_users
• proxysql_servers

How does it work?

It’s easy. When you make a change like INSERT/DELETE/UPDATE on any of these tables, after running the command

LOAD … TO RUNTIME

admin ((none))>SELECT name, version, FROM_UNIXTIME(epoch), checksum FROM runtime_checksums_values ORDER BY name;
+-------------------+---------+----------------------+--------------------+
| name              | version | FROM_UNIXTIME(epoch) | checksum           |
+-------------------+---------+----------------------+--------------------+
| admin_variables   | 0       | 1970-01-01 00:00:00  |                    |
| mysql_query_rules | 1       | 2018-04-26 15:58:23  | 0x0000000000000000 |
| mysql_servers     | 1       | 2018-04-26 15:58:23  | 0x0000000000000000 |
| mysql_users       | 4       | 2018-04-26 18:36:12  | 0x2F35CAB62143AE41 |
| mysql_variables   | 0       | 1970-01-01 00:00:00  |                    |
| proxysql_servers  | 1       | 2018-04-26 15:58:23  | 0x0000000000000000 |
+-------------------+---------+----------------------+--------------------+

Internally, all nodes are monitoring and communicating with all the other ProxySQL nodes. When another node detects a change in the checksum and version (both at the same time), each node will get a copy of the table that was modified, make the same changes locally, and apply the new config to RUNTIME to refresh the new config, make it visible to the applications connected and automatically save it to DISK for persistence.

The following setup creates a “synchronous cluster” so any changes to these 4 tables on any ProxySQL server will be replicated to all other ProxySQL nodes. Be careful!

How can I start testing this new feature?

1) To start we need to get at least 2 nodes. Download and install ProxySQL 1.4.2 or higher and start a clean version.

2) On all nodes, we need to update the following global variables. These changes will set the username and password used by each node’s internal communication to cluster1/clusterpass. These must be the same on all nodes in this cluster.

update global_variables set variable_value='admin:admin;cluster1:clusterpass' where variable_name='admin-admin_credentials';
update global_variables set variable_value='cluster1' where variable_name='admin-cluster_username';
update global_variables set variable_value='clusterpass' where variable_name='admin-cluster_password';
update global_variables set variable_value=200 where variable_name='admin-cluster_check_interval_ms';
update global_variables set variable_value=100 where variable_name='admin-cluster_check_status_frequency';
update global_variables set variable_value='true' where variable_name='admin-cluster_mysql_query_rules_save_to_disk';
update global_variables set variable_value='true' where variable_name='admin-cluster_mysql_servers_save_to_disk';
update global_variables set variable_value='true' where variable_name='admin-cluster_mysql_users_save_to_disk';
update global_variables set variable_value='true' where variable_name='admin-cluster_proxysql_servers_save_to_disk';
update global_variables set variable_value=3 where variable_name='admin-cluster_mysql_query_rules_diffs_before_sync';
update global_variables set variable_value=3 where variable_name='admin-cluster_mysql_servers_diffs_before_sync';
update global_variables set variable_value=3 where variable_name='admin-cluster_mysql_users_diffs_before_sync';
update global_variables set variable_value=3 where variable_name='admin-cluster_proxysql_servers_diffs_before_sync';
load admin variables to RUNTIME;
save admin variables to disk;

3) Add all IPs from the other ProxySQL nodes into each other node:

INSERT INTO proxysql_servers (hostname,port,weight,comment) VALUES ('10.138.180.183',6032,100,'PRIMARY');
INSERT INTO proxysql_servers (hostname,port,weight,comment) VALUES ('10.138.244.108',6032,99,'SECONDARY');
INSERT INTO proxysql_servers (hostname,port,weight,comment) VALUES ('10.138.244.244',6032,98,'SECONDARY');
LOAD PROXYSQL SERVERS TO RUNTIME;
SAVE PROXYSQL SERVERS TO DISK;

At this moment, we have all nodes synced.

In the next example from the log file, we can see when node1 detected node2.

[root@proxysql1 ~]# $ tail /var/lib/proxysql/proxysql.log
...
2018-05-10 11:19:51 [INFO] Cluster: Fetching ProxySQL Servers from peer 10.138.244.108:6032 started
2018-05-10 11:19:51 [INFO] Cluster: Fetching ProxySQL Servers from peer 10.138.244.108:6032 completed
2018-05-10 11:19:51 [INFO] Cluster: Loading to runtime ProxySQL Servers from peer 10.138.244.108:6032
2018-05-10 11:19:51 [INFO] Destroyed Cluster Node Entry for host 10.138.148.242:6032
2018-05-10 11:19:51 [INFO] Cluster: Saving to disk ProxySQL Servers from peer 10.138.244.108:6032
2018-05-10 11:19:52 [INFO] Cluster: detected a new checksum for proxysql_servers from peer 10.138.180.183:6032, version 6, epoch 1525951191, checksum 0x3D819A34C06EF4EA . Not syncing yet ...
2018-05-10 11:19:52 [INFO] Cluster: checksum for proxysql_servers from peer 10.138.180.183:6032 matches with local checksum 0x3D819A34C06EF4EA , we won't sync.
2018-05-10 11:19:52 [INFO] Cluster: closing thread for peer 10.138.148.242:6032
2018-05-10 11:19:52 [INFO] Cluster: detected a new checksum for proxysql_servers from peer 10.138.244.244:6032, version 4, epoch 1525951163, checksum 0x3D819A34C06EF4EA . Not syncing yet ...
2018-05-10 11:19:52 [INFO] Cluster: checksum for proxysql_servers from peer 10.138.244.244:6032 matches with local checksum 0x3D819A34C06EF4EA , we won't sync
...

Another example is to add users to the table mysql_users. Remember these users are to enable MySQL connections between the application (frontend) and MySQL (backend).

We will add a new username and password on any server; in my test I’ll use node2:

admin proxysql2 ((none))>INSERT INTO mysql_users(username,password) VALUES ('user1','crazyPassword');
Query OK, 1 row affected (0.00 sec)
admin proxysql2 ((none))>LOAD MYSQL USERS TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

In the log file from node3, we can see the update immediately:

[root@proxysql3 ~]# $ tail /var/lib/proxysql/proxysql.log
...
2018-05-10 11:30:57 [INFO] Cluster: detected a new checksum for mysql_users from peer 10.138.244.108:6032, version 2, epoch 1525951873, checksum 0x2AF43564C9985EC7 . Not syncing yet ...
2018-05-10 11:30:57 [INFO] Cluster: detected a peer 10.138.244.108:6032 with mysql_users version 2, epoch 1525951873, diff_check 3. Own version: 1, epoch: 1525950968. Proceeding with remote sync
2018-05-10 11:30:57 [INFO] Cluster: detected a peer 10.138.244.108:6032 with mysql_users version 2, epoch 1525951873, diff_check 4. Own version: 1, epoch: 1525950968. Proceeding with remote sync
2018-05-10 11:30:57 [INFO] Cluster: detected peer 10.138.244.108:6032 with mysql_users version 2, epoch 1525951873
2018-05-10 11:30:57 [INFO] Cluster: Fetching MySQL Users from peer 10.138.244.108:6032 started
2018-05-10 11:30:57 [INFO] Cluster: Fetching MySQL Users from peer 10.138.244.108:6032 completed
2018-05-10 11:30:57 [INFO] Cluster: Loading to runtime MySQL Users from peer 10.138.244.108:6032
2018-05-10 11:30:57 [INFO] Cluster: Saving to disk MySQL Query Rules from peer 10.138.244.108:6032
2018-05-10 11:30:57 [INFO] Cluster: detected a new checksum for mysql_users from peer 10.138.244.244:6032, version 2, epoch 1525951857, checksum 0x2AF43564C9985EC7 . Not syncing yet ...
2018-05-10 11:30:57 [INFO] Cluster: checksum for mysql_users from peer 10.138.244.244:6032 matches with local checksum 0x2AF43564C9985EC7 , we won't sync.
2018-05-10 11:30:57 [INFO] Cluster: detected a new checksum for mysql_users from peer 10.138.180.183:6032, version 2, epoch 1525951886, checksum 0x2AF43564C9985EC7 . Not syncing yet ...
2018-05-10 11:30:57 [INFO] Cluster: checksum for mysql_users from peer 10.138.180.183:6032 matches with local checksum 0x2AF43564C9985EC7 , we won't sync.
...

What happens if some node is down?

In this example, we will see and find out what happens if one node is down or has a network glitch, or other issue. I’ll stop ProxySQL node3:

[root@proxysql3 ~]# service proxysql stop
Shutting down ProxySQL: DONE!

On ProxySQL node1, we can check that node3 is unreachable:

[root@proxysql1 ~]# tailf /var/lib/proxysql/proxysql.log
2018-05-10 11:57:33 ProxySQL_Cluster.cpp:180:ProxySQL_Cluster_Monitor_thread(): [WARNING] Cluster: unable to connect to peer 10.138.244.244:6032 . Error: Can't connect to MySQL server on '10.138.244.244' (107)
2018-05-10 11:57:33 ProxySQL_Cluster.cpp:180:ProxySQL_Cluster_Monitor_thread(): [WARNING] Cluster: unable to connect to peer 10.138.244.244:6032 . Error: Can't connect to MySQL server on '10.138.244.244' (107)
2018-05-10 11:57:33 ProxySQL_Cluster.cpp:180:ProxySQL_Cluster_Monitor_thread(): [WARNING] Cluster: unable to connect to peer 10.138.244.244:6032 . Error: Can't connect to MySQL server on '10.138.244.244' (107)

And another check can be run in any ProxySQL node like node2, for example:

admin proxysql2 ((none))>SELECT hostname, checksum, FROM_UNIXTIME(changed_at) changed_at, FROM_UNIXTIME(updated_at) updated_at FROM stats_proxysql_servers_checksums WHERE name='proxysql_servers' ORDER BY hostname;
+----------------+--------------------+---------------------+---------------------+
| hostname       | checksum           | changed_at          | updated_at          |
+----------------+--------------------+---------------------+---------------------+
| 10.138.180.183 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:39 | 2018-05-10 12:01:59 |
| 10.138.244.108 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:38 | 2018-05-10 12:01:59 |
| 10.138.244.244 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:39 | 2018-05-10 11:56:59 |
+----------------+--------------------+---------------------+---------------------+
3 rows in set (0.00 sec)

In the previous result, we can see node3 (10.138.244.244) is not being updated; the column updated_at should have a later datetime. This means that node3 is not running (or is down or network glitch).

At this point, any change to any of the tables, mysql_query_rules, mysql_servers, mysql_users, proxysql_servers, will be replicated between nodes 1 & 2.

In this next example, while node3 is offline, we will add another user to mysql_users table.

admin proxysql2 ((none))>INSERT INTO mysql_users(username,password) VALUES ('user2','passwordCrazy');
Query OK, 1 row affected (0.00 sec)
admin proxysql2 ((none))>LOAD MYSQL USERS TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

That change was propagated to node1:

[root@proxysql3 ~]# $ tail /var/lib/proxysql/proxysql.log
...
2018-05-10 12:12:36 [INFO] Cluster: detected a peer 10.138.244.108:6032 with mysql_users version 3, epoch 1525954343, diff_check 4. Own version: 2, epoch: 1525951886. Proceeding with remote sync
2018-05-10 12:12:36 [INFO] Cluster: detected peer 10.138.244.108:6032 with mysql_users version 3, epoch 1525954343
2018-05-10 12:12:36 [INFO] Cluster: Fetching MySQL Users from peer 10.138.244.108:6032 started
2018-05-10 12:12:36 [INFO] Cluster: Fetching MySQL Users from peer 10.138.244.108:6032 completed
2018-05-10 12:12:36 [INFO] Cluster: Loading to runtime MySQL Users from peer 10.138.244.108:6032
2018-05-10 12:12:36 [INFO] Cluster: Saving to disk MySQL Query Rules from peer 10.138.244.108:6032
...

We keep seeing node3 is out of sync about 25 minutes ago.

admin proxysql2 ((none))>SELECT hostname, checksum, FROM_UNIXTIME(changed_at) changed_at, FROM_UNIXTIME(updated_at) updated_at FROM stats_proxysql_servers_checksums WHERE name='mysql_users' ORDER BY hostname;
+----------------+--------------------+---------------------+---------------------+
|      hostname  |     checksum       |     changed_at      |      updated_at     |
+----------------+--------------------+---------------------+---------------------+
| 10.138.180.183 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:39 | 2018-05-10 12:21:35 |
| 10.138.244.108 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:38 |2018-05-10 12:21:35  |
| 10.138.244.244 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:39 |2018-05-10 12:21:35  |
+----------------+--------------------+---------------------+---------------------+
3 rows in set (0.00 sec)

Let’s start node3 and check if the sync works. node3 should connect to the other nodes and get the last changes.

[root@proxysql3 ~]# tail /var/lib/proxysql/proxysql.log
...
2018-05-10 12:30:02 [INFO] Cluster: detected a peer 10.138.244.108:6032 with mysql_users version 3, epoch 1525954343, diff_check 3. Own version: 1, epoch: 1525955402. Proceeding with remote sync
2018-05-10 12:30:02 [INFO] Cluster: detected a peer 10.138.180.183:6032 with mysql_users version 3, epoch 1525954356, diff_check 3. Own version: 1, epoch: 1525955402. Proceeding with remote sync
…
2018-05-10 12:30:03 [INFO] Cluster: detected peer 10.138.180.183:6032 with mysql_users version 3, epoch 1525954356
2018-05-10 12:30:03 [INFO] Cluster: Fetching MySQL Users from peer 10.138.180.183:6032 started
2018-05-10 12:30:03 [INFO] Cluster: Fetching MySQL Users from peer 10.138.180.183:6032 completed
2018-05-10 12:30:03 [INFO] Cluster: Loading to runtime MySQL Users from peer 10.138.180.183:6032
2018-05-10 12:30:03 [INFO] Cluster: Saving to disk MySQL Query Rules from peer 10.138.180.183:6032

Looking at the status from the checksum table, we can see node3 is now up to date.

admin proxysql2 ((none))>SELECT hostname, checksum, FROM_UNIXTIME(changed_at) changed_at, FROM_UNIXTIME(updated_at) updated_at FROM stats_proxysql_servers_checksums WHERE name='mysql_users' ORDER BY hostname;
+----------------+--------------------+---------------------+---------------------+
| hostname | checksum | changed_at | updated_at |
+----------------+--------------------+---------------------+---------------------+
| 10.138.180.183 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:39 | 2018-05-10 12:21:35 |
| 10.138.244.108 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:38 |2018-05-10 12:21:35 |
| 10.138.244.244 | 0x3D819A34C06EF4EA | 2018-05-10 11:19:39 |2018-05-10 12:21:35 |
+----------------+--------------------+---------------------+---------------------+
3 rows in set (0.00 sec)admin proxysql2 ((none))>SELECT hostname, checksum, FROM_UNIXTIME(changed_at) changed_at, FROM_UNIXTIME(updated_at) updated_at FROM stats_proxysql_servers_checksums WHERE name='mysql_users' ORDER BY hostname;
+----------------+--------------------+---------------------+---------------------+
| hostname | checksum | changed_at | updated_at |
+----------------+--------------------+---------------------+---------------------+
| 10.138.180.183 | 0x3928F574AFFF4C65 | 2018-05-10 12:12:24 | 2018-05-10 12:31:58 |
| 10.138.244.108 | 0x3928F574AFFF4C65 | 2018-05-10 12:12:23 | 2018-05-10 12:31:58 |
| 10.138.244.244 | 0x3928F574AFFF4C65 | 2018-05-10 12:30:19 | 2018-05-10 12:31:58 |
+----------------+--------------------+---------------------+---------------------+
3 rows in set (0.00 sec)

Now we have 3 ProxySQL nodes up to date. This example didn’t add any MySQL servers, hostgroups, etc, because the functionality is the same. The post is intended as an introduction to this new feature and how you can create and test a ProxySQL cluster.

Just remember that this is still an experimental feature and is subject to change with newer versions of ProxySQL.

Summary

This feature is really needed if you have more than one ProxySQL running for the same application in different instances. It is easy to maintain and configure for a single person and is easy to create and attach new nodes.

Hope you find this post helpful!

References

http://www.proxysql.com/blog/proxysql-cluster
http://www.proxysql.com/blog/proxysql-cluster-part2
http://www.proxysql.com/blog/proxysql-cluster-part3-mysql-servers
https://github.com/sysown/proxysql/wiki/ProxySQL-Cluster

The post ProxySQL Experimental Feature: Native ProxySQL Clustering appeared first on Percona Database Performance Blog.

Sequences are used to requesting unique values on demand, The best use case of sequences is to have a unique ID. , that can be used across multiple tables. In some cases sequences are really helpful to have an identifier before an actual row is inserted. With the normal way of having an automatically incrementing identifier, the identifier value will only be available after insert of the row and the identifier will only be unique inside its own table. MariaDB Server 10.3 follows the standard and includes compatibility with the way Oracle does sequences introduced in Oracle Database Server on top of the standard.

Simple steps to create a sequence in MariaDB 10.3 onwards, a create statement is used:

MariaDB [MDB101]> CREATE SEQUENCE Seq1_100
    -> START WITH 100
    -> INCREMENT BY 1;
Query OK, 0 rows affected (0.015 sec)

This creates a sequence that starts at 100 and is incremented with 1 every time a value is requested from the sequence. The sequence will be visible among the tables in the database, i.e. if you run SHOW TABLES it will be there. You can use DESCRIBE on the sequence to see what columns it has.

To test out the usage of sequences let’s create a table:

MariaDB [MDB101]> CREATE TABLE TAB1 (
    -> Col1 int(10) NOT NULL,
    -> Col2 varchar(30) NOT NULL,
    -> Col3 int(10) NOT NULL,
    ->  PRIMARY KEY (Col1)
    -> );
Query OK, 0 rows affected (0.018 sec)

Since we want to use sequences this time, we did not put AUTO_INCREMENT on the Col1 column. Instead we will ask for the next value from the sequence in the INSERT statements:

MariaDB [MDB101]> INSERT INTO TAB1 (Col1, Col2, Col3) VALUES (NEXT VALUE FOR Seq1_100, 'India', 10);
Query OK, 1 row affected (0.011 sec)

MariaDB [MDB101]> INSERT INTO TAB1 (Col1, Col2, Col3) VALUES (NEXT VALUE FOR Seq1_100, 'Jakarta', 20);
Query OK, 1 row affected (0.008 sec)

MariaDB [MDB101]> INSERT INTO TAB1 (Col1, Col2, Col3) VALUES (NEXT VALUE FOR Seq1_100, 'Singapore', 20); 
Query OK, 1 row affected (0.016 sec)

MariaDB [MDB101]> INSERT INTO TAB1 (Col1, Col2, Col3) VALUES (NEXT VALUE FOR Seq1_100, 'Japan', 30);
Query OK, 1 row affected (0.007 sec)

Instead of having the NEXT VALUE FOR in each INSERT statement, it could have been the default value of the column in this way:

MariaDB [MDB101]> ALTER TABLE TAB1 MODIFY Col1 int(10) NOT NULL DEFAULT NEXT VALUE FOR Seq1_100;
Query OK, 0 rows affected (0.007 sec)
Records: 0  Duplicates: 0  Warnings: 0

Running a SELECT over the TAB1 table will look like this:

MariaDB [MDB101]> SELECT * FROM TAB1;;
+------+-----------+------+
| Col1 | Col2      | Col3 |
+------+-----------+------+
|  100 | India     |   10 |
|  101 | Jakarta   |   20 |
|  102 | Singapore |   20 |
|  103 | Japan     |   30 |
+------+-----------+------+
4 rows in set (0.000 sec)

As we can see the Col1 column has been populated with numbers that start from 100 and are incremented with 1 as defined in the sequence’s CREATE statement. To get the last retrieved number from the sequence PREVIOUS VALUE is used:

MariaDB [MDB101]> SELECT PREVIOUS VALUE FOR Seq1_100;
+-----------------------------+
| PREVIOUS VALUE FOR Seq1_100 |
+-----------------------------+
|                         103 |
+-----------------------------+
1 row in set (0.000 sec)

MariaDB 10.3 shipped another very useful option for sequences is CYCLE, which means that we start again from the beginning after reaching a certain value. For example, if there are 5 phases in a process that are done sequentially and then start again from the beginning, we could easily create a sequence to always be able to retrieve the number of the next phase:

MariaDB [MDB101]> CREATE SEQUENCE Seq1_100_c5 
    ->   START WITH 100
    ->   INCREMENT BY 1
    -> MAXVALUE = 200
    -> CYCLE;
Query OK, 0 rows affected (0.012 sec)

The sequence above starts at 100 and is incremented with 1 every time the next value is requested. But when it reaches 200 (MAXVALUE) it will restart from 100 (CYCLE).

We can also set the next value of a sequence, to ALTER a sequence or using sequences in Oracle mode with Oracle specific syntax. To switch to Oracle mode use:

MariaDB [MDB101]> SET SQL_MODE=ORACLE;
Query OK, 0 rows affected (0.000 sec)

After that you can retrieve the next value of a sequence in Oracle style:

MariaDB [MDB101]> SELECT Seq1_100.nextval;
+------------------+
| Seq1_100.nextval |
+------------------+
|              104 |
+------------------+
1 row in set (0.009 sec)

You can read about MariaDB sequences in the documentation, MariaDB documentation

The post MariaDB 10.3 support Oracle mode sequences appeared first on MySQL Consulting, Support and Remote DBA Services By MinervaDB.

In this 101 article, I will show how to drop/remove a column from a table in MySQL.

In this article I will use an example table:

CREATE TABLE tb(
c1 INT PRIMARY KEY,
c2 char(1),
c3 varchar(2)
) ENGINE=InnoDB;

To remove a column we will make use of ALTER TABLE command:

ALTER TABLE tb DROP COLUMN c2;

The command allows you to remove multiple columns at once:

ALTER TABLE tb DROP COLUMN c2, DROP COLUMN c3;

If you are running MySQL 5.6 onwards, you can make this operation online, allowing other sessions to read and write to your table while the operation is been performed:

ALTER TABLE tb DROP COLUMN c2, ALGORITHM=INPLACE, LOCK=NONE;

Reference:

https://dev.mysql.com/doc/refman/5.7/en/alter-table.html
https://dev.mysql.com/doc/refman/5.7/en/innodb-create-index-overview.html

We build highly available and fault tolerant MySQL database infrastructure operations for some of the largest internet properties in this planet,  Our consulting team spend several hours daily researching on MySQL documentation and MySQL blogs to understand what are the best possible ways we can build optimal, scalable, highly available and reliable database infrastructure operations for planet-scale web properties. The most common approach towards building a fault-tolerant system is to make all the components in the ecosystem redundant, To make it even simple, component can be removed and system should continue to operate as expected.  MySQL replication is an proven method to build redundant database infrastructure operations, operationally these systems are highly complex, requiring maintenance and administration of several servers instead of just one, You need Sr. DBAs to manage such systems.

MySQL Group Replication can operate in both single-primary mode with automatic primary election, where only one server accepts updates at a time and multi-primary mode, where all servers can accept updates, even if they are issued concurrently. The built-in group membership service retains view of the group consistent and available for all servers at any given point in time, Servers can leave and join the group and the view is updated accordingly. If servers leave the group unexpectedly, the failure detection mechanism detects this and notifies the group that the view has changed, All this happens automatically !

For any transaction to commit, the majority of group members have to agree on the order of a given transaction in the global sequence of transactions. Decision to commit or abort a transaction is taken by respective servers, but all servers make same decision. In the case of systems built on network partition, When the members of the group are unable to reach agreement on a transaction, the system halt till the issue is resolved, This guarantees split-brain protection mechanism.

All these fault tolerance mechanisms are powered by Group Communication System (GCS) protocols, a group membership service, and completely safe ordered message delivery system. Group Replication is powered by Paxos algorithm (https://en.wikipedia.org/wiki/Paxos_(computer_science)), which acts as the group communication engine.

Till now, we have just posted about the capabilities of Group Replication, Now we have mentioned below (in bullets) about all the limitations of Group Replication:

• set –binlog-checksum=NONE – Group replication cannot benefit from –binlog-checksum due to the design limitation of replication event checksums.
• Gap Locks not supported – The information about gap locks are not available outside InnoDB so certification process cannot acknowledge gap locks  .
• Table Locks and Named Locks not supported – Certification process will not acknowledge table / named locks.
• Concurrent DDL versus DML Operations – Concurrent data definition statements and data manipulation statements executing against the same object but on different servers is not supported when using multi-primary mode. During execution of Data Definition Language (DDL) statements on an object, executing concurrent Data Manipulation Language (DML) on the same object but on a different server instance has the risk of conflicting DDL executing on different instances not being detected.
• Very Large Transactions.  Individual transactions that result in GTID contents which are large enough that it cannot be copied between group members over the network within a 5 second window can cause failures in the group communication. To avoid this issue try and limit the size of your transactions as much as possible. For example, split up files used with LOAD DATA INFILE into smaller chunks.
• Multi-primary Mode Deadlock.  When a group is operating in multi-primary mode, SELECT … FOR UPDATE statements can result in a deadlock. This is because the lock is not shared across the members of the group, therefore the expectation for such a statement might not be reached.

The post MySQL 8.0 Group Replication Limitations appeared first on MySQL Consulting, Support and Remote DBA Services By MinervaDB.

Let them stay together.

In the last YEARS, I have seen quite often that users, when installing a product such as PXC, instead of spending five minutes to understand what to do just run

iptables -F

In short, they remove any rules for their firewall.

With this post, I want to show you how easy it can be to do the right thing instead of putting your server at risk. I’ll show you how a slightly more complex setup like PXC (compared to MySQL), can be easily achieved without risky shortcuts.

iptables is the utility used to manage the chains of rules used by the Linux kernel firewall, which is your basic security tool.
Linux comes with a wonderful firewall built into the kernel. As an administrator, you can configure this firewall with interfaces like ipchains  — which we are not going to cover — and iptables, which we shall talk about.

iptables is stateful, which means that the firewall can make decisions based on received packets. This means that I can, for instance, DROP a packet if it’s coming from bad-guy.com.

I can also create a set of rules that either will allow or reject the package, or that will redirect it to another rule. This potentially can create a very complex scenario.

However, for today and for this use case let’s keep it simple…  Looking at my own server:

iptables -v -L
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
 250K   29M ACCEPT     all  --  any    any     anywhere             anywhere             state RELATED,ESTABLISHED
    6   404 ACCEPT     icmp --  any    any     anywhere             anywhere
    0     0 ACCEPT     all  --  lo     any     anywhere             anywhere
    9   428 ACCEPT     tcp  --  any    any     anywhere             anywhere             state NEW tcp dpt:ssh
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere             state NEW tcp dpt:mysql
    0     0 ACCEPT     tcp  --  any    any     anywhere             anywhere
  210 13986 REJECT     all  --  any    any     anywhere             anywhere             reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 REJECT     all  --  any    any     anywhere             anywhere             reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT 241K packets, 29M bytes)
 pkts bytes target     prot opt in     out     source               destination

That’s not too bad, my server is currently accepting only SSH and packets on port 3306. Please note that I used the -v option to see more information like IN/OUT and  that allows me to identify that actually row #3 is related to my loopback device, and as such it’s good to have it open.

The point is that if I try to run the PXC cluster with these settings it will fail, because the nodes will not be able to see each other.

A quite simple example when try to start the second node of the cluster:

2018-05-21T17:56:14.383686Z 0 [Note] WSREP: (3cb4b3a6, 'tcp://10.0.0.21:4567') connection to peer 584762e6 with addr tcp://10.0.0.23:4567 timed out, no messages seen in PT3S

Starting a new node will fail, given that the connectivity will not be established correctly. In the Percona documentation there is a notes section in which we mention that these ports must be open to have the cluster working correctly.:

• 3306 For MySQL client connections and State Snapshot Transfer that use the mysqldump method.
• 4567 For Galera Cluster replication traffic, multicast replication uses both UDP transport and TCP on this port.
• 4568 For Incremental State Transfer.
• 4444 For all other State Snapshot Transfer.

Of course, if you don’t know how to do it that could be a problem, but it is quite simple. Just use the following commands to add the needed rules:

iptables -I INPUT 2 --protocol tcp --match tcp --dport 3306 --source 10.0.0.1/24 --jump ACCEPT
iptables -I INPUT 3 --protocol tcp --match tcp --dport 4567 --source 10.0.0.1/24 --jump ACCEPT
iptables -I INPUT 4 --protocol tcp --match tcp --dport 4568 --source 10.0.0.1/24 --jump ACCEPT
iptables -I INPUT 5 --protocol tcp --match tcp --dport 4444 --source 10.0.0.1/24 --jump ACCEPT
iptables -I INPUT 6 --protocol udp --match udp --dport 4567 --source 10.0.0.1/24 --jump ACCEPT

Once you have done this check the layout again and you should have something like this:

[root@galera1h1n5 gal571]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- 10.0.0.0/24 anywhere tcp dpt:mysql
ACCEPT tcp -- 10.0.0.0/24 anywhere tcp dpt:tram
ACCEPT tcp -- 10.0.0.0/24 anywhere tcp dpt:bmc-reporting
ACCEPT tcp -- 10.0.0.0/24 anywhere tcp dpt:krb524
ACCEPT udp -- 10.0.0.0/24 anywhere udp dpt:tram
ACCEPT icmp -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere tcp dpt:ssh
ACCEPT tcp -- anywhere anywhere tcp dpt:mysql
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain FORWARD (policy ACCEPT)
target prot opt source destination
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Try to start the secondary node, and — tadaaa — the node will connect, will provision itself, and finally will start correctly.

All good? Well not really, you still need to perform a final step. We need to make our server accessible also for PMM monitoring agents.

You have PMM right? If you don’t take a look here and you will want it.

Anyhow PMM will not work correctly with the rules I have, and the result will be an empty set of graphs when accessing the server statistics. Luckily, PMM has a very easy way to help you identify the issue:

[root@galera1h1n5 gal571]# pmm-admin check-network
PMM Network Status
Server Address | 192.168.1.52
Client Address | 192.168.1.205
* System Time
NTP Server (0.pool.ntp.org) | 2018-05-24 08:05:37 -0400 EDT
PMM Server | 2018-05-24 12:05:34 +0000 GMT
PMM Client | 2018-05-24 08:05:37 -0400 EDT
PMM Server Time Drift | OK
PMM Client Time Drift | OK
PMM Client to PMM Server Time Drift | OK
* Connection: Client --> Server
-------------------- -------
SERVER SERVICE STATUS
-------------------- -------
Consul API OK
Prometheus API OK
Query Analytics API OK
Connection duration | 1.051724ms
Request duration | 311.924µs
Full round trip | 1.363648ms
* Connection: Client <-- Server
-------------- ------------ -------------------- ------- ---------- ---------
SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD
-------------- ------------ -------------------- ------- ---------- ---------
linux:metrics galera1h1n5 192.168.1.205:42000 DOWN NO NO
mysql:metrics gal571 192.168.1.205:42002 DOWN NO NO
When an endpoint is down it may indicate that the corresponding service is stopped (run 'pmm-admin list' to verify).
If it's running, check out the logs /var/log/pmm-*.log
When all endpoints are down but 'pmm-admin list' shows they are up and no errors in the logs,
check the firewall settings whether this system allows incoming connections from server to address:port in question.
Also you can check the endpoint status by the URL: http://192.168.1.52/prometheus/targets

What you want more? You have all the information to debug and build your new rules. I just need to open the ports 42000 42002 on my firewall:

iptables -I INPUT 7 --protocol tcp --match tcp --dport 42000 --source 192.168.1.1/24 --jump ACCEPT
iptables -I INPUT 8 --protocol tcp --match tcp --dport 42002 --source 192.168.1.1/24 --jump ACCEPT

Please note that we are handling the connectivity for PMM using a different range of IPs/subnet. This because it is best practice to have PXC nodes communicate to a dedicated network/subnet (physical and logical).

Run the test again:

* Connection: Client <-- Server
-------------- ------------ -------------------- ------- ---------- ---------
SERVICE TYPE NAME REMOTE ENDPOINT STATUS HTTPS/TLS PASSWORD
-------------- ------------ -------------------- ------- ---------- ---------
linux:metrics galera1h1n5 192.168.1.205:42000 OK YES YES
mysql:metrics gal571 192.168.1.205:42002 OK YES YES

Done …  I just repeat this on all my nodes and I will have set my firewall to handle the PXC related security.

Now that all my settings are working well I can save my firewall’s rules:

iptables-save > /etc/sysconfig/iptables

For Ubuntu you may need some additional steps as for (https://help.ubuntu.com/community/IptablesHowTo#Using_iptables-save.2Frestore_to_test_rules)

There are some nice tools to help you even more, if you are very lazy, like UFW and the graphical one, GUFW. Developed to ease iptables firewall configuration, ufw provides a user friendly way to create an IPv4 or IPv6 host-based firewall. By default UFW is disabled in Ubuntu. Given that ultimately they use iptables, and their use is widely covered in other resources such as the official Ubuntu documentation, I won’t cover these here.

Conclusion

Please don’t make the mistake of flushing/ignoring your firewall, when to make this right is just a matter of 5 commands. It’s easy enough to be done by everyone and it’s good enough to stop the basic security attacks.

Happy MySQL (and PXC) to everyone.

The post PXC loves firewalls (and System Admins loves iptables) appeared first on Percona Database Performance Blog.

Please join Percona’s CEO, Peter Zaitsev as he presents Performance Analysis and Troubleshooting Methodologies for Databases on Wednesday, June 13th, 2018 at 11:00 AM PDT (UTC-7) / 2:00 PM EDT (UTC-4).

Have you heard about the USE Method (Utilization – Saturation – Errors)? RED (Rate – Errors – Duration), or Golden Signals (Latency – Traffic – Errors – Saturations)?

In this presentation, we will talk briefly about these different-but-similar “focuses”. We’ll discuss how we can apply them to data infrastructure performance analysis, troubleshooting, and monitoring.

We will use MySQL as an example, but most of this talk applies to other database technologies too.

Register for the webinar

About Peter Zaitsev, CEO

Peter Zaitsev co-founded Percona and assumed the role of CEO in 2006. As one of the foremost experts on MySQL strategy and optimization, Peter leveraged both his technical vision and entrepreneurial skills to grow Percona from a two-person shop to one of the most respected open source companies in the business. With over 140 professionals in 30 plus countries, Peter’s venture now serves over 3000 customers – including the “who’s who” of internet giants, large enterprises and many exciting startups. Percona was named to the Inc. 5000 in 2013, 2014, 2015 and 2016.

Peter was an early employee at MySQL AB, eventually leading the company’s High Performance Group. A serial entrepreneur, Peter co-founded his first startup while attending Moscow State University where he majored in Computer Science. Peter is a co-author of High Performance MySQL: Optimization, Backups, and Replication, one of the most popular books on MySQL performance. Peter frequently speaks as an expert lecturer at MySQL and related conferences, and regularly posts on the Percona Database Performance Blog. He has also been tapped as a contributor to Fortune and DZone, and his recent ebook Practical MySQL Performance Optimization Volume 1 is one of percona.com’s most popular downloads. Peter lives in North Carolina with his wife and two children. In his spare time, Peter enjoys travel and spending time outdoors.

The post Webinar Weds 6/13: Performance Analysis and Troubleshooting Methodologies for Databases appeared first on Percona Database Performance Blog.

In this article I will explain the partial update optimizations for smaller (LOBs) in InnoDB. Small here qualifies the size of the modification and not the size of the LOB.  For some background information about the partial update feature, kindly go through our previous posts on this (here, here and here).…

In this post, I will cover a bit of ProxySQL. ProxySQL is a proxy which implements MySQL protocol, allowing it to do things that other proxies cannot do. ProxySQL is gaining a lot of traction nowadays and it’s capable to integrate with a variety of products from MySQL ecosystems, such as replication (master – slave / master – master), Percona XtraDB Cluster and Group Replication.
One of its many functionalities (which IMHO makes it awesome) is the ability to do read/write split seamless to the application. You can start sending reads to your slave without doing a single line of code.

In this article, I will cover a few important points to understand how it works.

Instalation

The instalation is easy, you can do it by downloading the corresponding package for your OS from its official github repo https://github.com/sysown/proxysql/releases

sudo yum install https://github.com/sysown/proxysql/releases/download/v1.4.9/proxysql-1.4.9-1-centos7.x86_64.rpm

Now we just need to start the service

sudo service proxysql start 

Interfaces

ProxySQL splits application interface from the admin interface. It will listen on 2 network ports. Admin will be on 6032 and application will listen on 6033 (reverse of 3306 ).

Layers

Other important part to understand how the proxy works is to understand its layers. I am gonna show you a diagram that can be found on its official documentation:

+-------------------------+
|      1. RUNTIME         |   
+-------------------------+
       /|\          |
        |           |
        |           |
        |          \|/
+-------------------------+
|     2.  MEMORY          |   
+-------------------------+ 
       /|\          |      
        |           |      
        |           |       
        |          \|/      
+-------------------------+  
|     3.   DISK           |  
+-------------------------+  

ProxySQL will always read information from 1.Runtime layer, which is stored in memory.
Every time we connect to the admin port (6032) we are manipulating information from layer 2.Memory. As the name infers, its also stored in memory.
We then have the layer 3.Disk. As the other two layers are stored in memory, we need a layer to persist information across service/server restarts.

What is the benefic of this layout?
It allows us to manipulate different areas and apply the changes at once. We can think about how a transaction works, where we run multiple queries and commit them at once. When we alter something, we will be manipulating the Memory layer, then we will run a command LOAD MYSQL [SERVERS | USERS | QUERY RULES] TO RUNTIME to load this information to runtime and we will save the information to disk layer by issuing SAVE MYSQL [SERVERS | USERS | QUERY RULES] TO DISK.

Hostgroups

ProxySQL group servers in something named hostgroup. In a topology which we have a master and two slaves, we will create a hostgroup(HG) 1 and specify that our master is part of that HG and we will create a HG 2 and specify that both slaves belong to that HG. Hostgroup creation is done at the time we specify servers on mysql_servers table. There is not fixed enumeration, you can create your HG with any ID you want.

Later we will configure user and queries to identify if the query coming in is a read it should be answered by one of the servers from HG 2 (where we configured our slaves). If the query is not a read, then our server on HG 1 will receive it.

Authentication

ProxySQL has functionalities like firewall, in which it has the capability of blocking a query even before it reaches our backend server. To do it it’s required to have the user authentication module also present on the proxy side. So we will be required to create all users that we wish to connect via ProxySQL also create on the proxy side.

Now that you understand a bit of the basic of how ProxySQL works, you can start playing with it.

I think you’ll like the story I’m about to tell you. I’m going to show you how to build a GraphQL API with Vesper framework, TypeORM, and MySQL. These are Node frameworks, and I’ll use TypeScript for the language. For the client, I’ll use React, reactstrap, and Apollo Client to talk to the API. Once you have this environment working, and you add secure user authentication, I believe you’ll love the experience!

Why focus on secure authentication? Well, aside from the fact that I work for Okta, I think we can all agree that pretty much every application depends upon a secure identity management system. For most developers who are building React apps, there’s a decision to be made between rolling your own authentication/authorization or plugging in a service like Okta. Before I dive into building a React app, I want to tell you a bit about Okta, and why I think it’s an excellent solution for all JavaScript developers.

What is Okta?

In short, we make identity management a lot easier, more secure, and more scalable than what you’re used to. Okta is a cloud service that allows developers to create, edit, and securely store user accounts and user account data, and connect them with one or multiple applications. Our API enables you to:

• Authenticate and authorize your users
• Store data about your users
• Perform password-based and social login
• Secure your application with multi-factor authentication
• And much more! Check out our product documentation

Are you sold? Register for a forever-free developer account, and when you’re done, come on back so we can learn more about building secure apps in React!

Why a Health Tracking App?

In late September through mid-October 2014, I'd done a 21-Day Sugar Detox during which I stopped eating sugar, started exercising regularly, and stopped drinking alcohol. I'd had high blood pressure for over ten years and was on blood-pressure medication at the time. During the first week of the detox, I ran out of blood-pressure medication. Since a new prescription required a doctor visit, I decided I'd wait until after the detox to get it. After three weeks, not only did I lose 15 pounds, but my blood pressure was at normal levels!

Before I started the detox, I came up with a 21-point system to see how healthy I was each week. Its rules were simple: you can earn up to three points per day for the following reasons:

1. If you eat healthy, you get a point. Otherwise, zero.
2. If you exercise, you get a point.
3. If you don't drink alcohol, you get a point.

I was surprised to find I got eight points the first week I used this system. During the detox, I got 16 points the first week, 20 the second, and 21 the third. Before the detox, I thought eating healthy meant eating anything except fast food. After the detox, I realized that eating healthy for me meant eating no sugar. I'm also a big lover of craft beer, so I modified the alcohol rule to allow two healthier alcohol drinks (like a greyhound or red wine) per day.

My goal is to earn 15 points per week. I find that if I get more, I'll likely lose weight and have good blood pressure. If I get fewer than 15, I risk getting sick. I've been tracking my health like this since September 2014. I've lost weight, and my blood pressure has returned to and maintained normal levels. I haven't had good blood pressure since my early 20s, so this has been a life changer for me.

I built 21-Points Health to track my health. I figured it'd be fun to recreate a small slice of that app, just tracking daily points.

Building an API with TypeORM, GraphQL, and Vesper

TypeORM is a nifty ORM (object-relational mapper) framework that can run in most JavaScript platforms, including Node, a browser, Cordova, React Native, and Electron. It’s heavily influenced by Hibernate, Doctrine, and Entity Framework. Install TypeORM globally to begin creating your API.

npm i -g typeorm@0.2.7

Create a directory to hold the React client and GraphQL API.

mkdir health-tracker
cd health-tracker

Create a new project with MySQL using the following command:

typeorm init --name graphql-api --database mysql

Edit graphql-api/ormconfig.json to customize the username, password, and database.

{
    ...
    "username": "health",
    "password": "pointstest",
    "database": "healthpoints",
    ...
}

TIP: To see the queries being executed against MySQL, change the "logging" value in this file to be "all". Many other logging options are available too.

Install MySQL

Install MySQL if you don’t already have it installed. On Ubuntu, you can use sudo apt-get install mysql-server. On macOS, you can use Homebrew and brew install mysql. For Windows, you can use the MySQL Installer.

Once you’ve got MySQL installed and configured with a root password, login and create a healthpoints database.

mysql -u root -p
create database healthpoints;
use healthpoints;
grant all privileges on *.* to 'health'@'localhost' identified by 'points';

Navigate to your graphql-api project in a terminal window, install the project’s dependencies, then start it to ensure you can connect to MySQL.

cd graphql-api
npm i
npm start

You should see the following output:

Inserting a new user into the database...
Saved a new user with id: 1
Loading users from the database...
Loaded users:  [ User { id: 1, firstName: 'Timber', lastName: 'Saw', age: 25 } ]
Here you can setup and run express/koa/any other framework.

Install Vesper to Integrate TypeORM and GraphQL

Vesper is a Node framework that integrates TypeORM and GraphQL. To install it, use good ol' npm.

npm i vesper@0.1.9

Now it's time to create some GraphQL models (that define what your data looks like) and some controllers (that explain how to interact with your data).

Create graphql-api/src/schema/model/Points.graphql:

type Points {
  id: Int
  date: Date
  exercise: Int
  diet: Int
  alcohol: Int
  notes: String
  user: User
}

Create graphql-api/src/schema/model/User.graphql:

type User {
  id: String
  firstName: String
  lastName: String
  points: [Points]
}

Next, create a graphql-api/src/schema/controller/PointsController.graphql with queries and mutations:

type Query {
  points: [Points]
  pointsGet(id: Int): Points
  users: [User]
}

type Mutation {
  pointsSave(id: Int, date: Date, exercise: Int, diet: Int, alcohol: Int, notes: String): Points
  pointsDelete(id: Int): Boolean
}

Now that your data has GraphQL metadata create entities that will be managed by TypeORM. Change src/entity/User.ts to have the following code that allows points to be associated with a user.

import { Column, Entity, OneToMany, PrimaryColumn } from 'typeorm';
import { Points } from './Points';

@Entity()
export class User {

  @PrimaryColumn()
  id: string;

  @Column()
  firstName: string;

  @Column()
  lastName: string;

  @OneToMany(() => Points, points => points.user)
  points: Points[];
}

In the same src/entity directory, create a Points.ts class with the following code.

import { Entity, PrimaryGeneratedColumn, Column, ManyToOne } from 'typeorm';
import { User } from './User';

@Entity()
export class Points {

  @PrimaryGeneratedColumn()
  id: number;

  @Column({ type: 'timestamp', default: () => 'CURRENT_TIMESTAMP'})
  date: Date;

  @Column()
  exercise: number;

  @Column()
  diet: number;

  @Column()
  alcohol: number;

  @Column()
  notes: string;

  @ManyToOne(() => User, user => user.points, { cascade: ["insert"] })
  user: User|null;
}

Note the cascade: ["insert"] option on the @ManyToOne annotation above. This option will automatically insert a user if it's present on the entity. Create src/controller/PointsController.ts to handle converting the data from your GraphQL queries and mutations.

import { Controller, Mutation, Query } from 'vesper';
import { EntityManager } from 'typeorm';
import { Points } from '../entity/Points';

@Controller()
export class PointsController {

  constructor(private entityManager: EntityManager) {
  }

  // serves "points: [Points]" requests
  @Query()
  points() {
    return this.entityManager.find(Points);
  }

  // serves "pointsGet(id: Int): Points" requests
  @Query()
  pointsGet({id}) {
    return this.entityManager.findOne(Points, id);
  }

  // serves "pointsSave(id: Int, date: Date, exercise: Int, diet: Int, alcohol: Int, notes: String): Points" requests
  @Mutation()
  pointsSave(args) {
    const points = this.entityManager.create(Points, args);
    return this.entityManager.save(Points, points);
  }

  // serves "pointsDelete(id: Int): Boolean" requests
  @Mutation()
  async pointsDelete({id}) {
    await this.entityManager.remove(Points, {id: id});
    return true;
  }
}

Change src/index.ts to use Vesper's bootstrap() to configure everything.

import { bootstrap } from 'vesper';
import { PointsController } from './controller/PointsController';
import { Points } from './entity/Points';
import { User } from './entity/User';

bootstrap({
  port: 4000,
  controllers: [
    PointsController
  ],
  entities: [
    Points,
    User
  ],
  schemas: [
    __dirname + '/schema/**/*.graphql'
  ],
  cors: true
}).then(() => {
  console.log('Your app is up and running on http://localhost:4000. ' +
    'You can use playground in development mode on http://localhost:4000/playground');
}).catch(error => {
  console.error(error.stack ? error.stack : error);
});

This code tells Vesper to register controllers, entities, GraphQL schemas, to run on port 4000, and to enable CORS (cross-origin resource sharing).

Start your API using npm start and navigate to http://localhost:4000/playground. In the left pane, enter the following mutation and press the play button. You might try typing the code below so you can experience the code completion that GraphQL provides you.

mutation {
  pointsSave(exercise:1, diet:1, alcohol:1, notes:"Hello World") {
    id
    date
    exercise
    diet
    alcohol
    notes
  }
}

Your result should look similar to mine.

You can click the "SCHEMA" tab on the right to see the available queries and mutations. Pretty slick, eh?!

You can use the following points query to verify that data is in your database.

query {
  points {id date exercise diet notes}
}

Fix Dates

You might notice that the date returned from pointsSave and the points query is in a format the might be difficult for a JavaScript client to understand. You can fix that, install graphql-iso-date.

npm i graphql-iso-date@3.5.0

Then, add an import in src/index.ts and configure custom resolvers for the various date types. This example only uses Date, but it's helpful to know the other options.

import { GraphQLDate, GraphQLDateTime, GraphQLTime } from 'graphql-iso-date';

bootstrap({
  ...
  // https://github.com/vesper-framework/vesper/issues/4
  customResolvers: {
    Date: GraphQLDate,
    Time: GraphQLTime,
    DateTime: GraphQLDateTime
  },
  ...
});

Now running the points query will return a more client-friendly result.

{
  "data": {
    "points": [
      {
        "id": 1,
        "date": "2018-06-04",
        "exercise": 1,
        "diet": 1,
        "notes": "Hello World"
      }
    ]
  }
}

You've written an API with GraphQL and TypeScript in about 20 minutes. How cool is that?! There's still work to do though. In the next sections, you'll create a React client for this API and add authentication with OIDC. Adding authentication will give you the ability to get the user's information and associate a user with their points.

Get Started with React

One of the quickest ways to get started with React is to use Create React App. Install the latest release using the command below.

npm i -g create-react-app@1.1.4

Navigate to the directory where you created your GraphQL API and create a React client.

cd health-tracker
create-react-app react-client

Install the dependencies you'll need to talk to integrate Apollo Client with React, as well as Bootstrap and reactstrap.

npm i apollo-boost@0.1.7 react-apollo@2.1.4 graphql-tag@2.9.2 graphql@0.13.2

Configure Apollo Client for Your API

Open react-client/src/App.js and import ApolloClient from apollo-boost and add the endpoint to your GraphQL API.

import ApolloClient from 'apollo-boost';

const client = new ApolloClient({
  uri: "http://localhost:4000/graphql"
});

That's it! With only three lines of code, your app is ready to start fetching data. You can prove it by importing the gql function from graphql-tag. This will parse your query string and turn it into a query document.

import gql from 'graphql-tag';

class App extends Component {

  componentDidMount() {
    client.query({
      query: gql`
        {
          points {
            id date exercise diet alcohol notes
          }
        }
      `
    })
    .then(result => console.log(result));
  }
...
}

Make sure to open your browser's developer tools so you can see the data after making this change. You could modify the console.log() to use this.setState({points: results.data.points}), but then you'd have to initialize the default state in the constructor. But there's an easier way: you can use ApolloProvider and Query components from react-apollo!

Below is a modified version of react-client/src/App.js that uses these components.

import React, { Component } from 'react';
import logo from './logo.svg';
import './App.css';
import ApolloClient from 'apollo-boost';
import gql from 'graphql-tag';
import { ApolloProvider, Query } from 'react-apollo';
const client = new ApolloClient({
  uri: "http://localhost:4000/graphql"
});

class App extends Component {

  render() {
    return (
      <ApolloProvider client={client}>
        <div className="App">
          <header className="App-header">
            <img src={logo} className="App-logo" alt="logo" />
            <h1 className="App-title">Welcome to React</h1>
          </header>
          <p className="App-intro">
            To get started, edit <code>src/App.js</code> and save to reload.
          </p>
          <Query query={gql`
            {
              points {id date exercise diet alcohol notes}
            }
          `}>
            {({loading, error, data}) => {
              if (loading) return <p>Loading...</p>;
              if (error) return <p>Error: {error}</p>;
              return data.points.map(p => {
                return <div key={p.id}>
                  <p>Date: {p.date}</p>
                  <p>Points: {p.exercise + p.diet + p.alcohol}</p>
                  <p>Notes: {p.notes}</p>
                </div>
              })
            }}
          </Query>
        </div>
      </ApolloProvider>
    );
  }
}

export default App;

You've built a GraphQL API and a React UI that talks to it - excellent work! However, there's still more to do. In the next sections, I'll show you how to add authentication to React, verify JWTs with Vesper, and add CRUD functionality to the UI. CRUD functionality already exists in the API thanks to the mutations you wrote earlier.

Add Authentication for React with OpenID Connect

You'll need to configure React to use Okta for authentication. You'll need to create an OIDC app in Okta for that.

Log in to your Okta Developer account (or sign up if you don’t have an account) and navigate to Applications > Add Application. Click Single-Page App, click Next, and give the app a name you’ll remember. Change all instances of localhost:8080 to localhost:3000 and click Done. Your settings should be similar to the screenshot below.

Okta's React SDK allows you to integrate OIDC into a React application. To install, run the following commands:

npm i @okta/okta-react@1.0.2 react-router-dom@4.2.2

Okta's React SDK depends on react-router, hence the reason for installing react-router-dom. Configuring routing in client/src/App.tsx is a common practice, so replace its code with the JavaScript below that sets up authentication with Okta.

import React, { Component } from 'react';
import { BrowserRouter as Router, Route } from 'react-router-dom';
import { ImplicitCallback, SecureRoute, Security } from '@okta/okta-react';
import Home from './Home';
import Login from './Login';
import Points from './Points';

function onAuthRequired({history}) {
  history.push('/login');
}

class App extends Component {
  render() {
    return (
      <Router>
        <Security issuer='https://{yourOktaDomain}.com/oauth2/default'
                  client_id='{yourClientId}'
                  redirect_uri={window.location.origin + '/implicit/callback'}
                  onAuthRequired={onAuthRequired}>
          <Route path='/' exact={true} component={Home}/>
          <SecureRoute path='/points' component={Points}/>
          <Route path='/login' render={() => <Login baseUrl='https://{yourOktaDomain}.com'/>}/>
          <Route path='/implicit/callback' component={ImplicitCallback}/>
        </Security>
      </Router>
    );
  }
}

export default App;

Make sure to replace {yourOktaDomain} and {yourClientId} in the code above. Your Okta domain should be something like dev-12345.oktapreview. Make sure you don't end up with two .com values in the URL!

The code in App.js references two components that don't exist yet: Home, Login, and Points. Create src/Home.js with the following code. This component renders the default route, provides a Login button, and links to your points and logout after you've logged in.

import React, { Component } from 'react';
import { withAuth } from '@okta/okta-react';
import { Button, Container } from 'reactstrap';
import AppNavbar from './AppNavbar';
import { Link } from 'react-router-dom';

export default withAuth(class Home extends Component {
  constructor(props) {
    super(props);
    this.state = {authenticated: null, userinfo: null, isOpen: false};
    this.checkAuthentication = this.checkAuthentication.bind(this);
    this.checkAuthentication();
    this.login = this.login.bind(this);
    this.logout = this.logout.bind(this);
  }

  async checkAuthentication() {
    const authenticated = await this.props.auth.isAuthenticated();
    if (authenticated !== this.state.authenticated) {
      if (authenticated && !this.state.userinfo) {
        const userinfo = await this.props.auth.getUser();
        this.setState({authenticated, userinfo});
      } else {
        this.setState({authenticated});
      }
    }
  }

  async componentDidMount() {
    this.checkAuthentication();
  }

  async componentDidUpdate() {
    this.checkAuthentication();
  }

  async login() {
    this.props.auth.login('/');
  }

  async logout() {
    this.props.auth.logout('/');
    this.setState({authenticated: null, userinfo: null});
  }

  render() {
    if (this.state.authenticated === null) return null;
    const button = this.state.authenticated ?
        <div>
          <Button color="link"><Link to="/points">Manage Points</Link></Button><br/>
          <Button color="link" onClick={this.logout}>Logout</Button>
        </div>:
      <Button color="primary" onClick={this.login}>Login</Button>;

    const message = this.state.userinfo ?
      <p>Hello, {this.state.userinfo.given_name}!</p> :
      <p>Please log in to manage your points.</p>;

    return (
      <div>
        <AppNavbar/>
        <Container fluid>
          {message}
          {button}
        </Container>
      </div>
    );
  }
});

This component uses <Container/> and <Button/> from reactstrap. Install reactstrap, so everything compiles. It depends on Bootstrap, so include it too.

npm i reactstrap@6.1.0 bootstrap@4.1.1

Add Bootstrap's CSS file as an import in src/index.js.

import 'bootstrap/dist/css/bootstrap.min.css';

You might notice there's a <AppNavbar/> in the Home component's render() method. Create src/AppNavbar.js so you can use a common header between components.

import React, { Component } from 'react';
import { Collapse, Nav, Navbar, NavbarBrand, NavbarToggler, NavItem, NavLink } from 'reactstrap';
import { Link } from 'react-router-dom';

export default class AppNavbar extends Component {
  constructor(props) {
    super(props);
    this.state = {isOpen: false};
    this.toggle = this.toggle.bind(this);
  }

  toggle() {
    this.setState({
      isOpen: !this.state.isOpen
    });
  }

  render() {
    return <Navbar color="success" dark expand="md">
      <NavbarBrand tag={Link} to="/">Home</NavbarBrand>
      <NavbarToggler onClick={this.toggle}/>
      <Collapse isOpen={this.state.isOpen} navbar>
        <Nav className="ml-auto" navbar>
          <NavItem>
            <NavLink
              href="https://twitter.com/oktadev">@oktadev</NavLink>
          </NavItem>
          <NavItem>
            <NavLink href="https://github.com/oktadeveloper/okta-react-graphql-example/">GitHub</NavLink>
          </NavItem>
        </Nav>
      </Collapse>
    </Navbar>;
  }
}

In this example, I'm going to embed Okta's Sign-In Widget. Another option is to redirect to Okta and use a hosted login page. Install the Sign-In Widget using npm.

npm i @okta/okta-signin-widget@2.9.0

Create src/Login.js and add the following code to it.

import React, { Component } from 'react';
import { Redirect } from 'react-router-dom';
import OktaSignInWidget from './OktaSignInWidget';
import { withAuth } from '@okta/okta-react';

export default withAuth(class Login extends Component {
  constructor(props) {
    super(props);
    this.onSuccess = this.onSuccess.bind(this);
    this.onError = this.onError.bind(this);
    this.state = {
      authenticated: null
    };
    this.checkAuthentication();
  }

  async checkAuthentication() {
    const authenticated = await this.props.auth.isAuthenticated();
    if (authenticated !== this.state.authenticated) {
      this.setState({authenticated});
    }
  }

  componentDidUpdate() {
    this.checkAuthentication();
  }

  onSuccess(res) {
    return this.props.auth.redirect({
      sessionToken: res.session.token
    });
  }

  onError(err) {
    console.log('error logging in', err);
  }

  render() {
    if (this.state.authenticated === null) return null;
    return this.state.authenticated ?
      <Redirect to={{pathname: '/'}}/> :
      <OktaSignInWidget
        baseUrl={this.props.baseUrl}
        onSuccess={this.onSuccess}
        onError={this.onError}/>;
  }
});

The Login component has a reference to OktaSignInWidget. Create src/OktaSignInWidget.js:

import React, {Component} from 'react';
import ReactDOM from 'react-dom';
import OktaSignIn from '@okta/okta-signin-widget';
import '@okta/okta-signin-widget/dist/css/okta-sign-in.min.css';
import '@okta/okta-signin-widget/dist/css/okta-theme.css';
import './App.css';

export default class OktaSignInWidget extends Component {
  componentDidMount() {
    const el = ReactDOM.findDOMNode(this);
    this.widget = new OktaSignIn({
      baseUrl: this.props.baseUrl
    });
    this.widget.renderEl({el}, this.props.onSuccess, this.props.onError);
  }

  componentWillUnmount() {
    this.widget.remove();
  }

  render() {
    return <div/>;
  }
};

Create src/Points.js to render the list of points from your API.

import React, { Component } from 'react';
import { ApolloClient } from 'apollo-client';
import { createHttpLink } from 'apollo-link-http';
import { setContext } from 'apollo-link-context';
import { InMemoryCache } from 'apollo-cache-inmemory';
import gql from 'graphql-tag';
import { withAuth } from '@okta/okta-react';
import AppNavbar from './AppNavbar';
import { Alert, Button, Container, Table } from 'reactstrap';
import PointsModal from './PointsModal';

export const httpLink = createHttpLink({
  uri: 'http://localhost:4000/graphql'
});

export default withAuth(class Points extends Component {
  client;

  constructor(props) {
    super(props);
    this.state = {points: [], error: null};

    this.refresh = this.refresh.bind(this);
    this.remove = this.remove.bind(this);
  }

  refresh(item) {
    let existing = this.state.points.filter(p => p.id === item.id);
    let points = [...this.state.points];
    if (existing.length === 0) {
      points.push(item);
      this.setState({points});
    } else {
      this.state.points.forEach((p, idx) => {
        if (p.id === item.id) {
          points[idx] = item;
          this.setState({points});
        }
      })
    }
  }

  remove(item, index) {
    const deletePoints = gql`mutation pointsDelete($id: Int) { pointsDelete(id: $id) }`;

    this.client.mutate({
      mutation: deletePoints,
      variables: {id: item.id}
    }).then(result => {
      if (result.data.pointsDelete) {
        let updatedPoints = [...this.state.points].filter(i => i.id !== item.id);
        this.setState({points: updatedPoints});
      }
    });
  }

  componentDidMount() {
    const authLink = setContext(async (_, {headers}) => {
      const token = await this.props.auth.getAccessToken();
      const user = await this.props.auth.getUser();

      // return the headers to the context so httpLink can read them
      return {
        headers: {
          ...headers,
          authorization: token ? `Bearer ${token}` : '',
          'x-forwarded-user': user ? JSON.stringify(user) : ''
        }
      }
    });

    this.client = new ApolloClient({
      link: authLink.concat(httpLink),
      cache: new InMemoryCache(),
      connectToDevTools: true
    });

    this.client.query({
      query: gql`
        {
          points {
              id,
              user {
                  id,
                  lastName
              }
              date,
              alcohol,
              exercise,
              diet,
              notes
          }
        }`
    }).then(result => {
      this.setState({points: result.data.points});
    }).catch(error => {
      this.setState({error: <Alert color="danger">Failure to communicate with API.</Alert>});
    });
  }

  render() {
    const {points, error} = this.state;
    const pointsList = points.map(p => {
      const total = p.exercise + p.diet + p.alcohol;
      return <tr key={p.id}>
        <td style={{whiteSpace: 'nowrap'}}><PointsModal item={p} callback={this.refresh}/></td>
        <td className={total <= 1 ? 'text-danger' : 'text-success'}>{total}</td>
        <td>{p.notes}</td>
        <td><Button size="sm" color="danger" onClick={() => this.remove(p)}>Delete</Button></td>
      </tr>
    });

    return (
      <div>
        <AppNavbar/>
        <Container fluid>
          {error}
          <h3>Your Points</h3>
          <Table>
            <thead>
            <tr>
              <th width="10%">Date</th>
              <th width="10%">Points</th>
              <th>Notes</th>
              <th width="10%">Actions</th>
            </tr>
            </thead>
            <tbody>
            {pointsList}
            </tbody>
          </Table>
          <PointsModal callback={this.refresh}/>
        </Container>
      </div>
    );
  }
})

This code starts with refresh() and remove() methods, which I'll get to in a moment. The important part happens in componentDidMount(), where the access token is added in an Authorization header, and the user's information is stuffed in an x-forwarded-user header. An ApolloClient is created with this information, a cache is added, and the connectToDevTools flag is turned on. This can be useful for debugging with Apollo Client Developer Tools.

componentDidMount() {
  const authLink = setContext(async (_, {headers}) => {
    const token = await this.props.auth.getAccessToken();

    // return the headers to the context so httpLink can read them
    return {
      headers: {
        ...headers,
        authorization: token ? `Bearer ${token}` : '',
        'x-forwarded-user': user ? JSON.stringify(user) : ''
      }
    }
  });

  this.client = new ApolloClient({
    link: authLink.concat(httpLink),
    cache: new InMemoryCache(),
    connectToDevTools: true
  });

  // this.client.query(...);
}

Authentication with Apollo Client requires a few new dependencies. Install these now.

npm apollo-link-context@1.0.8 apollo-link-http@1.5.4

In the JSX of the page, there is a delete button that calls the remove() method in Points. There's also 'component. This is referenced for each item, as well as at the bottom. You'll notice both of these reference therefresh()` method, which updates the list.

<PointsModal item={p} callback={this.refresh}/>
<PointsModal callback={this.refresh}/>

This component renders a link to edit a component, or an Add button when no item is set.

Create src/PointsModal.js and add the following code to it.

import React, { Component } from 'react';
import { Button, Form, FormGroup, Input, Label, Modal, ModalBody, ModalFooter, ModalHeader } from 'reactstrap';
import { withAuth } from '@okta/okta-react';
import { httpLink } from './Points';
import { ApolloClient } from 'apollo-client';
import { setContext } from 'apollo-link-context';
import { InMemoryCache } from 'apollo-cache-inmemory';
import gql from 'graphql-tag';
import { Link } from 'react-router-dom';

export default withAuth(class PointsModal extends Component {
  client;
  emptyItem = {
    date: (new Date()).toISOString().split('T')[0],
    exercise: 1,
    diet: 1,
    alcohol: 1,
    notes: ''
  };

  constructor(props) {
    super(props);
    this.state = {
      modal: false,
      item: this.emptyItem
    };

    this.toggle = this.toggle.bind(this);
    this.handleChange = this.handleChange.bind(this);
    this.handleSubmit = this.handleSubmit.bind(this);
  }

  componentDidMount() {
    if (this.props.item) {
      this.setState({item: this.props.item})
    }

    const authLink = setContext(async (_, {headers}) => {
      const token = await this.props.auth.getAccessToken();
      const user = await this.props.auth.getUser();

      // return the headers to the context so httpLink can read them
      return {
        headers: {
          ...headers,
          authorization: token ? `Bearer ${token}` : '',
          'x-forwarded-user': JSON.stringify(user)
        }
      }
    });

    this.client = new ApolloClient({
      link: authLink.concat(httpLink),
      cache: new InMemoryCache()
    });
  }

  toggle() {
    if (this.state.modal && !this.state.item.id) {
      this.setState({item: this.emptyItem});
    }
    this.setState({modal: !this.state.modal});
  }

  render() {
    const {item} = this.state;
    const opener = item.id ? <Link onClick={this.toggle} to="#">{this.props.item.date}</Link> :
      <Button color="primary" onClick={this.toggle}>Add Points</Button>;

    return (
      <div>
        {opener}
        <Modal isOpen={this.state.modal} toggle={this.toggle}>
          <ModalHeader toggle={this.toggle}>{(item.id ? 'Edit' : 'Add')} Points</ModalHeader>
          <ModalBody>
            <Form onSubmit={this.handleSubmit}>
              <FormGroup>
                <Label for="date">Date</Label>
                <Input type="date" name="date" id="date" value={item.date}
                       onChange={this.handleChange}/>
              </FormGroup>
              <FormGroup check>
                <Label check>
                  <Input type="checkbox" name="exercise" id="exercise" checked={item.exercise}
                         onChange={this.handleChange}/>{' '}
                  Did you exercise?
                </Label>
              </FormGroup>
              <FormGroup check>
                <Label check>
                  <Input type="checkbox" name="diet" id="diet" checked={item.diet}
                         onChange={this.handleChange}/>{' '}
                  Did you eat well?
                </Label>
              </FormGroup>
              <FormGroup check>
                <Label check>
                  <Input type="checkbox" name="alcohol" id="alcohol" checked={item.alcohol}
                         onChange={this.handleChange}/>{' '}
                  Did you drink responsibly?
                </Label>
              </FormGroup>
              <FormGroup>
                <Label for="notes">Notes</Label>
                <Input type="textarea" name="notes" id="notes" value={item.notes}
                       onChange={this.handleChange}/>
              </FormGroup>
            </Form>
          </ModalBody>
          <ModalFooter>
            <Button color="primary" onClick={this.handleSubmit}>Save</Button>{' '}
            <Button color="secondary" onClick={this.toggle}>Cancel</Button>
          </ModalFooter>
        </Modal>
      </div>
    )
  };

  handleChange(event) {
    const target = event.target;
    const value = target.type === 'checkbox' ? (target.checked ? 1 : 0) : target.value;
    const name = target.name;
    let item = {...this.state.item};
    item[name] = value;
    this.setState({item});
  }

  handleSubmit(event) {
    event.preventDefault();
    const {item} = this.state;

    const updatePoints = gql`
      mutation pointsSave($id: Int, $date: Date, $exercise: Int, $diet: Int, $alcohol: Int, $notes: String) {
        pointsSave(id: $id, date: $date, exercise: $exercise, diet: $diet, alcohol: $alcohol, notes: $notes) {
          id date
        }
      }`;

    this.client.mutate({
      mutation: updatePoints,
      variables: {
        id: item.id,
        date: item.date,
        exercise: item.exercise,
        diet: item.diet,
        alcohol: item.alcohol,
        notes: item.notes
      }
    }).then(result => {
      let newItem = {...item};
      newItem.id = result.data.pointsSave.id;
      this.props.callback(newItem);
      this.toggle();
    });
  }
});

Make sure your GraphQL backend is started, then start the React frontend with npm start. The text squishes up against the top navbar, so add some padding by adding a rule in src/index.css.

.container-fluid {
  padding-top: 10px;
}

You should see the Home component and a button to log in.

Click Login and you'll be prompted to enter your Okta credentials.

And then you'll be logged in!

Click Manage Points to see the points list.

It's cool to see everything working, isn't it?! :D

Your React frontend is secured, but your API is still wide open. Let's fix that.

Get User Information from JWTs

Navigate to your graphql-api project in a terminal window and install Okta's JWT Verifier.

npm i @okta/jwt-verifier@0.0.12

Create graphql-api/src/CurrentUser.ts to hold the current user's information.

export class CurrentUser {
  constructor(public id: string, public firstName: string, public lastName: string) {}
}

Import OktaJwtVerifier and CurrentUser in graphql-api/src/index.ts and configure the JWT verifier to use your OIDC app's settings.

import * as OktaJwtVerifier from '@okta/jwt-verifier';
import { CurrentUser } from './CurrentUser';

const oktaJwtVerifier = new OktaJwtVerifier({
  clientId: '{yourClientId},
  issuer: 'https://{yourOktaDomain}.com/oauth2/default'
});

In the bootstrap configuration, define setupContainer to require an authorization header and set the current user from the x-forwarded-user header.

bootstrap({
  …
  cors: true,
  setupContainer: async (container, action) => {
    const request = action.request;
    // require every request to have an authorization header
    if (!request.headers.authorization) {
      throw Error('Authorization header is required!');
    }
    let parts = request.headers.authorization.trim().split(' ');
    let accessToken = parts.pop();
    await oktaJwtVerifier.verifyAccessToken(accessToken)
      .then(async jwt => {
        const user = JSON.parse(request.headers['x-forwarded-user'].toString());
        const currentUser = new CurrentUser(jwt.claims.uid, user.given_name, user.family_name);
        container.set(CurrentUser, currentUser);
      })
      .catch(error => {
        throw Error('JWT Validation failed!');
      })
  }
  ...
});

Modify graphql-api/src/controller/PointsController.ts to inject the CurrentUser as a dependency. While you're in there, adjust the points() method to filter by user ID and modify pointsSave() to set the user when saving.

import { Controller, Mutation, Query } from 'vesper';
import { EntityManager } from 'typeorm';
import { Points } from '../entity/Points';
import { User } from '../entity/User';
import { CurrentUser } from '../CurrentUser';

@Controller()
export class PointsController {

  constructor(private entityManager: EntityManager, private currentUser: CurrentUser) {
  }

  // serves "points: [Points]" requests
  @Query()
  points() {
    return this.entityManager.getRepository(Points).createQueryBuilder("points")
      .innerJoin("points.user", "user", "user.id = :id", { id: this.currentUser.id })
      .getMany();
  }

  // serves "pointsGet(id: Int): Points" requests
  @Query()
  pointsGet({id}) {
    return this.entityManager.findOne(Points, id);
  }

  // serves "pointsSave(id: Int, date: Date, exercise: Int, diet: Int, alcohol: Int, notes: String): Points" requests
  @Mutation()
  pointsSave(args) {
    // add current user to points saved
    if (this.currentUser) {
      const user = new User();
      user.id = this.currentUser.id;
      user.firstName = this.currentUser.firstName;
      user.lastName = this.currentUser.lastName;
      args.user = user;
    }

    const points = this.entityManager.create(Points, args);
    return this.entityManager.save(Points, points);
  }

  // serves "pointsDelete(id: Int): Boolean" requests
  @Mutation()
  async pointsDelete({id}) {
    await this.entityManager.remove(Points, {id: id});
    return true;
  }
}

Restart the API, and you should be off to the races!

Source Code

You can find the source code for this article at https://github.com/oktadeveloper/okta-electron-react-example.

Learn More About React, Node, and User Authentication

This article showed you how to build a secure React app with GraphQL, TypeORM, and Node/Vesper. I hope you enjoyed the experience!

At Okta, we care about making authentication with React and Node easy to implement. We have several blog posts on the topic, and documentation too! I encourage you to check out the following links:

• Build User Registration with Node, React, and Okta
• Build a React Application with User Authentication in 15 Minutes
• Build a React Native App and Authenticate with OAuth 2.0
• Add Okta Authentication to Your React app
• Build a Basic CRUD App with Vue.js and Node

I hope you have an excellent experience building apps with React and GraphQL. If you have any questions, please hit me up on Twitter or my whole kick-ass team on @oktadev. Our DMs are wide open! :)

MariaDB DBAs are accountable for auditing database infrastructure operations to proactively troubleshoot performance and operational issues, MariaDB Audit Plugin is capable of auditing the database operations of both MariaDB and MySQL. MariaDB Audit Plugin is provided as a dynamic library: server_audit.so (server_audit.dll for Windows).  The plugin must be located in the plugin directory, the directory containing all plugin libraries for MariaDB.

MariaDB [(none)]> select @@plugin_dir; 
+--------------------------+
| @@plugin_dir             |
+--------------------------+
| /usr/lib64/mysql/plugin/ |
+--------------------------+
1 row in set (0.000 sec)

There are two ways you can install MariaDB Audit Plugin:

INSTALL SONAME statement while logged into MariaDB, You need to use administrative account which has INSERT privilege for the mysql.plugin table.

MariaDB [(none)]> INSTALL SONAME 'server_audit';
Query OK, 0 rows affected (0.012 sec)

MariaDB [(none)]>

Load Plugin at Start-Up 

The plugin can be loaded by setting -plugin_load system variable in my.cnf (my.ini in windows)

[mysqld]
#
# include all files from the config directory
#
!includedir /etc/my.cnf.d

plugin_load=server_audit=server_audit.so

System variables to configure MariaDB Audit Plugin

MariaDB Audit Plugin is highly configurable, Please fine below the system variables available for MariaDB Audit Plugin:

MariaDB [(none)]> SHOW GLOBAL VARIABLES LIKE '%server_audit%';
+-------------------------------+-----------------------+
| Variable_name                 | Value                 |
+-------------------------------+-----------------------+
| server_audit_events           |                       |
| server_audit_excl_users       |                       |
| server_audit_file_path        | server_audit.log      |
| server_audit_file_rotate_now  | OFF                   |
| server_audit_file_rotate_size | 1000000               |
| server_audit_file_rotations   | 9                     |
| server_audit_incl_users       |                       |
| server_audit_logging          | OFF                   |
| server_audit_mode             | 0                     |
| server_audit_output_type      | file                  |
| server_audit_query_log_limit  | 1024                  |
| server_audit_syslog_facility  | LOG_USER              |
| server_audit_syslog_ident     | mysql-server_auditing |
| server_audit_syslog_info      |                       |
| server_audit_syslog_priority  | LOG_INFO              |
+-------------------------------+-----------------------+
15 rows in set (0.001 sec)

configure system variable server_audit_events for auditing MariaDB transaction events:

MariaDB [(none)]> SET GLOBAL server_audit_events = 'CONNECT,QUERY,TABLE';
Query OK, 0 rows affected (0.008 sec)

Enable MariaDB Audit Plugin

MariaDB [(none)]> set global server_audit_logging=on;
Query OK, 0 rows affected (0.007 sec)

MariaDB Audit Plugin creates audit log file “server_audit.log” on path  /var/lib/mysql/ 

Testing MariaDB Audit Plugin 

MariaDB [employees]> update employees
    -> set last_name='Gupta'
    -> where emp_no= 499999;
Query OK, 1 row affected (0.010 sec)
Rows matched: 1  Changed: 1  Warnings: 0

[root@localhost mysql]# tail -f  server_audit.log
20180612 20:32:07,localhost.localdomain,root,localhost,16,433,QUERY,,'SHOW GLOBAL VARIABLES LIKE \'%server_audit%\'',0
20180612 20:32:26,localhost.localdomain,root,localhost,16,434,QUERY,,'update employees set last_name=\'Gupta\' where emp_no= 499999',1046
20180612 20:32:37,localhost.localdomain,root,localhost,16,435,QUERY,,'SELECT DATABASE()',0
20180612 20:32:37,localhost.localdomain,root,localhost,16,437,QUERY,employees,'show databases',0
20180612 20:32:37,localhost.localdomain,root,localhost,16,438,QUERY,employees,'show tables',0
20180612 20:32:41,localhost.localdomain,root,localhost,16,447,WRITE,employees,employees,
20180612 20:32:41,localhost.localdomain,root,localhost,16,447,READ,employees,dept_emp,
20180612 20:32:41,localhost.localdomain,root,localhost,16,447,READ,employees,dept_manager,
20180612 20:32:41,localhost.localdomain,root,localhost,16,447,QUERY,employees,'update employees set last_name=\'Gupta\' where emp_no= 499999',0

How can we block UNINSTALL PLUGIN ?

The INSTALL PLUGIN statement can be used to uninstall a plugin but you can disable this by adding following line in my.cnf after plugin is loaded once:

[mysqld]
#
# include all files from the config directory
#
!includedir /etc/my.cnf.d

plugin_load=server_audit=server_audit.so
server_audit=FORCE_PLUS_PERMANENT

The post MariaDB Audit Plugin appeared first on MySQL Consulting, Support and Remote DBA Services By MinervaDB.