MySQL Audit Logging — How to Avoid Data Overload

MySQL Audit Logging — How to Avoid Data Overload

In the last few months, our solution engineers have had many discussions with security and operation teams about how to implement MySQL auditing. In some cases, these teams have been asked to implement new database auditing in response to a security breach or to meet a regulatory requirement (PCI, HIPPA, GDPR or CCPA).

More often, however, these companies are switching audit strategies because their existing method has led to data overload. They typically start by monitoring everything, sometimes even using the general query log. They quickly learn that even with moderate database activity, the amount of generated data soon becomes overwhelming.

Below are a couple considerations for selecting your audit strategy.

1. Review your schema and identify data that *must* be audited. It’s likely your company’s personal identifiable information (PII) is only in a few tables. MySQL Workbench has a great reverse engineer feature to create an ER Diagram, useful for double-checking where your sensitive data might reside.

#2. Choose an audit plugin that has the following capabilities:

• audit log encryption
• log rotation
• log compression
• fine-grained filtering capabilities

While there are several audit plugins available, I’ll review the options available with MySQL Enterprise Audit.

MySQL Enterprise Audit can log very detailed information about who is connecting to MySQL and what they are doing.

Equally important, is its capability to set very fine-grained filters. Here are some common requests for which we’ve helped to build filters:

• audit specific users, tables or databases
• audit access denied errors (see who may be searching for access)
• audit failed connections
• audit non-local subnets
• audit insecure connections

Additionally, these filters are “stackable”. You can apply multiple filters to a single user as well as different filters to different users. For example, you may want to log connects and disconnects for anyone outside of your domain but log any activity related to sensitive tables.

I’ll conclude by offering three sample filters: 1) a verbose filter that shows you all the options you can turn on and off, 2) a filter to only log access to a specific table and 3) a filter that will monitor failed connections. If you’d like to try these out, download an evalution version of MySQL Enterprise ( Download from Oracle eDelivery ). Refer to the documentation for setup ( https://dev.mysql.com/doc/refman/8.0/en/audit-log.htm) or the the following blog ( http://dasini.net/blog/2018/04/04/mysql-security-mysql-enterprise-audit/ ).

Sample #1: Verbose filter to show true/false options at multiple levels. The below filter has general logging set to true.

# Change any option to true to enable logging.  Useful for testing 
# options.  More examples in documentation 
SET @f='
{
  "filter": {
    "log": false,
    "class": [
      {
        "name": "connection",
        "event": [
          { "name": "connect", "log": false},
          { "name": "disconnect", "log": false }
        ]
      },
      { 
        "name": "general",
        "event": { "name": "status", "log": true}
      },
   {
        "name": "table_access",
        "event":[
            { "name": "insert", "log":false },
            { "name": "delete", "log":false },
            { "name": "update", "log":false },
            { "name": "read", "log":false }
        ]
    }
 ]
 }
}';

Sample #2: This filter logs table access for a specific table.

SET @f='                                                       
{
   "filter": {
    "class":
  {
     "name": "table_access",
     "event":
    {
      "name": [ "insert", "update", "delete", "read" ],
      "log": {
       "and": [ { "field": { "name": "table_database.str",
                "value": "employees" } },
                { "field": { "name": "table_name.str",
                "value": "salaries" }]
       }
     }
   }
 }
}';

Sample #3. Finally, this filter will log details for anyone whose connection was denied.

SET @f='
{
  "filter": {
    "log": false,
    "class": {
        "name": "connection",
        "event": [
         { "name": "connect", "log" : 
         { "not": { "field": { "name": "status", "value": 0 } } } },
         { "name": "disconnect", "log": false }
        ]
    }
  } 
}';

I’ll try to post other useful filters as they arise. Fortunately, The MySQL Enterprise Edition, which provides access to the audit plugin, also includes access to MySQL Support engineers who can help you construct filters as well.