Big part of the content on the Internet is stored on databases for which MySQL is a popular choice. But what to do when suddenly your dynamic content doesn’t load, or when returning to your website you are greeted by a nearly empty white page with message “Error establishing a database connection”. This guide is aimed at helping with troubleshooting MySQL on Cloud Servers, and by following the steps listed here you’ll hopefully be able to restore your database functionality.

Check that the service is running

If your website cannot connect to your database, it is possible the service is simply not listening. Check your MySQL state, on Ubuntu and Debian systems this can be done with

sudo service mysql status

CentOS and other Red Hat variants use MySQL as well, but it is name MariaDB instead, so use this following command

sudo service mariadb status

The output from the status check on CentOS and Debian will show something along the lines of this example from CentOS below, Debian output would be nearly identical with the exception of different service name.

mariadb.service - MariaDB database server
    Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled)
    Active: active (running) since Wed 2015-08-05 11:53:38 EEST; 3h 23min ago
  Main PID: 2451 (mysqld_safe)
    CGroup: /system.slice/mariadb.service
            ├─2451 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
            └─2609 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql...

The printout is rather verbose, but the important part is usually coloured to stand out better. In green ‘active (running)’ means the service should be running normally, if instead it says ‘active (exited)’ or ‘inactive (dead)’ the process has been stopped or killed.

Ubuntu condenses the same information to a one liner like the example output underneath.

mysql start/running, process 5897

If your service status says something other than ‘running’, try to restart the process using the same service command as before but with ‘restart’ instead of ‘status’.

sudo service mysql restart

sudo service mariadb restart

Should the database service restart without encountering errors, you can try to connect to it using the command below. Enter the root password when prompted.

mysql -u root -p

If you are greeted with “Welcome to the MySQL/MariaDB monitor” the connection was successful and the database service is running. If instead you get an error like this example below you probably mistyped the password for root user. Try again, or if you are not sure about the root password, log in with an other user account you do know the access to by just replacing root with an other username.

ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

If you have your database set up on a separate server from your webhost, make sure the two servers can reach each other. You can test the database connection from your web server with the command underneath using the correct username for your installation.

mysql -u <user name> -p -h <database server private IP>

Check the configuration

When MySQL is running but your website still doesn’t load as it should, or if when attempting to connect to your database manually you get an error message like the one below, you should take a look at the service configuration.

ERROR 2002: Can't connect to local MySQL server through socket '/tmp/mysql.sock' (111)

On Debian and Ubuntu servers the configuration fine for MySQL is usually saved at /etc/mysql/. It’s also possible to have user specific settings stored at /home/<user>/.my.cnf, which would override the global configurations. Check if any user level overrides have been set. It is commonly advice to have separate usernames for different web applications, so check at least those relevant to your page loading issues. You can open the global configuration file with first of the following two commands below, and the user specific with the latter by replacing the <user> with a database username.

sudo nano /etc/mysql/my.cnf

sudo nano /home/<user>/.my.cnf

By scrolling down past [client] and [mysqld_safe] settings you’ll find something like the example here.

[mysqld]
#
# * Basic Settings
#
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
lc-messages-dir = /usr/share/mysql
skip-external-locking
#
# Instead of skip-networking the default is now to listen only on
# localhost which is more compatible and is not less secure.
bind-address            = 127.0.0.1

With CentOS and other Red Hats the primary configuration file is stored at slightly different location, open it for inspection with

sudo vi /etc/my.cnf
[mysqld]
datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock

The lines here to pay close attention to are ‘socket’, ‘datadir’ and ‘bind-address’. The parameters in the example above are in their default values, and in most cases your configuration would look the same. Make sure the settings point to the correct directories so that MySQL can actually find the required files. The easies way to check the ‘datadir’ is to use this command below

sudo ls -l /var/lib/mysql/

The output will list all files in that directory, it should contain at least the following plus any databases you have created.

drwx------ 2 mysql root      4096 Aug  5 12:23 mysql
drwx------ 2 mysql mysql     4096 Aug  5 12:29 performance_schema

If the data directory or socket has been moved and MySQL doesn’t know where they are, fix the configuration file to point to the correct directories. You can search for the folders with the following command.

sudo find / -name performance_schema  && sudo find / -name mysql.sock

The third parameter you’ll need to check is the bind-address, this is only really relevant if your database needs to be accessed remotely. In Debian and Ubuntu installations the bind is by default set to the loopback address, which prevents database calls from outside the local host. CentOS doesn’t have the same parameter unless manually set. For any setup where your web service is on a different server to the database this bind-address should be set to the server’s own private IP.

Check the error logs

If the configuration seems correct and the service is running, but your website still doesn’t load as it should, try checking the logs for any hints to as what might be the cause.

Debian and Ubuntu servers store error logs to /var/log/mysql/error.log. You can read though the logs with ‘less’, but this might not be very convenient as the log includes more than just critical errors. Instead search the logs using ‘grep’ with

sudo grep -i error /var/log/mysql/error.log

Should you not be able to find anything within the most recent logs, check the archived ones as well. To do this, use ‘zgrep’ with otherwise the same command as regular ‘grep’

sudo zgrep -i error /var/log/mysql/error.log.1.gz

Since the database under CentOS is named MariaDB instead of MySQL, the logs are also saved under different name. You can search the logs with the following command

sudo grep -i error /var/log/mariadb/mariadb.log

Debian systems also report MySQL events to /var/log/syslog, to filter out everything else, use ‘grep’ with two keywords separated by .* to express ‘and’ like in the command below

sudo grep -i -E 'mysql.*error' /var/log/syslog

If you are having difficulties finding anything helpful, try different keywords such as ‘start’ to see when the service was last restarted, or ‘failed’ to find any less critical problems that might not be reported as errors.

Ask for help

Optimistically by now your database should be up and running again, but in case you’ve encountered a more persistent error, feel free to ask for help. To contact UpCloud Support, please follow the instructions in your UpCloud Control Panel, under My Account and Support –tab. Don’t forget to explain the problem to the best of your ability, also include the steps you’ve taken with their results while troubleshooting the issue, it will help others in helping with your problem.