The general query log is a general record of what mysqld is doing. The server writes information to this log when clients connect or disconnect, and it logs each SQL statement received from clients. The general query log can be very useful when you suspect an error in a client and want to know exactly what the client sent to mysqld.

mysqld writes statements to the query log in the order that it receives them, which might differ from the order in which they are executed. This logging order contrasts to the binary log, for which statements are written after they are executed but before any locks are released. (Also, the query log contains all statements, whereas the binary log does not contain statements that only select data.)

To enable the general query log, start mysqld with the –log[=file_name] or -l [file_name] option.

If no file_name value is given for –log or -l, the default name is host_name.log in the data directory.

Server restarts and log flushing do not cause a new general query log file to be generated (although flushing closes and reopens it). On Unix, you can rename the file and create a new one by using the following commands:

shell> mv host_name.log host_name-old.log
shell> mysqladmin flush-logs
shell> cp host_name-old.log backup-directory
shell> rm host_name-old.log

Before 5.0.17, you cannot rename a log file on Windows while the server has it open. You must stop the server and rename the file, and then restart the server to create a new log file. As of 5.0.17, this applies only to the error log. However, a stop and restart can be avoided by using FLUSH LOGS, which causes the server to rename the error log with an -old suffix and open a new error log.

You may remember a previous post about MySQL replication.

I decided to make a revised post detailing the different steps required in order to implement a master / slave replication relationship within two or more MySQL servers.

The steps required are slightly different and I think its important to outline the necessary steps in order to accomplish this task — it may actually save you some troubleshooting!

Replication of Existing DBs

If you have existing data on your master that you want to synchronize on your slaves before starting the replication process, then you must stop processing statements on the master, obtain the current position, and then dump the data, before allowing the master to continue executing statements.

If you do not stop the execution of statements, the data dump and the master status information that you use will not match and you will end up with inconsistent or corrupted databases on the slaves.

PREPARATION OF MASTER SERVER

1. Select a master server. It can be either one.

2. Make sure all databases that you want to replicate to the slave already exist! The easist way is to just copy the database dirs inside your MySQL data directory intact over to your slave, and then recursively chown them to “mysql:mysql”. Remember, the binary structures are file-system dependant, so you can’t do this between MySQL servers on different OS’s. In this instance you will want to use mysqldump most likely.

3. Create /etc/my.cnf if you do not already have one:

[mysqld]
socket=/tmp/mysql.sock [enter YOUR path to mysql.sock here]
server-id=1
log-bin=mysql-bin
binlog-do-db=bossdb     # input the database which should be replicated
binlog-ignore-db=mysql1 # input the database that should be ignored for replication
binlog-ignore-db=mysql2  # input the database that should be ignored for replication

4. Permit your slave server to replicate by issuing the following SQL command (substituting your slave’s IP and preferred password):

mysql> GRANT SUPER,REPLICATION CLIENT,REPLICATION SLAVE,RELOAD ON *.* TO 'replicate'@'192.168.1.1' IDENTIFIED BY 'somepass';

5. Flush all talbes and block write statements :

mysql> FLUSH TABLES WITH READ LOCK;

6. Use the SHOW MASTER STATUS statement to determine the current binary log file name and offset on the master:

mysql > SHOW MASTER STATUS;
+---------------+----------+--------------+------------------+
| File          | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+---------------+----------+--------------+------------------+
| mysql-bin.003 | 73       | test         | manual,mysql     |
+---------------+----------+--------------+------------------+

Copy the file + position for use in Step 4 of the slave configuration.

7. Create data snapshot to import into slave with mysqldump :

shell> mysqldump -u root -p db_name --lock-all-tables >dbdump.sql

8. Unlock the tables of the database :

mysql> UNLOCK TABLES;

9. Transfer & import the db into the slave

10. Shut down and restart MySQL daemon and verify that all is functional.

PREPARATION OF SLAVE

1. Create /etc/my.cnf if you do not already have one:

[mysqld]
socket=/tmp/mysql.sock [enter YOUR path to mysql.sock here]
server-id=2 [MUST be different to master]
master-host=192.168.1.1
master-user=replicate
master-password=somepass

2. Shut down and restart MySQL on slave.

3. Log into mysql and stop slave :

mysql> stop slave;

4. Set the master configuration on the slave :

mysql> CHANGE MASTER TO
    ->     MASTER_HOST='master_host_name',
    ->     MASTER_USER='replication_user_name',
    ->     MASTER_PASSWORD='replication_password',
    ->     MASTER_LOG_FILE='recorded_log_file_name',
    ->     MASTER_LOG_POS=recorded_log_position;

3. Issue the following SQL command to check status:

mysql> show slave status\G;

Ensure that the following two fields are showing this :

Slave_IO_Running: Yes
Slave_SQL_Running: Yes

If not, try to issue the following command :

mysql> start slave;

This will manually start the slave process. Note that only updated tables and entries after the slave process has started will be sent from the master to the slave — it is not a differential replication.

TESTING

Just update some data on the master, and query that record on the slave. The update should be instantaneous.

Test creating a table on the master MySQL server database :

mysql> use replicateddb;
Database changed

mysql> CREATE TABLE example4( id INT NOT NULL AUTO_INCREMENT,  PRIMARY KEY(id),  name VARCHAR(30),   age INT);
Query OK, 0 rows affected (0.04 sec)

Once in a while something will happen to a server and the mysql database will get corrupted.

A specific instance comes to mind on one of our Cacti monitoring servers.

The /var partition filled up due to too many messages being sent to the root user in /var/spool/. This caused MySQL to crash as well since the cacti poller couldnt write to the poller_output table in MySQL.

The result was all graphs being blank within cacti.

In any case, a thorough analysis of the mysql database was in order and I decided to post this quick tutorial for performing quick / lengthy table checks for offline and online MySQL databases.

Stage 1: Checking your tables

Run:

or

if you have more time. Use the -s (silent) option to suppress unnecessary information.

If the mysqld server is stopped, you should use the –update-state option to tell myisamchk to mark the table as “checked.”

You have to repair only those tables for which myisamchk announces an error. For such tables, proceed to Stage 2.

If you get unexpected errors when checking (such as out of memory errors), or if myisamchk crashes, go to Stage 3.

Stage 2: Easy safe repair

First, try :

(-r -q means “quick recovery mode”).

This attempts to repair the index file without touching the data file. If the data file contains everything that it should and the delete links point at the correct locations within the data file, this should work, and the table is fixed. Start repairing the next table. Otherwise, use the following procedure:

1. Make a backup of the data file before continuing.

2. Use

(-r means “recovery mode”)

This removes incorrect rows and deleted rows from the data file and reconstructs the index file.

3. If the preceding step fails, use

Safe recovery mode uses an old recovery method that handles a few cases that regular recovery mode does not (but is slower).

Note: If you want a repair operation to go much faster, you should set the values of the sort_buffer_size and key_buffer_size variables each to about 25% of your available memory when running myisamchk.

If you get unexpected errors when repairing (such as out of memory errors), or if myisamchk crashes, go to Stage 3.

Stage 3: Difficult repair

You should reach this stage only if the first 16KB block in the index file is destroyed or contains incorrect information, or if the index file is missing. In this case, it is necessary to create a new index file. Do so as follows:

1. Move the data file to a safe place.

2. Use the table description file to create new (empty) data and index files:

   shell> mysql db_name
     mysql> SET AUTOCOMMIT=1;
     mysql> TRUNCATE TABLE tbl_name;
     mysql> quit

3. Copy the old data file back onto the newly created data file. (Do not just move the old file back onto the new file. You want to retain a copy in case something goes wrong.)

Go back to Stage 2. :

(This should not be an endless loop.)

You can also use the REPAIR TABLE tbl_name USE_FRM SQL statement, which performs the whole procedure automatically. There is also no possibility of unwanted interaction between a utility and the server, because the server does all the work when you use REPAIR TABLE. See Section 12.5.2.6, “REPAIR TABLE Syntax”.

Stage 4: Very difficult repair

You should reach this stage only if the .frm description file has also crashed. That should never happen, because the description file is not changed after the table is created:

1. Restore the description file from a backup and go back to Stage 3. You can also restore the index file and go back to Stage 2. In the latter case, you should start with

2. If you do not have a backup but know exactly how the table was created, create a copy of the table in another database. Remove the new data file, and then move the .frm description and .MYI index files from the other database to your crashed database. This gives you new description and index files, but leaves the .MYD data file alone. Go back to Stage 2 and attempt to reconstruct the index file.

Thats it!

When implementing redundancy as far as DNS is concerned, automated is always better. In a hosting environment, new zone files are constantly being created.

This need for a DNS master/slave implementation where new zone files are transferred between the master nameserver and the slave became apparent as operations grew and geographic DNS redundancy became apparent.

Obviously some commercial dns products provide this type of functionality out-of-the-box, but I will show you how to do this with a simple Bind DNS distribution.

I wrote this tutorial to help you, hopefully, to create an automated DNS slave / zone file transfer environment. Obviously you can create as many slave servers as you feel necessary.

MASTER Server

1. Edit /etc/named.conf and add the following to the options section where xx.xx.xx.xx is the ip of your slave server.:

2. Create a script with the following, where somedirectory is the directory on your SLAVE server to store the slave zones and where yy.yy.yy.yy is your MASTER server ip and somewwwdir is a directory browsable via http and finally someslavefile.conf is the output file to write you slave config:

3. Test the script to ensure it is writing out the appropriate format.

4. Run the script as any user with permission to write to an http visible directory via cron.

SLAVE SERVER

1. Transfer the rndc.key file from your master server to the slave :

2. Edit ns1rndc.key and change the name of the key definition.

3. Edit named.conf and add the following to the options section:

4. Append the following to the named.conf file:

5. Run the following commands

6. Create a script:

7. Add to root’s crontab

In the second slave script, you see that the transfer is done via wget. This can be replaced by many other more secure methods. If ssh based key authentication is employed, a simple scp or even rsync can be utilized to accomplish the actual zone transfer.

It is often necessary, when designing high availability environments to implement a database replication scenario with MySQL.

This simple how-to is intended to setup a simple master / slave relationship.

PREPARATION OF MASTER SERVER

1. Select a master server. It can be either one.

2. Make sure all databases that you want to replicate to the slave already exist! The easist way is to just copy the database dirs inside your MySQL data directory intact over to your slave, and then recursively chown them to “mysql:mysql”. Remember, the binary structures are file-system dependant, so you can’t do this between MySQL servers on different OS’s. In this instance you will want to use mysqldump most likely.

3. Create /etc/my.cnf if you do not already have one:

4. Permit your slave server to replicate by issuing the following SQL command (substituting your slave’s IP and preferred password):

5. Shut down and restart MySQL daemon and verify that all is functional.

PREPARATION OF SLAVE

1. Create /etc/my.cnf if you do not already have one:

2. Shut down and restart MySQL on slave.

3. Issue the following SQL command to check status:

Ensure that the following two fields are showing this :

If not, try to issue the following command :

This will manually start the slave process. Note that only updated tables and entries after the slave process has started will be sent from the master to the slave — it is not a differential replication.

TESTING

Just update some data on the master, and query that record on the slave. The update should be instantaneous.

Test creating a table on the master MySQL server database :

And check the database on the slave to ensure that the recently created table on the master was replicated properly.