MySQL replication failure

YellowHummingbird

Hello

Recently the replication on my MySQL servers appears to have broken for no reason I can tell.

In the mySQL log, I can see the following:

[ERROR] Slave I/O: error connecting to master 'slave_user@[host]:3306' - retry-time: 60 retries: 86400, Error_code: 2003

150408 20:01:37 [Note] Error reading relay log event: slave SQL thread was killed

150408 20:01:37 [Note] Slave I/O thread killed while connecting to master

150408 20:01:37 [Note] Slave I/O thread exiting, read up to log 'mysql-bin.000057', position 129754

150408 20:01:38 [Note] Slave SQL thread initialized, starting replication in log 'mysql-bin.000057' at position 129

Doing SHOW SLAVE STATUS\G shows that status as:

Slave_IO_State: Connecting to master

Do you have any tips or advice? I've tried the following:

• Verified that the host is reachable via ping from the slave
• Restarted mysql, restarted the slave.

PrincessOfCats

Can you login using slave_user on the master with the set password?

I.e.

mysql -uslave_user -p -h [host]

YellowHummingbird

@StartledPhoenix said:
Can you login using slave_user on the master with the set password?

I.e.

mysql -uslave_user -p -h [host]

Thanks for the suggestion It appears to just hang after entering the password. I'm guessing there might be some kind of firewall issue, but I really haven't changed anything at all.

PrincessOfCats

@YellowHummingbird said:
Thanks for the suggestion It appears to just hang after entering the password. I'm guessing there might be some kind of firewall issue, but I really haven't changed anything at all.

tcpdump time

Maybe some telnet as well

YellowHummingbird

@StartledPhoenix said:
Maybe some telnet as well

Yay, fixed it! Thanks for your help. Turns out, I'd saved my iptables rules into a file, and blocked all IP's except for those that were running services that needed the databases. I'd got a new server, and added the rule, but not added it to the file. Hence, when I rebooted for a kernel update, the old rules got added (so all the services I was monitoring carried on working) but the replication stopped.

Thanks once again, and have an amazing day.

If a mod sees this and wants to lock it, that's fine by me

PrincessOfCats

np, glad I could help

Btw, for no rebooting, try using KSplice or KernelCare, will make it so that you don't have to reboot for security patches.