Power Outages Suck Balls. Mysql Database Problems.
I came home today to find that my computers were powered off. All my clocks were blinking. Power outages suck balls.
I don’t have a UPS but I am running a test web site on my local machine that has been getting some nice traffic lately so I should probably invest. After starting everything up I tried running one of my php scripts that access my mysql database. Error.
Warning: mysql_connect(): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) in /home/ctaylor/scripts/webcrawl.php on line 8
Why isn’t my database running? I have seen this before. Possible database corruption? I prayed that this wasn’t the case and executed:
codytaylor@desktop:~$ sudo /etc/init.d/mysql start
* Starting MySQL database server mysqld [fail]
codytaylor@desktop:~$ sudo /etc/init.d/mysql restart
* Stopping MySQL database server mysqld [ OK ]
* Starting MySQL database server mysqld [fail]
Awesome. So now what?
Hoping to get a handle on what exactly is causing it not to start I first checked /var/log/mysql.err and /var/log/mysql.log.
They were both empty. Useful right?
I then did decided to check syslog. I didn’t find anything at first but then I followed syslog while trying to restart mysql.
codytaylor@desktop:/var/log$ tail -f syslog
Apr 24 19:59:54 desktop mysqld_safe[8173]: started
Apr 24 19:59:54 desktop mysqld[8176]: 090424 19:59:54 [Warning] Can't create test file /var/lib/mysql/desktop.lower-test
Apr 24 19:59:54 desktop mysqld[8176]: 090424 19:59:54 [Warning] Can't create test file /var/lib/mysql/desktop.lower-test
Apr 24 19:59:54 desktop mysqld[8176]: 090424 19:59:54 [Warning] One can only use the --user switch if running as root
Apr 24 19:59:54 desktop mysqld[8176]:
Apr 24 19:59:54 desktop mysqld[8176]: 090424 19:59:54 InnoDB: Operating system error number 13 in a file operation.
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: The error means mysqld does not have the access rights to
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: the directory.
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: File name ./ibdata1
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: File operation call: 'open'.
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: Cannot continue operation.
Apr 24 19:59:54 desktop mysqld_safe[8183]: ended
Apr 24 20:00:08 desktop /etc/init.d/mysql[8336]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
Apr 24 20:00:08 desktop /etc/init.d/mysql[8336]: Could not open required defaults file: /etc/mysql/debian.cnf
Apr 24 20:00:08 desktop /etc/init.d/mysql[8336]: Fatal error in defaults handling. Program aborted
Apr 24 20:00:08 desktop /etc/init.d/mysql[8336]:
I executed the restart of mysql as root didn’t I? All the folders and files are owned by the mysql user. WTF?
So it’s saying that first it can’t create a test file /var/lib/mysql/dektop.lower-test and then that it can’t open ./ibdata.
My first instinct was to set everything to 777 and just go get a beer. But probably not a good idea.
So obviously this is a permissions error. But seeing as the mysql user owns everything in this folder it’s a little confusing.
I then executed:
codytaylor@desktop:/var/lib/mysql$ sudo chmod -R 755 ./*
I tried starting mysql again. Same error.
I got frustrated and said screw it for the night. This morning while I was trying to get rid of the hangover I tried again to start mysql again and I got a totally different error in my syslog.
Apr 25 20:49:48 desktop mysqld_safe[31095]: started
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48 InnoDB: Started; log sequence number 0 60274
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48 [ERROR] Can't start server: Bind on TCP/IP port: Cannot assign requested address
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48 [ERROR] Do you already have another mysqld server running on port: 3306 ?
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48 [ERROR] Aborting
Apr 25 20:49:48 desktop mysqld[31098]:
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48 InnoDB: Starting shutdown...
Apr 25 20:49:50 desktop mysqld[31098]: 090425 20:49:50 InnoDB: Shutdown completed; log sequence number 0 60274
Apr 25 20:49:50 desktop mysqld[31098]: 090425 20:49:50 [Note] /usr/sbin/mysqld: Shutdown complete
Apr 25 20:49:50 desktop mysqld[31098]:
Apr 25 20:49:50 desktop mysqld_safe[31119]: ended
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: ^G/usr/bin/mysqladmin: connect to server at 'localhost' failed
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]:
All of a sudden it’s only a problem with the bind address? What happened to the permissions error? I didn’t reboot or anything.
Easy enough to fix though.
I open up the my.cnf file which resides in /etc/mysql/.
I look for the bind-address and noticed it was set to 192.168.2.10 instead of my new ip 192.168.1.11.
DHCP assigned it a new ip address which sucks.
So I change the line to reflect the change and restart again. All is now well.
-Cody Taylor
