Oct 27 2009

MySQL SUM() Doesn’t Play Well With Floats

I had to write some reports for some legacy software today and I was unpleasantly surprised with the results of my SQL queries. I was selecting dollar values and summing them to for the monthly spending of certain individuals. Easy enough right? I wrote a query something like this :


SELECT SUM(t.money_spent) as sum_of_spent,
c.customer_name 
from transactions t 
join customers c on t.customer_id=c.customer_id 
group by customer_name order by c.customer_name asc

I ended up getting numerical values that were 10 decimal places long with seemingly random numbers. After checking to make sure the database didn’t have any odd entries I stumbled on this bug report.
The ‘money_spent’ column had a data type of float, which is a waste, but I still don’t think that it should sum up incorrectly. When I select individual values I get proper two decimal results.
Apparently floats and doubles use floating point math, which deals with approximate values for numbers and can thus result in confusion like this. It seems that it isn’t really possible to store 0.1 in a column of type float. You can only store 0.00999999977648258. This behavior is a little silly but easily fixed by using the ROUND() function :

SELECT SUM(ROUND(t.money_spent)) as sum_of_money_spent,
c.customer_name from transactions t 
join customers c on t.customer_id=c.customer_id 
group by customer_name order by c.customer_name asc

Share

Sep 20 2009

Bash Shell Script Error. “bad interpreter: No such file or directory error”

Today I created a simple shell script and I was getting a few odd errors:


cody@taylor:/var/some_folder/server$ ./process_xml.sh
-bash: ./process_xml.sh: /bin/sh^M: bad interpreter: No such file or directory

I figured it was probably a permissions error or an issue with the shebang (#!/bin/sh) line. I tried removing the shebang line, changing it to use dash or bash explicitly, chmoding to 777 and still no luck and another odd error.


cody@taylor:/var/some_folder/server$ sh process_xml.sh
: not found.sh: 4:

I then checked the log file that the commands were supposed to be writing to and it was filled with ‘^M’ on every line break and the log name itself was followed by a ‘?’. Took a minute or two but I finally clued in that I wrote that script on a windows machine and then exported it to an ubuntu linux server via subversion. It was just a basic text format issue.

Under DOS (Windows/PC) the end of a line of text is signalled using the ASCII code sequence CarriageReturn,LineFeed. Alternately written as CR,LF or the bytes 0x0D,0x0A. On the Macintosh platform, only the CR character is used. Under UNIX, the opposite is true and only the LF character is used.

After a quick :


cody@taylor:/var/some_folder/server$ apt-get install tofrodos
cody@taylor:/var/some_folder/server$ dos2unix process_xml.sh

Everything worked fine.

Share

Apr 25 2009

Power Outages Suck Balls. Mysql Database Problems.

I came home today to find that my computers were powered off. All my clocks were blinking. Power outages suck balls.
I don’t have a UPS but I am running a test web site on my local machine that has been getting some nice traffic lately so I should probably invest. After starting everything up I tried running one of my php scripts that access my mysql database. Error.


Warning: mysql_connect(): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) in /home/ctaylor/scripts/webcrawl.php on line 8

Why isn’t my database running? I have seen this before. Possible database corruption? I prayed that this wasn’t the case and executed:


codytaylor@desktop:~$ sudo /etc/init.d/mysql start
 * Starting MySQL database server mysqld                                                                 [fail] 
codytaylor@desktop:~$ sudo /etc/init.d/mysql restart
 * Stopping MySQL database server mysqld                                                                [ OK ] 
 * Starting MySQL database server mysqld                                                                 [fail] 

Awesome. So now what?
Hoping to get a handle on what exactly is causing it not to start I first checked /var/log/mysql.err and /var/log/mysql.log.
They were both empty. Useful right?
I then did decided to check syslog. I didn’t find anything at first but then I followed syslog while trying to restart mysql.


codytaylor@desktop:/var/log$ tail -f syslog
Apr 24 19:59:54 desktop mysqld_safe[8173]: started
Apr 24 19:59:54 desktop mysqld[8176]: 090424 19:59:54 [Warning] Can't create test file /var/lib/mysql/desktop.lower-test
Apr 24 19:59:54 desktop mysqld[8176]: 090424 19:59:54 [Warning] Can't create test file /var/lib/mysql/desktop.lower-test
Apr 24 19:59:54 desktop mysqld[8176]: 090424 19:59:54 [Warning] One can only use the --user switch if running as root
Apr 24 19:59:54 desktop mysqld[8176]: 
Apr 24 19:59:54 desktop mysqld[8176]: 090424 19:59:54  InnoDB: Operating system error number 13 in a file operation.
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: The error means mysqld does not have the access rights to
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: the directory.
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: File name ./ibdata1
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: File operation call: 'open'.
Apr 24 19:59:54 desktop mysqld[8176]: InnoDB: Cannot continue operation.
Apr 24 19:59:54 desktop mysqld_safe[8183]: ended
Apr 24 20:00:08 desktop /etc/init.d/mysql[8336]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
Apr 24 20:00:08 desktop /etc/init.d/mysql[8336]: Could not open required defaults file: /etc/mysql/debian.cnf
Apr 24 20:00:08 desktop /etc/init.d/mysql[8336]: Fatal error in defaults handling. Program aborted
Apr 24 20:00:08 desktop /etc/init.d/mysql[8336]: 

I executed the restart of mysql as root didn’t I? All the folders and files are owned by the mysql user. WTF?
So it’s saying that first it can’t create a test file /var/lib/mysql/dektop.lower-test and then that it can’t open ./ibdata.
My first instinct was to set everything to 777 and just go get a beer. But probably not a good idea.
So obviously this is a permissions error. But seeing as the mysql user owns everything in this folder it’s a little confusing.
I then executed:


codytaylor@desktop:/var/lib/mysql$ sudo chmod -R 755 ./*

I tried starting mysql again. Same error.
I got frustrated and said screw it for the night. This morning while I was trying to get rid of the hangover I tried again to start mysql again and I got a totally different error in my syslog.


Apr 25 20:49:48 desktop mysqld_safe[31095]: started
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48  InnoDB: Started; log sequence number 0 60274
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48 [ERROR] Can't start server: Bind on TCP/IP port: Cannot assign requested address
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48 [ERROR] Do you already have another mysqld server running on port: 3306 ?
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48 [ERROR] Aborting
Apr 25 20:49:48 desktop mysqld[31098]: 
Apr 25 20:49:48 desktop mysqld[31098]: 090425 20:49:48  InnoDB: Starting shutdown...
Apr 25 20:49:50 desktop mysqld[31098]: 090425 20:49:50  InnoDB: Shutdown completed; log sequence number 0 60274
Apr 25 20:49:50 desktop mysqld[31098]: 090425 20:49:50 [Note] /usr/sbin/mysqld: Shutdown complete
Apr 25 20:49:50 desktop mysqld[31098]: 
Apr 25 20:49:50 desktop mysqld_safe[31119]: ended
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: 0 processes alive and '/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf ping' resulted in
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: ^G/usr/bin/mysqladmin: connect to server at 'localhost' failed
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: error: 'Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)'
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: Check that mysqld is running and that the socket: '/var/run/mysqld/mysqld.sock' exists!
Apr 25 20:50:02 desktop /etc/init.d/mysql[31263]: 

All of a sudden it’s only a problem with the bind address? What happened to the permissions error? I didn’t reboot or anything.
Easy enough to fix though.
I open up the my.cnf file which resides in /etc/mysql/.
I look for the bind-address and noticed it was set to 192.168.2.10 instead of my new ip 192.168.1.11.
DHCP assigned it a new ip address which sucks.
So I change the line to reflect the change and restart again. All is now well.
-Cody Taylor

Share

Apr 23 2009

Unable to load dynamic library php_pgsql.dll

I was installing a postgres database to work with apache from uniform server. It was very slow due to running both mysql and postgres on a laptop. Since it was only for a tradeshow I didn’t really care. But I did get this error:


Unable to load dynamic library php_pgsql.dll The application has failed to start because its side-by-side configuration is incorrect. 

After some reading about similar errors which were mostly about mysql because no one likes postgres. I managed to fix it by replacing the dll’s in the apache folder with the dll’s from the postgres folder.
Nice and Easy.

Share

Apr 22 2009

Fix “Shared Key is Invalid” Error on a Bell 6520 Modem/Router.

WPA-PSK Wireless Network Error on Speedstrem 6520 Modem/Router.

I went to set up a wifi network today on my Speedsream 6520 Router/Modem. I chose the WPA-PSK encryption option because WEP is stupidly easy to crack. It seemed that whatever I entered for the shared key gave me this “Shared Key is invalid” error. This was a problem. I immediately started cursing the developers of the web interface for not making it clear what the length or format should be.

After trying a few times I was getting ready for a night of frustration.
I looked a little closer at the page that prompts for the shared key for the WPA-PSK Shared Key.
The page has a fair amount of text on it describing the different encryption schemes and right under that there is a bullet labeled Shared Key.

The third sentence tells us that the password be a minimum of 10 characters.
This got me thinking because every key I used for the TKIP network was as many characters as the text box would allow, so I tried a Key that was 8 characters in length. This got rid of the error but left the WPA-PSK hackable due the the lack of strength of the Key. I put another 4 characters onto the WPA-PSK Shared Key and tried again.This also works. Not really sure why they would make the text box hold more characters than the firmware would allow but it happens. Anyway with a WPA-PSK wireless network encrypted with a Shared Key of 12 characters my network should be secure enough for now.

Share

Apr 20 2009

Solved: The iPhone cannot be synced. The required disk cannot be found.

I got an error tonight that read “The iphone cannot be synced. The required disk cannot be found.”
Of course, having had almost all my gadgets die on me many times before I jumped to the though of just reinstalling the original firmware for my iphone 3G. But I did some research first on the sync error and why it can’t find the required disk.

My initial assumption was that the “required disk” was the iphone drive itself and I’m very glad that I was wrong.
This sync error was happening on my iphone due to trying to sync the iphone with a ringtone that was on a network drive that I recently reformatted and reorganized. Disabling the files that were on the network drive in itunes completely solved my problem. So if you ever read “The iphone cannot be synced. The required disk cannot be found.” Make sure that all the files that you are trying to sync are actually accessible.

Share