Cody Taylor

My webhost recently moved all my sites to a ‘stabilization’ server because my sites were using far to much CPU time and Memory. After reviewing the logs it looked like some bot from India decided to repeatedly scrape one of my sites in it’s entirety without any delays between requests. So the support team over there either requires me to correct the problem or upgrade to a dedicated server plan at ridiculous costs.

Since I didn’t really think that there was a problem I emailed back about the single IP address that was causing all the issues and took steps to prevent requests from that IP address from accessing the site. The support team replied saying that my usage was still high and that I still needed to correct the problem. A little frustrated, I did some research on how to improve my site’s load time and hopefully reduce CPU and memory usage.

Most of my sites use wordpress so I found a large number of articles geared specifically to optimizing wordpress blogs. Before I tried anything I backed up my entire public_html directory and did a dump of all my mySQL databases (took almost 20 minutes for the dump).

Dealing with Plugins
The first thing I did was upgrade all my plugins. Most wordpress plugins allow you to upgrade automatically so all you really have to do is click a button and all the work is done for you. I also deactivated and deleted a surprising number of plugins that I haven’t really had any use for recently. Apparently a lot of free plugins can cause large amounts of unneccesary load on your server due to the authors not really knowing or caring how well their software performs.

Dealing with spam bots
I have been using the Akismet plugin for awhile and it has been reporting large amounts of spam comments and pingbacks. It’s not really something that most people worry about because the spam is automatically deleted after a period of time. It does however increase server load, especially if it’s in the thousands of messages a day. I found this little mod_rewrite snippet to deny any blatent spammers that don’t have a proper referer :

RewriteEngine On
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} .wp-comments-post\.php*
RewriteCond %{HTTP_REFERER} !.*codytaylor.org.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule ^(.*)$ ^http://lemonparty.org//$ [R=301,L]

Cache and Compress
Since most of my pages rarely change it’s silly to generate every page for every request dynamically. After some reading I decided to use WP Super Cache to help optimize my WordPress sites. Of course just enabling Super Cache in the WP Super Cache plugin didn’t really improve load times for the end user but it should reduce server load immensely. What did improve load times drastically was the Super Cache Compression. This was a little more involved to get going but if you’re comfortable with copying and pasting code into a .htaccess file then it shouldn’t be difficult as long as your host supports mod_mime, mod_rewrite, and mod_deflate.

After going through all that, my sites now average at about half the load time they used to. Hopefully my web host feels that I’ve done enough to get off the ‘stabilization’ server so I don’t have to transfer all my stuff to another company.

I was reading about some php performance tips at work today and decided to throw together some of the ways to make your php site faster.

Here’s a couple of short tips to avoid making your site lag for the user:

1. Unset your variables. This frees up a surprising amount of memory
2. Use memcache
3. Turn on apache’s mod_deflate
4. If your database is local then close your connections when you’re done with them
5. If your database is remote then use persistant connections
6. Don’t use functions inside of a for loop control expression unless absolutely necessary. Example :
   

   
   for($i=0;$i<some_big_calculation(20);$i++)
   

   
   The some_big_calculation function will be called every iteration.
7. Use string concatenation instead of embedding variables in strings.
   

   
   $some_string = "asdf $asdf2 asdf"; //bad
   $some_string = 'asdf '.$asdf2.' asdf'; //good
   

8. When including files, use at least a relative path so PHP doesn’t have to look in the entire path.
   

   
   include("./some_file.php"); // looks in the current directory
   include("wrong.php"); // searches every directory in the path
   

9. Try not to use include_once or require_once. They are more expensive and it really isn’t that hard to only include files once
10. Not everything needs to be an object. There is a fair amount of overhead when doing everything OOP.
11. Use output buffering to make everything seem faster.

Hopefully I didn’t miss any of the big ones.

PHP makes life a lot easier for quick or dirty maintenance scripts, cron jobs or web applications but how does it do for older, not so straight forward problems dealing with bits and bytes? I was surprised how easy it was to manipulate bits in a byte with php. Here is an function that made my life a fair amount easier when having to check for a specific bit in a byte.

This function checks whether a certain bit is set or not given a byte and an index. It returns true if the chosen bit is set. It casts the $value argument to a integer just in case. The index $n goes from left to right so the most significant bit is bit one and the least significant is bit eight. This function will only work for integers between 0 and 255 because that was all I needed at the time. It would be trivial to write either a function to separate bytes in an integer or to increase the amount of bits that this function checks. I originally had a different function here but the Internet quickly told me that there was an easier way to do this.

<?php

function check_bit($value,$n=8)
{
    $value = (int)$value;
    if($value & (1<<(8-$n))) { return true; }
    else { return false; }
}

//Check Bit Usage Example
$test_byte1 = "4";

if( check_bit($test_byte1,2))
  echo "Bit 2 is set in ".decbin($test_byte1);
else
  echo "Bit 2 is not set in ".decbin($test_byte1);

?>

Here are the php bitwise operator definitions from the php documentation. Look like C much?

$a & $b	And	Bits that are set in both $a and $b are set.
$a | $b	Or	Bits that are set in either $a or $b are set.
$a ^ $b	Xor	Bits that are set in $a or $b but not both are set.
~ $a	Not	Bits that are set in $a are not set, and vice versa.
$a << $b	Shift left	Shift the bits of $a $b steps to the left (each step means “multiply by two”)
$a >> $b	Shift right	Shift the bits of $a $b steps to the right (each step means “divide by two”)

Because I’ve been starting to keep a closer eye on my traffic I’ve been logging everything to my databases. When writing reports for this information I noticed that there was huge amounts of traffic from bots. Yahoo, Google, MSN and many others have been hammering my sites quite a lot lately. Since I’m writing my reports in PHP I needed a quick little function to identify whether the visitor was real traffic or some machine scraping my site. I couldn’t find one after a few quick Google queries but the solution was trivial so I wrote my own. Hopefully this will save someone a few minutes. The function is as simple as possible and it seems to be working so far. I’ve been watching it for awhile and it hasn’t missed any yet. I’m assuming that it’s not going to catch everything but it would be nice to get most. Any bot user agent string suggestions would be helpful.

//returns 1 if the user agent is a bot
function is_bot($user_agent)
{
  //if no user agent is supplied then assume it's a bot
  if($user_agent == "")
    return 1;

  //array of bot strings to check for
  $bot_strings = Array(  "google",     "bot",
            "yahoo",     "spider",
            "archiver",   "curl",
            "python",     "nambu",
            "twitt",     "perl",
            "sphere",     "PEAR",
            "java",     "wordpress",
            "radian",     "crawl",
            "yandex",     "eventbox",
            "monitor",   "mechanize",
            "facebookexternal"
          );
  foreach($bot_strings as $bot)
  {
    if(strpos($user_agent,$bot) !== false)
    { return 1; }
  }
  
  return 0;
}

Using my previous wordpress example plugin I’m going to demonstrate how to use ajax within the admin panel. WordPress uses the Simple AJAX Code-Kit (SACK) which is relatively easy to use and understand.

For this example I’m going to expand on my previous example and add a simple javascript function that queries some data from the server. WordPress forces us to do this in a roundabout way. First we have to add two new function hooks. One prints my javascript function in the scripts section of the admin panel and the other is the code that gets called by the ajax sack request.

//add my custom ajax function to the scripts section in the admin panel
add_action('admin_print_scripts', 'ajax_request');

//add data returning ajax refresh table function
add_action('wp_ajax_do_something', 'get_random_number');

The ajax_request function is a shell function for the javascript ajax call get_random_number_from_server which is our sack ajax request. Note that we are calling the admin-ajax.php script. This script will handle the calling of the get_random_number function for us because of the action defined above.

//This function will print out in the header section. 
//Put all your javascript in this function.
function ajax_request()
{
        //print out the sack ajax library
        wp_print_scripts( array( 'sack' ));  
  ?>
  <script type="text/javascript">
  //<![CDATA[
    
  function get_random_number_from_server()
  {
    //creates the sack object and 
                //gives it the url that it should request to.
    var mysack = new sack( '<?php 
                bloginfo( "wpurl" ); ?>/wp-admin/admin-ajax.php' );    
        
    mysack.execute = 1;   //execute whatever is returned
    mysack.method = 'POST';
    
    //Set POST fields
    mysack.setVar( "action", "do_something" );
        
    mysack.encVar( "cookie", document.cookie, false );
    mysack.onError = function() { alert('Ajax Error')};
    mysack.runAJAX();  //run the result

    return true;

  } // end of JavaScript function myplugin_ajax_elevation
  //]]>
  </script>
  <?php
  
}

This is the php function that is called by the javascript function above. It spits out a alert javascript function and then dies. Not sure why but it is recommended to die in this function.

//This is the server side code for the ajax sack request
function get_random_number()
{
  $minimum_number = 0;
  $maximum_number = 100;
  die("alert('".rand($minimum_number,$maximum_number)."');");
} 

This code can now be run like the following in your plugins admin page.

<input type='button' value='Get Random Number' 
onclick='get_random_number_from_server();'>

When the button is hit a popup with a random number in it will pop up. Here’s a link to the entire example plugin code.

I wanted to do a few custom things with one of my site’s stats. PHP made this nice and simple with the $_SERVER array. When someone visits one of your scripts the $_SERVER array is automatically filled so as long as you know the keys then you can get all the information needed. These are the most useful items for identifying a visitor that I found in this using this array.

$_SERVER[‘REMOTE_ADDR’] gives you their ip address.
$_SERVER[‘HTTP_USER_AGENT’] is their user-agent (What browser they are using).
$_SERVER[‘HTTP_REFERER’] is where they came from.
$_SERVER[‘REQUEST_URI’] is the page they want to view.

This data is kinda useless unless you put it in a database so here’s a create table mySQL statement.

$sql = "CREATE TABLE visitors (
      `visitor_id` mediumint(11) NOT NULL AUTO_INCREMENT,
      `ip_address` VARCHAR(16)  NOT NULL,
      `user_agent` varchar(100) NOT NULL default '',
        `time_of_visit` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `referrer` varchar(256) NOT NULL default '',
      `target_url` VARCHAR(256) NOT NULL,
      PRIMARY KEY  (`visitor_id`)      
  );";

If you’d like to explore the $_SERVER array more there is very detailed info at the php web site.

I’ve been doing a bit of site scraping using curl and PHP lately. I’ve found that most sites will ban your ip if they think you’re a bot (good thing I’m on DSL) so you need to make them think that your script is a browser. The easiest way to do this is to add a user agent header to your script. Here is an example of getting a results page from google for a specific search query.

function get_google_result($search_term)
{
  $ch = curl_init();
  $url = 'http://www.google.ca/search?hl=en&safe=off&q='
                  .urlencode($search_term);
  $useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1)".
                            " Gecko/20061204 Firefox/2.0.0.1";
  
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_HEADER, 1);
  curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  
  $google_string = curl_exec($ch);
  
  curl_close($ch);
  $google_string = utf8_encode($google_string);
  return $google_string;
}

So this function will take the search term that was provided and request the results from google for it. Our user agent header makes the script look like firefox.
I put the returned string (The entire google results page) into the google_string variable and return it for parsing out what is needed. My newest scrape experiment is a site called Quick Content and basically scrapes some results from google based on some parameters from google hot trends and posts it all as a feed. It was fun to code up but it is desperate need of a makeover.

I used JSON for the first time today and it’s really nothing special. I’ve heard about it a few times but never really given it much thought and theres no reason I should have. Here is the list of all you really need to know about this syntax for passing around name value pairs and arrays to javascript.

1. JSON is an acronym for ‘Javascript Object Notation’.
2. JSON is fast. Mostly because it is recognized natively by Javascript so there’s no processing overhead.
3. JSON is an ordered list of name value pairs.
4. JSON is so much easier to read and write than XML due to it’s simplicity.
5. Apperently (Untested by me) data is formatted as JSON then Ajax can travel across domains.
6. Almost every language used in web development either already has a JSON library or set of functions. If one doesn’t, then creating functions is a trivial task.
7. ‘var jsonObject = { ‘cody : ‘taylor’ };’ is referenced by ‘jsonObject.cody’ which gives us ‘taylor’.
8. ‘var jsonObect = {‘javascript’ : {‘json’ : ‘not xml’ };’ is referenced by ‘jsonObject.javascript.json’ which gives us ‘not xml’.
9. You can also reference the JSON object as if it was an associative array like ‘jsonObject[‘cody’]’ or ‘jsonObject.javascript[‘json’]’ which gives the same values as previously.
10. If you don’t want to use key/value pairs you can define a normal data array. ‘jsonObject = {‘arrayOfData’: {‘numbers’ : [‘1’, ‘2’, ‘3’]}};’ We use indexes for this dataset. Don’t use indexes for the collections defined in the ‘{ }’. ‘jsonObject.arrayOfData.numbers[1]’ will give us ‘2’.
11. You can put functions in the dataset to pass around executable code.

So now you know basically all there is to know about JSON.

So it seems that most of the popular web browsers will allow you to write javascript where you normally put the url address. I thought this was kinda neat.

If you put something like ‘javascript:alert(“it works”);’ into the url field of your browser you should get a popup that says ‘it works’. This can be used to change any attribute on whatever page you are currently looking at. All you need to do is browse to the desired page and put something like the following into the url bar in your browser.

javascript:document.getElementById(“title”).innerHTML=”Fake Title”;

Of course this will only work on pages that contain a div or p element with the id of title (which most sites have).
I did have some issues with this depending on the browser used. Some browsers, like firefox 3.0.10 on Ubuntu Linux would just show me a blank page with Fake Title on it. This was solved by putting the javascript statements in an anonymous function in the url line.

This is all good and fun for small little tweaks but what if you wanted to do something a little more… crafty.
It is possible to include an external javascript file and alter the page so that you can execute your own little script from a specific domain. Have you ever been playing around with some ajax and gotten an ‘Access to restricted URI denied (NS_ERROR_DOM_BAD_URI)’ error message? Well the method outlined below would allow you to put some proper ajax code into the site that isn’t allowing your requests.

While I’m not going to explain explicitly how to do anything fun I will show you how to include an external javascript file into a page in your browser that will allow you to call functions from that file on the page. Even though this will include the script you will still have to call a function from the script in the url bar. The code in the script will not execute by itself just because you included it.

javascript:{{ var e=document.createElement(“script”); alert(“here”);e.src = “http://quick-content.com/include_me.js”;e.type=”text/javascript”; document.getElementsByTagName(“head”)[0].appendChild(e);} test();}

The snippet above should all be one line. So if you paste that long one liner into the url bar of your preferred browser the javascript file from the url specified (http://quick-content.com/include_me.js) will be included into the html. At the very end of that statement I call the test() function that just changes the title of my blog to ‘Tech Stuff From Null’ as opposed to ‘Tech Stuff From Cody Taylor’. Try it out. Paste that code into your browser as you’re looking at my blog and check the title. This will work for any js file so have fun and don’t break anything.

I couldn’t find an easy to follow example to do this anywhere on the web so I did my own up. Basically I wanted a list where each item has an up and a down button that will dynamically reorder the list. I tried swapping the nodes themselves but it didn’t work so it seems that I have to use the removeChild and insertBefore methods to actually see the results on the page.
Here is the Javascript DOM Example

<ul id='list' name='list'>
</ul>

<script type='text/javascript'>

//swap the current li node with the one above it
function up(button_node)
{
  var current_node = button_node.parentNode;
  var list_children = document.getElementById('list').childNodes;
  var list_node = document.getElementById('list');
  
  var temp_node;
  var previous_node;
  
  for(var i=0;i<list_children.length;i++)
  {
    if(current_node.id == list_children[i].id && previous_node)
    {
      temp_node = list_children[i].cloneNode(true);
      list_node.removeChild(list_children[i]);
      list_node.insertBefore(temp_node,previous_node);
    }
    if(list_children[i].tagName=='LI')
      previous_node=list_children[i];
  }
}

//swap the chosen li node with the one below it
function down(button_node)
{
  var current_node = button_node.parentNode;
  var list_children = document.getElementById('list').childNodes;
  var list_node = document.getElementById('list');
  
  var temp_node;
  var previous_node = "";
  
  for(var i=0;i<list_children.length;i++)
  {
    if(current_node.id == list_children[i].id)
    {
      previous_node=list_children[i];  
    }
    else if(previous_node && list_children[i].tagName=='LI')
    {
      temp_node = list_children[i].cloneNode(true);
      list_node.removeChild(list_children[i]);
      list_node.insertBefore(temp_node,previous_node);
      previous_node = "";
    }
  }
}

//create the li elements
function create_list()
{
  for(var i=0;i<6;i++)
  {
    add_li('list', "Item "+i," Item "+(i+1)+" \
    <input type='button' id='up_button1' value='Up' onclick='up(this);'>\
    <input type='button' id='down_button1' value='Down' onclick='down(this);'>");
  }  
}

//add a li element
function add_li(list, id, text) {
  var list = document.getElementById(list);
  var li = document.createElement("li");
  li.innerHTML = text;
  li.id=id;
  list.appendChild(li);
}

create_list();
</script>

So it is a little longer than I expected but it works and it’s pretty easy to understand.