Cody Taylor

I wanted to do a few custom things with one of my site’s stats. PHP made this nice and simple with the $_SERVER array. When someone visits one of your scripts the $_SERVER array is automatically filled so as long as you know the keys then you can get all the information needed. These are the most useful items for identifying a visitor that I found in this using this array.

$_SERVER[‘REMOTE_ADDR’] gives you their ip address.
$_SERVER[‘HTTP_USER_AGENT’] is their user-agent (What browser they are using).
$_SERVER[‘HTTP_REFERER’] is where they came from.
$_SERVER[‘REQUEST_URI’] is the page they want to view.

This data is kinda useless unless you put it in a database so here’s a create table mySQL statement.

$sql = "CREATE TABLE visitors (
      `visitor_id` mediumint(11) NOT NULL AUTO_INCREMENT,
      `ip_address` VARCHAR(16)  NOT NULL,
      `user_agent` varchar(100) NOT NULL default '',
        `time_of_visit` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `referrer` varchar(256) NOT NULL default '',
      `target_url` VARCHAR(256) NOT NULL,
      PRIMARY KEY  (`visitor_id`)      
  );";

If you’d like to explore the $_SERVER array more there is very detailed info at the php web site.

I’ve been doing a bit of site scraping using curl and PHP lately. I’ve found that most sites will ban your ip if they think you’re a bot (good thing I’m on DSL) so you need to make them think that your script is a browser. The easiest way to do this is to add a user agent header to your script. Here is an example of getting a results page from google for a specific search query.

function get_google_result($search_term)
{
  $ch = curl_init();
  $url = 'http://www.google.ca/search?hl=en&safe=off&q='
                  .urlencode($search_term);
  $useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1)".
                            " Gecko/20061204 Firefox/2.0.0.1";
  
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_HEADER, 1);
  curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  
  $google_string = curl_exec($ch);
  
  curl_close($ch);
  $google_string = utf8_encode($google_string);
  return $google_string;
}

So this function will take the search term that was provided and request the results from google for it. Our user agent header makes the script look like firefox.
I put the returned string (The entire google results page) into the google_string variable and return it for parsing out what is needed. My newest scrape experiment is a site called Quick Content and basically scrapes some results from google based on some parameters from google hot trends and posts it all as a feed. It was fun to code up but it is desperate need of a makeover.

So it seems that most of the popular web browsers will allow you to write javascript where you normally put the url address. I thought this was kinda neat.

If you put something like ‘javascript:alert(“it works”);’ into the url field of your browser you should get a popup that says ‘it works’. This can be used to change any attribute on whatever page you are currently looking at. All you need to do is browse to the desired page and put something like the following into the url bar in your browser.

javascript:document.getElementById(“title”).innerHTML=”Fake Title”;

Of course this will only work on pages that contain a div or p element with the id of title (which most sites have).
I did have some issues with this depending on the browser used. Some browsers, like firefox 3.0.10 on Ubuntu Linux would just show me a blank page with Fake Title on it. This was solved by putting the javascript statements in an anonymous function in the url line.

This is all good and fun for small little tweaks but what if you wanted to do something a little more… crafty.
It is possible to include an external javascript file and alter the page so that you can execute your own little script from a specific domain. Have you ever been playing around with some ajax and gotten an ‘Access to restricted URI denied (NS_ERROR_DOM_BAD_URI)’ error message? Well the method outlined below would allow you to put some proper ajax code into the site that isn’t allowing your requests.

While I’m not going to explain explicitly how to do anything fun I will show you how to include an external javascript file into a page in your browser that will allow you to call functions from that file on the page. Even though this will include the script you will still have to call a function from the script in the url bar. The code in the script will not execute by itself just because you included it.

javascript:{{ var e=document.createElement(“script”); alert(“here”);e.src = “http://quick-content.com/include_me.js”;e.type=”text/javascript”; document.getElementsByTagName(“head”)[0].appendChild(e);} test();}

The snippet above should all be one line. So if you paste that long one liner into the url bar of your preferred browser the javascript file from the url specified (http://quick-content.com/include_me.js) will be included into the html. At the very end of that statement I call the test() function that just changes the title of my blog to ‘Tech Stuff From Null’ as opposed to ‘Tech Stuff From Cody Taylor’. Try it out. Paste that code into your browser as you’re looking at my blog and check the title. This will work for any js file so have fun and don’t break anything.

I couldn’t find an easy to follow example to do this anywhere on the web so I did my own up. Basically I wanted a list where each item has an up and a down button that will dynamically reorder the list. I tried swapping the nodes themselves but it didn’t work so it seems that I have to use the removeChild and insertBefore methods to actually see the results on the page.
Here is the Javascript DOM Example

<ul id='list' name='list'>
</ul>

<script type='text/javascript'>

//swap the current li node with the one above it
function up(button_node)
{
  var current_node = button_node.parentNode;
  var list_children = document.getElementById('list').childNodes;
  var list_node = document.getElementById('list');
  
  var temp_node;
  var previous_node;
  
  for(var i=0;i<list_children.length;i++)
  {
    if(current_node.id == list_children[i].id && previous_node)
    {
      temp_node = list_children[i].cloneNode(true);
      list_node.removeChild(list_children[i]);
      list_node.insertBefore(temp_node,previous_node);
    }
    if(list_children[i].tagName=='LI')
      previous_node=list_children[i];
  }
}

//swap the chosen li node with the one below it
function down(button_node)
{
  var current_node = button_node.parentNode;
  var list_children = document.getElementById('list').childNodes;
  var list_node = document.getElementById('list');
  
  var temp_node;
  var previous_node = "";
  
  for(var i=0;i<list_children.length;i++)
  {
    if(current_node.id == list_children[i].id)
    {
      previous_node=list_children[i];  
    }
    else if(previous_node && list_children[i].tagName=='LI')
    {
      temp_node = list_children[i].cloneNode(true);
      list_node.removeChild(list_children[i]);
      list_node.insertBefore(temp_node,previous_node);
      previous_node = "";
    }
  }
}

//create the li elements
function create_list()
{
  for(var i=0;i<6;i++)
  {
    add_li('list', "Item "+i," Item "+(i+1)+" \
    <input type='button' id='up_button1' value='Up' onclick='up(this);'>\
    <input type='button' id='down_button1' value='Down' onclick='down(this);'>");
  }  
}

//add a li element
function add_li(list, id, text) {
  var list = document.getElementById(list);
  var li = document.createElement("li");
  li.innerHTML = text;
  li.id=id;
  list.appendChild(li);
}

create_list();
</script>

So it is a little longer than I expected but it works and it’s pretty easy to understand.

Django is a high level python web framework that I’ve been hearing a lot about lately. I decided to try it out this weekend.
It took a little reading to get up and running so I documented the steps I took so others can get up and running a little more quickly.

sudo apt-get install django
django-admin –version

Create the appropriate directories and start the django project.

For no real reason I decided to put my new projects in /var/django.
sudo mkdir /var/django
sudo chmod 755 /var/django
cd /var/django
django-admin startproject mysite
cd mysite
mkdir apache

It seems that mod-wsgi is the best way to serve up python web apps due to mod_python being a little outdated. Note that django does come with a built in webserver that is really easy to get going. So if you’re just planning on evaluating the framework and not actually do any production aplications then that would be the way to go. mod_wsgi is an Apache module which can be used to host Python applications.

Install the module:
sudo apt-get install libapache2-mod-wsgi

Now for the configuration. It took me a few tries to get this right.
For now I just put my django site on port 8080 so I can play around without it being public to anyone else.

In the /var/django/mysite/apache directory I created a file called django.wsgi and put this in it:

import os, sys
apache_configuration= os.path.dirname(__file__)
project = os.path.dirname(apache_configuration)
sys.path.append(‘/var/django’)
os.environ[‘DJANGO_SETTINGS_MODULE’] = ‘mysite.settings’
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

Make sure that the sys.path.append line contains the directory above the project directory. This one took me awhile to figure out.
I was getting this error until I got it right.

[Sat May 30 15:33:24 2009] [error] [client 127.0.0.1]     raise ImportError, “Could not import settings ‘%s’ (Is it on sys.path? Does it have syntax errors?): %s” % (self.SETTINGS_MODULE, e)
[Sat May 30 15:33:24 2009] [error] [client 127.0.0.1] ImportError: Could not import settings ‘mysite.settings’ (Is it on sys.path? Does it have syntax errors?): No module named mysite.settings

Next I added a new virtual host in my sites-enabled folder like this:

<VirtualHost *:808http://localhost:8080/0>
ServerAdmin root@mysite.com
ServerName mysite.com
ServerAlias mysite.com
<Directory /var/django/mysite/apache>
Allow from all
</Directory>

WSGIDaemonProcess www-data
WSGIProcessGroup www-data
WSGIScriptAlias / /var/django/mysite/apache/django.wsgi
</VirtualHost>

After restarting apache :
sudo /etc/init.d/apache2 restart

I got an “It Worked” page when I point my browser to http://localhost:8080/

Of course this doesn’t allow you to use a database in your applications yet but it’s a start.

Firstly check out the effect example. Hover over the list in the middle to see the effect. This is called the fisheye effect in dojo and it’s one of my favorites because it’s the easiest to implement. It’s ridiculously easy to implement. I’m going to assume that you know what a basic html page looks like, so in a quick three parts here’s the code:

First the Javascript:

<script type=”text/javascript” src=”includes/dojo/dojo/dojo.js” djConfig=”parseOnLoad: true”></script>

<script type=”text/javascript”>

dojo.require(“dojox.widget.FisheyeLite”);
dojo.addOnLoad(function(){
//make the list items into fisheye items
dojo.query(“li.fish”, dojo.byId(“fishyList”)).forEach(function(n){
new dojox.widget.FisheyeLite({ },n);
});
});
</script>

Next the CSS:

<style type=”text/css”>
@import “includes/dojo/dojo/resources/dojo.css”;
@import “css/dojo_lists.css”;
#fishyList ul {
width:600px;
list-style-type:none;
}
.fisheyeTarget {
font-weight:bold;
font-size:19px;
}
</style>

None of the CSS really matters as far as functionality goes. It does go a long way into making it look a lot slicker.

Now the HTML:

<ul id=”fishyList”>
<li class=”fish”><span class=”fisheyeTarget”>Cody Taylor’s<br><br></span></li>
<li class=”fish”><span class=”fisheyeTarget”>Dojo Javascript <br><br></span></li>
<li class=”fish”><span class=”fisheyeTarget”>Slick FishEye Effect<br><br></span></li>
</ul>

Of course you’ll have to download and put the dojo framework on your webserver and alter the include statements but other than that it’s that simple. You may want to look at the dojo documentation for the dojo.query and forEach functions.

Functions are objects. This makes recursion short and sweet in javascript using anonymous functions.
In your html, php, or asp file try this out.

<input type=’button’ id=’myButton’ value=”5″>

<script type=’text/javascript’>
var myButton = document.getElementById(“myButton”);
var i=myButton.value;

myButton.onclick = function() {
if(i==0)
{
alert(“DONE”);
myButton.value = i;
return i;
}
alert(i);
return arguments.callee(–i);
};

</script>

I attached the anonymous function onto the onclick event for the button that was created right above the javascript. When that button is clicked the anonymous function will call itself and decrement ‘i’ until it is zero. When ‘i’ becomes zero the value of the button is set to 0 and the function returns. Nice and simple.

The only possible confusing thing here is the arguments.callee line. Remember I said that functions are objects? The arguments.callee actually calls itself. We need this because we don’t have a handle for the anonymous function. The arguments object is a local variable that all functions have. It can be used like an array to access the values passed to the function. It also holds ‘callee’, ‘caller’, and ‘length’. Since the callee is the function itself then caller is obviously the function that called it and length is the number of arguments passed to the function.

Learning javascript by copying and pasting useful snippets off the web, the only aspects of Javascript that I’ve ever looked closely at were to change css properties, validate forms, or get some values with ajax. Because of this, there is still  a large portion of the language that I’m still learning. I’ve never used anonymous functions in any language, but they’ve always looked like an interesting concept, so I decided to give it a shot. This is what I learned with a few short examples.

This little javascript feature can be hugely useful for passing functions as variables or declaring quick functions inline for the programmer who is to lazy to switch files or scroll to wherever that annoying javascript declaration is.

To pass a function as a variable either with a reference or inline:

<script type=’text/javascript’>

//garbage example data
var array = [ 3, 14, 15, 9, 26 ]
var array2 = [ 3, 14, 15, 9, 26 ]

//This function takes an array and a function as parameters.
//It then executes the passed function on every element of the array.
function execute_on_array( array_of_stuff, anonymous_function )
{
    for (var i=0; i<array_of_stuff.length; i++)
    {
        alert(“before ” + array_of_stuff[i]);
        array_of_stuff[i] = anonymous_function(array_of_stuff[i]);
        alert(“after” + array_of_stuff[i]);
    }
}

//Here’s the anonymous function that is now referred to as nullify_function
var nullify_function = function (x) { if(x) return 0; else return x; }

//Passing an anonymous function in as a variable
execute_on_array( array_of_numbers, nullify_function);

//Here is the call to the previous function with an anonymous function defined inline
function execute_on_array( array2, function (x) { x=0; return x; } );

</script>

I could see how this would make recursion in javascript really easy.

Create Custom Iphone or Ipod Touch Web Applications Using iWebKit

For the impatient people like myself, heres what it looks like. You want to point your iphone browser to this location and hit the ‘+’ button at the bottom and select add to the homescreen. This will get rid of the navigation bar on in safari and make the iwebkit example look like a native iphone app.

Here are the steps to take in order to get this working.
Get the frame work: iWebKit framework (I used 4.5.3 for this guide)
Extract the iwebkit and take a look at the manual. Iwebkit comes with a few examples of how to use the key framework elements.We’re going to make a very rudimentary application for the purpose of iphone iwebkit example. Ajax integration with iwebkit will come later.

First you will need to create a directory on your web server and extract the iwebkit files into it.
The webkit contains a few examples but most of them didn’t show up properly for me when I used the Iphone.
Most webapps are built with just really basic html. The custom elements that iwebkit provides are called by normal divs with certain ids.

First start with an extremely basic html skeleton page :

<html>
<head>
<meta content=”yes” name=”apple-mobile-web-app-capable” />
<meta content=”text/html; charset=iso-8859-1″ http-equiv=”Content-Type” />
<meta content=”minimum-scale=1.0, width=device-width, maximum-scale=0.6667, user-scalable=no” name=”viewport” />
<title>iphone iwebkit example</title>
</head>
<body>

</body>
</html>

Hopefully no explanations are needed there. Notice the meta tags? So now we need to start adding the iphone javascript and css.
To get started with iwebkit for the iphone or iphone touch you only need to include two files into this basic html page but you will need to take all four folders from the Framework folder and put them into the directory where you want to host your web app.
Add these two lines to the head portion of the skeleton html file outlined above.

<link href=”css/style.css” rel=”stylesheet” type=”text/css” />
<script src=”javascript/functions.js” type=”text/javascript”></script>

Now to start putting the application together using iwebkits basic elements.

Top Bar
This section will outline the options for the navigation bar at the top of the page on the iphone or ipod touch.
Using iwebkit for the top bar we will normally put navagation buttons and a title like this.

<div id=’topbar’>
<div id=”leftnav”>
<a href=”index.html”><img alt=”home” src=”images/home.png”/></a>
</div>
<div id=’title’>
iphone iwebkit example
</div>
<div id=”rightnav”>
<a href=”index.html”>Next</a>
</div>
</div>

There are four div elements there that make up this top bar. The first is the bar itself and it encompasses the other three elements.
The second is the leftnav div. This specifies that the elements within this div are to appear at the left side of the bar (Big surprise).
The third is the title. The fourth is the rightnav which will appear on the right side. You’re probably going to want to put links and pictures in the leftnav and rightnav divs.
Instead of the rightnav div it is possible to put a rectangle button instead of the arrow button just by changing rightnav to rightbutton in the id.

Content
This section is for basically anything you want to put in your iwebkit page. I’m going to demonstrate a simple iphone-looking list here:

<div id=”content”>
<span class=”graytitle”>Iphone iwebkit example</span>
<ul class=”pageitem”>
<li class=”textbox”>
<p>Some Text for the iwebkit example</p>
<p>More text for the Iphone</p>
</li>
<li class=”textbox”>
<span class=”header”>Iphone iwebkit</span>
<p>Iphone Iphone</p><p>WebKit WebKit</p>
</li>
<li class=”menu”>
<a href=”index.php”>
<img alt=”Description” src=”thumbs/basics.png” />
<span class=”name”>Iphone iwebkit Example</span>
<span class=”comment”>Comment about iwebkit</span>
<span class=”arrow”></span>
</a>
</li>
<li class=”store”>
<a class=”noeffect” href=”index.php”>
<span class=”image” style=”background-image: url(‘image.jpg’)”></span>
<span class=”name”>Iphone Song</span>
<span class=”comment”>iWebKit Comment</span>
<img alt=”rating” class=”stars” src=”images/4stars.png” />
<span class=”starcomment”>13 Reviews</span>
<span class=”arrow”></span>
</a>
</li>

</ul>
</div>

There are a lot of options when using iwebkit. Most are pretty self explanatory, like the <ul> class and the <span> and <li> classes, but who wants to remember all that.
So go check the source of some feature or look that you want on your iwebkit page at <a href=’http://m.iwebkit.net/index.html’>this demo</a>.

The first issue I had when looking at my iwebkit example on the iphone was that the example iwebkit application wasn’t fullscreen in safari.
The only way that I’ve found so far to make this web application look like a native iphone application is to go to the page in safari and then hit the ‘+’ button at the bottom and select add to the homescreen.
This will get rid of the navigation bar on the iphone and make the iwebkit example look like a native iphone app.

This example and the full source for this iwebkit iphone example  can be looked at here.

DJRumpy writes in to alert us that Apple’s new OS, Snow Leopard, is apparently nearing completion. “Apple this past weekend distributed a new beta of Mac OS X 10.6 Snow Leopard that altered the programming methods used to optimize code for multi-core Macs, telling developers they were the last programming-oriented changes planned ahead of the software’s release. …`Apple is said to have informed recipients of Mac OS X 10.6 Snow Leopard build 10A354 that it has simplified the`… APIs for working with Grand Central, a new architecture that makes it easier for developers to take advantage of Macs with multiple processing cores. This technology works by breaking complex tasks into smaller blocks, which are then`… dispatched efficiently to a Mac’s available cores for faster processing.”

Read more of this story at Slashdot.