Cody Taylor

After writing the backend for a new web app in python I went to start working on the Django portion. I was planning on hosting this application with a Dreamhost shared hosting plan which already has Python 2.5 installed. After trying unsuccessfully to install some new middleware with easy_install, I started looking for a solution that gives me more control over what I want to do with Python without having to purchase dreamhost vps hosting. It seems that you can set up a virtual python install in your home directory and it was surprisingly easy. I’ve only had the need to use this virtual python install on Dreamhost, which is using Debian, but I can’t see any reason why it wouldn’t work on other environments.

Note that this is completely unnecessary if you have root access.

Assuming you already have a shell account and you are ssh’d in, execute these commands to install your virtual python environment:

$ wget http://peak.telecommunity.com/dist/virtual-python.py 
$ python virtual-python.py

Those commands copy the Python binary to your /home/user/bin directory and sets up symbolic links to the system wide libraries. This means that the ~/bin/python executable will have access to the same libraries as the system Python but that any extra installed software will not affect the system wide Python install.

Next you should add the ~/bin directory to your PATH by adding this block to your .bash_profile:

 if [ -d ~/bin ] ; then     
    PATH=~/bin:"${PATH}" 
fi 

You’ll have to log out and back in for this to take effect.
After you’re logged back in, run these commands to install easy_setup:

$ wget http://peak.telecommunity.com/dist/ez_setup.py 
$ python ez_setup.py 

Now you should be able to install whatever you want using easy_install. The first thing I did was install django-db-log using this command:
$ easy_install django-db-log

After getting my beta key for Starcraft 2 I’ve been spending a lot of time in Windows. Windows 7 is a huge improvement over Windows XP but I really miss the transparent tabbed terminals in Linux. I’ve become so used to having numerous SSH tunnels and sessions going in multiple tabs that the classic Putty Client in Windows 7 seemed very restricting. I’ve explored a few Putty alternatives and it seems that a transparent tabbed SSH client isn’t an option in Windows. I hope that there is an application that I’ve missed, but at the moment it seems you can either have a transparent Putty client with text that also becomes transparent and illegible with Putty Tray, or you can have a buggy tabbed SSH client with Putty Connection Manager. Both of these applications are less than desirable, but I have settled on Putty Tray. With Putty Tray, the console windows will minimize to the tray and not take up valuable real estate in the task bar. If used with Pageant, Putty Tray is far superior to Putty Connection Manager and it doesn’t have the usability issues that Putty Connection Manager does.

I’m still tempted to install a virtual machine and run some flavour of Linux just for the clean and visually appealing consoles.

If you have root access to a linux server and you don’t have the root mysql password, but need it, then you can easily reset the root mysql password in just a few commands. These commands probably differ depending on what linux distro you use. I was using Ubuntu 9.04 (Jaunty Jackalope) when I wrote this.

Firstly you will want to turn the mysql service off.

codytaylor@server:~$ sudo /etc/init.d/mysql stop
 * Stopping MySQL database server mysqld   

Now we restart the mysql server with the ‘skip-grant-tables’ option which basically allows anyone to do whatever they like. It’s usually preferable to include the ‘skip-networking’ option so that only localhost (you) have access to the naked database.

 codytaylor@server:~$ sudo mysqld_safe --skip-grant-tables --skip-networking &

Now all that is left is actually changing the root password. Log into the mysql monitor and change the root password.

codytaylor@server:~$ mysql -u root mysql
mysql> UPDATE user SET Password=PASSWORD('password') WHERE User='root';
mysql> FLUSH PRIVILEGES;

Those commands will reset the root mysql password to ‘password’. Now you’ll probably want to restart the mysql service and have it run normally.

codytaylor@server:~$ sudo /etc/init.d/mysql restart

If you are using windows and you want to reset the mysql root password then check the mysql documentation.

I recently had to move all my sites over to a new host due to Lunarpages basically kicking me off. I chose DreamHost because they were having a promotion where I get a full year for $10. One of my coworkers uses them with no complaints and they give free shell access.

Vim is my editor of choice but I hate coding without syntax highlighting and DreamHost doesn’t have Vim Syntax highlighting on by default. It is possible to type “:syntax enable” when already in the editor to enable syntax highlighting on the current file that you’re editing but after closing it and opening another file you will be forced to do it again.

To enable syntax highlighting forever I had to edit the .vimrc file which lives in my home directory. It didn’t exist so I had to create one. After I created this file I added the lines :

syntax enable
set background=dark
set nocompatible
set autoindent
set smartindent
set tabstop=4
set shiftwidth=4
set showmatch
set ruler
set virtualedit=all

Alternatively you could put ‘background=light’ which will make the colors more readable if your console is black text on a white background. The rest of those lines are just personal preference and put there so I can copy and paste them elsewhere. More info and a great example can be found at this vim site.

If you spend a lot of time on the linux command line you quickly find that it requires a lot of typing and retyping commands. I used to find myself using the exact same lengthy command multiple times a day and to get there I would type “history | grep some_command” and then execute it from there. Since I knew enough to get the job done I hadn’t really tried to find more efficient ways of doing the same old thing. But when I found out about the Linux bang (!) commands I realized how wasteful what I was doing really was.

The exclamation mark, in this case, is referred to as a ‘bang’.

• !!
  This bang command, when entered into the bash shell will run the previous command. It basically does the same thing as hitting the up arrow to take you to the previous command and then hitting enter.
   
• !ls
  This will run the last command that started with ‘ls’. If you ran ‘ls -al /etc/init.d’ a few commands ago and then you type ‘!ls’ the full command will be run again, assuming that you haven’t used that command since then.
   
• !ls:p
  This will display the command instead of running it.
   
• !$
  This one will run the last word of the previous command. This one is mainly useful for substitutions.
   
• !$:p
  Instead of running the last word of the previous command this will print it out.
   
• !*
  This bang command will run the previous command without the first word. This one is also only really useful for substitutions as we see in the examples that follow.
   
• !*:p
  This will print the previous command without the first word.
   

Here are a few examples of how to use these bash bang commands in everyday command line usage :

For the purposes of these examples, every example will assume these are the last three commands you ran:

    % which firefox
    % make
    % ./foo -f foo.conf
    % vi foo.c bar.c

Getting stuff from the last command:

    Full line:     % !!            becomes:   % vi foo.c bar.c
    Last arg :     % svn ci !$     becomes:   % svn ci bar.c
    All args :     % svn ci !*     becomes:   % svn ci foo.c bar.c
    First arg:     % svn ci !!:1   becomes:   % svn ci foo.c

Accessing commandlines by pattern:

    Full line:     % !./f          becomes:   % ./foo -f foo.conf
    Full line:     % vi `!whi`     becomes:   % vi `which firefox`
    Last arg :     % vi !./f:$     becomes:   % vi foo.conf
    All args :     % ./bar !./f:*  becomes:   % ./bar -f foo.conf
    First arg:     % svn ci !vi:1  becomes:   % svn ci foo.c

I found those examples here.

Spending a lot of time on the command line lately I noticed that Ubuntu does not come with Vim syntax highlighting by default. Apparently it installs a version of Vim called vim-tiny which doesn’t include any syntax highlighting.

There are two packages that you can install to get syntax highlighting to work in Vim: vim-full and vim-common. Because I didn’t have gnome installed vim-full was a very large download (like 50MB) and it errored out anyway. vim-common is definitely the way to go.

sudo apt-get install vim

The above line will replace vim-tiny with vim-common and will allow for syntax highlighting. A lot of the time you will have to enable syntax highlighting by editing the vimrc config file either in /etc/vim or in yur home directory. You will need to uncomment the line “syntax on”.

Today I created a simple shell script and I was getting a few odd errors:

cody@taylor:/var/some_folder/server$ ./process_xml.sh
-bash: ./process_xml.sh: /bin/sh^M: bad interpreter: No such file or directory

I figured it was probably a permissions error or an issue with the shebang (#!/bin/sh) line. I tried removing the shebang line, changing it to use dash or bash explicitly, chmoding to 777 and still no luck and another odd error.

cody@taylor:/var/some_folder/server$ sh process_xml.sh
: not found.sh: 4:

I then checked the log file that the commands were supposed to be writing to and it was filled with ‘^M’ on every line break and the log name itself was followed by a ‘?’. Took a minute or two but I finally clued in that I wrote that script on a windows machine and then exported it to an ubuntu linux server via subversion. It was just a basic text format issue.

Under DOS (Windows/PC) the end of a line of text is signalled using the ASCII code sequence CarriageReturn,LineFeed. Alternately written as CR,LF or the bytes 0x0D,0x0A. On the Macintosh platform, only the CR character is used. Under UNIX, the opposite is true and only the LF character is used.

After a quick :

cody@taylor:/var/some_folder/server$ apt-get install tofrodos
cody@taylor:/var/some_folder/server$ dos2unix process_xml.sh

Everything worked fine.

Following the installation of Windows 7 on my desktop machine that was previously running Ubuntu 9.04, I realized that one of my partitions was missing. It was formatted as ext3 so of course it wouldn’t show up in windows. When I went to boot back into linux the grub boot manager was gone. The machine just booted straight into windows 7.

To re-enable grub was a lot simpler than I thought it would be. Since the Ubuntu 8.04 install disc that I had lying around also works as a live disc I booted my desktop off that. Once Ubuntu started up I opened up a terminal and typed these commands:

sudo grub
root(hd0,7)
setup(hd0)
quit

I then rebooted and grub was back to normal with both my Ubuntu operating system and windows 7 listed. Note that hd0,7 was the partition where I installed Ubuntu. If you installed linux before windows then you will probably want to use hd0,0. I installed windows 7 over a broken windows xp partition that I had installed before everything else.

If you installed ubuntu before you installed windows then you’re probably going to have to edit your ‘/boot/grub/menu.lst’ file. The grub documentation may be helpful.

Do you ever download those files that are comprised of multiple zip files that always seem to extract individual rar files to their own folders. This would normally require the user to cut and paste all these rar files back into the original folder and then unrar them. This is very tedious, especially when there is lots of these files and folders. I’ve written this little shell script to automate this task for myself and figured that I would share. This script is extremely simple but it has worked great for every double archive that I have tried it on. As always any improvements and comments are welcome.

Call this script from the command line when your current working directory is the folder with all the zip files in it. To get your current working directory use the command ‘pwd’.

unzip -o \*.zip
for f in *.rar;do unrar -o- e “$f”;done
rm *.rar
rm *.zip

You may be asking why I was forced to escape the ‘*’ in the first line. If I didn’t do this then I got a lot of “caution: filename not matched:” errors on the unzip line. Not sure why and if anyone can shed some light on that it would be great. As for the option to the commands, read the man pages. And the for loop is pretty self explanatory.

Enjoy those giant image sets.

If you have access to a ssh server you can easily tunnel your web browsing and newsgroup/IRC activity to make sure that your IP is never associated with that ..stuff you do. You can basically tunnel any port through the any ssh server. These methods can be used to build up complex chains that are almost impossible to track.

To tunnel your basic browsing through a ssh “proxy” use this command:

ssh -D 8080 -p 22 codytaylor@someserver.com

I used port 8080 instead of the default 80 becuase I run a webserver on my local machine and I didn’t want to mess that up.

The -D in the command specifies a local dynamic application-level port forwarding which creates a socket to listen to the specified port on the local machine. Ancy connections made to this local port are forwarded over the secure channel.

This makes it easily possible to forward all web traffic on the local system through the secure channel by setting up a simple iptables rule:
–iptables -A PREROUTING -t nat -i eth0 -p tcp –dport 80 -j REDIRECT –to-port 8080

If you want to set up a secure ssh tunnel to connect to a specific newsgroup server you can use this command:

sudo ssh -N -p 22 codytaylo@sshserver.net -f -L 443:111.111.111.111:443

The -L specifies that the given port on the local client is to be forwarded to the given host and port on the remote side. This is basically a very specific instruction that tells all traffic that is directed to localhost at port 443 (119 if you don’t like double encryption) to go to the ip address 111.111.111.111:443. Note: I couldn’t get this to work with domains for some reason.

The -N tells ssh to not execute any remote commands. So you do not get the annoying ssh session stuff that makes you leave the terminal open.

The -f tells ssh to go into the background.