The below is pretty much directly from the online manuals for PHPUnit and WordPress but copied here for my personal usage.

Install PHPUnit

1. Download:
   wget https://phar.phpunit.de/phpunit.phar
2. Make executable:
   chmod +x phpunit.phar
3. Move globally:
   sudo mv phpunit.phar /usr/local/bin/phpunit
4. Test
   phpunit --version

Optionally, if you want to verify the signature, between steps #1 and #2:

1. Download signature file
   wget https://phar.phpunit.de/phpunit.phar.asc
2. Add the public key to our local file:
   gpg --keyserver pgp.uni-mainz.de --recv-keys 0x4AA394086372C20A
3. Verify:
   gpg phpunit.phar.asc

Install WordPress

1. Get a local copy of WordPress
   cd ~/
svn co http://develop.svn.wordpress.org/trunk/ wordpress-develop
cd wordpress-develop/

2. Create a MySql database (replace DATABASE_NAME_HERE, USERNAME_HERE and PASSWORD_HERE)
   mysql -uroot -p -e "CREATE DATABASE DATABASE_NAME_HERE; GRANT ALL PRIVILEGES ON DATABASE_NAME_HERE.* TO USERNAME_HERE@localhost IDENTIFIED BY 'PASSWORD_HERE';FLUSH PRIVILEGES;"
3. Copy the test config file
   cp wp-tests-config-sample.php wp-tests-config.php
4. Modify the variables DB_NAME, DB_USER and DB_PASSWORD in wp-tests-config.php REMEMBER!!!This database gets erased every time a unit test is run so DO NOT POINT IT AT A PRODUCTION DATABASE
   vi wp-tests-config.php.
5. Sanity check by running the official test suite:
   phpunit