Here’s the basic code for creating a snapshot on the pool:

#!/usr/bin/python
 
import subprocess
from datetime import date
 
def take_snapshot(pool, snapshot_name):
    command = "zfs snapshot -r %s@%s" % (pool, snapshot_name)
    p = subprocess.Popen(command.split(' '))
    p.wait()
    if p.returncode:
        raise Exception("Error executing '%s': %d" % (command, p.returncode))
 
try:
    snapshot = date.today()
    snapshot_name = "%d-%02d-%02d" % (snapshot.year, snapshot.month, snapshot.day)
    take_snapshot("vault2", snapshot_name)
except:
    exit(1)

This is coupled with this crontab entry:

5  0 * * * /root/zfs-snap &> /tmp/zfs-snap.output \
 || cat /tmp/zfs-snap.output | mailx -s "zfs-snap error" me@example.com

If you want a full-featured solution, you might take a look at zfsbackup

I also am splitting my blog – music posts will be at thedavidofmusic.com

Export/Import

Export was very straightforward – I just went to Tools->export on my old site, but the import was more complicated.

First, I had to install the import plugin, then when I went to do the import, wordpress complained about permissions. I set the wp-content/uploads folder to 777 temporarily, at which point the import worked.

I then exported the import, and found a few links that had not been migrated (all of which were links within the regular text in posts), which I manually fixed, then re-imported.

I also went through and modified posts containing code to use WP-Syntax instead of the old code blocks.

Since then, technologies have advanced and the structure of the internet has progressed. Many services provide encrypted peer to peer chat, which is inherently resistive to wiretapping.

The Obama administration is drafting a bill requiring that all communications services be capable of complying with wiretap orders. Setting aside privacy implications (especially in the context of the Bush administration’s warrantless wiretapping), the added burden on communications services will slow innovation and herald an era of unnecessary beaurocratic regulation on what is our generation’s frontier.

The creation process will modify the build server, though the resulting tarball will be completely clean (can be untarred into any similar server with identical software versions and file locations without modification).   You may want to wipe your environment after successfully testing deployment.

Creation

export CDIR=/opt/python-2.6
mkdir py-tmp && cd py-tmp
# get needed libraries
# if any of these fail to install for whatever reason, you will need to grab the source for the closest available version to what is running on your system, and install it manually into its standard location (specific examples found below).
yum install -y wget zlib-devel openssl-devel readline-devel mysql-devel gcc-c++
 
# now for Python itself
wget http://www.python.org/ftp/python/ (get your own updated link)
tar xf Python-2.6.*.tgz
cd Python-2.6.*
./configure --prefix=$CDIR
make
make install
cd ..
 
# install setuptools (easy_install)
wget http://pypi.python.org/packages/2.6/s/setuptools/ (get your own link)
PREV_PATH=$PATH
PATH=$CDIR/bin:$PATH
sh setuptools-*.egg
PATH=$PREV_PATH
 
# install the coveted mysql-python package
wget http://downloads.sourceforge.net/project/mysql-python/ (get your own link)
tar xf MySQL-python-*.tar.gz
cd MySQL-python-*
$CDIR/bin/python setup.py build
$CDIR/bin/python setup.py install
cd ..
 
tar -czf python-2.6.tgz $CDIR

Deployment

tar -xzf python-2.6.tgz -C /

To use this python binary, you can modify your $PATH, adding $CDIR/bin before /usr/bin or wherever your other version is installed. To make a python script use this version, either specify the full path on the scripts #! line, or run this:

$CDIR/bin/python script

Issues
If any of the libraries fail to install (possibly due to crufty servers), you will need to install them manually to their standard location. Here are some examples of what I’ve had to do. Be sure to grab the same version of the software as is already installed.

wget http://www.zlib.net/zlib-1.2.3.tar.bz2
tar -xjf zlib-1.2.3.tar.bz2
cd zlib-1.2.3
./configure
make
make install
cd ..
 
wget http://www.openssl.org/source/openssl-0.9.7a.tar.gz
tar -xzf openssl-0.9.7a.tar.gz
cd openssl-0.9.7a
./config
make
make install
cd ..

Here is something quite important: if you cannot install mysql-devel, I recommend just grabbing the devel rpm from mysql.com. Be sure that the mysql version matches as closely as possible to that installed in your system (the version should be spit out when you try to run yum install mysql-devel). Also, the glibc version must match your system exactly.

wget http://dev.mysql.com/get/Downloads/MySQL-4.1/MySQL-devel-4.1.22-0.glibc23.i386.rpm/from/http://mirror.services.wisc.edu/mysql/
yum install Mysql-devel*

Being at a university where the first language taught is Java, but transferring from an institution where C++ was dominant was a bit of a pain.  IntelliJ got me through my Java-based courses.  I tried and managed with Eclipse and NetBeans for the first semester – but we were given a significant initial framework.

Enough of my praise for IntelliJ.  Let’s get started!

• 1. Download IntelliJ

• 2. Install IntelliJ

• 3. Let IntelliJ know where the JDK is (Linux only)

Assuming that a JDK is already installed, run

updatedb; locate jdk | grep jre$

Do something like the following based on those results

echo "export JDK_HOME=\"/usr/lib/jvm/java-6-openjdk/jre\"" >> ~/.bashrc

• 4. Install ‘La Clojure’ Plugin

Run IntelliJ (linux:bin/idea.sh), open the plugin manager, and search for ‘la clojure’.   Select it for installation, and click OK.

• 5. Create a Clojure Project

Click ‘Create New Project’, select ‘Create project from scratch’, give it a descriptive name, and select ‘Clojure’ as a desired technology.

• Start a REPL!

Tools -> Clojure REPL Actions -> Add new Clojure REPL

This post is password protected. To view it please enter your password below:

Password:

Furthermore, if any of these steps would cause you trouble (deleting .emacs* comes to mind), you probably already know how to handle it.

Ubuntu:

sudo aptitude install emacs git-core ant openjdk-6-jdk

- Evaluate the install code
- OR evaluate at the cli (warning: rm -Rf ~/.emacs* might not do what you want)

rm -Rf ~/.emacs*
git clone git://github.com/technomancy/emacs-starter-kit.git ~/.emacs.d

emacs

M-x package-install

(M-x corresponds to ALT-x in emacs)

M-x clojure-install

echo "(eval-after-load 'clojure-mode '(clojure-slime-config))" >> ~/.emacs.d/user'name.el

emacs foo.clj

M-x slime

Other notes:
- For cli setup, see Tim Riddell’s tutorials – I recommend the bash startup alias, in particular.
- For a longer explanation, see Phil Hagelberg’s post.

OpenDNS:
basic web filtering

Gizmo:
make and receive online calls

Google Voice:
very nifty phone enhancements

(the above two can be combined to give free skype-in/out type service)

One of my hard drives is making sounds that indicate that it could fail at any time; In order to prevent any losses, I ordered a new drive. I spent a while copying all of my data from one drive to the next (I generally have 2 drives in my computer at a given time; the drive closer to failure contains data that is broadly available online, so losses should be minimal if anything happens, while my personal data is semiannually burned to disc and regularly synced to my brother’s computer). At any rate, I finished the migration, and started updating my install on the new drive.

The next morning I got up and found that my computer was making a horribly disturbing sound. The update process had blocked on a prompt for my acceptance of a license agreement. Gkrellm told me that data was not being transfered, so the sound was simply the spinning of the platters. I ran “hdparm -y /dev/olddrive” and “hdparm -C /dev/olddrive” to stop my old drive to make sure it wasn’t responsible for the sound, and then recorded the sound for documentation purposes.

I have a pretty good recording setup, so the recording is probably about as good quality as is practical – or even better. I heard it as soon as I entered the room, so it was fairly loud – but my system is also fairly quiet. The sound at the very end is the sound of me clicking a (fairly loud) mouse from a foot away from the mic – which is about the same distance as the mic was from the disk.

At any rate, I’m in the habit of keeping multiple copies of my data around until I trust my drives and I need more space, so I was able to undo a few changes I had made to my older drive to make it usable as the master – so I should have a marginally useful system until the real replacement is mailed to me.

Wiping the Disk for RMA
As I had put nearly all of my data on the new drive, I want to wipe it before returning it – but I also want to return the drive asap. I had always used DBAN to wipe drives, but I figured I should be able to do it from my normal install.

I first found shred, which seemed like it should be quite effective (and useful for shredding specific files, depending on the file system). It puts random data over the drive in several passes, and it optionally zeros the drive on the final pass. Unfortunately, I noticed that it was taking a long time – it uses data from /dev/urandom, which means that the random number generator was cranking like mad. To sum it up, the bottleneck was the CPU, not the hard disk.

I next tried dding data from /dev/urandom over the file (this was before I completely understood how shred works). Not surprisingly, it was CPU bound as well (I’m guessing that disk throughput was at about 10% of its capacity).

Finally, I found wipe, which uses special patterns, much like DBAN, as opposed to pseudorandom data like shred. As a result, the data is probably slightly less securely wiped (although it should also be more consistent in its results), but the bottleneck is now the disk. I expect that it will take 3-4 hours per pass, as opposed to 30-40 hours per pass.

New Seagate Barracudas (.12)
This reminds me, the new barracudas seem to be getting great throughput, at least – though I can’t speak for their reliability. However, I should mention that the packing job was not as good as it could have been, so my issues should probably not be blamed on Seagate, at least not in their entirety. I bought the OEM drive from Newegg, but the bubble wrap they used has perforations so it can be torn easily. Unfortunately, one of the perforated lines lined up perfectly with the edge of the disk – not a good thing, as the perforations pop the bubbles along the line.

I actually took the bubble wrap and tested it to see how much room for error they had in packing. As it turns out, they have about 1.25 inches of give on either side before one of the corners will not be protected by the bubble wrap. It’s enough allowance so that it’s not generally a problem, but just the same, it may easily have been a factor in my experience.