How to enable `git grep -P` on OS X using Homebrew

If you're using Homebrew to manage software on your OS X Lion box, and using the version of git it provides, you may have noticed that you can't use the -P flag to git grep to enable Perl-compatible regular expressions (PCRE). It turns out the default formula doesn't enable that option when building git. Rather than have to override this every single time you upgrade git, it's far easier to add this line to your .bashrc (or .zshrc):

export USE_LIBPCRE=yes

After you do that, you also need to brew install pcre, because if you don't you'll get a build error when you next attempt to build git. It will look something like:

==> Upgrading git
==> Downloading http://git-core.googlecode.com/files/git-1.7.8.3.tar.gz
File already downloaded in /Users/hank/Library/Caches/Homebrew
==> make prefix=/usr/local/Cellar/git/1.7.8.3 install
GIT_VERSION = 1.7.8.3
    * new build flags or prefix
    * new link flags
./generate-cmdlist.sh > common-cmds.h+ && mv common-cmds.h+ common-cmds.h
/usr/bin/llvm-gcc -o hex.o -c -MF ./.depend/hex.o.d -MMD -MP  -O3 -w -pipe  -march=core2 -msse4.1 -I. -DUSE_LIBPCRE -DUSE_ST_TIMESPEC  -DSHA1_HEADER=''  -DNO_MEMMEM  hex.c
/usr/bin/llvm-gcc -o ident.o -c -MF ./.depend/ident.o.d -MMD -MP  -O3 -w -pipe  -march=core2 -msse4.1 -I. -DUSE_LIBPCRE -DUSE_ST_TIMESPEC  -DSHA1_HEADER=''  -DNO_MEMMEM  ident.c
/usr/bin/llvm-gcc -o kwset.o -c -MF ./.depend/kwset.o.d -MMD -MP  -O3 -w -pipe  -march=core2 -msse4.1 -I. -DUSE_LIBPCRE -DUSE_ST_TIMESPEC  -DSHA1_HEADER=''  -DNO_MEMMEM  kwset.c
/usr/bin/llvm-gcc -o levenshtein.o -c -MF ./.depend/levenshtein.o.d -MMD -MP  -O3 -w -pipe  -march=core2 -msse4.1 -I. -DUSE_LIBPCRE -DUSE_ST_TIMESPEC  -DSHA1_HEADER=''  -DNO_MEMMEM  levenshtein.c
/usr/bin/llvm-gcc -o list-objects.o -c -MF ./.depend/list-objects.o.d -MMD -MP  -O3 -w -pipe  -march=core2 -msse4.1 -I. -DUSE_LIBPCRE -DUSE_ST_TIMESPEC  -DSHA1_HEADER=''  -DNO_MEMMEM  list-objects.c
/usr/bin/llvm-gcc -o ll-merge.o -c -MF ./.depend/ll-merge.o.d -MMD -MP  -O3 -w -pipe  -march=core2 -msse4.1 -I. -DUSE_LIBPCRE -DUSE_ST_TIMESPEC  -DSHA1_HEADER=''  -DNO_MEMMEM  ll-merge.c
In file included from revision.h:5,
                 from list-objects.c:8:
grep.h:43: error: expected specifier-qualifier-list before ‘pcre’
make: *** [list-objects.o] Error 1
make: *** Waiting for unfinished jobs....
==> Exit Status: 2
http://github.com/mxcl/homebrew/blob/master/Library/Formula/git.rb#L51
==> Environment
HOMEBREW_VERSION: 0.8.1
HEAD: 730f9b7fbc30a74348223fa78e1ede295dc73340
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CELLAR: /usr/local/Cellar
Hardware: dual-core 64-bit penryn
OS X: 10.7.2
Kernel Architecture: x86_64
Ruby: 1.8.7-249
/usr/bin/ruby => /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby
Xcode: 4.2.1
GCC-4.0: N/A
GCC-4.2: N/A
LLVM: build 2336
Clang: 3.0 build 211
MacPorts or Fink? false
X11 installed? true
==> Build Flags
CC: /usr/bin/llvm-gcc => /usr/llvm-gcc-4.2/bin/llvm-gcc-4.2
CXX: /usr/bin/llvm-g++ => /usr/llvm-gcc-4.2/bin/llvm-g++-4.2
LD: /usr/bin/llvm-gcc => /usr/llvm-gcc-4.2/bin/llvm-gcc-4.2
CFLAGS: -O3 -w -pipe  -march=core2 -msse4.1
CXXFLAGS: -O3 -w -pipe  -march=core2 -msse4.1
MAKEFLAGS: -j2

Error: Failed executing: make prefix=/usr/local/Cellar/git/1.7.8.3 install
These existing issues may help you:
    https://github.com/mxcl/homebrew/issues/6820
    https://github.com/mxcl/homebrew/issues/6971
    https://github.com/mxcl/homebrew/issues/7462
    https://github.com/mxcl/homebrew/issues/8030
    https://github.com/mxcl/homebrew/issues/8913
    https://github.com/mxcl/homebrew/issues/8977
    https://github.com/mxcl/homebrew/issues/9017
    https://github.com/mxcl/homebrew/issues/9023
    https://github.com/mxcl/homebrew/issues/9435
    https://github.com/mxcl/homebrew/issues/9538
    https://github.com/mxcl/homebrew/issues/9574
    https://github.com/mxcl/homebrew/issues/9618
Otherwise, please report the bug:
    https://github.com/mxcl/homebrew/wiki/checklist-before-filing-a-new-issue

It's that little line grep.h:43: error: expected specifier-qualifier-list before ‘pcre’ that clues you in that you have a PCRE problem.

Back to flipping out...

Book Review: Python 3 Web Development Beginner's Guide

I recently received a free review copy (eBook version) of

from Packt Publishing. I was looking forward to this book, because I haven't really done much Python 3 work yet, and I wanted to see how it could make my life as a web developer better. However, the book wasn't what I expected. Instead of covering the basics of web development and how Python 3 applies, it is more of an introduction to the sorts of concerns that come up when you build a web framework on top of CherryPy. The sample code just happens to be in Python 3.

The Good

The two best parts of the book, to me, were the coverage of writing a jQuery plugin, and growing an ORM that uses metaclasses to provide a compact, readable way to define the models.

The Bad

I have a rather long list of things I didn't like about the book, some of which are a function of the title setting misleading expectations, and some of which I think are just problematic in general.

In general, I didn't care for the examples. Some of this is personal preference: I find that many people (myself included) learn better when they must type in the examples instead of opening up the code and reading through a completed solution. While the book sometimes indicated that something had been left as an exercise to the reader, opening up the sample code showed that the exercise actually had not been left to the reader. This mismatch between what the text of the book says will be in the sample code and what is actually in the sample code occurs in multiple places throughout the book, and gives a sense that the book was sloppily edited.

I also felt the examples in general were too complicated. It's fine to build up a complicated example over the course of a book, but instead we got a task list, a wiki, a Customer Relationship Management (CRM) tool, a spreadsheet, and more. That's an awful lot to distract you from the beginner's principles that you would expect in a book with this title.

I also didn't care for many of the shortcuts taken in the book. In most instances, the book did acknowledge that the approach taken was not appropriate in the real world, but then proceeded with little or no justification for why it was done the way it was. The two examples that really leap out in this category are the password hashing scheme and the failure to use a template engine.

When the book first introduces authentication, it explains that you should never store passwords in plaintext. This is absolutely correct, but the book then goes on to demonstrate a completely insecure password hashing scheme: UNSALTED SHA1. The author only provides a cursory link to explain what you should actually be doing. In this day and age, demonstrating anything less than a bcrypt-based solution is wrong. Read Enough With The Rainbow Tables and How To Safely Store A Password for a far better explanation than I can provide. There's really no excuse for this: the added complexity of using py-bcrypt instead of writing your own (insecure) SHA1-based solution is trivial at worst; there's a strong case to be made that it would actually be simpler.

The failure to use a template engine (also a weakness acknowledged by the book) really makes the code harder to follow than it should be. Virtually any serious web development effort is going to take advantage of a template engine, and for good reason. This code gives me flashbacks to my days of writing Java servlets before the advent of JSP, and I saw where one other reviewer invoked the specter of PHP. The fact that this style of coding draws such comparisons should give you an idea of just how unpythonic it is. I would be sympathetic to claims of not wanting to add too many external dependencies if the book did not already rely significantly on the magic of jQuery UI.

My last major complaint is simply one of focus: the book spends substantial amounts of time growing an ORM and teaching Python metaclasses (and doing a good job of it), but spends little more than the bare minimum required on CherryPy (which is at the core of the code), and essentially none on understanding HTTP. In fact, the few times it comes up is usually in relation to GET vs. POST, where the decision is usually made based on inane implementation details such as whether request arguments are logged by default instead of HTTP fundamentals such as idempotency, safety, or cacheability (although caching is mentioned elsewhere, in the context of how to prevent it). Also, the book does mention security, but it does not give it the sort of omnipresent emphasis that is necessary to write good web applications, given the hostile nature of the domain. XSS, CSRF, and SQL injection attacks all deserve much more attention than they were given.

The Summary

The book has some good content mixed in with the stuff I didn't like. Unfortunately, the good content is rarely specific to web development. For example, the chapter that uses metaclasses to clean up the ORM is one of the better resources on metaclasses that I've seen, but metaclasses are clearly not specific to web development. Furthermore, the impression of sloppy editing makes it hard to put as much faith in the content as it probably deserves. Given these flaws, I really don't think I'd recommend this book to a friend who was looking to get started with web development.

Back to flipping out...

Now Reading: Python 3 Web Development Beginner's Guide

Packt Publishing has kindly given me a free review copy of the eBook edition of

I'll be reading it and publishing my review here and on the PyATL Book Club blog.

Back to flipping out...

Announcing My First Real Open Source Project

I ported the dark-background variant of the Solarized project to Pygments. You can see the results here.

Back to flipping out...

Note to Self: A NetBSD domU Doesn't Have An Entropy Source By Default

I discovered this after a curious problem led me to email my VPS provider and get back a pointer to mailing list exchange. Possibly more interesting than the original problem, I also learned that this exists.

Back to flipping out...

Note to Self: Source Django's bash Completion Automatically

If you're using Doug Hellman's awesome virtualenvwrapper to manage your Django projects—and you really should be—try adding the following line to your $VIRTUAL_ENV/bin/postactivate script:

source "$VIRTUAL_ENV/build/Django/extras/django_bash_completion"

If you used the --no-site-packages option to create the virtualenv, that should automatically source the Django bash completion script everytime you workon into your project. If you didn't, you just need to figure out where the Django bash completion script is squirreled away on your system and use that path instead. BTW, --no-site-packages should really be the default.

Back to flipping out...

Note to Self: Changing info for the postgres system user

Do you use MacPorts? Ever switch from one version of PostgresQL to another, e.g., 8.4 to 9.0? Did you notice that it started complaining about a non-existent home directory for the postgres system user? Me, too.

My first instinct was to edit the user, but there is no entry for postgres in /etc/passwd. That led me to this excellent article. Using that info, I cooked up the following command:

sudo dscl localhost change /Local/Default/Users/postgres NFSHomeDirectory /opt/local/var/db/postgresql84 /opt/local/var/db/postgresql90

Back to flipping out...