Aren't app specific passwords simply single-factor auth tokens, thus defeating the purpose of using two factor auth? Wouldn't a sufficiently complex pass-phrase be just as secure?
No, because each app has a different token and you can revoke them individually.
Google does want people to use OAuth instead of app-specific passwords as much as possible, but sometimes that's not possible or just isn't done. Then you at least get some security from not posting the same password to a million places.
The latest version of Chrome dev (21.0.1155.2 dev-m) does not require an app-specific password - it works with the OTP if you have it enabled, but at least for me the sync is broken at this point.