Use pyramid. It's exceptionally well designed. It has very sensible defaults out of the box, but literally everything about the framework is swappable if you need it to be.

I can't comment on rails, but I spent quite a long time with django before switching to repoze-bfg (pyramid's previous name). In my experience you don't hit the same wall that you tend to with django when you need to step off the the common path.