> Decorators are a very "high" level of abstraction. Leave aside dependency injection, clojures and currying and all that jazz which worms its way in here.
I don't follow. They're magic. Magic (and fancy programming) is generally bad in a professional, software engineering environment. If you're writing software for you, do whatever you want. But I'd you're writing software for your peers, dependency injection, closures and currying (and all that jazz) are significantly more literate and easier to follow.
Decorators nearly universally have hidden side effects through hidden state.
Hidden state is not bad. That's the entire premise of Object Oriented programming: we hide the state with encapsulation and only expose methods that act on that hidden state. Closures are exactly equivalent to objects. (Makes sense, given how JS uses closures to implement objects and how Java uses classes to implement lambdas...)
What's bad is global state, but in Python specifically - and in any language supporting modules and/or being class-only like Java - the globals are actually confined to their own Singleton instances (ie. the modules or static members of classes).
Hiding state in a closure is exactly equivalent to hiding the state in private fields on an object. It's not magic. It's just an alternative formulation of the same idea. In Python, it's slightly less convenient to use due to some peculiarities around scopes, but it's a valid technique nevertheless. You seem to agree, saying that "closures and currying [...] are significantly more literate and easier to follow".
But then, I don't understand what is your problem with decorators. They're just a convenient syntax for higher-order functions, that often happen to be also closures. You said they are easy to follow; the same should be true for decorators, then.
Dependency injection, included in your list of easy to follow things, is often implemented with decorators in Python.
Basically, decorators are just a special syntax for the following pattern:
def f(): pass
f = some_higher_order_function(f)
and nothing more. How complex or "magical" you'd like your `some_higher_order_function`s to be is your choice, but that has literally nothing to do with decorators syntax...
I would agree if you singled out descriptors, metaclasses, or stack manipulation, but things like decorators or context managers are simple enough to be your everyday tool for writing Python code.
Tiers of information abstraction and interfaces are a very important part of programming.
> it's slightly less convenient to use due to some peculiarities around scopes
The cognitive load of closures and currying is pretty extreme for newer programmers.
I'm sure, given a certain level of experience and ability, people can stare at closures and not blink.
However, I'd be hesitant to use closures in a team of other developers. If you don't have an education in functional programming, a closure definition is quite a lot to dump on folk.
I tend to avoid decorators etc in the majority of my applications where I'm working on a shared codebase with coworkers etc.
All abstractions are a trade-off between "how easy is it for the writer " vs "how easy is it for the reader."
Decorators have a "hurdle" rate of complexity for the writer, and need a sufficient number of readers to justify.
> If you don't have an education in functional programming, a closure definition is quite a lot to dump on folk.
That's true, I agree. It's not good when you have to explain a piece of code to every single developer (or even a majority of them) that comes to the project.
It would be different in Ruby or JS, but Python developers tend to be more procedural and OO oriented, with FP being less widely known and used.
> All abstractions are a trade-off between "how easy is it for the writer " vs "how easy is it for the reader."
Yes, definitely. If you can solve a problem with simpler tools, you should do that. If you can solve a problem in a way that's easier to follow, you should do that, even at the cost of some repetition. All abstractions leak, and if you can solve a problem without introducing yet another leaky abstraction - you should do that.
On the other hand, if your abstraction is tight enough not to leak too much, you can often get away with it by simply documenting how to use it. Huge masses of Python programmers use Django ORM, for example, without ever caring about the metaclasses and descriptors that make them work. Many of the more junior devs don't even understand that the ORM is basically a builder for SQL queries. It's not ideal and they sometimes introduce problems to the codebase by doing N+1 queries where 1 query would suffice (for example) and they become lost at the first sight of `QuerySet.raw` method. Until they hit one of these, though, they can still add value to the project.
> Decorators have a "hurdle" rate of complexity for the writer, and need a sufficient number of readers to justify.
Agreed. The complexity of creating a closure that wraps a given function to add a useful logic to it is definitely higher than simpler solutions, like template method (for example). And while using a decorator has little syntactic overhead, there's a cognitive overhead should the new developer need to change the decorator's logic.
In other words: simple is best. No argument here :)
I don't follow. They're magic. Magic (and fancy programming) is generally bad in a professional, software engineering environment. If you're writing software for you, do whatever you want. But I'd you're writing software for your peers, dependency injection, closures and currying (and all that jazz) are significantly more literate and easier to follow.
Decorators nearly universally have hidden side effects through hidden state.