I see this kind of opinion a lot, and it's clearly resonating in the comments. I'm beginning to think most people are just very bad at writing reusable code. Despite all the negativity, it is actually possible to write highly-reusable code that is clean, understandable, and not "over-engineered". Why have you all given up on that?
My take is simply that the easiest code to repurpose is small code - because small programs are easy to read and easy to change.
Putting something behind an interface / abstract factory / whatever ironically often makes it harder to change because you have to read more lines of code before you can start modifying the program. Programs with a lot of lines of code are intimidating.
There are situations where you have clean module boundaries, which lets you separate your code into separate independent modules. (Eg POSIX separating user applications and libc / the kernel). But the downside is that once you've separated your code into modules (with documented APIs) the module boundary itself becomes harder to change. You're essentially betting that you have the right API. You lose flexibility at the boundary in exchange for flexibility in the code on either side of that API.
But all that said, I don't think there's any way to learn this stuff without experience. "The dao that can be spoken is not the eternal dao". There's no set of rules for this stuff that can be explained simply, and which will give you the ideal outcome in all situations.
Putting something behind an interface / abstract factory / whatever ironically often makes it harder* to change because you have to read more lines of code before you can start modifying the program.*
I think the “secret” is proportionality. A useful abstraction hides significantly more complexity than it introduces. That could be a single function, if that gives a name to a simple but common algorithm on a particular data structure like map or reduce. Or it could be a 100 function API with 100,000 more lines behind it, if that provides a correct, efficient implementation of some entire field of programming that your application rests on. But it’s probably not a one-line function that needs a longer name to describe what it does in words than its whole implementation, and it’s probably not a library with a 100 function API that only obfuscates a few standard data structures and custom types that are locked away in the implementation for little benefit.
> it’s probably not a library with a 100 function API that only obfuscates a few standard data structures and custom types that are locked away in the implementation for little benefit.
Thanks for expressing this - I think this is a fantastic insight. I love C as a language, and I've been really enjoying Rust lately too. Both languages make it easy to design around simple, raw structs. And a simple struct with 3 public fields is often a much easier interface to work with than 100 getter & setter methods hiding the same struct which has private fields.
I have more success writing reusable libraries that do very, very fundamental stuff or add ergonomics to existing features. For example, in Go when generics landed I immediately wrote a few small libraries using them. They're in use all over the system I work on.
When I try to build similar abstractions over business processes, they typically don't survive the next wave of changing requirements.
I have more success writing reusable libraries that do very, very fundamental stuff
I concur. If we look at abstractions that have stood the test of time, they are often relatively simple to describe, but they express powerful ideas that are widely applicable. A few examples come to mind:
• Structured programming
• Iterators and generators
• Promises
• Atomic transactions on databases
• Model-view-whatever patterns in UI code
• Regular expressions
• Pipelines
• Continuations
• Haskell’s standard typeclasses (particularly Functor, Applicative, Monad, Monoid, Foldable and Traversable)
Many of these have proved so useful that popular languages now directly incorporate them or perhaps, for the most abstract ones, incorporate features that are more specific instances of the general idea.
I think the last example is particularly telling. The patterns Haskell developers work with are in one sense very abstract: each is ultimately just a short list of mathematical properties that some type can have. This turns out to be quite a high barrier to entry and understanding their true nature is something of a rite of passage that many fledgling Haskell developers never complete. It also took some very smart people many years of thought and discussion before the community eventually reached the list above.
However, once you do recognise the patterns, you see them everywhere. They create some very clear seams in your software design that allow separation of concerns and composability to a degree that less powerful abstractions only dream of. And because they’re rooted in relatively simple properties, they’re about as stable and future-proof as anything in programming can be.
Maybe one day they’ll be supplanted by a more effective abstraction with better language support. Effect systems, perhaps? But for now, you can express ideas in a few lines of Haskell that would take 10x that in many other programming languages, because you can compose tools from a vast toolbox of data structures, algorithms and design patterns in almost arbitrary ways. But you have to learn some abstractions that aren’t widely known and have silly names first. :-)
As an old coworker used to say: code reuse in the small (libraries) and in the large (databases) are solved problems, but code reuse in the medium is Hard. I agree, though I find it difficult to understand or explain why.
p.s. I also wrote that same library when go generics landed.
The team manager at work likes to bring up the list of "time investments which we will benefit from later" which we never benefited from later. It's a good point that usually its better to wait until the benefits are obvious rather than predicting at some point there will be some benefit later.
My general strategy - usually applied on a much smaller scale - was the first time I needed to do something similar (or the same) I'd copy the code. The second time, I'd consider abstracting it. By that time I had three applications and could better identify what was common and what was specific to a particular usage. I wonder if something like that can be applied to larger libraries.
> it is actually possible to write highly-reusable code that is clean, understandable, and not "over-engineered"
You might be able to do that. That code (most likely) still needs to solve a complex issue and needs documentation to survive next to it. Also, there is a good chance that the next guy coming to fix it is not going to investigate the extra time to fully understand the code if he can fix that one crash by throwing in a hotfix at the specific line accessing a nullpointer. Lastly, at some point, if you need to solve a difficult problem, the code to do so is likely going to be more difficult to read, too.
It's not impossible to do this, but having the time to do a clean solution, having the foresight to be right about the necessary complexity and also have the environment for it to stay clean is simply rare.
> it is actually possible to write highly-reusable code that is clean, understandable, and not "over-engineered"
This is true for systems where you already have working code and you have multiple concrete examples of code that would benefit from refactoring or rewriting a reusable module.
I think the skepticism is more toward the idea that you should design for reuse up front before you have working code. There are two failure modes. One is that you spend time making something reusable that you only end up using in one place. The other is that you bake assumptions into the code that later turn out to be false. In both cases, the resources spent on reusability could have a net negative impact on the project.
Part of the problem is discoverability. As the library gets larger, without adequate time invested into docs or standardised naming to enable faster code-based search, people can end up writing their own functions when one already exist. Often these are not discovered unless PRs are reviewed by someone that either wrote or has experience with a reusable function.
