I'm well aware that it doesn't stop a devoted attacker. But it needs a devoted attacker. That is the import point which you also recognized (beside many others). Even a devoted attacker has it harder because cracking a code with known algorithm and unknown key is easier than cracking a code with unknown algorithm and unknown key. Even more if the unknown algorithm is not available.
I think your argument that the effort put in custom ciphers maybe should be put in mainstream ciphers instead is interesting.
Tinkering around with custom ciphers can teach you a lot. Maybe you have not the knowledge or no idea how to attack/improve mainstream ciphers.
However, if you can make a difference for mainstream ciphers, of course that's what we need.
Just to put the issue in an extreme perspective, suppose that the best mathematically possible attacks against AES-256 in some setting only reduce the attacker's work by the same factor as the best mathematically possible attacks against AES-128. (There's no proof of this now, but it's conceivable that it's true.) In that case, the decision to use AES-256 in a particular application instead of AES-128 improves security against cryptanalysis of AES by a factor of 2¹²⁸ steps for the attacker. (Maybe cryptanalysis of AES isn't actually the weak point anyway, but let's set that aside because that's what inventing new ciphers tries to address.)
If this hypothesis is true, the work that Daemen and Rijmen did to invent AES-256 and the work that a particular implementer did to implement it will produce an almost inconceivably vast security benefit against this particular threat.
The reason this is important is the kind of disproportionality between the effort of Daemen and Rijmen and the AES reviewers and implementers, and the magnitude of the resulting security benefit. They might have spent a total of 500 person-years on making AES-256 work well, and received a security improvement of 340 trillion trillion trillion trillion-fold relative to whatever the security of AES-128 is. Whereas a homegrown cipher that isn't very mathematically sound might be developed with 1 person-year of effort and end up make an attacker do, let's say, 100 trillion operations. In my hypothesis, Daemen and Rijmen and other folks then got somewhere between a trillion trillion trillion and a trillion trillion trillion trillion trillion trillion trillion times better security return on their effort.
Now you might reasonably point out that if you use a standard, known cipher, the attacker's costs for a direct brute force attack are purely computational and don't involve research and development, or attempting to suborn or hack your correspondents or colleagues to discover the principles of operation or your system. Whereas if you do have a homegrown mechanism in play, an attacker incurs these other kinds of novel and sort of one-off costs, notably including making other human beings think about stuff more.
The point that I've taken from a lot of the security experts who've talked about this, though, is that the scaling benefits are the important factor here, again especially if you want to make a system that many people could use for a long time. When the limiting factor is computer time, which is really only likely to be true for systems created, refined, and reviewed by experts, you can sometimes get the really absurd security ratios that are hard to even think about, and require your adversary to spend more money than exists in the world, build more computers than can be made from all the silicon on Earth, consume more energy than the Sun outputs, etc., etc. When the limiting factor is human reasoning, you might say "but that would require human cryptographers to think about my system for 1 year!". But if that's so, that may actually happen, and in any case you can't easily get the order of magnitude of the costs and resources required up to "inhuman" levels.
The point I'd take from your idea is that it could be valuable to try to make adversaries incur diverse costs in attacking your system, especially if you don't know what capabilities and resources your adversaries do and don't have. This is kind of akin to what's happened with key derivation, where people have proposed KDFs that are very CPU-intensive and also KDFs that are very memory-intensive, and if there are other sorts of resources that you could make an attacker burn, there are probably people trying to invent KDFs that burn those, too. It's not clear to me that there's a genuinely scalable way to require human analytical effort as one of those resources, but if there is, that could be a useful property for communications systems to have for defense in depth. But making up a new cipher by hand for every system is probably not going to provide that property very reliably, or be a very effective use of resources, again when other uses of resources can improve security to a staggering extent.
I think your argument that the effort put in custom ciphers maybe should be put in mainstream ciphers instead is interesting.
Tinkering around with custom ciphers can teach you a lot. Maybe you have not the knowledge or no idea how to attack/improve mainstream ciphers.
However, if you can make a difference for mainstream ciphers, of course that's what we need.