Docs engineer here. Nice to read about writing style: would you consider creating a writing benchmark at some point? I guess y'all are painfully aware of the load-bearing issues (pun intended).
Agreed on the first experiment, that's why the post says it hit the ceiling. The others are not saturated. I'd love to find a way of making the first experiment more discriminating.
It also redirects the agents from adding enormous sloppy comments in the code - I have a linter that only allows 1 line comments with a reference to a docs folder tag `See BOB-IS-UNCLE.md`
Then you can ruthlessly prune the docs folder on PR review and regular passes when e.g. claude really wants to write a novel about the glorious intricacies of bugs it's solved.
Indeed. Another thing worth exploring is how much the position of instructions matter; Claude claims to read docs as a whole unless they're very large, though it might be that it only grabs the first lines of a doc sometimes, in which case there is an "above the fold" effect.
ha this is the first time i've seen this but honestly yes, if there was an accessible enough for a 4 year old OS to "explore" around in to find the applications I would be throwing it on the USB as I type
reply