I've had this go both ways before, too. I think the point to take away from this sort of experience is that the best you can do is apply your best judgment and re-evaluate as you go along. Often opaque documentation makes sense after you gain some context by reading the source; it works the other way, too. With experience, you gain a sense for what path to go down; sometimes that (subconscious) heuristic is totally wrong, which is why it's important to consciously re-evaluate on top of that. The current approach isn't making you any practical headway, even though you're learning a lot about the system in question? Take a step back and try looking at the bigger picture; what are you really missing? If you can, sleep on it for a bit.

Most of all, if you're getting it right more than 50% of the time (and I'm finding it hard to imagine a way you'd do worse than that) there really is no point beating yourself up about it.