If a comment really is factually incorrect then a downvote is appropriate, but some comments that at first glance appear to be factually incorrect are really just opinions.
I'm continually downvoted on HN because I disagree with most on here on things related to copyright.
It is becoming pretty annoying, mainly because HN is becoming more like /r/politics (NSA, government, big business are all evil), rather than talking about cool hacker tech.
Of course, one should reply with a correction, but in reality, maybe that's not always what happens.