You need to have enough HM karma to see the downvote button. A long time back it used to be 100, no idea if it still is. Have an upvote to get you a step closer.

You'll get a downvote at 500 karma.

I’m quite liberal with them, I think it’s one of the things that stops this place turning into Reddit.