Well, maybe that poster that is "perceived to be an enemy" will work harder to gain more respect. Or, they just won't care. There wouldn't be any negative consequences for having a low Rep, but those that want a higher rep would be more careful to not upset others. 
 
Plus, if you limit how often someone can vote a rep point, like one per poster per day, or even just one vote a day, you eliminate the ability to gang up on someone. Who cares if a person gives a bad rep vote to someone, because it would be a one off vote. If that Poster has friends, it could make up the negative vote. But, if everyone just doesn't like the quality of the post, that would affect the rep points, and that would tell the poster he/she should clean up their act. And, that is only if they care about it to begin with. 
 
Just like on Xbox Live, if you have a crappy gamer, that no one has fun playing with, others will vote him down, warning others not to play with him. He'll have to act better to get higher votes. Letting others know that he is ok to play with. 
 
On the hype, if you are having a conversation with someone with a low Rep, you'd know that they can be aggressive, or just mean, but if they have a high rep, then you know you can have a pleasant convesation with them. Like, I'd give you a postitive vote in a heartbeat.