Having never been a man, I wouldn't know. And assuming that you have never been a woman, how could you know?
I know both girls and boys who take "chastity vows", and they both say it's hard to keep them; especially if they date.
It's not something that's harder for one or the other. My roomate didn't start dating until recently and now has sex on a regular basis, now he has a whole different perspective on things. Thinks it's foolish to not have sex within a relationship or before marriage.
So it's all a matter of situation and perspective. It's only hard if your life makes it hard. Someone who takes chastity vows but still wishing to date around is going to have a lot harder time than a girl I knew who used to shun every form of male attention under the sun. She's not having a hard time remaining celebate, guys just don't like her.
Similarly your attractiveness, your ability to deal with people, the places you hang out, what you do for fun, your parents are going to have an affect on this as well.
It's pretty easy to remain celebate, just lock yourself up in a room all day...you'll be celebate for a while. But at the same time, if you embrace and experience the non-celebate world around you on a day to day, up close and personal way that is going to affect you.