Actually, several people in my circle, including myself, thought well of Ruffalo's performance. We certainly enjoyed the spectacles that were the fights, but also enjoyed character development parts of the movie.
I think the Hulk has a head start in popularity because he has been in several movies and TV shows at this point. And that's probably because his powers can be written to be easier on a special effects budget.
I also wonder if Hulk was more popular in the books from way back when. (I don't know for certain, I just suspect so). So, going back to before there were any superhero movies (which almost would have been even before the Hulk TV show), who was selling more books?
Having said all this, I'm sure people will love Thor when his second movie comes out. They certainly enjoyed the first movie!