I think honestly audiences didn't connect with Hulk very much, and Hulk does very little in the movies to make himself cooler, more sympathetic with audiences. Hulk could've been more of a character in the movies.
I think the other thing is that since he's CG the entire time, it's hard to get into him. Not that audiences can't find footing with CG characters, but when they know that and the character isn't absolutely amazing it can be troublesome.
The problem is they wouldn't let Hulk talk more, even though he at least said Hulk Smash this time. But we don't get enough of the idea that Hulk is really just the psyche of a scared, abused kid. It's almost like they felt if Hulk spoke more often, the audience wouldn't like it.
The other thing is that Hulk also could've been shown to be more heroic. Such as saving innocent bystanders during those fights or trying to protect small animals or children. That CG promotional spot at conventions with Hulk petting the puppy. Stuff like that is great, "Puppy." It also kind of expresses that in many ways Hulk is like an angry toddler.