I think QOS suffers from what I call the "hype" factor. Look at any movie that comes after a highly anticipated sequel, and if it doesn't live up, look at how intense the hate can be.
Spider-man 3 is a perfect example. The hype for that was metoric, only being surpassed by TDK. And the fallout was equally huge. There were people calling SM3 the wost Superhero movie ever, which is ridiculous when you look at some of the truly bad superhero flicks out there (Steel, Supergirl, Howard the Duck...ect) But the anticipation was so high that people felt supremely let down by SM3, which is why they reacted so harshly.
I think Quantum suffers from the same effect. CR was very well recieved and people expected a sequel with the quality that Skyfall produced. So people really reacted strongly against it.
Was QOS a great movie, heck no, but it was okay. Certainly nowhere near one of the worst Bond films ever. Heck, I'd take it over almost any Moore film.