I can add some more fuel to that fire: I read an interview with him just as X2 was being released, where he said he regarded movies like that as "bank" movies--where he could make a lot of money, put it away, and move on to smaller projects where he wouldn't have to worry about the money as much.
I have read that he hated the makeup, I've also read that he possibly didn't get along with Bryan Singer, but that's all gossip, so who knows what's true.
And I can't diss the guy....I saw him on Broadway in "Cabaret" a few years before X2, and he was amazing. My friends waited for his autograph after the show, and he was really nice. Plus I saw him at a CD signing for BC/EFA where he did a Q&A, and again...he was really nice.