Also lets not forget that he was filming Superman Returns at the time they were filming X3 so thats another factor. I'm kinda annoyed that Bryan Singer hired him for that film when it was obvious that foX was releasing X3 in 2006. Same thing with Anna Paquin as she was filming something else during X3/X4. Bad timing also played a factor.
However that being said, all of the original cast eXcept the ones that appeared in Logan were done dirty. They should have headlined more films especially after Days of Future Past. There were so much they could have done with the present day Storm, Beast, Cyclops, Rogue, Kitty, Iceman and Dr. PhoeniX Grey.
As for Mystique, I don't think she was done "dirty". Her film adaptation was inaccurate for the most part but people who have zero clue about the comiXs think she's one of the biggest X-Men characters because of the movies. Inaccurate and all Mystique was still NOT undershown.