I think the hate for X3 has more to do with its deviation from the source material than the quality of the film itself. I went back and rewatched it after Singer was announced for Days of Future Past and found it to be very entertaining, suspenseful and epic in the right moments. For 2006, the CG work looks terrific on bluray and I enjoyed the film as much as I did The Avengers.
I think if you can look past the changes made to the Phoenix, you get a pretty great summer blockbuster out of X3. I'd still prefer a longer film with all the footage excised restored in an extended cut some day.
I can't say I have any complaints with the acting as Stewart and McKellen were selling every line they spoke IMO. I could do without Dania Ramirez altogether though.