I don't have an issue with how they handled the supporting cast in FC because the film is not about them. It's primarily centered around Xavier and Magneto's relationship. Even so, Vaughn still did a better job with the supporting cast then Singer could even dream of when he worked on the first two X-Men films. Jean, Scott and Storm were really nothing more than window dressing and that is a fact that no one can deny. Even if the film had been titled Wolverine you would still need likable and relatable characters in your supporting cast. The rest of the X-Men had so little to do that you barely even got to know them.
As for budget, you keep bringing this up, but 75 million was a very decent budget in those days. Schwazenegger's the 6th Day was only 80 plus million. Let's face it, X-Men as a whole would not have been improved with an additional 20, or even 40 million in Singer's hands. He's proven that even with a monster budget he doesn't really know how to utilize the money properly. Both SR and Avatar cost the same! The only difference is Avatar's marketing budget was $150 million, but that has nothing to do with the budget for the film itself.
Kate Bosworth, Brandon Roth, Halle Berry, Rebecca Romijin were absolutely laughable as casting choices. Even Famke and James Marsden were bad casting choices for their respective roles. Casting better known established actors like Spacey and McKellen hardly takes a huge leap of faith. Patrick Stewart has been the choice to play Prof X since TNG came out and was a no brainer. I would never trust a director who makes the types of bad decisions in casting that Singer does.