Fox likes the X-Men movies to be huge and have more of a focus on large scale action and plot than intricate character interaction. It's the exact opposite for Avengers. I would like to see an X-Men film that kept the team to the traditional Chris Claremont group (Professor X, Wolverine, Storm, Cyclops, Jean, Beast, Angel, Iceman, Colossus, Rogue, Kitty Pryde) fighting against a singular enemy instead of another team up group (Brotherhood, Hellfire, etc).
They haven't even attempted to touch on the villains outside of those groups in X-Men mythos, and some of those are far better and more interesting concepts.