Yeah, they did appear in X-Men first, but that does not make them X-Men characters. They've literally only been in a couple dozen X-Men issues since they were introduced 50 years ago and only recently started being heavily used in X-Men/mutant storylines, especially Wanda. People don't even realize that Fox will have to alter their actual origin, since it involves things Marvel has the rights to. But, since they're known as being Magneto's children, it's Marvel who are "needlessly" using them and are "handicapped". >.>
I realize those characters can be worked into the script, but it's a matter of whether the time is right and if it makes sense. People don't think about those things. Why would they have Ms. Marvel be in Avengers 2 before anything Kree and Captain Marvel related have been introduced? Why would they have Ant-Man and Wasp in Avengers 2 right before their origin movie? And why would they have those things together? Those are big characters and they need time to be introduced less you waste their potential. It's just not time for them and people need to be patient about it. Not to mention the Maximoffs joined the Avengers before any of those characters sans Wasp and Ant-Man and were a part of only the second incarnation of the Avengers roster, so from even that perspective it makes sense to have them in the sequel.
/rant