Out of the six Avengers six of them are white. That's not "pretty well". That's "antebellum south".
I, for one, absolutely love the actors we have in all four "big" roles right now and think they are perfect for their given roles, but how many African-Americans do you think they auditioned for the role of, well, Captain America? How many folks who would be considered to have a middle easterner appearance do you think had even the smallest chance of landing the lead role in Thor? I would suggest that there is probably an Asian actor with the talent and charisma to have taken on the role of Stark but I don't even have to go that far because Banner has been played by three different actors in various quality roles that almost certainly could have been filled by others, and yet all three have been white.
The problem with saying that the best person should be chosen for any given role is that Hollywood almost always casts white, and then THAT gets tossed out as a defense. "Oh, but they were just casting the best possible actor!" The fact is that generations of prejudice have given rise to an unfair system so that even when an honest attempt to just cast the most talented actor is made white actors already have a step up over any minority actor in the same role. It's a cultural bias that is so deeply embedded that they have an undo amount of privileges inherent in the system.
Which is exactly why it's so vital for people to take notice of things like this and to demand change. Because what we have now is base on these little subconscious biases that, in some ways, are harder to deal with that all out racism. At this point if there was someone standing in a film studio saying "we're only going to cast white males for leads - those other people should be glad we're letting them be sidekicks in so many of these movies nowadays!" we could just call that person out and publicly shame them because we all know that is messed up. But no one is saying that. Instead it's predicated on things like "star power" and "biggest comic book name". Well, most of the stars are white males and most of the biggest superheroes are white males because most of our culture is based on old, outdated ideas. The only way to get change is to invite it.
Plus, Black Panther looks so awesome. And he can climb walls! He's like Spider-man if Spider-man were a cat! And king of his own country! So nothing like Spider-man, that's fair.
Sidenote-That-Is-Actually-On-Topic: I had the strangest crush on Quicksilver when I was a kid (I also have this thing about sibling relationships in media - maybe it's just that I'm an older sister myself). I am still in shock that he's in two live action movies. Has he even been in non-live action films? My only regret is that we'll never see Stark and him get into a pissing match over whose father failed them more. Le sigh. But, yeah, I haven't really seen an X-film since X3, and I'm still cool with them sharing.