For me, it takes away from the specialness of the core Avengers to have loooooads of other super powered people.
"Whoa...that building just exploded and people are in danger...let's call Stark."
(Yay we get to see Iron Man in action...I love it when that happens)
"Nah...Fire Guy's right here...let's just call Fire Guy"
(aw...it's just Fire Guy

)
It just feels like Tron:Legacy...I couldn't care less about Flynn's SON...I wanna see Flynn...or the Cosby Show with Olivia...you get me, right?
It takes the attention and screen time off the main characters I've come to know and want to see.