Not to mention Heroes for Hire and The Inhumans! I'm all for expanding the MCU beyond the Avengers.
I'm with you on not talking about it until later, but it's probably wise to expect (and accept) that it will be brought up pretty much every other day. By the same people.
I think we are going to see a pretty quick expansion of the MCU when SHIELD starts airing in the fall. We'll get more heroes and villains which will make the world seem bigger, which is a great thing. However, the reason this keeps getting brought up is because no matter how big and comprehensive they make the MCU, it will continue to have a large hole in it without Spidey (Wolverine less so). And here is the thing about Spidey, the current iteration of Spider-Man fits pretty well into the MCU. If some agreement could be reached to reference him, Spider-Man could continue to have his own adventures apart from The Avengers.