I think the people being emotional about this need to get over it.
I think recently Favreau has shown that he's sort of done with Iron Man and ready to move on.
For starters these things:
1. Favreau didn't want to do Demon in A Bottle. According to Bob Layton, RDJ really wants to do it, but Favreau does not.
2. Favreau doesn't want to do stories that have types of magical or fantasy stories in the Iron Man realm, so nothing that isn't tech-based or isn't realistic. He couldn't really get over that hump and I think that's why he wasn't picked for Avengers either.
3. He's got lots of other projects lined up right now, so he should be fine.
4. Its just, he just hasn't acted serious about wanting to lead and be the visionary for the franchise or other Marvel movies. I think he basically had his take and he's stepping aside now. What is wrong with that?
I think the only thing that would hurt the franchise is losing Robert Downey Jr.