I agree with all of this. And there's some definite solid reasoning for believing this.
1. Thanos is The Avenger's greatest threat (that Marvel owns the film rights to

) This seems to be the main argument around these boards as for why he should be saved for Avengers 3 and built up over both Phase 2 and 3.
2. Loki's scepter could contain the Mind Gem, "He who put the scepter in your hand." Remember Loki did just fine with his own spear in Thor and it had no blue gem in it to speak of. Assuming it was the Mind Gem, we will likely see the rest of the gems in future films. But there does not seem to be enough films in Phase 2 to introduce them all.
3. Joss Whedon states, "By being smaller. More personal, more painful. By being the next thing that should happen to these characters, and not just a rehash of what seemed to work the first time." Thanos is not smaller than Loki and his army. Nor is he more personal.
4. Ant-Man is several years away from Avengers 3. To tease or setup Ultron in Ant-Man (2015) will lose most of its spark by the time we see Ultron in A3.
5. Remaining baddies (other than Ultron and Thanos) include MoE, Kang, Korvac and several others that don't quite have the impact of Thanos, let alone greater impact.
If Avengers 2 was more conclusive, I'd say hell yea bring Thanos on and thrill us all. But since we already know Phase 3 exists, I believe the Mad Titan will bide his time until Avengers 3.