There's plenty of other ways. Ultron could just be on a much larger scale than we are assuming.
Now maybe I'm being over analytical that's just how I am, school, life, whatever..but RDJ posted a picture somewhere of the cast captioned something fun along the lines of "where the ones saving the universe"
Hear me out,
I think Ultron and Wanda know what's coming, and what these series of events are going to lead to and perhaps Ultron is trying to "protect the world from that" by establishing peace and order. Thor goes on his own quest to probably learn that the Infinity War is coming. During that quest, we hear of Thanos, maybe see a future vision of him. And at some point, maybe we'll see what he's doing currently.
Joss said they have to anticipate that general audiences may have just seen avengers 1, or not even that. Something as big as the ininity gems, and what all of them together can do is so big that it has to be introduced slightly in an earlier movie, they're part of the MCU
Now for the Avengers franchise there are three stories (the final of which is split into two films), we SEE two infinity stones in A1, and realized Thanos pulled the strings, but it's really not directly PART of that Avengers story, we don't realize what's going to happen till after the credits.
Avengers 2, Thanos HAS to appear, and the Infinity Stones seem a bit necessary to transition into Avengers 3 because it has to stay true to an exclusive franchise so that way people who don't watch the other films aren't confused. Especially with infinity stones being the maguffins of two phase 2 movies, potentially three now. Thanos had to be part of it and obviously Joss thought that too.