After a few re-watches of this film, I really think that this should have been 2 films, a-la the last Harry Potter or Hunger Games. The avengers is meant to be the big shiny jewel of the MCU, and quite frankly topping the first avengers was going to be an impossible task. By making it into 2 movies Ultron could have become a truly fantastic villain, the new characters could have been better fleshed out.
The reason I believe that AOU didn't succeed as well as the avengers was mainly because the avengers had very little to introduce, basically every character/concept/plot point had been introduced in previous movies. The only new things were Maria Hill, Hawkeye (despite his Thor cameo), and the Chitauri (didn't need much explanation). They were able to dive right into the story. AOU had a lot more to introduce. Strucker, the twins, Ultron, Vision, Klaw, Dr Cho. All that introduction chewed up a lot of time.
My broad concept would have been using Strucker as the bad guy in the first movie (instead of being totally wasted). The movie opens the same, but Strucker gets away instead of being captured. Ultron is created, but is initially more child-like, and works with the avengers to help bring down Strucker. This would allow James Spader to develop Ultron from a neutral, naive character to the 'evil' entity that we ultimately see. Over the film Ultron would slowly make his decision that humanity are the planets real enemy. In the final battle Ultron would kill Strucker, despite being told not to by the avengers. They would then take him down, but his essence would escape in another robot/computer. This is the point that twins would decide that they need to join up with avengers. The film would close with Ultron showing up in a new, better body.
In the 2nd movie, the opening would be the avengers taking down ultrons minions in various battles, and Ultron deciding he needs his own super soldier to help destroy the avengers. He creates Vision, but Tony manages to interject JARVIS into the body during its awakening, giving the new creature conflicting thoughts. Much as Ultron came to his realization about what constitutes good and bad, Vision does too, but comes to a different conclusion. During the final fight with the avengers, Ultron kills Quicksilver and this finally convinces Vision that Ultron is the true evil. He joins with the avengers to defeat Ultron once and for all.