I never enjoyed the first Avengers. Far too much humor, so much so that it undermined the drama, and the plot was threadbare. There wasn't very much balance in how the characters were treated (felt like Iron Man & Friends) and Loki was turned into a caricature and a joke. Another gripe I had with it was that it felt like a TV movie. The sets were small and self-contained, the CGI was crap and the final battle was very underwhelming.
To me, Age of Ultron improved on all of those points. Maybe I've just gotten used to the comedy, but it didn't bother me so much this time. There was a lot more going on in the story and I actually cared about the characters for a change (aside from Natasha & Bruce). Ultron was a great villain, maybe the MCU's best (aside from Loki in the first Thor). Even with the number of characters this film had, to me short-changed was Thor (and even there, he still has another solo film to expand with); I really grew to like the new additions. This film was considerably larger in scope, and the sets & locations helped in emphasizing that. Whedon's skill-craft has clearly grown since the first film. The CGI was still wonky at times (the opening, Ultron and his horde in the final battle), but it was tolerable this time. It didn't have any outright bad shots like Tony & Pepper in Stark Tower at the end of the first film.
So for someone who hasn't liked the way the MCU has moving in the last few years, and went into this film expecting to dislike it, I was pleasantly surprised by how much I enjoyed it. Easily my third favorite MCU film, maybe even my second (Iron Man 1 is still the best for me).