The Avengers is much better. The storyline was better, it was more balanced, had better structure and was more fully formed. The Avengers felt much more organic and natural. The pacing and editing in Avengers was alot better. Alot of things in AoU felt forced, rushed and underdeveloped.
Loki was a better written and acted villain and was given the proper time to develop. With Ultron they skipped the reasoning of why he was "born" evil in the first place. I thought they skipped over important parts of his development and left alot of his storyline up to us to piece together instead of showing it. Example being how did he grab the scepter when it was said he escaped the party via the internet, where did he get the material to built his first body, didn't show much about how he built an entire army of robots, didn't do a great job showing how much time had past for him to do all of that, etc. Yes, these are all things that are fairly easy to figure out but I would have liked to see these things play out on screen, it would have given Ultron more texture/depth. Plus he pretty much fails at every turn, doesn't accomplish much and gets defeated relatively easily.
Loki's plan was better planned, better developed, better executed and came closer to being realized. I still think it was Hiddleston's best performance of the MCU by far as well, while James Spader was rather bland to me. Ultron wasn't very intimidating or menacing, and actually came off kind of incompetent half the time. There were two many jokes from Ultron to take him seriously, it felt like he was always trying to make you laugh instead of intimidating you. Couldn't take him seriously.
The Banner/Widow relationship felt extremely forced, illogical and unnecessary. Basically feel the same way about Hawkeye's secret family. Thor's vision quest was silly, sloppy and unnecessary. I don't like that that is how Thor figured out someone is out there behind the scenes manipulating the Infinity Stones, or that he figured that out at all, I would have preferred it be a complete surprise to the Avengers that Thanos is behind it all when he starts to attack people in possession of the Stones. I also thought it was pretty stupid how easily the Twins decided to join forces with Ultron and it was awfully convenient that Nick Fury showed up exactly when they needed a way off the floating City.
Don't get me wrong, AoU is still a pretty cool movie, the Hulkbuster scene looked amazing, Scarlett Witch was a cool addition to the team, the story was good enough that I never lost interest and it had alot of cool moments. But I liked the writing and the acting of the characters better in Avengers, the storyline was better, the villain was better, Loki/Chitari felt like a much bigger threat, tension was higher, the comedy hit it's marks better and wasn't as repetitive or overused and it felt far more natural and developed overall. Avengers is one of my Top 5 favorite comic book movies ever, AoU just makes my Top 5 from the MCU itself, and with Civil War coming up it probably won't last long. AoU is good, Avengers is great.