And also, Captain America: The Winter Soldier. No one expected it to be a big hit and, although CA got a little popularity due to the first Avengers movie, it wasn't enough to justify The Winter Soldier's box office success.
The one thing Deadpool, Guardians and TWS have in common--they all had great scripts. There is the formula. You want to bet on something, bet on that.
Here's the bottom line--Apocalypse just wasn't very good. I know there are good moments; I was grinning big during Quicksilver's scene and LOVED the opening scene in Egypt. But overall, the script was weak, there were so many characters that no one got a decent arc--none of it coalesced. None of the victories felt earned (and don't get me started on how Magneto could haphazardly murder thousands of people, and then get PRAISED for helping to take down Apocalypse at the very last second--what a joke).
Forget about getting rid of Singer--he's proven he can make a solid X-movie (X1, X2, DOFP, and was greatly involved in FC). Fire Kinberg. There's your weak link. When he's on his A-game, he's good, but there's no telling when that will be. He's so inconsistent, and when he doesn't know what to do, he just decides to kill people and create melodrama. He's so overrated--he has to go.
You want to good X-men movie? First and foremost, it's about the writing. If you have a solid script, the audience will follow. It really is that simple.