I see that point and agree to an extent. I think series with strong continuity--I would also add Indiana Jones, Nolan's Batman movies, and Back to the Future off the top of my head--have a richer texture because of it. Then again, most of those have only several installments.
While I am not one of the great lovers of the MCU around here, I respect that they have lasted 11 films without any major gaff.
Be that as it may, the real problems here (other than a few dropped lines in X1) are a prologue in the despised X3 and the plot of the universally loathed and ignored Origins movie. The choice to ignore them led to First Class and Days of Future Past, and Deadpool being much better movies as a result. When people devalue that we have three great movies simply because they contradict two bad ones, I just shrug.