He's the guy who got the cast together, knew who the right kids were, created the look of Hogwarts, and found the cinematic way to present Quidditch. Not to mention everything else that's remained a staple to the very end of the series. So, in my view - he set everything up for the others to come in on.
All the other directors seemed to bring were muddy camera lenses, a constant gray sky and needless gothic design patterns. While the books did get darker, I highly doubt they got to the point where that would be the right palette. I'm not even sure how many directors got brought on between him and the last guy because they all blended together way too much. What I do know is that after Chris, the only guy to last was the last guy they brought on. Which tells me he was the only one similarly capable of capturing both the studio's, the author's, and the audience's attention or else one of those in-between directors would have stuck around. You want maturity and growth, you go after the themes and character dynamics - not making something look overly gothic at times. Maturity is one thing, scarier creatures is one thing, unnecessary darkness is completely another. To me the in-between HP films are, at times, even darker than the darkest scenes of LOTR - which wasn't the case for the guy who introduced it to film or the guy who finished it in film, which tells me that probably wasn't the case of the books in-between and rather what the other directors did.