I hate to just baselessly speculate, because we have no inside information to draw any conclusions from, but to me it seems like Druckman and Gross would rather put their energy into projects that they have more creative control over than a show where they're basically work for hire. Druckman might've had more influence than Gross, but at the end of the day this is Mazin's show, and they're nothing more than players in his sandbox.
I won't speak for them as to their feelings on certain changes made in the adaptation process, but there were clearly issues in making Ellie's timeline fit in just seven episodes. With session three at least eighteen months away, and likely another eighteen until the story of Part II is completed, working on the show might end up feeling like a distraction that's not worth the mental real estate, when that time could be spent more productively on personal projects. Whether that's a potential Part III or something else, I don't blame them for leaving. It does, however, leave me less confident in the show.
I thought season one did a respectable job of adapting the game, but it's clear that the fragmented storytelling of Part II was too much of a challenge for the realities of TV production, and mistakes were made in deciding how to best allocate their limited time and budget. Did we really need the Jackson battle if it meant sacrificing Hillcrest and more time developing Jesse as a character? I can only hope that the relatively straightforward nature of Abby's section will lend itself more easily to the small screen.