Blofeld killed Tracy too, right in front of Bond and then he went to ground. Bond's reaction in the novels wasn't at all what it was in the Diamonds Are Forever PTS or the rage driven pursuit that we saw in Licence to Kill and Quantum of Solace. When Vesper died, Bond was angry at her and at himself, but when Tracy died it was pure grief. He was utterly depressed and self-medicated with pills, he drank to excess, buried himself in unfulfilling sex. The point, is that he had no reason to live anymore. M sees that plainly and gives him the mission in Japan, the impossible mission, as a chance to find redemption. And through that mission and his immersion in the Japanese culture and with the help of Henderson, Tiger Tanaka and Kissy Suzuki, Bond regains the will to live. That's the entire concept of You Only Live Twice. It's not a revenge story, it's the story of how James Bond found a reason to live. The Blofeld stuff is only coincidental, and just a form of final catharsis before Bond moves on with his new life.
I don't see any reason why M couldn't catch wind of the problems in Japan and then decide to track down Bond and offer him that same chance at salvation, even if he did leave the service.