In the novels, Mr. Tumnus appears in The Horse and His Boy and briefly in The Last Battle, with the former taking place during the reign of the Pevensies in Narnia, which begins and ends in the last chapter of The Lion, the Witch and the Wardrobe. So, you're probably right about him being dead in Prince Caspian, as it's set approximitely 1,300 years after The Horse and His Boy/The Lion, the Witch and the Wardrobe... don't quote me on this, though.
The books don't follow sequentially, e.g. The Magician's Nephew (publication order: 6/7) is a prequel to The Lion, the Witch and the Wardrobe.