Didn't Nolan say TDK was 'the rise of The Joker'?
I take it that Joker got all of his henchmen when he killed the Chenchen. "Tell your men they work for me now." His followers, the sadists, and mentally ill of gotham who mistakenly look up to Joker "His name is Schiff, Thomas. A former patient at Arkham. The kind of mind the Joker attracts" are probably the ones who will bust Joker out, and probably end up getting killed by him shortly after.
Also, why does everyone think Joker is going to Arkham? Remember? The Narrows was destroyed in Batman Begins. Arkham's in the Narrows. Arkham's gone, people.