Phantasm shouldn't be in a live-action movie. It wouldn't really make sense. It was a character that worked for one specific story, so in order to really put the character in one of Nolan's movies, they'd have to actually adopt that story. So basically we'd be getting a live action Mask of the Phantasm, which seems kinda pointless.
Only long-term, recurring villains should be used. You can take Joker or Scarecrow and adapt them to film making minor changes to their first encounter with Batman and such. Joker and Batman's first meeting doesn't have to be exactly like The Man Who Laughs. But taking someone like Holiday and putting them in a film that's not The Long Halloween seems kinda off, do you get what I mean?