That scene, in my opinion, sums up why Heath Ledger's performance was so brilliant.
Yes, Joker was intimidating in that scene, but he was downright hilarious, too. Ledger balanced the two magnificently.
I'll never forget seeing The Dark Knight in theaters for the first time (it was a packed theater, at a midnight/premiere screening). The audience reaction to the "magic trick" was amazing. First, there was that scared "oh," then there was laughter, and then there was loud applause.
Yeah. Ledger's Joker=lightning in a bottle. I don't see something of that caliber happening again anytime soon.
EDIT: Whoops, my brain was reading Joker's first scene with Gambol, not the "Why So Serious" scene, for some reason. My mistake! But yeah, you're right. He was frightening in that sequence. First time I saw the movie, I had to advert my eyes because I didn't know what was going to happen next. A+ example of a movie building up tension, suspense, and fear without having to resort to jump scares and buckets of gore.