I'd be inclined to agree with you except for two things:
1. Michael Massee, who plays the Man in the shadows, also appears to have served as the silhouette of Norman Osborn for the hologram in the Oscorp lobby. The resemblance is pretty uncanny, so that's the general consensus, although it is possible that the general consensus is wrong.
2. The Man in the shadows shares many traits with what many Spidey fans would recognize as Ditko-era Norman Osborn--namely the trench coat, hat, and penchant for hiding in the shadows.
I will say this, though. The man in the shadows clearly disappeared, literally during a flash of lighting. This indicated that although the appearance is likely of Osborn, he was not actually there in the flesh. I previously listed by guesses: hallucination, Mysterio's hologram, or Chameleon with legitimate chameleon-like powers. If any of these are the case, then it's Osborn, but technically not Osborn, so Ifans was telling the truth...kinda.