Because of his change in appearance as he ages while he's gone, nobody ever has a chance to become acquainted with Clark's more matured appearance. The only time they see Clark as a full-grown man is while he wears glasses and his hair is neat and parted. If, instead, Clark had become Superman while in college, where many many people had a chance to memorize his face without glasses, they'd just see Superman's picture in the paper or on the news or even up close and they'd recognize him as Clark. The sudden change in personality would be jarring too. The 12 year gap makes it believable that Clark's personality could have changed over the years.