NOBODY looks like that. Somebody who knows more can correct me if I'm wrong, but I'm pretty sure you can get insanely strong just lifting weights if you do it right and long enough, but a bodybuilder's physique is another level entirely.
It requires a strict diet, an exact exercise regimen, etc. Nobody who spends his life travelling the wilderness is going to look like that. Big and strong aren't necessarily the same things. And of course, there's the fact that though bodybuilding existed back then, no one back then looked like today's bodybuilders, so why would Howard envision something like that art?
Besides, Howard at one point called Conan "sinewy." Look it up and see if it makes you think of Arnold. People's perception of Conan is largely influenced more by Arnold and Frazetta than Howard's words.
Here is a thread with more knowledgeable Conan fans than I:
http://www.conan.com/invboard/index.php?showtopic=6135