Who here remembers how badly Peter kicked the @ss of Flash Thompson in Spider-Man 1?
Flash Thompson was a very big, athletic guy. And even throwing extremely rapid and well-placed punches, he looked to be moving in slow motion to Peter.
Without the serum, Harry won't even get one punch in on Peter. And even if he did... it wouldn't even hurt Peter. Parker can take hits from a Green Goblin who is 8 times stronger than a normal man.
For those who don't think 8 times stronger is a big deal, imagine being in an arm wrestling match with someone TWICE as strong as you (and uses technique equal to yours). You'd lose almost instantly. A guy with three times your strength would probably smash your hand when he slams it down in victory. At 8 times stronger, your arm is probably gonna just tear off from the force.
So to Parker, a punch from a normal person is practically a pat on the back. The fight would be over before Harry knew it because Peter would be able to attack him so much faster and stronger.
So yeah... no way Harry has an intial fight without serum. Because he'd be defeated in literally 10 seconds.