Angel has 200+ years of fighting under his belt. Both with and without a soul... Plus the number of years he spent in that hell dimension after Buffy defeated him in the season 2 finale. I mean sure, Connor was raised in a horrible place, but he only spent about 16 years there.
Also, I always thought although Angel had his soul, he still had the same urges Angelus had... So a portion of his will power, or strength was used to keep Angelus at bay. Perhaps when he fights, and he has no reason to hold Angelus back, he's that more powerful.
That's the way I saw it anyway. As powerful as Connor was, he wasn't very disciplined. He let his emotions get the better of him during fights countless times. So let's say he is more powerful than Angel. Angel having more experience in both battles and manipulation could exploit Connor's weaknesses and easily defeat him.
Maybe I'm over thinking this, but that's the way I've always seen things.