Except he never does anything to make him the most feared bounty hunter. In ESB, all he does is follow the Falcon to Cloud City. He may have alerted Vader, he may not have. Either way, Vader and the Stormtroopers did all the work for him. Fett was nothing more than a delivery boy bringing the frozen Han to Jabba. Could he have made it more easy for him?
Then in ROTJ, he's a complete failure when he does attempt to do anything. Just look at the clip below. He flies towards Luke, pulls out his blaster but doesn't even fire which allows Luke to cut it. Then he fires a cable on Luke who just uses it to pull him causing him to fall. He gets up, turns around, Han whacks his jetpack and he smashes into Jabba's ship and rolls into the Sarlac pit. Everyone makes escaping from it seem like some incredible feat to make Fett seem like less of a loser. The only thing missing from that scene is the Benny Hill music.
The only thing Fett has ever had going for him was his name and his armor.