Navy Seals, Rangers, Spetnaz whatever other type commandos out there are all overrated in my opinion. It doesnt matter if they have over $100,000 in education and training put into them (each), they can be taken out by a 5 dollar, 20 year old AK-47.
Batman would wipe the floor with Punisher in a fist to fist fight, in the comics or even outside the comics (ie. Movie), but again Batman wouldnt kill him either, because he's almost godlike and forgiving like that.
Punisher, if brainwashed like he was back then (late 90s) by Spook (when he killed Nick Fury thinking he killed his family), would probably know that he wouldnt stand toe to toe, and would sneak up and probably find his batcave and blow it to bathell and then shoot his butler, and all his friends before he even got to Batman. When batman was filled with rage and desperation, he would shoot him in the back of the head when he didnt even know it.
In terms of a fight I'd say - Batman. In terms of life or death - Punisher.