Micheal Keaton was hardly the embodiment of male physical perfection. He was pretty slim, not especially muscular, and not especially tall. Did it stop us accepting him as a believeable Batman? No.
Same goes for George Clooney and Val Kilmer at the times. Good looking guys with decent physiques, but they certainly weren't ripped or hugely muscular.
It was only when Christian Bale came along that we got a Batman who was pretty muscular, and looked strong even without the suit - like he does in the comics.
However ........................... I agree that Batman's physical appearance should be something a bit more out of the ordinary than just an average slim guy; it's not critical to the role, but it certainly helps. He's obsessed with what he does, and that obsession should extend to his training regime. Even without training, he spends his nights doing hugely physical things - violent confrontations, swinging off rooftops, etc - all of that will have aided his physique in some way too.
With Snyder's particular interpretation of Batman in mind (older, seasoned), I don't think he needs to be cut and chiselled in the same way that Cavill is. Cavill's Superman is genetically perfect and that shows in his body. I think they should focus more on Batman just being big and solid, with real power behind his punches and kicks. I liked the approach in Rocky Balboa where he got back into boxing as an older man, and his trainer said he no longer had speed, or could run or spar like a young guy ......... but he had power, and they could rely on blunt-force-trauma. Batman isn't that old yet, but if he is seasoned and late 30s/approaching 40, he won't be as quick as when he was younger. Having more power balances that out.
I like the idea of approaching Batman in a similar way, where his aesthetic matches his physicality - he's not a sharply chiselled sculpture, more a big solid block of granite which blocks out the light! Affleck is tall, and broad, which is a great start ........... and as long as he can pack on muscle and size (which he looks to have been doing) I wouldn't particularly care if he has a visible six-pack or if we see him shirtless at all. He will look powerful, which is all that's needed for this Batman.