Okay I tracked down some sources myself. He talked about it on the Howard Stern show
Stern reportedly asked Baldwin why he’s never been offered a super-villain, which set off the following exchange.
I have, but I won’t mention which one. I don’t want to undo somebody because they hired someone else to do it. But they asked me to play a villain in one of these Marvel Comics movies.After a follow up, Baldwin said:
I wasn’t available, they wanted me to work a certain time. And because of my wife’s pregnancy, I had to turn down two films this summer that I wanted to do.
The speculated possibilities were: GotG (The Collector), Winter Soldier, X-Men (I'd guess Bolivar Trask), ASM 2 (Rhino or Osborn).
I'm not familiar enough with GotG to judge there, but that aside I'd guess Trask or Osborn. Baldwin to Redford just seems like a really strange jump in casting.