I was sticking up for the recast of Banner by pointing out that of all the solo films leading in to the Avengers, TIH was probably the least important of them. Iron Man 2 includes major setup as to what the initiative is looking for and what it may consist of so the Iron Man series is very important, Captain America (I assume) is going to have a plot point that is going to lead into or be resolved in The Avengers, and Thor is a character somewhat unfamiliar to the GA, and a bit complicated since he brings otherworldly elements to the series, hence why his own solo film is important.
People know who the Hulk is, they know he doesn't enjoy being a heartbeat away from pandemonium at all times, and there was all ready a film of him within a decade from The Avengers release. Therefore, if someone had to be recast before The Avengers film came out, it's probably best it's Banner since TIH is the least pivotal of those movies.