Indeed; I also remembering reading on how the director for TIH said that if they didn't get a sequel for the Hulk before the Avengers, then the plan was to go with Hulk becoming a menace.
Also, I remember Kevin Feige commenting on how they (Marvel) had realized that they had a problem later with how the scene with Tony and General Ross was because when they had later realized that they wanted to go down the direction with a more heroic Hulk in the Avengers, they had to figure out a way on how they were going to explain the last scene in TIH, hence why we got "The Consultant" short in "Thor".