Well, while the actors may be different (which I know can be irksome for the sake of continuity), TIH for whatever purposes, is still being treated as a part of the large continuity in the MCU.
I mean, it was stated in the SHIELD website that had been opened prior to the release of Thor that both Erik Selvig and Jane Foster are from the same university that Bruce Banner and Betty Ross work at, and Erik Selvig is also a known associate of Banner as well.
Plus, we have the footage of the campus attack being shown in the SHIELD Warehouse in Iron Man 2, we also have that special short film that's being released alongside Thor called "The Consultant" that ties the gap between Iron Man 2 and the end of TIH.
Hell, the Super Soldier Serum is also a main element in TIH as well.
Personally, I just think of this case being similar to cases like how we had two actors playing Dumbledore in the Harry Potter Series, and how despite the fact that they had a new actor playing the character after the second film (due to the first one having passed away), they still used clips from the first two films in latter films for flashbacks and even had flashback scenes that took place before the events of the first film using the second actor.