I think there are a couple of factors here.
First, yeah that audience was always interested in spectacle and with the advances in modern special effects Hollywood can deliver more spectacle than ever before. Even the best television can't deliver spectacle the way movies can.
Also, the rise of international box office has lead to an emphasis on image over dialogue. Also, given the tough times since 9/11 and the recession, I think the worldwide audience is in the mood for escapism. I think it's a bit of a coincidence that the mix of top earners turned to fantasy and science-fiction in 2001, 2000 seems to have a good mix of genres among the top earners, but not total coincidence.
The 24-hour news cycle perhaps has lead to less of a desire for topical films, such as All the President's Men, as topics are beat into the ground long before a movie production can get off the ground.
Merchandising has become more important since Star Wars. You can't sell Remains of the Day lunchboxes or My Dinner With Andre action figures, but you can sell all the superhero action figures you want.
Lucas (in the 70s), Spielberg, Scott, and Cameron were incredible filmmakers who created the taste of a generation or two.
The star system has collapsed to a great extent. There are still stars, but they can only do so much to get people into the seats for dramas. Special effects also don't have the luxury of demanding raises.
The long form drama on television, specifically HBO, has proven to be tremendously satisfying for a lot of viewers.
Demographics of movie going have shifted so that it's teens instead of adults that go to the movies and there's no real reason for teens to be interested in adult dramas. Maybe as more Baby Boomers retire and have time on their hands, it will shift somewhat. Also, with the window between release and home video decreasing, opening weekends have become more important than ever. How many adults with jobs are really going to go to a Thursday, midnight opening?
I do think it's probably peaked to an extent that now that everything is possible, just throwing special effects on the screen, save for some real breakthroughs such as Avatar and Gravity, has lost a lot of special-ness. We've also gotten to the point where most of the classics of fantasy and science fiction have gotten to the big screen. I don't think fantasy and science fiction is going away, for all of the reasons above, but I don't think unlimited growth is in the cards either.