Um. . . the Fantastic Four probably spend more time off-world than anyone but the Silver Surfer.
I guess it just depends upon the which issues you are referring to - or what decades, etc.
When you look at their first 100 issues - those that really defined the book - they were in orbit in #6,
on planet x in #7,
the moon in #13,
the micro-world in #16 [but still on earth],
the Skrull planet in #37,
the Negative Zone in #51 and #62,
the micro-verse for a few issues in the 70s [again, still on earth],
a Skrull planet for a few issues in the 90s,
and the Negative Zone for annual #6.
So that's about 15 % of the first 100 issues - not really "most" of their adventures.
Again, I guess it changes depending upon the creative team, etc.
If it's a good story, I'm all for it...