I was in the exact same situation as you a few years ago. And, yeah, just find some new friends. You don't have to COMPLETELY ditch them. I see my "old" group once every month or so for a few hours and still talk to them via facebook. But I know what you mean. You just sit there with nothing to say because you simply can't relate. Same thing happened with this girl I was seeing. She was big into drinking and trying new drugs etc and I just broke it off because besides seeing a movie and going to eat every now and then there was nothing else we had in common. She was funny and I liked doing the little we did together but we really had 2 different worlds.
I've been hanging out with some people now for the past 2 years or so with interests very similar to me and it's great. It's great to be surrounded by people that like what you like and respect what you do. Makes life much happier.
Just leave your old friends while you still can. Don't give into what they're offering. And don't feel bad. When they hit 30 or so they'll ask, "Where the hell did my life go?".