Practice makes perfect.

That said, if you don't see yourself having a car or moving to a place where you'll need a car any time soon, it'll kind of be a waste of time. My sister doesn't own a car, but still drives out of the city to see friends (where public transportation doesn't reach easily) a few times a month and it keeps her skills fresh.
She still wouldn't dare to drive in LA though.
I mean, you wouldn't forget HOW to drive, but 90% of driving is knowing how to deal with other drivers. Or pedestrians, or cyclists, etc etc...