I'm not sure why they call them that. They're places where pokemon trainers go to have their pokemon challenge those of the person in charge of the gym. If they win, they get money, skills they can teach their pokemon, the right to use certain moves outside of battle, and sometimes items. It's useful to gain Fly and Surf and the ability to use them, at least, because it can decrease travel time if you have a pokemon who can use Fly, or allow you to travel in water without a boat. But the pokemon will need to rest like anyone else if they do use them. The pokemon seem to like fighting, or at least mine do; otherwise I would have just continued to walk everywhere.

Otherwise, there are things called warps in the cities that will take you to other towns, but only the ones nearest the city you're in and only if you've been to the other cities before. You're not allowed to warp to places you've never been.