The Homeward Bounders by Diana Wynne Jones. Yet another of her sadly out-of-print titles, I think I found this one on Powell's website at some point. This was a very strong story with an interesting twist at the end.
A boy named Jamie is wandering through his city when he hops the fence of a strange building. The garden is completely silent and the building is occupied by men in hooded robes. Jamie comes back another time, determined to find out what is going on in this strange place. Unfortunately, the men are Them, beings that play with the fates of those on many worlds. Jamie's discovery has made him a liability to their game and so they throw him out into the boundaries of another world. He is able to travel between worlds but cannot ever join in to their lives in any meaningful way. He is always tugged out of the world when a move is completed -- after hours or months. He, and others like him, are all "homeward bound". If they make it back to their home world, they can rejoin the game.
This was actually quite a sad book. Jamie is only twelve when he begins wandering through worlds of danger, war and desolation. And from the first pages of the book, we know that things aren't exactly going to work out for him. Still, it's heartening when he finds comrades in his journey and Jones really does a fantastic job with this story. It was well built and compelling. It didn't get me to push through until 5am but I did power through it the next day!
Safe and home,
You can find a used copy of The Homeward Bounders on Amazon or Powell's or at your local library. We bought a used copy.