Suppose the further back in time one goes the more likely it is one is dropped back in to the future.
A first moment at t = 0 is impossible to get back to since one gets probabilistically, randomly, dropped back to the future where (when) one just came from (you get to keep the memories of your attempts to go back in time).
So going back in time would be like: 1. going back in time, 2. getting kicked back to some time away from t = 0. In our spaceship we note 'we've just been here...' 3. going back again, 4. probabilistically dropped back out to the future again, repeat.
Note: this idea was motivated by quantum mechanics, but I'm not claiming it's true, just that it's interesting.