Bio shock infinite is what comes to mind when I read this. The cycle keeps repeating itself. You know the outcome, you want to change it, you can but you don't. Repeating the same thing again
I think altering anything on purpose in the past would always create a paradox. Here I go.
You want to change something in the past. You time travel back to said event. You change it to how you think the outcome would be better/best/the way you want it. Here's where the paradox starts. If you change it, then in the future the outcome would be the way you want it. If it's the way you want it then you would not have needed to go back and change it. If you didn't go back and change it then the change never would have happened because you never went back to change it. This makes the original event happen which restarts the whole cycle again.
Yikes! There are so many things wrong with this story. For starters, people born with both sex organs are %~90 sterile, %~9 male sex organs work, less than %1 female sex organs work, but under no circumstance will a person have both sex organs that are fertile. Second, wouldn't the drifter remember what s/he looked like when s/he was younger? I mean, we're only talking about 7 years passing. And would you honestly want to stick your 25yo penis into your own 18yo vagina? But let's pretend that that's OK, do you realize that if you could some how become pregnant with your own child over and over again, each time your child would be more and more severely inbreed. We're talking worse inbreeding than if a brother and sister had kids who then had kids with each other. Eventually, your offspring would be so severely mentally and physically handicapped, it no longer be able to breed and would be in constant pain.
You want to change something in the past. You time travel back to said event. You change it to how you think the outcome would be better/best/the way you want it. Here's where the paradox starts. If you change it, then in the future the outcome would be the way you want it. If it's the way you want it then you would not have needed to go back and change it. If you didn't go back and change it then the change never would have happened because you never went back to change it. This makes the original event happen which restarts the whole cycle again.