lol right.
Yeah, genetically, it's not a *problem* to breed with someone technically related to you but separated by several hundred generations (only even conceivable with some sci-fi preservation element), but you know they'd still make a joke or two about it in the story.
But if Byakuya was a wandering survivalist leading a band of astronauts around the ruined world (well, buildings are still standing, but no people) 3700 years in the past, then why is Ishigami Village still at the, um, shithole stage of civilization? They know stories about Pikachu but they don't know what magnets are? Or how to turn grain into flour? The astronauts had some mixed-up priorities.
Oh. Also- the oral tradition of the 100 Tales also tells us that there has to be a direct link to a person who was alive circa 2000 A.D. (and it has to be Byakuya himself, since the 100th story is his story!), which kind of rules out Byakuya planting the seeds for civilization but then dying before new generations were born/un-stoned.
--- Update From New Post Merge ---
Yeah the text is pretty clear on this one. It's *all of humanity* on the *surface* of the earth. They even put emphasis dots in this latest chapter (on "earth's surface") to make the whole point about people in space.