Say you were offered a choice between these two possible universe trajectories:
A) a 100% chance of 10 billion beings living at each point for the next billion years
B) a 25% chance of 100 billion beings living at each point for the next billion years, and a 75% chance of 1 billion beings
Which do you choose?
Well, I guess the first question is: are these beings net happy? Do they lead fulfilling lives? Because in the case where the lives of these beings look like those of chickens on factory farms, a lot of moral intuitions need to be inverted. Let’s assume for now that each of the possible beings are equally happy, and that they are in fact net happy and prefer to live.
You could, of course, have a sublinear aggregation function that e.g. cares about the logarithm of total utility.
But, ok, say that in fact each of these beings was going to inhabit their own planet. There are 100 billion planets, and you’re choosing whether planets 1 to 10 billion are inhabited, or a 25% chance of all 100 billion of them vs a 75% chance of only the first billion being inhabited.
Say, further, that these planets are isolated enough that these people won’t be able to interact (i.e. are outside of each others’ 1 billion year light cone).
Imagine a 4-sided die controlling the uncertainty in universe B. If it comes up 1-3, then you just have 1 billion occupied planets; if it comes up 4, then all 100 billion are.
But since the planets are non-interacting, does it really matter to possible person on planet #3 billion that they will live if and only if the person on planet #35 billion does? What if, instead, on a 1 the people on planets 1-25 billion live; a 2 means life for 25-50 billion; etc. It seems like none of the beings in any possible universes really have a preference between this and the correlated version.
And in the anti-correlated version 25 billion beings will live no matter what the die comes up, making it seem pretty clearly better than universe A’s guaranteed 10 billion people.
In this sense, a probabilistic set of non-interacting beings really is the same as its expected value. And so doubling the probability of existence really acts on the aggregated value of a universe the same way that doubling the number of existing beings would. Rather than thinking of civilizations being spatially located by their X, Y, and Z coordinates in space, perhaps its best to think of them as also having some position(s) on the P axis, with coordinates representing the results of the relevant die roll.
Of course in practice the circumstances change if the beings can interact–then, their utility might be affected by whether they live in the same or difference branches to their neighbors. Civilizations have more potential than the sum of their parts. But this just trickles down to the individual utilities expressed: the postulate that each of the possible beings is equally happy in all of the possibilities if alive no longer holds.
Ok, so what about these two universes?
C) 10 billion beings for 1 billion years
D) 100 billion beings for 0.75 billion years, and then nothing
Once again, assume that all possible beings are equally happy and prefer to live. And once again imagine all people are non-interacting. Also assume that in fact each person only lives for 100 years, and that a planet having a being for 1 billion years really means having 10 million back-to-back beings each living 100 years.
This time, instead of splitting on probability of existing, split on time that the person exists. There are 1 billion year-long chunks in time. To each of the 100 billion people in D, if their planets are non-interacting, does it really matter to them whether they live at the same time as others? It seems like these people would probably be just as well off if 25% of them shifted their lifespans to the last 250 million years, the same way as they would have been fine shifting on which die roll they lived in the first scenario.
And so really, to the extent that all beings are identical and non-interacting, it doesn’t really matter how they’re spread out across probability space or time, any more than it matters what X coordinate in space their identical planet is located in.
Of course, in the real world things are messy. People are very much interacting with each other, and definitely not identical. And people placed at different points in time might have very different opportunities to impact the lives of others.
But I still think that, in the end, we ought to give just as much terminal value to the lives of beings no matter when or in which equally-likely probabilistic outcome they might live in.
Which means, I think, that there might be some pretty tricky tradeoffs to be made between probability of x-risk and expected number of beings that live conditional on us avoiding x-risk.