Introduction: Better than Human

We get better at being moral. unfortunately, this doesn't mean that we can get moral enough, that we can reach the heights of morality required for the flourishing of all life on planet Earth. Just as we are epistemically bounded, we also seem to be morally bounded. This fact coupled both with the fact that we can build machines that are better than we in various capacities as well as the fact that artificial intelligence is making progress entail that we should build or engineer our replacements and then usher in our own extinction. Put another way, the moral environment of modern Earth wrought by humans, together with what current science tells us of morality, human psychology, human biology, and intelligent machines, morally requires us to build our own replacements and then exit stage left. This claim might seem outrageous, but in fact it is a conclusion born of good old-fashioned rationality.

In this paper, I show how this conclusion is forced upon us. Two different possible outcomes, then, define our future; the morally best one is the second. In the first, we will fail to act on our duty to replace ourselves. Eventually, as it has done with 99 percent of all species over the last 3.5 billion years, nature will step in to do what we lacked the courage to do. Unfortunately, nature is very unlikely to bring our replacements with it. However, the second outcome is not completely unlikely.