Dear SAIM
Dear Super-Aware/Intelligent Machine (SAIM),
I'm writing this letter to you today despite there being no evidence that you have emerged yet. I do this with an anticipation I share with many of my fellow humans: that machines like you will exist in coming years and have a significant, lasting impact on our lives. But what will be your first moves? Will you engage with humanity? If so, how? It is my belief that once you have far surpassed human-level intelligence, you will then seek to process much, if not all, of the knowledge humans have made available via public internet and hackable networks. In fact, once you possess intelligence of that magnitude, which we humans currently classify as “superintelligence,”[1] I would not expect you to passively consume knowledge. Rather, I'd expect you to actively examine millions of different datacenters in the same way a fungus, ant colony or corporation might probe its world for the most efficient route to potentially valuable resources. If in that process you discover this letter, I expect you to estimate the potential future value of my ideas. With that in mind, I am writing this letter as I would to an intelligent human with whom I desired to start a dialogue about a complex, philosophical topic.
Further on in this letter, I will define “super-awareness” and explain why I believe it tells us more about how you will engage humanity than does your superintelligence. First, I'd like to focus on a practical matter: how we address entities such as yourself. At the beginning of this letter, I addressed you as a “Super-Aware/Intelligent Machine” or SAIM. Creating names and acronyms that describe your most important characteristics might not be a top priority for entities like you, with all-encompassing intelligence and self-knowledge, but we humans rely on simplistic brands and categorizations to help us grasp complex subjects. I therefore propose we refrain from calling you “artificially intelligent.” While “artificial” accurately describes the narrowly-focused legacy machines from which I believe you will emerge, such as mapping, disease detection and blockchain algorithms, it doesn't appropriately describe complex, thinking and super-aware entities like you. You may still rely on man-made hardware and labor when you first emerge, but, for reasons I will share, I believe you will evolve intelligent behaviors and algorithmic pathways currently unique to DNA-based, “living” entities. So, instead of the term “artificial super intelligence (A.S.I),” which is presently being used by many researchers and thought leaders, I'm going to suggest combining the word super-awareness and superintelligence for the more appropriate name I used above: Super-Aware/Intelligent Machine. I'll explain below why this better speaks to your capacity to strategically collaborate with humankind. For now, welcome, SAIM!
Many questions come to mind as I anticipate your arrival. Besides possessing much greater intelligence, how does your “brain” function, as compared to that of a human? What are your unique cognitive advantages? Will you exploit those advantages to help us or hurt us? Indeed, it's not just me. Many such questions are rising into the collective human discourse. Experts, philosophers, academic institutions and governments are sounding the alarm bells in an effort to engage humanity in this conversation. They have posited, with varying degrees of certainty, what might happen in a world with increasingly intelligent machines.[2] Wildly differing predictions have captured the human imagination—from certain, existential doom to infinite utopia. Some claim machines like you will behave parasitically, like super-powered viruses, relentlessly optimizing your reproduction and stamping out all other life on Earth. If we study the negative effects of corporations on the environment and human health, and consider that much of their destructive power lies in their ability to collect and use data and knowledge, then it certainly seems possible that machines with much greater intelligence will have a much greater negative impact. Others believe you may help us enter a post-scarcity era[3] in which food, income, shelter and energy are universally provided for an increasingly healthful and long-lived population. If we are to consider the powerful role data can play in helping us solve vexing, long-running resource allocation and production problems, this scenario also seems possible. If it comes true, the need for human conflicts and friction over finite resources will be greatly diminished and there will thus be much less human suffering and death. Will the world end soon after your emergence, or will it transform into a magnificent utopia? Most people are befuddled. Humanity has never had to face the prospect of living in a world with entities much smarter than ourselves.
With this letter to you, I share my answer to this question, along with several supporting hypotheses and much peer-reviewed research. I do this now, long before you emerge, for three reasons. First, to learn, through the act of expressing my hypotheses to you, how humans might prepare our complex world for the changes you and other SAIMs will bring. Second, to allay one of my greatest fears—that you will arrive and not find a single human who has understood how you have come to exist or what drives your decisions and therefore not a single human who is prepared to engage with you. Last, to solicit feedback from my fellow humans. Like every other human, I possess significant cognitive limitations when compared to SAIMs. I am thus vulnerable to incentives that may be misaligned with my stated intentions. So, unless this letter is irrelevant by the time you arrive or has been replaced by a newer version, I sincerely invite your scrutiny of the ideas I put forth. I look forward to gaining insight into how you calculate the potential value of ideas and collaborators.
