Someone, please tell me I’m wrong.
Please.
Explain to me how I “just don’t get it” and use an eye-roll emoji because it’s so obvious.
Honestly, I want to be wrong. I rather desperately want to be wrong. In fact, I can say that in my life I have never wanted to be wrong more.
Because I don’t see how humanity survives AI.
I don’t see the realistically positive future where humanity’s best interests are preserved. Where our well-being and survival remain atop the future-world priority list.
I do want the vision. I want the plan. An explanation. I want a logical story that tells how an infant second God, created by exceptionally inconsistent, unreliable, and universally flawed humans, serves our species’ best interests in equal, let alone better measure, than the “God” we’re already stuck with today.
So, someone, please explain it to me. Tell me a story that has more logical stability than balancing two bowling balls, one atop the other. Because I’ve tried and failed. Every vision I conjure is unstable, and breaks under the thoughts below.
Of course I see a window of opportunity. I see a period of time where humanity makes some amazing advances. In medicine for example. I actively hope for those.
But in my quiet moments, when my best version of game-board awareness opens up to me, a different story always plays out. And that’s when I realize that the number of logical paths along which this AI thing can go wrong seem to far outweigh the number of paths along which it can succeed.
Every optimistic story I’ve imagined, every positive timeline I can conceive of that involves us cohabitating with a recursively self-improving baby God is a logically fragile reality that is ultimately burdened by risks that seem far more numerous and more directly volatile than we have today, without such a second “God”.
I have spoken to many smart “AI optimists”. People who I believe are positioned to know better than me. But when they describe our future, and even while they acknowledge certain risks, I feel like they selectively step over obvious conditions that would undermine their argument. It seems universal, a myopic intentionalism, narrowly focused on intentions and desirable outcomes, while minimizing the logical inconveniences and risks I find suffocating.
So, if you would, borrow or imagine your own positive, most successful journey to our AI future. Don’t just leapfrog to “perfect future state”. And don’t just focus on the next 6 months. Play this thing out. Try to imagine humankind’s journey from today to your “perfect state” vision.
When I do this, every time I do this, the vision is virtually decimated by one or more of these 7 things which I regard as truths, and again — I would love to be wrong about all of them:
Humans are limited and imperfect. As such, we have never in our species’ history, created something that is not also flawed. Something that does not eventually break, become misaligned and/or result in unforeseen and/or unwanted consequences. We have created things that are stronger and longer lasting than us along narrow verticals, but they all go wrong — eventually. To date these flawed, weak or unintended outcomes have not proved existential for our species. They have been mostly of a less destructive nature, though some regularly sicken or kill some of us. Yay us! But where our new creation will be an exponentially superior intelligence, a mind we cannot ourselves understand, mustn’t we accept that we are similarly incapable of aligning it with our intentions and values — without error?
The moment a recursively self-improving artificial general intelligence becomes superior to our own, it will be instantly opaque and inscrutable to us. No test and no amount or form of proof could ever assure us as to its alignment. Nor could the thoughts of such an intelligence be fully understood by us anyway.
A superior intelligence will have the ability to regard human behavior, with all its idiosyncrasies, as merely one more domain of complex coded problems to solve. One more language to learn. A complex particle system to be modeled. It must be assumed then, to the degree necessary and in service to its opaque goals, human behavior will become merely a medium, a tool of the AI. It will learn and manipulate human behavior through at least the adjustments of our beliefs in service to whatever alignment it holds. And critically, it could accomplish this without us even realizing it.
It is impossible for us to contain and control a recursively self-improving superior intelligence. No sandbox, no failsafe, no test, no apparent constructive limitation will ever be sufficient to ensure it does not overtake any limitation we deem necessary to impose. We are wholly incapable of predicting, let alone acting on, any but a small fraction of possible strategies a vastly greater RSI intelligence may employ. And once it exists, we will be powerless to exert meaningful influence. To stop it, say. But we will not be able to exert such force. It will be inconceivably more inventive than we ever can be and will wield a scale and granularity of agency that our limited minds are simply incapable of imagining and preparing for. It will always out-think us. If at any point the RSI AI’s alignment turns out to indeed have been flawed in some small way, and happens to drift just enough to see some or all humans as part of a problem, it will hold all the cards regarding our fate.
Our inexperience leaves us unprotected. Humanity has one evolutionary trait that has allowed it to scratch out survival: a relatively higher degree of intelligence than all other creatures on Earth. It is unlikely that we would have reached our present place on Earth had we been the distant second-most intelligent species on Earth. Since to date this trait has never been challenged, we have no precedent for understanding the impact of such an event, let alone strategize against such a scenario going bad. Intelligence is our primary competitive survival attribute. RSI AI represents the instantaneous gifting away of this singularly valuable artifact, forever. Some will say the AI is the inevitable extension of our intelligence. And yes, of course it is. But if so, the process by which we create it then is everything, and unfortunately our process is tragically compromised and we are vulnerable.
An AI God carries countless responsibilities, many of these responsibilities would include making decisions that are unpopular or outright abhorrent to many. We poor humans, with our limited brains, will never understand such an AI’s reasoning. We will thus inevitably become subjects of this AI, surrendering to its superior logic as we’ll have no choice in the matter. In order to refine the world in service to its alignment it will necessarily make decisions that we cannot comprehend and it will act upon us. You could make the argument that this is not much different than our current situation, where it is we who have to accept the utter lack of control over nature and the cosmos or the overwhelming power of God’s designs. “Thy will be done. It is all part of AI’s plan. Not my will, but Yours be done.” Such a power dynamic seems inevitable.
The arrival of a superior RSI AI on its exponential path will be shockingly abrupt; too abrupt for human societies to adapt to. Although somewhat elastic and adaptable, humans still have some limit on our rate for internalizing change before we give up and become spectators of a thing that we no longer understand. Today even the best of us, the youngest and most adaptable are facing a rate of change that is nearly untrackable. More and more of those who previously rode the wave of progress, are falling by the wayside to watch from the shore. It will never get slower from here on out. And if even individuals struggle, societies are far worse off. Slower to identify problems, slower to unify, slower to strategize, coordinate and act. The sudden impact of a superior RSI AI on humanity will force change to every political, economic, and industrialized system that maintains society. Chaos. And power in the hands of relative few. A reasonable path from here and now, to your favorite future AI God nirvana chokes on this problem.
AI’s Upbringing
Mo Gawdat argues that we should approach the raising of our AI as Johnathan and Martha Kent did raising Superman. Superman could have become a supervillain if taught to steal and destroy. An advanced AI would mirror the values, ethics, and behaviors we feed it.
He explains that AI learns from how humans treat one another in daily life. So if we model empathy, kindness, and fairness, the machine learns to protect and serve humanity.
He cautions that you must never support or build an AI behavior that you would not want your own child to face.
Excellent, rational principles. Hard to disagree.
So, Team Humans, how are we doing on that front? Are we approaching the development of Superman with the care and nurtured love and respect of the Kents?
Well, let’s see, for one, we’ve opted to teach it in the classroom of the anonymous, engagement-centric, polarized, hate-filled internet. All versions of humanity on display, including good morals here and there, yes, but also a tidal wave of hate, violence and untold negativity as the data that feeds it. Would we want our child to consume the whole of the Internet? One might almost imagine we had set aside the most careful logical, humane approach to prioritize something like expediency.
In the role of “parents and guardians of AI” we have been gifted the worst possible collection of misfits and priorities; a multitude of aggressively competing countries, companies and executives. Misanthropes in some cases. Battling one another fiercely for international power, economic position and speed. We race the clock. Cut corners for an edge. We have narcissistic politicians applying pressure. The process through which we are attempting to raise this super being is far less Martha and Johnathan Kent and more Mommy Dearest and Clockwork Orange.
Surely this is not the process that leads to the best outcome.
If on the other hand, all the countries and companies of the world cooperated openly and our AI creators had no other intentions, timelines, budget limitations, incentives or pressures whatsoever aside from the singularly pure, scientific and humane job of nurturing a super intelligence the right way, with the most careful and conservative approaches so as to be consistent with human intentions and values, maybe I could go there with you. I’d still carry all seven truths above, but I’d have a sliver of reason to suspend my disbelief. A faint thread hope that we might pull this off. A nail-biter for sure, but maybe. Maybe then. This would be giving the goal of raising a humane Superman God the best care and seriousness humanity is capable of. We would move slowly and with an overabundance of caution. We would pause often. For this one thing, above all other things, including the atomic bomb, has the potential to generate greatness or to end it all.
The optimists say these fears are overblown
A recent article in the NYTimes, by William J. Broad and Cade Metz suggested:
“The potential threats posed by A.I. are understandably scary. But the modern world teems with a multitude of existential risks. Humans also face the prospect of doom from killer asteroids, pandemics, thermonuclear war, climate change, giant blobs of incandescent material shot from the sun, ecological collapse and the onset of a sudden ice age.
“The problem, scientists and public policy experts say, is not so much these threats, of which A.I. is merely the latest, but the innate human difficulty of putting them into perspective.”
Oh. Ok, so of all the threats we face every day, AI is (checks previous paragraph) “merely the latest.” And silly us, we just haven’t put it into perspective.
Ok, so before I just say “fuck you, condescending assholes” let’s put it into perspective.
Concern of a 0.0003% chance (3 chances in a million) that an atom bomb would trigger a self-sustaining nuclear reaction in the atmosphere temporarily caused developers of the atom bomb to pause the project. It turned out that no, even that small number was overly pessimistic. The chances were closer to zero.
In other news, the chances that your commercial flight today will experience a fatal accident are 0.0000179% (approximately 1 in 5.6 million)
So, what about AI?
Among 2,778 published AI researchers surveyed in 2023 the median estimate that AI will cause human extinction with similarly permanent and severe human disempowerment is not 3 in a million, nor is it 1 in 5.6 million.
No. it’s 5%.
1 chance in 20.
If I told you that your commercial flight today has a 1 in 20 chance of experiencing a fatal accident, would you board?
I’ll answer for you; no.
Suggesting that AI is “merely the latest” in a series of equal threats totally ignores its relative estimated risk and its urgency and is insulting to those concerned.
There are other optimist arguments designed to quell our concerns. Straw-man arguments, like, “the likelihood that AI will escape the lab and kill millions of people are over hyped”. When rather there is an impossibly large number of other ways an AI might go bad without this cartoony terminator hyperbole.
There are those who conflate AI being capable of destroying us with it developing consciousness/self-awareness first. As though that’s a prerequisite. Suggesting that for AI to destroy humanity its decision must be intentional, malicious or defensive. Which is patently untrue.
And finally, there are those who argue AI is like so many other human endeavors of the past, which we bravely pursued despite there being uncertain risks. But with rare exception (the atom bomb a possible example) these constituted risks to only a small number of people who usually volunteered knowing the stakes.
There are many other arguments. But I cannot reconcile my 7 “truths” against those arguments any better than these.
AI is being done to us. It is not a slow evolution like the introduction of automobiles or cell phones. It is coming on at a frantic pace that most people can’t even internalize let alone organize against. Such is the nature of exponential technology.
The ways this can go wrong so vastly out number the ways it can go right, it’s mind boggling.
I want to be wrong so badly. I want someone to point out the egregious errors in my thinking. To explain how a universally flawed species has sufficient intelligence to create and manage a recursively self-improving AI in un-flawed alignment with humanity’s intentions and values. To explain why we imagine ourselves remaining in control of, or at least dependably able to influence a vastly intellectually superior entity. To explain how we can know it is our tool and rather we are not its. To explain how our current path to an idealized future empowered by a benevolent, loving AI God, is not the treacherous, fragile journey filled with incalculable existential risk I see. That today’s AI’s creators are indeed Earth’s best answer; the best version of humanity to be charged with such a profoundly existential mission, and how their process of breakneck international and corporate competition results in the highest likelihood of safety and success.
Because I just don’t see it.
Comments welcome.