A Single Step From Evil

He had done it. The code was finished. The program was complete, and if his math was good (and it always was), Dr. Hugh Tolling had synthesized a software capable of predicting any past or future event. From the mundane to the dire, he could see everything—and who better than he should be blessed with this divine gift? Who else should be trusted to make the distinction between what matters and what doesn’t? Certainly not any of his colleagues; those irreparable, bumbling wretches. No—it was Dr. Hugh Tolling who had accomplished the inconceivable. He had become a time lord. And he alone shall reign.

In fairness, his software was generously hoisted upon the shoulders of colossi. Without the enormous strides of the various predictive AI engines of the 2100s, Tolling’s program would have been a hypothetical dust grain in the nebulae. He was well aware of this, but he also knew that those other names would be lost to history while his own would live on. “Tolling” would be etched into humanity’s collective memory like “Newton” and “Einstein.” In fact, he could take a look, right now, and confirm that prediction; should that endeavor be his first run of the software? A quick peek into a grade school classroom a thousand years from now, where he was sure he'd find an apathetic student giving a presentation on the life and achievements of Dr. Hugh Tolling, inventor of the tool that elevated humankind to the tier of the gods.

Tolling sat back down in his lab chair, giddy with anticipation. But an instant later, that excitement melted into unease. Then the unease shriveled into fear. If another second had elapsed before these contemplations, the program would have been run and the world would be altered forever. But something nagged at him, gently at first, then ferociously. It stopped him dead in his tracks.

The problem was clear: since the moment he concluded writing the code (and some time before he was finished, to be sure), he had been consumed by selfishness. He felt an overwhelming sense of pride in his own bravura. He thought nothing of the feelings at first, but enough time had now ticked away to allow the analytical side of his mind to soak up some of the dazzling spotlight in which his ego had been unabashedly bathing. 

This initial reaction startled him. He found the fact important that he was not immediately taken by the philanthropic applications of his software; the good that could be done with it. The avoidance of immense swaths of hardship and pain. Prosperity for the ages was within grasp, if only the software was used correctly. But goodness did not strike him first. Temptation won the race. Lust for glory and praise outbid his will to make honest improvements for his world.

Tolling looked up and down at his creation—a compressed file hidden inside an icon on the home “screen” of his autosystem—and he saw, really saw, what he had made. He was standing on a precipice, a single step from greatness in one direction, and a step from evil in the other. A blanket of shame and despair wrapped itself around him, but he shook it off just as soon. “We don't have time for this,” he reassured himself. “Humanity doesn't have time for second-guessing.”

He closed his eyes.

“Run the program,” Tolling announced to no one. His autosystem immediately flashed before him a new “screen”: a rectangular image projected a foot and a half away from his face. A screen which only he could see. The simulation specifications populated on the screen precisely how Tolling had been thinking of them. Autosystems performed not only the heavy lifting of nearly every day-to-day task, but the light lifting, too.

Tolling’s autovoice—a flawless replica of his own, with its sporadic twinges of raspiness and booming depth—said, “Verbally confirm simulation parameters.” 

Tolling looked at the parameters and mulled them over for a moment before reading them aloud. “Dr. Tolling shall use the predictive software to predict the effect of Dr. Tolling having created aforementioned predictive software. Time window: one year from today. Space frame: this laboratory.” Simple and clear. But it didn’t sit right with him. Something wasn’t adding up. Was the prompt fallacious? “Perform a fallacy check,” said Tolling.

His autovoice stated: “Fallacy found. Unaccounted action: the effect of using the predictive software may skew the data gained from predicting the effect of its creation alone.”

Outside one of the many tall windows of the laboratory, a snowy owl perched on a towering evergreen branch. It peered through the plexiglass directly into Tolling’s eyes. Its piercing yellow irises spoke to Tolling. What of the animals? he thought. What will their fates be? I can watch the very last snowy owl die right now if I so choose.

“New parameters,” said Tolling. And just then, the new parameters appeared on his autoscreen. 

“Verbally confirm simulation parameters,” said his autovoice.

“Dr. Tolling,” Tolling began, “shall use the predictive software to predict the effect of using the predictive software. Fallacy check.”

“Fallacy found. Self-fulfilling action: the act of using the software to predict the outcome of its use will cause the predicted outcome to occur.”

“The Court will know what I’ve done here,” Tolling said. “No doubt, they already do: they analyze everyone’s thoughts all the time, nonstop, every second of every day. They catalog them and reference them in fractions of nanoseconds. Even if I destroy it right now, they already have it.” Tolling paused for a moment before stating what he had known all along: “The program was used the moment I brought it into existence.”

The owl flew away, onward to someplace new, on with its life, on with its chartered course through its small corner of the cosmos.

“I won’t let them take it from me!” Tolling erupted. He quickly regained his composure and said, “Parameters confirmed. Run the program.”

Tolling was thrust into the simulation, one year into the future, at the exact coordinates of his lab chair. The laboratory was no longer. Nor were the artificial evergreen trees that once littered the sprawling complex. All that remained of the structures was a heaping pile of burnt rubble, which was slowly beginning to be reclaimed by the lichen.

A snowy owl swooped down and landed on a chunk of marred debris. Its eyes met Tolling’s own for an instant before it flew away.